Sunk by Complexity

Monday, August 16th 2010

If it takes effort to understand something, it is is hard to believe the effort was not worth it. That the complexity was pointless. The effort has been made. You understand it. So you necessarily feel attached to the complexity to one degree or another.

One of the problems of modern life is the bitterness and resentments of those forced to learn things that seem pointlessly complex. Such as a granny trying to work out how to use a web browser. Or learning a new set of guidelines from ...

Argh, you get the idea. I don't want to think about it.

Complexity & Vested Interests

If something is complex it means someone, somewhere, understands it - to one degree or another. And that person has invested time and effort in that complexity. In this way, complex things represent "sunk costs" to those understand them.

(I'm leaving out the complexity no-one understands. Which is most of it, but oh well. Some people understand some of it, but know they don't know it all. So they are annoyed they can't grasp it, and worried someone else will come along and expose their ignorance by knowing just that little bit more. Hence the unwillingness to admit you don't know by handing out irrelevant pieces of information in the form of sideways flummery. It also explains why the "confident power play" of saying: "Hey, what does this mean?" can be so dangerous.)

Further, the complexity means that others cannot easily replace that person. It makes the person valuable (as long the complexity is extant).

I have written before that this partially explains a great of impenetrable academic writing. It also explains why computer software often ends up so complex and hard to extricate your data from. Or how this suits financial systems that need to obscure their shenanigans behind "complex financial instruments". (The Yen Carry Trade Oboe Anyone?)

But there's another aspect to it, too.

The Curse of Knowledge

The Heath brothers, in their book "Made to Stick", refer to the idea of the "Curse of Knowledge". That is:

"... we start to forget what it's like not to know what we know. At that point, making something simple can seem like "dumbing down". As an expert, we don't want to be accused of propagating sound bites or pandering to the lowest common denominator. Simplifying, we fear, can devolve into oversimplifying." (Made to Stick, p. 46)

More than this, though, abstract complexity can seem like a fact to you because it is a fact to you. The abstractions are derived from less abstract, concrete experience. You pin abstract ideas to actual experiences.

Yet, to generalise from that experience means abstraction. But underneath that abstraction are the memories used to form the basis for that abstraction. But only for you.

To others, they have to fill in their own memories if they have them. Otherwise it is just noise.

Most abstraction is just noise.

Is what I'm writing noise to you?

Probably! How can I tell the difference? How do i escape the "Curse of Knowledge"? I'm not sure.

The Dunning Kruger Effect

John Cleese, in a lecture on creativity in Belgium, puts it this way:

" ... to know how good you are at something requires the same skills as it does to be good at that thing. Which means if you're absolutely hopeless at something you lack exactly the skills that you need to know that you're absolutely hopeless at it. And this is a profound discovery. That most people who have absolutely no idea what they're doing have absolutely no idea that they have no idea what they're doing."

This is also known as the "Dunning-Kruger" effect. John Cleese went on to observe:

"... it explains a great deal of life. It explains particularly Hollywood. But it also explains why so many people in charge of so many organisations have no idea what they're doing. They have a terrible blind spot. And the problem with the teachers may be that the teachers do not realise that they themselves are not very creative. And therefore they may not value creativity even if they can recognise it."

It may also be that someone who enjoys complexity, or has an attachment to it, is also a victim of the Dunning Kruger effect. Or, at least, a strange variation on the idea.

If making things intelligible to the layman is a skill, and people without that skill do not realise they do not have the skill, then those people are a victim of the Curse of Knowledge.

We could call this is the "Inverted Dunning Kruger effect" or something like that :-).

Now, use that term in a conversation and, voila, instant introduction of over-complication to a very simple idea:

People who aren't good at making things intelligible don't often realise that what they're saying is unintelligible.

Now imagine I didn't realise that the Curse of Knowledge and the Inverted Dunning Kruger effect were unintelligible gibberish. And proceeded to use the ideas to explain something.

Why would I do it?

  • Because I don't realise it's over complicated;
  • Because I want to sound clever;
  • I want to exclude people who don't share my Curse of Knowledge from the discussion.

One of those must be true :-).