Thursday, July 30th 2015

The distinction between metadata and "content" is one of those superficially appealing ideas that doesn't stand up to scrutiny.

Metadata is data about data. Examples from wikpedia:

  • Means of creation of the data
  • Purpose of the data
  • Time and date of creation
  • Creator or author of the data
  • Location on a computer network where the data was created
  • Standards used

But as observational computing emerges, most data will be data about data. That is: it will be data collected by machines, not entered directly by people.

For instance: if I make a face, and this creates data points for a machine, is that "content" - i.e. something I created and communicated? What about if I ask a question of a virtual assistant? What if the virtual assistant starts to predict my requirements? Does this count as metadata?

That is, the metadata needs to completely computer-generated. It cannot contain content - content being something created by a person consciously.

The argument is that a phone call is a comprised of data (a phone number, for instance) and what was said.

But the truth is I typed the phone number in. It is "content" in the sense that I entered it.


The trend is for more and more technology to autonomously gather data via observational computing. As machines become more autonomous they gather more and more data. Less and less information entered by or directly created by humans as "content" (e.g. email content, phone calls, etc).

As less and less data is entered by people (and classified as "content") and more data is generated by observing of people and their use of technology ("metadata" - data about content), the metadata/content divide will end up irrelevant by dint of the fact hardly any useful data is "content".

You will be able to know everything you want to know about people from data about them no created by them.

The internet has only been ubiquitous for the last twenty years (for most people). Sophisticated internet-enabled phones (post iPhone devices) in the last 10. Internet-enabled devices (TVs, fitness bands, watches, car sensors, etc) in the last 5 or so. So there is an order of magnitude more data available concerning peoples' everyday activities than there was before.

The scope of what "metadata" encompasses has increased significantly and is likely to continue to increase; it will include much more information about citizen's lives. Indeed, as sensors improve, machine-generated observations of people will, particularly once it is analysed by algorithms, end up being more intrusive than human-generated "substance".

Terrorism Lite

Wednesday, July 29th 2015

"Terrorism" comes from the French word terrorisme, and originally referred specifically to state terrorism as practiced by the French government during the 1793–1794 Reign of terror. The French word terrorisme in turn derives from the Latin verb terreō meaning "I frighten".[6] The terror cimbricus was a panic and state of emergency in Rome in response to the approach of warriors of the Cimbri tribe in 105 BC. The Jacobins cited this precedent when imposing a Reign of Terror during the French Revolution.[7][8] After the Jacobins lost power, the word "terrorist" became a term of abuse.[9] Although "terrorism" originally referred to acts committed by a government, currently it usually refers to the killing of innocent people[10] for political purposes in such a way as to create a media spectacle. This meaning can be traced back to Sergey Nechayev, who described himself as a "terrorist".[11] Nechayev founded the Russian terrorist group "People's Retribution" (Народная расправа) in 1869. Wikipedia

Most people don't cooly appraise the risks of certain classes of events occurring or the costs thereof. They don't use statistics as an actuary would. Instead some risks are seen as scarier than others, even if the physical effects of those things are much the same. E.g. The risk and costs of people being blown up by terrorists is not assessed in the same way the risk of being in a car accident is assessed.

So there are two types of risk.

  • "shit happens" risks;
  • "the only thing to fear is fear itself" risks.

A car accident is a "shit happens", the price of using technology that otherwise is very positive. It is, across a large population of motorists, reasonably predicable. People accept the risk of driving.

An act of political violence is different. It taps into everything from atavistic tribalism to media hysteria. A hysterical response to a terrorist act is what gives the act its power. And a hysterical response is inevitable if the outrage is well calculated enough. The end result is damage to the social consensus that supports the institutions that keep society running.

In theory, if terrorist acts were treated as mere crimes then the threat would dissipate. (Or escalate until would-be terrorists discovered an act heinous enough to engender the suitable level of hysteria.)

But the main reason terrorism represents a Big Risk is because a consensus might emerge that it is Big Risk. The psychological reaction is the real damage. It is impossible to discuss this stuff rationally. In the sense that there are so few incidents to call upon in real life that the only basis of discussion is wild speculation. Rational thought is lost in an explosion of anecdotes, spectacle, and hyperbole. The paranoid imagination knows no bounds! It keeps inventing new scenarios to scare itself. It scares itself stupid.

That's why terrorism can work -- it scares a more powerful enemy into irrational decision making. Into strategic errors. Errors a more deliberative enemy would not make. It's the tool of the otherwise weak or powerless.

Or, at least, it seems that way.

But people in positions of power are actually only in nominal positions of power. That is, they rule with the tacit consent of the people who support or work for the institutions that give them their power. A dictator without his army and security services is no dictator at all!

Indeed, all power structures rely on external threats to garner support. You accept the authority of the police because they protect you from criminals. You respect the legal system because it tries to resolve conflict without actual, direct, arbitrary violence. You respect X (authority) because if Y (threat). It's a tradeoff.

All power structures try to shift things toward authority. That requires a external threat. The bigger the external threat, the power you can accrue.

Every system of power and authority needs an external threat to remain viable. The more extensive the power the bigger that threat needs to feel.

It follows that terrorism by the powerless usually supports the powerful. (A shortcut to power is the manufacture a threat, of course. Then provide a solution to it by asking people to give you power.)

Cynical users of power know that the threat of terrorism/child molestation/invasions by immigrants/mass murder/any other highly emotional issue usually bypasses the public's rational thought and shifts things closer the authority end of the continuum. That is: it can be used to trick the (dormant but powerful) public into supporting their increasing authority.

In this way cynics can trick you into mistaking the interests of the powerful with your own interests. The threat to the power or status of the powerful is transmogrified into a threat to you.

It is all about scaring people into granting new powers to politicians/police/security organisations. Powers that people would not agree to if they looked at the actual threat rationally; in relation to with all the other threats to life, liberty, etc (car accidents, pollution, and so on).

Although not necessarily the use of direct violence, as this process expands beyond certain reasonable bounds (authority to deal with genuine threats) it becomes about the exploitation of fear of violence for political ends. Which is a hop, skip and jump away from actual violence for political ends. E.g. a hop, skip and jump away from terrorism.

Nothing new here, either.

The things power systems of all stripes fear most is a dissolution of the "uh-oh! threats!" narrative they have wrought. A dissolution that shifts things away from authority on the continuum. Neither terrorists or the powerful want this to occur. If it does, they lose support and credibility.

And so a macabre dance begins.

The equation is simple:

  • Media spectacle happens or is contrived to happen
  • "We need new powers to combat [insert highly charged issue here]!"
  • Question: "so you'll be restricting the use of that power to only certain classes of crimes?"
  • "Erm, mumble, mumble, blah" (Upshot: no, we will not)
  • What about the problems that led to this terrorist activity ...
  • "What! Are you a terrorist sympathiser?"
  • Result: new powers used much more widely.

People invariably follow these processes to secure themselves from scrutiny or change.

It's Terrorism Lite. Taken to its logical conclusion, it leads to actual the types of systematic outrages terrorists only dream of.


  • Poverty, injustice or some historical grievance.
  • The ruling powers are hopelessly corrupt and must be destroyed!
  • "We need to take brutal steps to make or repressors make strategic errors!
  • Question: "won't this make us as brutal as them?"
  • No! They are the enemy. Unlike them it they are brutal means to a just end!
  • "But isn't that what they say?"
  • "Erm, mumble, mumble, blah - what are you, a collaborator?"
  • Result: people are compelled to fight for the cause.

Beyond all that terrorism rarely causes more harm that an unpredicted accident. (Almost be definition, terrorists are not a real political or military threat. People are only terrorists if they can't fight and win a conventional battle or, perhaps, political debate.)

On all sides it's just violence.

Inner Fish

Tuesday, July 28th 2015

"These fish-derived life-forms are unimpressive," said Captain Zaaarl Qasinoonoo the Seventh Quandant of Tersog. "I always said that the Neems of Ungrum II should use a more efficient algorithm to generate their theme park on Terra, but no! They had to use evolution."

Your Inner Fish:

Your Inner Reptile:

Investment-based Welfare & c.

Monday, July 27th 2015

Some see better rehab for prisoners as indulging miscreants, "lax drugs laws" as "sending the wrong message", "permissive welfare" as rewarding laziness, etc, etc. They see these rehab, lax drug laws, and income support as costs. (And, sure, there are some valid concerns. Marijuana can cause psychosis in some cases, for example. Do you want to let people buy it?)

For these people the "state" (or "society") is actually a stand-in for for the person making the argument. What they really mean is:

  • I give you unemployment support. What do I get? Nothing. Bad deal!
  • I give you free education! What do I get? Nothing! Bad deal!
  • You committed a crime! You must be punished as a deterrent!
  • If I tell you drugs are okay and don't punish you'll take them and drain me of cash with irresponsible behaviour!

The approach is primarily one dimensional, narrative and transactional.

This mentality tends to only look at the costs of programs because they see the transaction as one way. From the person (via the State) to the "miscreant" and that's it.

Perhaps this is because costs are easily measured and benefits are more distributed and harder to measure.

The results are failed "wars on drugs", welfare policies that lead to poverty traps, non-violent drug addicts being jailed, and so on.

But deaths due to lack of clean needles are a thing. People being raped in a badly run prisons are a thing. Higher recidivism. Etc. Etc. These are all costs. Policy-makers need to see the more the equation.

Sometimes returns are not be easily measured. But that doesn't mean they aren't there!

The idea is:

  • better rehab for prisoners reduces crimes and prison costs;
  • better drug laws that treat drug abuse as a health problem rather than a crime and reduce drug-fuelled crime;
  • getting homeless people into housing reduces the cost of policing homelessness, and so on;
  • more effective unemployment support means less spending on unemployment dollars, removes perverse incentives (such as high effective marginal tax rates);
  • return on investment on refugees over a 25 year period (e.g. do tax returns from those refugees exceed the cost of processing and helping them in their early years?)
  • and so on.

Rather than simply looking at the moral dimension (whatever your moral position around these things are), looking at the evidence of whether a dollar spent on better rehab is a better investment than a dollar spent on prisons is worth looking into and a possible way to find agreement outside the traditional moral posturing and positional negotiating.

These things need to be seen as investments. "Investment based welfare", for instance.

Just the the sort of thing business people wrack their brains over.

(I wrote yesterday about the problems with this idea, too (Markets and Legibility).

In Utah, for instance, they discovered it was cheaper to give homeless people housing and social support than police them on the streets. Morality aside, it is a simple business decision.

The Australian NEIS scheme, supporting small start-up business owners with income support, in the early years actually pays off. Dollars spent on NEIS are actually an investment in business tax revenue.

There will be problems with a simple equations like this.

But I can't think it could be any worse that the moralising (on all sides) about these things. Sure, the morality has its place, I think. But it blinds people to compromise. Some people tend to think more in terms of injustice, empathy and fellow feeling. Often they are performers of a kind. Others think transactionally. "Investment Based Welfare" could help get people talking.

Markets & Legibility

Saturday, July 25th 2015

The thing that mystifies me about some people is that they think virtually anything can be improved by applying business notions to it. "The discipline of market signals should be applied to extract more efficiency", they might say.

But business techniques can only be really applied to solve a limited set of problems in certain domains.

Where the business is involved in an area where economic externalities are small, there is strong competition, natural monopolies are minimal, people can make informed decisions ("information asymmetries" are low) and negative externalities can be remediated with regulation.

That's because the business world - as whole - tends to optimise for one thing: profit. The rest is marketing.

How a businesses optimises for profit might involve doing really useful, long term things. Or it could involve being parasitic or predatory. Or both! But a business that doesn't optimise for profit is usually going to fail. There are, of course, non-profits, mutual societies, and so on ... but even they try to expand and accrue the cash they need to operate -- most organisations tend to want to perpetuate themselves.

The very essence of making a profit (or, at the very least, self perpetuation) is to minimise your exposure to negative externalities. To push costs on to others. Whether through concerted activity or simply by inaction. Businesses also want to eliminate competition. If they are profitable they want to avoid change -- so they end up very conservative indeed.

It is just how it works.

I don't see this as a problem per se. It leads to good and bad results depending on how society regulates the process.

Some claim that the market system as self-organising entity. Something almost independent of any organised social order; something that inevitably emerges if people are left to their own devices. A consequences of a human tendency to truck, bater and trade. A simple consequence of human nature.

But even if humans have a natural tendency to truck, barter and trade (which may or may not be true), once a society gets to any size, self-organisation has to take a more structured, directed form; norms are established, rules enforced, and so.

A market of this kind (and the "free market" that most economists talk about is this type of market) is a creation of the State (it enforces laws, legal entities such as corporations, property right, etc). It is as much a creation of the state as the police, public schools, unemployment support payments or vast tax bureaucracies.

Venkatesh Rao, talking about "Seeing like State" and Stephen Johnson's "Mind Wide Open". He says Johnson is:

" ...a guy who wrote a book about subjecting himself to various biological tests. And one of the things he did was put himself through a fMRI machine. And, before he started he asked the technician "well, maybe you should show me some random white noise to make my brain into a neutral, normal state. And the technician told him that white noise tends to send peoples' brains into complete craziness. Your mind goes crazy trying to find some pattern in the chaos. If you show a person a straight chessboard type of pattern that makes the mind calm down completely and the fMRI machine starts at zero."

It's my thinking that the mind naturally seeks pictures [like the chessboard]. Even if it makes no sense to put that much order or structure into something we impose that kinda structure just because it makes our anxieties go away.

He talks about how imposing order and structure to sooth this anxiety and to make things more "legible".

He refers to the discussion about reorganising land ownership during the Stolypin reform (a study from a book "Seeing like a State"), the idea is that before the reforms land was organised around water sources, local walking trails, and so on. After the reforms, the the centrally determine land boundaries ignored all this local intelligence and went for a simpler-looking, more logical grid.

He sums it up:

Centralised intelligence tends to ignore local knowledge.

Something is "illegible" according to Roa if "it serves many purposes and functions in complex ways. So a locality has thousands of people doing different things ... whereas when a central planner comes in they typically only want to do one thing like maximise the yield of a forest, maximise the revenue from a piece of land ... they impose a legible model and cause that organic reality to simply collapse because they don't understand how it really functions".

There's a bit more to it, though.

One of the effects of seeking legibility is standardisation. The benefits are many (everything from global trade, container sizes, to coordinated times, etc), but the costs can a significant loss of important local information.

Any State organising a market system at scale (or organisation operating within a market system at scale) invariably turns into a vast, centralised bureaucracy that seeks out standardisation. It tries to make reality "legible" around the notion of market transactions and its own internal structures. Although it makes an attempt to integrate local information into transactions (via prices and other market mechanisms), these systems are missing a lot of information (due to externalities, etc).

Its the story of the bureaucratic age.

(That is not to say these systems are not fit for certain purposes. They most certainly are. But, for a traditional market system to survive (long term), the market bureaucracies have to be treated as a lossy information processing system.)

Scale and complexity go together. As the market system expands its reach, it has to process more and more information. Not only because it becomes more complex itself, it has increasingly complex feedback loops with other systems (everything from ecosystems to collective psychology).

As more information is generated by everything from observational computing and more densely networked human populations to complex feedback loops with natural ecosystems, the information load generated by society increases. For market systems, the price mechanism will have to become more complex and multi-dimensional to cope. To the point that transactions need to automated because humans can no longer calculate them. (This is already happening.)

As long as complexity is increasing, a society that doesn't adopt higher and higher throughput modes of organisation (and abandon lossy systems or slower processing systems) will drown in information. Eventually negative externalities (and, often lack of recognition of positive externalities) overwhelm the system. The stagnant society starts to fail. It suffers from information overload which leads to increasingly unpredictable natural, technological and social systems. (Think political deadlock, financial systems failing, etc, etc.)

Note I said scale and complexity go together, I didn't say centralisation went with them. I think the post-bureaucratic age will be when decentralised systems start to emerge to replace overwhelmed centralised bureaucracies. I think this will probably start to happen during the next major financial crisis.

Here's Rao's talk:

"Temporal Legibility and Organizational Robustness":

Well worth a look.

The New Explorers

Friday, July 24th 2015

The next big form of entertainment will be generated. Generative worlds will create new universes and game mechanics.

They will generate passive stories (movies), universes to explore (games), virtual agents (generated personas), and so on.

The distinction between reality and generated reality will gradually blur.

Generated worlds will be windows into the software of the future.

Many new forms of work will be as various types of explorers of generated universes.

(It's a distinct possibility that we're already in one of those universes.)

We're at the bottom of page 5.

Click a page number above to go to that page.