Friday, May 23, 2008

Wisdom of Crowds

"The Wisdom of Crowds", by James Surowiecki, is an interesting book. In a mathematical sense, the wisdom of crowds can be (grossly) summarized as the elimination of error through the random exclusion of outlying, erroneous information, leaving only the coherent truth remaining. It describes some pretty amazing examples, but endless reruns of "Who Want to be a Millionaire" audience voting lifelines are perhaps the most accessible.

Is this phenomenon true for political issues as well?
What about mob behavior, or someone shouting 'fire!' in a theater?
The wisdom of crowds is best extracted under friendly circumstances.

It would be an interesting experiment to see under what circumstances the crowd returns a better answer than, say, a less direct government structure. The US democracy creates a decision layer which 'represents' a constituency, after being elected. Of course the US government has departed from the ideal, given gerimandering, and special interest lobbying. In any case, the indirect representation created by the country founders was always geared towards the preferential treatment of the landed gentry. The 'town meeting' format is a closer comparison, with more participation of common citizens. It would be interesting for an experimental town meeting to take a silent vote prior to discussion, and compare the resulting decision to the one arrived at after extended discussion.

From Surowiecki, "There are four key qualities that make a crowd smart. It needs to be diverse, so that people are bringing different pieces of information to the table. It needs to be decentralized, so that no one at the top is dictating the crowd's answer. It needs a way of summarizing people's opinions into one collective verdict. And the people in the crowd need to be independent, so that they pay attention mostly to their own information, and not worrying about what everyone around them thinks."

The internet can provide an avenue to satisfy these conditions, no question. What can defeat this potential wisdom?
"Essentially, any time most of the people in a group are biased in the same direction, it's probably not going to make good decisions. So when diverse opinions are either frozen out or squelched when they're voiced, groups tend to be dumb. And when people start paying too much attention to what others in the group think, that usually spells disaster, too."
Hmmm, sounds like groupthink frequently exhibited in politics, religion, and corporations!
A mindless conformity can yield some pretty outrageous results...

Line in the Sand

Read the book for a lot more detail and some incredible examples. At our best, humans have some amazing abilities.
Will the coming singularity empower this potential, or muffle it among a barrage of shared biased opinions?

Wisdom of Crowds