Intelligence In Numbers

That crowds are dumb is a given – an immutable truth that's not worth arguing about. Meetings are to be sneered at, group-think deserves contempt, committee and procrastination are synonyms. As Nietzsche puts it, “Madness is the exception in indiviudals but the rule in groups.” And Thoreau: “The mass never comes up to the standard of its best member, but on the contrary degrades itself to a level with the lowest.” (etcetera: We are good at googling).

The smartest (ok, ok, funniest) philosopher of them all, “Dave Barry sums up our collective feelings in one (slightly long) line, “If you had to identify, in one word, the reason why the human race has not achieved, and never will achieve, its full potential, that word would be meetings. When on quotes, here's another one: “Anyone taken as an individual is tolerably sensible and reasonable – as a member of a crowd, he at once becomes a blockhead.” – Friedrich Schiller, through Amit Varma.

New Yorker columnist James Surowiecki wanted to prove Barry wrong, and wrote a whole book about the value of meetings. Ok, half a book maybe. Seriously though, Surowiecki's erudite, well-researched, and interesting book provocatively takes on a widely held notion – that individuals are better decision makers than groups – and proves how wrong it is. Not conclusively, but there's enough empirical evidence in the book to force a rethink. The book was an instant bestseller and won a lot of acclaim – but most importantly it is an enjoyable read.

So, why are groups good? A long time back, somewere in England, a philosopher decided to run an experiment involving a few hundred men and a dead ox. The philosoper strongly believed that most people were dorks, and that crowds of dorks were dorkier and this experiment was going to be conclusive proof of his belief. The dead ox was placed in a shandy , and people were asked to guess the weight of the ox. At the end of the day, the estimates were collected and averaged. The expected outcome was that the mean guess would be far off, thus proving that crowds were stupid.

Unfortunately for the philosoper though, the mean of all the guesses was almost exactly equal to the weight of the ox, and much better than most individual guesses. In additon to thorougly confusing the philosopher, the experiment had a point to it: It was an early indicator of the idea that forms the basis of Surowiecki's book: A collective decision made by a group of people, each of whom thinks independently, will almost always be better than the best individual decision. This is not the same as consensus, which is the dumbing down of individual opinion to satisfy the group. You cannot find a better example of the Wisdom of Crowds than the Internet: Google, Linux, Firefox and Wikipedia are examples of decentralized mechanisms that have delivered outstanding results. The Blogosphere, in conjunction with Google's Pagerank is quite possibly the best example of a perfect democracy: the more people that like a blog, the more links it gets, increasing its relative importance and readership.

Drawing on examples from everyday life – stock market bubbles and traffic jams; the Colombia crash and overcrowded bars – Surowiecki paints a compelling picture of smart groups, and provides suggestions for possible ways to harness the potential. One of the methods discussed is a “decision market” like the Iowa Electronic Market – a stock market like system where you bet on a certain person winning an election. The accuracy of the IEM's predictions are better than even polls – which is surprising considering the small number of people that participate in it. Fascinating. By the way, what do you think was the most reliable lifeline on Who Wants to be a Millionaire?

Get a couple of people in a room. Give one of them $10, and ask him to share it with the other guy. If the other guy accepts his offer, they both get to keep the money, otherwise no money for either of them. Most of the time, the offer was$5 apparently, while people rejected lowball offers (anything less than $2), even though it was free money. Surowiecki attributes this to an innate sense of what constitutes fair that we all have – and uses it to explain why we vote in elections and why we tip at restaurants in distant towns that we know we are not going to go back.

The book has been accused of not being rigorous enough (as in providing statistics for why crowds are smarter), and sometimes you get the feeling that the author is hand waving his way to a pre-determined conclusion using anecdotes, but you know what, if you judge a book by the amount of pleasure you get from reading it, then this is a good book. I've not enjoyed non-fiction more (if you don't count Starry Nights, that is). Plus I now know all about the virtues of a decentralized, diverse corporate power structure and ways to run productive meetings.

Excerpts can be found here and here.