Cosma Shalizi hosts a discussion

on The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies
by Scott E. Page at FDL FDL Book Salon Welcomes Scott Page: The Difference. From Cosma Shalizi's opening remarks:
"The problems we face in the real world are hard. There are always an immense array of variables which might be relevant, and an at least equally huge number of ways we might respond to them, plus overlapping constraints on our actions. Moves which seem to help part of the problem typically screw up others, because everything is interdependent, everything is connected to everything else. Even when problems are (more or less artificially) tidied up into well-defined mathematical tasks of locate-the-optimum, it is, generically, quite intractable to actually find that One Best Answer.

The Difference draws on about half a century of research into how to deal with complex, intractable problems. Faced with an overwhelming array of considerations that could be relevant, people assume perspectives from which some variables are visible and the rest are hidden. Within those perspectives, they apply heuristics, approximate rules for getting from where you are now to someplace that is usually better, though not guaranteed to be the best. A heuristic for a problem will do better or worse on different instances of the problem, and different heuristics will do better or worse on the same instance. One heuristic may do better than another on most instances, which makes the former stronger; a weak heuristic does only a little better, on average, than a random guess.

The key insight is that a single strong heuristic will do worse than a collective of individually weak but diverse heuristics. The problems are too hard for any one heuristic to solve perfectly, but the diverse heuristics can, so to speak, cover each others' weaknesses and help each other out when they get stuck; a single strong heuristic can't. A collection of diverse strong heuristics would be even better, but the strong heuristics for a problem tend to be similar to each other, so a group of them lacks diversity. In problem solving and prediction, diversity is exactly as important as individual ability.

This is the answer to "why do we need diverse X-ers instead of just the best X-ers?": if there's going to be one person who does X and they'll solve all their problems by their lonesome, sure, go for the one who's individually best at it, the one with the strongest heuristics. Meanwhile, on Planet Earth, everyone works in teams and groups and networks, and even if all you want is the very best job of X-ing possible, you have no reason to ignore diversity and every reason to seek it out, foster it, and make sure that all voices are heard and aren't just token presences.

I doubt it's escaped the reader that the problems facing progressives (how to we get universal health care? how do we keep our country from getting into more stupid wars?) are full of the kind of difficulty and complexity I described. Which means it's no good hoping for an inspired leader to figure out how to do them, or even a small clique of super-geniuses... There are lots of us, from many backgrounds, and we're self-selected to agree, mostly, on what we want to happen. So there should ways to use our internal diversity, our range of perspectives and approaches, to better accomplish progressive goals. But before we can do that we need to understand how and when diversity improves problem-solving capacity, and we really need to think about the ways in which we can bring that diversity to bear on our common problems — which is the whole "better societies" part of the book."
And much discussion. There is much about the sort of problems that I have been wondering about and I hope to read the rest of the discussion over the next few days.(Via 3quarksdaily)