Sunday, August 16, 2009

Rational Fortune Telling?

Earlier Julie Rehmeyer has written about the work of Bruce Bueno de Mesquita:Mathematical Fortune-Telling :
"The New York University political science professor has developed a computerized game theory model that predicts the future of many business and political negotiations and also figures out ways to influence the outcome. Two independent evaluations, one by academics and one by the U.S. Central Intelligence Agency, have both shown that about 90 percent of his predictions have been accurate. Most recently, he has used his mathematical tools to offer approaches for handling the growing nuclear crisis with Iran." (mentioned in this blog :Synthesizing the wisdom of experts). Recently, there is a more detailed article in The New York Times:
Can Game Theory Predict When Iran Will Get the Bomb? (via 3quarksdaily). Some of it is a bit scary:
"For Bueno de Mesquita, the first prominent use of the model came in 1979, when the State Department was canvassing academics with expertise on India, including Bueno de Mesquita, to see how some parliamentary maneuverings would unfold. Bueno de Mesquita decided to use his first version of the software (which was, as he puts it, “barely working”) and his own knowledge of India to determine the power players and each of their numbers. Then the university’s mainframe computer worked on the data all night.

In the morning, Bueno de Mesquita said, he was astonished: the predicted victor was a seemingly minor figure, someone discounted by the experts. Bueno de Mesquita shared their opinion, he told me, but he accepted the computer’s verdict anyway. “So I called the person back at the State Department, and told him what I had concluded,” Bueno de Mesquita went on. “And there was a long, quiet period and some laughing. He said: ‘How did you arrive at that? Nobody’s saying that.’ So I told him I had a little computer model. He just guffawed. He said, ‘I wouldn’t repeat that if I were you.’ ”

Three months later, according to Bueno de Mesquita, his prediction turned out to be right....
Soon Bueno de Mesquita and Organski (who died in 1998) acquired clients ranging from Arthur Andersen to Union Carbide, which tapped them for advice on placating the Indian government after the Bhopal chemical spill. Today Bueno de Mesquita’s firm essentially consists of himself and Harry Roundell, a former banker at J. P. Morgan who met Bueno de Mesquita when Roundell hired him in 1995 to help the bank figure out how to push for new, favorable regulations in the U.S. They charge $50,000 and up to do a prediction and offer negotiating tips, and they take on 18 to 20 of these assignments a year. Beyond saying it was “a reasonable amount of money,” Bueno de Mesquita would not describe his income from the company."
However, there are doubts about his predictions:
"Stephen Walt, a Harvard professor of international affairs, says that Bueno de Mesquita’s nonprediction work — like his theory of the “political survival” of heads of state — make him a “respected scholar, deservedly so.” It’s the predictions that Walt doesn’t trust, because Bueno de Mesquita does not publish the actual computer code of his model. (Bueno de Mesquita cannot do so because his former firm owns the actual code, but he counters that he has outlined the math behind his model in enough academic papers and books for anyone to replicate something close to his work.) While Bueno de Mesquita has published many predictions in academic journals, the vast majority of his forecasts have been done in secret for corporate or government clients, where no independent academics can verify them. “We have no idea if he’s right 9 times out of 10, or 9 times out of a hundred, or 9 times out of a thousand,” Walt says. Walt also isn’t impressed by Stanley Feder’s C.I.A. study showing Bueno de Mesquita’s 90 percent hit rate. “It’s one midlevel C.I.A. bureaucrat saying, ‘This was a useful tool,’ ” Walt says. “It’s not like he’s got Brent Scowcroft saying, ‘Back in the Bush administration, we didn’t make a decision without consulting Bueno de Mesquita.’ ” Other academics point out that rational-actor theory has come under increasing criticism in recent years, as more evidence accumulates that people make many decisions irrationally."

No comments: