October 31, 2007

90% future prediction accuracy for Game theory model

Bruce Bueno de Mesquita has led a shift in political science toward quantitative models. Analyses of his model of political decision-making show that it has a 90 percent accuracy rate.

Like a 0.1 version of the fictional character Hari Seldon of the Asimov Foundation series.

The elements of the model are players standing in for the real-life people who influence a negotiation or decision. At each round of the game, players make proposals to one or more of the other players and reject or accept proposals made to them. Through this process, the players learn about one another and adapt their future proposals accordingly. Each player incurs a small cost for making a proposal. Once the accepted proposals are good enough that no player is willing to go to the trouble to make another proposal, the game ends. The accepted proposals are the predicted outcome.

To accommodate the vagaries of human nature, the players are cursed with divided souls. Although all the players want to get their own preferred policies adopted, they also want personal glory. Some players are policy-wonks who care only a little about glory, while others resemble egomaniacs for whom policies are secondary. Only the players themselves know how much they care about each of those goals. An important aspect of the negotiation process is that by seeing which proposals are accepted or rejected, players are able to figure out more about how much other players care about getting their preferred policy or getting the glory.

The main reason that the model generates more reliable predictions than experts do is that "the computer doesn't get bored, it doesn't get tired, and it doesn't forget," he says. In the analysis of nuclear technology development in Iran, for example, experts identified 80 relevant players. Because no individual can keep track of all the possible interactions between so many players, human analysts focus on five or six key players. The lesser players may not have a lot of power, Buena de Mesquita says, but they tend to be knowledgeable enough to influence how key decision-makers understand the issues. His model can keep track of those influences when a human can't.

"Given expert input of data for the variables for such a model, it would not surprise me in the least to see that it would perform well," says Branislav L. Slantchev, a political scientist and game theorist at the University of California at San Diego.

He points out that the model relies on having a considerable amount of expert input. "Honestly, if you had all this information," Slantchev says, "you should be able to predict fairly well how the issue would be resolved." The main reason that the model does this better than experts is that it "strips ideological blindfolds, cultural prejudice, and normative commitments that very often color the view of experts."

So this shows that by training oneself and learnnig what the proper inputs are in determining a future outcome and then rigorously reducing biases and focusing solely on an accurate assessment and prediction then an expert person could also achieve near 90% accuracy in predictions.