September 06, 2009

Bruce Bueno de Mesquita: Learning the Methods for Making Accurate Predictions

Nextbigfuture previously looked at Bruce Bueno de Mesquita's predictions on Iran and his method of prediction. Bruce is a famous math professor who uses a computerized game theory model to make accurate political predictions.

More Predictions

The COP-15 (copenhagen Dec 2009) climate talks in Copenhagen will fail.

"The New Nostradamus article by Good Magazine

Though controversial in the academic world, Bueno de Mesquita and his model have proven quite popular in the private sector. In addition to his teaching responsibilities and consulting for the government, he also runs a successful private business, Mesquita & Roundell, with offices in Rockefeller Center. Advising some of the top companies in the country, he earns a tidy sum: Mesquita & Roundell’s minimum fee is $50,000 for a project that includes two issues. Most projects involve multiple issues. “I’m not selling my wisdom,” he says. “I’m selling a tool that can help them get better results. That tool is the model.”

“In the private sector, we deal with three areas: litigation, mergers and acquisitions, and regulation,” he says. “On average in litigation, we produce a settlement that is 40 percent better than what the attorneys think is the best that can be achieved.

“We have a corporate policy that we will not, on a commercial basis, use the model in campaigns,” he says. “We don’t think it’s appropriate to manipulate the democratic process. We won’t take a client who wants to manipulate U.S. government policy, even if we agree with the manipulation. And we won’t take a foreign client whose objectives are contrary to the objectives of the United States government.”

World governments are set to meet this December in Copenhagen to commit to firm CO2-reduction levels—but when Bueno de Mesquita modeled the future of these targets, most countries renege on them. No democratic government will seriously limit CO2 if it will hurt its citizens economically.

Predictions on the Final Compromise and Coalition
The New York Times provides some more particulars of the model.

Bruce Bueno de Mesquita has a spreadsheet included almost 90 players. Some were people, like the Iranian president, Mahmoud Ahmadinejad, and Supreme Leader Ali Khamenei; others were groups, like the U.N. Security Council and Iran’s “religious radicals.” Next to each player, a number represented one variable in Bueno de Mesquita’s model: the extent to which a player wanted Iran to have the ability to make nuclear weapons. The scale went from 0 to 200, with 0 being “no nuclear capacity at all” and 200 representing a test of a nuclear missile.

Amid the thousands of rows on the spreadsheet, there’s one called Forecast. It consists of a single number that represents the most likely consensus of all the players. It begins at 160 — bomb-making territory — but by next year settles at 118, where it doesn’t move much. “That’s the outcome,” Bueno de Mesquita said confidently, tapping the screen.

What does 118 mean? It means that Iran won’t make a nuclear bomb. By early 2010, according to the forecast, Iran will be at the brink of developing one, but then it will stop and go no further. If this computer model is right, all the dire portents we’ve seen in recent months — the brutal crackdown on protesters, the dubious confessions, Khamenei’s accusations of American subterfuge — are masking a tectonic shift. The moderates are winning, even if we cannot see that yet.

He has published a large number of startlingly precise predictions that turned out to be accurate, many of them in peer-reviewed academic journals. For example, five years before Ayatollah Khomeini died in 1989, Bueno de Mesquita predicted in the journal PS that Khomeini would be succeeded by Ali Khamenei (which he was), who himself would be succeeded by a then-less-well-known cleric named Akbar Hashemi Rafsanjani (which he may well be). Last year, he forecast when President Pervez Musharraf of Pakistan would be forced out of office and was accurate to within a month. In “The Predictioneer’s Game,” a book coming out next month that was written for a popular audience, Bueno de Mesquita offers dozens more stories of his forecasts.

Bueno de Mesquita was enthralled by the idea of rendering the messy business of politics and history into precise, logical equations. He began his signature academic work on “the selectorate,” or the group of actors who run a country. In Bueno de Mesquita’s worldview, there is no such thing as a “national interest” (or “state”). There are just leaders trying desperately to stay in power by building coalitions within their selectorate — buying off cronies in the case of a dictatorship, for example, or producing enough good works to keep hoi polloi happy in a democracy.

To predict how leaders will behave in a conflict, Bueno de Mesquita starts with a specific prediction he wants to make, then interviews four or five experts who know the situation well. He identifies the stakeholders who will exert pressure on the outcome (typically 20 or 30 players) and gets the experts to assign values to the stakeholders in four categories: What outcome do the players want? How hard will they work to get it? How much clout can they exert on others? How firm is their resolve? Each value is expressed as a number on its own arbitrary scale, like 0 to 200. (Sometimes Bueno de Mesquita skips the experts, simply reads newspaper and journal articles and generates his own list of players and numbers.) For example, in the case of Iran’s bomb, Bueno de Mesquita set Ahmadinejad’s preferred outcome at 180 and, on a scale of 0 to 100, his desire to get it at 90, his power at 5 and his resolve at 90.

If you merely sort the players according to how badly they want a bomb and how much support they have among others, you will end up with a reasonably good prediction. But the other variables enable the computer model to perform much more complicated assessments. In essence, it looks for possible groupings of players who would be willing to shift their positions toward one another if they thought that doing so would be to their advantage. The model begins by working out the average position of all the players — the “middle ground” that exerts a gravitational force on the whole negotiation. Then it compares each player with every other player, estimating whether one will be able to persuade or coerce the others to move toward its position, based on the power, resolve and positioning of everyone else.

After estimating how much or how little each player might budge, the software recalculates the middle ground, which shifts as the players move. A “round” is over; the software repeats the process, round after round. The game ends when players no longer move very much from round to round — this indicates they have compromised as much as they ever will. At that point, assuming no player with veto power had refused to compromise, the final average middle-ground position of all the players is the result — the official prediction of how the issue will resolve itself.

Theory talk interview of Bruce Buene de Mesquita from June 2009

The accuracy of a model such as my forecasting model provides a tool for IR scholars to use that may help us understand, explain, and predict events. It also provides policy analysts and decision makers with assessments that have transparent logic so they can argue with its conclusions, generally based on their own data.

One of the most common errors made by anti-rational choice analysts is to engage in post hoc, ergo propter hoc false reasoning. Let’s take the two examples offered in the question. Did Saddam Hussein act against his self-interest? Well, it certainly turned out badly for him but could he have known that ex ante when he had to make choices? I contend that the answer is no and I explain why.

First, we know now that things turned out badly for Saddam Hussein but neither we nor he could have known that before the fact when he had to make choices. Indeed, based on what he could know (such as the prior history of the United States government in dealing with him) he chose actions that were rational; that is consistent with what appear to have been his interests in survival. In the first Gulf War (1991), despite his army having been completely routed (and Colin Powell arguing to the Congress before that war that the United States would suffer perhaps tens of thousands of casualties and deaths), the US did not march on Baghdad and overthrow him or his government. Indeed, the Bush 41 administration did not even compel an unconditional surrender. Based on that experience, Saddam would have had solid reason to doubt the US government’s resolve to remove him from power. Second, we know now that he did not have WMD, but many thought he did before the 2003 invasion. It is quite possible that he thought he had WMD, we do not know. What we do know, is that some arguments against the 2003 war (which I opposed for other reasons at the time – namely that I saw no clear and present danger that would justify a pre-emptive attack by the US government) revolved around concern that there would be massive American casualties because Hussein was likely to use his WMD capacity (he had, after all, used nerve gas against the Iraqi Kurdish population and in 2003 American soldiers were deployed with anti-chemical weapons gear, apparently indicating that this was seen as a credible threat). Thus, by interfering with international inspections he was able to increase the belief at the time that he had WMD (see my textbook, third edition, for an explanation of the Bayesian updating calculations) and this could have deterred an American invasion. Thus, his actions seem to have been his best hope of political survival given that exile was not a good option (Bush was against it and, as the Pinochet experience surely taught Saddam Hussein, just because one is promised a secure exile does not mean that the promise will be kept – such promises lack credible commitment or enforcement). Once he knew that Bush 43 was serious and not bluffing it was too late for Saddam Hussein to extricate himself or to alter his earlier policies which had, after all, worked well for him for nearly a quarter of a century.

His Own June 2009 Assessment of The Feb 2009 Iran Prediction

The model so far has done rather well. I predicted that the Iranian presidential race would be close but that Ahmadinejad would win. We don’t actually know how close it was – we do know that the reported results are inconsistent with polling and with a reasonable statistical projection from the previous election. The modeling results also predicted that Ahmadinejad and Khamenei’s power was going to enter a period of significant decline even though Khamenei would remain a major political force probably until he retires. Clearly the events since the election indicate that Ahmadinejad and Khamenei have faced an unprecedented political challenge. We have to go back to the 1979 revolution to see something comparable so I think the evidence supports the prediction that they are entering a period of declining political power (which should not be confused with saying they will be ousted any time soon). Having gone back into my output to look at other details, I was intrigued to see (as a non-expert on Iran) that the model predicted a sharp increase in the political power of students and dissidents starting now and continuing for several months, although with fits and starts (I can send the graph if you like – it is just an excel plot of results produced on November 1, 2008 as I prepared my TED talk).

On the nuclear front, I am predicting that by around 2010 or early 2011 (the model is not as precise as I would like about timing; it is better at sequencing) there will be an agreement that limits Iran to producing small, research-quantities of weapons-grade fuel. I have not modeled the inspection regime that would be required to support such an agreement. It is worth noting that President Obama has acknowledged publicly that Iran has the right (under the NPT) to produce civilian nuclear energy (and, implicitly, to enrich uranium for this purpose), something apparently denied by the Bush administration. So Obama has moved the discussion forward toward the outcome predicted in my TED talk so as I see it things are moving as predicted both on the political influence front and on the nuclear front.

2007 Prediction of What North Korea Needed

Bueno de Mesquita’s model suggested: Kim agrees to dismantle his existing nuclear weapons but not his existing nuclear capability. “He puts it in mothballs with IAEA (International Atomic Energy Agency) inspectors on site 24 hours a day, 365 days a year. And in exchange, we provide him with $1.2 billion a year, which we label ‘foreign aid,’ of course.” The “foreign-aid” figure published in the newspapers was $400 million, which concerns Bueno de Mesquita. “I read that and I said, I hope that’s not the deal because it’s not enough money. He needs $1.2 billion, approximately, to sustain the loyalty of his cronies in the military and so forth.

How to use game theory to buy a car

Research the car carefully on the internet and decide exactly what you want.

Determine the colour, the extras, everything. Then, call every dealer within, say, a 20-mile radius.

When they answer, tell them exactly the car that you want. Then inform them that you are calling all the dealers in the area and asking about the same car.

You are going to buy the car at 5pm from the dealership offering you the best deal. You will ring back soon and seek a price — the full price, with nothing at all left to be added on later.

The dealer may object that if he gives you a quote over the phone, the next dealer will just come in £50 lower. You simply tell him that, yes, this might indeed happen.

That is why, you explain, he has to give you the very lowest price he humanly can, so as to avoid anyone underbidding with a price the dealer would have been willing to accept.

When the witching hour arrives, you go to the dealer with the best offer, cheque in hand, and pick up your car. If there is any change in the terms, you go to the second-best showroom, although this shouldn’t be necessary.

What has happened here? You have forced the salesman to provide you, in the form of his lowest price, all the information he has about the real cost of the car. The advantage has moved from the dealer to you.

List of publications by bueno de mesquita

2009 Publications:
Bueno de Mesquita B, Smith A
Political Survival and Endogenous Institutional Change
COMPARATIVE POLITICAL STUDIES. 2009 FEB; 42 (2): 167-197 1 1 2 26

Bueno de Mesquita B, Smith A
A Political Economy of Aid

blog comments powered by Disqus