Forecasting in Conflicts: How to Predict What Your Opponent Will Do

J. Scott Armstrong is a marketing professor at Wharton and author of the  recent book, Principles of Forecasting. In the article that follows, he talks about his research into role-playing as a tool for leaders in both business and government who face crucial decisions in situations ranging from military clashes to marketing challenges.

How should the war on terrorism proceed? How will other countries respond? Since September 11, I have watched as military and political experts have described what we should do and what the consequences will be. Although these forecasts sound impressive and come with convincing arguments to support their conclusions, they are often wrong. This is not surprising; research tells us that experts are not good at forecasting decisions in conflict situations. The reason is that conflicts are complex and often involve several rounds of action and reaction. Fortunately, there is an effective alternative: role-playing. For conflict situations, research shows that role-playing yields the most accurate predictions.

Kesten Green, a colleague at Victoria University in New Zealand, and I have been studying how to make accurate predictions in conflict situations. We presented 290 participants with descriptions of six actual conflicts and asked them to select the most likely decisions. The conflicts involved labor-management, commercial and civil disputes. In five of these conflicts, we were able to role-play the interactions. When unaided, the participants did no better than mere chance; they were correct on 27% of the decisions. We then asked 21 game theorists from around the world to make predictions, reasoning that their greater understanding of conflicts and knowledge of game theory would produce better forecasts. Surprisingly, they were correct on only 28% of their decisions.

When we instructed 352 students to role-play, forecast accuracy improved in all five situations. On average, there were 61% correct predictions versus the 27% when similar participants made unaided predictions. I have been involved in forecasting since 1960 and have never before encountered a forecasting method that produces such large improvements over other procedures.

Role-playing can and should be used to simulate conflicts in a realistic manner. After receiving brief descriptions of their roles, participants read about the situation. The partisans meet with their confederates to discuss strategy and act out interactions with the other parties. While encouraged to improvise, they must always stay within their roles. Typically, 10 independent simulations will be sufficient, but more can be conducted if the decisions vary substantially across simulations. Predictions are based on the frequency with which decisions occur. For example, in our role-play of the 1982 conflict between National Football League players and owners, role-playing resulted in strikes 60% of the time. There was a strike.

Interestingly, neither instructing decision makers to think like their opponent nor giving them information about the roles of the parties improves accuracy. Role-playing must simulate the complex interactions.

Militaries have used role-playing since at least 1929. David Halberstam described its use in the Vietnam War in The Best and the Brightest. Unfortunately, top government officials did not believe the conclusion from the role-playing that moderate bombing was the worst strategy the U.S. could follow.

Better predictions of how other parties will respond can lead to better decisions. For example, role-playing might have led the United Kingdom to accept a 1975 offer from government-backed Argentine businessmen to buy most of the Falkland Island resources, and it might have helped the three Argentine generals to see how the U.K. leaders would respond to Argentina’s occupation of the islands.

Our studies have found that unaided expert judgment is generally relied upon when analyzing conflict situations in business. Although role-playing is seldom used in a formal way in business, we see a great potential. For example, it was used by the Lockheed Corporation to forecast the reactions of their major customers to proposed changes in the design of their planes; this allowed the company to examine various options before making a final design decision.

Role-paying is especially useful in that it can lead to predictions that are not obvious to experts. Consider the case of Contact Energy in New Zealand. Contact was formed in 1996 when the New Zealand government transferred some of the assets of its monopoly electricity generator to a new private sector company. The electricity market changed again in 1999 when the government split the residual into three new companies. In an effort to forecast the behaviour of the competitive market for wholesale electricity, Contact management organised electricity trading simulations and Contact executives role-played the managers of the rival generator companies in various company offices. However, because decisions suggested by the role-playing were so at odds with the beliefs held by Contact management about how the market would behave, the managers ignored them. The company then turned to game theory but found the exercise unhelpful. As it later turned out, the role-playing exercise had accurately foretold the future and the forecast had been ignored to the company’s detriment.