In business, forecasting is usually used to predict number-oriented outcomes, such as future sales, profits or economic growth. Efforts to predict the results of more complex situations, such as labor negotiations, are exceedingly difficult. Just how do you forecast situations where the parties’ actions and reactions pose a profusion of possible permutations?
Efforts to forecast the outcomes of situations involving conflict include the use of “expert” opinions or “intention” surveys, in which participants are asked how they would act in a certain situation. For a variety of reasons, however, the jury is still out on how accurate these techniques are at predicting actual human behavior.
Extensive research conducted since the mid-1970s by Wharton’s J. Scott Armstrong suggests an alternative. Armstrong provides strong evidence that forecasts based on the results of role-playing sessions can make accurate predictions of human responses to conflict or change.
In role-playing, researchers ask study subjects to play the part of decision-makers in certain situations. For instance, a research team trying to forecast the effect of a price increase on sales of an industrial product might ask subjects to play the role of corporate buyers. After being tutored in the objectives and function of the part they are playing, the “buyers” might then be asked to describe how they might react to a price hike for the product in question. The research team then uses those responses to predict the effect of an actual price hike.
The accuracy of role-playing in predicting decisions has been confirmed in several applications. For instance, it has been used to help predict jury verdicts and to forecast the presidential nominee of political parties.
Role-playing has also been used as a forecasting tool in military applications, as when high-ranking U.S. officers used role-playing during the Vietnam war to test the strategy of bombing North Vietnam. (Role-play showed that no bombing would achieve superior results to either limited or full-scale bombing. Unfortunately, the Administration at the time ignored that prediction.)
Armstrong cites his personal experience from five role-playing situations to show that the technique is more accurate than expert opinions for predicting groups’ responses to important changes or conflicts. Each role-play reenacted recent events that had actually confronted various organizations, whose identities were disguised so that the role-players would not recognize them.
In each exercise, role-players were randomly assigned to play the part of two factions involved in a conflict or major change. They were given enough background information to simulate the original situation as closely as possible and to help each party understand its position. The parties’ decisions were then compared to the outcomes of the actual events.
The situations the role-players were asked to emulate were:
1. The board of directors of a pharmaceutical manufacturer met to consider a request by the U.S. Food and Drug Administration to discontinue marketing a drug that the agency suspected was unsafe. Of role-players acting as board members, 76% decided to reject the government’s request and to take a variety of aggressive efforts to prevent the FDA from banning the drug. That decision was, in fact, the actual board’s final ruling. Among “experts” polled by the research team, however, only 34% correctly predicted the actual board’s response.
2. An appliance corporation in the early 1960s endeavored to convince a major supermarket chain to sell its products..In role-play, supermarket representatives accepted the company’s plan 75% of the time, while only 3% of those offering expert opinions predicted that result. In fact, the supermarket chain had accepted the offer.
3. In 1982, contract negotiations between the players’ union and NFL owners failed to reach agreement. Role-players (again, unaware of the actual outcome) reached the same conclusion 60% of the time. Only 27% of 15 “experts” predicted that outcome.
4. Dutch artists staged a sit-in a museum seeking government subsidies. Out of more than 500 possible outcomes, 29% of role-players predicted what actually had happened: The government gave in to the demands. Only 3% of “experts” forecast that outcome.
5. In contract negotiations between contributing editors of an academic journal and the journal’s management, the editors ultimately failed to secure better payment terms. Among role-players, 42% predicted that outcome, compared to 12% of “experts.”
All told, role-playing accurately predicted decision-making outcomes 56% of the time, compared to 16% for opinions.
Other studies confirm those findings. In making hiring decisions, for example, researchers found that role-playing is consistency superior to more traditional methods of predicting the success of new doctors, military recruiters and life-insurance salespeople.
Role-playing can be expensive compared to some other techniques for predicting decisions. Moreover, its validity compared to some other predictive methods such as game theory has yet to be tested. But compared to the cost of business failures, its potential for savings may be enormous. It can only be imagined whether role-playing could have helped avert some expensive marketing faux pas of the past, such as Coca-Cola’s red-faced retreat from its formula change in the early 1980s.
