Systems for school assignment, immigration visa lotteries, and affirmative action programs more broadly have used reserve systems to help applicants from underrepresented or otherwise targeted groups. In these systems, some positions are open to all applicants, while others are reserved for the people targeted for affirmative action.

But new research co-authored by Wharton economics and public policy professor Alex Rees-Jones and co-authors reveals that a simple feature of reserve systems is often misunderstood. This can lead individuals to support policies that do not best advance their own goals, according to the paper titled “Reversing Reserves,” which was published in the journal Management Science.

“We view reserve systems as good tools, but like many tools, there are dangers associated with them,” Rees-Jones said. “Our work illustrates some of those dangers.”

The simple feature that is so often misunderstood is the importance of the order in which reserve seats are processed. Rees-Jones explains the issue like this: Imagine you are running a system with 100 seats to fill and 20 are saved for a targeted group. Also imagine each applicant gets in based on some measure of merit — for example, a test score.

Consider what happens if you fill the 20 reserve seats first. If they’re filled first, the seats go to the 20 highest-scoring members of the target group. If you would have admitted fewer than 20 of these students in the first place, this still helps members of that group a bit. But if you would have admitted more than 20 of these students without a reserve system, these reserved seats are filled by students who would have got in anyway.

If you fill the 20 reserve seats last, after the seats available to all are filled, the same 20 seats have a different impact. In the course of first filling the 80 seats that are open to all, many of the highest-scoring people in the target group can fill unreserved seats. Once you get to filling the 20 reserve seats, the 20 best students from the target group remaining are not those with the absolute highest scores. Instead, they are those with the highest scores who were not among the applicants who got in anyway. In this way, the reserve seats now give a boost to students just on the margin of admission.

“While it might initially feel like all that matters are the number of seats reserved,” Rees-Jones said, “simply switching the order that the seats are filled can significantly change how much affirmative action is achieved with those seats.”

“We view reserve systems as good tools, but like many tools, there are dangers associated with them.”— Alex Rees-Jones

Examples of Confusion in Reserve Systems

The paper was co-authored by Parag A. Pathak, an economics professor at Massachusetts Institute of Technology, and Tayfun Sӧnmez, an economics professor at Boston College. They wrote a previous paper on the unintended consequences of the Boston Public Schools’ choice program, which helped inform the current research. In the early 2000s, the Boston school system enacted a walk-zone reserve as a compromise between parents demanding unrestricted school choice and those demanding neighborhood schooling. The policy reserved 50% of seats at a school for students living within walking distance of that school. After more than a decade, the policy was rescinded when it became apparent that parents had been confused about the role of processing order.

“At the time the policy was put in place, it doesn’t appear that people realized that they needed to agree both on the number of seats and the order they’re filled. And my co-authors’ work showed clearly that saving 50% of seats and processing them first — which is what happened — provided very little boost to the admission of walk-zone applicants. When the policy went into place, I think many people didn’t realize that. It was perceived as more of a compromise,” Rees-Jones said.

“Reversing Reserves” also builds on a previous paper by Rees-Jones, Pathak, and Sӧnmez that examined reforms to the U.S. immigration lottery system, which holds a reserve cap for H-1B visas granted to people with advanced degrees. Following an executive order issued by President Donald Trump, the system was changed to process the advanced degree cap last, rather than first, in 2019. This change resulted in more advanced-degree holders getting a larger fraction of all H-1Bs.

“The reform in 2019 got a fair bit of attention and faced some public opposition, which was surely influenced by the Trump administration speaking on the issue. But there were other points in the last 20 years where the processing order of the reserve cap was flipped, and with bigger effects on who received visas. In those cases, the policy change received very little public attention. Once again, it looks like many members of the public miss the importance of this feature when they’re not given some help in noticing,” Rees-Jones said.

In “Reversing Reserves,” the co-authors sought to carefully test this theory. To do so, they conducted an experiment with more than 1,000 participants who were assigned a color, either blue or green, then asked to choose between a reserves-first or reserves-last policy that maximized their chances of admission.

“We find that participants rarely apply a decision rule that’s always in their best interests,” Rees-Jones said. “In contrast, we find that many choices — 40% in our primary estimates — are rationalized by a decision rule that treats more seats as better but that entirely ignores processing order.”

The scholars said there may be a number of reasons why people don’t choose correctly, but the most likely culprit is the difficulty with statistical reasoning. It’s well-established in economic research that random sampling makes sense to most people, but they aren’t so good at comprehending non-random or selected samples.

“It’s not hard to imagine making this mistake,” Rees-Jones said. “When things seem super simple, people might think there is nothing more to understand. Thinking `more seats is better’ sounds like a pretty compelling complete story, and one could see how someone could think there was nothing else to pay attention to.”

“When things seem super simple, people might think there is nothing more to understand.”— Alex Rees-Jones

The Objective Value of Better Market Design

The authors don’t take a political stance in their paper or advocate for or against reserve systems. Instead, they want policymakers and the public to understand the critical role of market design.

Better understanding of market design can help benevolent policymakers avoid unintended consequences, and it can prevent malevolent policymakers from duping the public into a program that is not as beneficial as promised. In fact, a key feature that sets the paper apart from other behavioral economics research is that it rejects the assumption that the market designer is always smart enough to see the flaws.

“The way the market is designed itself can be influenced, and that’s a pretty big break from the way that economists think about this,” Rees-Jones said.

He said he expects the adoption of reserve systems to increase, especially by companies and universities concerned about the demographics of their workforce. And with the U.S. Supreme Court expected to rule in June on race-based college admissions, the controversy over reserve systems doesn’t appear to be cooling.

“Just because a system is complicated does not mean you shouldn’t use it,” Rees-Jones said. “But it does make it really important that there is some understanding among the public about those complications. We hope that knowledge of these subtleties helps both policymakers and the public better assess the programs they want.”