In competitive sales environments, letting employees know how they rank compared to their colleagues is common. But does it actually affect the bottom line? According to a big data study performed by Wharton management professor Iwan Barankay on the motivational effect of employee rankings, the answer is yes. Barankay says these rankings absolutely do affect the bottom line and individual sales figures — but mostly in the wrong direction. His findings are detailed in the paper, “Rank Incentives: Evidence from a Randomized Workplace Experiment.”

In an interview on the Knowledge at Wharton show on Wharton Business Radio (SiriusXM channel 111), Barankay talked about the many ways companies try to incentivize higher performance and smarter choices from their workers, the pitfalls and positives to gamifying the workplace, and what he learned about how people respond to being told how they’re really doing.

An edited transcript of the conversation appears below.

Knowledge at Wharton: Let’s start with an overall view as to what your research project was, and what you were looking for.

Iwan Barankay: As someone who’s interested in how to shape the workplace environment — either by incentives or what information is available or what tasks to allocate to what people — there’s been a growing interest in looking beyond simple monitoring centers. We’ve started bringing — not just me, but the research community broadly — ideas from psychology, sociology and what people often refer to as behavioral economics.

There have been some principal forces behind why this has become so important. One is, many jobs are really complicated. They involve, for instance, a sense of mission or quality, so people often felt that addressing this with a bonus at the end of the year was not the right way to do it. It might lead to unintended consequences. People do what it takes to earn more money, but don’t necessarily work along the spirit of the mission. Another issue, however, is that companies just thought, “Well, you know, sure, I could offer people a bonus. But maybe there’s a cheap little trick here or there I could borrow from psychology. Maybe I put a smiley face on somebody’s door or tell them how great they are, or give them a little present of little monetary value but that conveys status and recognition.” So people look out for these little tricks. And either they are done to enhance the mission of the workplace or to find a cost effective alternative to monetary incentives. And there have been some remarkable success stories.

I think the one uncontestable success story was default options. People have a strong bias towards their initial position, and that’s difficult to overcome. Here, the principal example is pension savings. So in the past, when people joined a company, they would actually have to tick a box or fill out a form to say, “Yes, I want to enroll in the 401(k) plan.” Then companies said, “Well, why is it that so few people save money? Why don’t we just, by default, when they join us, we automatically enroll them.” They can choose to leave at no cost, they just have to fill out a form or call a number. What they found was that just simply changing this default substantially increased saving rates among people.

But going beyond that, the evidence is a bit more mixed. I think my colleagues working in organizational behavior, they have a lot to say on how to enhance a sense of mission and meaning in the workplace, to make people be more respected or recognized. There is some interesting research on that.

Another interest that people have had is, wow, we have all this data. We can now, at the click of a button, generate fancy reports that we can send to every employee at any point in time. So is there something we can do with all this data to, perhaps, inform the employees or make them work harder or focus their attention on something?

This has inspired the idea of gamification in the workplace. You have some little fun contests about new ideas, or perhaps who completes a project or acquires more clients more quickly. Another thing that can easily be done is to just spit out a report every week or every month on how you perform in the company compared to others. Along these lines, I’ve been teaming up with a very large office furniture company here in the U.S. where we have been looking at this matter.

Knowledge at Wharton: There are some types of jobs, such as sales, that are obviously the most affected by that sort of data. The quota always comes up when you’re talking about sales: If you’re not meeting your quota, there’s a chance that you’re going to lose your job.

Barankay: Yes, that is absolutely true. And actually many companies resolve this very, very simply by basically giving all the risk to the employees, in the way you mentioned. They pay them sometimes no base salary or a very, very small base salary. Their income is entirely based on commissions. That is what this [office furniture] company has been doing for a long time. People are paid a percentage cut of the dollar value of their sales, And this is how they are making their money. They have been using this for a long time and very successfully. Clearly, some people do better at that than others, so there’s a big variation, but this is how they have been addressing this issue.

Why do we pay them by commission? Because, you know, it is actually a relatively tough job. You have to look after clients — figure out what budget they have, how to satisfy them. I can’t write a contact telling you what to do at every point in time, so the best thing to do is just to give you a reward for closing. And “always be closing,” as they say, ABC.

“We thought: This can’t be, everybody tells us social comparison is great and fun. There must be a group for which this is better.”

Knowledge at Wharton: So then with the research that you did with the furniture company, what approaches did you take in terms of how to present information — or not present it, — to the employees?

Barankay: As I started working with the company, we were first over-viewing their incentive system, which they basically had done quite well already. But then, one thing I noticed was that as the salespeople logged into the webpage where they record the sales they have made, and they can check whether they have correctly calculated their payouts — on the splash first page, they saw their name. And next to their name, they saw how they ranked in the U.S. in terms of year-to-date sales.

And I asked them about that. “Oh, this is interesting. Why do you do this?” And then they said — and this is what a lot of researchers invoke as well — that people have an urge to compare themselves to others. This is the social comparison idea.

If you start thinking about it more and more deeply — why do we do this? — part of the reason is not just because we have an urge for status and to be better than others, but because you can learn something from others. You see someone else doing a successful show, a radio show or podcast — what does it do to get so many listeners? And you compare. So you can learn a lot of information from that.

But another quite important urge is that if you show people how they rank, then this stimulates them to catch up, if perhaps they are falling behind on their rankings. Or if they’re doing well, that encourages them to work even harder, because it’s great to be ahead of the others. This is the overwhelming intuition of many people in C suites — that these rankings are great because they instill a healthy sense of competition. They make the job perhaps more fun. And you know, if a job is really boring and terrible, then perhaps gamifying it this way could be a way to do it. But there are other ways, perhaps, to achieve this as well.

Knowledge at Wharton: But as you found out, there were lots of negative effects when you take some of those approaches, because people’s emotions get put into play in this formula. And obviously, if they’re struggling, that heaps another level of pressure on them that you don’t want to have.

Barankay: Yes. Just to clarify, what I’ve been doing is not a tournament in the sense of, if you’re number one you’ll get $1 million or something like that. They were already paid for performance. The only thing that we varied is to tell them how they ranked.

Think about it: Most people, if you ask them, “Are you a good driver?” 90% of them would say, “Yes.” [If we asked,] “Do you think you’re better than the average driver?” 95% of people would say they’re better than the average. Now this can’t possibly be the case. People have a healthy predisposition to be optimistic about their performance, and that almost has substantial biological routes because life is tough and it’s helpful to be optimistic.

But if you give people hard feedback, by definition, many people will get disappointed. If I tell you, “Good job. You are just in the top 50% of the distribution, and you’re not in the top 10% as you thought you were,” this is disappointing news. Also, if people are very optimistic and you actually tell them, “Oh, you’re doing great,” then perhaps they become complacent.

“People do what it takes to earn more money, but don’t necessarily work along the spirit of the mission.”

Knowledge at Wharton: Slack off a little bit. Yeah.

Barankay: Right. [They think,] “I’m already No 1. Why try harder?” This complacency may kick in. And this is actually what we found when we did this experiment. And by experiment, I really mean it in the scientific way. We randomized salespeople. Some of them continued to see their rankings. For others, all we did was not to display the ranking next to their name. This was a purely random thing. We did not change anything else.

Once we removed these rankings, they actually sold significantly more furniture afterwards. And the reason for that is people didn’t “check out” as much. When they saw their ranking and they didn’t do well, they thought, perhaps this is not what I should be doing. Or they gave up. Or they became complacent. But when we looked deeper into the result, what was interesting as well is that we found a way to understand whether people exceeded their expectations or fell short of their expectations. We did this because we saw the overall improvement in performance when we no longer showed the ranking. And we thought: This can’t be, everybody tells us social comparison is great and fun. There must be a group for which this is better. What about the top people? What about people who get a positive update, who learn that they’re doing better than they expected?

But what we found is that people who did better than expected showed no change in their behavior. This is the complacency effect, almost where they say, ”Well, you’re just confirming to me that I’m great, but I knew this all along.” So there was nothing new coming to them. But for the people who fell short of their expectations who performed less well than before, these are the people who really drove the results down. So this gives some pause about whether companies should show rankings to their employees.

Other studies basically did what amounts to a tournament. They had some stores compete against each other, and the employees could earn some money. And they did this, showed them how they were doing, in part by showing posters to them. But in my study, it was just private feedback, not a public display of ranking. Private feedback to people and they were free to talk about it.

But so many interesting things happened about this. For instance, we were contacted by some salespeople from all over the states and from all over the distribution: top guys, medium guys, bottom guys. And they said, “You know, I used to see my ranking and now I no longer do. And this really motivated me in the past.” But then I looked into the data of these people who contacted us, and actually they sold more when we no longer showed them the ranking. So their personal recollection of whether they did well or not did not actually square up with the data.

Knowledge at Wharton: In some respects, when you don’t know the ranking it just focuses you on sales and not where you are in the grand scheme. And for the company, when you don’t provide those rankings, it does provide maybe a little bit more of a team aspect to the entire sales crew.

Barankay: That is very true. And I think this is where the limit of the competitive spirit lies — when these incentives or the competition interfere with the team spirit and interfere with an efficient allocation of information or effort. When people are better, when they work together as a team, you don’t want to have a competition between team members, so this is very, very important.

On many levels, sometimes we have a sense of protecting ourselves against bad information. We do not necessarily expose ourselves to situations we know would be unpleasant or disappointing. For example, my son loves to play chess. He finds it very entertaining and very fun and he loves to go to chess tournaments. I’ve also played chess a little, but I don’t have a chess brain. So every time I play chess, I feel like a complete loser. And therefore, I don’t crave going to chess tournaments because I know if I went there, people would just be laughing at me. So I choose not to participate in those tournaments — although I enjoy doing some chess problems.

“When people are better, when they work together as a team, you don’t want to have a competition between team members.”

Knowledge at Wharton: What was the reaction of the company to all of this data? Did they want to make changes in terms of the information they provide to their employees? Did it make enough of an impression to change their philosophy about how they run their business?

Barankay: Yes. This was really a fascinating interaction with this company. This company is, in a sense, like Google: They always try to experiment with innovations. And they went with it because, for one thing, their sales people demanded this information, or at least the vocal ones wanted to hear their rankings. And they saw other companies do it. So they thought, “Well, if others do it and people ask for it, it must be right or it must work well.” Then we did this experiment and showed them: You actually sell significantly more when you don’t show the rankings, and by significant, it was quite an improvement in their sales. So now they no longer show the rankings.

Knowledge at Wharton: If they can improve performance without having to significantly increase the payroll, then they’ve reached, in some respects a level of nirvana because they’re getting better performance from all of their people and they’re getting better profits without having to do anything to the cost level?

Barankay: Yes. This is absolutely right. But I would also encourage all these companies that consider this to do it in a more thorough and systematic way. And understand that their circumstances might be different from what they see elsewhere. This is why in my research, I always use ways to pilot or experiment, meaning you take either the whole company or a part of it and then randomly change something so you can understand whether this innovation is helpful or not before you roll it out to the whole company.

There are always health apps that try to use some insight or some idea from behavioral economics to help us form better habits and lose weight without any consequence or cost. And maybe it works. For anyone for whom this actually was a success, good for you guys. But for a company, they should not be so quick at adapting these changes.

Knowledge at Wharton: I’m guessing that because of the nature of the type of company we’re talking about, where the sales element is such a very important piece to this, that this would play out with, say, the auto industry. Because, in some respects, it’s the same type of business model.

Barankay: Yes. Wherever you can easily measure performance and compare to each other. Of course there are variations. Some markets are just deeper and there are more sales opportunities. If you’re in some remote location, you have less opportunities to service customers than when you’re in Manhattan. So many times, these rankings can be unfair. But people can adjust this. If you know that you’re in a remote location, you can adjust to that.

Knowledge at Wharton: And your study went over a couple of years, correct?

Barankay: Yes, it went over three years.

Knowledge at Wharton: So there was really no seasonality to it in terms of the sales. If you’re talking about furniture, people may be buying furniture in the spring more so because the weather is nice and they want to make changes to their house, that type of thing.

Barankay: Yeah. And that is the beauty of doing a randomized control trial like I did, where I basically was able to compare the growth in sales with people where we made a change, compared to those where we did not make a change. This allows us to adjust for all seasonal variations as well, in addition to anything that is, perhaps, specific to a market or to a person, because we’re just looking at the growth of sales.

Knowledge at Wharton: As this research becomes more public and maybe companies change their policies in light of it, how do you think this will affect employees in general? Because I remember being in that type of industry many, many years ago, and it is part of your nature to want to know how well you are doing. People love the fact that they may be the top salesperson in that particular office. And, obviously, they don’t like being told if they’re toward the bottom part of that room. That’s a hard thing to break, just on people’s natural intentions.

Barankay: Yes, that is absolutely true. I think it’s important to remember that it is very easy to give somebody a piece of information, but almost impossible to take that information back. You can’t make people un-know something that they were already told.

Knowledge at Wharton: Once it’s out there, it’s done.

Barankay: Yeah. And this is why it’s useful to be cautious. Apart from some other discriminatory or legal situations and implications, it’s useful to think before you divulge information like this: Is this really helpful? This is why you should really test these things out. But you should not just talk to some of the employees whom you trust, whose opinions may matter, because these may be just employees who are more vocal than others. You might actually miss out on the reaction of the median employee, or even of the more silent but top performers. And what you really don’t want to do in a company is to annoy the top performers, because in most companies, it may not be pleasant information for most, but you know, the top 10% or 20% of employees, they generate 80% of the revenue. It is so unequal. And that was true in this company or in this industry. So alienating any of these people with information that might upset them, they’ll just quit and go elsewhere where it’s a more pleasant environment. You have to therefore be very careful about what information to divulge.

“I think it’s important to remember that it is very easy to give somebody a piece of information, but almost impossible to take that information back.”

Knowledge at Wharton: Do you think we will see more of a trend of companies wanting to adopt this type of philosophy in terms of providing information to their employees, maybe not doing it as much as they have over the last several decades?

Barankay: This is a very important trend. And it’s actually particularly strong in the health care domain. I’ve been approached, but also I’ve been talking to a lot of physicians who say “My hospital or my head of department or my chief is actually sending me an e-mail every week or every month, telling me how I perform in terms of benchmark numbers.” Partly this is because it’s tied to the reporting requirements that hospitals now have to the government. But this is becoming very routine and very common. And again, this may lead to some unintended consequences among physicians if they know, well, if I take on this high risk patient then perhaps my benchmark numbers will deteriorate. So I’d rather have one of my junior colleagues to take on this patient so that his numbers suffer and not mine. These unintended consequences are really dangerous, and therefore, again, we should be careful about this information.

Knowledge at Wharton: Especially if you’re talking about passing it down to the junior colleague who doesn’t have your experience level and may not perform in the same manner that you would.

Barankay: Yes, exactly. If you look at cardiac surgeons, if you look at the numbers of cardiac surgeons who have the highest mortality rate, often you will see it’s the most senior guy. But that is because he gets the most difficult patients to operate on. So it is a poor idea to think that just looking at these numbers gives a fair picture, because these patients or these cases are not allocated by chance, but really allocated knowing that the most senior person is more capable at the operating table.

Knowledge at Wharton: So it sounds like we know what is the next step for you in terms of this realm, that maybe to look to see how the effect happens in the medical community.

Barankay: Yes, that’s the next goal and that’s what I’m working on at the moment as well.