Tomas Chamorro-Premuzic, chief talent scientist at Manpower Group, examines the myths about masculine management and makes the case for talent over gender.

When New Zealand Prime Minister Jacinda Arden took office in late 2017, she made worldwide headlines for being the youngest female head of state and for giving birth to a daughter while executing her duties. She grabbed global attention again in March for her response to a mass shooting that killed 50 Muslim worshippers at a mosque in Christchurch. Pundits and political observers alike praised Arden for her compassion and strength following the tragedy, and her ability to effect immediate change in the nation’s gun laws.

Her actions re-ignited a longstanding debate: Are women better leaders than men? The answer is often a “yes,” according to Tomas Chamorro-Premuzic, chief talent scientist at Manpower Group and professor of business psychology at University College London and Columbia University. His new book, Why Do So Many Incompetent Men Become Leaders? (And How to Fix It), explores the fallacy of confidence. A man who displays a confident, charming personality can mask incompetence in his job, yet there is scientific evidence that personality traits often associated with women are the building blocks of better leadership. Chamorro-Premuzic joined the Knowledge at Wharton radio show on SiriusXM  to talk about his book and why he thinks German Chancellor Angela Merkel is a great example of a good leader.

An edited transcript of the conversation follows.

Knowledge at Wharton: How prevalent is the incompetent boss?

Tomas Chamorro-Premuzic: It’s very prevalent, certainly much more common than people think. I used to bore people with data and statistics. Now I recommend a much quicker way to find out about the prevalence of incompetent managers, which is to just to go to Google and type, “My boss is” or “My manager is,” and see what people think of their leaders. “Crazy,” “abusive,” “unbearable,” “toxic” and a lot of other things that are too rude to repeat here come up. We also know from engagement surveys that only about 30% of people like their jobs. Most people who dislike their job and leave organizations do so because of a direct line manager or supervisor. People sometimes decide to leave traditional employment altogether and become entrepreneurs or work for themselves because they have been traumatized by their previous boss.

Incompetence is the norm when it comes to leadership, and we also know that masculinity and being male is a norm as well. My book explores whether these two things are connected, whether the fact that most leaders are males and most leaders are incompetent are causally linked or not. And I do find some causal connections.

“What’s somewhat peculiar is that there are slight advantages that favor women rather than men, yet there is this disproportionate overrepresentation of men in leadership positions.”

Knowledge at Wharton: As we see more female leaders taking executive roles, will some of this problem dissipate on its own?

Chamorro-Premuzic: No, because if the current rules of the game don’t change and we focus on gender as opposed to talent or potential, then even though the number of biological females that might occupy leadership positions will increase, dispositionally from a style/personality perspective they will be hyper-masculine. They will out-male males in masculinity, and they will still show these reckless, narcissistic, bold, overconfident and even deluded tendencies that are responsible for poor levels of leadership when the leaders are men, as opposed to women.

Knowledge at Wharton: You’re saying that focusing on the role of the leader is important?

Chamorro-Premuzic: We have to understand that leaders are responsible for creating culture. Culture doesn’t just emerge out of the blue. When companies have a toxic, counterproductive, low-performance culture, that is a direct result of the behaviors and values of the leader.

There are decades of systematic research as to what good leaders look like. They are, on average, more competent, more humble, more self-aware, more coachable, they have better people skills, and they have more integrity. Even though most people agree theoretically with these qualities or competencies as being pivotal to leadership effectiveness, we keep on selecting leaders with a very different profile. They tend to be confident rather than competent. They tend to be charismatic rather than humble. And they are often narcissistic, as opposed to having integrity.

Knowledge at Wharton: Are employees not doing enough to call out these situations when they occur, or is it the fear of calling out the boss?

Chamorro-Premuzic: Yes, absolutely. It makes sense that this is the case because everybody advances their career based on managing up. This is particularly problematic with leaders because leadership should be about managing down and turning a group of individuals into high-performing teams. When leaders are more focused on politicking, on sucking up to their bosses or managing up, they get rewarded. The same goes for employees. We often ask employees in our research, how many of you feel that you can provide your boss with honest, candid, critical feedback on their performance? Can you critique them? Only a minority of people say yes. Those people are probably about to lose their jobs because even though leaders should be evaluated by their followers and direct reports, their direct reports and followers get rewarded if they actually lie to their leaders and suck up to them.

Knowledge at Wharton: In the book, you bring up the fact that men generally think they are smarter than women.

Chamorro-Premuzic: Yes, and it’s important to understand that everyone is over-confident to begin with. There is a general tendency for humans to be over-confident, to be overly optimistic. If we could choose between a painful, maybe even traumatic and hard-to-digest reality check or a distorted view of reality that makes us feel better about ourselves, most of us would choose the latter. But this tendency to self-deceive is particularly striking in men, more so than in women.

Humans are over-confident in general, but men are more over-confident than women and get rewarded more for being over-confident. If you are unjustifiably pleased with yourself, unaware of your limitations and come across as over-confident — but you are a man — people would be more likely to say, “Wow, this person is leadership material. Let’s hire this guy. He seems to know what he’s talking about.” If displays of over-confidence are detected or perceived in women, we call those individuals “pathologically ambitious,” “bulldozers,” and we are afraid of them. What we should do is not to encourage women to be as over-confident as men, but to correct our standards and our evaluation criteria so we actually focus on competence rather than over-confidence.

Knowledge at Wharton: Does this perception also affect how the board of directors is brought together?

Chamorro-Premuzic: Yes, everything starts at the top and trickles down. If you want to diagnose the problem and especially change the problem, it starts at the top. I remember a story from David Ogilvy, the big advertising tycoon of the 1960s who wrote the book Confessions of an Advertising Man. He was the founder, chair and CEO of the company and said that the only on-boarding process whenever they recruited someone was to give that individual a [Russian nesting doll]. He would say, “If you hire people who are smaller than you, we will become a company of dwarfs. If you hire people who are bigger than you, we will become a company of giants.” If people are incompetent at the top, then that only snowballs down, and you have not just people who are worse and worse, but also toxic cultures that make things worse even for people who are talented and well-intended.

“…  Companies that are more meritocratic, less nepotistic, less political and more data-driven in their talent identification processes end up with leaders with higher IQ and EQ, and they out-perform their competitors.”

Knowledge at Wharton: Why aren’t we seeing enough combining of IQ and EQ, or the emotional intelligence that you talk about?

Chamorro-Premuzic: We have to understand that we do see it, because this isn’t an issue of whether companies are getting it right or wrong, but of how well companies are doing talent management, particularly leadership selection and development.

The evidence, both from independent, scientific studies and from studies exploring the drivers of effective corporations and business resolve, shows that companies that are more meritocratic, less nepotistic, less political and more data-driven in their talent identification processes end up with leaders with higher IQ and EQ, and they out-perform their competitors. They have higher revenues, higher profits, higher market cap. Their staffs are more engaged. They have higher net promoter scores. They have higher productivity and innovation indices. There is no secret; the ROI is there. Some companies understand it better than others and are better able to apply the science to their practices.

Knowledge at Wharton: Isn’t EQ also a factor in leaders who are honest and ethical?

Chamorro-Premuzic: Absolutely. I think these things are also somewhat cyclical. We have spent decades selecting leaders with very over-confident personalities who are likely to indulge in reckless risks, who are very focused on results and maybe have a lot of technological expertise but neglect things like empathy, humility, integrity. I think that’s why there is a premium today for leaders who do display these more feminine features, who are able to connect with their followers, who are able to mentor others, who are able to put themselves aside and care about the interests of their teams and followers first.

By the way, the more artificial intelligence will automate the kind of algorithmic aspects of leadership and things like processing and managing data, the more we will require leaders with high EQ, because empathy is the last frontier for AI.

Knowledge at Wharton: Is there a generational component to this? As the millennial cohort takes over management, the traditional thoughts around leadership are changing.

Chamorro-Premuzic: I have mixed views. I think the evidence is more nuanced than the media often suggests. First, there’s not that much data to look at generations. A lot of these data report simply on age differences, and we know that younger people are typically more curious, more open-minded, less hierarchical and more anti-authoritarian than older people. But we don’t know if these millennials, who are already in management and leadership positions, won’t become as conservative, traditional and hierarchical as their parents or grandparents as they grow older.

On the other hand, there is some good data suggesting that narcissism levels have been rising for about seven or eight decades, and that is generational. It’s not something that we need to blame millennials for. If the trend continues, then the next generation will be more narcissistic, the one after that more narcissistic, and in 50 years’ time, when we look back at Kim Kardashian or Kanye West, we will look at them as examples of very humble, modest people.

Just look at the typical advice for people who want to be leaders: “Don’t worry about what people think of you. If you think you’re great, you are. Just focus on yourself. You can be anything you want if you dream it,” etc. All those things promote a self-centered, narcissistic mindset. In contrast, we don’t hear things that we should hear if we’re giving people sensible advice: Care a lot about the reputation that you have. You should try to manage your image and your reputation so that others find you rewarding to deal with and that you are considerate, altruistic, pro-social. Clearly, no society can function if everybody is focused on themselves and everybody thinks they can do whatever they want because they’re entitled to do so.

Knowledge at Wharton: Do narcissistic leaders have the ability or the recognition to change?

“Incompetence is the norm when it comes to leadership, and we also know that masculinity and being male is a norm as well.”

Chamorro-Premuzic: I think that’s a very underrated quality in effective leaders or people with leadership potential. I could call it coachability, the ability to be curious, humble, self-critical; to pay attention to feedback and admit when you’re doing things wrong; to not be deluded or over-confident, and even feel a little bit uncomfortable about that gap between where you want to be and where you are. I think all exceptional achievers have an inherent inferiority complex or an impostor syndrome because they never believe in their own hype. They always want to keep going and achieve something that is for the greater good or progress or society. Fundamentally, they have this ability to remain dissatisfied with their accomplishments.

Knowledge at Wharton: One chapters is titled “Learning to Distrust Our Instincts.” What do you mean?

Chamorro-Premuzic: It speaks to the more complex nature of leadership today. I’m not comparing it to 10 or 50 years ago. I’m comparing it to 200,000 years ago, when leadership was very easy to observe directly. If somebody looks strong, brave, tall, fearless — and these people were usually men — we knew we should follow them and they would protect us from the threats.

Today, leadership is about a range of abstract, intellectual, even emotional skills and competencies that can’t be observed directly. But we still think that we can. Even smart, professionally skilled and sophisticated executives will tell you, if you ask them, “How do you know whether somebody has talent for leadership? “Oh, I know it when I see it. I just know.” The evidence shows that most of us are not as intuitive as we think. There is no excuse today for playing it by ear and not looking at data, not looking at evidence. There is a well-established science for identifying leadership potential. We should distrust our instincts and follow the data.

Knowledge at Wharton: What does a good leader look like today, and what will that leader look like in the future?

Chamorro-Premuzic: I always use the example of Angela Merkel because I think she epitomizes a lot of the qualities that, on the one hand, are very obviously relevant for leadership potential and, on the other hand, are so often neglected or not glamorous enough for us to pay attention to or remember.

First of all, they need to be competent. Secondly, they need to be quite objective, fair, unbiased, non-ideological. But then there’s a high degree of people skills, altruism, humility, coachability to the point that you can make non-ideological, non-emotional, non-impulsive decisions. If you think about the average high-performing manager today, they are mostly quite boring — like Angela Merkel. There are no thrills or scandals about them. They are very predictable. Their staff knows what to expect. If you couple that with integrity and competence, you get a high-performing manager, and these things are not lost at the higher level of leadership. People want to work for others who are not self-centered, who are competent and objective, and who are able to understand them and judge them objectively to get the best out of them.

Knowledge at Wharton: Are there situations where women leaders have a distinct advantage? Does it have to evaluated on individual-by-individual basis?

Chamorro-Premuzic: It’s always on an individual-by-individual basis. The gender discussion always gets people excited or passionate, but I argue for talent identification and the evaluation and leadership potential at the individual level. It does follow, when you measure something well at the individual level, that there are group differences. What’s somewhat peculiar is that there are slight advantages that favor women rather than men, yet there is this disproportionate overrepresentation of men in leadership positions.

“I think all exceptional achievers have an inherent inferiority complex or an impostor syndrome because they never believe in their own hype.”

Ultimately, what I’m arguing for is that the best gender diversity intervention is to focus on talent, rather than gender. If you are able to identify and select and develop individuals with the biggest talent or potential for leadership, you will not just have more women in leadership, but you also will have slightly more women than men in leadership.

Finally, I think a lot of men are unfairly overlooked or rejected or excluded from leadership identification programs today because they don’t match the traditional masculine archetypes of a leader that we have in mind. In a way, they are more feminine than we expect them to be. But that also means that they are more self-critical, more humble, more coachable and more self-aware. And we need leaders with these characteristics.

Knowledge at Wharton: In the end, you want that leader to have the most positive impact that he or she possibly can.

Chamorro-Premuzic: Exactly. The only way to judge whether somebody is a good leader or a bad leader is how they impact their teams, subordinates, followers, organizations, nations. It is the same at the level of political leaders. Yet we have a tendency to focus too much on the individual success that the leader attains. Which is why if you walk into the average organization and ask, “Who are your best leaders?” — immediately, they will point at the ones who are the most successful in terms of how much they earn, what titles they have. But that just means that they have managed up well. It usually doesn’t tell us what effect they have on their teams. We only do this well when it comes to professional sports teams where we compare different coaches or managers of football, basketball, baseball, soccer. There are very, very clear KPIs to judge how leaders affect their teams, and we should be doing the same in business.