Unlike the corporate world, social enterprises traditionally have not been outcome driven. In part this is because it is often difficult to define and measure outcomes in these complex areas. However, in the data-driven world, focusing on results has become an imperative for every sector.
According to Trent Kaufman, CEO of management consulting firm Cicero Group, social enterprises are increasingly starting to discover that successful fund-raising is often dependent on the ability to show indicators and measures of success. Funders, he adds, are becoming savvy and want to see quantifiable outcomes.
In a conversation with Knowledge at Wharton and Rohan Valrani, a senior at the University of Pennsylvania, Kaufman discusses the challenges of developing and applying data-driven strategies in fields such as education and social impact.
An edited transcript of the conversation appears below.
Knowledge at Wharton: Could you give us an overview of how data-driven strategy can be applied to the education and social impact industries?
Trent Kaufman: At the broadest level, social impact and educational organizations are no different than other types of entities, including government, for-profit and so on. They strive to deliver impact to a set of end users or customers or clients, and they can always work to improve their ability to deliver that impact. How an organization goes about improving its impact varies quite a bit. Within the social sector, education, not-for-profit government entities, non-government organizations, foundations and philanthropies, use of data as a catalyst for improvement is on the rise. In the private sector, however, it has been part of the culture for a very long time.
The strategy around using data to improve social outcomes is complex. This is because there aren’t always bottom-line metrics that are easy to find, easy to collect, and clearly agreed upon by all stakeholders. That’s perhaps why the social sector is a few years behind the private sector. But that is also why it’s such an exciting field. It’s why I’m passionate about it. It starts with defining strategy and key metrics and agreeing upon those across a variety of stakeholders that are sometimes hard to coalesce and hard to bring together. It ends with an organization running smoothly with consistent and productive windows into their impact, leading and lagging indicators, and continuous improvement.
Knowledge at Wharton: Could you explain this with an example of how data-driven strategy has been used in the social impact or education field?
“The strategy around using data to improve social organizations is complex because there aren’t always bottom-line metrics that are easy to find, easy to collect, and clearly agreed upon by all stakeholders.”
Kaufman: Take the K-12 education space. This has multiple stakeholder groups. We have kids. We have parents. We have governments that are funding education – these range from the local municipality to county, state and the federal government. We have employers who will eventually employ these students. And then there are universities who accept students who are trying to continue their education. Every one of those stakeholder groups would describe a different desired outcome that would be ideal for our K-12 education system, even down to a specific school.
For the first 100 years or so of public education in the United States, from the days of [American educational reformer] Horace Mann all the way through the 1970s, no one tried to fix that problem. We allowed the varied stakeholders and interests to create a system in which we had very little data because we couldn’t clarify the outcome. We would know things like attendance and graduation rates, but we couldn’t track students from high school into college. There was no single indicator system that would tell us which high schools were producing the most college-ready students, or which types of programs in K-12 led to higher college or career success. The U.S. public education system was humming along, receiving and allocating a lot of funding, but there were no agreed-upon outcomes or measurements to know whether or not the money was being used wisely, which money was being used the most wisely, and in what ways.
Since then, U.S. public education, through a series of federal mandates as well as some funding with strings attached, has gotten more clarity on what it is that we should measure as the ultimate outcome of a K-12 education. We call that “mastery” or “proficiency” of certain subject areas. Since the late 1990s, we’ve been measuring this pretty consistently and now we’re able to track students through different levels of the system. We’re able to draw some conclusions about which things are and aren’t working, and which students are and aren’t succeeding and why. That single measure has done more to shed light on the issues in education than anything else in the history of education. So there’s a huge new industry around data in education and data-driven strategy in education, all because we were able to identify a bottom line and begin to measure it.
Knowledge at Wharton: You’ve spoken from a management consultant’s perspective. What about from the perspective of schools and social impact organizations themselves? What are the top priorities for them?
Kaufman: Foundations and not-for-profits are finding that securing donations and funding is dependent on indicators and measures of success. Funders are becoming savvy and wanting to see more outcomes. A good analogy is from back in the 1990s; you see some of it even today. In almost every opportunity, you had to donate money — whether it was dropping a few coins at the checkout stand of the grocery store, or through a formal donation at your local church. One of the things the organization would often report on — the data they would give you — was what percentage of your dollar was going to go to the starving child in Ethiopia or the uneducated female teenager in Kenya. That metric was supposed to tell you how lean the organization was and how little money was being spent on overhead. People determined their willingness to pay based on leanness of overhead. From our perspective, that’s a perverse incentive. It basically tells organizations that they should have very little structure, very little infrastructure, with minimal leadership that does very little planning and gets as much money to the end user as possible. It ignores impact entirely.
Whether that money going to that child provides long-term nutrition or creates the right environment for that child to be nourished and to succeed long-term was ignored entirely. You now see social organizations becoming more savvy themselves at determining, “What is the impact we’re seeking? How do we measure it?” And you also see funders making their donations more dependent on outcomes versus inputs or other measures that these organizations previously used.
Knowledge at Wharton: What do you see as some of the main challenges that consulting firms face in dealing with this sector, relative, say, to other areas that they might focus on?
Kaufman: The challenge is to get the right balance of consulting skills and deep industry knowledge. In some respects, every industry has faced this. What we see specifically in social impact is that the foundations, the funders and the non-governmental organizations are demanding content experts. They’re demanding folks who have been there, who have seen outcomes, who have worked in the developing world and who have achieved success as practitioners in the field. But what is required to deliver effective consulting is some of the traditional consulting methods that consulting firms have always used. You need a clear set of strategic tools that firms are good at to be able to deliver value to these foundations.
“Foundations and not-for-profits are finding that securing donations and funding is dependent on indicators and measures of success. Funders are becoming savvy and wanting to see more outcomes.”
When the consulting team is trained on shareholder value and bottom line as the driving force, it can sometimes struggle to wrap its head around a much more complex set of stakeholders and bottom-line data. So the biggest challenge I see around consulting in this sector is how do we balance the skill sets required to help the sector with the industry expertise that’s required. It’s something that consulting firms have succeeded in doing in other industries, and I think we’re on the right track on the social sector, as well.
Knowledge at Wharton: How does the Cicero Group navigate this expertise challenge posed by the social impact industry?
Kaufman: Consulting firms are the home of cross-trained analysts. You can pick up any analyst from the firm and assign them to any project. In essence, that was the business model of some of the early pioneers in this industry. We train people to go attack any problem for any industry. The business was dependent on that because once you start specializing it’s much harder to manage the inbound flow of consulting work. Typically you get a bunch of projects one month in one sector, and then fewer in that sector and more in another sector the next month. That’s hard for a management-consulting firm to manage unless their resources are cross-trained.
What we’re trying to do is keep a bench of cross-trained consultants at the analyst level, so that’s typically right out of college, for the next three or four years prior to a graduate degree. In our organization, we don’t allow them to specialize. We know that what they want is cross training. That’s why they come here. It sets them up for success in anything they want to do in the future. Earlier, we required the same of the next level, which are the associates. They’re typically post-graduate degree, pre-management level. We used to keep them also free from specialization. But we have changed that. All our associates now have a practice area within which they focus. They start to develop that specialization a little sooner in their career than they used to. It used to be that we would allow managers or encourage managers to specialize and evolve. So partners, principals and engagement managers would specialize. We’ve pushed that down a layer. Our ratio of industry expertise to generic consulting expertise at present is different than it was even five years ago.
Knowledge at Wharton: How does Cicero differentiate itself from other firms operating in the social impact industry?
Kaufman: Our main differentiation is our focus on impact and our track record of impact. Let’s take K-12 education as an example. We are focused on student outcomes as measured by proficiency and mastery exams administered by local states. We can show the direct impact of our work on student achievement. McKinsey talks about a 10x number. It’s something that I’ve always admired. They basically try to make 10x the financial impact as their fees — at a minimum. When they evaluate a project, they say, “Okay, this is going to be a $1.5 million project. We have to have a line of sight for how this is going to make a $15-million bottom line impact for our client.” It’s a great exercise and makes selling consulting pretty easy if you can convince your clients that they’re going to get a 10x return on their investment in you.
How do you have that same kind of conversation in the social sector? In K-12 education, we’ve been able to track our progress, and we can say, for this $500,000 project, we can increase your proficiency rates by, let’s say 15% over the next two years for 25,000 students. It’s not quite as financially quick and easy to measure and to describe, but the school district might be able to say, “Well, for that $500,000, we’d be able to hire two more assistant superintendents, or a principal and an assistant superintendent. Could we expect 15% proficiency growth across our entire student body by adding full-time employees?”
So we’re getting closer to being able to show directly what impact we can have for what price. I think that sets us apart — our track record and our fastidiousness around measuring impact.
“The challenge is to get the right balance of consulting skills and deep industry knowledge.”
Knowledge at Wharton: What are some examples in which Cicero has failed in certain education and social impact projects and what are some lessons that Cicero has learned from that?
Kaufman: Our biggest failure was probably early on. We were under the assumption that if we provided great content and great training in this sector, we could very affordably make impact. From about 2007 to 2011, I was on an airplane teaching a model called “Data Wise” to social impact leaders all over the U.S. I had helped create this model while in grad school. It was a nice little business. I got great pats on the back, really awesome end-of-session evaluations. I wrote another book during that time that got great accolades. It was a high point of my career in terms of feeling good about what I was doing and the recognition I was getting. And the business was growing.
We weren’t as fastidious then at measuring impact with every single client, so we had to do a rearview-mirror analysis. I asked my team to check the publicly available information about all of our clients over the last four years and draw the hockey stick chart that showed either steady or declining proficiency rates prior to our arrival, and then the dramatic increase in proficiency rates after our training. The irony is that our clients would tell us that what we were doing was making an impact. We just assumed that they meant they were seeing improvements in student learning. But what my team brought back to me was a straight line, not a hockey stick. We had to do some serious reflection about the fact that we, at the time, had a good business but a bad product. There was a lot of activity going on, but not a lot of actual impact.
It’s the same problem we mentioned at the beginning. If you don’t have clear measurements and agreed-upon focus on those measurements, it can be problematic. It was problematic for us as well. We had to switch our thinking, fix our product and then build a business around our new product. We are now much more focused on changing culture and changing behavior at these organizations, instead of just offering training. So there are bigger projects, and there’s much more intensive coaching than sage-on-the-stage kind of training.
Knowledge at Wharton: What sort of culture did Cicero itself adopt in order to combat failures like these and to drive solutions effectively?
Kaufman: Culture is probably our biggest differentiator in terms of recruiting. It demonstrates itself in our product offering; it’s in front of us every day. Here’s an example. We’re a small consulting firm that will be hiring maybe 20 people this year — 20 full-time consultants. We had more than 2,000 applications. But it’s amazing how easy it is for us to find awesome, amazing new talent. You don’t hear that from other firms. I think our culture is a differentiator, and I’d say it’s centered on what we call the “Cicero way,” It can be boiled down and summarized as follows: We all want to be the best we can be. To do that, we need constant coaching. We need transparency. We need vulnerability. We need feedback. But that’s hard for adults to do — even those who are the best and brightest. You build bad habits early in your career, and you hide and try to brush over the things you’re weak at. We try to have them be part of the constant conversation. I’m the CEO, I have senior partners here and I know exactly the areas they want me to improve in. I know the areas where I’m progressing well and the areas where I’m kind of lagging. It’s part of regular and consistent dialogue. We model it from the top. I’d say that’s probably a good description of our culture and why it was that we’re able to fail and then fix ourselves along the way.
“Some not-for-profits are scared to look at outcomes because they are worried that maybe they aren’t making a difference…instead we must humbly admit when we are not making a difference. I think consulting firms can play a role in providing a third-party rigorous lens to that endeavor.”
Knowledge at Wharton: Could you give us a sense of what your career path was like before you became the CEO of Cicero?
Kaufman: I became a teacher right out of undergrad. I was a regular high school teacher in Northern California. I got passionate about using the data that was available to me to make improvements in my classroom, and I was able to have some pretty cool accomplishments. For example, at the time the end-of-year assessment in each subject in California was called the Golden State Exam. Roughly 20% of students would achieve — I think the category was called “highly proficient” on the exam at my school. After my second year, I was getting 80% students receiving “highly proficient.” It was just basic data-driven strategy. I was taking the outcome measure, breaking it down, creating predictive measures, and strategically planning how I would allocate time throughout the year. It was not rocket science.
Because of those early results, I was quickly promoted to be a principal. Well, an assistant dean, an assistant principal, a principal. In the meantime, I was doing a graduate program at UC Berkeley. I fell in love with statistics, decided to pursue research-based doctoral programs, left school administration, went back and pursued the research doctorate at the Harvard Graduate School of Education. There I learned that there is a way to have an impact from the outside. I started a small consulting firm right out of grad school focused on this Data Wise work as I described. Not long after that, I sold my growing consulting firm to Cicero. It became Cicero’s K-12 practice. It’s called “Ed Direction.” Over the next 10 years, I worked my way up at Cicero. I stayed focused on education and social impact, but tried to influence the whole organization as much as I could. I was named CEO on January 1, 2017.
Knowledge at Wharton: If you think back on your journey, what would you say is the biggest leadership challenge that you faced, how did you deal with it, and what did you learn from it?
Kaufman: The biggest issue we face as leaders is helping the people we work with see the potential that they have. It’s helping our organizations think bigger and see bigger. I feel blessed that I got to be a teacher. You get to work with humans before their minds and brains and bodies are fully developed. And they’re not your own kids. You can’t learn some of these things as a parent, because you’re too emotionally invested in your own kids. I’ve experienced that myself now. As a teacher, in every semester, every year, you get a new set. They come from all walks of life. You see passion and interest for being great in every one of them. And then you fast-forward a few years, and we somehow develop this mindset that we can’t keep growing in the same way we did when we were young. But I just don’t believe that. I’ve seen in my own development and I see in others who have more of a growth mindset.
This passion I have for kids and for their development, for becoming the amazing people they want to become, translates easily into adults. It’s just sometimes hard to convince each person that they have that potential, that they can do great things, that they can be better, to set their sights higher. I saw a funny quote the other day. “We believed the world was flat for thousands of years, and somehow we can’t believe in ourselves for five minutes.” That rings true to me. So the leadership challenge I have is how do I inspire people to see in themselves what I see in them and make the leaps in growth that they have the ability to make.
Knowledge at Wharton: What do you think management consulting firms can do better to improve the education and social impact industries?
Kaufman: I think data is the key — quickly identifying the outcomes that matter and focusing on those outcomes. Also, it is important to admit when you’re not making the difference you think you are making. And this is huge. Some not-for-profits are scared to look at the outcomes because they’re worried that maybe they aren’t making a difference. But we’ve got to choose our outcomes. We’ve got to focus on them. We’ve got to humbly admit when we’re not making a difference. I think consulting firms can play a real role in providing a third-party rigorous lens to that endeavor.
