Companies have access to more data than ever before. But how can they optimize it without getting lost in the weeds – or losing sight of the customer? Evite CEO Victor Cho and Wharton marketing professor Raghuram Iyengar offered advice from their own experiences during a recent conversation with Knowledge at Wharton. Cho was on campus to host a Datathon with the Wharton Customer Analytics Initiative, which Iyengar co-directs. Penn students from multiple academic majors were given datasets from Evite and asked to come up with solutions based on the data for improving Evite’s platform and increasing revenue. Evite is among the participants in WCAI’s corporate partner program, which seeks to help companies find ways to better use their data through collaborations with academic researchers, student projects and other initiatives.
An edited transcript of the conversation follows.
Knowledge at Wharton: Victor, you helped engineer a turnaround at Evite after a period of declining user growth and increased competition How did you use analytics to identify what was going wrong at the company?
Victor Cho: To give you two seconds of quick history, Evite between the period of around 2008 to 2014, which is when I joined, lost its focus on customers. And this is not something that you want to do when things like Facebook become mainstream, which happened in that period, or when things like the mobile phone become ubiquitous, which happened in that period.
Evite at its core is a social service, highly mobile-centric. So analytics drove our turnaround in really two big prongs. The first — and probably most important — core engine that fixed Evite and brought us back into robust growth, which is where we are now again, was focusing on the customer.
This wasn’t hardcore quant analytics, but it was absolutely customer focused. It was understanding what was the customer usage through the site, what is the Net Promoter score, which is the metric that we use, where are the pain points in the experience. So all of that I would argue is an analytic function applied to customer learning.
The second thing that was critical for us was really building what I call the longitudinal model, but dissecting the customer segments: Who is coming in, where are they coming in from, what is their return rate over time?
We have a model that can project out over two to three years whether the fixes we are putting in place are actually going to translate into downstream growth as opposed to short-term Net Promoter impact. So those are the two big levers that we use.
Knowledge at Wharton: I would imagine that you have a lot more data available to you today certainly compared to when Evite launched, and maybe even from a year or two ago. How do you try to wade through that data and identify what is important, what is not important, what should be focused on, and what shouldn’t?
Cho: It’s funny, we have a ton of data available, and one of the things that I have to do perpetually is to tell people — this sounds weird, but not to look at the data. They are going way too granular into datasets when they are actually missing the bigger question.
One great example of that is the first year that I joined, as I mentioned, we used Net Promoter, which is actually at a high level a fairly crude system for fixing experience. I mean, it is nothing more sophisticated than getting a raw stream of input from your customers [saying] what is working, what is not working, and you calibrate that against a 10-point scale. You don’t need statisticians to do this work.
And the teams wanted to always go deep. It was like, well let’s go deep in this conversion funnel, and I kept having to say, no, no, no, I don’t care – again, this sounds weird, I don’t care about the conversion on the site because we have 50 bugs and hundreds of customers coming in and saying that the mobile phone [app] doesn’t work. So we’re not going to look at any of that until we fix these things at a high level.
Knowledge at Wharton: So it’s thinking about what data can be used to actually fix customers’ pain points before you go into something really complex?
Cho: Exactly. We literally didn’t start looking at deep conversion funnels until year three — until our Net Promoter got up into the 80 range, which is now world-class, and I felt comfortable that, yes, the core foundational experience is good. Now let’s go optimize and tweak. And then of course, we built all of the deep conversion funnel analysis that you would expect.
“One of the things that I have to do perpetually is to tell people — this sounds weird, but not to look at the data.”–Victor Cho
Knowledge at Wharton: Because if I am the end user and my app doesn’t work, I don’t care about the funnel.
Cho: Yeah, you don’t care if the button is green or blue, exactly. You had people saying, oh let’s go change, let’s go test AB, let’s go put in this crazy AB testing functionality, [but we thought] we don’t need to do that either; we just need to get the apps working.
Knowledge at Wharton: Now Raghu, the Wharton Customer Analytics Initiative partners with companies like Evite regularly to look at datasets and examine how they can use analytics to achieve business goals. How does what Victor described about the analytics journey at Evite compare to some of the questions you get from other companies?
Raghuram Iyengar: What Victor said is pretty much on the money. What I have seen companies doing many times is they get so deep into the data they forget the problem itself. While on the one hand there is a lot of richness in looking at data, I think it is very easy to in some sense get lost in the weeds.
You have to step back a little bit and think about what the business problem is, understand what the low-hanging fruits are, and then start digging deeper. Start with a business problem, work backwards, and then understand what analytics is required at what point in time, and structure the analytics in that way.
So I very much agree with what Victor did in terms of trying to understand what the end user experience is. What are the big pain points? Let’s first figure that out because before figuring that out, that data actually might not be very good quality. So let’s clean that out first, and then start looking deeper into optimizing any experience that might be there.
It is absolutely very much the experience that I have seen over the years, that many companies get so deep into the data that they start forgetting what the big-picture managerial problem is. I am glad to hear that at Evite for example, they started with the problem first, cleaned up the data after that, and then started optimizing.
Cho: Yes, and I am a huge data junkie, so I also want to make it clear that there is massive power in the more sophisticated data work. I think of it almost like a wedding cake. The very bottom of the cake is — we use Net Promoter, right? Your core engine: Is the customer experience good?
The next layer of the wedding cake is your tactical optimizations. Things like conversions, AB testing, etc. We’re actually past those two layers, and now we’re in the third layer, which is how do you actually untap really innovative business opportunity from data. We just recently hired a VP of data science and business intelligence. We weren’t ready for that role two or three years ago. Now we are; we are building out a data science team and a higher order team, because we do think at this stage mining our data will take us to the next step of innovation.
Knowledge at Wharton: Tell me a little bit more about the top of that wedding cake. With this new role and trying to integrate that into the rest of your operations, what do you think is next in terms of data challenges for Evite?
Cho: The great thing for us is we actually don’t even know what it is — we have concepts of what is possible, but there is not a singular focus where I am going to tell — his name is Jerry — where I am going to tell Jerry, go do x, right? To a large degree, we are going to be in an exploratory mode of, hey we have this incredibly rich dataset, there are lots of different vectors, let’s go figure out which ones might bear fruit.
That’s why I am super excited about this Datathon. I am actually selfishly hoping we get some interesting tidbits, that [sense of] ‘Wow we never even thought of that.’ Maybe that becomes a vector of exploration for untapping business opportunity.
Knowledge at Wharton: That was actually what I was going to ask you next. Tell me a little bit about what led you to do the partnership with Wharton Customer Analytics?
Cho: The history is I got invited to speak at one of the conferences here [in 2018] and got introduced to the organization. And I love what it is doing at a high level. I was a weird undergrad in that I actually constructed a nonexistent concentration at Wharton around statistics, because I felt like that was really the more powerful learning versus getting deep into finance. To me, finance was just an application of statistics.
I told Raghu I was jealous; if I were coming through Wharton now I would want to go through this curriculum and program because I think it is so powerful. I was enamored by the work that they were doing, and I love giving back. There was an opportunity for us, Evite, to partner with Wharton in a way that creates value for both sides. Help students, help us — it’s just a win-win.
Knowledge at Wharton: Raghu, what is the value for Wharton Customer Analytics and for the students to work with these real-world datasets?
“You have to step back a little bit, and think about what the business problem is, understand what the low-hanging fruits are, and then start digging deeper.”–Raghuram Iyengar
Iyengar: We need organizations like Victor was mentioning who really want to give back to the school in the sense of enriching the next generation of people who would become data analysts, become people who would transform the analytics world.
And the way of doing that is learning by doing. And so that is what we firmly believe in Wharton Customer Analytics, that the only way of learning analytics is by doing it. And how you do it is by partnering with companies who are facing real challenges, so students get exposed to real datasets with real managerial problems.
They work on them, they understand what the different pain points are, they understand what the different levers are, and then they come back with actionable solutions. So working with companies like Evite, that is the only way to move forward in terms of exposing our students to real-world problems.
Knowledge at Wharton: What type of data did the students work with for this event?
Cho: At Evite, we have an interesting site in that we have a long tail of different party types that are flowing through our system. So it was kind of a trick to find out — you don’t want to dump massive petabytes of data on these guys. And so the question became, what subset of data are we going to give them?
We ended up with effectively a three-pronged dataset. We have some parties in our system that are what we call seasonal, they kind of happen in bursts. For that, we give just one party type, which is barbecues, which tend to happen in the summer. So the students will get a dataset around barbecue parties.
We have some parties that happen throughout the year, they’re enduring, and we gave them a dataset of a party that I didn’t even know existed, but there is this thing called pet parties. People throw parties for their pets. It is actually a fairly large category, so they get to play around with that.
And then we have these spiky, kind of one-time events. For that, we are giving them a Father’s Day dataset. They will have access to that data, and they will also have access to — in some ways what is more interesting — the downstream behaviors of people who were invited to those parties so that they can hopefully help us understand what are the dynamics driving exposure to a party versus downstream behavior.
Knowledge at Wharton: When you say downstream behavior, is that just RSVPs, or is that how they interact with the invitation, or both?
“We just recently hired a VP of data science and business intelligence. We weren’t ready for that role two or three years ago.”–Victor Cho
Cho: It is all of their subsequent behavior. So for someone who goes to a pet party who wasn’t a customer and now becomes a customer, what does that curve look like?
We have a pretty simple host-to-guest engine. So you are a host, you throw the party. You become a guest, and then we hope that by being exposed to our wonderful service, that at some point when then you go throw a party you will think, oh yeah, Evite, that was a great experience, I want that experience for my guests. So you convert from a guest to a host. So yeah, [the students] will have a dataset that tracks all of those behaviors to understand some of the longitudinal dynamics.
Iyengar: From a student’s perspective, they get to see what the different customers are like. In the Evite case, for example, you might start out by looking at some customers who are guests to a party, kind of going along with what Victor was saying.
At some point they might say, well, I just went to a pet party. I also have a pet; I would like to host a party. So what do students see from this, how do customers transition in different types of dynamics? So from a student’s perspective, it is seeing a dataset that is very dynamic, very rich, and very similar to perhaps other kinds of datasets that they might see in their own careers.
Secondly, and perhaps more importantly, what they get out of this is not just the analytics per se. It is also how to present back to the company. This afternoon, [after] these teams have analyzed the data, and they will come in and present back to Victor and Jay [Neuman, Evite’s vice president of data science]. They have to be very articulate about what the problem was, how they went about doing it, and translate what they have found into actionable insights.
I think that is a very important piece that many analysts somehow don’t seem to gather. They are very good sometimes at analyzing the data, but a very critical step, in terms of actionability, is asking how do you convey the results back in a way that a company can actually work with?
Knowledge at Wharton: You have done other Datathons in the past with other companies. What are some of the things that students have come up with?
Iyengar: There are many, many examples. One of the Datathons we did was for the Hertz [rental car] company. They had given us a dataset which was quite rich, extensive, very much like the Evite dataset, where they had information on the Net Promoter score for example, of people who were serving customers. So this is from the salesperson’s perspective. How did the salesperson do in different locations like, for example, on airport, off airport…
And they wanted to get a sense of the dynamics. How are people, for example, changing over time? Do some locations have better service quality than others?
Students, when they first see data of this kind, tend to be overwhelmed a little bit. So they have to understand how to sit back a little bit, understand what the data structure is, understand what the key problem is, and then dive into the data to do the appropriate kind of analysis, as opposed to kind of trying to find the needle in the haystack.
Knowledge at Wharton: Victor, do you have any expectations as far as having fresh eyes on Evite’s data?
Cho: I am expecting a revolutionary innovation. I’m just kidding — I have zero expectations. I just love seeing fresh eyes on data. I think I would be incredibly happy even if we get a hint of a direction of exploration that we haven’t thought of. That would be a crazy success from my perspective.
Iyengar: One of the powers of Wharton Customer Analytics is that we expose a lot of students to a dataset of that kind. And when I mean a lot of students, it is students coming from different backgrounds. These are the Wharton students; they are also people from engineering, they are people from economics.
“A very critical step, in terms of actionability, is asking how do you convey the results back in a way that a company can actually work with?”–Raghuram Iyengar
To Victor’s point, I think what is always great when we look at Datathons of this kind is that the business school students from Wharton have one perspective, the engineering school students have another perspective. So, many different perspectives converge to then find an actionable solution.
Cho: We find there are some very smart people who will come in, and they are just missing that fundamental, for lack of a better term, business acumen [for understanding whether] we are solving the right problem at the right level, or whether we should be thinking about this problem differently.
A lot of times, if you have a toolkit, you are just excited to go in and show that you can do stuff. And we’re [thinking], well that’s cute but what are we going to do with this? It is just going to sit there on the shelf as an interesting analysis, as opposed to, oh wow, here is a new product experience that we can actually bring to market because of this insight.
Knowledge at Wharton: After the Datathon, what’s next for the partnership for both of you?
Cho: We are a multi-year sponsor with the program. So we actually just had a breakfast this morning where we were talking about all of the different potential ways we might be able to come together in future venues. It will be an ongoing, robust relationship.
Iyengar: There are multiple ways in which we collaborate with companies like Evite.
We have the Analytics Accelerator program. This takes place typically every fall. Let’s say Evite, for example, wants to participate in that: This would be a situation where Evite works with a group — there’s about four to five students — over a month-long period where they deep dive into the data and come back with actionable solutions. So that is one example.
Datathons, of course, are another example. We also have an Executive Education program, where we are targeting, for example, mid-level executives who want to be data translators. These are people who are not analysts, but who want to take the results that analysts are doing and make them actionable, make them understandable to the C-suite.
We have been talking to Evite to see if they would like to partner with us in those programs as well. A host of different things. Another example is you have the Wharton Analytics Fellows. These are groups of undergrad and MBA students who have the analytics club. They work with companies as well to solve business problems. There are lots of different exciting opportunities going forward.
