Close ×

Transcript: Episode 1

Jonathan Halvorson 00:01

With everyone having these tools to be able to produce creative, what differentiates people? I think what matters a lot is going to be the differentiation of the brand and the foundations. What is the product truth that is core to the brand? What is the brand purpose? What is the tension it plays against? And if you define those things clearly, then I don’t think that you’re going to have that swell to the average and to the mean. But for brands who aren’t clear on exactly who they are in their foundations, I think they will find themselves in a sea of sameness.

Kartik Hosanagar 00:34

Hello and welcome to the premiere episode of Where AI Works, conversations at the intersection of AI and industry, brought to you by the Wharton School and sponsored by Accenture. I’m Kartik Hosanagar. I’m a professor of technology and digital business at the Wharton School and also co-Director of our AI research center, and it’s my pleasure to be your host on this season of the podcast, we’re going to tackle some big questions about how AI is shaping the world of business today and where it’s going to take us. It’s our goal to cut through the noise and to deliver actionable insights for business leaders. On every episode, we’ll combine cutting edge research with real world case studies and uncover how the best companies are using AI to upskill their workforce and transform their business. Things are changing fast, so without further ado, let’s dive in.

News Clip One 01:24

We are definitely in an AI hype cycle, and artificial intelligence is a very big and very important macro trend.

News Clip Two 01:31

The race is no longer just about scale or speed, it’s about intelligence that can think and reason.

News Clip Three 01:37

AI is replacing human tasks faster than you might think.

Kartik Hosanagar 01:41

AI has been around since the 1950s and computer scientists have been working on it for several decades now, but it’s really taken off in the last two years, and all of a sudden it’s become extremely important for the C suite. It’s likely to impact everything from marketing and sales to software development, R and D and other functions, and likely across all industries. This inaugural episode of Where AI Works starts with marketing, and in particular, how is AI super charging the way brands are creating content to help me unpack the answers, I am pleased to introduce the global Senior Vice President of consumer experience at Mondelez International. Jonathan Halvorson, Jon, welcome to Where AI Works.

Jonathan Halvorson 02:25

Thank you so much for having me.

Kartik Hosanagar 02:26

Not everyone is likely familiar with Mondelez, but I suspect the brands that are within Mondelez are known to all of us. So maybe I’ll ask you to first just introduce Mondelez and tell us a little bit more about the company.

Jonathan Halvorson 02:39

Sure, Mondelez is the global leader in snacking. We operate in a few core categories that you’re very familiar with: chocolate, biscuits, baked snacks, you’d be familiar with: Oreo, Milka, Cadbury, Ritz. So, a true combination of global and local brands operating everywhere around the world,

Kartik Hosanagar 02:57

From what I understand, your background in college was investigative journalism. So I’m kind of curious how you go from investigative journalism to advertising, and in particular, Mondelez as well.

Jonathan Halvorson 03:08

So I attended the University of Missouri undergraduate program in school of journalism, and why I was there, I was sitting in history of American journalism, and they were doing a preview of all the sections. It’s that moment where you decide, am I going to be a newspaper or a broadcast? And a professor, Steve Capch, came in to present what the vision was for STRATCOM, and he had an unbelievably brilliant articulation of exactly what STRATCOM was. And it was the just really the compelling nature of it, how he talked about writing, the clarity of thought, and just really building brands. And I sat there, and I scrapped all my dreams of being an investigative reporter or working a broadcast beat, and I signed up for advertising. A few short years later, I would go to work at Leo Burnett and later on, in media organizations, OMD, Publicis, working around the world with a lot of blue chip clients. And fortunately, I had Mondelez as a client, and later on, had a chance to join them about eight years ago in the capacity I’m in now running global media, digital data, creative and digital commerce. For us, I think ultimately AI is going to work from the bottom of the funnel and come up. So the beginning uses of AI that we see are really in D commerce, building product display pages. So, the images for that, the text optimizing that, for search engine optimization inside retailers. The second use case that we see is really in producing everyday advertising content, the display assets and banners you see that are meant to drive conversion and are slowly working way up to social assets. So lifestyle imagery that you would see and Instagram posts and Tiktok, and we continue to progress up, and we see a clear path to doing video and even our above the line TV assets.

Kartik Hosanagar 04:09

Got it and in fact, I think advertising and content creation is a great place for us to kick off this series of conversations we’ll have on the show on AI and marketing. And I want to start by sharing a study that some of my colleagues at Wharton did where they were looking at which industries and job types are going to be most impacted by AI, and the top two in that list were media and entertainment, and legal. And of course, media and entertainment includes advertising. Where I want to begin the conversation is perhaps even the very basics, which is, where exactly can you apply AI within the content creation space in advertising? Where are you folks using it today?And it’s interesting, you said you’re starting at the bottom and working way up, meaning like these asset creation and hopefully impacting strategic choices over the long run.

Jonathan Halvorson 05:37

Yeah, you’re starting with lower funnel assets that are more conversion, but I think the systems you’re building are really end to end, and I think that’s where you’re going to get the value. And I think your Wharton colleagues are speaking to that. That’s why it’s not augmentation, but true disruption. Because I think AI is going to help you do everything from creation of the brief all the way to actually traffic that asset and put it out into market.

Kartik Hosanagar 06:00

That makes sense. But one question I have for you is, in this early phase, are you already seeing any evidence of ROI? And I ask that because there’s so much hype around AI, but many skeptics are also starting to raise these questions about all the spend, where’s the return? What are you folks saying?

Jonathan Halvorson 06:17

Over the last two years, we have done 40 campaign based AI activations. So this is example wearing campaigns. We’re using AI technology to augment and deliver a higher efficacy or a higher efficiency of our assets in those campaigns. We can clearly point to that not only being some of our strongest work, but also in those campaigns, delivering higher brand awareness, higher market share gains and higher net revenue. I think that you see a lot of the best of that work inside Mondelez on Cadbury. I’d point to two specific examples. First is give a cheer for a volunteer in Australia. Last year, our top performing BU was Australia, and the top performing brand Cadbury, at the center of their campaign activation was a program that allowed consumers to create an AI film that celebrated a volunteer in their life. And it’s something that wouldn’t have been possible without AI. We can’t hire an artist or a film director to produce all those films and be able to make that possible. Similarly, inside India, last year for Valentine’s Day, we created an ability, through packtivation, for consumers to create Valentine’s stories and tell the entire story of their love. It was really geared towards people who are, perhaps not the planners, but rather last minute gift givers and a little bit of also helpless romantics who want to tell those stories. Those campaigns aren’t possible without it, and there are two of our highest performing campaigns, both internally as well as externally celebrated.

Kartik Hosanagar 07:39

Jon, what I really like about the examples you just shared is that the examples are not just about cost cutting with AI use, but it’s about meaningfully getting a lift in outcomes like consumer engagement or attention and things like that. And we’ve seen that actually, we organize an annual generative AI workshop where researchers share the latest research in these spaces, and multiple studies in our workshop have shown that it’s, again, not just a cost reduction standpoint. For example, one study by researchers at MIT and Indian Institute of Management showed that there was a 6 to 9% lift and click through rate of ads, and another one by researchers at University of Hamburg and Emory University compared human-generated ads against AI-created ads for automotive companies, and they used purchase funnel metrics like: Does the ad get attention? Does it generate interest? Does it generate desire? Does it create action, like downloading a brochure and so on. And on a seven point scale, they found those purchase funnel metrics went up by nearly a point, about 0.75 points. So strong evidence. Have you guys tried to measure this as well?

Jonathan Halvorson 08:47

Yeah, the same thing. I think in the very beginning you see higher ROIs, higher conversion rates. But I think it’s also important to note, this is just the beginning. We’re in the early rounds of the ball game. What I think the potential that gets people excited about AI is for that to only improve over time. So today it’s seven points. And just to be clear to any of the listeners, that’s material to the business, when you start delivering 250 trillion impressions in a year, and if you can do that 1% better, that adds up to significant volume gains and significant growth in terms of net revenue, but then it’s just the beginning. Over time, AI is going to cumulatively learn and get better and get better. And I think that’s what really excites us. If we can start from that point, have really good success signals and train the systems, we’re only going to achieve higher levels of efficacy and excellence.

Kartik Hosanagar 09:37

It’s great to see that kind of impact with AI optimized or AI-generated content. I’m also curious about what is the role of the human in this? So what is the human in the loop process here? Because I’m assuming it’s not like a bunch of AI agents doing everything there are humans prompting and humans selecting and so on. So tell me about that role.

Jonathan Halvorson 09:57

A lot of people have a different point of view about this, and I’ve had a lot of good debates with practitioners, colleagues, peers, academics, about where it’s at. And I think where I’m at on this is AI is going to impact every single part of the value chain. And I think it’s going to help humans do their jobs better. And if we look at it the very beginning, like we use AI internally to help us write better briefs. So do we write good creative briefs today? Yes. Can they be better? Yeah. I want them to be perfect every single time. And I think with AI inputs, we’re raising the quality of our briefs. The second thing is coming up with concepts. So we use our tool, our proprietary tool at Mondelez is called Aida, AI D A. It stands for artificial intelligence, data and analytics, and that tool allows us to generate concepts. Are any of those concepts perfect? No, but they help inspire our creative teams and our marketers around the world as to what’s possible. So I think that then gives a jumping off point, and this is where our agency creative directors, at WPP, Publicis, the Martin agency, VCCP, provide their own ideas, elevate those ideas to get to really good scripts, then we can quickly test those scripts, ultimately, by generating a series of images that represent that we test it. What used to take two weeks, can now take a few hours, get a score back, and then we can go produce that asset at a higher quality. So I think the human in the loop really is in a few key areas, like it is giving humans a lot more time to spend on creative excellence, to elevate the ideas and a lot of the place where that uniquely provides value.

Kartik Hosanagar 11:27

Yes, that makes so much sense. I think AI can be a great tool from an idea generation standpoint, idea enhancement and, of course, the actual execution pieces as well. We actually did a study recently where we were looking at AI use in creative writing. So we had a bunch of students who wrote without AI and with AI, and we noticed that AI use, of course, reduced the time it took to write, it increased the quality of the writing as well. But one of the interesting side effects we found was that human-created content in aggregate was more diverse than AI-created content. When everyone is using the same AI tools to write, they’re coming up with similar ideas, whereas all of us have very different biological neural networks in our head, your influences are different from my influence. So we come up with different ideas, but if we both use the same kinds of tools, you end up with similar ideas. We see it now with like PowerPoint presentations, like everyone’s PowerPoint is starting to look similar. Do you worry that if all your people are using these same tools, the diversity of ideas might go down? And eventually what you’re seeing as a lift in performance might actually go down, because everything starts to look the same?

Jonathan Halvorson 12:42

Yeah. Look, this is the huge [unintelligible] challenge. Like everyone’s getting ice cream. It’s the best ice cream, but everyone’s getting vanilla is essentially a good summary of the problem. I think about this a lot, because there will be a massive democratization of AI tools, and anybody, any marketer, any company, no matter how big or small, will be able to produce Super Bowl level creative instantly off of their laptop, if not their phone. 12, 18, 24 months at most from today. So then, with everyone having these tools to be able to produce creative, what differentiates people? And I think what matters a lot is going to be the differentiation of the brand and the foundations. At Mondelez, we’re really blessed. We have an amazing portfolio of over 250 brands that are iconic taste of nation. But what will be increasingly important in the journey we’ve gone on over the last eight years is to very clearly define those brands. What is the product truth that is core to the brand? What is the brand purpose, what is the tension it plays against? And if you define those things clearly, then I don’t think that you’re going to have that swell to the average and to the mean. But for brands who aren’t clear on exactly who they are in their foundations, I think they will find themselves in a sea of sameness. So I spend a lot of time thinking about this, and I think the safeguard on it is: one, the clear brand foundations;two, humans in the loop; and three, just the discipline you put in, just the intentionality of how you use AI and just recognizing the concern.

Kartik Hosanagar 14:10

Yeah, I think that’s a really important point. And I just want to dig a little deeper into that. How do you in practice, implement brand distinctive features in your AI augmented or AI generated content?

Jonathan Halvorson 14:23

There’s a few things. First, it starts by being really clear on what are the distinct features of your brand. And that sounds like a very obvious thing, but it’s a very intentional exercise that has to take place before you ever open up AI or open up a laptop, you know. And so in the case of Oreo, it is a black and white sandwich cookie. It has a very clear embossment. There’s a very clear logo, and look to that brand that has to be clear, no matter what the instance is. Then having defined all those assets that are truly distinct, then you have to train a large language model, and you have to explain it. And we’ve had a lot of work that we’ve done on this across all of our agency networks. Training for Milka, training for Oreo. And it just requires time to essentially upload the entire brand history. And really it’s a technical exercise of training the system as to exactly who you are. So it starts by uploading all those awful brand playbooks that we’ve built over the past few years. Those have immense value. Two, is your archive. You know, the really great brands keep very good archives of all the work that they’ve done. And that’s because often, when you’re in a challenge, you can find your solution in the archive. But also that becomes really powerful training data to explain to the AI system your performance over time, what’s worked, what’s not. Then it’s a matter of as you train your large language model, sitting down and giving it back and forth. Just like you train in machine learning or any other application, there is a back and forth exchange of when a system is showing you something, and you have to provide feedback on that. Yes, that’s right. And no, it’s not. And through all that training exercises, you will create a custom LLM that represents your brand.

Kartik Hosanagar 15:59

Yeah, thanks for that helpful peek under the hood. It’s important to understand that process that goes into that and it’s not just going to ChatGPT and writing a prompt and voila, you have your ad. So we’ve talked a little bit about the process. Let’s talk a little bit about the outcomes. I know you’ve run dozens of campaigns that are AI-supported, but I’d love to hear about one or two in particular. What was actually done? What was the new customer experience and things like that?

Jonathan Halvorson 16:27

Let me answer that question in two parts. First, let’s talk about the journey, and then let’s talk about a specific example. So let’s talk about the journey.

Kartik Hosanagar 16:34

Great.

Jonathan Halvorson 16:34

So 18 months ago, we start doing our first work on this. We trained a large language model. I’m sitting with the agency, and I’m super excited. We’ve done a little pilot, and the first thing it spits out is humans that have six fingers and people who have giraffe necks. And then we would see Milka bars that were missized. Everything was wrong. We tried to prompt it on Halloween, and instead, I get things that looked like Christmas. There were people who were joined at the shoulder. I could have cried right then and there, and I was like, man, maybe this isn’t for us.And the early thing that we’ve done in the small little pilot just showed how hard this was gonna be when you talk about some specific examples, and what does that really enable? I like to go to the India BU, because I think they’ve done some of our most advanced work on Cadbury. The first work that they did was actually in music, and it was for Cadbury birthday song. And so the idea behind Cadbury birthday song was a simple insight: There are literally billions of people in this world, and yet there’s only one song to celebrate them. Happy birthday to you. And so their insight was, wouldn’t it be powerful if you could create a custom birthday song? But the thing is, is you can’t hire a musician to write a billion birthday songs for any iteration, but using AI and some large language models, in partnership with the Ogilvy team, they created a tool where, if you uploaded some basic information about yourself, you got to pick a type of music, you got to pick a language. Ultimately, you would produce a custom birthday song for you. We did work promoting it and above the line, as well as below the line, and it became huge hit across India, with literally millions of people participating, driving incremental sales celebrations outside of the core festive season, when typically almost all of our chocolate is sold. So really extending the chocolate season from just Diwali and Valentine’s Day to really being something that’s 365 days a year. And that’s a powerful example.

Kartik Hosanagar 17:08

Right.Yeah, and by the way, I grew up in India, and so Cadbury was a huge brand in India when I was growing up.

Jonathan Halvorson 18:38

I have to thank Brot Purdy and many of the great marketers that come before us for the great tradition we have in India in marketing, and it’s laid a great foundation.

Kartik Hosanagar 18:46

It’s been great to hear about what’s gone well, how things are producing positive impact and results for you. But let’s talk a little bit about lessons learned, because a lot of companies are just getting started with their AI and content journeys. And I’m curious what’s been the hard part of all this? What have you learned in terms of both what’s been hard that you’ve solved, but also what you’re still trying to solve?

Jonathan Halvorson 19:11

So many lessons learned. I mean, what started as an insight at a moment where we had the flash of brilliance to being here today, has been a long road and a lot of work. I think, you know, lessons learned, and things I would tell one is, one is find a strategic partner to be your sounding board and stick with one all the way consistently. I would say, throughout this entire journey, I have had really good strategic partners, and it’s the one voice in my head and that makes life really simple. The second thing is, I think it’s very important for you to have a clear narrative. Every company will approach AI differently and even the use case of content. I’m very focused on increasing the effectiveness of that content and the quality of it. Others, their business case is more built around speed. How can they take global assets, localize them, get to market faster? Other people are all about cost savings. What I think is important is that you have a very clear business case and narrative around that that you consistently talk about through the organization, and that becomes really important. Third is, most of this is change management. And I’d say that the change management is probably half of the cost and definitely more than half of the time. And so I think it’s important to get people in early to see this and talk about where you’re going. The last thing is, I think you learn from us, is the power of a demo. We did a small little pilot. We had a little example of a Milka large language model that simulated the brand, and we produced this video. And so when we went to go talk to the marketing organization, we went to talk to the C suite, it was an asset we could show. And it was powerful, because often when you start talking about AI, large language models, you can get very quickly lost in all the junk in a series of acronyms that aren’t really going to help you. But this little example became a real shining light, and for people who weren’t even marketers or weren’t familiar with AI, they saw it and it changed their mood. There was a moment, a pivotal moment, where we were deciding if this was gonna be part of our strategy or not, and one of our region presidents thought,he goes, “that’s the future”.

Kartik Hosanagar 21:06

Change management, to bring people along, demos to win people over. The interesting thing, and that’s where I wanna take this last question, is that advertising, at the end of day is all about people. It’s about people with creative ideas who can help brands connect with real people, and now you have AI, native agencies, AI automating work and all of that. I’m curious, what are your hopes, or what’s your vision for how people and human creativity can still be at the center of this conversation around AI.

Jonathan Halvorson 21:40

There is an amazing tension and cycle that happens between technology and creativity. Technology gets out ahead, and it inspires new level of creativity, that inspires the next level of technology, and you get a very virtuous cycle, and I think you see that playing out over the last several decades. And I always love moments where technology is out ahead, because it means that the things I dream up in my willy wonka mind are possible. And whereas, when creativity is ahead of technology, you’re like, this would be really cool, but we can’t do it. And I find that to be a lot more frustrating. But my hope and dream is that I always believe that the best advertising is yet to be done. And therefore there’s a constant way to do better and cooler things. And every time we do this, I think for us, it was our Shah Rukh Khan, your ad moment.

Kartik Hosanagar 22:29

By the way, for our listeners who don’t know, maybe we should say who is Shah Rukh Khan is. He’s, because I grew up in India, I can say he’s like the Brad Pitt of India, only 10 times bigger in India, or something like that, right?

Jonathan Halvorson 22:39

Tom Brady, plus Tom Cruise, plus every good imaginable human like, you know, all rolled into one. I mean, and this asset we just we shot with him for a day, and it allowed you to create personalized versions of it for every local retailer. And it was just a dynamic creative optimization on steroids. And when you saw that, you were like, wow, that wouldn’t be great if everything we did was that good. I think you have moments like that where you see a piece of work, it inspires you as to what could be. And I think every day, I hope that the work that I produce at Mondelez inspires other marketers to create things. Because I know as Mita at L’Oreal, Tamara at Heliot, they’re constantly producing things that are inspiring me, and I go to my teams, I go look at this, and wouldn’t that be so cool? And I’m pushing that to the next frontier.

Kartik Hosanagar 23:28

Jon, this has been great. I think we created some good podcast content here, and we didn’t even need AI for this.

Jonathan Halvorson 23:34

Nope.

Kartik Hosanagar 23:36

Thank you so much for being guest number one on Where AI Works.

Jonathan Halvorson 23:40

Thanks so much for having me.

Kartik Hosanagar 23:42

Now let’s review the key learnings from my conversation with Jonathan. A big initiative like AI driven change is likely to throw some early challenges. Like Jon was talking about images where people had giraffe necks, or six or seven fingers. What’s really important is one needs to have patience, because you’re investing in AI for the long run. The other learning was that AI value is not just about cost reduction. It’s also about consumer engagement and revenue impact and increasing top line increasing consumer retention and things like that. And finally, it’s important for companies to find the right and consistent strategic partner, because AI driven transformation is as much about change management. How do you convince people? How do you bring them along for the ride? How do you create demos to get some early excitement about what you’re trying to do, and how do you make sure your people are with you in this process? That brings us to the end of the series premiere. Thanks so much for listening. Please follow us so you don’t miss an episode, and be sure to tune in next time when I sit down with Jill Kramer, CMO at Accenture. We’re going to explore how AI is reshaping marketing. This has been Where AI Works, brought to you by Wharton and sponsored by Accenture. I’m Kartik Hosanagar. See you next time.

Close ×

Transcript: Episode 2

Jill Kramer00:01

Curiosity and creativity is driven by exposure, by options, by the least restrained possibilities, and Gen AI unleashes that. So if you allow it to be short-handed too “Can’t Gen AI just write that for you?”, it’s an incredible disservice to the technology and the potential of it being applied to a function as important to growth and brand and the strategy of any given company as marketing is.

Kartik Hosanagar 00:31

Hello, and welcome back to Where AI Works, Conversations at the Intersection of AI and Industry. Brought to you by Wharton and sponsored by Accenture, I’m your host, Kartik Hosanagar. I’m a professor at the Wharton School, and my work focuses on AI and business transformation. It is our goal to cut through the noise and deliver actionable insights for business leaders by combining cutting edge research with real world case studies. Things are changing fast, so without further ado, let’s dive in.

News Clip One 01:02

I’m getting the same question a lot nowadays. Should we be scared of AI?

News Clip Two 01:06

When we’re talking about trust, it means so many things, and it’s a whole world that everybody will have a different interpretation of it.

News Clip Three 01:12

One of the challenges that we have is that the space is moving incredibly quickly.

Kartik Hosanagar 01:16

On this episode, I’m excited to explore how global brands are harnessing AI and also what it takes to take large teams along on that journey. Multiple studies have shown that AI dramatically improves human productivity. These studies range from settings like software to sales, consulting, customer support, professional writing and many more. Now, while we have that evidence at the individual level, hard evidence on return to AI investments at the organizational level haven’t yet emerged, and I think that’s because there’s a difference between a few individuals adopting and seeing their returns on that versus large, complex organizations, because you’ve got to manage cultural change. Re-skilling, up-skilling, strategic prioritization; lots of issues here. To help us think through the challenges and the opportunities around AI, it’s my great pleasure to welcome Jill Kramer. Jill is the Chief Marketing and Communications Officer at Accenture. Jill, welcome to the podcast.

Jill Kramer 02:15

Hello, Kartik. Thank you so much for having me. I’m excited for our conversation.

Kartik Hosanagar 02:19

Likewise, super excited to have you here. I guess even before I get to the topic of returns from AI and how are you managing AI driven change at Accenture, I’m really curious about what’s been your relationship with technology.

Jill Kramer 02:34

So my personal technology journey is one of avoidance and fear that turned to a deep, unwavering love. I would love to tell you that I was fearless, that I was first in line with my hand up, saying, let’s do this specifically with the most recent wave with Gen AI and the speed at which it’s moving. I did start this thinking, wow. Like, is this an existential crisis, and I made the decision, and I was very transparent about it with the full marketing and communications team at Accenture that I desperately wanted to be in the driver’s seat. And the only way you can do that is by getting really close, getting very hands on, deeply understanding, and taking some risks on behalf of yourself, your company, your brand, your team. So that’s where I went over to the love side. And it is something that I truly believe now is invigorating, exciting, drives more creativity than I ever could have imagined.

Kartik Hosanagar 03:35

Yeah, that’s great to hear. And in fact, later in the conversation, I’d love to explore how you have undertaken that journey. But before I get into that, perhaps where I want to start off is maybe at the organizational level, before we get into like the marketing team and then you individually. Accenture has been talking about being early and aggressive with transforming its business around AI, so help us understand what progress has Accenture made in that AI transformation journey, and where is it currently positioned relative to others in the market?

Jill Kramer 04:08

The journey for us is really the journey we’ve taken with previous technologies, really, which is understanding each wave of technology and the impact it can have on work and workers, and in this particular wave, one thing that was evident very early in the process with Gen AI was that if you apply it to existing work, existing processes, you would see incremental improvement. But if you said, I want to look at the work in a zero based way, now that I know that this technology exists, how would I do it given these new circumstances? And that is something we have done across the board. With every function that helps run this corporation, we have looked at rethink the work now that you know that Gen AI exists. Because one of the things you learn, when you embrace technology like this, is those areas of overlaps and adjacency, there’s magic in reimagining those handoffs, those shared services, those processes from legal to HR to operations to marketing to sales. So it’s something that has to be done intentionally. It’s something that has to be done with a clean sheet of paper, you have to rethink and reinvent, and it’s something that has to be done very collaboratively across all of the functions that run the enterprise. Let’s go from Accenture as an organization to your marketing organization. There are many things under your purview which AI could touch upon and does indeed impact and influence today, whether it’s advertising, content, digital marketing, social media, corporate communications, AI’s applications in all of those areas. I’m curious, for you, what have been the first few areas that you have brought AI into, and why have you picked these specific applications or use cases for AI? So I’m actually going to tell you a quick recap of the chapter that preceded our application of Gen AI.

Kartik Hosanagar 06:01

Okay.

Jill Kramer 06:01

And that is when you run a marketing function at a large multinational company, you have projects being asked from every part of the organization, lines of business geographies, you name it, you have requests. A lot of marketing organizations execute those requests, yes, in service of a business strategy, a brand strategy, a marketing strategy, but you don’t necessarily know every project that’s going on at any one moment.

Kartik Hosanagar 06:28

Right.

Jill Kramer 06:29

And you can’t necessarily reconcile, prioritize. So I wanted to be able to say at any point in time, how many events have we done this year, which are the most successful? How many pieces of content have we put in the market? How many new pieces of content have I put on our website? What’s performing the most? So the first thing we did unknowing about the Gen AI movement that was about to happen, we really got our act together in terms of knowing the work that happens in marketing using free Gen AI AI, so regular AI, to create algorithms and prioritize and sort work in a very simple but strategic and intentional fashion. We also brought all of our data together, because if I wanted to look at the work across the function, I had to look at the KPIs. That meant that when Gen AI started ramping up, we were able to be pretty confident about which we were going to adopt first. So that brings me back to where you began with your question. Some of the first things we did was we had really done a very rigorous reinvention of our internal communications. We had created a very highly personalized approach to communicating with the then 700,000 or so people of Accenture. So one of the first things we did was we applied AI to our writing so that, as we were highly personalizing, we could use Gen AI to quickly version. Similarly, we did it with content, not the entire content supply chain, but the content versioning. We did that very quickly. So here’s the long form report, using Gen AI to assist in the medium size and the short size, the quick synops that would appear on the website, the quick synops that might appear on our app. What really changed was when we decided that we were going to go all the way to agentic AI. When you do that, you have to go all the way back to your process reinvention and say that if I looked at the steps in the marketing process, and I knew I had an agent or agents who could do work simultaneously with me going to parallel work streams, instead of everything having to be so linear and sequential, how might I design the work? We ramped that up this past September, and we now have 1000 marketers all onboarded using agents in their daily work.

Kartik Hosanagar 08:47

For our listeners who are not as familiar with agenetic AI and where it’s headed, I think the way to think about it is that lot of AI systems today can get individual tasks done. Agent-based systems are systems that can function autonomously. They can communicate with each other, they can interact with the outside world, get information from there. They can transact. And so once you start to build agentic systems, you’re not talking about automation of individual tasks. You’re talking about automation of full workflows where an agent based system can go fetch information from some other place, maybe interact with a vendor or a partner, negotiate with them, complete full tasks. So that’s an interesting direction that AI is headed. And it’s great to hear that you’re talking about some of these implementation of agentic systems. We’ll come back to that in a moment, but coming back to what you were saying with this journey from starting with AI for writing and then for image-based content, to other workflow products and now to agentic systems. Are you seeing any results in terms of costs or business agility, or what are you seeing in terms of what it’s doing for the business or for your org?

Jill Kramer 09:55

What we’re tracking very specifically is speed to market.And we’re tracking reduction of steps taken in a given process across the function, but also within sub-functions. Then there’s two waves of KPIs that I’m tracking. One is the prioritization of the work, which is Gen AI assisted, because the way we’re processing our reports and our measurement, it allows us to make better decisions, which has led to, like, 50% content reduction. Using Gen AI allows me to now do what we call Good Morning Accenture, which is a highly personalized multi-channel way of greeting each person within Accenture and letting them know what they need to know and what they need to do for that day in order to be productive, focused, etc. On the flip side, when you look at the longer flow processes, we’re seeing about a 35% reduction in steps taken within the application of the agentic workflows, and as much as 30 to 50% increase in speed to market.

Kartik Hosanagar 09:59

Yep. Wow. Those are very impressive numbers, I’m sure, an organization of Accenture’s scale, many of these systems are often built in house, but at the same time, there’s probably tons of AI products that you’re consuming. One of the questions I have for you, which is something I keep hearing from a lot of CMOS, and in fact, across different functions, but certainly from CMOS as well is that there’s so many different vendors out there, and so when you’re bombarded with so many tools to potentially use or pilot, how do you approach the decision of which tools to prioritize? How do you pilot them? And there’s possibly IT approvals and AI committees that need to approve things, that slows things down. So how do you stay nimble within your marketing org, where you can experiment, try new tools, but at the same time be compliant within the organization’s broader mandates and restrictions about what AI is allowed, what’s not allowed, and things like that?

Jill Kramer 11:56

You bring up so many important things and points, but you bring me back to my driver’s seat decision. There are so many solutions. They’re exciting. People want to get their hands on them, and so I’ve got to be in the driver’s seat for myself and for my team on this because people were adopting solutions, bringing them forward. We can use this over here. This is now embedded in this solution we already use, and it felt like a beautiful kind of chaos, but I realized that it was going to vary. We’d just gone through all the work to get our tech stack cleaned up, our data cleaned up, our processes cleaned up. So it was like, how do we bring our people with us? Use everything that they’re understanding? I recently had another CMO saying to me, the people in your marketing organization and or your customers are going to start using these tools, whether you sanction them and intentionally do them or not. And that’s a very, very true statement. And we made pretty bold decisions, like giving access to writer.ai, to every marketer. We could have done a much smaller group. We could have trialed, but we said, you know what, we didn’t want people to feel left behind. And then in other cases, we said, this is going to be the landscape. So we created a leader on my team who was going to oversee all the Gen AI ideas. Let all those beautiful ideas bloom, bring them back to our team and our IT advisors, because no CMO should be doing this without right your IT advisor. They’re your best friend.

Kartik Hosanagar 13:22

Yep.

Jill Kramer 13:22

The other thing about these AI systems is that they are designed to be more modular. You know, like, whether it’s the ability to switch between LLM models, whether it’s the ability to click in new modules of agent types or support with out of the box solutions. There’s an agility and a flexibility if you design your approach to AI and Gen AI in the right way. My advice is set up a system where your people are bringing you their best ideas, but you do have a formal way to assess pilot and scale. You must change the work, because if you don’t change the work, the scale and the actual benefit from going to scale won’t be there. And then the last one is that you need to have your IT department with you, just like we wouldn’t want anybody coming up with their own brand tagline. So you have to merge the craft. Then what is exciting about the technology for the craft of marketing and communications with the expertise of technology experts who can say, here’s how you build this, and don’t worry, because it’s modular, it’s agile, and you’re going to want that flexibility to evolve as you go forward.

Kartik Hosanagar 14:26

I want to move the conversation from the technology to the people, because a lot of this is really a story of people, how comfortable they are with how empowered they feel with these technologies and so on. I’m curious how big is your marketing org, Jill? How many people are we talking about?

Jill Kramer 14:45

We’re at about 17, 18 hundred people.

Kartik Hosanagar 14:47

Wow. Okay, yeah, that’s a lot of people you’re sort of bringing into this AI journey. I feel like when it comes to an org with as many people, you’re going to have some people who are maybe early adopters, who are jumping in trying things out, and then there are going to be people who feel threatened by this. With my previous AI startup, you know, I had these conversations with the C-suite about the products, and I remember one conversation in particular where one of the C-suite leaders was talking about how this tool that we had could potentially be threatening for people in their org. And specifically, he kind of said that, you know, I don’t see our employees using this at all. And the reality was, we had several dozen of the employees already using the free version of the tool, and then we had to walk them through. Hey, listen, you know, your employees are actually further ahead than you think they are. So tell me a little bit about both sides of this coin. One being there are many instances where the employees are ahead of where the leadership thinks they are. And then there are also instances or individuals who are feeling very threatened and feeling left behind because the pace of change is so fast. How has it played out in your arc, both in terms of people feeling threatened, but also empowering people who are moving faster?

Jill Kramer 16:04

Very similar. I mean, a lot of people were proactively coming to us saying “Hey, I’ve been using this tool on the side, and look at how much faster it helps me do this brief” and “look at how much more information I was able to collect”, or “Look at how quickly I did these” and you want to embrace those things, but also make sure you’re within your responsible AI guidelines. Is everything legally sanctioned? On the flip side, I always talk about marketing is a very generous function, right? You give your ideas, your words, your perspective on imagery, to internal and external clients, right, to form marketing materials. But a lot of people think they are marketers, right? Oh, I would have written the headline this way, or I shouldn’t have done this this way. So when you end up with a technology who now says, oh, it can write the headline, it can do the image. It was a really natural reaction to protect a craft, to protect the function, and not allow it to be unintentionally devalued by saying anybody or anything could do this craft that we all love so dearly. So my take on this is, and I was very clear with my journey with the team was that a good marketer is a curious marketer. If you watch someone write a brief, they will spend as much time as you will give them, researching competitors, looking back at everything we have done and not done. What was the best thing we ever did? What was the biggest failure? Same thing with creatives, they want to write, rewrite. I mean, how many times have you had to say to a creative team “okay, it’s time to put the pen down. We got to go produce this.” They want to find every image they can.When you think about Gen AI through that lens, these agents we’re using help you go out and get every bit of analysis, competitive intel, activation schema that you could possibly think of. If you’re using it as a creative, the ability to seek images, to change parameters, to look at the same thing 10 different ways, is now at your fingertips.So I think that’s the biggest thing is curiosity and creativity is driven by exposure, by options, by the least-restrained possibilities. And Gen AI unleashes that on behalf of the human, on behalf of the craft. So if you allow it to be shorthanded too “Can’t Gen AI just write that for you?”, it’s an incredible disservice to the technology and the potential of it being applied to a function as important to growth and brand, and you know, the strategy of any given company as marketing is.

Kartik Hosanagar 18:04

Yeah.Yeah, I’m gonna remember that statement you said, which is, curiosity and creativity are fueled by having options, having this information and so on, and AI enabling that. So that’s great. Walk us through the up-skilling and re-skilling process. You know, 1800 people who you’re now trying to train on new systems, new tools like writer.ai, new ways to approach work where you’re giving up part of what is core to your craft to an AI system, and focusing on other pieces where you can add unique and differentiated value as a human creative. What has that re-skilling process been like?

Jill Kramer 19:13

So there’s a couple of things that are absolutely core to it. The first one is the concept of creating cohorts. When you are bringing a large team along on a very new journey, don’t try to do it all at once. Don’t try to spread yourself, your ability to be attentive, to deeply train and to listen to people by trying to do it all in one fell swoop.So we’ve created cohorts for every wave of transformation. Each cohort takes the next one with them. They create a bigger community of people who can listen, who can help, who can energize and who can address problems. The second thing is communication. Very often, as a leader, you’re already sold and you know all of the reasons in your head. It’s very easy to quickly think “Well, I’ve already told them this, like they know this, they know the why.” You have to say it over and over and over again, and you have to gather together regularly to listen, to communicate progress, communicate advancements, be incredibly transparent, and take every question that comes your way. And the only way that that changes the culture is by repetition and by people realizing they truly can ask any question that they want to or need to. And then the last one are use cases. When the first two things go well, the use cases that are created by your people will be better than the ones you ever imagined. Mark my words. That’s where the curiosity and the creativity is on steroids, so you need to find a way to bring those use cases back, because that’s the very essence of change management.

Kartik Hosanagar 19:34

Right?Right. Makes sense. So, Jill, we’ve talked about the employee side of things. Now, employees are part of the journey, but so are customers, and one of the interesting things is customer behavior is changing. So for example, many of us are no longer just going to search engines and searching and finding our answers. We’re going to LLMs and finding answers, including, what should I buy, or what are the best brands for, whatever the use is. So how do you think about that, and in particular, how does the marketing org prepare for and keep up in a world where customers are embracing AI and making decisions informed by AI?

Jill Kramer 21:27

This is a case where I think it is similar to previous waves of technology, when customers were going to digital, if you decided not to understand that, you would get left behind. When search became a big component if you were not intelligently constructing your content and your digital experiences for search results, you would get left behind. It’s very similar. So, you know, one of the beautiful things about Gen AI is, for example, when you are creating something or writing something, you ask it for a summary.

Kartik Hosanagar 21:54

Yep.

Jill Kramer 21:54

And it makes you realize, oh, the points I thought I was making, I actually obscured. And some points were taken away as top level, and I meant them to be second or third level. If you’re like us, B2B professional services, and you’re creating long form content, what’s the first thing a person’s going to do now that they have these tools at their disposal? They’re going to create summaries, they’re going to merge documents. They’re going to go across and look for the best. So how do you create the experience that’s going to do that for them and with them? It’s just like anything else, you recognize a new condition and you create for it.

Kartik Hosanagar 22:31

Jill, my last question for you is where we started this conversation. You talked about how you started from a place of fear, and over time, it’s been one where you feel like you’re in the driver’s seat, in control and loving it. Tell us how that change happened, and in particular, what advice do you have for other marketing leaders that are facing this change and are perhaps a little behind in that journey and facing some of those fears themselves?

Jill Kramer 22:57

It has to do with human nature and how you overcome fear and misperceptions in anything, and that is through proximity, through exposure, through empathy and through a desire to learn. I will give this single piece of advice Kartik, and that is be hands on with this technology. It is not the same as previous waves, where someone else created a digital environment, and we just had to put the content into it.

Kartik Hosanagar 23:24

Right.

Jill Kramer 23:24

You need to know this intimately and deeply, and that comes from being very close and very hands on.

Kartik Hosanagar 23:32

That’s great advice. I mean, that’s what I discuss with my students as well, that you have to get into the black box, you have to use it, you have to touch it, feel it, and you can’t really poke at it from a distance.

Jill Kramer 23:43

It’s a lot less scary and a lot more exciting when you do that.

Kartik Hosanagar 23:46

Jill, we’ve covered a lot of good ground here today. Thank you so much for your time, and thank you for sharing your insights here on Where AI Works.

Jill Kramer 23:54

Thank you. I’ve enjoyed every minute of it.

Kartik Hosanagar 23:57

One interesting takeaway for me is about how we need to approach AI, not with a lens of fear, but with a lens of curiosity. And her point about how curiosity and creativity are both fueled when we approach AI with that lens. The other interesting takeaway for me is Jill’s framework of cohorts, communication and use cases. When you’re trying to pull off change in large organizations, doing so in stages with cohorts, makes it more achievable, realistic, at the same time, creating use cases, communicating that celebrating them helps bring people forward, and I think that’s a very practical, easy to implement framework for our leaders today. That’s all for today. Thank you so much for listening. Please follow us so you don’t miss an episode, and be sure to tune in next time when I sit down with David Droga, the founder of ad agency Droga5, and CEO of Accenture Song, which is Accenture’s creative agency. We’ll be discussing AI’s transformative impact on the creative industry. This has been Where AI Works, brought to you by Wharton and sponsored by Accenture. I’m Kartik Hosanagar. Bye for now.

Close ×

Transcript: Episode 3

David Droga00:00

I think people always expect me as a creative person, even though I’m CEO, to sort of be defensive and push back and think that AI is going to gobble up everybody’s jobs and all that. I’m like, look, history’s proven us over and over that it moves us forward, and then the creative minds, it’s what they do with it, their interpretation, misinterpretation of it, that moves us forward. I also think that, which is controversial, but I mean it when I say it, not all creativity is worth saving.

Kartik Hosanagar00:31

Hello and welcome back to Where AI Works, Conversations at the Intersection of AI and Industry, brought to you by Wharton in collaboration with Accenture, I’m your host, Karthik Hosanagar, Professor of Operations, Information and Decisions at Wharton. My work focuses on AI and business transformation and how AI is affecting creative industry. This is the show where we explore the implications of AI for business leaders by combining cutting edge research with real world case studies. Things are changing fast, so without further ado, let’s dive in.

News Clip One01:04

We are at an inflection point for human creativity. These happen every so often, usually caused by some sort of technology innovation. The innovation this time is artificial intelligence.

News Clip Two01:14

Generative AI is transforming the core of marketing activities.

News Clip Three01:19

It’s not just how things are changing and how we work, but how we create, connect, and tell stories.

Kartik Hosanagar01:25

On this episode, we’ll discuss AI’s transformative impact on creative industries with a focus on how AI is redefining the value proposition for agencies and their clients. The motivating question for today really is: is AI the best creative partner for agencies and creative workers, or is it potentially a threat in certain ways. To help us explore that, I am thrilled to introduce our guest, the CEO of Accenture song and the founder of creative agency, Droga5: David Droga. David, welcome to Where AI Works.

David Droga01:57

Thank you very much. Very happy to be here.

Kartik Hosanagar01:59

Before we even jump in and talk about AI, I’d like to just start off with your decision to start your own agency and become an entrepreneur. I’m just curious about that piece, being an entrepreneur myself. What was the gap you saw in the market, or what was the need for you personally that took you in the direction of starting your own agency, and you ran it for over a decade before it was acquired by Accenture.Absolutely. Yeah. And by the way, that makes the two of us. My most recent company, I gave myself the Chairman title, and it was AI for creative industry, specifically Hollywood. So there’s some of what you’re saying that resonates.

David Droga02:22

Great question, and I wish that it was built around a spectacular business plan. Now, you can probably tell from my accent that I’m from Australia. So, you know, I sort of grew up looking at both what was happening in the US and it was influenced by the UK. And my background is as a writer. So I sort of moved into the advertising industry just because I wanted to be a writer. And I moved up the ranks rather quickly, and I ended up running Saatchi and Saatchi in Asia, and then Saatchi and Saatchi in London as their Chief Creative Officer, and that brought me to the US. And for all the successes that I had and the learnings that I had along the way, and worked with some spectacular people, as someone who was a very passionate, creative person, I kept on feeling a love of the industry, but a sort of a disdain of the industry, because I felt like advertising had become almost like a tax for people, and most of it felt quite pollutive. But I could also see the power of it when it was done very, very well. And I always sort of go to this, this go-to observation that, you know, people in Hollywood and the music industry, people invent technology to steal what they create. But people were inventing technology to avoid what we were creating. And that, for me, was such a stark wake up call that we need to create better, more relevant, more inspiring, more effective things. I didn’t like working for other people so much, and I decided that, you know, if you’re going to start something, New York is the place. I didn’t have to be for everybody, but I knew I could be for somebody. The scale makes that possible. And my business plan, which really wasn’t a business plan, was I just wanted to make work that I thought was relevant and was making the most of the technology, because this was sort of the start of the digital age, and I wanted to work with people that I wanted to work with. And right from the get go, we just sort of started doing work that was amplified by digital, because digital suddenly made it that you were judged by the caliber of your ideas, not by the media budget that you had to spend. So it sort of gave opportunity and advantage to people that were putting the consumer first, not the media budgets first. And I did that for about 15 years as Creative Chairman, which was a made-up title, because I didn’t like the idea of being a CEO, the irony of now that I’m a CEO. So that was sort of my backstory, but it was I just wanted to make work that I believed in, and technology was an enabler, which is relevant to what we’re sort of talking abouttoday.Exactly.

Kartik Hosanagar04:03

You just said that one of the motivations was to make sure what you’re making is relevant in making use of the technology that was becoming available. Obviously, over the last couple years, there’s been a big shift in the technology that’s available. We’re starting to see, even in these early two, three years, AI’s impact in creative industries, and projecting out 5/10, years, the impact could be huge. And generally, I’m seeing its impact in many places. Now, you can talk about, say, in Hollywood, somebody with a great idea can go make short films or make movies without the blessings of the gatekeepers. It’s already affecting jobs for freelancers and simple creative tasks, like, say, logo design and so on. So I have a two part question for you. I think the first question really is, what does this technology mean for creative folks? Is it a tool that you see that’s going to augment and enable them, or is it a tool that’s going to displace them, and we can talk about it generally or in the context of advertising, which is where I want to focus this discussion.

David Droga05:46

My response to that is, I don’t think it’s quite as simple as that, because I think really it’s going to change jobs, but not the necessity of what the output of the jobs themselves. There’s always going to be a need for communication and brand and design and storytelling and, you know, all these things, how we get there, that is definitely changing. So when we stop holding on to a business model or a role and understand that creativity, taste, strategy, they’re liquid. They go wherever they’re needed. You know what I mean? And that’s why I’m very bullish and optimistic about it. You know, I know we’re in the infant stages, but I think people always expect me as a creative person, even though I’m a CEO, to sort of be defensive and push back and think that, you know, AI is going to gobble up everybody’s jobs and all that. I’m like, look, history has proven us over and over, you know, that it moves us forward. And then the creative minds, it’s what they do with it, their interpretation, misinterpretation of it, that moves us forward. I also think that, which is controversial, but I mean it when I say it, not all creativity is worth saving.

Kartik Hosanagar06:47

Interesting.

David Droga06:48

Now, that’s a provocative thing to say, but there’s so much things that are deemed creative that are just not. You know, I look at architecture, I look at majority of advertising, I look at a lot of journalism, that can be better. That, you know, that’s quite formulaic. I mean, they’re informed and influenced by something far more scary, which is conformity and research. Playing it safe. So you know, if we’ve created tools of get rid of the mediocre middle then that’s going to accelerate the people that actually have the talent and the know-how, and the ambition and the creativity to do more with it. Because every time creativity has been enhanced by a tool, yes, we all get intimidated by it, and obviously I understand the scale and pace of this is unlike anything we’ve seen. But I think of even when I started in the industry, I’m not looking back thinking, I feel terrible that we don’t have rooms full of typographers or still have rooms filled with illustrators who are doing storyboards. The output is what matters. And when you focus on the output, creatives will always be relevant. The tasks are changing, not the necessity of what people need.

Kartik Hosanagar07:47

Yeah, I like the way you put that, and I have the same belief that we can break down work into a set of tasks. And no doubt some tasks will go and some tasks will stay, and some new tasks will get created. You brought up the point of taste, I think that’s really interesting to me. It reminds me of a conversation I had with producer some time back, and that context was filmmaking, as I mentioned earlier, and when it came to AI tools, say, for producers to analyze scripts and so on. Initially, the reaction from some studios was: there’s no way our creative people will use AI because AI cannot be creative. Then it was: they feel threatened by it. But then you actually look at the job that’s happening, and you talk to the people. Turns out a typical person is overwhelmed with the number of scripts coming in, four books, six scripts to read in a weekend. And AI actually is great for them to understand. What is this about? Does this fit your mandate? Is it too similar to what you’re working with? And when it comes to what does this make me feel, and where do I want stories to go in the future that taste-making role, it’s very hard to not see humans being relevant in that setting.

David Droga08:54

Completely. I mean, again, a lot of what AI is doing is sort of bringing together all the best practices and all the knowledge that exists to formulate something new. And again, it could probably write scripts and movies that are on point with some of the average films that are out there. Of course it could. But it’s never going to create something and you take some people who push into new territories or have context and references that only a human that’s gone through the human experience can get to in certain pieces of dialog and and I think that’s why I feel I’m cautious and I care about when I say, you know, not all creativity and not all jobs are worth preserving. I don’t say that in a cold-hearted way. I mean, everyone deserves to have a job. I just think it’s going to give people new opportunities, and it’s going to industrialize the imagination. Because when everyone has access to best practices, then something new has to happen. You know what I mean, to sort of elevate beyond that. And we look out in the real world. As I said, taste is actually a real thing. It really is. Understanding and context are real things, making connections are real things. And our past is littered with things that allowed us to do things faster and more. And at the moment, it’s just allowing us to do things cheaper and faster. There was a big to do about some big fashion brand that had spent $600,000 making a esoteric commercial with someone diving in the water and swimming and whales and all this sort of stuff. And it was for, you know, a perfume or something like that. And I think they spent two months making the ad or three months, and it cost $600,000 and someone had used AI and recreated it in a matter of hours for 100 bucks. Which is extraordinary.

Kartik Hosanagar10:19

Right.

David Droga10:19

What no one’s talking about is both of them were crap. Now, what the advantage of that is you can get to crap faster, so you can move on from that. But the parlor trick of the technology is never going to be the thing that just makes the connection. It still has to have strategy and insight, particularly if you talk about things that motivate people and you want to move people and stuff like that, but it’s same with architecture, same with literature, same, you know, it can do great things, and we shouldn’t begrudge that. But it’s what we do with it, that’s the great thing about humanity, our interpretation of things, as I said, and we’re the ones who enforce ourselves into that.

Kartik Hosanagar10:53

Yeah, and I’m curious how you infuse that taste, that originality, in your own work, and I’m asking specifically whether there’s a trade off, especially when it comes to AI use, but there’s a trade off between efficiency and originality.

David Droga11:08

Well, at this stage, majority of people are still very much in their table stakes, and so they should be. There’s great efficiencies to be made: time, money and all those things. And so a lot of people think that, oh, this is going to solve everything. I’m never going to need to deal with a creative organization or a design company or get, you know, of course. So they’re sort of enjoying the- reaping the benefits of that, but that’s only going to get you so far. Again, our world is to sort of connect with consumers from a emotional point of view, from a commercial point of view, and brands have to be distinctive, they have to be meaningful. They have to be memorable. AI, can get you so far. But if everyone has access to the same tools, you’ll get the same outcomes all the time. So we sort of add that layer of context and differentiation.

Kartik Hosanagar11:51

I think your point about everyone having access to the same tools, you’ll have the same outcome if you’re not careful, is an important one. We actually had a study recently in our research group where we had 250 plus participants engage in creative tasks, mostly creative writing, some with AI, some without AI. And what you observe is that AI used reduced task completion time. It also increased the quality of the writing as rated by independent evaluators. But when all of them were using the same kinds of AI. What we observed was that the diversity of the writing went down.

David Droga12:25

Oh, of course. I mean, you know, think about when search first started. Everyone thought that everyone was going to be lazy and wouldn’t be able to do anything. And you know that, for instance, suddenly everyone had access to everything. And, you know, people cut corners and all that sort of stuff. But it still opens up new avenues. It opens up new quadrants of exploration. And I think, I also have to believe this sort of being transparent, you know, my wife is in film, my eldest son graduated USC film schools. My daughter got accepted last night into USC film school.

Kartik Hosanagar12:51

Oh, wow.

David Droga12:52

So I have to believe. Now, the cynic in me would be, and I’m a creative thinking, oh, are they going into an industry that’s going to be irrelevant? I’m like, the business model of the industry is maybe going to be irrelevant, but the need for storytelling and movies and evocative and emotional things is never going to be. So I really want them to understand the technology, work out how to bend it and use it and break it and sometimes ignore it. But I’m not so bullish that I’m not aware of the pitfalls. And you know that we have to be very thoughtful about it and consider- we have to have a little bit of paranoia that’s also important, to make sure that we sort of utilize it and use it in a way that’s responsible and fair, and it’s, you know, because there’s a lot of things that could go wrong as well.

Kartik Hosanagar13:32

So we’ve talked a little bit about what AI means for creativity. I want to talk about what it means for the business of creativity. Let’s start with the client side of things. What are you seeing in terms of how technology in general and AI in particular, is making people rethink roles and responsibilities within the enterprise. For example, what does it mean for CMOs, and how is it changing what they need from their marketing partners?

David Droga13:57

The CMO’s role has changed. It was already morphing before the emergence of AI, and that sort of moved from being, you know, responsible for just growing brand, to sort of growing companies. And I think that the voice of the CMO carries a lot more weight than it used to, or should, definitely it should need to be, sort of, you know, really, in the C suite. I think what AI is allowing us to do, and I’m seeing it firsthand with Song, which is part of the reason I sold my company to Accenture, and sort of the belief about the necessity of technology just to weave together all the facets of the world I’d come from. You know, we can actually do things. But the excitement of the output is tapered a little bit in the beginning, because it’s sort of exposing people like: Are they are they ready for it? Is their corporation ready for it? Is their data clean enough? And at the moment, it’s all fragmented and in silos. So there’s a lot of enthusiasm for it, but it’s also revealing, when you have the conversations about just how ready is your organization, your company, to do that. What’s happened is every company out there needs to have sort of a unified customer experience across every facet of it. You can’t separate marketing from the commerce or from the sales, orr from the products. You know, they have to be united, and AI is a way to sort of make that tangible and a reality. But as I said, there’s a lot of questions. It exposes a lot of flaws that people need to fix their digital core and sort of get their heads around what their data capabilities and is it ready for it?

Kartik Hosanagar15:15

Yeah. So if the challenge for a lot of CMOs is quite heavily in the data and technology camp, meaning, help me get a unified view of my customer, help me understand the customer journey and so on. Does it change anything in terms of how they work with partners like you, like agencies? Does it change anything in terms of what agencies do? Because what you’re describing primarily sounds like a technology issue, a data issue, more for the CMO, interacting with their CIO, CTO, versus with their agency partner.

David Droga15:46

Well, the thing is, and I say to you, as someone who built a well known creative agency, the model of that just being by itself from any agency can no longer exist, or it’s going to be sort of not irrelevant, but its power and influence is going to shrink.

Kartik Hosanagar15:57

Oh, I see.

David Droga15:58

So that’s why they have to morph it in and again, being transparent, 100% why I sold my company to Accenture and why even why I said yes to this job because I believed that a creative person should be at the head of a company like this, making decisions that are about brand and consumer, but make sure that strategy and creativity is woven into the data and AI, as opposed to a nice-to-have or an addition.

Kartik Hosanagar16:20

I see.

David Droga16:20

So there is consistencies, and there is ambition, and there is an understanding of humanity. Because, you know, creativity needs technology to scale, and technology needs creativity be more human and relatable.

Kartik Hosanagar16:31

You know, it makes me wonder what you think an ad agency will look like in five years. The kind of vision that you are espousing, I don’t mean to put words in your mouth, but at least what I’m taking away from what you’re saying is that it is hard to separate the creative insight and the creative strategy from the data and the technology piece. And part of what you’re doing is, you know, coming together with Accenture and now heading Accenture Song is you’re bridging the two together simultaneously and taking it to your clients.

David Droga17:01

100%. I mean, that’s our point of difference. I think that’s why we’ve grown exponentially. I’d love to think it was my genius, but it’s not. It’s sort of the alchemy of what we have, the sort of unified customer positioning and understanding. And it doesn’t sort of leave something on the sidelines. It’s consistent. Complexity is something that is one of the great stiflers, particularly at the pace that everyone’s moving now, and it does take out a lot of the complexity as well. Everyone’s still in the learning phase. I mean, I’m not pretending that we have all the answers, but we have a lot of the people that have the answers. And know, you know, we spend as much time trying to have clients get business ready, as well as customer relevance. You know, we’re putting those two things together. Sometimes they’ve been separated, but I think some of the Holdcos understand customer relevance, and I think some of the other consultancies understand business readiness. I think we’re the ones who sort of interlock those together because they influence each other and they need each other. And having a creative person at the helm, really, that means I’m just a product person. So I’m thinking about not just the integration and the seamless nature of it, but the output. You know, I always start with what’s going to work for the consumer, and that’s the best thing you can do for your client, is care about their consumer.

Kartik Hosanagar18:05

Yeah, it’s interesting to hear the blurring of these lines between the creative person and the product person. I think that’s quite fascinating.

David Droga18:11

It’s necessary, and again, I bang this drum too often. I’m not down on creative agencies, as I said, as a creative person, I’m bullish on creative people. I just keep on telling the agencies out there: change your business model, embrace the thing. You know, some of the companies we bought, a great company called Work and Co, and they’re a digital design products company. You know, they’ve created this new model where the designers and the engineers are partners in crime, so that just that sort of harmony between that. And then you’ve got the sort of agentic thing, which is going to change so much as well, because we spend a lot of time building that for our clients and making sure that they’re built in a way that has understanding in it, you know what I mean.And, you know, look, who knows in five years what we will be. But I like the idea that people who understand people, and not just engineers, are making decisions.

Kartik Hosanagar18:53

You’ve talked about, strategically, structurally, what changes you see coming in this space. Let’s talk about operationally, like inside an agency, day-to-day work, what are the changes you imagine? And one of the things that AI is allowing us to do broadly is reimagine work in general. And we’re seeing tons of money being poured in by VCs, and entire industries and functions are being re-imagined. With all these investments coming in, no doubt that many of these experiments will fail, but also many will succeed. And so I’m kind of wondering, what are the experiments you’re running in-house? You talked about agents, for example. So I’m kind of very curious internally, what are the changes that you see in terms of how the creative work gets done?

David Droga19:39

Well, I think that people who can ideate and have taste, I’m going to belabor those things are still necessary. You know, do I need reams of people who can do translations and storyboards and, you know, things that are time consuming? No, it allows people to get stuff out of their head faster, good or bad, right? So we can just operate at a different speed. And what’s the biggest change that I’m seeing in the agency space is, you know, there was that classic triangle that we used to say to our clients about speed, quality and cost. You can only pick two, that whole thing. What AI has allowed us is to take that out of the equation. You can actually do all three. Now, you can’t outsource to it all completely and just think that it’s going to solve everything but the repeatable, scalable part as one now. You know what I mean. Part of what I said to Droga5 is, you know, part of their superpower now is, yes, they still have the DNA core of being creatively, strategically-led. But the conversations and the people that have a voice at the table are technologists as well, and data scientists and people that actually understand how things are made and scaled anddeployed. And that makes a big difference. And then once creatives get over the nervousness of this is going to make me redundant or compromise me. And they go actually, I can do things that I could never do before. Yeah, that’s where it gets really exciting. And we’re still at that early stages of that. As I said, everyone out there is still at the, mainly at the efficiencies and speed phase of things. But like with all technology, soon we’re on the cusp of getting to the “I could never dream of doing this before”, now we can actually do that.

Kartik Hosanagar21:03

I guess, one closing question for you, David, is: what’s your advice for creative folks who are starting their career now? The world is so different than when you were starting so for most industries, most jobs, the old ways no longer work. So what would you ask them to do? What kinds of skills or what kinds of approaches they should be using early in their career?

David Droga21:23

Well, success is not nostalgic, right? We are, but success is not. Okay. You can be nostalgic for emotional things and again, the ramifications of things, but don’t look back and think, okay, you know, I have to be making a TV commercial, or the best team is a writer and an art director sitting together, or be open to experimenting more, as I said, have more faith in creativity, but also insert yourself in more places and learn more about it. When I took this job, it was the first time I’d been intimidated and learning in the last 20 years, and it was interesting and fascinating, because once I got through the sort of understanding and sort of I don’t pretend to have a grasp of everything, but I’ve moved past the instruction manual phase into the oh, what this means, and what could we do with it? Because I know that I, or young creatives even more so, look at things and see the world in different ways. You know, remember, engineers make things and expect the world to work out how to use them. Yeah, creatives and designers make things around how the people work, and what works for people, and it’s just a different mindset. So I just, I keep on saying to them, just know that you’re more relevant in more places than you’ve ever been before, and put yourself out there, push your boat out. Becausecreative voices, we ask different questions. You know, we don’t just give different answers, we give different questions, ask different questions. And that’s what sets us apart. So I’m the one who’s saying the model may be broken, but the need for your role, if you’re a genuine, creative person or a strategist, has never been more in need.

Kartik Hosanagar22:48

We’ll end on that very positive, uplifting, inspirational note. I think that’s a good call to arms for creative folks. David, thank you so much for joining. It’s been a great pleasure to have you on the show.

David Droga22:59

Really loved chatting to you, Kartik, thank you.

Kartik Hosanagar23:02

I thought there were two really interesting takeaways from this conversation with David. The first thing is that while AI is helping automate and augment a lot of creative tasks, the role of the creative still remains. This is because while AI can add a lot of efficiency into generating drafts, into creating early mock-ups, even creating ultimate visual designs; there’s an important role for humans to be taste-makers, to see what actually gets emotional response from the audience and so on. The other takeaway is the blending together of the creative and the product person. David talked about how in many ways, the creative person has to almost be the doer as well. I brought up the analogy that entrepreneurs need to be able to build product prototypes themselves in other domains. Similarly, I think in creative domains, it’s the creative, the idea generator, who should be able to use the right tools and create prototypes of their ideas. And ultimately, that ensures that they can go from idea to the final product in a more effective manner.

That’s it for today’s conversation. Thank you so much for listening. Please follow us so you don’t miss an episode, and be sure to tune in next time for our season finale. I’ll be chatting with Lan Guan, who is Accenture’s Chief AI Officer, and we will look at how AI is transforming enterprise strategy, customer interactions and business growth. This has been Where AI Works, Conversations at the Intersection of AI and Industry, brought to you by Wharton in collaboration with Accenture. I’m Kartik Hosanagar. Bye for now.

Kartik Hosanagar 23:42

Now let’s review the key learnings from my conversation with Jonathan. A big initiative like AI driven change is likely to throw some early challenges. Like Jon was talking about images where people had giraffe necks, or six or seven fingers. What’s really important is one needs to have patience, because you’re investing in AI for the long run. The other learning was that AI value is not just about cost reduction. It’s also about consumer engagement and revenue impact and increasing top line increasing consumer retention and things like that. And finally, it’s important for companies to find the right and consistent strategic partner, because AI driven transformation is as much about change management. How do you convince people? How do you bring them along for the ride? How do you create demos to get some early excitement about what you’re trying to do, and how do you make sure your people are with you in this process? That brings us to the end of the series premiere. Thanks so much for listening. Please follow us so you don’t miss an episode, and be sure to tune in next time when I sit down with Jill Kramer, CMO at Accenture. We’re going to explore how AI is reshaping marketing. This has been Where AI Works, brought to you by Wharton and sponsored by Accenture. I’m Kartik Hosanagar. See you next time.

Close ×

Transcript: Episode 4

Lan Guan  00:01

When ChatGPT Phenomenal first happened at the beginning of 2023,everybody was scratching their head, oh, wow, this is so disruptive, there’s no playbook. But I think now, two and a half years later, we have learned a lot. We’re still learning. Now it’s our job to go to all these clients to say, okay, let’s focus on the how.

Kartik Hosanagar  00:25

Hello and welcome to the season finale of Where AI Works, Conversations at the Intersection of AI and Industry, brought to you by Wharton, in collaboration with Accenture. I’m your host, Kartik Hosanagar, Professor of Operations, Information and Decisions at Wharton and Co-Director of Wharton Human AI Research Initiative. It is our goal to cut through the noise, to deliver actionable insights for business leaders by combining cutting edge research with real world case studies. Things are changing fast, so without further ado, let’s dive in.

News Clip One  01:01

What we see in the data is that the executive urgency to incorporate AI is at an all time high.

News Clip Two  01:06

There’s been a lot of buzz, maybe even hype, around if you don’t adopt this now, you’re going to be left behind.

News Clip Three  01:13

Business is a game, and if you want to compete, if you want to win, you have to use AI the right way.

Kartik Hosanagar  01:19

On this final episode of our season, we’re going to transition from AI strategy and use cases in marketing, to implementation at scale. To help me unpack the challenges and opportunities for business leaders, I’m thrilled to introduce someone who has helped lead her company to its fastest growth in its history by booking over $3 billion in generative AI related business last year. Lan Guan is the chief AI officer at Accenture. Lan, welcome to Where AI Works.

Lan Guan  01:49

Thank you for having me. I’m so excited to talk to your audience about AI

Kartik Hosanagar  01:56

There’s really a lot to talk about today, but where I want to start off is with your role, your title, Chief AI Officer. It’s a relatively new role in organizations. It’s not too long back, I remember even the CTO and CIO roles were new in organizations. Until then, IT was like a back-office enabling function, and then it suddenly became strategic to drive business transformation. So what is the role all about, and how did it get formed within Accenture?

Lan Guan  02:24

So I’ve been playing this role for a little bit over two years now. First and foremost, I’m responsible for our company’s overall AI strategy in the market. So that means things like: What do we want to be famous for? Where do we invest and how do we work with ecosystem partners? Who do we work with? I’m also responsible for developing highly differentiated industry solutions, assets, platforms, accelerators for Accenture, so that our practitioners can be bringing this kind of tangible capabilities to our cross-industry clients. Last but not the least, I’m also responsible for leading 65,000 data and AI practitioners. It’s different from the typical IT role, because I always talk about AI, comparing to data, comparing to cloud, is so much closer to the business.

Kartik Hosanagar  03:23

Makes sense. I think it’s so exciting that you get to help define what this role ends up being, not only within your organization, but over time, other companies will draw inspiration, and they’ll figure out what that role means for their firms as well. Where I wanted to really take this conversation too is understanding what is the current status of AI implementation today? Where are enterprise AI investments currently being directed? What kinds of applications are companies buying and building? I was wondering if you can talk about that both in terms of within Accenture, but you work with so many clients with their AI implementation. So what does it look like across hundreds of thousands of companies?

Lan Guan  04:03

Yeah, sure. You know, oftentimes we get asked, is this reality or hype, right. So I think I can be very upfront here. In this case, given the results that we have seen, this is quite real. You mentioned 3 billion in revenue that we have been booking for gen AI. In fact, it’s 5.6 billion over the last 18 months. Okay.

Kartik Hosanagar  04:25

Oh, amazing. That’s amazing.

Lan Guan  04:27

This is all public.

Kartik Hosanagar  04:28

Good for you.

Lan Guan  04:28

And also, we have completed 2000 gen AI now agentic AI projects. The progress that we’ve been making in the market is just enormous. Let me also give you a couple of data points here. So our latest research shows that 86% of the C-suite clients, they’re actually planning to increase their investment in AI. 60% actually expect their gen AI solutions to be scaled across their organization, up from 36% in 2024. So I think the point here is this is real. They are spending money on this. They’re shifting their spend from other areas or into AI, or they are just increasing the net new spend. Another piece of the research we have done also indicated less than 10% of the clients have actually scaled. In fact, the number is 8% so everybody’s asking, okay, what happened to the other 92%? So it’s our responsibility to collectively go help them. I think the second part of your question is, what are some common use cases, right? So we actually tell our clients that you should be thinking about your AI investments in two categories. The first category is what we call table stake, right? Think about those no-brainer areas that every company should be investing in. For example, in marketing space. The second category is what we mean by more strategic bets, right? So these are the areas that tend to be associated with the client’s industry value chain. Okay, like, for example, in life science, one strategic bet that we’ve been investing in is actually helping a lot of life science companies to speed up their clinical review process, leveraging AI, leveraging data, so that they can release the new drug faster into the market. In telcos case, using gen AI and deep dura network to help improve their network operations. In banking industry, for example, deepen the bank’s ability in anti-money-laundering, so all these areas. So I think holistically, we believe this technology is ripe for enterprise reinvention, and we go to clients very strategically. Help them identify low hanging fruit area, table stake areas, so that you can reap the benefits faster, but at the same time, also help them develop this kind of a roadmap so that they can see continuous reinvention, continuous streams of ROI coming in by also tapping into strategic bets.

Kartik Hosanagar  07:09

I think one of the things that companies have to contend with is a problem of plenty in this space, meaning that almost every exec, especially CTOs I talk to, talk about how they’re being bombarded by lots of new vendors who have new AI-powered substitutes for software that already use, or new workflow automation kinds of solutions. But how does one think about where to prioritize one’s attention with adoption decisions, and especially on the strategic side. I’m understanding that on the table stakes side, yes, you know you have opportunities to increase efficiency. You should certainly go pursue those. But I’m really curious how you advise your clients, or how should one think about how to prioritize the attention when you’re being bombarded on a daily basis with new AI software?

Lan Guan  07:59

Yeah, people can easily get lost, right? Even just in the year of 2024 we counted more than 200 large language models being developed by all kinds of companies, not to mention, now it’s all about agentic AI. So I think from the C-suite perspective, we basically asked them the question, right? What is your strategy? What are you trying to achieve? Are you looking at AI as a competency-enhancing technology or something that is actually competency-diminishing? So this kind of question, it’s almost like the light bulb moment to them. Because it’s very easy for people to get enamored by this kind of a tool conversation and then forget about, what is the “so what?” That’s why we lay down five tricks that everybody should be talking about. I think the first one is, like I said, focus on the problem, not the tool. What is that strategic needle that you want to move? What is the competitive advantage that you want to be famous for? The second one not to get carried away by a lot of this technology conversation. So you stay calm, you stay focused. It’s actually about taking advantage of this kind of platform-based approach, as opposed to chasing all kinds of use cases. So let me tell you one example, a very large energy client that I was working with in the Middle East, okay, they’re basically telling me that, within their company, because of the size, because everybody is so interested in using gen AI, on a monthly basis, they are getting more than 800 use cases coming from all parts of the company. Everybody’s submitting this kind of use cases. So they came to us saying: Hey, should we be catering to all these use cases? So their approach is actually quite smart. They look under the cover, with our help, across all these use cases, they realize that there are a lot of the common building blocks. For example, most of these use cases involving working with some kind of unstructured data, so they had that light bulb moment. Hey, now in this case, why can I just invest in some kind of vector database? Why can I just go build, on the enterprise level, some kind of vectorized embedding pipeline, not just working with the textual data, also working with multimodal data, so that this kind of capabilities become the foundation of the platform. Then every single use case from this repository that they’re getting from their employees and different business functions is leveraging this industrialized, commonly governed capabilities. So we’re seeing this across the board. Now, sometimes I say, okay, when chatGPT Phenomenal first happened at the beginning of 2023, everybody was scratching their head, oh, wow, this is so disruptive, there’s no playbook. But I think now, two and a half years later, we have learned a lot. We’re still learning. Now it’s our job to go to all these clients to say, okay, let’s focus on the how, and look at this kind of platform-based approach, look at this kind of a reference architecture, and start investing in this kind of strategic bet. I think that’s something that is super, super important.

Kartik Hosanagar  09:29

So, you mentioned, focus on the problem, don’t get carried away by the technology. So that’s two. I think you said there were five.

Lan Guan  11:25

Yeah, I’ll quickly go through the rest of them. Also be pragmatic about the embedded AI. A lot of CIO/CTO clients, the first question they tell me is: I’ve been investing in a lot of technology in the past. So how do you make sure that Gen AI, or agentic AI is working with my existing technology landscape. If they have the enterprise platform, if they are working with Salesforce, if they are working with SAP or ServiceNow, they all have embedded AI capabilities. So they work quite well with the data that is specific to the platform. So I always tell my client, get started, turn on those capabilities. So that’s the third one. The fourth one is that you need to be demanding substance from the vendors, okay, meaning, it’s not just the buzzword, it’s focused on asking the tough questions, right? How did it works? How is that trained? What are the differentiators between your agents and the other agents? So I think asking these kinds of questions from all kinds of vendors is super crucial. The last one is actually about aligning with your data and talent readiness. The old phrase of garbage in, garbage out, is becoming even more relevant in this case. So I have seen so many examples, so many clients, they got excited to start AI, they thought that they are on the right track, and then they hit the road bump because their data is not ready. Their talent is not ready. Let me give you one quick example. One of the very large telco clients I was were personally working with, this is exactly what’s happening to them, right? They are very anxious to implement contact center agent solution. And then right at the moment of pushing this into production, they realized the model performance degraded, right, drifted. And then they realized: Oh, it was because the training data that they used did not have the good quality. They actually have 37 versions of standard operating procedures for their human employees on the floor. If it’s a standard operating procedure, SOP, how can you have 37 versions? It’s supposed to be one single source of truth. What we are dealing with every day is this enterprise messiness, meaning, data is messy, process is messy because of all kinds of merger acquisition that is happening, right. Talent shortages, right. I work with a lot of clients, they basically have been investing in the last two decades, building data science talents, building data engineers. Now I’m going to them say: Did you know your data science talents need to get re-skilled? Because now, in order for talent to thrive in the age of AI, they need to learn software engineering, they need to learn design, they need to gain this kind of a full stack capabilities, right? So a lot of times clients are like: Okay, that is a gap. How can you help me?

Kartik Hosanagar  14:16

Yeah, it’s so interesting you say that. It makes sense, but at the same time, is something that companies haven’t necessarily paid a lot of attention to. This point about to get those returns in AI investments, you need first the investments in data and the talent. And in fact, at Wharton in our center, we have several studies. My colleague Sonny Tambe has studies that have looked at AI investments by public companies and then their returns in terms of stock market returns, their revenues, profitability and so on. And it’s pretty clear that firms that are getting those returns, their AI investments are preceded by the data infrastructure investments, and they also tend to be companies that are in regions or geographies where there’s a large talent base that they can recruit from. Lan, I heard you mention multiple times agents. Now, in the previous conversations that we’ve had on this show, it’s come up, but nowhere near as many times as you, which makes sense, you’re the Chief AI officer. So we should spend some time talking about that. A lot of people believe agents will transform software as a service itself, because now you can replace traditional software applications with agents that are able to take action, listen to a sales call, analyze the call, and take some follow up action, like sending a follow up email to the customer. So even the human interaction with software sometimes becomes irrelevant. There’s so much hype around this, but it’s also not a mature space, it’s very early. So I’m really curious to hear, what kinds of experiments are you running in the agent space, and what are the big challenges and roadblocks to deploying large scale agent-based systems in organizations.

Lan Guan  15:58

So Kartik, we are the early adopter of HFT AI. So one of the best examples is actually the marketing reinvention that we were doing to ourself, because you know, as a large organization, we have lots of marketers, B2B marketing is something that is very important to us. Last year, we built AI refinery together with one of our partners in media, in this case, and we built 15 agents to basically reinvent the strategic planning, because we believe that’s the area that is almost like the crux of the marketing, right? Because it involves a lot of a cognitive task. It involves a lot of a permanent task, a lot of coordination, right? All these characteristics that matches the agent’s capabilities. So let me give you one example of the agent, utility agent level, like calendar agent. Calendar agent, why is that important? Because in B2B marketing’s case, we always have multiple campaigns coming out, multiple campaigns in multiple geographies. So how do you avoid the crashing calendar, so that we don’t have campaigns with the different objective, different messagings hitting the same audience. Today, it’s usually done by people, manually creating marketing campaign calendar. Oh, okay, this sounds like a classic assignment problem. Can we build operational research mini agent to solve, resolve a lot of this calendar conflicts?To optimize audience impression, optimize the marketing budget, to make this much more comprehensive and much more optimal. In terms of the challenges, right, the second part of your question, I think there’s also three things that I think is slowing us down. The first part is because of this confusion around what is agent, how is that different from RPA, should I be using agent together with large language models, especially thinking about reasoning capabilities? I think we, as an industry, we need to come together and actually put the stake in the ground to say: Okay, this is the definition: Agent needs to have these kinds of four characteristics. Agent needs to be able to perceive the outside world. Agent needs to have the cognition, reasoning capabilities to troubleshooting, do diagnostic, breaking down larger problems into smaller chunks. But that’s not just it. Also taking action, to using a function calling, giving the tooling part of the agentic capability is super critical, right? Connecting with your enterprise systems. Last one is learning, right? Because ultimately, agent is not perfect. So how do you institute this kind of self reflection so the agent can continuously learn, be environmentally aware. We need a clean way, very crystal clear way, to describe the power of agents, so that a lot of noise will actually be not there. I think that’s one challenge. The second challenge, I would say, is agent interoperability. I still think that there are a lot of gaps that we need to be closing. We don’t want this technology, super powerful, to also create fragmentation, to create silos. To me, that’s more tangibly, how do we solve the agent interoperability is a big challenge. The third one is, I would say the agent trust. Yes, we talk about responsible AI, explainable AI, probably for the last two decades, it became another buzzword when LLM came out, everybody started talking about AI safety. But Kartik, I can tell you, the importance of AI trust is paramount now, simply because of the proliferation of AI agents, not everybody will be creating LLMs, not everybody will be creating SLM, right? Model Customization still seems to be limited to the technical resources, but that’s not the case with AI agents. I have business users creating AI agents. I have power users who want to create co-pilot agents. Then you have the out-of-box agents. I’m just telling you, the proliferation of the agents make this so critical to actually bring the trustworthiness to everything we do. Because how do I know this agent created by Mary in supply chain should have the right role level access. Is this the authoritative one for research? All these other questions that my client is asking.

Kartik Hosanagar  20:12

It brings up a lot of questions about governance. I mean, you brought up proliferation of agents. You brought up how agents are being created and used by business teams and so on. But let’s talk about what can go wrong. And one, I guess, news story that is very fresh in my mind was, I don’t know if you saw the one about Cursor AI, they had the customer support agent Sam that was automating a bunch of customer support functions and sent people emails saying that you can only use our tool on one device which was incorrect, caused a bunch of people to unsubscribe, and then the company had to then follow up and say “No, that’s not our official policy. Our AI agent did that incorrectly.” So the question here is, there’s a huge opportunity around AI, but also a lot of things can go wrong when you do this level of automation, and AI has access to resources, can make decisions, and so on. How do you think about that governance? What needs to be in any governance framework for these companies?

Lan Guan  21:09

That’s a very good question. I think the example that you gave is very real, right? One of my similar examples I always talk about is a terrible story, or horror story that, okay, now in contact centers case, now you have agent out of the control. Agent just give refund to every customer. So to me, that’s not the kind of things we want to see happening. AI is surely becoming so autonomous. But how do we actually not slow down the progress, but also put a guardrail around this and basically say “Okay, how do we balance the risk and reward for this kind of imperfect technology?” So our point of view on AI governance, I think the entire playbook needs to be rewritten simply because of all the things that we’ve been talking about. A lot of cases, we tell the client, this is the area that you need the professionals help, doing this homegrown things can get out of control very easily. Professionalism of the gen AI, agentic AI, is quite important, right. Which means asking for outside help, right, don’t try to do all this by yourself. I think the second one is, how do you automate a lot of this instrumentation? How do you use algorithmic way to actually measure agent trustworthiness, right? How do you break down something like this into things like explainability, things like access control, security, safety, ethics. To me, that is the new way of addressing this kind of AI trust, and how do you do that in the highly automated way, so that you’re not just talking about theory, it’s not just talking about the guardrail, it’s not just talking about the design principle, because this is what every single client would need. It’s another huge gap area that I think we, as an industry, need to come together to close.

Kartik Hosanagar  22:49

Lan, this has been a great way to wrap up our first season. Thank you so much for joining us and sharing your insights here on Where AI Works.

Lan Guan  22:58

My pleasure, Kartik.

Kartik Hosanagar  22:59

Now I would like to highlight a few takeaways from my conversation with Lan. Lan brought up the fact that many companies are investing heavily in AI, especially generative AI, but many struggle with execution. The biggest roadblocks include integrating AI with proprietary data, building the right data and talent infrastructure in-house, and scaling beyond small experiments. We also talked about AI governance and trust, and there are many dimensions to it. Companies must prioritize explainability, transparency and ethical considerations to build trust, especially as AI becomes more autonomous with agent-based systems, organizations will need to rethink their trust models and establish clear guardrails.  That’s a wrap on Season One of the podcast. Thanks so much for listening. It’s been a great pleasure taking part in these conversations, and I hope you have enjoyed them as much as I have. Season Two is just around the corner, so be sure to follow us so you don’t miss an episode. This has been Where AI Works, Conversations at the Intersection of AI and Industry, brought to you by Wharton in collaboration with Accenture. I’m Kartik Hosanagar, bye for now.

Close ×

Transcript: Episode 5

Jonathan Halvorson 00:02

For brands who aren’t clear on exactly who they are in their foundations, I think they will findthemselves in a sea of sameness.

Lan Guan 00:09

People can easily get lost. Focus on the problem, not the tool. What is that strategic needle that you want to move?

Speaker 1 00:17

Cohorts, communication, and use cases. They’re my quick tips to bringing people along on this AI journey.

David Droga 00:25

Creatives will always be relevant. The tasks are changing, not the necessity of what people need.

Kartik Hosanagar 00:33

Hello and welcome to Where AI Works, conversations at the intersection of AI and industry, brought to you by Wharton in collaboration with Accenture. I’m Karthik Hosanagar, back in the hosting chair one last time for a short recap and review of our inaugural season. We covered a lot of ground in those first four episodes, so I want to look at some highlights and break down my key takeaways for you. As I’ve been saying all season, things are changing fast, so let’s dive in. We kicked off the season by speaking with Jonathan Halvorson, the global SVP of consumer experience at Mondelez International. We talked about AI-driven content creation from a marketing perspective. My research shows that while AI can boost content efficiency, it may also lead to more similarity in content as everyone uses the same tools.

Jonathan Halvorson 01:22

I think in marketing there are some really exciting and clear use cases, and I think that’s why a lot of companies are starting in marketing. We get excited about focusing on content because we see the size of the opportunity. One of the biggest line items in any company’s budget is going to be what they spend on media to place advertising and what they spend on content creation. And if a company can improve the effectiveness of that content in media, or can improve the cost that they purchase that with, it’s going to have a big impact on the P and L.

Kartik Hosanagar 01:52

Jonathan emphasized the importance of embedding brand distinctiveness in AI-powered content, meaning that you want to make sure that your brand’s truths don’t get lost in AI-enabled content creation.

Jonathan Halvorson 02:05

What is the product truth that is core to the brand? What is the brand purpose? What is the tension it plays against? And if you define those things clearly, then I don’t think that you’re going to have that swell to the average and to the mean. But for brands who aren’t clear on exactly who they are in their foundations, I think they will find themselves in a sea of sameness. So I think the safeguard on it is: one, the clear brand foundations; two, humans in the loop; and three, just the discipline you put in, just the intentionality of how you use AI, and just recognizing the concern.

Kartik Hosanagar 02:40

On episode two, I sat down with Accenture’s Chief Marketing and Communications Officer, Jill Kramer. Jill talked about moving from fear and avoidance to excitement and curiosity about AI. Her journey felt deeply human to me.

Jill Kramer 02:54

I would love to tell you that I was fearless, that I was first in line with my hand up, saying let’s do this. I did start this thinking, wow, like, is this an existential crisis? And I made the decision, and I was very transparent about it with the full marketing and communications team at Accenture that I desperately wanted to be in the driver’s seat. And the only way you can do that is by getting really close, getting very hands on, deeply understanding, and taking some risks on behalf of yourself, your company, your brand, your team.

Kartik Hosanagar 03:27

We also got into the nuts and bolts of re-skilling teams at scale. A few of Jill’s insights: one, start small, move in cohorts. Don’t try to boil the ocean. Two, over communicate. The vision may be clear to you, but it takes repetition to make it resonate across the org. Let me repeat that. The vision may be clear to you, but it takes repetition to make it resonate across the org. And three, choose early use-cases, carefully. Nail the first two and employing curiosity and creativity, it will take off from there.

Jill Kramer 04:01

Curiosity and creativity is driven by exposure, by options, by the least restrained possibilities. And Gen AI unleashes that on behalf of the human, on behalf of the craft. So if you allow it to be shorthanded too, can’t Gen AI just write that for you. It’s an incredible disservice to the technology and the potential of it being applied to a function as important to growth and brand and, you know, the strategy of any given company as marketing is.

David Droga 04:33

Not all creativity is worth saving. Now, that’s a provocative thing to say, but there’s so much things that are deemed creative that are just not. You know, I look at architecture, I look at majority of advertising, I look at a lot, a lot of journalism – that could be better, that’s quite formulaic. I mean, they’re informed and influenced by something far more scary, which is conformity and research. So if we’ve created tools that allow us to sort of raise up and get rid of the mediocre middle, then that’s going to accelerate the people that actually have the talent and the know how and the ambition and the creativity to do more with it.

Kartik Hosanagar 04:33

For episode three, it was a real pleasure to chat with David Droga, the founder of Droga5, and the CEO of Accenture song. We talked about how AI changes the game for creative teams. David also mentioned that it’s natural to be fearful, but he doesn’t think AI will replace or eliminate human creativity.

David Droga 05:28

Every time creativity has been enhanced by a tool, yes, we all get intimidated by it, and obviously I understand the scale and pace that this is unlike anything we’ve seen, but I think of even when I started in the industry, I’m not looking back thinking I feel terrible that we don’t have rooms full of typographers or illustrators who are doing storyboards. The output is what matters, and when you focus on the output, creatives will always be relevant. The tasks are changing, not the necessity of what people need.

Kartik Hosanagar 05:54

And for the fourth episode, I was joined by Accenture’s Chief AI Officer, Lan Guan, to discuss AI at scale. She mentioned that many companies are investing heavily in AI, especially generative AI, but that execution can still be a struggle.

Lan Guan 06:09

Our latest research shows that 86% of the C-suite clients, they’re actually planning to increase their investment in AI. So this is real, they are spending money on this. They’re shifting their spend from other areas into AI, or they are just increasing the net new spend. Another piece of the research we have done also indicated less than 10% of the clients have actually scaled, in fact the number is 8%. So everybody’s asking: Okay, what happened to the other 92%? So it’s our responsibility to collectively go help them.

Kartik Hosanagar 06:44

We also talked about the biggest roadblocks to implementation, including issues around trust and governance, as AI becomes more autonomous with the rise of agentic systems.

Lan Guan 06:53

Yes, we talk about, you know, responsible AI, Explainable AI, probably for the last two decades it became another buzzword. When LLM came out, everybody started talking about AI safety, but I can tell you, the importance of AI trust is paramount now, simply because of the proliferation of AI agents. Not everybody will be creating LLMs, not everybody will be creating SLMs. Model customization still seems to be limited to the technical resources, but that’s not the case with AI agents.

Kartik Hosanagar 07:27

Our guests shared a lot of valuable advice and insights over the course of these conversations, but my main takeaway for business leaders is that AI-driven business transformation is not just a technology- based exercise. In fact, I would argue it’s primarily a people exercise, because one of the things that emerged from these conversations is, while the technology is there, we need to make sure that our people can embrace the change without fear. We need to make sure that culture change is feasible, and all of that requires much more from an organization than turning on a switch of technology. And of course, if you haven’t listened to the first four episodes in full, I would also very much encourage you to do so. This has been Season One of Where AI Works, conversations at the intersection of AI and industry. Brought to you by Wharton, sponsored by Accenture. Thanks so much for listening. Please follow so you don’t miss an episode. In just two weeks, we’ll be kicking off season two, hosted by my very own colleague, Serguei Netessine, and he’ll be focusing on the challenges surrounding the monetization of AI, an extremely important topic. I’m Kartik Hosanagar, bye for now.