Brought to you by the Wharton School in collaboration with Accenture, Where AI Works explores AI’s real-world impact on business. Each season takes a fresh approach, led by a different Wharton faculty expert who brings their own AI-focused expertise to the conversation, alongside practitioners actively shaping AI’s role in innovation, strategy, and transformation.
“AI is rewriting the playbook for business and society. In Season 1 of Where AI Works, I’m excited to explore what that means for decision-making, marketing, and beyond.
Kartik Hosanagar
Professor of Technology & Digital Business,
Co-Director of Wharton Human-AI Research
Host, Season 1: The Impact of AI in Marketing
Brought to you by the Wharton School in collaboration with Accenture, Where AI Works explores AI’s real-world impact on business. Each season takes a fresh approach, led by a different Wharton faculty expert who brings their own AI-focused expertise to the conversation, alongside practitioners actively shaping AI’s role in innovation, strategy, and transformation.
“AI is rewriting the playbook for business and society. In Season 1 of Where AI Works, I’m excited to explore what that means for decision-making, marketing, and beyond.
Kartik Hosanagar
Professor of Technology & Digital Business,
Co-Director of Wharton Human-AI Research
Host, Season 1: The Impact of AI in Marketing
Listen to Season 1
Episode 1: The Future of Marketing: AI-Driven Content Creation
Episode 2: Balancing the Algorithm: How AI Is Reshaping Marketing
New episodes will drop every two weeks
Season 1: The Impact of AI in Marketing
In Season 1, Wharton Professor Kartik Hosanagar explores AI’s transformative role in marketing—how AI-powered social listening, digital advertising, and creative automation are reshaping strategy, customer engagement, and brand trust.
Episode 1
Guest: Jonathan Halvorson, SVP, Consumer Experience; Mondelēz International
Transcript: Episode 1
Jonathan Halvorson00:01
With everyone having these tools to be able to produce creative, what differentiates people? I think what matters a lot is going to be the differentiation of the brand and the foundations. What is the product truth that is core to the brand? What is the brand purpose? What is the tension it plays against? And if you define those things clearly, then I don’t think that you’re going to have that swell to the average and to the mean. But for brands who aren’t clear on exactly who they are in their foundations, I think they will find themselves in a sea of sameness.
Kartik Hosanagar00:34
Hello and welcome to the premiere episode of Where AI Works, conversations at the intersection of AI and industry, brought to you by the Wharton School and sponsored by Accenture. I’m Kartik Hosanagar. I’m a professor of technology and digital business at the Wharton School and also co-Director of our AI research center, and it’s my pleasure to be your host on this season of the podcast, we’re going to tackle some big questions about how AI is shaping the world of business today and where it’s going to take us. It’s our goal to cut through the noise and to deliver actionable insights for business leaders. On every episode, we’ll combine cutting edge research with real world case studies and uncover how the best companies are using AI to upskill their workforce and transform their business. Things are changing fast, so without further ado, let’s dive in.
News Clip One01:24
We are definitely in an AI hype cycle, and artificial intelligence is a very big and very important macro trend.
News Clip Two01:31
The race is no longer just about scale or speed, it’s about intelligence that can think and reason.
News Clip Three01:37
AI is replacing human tasks faster than you might think.
Kartik Hosanagar01:41
AI has been around since the 1950s and computer scientists have been working on it for several decades now, but it’s really taken off in the last two years, and all of a sudden it’s become extremely important for the C suite. It’s likely to impact everything from marketing and sales to software development, R and D and other functions, and likely across all industries. This inaugural episode of Where AI Works starts with marketing, and in particular, how is AI super charging the way brands are creating content to help me unpack the answers, I am pleased to introduce the global Senior Vice President of consumer experience at Mondelez International. Jonathan Halvorson, Jon, welcome to Where AI Works.
Jonathan Halvorson02:25
Thank you so much for having me.
Kartik Hosanagar02:26
Not everyone is likely familiar with Mondelez, but I suspect the brands that are within Mondelez are known to all of us. So maybe I’ll ask you to first just introduce Mondelez and tell us a little bit more about the company.
Jonathan Halvorson02:39
Sure, Mondelez is the global leader in snacking. We operate in a few core categories that you’re very familiar with: chocolate, biscuits, baked snacks, you’d be familiar with: Oreo, Milka, Cadbury, Ritz. So, a true combination of global and local brands operating everywhere around the world,
Kartik Hosanagar02:57
From what I understand, your background in college was investigative journalism. So I’m kind of curious how you go from investigative journalism to advertising, and in particular, Mondelez as well.
Jonathan Halvorson03:08
So I attended the University of Missouri undergraduate program in school of journalism, and why I was there, I was sitting in history of American journalism, and they were doing a preview of all the sections. It’s that moment where you decide, am I going to be a newspaper or a broadcast? And a professor, Steve Capch, came in to present what the vision was for STRATCOM, and he had an unbelievably brilliant articulation of exactly what STRATCOM was. And it was the just really the compelling nature of it, how he talked about writing, the clarity of thought, and just really building brands. And I sat there, and I scrapped all my dreams of being an investigative reporter or working a broadcast beat, and I signed up for advertising. A few short years later, I would go to work at Leo Burnett and later on, in media organizations, OMD, Publicis, working around the world with a lot of blue chip clients. And fortunately, I had Mondelez as a client, and later on, had a chance to join them about eight years ago in the capacity I’m in now running global media, digital data, creative and digital commerce. For us, I think ultimately AI is going to work from the bottom of the funnel and come up. So the beginning uses of AI that we see are really in D commerce, building product display pages. So, the images for that, the text optimizing that, for search engine optimization inside retailers. The second use case that we see is really in producing everyday advertising content, the display assets and banners you see that are meant to drive conversion and are slowly working way up to social assets. So lifestyle imagery that you would see and Instagram posts and Tiktok, and we continue to progress up, and we see a clear path to doing video and even our above the line TV assets.
Kartik Hosanagar04:09
Got it and in fact, I think advertising and content creation is a great place for us to kick off this series of conversations we’ll have on the show on AI and marketing. And I want to start by sharing a study that some of my colleagues at Wharton did where they were looking at which industries and job types are going to be most impacted by AI, and the top two in that list were media and entertainment, and legal. And of course, media and entertainment includes advertising. Where I want to begin the conversation is perhaps even the very basics, which is, where exactly can you apply AI within the content creation space in advertising? Where are you folks using it today?And it’s interesting, you said you’re starting at the bottom and working way up, meaning like these asset creation and hopefully impacting strategic choices over the long run.
Jonathan Halvorson05:37
Yeah, you’re starting with lower funnel assets that are more conversion, but I think the systems you’re building are really end to end, and I think that’s where you’re going to get the value. And I think your Wharton colleagues are speaking to that. That’s why it’s not augmentation, but true disruption. Because I think AI is going to help you do everything from creation of the brief all the way to actually traffic that asset and put it out into market.
Kartik Hosanagar06:00
That makes sense. But one question I have for you is, in this early phase, are you already seeing any evidence of ROI? And I ask that because there’s so much hype around AI, but many skeptics are also starting to raise these questions about all the spend, where’s the return? What are you folks saying?
Jonathan Halvorson06:17
Over the last two years, we have done 40 campaign based AI activations. So this is example wearing campaigns. We’re using AI technology to augment and deliver a higher efficacy or a higher efficiency of our assets in those campaigns. We can clearly point to that not only being some of our strongest work, but also in those campaigns, delivering higher brand awareness, higher market share gains and higher net revenue. I think that you see a lot of the best of that work inside Mondelez on Cadbury. I’d point to two specific examples. First is give a cheer for a volunteer in Australia. Last year, our top performing BU was Australia, and the top performing brand Cadbury, at the center of their campaign activation was a program that allowed consumers to create an AI film that celebrated a volunteer in their life. And it’s something that wouldn’t have been possible without AI. We can’t hire an artist or a film director to produce all those films and be able to make that possible. Similarly, inside India, last year for Valentine’s Day, we created an ability, through packtivation, for consumers to create Valentine’s stories and tell the entire story of their love. It was really geared towards people who are, perhaps not the planners, but rather last minute gift givers and a little bit of also helpless romantics who want to tell those stories. Those campaigns aren’t possible without it, and there are two of our highest performing campaigns, both internally as well as externally celebrated.
Kartik Hosanagar07:39
Jon, what I really like about the examples you just shared is that the examples are not just about cost cutting with AI use, but it’s about meaningfully getting a lift in outcomes like consumer engagement or attention and things like that. And we’ve seen that actually, we organize an annual generative AI workshop where researchers share the latest research in these spaces, and multiple studies in our workshop have shown that it’s, again, not just a cost reduction standpoint. For example, one study by researchers at MIT and Indian Institute of Management showed that there was a 6 to 9% lift and click through rate of ads, and another one by researchers at University of Hamburg and Emory University compared human-generated ads against AI-created ads for automotive companies, and they used purchase funnel metrics like: Does the ad get attention? Does it generate interest? Does it generate desire? Does it create action, like downloading a brochure and so on. And on a seven point scale, they found those purchase funnel metrics went up by nearly a point, about 0.75 points. So strong evidence. Have you guys tried to measure this as well?
Jonathan Halvorson08:47
Yeah, the same thing. I think in the very beginning you see higher ROIs, higher conversion rates. But I think it’s also important to note, this is just the beginning. We’re in the early rounds of the ball game. What I think the potential that gets people excited about AI is for that to only improve over time. So today it’s seven points. And just to be clear to any of the listeners, that’s material to the business, when you start delivering 250 trillion impressions in a year, and if you can do that 1% better, that adds up to significant volume gains and significant growth in terms of net revenue, but then it’s just the beginning. Over time, AI is going to cumulatively learn and get better and get better. And I think that’s what really excites us. If we can start from that point, have really good success signals and train the systems, we’re only going to achieve higher levels of efficacy and excellence.
Kartik Hosanagar09:37
It’s great to see that kind of impact with AI optimized or AI-generated content. I’m also curious about what is the role of the human in this? So what is the human in the loop process here? Because I’m assuming it’s not like a bunch of AI agents doing everything there are humans prompting and humans selecting and so on. So tell me about that role.
Jonathan Halvorson09:57
A lot of people have a different point of view about this, and I’ve had a lot of good debates with practitioners, colleagues, peers, academics, about where it’s at. And I think where I’m at on this is AI is going to impact every single part of the value chain. And I think it’s going to help humans do their jobs better. And if we look at it the very beginning, like we use AI internally to help us write better briefs. So do we write good creative briefs today? Yes. Can they be better? Yeah. I want them to be perfect every single time. And I think with AI inputs, we’re raising the quality of our briefs. The second thing is coming up with concepts. So we use our tool, our proprietary tool at Mondelez is called Aida, AI D A. It stands for artificial intelligence, data and analytics, and that tool allows us to generate concepts. Are any of those concepts perfect? No, but they help inspire our creative teams and our marketers around the world as to what’s possible. So I think that then gives a jumping off point, and this is where our agency creative directors, at WPP, Publicis, the Martin agency, VCCP, provide their own ideas, elevate those ideas to get to really good scripts, then we can quickly test those scripts, ultimately, by generating a series of images that represent that we test it. What used to take two weeks, can now take a few hours, get a score back, and then we can go produce that asset at a higher quality. So I think the human in the loop really is in a few key areas, like it is giving humans a lot more time to spend on creative excellence, to elevate the ideas and a lot of the place where that uniquely provides value.
Kartik Hosanagar11:27
Yes, that makes so much sense. I think AI can be a great tool from an idea generation standpoint, idea enhancement and, of course, the actual execution pieces as well. We actually did a study recently where we were looking at AI use in creative writing. So we had a bunch of students who wrote without AI and with AI, and we noticed that AI use, of course, reduced the time it took to write, it increased the quality of the writing as well. But one of the interesting side effects we found was that human-created content in aggregate was more diverse than AI-created content. When everyone is using the same AI tools to write, they’re coming up with similar ideas, whereas all of us have very different biological neural networks in our head, your influences are different from my influence. So we come up with different ideas, but if we both use the same kinds of tools, you end up with similar ideas. We see it now with like PowerPoint presentations, like everyone’s PowerPoint is starting to look similar. Do you worry that if all your people are using these same tools, the diversity of ideas might go down? And eventually what you’re seeing as a lift in performance might actually go down, because everything starts to look the same?
Jonathan Halvorson12:42
Yeah. Look, this is the huge [unintelligible] challenge. Like everyone’s getting ice cream. It’s the best ice cream, but everyone’s getting vanilla is essentially a good summary of the problem. I think about this a lot, because there will be a massive democratization of AI tools, and anybody, any marketer, any company, no matter how big or small, will be able to produce Super Bowl level creative instantly off of their laptop, if not their phone. 12, 18, 24 months at most from today. So then, with everyone having these tools to be able to produce creative, what differentiates people? And I think what matters a lot is going to be the differentiation of the brand and the foundations. At Mondelez, we’re really blessed. We have an amazing portfolio of over 250 brands that are iconic taste of nation. But what will be increasingly important in the journey we’ve gone on over the last eight years is to very clearly define those brands. What is the product truth that is core to the brand? What is the brand purpose, what is the tension it plays against? And if you define those things clearly, then I don’t think that you’re going to have that swell to the average and to the mean. But for brands who aren’t clear on exactly who they are in their foundations, I think they will find themselves in a sea of sameness. So I spend a lot of time thinking about this, and I think the safeguard on it is: one, the clear brand foundations;two, humans in the loop; and three, just the discipline you put in, just the intentionality of how you use AI and just recognizing the concern.
Kartik Hosanagar14:10
Yeah, I think that’s a really important point. And I just want to dig a little deeper into that. How do you in practice, implement brand distinctive features in your AI augmented or AI generated content?
Jonathan Halvorson14:23
There’s a few things. First, it starts by being really clear on what are the distinct features of your brand. And that sounds like a very obvious thing, but it’s a very intentional exercise that has to take place before you ever open up AI or open up a laptop, you know. And so in the case of Oreo, it is a black and white sandwich cookie. It has a very clear embossment. There’s a very clear logo, and look to that brand that has to be clear, no matter what the instance is. Then having defined all those assets that are truly distinct, then you have to train a large language model, and you have to explain it. And we’ve had a lot of work that we’ve done on this across all of our agency networks. Training for Milka, training for Oreo. And it just requires time to essentially upload the entire brand history. And really it’s a technical exercise of training the system as to exactly who you are. So it starts by uploading all those awful brand playbooks that we’ve built over the past few years. Those have immense value. Two, is your archive. You know, the really great brands keep very good archives of all the work that they’ve done. And that’s because often, when you’re in a challenge, you can find your solution in the archive. But also that becomes really powerful training data to explain to the AI system your performance over time, what’s worked, what’s not. Then it’s a matter of as you train your large language model, sitting down and giving it back and forth. Just like you train in machine learning or any other application, there is a back and forth exchange of when a system is showing you something, and you have to provide feedback on that. Yes, that’s right. And no, it’s not. And through all that training exercises, you will create a custom LLM that represents your brand.
Kartik Hosanagar15:59
Yeah, thanks for that helpful peek under the hood. It’s important to understand that process that goes into that and it’s not just going to ChatGPT and writing a prompt and voila, you have your ad. So we’ve talked a little bit about the process. Let’s talk a little bit about the outcomes. I know you’ve run dozens of campaigns that are AI-supported, but I’d love to hear about one or two in particular. What was actually done? What was the new customer experience and things like that?
Jonathan Halvorson16:27
Let me answer that question in two parts. First, let’s talk about the journey, and then let’s talk about a specific example. So let’s talk about the journey.
Kartik Hosanagar16:34
Great.
Jonathan Halvorson16:34
So 18 months ago, we start doing our first work on this. We trained a large language model. I’m sitting with the agency, and I’m super excited. We’ve done a little pilot, and the first thing it spits out is humans that have six fingers and people who have giraffe necks. And then we would see Milka bars that were missized. Everything was wrong. We tried to prompt it on Halloween, and instead, I get things that looked like Christmas. There were people who were joined at the shoulder. I could have cried right then and there, and I was like, man, maybe this isn’t for us.And the early thing that we’ve done in the small little pilot just showed how hard this was gonna be when you talk about some specific examples, and what does that really enable? I like to go to the India BU, because I think they’ve done some of our most advanced work on Cadbury. The first work that they did was actually in music, and it was for Cadbury birthday song. And so the idea behind Cadbury birthday song was a simple insight: There are literally billions of people in this world, and yet there’s only one song to celebrate them. Happy birthday to you. And so their insight was, wouldn’t it be powerful if you could create a custom birthday song? But the thing is, is you can’t hire a musician to write a billion birthday songs for any iteration, but using AI and some large language models, in partnership with the Ogilvy team, they created a tool where, if you uploaded some basic information about yourself, you got to pick a type of music, you got to pick a language. Ultimately, you would produce a custom birthday song for you. We did work promoting it and above the line, as well as below the line, and it became huge hit across India, with literally millions of people participating, driving incremental sales celebrations outside of the core festive season, when typically almost all of our chocolate is sold. So really extending the chocolate season from just Diwali and Valentine’s Day to really being something that’s 365 days a year. And that’s a powerful example.
Kartik Hosanagar17:08
Right.Yeah, and by the way, I grew up in India, and so Cadbury was a huge brand in India when I was growing up.
Jonathan Halvorson18:38
I have to thank Brot Purdy and many of the great marketers that come before us for the great tradition we have in India in marketing, and it’s laid a great foundation.
Kartik Hosanagar18:46
It’s been great to hear about what’s gone well, how things are producing positive impact and results for you. But let’s talk a little bit about lessons learned, because a lot of companies are just getting started with their AI and content journeys. And I’m curious what’s been the hard part of all this? What have you learned in terms of both what’s been hard that you’ve solved, but also what you’re still trying to solve?
Jonathan Halvorson19:11
So many lessons learned. I mean, what started as an insight at a moment where we had the flash of brilliance to being here today, has been a long road and a lot of work. I think, you know, lessons learned, and things I would tell one is, one is find a strategic partner to be your sounding board and stick with one all the way consistently. I would say, throughout this entire journey, I have had really good strategic partners, and it’s the one voice in my head and that makes life really simple. The second thing is, I think it’s very important for you to have a clear narrative. Every company will approach AI differently and even the use case of content. I’m very focused on increasing the effectiveness of that content and the quality of it. Others, their business case is more built around speed. How can they take global assets, localize them, get to market faster? Other people are all about cost savings. What I think is important is that you have a very clear business case and narrative around that that you consistently talk about through the organization, and that becomes really important. Third is, most of this is change management. And I’d say that the change management is probably half of the cost and definitely more than half of the time. And so I think it’s important to get people in early to see this and talk about where you’re going. The last thing is, I think you learn from us, is the power of a demo. We did a small little pilot. We had a little example of a Milka large language model that simulated the brand, and we produced this video. And so when we went to go talk to the marketing organization, we went to talk to the C suite, it was an asset we could show. And it was powerful, because often when you start talking about AI, large language models, you can get very quickly lost in all the junk in a series of acronyms that aren’t really going to help you. But this little example became a real shining light, and for people who weren’t even marketers or weren’t familiar with AI, they saw it and it changed their mood. There was a moment, a pivotal moment, where we were deciding if this was gonna be part of our strategy or not, and one of our region presidents thought,he goes, “that’s the future”.
Kartik Hosanagar21:06
Change management, to bring people along, demos to win people over. The interesting thing, and that’s where I wanna take this last question, is that advertising, at the end of day is all about people. It’s about people with creative ideas who can help brands connect with real people, and now you have AI, native agencies, AI automating work and all of that. I’m curious, what are your hopes, or what’s your vision for how people and human creativity can still be at the center of this conversation around AI.
Jonathan Halvorson21:40
There is an amazing tension and cycle that happens between technology and creativity. Technology gets out ahead, and it inspires new level of creativity, that inspires the next level of technology, and you get a very virtuous cycle, and I think you see that playing out over the last several decades. And I always love moments where technology is out ahead, because it means that the things I dream up in my willy wonka mind are possible. And whereas, when creativity is ahead of technology, you’re like, this would be really cool, but we can’t do it. And I find that to be a lot more frustrating. But my hope and dream is that I always believe that the best advertising is yet to be done. And therefore there’s a constant way to do better and cooler things. And every time we do this, I think for us, it was our Shah Rukh Khan, your ad moment.
Kartik Hosanagar22:29
By the way, for our listeners who don’t know, maybe we should say who is Shah Rukh Khan is. He’s, because I grew up in India, I can say he’s like the Brad Pitt of India, only 10 times bigger in India, or something like that, right?
Jonathan Halvorson22:39
Tom Brady, plus Tom Cruise, plus every good imaginable human like, you know, all rolled into one. I mean, and this asset we just we shot with him for a day, and it allowed you to create personalized versions of it for every local retailer. And it was just a dynamic creative optimization on steroids. And when you saw that, you were like, wow, that wouldn’t be great if everything we did was that good. I think you have moments like that where you see a piece of work, it inspires you as to what could be. And I think every day, I hope that the work that I produce at Mondelez inspires other marketers to create things. Because I know as Mita at L’Oreal, Tamara at Heliot, they’re constantly producing things that are inspiring me, and I go to my teams, I go look at this, and wouldn’t that be so cool? And I’m pushing that to the next frontier.
Kartik Hosanagar23:28
Jon, this has been great. I think we created some good podcast content here, and we didn’t even need AI for this.
Jonathan Halvorson23:34
Nope.
Kartik Hosanagar23:36
Thank you so much for being guest number one on Where AI Works.
Jonathan Halvorson23:40
Thanks so much for having me.
Kartik Hosanagar23:42
Now let’s review the key learnings from my conversation with Jonathan. A big initiative like AI driven change is likely to throw some early challenges. Like Jon was talking about images where people had giraffe necks, or six or seven fingers. What’s really important is one needs to have patience, because you’re investing in AI for the long run. The other learning was that AI value is not just about cost reduction. It’s also about consumer engagement and revenue impact and increasing top line increasing consumer retention and things like that. And finally, it’s important for companies to find the right and consistent strategic partner, because AI driven transformation is as much about change management. How do you convince people? How do you bring them along for the ride? How do you create demos to get some early excitement about what you’re trying to do, and how do you make sure your people are with you in this process? That brings us to the end of the series premiere. Thanks so much for listening. Please follow us so you don’t miss an episode, and be sure to tune in next time when I sit down with Jill Kramer, CMO at Accenture. We’re going to explore how AI is reshaping marketing. This has been Where AI Works, brought to you by Wharton and sponsored by Accenture. I’m Kartik Hosanagar. See you next time.
Episode 2
Guest: Jill Kramer, Chief Marketing and Communications Officer; Accenture
Transcript: Episode 2
Jill Kramer 00:01
Curiosity and creativity is driven by exposure, by options, by the least restrained possibilities, and Gen AI unleashes that. So if you allow it to be short-handed too “Can’t Gen AI just write that for you?”, it’s an incredible disservice to the technology and the potential of it being applied to a function as important to growth and brand and the strategy of any given company as marketing is.
Kartik Hosanagar 00:31
Hello, and welcome back to Where AI Works, Conversations at the Intersection of AI and Industry. Brought to you by Wharton and sponsored by Accenture, I’m your host, Kartik Hosanagar. I’m a professor at the Wharton School, and my work focuses on AI and business transformation. It is our goal to cut through the noise and deliver actionable insights for business leaders by combining cutting edge research with real world case studies. Things are changing fast, so without further ado, let’s dive in.
News Clip One 01:02
I’m getting the same question a lot nowadays. Should we be scared of AI?
News Clip Two 01:06
When we’re talking about trust, it means so many things, and it’s a whole world that everybody will have a different interpretation of it.
News Clip Three 01:12
One of the challenges that we have is that the space is moving incredibly quickly.
Kartik Hosanagar 01:16
On this episode, I’m excited to explore how global brands are harnessing AI and also what it takes to take large teams along on that journey. Multiple studies have shown that AI dramatically improves human productivity. These studies range from settings like software to sales, consulting, customer support, professional writing and many more. Now, while we have that evidence at the individual level, hard evidence on return to AI investments at the organizational level haven’t yet emerged, and I think that’s because there’s a difference between a few individuals adopting and seeing their returns on that versus large, complex organizations, because you’ve got to manage cultural change. Re-skilling, up-skilling, strategic prioritization; lots of issues here. To help us think through the challenges and the opportunities around AI, it’s my great pleasure to welcome Jill Kramer. Jill is the Chief Marketing and Communications Officer at Accenture. Jill, welcome to the podcast.
Jill Kramer 02:15
Hello, Kartik. Thank you so much for having me. I’m excited for our conversation.
Kartik Hosanagar 02:19
Likewise, super excited to have you here. I guess even before I get to the topic of returns from AI and how are you managing AI driven change at Accenture, I’m really curious about what’s been your relationship with technology.
Jill Kramer 02:34
So my personal technology journey is one of avoidance and fear that turned to a deep, unwavering love. I would love to tell you that I was fearless, that I was first in line with my hand up, saying, let’s do this specifically with the most recent wave with Gen AI and the speed at which it’s moving. I did start this thinking, wow. Like, is this an existential crisis, and I made the decision, and I was very transparent about it with the full marketing and communications team at Accenture that I desperately wanted to be in the driver’s seat. And the only way you can do that is by getting really close, getting very hands on, deeply understanding, and taking some risks on behalf of yourself, your company, your brand, your team. So that’s where I went over to the love side. And it is something that I truly believe now is invigorating, exciting, drives more creativity than I ever could have imagined.
Kartik Hosanagar 03:35
Yeah, that’s great to hear. And in fact, later in the conversation, I’d love to explore how you have undertaken that journey. But before I get into that, perhaps where I want to start off is maybe at the organizational level, before we get into like the marketing team and then you individually. Accenture has been talking about being early and aggressive with transforming its business around AI, so help us understand what progress has Accenture made in that AI transformation journey, and where is it currently positioned relative to others in the market?
Jill Kramer 04:08
The journey for us is really the journey we’ve taken with previous technologies, really, which is understanding each wave of technology and the impact it can have on work and workers, and in this particular wave, one thing that was evident very early in the process with Gen AI was that if you apply it to existing work, existing processes, you would see incremental improvement. But if you said, I want to look at the work in a zero based way, now that I know that this technology exists, how would I do it given these new circumstances? And that is something we have done across the board. With every function that helps run this corporation, we have looked at rethink the work now that you know that Gen AI exists. Because one of the things you learn, when you embrace technology like this, is those areas of overlaps and adjacency, there’s magic in reimagining those handoffs, those shared services, those processes from legal to HR to operations to marketing to sales. So it’s something that has to be done intentionally. It’s something that has to be done with a clean sheet of paper, you have to rethink and reinvent, and it’s something that has to be done very collaboratively across all of the functions that run the enterprise. Let’s go from Accenture as an organization to your marketing organization. There are many things under your purview which AI could touch upon and does indeed impact and influence today, whether it’s advertising, content, digital marketing, social media, corporate communications, AI’s applications in all of those areas. I’m curious, for you, what have been the first few areas that you have brought AI into, and why have you picked these specific applications or use cases for AI? So I’m actually going to tell you a quick recap of the chapter that preceded our application of Gen AI.
Kartik Hosanagar 06:01
Okay.
Jill Kramer 06:01
And that is when you run a marketing function at a large multinational company, you have projects being asked from every part of the organization, lines of business geographies, you name it, you have requests. A lot of marketing organizations execute those requests, yes, in service of a business strategy, a brand strategy, a marketing strategy, but you don’t necessarily know every project that’s going on at any one moment.
Kartik Hosanagar 06:28
Right.
Jill Kramer 06:29
And you can’t necessarily reconcile, prioritize. So I wanted to be able to say at any point in time, how many events have we done this year, which are the most successful? How many pieces of content have we put in the market? How many new pieces of content have I put on our website? What’s performing the most? So the first thing we did unknowing about the Gen AI movement that was about to happen, we really got our act together in terms of knowing the work that happens in marketing using free Gen AI AI, so regular AI, to create algorithms and prioritize and sort work in a very simple but strategic and intentional fashion. We also brought all of our data together, because if I wanted to look at the work across the function, I had to look at the KPIs. That meant that when Gen AI started ramping up, we were able to be pretty confident about which we were going to adopt first. So that brings me back to where you began with your question. Some of the first things we did was we had really done a very rigorous reinvention of our internal communications. We had created a very highly personalized approach to communicating with the then 700,000 or so people of Accenture. So one of the first things we did was we applied AI to our writing so that, as we were highly personalizing, we could use Gen AI to quickly version. Similarly, we did it with content, not the entire content supply chain, but the content versioning. We did that very quickly. So here’s the long form report, using Gen AI to assist in the medium size and the short size, the quick synops that would appear on the website, the quick synops that might appear on our app. What really changed was when we decided that we were going to go all the way to agentic AI. When you do that, you have to go all the way back to your process reinvention and say that if I looked at the steps in the marketing process, and I knew I had an agent or agents who could do work simultaneously with me going to parallel work streams, instead of everything having to be so linear and sequential, how might I design the work? We ramped that up this past September, and we now have 1000 marketers all onboarded using agents in their daily work.
Kartik Hosanagar 08:47
For our listeners who are not as familiar with agenetic AI and where it’s headed, I think the way to think about it is that lot of AI systems today can get individual tasks done. Agent-based systems are systems that can function autonomously. They can communicate with each other, they can interact with the outside world, get information from there. They can transact. And so once you start to build agentic systems, you’re not talking about automation of individual tasks. You’re talking about automation of full workflows where an agent based system can go fetch information from some other place, maybe interact with a vendor or a partner, negotiate with them, complete full tasks. So that’s an interesting direction that AI is headed. And it’s great to hear that you’re talking about some of these implementation of agentic systems. We’ll come back to that in a moment, but coming back to what you were saying with this journey from starting with AI for writing and then for image-based content, to other workflow products and now to agentic systems. Are you seeing any results in terms of costs or business agility, or what are you seeing in terms of what it’s doing for the business or for your org?
Jill Kramer 09:55
What we’re tracking very specifically is speed to market. And we’re tracking reduction of steps taken in a given process across the function, but also within sub-functions. Then there’s two waves of KPIs that I’m tracking. One is the prioritization of the work, which is Gen AI assisted, because the way we’re processing our reports and our measurement, it allows us to make better decisions, which has led to, like, 50% content reduction. Using Gen AI allows me to now do what we call Good Morning Accenture, which is a highly personalized multi-channel way of greeting each person within Accenture and letting them know what they need to know and what they need to do for that day in order to be productive, focused, etc. On the flip side, when you look at the longer flow processes, we’re seeing about a 35% reduction in steps taken within the application of the agentic workflows, and as much as 30 to 50% increase in speed to market.
Kartik Hosanagar 09:59
Yep. Wow. Those are very impressive numbers, I’m sure, an organization of Accenture’s scale, many of these systems are often built in house, but at the same time, there’s probably tons of AI products that you’re consuming. One of the questions I have for you, which is something I keep hearing from a lot of CMOS, and in fact, across different functions, but certainly from CMOS as well is that there’s so many different vendors out there, and so when you’re bombarded with so many tools to potentially use or pilot, how do you approach the decision of which tools to prioritize? How do you pilot them? And there’s possibly IT approvals and AI committees that need to approve things, that slows things down. So how do you stay nimble within your marketing org, where you can experiment, try new tools, but at the same time be compliant within the organization’s broader mandates and restrictions about what AI is allowed, what’s not allowed, and things like that?
Jill Kramer 11:56
You bring up so many important things and points, but you bring me back to my driver’s seat decision. There are so many solutions. They’re exciting. People want to get their hands on them, and so I’ve got to be in the driver’s seat for myself and for my team on this because people were adopting solutions, bringing them forward. We can use this over here. This is now embedded in this solution we already use, and it felt like a beautiful kind of chaos, but I realized that it was going to vary. We’d just gone through all the work to get our tech stack cleaned up, our data cleaned up, our processes cleaned up. So it was like, how do we bring our people with us? Use everything that they’re understanding? I recently had another CMO saying to me, the people in your marketing organization and or your customers are going to start using these tools, whether you sanction them and intentionally do them or not. And that’s a very, very true statement. And we made pretty bold decisions, like giving access to writer.ai, to every marketer. We could have done a much smaller group. We could have trialed, but we said, you know what, we didn’t want people to feel left behind. And then in other cases, we said, this is going to be the landscape. So we created a leader on my team who was going to oversee all the Gen AI ideas. Let all those beautiful ideas bloom, bring them back to our team and our IT advisors, because no CMO should be doing this without right your IT advisor. They’re your best friend.
Kartik Hosanagar 13:22
Yep.
Jill Kramer 13:22
The other thing about these AI systems is that they are designed to be more modular. You know, like, whether it’s the ability to switch between LLM models, whether it’s the ability to click in new modules of agent types or support with out of the box solutions. There’s an agility and a flexibility if you design your approach to AI and Gen AI in the right way. My advice is set up a system where your people are bringing you their best ideas, but you do have a formal way to assess pilot and scale. You must change the work, because if you don’t change the work, the scale and the actual benefit from going to scale won’t be there. And then the last one is that you need to have your IT department with you, just like we wouldn’t want anybody coming up with their own brand tagline. So you have to merge the craft. Then what is exciting about the technology for the craft of marketing and communications with the expertise of technology experts who can say, here’s how you build this, and don’t worry, because it’s modular, it’s agile, and you’re going to want that flexibility to evolve as you go forward.
Kartik Hosanagar 14:26
I want to move the conversation from the technology to the people, because a lot of this is really a story of people, how comfortable they are with how empowered they feel with these technologies and so on. I’m curious how big is your marketing org, Jill? How many people are we talking about?
Jill Kramer 14:45
We’re at about 17, 18 hundred people.
Kartik Hosanagar 14:47
Wow. Okay, yeah, that’s a lot of people you’re sort of bringing into this AI journey. I feel like when it comes to an org with as many people, you’re going to have some people who are maybe early adopters, who are jumping in trying things out, and then there are going to be people who feel threatened by this. With my previous AI startup, you know, I had these conversations with the C-suite about the products, and I remember one conversation in particular where one of the C-suite leaders was talking about how this tool that we had could potentially be threatening for people in their org. And specifically, he kind of said that, you know, I don’t see our employees using this at all. And the reality was, we had several dozen of the employees already using the free version of the tool, and then we had to walk them through. Hey, listen, you know, your employees are actually further ahead than you think they are. So tell me a little bit about both sides of this coin. One being there are many instances where the employees are ahead of where the leadership thinks they are. And then there are also instances or individuals who are feeling very threatened and feeling left behind because the pace of change is so fast. How has it played out in your arc, both in terms of people feeling threatened, but also empowering people who are moving faster?
Jill Kramer 16:04
Very similar. I mean, a lot of people were proactively coming to us saying “Hey, I’ve been using this tool on the side, and look at how much faster it helps me do this brief” and “look at how much more information I was able to collect”, or “Look at how quickly I did these” and you want to embrace those things, but also make sure you’re within your responsible AI guidelines. Is everything legally sanctioned? On the flip side, I always talk about marketing is a very generous function, right? You give your ideas, your words, your perspective on imagery, to internal and external clients, right, to form marketing materials. But a lot of people think they are marketers, right? Oh, I would have written the headline this way, or I shouldn’t have done this this way. So when you end up with a technology who now says, oh, it can write the headline, it can do the image. It was a really natural reaction to protect a craft, to protect the function, and not allow it to be unintentionally devalued by saying anybody or anything could do this craft that we all love so dearly. So my take on this is, and I was very clear with my journey with the team was that a good marketer is a curious marketer. If you watch someone write a brief, they will spend as much time as you will give them, researching competitors, looking back at everything we have done and not done. What was the best thing we ever did? What was the biggest failure? Same thing with creatives, they want to write, rewrite. I mean, how many times have you had to say to a creative team “okay, it’s time to put the pen down. We got to go produce this.” They want to find every image they can.When you think about Gen AI through that lens, these agents we’re using help you go out and get every bit of analysis, competitive intel, activation schema that you could possibly think of. If you’re using it as a creative, the ability to seek images, to change parameters, to look at the same thing 10 different ways, is now at your fingertips. So I think that’s the biggest thing is curiosity and creativity is driven by exposure, by options, by the least-restrained possibilities. And Gen AI unleashes that on behalf of the human, on behalf of the craft. So if you allow it to be shorthanded too “Can’t Gen AI just write that for you?”, it’s an incredible disservice to the technology and the potential of it being applied to a function as important to growth and brand, and you know, the strategy of any given company as marketing is.
Kartik Hosanagar 18:04
Yeah. Yeah, I’m gonna remember that statement you said, which is, curiosity and creativity are fueled by having options, having this information and so on, and AI enabling that. So that’s great. Walk us through the up-skilling and re-skilling process. You know, 1800 people who you’re now trying to train on new systems, new tools like writer.ai, new ways to approach work where you’re giving up part of what is core to your craft to an AI system, and focusing on other pieces where you can add unique and differentiated value as a human creative. What has that re-skilling process been like?
Jill Kramer 19:13
So there’s a couple of things that are absolutely core to it. The first one is the concept of creating cohorts. When you are bringing a large team along on a very new journey, don’t try to do it all at once. Don’t try to spread yourself, your ability to be attentive, to deeply train and to listen to people by trying to do it all in one fell swoop. So we’ve created cohorts for every wave of transformation. Each cohort takes the next one with them. They create a bigger community of people who can listen, who can help, who can energize and who can address problems. The second thing is communication. Very often, as a leader, you’re already sold and you know all of the reasons in your head. It’s very easy to quickly think “Well, I’ve already told them this, like they know this, they know the why.” You have to say it over and over and over again, and you have to gather together regularly to listen, to communicate progress, communicate advancements, be incredibly transparent, and take every question that comes your way. And the only way that that changes the culture is by repetition and by people realizing they truly can ask any question that they want to or need to. And then the last one are use cases. When the first two things go well, the use cases that are created by your people will be better than the ones you ever imagined. Mark my words. That’s where the curiosity and the creativity is on steroids, so you need to find a way to bring those use cases back, because that’s the very essence of change management.
Kartik Hosanagar 19:34
Right? Right. Makes sense. So, Jill, we’ve talked about the employee side of things. Now, employees are part of the journey, but so are customers, and one of the interesting things is customer behavior is changing. So for example, many of us are no longer just going to search engines and searching and finding our answers. We’re going to LLMs and finding answers, including, what should I buy, or what are the best brands for, whatever the use is. So how do you think about that, and in particular, how does the marketing org prepare for and keep up in a world where customers are embracing AI and making decisions informed by AI?
Jill Kramer 21:27
This is a case where I think it is similar to previous waves of technology, when customers were going to digital, if you decided not to understand that, you would get left behind. When search became a big component if you were not intelligently constructing your content and your digital experiences for search results, you would get left behind. It’s very similar. So, you know, one of the beautiful things about Gen AI is, for example, when you are creating something or writing something, you ask it for a summary.
Kartik Hosanagar 21:54
Yep.
Jill Kramer 21:54
And it makes you realize, oh, the points I thought I was making, I actually obscured. And some points were taken away as top level, and I meant them to be second or third level. If you’re like us, B2B professional services, and you’re creating long form content, what’s the first thing a person’s going to do now that they have these tools at their disposal? They’re going to create summaries, they’re going to merge documents. They’re going to go across and look for the best. So how do you create the experience that’s going to do that for them and with them? It’s just like anything else, you recognize a new condition and you create for it.
Kartik Hosanagar 22:31
Jill, my last question for you is where we started this conversation. You talked about how you started from a place of fear, and over time, it’s been one where you feel like you’re in the driver’s seat, in control and loving it. Tell us how that change happened, and in particular, what advice do you have for other marketing leaders that are facing this change and are perhaps a little behind in that journey and facing some of those fears themselves?
Jill Kramer 22:57
It has to do with human nature and how you overcome fear and misperceptions in anything, and that is through proximity, through exposure, through empathy and through a desire to learn. I will give this single piece of advice Kartik, and that is be hands on with this technology. It is not the same as previous waves, where someone else created a digital environment, and we just had to put the content into it.
Kartik Hosanagar 23:24
Right.
Jill Kramer 23:24
You need to know this intimately and deeply, and that comes from being very close and very hands on.
Kartik Hosanagar 23:32
That’s great advice. I mean, that’s what I discuss with my students as well, that you have to get into the black box, you have to use it, you have to touch it, feel it, and you can’t really poke at it from a distance.
Jill Kramer 23:43
It’s a lot less scary and a lot more exciting when you do that.
Kartik Hosanagar 23:46
Jill, we’ve covered a lot of good ground here today. Thank you so much for your time, and thank you for sharing your insights here on Where AI Works.
Jill Kramer 23:54
Thank you. I’ve enjoyed every minute of it.
Kartik Hosanagar 23:57
One interesting takeaway for me is about how we need to approach AI, not with a lens of fear, but with a lens of curiosity. And her point about how curiosity and creativity are both fueled when we approach AI with that lens. The other interesting takeaway for me is Jill’s framework of cohorts, communication and use cases. When you’re trying to pull off change in large organizations, doing so in stages with cohorts, makes it more achievable, realistic, at the same time, creating use cases, communicating that celebrating them helps bring people forward, and I think that’s a very practical, easy to implement framework for our leaders today. That’s all for today. Thank you so much for listening. Please follow us so you don’t miss an episode, and be sure to tune in next time when I sit down with David Droga, the founder of ad agency Droga5, and CEO of Accenture Song, which is Accenture’s creative agency. We’ll be discussing AI’s transformative impact on the creative industry. This has been Where AI Works, brought to you by Wharton and sponsored by Accenture. I’m Kartik Hosanagar. Bye for now.
Season 2: AI & Productivity
Premieres June 12
In Season 2, Wharton Professor Christian Terwiesch examines how AI-driven efficiencies integrate into workflows, with a particular focus on the customer experience and the new product offerings that will be enabled by recent advances in AI technology.
Season 3: AI & Workforce Transformation
Premieres August 7
In Season 3, Wharton Professor Peter Cappelli unpacks how AI is reshaping the workforce—shifting job roles, redefining skills, and driving human-AI collaboration while businesses navigate job displacement and ethical AI challenges.