Large language models (LLMs), the AI programs that churn vast amounts of data to generate content, will have large impacts on jobs, but they will take a long time to do so. That’s the conclusion of a team of experts at Wharton and elsewhere who analyzed tasks across occupations for their exposure to LLMs and their potential to be transformed by AI. For example, cooks, carpenters, and motorcycle mechanics are unlikely to have exposure to LLMs. But interpreters, poets, and proofreaders are among those with the highest likelihood of LLMs performing their functions.
The study covered U.S. government data on 21,000 tasks across 1,016 occupations over three years from 2020. Its findings are captured in a recent paper titled “GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Models.” Daniel Rock, Wharton professor of operations, information and decisions, co-authored the paper with Tyna Eloundou and Pamela Mishkin, both technical staff members at OpenAI, and Sam Manning, senior research fellow at the Centre for the Governance of AI.
“LLMs are much more likely to affect knowledge workers such as lawyers, analysts, scientists, and technologists,” Rock said during a recent episode of the Wharton Business Daily radio show. (Listen to the podcast.) “We asked a simple question: Could a large language model cut the time it takes to do a task in half without sacrificing quality?” LLMs can perform tasks either as standalone technologies or with additional software and retraining and change management. In case of the latter, “the labor market impact will take a while, even if it has high potential,” Rock said.
The paper is focused on two types of GPTs: General-purpose technologies such as printing, the steam engine and computing, and the generative pre-trained transformers that power gen AI software. Both GPTs are marked by continuous improvement and innovation with resulting productivity increases and other efficiencies.
The paper is an advancement over existing research into LLMs with its focus on the complementary innovations that will be needed to unlock new applications. The authors noted that their primary contribution is “to motivate consideration of the difference in potential impact with and without additional software tools that complement LLMs.” That approach helps to evaluate LLMs as general-purpose technologies, particularly in light of the increased exposure of R&D-related occupations to these systems, they explained. “By employing task-level information to assess whether LLMs fulfill the criteria of a general-purpose technology, we seek to merge the two perspectives for understanding the technology-labor relationship.”
The study found that LLMs meet the criteria to be general-purpose technologies. “They’re pervasive. They improve over time, and then they necessitate and spawn complementary innovations, new techniques, new training, and new organizational designs [ required to] unlock the benefits of the technology,” Rock said.
“A lot of the story with large language models and labor in general has been alarmist. But it should be instead exciting and invigorating for people who are doing knowledge work.”— Daniel Rock
According to Rock, the idea of LLMs replacing jobs is overblown. “I don’t think job losses should be our operating model,” he said. At the same time, it is important to pay attention to where job losses might be occurring, he added. “A lot of the story with large language models and labor in general has been alarmist. But it should be instead exciting and invigorating for people who are doing knowledge work.”
Different Impact for Different Functions
The labor impact of LLMs depends on the market for specific functions, Rock said. For instance, the impact on customer service tasks will be very different than the impact on financial analyst tasks, he noted. In some cases, LLMs may drive an expansion of demand because it makes people more productive, but in other cases, the volume of work itself may not be large enough to increase demand for it, he explained. None of those findings are set in stone, especially because the world of LLMs is evolving. For instance, a breakthrough might warrant a review, which is why “forecasts of the future are very murky,” Rock said.
Forecasting the potential impacts of LLMs will get better with refinements of the metrics in this new field. “We need to improve our measurement apparatus to make sure that we’re tracking where workers might be facing tougher situations, where they might need opportunities to transition their roles, and then also pay attention to where things are getting turbocharged or going super well,” Rock said. “Coincidentally, LLMs can help us measure the economic [outcomes] better by using unstructured data.”
Getting adjusted to the changes that LLMs bring to the workplace calls for actions both within firms and outside, say in terms of government policies. Internally, firms would have to focus on removing frictions on the adoption curve with training and integration with existing processes, Rock said. Government policy could focus on helping workers who may be displaced by automation, although those cases will be rare, he added. “If we have displaced workers, it shouldn’t necessarily matter that it’s from technology. Technology might increase the urgency, but if there’s a displaced worker, the policy should be designed to help them, no matter what the reason was.” He pointed to recent studies that looked into wage insurance, for example, as a possible policy response.
One surprising finding for Rock was that higher-wage workers were more exposed to LLMs, which changed earlier perceptions that lower-wage work, such as clerical functions, tended to be more exposed to automation. But that exposure could have either good or bad outcomes, he clarified. For instance, LLMs can help entrepreneurs perform better if they were already skilled, whereas new employees at call centers can catch up with their most talented colleagues very quickly, he explained.
