One of the big questions about genAI’s implication for work is whether it gives a greater performance boost to less experienced, lower-performing workers or more experienced, higher-performing workers. Research skews toward the former, but a few studies have found that access to genAI tools like ChatGPT help high performers more.
One of the most well-known studies in the second camp has now been discredited. The paper—which was widely covered in media outlets, including Charter—found, among other things, that a genAI tool that helped material scientists discover new materials helped high-performing scientists significantly more than it helped their lower-performing peers.
The author of the paper is no longer at MIT, and the university said it now “has no confidence in the veracity of the research contained in the paper.”
This came as a shock to us, as that research was important to our understanding of how AI may impact work and inequality. Given its retraction, we wanted to synthesize what the other current research says about this question of whether AI will widen or narrow performance gaps between workers.
The question matters for understanding whether AI will increase inequality, explains Rembrand Koning, an associate professor at Harvard Business School, who co-authored a paper on this topic. But, he adds, it also has important implications for how AI might impact less experienced workers. “If [AI] helps people with more experience, [who] tend to be the people who can do things better, it might take out the lowest rungs of the labor market because we think AI can have such large effects.” Longer term, Koning adds, “there may be policy solutions or organizational changes that can get around some of these problems…we need to know the facts to start understanding how the economy might respond.”
Most studies that look into this question find that genAI has an equalizing effect, helping less experienced, lower-performing workers more than their more experienced, higher-performing peers. With the retraction of the material sciences paper, I know of only a few papers that have found the opposite to be true.
The AI-as-an-equalizer result has shown up in studies of everything from customer-support work to writing and consulting tasks. There are two important aspects of these studies to keep in mind when applying their results to other settings. First, the tasks in question are areas where we know AI already performs very well. Second, in these studies, what the genAI tool produces is often relatively close to—or, in some cases, is—the final product. So the mechanism through which AI levels the playing field in these studies is essentially by doing much, sometimes most, of the work.
The AI-widens-performance-gaps result has shown up in a few papers, including one that looked at college students in a debate competition and another that studied entrepreneurs in Kenya. Both studies suggest judgment plays a crucial role in whether or not someone benefits from AI. In the Kenya context, for example, the AI tool provided a range of advice to entrepreneurs and it was up to them to decide what to do with it. “Those who have the judgment do better because they’re able to be like, ‘Yeah, these 10 pieces of advice—six are terrible, two are neutral, two are actually really good. Let me focus on the two that can actually help my business,’” explains Harvard Business School’s Koning, one of the study’s co-authors.
So, will AI widen or narrow performance gaps?
My view is that it depends on a job’s level of autonomy. If you’re in a job where you help set your goals, determine how to reach those goals, and make many decisions throughout your day, AI likely helps you more if you’re a high performer with strong judgment. If, however, you’re in a role that’s more constrained, requires less decision-making, and performs a pre-defined set of tasks that AI can do well, then AI likely narrows the performance gap.
“The way I think of that is when you start your day and you’re looking at the possible number of paths you can go down…how many paths could I possibly choose? And then how different are the end destinations of those paths?” explains Ben Weidmann, director of research at the Harvard Skills Lab and co-author of a related paper. “My hypothesis, which is very similar to yours…is that jobs where there are more paths are going to disproportionately have the AI gains go to people who are high-skilled” in their decision-making capabilities, says Weidmann.
“The other way to think about it is [whether] the AI [is] producing the inputs or the outputs for your job,” says Koning. In many of the studies that find that AI narrows performance gaps, the AI’s output becomes the worker’s output, with varying degrees of editing. Koning points out that in the debate and entrepreneurship studies, the AI’s answers were the inputs for the tasks of debating an opponent or making critical business decisions and implementing them.
—
This is a condensed version of our analysis of this question. Charter Pro subscribers can read the full version here, including further analysis of the implications for how AI might impact different types of jobs.