Researchers at MIT’s Computer Science and Artificial Intelligence Laboratory tested current large language models against thousands of real-world labor tasks and found that AI “workers” succeed only about half the time. Even when they do succeed, the quality of their output tends to land at the lowest acceptable threshold rather than matching human-level performance. The findings challenge popular predictions that AI systems will rapidly displace human workers across white-collar professions within the next few years.
What is verified so far
The study, titled “Crashing Waves vs. Rising Tides,” drew its task list from the U.S. Department of Labor’s O*NET database, selecting more than 3,000 tasks that represent the kind of work people actually do in offices, at desks, and on screens. Human workers then judged AI outputs across more than 17,000 individual evaluations, making this one of the larger empirical assessments of AI labor capability to date.
The headline number: AI succeeded at roughly 50% of the tasks it attempted. That figure alone might sound promising, but the quality breakdown tells a different story. According to an overview from MIT CSAIL, about 60% of successful outputs were rated “minimally sufficient” when the model received the right contextual information. Only 26% of outputs earned a “superior” quality rating. That gap between passing and excelling is where the practical limits of current AI become clear. A system that clears the bar by inches is not the same as one that reliably produces expert-grade work.
The researchers also noted that 63% of the tasks they studied were text-based, which is the domain where large language models perform best. Performance on non-text tasks, such as those requiring visual judgment or physical manipulation, was not a focus of this study. That means the 50% success rate likely represents something close to a ceiling for current models, rather than an average across all job types.
The paper’s central metaphor frames AI progress as a “rising tide” rather than a “crashing wave.” Capability gains are spreading broadly across many task categories at once, but they are doing so gradually. No single occupation or skill cluster faces an abrupt wave of full automation, according to the MIT FutureTech summary hosting the research. The study projects that broader workforce effects may not materialize until around 2029, which pushes back against earlier timelines that suggested AI could overtake human workers as soon as 2027.
What remains uncertain
The biggest open question is how these lab-style task evaluations translate to real working conditions. A separate benchmark called the Remote Labor Index, which tested AI agents on end-to-end freelance projects with human gold-standard deliverables, found that AI failed more than 95% of the time. That number sits in sharp tension with the MIT study’s 50% task-level success rate. The difference likely comes down to scope. Completing a single defined task (write a summary, classify a document) is far simpler than managing an entire freelance project from brief to delivery, which requires sequencing decisions, handling ambiguity, and recovering from errors.
This conflict in the data deserves careful attention. If someone reads only the MIT study, they might conclude AI is halfway to replacing knowledge workers. If they read only the Remote Labor Index, they might conclude AI is nearly useless for real jobs. Neither reading captures the full picture. The MIT researchers themselves cite the Remote Labor Index as context, suggesting they view both findings as compatible parts of a more complex reality. AI can handle isolated subtasks at a passable level but struggles to chain those subtasks into the kind of coherent, adaptive performance that actual employment demands.
There is also uncertainty about how quickly the “rising tide” is actually rising. The study’s 2029 projection for broader workforce impacts comes from the MIT FutureTech team, but the specific assumptions behind that timeline (such as the rate of model improvement and the pace of corporate adoption) are not fully detailed in the publicly available materials. Earlier economic modeling work on automation and labor markets has used different frameworks, and it is not yet clear how the MIT team’s empirical findings integrate with those prior projections.
Related research on labor and technology policy has emphasized that institutional choices, not just technical capability, shape how quickly automation affects employment. Similarly, studies of macroeconomic impacts of new technologies point out that productivity gains can be offset or amplified by changes in demand, investment, and regulation. The MIT results therefore sit inside a broader, still unsettled debate about whether AI will trigger rapid displacement, gradual restructuring, or some combination of both.
How to read the evidence
The strongest evidence in this discussion comes from the MIT study itself, which used a large sample, drew tasks from a well-established federal occupational database, and relied on human evaluators rather than automated scoring. Those design choices make the core findings (the 50% success rate and the “minimally sufficient” quality pattern) relatively trustworthy as a snapshot of where AI stands on defined, text-heavy work.
The Remote Labor Index offers a useful counterweight because it measures something different: not whether AI can do a task in isolation, but whether it can do a job that resembles actual employment. In that benchmark, AI agents had to interpret client briefs, plan their own workflows, and deliver complete packages of work. The fact that they failed the overwhelming majority of the time suggests that orchestration, context management, and error recovery remain weak points even when individual subtasks are well within the model’s capabilities.
Putting the two results together yields a more nuanced picture. Current AI systems are already capable of helping with many white-collar subtasks—drafting, summarizing, editing, translating, and basic analysis, at a level that human evaluators often deem barely acceptable. That is enough to support productivity tools and co-pilot-style applications, where a human remains responsible for oversight and final judgment. It is not enough, at least for now, to support fully autonomous agents that can reliably replace a human worker end to end.
For workers and employers, the practical takeaway is to think in terms of task bundles rather than job titles. The MIT study shows that within most occupations, some tasks are already within AI’s reach while others remain stubbornly human. Jobs that can be decomposed into many independent, well-specified text tasks are more exposed to near-term automation or heavy augmentation. Roles that depend on cross-task coordination, tacit knowledge, interpersonal nuance, or accountability for outcomes are less vulnerable, even if parts of the work can be automated.
Policy makers reading these findings face a similar balancing act. The evidence does not support complacency: a system that can handle half of office tasks at minimal quality today may handle a much larger share at higher quality within a few model generations. But it also does not justify panic about an imminent, all-at-once collapse of white-collar employment. The “rising tide” framing implies time to adapt (through retraining, job redesign, and updated safety nets) if institutions move proactively rather than waiting for disruption to arrive.
For technologists and investors, the gap between task-level competence and project-level failure highlights where innovation is most needed. Improvements in reasoning, planning, and tool use could narrow that gap, as could better interfaces that keep humans in the loop for the kinds of decisions AI still handles poorly. The MIT data suggests that simply scaling models for more raw capability may not be enough; aligning those capabilities with the messy structure of real work is an equally important frontier.
Ultimately, the current evidence base is still thin relative to the scale of the claims being made about AI and the future of work. The MIT study and the Remote Labor Index offer rare, data-rich glimpses into how today’s systems perform on tasks that resemble actual jobs, and they both point away from simplistic narratives of either imminent mass unemployment or harmless productivity tools. As more empirical evaluations emerge, the central question will not be whether AI can do “a job,” but which specific tasks it can do well, under what conditions, and with what kinds of human partnership. Until that map is drawn in much finer detail, forecasts about AI-driven labor upheaval will remain more speculative than their confident tone often suggests.
More from Morning Overview
*This article was researched with the help of AI, with human editors creating the final content.
