AI Can Automate 60% of Tasks... But There's a Catch
MIT FutureTech director Neil Thompson revisits his group's 2020 computational limits paper and discusses his newer research on how AI automates tasks across the economy, covering the crashing-waves versus rising-tides survey, adoption bottlenecks, fixed-cost economics, apprenticeship risks, AI safety, and the open versus closed model debate.
Revisiting the Computational Limits of Deep Learning
Neil Thompson opens by scoring his group’s 2020 paper on the computational limits of deep learning, six years on. He credits it with correctly anticipating that scaling laws would drive models from inexpensive to very expensive systems, making cost pressure around chips, capital and electricity central. He admits the paper under-called algorithmic progress: the economic modeling implicitly assumed efficiency gains, but the team never named them. Transformers are the standout example, along with the later wave of serving and inference optimizations such as speculative decoding, distillation and quantization. He says the pace of inference efficiency gains surprised him. The team expected AI to drift back toward classical, provably optimal algorithms, but instead models learned to call tools (for example, a C library that sorts a list), so the efficiency lives in the tools rather than in the core learning method.
From Chatbots to Task Routing
Ben Lorica ties this to Jeff, a recently announced system that treats routing as a scoring and decision problem rather than a text-generation problem. In a DevOps incident, an LLM would simply say the site is down and recommend the infrastructure team, while a routing model would estimate urgency, churn risk and the probability that a given team should handle it. The discussion frames this as the system-one versus system-two distinction, with more upfront planning and optimization replacing free-form text answers.
Benchmarks and Measuring Real Impact
The conversation then moves to evaluation. Early benchmarks such as ImageNet were convenient samples, and Thompson argues that what matters is the effect on the real labor market and economy, so evaluation should move as close as possible to actual tasks.
Crashing Waves vs. Rising Tides
The second paper rests on a survey of workers rather than computer scientists. The team used more than 40 models, recruited people with real experience in their occupations through Prolific, and drew tasks from ONET, the Bureau of Labor Statistics catalog of job tasks. Because ONET tasks are broad, GPT-5 generated concrete task instances, which respondents screened for realism. Each instance was answered by several LLMs of varying size, and respondents rated the answers without being told which were AI-generated, to avoid bias. The sample spans more than half of US occupations, with roughly 70 to 80 percent having at least some tasks covered.
Thompson contrasts this with the Berkeley/METR-style finding that AI capability on software tasks has a steep horizon: short tasks are easy, long ones collapse. He calls that a crashing-waves world, where workers feel safe until a rogue wave suddenly covers them. The explanation is compounding error: software work requires many sequential steps to all go right, so a few percent error per step produces exponential decay. Many economic tasks, like writing a marketing plan, degrade gracefully, so the expected pattern is a flatter, gradual rise, a rising tide. The team proxies step count with human task duration, asking LLMs for tasks of different lengths, and empirically finds that longer tasks are performed worse.
Ben notes the parallel with mathematicians, who saw AI progress before the broader community. Thompson points out that math shares features that make it amenable to automation: it is largely written down, mostly mental, has abundant data, and is verifiable through tools like Lean. Even so, frontier labs still rely on mathematicians to pose questions and judge answers. The survey distinguishes between automating at the level of an average employee, which AI can do for about 60 percent of tasks, and at a superior quality level, which it reaches for only about 27 percent. The gap suggests that humans still matter at the frontier, though this area will likely move quickly.
The Apprenticeship Dilemma
Ben raises the worry that if AI absorbs entry-level work, the pipeline that trains future experts breaks, whether that is a graduate student, a junior lawyer, or a programmer. Thompson takes the concern seriously but argues that the apprenticeship model is inherently slow: a junior lawyer sees few instructive cases and gets sparse feedback from senior partners. AI could greatly accelerate learning by showing trainees many variants and asking which are good, and that training function need not live inside the firm. Academia faces its own version of this, as educators debate how to teach with and without AI. He cites Bill Gates’s notion of human-reserved roles and Terence Tao’s idea that some problems should be set aside as a non-renewable resource for training. Thompson’s response is that the answer depends on what is lost, and that a currently limited reserve need not remain limited.
Why Adoption Lags Capability
The team’s forthcoming paper finds that adoption lags significantly behind capability. In the study, AI is given all the information it needs, whereas in the real world information flows are incomplete, such as a dentist’s observations not yet recorded in the medical record. Adoption also carries friction: even calling an API requires setup and reliability work, and building full pipelines is harder still. Thompson estimates that roughly half of the potential gains are eroded by these costs, though the figure is falling year over year as tooling improves.
On bottlenecks, economists tend to argue that automating one step shifts the constraint elsewhere. Thompson cites work by Mert Demirer and co-authors showing that after tools like Claude Code arrived, lines of code rose by about 800 percent, commits and files rose far less, and actual releases rose only about 20 percent. Generation is cheap, but verification, acceptance and deployment are not keeping pace.
AI Scale and Task-Based Automation
The second new paper is more theoretical, using a general equilibrium model in which substitution between labor and machines depends not only on marginal cost but also on fixed costs. Those fixed costs include training frontier models and building customized systems on top of them, along with data collection, labeling and validation. Empirical evidence supports the scale argument: about 20 percent of firms with fewer than 50 employees report using AI in a business function, compared with 37 percent of firms above 250 employees. The paper estimates that roughly two-thirds of AI adoption is explained by fixed costs rather than marginal costs. Thompson notes that as inference costs fall, fixed costs may become the dominant barrier.
Ben adds that large firms also have talent, and that even with cheaper inference, safe and responsible deployment still requires some expertise. Thompson suggests that small firms can rent that expertise from consultancies such as Accenture, Deloitte or IBM for a few weeks at a time, and that the real question is whether a task is repeated often enough to justify automation. Across roughly 30 case studies, firms repeatedly found that gathering and labeling proprietary data was the largest cost.
On which sectors lead, Thompson expects the capability frontier to keep rising in the same domains, so the same firms may remain ahead, though competitive advantage can shift as AI erodes what made firms valuable. Firms with unique data may become more valuable. On management consulting, Thompson agrees that junior analyst work, the intellectual horsepower and long hours, is vulnerable, while the value of cross-industry knowledge and specialized AI expertise may persist and business models will evolve.
Safety Risks
The conversation closes on safety. Thompson is less AI-doomer-pilled than many in San Francisco and sees cybersecurity and bioterrorism as the most concrete concerns. He points to an incident at the Bank of Thailand in which an agent penetrated systems without successfully exfiltrating anything important. He is more worried about what labs see at the cutting edge, particularly biological risk, and about open-source release shortly after a capable model. His lab maintains the MIT AI Risk Repository, and a Delphi study of 272 experts estimated more than a 10 percent chance of a catastrophic outcome, defined as a million deaths or 100 billion dollars in damages, within five years for risks including cyber, weapons development, manipulation and competitive pressure to remove safeguards. Ben notes that AI could also improve security through rigorous testing and automated patching, while a powerful AI aimed at a system could still take it down. Many organizations in the developing world, such as a power generator or hospital, will not get early access to secure their systems.
Open vs. Closed Models
For the final question, Ben argues that open models will ultimately capture most compute, much as Linux and open-source databases prevailed over Solaris and Oracle. Thompson agrees that closed models will likely keep defining the frontier, but that open-weight models being good enough for most uses is plausible. His main uncertainty is distillation. If frontier labs can distill large models into a family of smaller ones that outperform comparable open models, proprietary providers could win on price and performance. The current data does not show a large gap between black-box and white-box distillation, but that could change. Ben counters that open models are already handling more inference and that scientific applications show proprietary models being shared, so the trend could run either way. Customization, control and data privacy remain strong arguments for open models, and competition among model providers benefits cloud platforms and infrastructure players.