925: AI, Automation and the Future of Work — with Oxford’s Prof. Carl Benedikt Frey

The Fragility of Progress

Technological progress is not an inevitable or guaranteed outcome. History shows that most technology leaders, from the Song dynasty in China to Victorian Britain, do not maintain their position indefinitely. Progress requires constant institutional adjustment as technology evolves. The collapse of the Soviet Union, for instance, can be partly attributed to its inability to adapt to and capitalize on the computer revolution, a technology that thrived in more decentralized systems.

The Two Phases of Innovation: Exploration vs. Exploitation

Technological progress unfolds in two distinct phases, and different economic systems are suited to each.

1. Exploration: This is the early, experimental phase of innovation, characterized by non-standardized, interactive discovery. It thrives in decentralized systems like the United States, where numerous actors can take different risks and explore diverse technological paths. If one venture capitalist declines an idea (like Bessemer Ventures initially passing on Google), many others can still provide funding.
2. Exploitation (Scaling): This phase focuses on execution, cost reduction, and scaling up a proven concept. Centralized systems, like the former Soviet Union, can be effective at mobilizing resources to scale specific industries (e.g., steel production). However, their rigidity hinders the exploration necessary for breakthrough innovation, which is why they struggle during technological shifts.

Networks and Discovery: The Power of Weak Ties

Innovation is often a product of our “collective brain,” a network of connections that circulates information and ideas. Sociological studies show that networks with many weak ties are more effective at fostering novel ideas than those with a few strong ties.

• Silicon Valley: The meeting of Larry Page and Sergey Brin at Stanford, a hub facilitating countless weak ties, laid the foundation for Google. The density of inventors and technologists in a community is a strong predictor of future innovation.
• mRNA Vaccines: The foundational science was developed over decades by Katalin Karikó, who, after leaving socialist Hungary, struggled for funding even in the decentralized US system but ultimately persisted and found the necessary collaborators to make her breakthrough.

The Corporate Stagnation Trap

Large, incumbent companies often face a predictable lifecycle that moves from innovation to protectionism.

“Once the easy gains from innovation are exhausted, big companies stop innovating and start lobbying.”

Initially, companies focus on product and process innovation. However, as they hit diminishing returns, the marginal return on investing in lobbying and protecting their market share against competition becomes higher than the return from incremental improvements. Incumbents are also hesitant to develop new technologies for fear of cannibalizing their existing business models.

• Kodak: Was at the forefront of digital camera technology but hesitated to launch it, fearing it would undermine its film business.
• Google: Despite being responsible for the Transformer architecture, it was slower to launch a generative AI product like ChatGPT, perhaps for fear of disrupting its lucrative ad-based business model.

This highlights the critical role of competition policy and antitrust measures. The US lawsuit against IBM, which forced the unbundling of hardware and software, and the breakup of AT&T were crucial for creating the competitive landscape that allowed the software and internet revolutions to flourish through new entrants.

Generative AI: An Engine of Consensus, Not Novelty

While impressive, current Large Language Models (LLMs) are unlikely to produce truly novel or inconceivable ideas. They are “engines of statistical consensus,” excelling at recombination and synthesis based on existing data.

• If you train an algorithm on Impressionist paintings, its output will be stylistically similar to Impressionism; it won’t invent Conceptual Art.
• An LLM trained on data up to 1900 would likely conclude human flight was impossible, given the long history of failed attempts and the biological limitations of birds. It could not have produced the blueprint for the Wright brothers’ airplane.

For now, humans retain an edge in frontier innovation, discovery, and the ability to make conceptual leaps that are not derivable from existing patterns.

AI’s Impact on the Labor Market

The distinction between a job changing and a job being eliminated is often blurry from the worker’s perspective. If a bus driver’s role is replaced by a vehicle supervisor who doesn’t need a driver’s license, the original driver is still displaced.

A key concern is that new technology firms are creating fewer jobs than in the past. For AI to be truly transformative and create shared prosperity, it must not only automate existing tasks but also create entirely new industries and sectors, as the railroad and automobile did.

AI also lowers the barrier to entry for many white-collar professions and makes them more globally tradable. If AI reduces the productivity gap between an accountant in New York and one in Manila, much of that work may be offshored, affecting the wages of professionals in high-cost countries as if their jobs were automated directly. This could lead to a future where skilled manual labor (plumbers, electricians) commands higher wages than some white-collar professionals.

Policy Lessons and Individual Strategy

• Government Policy: The goal should be to protect people, not jobs. This involves creating flexible labor markets combined with strong social safety nets (a “flexicurity” model, as seen in Denmark). Generous social support reduces resistance to technological adoption, as workers feel more secure during transitions.
• Individual Skills: To remain valuable, individuals should focus on skills that remain human-centric bottlenecks to automation:
  • In-Person Communication: As AI homogenizes written communication, the ability to differentiate oneself through direct, face-to-face interaction will become more valuable.
  • Complex Social Skills & Resilience: The ability to navigate complex social situations and respond to novel, real-world events remains a key human advantage.
  • Manual Skills: Jobs requiring dexterity and navigating unstructured physical environments, like those of plumbers and electricians, are less susceptible to automation in the near term.

• Tags:
• Future of Work
• Generative AI
• Feature
• Labor Market
• Innovation
• Job Displacement
• Economic Systems