Continual learning

Dec 23, 2025

What are we scaling?

A critical analysis of AI progress, arguing that short AGI timelines are unlikely given the current reliance on pre-baking skills via reinforcement learning. The author contends that true AGI requires on-the-job, continual learning—a capability current models lack. The modest economic impact of AI is presented not as a diffusion lag but as direct evidence of this capability gap. The future of AI will be a gradual, competitive race to solve continual learning, not a sudden takeoff.

Oct 04, 2025

Some thoughts on the Sutton interview

A reflection on Richard Sutton's "Bitter Lesson," arguing that while his critique of LLMs' inefficiency and lack of continual learning is valid, imitation learning is a complementary and necessary precursor to true reinforcement learning, much like fossil fuels were to renewable energy.

Sep 27, 2025

29.4% ARC-AGI-2 🤯 (TOP SCORE!) - Jeremy Berman

Jeremy Berman, winner of the ARC-AGI v2 public leaderboard, discusses his novel evolutionary approach that refines natural language descriptions instead of code. He explores the idea of building AI that synthesizes new knowledge by constructing deductive "knowledge trees" rather than merely compressing data into "knowledge webs," touching on the fundamental challenges of reasoning, continual learning, and creativity in current models.

Sep 26, 2025

Richard Sutton – Father of RL thinks LLMs are a dead end

Richard Sutton, a foundational figure in reinforcement learning, argues that Large Language Models (LLMs) are a flawed paradigm for achieving true intelligence. He posits that LLMs are mimics of human-generated text, lacking genuine goals, world models, and the ability to learn continually from experience. Sutton advocates for a return to the principles of reinforcement learning, where an agent learns from the consequences of its actions in the real world, a method he believes is truly scalable and fundamental to all animal and human intelligence.