A reflection on Richard Sutton's "Bitter Lesson," arguing that while his critique of LLMs' inefficiency and lack of continual learning is valid, imitation learning is a complementary and necessary precursor to true reinforcement learning, much like fossil fuels were to renewable energy.
Richard Sutton, a foundational figure in reinforcement learning, argues that Large Language Models (LLMs) are a flawed paradigm for achieving true intelligence. He posits that LLMs are mimics of human-generated text, lacking genuine goals, world models, and the ability to learn continually from experience. Sutton advocates for a return to the principles of reinforcement learning, where an agent learns from the consequences of its actions in the real world, a method he believes is truly scalable and fundamental to all animal and human intelligence.