AI That Can Change Its Mind? (New Architecture) [w/ Sakana CTO]

Llion Jones, a co-inventor of the Transformer, and his colleague Luke Darlow from Sakana AI argue that the AI industry is trapped in a local minimum by the Transformer's success. They discuss the architecture's fundamental limitations using the 'spiral problem' analogy and introduce their new, biology-inspired Continuous Thought Machine (CTM), an architecture designed for more human-like, sequential reasoning and adaptive computation.