Beyond Bigger Models: Recursion As The Next Scaling Law In AI

The Reasoning Ceiling of Standard LLMs

Standard Large Language Models (LLMs), built on the Transformer architecture, have a fundamental limitation when it comes to certain reasoning tasks. Unlike their predecessors, Recurrent Neural Networks (RNNs), which are naturally recursive, Transformers are feed-forward during training. This design choice brilliantly solves the backprop through time (BPTT) problem that plagued RNNs, such as vanishing or exploding gradients and massive memory requirements for long sequences.

However, this comes at a cost. A Transformer with a fixed number of layers has a fixed number of computational steps it can perform in a single forward pass. This creates a hard ceiling for “incompressible problems” that require more computational steps than the model has layers. A key example is sorting: a comparison sort has a theoretical lower bound of O(n log n) steps. If an LLM with 30 layers is given a list of 31 elements to sort in one pass, it is computationally impossible for it to perform the necessary comparisons. Other examples of such problems include solving Sudoku puzzles or navigating mazes.

The common workaround for this is Chain of Thought (CoT), which makes LLMs Turing-complete at test time. By generating intermediate reasoning steps as text, the model effectively uses its own output as a scratchpad. However, this is a “hack” that relies on the model having learned these reasoning patterns from human-generated text in its training data. It cannot discover novel, complex algorithms from first principles and is bounded by the limits of existing human knowledge.

HRM: Hierarchical Reasoning with Truncated Backpropagation

The Hierarchical Reasoning Model (HRM) paper introduces a recursive approach to grant models the necessary computational depth. It was a 27-million parameter model trained from scratch on only about a thousand examples for the ARC Prize challenge, yet it achieved state-of-the-art results, while models like GPT-3 scored zero.

The architecture features three levels of recursion using two main components: a low-level network (L-Net) and a high-level network (H-Net), each with its own hidden state (ZL and ZH).

1. Low-Level Loop: The L-Net iterates on the input and its hidden state TL times.
2. High-Level Loop: The H-Net executes once, then calls the entire low-level loop. This combined process repeats TH times.
3. Outer Refinement Loop: This entire hierarchical process is wrapped in an “outer refinement loop” that runs N_outer times. This final loop proved to be the most significant contributor to the model’s performance.

To avoid the pitfalls of BPTT across these many recursive steps, HRM employs a clever training trick inspired by Deep Equilibrium (DEQ) models. Instead of unrolling the entire computation graph, it uses a form of truncated backpropagation. The model is run on the same batch of data repeatedly (e.g., 16 times). Crucially, the hidden states (ZL, ZH) are not reset between these runs. The gradients are calculated and weights are updated after each run, but the backpropagation is truncated, only flowing through the final step of the L-Net and H-Net modules. This is justified by viewing the process as creating a new “mini-batch” in the latent space with each iteration, as the evolving hidden states provide a different starting point for the computation each time.

TRM: Simplifying and Scaling Recursive Reasoning

The Tiny Recursive Model (TRM) paper builds directly on HRM, simplifying the architecture while improving performance. TRM is a 7-million parameter model that achieved an 87% score on ARC Prize 1, surpassing the larger HRM.

TRM distills the key lessons from HRM and makes two primary changes:

1. Simplified Architecture: Instead of separate L-Net and H-Net, TRM uses a single, smaller, weight-shared network. However, it maintains two distinct hidden states (ZL and ZH) to represent low-level and high-level features, preserving the concept of variable scoping.
2. Deeper Truncated Backpropagation: While HRM backpropagated through only the final module call, TRM backpropagates through one full latent recursion loop. This provides a richer gradient signal while still avoiding the full cost of BPTT.

Core Concepts: Latent Space Reasoning and EM Analogy

The training process for both models can be viewed as a form of the Expectation-Maximization (EM) algorithm. The model iteratively refines its hidden states by:

1. Updating the low-level “scratchpad” state (ZL) conditioned on the input and the high-level state.
2. Updating the high-level state (ZH), which represents a “candidate latent answer,” conditioned on the newly updated low-level state.

This process allows the model to discover complex, multi-step reasoning procedures on its own, without being explicitly guided by human-written chain-of-thought examples. It operates in a continuous latent space, which is far more expressive than the discrete token space that LLMs are confined to for their external reasoning.

Future Directions: Combining Recursion and Scale

While HRM and TRM are incredibly powerful, they are task-specific models. The model trained on Sudoku cannot solve ARC Prize puzzles. The most exciting future direction is the synthesis of these recursive architectures with large-scale foundation models.

The proposed paradigm is to use a massive LLM for what it does best: learning a rich, semantically organized embedding space from vast amounts of data. Then, within that latent space, a small and efficient recursive model like TRM can be trained to perform complex, iterative reasoning. This approach would combine the immense world knowledge and representation power of LLMs with the deep, algorithmic reasoning capabilities of recursive models, potentially unlocking a new level of performance and efficiency in AI.

• Tags:
• Transformers
• Reasoning
• Recursive Models
• HRM
• TRM
• Inference Time Computation
• ARC Prize