RLM: Recursive Language Models for Large Codebases - Shashi, Superagentic AI

This talk introduces Recursive Language Models (RLM) as a solution for coding agents struggling with large codebases. RLM externalizes context management into a programmable REPL, allowing the model to write code, inspect the repository, and recursively delegate sub-questions via `llm_query` to curate relevant context. The speaker demonstrates `RLM Code`, an independent implementation, showcasing its operational loop, observability features, and practical applications for AI engineers dealing with complex software projects.

July 12, 2026 5 min read

RLM: Recursive Language Models for Large Codebases - Shashi, Superagentic AI — source: https://www.youtube.com/watch?v=8oyalrfwgjw

This talk introduces Recursive Language Models (RLM), a pattern originally published by MIT, designed to address the significant challenges AI coding agents face when working with large codebases. While traditional coding agents excel on smaller or mono-repositories, their performance degrades dramatically with increasing context in larger projects, leading to issues like context explosion and difficulty in managing extensive codebases. Traditional approaches—like using grep for searching, semantic search, context compression, or memory solutions—often fall short.

RLM’s Core Thesis: Externalizing Context Management

The fundamental idea behind RLM is to externalize context management into a programmable execution environment (REPL). Instead of attempting to load the entire codebase into the model’s limited context window, RLM treats the repository as data on which the model can operate. The model is empowered to:

Write code (e.g., Python scripts) to inspect the repository.
Slice and compute relevant chunks of information.
Curate and feed only the most pertinent context back into its main processing window.

The speaker draws an analogy to a lead software engineer onboarding to a massive codebase. Such an engineer wouldn’t read every line but would instead inspect the structure, make notes, identify dependencies, and, when stuck, consult other experts. This mirrors RLM’s approach, where the model makes “notes” (extracted context) and “asks experts” (delegates to llm_query).

The RLM Recursive Loop

RLM operates through a recursive loop:

Repository as Context: The entire codebase serves as the primary data source.
Programmable REPL: The model writes and executes code within this environment to extract relevant information.
Bounded Observation: Snippets of relevant context are identified and extracted.
LLM Query for Recursion: If the model requires further information or deeper analysis, it формулирует focused sub-questions and delegates them to another Language Model (or system) via an llm_query mechanism. This is where the recursion truly comes into play, as the sub-question might initiate another RLM loop.
Loop Continuation & Termination: Responses from the llm_query feed back into the main loop, which continues until a comprehensive answer or final synthesis is achieved. The loop terminates once the required results are obtained.

Why Codebases Are Uniquely Suited for RLM

The speaker emphasizes that codebases are distinct from generic text (like books or dictionaries) because they represent structured data. They include directories, tests, imports, dependencies, configuration files, and even images. A model must not only process text but also understand and reason over this intricate structure. RLM’s ability to programmatically inspect and interact with this structure makes it particularly effective for code-related tasks.

RLM Code: An Open-Source Implementation

Shashi introduces RLM Code, an independent, open-source reference implementation developed by Superagentic AI. It serves as a research playground to demonstrate RLM concepts under the hood.

Key distinctions:

RLM is a general concept and pattern.
RLM Code is a specific implementation.

Other implementations exist, including the official MIT versions (RLM and RLM minimal) and an RLM implementation within DSPy (authored by Omar, a co-author of the RLM paper). RLM Code was built to strictly adhere to the original RLM paper’s ideas, offering flexibility:

Compatibility with both local and cloud-based models.
Seamless integration with any observability framework for tracing and debugging.
Ability to be plugged into other frameworks (e.g., for specific use cases or custom harnesses).

Live Demo Highlights (Conceptual)

The presentation included a live demonstration of RLM Code operating on a large codebase. Key elements showcased were:

The model writing REPL code to extract snippets.
Building “evidence” from the code.
Making recursive llm_query calls to gain deeper context.
Obtaining final results.
Comprehensive observability: displaying tool calls, token usage, execution traces (in JSONL format), session runs, and full trajectories within the RLM loop, including REPL code, observations, and final outputs. Both a command-line interface (CLI) and a terminal user interface (TUI) were demonstrated.

Practical Applications for AI Engineers

AI engineers can leverage RLM concepts for various real-world scenarios, particularly when dealing with large and unfamiliar codebases:

Root Cause Analysis: Efficiently pinpointing issues within complex systems.
Repository Onboarding: Rapidly understanding new or unfamiliar codebases.
Designing Custom Harnesses: Building bespoke agent workflows that encapsulate planning, coding, observation, sub-calls, budget management, and final output generation.

Industry Adoption and Future Trends

The speaker notes that RLM concepts are gaining traction and are being implicitly or explicitly adopted in various proprietary and open-source systems:

Managed Agents & Dynamic Workflows: Concepts like those powering cloud-managed agents (e.g., Gemini’s managed agents) or dynamic workflows that spawn multiple specialized agents within separate sandboxes often derive from RLM principles.
CodeDex: Uses Python REPL execution for context curation, an example of RLM in practice.
Software Factories: Many software factory concepts are likely using RLM under the hood.
Anthropic’s Acknowledgment: Cloud code engineers from Anthropic have publicly acknowledged using RLM concepts in their work, underscoring its real-world utility in advanced AI development.