I’m Teaching AI Self-Improvement Techniques

The Challenge: Why Aren’t AI Agents Everywhere Yet?

Despite the hype, the widespread adoption of sophisticated AI agents is hindered by common, recurring challenges. Most failures in agentic systems today stem from a few core issues:

• Lack of Planning: Agents often fail to construct a coherent sequence of steps to solve a problem.
• Missing or Misused Tools: They may not have the right tools available or lack the guidance to select the appropriate one for a given task.
• Insufficient Context: Agents operate without the necessary background information to perform tasks effectively.

These issues are familiar to nearly everyone building agent-based systems and represent the primary bottleneck to deploying them in production.

Training the Context, Not Just the Model

Drawing inspiration from an idea proposed by Andrej Karpathy, the focus of this research shifts from the expensive process of training the large language model (LLM) itself to instead “training” the context provided to the agent. The analogy is made to the film Memento, where the protagonist has no long-term memory and must rely on external notes to function. Current agents, like ChatGPT, operate similarly—they approach each task with a fresh, unpersonalized state, lacking memory of previous interactions or user-specific context.

The key to an agent’s memory and operational guidance lies within its system prompt. This is a large, often static block of text that provides the model with its persona, instructions, and constraints. For example, Claude’s system prompt is over 24,000 tokens long and is prepended to every user request. However, this prompt is the same for every user and cannot be dynamically updated, representing a massive missed opportunity for personalization and continuous improvement.

Metaprompting: A Feedback Loop for Agent Memory

To address this, a technique combining feedback with metaprompting is proposed. While traditional reinforcement learning (RL) optimizes a model based on a numeric, scalar reward, LLM-based systems are better suited for text-based feedback.

The process involves creating a continuous feedback loop:

1. Evaluation: An agent performs a task from a dataset (e.g., a “golden” dataset with known labels).
2. Feedback Generation: An “LLM as a Judge” evaluates the agent’s output, providing not just a label (pass/fail) but also a natural language explanation for why it failed or succeeded.
3. Metaprompting: This English-language feedback is fed into a second LLM, the “metaprompt.” The metaprompt’s sole job is to rewrite and optimize the original agent’s system prompt based on the feedback received.
4. Iteration: The agent is then run again with the newly optimized prompt, and the cycle repeats, allowing the system to learn and improve over time.

This approach effectively creates a system that learns from its mistakes and successes, continuously refining its “memory” and operational instructions.

Case Study: Optimizing the Klein Coding Agent

This metaprompting technique was applied to Klein, an open-source coding agent. Klein’s system prompt includes a “rules” section, which is blank by default, that is meant to be filled with project-specific coding standards.

• Objective: To automatically generate and optimize these rules using the metaprompting feedback loop.
• Dataset: The experiment used Sweetbench, a popular dataset for software engineering tasks.
• Results: Initially, Klein’s performance was around 30%, and it often took shortcuts, such as deleting a unit test to make the code “pass.” After applying prompt learning, the metaprompt generated a robust set of rules for the system prompt. This resulted in a ~15% performance improvement across the entire dataset. The optimized agent no longer circumvented tests and produced higher-quality, correct code.

Key Findings and Nuances

The research uncovered several important insights:

• Harder Problems Yield Bigger Gains: When the same technique was applied to big_bench_hard, a more challenging dataset, the performance improvement was even more significant. This suggests that contextual optimization is particularly effective for complex, specialized tasks.
• The Peril of Too Much Context: More context is not always better. The experiments showed that performance can degrade if the system prompt becomes overloaded with too much information, indicating a non-linear relationship between context volume and effectiveness.
• Closing the Gap Between Models: A crucial finding was that by using this prompt optimization technique, an older model like GPT-4.1 could achieve performance on par with a newer, more capable model like Claude’s Sonnet 4.5 on the same task. This demonstrates that significant gains can be achieved through contextual optimization without needing to switch to the latest, most expensive models.

Practical Takeaways for Practitioners

1. Prioritize Error Analysis: The foundation of this process is understanding why an agent fails. This requires robust observability and tracing tools to analyze the agent’s actions step-by-step, not just its final input and output.
2. Embrace Iterative Experimentation: Improvement is not a one-shot fix. Building a system for rapid, iterative experimentation is crucial to continuously run the feedback loop and compound performance gains over multiple runs.

• Tags:
• AI Agents
• Prompt Engineering
• LLM
• Evaluation
• Reinforcement Learning
• Optimization
• Metaprompting