Beating RL With Reflection: GEPA and Optimize Anything — Lakshya A. Agrawal, GEPA
Lakshya Agrawal, creator of GEPA and a PhD student at UC Berkeley's Sky Computing Lab, presents reflective optimization as an alternative to scalar-reward reinforcement learning. GEPA lets a model read full rollout traces, including reasoning, tool calls, and error messages, and rewrite prompts from that feedback, with a Pareto candidate pool guarding against local optima. The talk covers the GEPA vs. GRPO sample-efficiency comparison, domain-knowledge discovery, the Optimize Anything API for text-based artifacts like code, harnesses, and skills, and production results from Databricks, Snorkel, and others.
Agrawal opens by framing the core problem: how to teach AI new tasks without the enormous data budgets that current methods assume. Weight updates through gradient descent work, whether in pre-training (trillions of tokens), supervised fine-tuning (tens of thousands of labeled examples), or reinforcement learning (hundreds of thousands of rollouts in domains like math and coding). Most teams have neither that data nor that compute. He describes two faces of the sample-efficiency bottleneck: scarce domain-specific knowledge, which makes offline methods like SFT impractical, and expensive rollouts, where either the LLM pipeline or the task metric is slow to run. Agents that work for hours make online learning with hundreds of thousands of rollouts infeasible, and real-world agents that call long-running tools make the problem worse.
What RL throws away
In the dominant reinforcement learning with verifiable rewards paradigm, a model runs many parallel rollouts, receives a reward at the end, and an algorithm like GRPO converts that reward into gradients. Agrawal’s critique is that a single scalar carries very little of what happened. Each rollout contains chains of thought, tool calls, environment responses, and error messages that have real diagnostic value, and in his words, ‘we learned almost nothing from all of that.’
Reflection in text space
The proposal has two parts. First, instead of collapsing a trace into a 0/1 signal, a language model or agent reads the entire rollout and reflects on what worked and what did not. That reflection can use all intermediate outputs and even make further tool calls, such as retrieving from a company knowledge base or a reference textbook. Second, instead of nudging weights with many small deltas, the method updates a prompt, where one natural-language edit can produce a large behavioral change. His example: changing a summarizer’s instruction from ‘generate a one-line summary’ to ‘generate a 10-line summary’ shifts behavior dramatically with a single word change. Achieving the same shift through gradient updates would take thousands of tiny sequential steps.
How GEPA works
The transcript refers to the method inconsistently (the transcription renders it as ‘JPEA’ or ‘Japa’), but it is GEPA, a reflective prompt optimization technique for agents. It is described as akin to reinforcement learning in text space, where each evaluation returns a score plus rich textual feedback that can be highly domain specific. The algorithm has three steps. It runs the AI pipeline, written in any agent framework or as raw LLM calls, on a few examples and collects domain-specific feedback. A reflection model reads that feedback and proposes an improved prompt. Most importantly, it maintains a Pareto pool that keeps every candidate that wins on at least one training example, not just the top scorer.
Agrawal explains why the pool matters. In a simple loop, an LLM keeps refining only the best prompt, which can get stuck in a local optimum and exhaust its search budget without improvement. His search-tree comparison shows a greedy loop repeatedly proposing edits that are no better than the current best. Pareto-based selection maintains a more balanced search and converges to a much higher score. Across four benchmarks, GEPA’s Pareto strategy roughly doubles the gains of applying the model in a loop.
GEPA versus GRPO
The headline comparison plots training steps (proportional to samples seen) against task performance. After a single reflection round using just three data points, GEPA achieves twice the gains that GRPO reached after 25,000 rollouts. Continuing to run GEPA widens the gap by another factor of roughly two. Notably, the optimizing model is Qwen 3 8B (transcribed as ‘Quen 38B’), and no external expert teacher is involved. The model improves itself.
What GEPA actually learns
Agrawal contrasts GEPA with earlier prompt optimizers that relied on model idiosyncrasies, such as tricks like ‘my grandmother will be angry if you don’t generate a good prompt.’ GEPA instead writes detailed problem specifications: how to interpret the input, what the purpose and context of this pipeline stage are, and what key observations and lessons the data reveals. In a multi-hop question answering example, GEPA discovered that first-hop documents typically cover one entity or aspect, so the second hop should retrieve documents related to that entity or aspect. Human teams often spend weeks manually tweaking prompts each time a new model appears, while this process is automated and takes roughly 30 to 60 minutes. Applied to GPT-4.1 mini, the same approach optimized it to outperform GPT-4.1 on a math task, with the distilled domain knowledge visible in the resulting prompt.
Surfacing domain knowledge: the AMD NPU case
AMD’s new XDNA2 NPU came with a new programming API that had almost no public information, and leading models such as GPT-4 failed at it. An existing agent that scored roughly 4.25% was improved, without any other changes, to about 30.5% after GEPA produced a prompt. One notable line instructed the agent to avoid including ADF.h. AMD does ship that library for NPU programming, but it did not work with this hardware generation, and GEPA discovered that in a single step. Agrawal uses this to argue that domain-specific information embedded in prompts can dramatically improve performance, and that GEPA can discover it automatically.
Optimize Anything: beyond prompts
Because prompts are just text artifacts that determine system behavior, the same reflective loop can optimize anything that can be written as text and scored. Agent harnesses are ultimately Python or JavaScript files, and CUDA kernels, serialized numeric parameters, and cloud scheduling heuristics are all candidates. The ‘Optimize Anything’ API takes a set of problems and an evaluator, a fitness function that returns a score along with arbitrary side information. That information might include expert feedback, compiler errors, profiler output, tool-call errors, documentation, job traces, or SLA violations. The dictionary is open-ended, and Agrawal’s central advice is to ‘bring actionable side information’ and expose as much domain-specific detail as possible to the optimizer.
Several demonstrations illustrate the range. A simple prompt asking for a 3D unicorn rendered as a PNG via Python code was compared against Claude Opus 4.6’s output, with the optimized result shown as a visual contrast. More substantively, a four-line Python program that merely asked a model for chain-of-thought reasoning was optimized over 16 reflection rounds into a six-step agent. That agent performs rule hypothesis induction, code synthesis, execution, tracing, debugging, and iterative revision, and it raised Gemini Flash’s ARC-AGI accuracy from 32.5% to 89.5%. Optimizing an agent file for MATH-500 likewise discovered a two-step agent that improved GPT-4.1 nano’s accuracy by 20%, with no architecture specified by the user beyond the objective and task.
Optimizing agent skills
Agent skills, the reusable instruction files used by coding agents such as Claude Code and Codex, are another target. The skill is learned from trajectories with the idea that a skill should help when a similar problem appears. Starting from a mini-SWE agent running GPT-5 mini (chosen for budget reasons), performance on Go repository issue resolution rose from 24% to 93%. The striking result is transfer: skills optimized on the cheap model were applied to Claude Sonnet 4.5, pushing issue resolution to 100% while cutting resolution time by roughly half and reducing token use. Agrawal explains that skills encode repository organization, how to invoke tests, where features are implemented, and which build systems are used. The feature, called GSkill, is open source in the GEPA repository.
Three optimization modes
Optimize Anything supports three modes. Single-task mode optimizes one problem, such as a matrix multiplication kernel. Multi-task search handles related problems that may share information, such as matrix multiplication alongside a dot product kernel. Generalization mode is for settings where training covers a set of problems but deployment will encounter many unseen queries, as in prompt optimization for math tasks.
Results in production
The applications span several areas. In cloud scheduling policy optimization, costs fell by almost 40% compared with expert heuristics. The Snorkel team improved internal benchmarks within 20 hours of release. For OCR with multimodal models, error rates dropped by roughly 35% in an externally validated report. Databricks tuned GPT OSS 120B to outperform Claude Opus at a 90x lower cost, and the improvement over Opus was larger than the improvement over the open-source models.
Better models need better prompts
Agrawal pushes back on the idea that stronger models make prompt optimization less important. As models get better at instruction following, precise task-specific instructions yield even larger gains. In the Databricks case, Opus improved substantially when given better instructions.
Learning evals from production traces
For subjective tasks that are hard to evaluate, GEPA can learn evaluators from production traces. The workflow is to collect agent traces, have humans annotate about 50 of them with detailed feedback (noting, for example, that a response is long, short, good, or uses a particular terminology), optimize an LLM-as-a-judge prompt against those annotations, and then use that judge to optimize the agent. The result is a data flywheel that teams are already running in production.
Fast-slow learning and adoption
The speaker briefly mentions the ’learning fast and slow’ paper, which co-optimizes model weights together with prompts and harnesses. It shows properties desirable for continual learning. GEPA has also been adopted in production by several companies, with the CEOs of Dropbox and Shopify discussing their use of it, and OpenAI published a blog post on building self-improving systems with GEPA.
Agrawal closes by noting that GEPA plugs into any framework and model with no hard dependencies. His final message is that many problems can be reframed as optimization, and that practitioners should surface as much actionable, domain-specific information as possible, because future optimizers will be designed to exploit it.