This Startup Beat Gemini 3 on ARC-AGI — at Half the Cost
Poetic, a startup by ex-DeepMind researchers, has significantly advanced performance on the ARC-AGI benchmark by applying a recursive self-improvement system to Gemini 3. Co-founder Ian Fisher discusses how their approach of automating prompt and system engineering provides a substantial performance boost without needing access to model weights, and explores its potential as a path toward AGI.
Poetic, a new startup founded by former DeepMind researchers, has achieved a significant breakthrough on the ARC-AGI benchmark. By layering their proprietary system on top of Gemini 3, they achieved a 54% score on the private test set, a substantial leap from Gemini 3’s baseline of approximately 33% and even surpassing the more advanced Gemini 3 Deep Think’s 45% at half the cost.
The Core Technology: Recursive Self-Improvement
The central idea behind Poetic’s success is a form of recursive self-improvement (RSI), which co-founder Ian Fisher describes as “the holy grail of AI.” The goal is to create a system where the AI actively makes itself smarter.
Unlike methods that require fine-tuning or access to model weights, Poetic’s approach operates purely at the system and prompt level. This is a crucial advantage when working with closed-source models available only through APIs. The methodology involves:
- Ensemble Methods: The system calls the underlying model (e.g., Gemini 3) multiple times.
- Independent Refinement: Each member of the ensemble works independently to refine its own answer.
- Advanced Voting Schemes: The refined answers are combined using a sophisticated voting mechanism to produce a final, more accurate solution.
This system-level optimization is what differentiates Poetic from other prompt engineering frameworks like DSPy, containing what Fisher refers to as “trade secret insights” that yield a significant performance difference. The entire ARC-AGI solver was an output of their system, which was trained on ARC-1 and then applied to ARC-2 without any specific training on the latter.
The Gemini 3 Catalyst
The release of Gemini 3 was a pivotal moment. While Poetic’s system showed promising results on ARC-1 with other models (reaching 89%), switching to Gemini 3 pushed their performance to 95%. When they applied this new combination to the more challenging ARC-2, they had a “holy cow moment” as the performance jumped to the state-of-the-art 54%.
Fisher attributes this leap to Gemini 3’s exceptional ability to generate code for visual problem-solving, a capability that surpassed previous models. He also notes that other powerful models like Anthropic’s Opus can be swapped in for Gemini 3 to achieve similar results, albeit at a higher cost.
A Path to AGI and Practical Applications
Fisher views RSI as both a practical tool for immediate performance gains and a credible path toward AGI.
- Immediate Value: The performance “bump” from Poetic’s system can be highly valuable. On the ARC-AGI benchmark, which allows for two solution submissions, their method provided a single, higher-quality solution that outperformed the underlying model’s two submissions, sometimes at a lower overall cost.
- Long-Term Vision: While not the only path, Fisher believes RSI is “the most exciting path to AGI and beyond.” The process on ARC-AGI was stopped manually due to cost constraints, suggesting that with more resources, the performance could have “hill-climbed” even further.
Automating the Prompt Engineer
The broader vision for Poetic is to automate the complex and often manual process of prompt engineering and agent creation. Fisher draws an analogy to the evolution of deep learning, which automated the manual process of feature engineering.
“We are quite intentionally automating ourselves, automating prompt engineers, automating people who are building agents. It’s a power tool.”
He contrasts their previous manual work at DeepMind—akin to building a car by hand—with Poetic’s technology, which is like “building a factory to build cars.” The goal is to create a system that automatically discovers the optimal prompts and system configurations, removing the human from the tedious trial-and-error loop. While continuing their research and targeting other high-impact benchmarks, the six-person team is now also focusing on bringing this capability to customers to solve real-world business problems.