Why 80% Reliability Isn't Good Enough — Felipe Blanes, Amazon AGI Lab
Felipe Blanes from the Amazon AGI Lab shares lessons from working directly with customers of Nova Act, Amazon's browser agent service, from research preview to general availability on AWS. He argues that static benchmarks leave teams hoping their evals match real usage, and proposes a customer-driven eval flywheel, the trust cliff around 90% reliability, the value of being open about limits, and principles for keeping evals fresh.
Felipe Blanes, a member of technical staff at the Amazon AGI Lab, opens with a problem many of us in AI have run into: an agent that scores well on benchmarks and then breaks the moment real customers use it. He calls this the “benchmark illusion.” Over the past year his role was customer enablement for Nova Act, Amazon’s service for building browser agents that automate anything you can do in a web browser. The product went from a research preview in March, to a new developer experience in July, to a general availability launch on AWS in December. Throughout that span, Felipe worked day to day with customers to understand their problems and how the product could solve them.
The benchmark illusion
Static evals, whether public benchmarks or synthetic datasets you build yourself, are optimized before shipping. The standard loop is familiar: receive requirements, build, run evals, optimize, and ship. Once the product reaches customers, a static eval offers little more than hope that it reflects what they will actually do. Customers try things nobody anticipated, the product fails, and the team is left with nothing to act on. Felipe is explicit that the fix is not better benchmarks but closing the customer loop.
The eval flywheel
Felipe proposes a four-step flywheel that repeats continuously.
- Define success the way the customer does, not the way the team imagines it.
- Capture signals. Engineers instinctively reach for instrumentation, but telemetry can mislead or miss the real problem. Talking to customers, scheduling meetings and visiting them, reveals what the data may obscure.
- Diagnose gaps. Signals sort into three buckets: model failures that feed back into evals, engineering or harness issues that become traditional bug fixes, and product issues around positioning, including whether customers understand what the product is for.
- Feed decisions. Prioritize the gaps and push them to science, engineering, and product teams, then repeat.
The lab has run this loop several times since March.
Four stages of the customer journey
Customers evolve as they use the product, and Felipe describes four stages. Use case discovery is about learning what problem the customer actually has. Early adopters try crazy things, showing what is possible and often surprising the team. Scaling reveals patterns, including a hero use case that most customers share. Product gaps, which surface along the way, become the roadmap for what to build next.
Lesson 1: the trust cliff
Felipe’s central observation is what he calls the trust cliff. At 80% reliability, customers do not feel they can hand off work. They feel they must keep monitoring the agent and handle failures manually, so the agent creates more work rather than less. Around 90%, the message changes to “okay, I can actually trust this,” and customers hand over the entire workflow. From the customer’s perspective, reliability is effectively binary: you trust it or you don’t, with little in between.
Lesson 2: be open about limits
Felipe argues for transparency along three lines: what the product does well and the reliability that justifies trust, what the team is actively improving, and what is out of scope. The third is the hardest because nobody likes to admit it, but customers who hit a failure they weren’t warned about will leave for one of the many alternatives on the market. His takeaway is that openness about limitations builds more trust than higher benchmark numbers do.
Lesson 3: surprising use cases
Working closely with customers surfaced uses no one expected. Amazon Leo, Amazon’s satellite internet company, led the team to build trajectory caching: successful runs are cached and the model is called only when a cached run fails, which lowers inference costs. Hertz used Nova Act for QA automation, but while some QA engineers could write Python scripts, many were not technical, which pushed the team to build a non-technical experience. Sola, an RPA company that automates processes for other businesses, needed more flexibility to customize the actuation stack. Felipe’s point is less about these specific features and more about the broader lesson that customers show you what to build next.
Four principles for keeping evals fresh
After running the flywheel repeatedly, Felipe distilled four principles. First, derive eval scenarios from production rather than imagination. Synthetic data matters at the start, but once customer signals arrive, they should drive the evals. Second, categorize gaps into actionable areas such as product, engineering, and research. Third, share capabilities honestly, including weaknesses. Fourth, run the flywheel as fast as you can. If your evals are not getting smarter every week, Felipe warns, they are going stale.
Two takeaways
Evals must reflect what customers care about, not what you think they care about. And evals are never built once. They form a flywheel that evolves as customers’ journeys grow more complex, so the evals must keep pace with what customers are trying to do now. Build the loop, ship evals that matter, and earn customer trust.