Building uReview, Uber’s Multi-Agent Code Review Engine — Will Bond & Ameya Ketkar, Uber
Uber's Will Bond and Ameya Ketkar present uReview, an in-house automated code review system developed to combat rising review times (3 to 9 hours for first review). They detail why Uber built it over buying—due to Phabricator, agentic SDLC integration, and distributed ownership—and its architecture, including comment deduplication. The presentation highlights their iterative approach using advanced observability (sentiment, addressal rate, agent trajectory) to tune model performance, acknowledging that "the model never knows that it is wrong." They discuss empowering hundreds of teams with custom review agents and skills, sharing impressive results: 25,000 comments weekly, 67% addressal rate, and 60% cost reduction. Finally, they explore the evolving role of human engineers in an agentic SDLC, predicting an "expanded outer loop" focused on architecture and domain expertise rather than direct code review.
Uber’s in-house automated code review system, uReview, was developed to tackle the escalating bottleneck of code reviews within its massive software engineering organization. With thousands of engineers across hundreds of teams and six language-specific monorepos, the “first time to review” metric surged from 3 hours in 2024 to 9 hours in 2026, underscoring an urgent need for efficiency and automation.
Why Uber Built uReview In-House
Uber opted for a custom solution over commercial offerings due to several unique constraints:
- Legacy Infrastructure: Continued reliance on Phabricator, with a migration to GitHub underway, while most vendor solutions lack Phabricator support.
- Agentic SDLC Integration: A core requirement was to provide a consistent code review experience for AI agents, ensuring they are reviewed against the exact same rules and standards as human engineers.
- Distributed Ownership: With hundreds of teams, centralized management of code review rules and customizations was impractical. uReview needed to integrate with existing team ownership models, allowing teams to manage their specific review logic.
- Risk Profiling & Complexity: The system must adapt review intensity based on code change risk and complexity, avoiding a one-size-fits-all approach.
- Consistency & Reliability: Ensuring critical reviews (e.g., security, compliance) are consistently applied, independent of human oversight.
uReview Architecture Overview
uReview acts as a central service that ingests review requests from various sources, including GitHub, Phabricator, and internal agent loops. It features:
- Generators: Multiple generators, tuned for different performance and cost profiles, create review comments. The system also supports plugging into third-party code review tools for benchmarking.
- Post-processing: A crucial component that rates, categorizes, filters, and deduplicates comments from various generators. This ensures engineers receive “only the highest confidence comments that are actionable for them to work on,” avoiding the common issue of overwhelming AI-generated feedback.
Evolution Through Observability and Evaluation
The development of uReview started humbly with a basic prompt for logic checks. Early observability was rudimentary, focusing on cost, NPS surveys, and basic feedback forms, which showed “quality to cost ratio was like all over the place.” This led to a significant shift towards more sophisticated metrics:
- Sentiment Analysis: Categorizing replies to uReview comments (positive, negative) identified systemic bugs and issues, allowing for targeted improvements.
- Addressal Rate: Tracking whether developers actually addressed comments made by uReview became a key indicator of actionable feedback.
- Agent Trajectory: Analyzing the agent’s runtime profile, including tool calls and internal thought processes, provided insights into why the agent made certain decisions, enabling fine-tuning for quality and cost.
A critical learning from this phase was that “the model doesn’t know that it’s wrong. It always confidently says 100% sure… But we saw that no, it actually needs a lot of guidance from the teams.” This highlighted the necessity for team-specific style guides, patterns, and anti-patterns to be baked into the agents, along with clear guardrails to prevent agents from spending time on unproductive tasks.
Empowering Team Customizations
A cornerstone of uReview’s success is its deep support for team-level customizations, acknowledging Uber’s decentralized engineering culture:
- Reviewer Types: The system supports general-purpose single-file reviewers (e.g., “find me all logic bugs per file”) and multi-file reviewers that incorporate monorepo-specific anti-patterns and style guides.
- AI Linters: Few-shot AI problems enable developers to define rules for systematic and mechanical issues with contextual awareness.
- Custom Agents: The most powerful feature allows teams to define their own custom agents, link them to knowledge bases, past PRs, and develop specific “skills” for review.
Implementing this at scale required:
- Integration with Ownership Model: Co-locating customizations with code to enable quick updates.
- Smart Deterministic Routing: Directing specific types of reviews to the right teams and models.
- Distributed Observability: Surfacing metrics like agent trajectory, addressal rate, and sentiment analysis directly to teams. This empowers teams to iteratively refine their custom rules, understanding which ones are effective and which require adjustments.
A key insight was that “writing the skill was very easy,” often by prompting LLMs. However, “the hard part was how to run these skills at scale with consistent quality and low cost,” requiring significant iteration from both the uReview platform team and individual engineering teams.
Tangible Results and Future Vision
uReview’s impact has been substantial:
- Scale: It generates approximately 25,000 comments per week.
- Engagement: Around 67% of comments are addressed, with almost three-quarters of high-severity issues being resolved.
- Efficiency: Cost fell by 60% compared to the naive initial implementation, while quality and accuracy increased by 70%.
Looking ahead, the presentation discusses the shift towards an “Agentic SDLC” and the evolving roles of human engineers, distinguishing between the “inner loop” (agent-authored code) and the “outer loop” (human oversight):
- Inner Loop Demands: Requires extremely high accuracy to avoid “cavitation” (agents fixing something only to introduce new issues due to poor feedback) and prevent agents from fixing trivial “nits” that would frustrate humans.
- Evolving Human Role: While some predict the “outer loop” (human engineers) might be eliminated, Uber believes in “expanding the outer loop.” Instead of humans focusing on implementation details, their role shifts “up a layer” to architecture, domain expertise, and product thinking. The agent excels at writing and reviewing code, freeing humans to guide the system at a higher level of abstraction. This transition means engineers will be “interacting with the system and guiding it” rather than being directly involved in every line of code review.
This concludes the detailed summary of the uReview system, its development, impact, and future implications for software engineering at Uber.