Agents, codebases, and teams — Aditya Khandelwal, Amazon AGI Lab

Aditya Khandelwal addresses the critical challenge of scaling AI agent adoption from individual use to team-wide integration in professional engineering environments. He argues that while ample content exists on setting up personal agent codebases, this advice often “breaks the moment you like actually try to use it with your team in your actual production setup.” Based on his experience leading a team of ten, he identifies that successful agent adoption is fundamentally a leadership problem, not an individual contributor (IC) issue.

The Agent Adoption Journey and Developer Sentiment

Khandelwal outlines the “enterprise journey” of agent adoption:

1. Early Hype: Initial awe and high leverage for a few individuals.
2. Mandate & Token Maxing: Companies mandate adoption, leading to excessive token use and budget overruns (“money is being lit on fire”).
3. “AI Slop” & Retraction: Poor quality outputs (“sev twos”) and a subsequent retraction from mandated usage.

This journey impacts individual developers across a spectrum of “fear vs. confidence.” Initially, fear is high, and utilization is low. Early adopters demonstrate leverage, reducing fear. However, mandates often lead to skepticism due to “slop,” lowering confidence. The goal is to shift developers towards high confidence and low fear, where they “are not fearful and they’re actually using it a whole lot more.”

Symptoms of a Dysfunctional Agent Setup

A common CLAUDE.md and basic skills are insufficient for team success. Khandelwal lists key symptoms indicating a broken setup:

• Babysitting Agents: Constant human intervention required for agent runs.
• “Model Got Dumb Today”: Blaming the model when underlying harness or setup issues are the real cause.
• Silent Context/Money Burn: Unintentionally consuming large contexts (e.g., “500k context” for simple tasks) and incurring high token costs.
• Long Sessions with Intervention: Agents requiring constant manual input over extended periods.
• “Slop Factory”: Consistent generation of low-quality or incorrect code.
• Slow Shipping Compared to Peers: Inability to match the rapid development pace of other companies using agents effectively.

Why Agent Adoption is a Leadership Problem, Not an IC’s

The core argument is that engineers working in isolation to adopt agents creates a “review burden trap.” High-producing early adopters (“10 PRs a day”) can overwhelm others (“one to two PRs”) with review requests for potentially lower-quality, agent-generated code. This leads to the latter group falling behind, seeing “worse code,” and concluding “the agents are the problem.”

The necessary changes for effective agent integration—such as restructuring a codebase for progressive disclosure, converging on a shared setup, or establishing team-wide best practices—cannot be made by a single IC. These are organizational changes requiring leadership buy-in and coordination. “Making engineers work well with their agents is truly the most impactful thing you could do as an organization.”

Principles of Harness Engineering

For a robust agent system, Khandelwal outlines three key principles:

1. Smart Prompt Injection: Treating the entire codebase as a mechanism to “smartly prompt inject the model with just the right context at just the right time.” This means context (e.g., runbooks, documentation) should reside immediately where it’s needed (e.g., in code comments) so the agent can discover it contextually.
2. Close the Loop: Building self-healing systems because “slop is inevitable.” This involves pipelines to detect and remove low-quality output and continuously improve the system.
3. Iterate Continuously: Recognizing that the underlying technology and best practices are constantly evolving. This isn’t a one-time setup but an ongoing organizational commitment.

Crucially, “treat it like a human problem,” acknowledging fear and emotions.

The Playbook for Team Adoption

Aditya’s team developed a playbook centered on:

1. Mastering the Basics with Progressive Disclosure:

   • Identifying best practices from top ICs and standardizing them into a “shared setup.”
   • Progressive disclosure is key: “your entire code base can be set up to encourage progressive disclosure.” This means the agent only gets the context it needs, when it needs it.

2. Investing in One High-Value Skill:

   • His team focused on a skill called “ship it,” which automates the entire process from “code done to PR ready for review”—handling descriptions, comments, CI failures, and merges.
   • Despite often running for “over an hour,” this skill built trust and demonstrated agents’ value, freeing up engineers.

3. Closing the Loop with Infrastructure:

   • Wiring “issues and boards into the repo,” adding agentic reviews, and implementing a “code gardener” that runs nightly to maintain code organization and quality.

4. Winning Over Skeptics & Iterating:

   • Crucially, involve skeptics by allowing them to “edit and play with the shared setup.” Their feedback is vital for iteration.
   • Accept that the setup will “never get to perfection” and dedicate a percentage of IC time to continuous improvement.

Challenges and Lessons Learned

The journey was not without issues:

• Issue Overload: Agents initially created an overwhelming number of issues (e.g., “4 or 500 issues within like a couple of weeks”).
• Lack of Agreement: Easy for engineers to revert to old habits when problems arise. Feedback must be integrated to improve skills.
• Agent Runtime: Long agent runtimes (e.g., “over an hour” for “ship it”) can initially scare people but indicate deeper reasoning.
• Merge Hell: Inevitable when many agents contribute; requires a strategy to manage.
• Prototype Exemption: Allow experimental or prototype code to “opt out of all the rigorous other standards” to avoid stifling innovation.
• Individual Variation: Developers vary in their comfort and skill; leadership must adapt to individual needs and feedback.

Khandelwal emphasizes a mindset shift: instead of “the model is so dumb,” ask “how can I make it smarter” or “how can I edit the shared setup.” He uses a skiing analogy: “You have to commit to the speed in order to actually get and feel like… how I can truly ski.” Full commitment, accepting failures, and learning from them are essential for truly harnessing AGI.

Q&A: Progressive Disclosure Strategies

Key strategies for progressive disclosure include:

• Skill File Limits: Setting a hard limit, such as “~100 lines in your skill.md” (which represents a folder), to prevent overloading context.
• Contextual Documentation: Embedding runbooks and relevant descriptions directly in code comments so agents find them when they grep into the code.
• Thin Index Files: Ensuring CLAUDE.md or agent index files are thin, primarily serving to point to the right files rather than containing all information.
• Measuring First Prompt Context Burn: A crucial test for effective progressive disclosure. If the first prompt burns “40K, 50K” tokens beyond the base, “something’s wrong.” This metric guides iterative improvements.

• Tags:
• Code Generation
• Developer Productivity
• MLOps
• Leadership
• AI Engineering
• LLM Agents
• Harness Engineering
• Team Collaboration
• Agent Adoption
• Progressive Disclosure