Lessons from 25 Trillion Tokens — Scaling AI-Assisted Development at Kilo
Scott Breitenother, CEO of Kilo, discusses the evolution of software development, where engineers are shifting from writing code to orchestrating AI agents. He shares lessons from processing 25 trillion tokens, emphasizing the critical role of trust, the importance of end-to-end ownership, and how this new paradigm leads to a 10x increase in shipping velocity.
The role of the software developer is undergoing a fundamental transformation. High-performing teams are moving away from the 2023 paradigm of manual coding and endless collaboration meetings toward a future where developers act as orchestrators for AI agents. This shift is not about working harder but about eliminating process bottlenecks that have traditionally slowed down development.
The New Engineering Paradigm: Orchestrator over Coder
The old paradigm saw developers as “code monkeys,” writing boilerplate, attending numerous alignment and review meetings, and constantly context-switching while waiting for input from others. The new paradigm redefines the developer as a manager and visionary who guides AI agents to handle the execution.
At Kilo, this philosophy is put into practice with a few core principles:
- End-to-End Ownership: Each feature is owned by a single engineer, from conception and coding to deployment and user feedback. This eliminates the diffusion of responsibility and communication overhead.
- Anti-Collaboration by Default: Collaboration is not the default mode of operation. It is reserved for situations where it adds clear value, preventing it from becoming a “safety blanket” that hinders velocity. The focus is on multiplying velocity, not killing it with unnecessary processes.
- Coding is the Easy Part: In the age of AI, the primary bottleneck is no longer writing code but the surrounding processes. By empowering individual engineers with AI agents, teams can achieve massive gains in speed. Kilo, with just 15 engineers, ships one to two features per week, a testament to this model.
“The proof is in production. I encourage you all to take a look at our product release notes every Friday. We ship and we ship fast.”
The Ladder of AI Adoption and the Centrality of Trust
Developers don’t instantly become AI power users; they ascend a “ladder” of adoption that is built on a foundation of trust.
- Autocomplete: The “gateway drug.” It’s a low-stakes introduction where the developer remains in full control, simply tabbing to accept suggestions.
- Chat: The developer begins to delegate by asking questions and providing context, but still directs the interaction.
- Single Agent: This is like a driving lesson where the developer gives the AI the wheel but keeps a hand on a “secret steering wheel and a secret brake.” You delegate a single task and watch it closely.
- Orchestration: The final step, where the developer’s role becomes hands-off, letting multiple agents run complex tasks. At this stage, “AI’s got the wheel, you better pray.”
Trust is fragile and can be broken at any point, causing a user to fall down the ladder.
- Latency Kills Trust: Even a latency of 200ms in autocomplete can cause a dip in usage. Slow suggestions break a developer’s flow.
- Errors Erode Confidence: An agent editing the wrong file path is scary and immediately undermines trust.
- Constant Interruptions Defeat the Purpose: An orchestrator that constantly asks for permission or review makes the developer a bottleneck, moving slower than before.
Three Pillars of Building Trustworthy AI Tools
To build and maintain trust, engineering platforms must focus on three key areas where adoption consistently breaks:
- Context is King: The AI tool must have the right information. This creates a “trust cycle”: the more context a user provides (e.g., access to multiple repositories), the more effective the AI becomes, which in turn builds more trust. For example, a Kilo data engineer built out the entire data infrastructure in two weeks by giving his agent access to both the data warehouse repository and the application code repository that generated the data.
- The Right Model for the Job: Using the most expensive, state-of-the-art model for every task is a mistake. It’s slow and costly. The key is to use a combination of models: a powerful model like Opus for high-level architecture and smaller, faster, more cost-effective models (like Kimmy, Miniax, or GLM) for coding, debugging, and autocomplete.
- The Tool Must Get Better Every Day: In a fast-moving field, users expect sharp edges to be smoothed out quickly. A continuous feedback loop, where the tool learns and improves from user interaction, is essential.
The Shifting Nature of Engineering Work
This new paradigm doesn’t eliminate work; it transforms it. The workload shifts from 80% coding and 20% deep thinking to 80% deep thinking and 20% coding. This is a more cognitively taxing balance. The developer’s job becomes that of an architect and conductor, making high-level decisions and setting quality gates, while the agent handles the tedious execution—the boilerplate, the hunt for a missing semicolon, and writing documentation.
This shift enables a 10x improvement in shipping velocity, fundamentally changing what a small team can accomplish. The engineer of the future is not a middle manager but a highly leveraged individual contributor, conducting an orchestra of AI agents to build and ship faster than ever before.