Building an Autonomous Engineering Org - Angie Jones, Agentic AI Foundation
Angie Jones details Block's journey in transforming its engineering organization into an autonomous one using AI agents. She outlines a five-stage AI maturity model, the strategic implementation of an 'AI Champions' program, and the technical steps taken to make repositories AI-ready and enable seamless agent delegation directly within developer workflows. The talk covers challenges in scaling multi-agent parallelism, including code review bottlenecks and resource management, and the development of an orchestrator ('Builder Bot') and 'Company World Model' to achieve full autonomy. It concludes with a critical reflection on the ethical implications of achieving such high levels of AI-driven automation.
Angie Jones recounts the ambitious journey of transforming Block’s 3,500-person engineering organization into an autonomous one, a challenge many tech companies are now facing. The journey began early, with Block building “Goose,” an internal coding agent, even before large language models (LLMs) natively supported tool calling. Their collaboration with Anthropic on the initial release of the Messages API (MCP) positioned Goose as a reference implementation, leading to 90% of engineers using AI for tasks like boilerplate generation and querying in their IDEs.
However, this widespread adoption didn’t translate to faster shipping, prompting a deeper investigation. Jones categorizes AI enablement into three phases: experimentation, adoption, and impact. While Block had excelled in experimentation, achieving true impact required integrating AI into how software was built and shipped, moving towards an “agentic engineering org” where engineers leverage AI agents as their primary means of producing engineering outcomes. This meant treating agents as core workflow members, capable of decomposing problems, delegating work, reviewing, and verifying tasks, with engineers directing the agents as their default mode of operation.
To guide this transformation, Jones developed a Maturity Model for engineer-AI agent interaction, later refined with insights from Steve Yegge’s Gastown article:
- Stage 0: No AI tools used.
- Stage 1: AI used for autocomplete; agent mode possibly unexplored.
- Stage 2: Chatting with agents, but not for generating pull requests (PRs).
- Stage 3: Delegating tasks to agents and reviewing their output.
- Stage 4: Running multiple agents in parallel.
- Stage 5 (Autonomous): Delegating complete tasks, with agents producing shippable results without direct human guidance.
Initially, most engineers were between Stages 1 and 2, with the ultimate goal of reaching Stage 5. The transition faced significant hurdles: the experimental nature of the field (no established playbook), the rapid pace of technological change leading to AI fatigue, and resistance from top-down mandates.
Recognizing that individual upskilling for 3,500 engineers was impractical, Jones leaned into the 90-9-1 rule, focusing on cultivating the “1%” of power users. She initiated an AI Champions Program, hand-picking 50 engineers from critical teams willing to dedicate 30% of their time and resilient to the non-deterministic nature of early AI tools. The initial target for these champions was Stage 3.
The first critical step was making repositories AI-ready. Early agent-generated code often lacked adherence to team conventions, leading to a lack of trust. The solution involved embedding AI directly into repos through assets like context files (agents.md, claude.md) for guidance, rules files for guardrails, repeatable workflows (slash commands, agent skills), enabled AI code reviewers, and AI attribution on PRs. This approach not only improved agent performance but also benefited the entire team. By involving champions from diverse engineering segments (front-end, back-end, mobile, mono repos, small services), Block identified scalable patterns, allowing teams to converge on solutions tailored to their specific needs rather than a one-size-fits-all mandate.
Next, the focus shifted to seamless delegation, making it easier for agents to receive work from common engineer touchpoints: issue trackers (Jira, Linear), GitHub issues, and informal Slack discussions. A compelling example demonstrated an entire bug fix cycle—from diagnosis to PR creation—occurring within Slack in five minutes, showing how agents could become integral to the sprint. This native integration made delegation feel intuitive and accessible, even for non-champions who benefited from the champions’ foundational work.
Within three months, the program yielded impressive results: 69% increase in AI-authored code, 37% reported time savings, and a 21-fold increase in automated PRs.
Moving towards Stage 4 (multi-agent parallelism) introduced new challenges, primarily a code review bottleneck due to the surge in agent-generated PRs, and resource contention as engineers’ machines struggled to handle multiple agents. To address reviews, Block improved AI code reviewers (shout out to Codex) and implemented an auto-fix loop where issues identified by an AI reviewer were automatically fixed and committed by another agent, ensuring PRs were in good shape. For resource management, dedicated cloud workspaces were provisioned for isolated agent environments, enabling parallel execution.
The culmination was Stage 5 (the autonomous engineering org), achieved through Builder Bot, a custom orchestrator. This required creating a Company World Model—a machine-readable representation of Block’s 25,000-repo codebase, detailing services and their interconnections. This model allowed Builder Bot and its delegated agents to pull context as needed, understand the system landscape, and execute plans spanning multiple codebases in parallel. The result was a truly autonomous system where agents produced shippable results without human intervention, accessible even to non-engineers via Slack for tasks like bug fixes or feature implementation.
However, the narrative takes a somber turn. The unprecedented efficiency and autonomy were followed by layoffs, leading Jones to profound ethical questions: “Was this my fault? Did enabling employees to do the most incredible work of their careers ultimately result in their dismissal?” She concludes by challenging the audience to reflect on the purpose and ultimate destination of these technological advancements.