Loop Engineering from First Principles — Kyle Mistele, HumanLayer

Kyle Mistele argues against the prevailing narrative that the solution to complex coding problems with AI agents lies in better prompts. Instead, he advocates for adopting a control theory-inspired approach: building better loops that enable agents to make incremental, verifiable changes, effectively addressing the issue of unmanageable, 40,000-line pull requests (PRs) that no human can adequately review. He criticizes the current trend of generating vast amounts of “read-only code” that is expensive, difficult to verify, and prone to stability issues, especially for critical systems.

The Principles of Control Theory Applied to Code

Mistele draws a parallel to control theory, which is about driving a dynamic system towards a desired stable state. A control loop consists of:

• Sensor: Measures the current state of the system.
• Set Point: The desired end state.
• Measured Error: The difference between the current and desired states.
• Controller: Reads the error and generates an incremental control signal.
• Actuator: Applies the incremental change to the system.
• Disturbances: External factors affecting the system.
• Feedback: Re-measuring the state after the change.

He illustrates this with everyday examples like thermostats and complex systems like Kubernetes autoscaling, Infrastructure as Code, and React’s virtual DOM. The core benefit is making incremental changes to avoid destabilizing the system, minimize risk, and allow for human review, contrasting this with “blind Ralph loops” that produce large, unmanageable diffs.

Designing Agentic Control Loops for Codebases

To build effective agentic control loops, Mistele outlines the practical implementation:

1. Define a Set Point: This is the desired end state for a specific property of the codebase (e.g., all RPC APIs use effect).

2. Implement a Sensor: This component identifies deviations from the set point. Options include:

   • Deterministic tools: ESLint rules, ASGrep, RipGrep.
   • Non-deterministic agents: Agents with natural language rules.
   • Hybrid approaches: Combining both, like Aiden’s React Doctor which identifies problems and suggests fixes.
   • ASGrep is highlighted for its power, language agnosticism, and ability to operate out-of-band from typical linters, allowing for granular rule sets and deterministic violation sorting.

3. Design the Controller: The controller decides what incremental change to make. Strategies include:

   • Simple deterministic picking: Selecting the first violation from a sorted list.
   • Risk reduction: Using ASGrep to find and prioritize the smallest unmigrated procedure.
   • Agent-driven decisions: An agent picks the procedure to migrate.
   • Telemetry-driven intelligence: Utilizing telemetry data (e.g., procedures with most errors, least instrumentation) to prioritize migrations that address actual pain points, allowing the agent to not just migrate but also improve the code.

4. Build the Actuator: This is the agent that applies the changes. It consists of an agent combined with a “skill.” Mistele emphasizes the importance of providing “golden patterns”—idiomatic, hand-written examples—to guide the agent, as agents are primarily pattern replicators and might otherwise only provide generic or documented solutions.

Practical Example: RPC API Migration to ‘Effect’

Mistele shares an internal use case at Human Layer: incrementally migrating their RPC API to effect. He details the loop:

• Sensor: An ASGrep rule finds unmigrated procedures. The violations are deterministically sorted and filtered.
• Disturbance Dampener: Before incremental migrations begin, a full scan on main branch tracks existing violations in version control. On new PRs, a check ensures no new unmigrated procedures are introduced, preventing teammates from undoing the loop’s work.
• Controller: Picks the next procedure to migrate, potentially using telemetry to prioritize problematic areas.
• Actuator: An agent applies the change based on the controller’s signal and golden patterns, then deterministically commits and creates a PR with the final message as the description.

Wiring the Loop into CI/CD and Human Feedback

• Execution: The loop is run via GitHub Actions (or similar CI/CD platforms) daily. Each run performs a single “sense, control, actuate” cycle, creating a small, low-risk PR for review.
• Human in the Loop: To address initial friction and allow human re-steering, a feedback file (Markdown) is tracked in version control. When a user adds a /iterate comment to a PR, the loop workflow loads the PR context (diffs, comments, reviews, feedback file) into the agent, instructs it to fix the code, and update the feedback file. This makes the feedback process auditable and revertible.

Adaptive Flow Control and Scaling

• Avoiding Stacked PRs: A crucial control mechanism is implemented where the loop checks if any PR with its label is currently open. If so, it shuts down, ensuring at most one PR per loop is open at a time. This prevents overwhelming reviewers with duplicate or conflicting work if humans fall behind on reviews.
• Scaling Velocity: To speed up migrations, the controller can be configured to pick multiple procedures (e.g., three or five), or the workflow can be run multiple times to distribute work across different team members or create separate implementation phases, each with its own context window for reliability and cost-effectiveness.

This approach results in a control loop that incrementally improves code, allows for human review, includes adaptive flow control, and provides a low-friction way to steer the agent’s work.

• Tags:
• AI Agents
• Software Engineering
• Developer Productivity
• Feature
• Human-in-the-Loop
• Continuous Integration
• LLM in Software Development
• Control Theory
• Incremental Code Changes
• Codebase Migration
• ASGrep