Automating Large Scale Refactors with Parallel Agents - Robert Brennan, AllHands

The nature of software development is undergoing a fundamental shift, moving beyond simple code completion to a new paradigm of agent-driven engineering. While today’s agents excel at small, atomic tasks, their true potential is unlocked in tackling massive, multi-file endeavors like major refactors, dependency updates, and framework migrations. These tasks, often too large for a single agent to “one-shot,” are highly amenable to automation through the orchestration of parallel agents with a human in the loop.

The Evolution of AI in Software Development

The journey to agent orchestration has progressed through several distinct stages:

1. Context-Unaware Snippets: Early models could generate isolated code blocks in a chat window but lacked any awareness of the broader project.
2. Context-Aware Generation: Tools like GitHub Copilot brought AI into the IDE, enabling it to generate relevant code by understanding the local context (variables, file content, etc.).
3. Autonomous Agents: The emergence of agents like Devin and Open Hands marked a significant leap. These agents can not only write code but also execute it, read error messages, search for solutions, and debug—automating the entire inner loop of development.
4. Parallel Agents & Orchestration: This is the current cutting edge, where multiple agents work in parallel, sometimes communicating with each other, to accomplish a large-scale goal. This approach is essential for automating the mountains of tech debt and maintenance work that burden modern software companies.

Why Large-Scale Tasks Require Orchestration

Attempting to solve a massive refactoring task with a single agent often fails due to several technological and human factors:

• Technological Hurdles:

  • Limited Context: Large codebases exceed the context windows of even the most advanced models.
  • Agent “Laziness”: Agents may complete a small portion of a repetitive task and then declare the rest requires a human team.
  • Lack of Domain Knowledge: Agents don’t possess the intuitive understanding of a codebase that a human developer does.
  • Compounding Errors: A small mistake made early in a long-running task can be repeated and amplified, leading to a flawed final result.

• Human Hurdles:

  • Difficulty Conveying Intuition: It’s challenging to translate a human’s mental model for a refactor into a prompt an agent can execute flawlessly.
  • Need for Intermediate Review: Large tasks cannot be a “fire and forget” process. Human oversight and validation at intermediate steps are crucial.

The goal of orchestration is not 100% automation but rather 90% automation, providing an order-of-magnitude productivity lift while keeping the human developer in a strategic, reviewing role.

A Workflow for Orchestrating Agent Swarms

A successful orchestrated task follows a more complex loop than a single-agent interaction:

1. Decompose: The human, perhaps aided by an AI, breaks the large problem into a series of smaller, independent tasks that a single agent can reliably complete.
2. Dispatch: An agent is assigned to each sub-task, often running in parallel in sandboxed cloud environments for security and scalability.
3. Execute & Review: Each agent performs its task. The human reviews the intermediate outputs (e.g., individual pull requests) to ensure correctness before they are merged into a main integration branch.
4. Collate: Finally, the results from all agents are combined into a single, cohesive change.

This process is often managed using a dedicated Git branch. Scaffolding code might be added to this branch to allow the old and new systems to coexist during the migration, enabling verification at each step.

Core Strategies for Orchestration

Two key strategic areas are crucial for success: task decomposition and context sharing.

Task Decomposition Strategies

• Piece-by-Piece: Iterate through every file, directory, or function. This is straightforward and works well for tasks like adding type annotations.
• Dependency Tree: Analyze the codebase’s dependency graph and start with the “leaf” nodes (files with no dependencies). Work your way up the tree, ensuring that when an agent works on a file, its dependencies have already been migrated.
• Scaffolding: Create temporary code that allows both the pre-migration and post-migration code to run simultaneously. For example, when migrating from Redux to Zustand, a scaffold can enable some components to use Redux while others use Zustand, allowing for incremental migration and testing.

Context Sharing Between Agents

When multiple agents are working in parallel, sharing learned information is vital to prevent them from repeatedly solving the same problem.

• Manual Sharing: A human can manually copy-paste relevant information or solutions between agent chat windows. This is effective but not scalable.
• Shared File: Agents can be given a tool to read from and write to a shared markdown file (e.g., agent.md). This acts as a persistent, shared “memory” for the swarm, though it may require human review to prevent agents from adding unimportant details.
• Agent-to-Agent Communication: The most advanced method involves giving agents the ability to send messages directly to each other. This is powerful but introduces non-determinism and the risk of agents getting stuck in unproductive loops.

Case Study: A Verifier-Fixer Pipeline

A concrete example of orchestration is a system built with the Open Hands SDK to eliminate code smells from a large codebase.

1. Visualize & Batch: The codebase was first broken down into manageable batches, often based on the directory structure. A dependency graph of these batches was created to determine the correct processing order.
2. The Verifier: An LLM-based “Verifier” was run on each batch. It programmatically checked for code smells and marked the batch as “completed” (green) if none were found or “failed” (red) if issues were present.
3. The Fixer: When a batch failed verification, a “Fixer” was triggered. This component used the SDK to spin up a new, dedicated agent with the sole task of fixing the specific code smells identified by the Verifier in that batch.
4. Human Review: The agent’s work resulted in a small, focused pull request, complete with a summary of the changes, ready for human approval.

By repeating this Verify -> Fix -> Review cycle, the entire codebase could be systematically refactored, with the batch-level dependency graph providing a clear visual of the project’s progress.

• Tags:
• AI Agents
• Software Engineering
• Code Refactoring
• Agent Orchestration
• Open Hands
• Tech Debt