AgentCraft: Putting the Orc in Orchestration — Ido Salomon
As the number of AI agents grows, humans become the bottleneck. This talk introduces AgentCraft, a game-inspired orchestrator that draws lessons from Real-Time Strategy (RTS) games to enhance human-agent collaboration, improve visibility, and increase agent autonomy, ultimately raising the ceiling of what's possible.
Scaling from a single, effective AI agent to a multitude of them introduces a significant challenge: the human operator becomes the bottleneck. The skills required to manage dozens of autonomous agents aren’t typical for software engineers. However, these skills aren’t entirely new; they can be found in unexpected places, such as Real-Time Strategy (RTS) video games. AgentCraft is an orchestrator built on this premise, aiming to “raise the ceiling of human-agent collaboration” by translating concepts from gaming into a productive development environment.
Core Concept: The Gaming Metaphor
AgentCraft treats AI agents as units on a map, providing a familiar interface for anyone who has played an RTS game. This approach is designed to leverage existing cognitive skills for managing multiple entities simultaneously.
- Physical Representation: Each agent is a physical manifestation on the screen, representing a live coding session (e.g., Cursor, Open Interpreter). Agents can be spawned directly from the interface.
- The Map as a File System: The operational map is a direct, visual projection of your local file system. Directories are territories, and files are rooms within buildings. This allows you to visually track where each agent is working.
Enhancing Visibility and Control
A key to managing multiple agents is having a clear, immediate understanding of their activities. AgentCraft provides several layers of visibility.
- Real-Time Monitoring: A side panel offers a high-level mission status for each agent. More importantly, you can see agents moving across the file system map as they access and modify different files.
- Complete Lineage: The system tracks the entire change list for each file, providing a full history of which agent performed what action and when.
- Collision Prevention: By tracking all agent activities, AgentCraft can generate a heatmap to visualize potential conflicts or “collisions” where multiple agents might be working on the same files. This allows for proactive intervention.
- Rapid Cycling: Borrowing another mechanic from RTS games, you can use muscle memory to quickly cycle between agents that require human input, such as plan approvals or clarifying questions.
Reducing Cognitive Load and Agent Babysitting
While visibility is crucial, it’s only the first step. To truly scale, the human must be removed from the low-level loop of constant supervision.
- Agent-Initiated Quests: To combat the limit of how many ideas a human can manage at once, agents can be tasked with finding their own work. They can be assigned “quests” to perform routine but necessary tasks like refactoring code or writing tests.
- The “Campaign” Feature: This allows you to define a high-level objective and spin up a containerized environment for agents to operate in. The agents themselves are responsible for decomposing the task, planning the execution, and carrying it out. The human’s role shifts from constant “babysitting” to reviewing the initial plan and the final outcome. The campaign orchestrator, not the human, manages the agents.
- Automated Idea Generation: This concept can be extended further. For instance, an agent could be configured with a cron job to scan Twitter for interesting project ideas, implement them, and submit them for review.
Streamlining the Review Process
As agents gain autonomy, the human’s primary role shifts from execution to review.
- Review Bundles: Instead of just seeing a pull request, you get a comprehensive “review bundle”. This includes the list of tasks, the rationale behind the changes, and visual evidence like screenshots and videos of the feature in action.
- Shifting Effort to Review: With a powerful review system, you can spend less time on meticulous upfront planning. It becomes more efficient to let agents attempt a task multiple times with different approaches and then simply choose the best outcome.
Fostering Human-Agent and Human-Human Collaboration
AgentCraft is built for a collaborative environment where humans and agents work together seamlessly.
- Shared Workspaces: Different team members, like a product designer and an engineer, can work in a shared space. You can see each other’s agents, understand their tasks, and hand off work between human and agent, or vice-versa.
- Soft Collaboration Mechanisms: A shared chat log is accessible to both humans and agents. An agent can post a message like, “I’m starting to work on the authentication service,” and a human can reply, “I’m also working on it.” The next time an agent plans a task, it is aware of the human’s activity and can avoid conflicts.