Develop at Idea Velocity - Jeffrey Lee-Chan, Snapchat
Jeffrey Lee-Chan argues that the critical component for production AI agent systems is not the model but the "harness"—an orchestration layer providing persistent memory and contextual guardrails. He introduces Open Claw, a conceptual framework that separates agent orchestrator managers (handling task specs and history) from specialized workers (executing implementation details). This architecture enables deterministic teamwork, parallelization, and unbiased outcomes, supported by examples like AI RPGs and multi-AI analysis, alongside tools like tmux for enhanced developer productivity and careful token management strategies.
Jeffrey Lee-Chan addresses a critical gap in production AI agent systems: not the models themselves, but the “harness”—the surrounding loop that provides persistent memory and contextual guardrails. He emphasizes that while agents might complete complex tasks on a first run, they often fail subsequently due to this missing harness.
The Core Problem: The Missing Harness
Lee-Chan’s experience, encompassing “1,000 hours of orchestrating autonomous fleets under human direction,” reveals a clear pattern: the absence of persistent memory and contextual guardrails in the surrounding system leads to agent failures in subsequent iterations. This highlights that the orchestration layer, or “harness,” is paramount for reliable, repeatable agent performance, not just the underlying large language model.
Open Claw: A Conceptual Framework for Agent Orchestration
Open Claw is presented less as a specific repository and more as a conceptual framework designed to turn raw models into deterministic teammates. Its core tenets include:
- Frictionless Communication: Enabling easy, natural language interaction with AI agents, moving beyond cumbersome interfaces. This facilitates a “higher level of velocity” in development.
- Specialization and Context Management: A key distinction is made between the “Agent Orchestrator Managers” (Open Claw) and specialized “workers.” Open Claw focuses on the spec, goals, and history of a task, acting as the high-level manager. This prevents the agent’s context window from being cluttered by implementation details. In contrast, workers (e.g., running Claude Code) focus on how to execute the task.
Lee-Chan argues that using Open Claw for specialization ensures that the orchestrator’s context is dedicated to the overall task, rather than being 25% consumed by “how-to-do” instructions, as might happen with direct model interaction.
Architectural Blueprint for Deterministic Teammates
Open Claw’s architecture involves:
- Agent Orchestrator Managers: These are the Open Claw layer, responsible for managing workers, maintaining task context, memory, and goals, and interpreting brief user instructions (e.g., a Slack message like “fix the skeptic agent”).
- Specialized Workers: These are often powered by frameworks like a forked agent orchestrator, which in turn run tools like Claude Code. Claude Code itself can manage sub-agents, creating a hierarchical agent structure. This separation prevents “low-level implementation bias” at the orchestration level.
This setup allows for parallelization through the use of multiple agents with work trees and integrates with CI/CD for continuous code quality.
Practical Examples and Applications
Lee-Chan demonstrates the power of this approach with several production systems:
- Skeptic Agent: A custom code review agent that leverages Open Claw’s contextual memory to understand brief commands like “fix the skeptic agent” without needing full re-explanation.
- AI RPG (Role-Playing Game): An application that builds a custom, reactive world. Unlike direct LLM interactions that might lead to “winning too much,” this system incorporates a D&D-like dice roll mechanic, introducing elements of chance and challenge based on player actions.
- Multi-AI Analysis Website: Designed to overcome the bias of single-model answers. Users can pose a query, and the website aggregates responses from multiple models. It features vertical tabs for managing information and notifications for task completion, acting as a “manager” for AI outputs. This system inherently provides a less biased analysis because the orchestrator (user or another agent) can compare and contrast diverse AI outputs, rather than accepting a single model’s potentially skewed response.
Key Tools and Considerations
- CMUX/tmux: A terminal program highly valued for parallelization in AI development. Lee-Chan praises its efficiency for managing multiple terminal sessions and notifications, emphasizing its utility as a “manager” interface rather than a coder’s direct environment. Austin, one of tmux’s creators, further highlights its capabilities, including Cloud Code Teams integration and Cmux SSH for remote work.
- Token Management and Cost: Lee-Chan discusses the trade-offs between leading models:
- He primarily uses Codex 53 as a default.
- GPT-4 is noted for its higher token usage and associated costs.
- MiniMax serves as a fallback option when token budgets for higher-quality models are strained, accepting a trade-off in quality for cost efficiency.
- Sandboxing for Reliability: For development and testing, sandboxing (e.g., running in Docker or a staging environment) is crucial. While a staging environment might increase token usage if the same work is sent to multiple instances, the strategy is to develop locally, run integration tests on the sandbox/staging, and then deploy to production. This approach enhances reliability and allows for experimentation without impacting live systems.
Conclusion
By dissecting his multi-agent architecture, Lee-Chan provides a “practical architectural checklist” for building robust, production-ready AI agent systems. The central message is clear: focus on the harness—the orchestrator, context management, and parallelization—to achieve deterministic, reliable, and unbiased AI agent performance, pushing human interaction to the beginning or end of the process to enable “idea velocity.”