The Golden Age of AI Engineering — Alexander Embiricos & Romain Huet & Peter Steinberger, OpenAI

The speakers, Romain Huet and Alexander Embiricos, open by framing Dev Day 2024 as a “World’s Fair” for AI engineering, where the future is being built in public by “AI engineers.” They contend that the notion of engineers becoming obsolete is mistaken; instead, “AI engineers are eating the world” by leveraging new capabilities to solve problems, marking a return to the foundational principles of engineering: combining science, design, taste, judgment, and imagination.

Accelerating Model Capabilities and Development Cycles

Model development cycles have dramatically accelerated, shifting from new releases every 15 months to approximately every six weeks. This rapid pace has led to significant advancements in model capabilities:

• Evolution of build-and-test loops: Initially, models offered completions and inline predictions. This progressed to “command K” functionality (making changes without testing), then to models testing their own work, and now to models autonomously tackling long, complex goals until completion.
• Demonstrated progress: Romain Huet recounts his experience at Dev Day 2024, where he built a drone interface using a preview model (01) that couldn’t verify its own code, necessitating “crossing fingers.” A year later, by Dev Day of last year, models could reliably test their work, enabling control of an entire camera and lighting system live on stage.
• Codex building Codex: The speakers emphasize that Codex itself is built using Codex agents. These agents can now perform “any task that you can do on your own computer,” extending their utility beyond coding to include pre-coding tasks (understanding project rationale) and post-coding activities (review, deployment). This ability to connect agents to the entire workflow (“why it has to be done,” “what you do afterwards”) is key to accelerating work and making better product decisions.

Empowering Engineers: The Codex Product Philosophy

The core mission for Codex is not to automate engineers but to “maximally empower engineers.” This philosophy is shaped by two key questions: how agents can interact with the world, and what the product experience should feel like.

• Dual Modalities: Inspired by sci-fi, the product experience revolves around two primary interaction modes:
  • Chat: An “underrated” modality for everyday communication, delegation, and allowing the agent to work autonomously without constant oversight.
  • Collaborative UI: A powerful interface for deep inspection, steering, and shaping work when granular control or detailed collaboration is required, akin to working closely with a teammate on a workbench.
• The Codex App: This application embodies the philosophy, offering a simple chat interface for coding and other tasks, with the flexibility to “go deep” and make specific changes or direct the model. This design overcame initial skepticism from engineers accustomed to CLIs and IDEs, as it uniquely enables a collaborative, chat-first workflow that is difficult to achieve in traditional environments.

An Open Ecosystem: Building with Shared Primitives

OpenAI is committed to building Codex as a layered, open product, ensuring that “Codex cannot be a closed product that only OpenAI can improve.” Their internal development mirrors what they provide to developers:

• Responses API: OpenAI uses the same API for its internal Codex development that it provides to external developers. New functionalities required by Codex, such as compaction for long contexts, are integrated into the API first for broader developer benefit.
• Open-Source Harness and Agent MD: The Codex harness, which orchestrates agent operations, is open-source, allowing inspection, forking, and adaptation. Similarly, “Agent MD” is an open-source file format for instructions, designed for universal agent use. This allows developers to use open models with the same agent loop and enables internal models to learn tool calling and environment navigation within an open-source context.
• App Server: An open-source solution used internally for products like the VS Code extension and Codex app, it provides a unified way to control the harness. This allows community members to build their own native applications leveraging existing Codex subscriptions.
• Extensible Primitives and Plugins: The app layer includes extensible primitives like an in-app browser and plugins (e.g., browser use, computer use, role-specific tools for data science or design). These are also open-source, promoting community innovation.
• Unified Experience: The goal is to allow users to leverage their existing Codex subscriptions across a growing ecosystem of tools, from Open Code and Pi to Xcode and JetBrains, underscoring the commitment to an open and collaborative future for AI engineering.

Value Maxing: Cost Efficiency and Blazing Speed

The focus has shifted from “token maxing” to “value maxing,” optimizing for both intelligence and practical utility.

• Cost Efficiency: OpenAI is delivering “frontier intelligence” at increasingly lower costs.
  • GPT 5.6 Terra achieves GPT 5.5-level intelligence at half the cost.
  • Luna beats many notable models at just $1 per million input tokens and $6 per million output tokens, offering “insane value.”
• Blazing Speed: The latest models offer unprecedented inference speeds.
  • GPT 5.6 Sol on Cerebras achieves 750 tokens per second, enabling substantial pull requests to be written in approximately 10 seconds.
  • This speed transforms agent capabilities, allowing them to explore multiple approaches in parallel and select the best solution in the time it would have taken to generate just one. The experience becomes less about waiting and more like a “coworker that’s like already showing you the results as it goes.”

The Future of AI Engineering: Agent Orchestration and Attention

The speakers outline a future where the distinction between local and cloud tasks vanishes. An agent should intelligently determine the appropriate environment for its work and utilize available resources, regardless of location.

• Peter Steinberger’s Perspective: Peter Steinberger, introduced as the “claw father,” shares his journey from manually juggling “10 or more terminal windows” (which felt like orchestrating but was actually “polling” and acting as scheduler, router, and memory) to managing a “manager” agent that delegates work to a team.
• Enabling Changes for Scalable Agent Loops:
  1. Server-side compaction: Made long-running tasks reliable, eliminating the need to optimize for short sessions.
  2. Coordination: Allowed a single thread to create and steer projects effectively.
  3. Automation/Triggers: Enabled the manager agent to be woken up by external events, creating persistent context, delegation, and automated triggers.
• The New Bottleneck: Human Attention: With tokens and compute becoming less restrictive, the primary constraint shifts to human attention. Steinberger notes that watching agents generate code is increasingly a “waste of time” due to their advanced understanding of intent.
• Automated Workflow Example: He envisions an open-source project workflow where a manager agent reads an issue, creates a worker to investigate, implement, and test a solution. Another agent reviews the result. The manager then presents a pull request, diff, video, or even a running build to the human. The human’s role becomes reviewing once and approving, making decisions in the “outer loop” while the agent handles the “inner execution loop.”
• Untethered Agents: Managers should be long-lived and untied from local machines, able to connect to any device and operate seamlessly between local and cloud environments. The ultimate vision is to “talk to my agent and have it design the whole loop for me.”
• The Next Engineering Problem: The rapid advancement of models means the harnesses and organizational structures around them are lagging. Designing these “better loops” is the next critical engineering challenge for the AI engineering community.

• Tags:
• Developer Tools
• Open Source
• OpenAI
• AI Engineering
• Product Strategy
• Codex
• Human-AI Collaboration
• Agent Orchestration
• LLM Development
• Agent Systems
• Cost Efficiency
• Inference Speed
• Dev Day 2024
• GPT-5.6