Build Hour: Responses API

The Responses API is a new, agentic primitive designed to replace the Chat Completions API, enabling more complex, multi-step, and multimodal workflows required by models like GPT-5. It combines the simplicity of previous APIs with the power needed for building sophisticated agents.

The Evolution from Completions to Responses

The journey of OpenAI’s core API reflects the evolution of model capabilities:

1. v1/completions (2020): Built for models like GPT-3 that were essentially “thought finishers,” continuing a prompt from where it left off.
2. v1/chat/completions (2022): Launched with ChatGPT and GPT-3.5 Turbo, this API introduced a conversational format with a sequence of messages. Features like tool calling and vision were later added, but the core design was limited.
3. Responses API (2024): Created for agentic and highly multimodal models like GPT-5. It is designed to handle everything from simple text-in/text-out requests to long-running, agentic rollouts that can last for minutes.

Core Advantages of the Responses API

The Responses API introduces several fundamental improvements over Chat Completions, addressing common developer pain points and unlocking new capabilities.

1. The Agentic Loop

At its core, the Responses API is an “agentic loop,” meaning it can sample from the model multiple times within a single API request. This allows the model to perform a sequence of actions before returning a final answer.

• Example: A user asks for the square root of a large number. The model can first use the Code Interpreter tool to write and execute Python code, see the output, and then sample again to formulate the final textual answer based on the code’s result. This entire sequence happens in one logical turn.

2. “Items In, Items Out” Paradigm

The API moves away from a message-centric design to a more flexible “item” structure. An item is a union of types representing things the model can do or say.

• Chat Completions: Everything was a message, and concepts like function calls were bolted on, making the code complex to handle.
• Responses API: message, function_call, mcp_call, and reasoning are all distinct types of item. This makes parsing the output much cleaner, often with a simple for loop and a switch statement based on the item type.

3. Persistent Reasoning

The API is purpose-built for reasoning models, allowing the chain of thought to be preserved across multiple API requests.

• When the model thinks, it emits a reasoning item. By passing this item back in the next request, the API can “rehydrate” the model’s previous thought process.
• This works both statefully (by default) and statelessly (by passing back encrypted_content).
• Benefit: This persistence boosts tool-calling performance significantly. On the toolbench evaluation, it results in a 5% performance increase compared to Chat Completions.

4. Superior Performance and Cost-Efficiency

For long, multi-turn tool-use rollouts, the Responses API is 20% faster at P50 and less expensive.

• Why? In Chat Completions, the model must “think again” at every step because the chain of thought is lost between requests. This generates more tokens and leads to worse cache hit rates.
• In the Responses API, the rehydrated reasoning allows the model to pick up its plan where it left off, avoiding redundant thinking, saving tokens, and improving latency.

5. Native Multimodal Workflows

The design simplifies working with images and other content types. You can easily pass Base64 encoded images, external URLs, or even upload files like PDFs for “context stuffing.” For example, you can upload a utility bill PDF and ask the model to analyze why it was so high.

6. Reimagined Streaming

Streaming has been redesigned from the ground up to be more developer-friendly.

• Chat Completions: Emitted “object deltas,” forcing developers to accumulate every event to reconstruct the final state.
• Responses API: Emits a finite number of strongly-typed events (e.g., output.text.delta, response.reasoning_summary.part.delta, web_search.call.searching). This allows developers to listen for specific events with a clean switch statement without needing to process every single one.

Live Demo: Building an “OpenAI Simulator”

A game simulating “a day in the life of an OpenAI engineer” was used to demonstrate key features.

• Reasoning Summaries: To improve the user experience while waiting for GPT-5 to think, the reasoning block was enabled with summary: "auto". This streams a summary of the model’s internal monologue (e.g., “Thinking about the critical path priorities…”) to the UI before the final answer is ready.

• MCP (Multi-tool Calling Protocol): The agent “Sam Altman” was given access to a Linear board via an MCP tool. This allowed the agent to perform real-world actions.

  1. The API first lists the available tools from the MCP server (linear.mcp.server.list_issues).
  2. The user approves the tool call.
  3. The agent executes the tool (linear.mcp.server.create_issue) with the necessary arguments.
  4. The agent receives the result and summarizes its action to the user (e.g., “Great, I’ve created issue AGI-32…”).

• Multi-Tool Rollouts: The agent “Wendy” demonstrated a multi-step, multi-tool workflow in a single turn.

  1. User Prompt: “I’ve never seen a French bulldog before. Can you search the web to find out what they look like and then draw me a picture of it?”
  2. Web Search: The agent autonomously used the web_search tool multiple times to gather information (breed standard, appearance, colors). The UI showed a “Searching the web…” status.
  3. Image Generation: After gathering sufficient information, the agent used the image_generation tool to create a picture of a French bulldog based on its findings.
  4. Final Output: The final response included both the generated image and a textual summary.

Migrating to the Responses API

To simplify the transition, OpenAI provides a migration pack that uses the Codex CLI for agentic coding. By running a single bash command, the pack can automatically refactor a codebase from using the Chat Completions API to the Responses API, handling changes in method calls, data structures (messages to input_items), and streaming logic.

Q&A Highlights

• Handling Hallucinations: Use few-shot prompting with varied, canonical examples. For factual hallucinations, provide the model with tools like web_search to ground its responses in real data.
• Managing Conversation History: There are three primary ways to pass context:
  1. Stateless: Pass the entire list of items in each request, similar to Chat Completions.
  2. Pointer-based: Use previous_response_id to chain requests without managing the full context yourself.
  3. Stateful: Use the Conversations API (the successor to Assistants’ Threads), where you create a conversation object and only pass incremental items in each call.
• Maximizing Prompt Caching: The cache works on a prefix match of tokens. To get the most benefit, treat your context as an append-only list. Modifying earlier parts of the conversation will break the cache for subsequent tokens.
• Common Mistakes:
  • For stateless/ZDR users, failing to request and pass back the encrypted_content, thereby losing the benefit of reasoning rehydration.
  • Re-implementing functionality that is already available and highly optimized in built-in tools (e.g., building a custom RAG pipeline instead of using file_search).
  • Not utilizing powerful platform objects like Prompts to define and version reusable tasks.

• Tags:
• Agentic AI
• Feature
• MCP
• OpenAI
• GPT-5
• Tool Calling
• Responses API
• Multimodal
• API Migration