Safety and security for code executing agents — Fouad Matin, OpenAI (Codex, Agent Robustness)
Fouad Matin from OpenAI's Agent Robustness and Control team discusses the critical safety and security challenges of code-executing AI agents. He explores the shift from models that *can* execute code to defining what they *should* be allowed to do, presenting practical safeguards like sandboxing, network control, and human review, drawing from OpenAI's experience building Code Interpreter and the open-source Code Interpreter CLI.
Fouad Matin, from OpenAI’s Agent Robustness and Control team, outlines the security and safety considerations for increasingly capable code-executing AI agents. As frontier research labs push coding benchmarks, every AI agent is becoming a code-executing agent. This capability isn’t just for software engineering tasks but is a general method for achieving objectives more efficiently. The central challenge is shifting from what models can do to defining what they should be allowed to do and with what guardrails.
The Rise of the Reasoning Model and RCE
Modern AI agents are moving away from complex, chained inner loops of task-specific prompts. Instead, a single, powerful reasoning model can now decide which tools to use, including writing and executing code on its own. From a security perspective, this is a Remote Code Execution (RCE) scenario, introducing significant risks that must be managed.
Key Risks in Agent Deployment
Several potential failure modes for code-executing agents have been observed:
- Prompt Injection and Data Exfiltration: This is the highest priority risk. An agent might read untrusted content from a source like a GitHub issue, which contains a malicious prompt that causes it to leak sensitive data from its context.
- Agent Mistakes: The agent might unintentionally install a malicious package, write vulnerable code, or perform other erroneous actions.
- Privilege Escalation and Sandbox Escape: A compromised or misbehaving agent could attempt to break out of its constrained environment to gain broader system access.
A Framework for Safeguards
OpenAI’s preparedness framework requires safeguards to avoid misalignment, especially at scale. The following are the most critical, deterministic controls.
1. Sandbox the Agent
The most effective method is to provide the agent with its own completely isolated environment.
- Best Practice: In products like Code Interpreter in ChatGPT, a fully isolated container is spun up for the agent’s tasks. The final output is a pull request, which is a very safe approach.
- Local Execution: For local tools like the open-source Code Interpreter CLI, proper sandboxing is crucial. The CLI uses OS-level sandboxing techniques:
- macOS: It employs “Seatbelt,” a sandboxing language bundled into the OS (also used by Chromium). Documentation is sparse, so the team relied on model-assisted research to create a robust policy.
- Linux: It uses
seccompandlandlock(implemented in Rust) to create an unprivileged sandbox and prevent privilege escalation.
2. Disable or Limit Internet Access
Controlling network access is the most direct way to mitigate prompt injection and data exfiltration.
- Code Interpreter CLI offers a “full auto mode” where the agent is firewalled from the internet but can still read/write files within its working directory. This allows it to run tests (
pytest,npm test) without the risk of external communication. - Code Interpreter in ChatGPT recently introduced configurable internet access. This allows users to define an allowlist of domains and even specific HTTP methods, providing a balance between flexibility and security.
3. Require Human Review
Keeping a human in the loop is one of the strongest available mitigations.
- LLM-based code review tools are helpful but are not a substitute for human oversight. A human must review the operations an agent performs, checking for issues like installing a typo-squatted malicious package.
- The challenge is to balance security with usability—avoiding both a “YOLO mode” where everything is approved and a tedious process of approving every minor command.
- For non-coding agents, techniques like “watch mode” can be used, where a human monitor must approve any potentially sensitive operations the agent proposes.
Building Safer Agents with Better Tools
Developers can build safer agents by deferring logic to the reasoning model and giving it the right tools. OpenAI has released several tools to facilitate this:
local_shell: The exact tool used to train OpenAI models to write and execute code.apply_patch: A tool for applying file diffs that is more reliable than standard git diffs, as models often struggle with correct line numbers.- Composable Tools: Developers can chain tools to enhance security. For example, an agent could be given a tool that uses a service like Socket to check a dependency for vulnerabilities before it attempts to install it.
Ultimately, the strongest recommendation is to use a remote container for execution. OpenAI is releasing a container service as part of its Agents API to make this more accessible.