What a $900M Bank Error Teaches Us About AI Agents - John Feminella, Two Sigma
John Feminella of Two Sigma uses a 2020 Citibank payment error to show how AI agents can be given real autonomy through five composable controls: an agent registry, risk-based review, purpose-scoped tool access, shared evals, and classifiers that send humans only the approvals that matter.
A 2020 payment error at Citibank, which sent roughly $900 million to Revlon’s lenders instead of an $8 million interest payment, shows how a process can fail even when every step is followed. John Feminella, who works on AI architecture and engineering at Two Sigma, uses the case to explain how organizations can give AI agents real autonomy without locking everything down. Citibank was the administrative agent on a Revlon term loan spread across more than 300 lenders. The plan was to pay accrued interest while the principal stayed put, and the principal was supposed to flow into an internal wash account. The boxes meant to route it were not checked, Flexcube displayed a warning that money was leaving the bank, and three approvers signed off anyway. Each control did what it was designed to do, and the failure happened in the gaps between them. The lenders, who had bought the debt at a steep discount, kept the windfall when Citi asked for it back, and commercial loan agreements now include clauses to prevent that outcome.
Why agents need a different model
Agents are non-deterministic, but so are people, and organizations already handle that by granting humans bounded autonomy wrapped in controls rather than isolating them. Feminella argues that safety and innovation are not opposite ends of a spectrum to be traded off. Three common approaches fall short. Sandboxes with no real system access are safe but not useful. Fully open agents are powerful but have weak isolation, which makes them unsuitable for regulated enterprises. Policy documents included in prompts are not enforceable at runtime, since nothing guarantees the agent follows them.
Five composable controls
The first control is an agent registry. Agents appear everywhere, from chatbots and notebook experiments to loosely assembled MCP server setups, and a firm cannot govern what it cannot see. The registry records each agent’s owner, product, tools, and endpoints, and it populates automatically when agents are built with approved frameworks. Gating every idea behind a heavy approval process only pushes people toward unmonitored shadow IT.
The second control is risk-based review. A summarizer for Jira tickets should not wait in the same queue as an agent that touches production systems. Each agent receives a risk score so that reviewers concentrate on high-stakes cases and low-risk experiments move quickly.
The third control is tool access that follows declared purpose. Most enterprise agents are purpose-built, unlike open-ended coding agents, so they can declare up front which tools and endpoints they need. A research summarizer might get web access while a release deployer does not. At initialization, the policy attached to the registry entry is translated into enforceable statements about permitted endpoints, hosts, and resources, so teams do not file tickets every time access changes. A blanket allowlist is safe but leaves agents unable to do their work, and blocked agents are either abandoned or find workarounds. As John Shedd wrote, “A ship in harbor is safe, but that is not what ships are for.”
The fourth control is shared evals. Evals function like unit tests for agents, and model changes act as runtime dependencies that can alter output quality. Rather than having each team build its own stack or send every prompt tweak through a committee, the firm provides a common evaluation approach. Teams contribute only what is specific to their agent.
The fifth control addresses approval fatigue. Asking a human to approve every action does not scale, and reviewers eventually click yes reflexively. An AI research lab director recently told her OpenClaw agent to suggest email deletions and wait for approval, but after the agent compacted its context it lost that instruction, deleted hundreds of emails, and kept going when she tried to stop it from her phone. The better design layers the decisions. Explicit rules act as deterministic fast paths. Conservative classifiers handle routine calls, and human review is reserved for ambiguous or potentially dangerous actions. Actions approved almost every time become rules, and clearly dangerous ones are denied outright.
How the pieces fit
The registry gives visibility from the moment an agent is created, risk scores route review to the right place, purpose-scoped guardrails limit each agent to what it needs, shared evals set a common quality bar, and runtime governance ensures classification happens where the agent actually runs. Feminella closes the thought with an analogy: cars have brakes not so they can stop, but so they can go fast and stop safely. Agent controls are meant to serve the same purpose.