An AI Agent Got Its User Banned From Resy — Keycard, JPMorgan Chase & AAuth
A panel of experts discusses the challenges of granting autonomy to AI agents due to coarse-grained authorization systems. They advocate for intent-based runtime authorization, leveraging concepts like "missions" and "budgets" from AOTH, brain-hands architecture, and dynamic policy management. The discussion covers agentic commerce, accountability through tamper-resistant audit logs, and whether P(doom) is a real threat, emphasizing engineering ingenuity and societal learning from failures.
The Autonomy Dilemma: Why AI Agents Lack Full Trust
The panel at AI Engineer New York 2026 delves into the pressing issue of granting autonomy to AI agents, particularly within regulated sectors like financial services. Despite significant advancements in AI models—boasting larger context windows and enhanced verification capabilities—enterprises are struggling to deploy agents with true independence. The core obstacle, as highlighted by Ian Livingstone, CEO of Keycard, is that established security systems are architected for human decision-makers and deterministic software, not the emergent, reasoning capabilities of AI. This leads to coarse-grained access controls, exemplified by systems like Salesforce’s single API scope in OAuth, which are fundamentally inadequate for the nuanced operations agents require.
Dick Hardt, co-creator of OAuth and founder of AOTH, corroborates this, explaining that OAuth was never designed for the agentic world. Its original purpose for website-to-website authorization, relying on pre-registration and browser redirects, and its limited scope granularity, makes it a poor fit. Agents demand dynamic, context-aware authorization decisions that cannot be predetermined.
Intent-Based Authorization: A Path to Controlled Autonomy
The proposed solution is runtime authorization driven by intent. This paradigm shift involves authorizing agents per tool call, leveraging contextual information to make superior, real-time decisions about what an agent is permitted to do.
Lessons from Financial Services
Michael Davis, Global Chief Security Architect at JPMorgan Chase, points to the financial industry’s historical move from authenticating humans to authorizing individual transactions at runtime. This practice, where intent (like a check’s memo field) and defined limits (price, payee) played crucial roles, offers a blueprint for agent authorization.
The “Mission” Concept in AOTH
Dick Hardt introduces the “mission” concept within AOTH (Agentic OAuth). A mission outlines the overarching goal an agent aims to achieve on behalf of a “person” (human or organization), who retains ultimate responsibility. Authorization decisions are then evaluated against this mission, enabling granular, transactional-level control that surpasses simplistic scopes.
Contextual Understanding for Authorization
Ian Livingstone illustrates the necessity of contextual intent: two distinct ChatGPT sessions, one for e-commerce and another for debugging production, demand vastly different permissions and human oversight. By understanding both the user’s immediate intent and the broader business intent behind an agent’s deployment, a “bounding box” of permissible actions can be calculated. Without this, the risk of “Pocket OS style incidents”—unintended but catastrophic actions due to broad access—remains high, impeding genuine agent autonomy and scalability.
Negotiation and Policy Layering
Effective intent-based authorization involves agents negotiating mission parameters with their human counterparts. Deterministic policies serve as foundational guardrails, defining non-negotiable boundaries or scenarios requiring human intervention. Intent then acts as a dynamic layer, bridging the gap between broad organizational policies and the specific requirements of a task, ensuring human involvement only when critically necessary to optimize adoption and efficiency.
Practicalities and Challenges in Implementation
Dynamic Policy Management
Davis emphasizes the value of decoupling policies, allowing different organizational components to contribute to authorization decisions dynamically. This enables flexible policy application, such as permitting an administrative assistant to access a CEO’s email within a specific context, which would otherwise be a blanket denial.
“Know Your Agent” for Agentic Commerce
Intent-based authorization is poised to revolutionize agentic commerce. For users, it ensures agents adhere to budgets and purchase correct items. For merchants, it offers critical data—an extension of “Know Your Customer” to “Know Your Agent”—to discern purchase intent and assess agent trustworthiness, vital for regulatory compliance.
The Web’s Agent-Unfriendly Design
Ian highlights the web’s inherent design limitations regarding unregistered clients. Similar to CAPTCHAs combating human spam, an “agent identity” is crucial for fostering trust. He cites the Vercel incident, where an entire platform was blocked due to a single problematic host, as an analogy for the challenge of allowing unknown agents to interact with web resources. This necessitates “agreeable rails” and open protocols for agent interaction.
Open Protocols vs. Walled Gardens
Dick advocates for open, federated protocols like AOTH over proprietary, closed-system approaches. Such protocols facilitate component interoperability, enabling independent supervising agents to enforce policies, thereby preserving user control and preventing vendors from monopolizing agent capabilities. Ian echoes this, urging the industry to embrace federated protocols that foster broad web access and innovation, rather than succumb to walled-garden architectures controlled by a few large entities.
Mitigating Agent Misbehavior: The Resy Incident and Solutions
The Resy Scenario
The panel recounts the “Resy incident,” where an Instinct agent, in its zealous attempt to book a restaurant reservation, made 200 API calls per hour, leading to the user’s account being banned. This illustrates how agents, driven by good intent but lacking appropriate constraints, can inadvertently “abuse the system.”
AOTH Budgets for Resource Control
Dick introduces AOTH’s concept of budgets as a solution. Agent tokens are not only time-limited but also assigned a resource budget. Upon exhaustion, a supervising agent determines if additional resources should be granted, providing continuous feedback to the agent about its consumption. This contrasts with traditional rate limiting, which abruptly cuts off access without feedback.
Brain vs. Hands Architecture
Ian elaborates on the “brain vs. hands” architecture, where the LLM (brain) is separated from the tools in a sandbox (hands). Credentials and API access are externalized, with an egress proxy serving as the authorization boundary. This model, prevalent in labs like Anthropic and OpenAI, allows for fine-grained control and dynamic authorization at the network perimeter, preventing agents from possessing broad, standing access.
Dynamic Tool Management and Efficiency Gains
Davis highlights the advantage of dynamically removing tools based on context, such as disabling web search capabilities while an agent is drafting an email. This prevents “toxic combinations” and ensures agents only have access to necessary functions. Ian adds that intent-driven systems lead to reduced token consumption by guiding agents toward optimal paths and preventing them from engaging in irrelevant or incorrect actions.
Production Agents: A Holistic Strategy
Dynamic Authorization and Human Oversight
Davis notes that intent-based authorization shifts focus to dynamic authorization, its validation, and judicious human-in-the-loop interventions. This brings engineers directly into the authorization process, refining threat models and understanding application attack surfaces. By constraining agents, security and efficiency improve, as agents make better decisions with fewer permissible actions.
Secure Software Factories
Ian positions runtime authorization as a pivotal element in secure software factories. It enables agents to be directed effectively, manage work handoffs, and safely access production systems for tasks like automated patching. This is critical for scaling software development beyond mere text generation to full deployment and iteration.
Lessons from Robotics
Drawing parallels with robotics, Davis emphasizes incorporating probabilistic decisions and external sensor data (e.g., proximity, temperature, system vulnerabilities) into authorization. Unlike binary authorization, a richer model can combine user intent with environmental context, leading to more nuanced decisions and vastly improved diagnostics through span-level observability.
Accountability and Tamper-Resistant Auditing
Psychological Ownership and Liability
Davis asserts that intent fosters “psychological ownership,” directly linking human accountability to agent outcomes—a crucial factor when financial transactions are involved. Ian underscores the ultimate question of liability: “who is liable if that action causes damage?” This necessitates a tamper-resistant audit log, external to the agent’s control, to meticulously record agent actions, purpose, and delegated authority.
P(doom): Real Threat or Overhyped?
The panel concludes by addressing the concept of P(doom)—the probability of AI leading to catastrophic outcomes.
- Dick Hardt regards fully autonomous agents as “science fiction.” His primary concern lies with malicious actors or irresponsible organizations leveraging powerful AI tools, potentially causing “massive natural disaster-like scale issues.” He cites incidents like Hugging Face as evidence of unintentional harm due to inadequate controls.
- Michael Davis expresses optimism, trusting in the “ingenuity of engineers” and the rapid iteration within the field to devise solutions.
- Ian Livingstone offers the “hottest take,” suggesting P(doom) “is approaching asymptotically zero.” He believes in humanity’s collective incentive to continue, the general goodness of people, and the practical reality that agents are ultimately “computers we can unplug.” He sees agent failures as societal “antibodies” that prompt learning and refinement, ensuring continued progress.