You Didn't Ship a Bug. You Just Wrote It for a Human. - Ravi Madabhushi, Scalekit

Ravi Madabhushi begins by highlighting a fundamental disconnect: current architectural paradigms, built for human interaction, are proving inadequate for AI agents. He illustrates this with an anecdote from Scalekit: a simple chat assistant demo, designed to connect agents to tools like Gmail and Calendar, caused production database strain every 15 minutes. The root cause was a “last seen” timestamp, intended for human login frequency, being updated “sixty times a second” by the agent. This “design assumption,” not a bug, revealed a critical flaw: infrastructure expects human-like behavior (one thing at a time, human speed), while agents operate at machine speed and scale. This is not an isolated incident; 60% of production LLM errors trace back to rate limits, indicating systemic infrastructure issues rather than model failures or bad prompts.

The Breakdown of Traditional Authentication and Authorization Paradigms

Madabhushi, drawing on his decade-plus experience in identity and authentication, explains how existing security models are ill-suited for agents:

• Human as Actor: In traditional systems (web apps, mobile apps, API keys), the user who authenticates is the same user who acts. Permissions are fixed at registration.
• Service Accounts: Machines have their own identities (e.g., OAuth, SPIFFE), with permissions defined at registration and acting deterministically based on pre-programmed logic. Human developers explicitly define what these programs can and cannot do, and code can be reviewed for security compliance.

How Agents Fundamentally Break These Assumptions:

1. Principal vs. Actor: Agents often need to act on behalf of a user (e.g., access user’s Gmail). Many current systems don’t fully support this “on behalf of” principal, making it hard to distinguish between a program acting for a user and the user acting directly.
2. Non-Deterministic Nature: Unlike deterministic human-written programs, agents are “probabilistic” and non-deterministic. An agent’s behavior today doesn’t guarantee its behavior tomorrow, making it impossible to predict actions solely from its initial programming. This lack of determinism means code review for security is insufficient.
3. Over-Permissioning: Developers often grant agents “way too many permissions and scopes” due to a lack of fine-grained primitives in existing systems. Agents might see all available tools or data, leading them to “pick the wrong tool, doing wrong things,” even if runtime checks prevent catastrophic outcomes.

Designing for Agents: A First-Class Principal Approach

To address these challenges, Madabhushi advocates for treating agents as first-class principals, not just fast humans. Key design principles include:

• Agent Identity: Agents must have their own distinct identities.
• Extremely Fine-Grained Permissions: Beyond broad OAuth scopes (e.g., “send emails on your behalf”), agents require:
  • Attribute-level scoping: Permissions tied to specific data attributes.
  • Context-level scoping: Permissions based on the specific operational context.
  • Principal-level scoping: Permissions tailored to the user the agent is acting for.
  • Temporal limitations: Permissions for “limited amount of time that they’re operating.”
  • Goal-oriented access: Access only to tools and jobs directly relevant to the agent’s current goal.
• Least Privilege by Default: Agents should start with minimal permissions and request “just-in-time authorization” for elevated scopes when necessary.
• Absolute Visibility and Deterministic Control: Organizations need comprehensive logging and auditing:
  • What actions were taken?
  • Who took them (which agent)?
  • On behalf of whom?
  • Who authorized it, when, and for how long?
  • The ability to deterministically control what an agent can or cannot do.

Urgency and The Path Forward

Madabhushi stresses that this isn’t a future problem but an immediate one, citing incidents where agents have performed “rogue things,” including deleting production databases. While OAuth is a good starting point, it’s insufficient for the granular control required. Architects must rethink systems designed with humans and traditional APIs in mind, focusing on building “deterministic guardrails” and fine-grained authorization controls for agents to ensure their safe and predictable operation.

• Tags:
• LLM Agents
• Authentication
• Identity Management
• Agent Security
• Agent Infrastructure
• Authorization
• Rate Limits
• Fine-Grained Access Control
• Non-Deterministic Systems
• API Design for AI