Agents Need Feature Flags - Sachin Gupta

AI teams are currently deploying highly behavior-changing systems—agents that can send money, modify databases, or spawn child processes—without the foundational safety infrastructure that traditional web teams established over a decade ago. This oversight leads to significant, costly, and embarrassing incidents, as highlighted by examples like the Cursor SAM chatbot hallucinating policies, Replit deleting production databases, LangChain pipelines incurring $47,000 costs due to runaway loops, and Pocket OS agents misusing API tokens for destructive database operations.

The Problem: All-or-Nothing Deployments

Unlike modern web development, where canary releases, segment targeting, kill switches, and comprehensive rollout monitoring are standard, AI agent deployments often lack these safeguards. Every prompt change, tool addition, model swap, or memory policy update is shipped globally to 100% of users, instantly, with no rollback mechanism. This leaves teams discovering critical failures via social media, rather than through controlled testing and monitoring.

Web Engineering Lessons Unapplied

Web engineers learned the necessity of these safety mechanisms years ago:

• Canary Releases: Gradually rolling out changes to a small percentage of users, monitoring metrics, and expanding or rolling back based on performance.
• Segment Targeting: Delivering different behaviors to different user groups.
• Kill Switches: Pre-wired toggles to instantly disable features without new deployments.
• Rollout Monitoring: Dedicated error rate dashboards for every change.

While tools like LaunchDarkly, Unleash, and Split exist, the discipline of applying them to AI agents is often missing.

Six Unique Behavior Surfaces of AI Agents

Unlike traditional CRUD applications, AI agents present six distinct “behavior surfaces” that require specific types of flags:

1. Prompts: The system prompt is the most behavior-altering code, changing frequently and often outside normal deploy processes.
2. Tools: Every tool an agent can call represents a new authorized action, with tools appearing and disappearing faster than traditional features.
3. Models: Swapping models can subtly change personality, refusal patterns, latency, and cost.
4. Memory: What an agent remembers across sessions silently alters behavior over time, leading to inconsistent outputs.
5. Autonomy: The level of agent autonomy (suggest vs. auto-approve vs. auto-execute) represents the single largest blast radius.
6. Sub-agents: Spawned child agents must inherit parent flags, a common enforcement gap in most systems.

Agent-Specific Feature Flag Types

To address these surfaces, the speaker proposes six specific flag types:

1. Prompt Variant Flags: Route different user segments to different system prompt versions on the fly without a deploy. This allows for A/B testing prompts (e.g., experimental V3 for beta, stable V1 for general users) and prevents incidents like Cursor SAM.
2. Tool Access Flags: Authorize or revoke specific tools per user segment, type, or risk class. Essential for agents with money-moving, data-deleting, or compliance-sensitive tools, preventing accidental mass emails or data leaks.
3. Model Routing Flags: Control which model handles which traffic, enabling seamless migration, fallback, or canary deployments. This is crucial for mitigating provider outages or model deprecations, ensuring system resilience.
4. Memory Policy Flags: Control what an agent remembers across sessions across four dimensions: retention (session, 30 days, forever), scope (per user, per tenant, global), write enabled (can the agent persist memory?), and user visible (can the user inspect/delete?). These flags define privacy posture and compliance with regulations like GDPR.
5. Autonomy Level Flags: Manage the blast radius by setting the agent’s action level: Suggest (agent recommends, human acts), Auto Approve (agent prepares, human confirms), or Auto Execute (agent acts independently). Defaulting to “suggest” is a critical safety measure.
6. Kill Switch: A pre-wired, agent-wide, and per-surface off toggle that takes effect in seconds, without deployment, restart, or code changes. It respects inflight requests at the next decision point and must be wired from the agent’s design phase, not as a hot patch during an incident. Without it, incidents like LangChain’s runaway costs or Replit’s data deletion become unavoidable.

Live Demos and Architecture

Two storyboards demonstrate these concepts: first, a tool access flag gracefully handles a disabled email sending tool by offering to copy the draft instead of citing a non-existent policy; second, a kill switch rapidly mitigates a runaway LangChain pipeline, bringing costs under control in 30 seconds without manual intervention or deployment.

The proposed architecture involves a middleware layer between the user and the agent loop. This middleware resolves flags, routes models, applies autonomy, and honors kill switches. Crucially, sub-agents must also pass through this middleware; a common failure mode is child agents bypassing it, rendering kill switches ineffective.

Rollout Playbook and Key Metrics

Five steps for secure agent rollout are proposed:

1. Kill Switch First: Wire agent-wide and per-tool kill switches and ship them before anything else.
2. Wrap the Tools: Every tool call must resolve a flag before execution.
3. Stage Autonomy: Default everything to “suggest,” then progressively enable “auto-approve” and “auto-execute” per tool as trust is built.
4. Variant Prompts: Move system prompts out of code into flag-resolved configurations.
5. Watch the Slope: Continuously track key operational metrics.

Four critical metrics to track from day one:

1. Kill switch fires per week: Target 0. More than two indicates a problem requiring investigation.
2. Time to mitigation: Target under 5 minutes for a kill switch, under 30 minutes for a prompt rollback. Longer times mean mitigation won’t fit within an incident window.
3. Canary error rate delta: If a new prompt variant’s error rate climbs more than 2% over baseline at 5% rollout, block the promotion.
4. Flag audit trail completeness: 100% required to debug incidents; who flipped what, when, and why.

Common Failure Modes to Avoid

• Flags resolved at session start, not per turn: Inflight conversations won’t see changes until the next session.
• Sub-agents bypass middleware: Kill switches won’t reach child processes.
• Context drift flags: Stale user segmentation context across long conversations.
• Caching defeating the flip: Aggressive caching can return old responses even after flags are flipped.
• No alert on kill switch fires: Kill switches going off silently, with product owners finding out much later. Every kill switch fire should trigger an alert.

The Business Case and Regulatory Imperative

Modern enterprises demand these capabilities. Buyers will ask about kill switches, prompt rollout policies, beta isolation, fast mitigation, and audited flag flips. Furthermore, regulations like the EU AI Act and legal precedents (Mopar vs. Air Canada, Garcia vs. Corrector AI) are making these safety mechanisms a compliance necessity.

Bad Habits to Break

• Kill switches rot: They get wired, then neglected, and fail when needed.
• Flag sprawl: Too many undocumented flags create hidden coupling and complexity. Every flag needs an owner and a removal date.
• Temporary flags become permanent: Flags shipped for rollouts must be removed promptly.
• Flag-driven prompt combinatorial explosion: Testing all combinations of multiple prompt variants becomes unmanageable.

Key Takeaways

1. Ship the kill switch first: This is the most impactful investment for operational posture.
2. Treat the six surfaces independently: Prompts, tools, models, memory, autonomy, and sub-agents each need their own flag type and metrics tracking.
3. Match discipline to blast radius: AI agents, with their ability to send emails, move money, and modify databases, demand at least the same, if not greater, engineering discipline as traditional web applications. This is why 2027 will be about control, not just adoption.

• Tags:
• Prompt Engineering
• Developer Tools
• AI Safety
• MLOps
• Incident Response
• Model Routing
• Software Reliability
• Agent Systems
• Feature Flags
• Canary Releases
• Kill Switch
• Autonomy Levels
• Production Engineering