CLAUDE.md, AGENTS.md, Skills, Hooks and Subagents: A Field Guide to Steering AI Agents (Ep. 1022)

This episode, a sequel to Episode 1020, delves into the critical art of steering AI agents, focusing on where instructions should reside to ensure reliable execution without bloating requests or incurring unnecessary costs. The core challenge addressed is why AI agents often ignore instructions given early in long sessions.

The Three Properties of Instruction Delivery

All methods for delivering instructions to an AI agent vary along three fundamental properties:

1. When the instruction loads into the model’s context.
2. Whether it survives long sessions, particularly during compaction, where the agent summarizes past conversation to free up context window space.
3. How much authority it carries.

Understanding these properties is key to reasoning about any instruction mechanism on any platform, explaining why early instructions can be lost when sessions run long.

Seven Methods for Delivering Instructions (Claude Code Taxonomy)

The Anthropic post outlines seven distinct methods, designed to make instructions cheap to carry and hard to lose:

1. claude.md file: A markdown file at the root of a project. It loads at session start, stays in context, and is reread after every compaction, making it suitable for always-relevant facts like build commands, directory layouts, or coding conventions. Anthropic advises keeping it under 200 lines, assigning an owner, and reviewing changes like code. A clever variant is a claude.md file in a subdirectory, loading only when the agent touches a file within that subdirectory, allowing for path-scoped, team-specific conventions.
2. Rules: Also markdown files, but they express specific constraints (e.g., all API handlers must validate input with a specific library). Their power lies in path scoping, loading only when relevant files are touched (e.g., an API rule loads only when working in the API directory).
3. Skills: A folder containing a procedure (e.g., a deployment workflow, release checklist). Only the skill’s name and a one-line description load at session start; the full body loads only when explicitly invoked or when the agent matches it to a task. This prevents lengthy procedures from cluttering the context window when not actively needed.
4. Sub-agents: These invert the problem by keeping context out of the main agent. A sub-agent is a defined assistant that runs a side task (e.g., deep search, log analysis) in its own fresh, isolated context window. Only the sub-agent’s final summary returns to the main agent, preventing intermediate results from cluttering the main conversation. Sub-agents can nest up to five levels deep, orchestrating complex multi-agent systems.
5. Hooks: Described as the most crucial method for data scientists to internalize. Hooks are deterministic code (commands, HTTP calls, checks) that fire reliably on lifecycle events (e.g., before a tool call, after a file edit). The critical distinction: an instruction is a probability, whereas a hook is a guarantee. If something must never happen or must always happen, enforcement requires deterministic code like a hook, not persuasive prose. This parallels the need for validation checks in data pipelines over comments.
6. Output Styles: A blunt instrument that modifies the system prompt to replace the agent’s entire default role, potentially stripping built-in behaviors (e.g., running tests before declaring work complete). Handle with care due to its broad impact.
7. Appending the System Prompt: This is an additive modification to the system prompt, best suited for tone and formatting preferences. It carries the highest instruction-following weight among all methods.

Generalization and Industry Standards

These ideas have transcended Anthropic’s specific tools:

• agents.md Standard: OpenAI recently released agents.md as an open, plain markdown standard for instructing any AI coding agent on how to build, test, and contribute to a project, similar to a root claude.md file. Stewardship has moved to the Agentic AI Foundation under the Linux Foundation, with backing from major tech companies. It’s natively read by many tools (OpenAI Code Interpreter, Cursor, GitHub Copilot, Google Gemini CLI) and adopted by tens of thousands of repositories. The emerging best practice is to make agents.md the single source of truth.
• Academic Study on Instruction Files (Zurich): A study across 138 real-world repositories found that developer-written instruction files modestly improved agent task success rates (approx. 4%) and drastically cut agent-introduced bugs (35-55%). Conversely, LLM-generated instruction files decreased success rates and increased inference costs (>20%). This highlights that the value lies in human judgment encoded in the file, not merely the file’s existence.
• Cross-Vendor Convergence: OpenAI’s Code Interpreter has adopted skills and hooks and supports sub-agents. Skills, originally an Anthropic format, are now an open standard. Even consumer-level tools like ChatGPT’s custom instructions (always-on root file), projects/gems (path-scoped rules), and Custom GPTs/gems (single-task skills) reflect these principles.

Three Practical Takeaways

1. Match persistence to relevance: Keep always-on files ruthlessly short for core facts. Use scoped mechanisms (like skills) for procedures and area-specific conventions that load on demand.
2. Instruction vs. Guardrail:

• Tags:
• AI Agents
• Prompt Engineering
• Feature
• Guardrails
• Context Window
• AGENTS.md
• Sub-Agents
• Skills
• Machine Learning Operations
• Instruction Steering
• Claude.md
• Hooks
• AI Best Practices