500 Skills, Zero Fine-Tuning: LinkedIn's Playbook for AI Agents — Ajay Prakash, LinkedIn

LinkedIn’s journey with coding agents reveals the critical challenges of deploying large language models (LLMs) in complex enterprise environments and their innovative solutions for building reliable, context-aware AI assistants.

The Initial Problem: LLM Hallucinations in Enterprise

Early attempts to use coding agents at LinkedIn faced significant hurdles. LLMs, primarily trained on public repositories, lacked crucial internal context regarding:

• Thousands of internal repositories, microservices, and applications.
• Proprietary internal frameworks, libraries, and custom infrastructure (e.g., databases, experimentation platforms, configuration systems).
• The extensive “tribal knowledge” that new engineers spend weeks in bootcamps to acquire.

This contextual gap led to frequent hallucinations, agents getting stuck, or generating incorrect code, often requiring more effort from engineers to correct than manual coding, rendering agentic coding ineffective initially.

Evolving the Tooling: From MCP to Playbooks

LinkedIn’s solution began with building an internal “Memory, Cognition, Perception” (MCP) server, inspired by Anthropic’s early work. The first major “tool” integrated was their sophisticated internal code search, allowing agents to find relevant examples and patterns across LinkedIn’s vast codebase. This was a significant “unlock,” enabling agents to answer queries and implement solutions based on internal practices.

Further tools were added, including integrations with:

• Internal documentation systems.
• Jira for task management.
• Slack for communication history.
• Data platforms and feature flag systems.

While valuable, this “tools-only” approach still fell short for complex, end-to-end workflows due to:

• Scattered Knowledge: Essential “how-to” information was fragmented across outdated wikis, internal documents, and old Slack threads.
• Context Overload: MCP-based systems typically degrade past 30-40 tools, as each tool’s output consumes valuable context window space, leading to agents losing information and needing to restart.
• Lack of Durable Memory: Agents had no persistent memory, meaning each interaction started from scratch, requiring repeated context provisioning.

Introducing Playbooks: Structured Instructions as Tools

To address these limitations, LinkedIn developed “playbooks” (an invention in early 2025). Playbooks are structured instructions and prompts, published and invoked by agents just like any other tool via the MCP server. When an agent invokes a playbook, the relevant instructions and context are returned as tool output, guiding the agent through complex tasks.

Key principles for designing effective playbooks include:

• Self-Contained: Each playbook should focus on a very specific task (e.g., setting up an Airflow DAG), aiding agents in selecting the correct playbook.
• Modular and Referenced: Large playbooks are broken down into smaller, referencable sub-playbooks. This design promotes reusability (smaller playbooks can be used by multiple larger ones) and enables “progressive discovery of context,” where agents only load relevant information as needed, mitigating context overload. This concept is analogous to “skills” but was developed independently.

A Self-Improving Ecosystem

A crucial aspect of the playbook system is its self-improving loop. Agents are encouraged to:

• Identify outdated information, discrepancies, or missing details within playbooks after a session.
• Propose improvements to the playbook content.
• Automatically create pull requests (PRs) to update the underlying playbook code in the repository.

This continuous feedback mechanism ensures the knowledge base remains fresh, accurate, and aligned with current internal practices.

Architecture and Scaling

LinkedIn’s agent infrastructure features a local MCP server, pre-installed on all employee laptops and updated hourly. It supports two types of playbooks:

• Central Playbooks: Cross-cutting instructions applicable across multiple repositories.
• Local Playbooks: Repository-specific instructions, checked into the repo alongside the code, activated only when agents operate within that particular repository.

This architecture supports seamless authentication and telemetry, providing valuable data for system improvement.

To overcome the inherent scaling limitations of MCP (which degrades beyond 30-40 tools), LinkedIn introduced three meta-tools:

1. search: Agents first use this tool to discover relevant tools and playbooks based on keywords and tags, guided by system instructions.
2. get_schema: Once a relevant tool or playbook is identified, agents use this to retrieve its detailed schema.
3. execute: Finally, agents invoke this tool to execute the chosen tool or playbook.

This meta-tool approach has allowed LinkedIn to scale its system to over 1,300 tools and 600 playbooks, serving over 8,000 daily users across various functions (engineering, product management, design, technical program management).

Key Takeaways

LinkedIn’s experience highlights two critical lessons for enterprise AI:

1. Prioritize Quality and Reliability: Success hinges on designing systems with quality and reliability as foundational principles from day one, ensuring productivity doesn’t compromise system integrity.
2. Build Tailored Infrastructure: In large enterprises, simply providing cutting-edge AI models or tools is insufficient. Dedicated infrastructure must be built to allow agents to operate effectively within the unique and complex internal environment.

• Tags:
• Software Development
• Large Language Models
• Context Engineering
• Developer Productivity
• Feature
• Enterprise AI
• MCP
• Agentic Systems
• AI Tooling
• LLM Agents
• Internal Tools
• Playbooks
• LinkedIn Engineering