Full Workshop: Build Your Own Deep Research Agents - Louis-François Bouchard, Paul Iusztin, Samridhi

The Problem with Modern AI Content

A significant challenge in the current AI landscape is the proliferation of “AI slop”—generic, shallow, and often meaningless content generated by language models. This content is characterized by repetitive phrases (“rapidly evolving landscape,” “it’s not about X, but about Y”), unsubstantiated claims (“most companies miss…”), and a general lack of depth. The core issue is the absence of a robust research and writing process. To address this, we can build a system that automates the creation of deeply researched, high-quality technical content, moving beyond simple chatbot interactions to a goal-directed pipeline.

This workshop presents a system designed to tackle this problem, born from the practical need to create technical courses and content. The system is split into two main components:

1. A Deep Research Agent that uses various tools to conduct thorough research on a given topic.
2. A Technical Writing Workflow that transforms the research findings into a polished, well-structured article.

AI Engineering: Choosing the Right Architecture

Building effective AI systems requires making deliberate architectural choices based on constraints like cost, latency, and quality. A key concept is the autonomy slider, which represents a spectrum of complexity:

• Simple Prompting: Suitable when the model already possesses the necessary knowledge.
• Context Engineering (RAG): Used when external, private, or real-time data is needed.
• Workflows: A sequence of predetermined steps. Workflows are ideal for deterministic processes where the order of operations is fixed and does not change. An example is a support ticket handling system that always classifies, routes, and drafts a response in the same order. Building this as an agent would add unnecessary overhead.
• Agents: Employed when the system needs to make autonomous decisions, plan a course of action, and react dynamically to its environment. Agents are suited for exploratory tasks where the exact steps are not known in advance.

A critical challenge, especially for agents, is context management. As an agent interacts and uses tools, its context window grows, which can lead to performance degradation, a phenomenon known as “context rot” or the “lost in the middle” problem. Effective context management involves techniques like summarization, strategic retrieval, and delegating tasks to sub-agents or tools, which naturally leads to multi-agent systems for complex problems.

System Design: A Tale of Two Components

The research and writing tasks have conflicting requirements, leading to a crucial design decision: splitting the system into two distinct parts.

1. The Research Agent: This part needs to be exploratory and flexible. It must be able to search the web, pivot its strategy based on findings, and iterate on its research plan. This calls for a more dynamic, agentic architecture.
2. The Writing Workflow: This part needs to be deterministic and consistent. It must adhere to a specific tone, structure, and quality standard, avoiding hallucinations and AI slop. This requires a more constrained, workflow-based architecture.

The two systems run sequentially. The research agent produces a research.md file, which serves as the input for the writing workflow. This simple, file-based communication avoids the need for complex orchestration.

Part 1: The Deep Research Agent

The research agent is built using the Modular Cognitive Protocol (MCP), with the agent acting as the “brain” for reasoning and planning, and an MCP server providing the “capabilities” (tools, prompts, and resources).

The agent is equipped with three primary tools:

• deep_research: Utilizes Gemini’s grounding capabilities to perform web searches and return precise, cited answers.
• analyze_youtube_video: Leverages Gemini’s multimodal features to process a YouTube URL directly, analyzing the video content (not just the transcript) to provide a summary.
• compile_research: Gathers the outputs from the other tools, which are stored in a memory/ directory, and synthesizes them into the final research.md artifact.

To guide the agent, we replace a monolithic system prompt with Agent Skills. Skills are a more maintainable and context-efficient method for defining workflows. They use “progressive disclosure,” loading only the skill’s name and description initially and the full prompt body only when the skill is invoked, thus keeping the context lean.

Part 2: The Technical Writing Workflow

The writing workflow is a deterministic pipeline designed for control and quality, transforming the raw research into a polished post.

Controlling the Generation

Three key elements are used to control the output:

1. Guideline (guideline.md): A user-provided file specifying the topic, angle, key points, narrative flow, and target audience. This is the primary input that shapes the what of the content.
2. Writing Profiles: Static Markdown files that define the how of the content. These include a structure profile (e.g., character count, post format), a terminology profile (banning slop words like “delve” and “tapestry”), and a character profile (defining the author’s voice and style).
3. Few-Shot Examples: A small number of high-quality, representative examples of the desired output (e.g., 3-5 successful LinkedIn posts). These help the model understand the target style and format implicitly.

The Evaluator-Optimizer Pattern

The most innovative part of the writing workflow is a multi-step refinement loop:

1. Initial Draft: The system first generates a draft based on the combined context (guideline, research, profiles, examples).
2. Review Loop: The draft then enters an iterative loop.
   • Reviewer: A separate LLM call with a fresh context is initiated. This “reviewer” assesses the draft against the guideline, research (to check for hallucinations), and writing profiles. Its feedback is structured as a list of Pydantic objects, each detailing a specific violation (e.g., “used banned term ’leverage’ in paragraph 2”). Using a separate context avoids the bias of an LLM favorably reviewing its own work.
   • Editor: The original LLM context, acting as the “editor,” receives this structured feedback and revises the draft. Reviews are prioritized to first address guideline violations, then research inaccuracies, and finally profile deviations.
3. Final Output: This loop runs for a fixed number of iterations (e.g., 3-4 times), progressively refining the post to meet all specified criteria.

Part 3: Observability and Evaluation

Moving a system from a cool demo to a reliable product requires robust monitoring and evaluation.

• Observability: Using tools like Opik, we can trace the entire execution of the agent and workflow. This provides visibility into every LLM call, tool invocation, latency, and cost, which is indispensable for debugging complex, multi-step processes where simple logs are insufficient.

• AI Evals: To quantitatively measure and improve the system’s performance, a systematic evaluation process is implemented:

  1. Create a Dataset: A high-quality dataset is built by taking ~20 real-world posts and reverse-engineering their corresponding guideline and research inputs. Each generated output is then manually labeled with a pass/fail status and a short critique.
  2. Build an LLM Judge: An evaluator model is created. This “judge” is a binary classifier that takes a generated post and its context (guideline, research) and outputs a pass/fail label and critique. The training split of the dataset is used to provide few-shot examples to the judge.
  3. Calibrate the Judge: The judge’s reliability is measured by computing its F1 score against the dev and test splits of the labeled dataset. The judge’s prompt and examples are tuned until its classifications align closely with the human expert’s labels.
  4. Continuous Improvement: Once calibrated, the judge can be used for regression testing and to continuously score the system’s output, ensuring that changes and new features do not degrade quality.

• Tags:
• Prompt Engineering
• RAG
• Gemini
• LLM Evaluation
• MCP
• AI Engineering
• Agentic Workflows