DSPy: The End of Prompt Engineering - Kevin Madura, AlixPartners

Building robust, testable, and enterprise-grade AI applications requires a paradigm shift from brittle “prompt engineering” to a more systematic approach of “programming with LLMs.” DSPy provides a framework for this shift, treating prompts as an optimizable implementation detail rather than the core focus of development. It enables developers to build modular, composable, and transferable programs that treat LLMs as first-class citizens.

The Core Philosophy: From Prompts to Programs

The central idea behind DSPy is to elevate the level of abstraction. Instead of tweaking prompt strings, developers should focus on defining the program’s logic and control flow. DSPy provides a set of primitives that allow you to declare your intent—what you want the program to do—while deferring the how to the framework and the underlying language models.

This “systems mindset” leads to programs that are:

• Modular: Logic is decomposed into clean, reusable components.
• Transferable: The same program logic can be retargeted to different and newer LLMs with minimal changes. An optimizer can then tune the program to regain performance on the new model.
• Optimizable: Performance can be quantitatively measured and improved by automatically tuning the prompts and few-shot examples.

The Primitives of DSPy

DSPy is built on a few core concepts that work together to create complex applications.

1. Signatures: Defining the Interface

Signatures are the most fundamental building block. They are a declarative way to specify the inputs and outputs of an LLM-powered function call. They define the contract for what a module will do, without specifying how.

• Shorthand Signatures: For rapid prototyping, you can use a simple string format like "text -> sentiment". DSPy infers the types and generates the necessary prompt structure.
• Class-based Signatures: For more complex and production-oriented use cases, you can define signatures using Pydantic-style classes. This provides strong typing for both inputs and outputs. Crucially, the field names and descriptions within the class become part of the prompt itself, guiding the LLM’s behavior.

class SentimentSignature(dspy.Signature):
    """Classifies the sentiment of a given text."""
    text = dspy.InputField(desc="The text to be classified.")
    sentiment = dspy.OutputField(desc="An integer from 1 (very negative) to 10 (very positive).")

2. Modules: Structuring the Logic

Modules are the Pythonic building blocks of a DSPy program, analogous to nn.Module in PyTorch. They encapsulate signatures and any surrounding business logic. You can compose modules to create complex pipelines.

DSPy includes several built-in modules that implement common prompting techniques:

• dspy.Predict: A standard, single-turn prediction.
• dspy.ChainOfThought: Instructs the model to reason step-by-step before providing an answer.
• dspy.ReAct: Enables the model to use tools (Python functions) to answer questions, similar to an agent.

You create custom modules by inheriting from dspy.Module and defining the data flow in the forward method, which can intersperse LLM calls with regular Python code.

3. Adapters: Controlling the Prompt Format

Adapters act as prompt formatters, sitting between the signature and the final string sent to the LLM. They translate the structured signature into a specific format that a model might prefer. This allows you to experiment with different prompt structures without changing your core program logic. For example, the BAMLAdapter can produce more token-efficient and sometimes more performant prompts compared to a standard JSONAdapter.

4. Optimizers: The Killer Feature

Optimizers are arguably DSPy’s most powerful feature. They automate the process of prompt tuning. Given a small dataset of input-output examples and a metric for success, an optimizer will iteratively test and refine the prompts for each module in your program to maximize performance.

The process involves:

1. Defining a Metric: You write a function that scores the output of your program (e.g., does the predicted answer match the gold-standard answer?). This can even use an LLM as a judge for more subjective tasks.
2. Running the Optimizer: The optimizer (e.g., MIPRO) runs the program on the dataset, evaluates the outputs using your metric, and generates new, improved prompts.
3. Compiling the Program: The output is a “compiled” version of your program with the optimized prompts baked in, which you can then save and deploy.

This enables incredible transferability. You can develop a program using a powerful model like GPT-4, then use an optimizer to compile it to run on a much smaller and cheaper model (e.g., Llama 3 8B or Gemini 2.5 Flash) while recovering a significant portion of the performance.

Real-World Applications

The talk demonstrated several practical examples built with DSPy:

• Complex File Routing: A program that ingests arbitrary files (e.g., SEC filings, legal contracts, images), classifies them by type, and routes them to different processing pipelines. An SEC filing might have structured data extracted, while a contract is summarized.
• PDF Boundary Detection: An advanced module that analyzes a PDF visually by processing its page images. It classifies each page (e.g., “Title Page,” “Table of Contents,” “Exhibit”) and then logically groups consecutive pages to identify the document’s structure—a task that is difficult with text alone.
• Structured Data Extraction: Using class-based signatures to extract deeply nested information from documents into clean, validated Pydantic objects.

• Tags:
• Prompt Engineering
• LLM
• AI Engineering
• DSPy
• Structured Output
• Prompt Optimization