Prompt Engineering for LLMs, PDL, & LangChain in Action

The role of a prompt engineer is evolving from a craft of finding “magic words” to a structured software engineering discipline. The core challenge is that Large Language Models (LLMs) are probabilistic, not deterministic. Unlike traditional functions in computing, an LLM’s output can vary with slight changes in wording, examples, or temperature, which is unacceptable for software that expects predictable, structured data.

For instance, when using an LLM to convert a free-form text bug report into a strict JSON format, the model might occasionally fail. It could return non-JSON text, wrap the JSON in a sentence like “Sure, here’s the reformatted report,” or drift off-schema by renaming a key from summary to synopsis. These inconsistencies can break software applications.

A Disciplined Approach to Prompt Engineering

To build reliable software with LLMs, prompt engineering must focus on three key areas:

1. Contract: Defining the exact shape and schema of the desired output upfront, including keys, types, and enums (e.g., severity must be one of “low”, “medium”, or “high”).
2. Control Loop: Implementing a system to validate every LLM response against the contract. If validation fails, the system should automatically retry, perhaps with more explicit instructions or a constrained decode, to repair the output.
3. Observability: Tracing which prompts produce which outputs. This allows for monitoring, debugging, and ensuring that changes to the system are safe and effective.

Two tools that facilitate this structured approach are LangChain and Prompt Declaration Language (PDL).

LangChain: A Code-First Pipeline

LangChain is an open-source framework for building LLM applications using a pipeline of composable steps called “runnables”. It allows developers to define what happens before and after a model call, not just the prompt itself.

Using the bug report triage example, a LangChain pipeline would look like this:

• Input: The user’s unstructured bug report text.
• PromptTemplate Runnable: This packages the core instructions (e.g., “You are a triage assistant, return JSON only”) and the expected JSON shape with the user’s input. This ensures consistency across calls.
• ChatModel Runnable: The formatted prompt is sent to an LLM, which returns a candidate JSON string.
• Validate Runnable: The candidate JSON is checked against the predefined schema.
  • Success Path: If validation passes, the clean JSON is sent to the application.
  • Failure Path: If validation fails, the process moves to a Retry or Repair runnable. This step can resend the request with stricter instructions or perform small fixes, like stripping extraneous text.
  • Fallback Path: If the retry/repair step also fails, a fallback can be triggered, such as using a different, potentially stricter, LLM.

This entire process ensures that the application ultimately receives clean, valid JSON, while also capturing traces and metrics to spot regressions and improve the system over time.

PDL: A Spec-First Declarative Approach

Prompt Declaration Language (PDL) offers a different, declarative paradigm for defining LLM workflows. The core idea is that since most LLM interactions are about producing data, you should declare the shape of that data and the steps to generate it within a single specification file.

PDL uses a YAML file to define the entire workflow, co-locating the prompt, the contract, and the control loop.

• Execution Model: A PDL interpreter runs the YAML file top-down. The spec is an ordered list of items, which can be literal strings or blocks that call a model.
• Context Building: As the interpreter processes the file, strings are appended to a running context. When a model block is encountered, this accumulated context is used as the default input.
• Type Safety and Control: PDL allows you to declare types for inputs and outputs, and the interpreter will enforce them, failing on shape violations. It also provides explicit control flow structures like conditionals and loops.
• Observability: Built-in tracing and a “live explorer” let you inspect the inputs, outputs, and exact context for each block in the workflow.

Conclusion: Code-First vs. Spec-First

LangChain and PDL are part of a growing toolbox that transforms prompt engineering into a mature software engineering practice.

• LangChain is a code-first framework where you wire together runnable components in your programming language of choice.
• PDL is a spec-first approach where the prompt, types, and control flow are defined together in a declarative YAML file that is then executed by an interpreter.

Together, these tools provide the necessary structure, reliability, and observability to move beyond simple “prompt whispering” and build robust, production-ready AI applications.

• Tags:
• Prompt Engineering
• LLM
• Feature
• LangChain
• PDL
• Structured Output
• LLM Workflows