Deploying Executable Agent Workflows

The traditional paradigm for agentic reasoning, centered around JSON-based tool calling popularized by models from OpenAI and others, has several fundamental limitations when applied to complex, real-world workflows. A new approach, called CodeAct, proposes a more flexible and powerful alternative: allowing Large Language Models (LLMs) to write and execute Python code directly to interact with tools and the environment.

The Problem with Traditional JSON-Based Tool Calling

While effective for simple, single-turn interactions, the conventional approach of an LLM generating a JSON object to specify a tool and its parameters breaks down in more sophisticated scenarios.

• Expressing Logic and Loops: It is difficult to express conditional logic (if/else) or loops (for). For a task like “get all users and deactivate each active user,” an LLM would fetch the full list of users and then attempt to generate a separate tool call for each deactivation. This approach fails at scale (e.g., with 10,000 users) due to context window limitations, the “lost in the middle” problem, and the inability of attention mechanisms to handle such a large volume of repetitive tasks.

• Dependency on Fine-Tuned Models: The ability of models to generate specific JSON schemas is the result of extensive fine-tuning. This creates a dependency on a specialized fine-tuning pipeline, which requires data collection, versioning, and periodic retraining. The pace of improvement for general code generation models often outstrips that of these specialized, format-constrained models.

• Representing Nested Objects: Translating complex, nested data structures (like Pydantic models representing a company with departments, employees, and addresses) into a JSON schema for an LLM can result in a “nasty,” verbose, and confusing representation. This adds significant noise to the model’s context, potentially degrading its reasoning capabilities.

• Lack of Composability: Chaining operations—using the output of one function as the input to another—is cumbersome. The LLM first calls getAllUsers(), gets the result back, and only then can it reason about what to extract from that output to pass into a subsequent computeHappiness() function. This requires multiple round trips and makes fluid, multi-step logic difficult to implement.

• Limited Action Space: The agent is restricted to the predefined set of tools. If a task requires a simple aggregation like counting the number of active users, the agent must first call a tool to get the full list and then rely on the LLM’s own reasoning capacity to perform the count. This is inefficient and relies on a capability (like counting) that LLMs are not always reliable at. With CodeAct, the agent could simply use Python’s built-in len() function on the list of users.

CodeAct: Generating Code Instead of JSON

The CodeAct paradigm shifts the agent’s output from declarative JSON to imperative Python code. The core loop becomes:

1. Plan: The LLM understands the user’s intent.
2. Write Code: Instead of emitting a JSON object, it generates a Python script to accomplish the task. This script can call predefined functions, use loops, conditional logic, and even leverage libraries like Pandas for on-the-fly data manipulation.
3. Execute: The generated Python code is run in a sandboxed execution environment.
4. Observe & Revise: The LLM observes the output (stdout) and any errors (stderr) from the execution. If an error occurs or the output indicates a wrong turn, the model can revise the code in the next step and re-execute, creating a powerful debugging and iteration loop.

Technical Implementation and Advantages

To implement CodeAct, you provide the LLM with the function signatures (without the implementation bodies) in a system prompt. The prompt also sets constraints on the environment, such as access to the internet or file system, to limit the action space and prevent convergence issues.

This approach offers significant benefits:

• Increased Power and Flexibility: The LLM can dynamically compose functions, perform data transformations, and implement complex logic without needing a pre-defined tool for every possible action.
• Improved Competitiveness: For workflow-based tasks, smaller models using CodeAct can be competitive with or even outperform larger, more expensive models that are constrained to JSON-based tool calling.

When to Use CodeAct vs. JSON

• Use CodeAct when your application involves complex workflows, requires branching and dynamic logic, and you are willing to invest time in customizing the execution environment. It is ideal for building powerful, stateful agents.
• Use JSON for simpler, “plug-and-play” solutions, stateless tasks, or when extremely low latency is a primary concern, as it avoids the overhead of code generation and execution.

To control the unconstrained nature of code generation, you can use tools like linters and Abstract Syntax Trees (ASTs) to analyze the code before execution, spot errors, and block forbidden functions. This, combined with careful prompt engineering to restrict the search space, provides the necessary guardrails for production deployment.

• Tags:
• LangGraph
• Python
• Tool Use
• LLM Agents
• CodeAct
• Agent Architectures