Building Agentic Applications with Spring AI • Matthew Meckes • GOTO 2025

Matthew Meckes argues that Java, contrary to being overshadowed by Python in the AI space, is poised to be a dominant force for building enterprise-scale agentic AI applications. The strengths that have made Java a cornerstone of enterprise development for decades—modularity, testability, a massive developer ecosystem, and proven scalability—are precisely what’s needed to move AI agents from experimental prototypes to production-ready systems that drive core business value.

What are AI Agents?

An AI agent is an autonomous software component that can reason, complete tasks, and achieve goals on behalf of users or other systems. The spectrum of agents ranges from simple generative AI assistants (e.g., for summarization) to complex, multi-agent swarms that can operate without human intervention.

A typical agent is composed of several key building blocks:

• LLM: The core reasoning engine.
• Prompt: System instructions that define the agent’s persona, constraints, and goals (e.g., “You are a travel agent. Do not write poetry.”).
• Memory: To maintain context across a conversation or workflow.
• Tools: Functions or services the agent can call to interact with the real world, such as booking a flight or querying a database.
• Retrieval-Augmented Generation (RAG): To provide the agent with access to external, up-to-date knowledge.

These components work together in an “agentic loop”: a user provides input, the LLM processes it, determines if a tool is needed, executes the tool, retrieves the result, and formulates a final response.

The Enterprise Case for Java and Spring AI

While much of the initial AI/ML research and development has occurred in the Python ecosystem, the next phase involves integrating these capabilities into the core applications that run businesses. These applications are overwhelmingly built in Java, with an estimated 60% of enterprises using Spring.

Spring AI provides the essential abstractions to bridge the gap, enabling Java developers to build agentic capabilities directly into their existing applications. It addresses the core challenges of working with LLMs:

• Statelessness: Spring AI offers abstractions for ChatMemory to handle conversational context.
• Lack of Domain Knowledge: It provides robust support for RAG, with flexible integrations for numerous vector databases (PGVector, OpenSearch, Pinecone, Bedrock Knowledge Bases), allowing developers to start simple and scale as needed.
• Inability to Act: It simplifies function calling and tool integration.
• Stale Knowledge: It allows for injecting real-time data into the model’s context.

Building with Spring AI: A Practical Approach

Integrating an LLM like AWS Bedrock into a Spring Boot application is remarkably straightforward. It requires adding a couple of dependencies, configuring the model in application.properties, and using an elegant ChatClient.

<dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-bedrock-starter</artifactId>
</dependency>

The real power for enterprise integration comes from exposing existing Java services as tools. With a simple annotation, any Java bean can become a tool available to the AI agent. Spring AI uses this to create a Model Context Protocol (MCP) server, automatically handling the conversion from natural language to a strongly-typed Java function call.

// Expose an existing Java method as a tool for the agent
@Tools
public class MyTimeService {
    @Bean
    public Function<String, String> currentTime(
        @Description("Gets the current time for a given timezone.") String timezone
    ) {
        // ... implementation ...
    }
}

This allows developers to spend less time on “LLM magic” and more time on what matters: leveraging their existing, robust, and well-tested business logic.

From Prototype to Production: Pragmatic Learnings

The promise of fully autonomous agents that can replace complex, hard-coded workflows is still a long way off. While it’s easy to create impressive demos, achieving the reliability needed for production (closing the final 20% gap) is incredibly difficult if you cede all control to the LLM.

A more pragmatic and production-ready approach is to “sprinkle” LLMs into existing, developer-owned workflows. The true secret power of LLMs in an enterprise context is their ability to convert natural language into structured data (JSON) for API calls.

Key principles for building effective agents, inspired by the “12-Factor Agent” concept, include:

1. Own Your Control Flow: Use traditional code (if/switch statements, loops, validation) to manage the overall process. Don’t rely on the LLM to orchestrate complex workflows.
2. Humans are First-Class Operators: Design systems with human-in-the-loop validation as a core feature, not an edge case.
3. Small, Focused Agents Beat Monoliths: Reduce the scope of each agent to a maximum of 3-10 steps to ensure reliability with the current state of LLM technology.

• Tags:
• Agentic AI
• LLM
• RAG
• Enterprise AI
• Java
• AWS Bedrock
• Spring AI