Building Agents at Cloud Scale — Antje Barth, AWS

At Amazon and AWS, the belief is that AI will reinvent virtually every customer experience, leading not just to the enhancement of existing applications but to the creation of entirely new ones. Internally, Amazon has over 1,000 generative AI applications in development or production. The complete reimagining of Alexa serves as a primary example of agents at a massive scale, with over 600 million devices. This new Alexa operates through hundreds of specialized “expert systems”—groups of capabilities, APIs, and instructions—that orchestrate tens of thousands of partner services to accomplish complex user requests.

The future will be composed of specialized agents, each with unique capabilities, working together seamlessly. The key challenge for developers is transitioning from building traditional web services to creating these “agentic services.”

The Shift to Model-Driven Development

A fundamental rethink in building agents has led to a model-driven approach. This strategy leverages the advanced reasoning, planning, and action-taking capabilities of modern LLMs. It allows developers to focus on defining what their agent should do, rather than prescribing how it should do it.

This approach dramatically accelerates development. For instance, the Amazon Q Developer Agent for the CLI—an agentic chat experience in the terminal that can debug issues, answer questions, and manipulate files—was built and shipped in just three weeks.

Strands Agents: An Open-Source SDK for Building Agents

To enable this rapid development for everyone, AWS has open-sourced Strands Agents, a Python SDK for building and running AI agents. Strands simplifies agent creation by connecting the two core components—the model and the tools—allowing developers to start with a simple prompt and a set of tools defined in code, test locally, and then deploy to the cloud.

Model and Tool Integration

• Model Providers: By default, Strands integrates with Amazon Bedrock (e.g., using Claude 3.5 Sonnet). However, it is provider-agnostic, with built-in support for local models like Llama, as well as APIs from Anthropic, Meta, and OpenAI. It can also integrate with any provider available through LightLLM and supports custom model providers.

• Pre-built Tools: Strands comes with over 20 pre-built tools for a variety of tasks, including:

  • File manipulation and API calls.
  • Memory and RAG: A key tool is retrieve, which performs semantic search over a knowledge base. This is powerful for tool selection at scale. For example, an internal AWS agent manages over 6,000 tools. Instead of overwhelming a model’s context window, the agent uses retrieve to find the most relevant tools based on their descriptions and only pulls those into the context for the final decision.
  • Multimodality: Support for handling images, video, and audio.
  • Multi-agent Workflows: Tools to orchestrate graph-based workflows or a swarm of sub-agents.

Scaling Agents with MCP and AWS Lambda

A significant challenge in production is moving beyond local agent architectures. The Model-Component-Protocol (MCP) initially focused on local tool connections via standard I/O. To operate at cloud scale, a decoupled, remote architecture is necessary.

A robust pattern for this on AWS is to deploy MCP servers as AWS Lambda functions using streamable HTTP. This architecture provides scalability, security, and integration with the broader cloud ecosystem.

A typical implementation involves:

1. MCP Server in Lambda: The core logic of the tool is encapsulated in a Lambda function. The open-source MCPLambdaHandler simplifies the setup.
2. API Gateway: An API Gateway is placed in front of the Lambda function to provide a stable, invokable HTTP endpoint.
3. Security and State: An authorizer (like a bearer token or AWS Cognito) is used to secure the endpoint. Session data can be persisted in a service like Amazon DynamoDB.

From the client-side, an agent built with Strands can then connect to this remote tool via its API Gateway URL, passing an authorization token and making the tool available for the agent’s tasks.

The Future: A Web of Interconnected Agents

This is just the beginning. The next frontier is enabling agents to communicate and collaborate with each other. AWS is actively contributing to open protocols like MCP to help build this future. Quoting Danielle’s talk from the previous day, “The atomic unit of all digital interactions will be an agent call.” We can envision a future where personal agents connect to an “agent store,” dynamically assembling the capabilities of other agents to accomplish tasks for users.

• Tags:
• AI Agents
• LLM
• Generative AI
• MCP
• Production AI
• Strands Agents
• AWS
• AWS Lambda