Underwriting Assist - A Multi Agent System // Somya Rai | Maria Zhang // Agents in Production 2025

Two leaders in applied AI, Somya Rai of EXL and Maria Zhang of Palona AI, share their experiences building and deploying sophisticated multi-agent systems for the insurance and restaurant industries, respectively. They offer a deep dive into architecture, scaling strategies, memory management, and cost optimization for real-world applications.

The Underwriting Assist Multi-Agent System

Somya Rai details EXL’s “Underwriting Assist” system, designed to tackle the new business submission process in B2B insurance. The traditional process was manual, requiring underwriters to conduct physical surveys, check sanction databases (like LexisNexis and OFAC), and assess risks, a workflow that could take weeks.

The multi-agent system automates this, reducing the turnaround time from weeks to just two days. The architecture consists of several specialized agents:

• Ingestion Agent: Parses incoming emails and attachments (multi-page documents, images, coordinates), identifies missing information, and communicates with brokers to retrieve it.
• Checker Agents: Once data is complete, these agents run in parallel to perform specific validation tasks, such as checking for sanctions against the company or individuals involved.
• Risk Profiler Agent: Analyzes the company’s risk profile, location (using geospatial tools), and business category. It compares this data against historical underwriting data for similar organizations to inform pricing and decision-making.

A crucial component is the Human-in-the-Loop (HITL) mechanism at every step. This ensures accuracy and provides the necessary audit trails and explainability required for regulatory compliance, justifying decisions like declining a bid or generating a specific quote.

Agent Architecture and Verticalization

Both speakers agree on the importance of vertical-specific solutions. As Maria Zhang notes, “vertical per industry is really important because there’s just a lot of nuances and expertise and…the rest of the ecosystem you need to kind of interface with.”

Their architectural approaches highlight a key design principle: structure agents by task, not by platform.

• Palona AI uses a single, unified ordering agent that can be accessed through various front-ends (voice call, text, WhatsApp, web widget). This avoids the redundancy of building separate agents for each platform.
• EXL learned that overloading a single agent with too many tools leads to failure points and hallucinations. They initially built a single agent for “validation and checking” but found that splitting them into distinct agents with specific tasks resulted in a more reliable system.

Scaling, Interfaces, and Technical Stack

A paradigm shift in scaling agentic systems is that the bottleneck is often not request volume but the complexity of integrations. The challenge lies in making the system “reusable and replicable” across different clients, each with unique interfaces.

To solve this, Somya’s team leverages a modern stack:

• LangGraph: Used as the core framework for building the agentic system, valued for its robust state and session management. This is critical for HITL workflows, allowing a session to be paused and resumed without losing context.
• NVIDIA’s AI Platform: The entire LangGraph system is packaged and served using an NVIDIA toolkit. This allows them to use a single configuration file to expose the agents through multiple interfaces (REST API, WebSockets, gRPC) without rewriting the core logic, greatly enhancing scalability and replicability.
• Ray: Used for optimizing the data ingestion pipeline. By using Ray for distributed, multi-threaded extraction from incoming documents, they significantly reduce GPU runtime and, consequently, compute costs.

Maria adds that real-world physical constraints, like the throughput of a kitchen’s ticket printer, are often the true bottlenecks, not the AI system itself.

Advanced Memory Management

Effective memory management is critical for creating contextual and intelligent agent interactions.

• Short-Term vs. Long-Term Memory: Maria introduces a clear distinction.
  • Short-Term (In-Context) Memory: Manages the immediate conversation. She advises against sharing this broadly across agents to prevent contextual confusion.
  • Long-Term (Persistent) Memory: Stores crucial, lasting information, like a customer’s food allergy. Palona AI has an internal service to intelligently determine which pieces of a conversation should be promoted to long-term memory, which is then shared across agents.
• State Management for Compliance: In regulated fields like insurance, memory is also about auditability. Somya’s team uses a two-level system:
  • A “working memory” (cache) for active sessions.
  • A persistent database (DynamoDB) to save session states if a user times out, allowing them to pick up exactly where they left off.
  • Kafka is used for event-based, schema-controlled communication between agents, ensuring data integrity and creating a complete, traceable record of how a decision was reached.

A Holistic Approach to Cost Optimization

While performance is the initial focus, cost becomes a factor at scale.

• Holistic Cost View: Maria advises looking beyond pure compute costs to the total cost of business, including payment processing and sales. She notes that compute costs are dropping drastically, so building architectural flexibility is more valuable than premature optimization. Her strategies include a multi-model architecture to swap cheaper models for simpler tasks and a provider-agnostic setup to leverage competitive pricing.
• Optimizing Runtime: Somya’s team focuses on reducing variable costs by minimizing GPU runtime. Using Ray for parallelized data processing and Triton Inference Server for efficient model serving are key strategies to control compute expenses.

• Tags:
• LLM
• AI
• Feature
• Multi-Agent Systems
• Memory Management
• Cost Optimization
• LangChain
• Ray
• Insurance