How BlackRock Builds Custom Knowledge Apps at Scale — Vaibhav Page & Infant Vasanth, BlackRock

At BlackRock, the world’s largest asset manager, investment operations teams are the engine ensuring that the vast flow of information is processed efficiently, enabling portfolio managers to execute trades and manage portfolios. These teams require complex, domain-specific internal tools, and accelerating the development of these applications is of utmost importance. The rise of LLMs presents a significant opportunity to augment or supercharge these tools, which typically fall into four categories: document extraction, complex workflow automation, Q&A systems, and agentic systems.

Use Case: New Issue Operations

A representative use case is the New Issue Operations team, responsible for setting up new securities following market events like an IPO or a stock split. The process is complex and manual:

1. Ingest a prospectus or term sheet.
2. Push it through a data pipeline.
3. Consult domain experts (e.g., equity, ETF teams) to interpret the complex financial instruments.
4. Generate a structured output.
5. Work with engineering teams to build transformation logic and integrate with downstream systems.

This entire lifecycle for a single application could take anywhere from three to eight months, highlighting significant bottlenecks in the development process.

Core Challenges in Scaling AI Knowledge Apps

Building and scaling these AI-powered “Knowledge Apps” presents a distinct set of challenges:

• Prompt Engineering and Management: Domain experts spend an inordinate amount of time on prompt engineering. A simple prompt for a financial instrument can quickly evolve into a multi-paragraph description. This introduces challenges in iterating, versioning, comparing, and evaluating prompts against standardized datasets to measure performance.
• LLM Strategy Selection: There is no one-size-fits-all approach. For a single task like data extraction, the optimal strategy varies based on the document’s complexity and size. A simple corporate bond might be handled with an in-context learning model, but a 10,000-page document exceeds token limits, necessitating a different strategy like Retrieval-Augmented Generation (RAG) or a Chain of Thought approach. Developers must constantly experiment by mixing and matching prompts with various LLM strategies and models from different vendors.
• Deployment and Operationalization: Moving an application from prototype to production introduces another layer of complexity. Beyond traditional challenges like access control and distribution, AI apps require specialized infrastructure considerations. A batch job analyzing 500 research reports might need a GPU-based inference cluster, while a real-time security setup app might use a burstable CPU cluster. Managing these resources, controlling costs, and integrating everything into a seamless CI/CD pipeline is a major hurdle.

A Modular Framework for Accelerated Development

To address these challenges, BlackRock developed a modular, Kubernetes-native AI framework that compresses the app development lifecycle from months to a couple of days. The architecture’s core components are the Sandbox and the App Factory, which federate and simplify the most significant bottlenecks.

The Sandbox: A Playground for Domain Experts

The Sandbox is a UI-driven environment that empowers domain experts—the people who understand the data best—to rapidly build and refine the core logic of a knowledge app without deep engineering involvement. Its key features include:

• Extraction Template Management: Experts can define templates specifying the fields to be extracted from a document. For each field, they can configure the expected data type (e.g., string, number, date), the source (extracted via LLM or derived from other fields), and associated prompts.
• Advanced Configuration: The Sandbox goes beyond simple prompt management. It allows for sophisticated configurations like defining inter-field dependencies (e.g., if a bond is “callable,” the “call date” and “call price” fields become required). It also supports setting up QC checks, validations, and constraints on the extracted values.
• Low-Code Transformations: A crucial feature is the ability for operators to build transformation and execution workflows directly within the Sandbox. This replaces the manual process of downloading a CSV, applying transformations, and pushing it to downstream systems. Operators can build an end-to-end pipeline within the tool.

The App Factory: From Sandbox to Production

Once an application’s logic is refined in the Sandbox, the App Factory takes over. It is a cloud-native operator that ingests the definitions, extraction templates, and workflows created in the Sandbox. It then automatically spins up a production-grade, standalone application. This decouples the domain logic from the deployment process, allowing users of the final app to simply upload documents and run extractions without needing to understand the underlying configuration.

Key Takeaways

The journey of building this framework yielded several key insights:

1. Invest in Prompt Engineering for Domain Experts: The complexity of financial documents makes prompt engineering a critical skill. Empowering the experts who understand the nuances of these documents is essential.
2. Educate the Firm on LLM Strategies: Stakeholders need to understand that building an AI application involves choosing from a menu of strategies (RAG, Chain of Thought, etc.) and that the right choice depends entirely on the specific use case.
3. Prioritize Human-in-the-Loop Design: In a highly regulated industry like finance, full automation via agentic systems is often not feasible or desirable. The need for compliance and “four-eyes” checks makes a human-in-the-loop design paramount. The system should be designed to augment human operators, not replace them entirely.
4. Evaluate ROI Carefully: Before building a custom AI solution, it’s critical to evaluate whether the return on investment justifies the cost and complexity, especially compared to off-the-shelf products.

• Tags:
• LLM
• Automation
• Kubernetes
• Fintech
• AI Framework
• Investment Management
• Document Extraction