GeoMind: A Multi-Agent Framework for Geospatial Decision Support

During disaster scenarios, such as the 2024 Kenya floods, decision-makers need rapid answers to critical questions: “Which roads are blocked in Nairobi?”, “How many buildings are damaged in Westlands?”, or “What is the closest hospital to the most affected zone?”. Traditional Geographic Information System (GIS) workflows are complex and require specialized expertise, creating a bottleneck when time is critical.

GeoMind is an open-source, multi-agent framework developed to address this challenge. It allows non-experts to query complex, multi-layered geospatial datasets using natural language and receive comprehensive, visualized answers. The system is designed to handle the limitations of existing tools, which often struggle with multi-step spatial reasoning, complex data joins, and the potential for LLM hallucinations in multimodal geospatial settings.

System Architecture: A Multi-Agent Approach

GeoMind’s architecture is built around a central Orchestrator Agent that manages a team of specialized agents, each designed for a specific task. This division of labor allows the system to handle complex queries that require interacting with different types of data and performing multiple operations.

When a user submits a query, the Orchestrator is responsible for:

• Fact Extraction: Identifying key entities from the query, such as the disaster type, location, and points of interest.
• Planning: Decomposing the complex query into a sequence of smaller, actionable steps.
• Agent Selection & Coordination: Assigning each step to the appropriate specialized agent and managing the flow of information between them.
• Progress Monitoring & Error Handling: Tracking the execution of the plan, handling errors, and re-planning if the system gets stuck in a loop.

The specialized agents are divided into two categories based on the data they handle:

1. Vector Data Agents: These agents work with points, lines, and polygons.

   • Maps Agent: Fetches real-time infrastructure and boundary data (e.g., roads, buildings, neighborhood polygons) from sources like Azure Maps and OpenStreetMap.
   • Vector Agent: Interacts with a PostGIS database containing historical disaster data, such as building damage assessments produced by the AI for Good Lab. It translates natural language requests into complex SQL queries to perform spatial joins and filtering (e.g., intersecting a flood map with a layer of school locations).

2. Raster Data Agents: These agents work with pixel-based data like satellite imagery.

   • Stack Agent: Connects to the Microsoft Planetary Computer to search and retrieve relevant satellite imagery (e.g., pre- and post-disaster images) based on the user’s query.
   • Raster Agent: Performs calculations and band math on the retrieved imagery to compute indices like the Normalized Burn Ratio (NBR) for wildfire intensity or the Normalized Difference Vegetation Index (NDVI) to assess vegetation loss.

Technical Challenges and Solutions

Developing a robust multi-agent system for geospatial analysis presented several unique challenges:

• Data Transfer and Context Window Limits: Geospatial data is often large and noisy (e.g., long coordinate numbers), making it impractical to pass directly into an LLM’s context window.

  The solution is a file-path handoff mechanism. Instead of passing raw data, an agent saves its output (e.g., a boundary polygon) to a file and returns only the file path and a concise metadata summary (e.g., feature count, data schema) to the Orchestrator.

• Inter-Agent Communication Reliability: Relying on the Orchestrator LLM to correctly pass file paths between agents is fragile. The LLM might forget a required parameter or pass the wrong file.

  To solve this, GeoMind implements a shared working memory or “response store.” Each agent writes its output artifacts to this central location. Subsequent agents can then retrieve the necessary files directly from the store, reducing the risk of errors.

• Dynamic SQL Generation: An LLM cannot write a correct spatial SQL query if it can’t “see” the vector data within a file.

  The system uses placeholders in its prompts. The Vector Agent generates an SQL query using a placeholder (e.g., [path/to/boundary.geojson]). In the backend, a separate process reads the actual data from the file and injects it into the SQL query before executing it against the database.

Lessons Learned

• Keep Logic out of the LLM: The more you can offload complex logic and strict parameter handling to traditional code, the more reliable the system becomes. Trusting an LLM to consistently remember and pass all required parameters for a function call is risky.
• The Importance of the Planner: The performance of the entire system is highly dependent on the capability of the planning agent. Using a more powerful model like GPT-4o for the Orchestrator leads to a significantly higher success rate compared to less capable models, even if the specialized agents use the more powerful model.
• Safety and Evaluation: The system is designed with safety in mind. The Vector Agent is restricted to read-only SQL queries, and any Python code executed by the Raster Agent runs in a sandboxed environment. A comprehensive benchmark is also under development to evaluate the system across multiple dimensions, including plan correctness, tool usage, and the accuracy of the final results.

• Tags:
• Geospatial AI
• LLM
• Multi-Agent Systems
• Natural Language Processing
• Disaster Response
• GIS