The Missing Piece in the AI for BI Puzzle
Yoni Leitersdorf, CEO of Solid, explains that directly applying Large Language Models (LLMs) to databases for text-to-SQL fails due to a lack of business context. He introduces the concept of a semantic layer as a critical "Rosetta Stone" that translates raw data into a meaningful format AI can understand, enabling reliable and accurate data interaction.
The Limitations of Simple Text-to-SQL
While modern foundation models have become proficient at generating SQL code, they cannot reliably interact with enterprise data warehouses on their own. The primary challenge is the lack of context. An LLM pointed at a database with multiple tables referring to “revenue” will simply guess which one to use. It doesn’t understand the business-specific acronyms (e.g., FSU in a column name, which it might mistake for “Florida State University”), internal terminology, or the nuances of how metrics are calculated. To overcome this, the model requires significant guidance in the form of both technical and business context, which is where the semantic layer becomes essential.
The Semantic Layer: A “Rosetta Stone” for AI
The semantic layer acts as a translation layer, providing the AI with the semantic understanding—the actual meaning—of the data. It serves as a “Rosetta Stone” that connects business terminology and questions to the underlying data structures. This layer addresses several key areas:
- Business Context: It defines what the data means, what business questions it can answer, and clarifies company-specific acronyms and terms.
- Data Quality: It helps the AI distinguish between high-quality, trustworthy data (the “gold layer”) and lower-quality or raw data (the “bronze” and “silver” layers). In enterprise environments, data quality is a significant problem, as responsibility for data is often diffuse and ownership changes, unlike curated content on the public internet.
- Trustworthiness: It provides signals to the AI about which data assets are reliable. Running AI against anything but the highest quality data leads to mistakes and erodes user trust.
Automating the Creation of the Semantic Layer
Building a semantic layer is not a “boil the ocean” project. The process starts by focusing on narrow, specific use cases within a particular domain (e.g., sales or marketing). The goal is to automate its construction by ingesting signals from a wide array of sources, creating a comprehensive understanding of the data landscape.
Key inputs for building the semantic layer include:
- Data Warehouse & BI Tools: Query logs and metadata from platforms like Snowflake and Databricks, as well as BI dashboards (e.g., a Tableau dashboard reviewed daily by the CEO is a strong signal of trust).
- Transformation Tools: Metadata from dbt repositories, including code and models.
- Collaboration & Project Management: Discussions and context from Jira tickets and Slack/Teams channels related to data projects.
- The “Social Graph”: Understanding who is querying the data. Queries from known star analysts or people in specific roles (e.g., a sales leader’s queries about sales data) are weighted more heavily.
This multi-source approach allows the AI to build a rich, interconnected graph of knowledge, determining which tables, columns, and sample queries are most relevant and trustworthy.
Implementation: A Gradual Process, Not a Turnkey Solution
Deploying an AI-for-BI solution is not an instant, turnkey process. Any vendor promising 100% accuracy on day one is setting incorrect expectations. Realistically, AI can achieve 75-85% accuracy on its own, with a human-in-the-loop required to validate results and handle the remaining complexity.
The deployment process is staged to build trust:
- Initial Warm-up (2 weeks): The system ingests historical data from various sources (query logs, Slack history, Jira tickets).
- Staged Rollout (6-8 weeks): The solution is first introduced to power users like data engineers and data modelers. They validate the system and help refine the semantic layer.
- Wider Release: Once trust is established with the technical teams, the tool is rolled out to analysts and, eventually, business stakeholders who cannot write SQL.
This gradual process is crucial because business users will quickly abandon the tool if they receive incorrect answers.
From Data Discovery to Direct Answers
The rise of chat-based interfaces marks a significant shift from the previous era of data catalogs. Data catalogs often failed to gain traction because they were a “push” from data teams, required tedious manual documentation that humans dislike, and forced users into another interface.
Today, the dynamic has inverted. Business users are now actively “pulling” for a “chat with your data” experience, creating a strong business case for investing in the underlying infrastructure, like a semantic layer. This new interface paradigm renders traditional data discovery obsolete for many use cases; users no longer want to find the right dataset—they want a direct answer to their question.
The Future: Autonomous Agents
The semantic layer is a foundational component for a future driven by autonomous agents. The vision is to move beyond reactive querying to proactive, agentic workflows. For example, an AI agent could be assigned to a marketing campaign, where it would continuously:
- Monitor the campaign’s performance by pulling and analyzing data.
- Identify areas for improvement.
- Propose specific adjustments to the campaign.
- Implement the changes upon approval from a human.
This represents the ultimate goal: leveraging a deep, contextual understanding of data to automate complex decision-making processes and drive business outcomes.