The End of Ad-Hoc BI Dashboards
Nick Schrock, CTO of Dagster, introduces Compass, a Slack-native tool for collaborative, exploratory data analysis, and discusses the rising importance of 'context engineering' as the new data pipeline in the AI era.
Orchestration in the Age of AI
In the current AI landscape, orchestration is bifurcating into two distinct areas. The first is dynamic, agentic orchestration designed for workflows in the critical path of user interactions, a domain Dagster does not address. The second, Dagster’s focus, is the orchestration of data pipelines and platforms, which is foundational to any enterprise AI strategy. Without structured, semantically understood data, the utility of AI and ML is severely limited. The data platform has effectively become the AI platform.
This is evident not just in model training, where customers like Poolside use Dagster to orchestrate their entire platform from ingest to training, but also in the application of AI. The management of context fed into LLMs is now a critical discipline.
Context Engineering: The New Data Pipeline
“Context engineering” has emerged as a crucial concept, framing the process of preparing and feeding information to AI agents as a data pipelining problem. Context is often expensive to compute, and its delivery requires precise scheduling, quality checks, and evaluation (e.g., LLM-as-eval). These are fundamentally data pipeline challenges, placing the orchestrator at the heart of modern AI applications.
Introducing Compass for Exploratory Data Analysis
Compass was born from the observation that data platform teams are often viewed as cost centers rather than value drivers by the business stakeholders they serve. The goal of Compass is to bridge this gap by empowering business users to interact with data directly. It is not a traditional Business Intelligence (BI) tool but rather a tool for exploratory data analysis (EDA).
The thesis is that a significant portion—perhaps 80%—of BI dashboards are created for ad-hoc, exploratory purposes and are rarely used after their initial creation. Compass aims to replace this inefficient workflow with a collaborative, conversational experience.
The Problem with Semantic Layers and the “Context Store” Solution
While the need for a semantic layer to provide context to LLMs for text-to-SQL is well-understood, adoption has been slow. Traditional semantic layers require a significant upfront investment to define metrics, measures, and metadata, and are often managed through bespoke tooling that is difficult for business stakeholders to use.
Compass introduces the concept of a Context Store as a more practical, lightweight solution:
- Collaborative and Governed: The Context Store is integrated directly into Slack. When a business user, who has the domain knowledge, identifies an error or ambiguity in the AI’s response, they can provide corrective feedback directly in the chat.
- Git-based Workflow: This feedback is submitted as a pull request to a Git repository that houses the Context Store. The data team can then review, approve, and merge the change, ensuring governance while treating context as code.
- Incremental Improvement: This workflow allows the semantic layer to be built accretively over time, solving the cold-start problem and fostering a sense of ownership within the data team. They transition from being reactive SQL writers to proactive “context engineers.”
How Compass Works
Compass is a Slack-native, agentic system designed for a “multiplayer” chat experience.
- User Experience: The primary interface is Slack, chosen because it’s where users already collaborate. This avoids the friction of adopting another web application. The channel abstraction is used to separate contexts (e.g., a marketing channel and a sales channel can point to different tables and even different context stores, as the same terms can have different meanings).
- Transparency and Trust: The agent is transparent about its process, stating what it’s searching for and how it’s constructing a query. This builds trust and teaches users how to guide the system effectively. The tool is designed to be informal and avoids overpromising precision.
- Agentic DataOps: If a user’s query requires data that isn’t available in the warehouse, Compass can automatically generate a detailed data request in a ticketing system. This request includes the full context of the user’s query, providing the data team with a high-quality starting point for building a new data pipeline in a tool like Dagster.
AI-Accelerated Software Engineering
The impact of AI on software development is transformative. Tools like Claude Code have fundamentally changed the development workflow, with a majority of code now being AI-generated, shifting the developer’s role towards review and integration. This is more accurately described as “AI-accelerated software engineering.”
This shift presents a new paradigm for career progression. Junior engineers can now onboard and learn complex codebases much faster by directly querying an AI about the system’s history and logic. Companies that are reducing junior hiring out of a belief that AI can replace them are making a strategic error. Younger engineers, being native to these tools, are learning machines who will quickly become highly proficient, representing a significant opportunity for companies that continue to invest in them.