Databricks CEO: Stop Scaring People About AI

Ali Ghodsi, co-founder and CEO of Databricks, joins Martin Casado and Sarah Wang to dissect critical topics in AI, challenging prevailing narratives around risk, adoption, and economic impact. He emphasizes the responsibility of leaders not to cause undue public alarm about existential AI risks, which he considers “close to zero” at present, arguing that such fear-mongering can lead to counterproductive federal intervention and public hysteria.

The “Pacing” Debate and Real AI Risks

Ghodsi is critical of the “pacing the frontier” discourse, calling it a PR misstep. He argues that “pacing” is orthogonal to genuine safety and security concerns, as companies continue to invest heavily in compute to accelerate development. He contrasts this with the real and immediate threat of AI-powered cyberattacks. The time from a vulnerability’s discovery to its weaponization has drastically decreased from years to mere hours. Given the vast, interconnected, and often insecure global infrastructure, AI agents could rapidly find and exploit vulnerabilities, spreading like viruses. He views this as a significant engineering challenge requiring automated cybersecurity solutions, as human teams cannot keep pace with the volume and speed of attacks.

Recursive Self-Improvement (RSI): A Deeper Look

Ali Ghodsi provides a rigorous definition for genuinely concerning recursive self-improvement (RSI), distinct from common “autocatalytic” effects (e.g., using AI to write code for AI components). True RSI, in his view, would entail all four of the following conditions simultaneously:

1. Reduced Resource Consumption: The next model requires significantly less computational resources (e.g., GPUs) to train.
2. Faster Training Time: The next model takes less time to train.
3. Increased Intelligence: The model’s intelligence or accuracy measurably improves.
4. Recursive Applicability: These improvements can be consistently replicated, leading to an accelerating cycle of advancement.

He argues that current frontier models exhibit the opposite trend, demanding more resources, human involvement, and becoming more brittle to train. He suggests that if labs are truly seeing these four conditions, they should share data for transparency. He believes that without such a dramatic and self-sustaining acceleration, the concern about superintelligence remains distant.

Industry Self-Policing vs. Federal Involvement

The conversation touches on the mechanisms for oversight. Ghodsi is skeptical of mutual “peer review” among highly competitive AI labs due to inherent biases and vested interests (e.g., IPOs). He advocates for independent, diverse inspectors to assess AI development, providing examples like Yann LeCun as a potentially credible figure whose changed perspective would carry significant weight. He warns that if AI companies themselves highlight existential risks, it makes federal regulation inevitable, suggesting the current period is a delicate balance before potential heavy-handed intervention.

Unlocking Enterprise AI Value: The “Context” Problem and Ontology

Ghodsi contends that today’s AI models are already powerful enough to automate far more work than enterprises currently leverage. The primary bottleneck isn’t model intelligence but the lack of organizational context. Models haven’t been in meetings, don’t understand internal decision-making, or possess the implicit institutional knowledge held by experienced employees. To bridge this gap, he proposes building an “ontology”: a digital, indexed graph representing all abstract concepts, relationships, departments, people, and projects within an organization. This ontology, similar to Google’s PageRank for the web, allows AI agents to efficiently access and synthesize internal knowledge, accelerating decision-making and automating complex tasks.

Databricks has implemented such an ontology internally (dubbed “Genie”), leading to massive productivity gains. He illustrates this with an anecdote where even the CFO uses “Genie” to answer board-level questions instantly, rather than relying on human employees. This approach streamlines information flow and decision-making within the organization, drastically reducing the time needed to gain insights.

Real-World AI Use Cases and Cost Management

Ghodsi highlights several impactful AI use cases beyond simple chatbots:

• Crisis Text Line: Using LLMs to detect self-harm intent in teenagers.
• Omnipod: AI-driven insulin delivery for diabetes patients.
• Zipline: Automated drone delivery of medical supplies (e.g., blood to refugees).
• Merck’s “Teddy”: A transformer-based model for drug discovery, predicting gene regulatory networks to reduce R&D costs.

These examples underscore the significant societal benefits of AI, which Ghodsi argues should not be “paced.” He also addresses the exploding costs of AI usage in enterprises. Databricks has developed solutions like its Unity Gateway and Omnient harness to manage these costs. The Gateway allows for budget constraints, smart routing to cheaper models for simpler tasks, and comprehensive analytics. The Omnient harness can multiplex between different models and even different harnesses, revealing that harness choice can yield a 2x cost difference for the same model. This leads to a paradigm shift towards using a multitude of models—combining powerful frontier models for complex tasks with more efficient, often open-source, models for mundane or specific applications. This strategy has helped Databricks keep AI costs stagnant despite increasing token usage.

Agents and Infrastructure Evolution

The discussion concludes with the evolution of infrastructure to support AI agents. Ghodsi points to Neon (Lakebase) as a prime example of a database optimized for agents. Neon’s success stems from its obsessive focus on agent needs: highly elastic, responsive, and lightweight databases that can spin up quickly, clone instantly, and offer branching capabilities. Its pricing model is also tailored for agent experimentation rather than traditional production use. This represents a new focus in database design, prioritizing agent-centric features over traditional DBA or application developer needs, leading to over 90% of Neon databases being created by agents, not humans.

• Tags:
• Cybersecurity
• AI Safety
• Enterprise AI
• Databricks
• AI Adoption
• Agentic Systems
• AI Regulation
• Recursive Self-Improvement
• LLM Costs
• Organizational Ontology