Arvind Jain on building Glean and the future of enterprise AI

Arvind Jain, founder of Glean and previously Rubrik, explains that Glean’s mission is to be the “chat GPD, but inside your company.” The platform connects to all of a company’s internal systems and knowledge, allowing employees to ask questions and receive answers synthesized from both internal context and the world’s knowledge. The same platform is evolving to allow the creation of custom AI agents for various enterprise workflows.

The Genesis of Glean: A Contrarian Bet on Enterprise Search

Glean was founded in early 2019, before “generative AI” was a common term. The initial focus was on solving the universal problem of enterprise search, where employees spend a significant amount of time just trying to find information scattered across hundreds of different systems.

While LLMs were not yet a focus, transformer models were. The Glean team saw the immediate impact of models like BERT on core search components. They were one of the first companies to use transformers in the enterprise, recognizing their power for conceptual understanding beyond simple keyword matching.

“We could conceptually understand information… we didn’t have to actually go and build a keyword based search system anymore which is sort of like quite dumb.”

This allowed them to pre-train what are now called “small language models” on an enterprise’s specific corpus. This gave the system a deep, semantic understanding of the business’s unique lingo and concepts, enabling powerful matching between user questions and relevant documents. As model capabilities evolved to include generation and reasoning, Glean’s product naturally evolved from a “Google for your work life” to a “ChatGPT for your work life.”

A Hybrid Model Strategy: When to Build, Fine-Tune, and Buy

Glean’s philosophy is to avoid reinventing the wheel and to maximally use the innovation happening in the broader LLM space.

• Off-the-Shelf LLMs: For high-level reasoning and answer synthesis, Glean uses large, out-of-the-box models from providers like OpenAI and Google. When a user asks a question, Glean’s system first retrieves the relevant documents and then passes that context to a large model to generate a synthesized answer.

• Custom-Trained Small Models: For the core task of semantic matching, Glean trains custom models on each customer’s enterprise corpus. These models create bespoke embeddings that understand the company’s specific vocabulary and concepts, leading to superior retrieval quality.

• Fine-Tuned Task-Specific Models: For smaller, specific tasks within the search stack—such as spell-checking or acronym expansion—Glean fine-tunes small, open-source models. While a large model like GPT could perform these tasks, using a small, fine-tuned model is significantly faster and more cost-effective.

Solving Core Enterprise Challenges: Security and Hallucinations

1. Security and Permissions: Enterprise AI must be safe and secure. A core part of Glean’s technology is its ability to understand and enforce data governance. As Glean connects to systems like Google Drive or Slack, it indexes not only the content but also the permission models for every single document. These permissions are baked directly into the core indexing system.

When a user performs a search, the retrieval step of the RAG (Retrieval-Augmented Generation) system only surfaces information that the specific user is authorized to see. This ensures that the AI never leaks sensitive information to unauthorized employees. For training custom embedding models, Glean uses a carefully curated, safe subset of the enterprise corpus to prevent any potential information leakage through the model itself.

2. Suppressing Hallucinations: Glean mitigates hallucinations with a strong emphasis on citations and reference checking. After a model generates an answer based on the provided source documents, Glean’s system verifies every statement in the response against the original input.

“Line by line you know we’ll take every… response of the model and we’ll try to see like you know if we can actually find you know that same piece of information and you know in the input that we gave to the model.”

This allows Glean to provide line-by-line citations, grounding the AI’s output in verifiable, human-generated information. If a piece of the generated text cannot be traced back to the source material, it is either suppressed or shown without a citation, signaling to the user that it may be less reliable.

The Future of Work: AI-Powered Teams and Evergreen Knowledge

Jain envisions a future where every employee, not just senior executives, has a “dream team” of AI assistants, coaches, and coworkers. This will democratize productivity and transform how work is done. Two powerful examples of this in practice are:

• Evergreen Documentation: A user can ask Glean for an up-to-date tutorial on an internal system. The AI agent will first read the original, often stale, design documents. It then goes further by analyzing recent JIRA tickets and code commits related to that system to synthesize a completely current and accurate explanation. This solves the massive problem of documentation becoming obsolete.

• Complex Business Analysis: A finance team member, without being an engineer, was able to ask Glean to perform a customer churn risk analysis. The agent was instructed to look at every customer, analyze sentiment in shared Slack channels, review product usage dashboards, and combine these structured and unstructured signals to create a comprehensive risk profile for each account. This produced a richer analysis than a traditional dashboard could.

This highlights AI’s ability to unlock insights from the 95% of enterprise data that is unstructured.

Measuring Progress and the Human Element

To measure model and system improvements, Glean creates “golden sets” of question-and-answer pairs for each customer. These can be automatically derived from real-world interactions, such as a question asked in Slack and the highly-rated answer it received. Any new model or system change is evaluated against this golden set to ensure it’s improving answer quality.

Jain concludes that AI should be viewed as a force multiplier, not a replacement for humans. The goal is not to shrink teams but to empower every individual to be more effective and productive, allowing the business to succeed in a much bigger way. The fundamental skills for success—hard work, dedication, and a desire to succeed—remain unchanged.

• Tags:
• AI Agents
• RAG
• Fine-Tuning
• Semantic Search
• Enterprise AI
• LLM Security
• Transformer Models
• Knowledge Management