LLM Knowledge Bases: a practical guide — Ben Holmes, Warp

Ben Holmes outlines a personal knowledge management system leveraging LLMs and automation to transform raw voice-dictated notes into an organized, browsable wiki and visual graph. The process emphasizes rapid, "sloppy" capture, followed by AI agents enriching notes with tags, web research, and backlinks, then generating structured wikis based on Karpathy's methodology, all automated via cloud sandboxes and Obsidian's headless CLI.

August 12, 2026 4 min read

LLM Knowledge Bases: a practical guide — Ben Holmes, Warp — source: https://www.youtube.com/watch?v=I3bpdgFJCUY

Ben Holmes presents an LLM-powered knowledge base pipeline designed to transform disorganized personal notes into a structured, browsable, and interconnected wiki. The core philosophy prioritizes raw thought capture over initial organization, acknowledging that “talking runs about 200 words a minute and typing does not”.

Capture: Getting Raw Thoughts Down

The pipeline begins with Capture, emphasizing voice dictation as the fastest input method. Holmes recommends local, on-device tools to avoid subscriptions, such as Handy (open-source) and Voice Ink (lifetime fee, mobile app, hotkey support). The initial input step is deliberately unformatted and rambling (e.g., after a podcast or meeting) to ensure sufficient “raw material” for subsequent processing. Markdown is used for notes.

Enrichment: Structuring with AI Agents

Once raw notes are captured, Enrichment occurs in passes using AI agents. A custom enrich note skill performs several actions:

Timestamping: Adds a timestamp to each file, indicating processing status and preventing redundant runs.
Tagging: Agents assign tags from a fixed reference list. This is crucial to prevent LLMs (specifically Claude is mentioned) from “inventing new tags” and ensuring consistency. Agents are explicitly instructed to be “reluctant to add new tags.”
Web Research: Agents use web tools (e.g., Exa) to research the source of the note, pulling down URLs and relevant context.
Backlinking: The skill identifies and adds backlinks by performing key term searches across the notes, creating an interconnected web of ideas. An example of an enriched note for a podcast about Ferrari’s founding is shown, featuring tags, source URL, enrichment timestamp, and related backlinks.

Wiki Generation: Browsable Knowledge

The enriched notes then form the basis for Wiki Generation:

Karpathy’s Gist: The idea for generating wikis is directly inspired by an “Andre Karpathy gist,” which outlines taking a raw directory of notes and combining it into a focused wiki.
Structured Wikis: These wikis automatically categorize entries by “people, concepts, organizations, and sources” relevant to a specific interest area (e.g., “latest in AI news,” “Bible in a year” podcast). This provides a browsable interface, akin to a “Wikipedia rabbit hole of your own thoughts,” making it easier to navigate connections and recall information. Use cases include tracking meeting notes, client interactions, or research topics.

Automation: A Self-Refreshing Knowledge Base

To ensure continuous organization, the entire process is Automated:

Scheduled Runs: The pipeline runs on a schedule (daily/weekly) in a cloud sandbox (e.g., Oz.dev).
Obsidian Headless CLI: This tool is used to sync markdown notes from a local machine to the cloud sandbox. Alternatively, a simple Git clone to a GitHub folder could work.
Agent Execution: Inside the sandbox, the agent executes the enrich note skill (and other wiki update skills) on any unprocessed notes.
Sync Back: After processing, the Obsidian CLI syncs the updated notes back to the local machine, so the user “wakes up to a fresh wiki” – described as “the daily paper, but it’s your own.”

Visualization: Mapping Your Thoughts

Finally, Visualization provides overarching insights:

Agent-Generated Graph View: An agent can be prompted to build an HTML and Tailwind-based graph view of the entire notebook. This offers a “bird’s eye view” of all notes, revealing common patterns, areas of interest, and crucially, “gaps in your own thinking.”
Interactive Exploration: The nodes in the graph are clickable, allowing users to drill down into specific notes and understand their connections. The visualization is highly customizable (e.g., “in space” theme, burndown charts for note-taking habits).

Key Tools Mentioned

Warp: Developer relations lead’s company, a terminal, and cloud platform.
Hubble (hub.md): The speaker’s own free and open-source notes app, designed to be “agent accessible.”
Oz.dev: A platform for setting up automated schedules and cloud sandboxes.
Handy, Voice Ink: Local voice dictation tools.
Exa: A web search tool for agents.
Obsidian Headless CLI: For syncing markdown notes.

This approach leverages LLMs to offload the burden of organizing and connecting information, turning raw input into a valuable, personal knowledge system.