I Gave an AI Agent the Keys to My Life (Here's What Happened) — Radek Sienkiewicz, Velvet Shark

The journey of building a personal AI agent is not a single leap but a series of small, incremental steps. It begins with a simple chat interface and gradually expands, one permission and one workflow at a time. This gradual approach builds trust and resilience, preventing the kind of catastrophic failures that can occur with a “big bang” implementation. Over time, a simple setup can evolve into a highly sophisticated system that manages emails, files, calendars, and even the operating system itself.

The Knowledge Base as the Agent’s Brain

A significant turning point in the agent’s utility comes from granting it access to a comprehensive knowledge base. In this case, a personal Obsidian vault containing over 3,000 notes on work projects, personal tasks, research, and articles serves as the agent’s long-term memory and contextual brain.

This setup transforms the agent from a simple task executor into a powerful knowledge processor, akin to the advanced “LLM OS” concepts described by Andrej Karpathy. The agent can:

• Enrich New Information: When a link (tweet, article, video) is added to an “inbox,” the agent analyzes the content, adds relevant tags, and, most importantly, finds and creates connections to existing notes within the vault.
• Surface Forgotten Knowledge: This process turns a passive bookmarking habit into an active knowledge-building exercise. The agent resurfaces related past research and notes, providing immediate context and preventing valuable information from being forgotten.

The 24/7 Autonomous Ecosystem

The agent operates around the clock, performing critical maintenance and preparation tasks, particularly during off-hours (e.g., 3-6 a.m.). This “ambient operations” ecosystem ensures the system is always ready, updated, and resilient.

This is what is happening when I’m sleeping. So when I’m sleeping again, my agent does everything so that it runs well.

Key overnight tasks include:

• Backups: Regularly backing up its own memory and the knowledge base to minimize data loss.
• Indexing: Refreshing all vector indexes for Qdrant, memory files, and the Obsidian vault to ensure search is fast and accurate.
• Self-Updating: The agent can update its own Open-Claw instance, with custom scripts and verification steps to check for breaking changes and ensure it can restart successfully.

Core Agent Jobs and Workflows

The agent’s responsibilities are categorized into several key areas, managed through dedicated Discord channels:

1. Ambient Operations: The overnight self-maintenance and updating tasks.
2. Attention Filtering: The agent proactively monitors incoming information (like emails) and, using context from the knowledge base, alerts on urgent and important events. Real-world examples include notifications for a failed Netflix payment and an upcoming domain renewal, both of which might have been missed in a busy inbox.
3. Execution Support: For tasks with sufficient context, the agent can take proactive steps. For instance, it can read an email related to a known project, understand the query based on project notes in Obsidian, and draft a reply, placing it in the drafts folder for review.
4. Knowledge Synthesis: This involves processing the inbox, enriching the knowledge base, and building connections between notes.

System Architecture and Best Practices

This level of automation relies on a system of interconnected parts that work together:

• LLM for Judgment: The Large Language Model is used for understanding context, making connections, and handling ambiguity.
• Scripts for Determinism: Simple, deterministic tasks are handled by scripts, which are faster and more reliable, bypassing the LLM entirely.
• Inspectable Memory: A key advantage of using Open-Claw is that its memory and configuration are stored in human-readable Markdown files (soul.md, memory.md). This inspectability is crucial for debugging and refinement.
• Critical Rules: For non-negotiable instructions that the agent must always follow, a critical_rules.md file is used to prevent the agent from “forgetting” or ignoring core directives.

Challenges at Scale

As the system grows, several challenges emerge:

• Compounding Bad Memory: A poorly structured memory becomes increasingly problematic as the knowledge base scales to thousands of notes.
• Brittle Automations: Long, multi-step automations are fragile and likely to break. The solution is to decompose them into simpler, more robust parts.
• Noisy Notes & Weak Boundaries: The knowledge base requires regular cleaning and curation. The agent’s guiding files (soul.md, etc.) must be continuously optimized to maintain clear boundaries.

Philosophy: Optimizing for the Future You

The ultimate goal of this system is to assist your future self. By automating tasks, organizing knowledge, and preparing for the day ahead, the agent acts as a partner in making your future life easier. It’s about creating a system where you wake up and the necessary groundwork has already been laid.

The job for me is to to become friends with the future me… the job of the agent. So I don’t need to do as much as I used to because the agent just helps the future me as much as possible.

• Tags:
• LLM
• AI Agent
• Automation
• Knowledge Management
• Open-Claw
• Obsidian
• Cron Jobs