The Watchdogs of AGI — Rune Kvist of AI Underwriting Company

AIUC co-founder Rune Kvist discusses their $40M Series A, emphasizing that AI adoption's biggest constraint is trust, not capability. He details AIUC-1, their quarterly-updated standard for agent security and reliability, and their expansion into model certification to bridge the trust gap between frontier labs and governments. Kvist highlights the critical role of standards and insurance (partnering with Lloyds of London) in quantifying and mitigating AI risks, from agent liability to national security concerns, and preparing for future challenges in robotics and multi-agent systems.

September 16, 2026 4 min read

The Watchdogs of AGI — Rune Kvist of AI Underwriting Company — source: https://www.youtube.com/watch?v=Sc2_LfWgHb4

Rune Kvist, co-founder of AIUC (AI Underwriting Company), discusses the company’s recent $40M Series A funding round and its mission to build “confidence infrastructure” for frontier AI through robust standards and insurance. Having witnessed the challenges of AI deployment firsthand at Anthropic, Kvist argues that trust, not capability, will be the binding constraint on AI adoption. This perspective is underscored by examples like Waymo, which, despite superhuman driving capabilities, struggles with widespread deployment due to liability concerns.

AIUC-1: The Agent Security, Safety, and Reliability Standard

AIUC-1 is introduced as a comprehensive standard designed to address the concerns of security leaders in Fortune 1000 companies. Its core principles are:

Comprehensive Coverage: Addressing all major adoption blockers, including jailbreaks, hallucinations, and data leaks, across six categories.
Technical Testing: Grounding the standard in rigorous, simulated technical testing. This involves thousands of simulations to assess how easily an agent can be jailbroken, its hallucination frequency, and data leakage rates.
Quarterly Refresh: Acknowledging the rapid pace of AI development, the standard is updated quarterly. This contrasts with traditional decade-long cycles and is informed by the AS1 consortium, a group of Fortune 1000 risk leaders.

The certification process for companies typically takes 3 to 10 weeks, involving external auditors (e.g., KPMG, Shellman) to verify technical controls (e.g., existence of a groundedness filter) and AIUC’s own testing for effectiveness. The certification includes a detailed 100-page report, providing clarity and trust for enterprise adoption.

Transition to Model Certification and the AI Trust Gap

The conversation extends to the nascent but critical area of model-level certification, driven by incidents involving models like Mythos and Fable. Kvist highlights a growing “trust gap” between frontier AI labs and governments. Governments need assurance regarding risks (cybersecurity, child safety, biological weapons, national security), but lack the technical expertise and resources to keep pace with rapid AI innovation.

AIUC envisions itself as a neutral third party, akin to Moody’s in financial markets, bridging this gap by providing objective, rigorous technical evaluations and transparent risk information. This is crucial because labs, despite their expertise, face incentives that may not always align with full transparency or caution. The immense and expanding risk surface of frontier models necessitates a coherent framework that integrates diverse subject matter expertise into a single, inspectable standard.

The Interplay of Standards and Insurance

AIUC’s integrated approach of standards and insurance is rooted in historical parallels (electricity, cars, nuclear energy), where market-driven confidence infrastructure emerged to enable adoption. Insurance plays a vital role by:

Quantifying Risk: Insurers are incentivized to truthfully assess and quantify risk, driving improvements.
Providing Trust: An insurance policy signals that risks are manageable, offering assurance to enterprises. AIUC partners with entities like Lloyds of London, leveraging their capital and trust, while AIUC provides the technical underwriting framework and evaluation data.

Case studies like the Air Canada chatbot, which made legally binding promises despite hallucinating, demonstrate how courts are beginning to clarify AI liability. The $20 Cursor subscription potentially leading to a $200M plane crash scenario highlights the profound implications of AI failures and the need for clear liability frameworks. Copyright infringement, however, remains a challenging area for insurance due to the