Why 70% of Companies Are FAILING at AI Safety (Shocking Survey Data): 2025 AI Governance Survey:

A recent AI Governance survey of over 350 respondents, primarily from US-based technology and healthcare companies, reveals key trends in AI adoption and the significant gaps that remain in governance and risk management.

AI Adoption: The Blurring Line Between Developer and Deployer

Approximately 30% of respondents report having at least one AI model in production. When asked to characterize their usage, the largest group—nearly 50% of technical leaders—identify as both a “developer” and a “deployer.”

This reflects a shift in what it means to be an AI developer. Today, it rarely involves pre-training a foundation model from scratch. Instead, development focuses on post-training activities:

• Supervised Fine-Tuning: Now largely considered a solved problem with many available services.
• Reinforcement Fine-Tuning: An emerging area with tools still in early stages.
• Contextual Enhancement: A growing realization is that providing models with additional context—through techniques like Retrieval-Augmented Generation (RAG), metadata, and access to external data sources—is often more impactful and simpler than fine-tuning. This approach avoids GPU investment for training and allows for real-time data updates.

Technical leaders are more aggressive in their deployment plans, aiming for 3-5 generative AI use cases in the next year, compared to the 1-2 planned by most other respondents. Small companies are the most conservative, with nearly 75% planning only 1-2 use cases, likely due to resource constraints.

Governance Frameworks and Regulatory Awareness

The survey highlights a significant disparity in governance maturity, particularly between large and small organizations.

• Policies and Staffing: Large companies are far more likely to have formal AI governance policies and dedicated staff (e.g., an AI Governance Officer) than their smaller counterparts. However, the existence of a policy doesn’t guarantee its substance; some are rudimentary, amounting to “don’t upload confidential data to ChatGPT.”
• Regulatory Knowledge: The NIST AI Risk Management Framework is the most recognized standard among respondents, especially technical leaders. David Talby notes its growing importance, suggesting it is on a path to becoming a de-facto legal standard for demonstrating “commercially reasonable” diligence, much like its predecessors in cybersecurity.
• Staying Current: The regulatory landscape is evolving at a breakneck pace, with over 700 state-level AI laws submitted and over 135 passed in the US in 2024 alone. Large companies are more likely to have a formal process for tracking these changes, but it’s a critical legal requirement for all organizations. A useful practice is to analyze regulatory actions, like the FTC’s letter to OpenAI, to build a “mind map” of the types of information and processes that require documentation.

Incident Response and Risk Management Practices

While over half of respondents claim to have an AI incident response playbook, the experts express skepticism about the depth of these plans. Small companies lag significantly, with only a third having such a playbook. A proper incident response plan is a multi-step process to minimize time to recovery when probabilistic models inevitably misbehave.

When asked about implemented risk management tools, the results were described as “alarmingly low”:

• Monitoring: This is the most common practice, with nearly 50% using it for accuracy, misuse, and drift.
• Risk Evaluation: The next largest group reported having a process for evaluating AI risk.
• Guardrails: Implementing input and output guardrails is a well-known design pattern to protect against PII leaks, proprietary data exposure, hallucinations, and sensitive topics. Custom, domain-specific guardrails (e.g., for healthcare ethics or financial regulations) are crucial for robust, real-world applications.

The speakers noted that if these low adoption numbers for monitoring and safety were applied to standard web development, it would be considered unacceptable. This suggests many AI projects are still in a pilot stage.

Barriers and the Path Forward

The primary barrier holding back AI governance adoption is speed to market, cited most often by technical leaders and large companies. For small companies, the main obstacle is a lack of budget.

The path forward involves moving away from fragmented tools and workflows managed in spreadsheets and Google Docs. The current friction between data science, legal, and compliance teams creates long delays. The vision is a unified platform that can:

1. Connect Roles: Provide a central environment for data scientists, product managers, legal, and compliance teams to collaborate.
2. Automate Best Practices: Embed frameworks like NIST into workflows, enabling teams to follow processes that ensure compliance without needing to be experts on the standards themselves.
3. Embed Industry Standards: Provide clear, standardized definitions and tools for complex tasks like bias testing and red teaming, establishing what “commercially reasonable” looks like in practice.

• Tags:
• Generative AI
• AI Safety
• AI Governance
• Feature
• AI Policy
• Risk Management
• NIST AI RMF
• LLM Operations
• Regulatory Compliance
• Model Monitoring
• Pacific AI
• The Data Exchange
• David Talby
• Ben Lorica