How to build an AI-Native Health Company — Dan Feng, Maven Clinic

Maven Clinic’s Dan Feng shares insights into their two-year journey to becoming an “AI-native” company, emphasizing that AI adoption is no longer optional. He uses the analogy: “tractors aren’t to replace farmers, but the farmers who can operate the tractor will replace the ones who cannot,” highlighting the necessity for individuals and organizations to adapt.

Defining “AI-Native”

Feng clarifies that there’s no single definition or playbook. For Maven Clinic, it involves three core aspects:

• Internal AI Tooling: Leveraging AI for daily tasks like generating summaries, managing meetings, or creating Jira tasks, promoting individual autonomy and efficiency. Leaders, including sales, now use AI for tasks they previously delegated.
• External Product Integration: Building AI into products to enhance user experience and reduce operational costs. AI-powered chatbots are cited as an example, offering 24/7 availability and cost savings over human agents.
• Cultural and Process Transformation: Adapting the way teams work to maximize AI’s benefits.

Navigating AI Adoption and Talent

Maven Clinic approaches AI adoption by segmenting users:

• Early Adopters: Provided tools and encouraged to share learnings.
• The Majority: Supported with shared AI infrastructure and easy-to-use tools, requiring constant feedback integration (e.g., adapting to engineers switching from Cursor to Cloud Code).
• Reluctant Adopters: Their concerns are heard, but the company’s clear AI direction is communicated.

The shift impacts talent and roles:

• Hiring: Focus is on individuals genuinely interested in AI, continuous learning, and a deep understanding of the product. The line between product management and engineering blurs, as engineers with product insight can make greater contributions.
• Delegation: Senior engineers, now equipped with AI tools, prefer solving implementation tasks themselves rather than delegating, as AI makes it faster and more efficient.
• Performance Reviews: AI leverage is integrated into performance reviews, rewarding individuals who use AI to multiply their impact.

Rethinking Planning: When Building is Cheap, Arguing is Expensive

A fundamental change is the shift in planning philosophy:

• Traditional: Weeks or months spent on detailed requirements and design due to expensive implementation costs.
• AI Era: “Building is super fast. It’s probably a couple minutes you can get it done. Arguing is really expensive one.” This means quick iteration is key.
• One-Year View: Becomes purely directional and aspirational, assuming AI models can accomplish anything by then.
• Short-Term Commitment: Real commitments are reduced to two to four weeks (sprint-level). PMs and designers define immediate needs, and engineers focus on rapid delivery.
• Documentation: Long requirement documents are replaced by short, one-to-two-page documents designed for rapid iteration and discussion rather than static definition.
• Mid-Term Plans (3-6 months): These become “awkward” and “very hard to plan” because the rapid evolution of AI models makes capabilities unpredictable.

Evolving Software Development Practices with AI

AI coding tools are seen as the “most successful AI application” and are particularly good for implementation. Maven Clinic’s adoption strategy:

• Phased Introduction: Started with low-risk tasks like writing unit tests and documentation, building confidence and internal best practices.
• Team-wide Push: Once confidence was established, AI coding tools were pushed across the engineering team, with feedback sought from those not adopting them.
• Engineer’s Role: Engineers now focus on reviewing, architecting, and evaluating rather than primary implementation.

Adapting Code Review for Increased Velocity

The tenfold increase in code output (hundreds to thousands of lines daily) necessitates a new approach to code review:

• Current AI Review Tools: While helpful, they aren’t 100% reliable, and human feedback remains valuable.
• Self-Certification: Engineers can self-identify simple PRs that don’t need a second reviewer, but remain accountable for their quality.
• PR Size Limits: Pull requests are capped at around 500 lines of code to ensure meaningful review.
• Stacked PRs: Large features are broken into multiple, smaller PRs, allowing parallel work and review.
• Avoiding “Rubber Stamp” Approvals: This is deemed the “worst case” as it creates false confidence; genuine review is paramount.
• Future Vision: The goal is an end-to-end automated software development lifecycle, including design, implementation, release, and AI-driven monitoring and self-correction in production.

Ensuring Reliability in Generative AI Systems

Recognizing that hallucination is inherent in generative AI, Maven Clinic implements a holistic reliability strategy:

• Failure Tolerance: Distinguishing between acceptable and unacceptable failures.
  • Tolerable: A scheduling action failing 1 in 1,000 times is acceptable; users can re-click.
  • Intolerable: Reimbursement claims have zero tolerance. If a user requests $50 and receives $200 (or vice versa), it’s an immediate escalation risk.
• Critical Task Safeguards: For high-stakes tasks like reimbursement claims, multiple models review the same input (e.g., receipt), and the process only proceeds if their results agree. If ambiguity persists, the system defaults to human escalation.
• Rigorous Testing: Hundreds of integration tests cover known use cases.
• Non-Deterministic System Testing: For LLM-based features, tests are run “many times” rather than once, requiring a consistent high pass rate (e.g., 90%) due to inherent non-determinism.
• Post-Launch Evaluation:
  • Automated Evaluation: An “auto-evolve” system evaluates conversations against predefined rubrics (good/bad), generating scores for review.
  • Human Spot Checks: A dedicated team conducts spot checks of conversations, adjusting rubrics or system improvements as needed. For new features, a higher percentage (e.g., 20%) of reviews are mandated to build confidence.

This comprehensive approach allows Maven Clinic to release features with confidence despite the known challenges of generative AI.

• Tags:
• Software Engineering
• Product Management
• LLM
• Generative AI
• Feature
• Reliability
• Machine Learning
• AI Native
• Code Review
• Engineering Culture
• Organizational Change
• Agile Development
• Maven Clinic