Andrew Burt of Luminos discusses how current AI risk evaluation methods are insufficient, advocating for a "high dimensionality" approach using granular sub-risks and diverse legal and technical expertise. He critiques common practices like guardrails and single-LLM evaluations, highlighting the need for multimodal systems and continuous, automated monitoring to address the evolving complexities of AI, particularly with the rise of open-weight models.
Jeff Crume explores why humanities are crucial in an AI-driven world. While AI generates sophisticated answers, it lacks human understanding, purpose, and judgment. He argues that STEM fields explain 'how' but not 'why,' making humanities essential for ethical decision-making, interpreting AI outputs, understanding bias, and effective prompt engineering. Ultimately, AI amplifies the need for human critical thinking and judgment.
SonderMind's AI mental health coach, Sonder, pioneers an eval-driven development approach balancing effectiveness and safety. This involves a clinical feedback loop turning human therapist insights into machine-readable evaluations, an Ethics Engine with modular, LLM-as-a-judge guardrails for evolving clinical guidelines, and a shift from single-prompt agents to a Supervisor/Executor/Evaluator architecture with human oversight to ensure safety and quality in high-stakes mental health conversations. They also open-source clinically reviewed datasets to foster community safety.
Joanna Bryson challenges popular AI assumptions, positing current generative AI as powerful tools for cultural knowledge compression, not autonomous intelligences. She emphasizes that AI's capabilities are nearing the human knowledge frontier, requiring focus on human coordination and governance. Bryson critically examines AI's impact on mental health and law, advocating for data-driven regulation and comprehensible systems. She calls for engineering activism, asserting human agency over technological determinism and stressing the importance of transparency and critical thinking in shaping AI's future.
The talk addresses the critical flaw in human-in-the-loop AI: humans often surrender cognitive effort, leading to "automation bias" rather than true discernment. Through a Duolingo English Test case study and various interaction design examples, it demonstrates how engineering the human-AI interface—rather than just the model or oversight—can elicit critical thinking, generate high-quality data, and foster a virtuous cycle of AI improvement.
Anjney Midha unpacks the critical bottlenecks in AI scaling beyond just GPU acquisition, advocating for responsible infrastructure, community-aligned data centers, and an independent system operator model for compute. He discusses the perils of research hoarding, the rise of researcher CEOs, and how Anthropic's culture of "preparedness" and "output maxing" led to its success, while also highlighting his personal mission to use AI for precise end-of-life prediction.