Are AI labs ignoring cybersecurity experts?
This podcast explores the friction between frontier AI labs and cybersecurity experts, who disagree on AI safety priorities (alignment vs. traditional controls). It discusses the dual risks of open-weight AI models enabling sophisticated hacks like the TikTok camera exploit, and the challenges of CISA's call for transparent breach disclosures amidst corporate self-interest and the complexities of global AI regulation.
The discussion centers on the growing chasm between frontier AI labs, primarily focused on AI alignment and existential risks, and seasoned cybersecurity professionals who feel excluded from critical safety conversations. Cyber experts, including Jeff Crume, Nikki Robinson, and Omari Jones, contend that many AI safety concerns are fundamentally cybersecurity problems that can be addressed with established security controls and practices.
The AI Cybersecurity Shutout: Alignment vs. Controls
Panelists challenge the notion of AI “doomsday scenarios” as often overlooking decades of cyber security infrastructure. Figures like Sier Martin from the UK’s NCSC note that warnings often “assume no monitoring of systems, no antivirus, no DOS protection, no network segmentation, no incident management, no nothing of any kind of the cyber security on the global internet of the type that has developed over the last 30 years.”
Nikki Robinson highlights a recurring “security second” pattern in technology adoption, where security is addressed only after issues arise. She argues that many perceived “new” AI security challenges—such as improper permissions or data disclosure—are merely existing cybersecurity issues rebranded for AI contexts. Jeff Crume reinforces this, noting that “Long established concepts that have been around for decades and even centuries like audit, inspections, accounting, evaluation, and safety reports do not have to be rethought from first principles just because it’s AI.”
The conversation also critiques the effectiveness of current containment strategies for frontier models. Jeff Crume emphasizes the necessity of robust containment, including genuine air-gapping (physical removal of network components, not just turning off Wi-Fi), especially for untested models. He compares it to a “vault within a vault” for handling highly dangerous entities, asserting that current approaches are inadequate given AI’s known ability to find zero-days. Omari Jones adds that while third-party audits are beneficial, they shouldn’t replace crucial pre-deployment security expertise and hygiene.
A key point of tension is the focus on “alignment” by AI labs versus “controls” by cyber experts. Nikki Robinson suggests this is partly a communication issue, where different terminology obscures shared underlying principles. Jeff Crume advocates for a “both/and” approach, comparing AI alignment to human policies and training, which still require technical controls. Omari Jones expresses concern that alignment, in practice, often means aligning models for “efficiency as possible and thus generating revenue for the business without necessarily contemplating the overall downstream impact,” leading to “alignment in the wrong direction.”
Open Models and the Risk Landscape: The TikTok Hack
The discussion transitions to the risks posed by open-weight AI models, exemplified by a TikTok camera hack where researchers used a modified open-source model (ZI’s GLM) to exploit vulnerabilities. This highlights a dichotomy: frontier labs restrict access, while open-weight counterparts are more permissive.
Jeff Crume reiterates his earlier prediction that controlling access to powerful AI models is futile. He points out that if proprietary labs restrict their models, others (e.g., from China) will develop similar capabilities, often with fewer guardrails. This makes it harder for “good guys” (defenders) to access and understand these tools, while “bad guys” (attackers) will inevitably find or create malicious versions (e.g., Worm GPT).
Omari Jones describes AI’s role in vulnerability discovery as a “double-edged sword.” While AI can uncover a vast number of vulnerabilities (e.g., 14,000, 99% previously unreported, by Unit 42), many lack proof of concept, overwhelming defenders. He argues that restricting these tools primarily impacts legitimate security researchers, while malicious actors will circumvent such restrictions.
Nikki Robinson and Omari Jones address consumer security. Nikki stresses that social media apps inherently involve significant privacy trade-offs, and AI simply accelerates and simplifies attack vectors. She advises users to be highly cautious of app permissions. Omari shares practical tips, like providing fake contact information or photos, given that many users, especially younger generations, grant broad access without understanding the risks.
Crisis Communications: CISA’s Call for Transparency
The final segment covers CISA’s advisory, “Communicating Under Pressure,” which advocates for greater transparency in breach disclosures. The goal is to move beyond minimal legal disclosure to providing “useful information for partners and the public over reputation management,” enabling stakeholders to manage their own risk.
Nikki Robinson differentiates this from “responsible disclosure,” which strategically limits information to prevent further exploitation while informing affected parties. She notes existing, secure channels for threat intelligence sharing. While supporting the intent to help organizations secure their platforms, she cautions against disclosing details that could arm other attackers.
Jeff Crume views CISA’s advisory as “high-minded stuff” but expresses skepticism about its practical impact without enforcement, as companies prioritize self-interest and reputation. He calls responsible disclosure an “ancient lesson” being rediscovered. Omari Jones agrees that while the intent is beneficial, the reality of corporate PR and brand reputation will likely hinder full transparency without stronger incentives or mandates.
The panel then considers how to “put teeth” on such advisories. Jeff Crume points out the challenge of regulation: laws are jurisdictional, but technology is global. Nikki Robinson adds that AI’s rapid evolution makes effective regulation difficult, as legal processes are too slow to keep pace with technological change, potentially rendering regulations obsolete before they are even enacted. Omari Jones suggests international cooperation, akin to a “NATO for AI,” could be a way forward, but ultimately emphasizes the need for organizations to proactively manage the risks of powerful AI models, potentially by keeping the most dangerous ones “100 miles beneath the soil and in some private secret vault that’s impenetrable to the world and truly air gap.”