Ideas: More AI-resilient biosecurity with the Paraphrase Project

The Paraphrase Project began nearly two years ago when a simple case study for a workshop on AI and biosecurity uncovered a significant vulnerability. Microsoft’s Eric Horvitz and Bruce Wittmann discovered that open-source AI protein design tools could be used to “paraphrase” or redesign toxic proteins. These new amino acid sequences could preserve the protein’s original structure and function while being different enough to bypass the biosecurity screening systems used by DNA synthesis companies. This finding was described by experts as the first “zero-day” vulnerability for AI and biosecurity.

The Science and the Security Gap

Proteins are the “workhorses of life,” with their function determined by their 3D structure, which in turn is determined by their amino acid sequence. The challenge for protein engineers is navigating the astronomical number of possible sequences (20^300) to find one with a desired function. Generative AI accelerates this process.

However, this creates a security challenge. Companies like Twist Bioscience, which manufacture physical DNA from digital sequences, are a critical control point. They screen incoming orders for hazardous sequences, such as those encoding toxins or viral components. As James Diggans of Twist explained, these screening systems have historically relied on homology—checking for similarity to known dangerous sequences. Generative AI breaks this model by creating novel, “paraphrased” sequences that are functionally equivalent but have low sequence similarity to their dangerous parent protein, allowing them to fly under the radar of traditional screening tools.

A Red-Teaming “CERT” Approach

Upon discovering the vulnerability using the toxin ricin, the team adopted a cybersecurity-inspired approach, forming a “Computer Emergency Response Team” (CERT) composed of a cross-sector group from industry, government, and nonprofits.

1. Generative Pipeline: Bruce Wittmann built an AI pipeline using open-source tools. By providing a known protein’s structure as “conditioning information”—analogous to giving an image generator the prompt “a dog playing fetch”—the model generated thousands of novel protein sequences predicted to fold into the same structure.
2. Vulnerability Confirmation: The team provided thousands of AI-generated variants of ricin to Twist Bioscience and other major DNA synthesis providers. The results were stark: the paraphrased sequences were largely undetected by the existing screening tools (referred to as Tool Series A).
3. Expansion and Patching: The effort was expanded from ricin to a challenge set of 72 hazardous proteins, including 62 toxins and 10 viral proteins. The screening providers used this data to develop and test updated, more “AI-resilient” screening tools (Tool Series B). These patched tools demonstrated a significantly improved ability to flag the high-quality paraphrased variants that were most likely to retain their harmful function.
4. Global Disclosure: Once the patches were validated, the findings and solutions were shared with over 40 companies worldwide through the International Gene Synthesis Consortium (IGSC) to strengthen global biosecurity.

Responsible Disclosure and Information Hazards

A significant portion of the project—nearly a year and a half—was dedicated to navigating the “information hazard,” or the risk of arming bad actors by publishing the methods. The team faced a dilemma between the scientific principle of openness and the security need for secrecy.

Working with Tessa Alexanian and IBBIS (International Biosecurity and Biosafety Initiative for Science), they developed a novel, tiered approach to data and code sharing, which was accepted by the journal Science.

• Public Data: A significant amount of data was made public to ensure the paper’s arguments were concrete and convincing.
• Gated Access: More sensitive information, such as the full set of paraphrased sequences and the code pipeline itself, was placed behind a “perpetual gatekeeping” system managed by IBBIS. Researchers can apply for access and are vetted through a transparent process, ensuring that those who need the data for legitimate defensive research can get it while creating a barrier for potential misuse.

The Path Forward: Layered Defenses

The participants agreed that while the Paraphrase Project was a successful model, the work is far from over. AI capabilities are constantly evolving, moving toward designing proteins for a specific function without a known template structure. This will require moving beyond homology-based screening entirely.

The consensus is that a “systems-level” approach with layered defenses is crucial. DNA synthesis screening is a powerful roadblock but should not be the only one. Other essential layers include robust “Know Your Customer” (KYC) policies, export controls, and government incentives that encourage the entire biotech supply chain to adopt best security practices. The project serves as a model for how proactive, cross-sector collaboration can help manage dual-use risks, ensuring that the benefits of AI can be realized while mitigating potential harms.

• Tags:
• Generative AI
• Red Teaming
• Feature
• Biosecurity
• Protein Design
• Dual-Use AI
• Responsible Disclosure