AI agents for therapeutic reasoning across biological contexts

AI agents for therapeutic analyses often struggle with preserving biological context, verifying computational steps, and reconciling conflicting evidence across diverse biological data. These limitations stem from issues like losing track of context in long workflows, failing to validate tool assumptions and parameter compatibility before execution, and aggregating studies without screening for contextual relevance. Michelle M. Li’s research aims to minimize the risk of contextual errors in AI, developing models grounded in biological and medical principles.

Medea, an AI agent for therapeutic reasoning, addresses these gaps by executing multi-step analyses with integrated verification throughout planning, execution, and evidence synthesis. Medea processes a therapeutic objective and optional experimental instructions, orchestrating four core modules:

• Research Planning: Generates a structured, multi-step analysis plan.
  • The research plan draft tool uses an LLM to distill objectives, performs LLM-augmented retrieval to identify relevant tools and their metadata, and assembles a coherent stepwise plan.
  • Context verification checks retrieved tool and data compatibility with the research objective, using an LLM to extract tool names/parameters and invoking internal context checker functions. It recommends alternatives if context is missing or invalid.
  • Integrity verification audits the plan’s feasibility, completeness, and logical consistency using an LLM judge. It evaluates clarity, tool use fidelity, parameter precision, hallucination risk, and logical coherence, triggering revision requests if criteria are not met.
• Analysis: Translates the plan into tool-calling analyses, leveraging databases, APIs, machine learning models, and other agents.
  • It performs pre-execution checks for tool and code dependencies, runs analyses in a sandboxed environment with error capture, and conducts post-execution verification.
  • The analysis quality checker employs an LLM judge to assess the scientific validity and completeness of executed code, evaluating informativeness, logical correctness, and alignment with research objectives, providing feedback for iterative refinement.
• Literature Reasoning: Retrieves and screens relevant literature using Semantic Scholar and Open Alex, synthesizing evidence from studies.
  • The relevance verification tool, powered by an LLM, systematically assesses a paper’s relevance by analyzing its title, abstract, and metadata (e.g., species, disease focus, cell type context), generating a binary relevance label and a detailed summary.
• Multi-Round Discussion: A panel of LLMs deliberates on evidence streams, identifies agreements and conflicts, and produces a consensus answer. If no consensus is reached due to weak tool outputs, conflicting literature, or panel disagreement, it abstains from providing a conclusion.
  • This process involves normalizing each module’s output into an evidence-grounded argument. The deliberation proceeds in four stages: independent evaluation by three distinct LLM panelists (assessing evidence strength, logical consistency, scientific rigor, etc., with confidence scores), preliminary consensus through confidence-weighted voting, iterative discussion (debate rounds with audit trails), and final summarization or abstention.

Medea’s capabilities were evaluated across three open-ended domains:

1. Cell Type Specific Target Nomination:
   • Addresses the limitation of traditional bulk tissue methods by reasoning about candidate targets within disease-relevant cellular contexts.
   • Evaluated across 29 cell types and five diseases (rheumatoid arthritis, type 1 diabetes, Sjogren syndrome, hepatoblastoma, follicular lymphoma). Medea leverages single-cell transcriptomic atlases and integrates differential expression with prior therapeutic evidence.
   • Medea significantly outperforms LLM-only and reasoning models, improving accuracy by up to 46% in rheumatoid arthritis. It also surpasses other biomedical AI agents like Cell Voyager in completion rate and accuracy, especially in handling fine-grained cell type distinctions through its context verification. Ablation studies confirm that all agentic modules contribute to Medea’s superior performance and calibrated abstention.
2. Synthetic Lethality Prediction:
   • Identifies gene pairs whose combined perturbation synergistically impairs cell viability, a strategy for cancer therapy.
   • Benchmarked using a multi-linear CRISPR screen across seven cancer cell lines, as well as a previously unpublished yeast epistatic miniarray profiling (EMAP) screen under DNA-damaging treatments (bleomycin and dimethyl sulfate).
   • Medea consistently achieves higher accuracy than LLMs, reasoning models, and state-of-the-art neural network-based predictors like PsL, even without using PsL as a tool. Medea’s verification enables it to correct LLM errors and make correct predictions where LLMs abstain or are incorrect, prioritizing interactions supported by convergent evidence. Its performance on the yeast screen, predicting experimentally measured synthetic lethal interactions, indicates biological relevance rather than information leakage from benchmark datasets.
3. Immunotherapy Response Prediction:
   • Links tumor intrinsic and microenvironment programs to treatment responses from multimodal patient profiles.
   • Evaluated using a patient cohort from the IMvigor210 clinical trial, predicting responsiveness and providing rationales based on tumor transcriptomic profile, microenvironment, and fungal profile.
   • Medea outperforms LLMs by up to 23.9% in accuracy and is robust to varying instruction specificity. Crucially, Medea effectively reasons over complex interactions between factors like tumor mutational burden (TMB) and tumor microenvironment, achieving high accuracy across diverse patient subgroups where standalone LLMs perform near-randomly. It also complements tool outputs by rescuing false predictions made by specialized ML models like Compass, with rescue rates ranging from 23% to 51%.

In summary, Medea demonstrates that verifiable AI agents can perform complex therapeutic analyses across diverse biological contexts. Its design, emphasizing verification, context preservation, grounding in OMIX data, and reconciliation of evidence, leads to improved performance, low failure rates, and calibrated abstention, addressing critical shortcomings of existing AI systems in biomedical research.

• Tags:
• AI Agents
• Drug Discovery
• Feature
• Context-Aware AI
• LLMs in Science
• Cancer Therapy
• Machine Learning in Biology
• Immunotherapy
• Therapeutic Reasoning
• Biological Context
• Multi-Modal Data Analysis
• Synthetic Lethality
• Single-Cell Transcriptomics
• Verification in AI