Publications

As a result of rapidly accelerating artificial intelligence (AI) capabilities, multiple national governments and multinational bodies have launched efforts to address safety, security and ethics issues related to AI models.

AI-enabled research might cause immense harm if it is used to design pathogens with worrying new properties. To prevent this, we need better collaboration between governments, AI developers and experts in biosafety and biosecurity.

Great benefits to humanity will likely ensue from advances in artificial intelligence (AI) models trained on or capable of meaningfully manipulating substantial quantities of biological data, from speeding up drug and vaccine design to improving crop yields.^1-3 But as with any powerful new technology, such biological models will also pose considerable risks.

Early detection of novel pathogens can prevent or substantially mitigate biological incidents, including pandemics. Metagenomic next-generation sequencing (mNGS) of symptomatic clinical samples may enable detection early enough to contain outbreaks, limit international spread, and expedite countermeasure development. In this article, we propose a clinical mNGS architecture we call “Threat Net,” which focuses on the hospital emergency department as a high-yield surveillance location. We develop a susceptible-exposed-infected-removed (SEIR) simulation model to estimate the effectiveness of Threat Net in detecting novel respiratory pathogen outbreaks. Our analysis serves to quantify the value of routine clinical mNGS for respiratory pandemic detection by estimating the cost and epidemiological effectiveness at differing degrees of hospital coverage across the United States. We estimate that a biological threat detection network such as Threat Net could be deployed across hospitals covering 30% of the population in the United States. Threat Net would cost between $400 million and $800 million annually and have a 95% chance of detecting a novel respiratory pathogen with traits of SARS-CoV-2 after 10 emergency department presentations and 79 infections across the United States. Our analyses suggest that implementing Threat Net could help prevent or substantially mitigate the spread of a respiratory pandemic pathogen in the United States.

Xie et al. recently described ‘a reverse genetic system that enables rapid synthesis of wild-type, mutant and reporter SARS-CoV-2 [severe acute respiratory syndrome coronavirus 2] strains’¹. Their goal was to enable researchers to assess the functional properties of sequence variants, including their susceptibility to countermeasures such as vaccine-induced immune responses. We agree that the rapid development of medical countermeasures is of utmost importance, especially during an ongoing pandemic. However, we believe that public dissemination of this protocol in its current form poses risks that outweigh the benefits. In providing detailed, step-by-step instructions, it enables anyone skilled in the art anywhere in the world to create novel variants of SARS-CoV-2 more quickly, including variants that have even more worrisome properties than those that have occurred naturally. Sometimes research should be slowed, not hastened, to ensure a proper discussion of goals and a full and public vetting of proposed plans. Detailed protocols pose special risks under certain circumstances like this one and should undergo special prepublication scrutiny.