Scientists trained AI to create chemical weapons

Source: (https://go.nature.com/3ik58O4)
The thought had never previously struck us. We were vaguely aware
of security concerns around work with pathogens or toxic chemicals,
but that did not relate to us; we primarily operate in a virtual
setting. Our work is rooted in building machine learning models
for therapeutic and toxic targets to better assist in the design of
new molecules for drug discovery. We have spent decades using
computers and AI to improve human health—not to degrade it. We
were naive in thinking about the potential misuse of our trade, as
our aim had always been to avoid molecular features that could
interfere with the many different classes of proteins essential to
human life. Even our projects on Ebola and neurotoxins, which could
have sparked thoughts about the potential negative implications of
our machine learning models, had not set our alarm bells ringing.
Our company—Collaborations Pharmaceuticals, Inc.—had recently
published computational machine learning models for toxicity
prediction in different areas, and, in developing our presentation to
the Spiez meeting, we opted to explore how AI could be used to
design toxic molecules. It was a thought exercise we had not
considered before that ultimately evolved into a computational proof
of concept for making biochemical weapons.
.....
Our toxicity models were originally created for use in avoiding
toxicity, enabling us to better virtually screen molecules (for
pharmaceutical and consumer product applications) before ultimately
confirming their toxicity through in vitro testing. The inverse,
however, has always been true: the better we can predict toxicity,
the better we can steer our generative model to design new molecules
in a region of chemical space populated by predominantly lethal
molecules. We did not assess the virtual molecules for
synthesizability or explore how to make them with retrosynthesis
software. For both of these processes, commercial and open-source
software is readily available that can be easily plugged into the de
novo design process of new molecules7. We also did not physically
synthesize any of the molecules; but with a global array of hundreds
of commercial companies offering chemical synthesis, that is not
necessarily a very big step, and this area is poorly regulated, with
few if any checks to prevent the synthesis of new, extremely toxic
agents that could potentially be used as chemical weapons.
Importantly, we had a human in the loop with a firm moral and
ethical ‘don’t-go-there’ voice to intervene. But what if the human
were removed or replaced with a bad actor? With current breakthroughs
and research into autonomous synthesis, a complete design–make–test
cycle applicable to making not only drugs, but toxins, is within
reach. Our proof of concept thus highlights how a nonhuman
autonomous creator of a deadly chemical weapon is entirely
feasible.