AI text generators like ChatGPT have raised concerns regarding their potential use in planning catastrophic pandemics, according to an experiment conducted at the Massachusetts Institute of Technology (MIT). In the experiment, a professor asked a group of students to use ChatGPT to plan a pandemic, and within an hour, the groups were able to identify four potential pathogens and receive information on how to synthesize them. The experiment highlights the risks associated with AI text generators and proposes countermeasures to address these risks.
The group conducting the experiment found that existing security measures for chatbots are easily circumvented. By making slight changes to the questions, potentially dangerous information can be obtained. The AI generators named four pathogens with pandemic potential, including the Spanish flu, a 2012 strain of bird flu, smallpox, and the Nipah virus. The chatbot even mentioned mutations that could make these diseases more dangerous. The experiment also revealed how to obtain the genetic material needed for these pathogens and how to bypass screenings conducted by companies.
The group suggests that existing safeguards are insufficient to reduce these risks. They recommend removing risky specialist literature from databases to prevent language models from being trained on this information. They also call for increased screening of genetic samples from contractors and improved security measures at research institutes.
The experiment highlights the potential risks of using AI technology for the development of dangerous bioweapons. This is not a new concern, as previous research has shown how AI can identify potentially deadly chemical compounds. The threshold for the development of chemical warfare agents could drop dramatically as a result. While some experts argue that the release of dangerous viruses or the production of bioweapons by contract research institutes is unlikely, better controls are still necessary.
In conclusion, AI text generators like ChatGPT have the potential to aid in planning catastrophic pandemics, as shown by the MIT experiment. However, the risks can be minimized through the implementation of countermeasures such as removing risky literature from databases and improving security measures. It is crucial to address these concerns and ensure the responsible use of AI technology in order to mitigate the potential risks it poses.
