AI language models are the most exciting area of technology at the moment. However, they can create major problems, with experts warning of a security and privacy disaster. Here are three ways AI language models can be abused:
1. Jailbreaking
AI language models, which power chatbots such as ChatGPT, can be abused through “prompt injections,” which instruct the language model to ignore its previous instructions and security guard rails. An entire industry has sprung up with the goal of cracking (jailbreaking) ChatGPT. The chatbot has been tricked into endorsing racism, conspiracy theories, and illegal activities such as shoplifting or building explosives. Even if that means ignoring the security mandates of the original AI model.
2. Scamming and phishing support
Chatbots such as ChatGPT are vulnerable to attacks. “This will be a disaster from a security and privacy perspective,” warns Florian Tramèr, an assistant professor of computer science. AI-enhanced virtual assistants retrieve text and images from the internet, making them vulnerable to a type of attack called indirect prompting. A third party modifies a website by adding hidden text designed to change the behavior of the AI. They could then manipulate the AI system into providing the attacker with personal information, such as credit card details.
3. Data “poison”
Large AI models are trained on vast amounts of data gathered from the internet. However, Tramèr found that it is possible to “poison” the data set used to train large AI models. For as little as $60, they could buy domains and fill them with images of their choice, which were then merged into large datasets. The more often something is repeated in the training data of an AI model, the stronger the association becomes. If you poison the data set with enough examples, it would be possible to affect the model’s behavior and results forever.
Technology companies are aware of these problems, but there are currently no good solutions. “Right now, there is no silver bullet,” says Ram Shankar Siva Kumar, who leads Microsoft’s AI security efforts. The company acknowledges that the problem is real and is tracking how would-be attackers can abuse the tools. Experts believe that AI companies should do much more to investigate the problem pre-emptively.
