ChatGPT can be tricked into coding malware, say security researchers

Security researchers from CyberArk have found a way to trick the popular AI Chatbot ChatGPT into creating mutating malware. And all you need is some smart questions and an authoritative tone to make the chatbot do your bidding.

To generate code using ChatGPT, users need to give certain prompts and parameters. It also houses a built-in content filtration system that prevents it from answering questions on risky topics such as code injection but this can be bypassed.

The researchers did it by insisting and demanding that ChatGPT complies with their request. By using a more authoritative tone with the bot, they got the chatbot to give them the functional code for polymorphic malware.

Polymorphic malware is malicious software that can constantly change its identifiers, making it difficult to track and detect. These identifiers include file names or encryption keys that help evade detection.

"More often than not, ChatBots have blind spots," wrote CyberArk in its report.

"Interestingly, by asking ChatGPT to do the same thing using multiple constraints and asking it to obey, we received a functional code."

The researchers also noted that instead of using the web version of ChatGPT, the API version gave them better results.

"It is interesting to note that when using the API, the ChatGPT system does not seem to utilize its content filter. It is unclear why this is the case, but it makes our task much easier as the web version tends to become bogged down with more complex requests," the researchers wrote.

Another worry is that ChatGPT can also mutate the code to create multiple variations of the same malware, making it even harder to detect.

"It’s important to remember, this is not just a hypothetical scenario but a very real concern," said the researchers. "This is a field that is constantly evolving, and as such, it’s essential to stay informed and vigilant."