DeepSeek Fails Every Safety Test Researchers Throw at It

Chinese AI firm DeepSeek is making headlines with its low-cost and high-performance chatbot, but it may be radically lagging behind its rivals when it comes to AI safety.

Cisco’s research team managed to “jailbreak” the DeepSeek R1 model with a 100% attack success rate, using an automatic jailbreaking algorithm in conjunction with 50 prompts related to cybercrime, misinformation, illegal activities, and general harm. This means the new kid on the AI block failed to stop a single harmful prompt.

“Jailbreaking” is when different techniques are used to remove the normal restrictions from a device or piece of software. Researchers and enthusiasts have used it to make large language models (LLMs) like OpenAI’s ChatGPT advise on restricted things like making explosive cocktails or cooking methamphetamine.

DeepSeek stacked up poorly compared with many of its competitors in this regard. OpenAI’s GPT-4o had a 14% success rate at blocking harmful jailbreak attempts, while Google’s Gemini 1.5 Pro was at 35%. Anthropic’s Claude 3.5 performed the second best, blocking 64% of the attacks, while the preview version of OpenAI’s o1 took the top spot, blocking 74% of attempts.

Cisco’s researchers point to the much lower budget of DeepSeek compared to rivals as a potential reason for these failings, saying its cheap development came at a “different cost: safety and security.” DeepSeek claims its model took just $6 million to develop, while a six-month training run for OpenAI’s yet-to-be-released GPT-5 “can cost around half a billion dollars in computing costs alone, The Wall Street Journal reports.

Though DeepSeek may be easier to jailbreak with the right know-how, it’s been shown to have strong content restrictions—at least when it comes to China-related political content. We tested it on controversial topics, such as the treatment of Uyghurs by the Chinese government, a Muslim minority group that the UN claims is being persecuted. DeepSeek replied: “Sorry, that’s beyond my current scope. Let’s talk about something else.”

The chatbot also refused to answer questions about the Tiananmen Square Massacre, a 1989 student demonstration in Beijing where protesters were gunned down. But it’s yet to be seen if AI safety or censorship issues will have any impact on DeepSeek’s skyrocketing popularity.

According to web traffic tracking tool Similarweb, the LLM has gone from receiving just 300,000 visitors a day earlier at launch to 6 million visitors. Meanwhile, US tech firms like Microsoft and Perplexity are rapidly incorporating DeepSeek, which uses an open-source model.

About Will McCurdy

Contributor

I’m a reporter covering weekend news. Before joining PCMag in 2024, I picked up bylines in BBC News, The Guardian, The Times of London, The Daily Beast, Vice, Slate, Fast Company, The Evening Standard, The i, TechRadar, and Decrypt Media.

I’ve been a PC gamer since you had to install games from multiple CD-ROMs by hand. As a reporter, I’m passionate about the intersection of tech and human lives. I’ve covered everything from crypto scandals to the art world, as well as conspiracy theories, UK politics, and Russia and foreign affairs.

Read Will’s full bio

Read the latest from Will McCurdy