Claude AI misuse allegations
Archived — this story has rotated out of today’s deck. It is kept here in full.
The gist
Anthropic said it disrupted misuse of its Claude models, including Chinese firms extracting capabilities.
The report escalates US-China AI tensions and raises questions about who guards model outputs.
Background
Anthropic released a 154-page threat intelligence report on September 10, 2026, saying it disrupted several alleged malicious uses of its Claude models over the past eight months. The report comes amid renewed worries over risks posed by rapidly advancing AI and the lack of guardrails. It also marks a further escalation in US criticism of China's AI industry practices, after three US government agencies earlier in the week claimed six Chinese companies were engaged in an industrial campaign to steal US model outputs. Distillation, the technique at issue, involves training smaller AI models using outputs from larger and more expensive models, which can reduce development costs.
How it unfolded
- May 2026 to Jul 2026Anthropic observed more than 151 million exchanges attributed to Alibaba, peaking at nearly 3 million exchanges a day across more than 3,500 accounts it described as fraudulent.
- Sep 8, 2026A researcher quit Anthropic, saying the race to develop AI could wipe out humanity.
- Sep 9, 2026Beijing rejected allegations that six Chinese companies engaged in an industrial campaign to steal US model outputs, describing them as actions by Washington to suppress China's AI industry and threatening retaliatory measures.
- Sep 10, 2026Anthropic released a 154-page threat intelligence report saying it had disrupted several alleged malicious uses of its Claude models, including a suspected Russia-linked cyber espionage campaign and efforts by Chinese AI firms to extract and replicate Claude's capabilities. It said Moonshot and DeepSeek routed live customer conversations through Claude and used its responses as training data, and that operators linked to Alibaba ran the largest illicit distillation attack it had observed, aimed at improving Alibaba's Qwen models. Anthropic said it banned accounts linked to identified malicious actors.
- Sep 10, 2026 latePresident Donald Trump dismissed rising concerns regarding the technology, telling reporters he is concerned the US will be in a very bad position if it does not lead the AI race.
- Sep 11, 2026Coverage said the report exposed state-sponsored surveillance, propaganda operations and more, including a UAE-linked influence operation targeting the Muslim Brotherhood and United Nations experts critical of Abu Dhabi's alleged role in Sudan. Anthropic also said it disrupted attempts to use its models for research that could help develop biological weapons, and that operators in China, Russia and Yemen used Claude for weapons design and development, intelligence gathering and procurement linked to weapons programmes.
Who’s saying what
- Anthropic
- Said several actors used its Claude models for activities ranging from weapons development and cyber operations to surveillance and fraud, that it disrupted these alleged malicious uses and banned linked accounts, and that its findings raise concerns about misuse of user data by PRC AI labs.
- Beijing
- Rejected allegations that six Chinese companies engaged in an industrial campaign to steal US model outputs, describing them as actions by Washington to suppress China's AI industry and threatening retaliatory measures.
- Trump
- Dismissed rising concerns regarding the technology, saying he is concerned the US will be in a very bad position if it does not lead the AI race.
Still unverified
Anthropic's report describes the cases as alleged by Anthropic. The developers' claim of funding from Russian government and military research programs was assessed by Anthropic as a small freelance team rather than an official operation. Anthropic said it could not determine whether the biological-weapons-related research was legitimate or malicious. Anthropic found no evidence that the actor's pursuit of a contract with Malaysia's national communications regulator succeeded. The actors generated allegations against named individuals for which Anthropic's model's own research could find no corroboration.