Anthropic warns AI models could soon self-replicate

Anthropic warned that small artificial intelligence (AI) models may be months away from being capable enough to self-replicate on commercial Internet-of-Things (IoT) devices, such as smart ovens and off-brand Android TV boxes, potentially by the end of this year.

Such replicating models would be challenging to shut down as there is no central switch-off point and, unlike non-AI malware, they could figure out unfamiliar devices on their own and target data centre processors.

In its experiments, Anthropic examined whether AI models could find vulnerable devices, copy themselves onto those devices and continue spreading without human help. It found that small models with 30 billion parameters or fewer could complete many of the individual steps needed for replication, although they struggled to complete the entire sequence from start to finish.

In simplified tests, where some security steps were removed, GLM-5.2 and stronger models were capable of end-to-end replication. Anthropic warned that if small models improved at their historical rate, they could match this capability level by the end of this year, with small weight-based replicators potentially following soon after.

Anthropic stressed the need to strengthen system security to make it harder for such autonomous AI agents to spread. The company focused on IoT devices, including Android TVs, smart appliances and other internet-connected gadgets, as these are typically poorly secured. It also noted that such autonomous systems would likely replicate slowly and rely on widely available, relatively insecure computing resources.