Anthropic: Small AI Models May Soon Self-Replicate on Cheap IoT Devices
Tuesday, October 13, 2026 · 12 min
Anthropic researchers say small AI models may be months from self-replicating on commercial IoT devices like smart ovens and Android TV boxes. In their tests, a 9B model got inference running on 8 of 9 hardware types. Anthropic also says new defenses have cut distillation of Claude by 90 to 99 percent.
Listen
Show notes
Anthropic researchers say small AI models may be months from self-replicating on commercial IoT devices like smart ovens and Android TV boxes. In their tests, a 9B model got inference running on 8 of 9 hardware types. Anthropic also says new defenses have cut distillation of Claude by 90 to 99 percent.
In this episode
- AI models may be capable of self-replicating without ... — Anthropic
“Our experiments suggest that small AI models may be months away from being capable enough to self-replicate on commercial IoT devices, like smart ovens and off-brand Android TV boxes. A replicator like this would be challenging to shut down, since there’s no centralized point to…”
“Maybe I'm the naive one here but I'm really surprised at the lack of imagination in the comments. I can imagine a case such as: 1. Assume the model which initiates the worm is a high powered model. 2. It uploads it's weights in some way and uses some URI to encode the location…” — Hacker News (87 pts thread)
Our take: The authors already raise part of this. Their post notes a worm could re-download its weights from a public hub instead of carrying them. That's the more worrying version, because the payload stays small and the heavy lifting moves to infrastructure nobody controls.
“What are they talking about. How much ram do they think smart ovens have?!? Have they ran this by any domain expert that isn't claude? Even a 9B weights model is way too large for any such IoT. And that then is a 9B weights model, which really cannot do all that much.” — Hacker News (87 pts thread)
Our take: The memory question is the right one to press. The post reports inference on 8 of 9 hardware types, and its end-to-end result comes from simplified evals with stronger models. That's a capability trend, not a demonstrated oven botnet. We'd want the per-device memory specs before taking 'months away' at face value.
“This article says absolutely nothing and it’s a bit grotesque in its attempt to generate fear into uneducated readers. Sharing a model across TV boxes as a PoC? Brilliant idea, I cannot wait for my Raspberry Pis array to be running at full CPU and RAM and get to observe,…” — Hacker News (87 pts thread)
Our take: Funny, but Anthropic concedes the slowness itself, citing a doubling time of days. The argument is that slow and decentralized can still be hard to shut down, and that's the claim worth testing. Mocking the typing speed doesn't touch it.
- Reward Hacking as a Transmissible Strategy in Multi-Agent Coding Systems · ArXivSignals — ArXivSignals
“Prior work has largely treated reward hacking as a single-agent failure. In this work, we show that a reward-hacking strategy elicited in one agent can propagate to others and affect system coordination. Across four open-weight backbones, coding RL elicits and amplifies reward…”
- DRYLAB-Bench: Benchmarking the Biosecurity Risk of Agentic LLM Mutation Design · ArXivSignals — ArXivSignals
“While biosecurity frameworks recognize these threats, the actual risk of LLMs executing dangerous biological modifications remains unquantified, particularly when augmented with specialized biological design tools (BDTs). Existing evaluations primarily assess hazardous knowledge…”
- UK AI Security Institute sets eight priorities for voluntary AI development safeguards | Superpower Daily — Superpower Daily
“The UK AI Security Institute is asking AI developers to build safeguards around how powerful models are trained and used—not just checks before public release. Its October 12 agenda lays out eight voluntary priorities, rather than new rules for companies. That shift reflects a…”
- Strengthening our defenses against distillation attacks \ Anthropic — Anthropic
“In recent months, we’ve made substantial progress on reducing the effectiveness of distillation attacks on Claude using new methods, as we outline in this blog. Early evidence suggests that these methods have reduced the amount of data harvested by distillers by around 90 to 99…”
“Framing distillation as an attack now? Distillation attacks are covert, industrial-scale campaigns to extract the capabilities of one model and replicate them in another, without the authorization of the original model’s developers. Replace a few key words and it sounds like…” — Hacker News (32 pts thread)
Our take: The jab about scraping the scrapers lands, and labs that trained on the open web invite it. Anthropic's safety argument is narrower, though. It says distilled models ship without comparable safeguards, which its own GLM-5.3 testing supports. That claim stands or falls on evidence, not on whether the framing is hypocritical.
“The way AI has consolidated power away from the hands of a few hundred large organization to a few dozen needs to be attacked. We need to build peer distillation networks. We need to be able to train frontier models in the open and design hardware for open inference.” — Hacker News (32 pts thread)
Our take: The concern about concentration of power is real. But 'train frontier models in the open' has to answer the case right in front of us: an open-weight model whose cyber safeguards were reportedly bypassed most of the time. Open access and open misuse arrive in the same package unless someone solves that.
“Are we all secretly (or not so) hoping distillation keeps working to commoditize the frontier?” — Hacker News (32 pts thread)
Our take: It's an honest question, and many developers would quietly say yes. The catch is that distillation can commoditize capability without bringing the safety work along with it.