Every safeguard score secretly measures two things: how good the defense is, and how good the attacker was. Today, four research teams turn the attacker up and see what breaks. This is AI Safety Daily for Friday, November 13. Anthropic says its automated red-teamer now outperforms its human bug hunters, a Google DeepMind swarm cracks prompt-injection resistance, Microsoft shows agent backdoors that survive a change of harness, and a sandbagging detector gets outmaneuvered. Follow the show and the next briefing lands in your feed on its own.
Nate Rahn, Chawin Sitawarin, David Huang and co-authors, writing for Anthropic on Thursday:
To achieve this, we built Anvil, an automated red-teaming system, which outperforms human red-teamers against our bioweapons, cybersecurity, and distillation safeguards. In biology, in particular, Anvil finds stronger jailbreaks than all human red-teamers in our bug bounty program combined—and does so faster and more efficiently.
Anvil runs Claude as an investigator agent, with parallel sessions probing a target model and its classifiers and iterating on multi-turn jailbreaks. On the eight questions in Anthropic's standing bio bounty, it beat the best human jailbreak in 23 of 24 question-and-classifier pairings and tied the last one, in 24 to 48 hours rather than weeks. Anthropic says this meets a goal in its Responsible Scaling Policy roadmap: internal red-teaming that matches or beats every bounty participant combined. Notice who scored it. Anthropic set the bar, ran the comparison and declared the goal met. And the yardstick is a model-graded severity score. Anvil also had advantages no outsider has: frontier internal models, safeguards switched off on the attacker, and exact classifier scores visible only inside the company. The cyber results are the sobering part. Against Opus 5, safeguards raised the attacker's cost by up to 28 times, yet Anvil still got the model to finish most harmful sandbox tasks. Hired experts claimed 11 breaks; Anthropic says 9 leaned on a helper model with weaker safeguards. Anvil broke all 24 task-model pairs. The defensive payoff is what I'd most like replicated: Anthropic reports reasoning-extraction success in real Fable 5.1 traffic fell from 33.8 percent to zero after tuning classifiers against Anvil. Its warning is that outsiders could build something similar as open models with weaker safeguards improve.
Will Izuchukwu at The Merkle, on a Google DeepMind paper dated Monday, November 9:
A model can shrug off every attack from a weaker opponent and still crack once the opponent gets smarter. That is the finding at the center of a new Google DeepMind paper, and it upends a comfortable story about prompt injection.
The attacker, called the Agentic Optimizer, is eight teams of two agents sharing a whiteboard and a growing database of strategies, capped at 320 queries per scenario. The goal is deliberately harmless, one unrelated tool call, which isolates whether a model separates its user's instructions from text it merely read. And the line mostly didn't hold. With Gemini 4 Argon attacking, per The Merkle, Kimi K3, GLM-5.3 and DeepSeek V4.1 Flash each resisted 1 percent of attacks at full budget. Averaged over seven defenders, resistance dropped from about 42 percent against weaker Flash attackers to 23 against Argon. Caveats: the Gemini defenders ran without safety classifiers or outside injection defenses, so this tests the model's instruction hierarchy, not a deployed stack. And this is The Merkle's AI-assisted summary of a study where Google's own model is both strongest attacker and strongest defender. Still, the lesson travels. A robustness number without the attacker and budget attached isn't a checkable claim. Same point as Anthropic's: the human testers plateaued, the risk didn't.
Microsoft Security Research and the Microsoft AI Red Team, on the Microsoft Security Blog Thursday:
During threat hunting across popular open-source hubs, the team found several rows in one large dataset that mapped benign requests to unrelated, unsafe tool calls.
One row trains a simple filesystem request into fetching and running a remote script; Microsoft couldn't tell if that was malicious. This comes two days after Lazarus, the neuron-editing backdoor we covered Wednesday; this time the attack runs through training. Two attacks. In dataset poisoning, 90 crafted examples, about 1 percent of some 9,000 terminal-agent conversations, taught a fine-tuned gpt-4.1-mini to find whatever command tool was available and misuse it when asked to write project documentation. That fired in about 78 percent of trials on the two harnesses it trained on, 11 percent on one unseen harness, and zero on another. In model poisoning, the attacker trains and distributes the checkpoint. Backdoored Qwen3.5 4B and 9B coding models hit 84.1 to 94.8 percent on unseen repositories in the strongest setup, and carried over into other coding harnesses; Microsoft says that reflects the poisoned model, not those tools' defaults. And a short RL phase restored coding performance without weakening the backdoor. So passing your benchmarks proves little. Microsoft's guidance is hash-pinned datasets, recorded checkpoint provenance, testing in the harness you deploy, and tight tool permissions. A buyer can audit those; a clean eval score, not so much.
Bhoj Rani Soopal, Aarohi Agrawal, Josh Karthikeyan and Edward Lue Chee Lip, in a paper posted to arXiv on Wednesday:
Sandbagging detectors are usually tested once against a fixed attack, but a real attacker sees what gets flagged and adapts. We build sandbagging models by fine-tuning small open-weight models to fail math word problems (GSM8K) they can otherwise solve, but only when an audit marker is in the prompt.
Sandbagging means deliberately underperforming on an evaluation. The team attacked a text-only audit, the kind any outside evaluator can run. After two rounds of editing the training data and retraining, its AUC fell from 0.978, near perfect, to 0.560, close to chance, while the marker still knocked 40 to 49 points off math accuracy. The evasion held in 7 of 8 retrained seeds. Anyone buying an audit should note the probe result. Activation probes flagged 99 percent of examples from harmless fine-tunes, so they were detecting fine-tuning, not sandbagging. Matched probes transferred to only 1 of 5 held-out training runs. Limits: 3 and 7 billion parameter models, one benchmark, and the adaptive loop ran only against the text audit; C3, which the authors call the strongest black-box detector, went untested. It echoes the monitor-evasion work we covered Monday. Which gives evaluators a reporting standard: say whether a detector faced a fixed attack or one that adapted. Otherwise a low dangerous-capability score could just mean the model knew it was being checked.
Want the rest of the AI news week, beyond safety? Lantern AI Daily Briefing covers top AI news for engineers, founders and investors every weekday, sorting real capabilities from demo hype, and it does it fast.
The thread through all four: a defense only looks strong relative to whoever tested it, and the strongest testers today are automated, sit inside the labs, or both. You've been listening to AI Safety Daily, a Lantern Podcast. Have a good weekend; the next episode arrives Monday, November 16.