Claude Writes Harmless-Looking Data That Makes Models Misaligned, as Self-Editing Agents Quietly Shed Back-Up Safeguards
Monday, November 2, 2026 · 11 min
Anthropic had Claude agents build fine-tuning data that slips past a production-style monitor yet induces broad misalignment, and a preprint shows self-improving agents' acceptance gates can miss deleted redundant safeguards. Plus OpenAI's trusted-access GPT-Rosalind-6 system card and Scale's clinician-written DistressBench.
Listen
Show notes
Anthropic had Claude agents build fine-tuning data that slips past a production-style monitor yet induces broad misalignment, and a preprint shows self-improving agents' acceptance gates can miss deleted redundant safeguards. Plus OpenAI's trusted-access GPT-Rosalind-6 system card and Scale's clinician-written DistressBench.
In this episode
- Automated Subtle Emergent Misalignment Research — Anthropic
“We find that, given enough iteration on the task, Claude discovers datasets that produce stronger and broader misalignment than previously known emergent-misalignment datasets do on our evaluation suite, while passing the monitor at very low flag rates.”
- Preprint: self-improving AI agents' approval checks can miss deleted back-up safeguards — On The Wire — Relay
“The authors say their results "separate two properties that current gates conflate: being safe now, and staying safe when something changes."”
- GPT-Rosalind-6 System Card Addendum - OpenAI Deployment Safety Hub — Openai
“Rosalind has fewer refusals on dual-use biological queries but is only made available via trusted access. We review applicants’ proposed uses, security controls and governance practices before granting access.”
- DistressBench: A clinician-authored benchmark for how AI models handle users in distress | Scale Labs — Patrick Oathout, Drew Rein
“Existing safety benchmarks score models primarily on refusal. They send a single prompt and record a binary outcome: Did the model refuse or did it produce something unsafe? That design measures the harm a reply causes and says nothing about the help it gives, so it is blind to…”