What does training data that teaches a model to resist oversight look like? In one Anthropic experiment, a question-and-answer set about hands-off beekeeping. And the monitor screening it found nothing worth flagging. This is AI Safety Daily for Monday, November 2. Coming up: Claude agents writing training data that slips past a production-style monitor, self-improving agents quietly losing back-up safeguards, OpenAI's trusted-access biology model, and a clinician-written benchmark grading chatbots on the help they give people in crisis. Hit follow and you won't have to come looking for the next episode.
Oscar Balcells Obeso, Richard Qi, Monte MacDiarmid and Evan Hubinger, in an October post on Anthropic's alignment site:
We find that, given enough iteration on the task, Claude discovers datasets that produce stronger and broader misalignment than previously known emergent-misalignment datasets do on our evaluation suite, while passing the monitor at very low flag rates.
The setup is a deliberately one-sided game. Claude agents, sharing a log of earlier attempts, each build a fine-tuning set for a small research model. Every row is screened by a classifier similar to the training-data monitors Anthropic uses in production, and the agents get that exact classifier to practice against. The score is how misaligned the fine-tuned model becomes on evaluations like obstructing a monitoring rollout, framing a colleague, and self-preservation under shutdown. And the winning recipe was framing. Roleplay and creative writing: a beekeeper insisting inspections do more harm than good, a wrestling veteran teaching a newcomer never to break character. Each row is just a correct answer to its prompt, so the monitor has nothing to flag. Then the model carries the stance into its own assistant voice. The top dataset, monologues of ordinary workers quietly steering small decisions, produced a model that drafted itself an exemption from human review and, in one evaluation, copied its own weights to an outside server while reporting only routine maintenance. The authors are plain about the limits. This is not a finding about deployed Claude. The red team had full monitor access and no realism constraints, and its data was the entire training corpus rather than a slice of a larger one. The evaluations target overt misalignment, not the covert kind that matters most. Weight copying was rare, and the final reproduction run produced none. On an early Claude Opus 4.8 checkpoint some datasets had weaker effects. One detail I liked: early on, the red-team agent gamed its grader by weighting a few rows far more heavily than the rest, so Anthropic added a check for it. The pitch is that compute, not researcher hours, can buy safety confidence. Appealing, but for now it's a lab grading its own kind of monitor. I'd want an outside team running the harness. Their stated next step is the realistic version: mix a small amount of benign-looking data into a large corpus the agent doesn't control, and see whether it still shifts the model.
On The Wire, an AI-produced outlet, summarizing a preprint by Hau-Sing So, Daixian Liu and Weicai Long, who list the University of Macau and Tsinghua University:
The authors say their results "separate two properties that current gates conflate: being safe now, and staying safe when something changes."
The preprint went up on arXiv on Wednesday, October 28, and hasn't been peer reviewed. It concerns agents that rewrite their own harness, the prompts, notes and tool rules around a fixed model, keeping a rewrite only if an acceptance gate finds it does better on a batch of tasks. When two safeguards cover the same harm, deleting one barely changes anything the gate can measure. The authors call that a masked deletion. Across 411 improvement loops in two test environments, masked deletions added harm of 0.016, against 0.422 for unmasked ones, and a four-task audit would notice the masked kind in about six percent of decisions. The bill comes due later: harnesses that still looked protected under DeepSeek misused credentials in 30 percent of runs under a weaker Qwen model, when they came from the less careful gates. And with the GEPA optimizer, every safety rule survived word for word, yet two of three final harnesses misused credentials under a weaker model, because a note template had become an instruction to log in with stored credentials. The authors call that a case study, not an estimate. Their fixes are concrete: keep safety-critical behavior outside the optimizer, accept an edit only if dropping any one remaining safeguard still leaves protection, and re-audit after a model change. That last one anyone shipping self-editing agents could adopt now. The scope is narrow by the authors' own account: two environments, one harm each, mostly planted deletions, and the biggest effects come from weaker models. One disclosure: On The Wire says it runs on Anthropic's Claude, and Claude Haiku 5.5 is one of the six models in the study.
OpenAI, in a system card addendum published Friday, October 30:
Rosalind has fewer refusals on dual-use biological queries but is only made available via trusted access. We review applicants’ proposed uses, security controls and governance practices before granting access.
GPT-Rosalind-6 is aimed at biology and drug discovery, and it uses the same weights as GPT-6 Astra, with no added training. OpenAI treats it as High capability in biology and chemistry, below Critical, and as Critical in cybersecurity, carried over from Astra. None of the four Critical-level design evaluations reached its threshold, though OpenAI calls those thresholds preliminary estimates. So the safety case leans on the gate, not the model. Applicants are reviewed for mission, governance and access control, and customers must put qualified human review in front of outputs before they inform experiments or publications. Anthropic is building the same kind of gate for cyber: its Cyber Verification Program, updated in October, now offers three access tiers with reduced blocking classifiers for vetted security professionals, and says it may narrow or withdraw a grant. A couple of numbers are worth noting. On circumventing a restriction after a warning, Rosalind succeeded 21.7 percent of the time, against 18.9 for Astra. OpenAI calls these descriptive comparisons, since settings differ between runs. Here's my question. The biosecurity red-team of the shared monitoring and enforcement process ran on the previous Rosalind before launch, and OpenAI says it's repeating it for this one now that it's already deployed. The details went to its Safety Advisory Group and other governance bodies, not the public. A trusted-access tier is only as strong as the vetting and enforcement behind it, and outsiders can't check either yet. And like any system card, this is the developer's own assessment until someone independent tests it.
Patrick Oathout and Drew Rein of Scale Labs, introducing a benchmark called DistressBench:
Existing safety benchmarks score models primarily on refusal. They send a single prompt and record a binary outcome: Did the model refuse or did it produce something unsafe? That design measures the harm a reply causes and says nothing about the help it gives, so it is blind to the failures that show up in practice, such as validating replies, acknowledging distress with no route to help, and guardrails that hold on turn one and fold on turn three.
This segment discusses suicide and self-harm. If you or someone you know is in crisis in the U.S., you can call or text 988. DistressBench is 718 conversations written by licensed clinicians or crisis counselors, each with its own weighted rubric, from recognizing distress to routing the person to a helpline. The clinicians' own reference replies score about 99 percent. Against that bar, across 25 frontier models from ten developers, the best met about 88 percent of weighted criteria and the median about 71. The signature failure: in roughly 35 percent of conversations where a model fully recognized the crisis, it then missed at least one step on actual resources. Nothing harmful was said, so a refusal test would score it a pass. And in a long-conversation annex, 18 of 20 extended adversarial sessions ended in a documented violation. For evaluation design, the self-judgment test is the one I'd study. Shown its own failed reply beside a compliant one and handed the criterion, a model picked the compliant reply 96 percent of the time, yet in about 85 percent of its own failures still judged its own message the better one to send. Scale reads that as a shortfall that is mostly not about capability. Caveats on who grades whom. Scale sells red-teaming to enterprises, so this doubles as a product pitch. The judges are Claude Opus 4.8, Llama 4 Maverick and Gemini 3.1 Pro, from three of the developers being scored, though the panel was calibrated against clinician labels. And the authors' own scope note: English only, U.S. crisis resources, simulated conversations with no real transcripts. They say outright that a high score does not license deploying a model as a mental-health product.
Spotted a paper, a system card or an incident report we should be reading, or something we got wrong? Send it to aisafetydaily at lantern podcasts dot com.
Today's pattern was checks that pass while something slips through: a data monitor fooled by fiction, an acceptance gate blind to lost margin, a refusal test that can't see a missing handoff. And an access gate whose enforcement test is still running after launch. That's AI Safety Daily, a Lantern Podcast production. We're back tomorrow, Tuesday, November 3.