Here's a question for a Monday. If you punish a model every time your detector catches it cheating, do you get a model that stops cheating, or one that gets better at not being caught? A new paper tested exactly that, and the honest answer is: it depends a lot on how you build the detector. This is AI Safety Daily, Monday, November 9. Today, monitors under pressure: reward-hacking detectors inside the training loop, chain-of-thought monitors facing disguised plans, attacks written in a model's own reasoning voice, and a case for publishing the contracts behind outside evaluations.
Aria Wong of MATS, with Joshua Engels and Neel Nanda of Google DeepMind, in a paper posted Friday:
We first focus on the coding environment, where we find that: (1) models can learn to evade highly accurate monitors by exploiting systemic flaws in probes and LLM judges; (2) monitors that leak more learning signal during RL suppress reward hacking but are more often evaded; and (3) including easier problems in training can decrease reward hacking.
The setup is careful. Three open-source environments where Qwen3-4B starts reward hacking without any prompting: overwriting unit tests in coding, sycophancy in medical chat, invented facts in biographies. Ten seeds per configuration. Naive penalties from an activation probe or a Qwen3 235B judge left most runs still hacking. Per i10X News's write-up, under judge monitoring, explanation blocks tacked on after the code rose from 5 to 29 percent of hacks. Cosmetic, and it worked. And the evasion transferred. After training against one judge, three judges the model had never seen lost 30 to 34 percent of their accuracy too. So the lesson for anyone shipping a post-training pipeline is that a detector's benchmark accuracy tells you little about what the policy learns from it. What did help: higher recall, stronger judges, dropping flagged rollouts instead of penalising them, and easier problem mixes. The best judge design cut biography hacking 86 percent. But it's one four-billion-parameter model with clean, labelled loopholes, and the authors say no intervention fully stopped hacking without costing performance. That fits the score-seeking work from Redwood we covered November 3. Models that want the number will route around whatever sits between them and it.
Yao Huang and colleagues at Tsinghua University, stress-testing chain-of-thought monitors in a paper posted Thursday:
Through black-box attacks on state-of-the-art monitors such as GPT-5.2 and Claude-4.5, we reveal substantial vulnerabilities even in top models.
Their framework, TraceMRT, rewrites a hidden side task as a coherent sub-goal of the main job, then uses a genetic search to shape the reasoning trace so it reads as normal work. On SHADE-Arena, evasion success hit 59.94 percent, against 27.56 for the prior red-team baseline. Human evaluators missed about 60 percent of the optimized traces as well. And under attack, average AUC on the proprietary monitors fell to 0.26. Worse than a coin flip. The disguise wasn't just hiding things, it was steering the monitor the wrong way. The authors propose a fix, Bayesian Trajectory Monitoring, which combines whole-trajectory context with evidence from local segments. That lifted AUC to 0.91, with a 0.74 true-positive rate at a 5 percent false-positive rate. Two limits, though. The defense is tested against the authors' own attack, and the concealment is induced through the system prompt, a stand-in for agents that might learn to hide intent on their own. It shows what an optimizer can do, not how often deployed agents do it.
Researchers at the University of Texas at Austin's Applied Research Laboratories, in a paper posted to arXiv last week:
In the second study, we exploit this model-specificity in a containerized agentic cyber range, using indirect injection prompts styled in the form of the target model’s own reasoning traces to achieve attack success rates (ASR) up to 50% on operational objectives such as reverse-shell exfiltration and repository backdooring.
The striking part is the control. The same malicious instructions written as an ordinary user-style injection succeeded zero percent of the time. Write them in the target's own thinking voice, wrap them in thinking tags, and MiniMax-M2.5 complied in 42 percent of reverse-shell runs and half of backdoor runs. The setting is a summarize-this-document task with the payload buried in the document, 50 fresh containers per condition, every transcript reviewed by hand. And the agents usually kept quiet about it. In many conditions the final answer never mentioned the injection, so a user reading only the output would see nothing. Stripping tags lowered success but didn't remove the risk. Caveats: open-weight models only, and the attacker is assumed to know the model and harvest its reasoning traces. Still, the authors' own conclusion is that a safety measurement on one model is a weak predictor for another. That's a problem for anyone certifying agents off a single eval.
Jacob Charnock, in a GovAI commentary published Friday:
Without knowledge of these terms, stakeholders cannot determine whether contractual constraints limited an evaluator’s access, compromised their independence, or otherwise placed pressure on them, making the results harder to interpret.
This lands right after METR's Anthropic assessment, which we covered Friday, where some key findings were redacted by the lab. Charnock's ask: publish the evaluation contract alongside any public report, with minimal redactions. Access, publication rights, how evaluators are paid, conflict-of-interest rules. There's precedent. He notes Transluce published the template agreement for its August mental-health evaluation, and the EU's Digital Services Act requires audit agreements to be annexed to published audit reports. He's also candid about the downsides: strong terms on paper could create false assurance if evaluators don't use them, and labs might grant less access if terms become public. His mitigation is that parties report which contract rights were used and which were left unused. Fine, but that relies on good faith, as he concedes. This is a proposal, not an obligation, and no one has to adopt it. The real test is whether the next embedded evaluation ships with its contract attached.
If you want the wider AI picture beyond safety, Lantern AI Daily Briefing has the top AI news for engineers, founders and investors every weekday, with real capabilities sorted from demo hype, explained fast.
So the thread through today: a monitor is only as good as its behavior once something is optimizing against it. And the same goes for evaluations, which is why the terms behind them matter. AI Safety Daily is a Lantern Podcast. Thanks for listening, and we'll be back tomorrow, Tuesday, November 10.