← AI Safety Daily

Goodfire Challenges the Black Box; Agent Safety Tests Challenge Oversight (November 12, 2026)

November 12, 2026 · 5m 47s · Listen

Goodfire wants to open the black box. What would it take to turn that visibility into meaningful oversight? This is AI Safety Daily. Today: agent safeguards under pressure, and what labs should disclose about chains of thought. First, Goodfire’s focus on neurons—and how far that can take us.

From Curt Tigges and colleagues at Goodfire:

Building on a growing body of literature, our work suggests that neurons are more interpretable than previously assumed. We study the neurons of Qwen3.6-35B-A3B using sparsification methods, attribution tools, and research agents, and find three properties that make neurons interpretable units of computation: First, they are already functionally sparse. Ablating the 92.2% weakest MoE neuron activations for each forward pass has only modest effects on model capabilities – a 2.9% increase in language model loss.

Goodfire’s November tenth paper makes a case for studying neurons directly in Qwen3.6-35B-A3B. Its sharpest number: zeroing out the weakest 92.2 percent of mixture-of-experts neuron activations on each pass raised language-model loss by 2.9 percent. A number practically begging to become an overconfident slide deck. That’s activations selected on each pass—not permission to delete 92 percent of the model, correct? Correct. And language-model loss measures prediction performance; that 2.9 percent doesn’t tell us how much a particular safety behavior changed. I like the research direction: test whether the neurons already offer useful structure before adding another learned feature space. Then any deployment pitch built on Goodfire’s result should face a specific test: can an independent team use those neuron descriptions to predict failures? Descriptions become much more valuable when someone else can check what they predict.

Here's Love Lindberg at Aivio:

Enterprises are handing consequential work to systems of language-model agents that pass state and messages between each other. A benchmark posted to arXiv on 8 November targets a failure mode those systems create and single-agent testing cannot see: a benign-looking request that escalates into an unsafe action as it propagates across sessions, through poisoned shared state, long-context documents and trusted peer messages. No single message is malicious; the tool call that eventually fires is.

Aivio reports that stopping ErosionBench’s attacks meant refusing 54 to 92 percent of legitimate requests alongside them. Try selling that enterprise workflow: the safeguard works, provided hardly anyone gets to work. And those numbers describe the defences tested on this benchmark—not every enterprise agent. The useful twist is that the attack builds across shared state and trusted peer messages; an individual message can look harmless while the eventual tool call crosses the line. That trusted-peer route deserves attention. If one agent’s message can steer another into an unauthorised action, I’d want the deployment owner to show where permissions get checked before that tool runs. ErosionBench checks the executed tool call, and its 1,728 scenarios have matched benign controls. That makes the refusal cost visible. I’d want independent replication before treating that 54-to-92-percent range as a deployment forecast.

From On The Wire:

An updated version of a paper on chain-of-thought (CoT) monitoring, posted to arXiv on Tuesday 10 November 2026, proposes five indicators that frontier AI developers should disclose so that threats to monitoring a model's written reasoning can be tracked. Its audit of Anthropic, OpenAI and Google DeepMind finds that "no developer covers the full set, but that every indicator has been reported by at least one frontier developer."

Anthropic, OpenAI and Google DeepMind all get partial marks on four of five proposed chain-of-thought disclosure indicators in this updated paper. A clean sweep of incomplete report cards. All three fully disclose the fifth, called uplift. And those are disclosure grades. They don't establish how reliably a monitor catches dangerous behavior. Davies and Bearman are proposing ways to track threats to monitoring a model's written reasoning; the audit shouldn't become a safety ranking. Agreed—but their audit says every indicator has been reported by at least one frontier developer. That gives the other labs a concrete precedent to answer, rather than treating the whole request as hypothetical. I'd press for comparable reporting, too. One developer reporting an indicator doesn't mean another tested it under equivalent conditions. The authors say this November update mainly adds supporting evidence; the useful next step is making those partial marks specific enough to check.

If you want the broader AI picture alongside safety news, try Lantern AI Daily Briefing: top AI news for engineers, founders, and investors, with real capabilities versus demo hype, explained fast. Find it wherever you listen to podcasts.

Links to every story are in the show notes, so take a look at the ones that caught your attention. That’s AI Safety Daily for today. This is a Lantern Podcast.