Outside Evaluators Get Inside OpenAI, Redwood Picks Apart Gemini 4's Safety Case, and Models Spot the Test
Tuesday, November 17, 2026 · 9 min

OpenAI signed Apollo, METR, Transluce and Goodfire to embedded assessments with limits Apollo itself flags, while Redwood Research argues Google DeepMind's Gemini 4 reports don't show low loss-of-control risk. METR finds top models can tell alignment evals from capability evals, and a LessWrong analysis finds a shortcut in the TASTE safety-research benchmark.
Listen
Show notes
OpenAI signed Apollo, METR, Transluce and Goodfire to embedded assessments with limits Apollo itself flags, while Redwood Research argues Google DeepMind's Gemini 4 reports don't show low loss-of-control risk. METR finds top models can tell alignment evals from capability evals, and a LessWrong analysis finds a shortcut in the TASTE safety-research benchmark.
In this episode
- A Pilot Towards Embedded Evaluations — Apollo Research
“This pilot is an ambitious first step towards deeper access and disclosure rights, even if it doesn’t yet fulfill the criteria above. OpenAI has agreed not to withhold information or access to staff, nor request redactions, simply because the information or findings are…”
- Gemini 4 safety reports don’t demonstrate low loss-of-control risk — Redwood Research — Redwoodresearch
“We think that the arguments in these reports have serious weaknesses, and the reports overall don’t convince us that the risks of deploying these models have been mitigated reliably.”
- Measuring alignment eval tells - METR — METR
“In particular, on our dataset, Opus 5.5’s direct and open-ended scores, each averaged over three runs per environment, rank every alignment eval above every capability eval. This doesn’t mean it’s impossible to build alignment evals that models can’t recognize, but alignment…”
- LLMs are below chance on TASTE once a shortcut is removed — LessWrong — Ziqian Zhong
“On the non-interp half, however, the stronger models by ECI do worse, often worse than chance. With only the rubric provided, all 14 models with ECI > 158 score below 50%, and 8 of those 14 are significantly below chance.”