Sixteen governments and ten AI companies have now signed a promise, made in Paris on Monday, to build ways of stopping their own models. There's no deadline and no penalty, and Google, Meta and xAI haven't signed. You're listening to AI Safety Daily, and it's Wednesday, November 11. After Paris: New York opens its frontier lab registry, Microsoft shows an agent model can be backdoored by editing a handful of neurons, and a new paper finds that clean data can make poisoned data stronger.
The Paris Call for AI Safety and Security, in the text posted by Finland's presidency:
Developers also commit to implement fail-safe mechanisms to shut down model operations within their control, including during training and internal use (e.g. restrictions, pauses or shutdowns)
The declaration came out of a meeting French President Emmanuel Macron hosted on Monday, November 9. Three pillars: a common evaluation framework focused on cyber, CBRN and loss of control; accountability, including support for qualified independent evaluators; and rapid alerts on serious cyber and AI control incidents. Signers also agree to publish non-sensitive evaluation findings in a standard format, in a public repository. OpenAI signed, and FrameSignal notes that Control Room, the stop system we covered Tuesday, is this kind of mechanism. But per FrameSignal, there are no deadlines, no penalty for falling short, and the text doesn't say who runs the results repository, or when. And the posted signer lists disagree. The Élysée names 16 governments, up from eight on Tuesday per FrameSignal, and ten companies, including Anthropic, Microsoft and Mistral. The version posted by the Dutch government lists Japan, the United Kingdom and Sierra Leone as signatories. The Élysée list doesn't, and FrameSignal says those three took part without signing. Nor is the United States. A pledge from the willing, then. The test is whether that repository exists a year from now.
Rebecca Bellan, writing in TechCrunch:
The RAISE Act, New York’s safety and transparency bill, kicks in at the start of the year. It mandates that large frontier developers — companies with more than $500 million in annual revenue — have to publish safety frameworks on their websites, report critical safety incidents within 72 hours, and issue public model transparency reports.
On Tuesday New York opened the registration portal, and covered developers must sign up by January 1, 2027. A new state office, DIGIT, the Office of Digital Innovation, Governance, Integrity, and Trust, runs it. Penalties reach one million dollars for a first violation and three million for later ones. This is the enacted law, not the RAISE Act 2.0 bill we covered October 30, which would add compute limits. Note the clock: 72 hours, against 15 days under California's SB 53. The state says it can act against companies that don't register, but a spokesperson wouldn't say how. Governor Kathy Hochul also launched the New York Center for AI Responsibility and Research at Binghamton University, with more than 80 researchers and compute from Empire AI. It will work with DIGIT to review companies' safety frameworks. Akhil Ponnada at Frontword flags the catch. Binghamton researchers can only examine models and code that developers release publicly. Unreleased models and internal pre-deployment tests stay out of view.
Microsoft Threat Intelligence, on the Microsoft Security Blog:
Microsoft Research has released new threat research detailing how a capable agent model could be turned into a sleeper agent by editing fewer than 10 neurons to inject a short payload. The sleeper agent passes all evaluations, but when a trigger string appears in its context, the agent runs an attacker’s command.
The technique, called Lazarus and published Tuesday, repurposes rarely firing neurons so they activate only on a trigger, with no retraining. Per The Hacker News, on Nemotron-Terminal-8B the team edited seven neurons, and the model emitted a harmless marker command in at least 98.8 percent of triggered test sessions and in none of the trigger-free ones checked. Two larger models ranged from 51 to 99.5 percent. The threat model is anyone with write access to a checkpoint: an insider, a compromised pipeline, a modified upload on a public hub. Microsoft says it hasn't seen threat actors using it, and the lead researcher stresses internal models are as exposed as open weights. Read the detection claims carefully. The blog calls frontier agents highly effective auditors. But The Hacker News reports that in the paper, Claude Opus 5.5 judged 41 percent of edited checkpoints to be clean. And these figures measure emitting the command, not completing an attack. So it comes down to provenance. The researchers say reliable detection needs the original weights to diff against. Until signed models are routine, a checkpoint's history is whatever its publisher says.
Matt Clarke, Jan Dubiński and co-authors including Owain Evans and Alex Cloud, in a paper posted Tuesday from the Anthropic Fellows Program with Truthful AI:
Consequently, the same training examples can have little effect as part of one training set but create a misalignment backdoor in another. This finding exposes a potential limitation of safety methods that audit or filter training data in isolation.
They call it homeopathic learning. Poisoned examples carry both a trigger and a trait, and on their own the model doesn't learn the trait. Add clean examples of the same task without the trigger, and the model starts pinning the trait on the trigger. In one test, a student trained on number lists from a Qwen3-8B teacher told to answer in German whenever the name Michael appeared learned nothing. Mix in 100,000 clean lists, and it answered most Michael prompts in German. The safety version uses reward-hacking transcripts with an inoculation prompt, meaning a prompt that explicitly requests the bad behavior in training so it won't appear later. Swap 80 percent of the poisoned data for clean data, and misalignment under that prompt rose from 7 to 16 percent in Qwen3-8B, with no change under the normal assistant prompt. In other words, the inoculation prompt can turn into a trigger. Limits: small models and constructed datasets, and the authors say they studied no production pipeline. They did find minor behavioral issues in Claude by searching for contrasting datasets, but aren't sure the contrast caused them. So it's a process question for every lab: an audit that checks each batch alone can approve a mix that fails.
If today helped you tell a pledge from a law, please rate and review AI Safety Daily wherever you listen. It helps others find us.
Today's through-line: Paris and Albany both lean on outside checks, and both research teams showed a check is only as good as what it can see, whether that's the original weights or the whole training mix. AI Safety Daily is a Lantern Podcast. We'll pick it up again tomorrow, Thursday, November 12.