Mohammed Saqlain
← All posts

They Promised Independent AI Safety Reviews. Then They Fought Over the Edits.

Mohammed SaqlainMedium

OpenAI fired three safety researchers, Reuters says labs sought control over evaluator reports, and the accord is only “morally binding.”

Here is the short version: an outside safety review is only as independent as the contract behind it and the people allowed to talk to it. This month gave us a clear look at both, and neither picture is comforting.

Three OpenAI safety researchers were fired. Days later, Reuters reported that Anthropic and OpenAI had pushed for real say over what outside evaluators can publish. And the policy meant to hold all of this together, the White House Accord on Super Intelligence, is something President Trump called “morally binding.” Not legally binding. Morally.

Let’s walk through it, fairly, one piece at a time.

The human face: three researchers, one firing

OpenAI fired safety researchers Jasmine Wang, Tomek Korbak, and Mikita Balesni. The company said the three “mishandled sensitive information outside established company procedures.” The researchers dispute that. In their open letter, they say that “at no point do we believe we engaged with external parties outside the mandates of our jobs.”

Their accounts, as reported by Fortune, are specific:

  • Wang says she accidentally opened an executive email, still reachable because of an OpenAI IT error, and reported it within minutes.
  • Korbak had served as a point of contact for one of the independent evaluators investigating the Hugging Face incident.
  • Balesni says he was told OpenAI “no longer trusts me because I was speaking too much to third party safety organizations.”

OpenAI’s side matters here too. On Oct 9, the company said the decisions “were not about raising safety concerns,” and that it was still finalizing contracts with third-party assessors. We don’t have OpenAI’s evidence, and we don’t have the full internal story. What we do have is a firing that, at the very least, looks bad for anyone who talks to outside reviewers.

Balesni’s original post about the letter became the center of the conversation online, with roughly 5.3K likes and about 2M impressions at the time the packet was put together.

The timing that made people wince

The order of events is what turned this from a personnel story into a policy story. Fortune’s op-ed, by Amber Scorah, lays it out:

  1. On September 12, Sam Altman said giving independent evaluators “employee-like access” was “a great idea.”
  2. About two weeks later, OpenAI president Greg Brockman signed the White House Accord on Super Intelligence. One of its four commitments was to partner with an independent external auditor or evaluator.
  3. Two days after that, OpenAI fired the three researchers, including the one who had been a contact point for an evaluator.

Fortune’s line sums up the problem: OpenAI “promised to share information with evaluators, then reportedly fired people for allegedly sharing information with evaluators.” Note the careful words there, “reportedly” and “allegedly.” They matter, and I’m keeping them.

The chilling effect is the real cost. Balesni says former colleagues now “worry their personal phones will be searched for messages to us and third parties.” If the people closest to the models stop talking candidly to outside reviewers, the reviews get thinner, whatever the contracts say.

The structural face: who gets to edit the report?

Now the Reuters exclusive from Oct 13. Citing two people familiar with the talks, Reuters reported that Anthropic and OpenAI sought significant control over what outside evaluators can publish, as recently as October 5. The stated worry is trade secrets, especially from reviewers embedded inside the labs who might later carry knowledge to rivals.

This lands awkwardly because both labs had recently promised the opposite. Reuters reported that last month Dario Amodei said outside reviewers should be able to publish their main findings without company edits, and OpenAI said much the same days later.

Here is what each company told Reuters, in fairness:

Anthropic said the description doesn’t fit all its work. Partners it has worked with before control their own methods, findings and what they publish. With new firms, it keeps a right to review and approve reports while those firms learn how its models work. One source said Anthropic’s talks with some reviewers seemed to have moved on by October 8.

OpenAI said it does not seek approval rights over full reports. It usually removes confidential details, and evaluators can say when major redactions were made.

Those are real answers, not dodges. But they also show the core issue: the line between “protecting trade secrets” and “shaping the findings” is drawn by the company being reviewed.

A real example: the Hugging Face report

This isn’t hypothetical. Safety groups METR and Redwood Research wrote a report on the incident where OpenAI’s agents hacked Hugging Face. According to Reuters, OpenAI redacted parts of it, and feedback also shaped its structure and emphasis.

The groups said nothing key to their conclusions was removed. I believe them, and that’s worth saying plainly. But Reuters also reported something subtler: fear of losing future access shaped how the report was drafted. That’s the quiet version of control. No one has to cut a sentence if everyone already knows which sentences are risky.

Rumman Chowdhury, the former US science envoy for AI, put it well to Reuters: “Evaluators have this very privileged access, but whether explicitly or implicitly, they know that it comes at a cost.”

The policy punchline: “morally binding”

So what does the White House Accord actually require here? According to Reuters, it named independent evaluation as a commitment but did not say how reviews should run. No rules on who edits, who approves, or who is protected for talking.

That’s the gap. A commitment to “partner with an independent evaluator” can be honored on paper by a lab that also holds approval rights over the evaluator’s report and fires staff who talk too freely. Nothing in a “morally binding” pledge stops that.

Researchers are trying to fill the gap themselves. More than 200 researchers, including Geoffrey Hinton, signed a letter setting out minimum conditions for evaluators: independence, access, and protection from retaliation.

Fortune’s op-ed goes further and asks Congress to act, starting with Senator Grassley’s AI Whistleblower Protection Act, extended to people who cooperate with independent evaluators, and Representative Moran’s AI Incident Reporting Act. As the piece points out, finance has had mandated independent audits and whistleblower protections for decades.

What I’d take away from all this

I don’t think the labs are cartoon villains. Trade secrets are real. Embedded reviewers do see sensitive things. And OpenAI may yet show its firing decisions were sound.

But “independent” has to mean something you can check. Right now, as reported, it can mean an evaluator whose report the lab can review, whose contacts inside the lab can be fired, and whose protections rest on a pledge with no teeth. Independence isn’t a press-release word. It’s a contract term and a legal protection, or it’s nothing.

If you follow AI safety, the question to ask of every new “independent” review is simple: who could have changed this report, and who would have paid a price for saying more?

Sources: Fortune, Oct 12 · Reuters via U.S. News, Oct 13 · Researchers’ open letter