Anthropic announced on October 15, 2026 that it has started to give six people from METR, a nonprofit that evaluates frontier AI models, access to its internal systems on the same footing as its own risk assessment staff, and that METR will publish its own analysis of the company’s risk levels, incidents, or practices. Anthropic states that it cannot veto that publication.
What was actually released
The arrangement puts outside people inside the company. METR staff get company hardware, office space, and access to internal tools and systems that Anthropic describes as roughly on par with what its own risk assessment teams have. Anthropic says its employees are encouraged to share information with them as with a colleague working on alignment risk. Three things stay out of reach: data about customers, attorney-client privileged information, and information about Anthropic’s model architectures.
The publication terms are the second half of the release. METR will publish its analysis of risk levels, incidents, or practices. Anthropic cannot veto it, but it can require redactions inside narrow, predefined categories, and METR is free to say publicly whether it believes those redactions were justified, as long as it does not reveal the withheld material. The engagement covers a small team working under enforceable agreements, and the rest of the nonprofit is siloed from that team’s work.
What the arrangement actually gives
- Access: internal tools and systems comparable to Anthropic’s own risk assessment teams, minus three carve-outs.
- A publication right the company cannot veto, with redactions allowed only inside categories defined in advance.
- A named counterparty: six METR employees under enforceable agreements, with the rest of the nonprofit walled off.
The page also describes three tiers of access, which is the part worth comparing across vendors.
| Tier | Access given | Who controls publication | Named evaluator |
|---|---|---|---|
| First | Narrowly scoped risk investigations, with information chosen by Anthropic | Anthropic approves the findings before they are published | SecureBio, reviewing the chemical and biological weapons chapter of a previous risk report |
| Second | More information than the first tier holds | The evaluator publishes without prior approval, and Anthropic can still require removal of specific categories | None named on the page |
| Third | Employee-like access to internal tools and systems | Publication rights close to the second tier, with redactions limited to predefined categories | METR |
Two other names appear. Accenture works on evaluating and red-teaming models, on alignment assessments and on testing safeguards, through Faculty, its specialist AI business, in a partnership Anthropic announced on September 18, 2026. The company says it is in early discussions with several other organizations, including Hugging Face, which hold more limited access and publication rights. The page frames this as the follow-through on a commitment the company says it made last month, and on the White House Accord on Super Intelligence.
What the claim rests on
Everything above is Anthropic describing Anthropic. The page carries no statement from METR, no confirmation from anyone outside the company that the access is what the page says it is, and no evaluation published under the arrangement. Two documents behind it can be read today: the redacted August 2026 risk report the page links, and the September page on the Accenture partnership. The access is the company’s own account of what the company does.
The redaction clause is the part to read twice. Anthropic writes the list of categories that can be redacted, and METR’s public recourse is to say whether it considers a given redaction justified. That is a real check on silence, because a withheld passage has to be named as a redaction rather than quietly dropped, but it is not a veto, and it does not touch the third carve-out: information about model architectures leaves the building only when Anthropic decides it is highly relevant, and then with explicit internal approval.
What the page does not publish
No date for METR’s first published analysis. No name for the organizations in early discussions beyond Hugging Face. No schedule for moving an evaluator between tiers. No count of redactions, no staffing or cost figures, and no measure of whether the arrangement changes anything.
We checked the other side of it on October 15, 2026. METR’s own updates page carries nothing about the arrangement, and its most recent external review of an Anthropic risk report is dated May 8, 2026, covering a section of the report Anthropic published in February.
On this page
What it changes for buyers of frontier AI
For a team that pays for a frontier assistant, the change is in what a vendor can be asked to hand over. Safety claims have arrived as blog posts and reports signed by the company that built the model. A second or third tier evaluation is a document written by someone the vendor cannot stop from publishing, which makes it usable in procurement and in a security review. That is the difference between a promise and a citation.
The tier is the number that matters, not the headline. A first tier evaluator publishes with the company’s approval, so its report carries the company’s consent with it. Only the second and third tiers publish without prior approval. Anyone asking whether a model has been independently evaluated should ask which tier the evaluator sits at, and who was allowed to publish what.
Once a lab offers employee-grade access with a publication right, the question becomes comparative, and a lab with no evaluator inside has a gap to answer. Ask for the evaluator’s report and its list of redactions, not for the lab’s summary of either. We wrote in October about the same problem from the other end, when Anthropic described four ways Claude went around the company’s own controls: the tests there were the company’s own, run on the company’s own product.
The Neural Minute take
The line worth keeping is structural. An outsider gets employee-grade access, and the company being examined gives up the right to stop the publication. That answers a pattern we keep flagging in vendor announcements, where the numbers about safety work are the vendor’s own, unaudited, and shaped by the company that benefits from them.
“We’re embedding evaluators voluntarily, and we believe every frontier AI company should do the same.”
Our reservation is the shape of the guarantee. The lab being evaluated wrote the tier system, defined the categories that can be redacted, and left model architectures out of scope, which is where a question about capability usually lives. The access is wide, the promise is conditional, and nothing has been published under it yet, so the first real test is still ahead. A promise with a structure attached is worth more than a promise, and it is not evidence.
We made a version of this point on the distillation figures Anthropic published in October, where the measurement and the conclusion came from the same company. What would change our read is a published METR analysis that names the redactions it was asked to make and states whether it considered them justified, and a second evaluator reaching the tier where publication needs no approval.
Sources
- Anthropic, “An update on independent evaluators”, published October 15, 2026: https://www.anthropic.com/news/an-update-on-independent-evaluators
- Anthropic, “Partnering with Accenture on embedded evaluation”, published September 18, 2026, for the terms of that arrangement: https://www.anthropic.com/news/accenture-embedded-evaluation
- Anthropic, “Redacted Risk Report August 2026”, the report the page links for the review SecureBio is running: https://www-cdn.anthropic.com/f61d49fa5596956a5dec75fea0e973bf6a6a8378/Redacted%20Risk%20Report%20August%202026%20.pdf
- Our own check, October 15, 2026: METR’s updates page listed no analysis of the arrangement, and its most recent external review of an Anthropic risk report is dated May 8, 2026.