Anthropic gives METR employee-like access to check its AI risks

What did Anthropic announce?

Anthropic said it has started to give employee-like access to a team from METR, a nonprofit that evaluates frontier AI models, according to Anthropic's announcement. It says this follows a commitment last month to stronger independent, third-party assessment of its models and safety practices.

Anthropic also says it signed the White House Accord on Super Intelligence, in which companies pledged to partner with independent external auditors or evaluators.

What does employee-like access mean?

Anthropic says it has onboarded six METR employees with company hardware, office space and access to internal tools and systems roughly on par with its own risk assessment teams. Outside a small number of defined categories, Anthropic staff are encouraged to share information with them as they would with a colleague.

Anthropic says the engagement is limited to a small team under enforceable agreements, and other METR staff are kept apart from that team's work.

Can Anthropic stop METR from publishing?

Anthropic says METR will publish its analysis of Anthropic's risk levels, incidents or practices. Anthropic cannot veto publication, but it can require METR to redact information in narrow, predefined categories, such as customer data, attorney-client privileged information or model architecture.

METR is free to say publicly whether it thinks the redactions were justified, as long as it does not reveal the redacted information.

Who else evaluates Anthropic?

Anthropic describes three tiers of access. In the first, Anthropic decides what access to give and evaluators publish with its approval. In the second, evaluators can publish without approval and see more. The third, METR's, adds employee-like access.

SecureBio is in the first tier and is reviewing the chemical and biological weapons chapter of Anthropic's August Risk Report. Anthropic says it is working toward a partnership with Accenture and Faculty on model evaluation, red-teaming and safeguard testing, and is in early discussions with others, including Hugging Face.

Why does it matter?

AI companies mostly check their own safety work. Here, Anthropic is letting an outside group look from inside and publish what it finds. Anthropic compares it to bank supervision, where regulators place resident examiners inside the largest banks. Whether the findings will be critical is not yet known.

What's next?

Anthropic says it hopes to have several evaluators at the third tier over time, each publishing its own analysis. It has not said when METR's first report will come.