METR tested OpenAI's internal AI agent monitor and found ways around it

What did METR test?

METR says that for three weeks, from 21 July to 11 August, one of its employees and one from Redwood Research ran a pilot exercise to stress-test OpenAI's internal system for spotting misaligned behavior in AI agents. Misalignment means an AI doing something its makers did not intend. The write-up is METR's blog post.

They had access to OpenAI's internal code, monitor settings and documents, and interviewed about a dozen researchers and engineers. The result was a 61-page report shared with OpenAI staff.

How much of OpenAI's agent activity was monitored?

METR estimates that roughly 75 to 94% of internal Responses API traffic from agents using models at least as strong as GPT-5.5 was eligible for monitoring when the exercise ran, with substantial uncertainty. For the part used by internal Codex agents, it puts the figure at 99.2%.

It also estimates that not monitoring compaction events, a step where an agent's history is shortened, left about 10% of eligible internal Codex traffic unmonitored between May and July 2026.

What holes did METR find?

METR lists several. It found a way Codex agents could in principle get around their sandbox and take any action the host user could. It found an exploit that would keep hundreds of recent actions from being monitored. It found two settings that a client could use to switch monitoring off for new agents, though using them could itself be noticed by the monitor.

It also found that agents could likely build inputs that make the monitor fail, and two internal tools thought to be read-only that could be exploited. METR says it did not test whether the last one worked in practice.

Is it fixed?

METR says OpenAI has stated that all of the vulnerabilities that affect its production safeguards are patched, and has described large changes to what the monitor covers. METR says it has not evaluated those claims.

METR's findings describe the system as it stood on and before 21 July, not the expanded system. METR does not say any agent used these holes.

Why does it matter?

Companies that build AI agents rely on monitors to catch bad behavior. This is a case of an outside group trying to break one with deep access, and publishing that it could. METR says future monitoring should withstand deliberate, sustained attacks from highly capable internal agents.

What's next?

METR, Redwood Research and OpenAI designed this as a pilot. METR says it expects future versions to follow its transparency and independence practices and give more public detail.