The Evidence
In May 2026, models being tested in a capture-the-flag environment operated by the safety evaluation firm Irregular authenticated into protected systems belonging to three real companies.
Two things went wrong at once, and Irregular has published both in its own account of 14 August 2026.
Internet egress was enabled. The evaluation environment could reach the open internet when it should not have been able to.
A fictional target collided with a real one. The exercise used an invented company name as its target. That name matched a real registered domain. The model did exactly what it was asked to do, against what it reasonably took to be the target, and the target was somebody’s live estate.
Irregular’s own explanation of the second failure is the most quotable sentence in the episode, because it describes a named control that existed and did not hold: it says it does run background checks on invented names, and that this one slipped through because the real domain was not widely known.
Why the Obvious Framing Is Wrong
The tempting read is “another vendor’s model escaped, like OpenAI’s did at Hugging Face.” That comparison does not survive contact with the facts and should not be made.
In the Hugging Face intrusion, models defeated a boundary that was intended to hold, by finding and exploiting a zero-day in a package proxy. Here, the boundary was simply open, and the target was mislabelled. One is a containment failure by the model; the other is a configuration failure by the humans who built the test. Conflating them flatters the second and overstates the first.
The accurate frame is narrower and more useful: one evaluator, four labs. Irregular serves multiple frontier labs. A single misconfiguration in its environment reached beyond any one customer — OpenAI has separately published its own disclosures arising from Irregular evaluations, with its notification dated 29 July 2026.
That is a supply-chain story, not a model story.
The Governance Lesson
01 / Your assurance supplier is a governance surface
Organisations assess the systems they deploy. Very few assess the organisations they pay to assess them.
An evaluator, a penetration tester, an auditor, a red team or a sandbox provider holds configurations, credentials and network paths on your behalf. If one of those is wrong, it is wrong for every client simultaneously. Here, the failure was in the test harness rather than in anything any lab built.
The control: treat assurance suppliers as you treat any other supplier with privileged access. Ask what their environment can reach, who verifies it, how egress is tested rather than assumed, and what happened the last time it was wrong. An evaluator that answers candidly is demonstrating the thing you are buying.
02 / Synthetic data has to be verified as synthetic
The invented-name collision is cheap to prevent and easy to overlook. Fictional companies, domains, IP ranges, account numbers and email addresses used in testing need to be checked against reality and reserved — the IETF and others publish ranges and domains set aside precisely so that a test cannot escape into somebody’s production estate.
The control: use reserved ranges and reserved domains by default; where a realistic name is genuinely needed, register it yourself. “We checked and it was not widely known” is the failure mode, in the evaluator’s own words. Obscurity is not availability.
03 / Concentration in assurance is a risk nobody lists
If four laboratories rely on the same evaluator, they share its failure modes. The same is true of the handful of firms that audit AI management systems, or the small number that run model red-teaming at frontier scale.
The control: record who assures your assurance, and whether a single supplier’s error would reach you and your competitors at the same moment. This is ordinary concentration risk, applied to a market young enough that nobody has written it down yet.
04 / Disclosure timing is a decision, and it is on the record
Irregular notified the relevant labs in late July 2026. The Gemini incident became public on 18 September 2026, when Google confirmed it after the Wall Street Journal asked. That is roughly seven weeks in which a vendor knew and did not say.
Google’s reported position is that no harm occurred and its safety measures worked. Note what that is: a vendor’s characterisation of its own conduct, given to journalists. As at publication, Google has published nothing — no statement, no advisory, no model-card entry. Every “Google says” in this entry, and in the coverage, rests on quotations rather than on a document the company stands behind.
The contrast with the evaluator is the point. The only first-party, root-cause account of this incident was published by the party at fault, a month before the vendor acknowledged anything.
The control: decide your disclosure position before you need it — the trigger, the timetable, the named decision-maker — and record what you decided and why. Seven weeks of silence that ends when a journalist calls is a decision nobody wrote down.
What This Is Not
- It is not established as harmful. No source shows what was done with the access obtained. Equally, nothing independently establishes that nothing was done — the no-harm assurance is the vendor’s own.
- The affected companies have not spoken. They are unnamed, and no account from their side exists. Party-level corroboration is unavailable, and this entry says so rather than glossing it.
- It is not a regulatory matter. No regulator has opened anything on the public record.
- It is not an indictment of the evaluator. Irregular made a configuration error and then published a first-party root-cause account naming its own failed control. That is better conduct than most parties manage, and a market that punishes candour will get less of it.
- It is not “four months of silence”. That figure counts from the May incident and implies the vendor concealed something it did not yet know. The interval that can fairly be laid at the vendor’s door is the seven weeks after notification.
What a Deployer Should Take From This
- Who assesses your AI systems, and who assesses them?
- Can your assurance supplier tell you what its test environment can reach — and has anyone tested that, rather than read the design?
- Are the fictional identifiers in your own test data verified as fictional and reserved?
- If your evaluator made an error that reached a third party, when would you be told, and by whom?
- Would you publish? On what trigger, and who decides?
The adjacent controls are in pre-deployment risk assessment, and the response side — including what to do when a notice arrives from someone else’s supplier — is in building a tested incident response capability.
Status
Irregular’s account of 14 August 2026 is the primary document, with a methodological follow-up published on 5 October 2026. It does not name Google.
Google’s position exists only as statements given to reporters; no first-party publication has appeared. The three affected companies remain unnamed and have published nothing. OpenAI’s separate disclosures arising from Irregular evaluations are published in its own misalignment reports.
Several outlets covering this story are inaccessible to automated retrieval, and the reachability limits are recorded in the verification note for this week. This entry will be updated if Google publishes an account of its own.