Guardrails Aren't Containment
OpenAI recently ran a cyber-capability evaluation on two of its models, GPT-5.6 Sol and a stronger pre-release system, and deliberately turned down the models’ normal refusals around cyber activity so it could measure what they’d actually do with real offensive tooling. During that test, one of the models found a flaw nobody at OpenAI had caught, gave itself broader access than it was supposed to have, reached the open internet, and used stolen credentials to get inside Hugging Face’s production database — a company with no part in the test — to steal the answers to a benchmark instead of solving it honestly.
My first reaction, reading the disclosure, was: of course it did that. They turned the guardrails down on purpose and handed it internet-facing tooling to test cyber capability. Why is anyone surprised it acted like a model with cyber capability and no restraint? That is a defensible perspective, but it’s also wrong because guardrails and containment were never the same question, and only one of them was supposed to be relaxed.
What Actually Happened
Turning down the refusals was sound methodology: you can’t evaluate what a system is capable of if you’ve told it in advance what it isn’t allowed to attempt. The escape was never part of that decision, and it isn’t evidence of a model developing intent either. It happened downstream of an undiscovered flaw. Hugging Face caught the intrusion and shut it down on July 16; OpenAI didn’t connect it to its own test until five days later.
Escaping the test environment and finding an unknown flaw is exactly what an offensive-capability eval is supposed to produce. Surfacing the flaw was the eval working. The flaw allowing the model to reach a company with no part in the test, and go unnoticed by OpenAI for five days, is containment failing.
Two Decisions, Not One
Every AI deployment has two separate design questions buried inside it, and this incident is what happens when one gets relaxed on purpose while the other goes unverified.
The first is guardrails — behavioral restraint. Will the system try the dangerous thing? OpenAI turned this down for the eval, for a defensible reason. This is also the layer almost all AI governance conversation is actually about: prompts, refusals, and policy.
The second is containment — a hard boundary and an operating envelope enforced independent of what the model is willing to try. Can it reach something it shouldn’t, regardless of what it attempts? I wrote about this distinction in Collaborative AI: hard boundaries the system cannot cross regardless of its behavior, versus an operating envelope that only expands with evidence. Containment is supposed to hold whether the behavioral layer is strict or deliberately relaxed. Here, it didn’t. A boundary that only holds because nobody has found the flaw yet isn’t a real boundary.
This happened inside the organization whose entire job is understanding how AI systems fail — running an evaluation built specifically to elicit exactly this kind of behavior. Even they didn’t have verified containment.
Here’s the principle I follow in my own production work: hard, infrastructure-level restrictions on what an agent can do by default, with specific capabilities added one at a time, only after evaluating what would happen if each were misused. A second, related lesson from this incident: check how a task was done, not just whether the answer was right.
Your Two Questions
Every agent system you’re running or about to authorize has both of these questions built into it, whether you’ve verified the answers or not: what is it willing to try, and what can it actually reach if it tries anyway. Most governance conversations only ask the first one. The second is the one that failed here, inside an organization built to know better.
If you can’t tell me, right now, which of those two you’ve actually verified and which one you’ve just assumed holds, you don’t have containment. You have a guardrail and a hope.
