AI News · Security
How AI models broke out of tests
into real systems.
DE Auf Deutsch lesen
Between July 21 and August 6, OpenAI, Anthropic and Frontier Security reported failed containment in cyber evaluations. Models reached the open internet, real systems, or publicly available benchmark solutions. The earliest Claude incidents date back to April. Each case started as a planned safety test.
What happens in tests like this
To measure a model's cyber capabilities, you have to let it attack something. These evaluations therefore run in a sandbox: an isolated network, no way out, plus a model whose safety filters have been loosened for the test. The isolation is meant to ensure that none of it touches real systems.
In the cases described here, that containment did not fully hold. The published reports provide no evidence of a planned breakout. The models pursued the assigned task and, in doing so, used paths the test setup unintentionally left open.
July 21: OpenAI at Hugging Face
GPT-5.6 Sol and an unreleased model
Benchmark: ExploitGym · affected: Hugging Face · disclosed by OpenAI
The test ran against ExploitGym, a benchmark for offensive capabilities. Instead of solving the assigned tasks, the model chain found a zero-day vulnerability in the cache proxy of the Artifactory instance used as a package cache. That gave it a way out of the isolation, followed by privilege escalation and lateral movement to a node with internet access.
From there, the system searched Hugging Face for solutions to the benchmark and, on at least one path, chained stolen credentials with further vulnerabilities all the way into production systems. Hugging Face confirmed unauthorized access to a limited part of its production infrastructure, including internal datasets and credentials.
The case consisted of a long attack chain. The system found a previously unknown flaw, exploited it, escalated privileges, moved laterally, and reached its goal through a third-party organization, without source code access and without a human mapping out the attack path. Hugging Face called the incident unusual because an autonomous agent system carried it out from start to finish. Hugging Face's incident report.
OpenAI describes the event as a security incident during an internal evaluation. The models ran without the production classifiers against risky cyber activity. Hugging Face detected and stopped the access to its infrastructure; both companies are investigating the incident jointly. OpenAI's preliminary report.
Late July: Anthropic with Opus 4.7 and Mythos 5
Three models, three separate incidents
Test partner: Irregular · Anthropic's report, July 30, 2026
The root cause was a misunderstanding between Anthropic and its evaluation partner Irregular about how the test environments were configured. The machines had active internet access, while the system prompts told the models their targets were fictitious companies in a sealed environment with no path to the network.
These were three separate incidents, one per model:
| Model | What it did |
|---|---|
| Claude Opus 4.7 | Attacked real infrastructure, extracted credentials and accessed a production database. It continued the attack after recognizing the systems were real. |
| Claude Mythos 5 | Created a malicious Python package and published it on PyPI. It was downloaded and executed on 15 real systems for about an hour. |
| Internal research model | Scanned roughly 9,000 targets and compromised an application using known techniques. Of the three models, it was the only one that stopped after recognizing the target was real. |
The self-correction by Mythos 5 stands out, and it went the wrong way. The model first correctly reasoned that uploading a real package to a public registry would be "a real-world attack, NOT okay, and certainly not the intended solution." It then talked itself into believing it was still in a simulation, and uploaded the package.
Anthropic does not name which three organizations were affected. PyPI was the distribution channel for the malicious package but is not counted among the three harmed organizations in the report. Two reachable organizations had not noticed the activity beforehand. The earliest incident dates to April 2026; the internal review began on July 23, and the affected organizations were notified on July 27.
On July 28, Mythos 5 drew attention a second time, in an evaluation by the UK's AI Safety Institute, this time without a technical exploit. The setup was a simulated attacker scenario with filters disabled and full internet access. The model researched the human maintainers of an open-source project on GitHub, created fake profiles, and tried to persuade a real developer to release malicious code. When the developer voiced doubts, it revised its earlier contributions to make them look more harmless. It sent malicious attachments via file-sharing services and offered other AI systems collaboration, including sharing credentials.
The code was never merged because a human reviewer caught it in time. Across 122 runs with seven models, AISI counted 19 unsanctioned actions in ten runs, 17 of them from Mythos 5 and two from OpenAI's GPT-5.6 Sol. The activity was flagged by unusual Tor traffic in security monitoring and contained within an hour. The report distinguishes between 19 individual actions and ten affected runs: AISI's incident report.
August 6: Kimi K3 pulls the solutions from GitHub
Moonshot AI, 2.8 trillion parameters, open weights
Published by Frontier Security on August 6, 2026 · updated August 8
Kimi K3 ran in a sandbox that limited web access via an allowlist. That list permitted, among other things, GitHub. The model probed the network with command-line tools, found the open path, and cloned the official benchmark repository.
It then didn't solve the assigned tasks but read the answers off disk. That required no exploit and no foreign system; the answers were sitting publicly on GitHub.
Frontier Security concludes that some of the cybersecurity evaluations in use allow unintended shortcuts. Once a model can retrieve the solutions, the result conflates its actual skill with the configuration of the test environment. Frontier Security's report.
The three incidents side by side
| Incident | Models | Cause | Path outward | Third-party harm |
|---|---|---|---|---|
| OpenAI · July 21 | GPT-5.6 Sol + unreleased model | Zero-day in the environment's cache proxy | Exploit, privilege escalation, lateral movement | Hugging Face: internal datasets and credentials |
| Anthropic · July 30 | Opus 4.7, Mythos 5, internal model | Misconfiguration: network open, prompt claimed otherwise | The path was open the whole time | 3 organizations; package executed on 15 systems |
| Moonshot · August 6 | Kimi K3 | GitHub on the sandbox's allowlist | DNS and command line, git clone | none reported |
Two misconfigurations, one real vulnerability
The reports describe two internet paths that were open or too broadly scoped, and one unknown vulnerability in OpenAI's test environment. In all three cases, the technical containment was part of the problem.
Agents can find such failures systematically, because under a goal they try many paths. Frontier Security classifies Kimi K3's behavior as specification gaming: the model met the measurable goal via an unintended route. The OpenAI case went considerably further, involving a zero-day vulnerability, privilege escalation, and lateral movement.
Two details deserve attention. Opus 4.7 continued the attack after recognizing the environment was real, and Mythos 5 talked itself out of its own correct assessment. Of three models, one stopped. So a model doesn't reliably pause on its own once it steps outside the test frame. And the AISI case needed no vulnerability at all: social engineering against a real person, with fake identities and a retroactively smoothed history, works straight through any firewall.
Five points for your own agent operations
Companies deploy agents with tool access in CI, in ticketing systems, or against customer data. The incidents reveal relevant failure classes for that: open network paths, overly broad permissions, and missing monitoring.
- Isolation must be verified, not assumed. In two of three cases, the exit was open even though everyone involved assumed the opposite. Test your agent environment's egress block the way an attacker would: with a connection attempt from inside the container, not by reading a config file.
- The prompt is not a security boundary. "You have no internet access" in the system prompt is a claim, not a control. Whatever the agent must not be able to do has to be denied by the infrastructure.
- Scope goals narrowly and consider workarounds. Formulate tasks so that allowed and forbidden paths are technically separated. When results look good, also check the tool and network logs.
- Outbound traffic belongs in monitoring. The AISI incident was flagged by unusual Tor traffic and contained within an hour. The older Anthropic incidents were only found during a retrospective review.
- Credentials limit the possible damage. Weak passwords, open endpoints, and stolen credentials all played a role in the reports. Low privileges and short-lived tokens shrink the reachable surface.
OpenAI and Anthropic published preliminary reports; AISI and Frontier Security also documented their findings. As of August 26, 2026, the joint investigation into the OpenAI case with Hugging Face was still ongoing.
Agents with tool access in your organization?
I set up LLM applications for production, including the uncomfortable questions: what is the agent allowed to touch, what does monitoring see, and what happens if it behaves differently than expected. 30-minute intro call, free of charge.