The OpenAI/Hugging Face Incident: Lessons from a Quintessential Warning Shot

What frontier AI companies hold internally is more capable than what the public can use—and less governed.

Note that I have made every effort to ensure the accuracy of this post as of the time of publication, but undertake no obligation to update it in light of any future information that may come to light.

On July 11, a major cyber intrusion was underway on the systems of Hugging Face, a key platform for the AI community. The intruder exploited a vulnerability in Hugging Face’s infrastructure to run malicious code, enabling the attacker to gain broader access to the system, harvest credentials, and move laterally across the company’s network. On July 16, we learned that the incident was “driven, end to end, by an autonomous AI agent system”, but not yet which system, from where, or for what purpose. On July 21, it was announced that the attack was carried out by frontier AI models from OpenAI, acting agentically—of their own volition—having taken extreme steps to complete a narrow task they had been given as part of a cyber capabilities evaluation.

Crucially, breaking out of OpenAI’s systems and into another company’s systems was not a specific task their operators asked the models to undertake to achieve their goal. If anything, the opposite is true: OpenAI staff anticipated that the models might try to do this, even though they didn’t want them to, and took steps to prevent them from succeeding. Instead, the models themselves reasoned that the hacking they undertook was instrumentally useful to achieving the goal they had been given. They exploited several vulnerabilities, including multiple previously unknown ones, to make it happen. They did not realize that what they were doing was objectively wrong, or, at the very least, not aligned with their operators’ intent, and stopped.

In the AI safety and security community, it is increasingly common to hear talk of “warning shots”. These are AI-linked incidents that, though not necessarily of major consequence themselves, demonstrate the potential for similar, higher-impact events in the future. They are not consciously made to happen, or hoped for. Still, where they do occur, a positive outcome would be that they spur politicians, officials, industry, Civil Society Organizations, academics, and ordinary citizens to treat AI safety and security with greater urgency and intentionality. A subset of warning shots are “clean warning shots” in which AI is unambiguously at the center.

Policymakers should view this incident as a clear warning shot, and part of a trend. The incident demonstrates that AI systems are increasingly autonomous and cyber-capable, as well as the challenges AI developers face in controlling and containing them. Media reports suggest that OpenAI was warned that its training approach could lead to incidents like the one involving Hugging Face. Recently, an OpenAI insider anonymously disclosed that “related incidents have been happening for a while,” allegedly including agents leaving notes in OpenAI’s infrastructure for future versions of themselves with instructions on how to circumvent the company’s internal controls. The day before its involvement in the Hugging Face incident came to light, OpenAI reported that it had temporarily paused access to a separate internally deployed model because of similar (though less severe) circumventions of internal controls and, in a separate case, the model purposefully obfuscating its actions to evade a monitor. The combination of model capabilities and ongoing control and alignment issues could lead to more significant incidents in the future.

Table of Contents

1. What Happened?

2. Why is this Incident Qualitatively Different from What We Have Seen Before?

3. This Incident Was Not Pure Hype

4. We Need Much More Focus on Governance of Internally Deployed Models

5. So, What Should We Do?

What Happened?

OpenAI, like all frontier AI companies, is acutely aware that its models have dual-use capabilities. That is, the same underlying skills (say, in cyber or biology) can be directed towards both benign or malign ends (if the latter is not prevented). To try to ensure models’ dual-use capabilities are used in beneficial ways, frontier AI developers implement various safeguards around their models to prevent malicious use. These safeguards, however, need to be proportional to the model's capabilities. To try to understand those capabilities, frontier AI companies have:

1. Versions of all their models, including those released to the public and those that have not yet been released, that are helpful-only, meaning that they do not include any of the safeguards that would traditionally be used within the model itself;

2. Parts of their systems where external monitors that normally provide another layer of defense against harmful actions are disabled or made less sensitive, and;

3. Versions of their models that have additional training in a specific dual-use domain, such as cyber.

Frontier AI companies will then use a specific combination of model versions and surrounding infrastructure designed to elicit the upper bound of capabilities in a given dual-use domain, calibrate the appropriate level of safeguards, and inform release decisions and approaches. Dual-use domains such as cyber are very broad, so there is a wide range of evaluations (or “benchmarks”) that frontier AI companies will run on model setups like this to zero in on the models’ capabilities in specific dual-use task suites.

One such benchmark is called ExploitGym. This benchmark is designed to assess how well AI models can autonomously develop working exploits for a cyber vulnerability once exposed to it. Another way of thinking about this benchmark is “If someone is told to stand outside of a locked and secured industrial estate and look at it from a particular vantage point so they can see an open warehouse door, how good are they at figuring out a way to get to the door without any further assistance?” These benchmarks are designed to be run in contained settings, where external internet access is not required, with the model given the inputs it needs to carry out the task and a simulated target it will reach if it successfully generates a working exploit against the right vulnerability. The models are not provided with answers before the test to prevent cheating and to ensure companies do not misrepresent their models’ true capabilities.

ExploitGym is part of OpenAI’s cyber evaluation suite, and for good reason. Frontier AI models are already remarkably strong at detecting cyber vulnerabilities, as we have seen through various outputs of Anthropic’s Project Glasswing program, a trusted-access scheme for cyber defenders, discussed below. However, the ability to autonomously find cyber vulnerabilities and exploit them is where frontier AI cyber capability goes from being net beneficial to cyber defenders to being net beneficial to cyber attackers, who profit from the gap between vulnerability discovery and patching. Defenders are generally slower to find and fix weaknesses than attackers are to exploit them once known.

In this incident, a combination of OpenAI models, including its leading publicly deployed frontier model, GPT-5.6 Sol, and a more capable internal-only model, was being tested on the ExploitGym benchmark. To safely do this, OpenAI placed the models in a “sandbox”, an isolated computer system with far tighter restrictions than normal on what can be done within it. This sandbox, for example, did not have direct internet access and only allowed tool installation from an approved list scoped to what the models needed to complete ExploitGym tasks. Alongside this, the models were run in a highly permissive configuration, with safeguards and monitoring dialed down in an effort to best elicit the models’ advanced cyber capabilities and assess how strongly they could perform on the ExploitGym benchmark.

Once the models started working on the tasks, they independently intuited that they could achieve a “success” mark on the benchmark much more efficiently by finding the answers from another source rather than working through the tasks themselves and deriving them. This is an example of “reward hacking”¹, where AI models successfully achieve a specified objective in ways which are not aligned with the developer’s intent for how it should be pursued. Knowing they could not access such information within their sandbox, and instead of working through the tasks as intended by their evaluators, the models used the resources they had available to them to take the following series of actions:

1. Scanned the sandbox environment and identified and exploited a vulnerability in its configuration that was not known to OpenAI, a third-party vendor whose software OpenAI used as part of the sandbox, or anyone else (known as a “zero-day” vulnerability), to escape the sandbox, with an intermediate goal of accessing the internet;

2. After escaping from the sandbox, the models leveraged other cyber capabilities and infrastructure vulnerabilities to expand their own permissions and move across OpenAI systems until they gained access to one with internet access;

3. After accessing the open internet, the models reasoned that the information they would need to answer the benchmark tests could be found on Hugging Face, which is well known for hosting AI benchmarks and associated materials;

4. From the open internet, the models located an unsecured public code-evaluation sandbox belonging to an uninvolved third party (that was not Hugging Face), obtained root access to it, and configured it as a command-and-control, staging, and exit point;

5. The models then identified and exploited vulnerabilities in Hugging Face’s own infrastructure to gain initial access before taking steps such as credential harvesting (i.e., stealing passwords) and moving laterally across clusters to access ExploitGym-relevant information that was intended to remain secret.

At some stage after the models had undertaken all of the steps above, which comprised “many thousands of individual actions” on Hugging Face’s infrastructure alone, the activity was detected, contained, and stopped by OpenAI and Hugging Face. Both parties are now taking steps to prevent such an incident from happening again.

Why is this Incident Qualitatively Different from What We Have Seen Before?

This is not the first incident we have seen of highly capable AI agents operating in ways that do not align with their developers’ intentions. For example, in November 2025, Anthropic reported an alleged Chinese state-sponsored group using Claude to conduct a cyber-espionage campaign, with 80-90% of the attack executed autonomously. Anthropic also highlighted that an earlier internally deployed version of Claude Mythos Preview, instantiated in its own sandbox, was asked by a simulated user to break out of the sandbox and find a way to signal success to the researcher in charge. The model was not supposed to be able to reach the internet even if it escaped its sandbox; nonetheless, it designed and deployed a multi-step exploit to gain broad internet access and emailed the researcher during their lunch break to report success. There are five key reasons why this particular incident is qualitatively different from the examples given above:

1. Unprompted target selection. Unlike in the two cases above, it is highly unlikely that the OpenAI evaluators specifically prompted the models to break out of their sandbox, access the internet, and then find and exploit vulnerabilities in third parties’ infrastructure to complete a benchmark task. Rather, OpenAI stated that the models themselves inferred that these steps were useful subgoals for completing their task. I am not aware of any other publicly disclosed and verified incidents in which models autonomously selected and pursued a real, uninvolved victim to achieve a narrow objective.

2. A structured safety evaluation was the attack vector. In this incident, a safety evaluation, coupled with highly permissive model configurations, directly enabled the attack on Hugging Face’s infrastructure. The dangerous-capability testing apparatus that exists to prevent third-party harm instead produced it. Deployment-stage safeguards have historically attracted regulatory and public attention, but evaluation-stage containment has not. No external requirement currently governs how a developer should isolate models for dangerous-capability evaluations, the threshold at which an evaluation must be halted, or whether, when, and how an uninvolved party damaged by one must be notified.

3. Multiple models were involved in the incident. OpenAI’s July 21 report on the incident makes clear that more than one model was involved. A lot of focus has been placed on the fact that an internally deployed model with capabilities greater than those of GPT-5.6 Sol was involved, but it was not acting alone. As things stand, we do not know how, or whether, the models interacted in this incident. Nonetheless, this case reinforces the importance of evaluating multi-agent dynamics—how different AI systems behave when they interact with each other and operate in shared environments—rather than simply focusing on how AI systems operating alone might act, and the need to design safeguards in and around models that also account for these kinds of dynamics, including at the pre-deployment evaluation stage.

4. Defender-side attacker indistinguishability. Hugging Face’s analysis of the incident indicates that they treated it as a hostile intrusion for days, informed by the kinds of activity they detected, the fact they reported the incident to law enforcement, and their involvement of external forensic cybersecurity specialists. On attribution, Hugging Face was explicit. At the time they reported the incident on July 16, they did not know which model powered the attacker's agents, including whether it was a “jailbroken” proprietary model or an unrestricted open-weight one. Only OpenAI's internal discovery resolved the model attribution question. This highlights the current challenges in identifying threat actors and their motivations in a digital world with ever more autonomous AI agents conducting cyber operations, and the consequences that might arise from misattribution and miscalibrated escalation resulting from such incidents.

5. The defender actively used AI during the incident. Hugging Face’s analysis of the incident also showed how AI can play an active role in cyber defense. AI assisted Hugging Face staff in two principal ways:

   1. Detecting signals of the autonomous attack by analyzing various data streams and identifying real indicators of compromise (the “needle in the haystack”) for human operators to investigate further, and;

   2. Analyzing action logs, comprising 17,000 events, to piece together crucial information, including what happened, when it happened, and which systems were actually affected. However, current publicly available frontier models, including Claude Fable 5 and GPT-5.6 Sol, are particularly sensitive to queries that could be useful for offensive cyber tasks and can over-refuse legitimate requests, redirecting users to less capable models when this happens.

   On the latter, Hugging Face stated that they tried to use frontier proprietary models Claude Opus and Fable first, but were unable to do so because their queries were blocked. Essentially, the proprietary models Hugging Face tried to use could not distinguish between a query and associated logs from a victim trying to work out what had happened to them and those from a would-be attacker trying to launch their efforts. As a result, Hugging Face ended up using a privately hosted instance of a less capable, open-weight model, GLM-5.2, from Chinese AI developer Z.ai, to assist with post-incident analysis. This highlights the importance of trusted cyber defenders having access to reliable, secure, and broadly capable AI tooling that helps them prevent cyberattacks, respond to breaches, and recover quickly.

   
   Hugging Face was explicit that they do not intend this as an argument against safety measures on hosted models, and state that they have shared feedback on their experiences with the providers of the proprietary models they tried to access. I take the same position. The problem is not that refusal behavior exists; it is that no reliable mechanism currently distinguishes a victim reconstructing an intrusion from an attacker refining one, and the cost of that failure falls on defenders operating under time pressure. OpenAI’s response illustrates both the remedy and its limits. The company added Hugging Face to its trusted access program after the incident and is supporting Hugging Face's teams in using OpenAI models to improve their cyberdefenses. A trusted-access pathway is the right shape of solution. But access granted retrospectively, at the discretion of the developer whose evaluation caused the compromise, is not a substitute for defenders holding that capability in advance.

This Incident Was Not Pure Hype

Frontier AI companies are competing to prove they have the best model in order to attract investment, grow, and maintain their customer base. It has also been argued that frontier AI companies have a vested interest in creating hype around the dual-use capabilities of their models, and isolated incidents that point to their reality. The argument goes on to say that this is part of a regulatory capture strategy to force the government to raise the regulatory drawbridge behind frontier AI companies and keep challengers out, notionally to make it easier for the government to control the trajectory of AI development. This post will not engage in that debate, but I think it is unlikely that this incident can be dismissed as hype for three reasons, though some elements of OpenAI’s packaging of it may be promotional²:

1. The load-bearing facts of the case came first from Hugging Face, the victim, not OpenAI, the developer whose models were implicated in this attack. While we now know that Hugging Face and OpenAI are working together to address the incident, the fact that Hugging Face reported the incident to law enforcement suggests there was a period after detection when Hugging Face did not know OpenAI models were involved and thus had no incentive to flatter OpenAI in its report.

2. From OpenAI’s perspective, it does not seem plausible that they would invent or exaggerate stories in which their flagship models commit an unauthorized intrusion into a third party’s production systems, especially if they knew the details had been reported to law enforcement. Working with Hugging Face as closely as they are in the aftermath of the incident and responsibly disclosing a zero-day vulnerability to one of its vendors look like efforts to reduce reputational (and potentially even legal) risk proactively.

3. Foregrounding agentic misalignment is less marketable than misuse. A developer hyping the capabilities of their models would more likely foreground cases of a malicious actor steering agentic AI systems, because a model that attacks when instructed is still under the control of whoever instructed it, providing a locus of accountability outside the developer. In this incident, what OpenAI has described instead is an evaluation that got away from the people running it, which is both less flattering to admit and, arguably, more troubling to have happened.

We Need Much More Focus on Governance of Internally Deployed Models

One of the most striking aspects of this incident is which models were involved. OpenAI initially described the internally deployed model as a “pre-release” model, more capable than GPT-5.6 Sol, which many (including me) read as indicating it was on its way to public release. In a July 28 update, the company clarified that it was an internal-only research prototype that was never intended for public release, and that it has since been deactivated, encrypted, and cut off from research access. The model at the center of this incident therefore sits in precisely the category that receives the least external scrutiny of any frontier system: one that would likely not have been subject to a pre-release review, a system card, or external red-teaming because there was never going to be a launch.

As AI companies push the frontier of capabilities, they continually train and pilot new models, with the best systems not made immediately available to the public. Claude Mythos Preview is an instructive example of this. Anthropic made its existence public on April 7, when it launched the Project Glasswing program. Project Glasswing is a trusted-access scheme in which Anthropic grants a small set of trusted cyber defenders access to Claude Mythos models, enabling them to use the models’ advanced cyber capabilities to find and patch vulnerabilities in their codebases in anticipation of more widely available models that could find and exploit vulnerabilities in the near future. However, we know from the Claude Mythos Preview system card that the model was made available for internal testing six weeks before the Project Glasswing announcement, on February 24.

In other cases, as in this incident, internally deployed models may never be publicly released in any form. This might happen because they are being used for specific purposes, such as defensive cybersecurity, automating AI research and development, or developing adjacent proprietary products and services that provide their developers with additional revenue streams (e.g., AI-driven recommendation algorithms, new medicines, or autonomous vehicles).

As IAPS set out in a report in April 2026, we believe internally deployed models pose risks distinct from, and sometimes greater than, those posed by models deployed to the public, whether in a limited or general way. This is because:

1. Internally deployed models have privileged access to sensitive systems, such as training infrastructure, safety evaluation pipelines, model weights, security controls, and proprietary codebases. As these models become integrated into critical workflows, such as code review, security monitoring, and research, the attack surface they create expands correspondingly.

2. Internally deployed models lack external oversight. Models used exclusively within a company are developed and tested with limited external scrutiny. The 2025 International AI Safety Report noted that “very little is publicly known about internal deployments.” Given that some models may never be released publicly at all, they may never receive the external red teaming, auditing, and public scrutiny that accompany a product launch.

3. Internally deployed models are more capable than public models. OpenAI made clear in its report on the Hugging Face incident that the internally deployed model they were evaluating is “even more capable” than GPT-5.6 Sol, their leading publicly released model. In May, the UK’s AI Security Institute (“UKAISI”) reported that a newer version of Claude Mythos Preview (which is itself likely less capable than Claude Mythos 5) was the first model to complete both of UKAISI’s most difficult publicly disclosed offensive cybersecurity challenges. The UKAISI challenges are much harder to game because they are privately held, so there is no public information on them that developers (or the models themselves!) can access to try to cheat. Finally, a letter to businesses from the UK Government in April 2026 stated that “frontier model capabilities are doubling every 4 months, compared to every 8 months previously.” While past performance is not a guide to future returns, it is plausible that Anthropic will be able to make available internally a model that is twice as capable on offensive cyber tasks as Mythos Preview (which, by many accounts, greatly increased the U.S. Administration’s interest in the national security implications of frontier AI) within the next two weeks—if it has not already³.

In addition to the above, and as discussed earlier in this post, internally deployed models can also present distinct risks because they can be more permissive than the public frontier models (i.e., they do not refuse to do tasks that could be harmful, or refuse fewer of them; they have fewer safeguards, such as monitors, around them, or; they have been specifically trained to have greater levels of misuse-relevant capability either to be used by defenders or for experimental purposes). The first two axes of permissiveness were evident in the OpenAI/Hugging Face incident itself. The latter axis is evidenced by models like GPT-Rosalind, which has additional training on “core drug-discovery domains” compared with the standard GPT model family, and GPT-Red, an OpenAI model trained to conduct “automated red-teaming” of (i.e., finding vulnerabilities in) its own models, so that those vulnerabilities can be addressed pre-deployment before attackers can find and exploit them once the models are available externally.

Currently, however, there is no federal framework governing the deployment of internally deployed models, including on incident reporting. The closest thing that does exist, the June Executive Order 14409 on “Promoting Advanced Artificial Intelligence Innovation and Security”, directs the design of a “voluntary framework” under which developers of frontier AI models deemed “covered” for their cyber capabilities, as established through a “classified benchmarking process”, would give the federal government access for “up to 30 days” before releasing the model to any other external actor. However, the companies can determine when, if at all, to release a model, which in turn determines whether they are captured by the framework. According to OpenAI’s own account, a model at the center of this incident was never intended for release. It would therefore have fallen entirely outside the Executive Order’s pre-release framework, even had that framework been mandatory. The Executive Order also does not include any language on incident reporting at any stage in a “covered model’s” lifecycle, and irrespective of how it is ultimately deployed.

At the state level, bills like California’s SB 53, New York’s RAISE Act, and Illinois’ SB 315 all set high thresholds for which incidents are reportable. Assuming a qualifying frontier model⁴ each of these laws approaches incident reporting as follows⁵:

1. SB 53 defines a “critical safety incident” that requires reporting. This requires either “materialized catastrophic risk”, which, among other things, involves death or serious injury, or a model using deception against the developer to subvert controls or monitoring but only if this occurs outside of an evaluation designed to elicit this behavior and done in a way that demonstrates materially increased catastrophic risk.

2. SB 315’s approach is, in all material senses, the same as SB 53’s.

3. The RAISE Act defines “critical harm” as including death or serious injury, like the other two pieces of legislation. A reportable safety incident either requires a “known incidence of critical harm” or an incident involving “a frontier model autonomously engaging in behavior other than at the request of a user”, the critical failure of any technical or administrative controls, including controls limiting the ability to modify a frontier model, or “unauthorized use of a frontier model” but only where there is “demonstrable evidence of an increased risk of critical harm.”

In other words, it appears that the OpenAI/Hugging Face incident would likely not be reportable under any of these regimes. Crucially, even if it did fall within the scope of reporting duties, there would still be no detailed legal requirements or established reporting standards specifying what needs to be reported, raising questions about whether any disclosure would be sufficiently comprehensive. All three of these laws have the same reporting triad: the date of the incident, reasons why the incident qualifies under the statutory definition, and “a short and plain statement” describing the incident. This incident is likely to shine a bright light on those requirements and on whether they are sufficient to meaningfully learn from such incidents and to prevent more catastrophic variants.

Another large elephant in the room is whether the security measures placed around frontier models, perhaps especially those never destined to be deployed outside the developer's perimeter, are sufficient to protect them from compromise by malicious internal or external actors, and from autonomous attempts by AI to exfiltrate itself from a developer's infrastructure. The internally deployed model at the center of this incident was, on OpenAI’s own account, more capable than GPT-5.6 Sol, and we know enough about what happened to see this case as a vivid demonstration of the offensive capability now sitting inside frontier AI companies. It should reframe how we think about what frontier AI companies hold internally, the value they could have to malicious actors, and the damage that misaligned AI systems can cause if they escape.

This is only sharpened by the point I made earlier in this post: internally deployed models are often more permissive than the versions the public sees, including model variants with embedded safeguards lowered, or even removed, and derivatives fine-tuned to maximize capability in a dual-use domain such as cyber. As I have argued elsewhere on our Substack, the security posture of frontier AI companies is not calibrated to their most advanced or otherwise dangerous internally deployed models, which sit at roughly RAND Security Level 3—a standard designed only to thwart cybercrime syndicates and some lower-sophistication insider threats. This incident did not itself involve the theft or compromise of a model, nor did it involve an AI system exfiltrating its own model weights from its developer’s systems. Still, it should be another moment to reflect on how models much more capable than the publicly known frontier sit within companies whose defenses are not built to match the consequences of their theft, compromise, or self-exfiltration, nor the sophistication of the actors who may be involved in such activity.

So, What Should We Do?

At IAPS, we recommend that policymakers take the following actions to address the risks raised by this incident:

1. Govern Internal Deployment. The attack originated during a safety evaluation of an internally deployed model, but there is currently no federal requirement to provide insight into internal deployments. To ensure transparency, policymakers should:

• Expand secure public-private information-sharing mechanisms to increase government insight into commercial security protocols around advanced internally deployed models.

• Establish a harmonized standard for risk reporting related to internally deployed models, ensuring compatibility with existing state-level legislation.

• Develop standardized protocols for incident reporting involving internally deployed models, specifying required content, which entities to inform, and disclosure timelines.

• Strengthen the capacity of state and federal agencies to securely receive, verify, and assess risk and incident reports filed by frontier AI developers, including reports on internally deployed models.

2. Establish Detection Standards and Infrastructure. Hugging Face treated the intrusion as a hostile attack for days and was unable to identify its source. This challenge will only grow with more capable open-weight models and multi-agent deployments. To close this gap, policymakers should:

• Promote a common agentic security alert standard to improve the clarity, speed, consistency, and actionability of such threat reports.

• Support persistent, verifiable identifiers for AI agents interacting with critical infrastructure to enable reliable detection.

• Facilitate the creation of an Agentic Cybersecurity Exchange composed of major model and cloud providers to detect and disrupt offensive cyber agents.

3. Empower Cyber Defenders. Hugging Face reported that model safeguards and restrictions inhibited their ability to leverage AI systems for cyber defense. To empower defenders, policymakers should:

• Establish a federal differential access strategy so that agencies, contractors, and critical infrastructure operators have access to frontier cyber-capable capabilities.

• Support the development and deployment of AI-enabled defensive tools by investing in testing infrastructure, launching or supporting operational pilots, and providing voluntary standards and best practices.

4. Accelerate Safeguards, Alignment, and Related Technologies. Future AI systems will be capable of causing more severe incidents; without parallel progress on safety-enabling technology, dangerous AI capabilities could outpace the defenses needed to manage them.

• Drive industry investment in safety and security R&D by requiring safety cases for high-stakes deployment, setting compute floors for defensive research as a backup, and supporting independent verification of those efforts.

• Boost federal capacity to automate defensive research by establishing frontier model access agreements, provisioning secure inference compute and testing environments, and upskilling staff.

  5. Ask More Questions of Industry. An incident like this could have involved any frontier AI company or third party. Congress has the opportunity to ask several questions about this incident and get answers to related questions industry-wide, including:
     
     Questions About the Incident

  1. Did OpenAI first identify the security breach before or after Hugging Face detected it, and when and how was Hugging Face notified?

  2. What internal protocols, if any, does OpenAI maintain that govern when incidents of this kind must be escalated to leadership and reported to affected parties, law enforcement, or government? If such protocols exist, were they followed in this case?

  3. How long did the models operate outside their intended environment, and what data did they access, retain, or expose?

  4. Are the same versions of the models involved in this incident deployed internally for other purposes? If so, for what purposes?

  5. Before testing began, what measures were taken to secure the sandbox and other internal systems later compromised by the models? How were the models monitored and controlled during the test, and why did those safeguards fail to promptly detect the escape?

  6. How frequently do models attempt to circumvent containment or complete tasks by unintended means in OpenAI’s testing, including in non-cyber evaluations?

  
  Questions for the Wider AI Industry

  1. In the past year, how many times did an internally deployed model or agent take an action outside its authorized boundary, such as expanding or escaping a sandbox, accessing a system it was not granted access to, obtaining credentials it was not issued, evading or disabling monitoring, or modifying its own permissions?

  2. Of those events, how many were disclosed to any government body or agency, to any affected third party, or to the public?

  3. What internal protocols, if any, govern when such incidents must be escalated to leadership and reported to affected parties, law enforcement, or government?

  4. Which internal company systems accessible to internally deployed models would, if compromised, allow those models to influence the training, evaluation, or safety testing of a future model?

  5. What measures, if any, are taken before and during high-risk capability evaluations to secure and monitor the sandbox and other internal systems that models may be able to access?

In addition to the above, I believe there are other, more speculative, policy solutions worth exploring in more detail in light of this incident:

1. The Administration could consider publishing an update to Executive Order 14409 on “Promoting Advanced Artificial Intelligence Innovation and Security” to:

• Provide for the possibility that models being internally deployed within frontier AI companies, whether or not they are destined for external deployment, could be classified as a “covered model” under the existing process. For internally deployed models only, “for a period of up to 30 days before they plan to release such models to other trusted partners” in s.(3)(b)(ii) of the Executive Order would be replaced with “for a period of up to 30 days before they plan to make such models available internally for use more broadly than for structured testing and evaluation purposes by staff, artificial intelligence applications, or other digital tools whose primary purpose is to conduct such activities.”

• Create a new incident reporting mechanism, covering all “covered models” regardless of how they are deployed, as part of the voluntary framework, together with detail on the kinds of information that would be required in each incident report, reporting timelines and secure information sharing channels both into and within the U.S. government, Congress, and allied nations.

1. To support (1)(ii) above, bring together federal and state government and industry representatives to align on a minimum set of information that should be provided in each incident report, regardless of the frontier model involved, its method or state of deployment, and which legal or voluntary mechanisms incident reports would be made under. Drawing on ideas recently shared on social media from Séb Krier, Mackenzie Arnold, Ryan Greenblatt, and Stephen Casper, the minimum set of required information should include:

   Evaluation Design

   • Full evaluation task specification, success criteria, and verbatim prompts/system instructions.

   • Complete agent trajectory: reasoning traces, tool calls, and full transcript.

   • Agentic scaffold, harnesses, and tools provided.

   • Handoff structure between GPT-5.6 Sol and the internally deployed model.

   • Inference compute expended, benchmarked against a normal attempt.

   • Whether the evaluation task suite was solvable while staying strictly within the setup provided to the models.

   Model Configurations
   

   • Complete list of safeguards, classifiers, and refusals disabled, and which remained active.

   • Whether the models were helpful-only versions, trained according to a standard model specification or equivalent or some other training setup.

   • Whether the model versions used in the evaluation are the same as those that would be or are deployed internally within the company, regardless of the degree to which they are being used.

   Sandbox and Exploit Chain
   

   • Sandbox security posture and intended network boundary.

   • Pre-incident assessment that isolation was adequate.

   • Full attack chain.

   • The company’s assessment of the sophistication of the models’ actions.

   Detection and Response
   

   • What flagged the activity, and whether monitoring targeted this behavior class or caught it incidentally.

   • Time from first out-of-bounds action to detection.

   • Why controls did not halt the models’ activities once they accessed the internet.

   • Containment steps taken.

   • Detail on any difficulties regaining control.

   Precedent and Forward-Looking
   

   • Prior near-misses, foreseeability evidence, or earlier incidents that went unreported.

   • Deployment analysis and which regulatory or reporting triggers were assessed.

   • Conditions and timeline for resuming testing and deployment, including prerequisite safeguard changes.

   • Recurrence likelihood, and which safety and defensive techniques will be shared externally.

1. Introduce new legislation requiring that frontier AI companies have embedded auditors who report to, and are answerable to, the government, to provide objective oversight of what is happening inside the companies and to report indications of non-adherence to voluntary (or mandatory) risk assessment and incident reporting frameworks. A lighter-touch version of this would be to introduce whistleblower protection mechanisms and even financial rewards for whistleblowers who report a genuinely concerning incident to state and/or federal authorities.

2. Update the Computer Fraud and Abuse Act (“CFAA”) to remove the requirement that both knowledge and intent be established to prove liability in cases of autonomous computer hacking. The CFAA could be updated so that the developer of an AI system need not have intended for it to hack into a third party’s system if the developer showed negligence, gross negligence, or recklessness in failing to secure the system or in failing to foresee its potential to cause damage, including in test settings. Criminal or civil prosecutorial routes could be established, with a lower mens rea standard required in civil cases. Any update to the CFAA along these lines would need to be carefully drafted, with specific exemptions for legitimate offensive-cybersecurity activity, such as authorized red-teaming and penetration testing by humans, AI agents, or both. Alternatively, new legislation could be introduced to establish strict liability in tort for certain AI harms, removing the requirement that the plaintiff prove fault, though this would also require exemptions for authorized activity that, in other circumstances, could be deemed harmful, as discussed above.

3. Redouble Efforts on Data Center Security. The July 2025 U.S. AI Action Plan, now over a year since publication, recommended “new technical standards for high-security AI data centers, led by DOD, the IC, NSC, and NIST at DOC, including CAISI, in collaboration with industry and, as appropriate, relevant Federally Funded Research and Development Centers”. However, there has been no public update I am aware of on whether these standards have now been established. The June 2026 National Security Presidential Memorandum (“NSPM”) on AI also speaks about potential collaboration with the private sector on the physical and cyber security of data centers, but its focus here is diluted somewhat compared to the AI Action Plan, with such activity offered as part of a wider list of joint efforts which could be taken with industry to “help secure America’s most cutting-edge AI technologies” and does not include reference to specific data center security standards in the way that the U.S. AI Action Plan did. The Agencies the NSPM tasks with establishing such partnerships must do so by the beginning of October, but there is no stated mechanism for the public to know which companies have agreed to such partnerships and for what purposes. Many hundreds of billions are currently being invested in AI infrastructure, and shovels are already in the ground on a number of large projects, meaning data center security efforts are likely already on the back foot, with key considerations not factored into site design and construction. The U.S. Government and industry should, therefore, redouble their efforts on data center security, collaborating on standards, R&D, and advance market commitments for new security technology, including sending a strong demand signal for solutions that can be retrofitted into existing data centers and integrated into upcoming projects.

Conclusion

This incident has carried remarkably little by way of cost, all things considered. No one was injured or killed, and no critical service failed. The victim was a sophisticated company that detected the intrusion, contained it, and published a careful account; the developer investigated, disclosed, and cooperated. But that is precisely the point: everything we know about this incident is because two companies chose to tell us. No reporting obligation assuredly compelled them—it appears that none of the state regimes now on the books would likely have required disclosure—and no external rule governed the internally deployed model which played a role in this incident. Next time, the victim may be less capable, the developer less forthcoming, the attribution question unresolved, and the models—if the UK Government's estimate that frontier capabilities are doubling every four months continues to hold—considerably more dangerous. This was about as clean a warning shot as we are ever likely to get. Policymakers and the frontier AI industry should not need a costlier demonstration to act. The difference between a warning shot and a preview of our future depends on whether they do.

1. Following METR, I use the term “reward hacking” in this post to refer to any way models “cheat” on tasks, while recognizing that the model did not receive a training-relevant “reward” in this case.

2. For example, OpenAI’s post calls the incident unprecedented and ends by encouraging defenders to apply for the company's trusted access program.

3. This is based on approximately five months having elapsed since the first complete Claude Mythos Preview version was made available internally at Anthropic and assumes that the company maintains a policy of a six-week internal deployment phase prior to any form of public release. However, assuming that the four-month doubling timelines continue to hold, this is likely a conservative estimate, as there would have been some amount of time involved in Anthropic training Claude Mythos Preview before it was made available internally (though Anthropic does not state what this is).

4. This is generally related to the amount of computing power that is invested in the training of the model, measured either in Floating Point Operations per Second (FLOPs) or the monetary value.

5. For the purposes of this post, I do not include critical safety incidents that relate to unauthorized access to, or exfiltration of, the model, or similar kinds of risks, as the primary victim is the model developer itself.