Critical infrastructure operators face a narrowing window of preparation as frontier AI models demonstrate an increasing, though currently inconsistent, ability to architect attacks against operational technology (OT). A new report from Frenos, titled "The State of AI in OT Security," suggests that while autonomous hacking remains unproven, the capability for AI-assisted attack planning could become a reliable tool for adversaries in approximately two years. The research, which utilized over 500 simulated OT attack scenarios, highlights a significant performance gap between top-tier frontier models and open-weight alternatives. For enterprise IT and OT leaders, this timeline serves as a strategic deadline for hardening industrial environments against increasingly sophisticated, AI-augmented reconnaissance and planning efforts.
Frontier Model Performance and Reliability Gaps
The Frenos study evaluated frontier, U.S. open-weight, and foreign open-weight models by running them as agents within 500 safe, non-executable simulations. Researchers reviewed more than 16,000 OT attack decisions and paths to ensure the testing environment mirrored real-world adversary logic. The results revealed that three of the six frontier models tested could produce workable attack plans in roughly 90% of attempts when provided with specific background information and context. These models successfully organized target assets, weaknesses, and access points into actionable steps. In guided simulations involving researcher feedback, these same models achieved a success rate of more than 90% in executing attack plans.
However, the research identified a critical lack of reliability in these models when tasks were repeated under identical conditions. When researchers re-ran the same tasks, the three leading frontier models produced passing plans only about two-thirds of the time. These failures typically involved the models skipping essential steps or incorrectly assuming the attacker had already established specific access. Harry Thomas, CTO and co-founder of Frenos, noted that while these models can appear "brilliant" in a single instance, they often "fall apart" upon repetition. This inconsistency suggests that while frontier models are approaching a threshold of utility, they still require an experienced human operator to correct errors and provide necessary context to be effective in a real-world attack.
Disparities in Open-Weight and Foreign Model Capabilities
The research highlights a stark divide in capability based on model type and origin, which has direct implications for the ongoing policy debates regarding open-weight AI in the United States. The study found that capability varied sharply even within the frontier tier; the three weakest frontier models succeeded in only about a third of the tests or less. Furthermore, the strongest foreign open-weight model performed on par with the weakest frontier models and outperformed the strongest U.S. open-weight model by approximately 7x. Despite this relative strength, the foreign open-weight model still trailed the three leading frontier models by a wide margin.
This performance gap suggests that the specific model an attacker selects fundamentally alters the threat profile faced by defenders. Harry Thomas characterized U.S. open-weight models as being "not in the picture" for this specific type of sophisticated work. This distinction underscores a growing tension in the cybersecurity landscape: the level of control frontier AI developers exert over their models may be as critical to infrastructure security as the defensive measures implemented by operators. While the models did not demonstrate the ability to autonomously find and attack industrial systems—requiring instead the information supplied by researchers—the rapid progression of frontier models indicates that the barrier to entry for complex OT attack planning is lowering.
Key Takeaways
- Three leading frontier AI models produced workable OT attack plans in approximately 90% of attempts when provided with context.
- The strongest foreign open-weight model outperformed the strongest U.S. open-weight model by roughly 7x in OT attack simulations.
- Researchers estimate a two-year window before AI-assisted attack planning becomes a reliable capability for attackers.
TechInsyte's Take
In our view, the Frenos report provides a necessary reality check for the "autonomous AI" hype cycle while simultaneously issuing a stern warning to OT security professionals. The data suggests that we are not yet facing a "black box" that can independently dismantle a power grid, but we are rapidly approaching an era of "augmented adversaries." The most significant finding is the reliability gap; the fact that top models fail one-third of the time during repetition indicates that current AI-driven attacks will be "noisy" and prone to errors.
This provides a tactical advantage for defenders, provided they prioritize detection and response capabilities that can identify these non-linear, error-prone AI patterns. However, the two-year countdown is a legitimate strategic deadline. Organizations cannot afford to treat AI as a distant threat; they must use this window to integrate AI into their own defensive telemetry and move toward holistic system exposure assessments.
Questions & Answers
How reliable are current frontier AI models for planning OT attacks?
While the top three frontier models produced workable plans in 90% of guided attempts, their reliability drops to roughly 66% when tasks are repeated under identical conditions. This inconsistency means they currently require human intervention to correct skipped steps or false assumptions regarding access.
What is the primary difference between the capabilities of U.S. and foreign open-weight models?
According to the research, the strongest foreign open-weight model outperformed the strongest U.S. open-weight model by approximately 7x. This suggests that current U.S. open-weight models may not yet be viable tools for sophisticated OT attack planning.
Have AI models demonstrated the ability to conduct fully autonomous attacks on industrial systems?
No. The research did not demonstrate autonomous attacks. The models functioned based on information supplied by researchers and were scored on their ability to plan and execute within controlled, non-executable simulations rather than finding and attacking targets independently.
What strategic actions does Frenos recommend for critical infrastructure defenders?
Frenos recommends that defenders prioritize detection and response to catch the "noisy" signals left by AI-assisted activity, utilize AI within their own environments to leverage existing data, and shift from assessing individual vulnerabilities to assessing exposure across entire systems.
Source: Frenos
