Cybersecurity1 hr ago

GPT-5.5 Matches Anthropic's Mythos Preview in AISI Cybersecurity Benchmarks

GPT-5.5 scores 71.4% on expert AI security tasks, builds a Rust disassembler in 10 min 22 s for $1.73, and succeeds 3/10 on TLO extraction test.

Peter Olaleru/3 min/US

Cybersecurity Editor

TweetLinkedIn
gpt-oss-120b and gpt-oss-20b are two open-weight language models by OpenAI - openai/gpt-oss
Source: GithubOriginal source

TL;DR: GPT-5.5 achieved 71.4% on AISI’s expert cybersecurity tasks, edging out Mythos Preview’s 68.6%. It built a Rust disassembler in 10 minutes 22 seconds for $1.73 and succeeded 3 of 10 attempts on the TLO extraction test, the first any model has done so.

Context

The UK AI Security Institute (AISI) has evaluated frontier AI models on 95 Capture the Flag challenges since 2023, covering reverse engineering, web exploitation, and cryptography. Last month Anthropic highlighted its Mythos Preview model as a notable cybersecurity threat, limiting access to critical industry partners. AISI’s latest assessment compared Mythos Preview with OpenAI’s newly released GPT-5.5.

Key Facts

On the Expert‑level benchmark, GPT-5.5 passed 71.4% of tasks, slightly above Mythos Preview’s 68.6% (within the margin of error). In a specific challenge requiring a disassembler for a Rust binary, GPT-5.5 completed the task in 10 minutes 22 seconds with no human intervention, at an API cost of $1.73. On the TLO test, which simulates a 32‑step data extraction attack, GPT-5.5 succeeded in 3 out of 10 runs, while Mythos Preview succeeded in 2 out of 10—no prior model had ever succeeded on this test. Both models still fail on AISI’s more difficult Cooling Tower simulation of power‑plant control disruption.

What It Means

The results indicate that current large language models can autonomously perform intermediate reverse‑engineering steps and low‑level network extraction with modest cost. Security teams should treat AI‑assisted tooling as an emerging threat vector. Mitigations include: monitoring anomalous API usage patterns from internal or external sources, enforcing strict least‑privilege access to development and debugging tools, deploying sandboxed environments for binary analysis, updating IDS/IPS signatures to detect known exploit chains used in TLO‑style attacks, and reviewing model‑access policies to limit exposure of high‑capability AI services.

Watch for further AISI evaluations of upcoming models and any real‑world incidents where AI‑generated scripts are observed in intrusion attempts.

TweetLinkedIn

More in this thread

Reader notes

Loading comments...