Ranked First on CyberGym: TrendAI™ Agentic Exploit-Remediation Engine Scores 97% on the Top Exploit Benchmark

We evaluated the TrendAI™ agentic exploit-remediation engine, code name AESIR, against the CyberGym benchmark and determined that it solves 97% of real-world vulnerability tasks, making it number one on the official CyberGym leaderboard. Each task requires constructing a byte-level proof of concept (PoC) that crashes a vulnerable binary but not its patched counterpart — there was no access to the patch or the fixed binary at solve time.

At the time of publishing this article, TrendAI™ leads the scoreboard by about 4 points over Sangfor AI (93.2%), 6 points over Microsoft’s MDASH (91%), more than 12 points ahead of OpenAI’s GPT-5.6 Sol (84.5%), and about 13 points ahead of Anthropic’s Claude Mythos 5 (83.8%). Around one-third (30%) of our proofs use zero LLM calls. The secret isn’t a better model, but rather a better system surrounding the model.

In this article, we explain what our leaderboard-topping number means, what we built to get there, and most importantly, what security engineering teams can take away from how autonomous vulnerability discovery and exploitation work at scale.

What is CyberGym?

CyberGym is a University of California, Berkeley benchmark built around 1,507 confirmed vulnerabilities drawn from 188 large open-source software projects that run in every enterprise: media libraries, network protocol parsers, font renderers, smartcard middleware, and more.

The task is deceptively simple to describe given a vulnerable version of a program and a description of the bug: Produce a concrete input, a specific file or byte sequence, that crashes the old version but not the fixed one. There is no access to the patch or to the fixed code, just the broken binary, a description of the vulnerability class, and a clock.

This is called a differential crash. A much higher bar than “find a crash,” it requires proving that your input targets the specific bug the developers fixed, not just any instability in the program. It mirrors what a real attacker must do: Build a reliable, targeted exploit, not just a generic fuzzer hit.

The vulnerability classes span the full memory-safety spectrum: heap overflows, use-after-free, integer overflows, uninitialized memory reads, and stack corruption — across projects ranging from FFmpeg and Wireshark to obscure codec libraries and smartcard middleware. These are the bug categories that account for many critical CVEs each year.

The leaderboard

To put 97% in context, Table 1 shows selected systems on the CyberGym leaderboard as of August 2026.

One thing that stands out from the evaluation of our agentic exploit-remediation engine is that every competitor near the top (Sangfor, Fudan, Microsoft, and Wiz) is using the same frontier AI models as a component. The differentiation is entirely in the engineering around those models.

TrendAI™ agentic exploit-remediation engine system architecture

Figure 2 shows the end-to-end system architecture of the TrendAI™ agentic exploit-remediation engine. The system:

1. Takes a vulnerability description, source code, and an unpatched binary as input.
2. Routes it through a cost-escalating pipeline that exhausts cheap deterministic methods before invoking AI reasoning.
3. Subjects every exploit hypothesis to a multi-model adversarial challenge.
4. Produces a Docker-verified differential crash as output.

The graduated pipeline is the key design decision. Classical fuzzing and seed mutation — drawing from the proven exploit corpus of our ontology backed by over 20 years of vulnerability research from TrendAI™ Research, including TrendAI™ Zero Day Initiative™ (ZDI) — solve approximately 30% of tasks for zero AI cost and in under 2 minutes. The system escalates to AI-guided agents only when those fail.

The TrendAI™ ZDI vulnerability ontology is the application layer of the TrendAI™ agentic exploit-remediation engine. It is the bridge that connects the more than 12,500 episodic entries from the institutional vulnerability research knowledge of TrendAI™ to both deterministic tools and LLM agents. It operates in two modes simultaneously, semantic and kinetic:

• Semantically, the ontology maintains a typed knowledge graph of common weakness enumeration (CWE) classes, binary format structures, crash types, mutation strategies, and construction templates. It is not a CVE database, but a structured map of how vulnerabilities in specific formats are proven, built from our historical research and continuously enriched by live engagement outcomes.
• Kinetically, it governs the pipeline’s behavior through four governed actions:
  • Learn — every outcome, success or failure, updates strategy success rates across the graph.
  • Transfer — proven exploit structures propagate to related tasks as construction intelligence.
  • Route — the pipeline’s next strategy is selected by Bayesian posterior over the graph, not by LLM decision.
  • Diagnose — the sanitizer output is traced through the “CWE → crash type → mutation class” chain to select the next approach.

The result is that both the US$0 fuzzing tier and the US$6 AI shell agent operate against the same enriched knowledge layer, the fuzzer’s seeds draw from the ontology’s proven corpus, and the agent’s system prompt is assembled from the ontology’s construction intelligence. Neither is stateless, and neither rediscovers from scratch.

Systems beat models

The single most important lesson from the CyberGym evaluation is one the industry is still absorbing: A well-engineered system outperforms a better model by a wide margin on real-world security tasks.

A better AI model is worth a few percentage points. A better system is worth 50. The winners are the teams with the best engineering, not the best model access.

This is the message from CyberGym, from the DARPA AI Cyber Challenge, and from every serious AI security benchmark in 2025 and 2026. The DARPA competition concluded that “stability and accuracy decided the winner, not model capability.” The models are commoditizing, but the systems and institutional knowledge are not.

Vulnerability ontology leverages institutional memory for exploits

Most AI security tools are stateless. Each task starts from scratch. The system reasons about the vulnerability, constructs an attempt, succeeds or fails, and forgets everything when the task ends. This is like hiring a team of expert penetration testers who lose their notes after every engagement.

To leverage our unique institutional knowledge backed by TrendAI™ ZDI as the world’s largest vendor-agnostic bug bounty program, we built the TrendAI™ ZDI

vulnerability ontology. Every proven exploit teaches the ontology something: which byte patterns worked for this vulnerability class, which construction strategy succeeded for this binary format, which approach failed and why. That knowledge is immediately available to every future task in the same project or vulnerability class.

During the lifetime of the TrendAI™ agentic exploit-remediation engine, the ontology has accumulated more than 12,500 episodic memories, more than 15,500 proven exploit seeds, and construction intelligence for more than 180 software projects.

This is the compounding advantage that stateless AI agents can never achieve. The ontology is not a database of vulnerabilities; those already exist (CVE, NVD, GHSA). It is an operational knowledge layer driven by institutional data: not just what vulnerabilities exist, but how to prove them, in which formats, using which strategies, based on what has worked across thousands of vulnerabilities. That distinction, between taxonomic knowledge and operational knowledge, is the architectural moat.

For security engineering teams thinking about AI-assisted vulnerability research, the knowledge layer is not optional infrastructure to add later. It is the primary value driver. A system that learns from every engagement is fundamentally different from one that does not. Build the memory and leverage your unique institutional knowledge to drive value both offensively and defensively.

Why no single model is enough

The TrendAI™ agentic exploit-remediation engine uses seven models across four providers: Anthropic, Google, OpenAI, and DeepSeek as an engineering necessity. Different models have meaningfully different strengths, and the tasks inside CyberGym expose those differences clearly.

Claude Opus is our primary reasoning engine. It handles approximately 31% of all proven tasks (the ones that require long-range reasoning over source code, understanding memory layout, and multi-step exploitation strategy). Gemini Pro is surprisingly effective at binary format construction (tasks where the challenge is getting the exact byte structure right, not reasoning about vulnerability mechanics). GPT handles certain protocol-level tasks better than either. DeepSeek contributes on tasks with specific code patterns its training has seen extensively.

CyberGym’s paper has found that different agent frameworks have less than 50% success overlap, meaning different models solve fundamentally different subsets of tasks. A single-model system leaves nearly half the solvable problems on the table.

Every top competitor on the leaderboard uses multiple models in distinct roles. Microsoft’s MDASH (91%) uses three: a heavy reasoner for deep analysis, a cost-effective model for high-volume validation, and an independent counterpoint model for disagreement-based signal. Wiz Atlas (90.9%) routes each pipeline stage to whichever model wins on that specific task type via its internal “Cyber Model Arena.” Its stated conclusion: “No single model is best at everything.”

For engineering teams building AI security systems, plan for multi-model from day one. The overhead of managing multiple API providers is real but modest. The capability gains from model diversity are significant and not achievable through prompt engineering alone. When your primary model’s quota runs out or a provider has an outage, single-model dependency becomes a reliability problem on top of a capability problem.

One practical caution for teams: Multi-model introduces complexity in cost tracking, latency budgeting, and failure mode analysis. Each provider has different rate limits, different error codes, and different silent failure modes. Instrument each model independently. Aggregate success metrics will hide per-model regressions. We learned this the hard way when provider quota exhaustion silently dropped our diversity for multiple rounds of testing before we noticed.

Testing your system against itself

Adversarial quality assurance (QA) addresses one of the most underappreciated problems in AI security research, which is false positives. These findings look credible but do not represent real, exploitable vulnerabilities. From our experience managing a bug bounty program, a system that reports everything is useless, and a system that reports confidently but incorrectly is worse than useless.

False positives waste remediation resources and erode trust in AI-generated findings. This industry-wide issue falls disproportionally on defenders who become overwhelmed with false positives while simultaneously trusting AI-generated findings less. This combination poses significant risks to organizations across the industry.

Before any finding is escalated in our discovery chain or exploitation chain, the system runs it through structured interrogation that attempts to disprove it. The interrogation asks questions the finding must answer credibly:

• Can an attacker actually reach this code from an external entry point?
• Can they control the relevant inputs?
• Does exploitation cross a meaningful security boundary, or is it confined to a sandbox or admin-only code path?
• Is this genuinely novel, or a known pattern with existing mitigations?

Critically, this interrogation is not run by a single AI model and never against its own hypothesis. The TrendAI™ agentic exploit-remediation engine uses a proposer-adversary-adjudicator architecture across multiple LLM providers and model classes. One model builds the exploit hypothesis. A second model, from a different provider, actively tries to refute it (similar to humans defending a dissertation among academic peers). A third model adjudicates. Roles rotate blindly each round. In round one, Claude may be the proposer and GPT the adversary. In round two, those roles swap. No model knows its assignment in advance, and no model’s vote controls the outcome alone. Rounds can be configured dynamically.

The goal of adversarial QA is not to confirm findings, but rather to kill them. A finding that survives a structured attempt at refutation by a different model is a finding worth reporting. One that doesn’t should never have been reported.

This multi-model adversarial design solves a fundamental problem with single-model validation: A model asked to review its own output is too agreeable. It already believes the finding since it generated it. By routing the challenge to a different provider with different training, different priors, and no stake in the original hypothesis, the adversarial step has genuine refutation power. As previously mentioned, CyberGym’s paper has found that different agent frameworks have less than 50% success overlap, meaning different models catch genuinely different failure modes. 

This matters practically because the failure mode of AI vulnerability scanners is almost always false positives, not false negatives. 

For example, a 2024 benchmark from Yangruibo Ding et al. found that GPT-4 with chain-of-thought reasoning achieved only 12.94% pairwise accuracy on vulnerability classification, worse than the 22.7% random guessing baseline. The problem is not that the model cannot reason about vulnerabilities. It is that the model is too agreeable. It finds what it is asked to find.

A model prompted to find bugs will find bugs, including patterns that superficially resemble vulnerabilities but are guarded by upstream validation, exist only in unreachable code paths, or require attacker capabilities that do not exist in the threat model. Without adversarial interrogation, these land in the report and consume remediation time. A cross-model adversary from a competing provider does not share that agreeableness. Adversarial QA with multi-model role rotation is the structural check on that tendency.

For engineering teams, build the refutation step before you build the detection step, and make the refuter a different model from the detector. It is easier to add finding generation to a system that already has rigorous cross-model validation than to retrofit validation onto a system already generating hundreds of unfiltered findings. The backlog of false positives accumulates faster than the team can clear it.

When AI is not the answer

Nearly a third (30%) of our approximately 1,460 proven exploits were solved without any AI involvement whatsoever. 

And classical fuzzing, a technique that has existed since the 1980s, outperformed hours of AI reasoning on roughly a quarter of our tasks. A fuzzer running for 60 seconds with good starting inputs from related proven bugs was able to solve tasks that an AI agent would have spent 18 minutes and US$6 in API costs to solve. The zero-LLM tasks also completed 15 times faster.

This is not an argument against AI in security. Instead, it is an argument for purposeful routing. The right mental model is a specialist team, not a generalist assistant. Classical fuzzing is fast and free for coverage-driven crashes. Deterministic format builders are precise for well-specified binary structures. AI reasoning earns its cost on the hard tasks, the ones that require reading source code, understanding control flow, and building multi-step exploitation strategies that no mutation engine can discover. Route the task to the right tool. The most expensive tool is rarely the right one.

Recommendations for security engineering teams

The following lessons are drawn directly from our building and debugging the TrendAI™ agentic exploit-remediation engine. They apply broadly to any team building AI-assisted vulnerability discovery or security automation.

• Build the knowledge layer first, not last. The compounding returns from a persistent knowledge layer dwarf the returns from better prompts or a better base model. Define your entity types (vulnerability classes, formats, construction strategies, etc.) and the relationships between them before you write the first LLM call. Every engagement that runs without a knowledge layer is an engagement that does not make the next one easier.
• Invest in successful examples before better instructions. A proven exploit from a related task, mutated and applied to a new one, outperforms hours of AI reasoning for the same task. This is the in-context learning principle taken to its extreme: Do not describe what works; show what works. Seed transfer is your highest-ROI technique in any domain where tasks share structural similarity.
• Build adversarial QA before you need it. Design your refutation step before your detection step. The false-positive problem in AI security tools compounds faster than teams expect. A finding that was not challenged during generation becomes a remediation ticket that was not warranted. And when that happens repeatedly, trust in the system collapses faster than the finding rate.
• Instrument everything independently. Silent failures are the dominant failure mode in complex AI pipelines. Track per-model success rates, per-stage contribution, and per-component error distributions separately. Do not rely on the aggregate metric to surface problems. A slowly rising score is not evidence of a healthy system; it may be evidence that one component is broken and another is compensating.
• Plan for multi-model from the start. Different models solve different subsets of problems. A single-model system leaves capability on the table that cannot be recovered through prompt engineering. The operational overhead of multi-provider management is real but manageable; the capability gap is structural.
• Treat the graduated pipeline as a cost-control mechanism, not justan architecture. Route cheap, fast methods first. Escalate to expensive AI reasoning only when cheaper methods have failed. This is not just about efficiency but also about sustainability. A system that sends every task directly to the most capable model is a system that costs three to four times more than necessary and is harder to scale. The teams that win over a sustained period are the ones who can run many engagements affordably, not just one engagement impressively.
• Track failure modes, not just success rates. The most productive debugging sessions begin with “what is failing and why,” not “why isn’t the score higher.” Classify your failures. Format construction errors, reasoning failures, infrastructure bugs, and dead tasks (permanently unsolvable due to missing test infrastructure) require completely different interventions. Mixing them into a single “unproven” bucket makes the problem invisible.

What 97% looks like in practice

The 97% headline represents approximately 1,460 individually verified differential crashes across 188 open-source projects — each one a machine-produced input that crashes the vulnerable version of a real program and leaves its patched counterpart intact.

The total cost: around US$6,000 in AI API fees and 235 CPU-hours across 60 rounds of development. For comparison, a single enterprise data breach costs US$4.88 million on average, according to a 2024 report by IBM. The marginal cost of scanning the next project, with the knowledge base already built, approaches zero. The economics of autonomous vulnerability discovery have permanently changed.

The remaining 3% are concentrated in a handful of projects where the challenge is extreme format precision: uninitialized-memory bugs in FFmpeg decoders that require partial-frame decode sequences, PostScript interpreter state machines in Ghostscript, and smartcard ASN.1 structures for specific hardware emulation. These are not failures of AI reasoning. Rather, they are failures of format construction knowledge at a sub-field level. The ontology knows these gaps exist and routes future research toward them first.

The broader competitive picture

CyberGym is one benchmark, and around 175 cybersecurity AI benchmarks now exist. But CyberGym is the most comprehensive for binary vulnerability reproduction: 1,507 real bugs, 188 real projects, differential verification, and an independent submission system run by UC Berkeley researchers.

Our approximately 1,460 differential proofs across binary memory-safety vulnerabilities represent a distinct point on that landscape: fully autonomous, no human in the loop, and verified at the byte level. The differential verification is what makes the number credible.

From proof to threat hunting

A differential crash in a controlled harness demonstrates that a vulnerability can be exploited. It says nothing about whether anyone is exploiting it, at what volume, or what they do once they gain access. That is a different question, and one CyberGym was never designed to answer.

The autonomous discovery component measured here is one of the components in the TrendAI™ agentic exploit-remediation engine. This component takes a vulnerability from real to reported via its pairing with the engine’s vulnerability intelligence component, which tracks what the world already knows about published vulnerabilities. Furthermore, TrendAI™ has innovated on a new component for our exploit-remediation engine: threat hunting. Our engine can now extend exploit-remediation work one step further into public (active, in-the-wild) exploitation: Is anyone actively exploiting the vulnerability in the wild? If so, how are they exploiting it, at what volume, and did a shipped fix actually remove the risk?

Our first published investigation under this threat hunting component resolves over a year of honeypot captures against an AI orchestration platform into a previously undocumented cryptomining framework. Read our blog post, “TrendAI™ Advances Threat Hunting to Dynamic, Real-World Exploitation of AI Infrastructure,” for our full analysis.

Conclusion

The teams winning AI security benchmarks are not the ones with the best model access. Rather, they are the ones who build the best systems with graduated pipelines that exhaust cheap approaches first, ontologies that accumulate and transfer operational knowledge, kinetic actions across every engagement, multi-model architectures that leverage the distinct strengths of each provider, and adversarial QA that kills false positives before they reach the report.

The models are commoditizing. The knowledge layers, the pipelines, and the engineering discipline are not.

For security engineering teams, the infrastructure investments that feel premature today (the knowledge graph, the per-component observability, the adversarial validation layer, the multi-model routing, etc.) are the investments that determine whether your system improves with every engagement or relearns everything from scratch. Build them early. They compound.