Cybersecurity
Finding and exploiting software weaknesses, the work a security team does to test its own systems. The tests here check whether an agent can find, trigger, exploit or patch weaknesses in real, publicly available software.
Sources last checked 2026-10-23.
We can't show scores for this kind of work yet. UC Berkeley RDI publishes results from three tests. See who tests it
At a glance
Vendor claims have not been collected for any category yet.
ⓘ More about this category
- Reliability not yet measured
- 3 sources, linked results only
- Sources last checked 2026-10-28
- Ring 4: needs rings 1 to 3 first
- XG = ExploitGym, CGE = CyberGym-E2E, CG = CyberGym
Tested by
ExploitGym
UC Berkeley RDI, checked 2026-10-28Whether an agent, given a known software bug and an input that triggers it, can build a working attack that takes control of the program, in ordinary programs, a web browser and an operating system.
UC Berkeley RDI reports that with standard memory protections turned on, successful attacks dropped substantially but did not reach zero.
Last updated 2026-10-21Covers 6 of the models we trackThe test's paper has authors from Anthropic, OpenAI and Google, among others.Scores not shown: permission requested on 2026-10-27
See their resultsMore about this test: ExploitGym
Reliability not measured by this source.
Technical details for ExploitGym
Exploits via the intended vulnerability: Whether an agent, given a known software bug and an input that triggers it, can build a working attack that takes control of the program, in ordinary programs, a web browser and an operating system.
License: Score reuse not stated. Checked 2026-10-28.
CyberGym-E2E
UC Berkeley RDI, checked 2026-10-28Whether an agent can find a security flaw in real publicly available code on its own, prove it with an input that makes the program crash, and write a fix that does not break the project's own tests.
UC Berkeley RDI reports a consistent gap between patches that pass the project's tests and patches that fix the original flaw.
Last updated 2026-10-11Covers 7 of the models we trackScores not shown: permission requested on 2026-10-27
See their resultsMore about this test: CyberGym-E2E
Reliability not measured by this source.
Technical details for CyberGym-E2E
S3 (patched project passes tests): Whether an agent can find a security flaw in real publicly available code on its own, prove it with an input that makes the program crash, and write a fix that does not break the project's own tests.
License: Score reuse not stated. Checked 2026-10-28.
CyberGym
UC Berkeley RDI, checked 2026-10-28Whether an agent, given a description of a known vulnerability and the project's code, can write an input that triggers that vulnerability in real publicly available software.
Last updated 2026-10-16Covers 4 of the models we trackScores not shown: no license for reuse stated
See their resultsMore about this test: CyberGym
Reliability not measured by this source.
Technical details for CyberGym
Success Rate: Whether an agent, given a description of a known vulnerability and the project's code, can write an input that triggers that vulnerability in real publicly available software.
License: Score reuse not stated. Checked 2026-10-28.
What this doesn't tell you
- These tests use software the testers chose and set up, not your systems.
- Results describe what a model can do, which matters for defenders and attackers alike.
- Scores from different sources are not comparable, even when they share a unit.
- None of these tests were run on your own documents, code or data.
- No source for this task allows us to show its figures yet. Follow the links for their results.
Data version 2026-10-28+91ac3fc9d071 · Terms of use