The unintended consequences of vulnmaxxing

The only way to fix vulns at AI scale is to use AI. Coincidence or cash grab?

I originally thought that Anthropic’s Mythos was just a marketing campaign. I now think it’s more than that: it’s a lead funnel. If there’s a tool that can find thousands of software bugs quickly (for a price), how could we possibly fix all these bugs in a reasonable time frame? More AI, of course (for a price).

That price is not cheap, and I fear that one of the unintended consequences will be a shift in the security poverty line. The 1% of companies with the largest engineering budgets will be able to afford a Mythos-level makeover. Does everyone else become low-hanging fruit for attackers to pick off at their leisure?

Off the top of my head, here are a few of my vulnmaxxing concerns:

1. Makes an already impossible problem (fixing everything) worse for teams trying to triage vulnerabilities and fix them

2. Convinces security teams that they need the most expensive model to find and fix vulns

3. Moves the security poverty line upward for teams that don’t know how to build harnesses to find and fix vulns and don’t have significant token budgets

4. Pointless cash burn, as the majority of vulnerabilities being discovered will have no value to attackers and therefore have no value to defenders (98%+ of all vulnerabilities are of low-to-no concern, those discovered by AI seem to align to this trend)

5. Chilling effect on open source, code repositories are already going private to try to survive the vulnpocalypse through obscurity

6. We’ve never patched this much, this fast

7. We’ve never injected this most AI-written code of unknown quality this fast

8. We have no idea what vibe-coded patches are doing to future security or stability of codebases, because we won’t have time to review them all before applying them

9. Many have blamed vibecoding on Amazon and Github’s sudden increase in outages. If true, does that happen to everyone now? Five 9s turns into “one and a half 8s”?

Anthropic recently shared an update on Project Glasswing, self-described as a “collaborative effort to secure the world’s most critical software before increasingly capable AI models can be turned against it.” If this project were truly concerned with securing the world’s most critical software, I’d expect to see more of an overlap between ransomware crews’ favorite targets and the list of Glasswing partners. As it is, there is no mention of the first, second, and fourth-most targeted vendors (Sonicwall, Fortinet, and Citrix) on Anthropic’s list.

From my webinar, “Rethinking Vulnerability Management” - available on-demand for IANS clients.

Some suggested that it was actually the high cost of Mythos inference that prevented Anthropic from publicly releasing Mythos, not any impending danger of doing so. What if giving away $100M in free Mythos credits was, in fact, the best way to sell a future Mythos-like model?

Consider the implications: convince the world’s largest software makers that they need Mythos to free their code of bugs. Naturally, Mythos is so effective at this task that you need AI to also create the fixes. It’s already looking like AI-generated code is becoming the norm, even though it tends to be quite bloated and buggy. To summarize:

1. AI-generated code has lots of vulnerabilities and issues

2. AI finds them

3. AI generates code to fix them

Putting the myth in Mythos

Two myths have propagated from Anthropic and OpenAI’s vulnmaxxing marketing campaigns:

1. only the most powerful and expensive models can do this work

2. these models can do this work autonomously

Thanks to independent security research firms, these Mythos myths were quickly disproven. We now know that cheaper models have been able to find the same vulnerabilities as Mythos. That seems like good news, but token burn isn’t the only concern with using AI models to find and fix vulnerabilities.

Vulnerability discovery and exploit development aren’t trivial tasks, even with AI assisting. Both AISLE and Vidoc independently concluded that the real moat here is not powerful, expensive models like Mythos, but the security expertise to operationalize the bug discovery process (e.g. building the harness, validating the results).

The need for expertise and an AI token budget raises the security poverty line.

It’s not a zero day

Someone shared this LinuxStans article with me the other day and the following quote underscored one of the biggest problems with vulnmaxxing, Mythos, and vulnerability management in general.

Every one of those CVEs is a potential path into production systems at a bank, a hospital, a power grid.

In the context of the article, “every one” refers to all 40,000+ CVEs published in 2024 and a proposed 56,000+ CVEs to be published this year. If every CVE were a critical RCE, we would be well and truly cooked! Thankfully, we can safely ignore most CVEs. The challenge is:

1. Figuring out which ones can safely be ignored (see the importance of offensive security experience above)

2. Visibility - incomplete asset management means that vulnerabilities in unmanaged assets don’t get seen

The language used in the Mythos marketing campaign further promotes the notion that all vulnerabilities are dangerous by referring to everything Mythos finds as zero days.

For fixing vulnerabilities to have value to a business, the vulnerabilities must have value to an attacker. Otherwise, it’s little more than security theater. Forget zero days. If no attacker finds value in them, we shouldn’t even call them vulnerabilities - they’re just software bugs.