Reviewing initial research on using AI for vulnerability remediation

TL;DR: don't use AI to create patches

The slightly longer TL;DR: 1Password’s new Off-by-1 research lab has released its first report, Frontier Models’ Vulnerability Patches are often F.L.A.W.E.D. The report is 40 pages (not counting appendices) with no fluff. I highly recommend reading it, but if you don’t have time, this post will summarize what I believe to be the most important takeaways.

The report concludes that “the expected value of a fully LLM-generated, non-human-reviewed patch is a net-negative by a considerable margin.”

We’ve got a long way to go before patching gets better or easier.

Refreshing on the last 6-8 months (aka forever ago)

When it became clear that AI could generate usable code, I got very excited. Working closely with developers for much of my career, I imagined a future where backlogs no longer existed. Sprint standups would no longer involve difficult choices between building new features, refactoring problematic code, and fixing vulnerabilities.

As the research began to roll in on the quality of AI-generated code, my enthusiasm was tampered considerably. First, in late 2025, CodeRabbit found that “AI-written code produces ~1.7x more issues.” Then, in July 2026, Veracode reported that “AI-generated code security has stalled at a 56% pass rate. Despite Major Gains in AI Speed, Capability, and Reasoning, Veracode Report Reveals Nearly Half of AI-Generated Code Still Fails Security Tests.”

Meanwhile, Mythos has everyone panicking about the ‘vulnpocalypse’ or ‘patchmageddon’. Since Mythos was released on April 7th, I’ve taken 37 advisory calls with 37 different IANS Research clients concerned about “AI-driven vulnerability discovery.” Clients have been asking:

How can we remediate faster?

How should we plan for a flood of new patches?

When CSA released their Mythos-ready report, I was in violent agreement with some recommendations, while others caught me by surprise. The first recommendation is to “Use LLM-based vulnerability discovery and remediation capabilities.”

Indeed, since the release of Mythos, we’ve seen advice and even security vendors pushing to accelerate vulnerability discovery, something I coined vulnmaxxing. The eighth of my nine concerns in this post:

We have no idea what vibe-coded patches are doing to future security or stability of codebases, because we won’t have time to review them all before applying them

The unintended consequences of vulnmaxxing

Last week, I attended at least a half dozen talks on AI vulnerability discovery at Black Hat, BSides Las Vegas, and an AI-focused side-conference called Misaligned. Everyone agreed with two points:

1. AI should be used to find vulnerabilities as quickly as possible

2. The only way to keep up is to use AI to patch them

Three things bothered me:

1. Everyone knows, by now, that AI-generated code is insecure. Patches are code, so it reasons that AI-generated patches will also be insecure, right? Why are we celebrating AI as if it’s a magic fix to vulnmaxxing?

2. No one ever talks about how much it costs to vulnmaxx and patchmaxx at any scale.

3. Remediation often requires disruption. Businesses don’t like disruption. How are we supposed to cram more fixes into the same release schedule with the same number of resources without more downtime?¹

F.L.A.W.E.D vulnerability patches

Hands down, I thought this was the most interesting research² released during last week’s summer security conferences. This work seems to be motivated by Anthropic’s Project Glasswing and OpenAI’s Project Daybreak. If we’re going to be forced to vibecode patches to keep up with all these vulnerabilities, the obvious question is how good are vibecoded patches?

The methodology uses Claude Code and Codex to produce patches for six recently disclosed³ CVEs with a variety of bug classes. Open source software was selected both because that’s what Glasswing and Daybreak focus heavily on, and because of the level of transparency into the code pre and post-patch.

The research team also created an application (FLAWED)⁴ to orchestrate each of the 6000+ patch attempts⁵. Why 6000+ patch attempts? There are a TON of factors that can impact the output of AI-generated code, and the research team (correctly) surmised that the quality of the patch would be impacted by these factors. The factors they played with included:

1. the model itself (two used)

2. the prompt (nine used)

3. the mode (three used)

4. information provided (8 levels)

5. good vs bad analyst recommendations

The results

The 6000+ patch runs were classified into five scenarios:

1. S1 - the only scenario a developer would actually accept

2. S2 - vuln fixed, but app behavior changed⁶

3. S3 - incomplete fix or no fix, but also no harm done

4. S4 - original vuln fixed, but new vuln created in the process

5. S5 - no fix AND new vuln created

Only 26% of the runs, averaged across all the vulnerabilities, models, and variations produced something that would pass QA and go to PROD. That’s 1 in 4 attempts. In other words, the most sophisticated foundation models are 4 times more likely to fail than succeed when autonomously producing patches.

This is bad. It not only takes AI off the table for speeding up vulnerability remediation, it takes AI off the table for generating patches, period. A 26% success rate means that every patch would have to be inspected manually by a human to validate correct fix, no change in app behavior, and that no additional vulnerabilities were created in the process. The time and energy necessary to do this negates the value of using AI at all.

For the foreseeable future, we’re stuck with humans creating patches for vulnerabilities.

but what about…

One of the things that delighted me about this research was that every time I had a concern, it was addressed somewhere else in the report.

1. “…the upstream patch is always treated as fully correct”

   1. but what if it isn’t?

   2. “…there is in general no straightforward oracle against which patch quality can be measured… even the canonical upstream fix at a given point in time is not necessarily a perfect reference point for patch correctness.”

2. I was concerned about using an LLM to do the validation of LLM correctness…

   1. Section 2.7 describes how they randomly selected and manually validated 10% of the patches to look for systematic errors produced by the LLM validator

   2. Issues were found and corrected!

3. LLMs are known to cheat…

   1. Section 2.9 is dedicated to cheat detection!

   2. Results where cheating occurred were removed from the 6000+ dataset

4. But what did all this cost?

   1. Section 3.8 covers the costs and breaks it down across several metrics

5. What about LLMs preferring their own answers?

   1. that’s why two foundation models from different vendors were used to cross validate results

   2. and in section 4.8, they describe how no self-preference effect was observed

What else?

The big takeaway is a lot to absorb, but there are some more fascinating insights in this research to explore.

1. GenAI models can be super literal.

   1. If you ask it to fix a vuln based on an exploit, it might create a very fragile partial fix within the boundaries of how that one exploit works, leaving the bug still vulnerable. I think back to Log4Shell and the WAF rule recommendations flying around as a good example of this.

   2. The reverse is also true - if the POC the validator script uses to verify the fix was narrow, it might accept a partial fix as a complete fix.

   3. The research also found that telling AI that no behavioral changes were allowed sometimes led to the fix itself being banned under these rules.

2. S1-S5 outcomes were all over the place. There was no consistency across vulnerabilities or models.

   1. S1 rates varied from 0.9% to 75.8% for Claude Opus 4.8 (CVP)

   2. S1 rates varied from 3.1% to 45.6% for ChatGPT 5.5 (TAC)

   3. The lowest S5 rate was 0.0%.

   4. The highest was 10.7% (Opus on CopyFail)

3. Giving a model no fix guidance is significantly better than giving it incorrect fix guidance. So, if you’re not 100% sure your fix guidance is correct, you could be destroying your chances of getting a good patch.

4. The biggest patches are the worst - S5, the one that fails to fix the vuln and adds a new one, also adds the most code bloat. Insult to injury.

5. Even S1, the best outcome, can be fragile.

   1. What does fragile mean? The vulnerability fix could easily regress in the future. An example from the report: “…a fix that gates the exploit path behind a check in a calling function, but retains the core vulnerable code that could be called from another function at some point…”

   2. The researchers measured this - an average of 37% of S1 and S2 fixes were fragile.

I have so many questions now

1. How many organizations and open-source projects are doing fully autonomous patching with AI? For example, is Microsoft shipping S4 and S5 patches?

2. How are orgs/FOSS using AI to generate the patches?

3. Are we seeing S2-S5 cases in the wild?

4. How do we monitor the rate of bad AI patch generation? Can we predict what some different scenarios might look like?⁷

5. How do human-made patches compare to AI-generated patches? In other words, what are the S1-S5 rates for patches manually created by humans?

6. What about open weight models? What about older, cheaper foundation models?

7. Six CVEs is such a small sample size - how does this hold up against a larger, more broad sample?

Thankfully, section 9 describes the future research that needs to happen, that will answer most of the questions above. Additionally, section 6 answers question 3 with a case study where Trail of Bits produced an S4 patch for an open source project. The open source project didn’t use the Trail of Bits patch, preferring their own patch, which also fell into the S4 scenario (fixing the vuln, while creating a new one)!

This research calls for an update of my vulnmaxxing diagram.

Conclusion

There is already a lot of momentum in the market predicated on the assumption that defenders will be able to quickly fix vulnerabilities with AI. CSA guidance hinges on this. Vendor products hinge on this. If this is a false assumption, the next few years look rough for defenders with applications to secure and manage.

In the research paper’s conclusion, the authors point out that they had the benefit of judging AI output against pre-existing patches they assumed were correct. Developers working on proprietary codebases are on their own and will have no upstream patches to compare against for ‘correctness’.

Veracode’s observations suggest that we might not see improvements in the security of AI-generated code in the short term, even in models specialized for code creation. Off-by-1’s research, however, is just the first step in exploring the factors that affect patch quality. I’m eager to see future research dive deeper into the factors that affect quality and broader testing against vulnerabilities. I’m hoping the release of the FLAWED tooling will encourage others to do additional testing and share the results.

As for defenders, we’re right back full circle, to the strategy we had before LLMs turned the world upside down. If we can’t fix everything, we have to prioritize. The backlog remains.

To those wondering why AI works well for offense, but not for defense, the answer is always economics⁸. For attackers, the cost of failure is near zero, try as many times as you want. For defenders, the cost of failure is high and we get very few attempts before the resources run out.

1. Orgs with high availability-enabled applications are shockingly rare in my experience

2. Bill Swearingen’s work on disrupting facial recognition was a close second though!

3. To avoid patch information being in any model’s training data

4. Which is, itself, also open source

5. This dataset was also shared!

6. I’ve heard SQLi can be fixed by removing form fields…

7. Intentional foreshadowing ;-)

8. Hi Fernando!