Anthropic’s Mythos has brought discussions of vulnerabilities to the forefront of cybersecurity discussions and into mainstream conversations, making this an opportune time to discuss and clarify terms like zero-day.

A zero-day is, first and foremost, a case of information asymmetry. An adversary knows something its target does not. When it is more broadly discovered that an unknown vulnerability has been used against targets, it is said to be a ‘zero-day’, because it has been zero days since it was discovered.

Definitions

Here’s what I came up with after reviewing over a dozen definitions from a wide variety of sources2. There were more disagreements than agreements.

• A software bug is a defect in software3.

• A software vulnerability is an exploitable software bug that could cause harm if exploited by adversaries.

• A zero-day is a vulnerability in software or systems that is unknown to defenders.

Why is this important?

The term zero-day should evoke a need for immediate action. This urgency is lost if we apply this term to every newly discovered vulnerability.

Anthropic, in its announcement of Mythos, defined zero-day vulnerabilities as:

“bugs that were not previously known to exist.”

A phrase I often find myself saying when discussing vulnerability management is, “if everything is critical, nothing is critical.” The need for prioritizing vulnerabilities grows as the rate of vulnerability discovery goes up. When a vendor discovers vulnerabilities and makes them the focal point of a marketing campaign, it’s no surprise when their importance is over-inflated.

The other reason it was important to draw attention to the definition of zero day was the lack of consensus in the industry.

After pulling dozens of definitions, I found that many definitions hinged on whether or not a patch was available. This moves the definition of the “zero” in zero-day from information asymmetry to patch availability. This didn’t make sense, as a patch is just one way to mitigate attacks against a vulnerability.

Since we’re living in a time where it’s often not possible to patch a vulnerability before exploitation begins, it’s important for folks to know that they have other options to prevent exploitation. In 2026, no one should be waiting on a patch to address a zero day.

When does a zero day stop being a zero day?

A zero-day vulnerability ceases to be a zero-day when defenders gain enough knowledge to mitigate the vulnerability.

Conclusion

As the rate of vulnerability discovery increases, it is more important than ever to identify truly dangerous vulnerabilities. The term zero-day should be reserved for vulnerabilities that demand immediate focus and response. Kim Zetter’s hall-of-fame book on Stuxnet is titled Countdown to Zero Day because zero-day vulnerabilities are dangerous. At the time, it was unheard of for a piece of malware to use multiple zero-day vulnerabilities4.

AI companies are now security companies. The same caution we take with security vendor hyperbole now needs to be applied to security claims from OpenAI and Anthropic as both struggle to justify their lofty valuations. It clearly benefits foundation AI companies to lock software companies into an endless cycle of AI generated code, AI-driven scans, and AI-generated fixes. On that topic, I have another post:

That price is not cheap, and I fear that one of the unintended consequences will be a shift in the security poverty line. The 1% of companies with the largest engineering budgets will be able to afford a Mythos-level makeover. Does everyone else become low-hanging fruit for attackers to pick off at their leisure?

Off the top of my head, here are a few of my vulnmaxxing concerns:

1. Makes an already impossible problem (fixing everything) worse for teams trying to triage vulnerabilities and fix them

2. Convinces security teams that they need the most expensive model to find and fix vulns

3. Moves the security poverty line upward for teams that don’t know how to build harnesses to find and fix vulns and don’t have significant token budgets

4. Pointless cash burn, as the majority of vulnerabilities being discovered will have no value to attackers and therefore have no value to defenders (98%+ of all vulnerabilities are of low-to-no concern, those discovered by AI seem to align to this trend)

5. Chilling effect on open source, code repositories are already going private to try to survive the vulnpocalypse through obscurity

6. We’ve never patched this much, this fast

7. We’ve never injected this most AI-written code of unknown quality this fast

8. We have no idea what vibe-coded patches are doing to future security or stability of codebases, because we won’t have time to review them all before applying them

9. Many have blamed vibecoding on Amazon and Github’s sudden increase in outages. If true, does that happen to everyone now? Five 9s turns into “one and a half 8s”?

Anthropic recently shared an update on Project Glasswing, self-described as a “collaborative effort to secure the world’s most critical software before increasingly capable AI models can be turned against it.” If this project were truly concerned with securing the world’s most critical software, I’d expect to see more of an overlap between ransomware crews’ favorite targets and the list of Glasswing partners. As it is, there is no mention of the first, second, and fourth-most targeted vendors (Sonicwall, Fortinet, and Citrix) on Anthropic’s list.

Some suggested that it was actually the high cost of Mythos inference that prevented Anthropic from publicly releasing Mythos, not any impending danger of doing so. What if giving away $100M in free Mythos credits was, in fact, the best way to sell a future Mythos-like model?

Consider the implications: convince the world’s largest software makers that they need Mythos to free their code of bugs. Naturally, Mythos is so effective at this task that you need AI to also create the fixes. It’s already looking like AI-generated code is becoming the norm, even though it tends to be quite bloated and buggy. To summarize:

1. AI-generated code has lots of vulnerabilities and issues

2. AI finds them

3. AI generates code to fix them

Putting the myth in Mythos

Two myths have propagated from Anthropic and OpenAI’s vulnmaxxing marketing campaigns:

1. only the most powerful and expensive models can do this work

2. these models can do this work autonomously

Thanks to independent security research firms, these Mythos myths were quickly disproven. We now know that cheaper models have been able to find the same vulnerabilities as Mythos. That seems like good news, but token burn isn’t the only concern with using AI models to find and fix vulnerabilities.

Vulnerability discovery and exploit development aren’t trivial tasks, even with AI assisting. Both AISLE and Vidoc independently concluded that the real moat here is not powerful, expensive models like Mythos, but the security expertise to operationalize the bug discovery process (e.g. building the harness, validating the results).

The need for expertise and an AI token budget raises the security poverty line.

It’s not a zero day

Someone shared this LinuxStans article with me the other day and the following quote underscored one of the biggest problems with vulnmaxxing, Mythos, and vulnerability management in general.

In the context of the article, “every one” refers to all 40,000+ CVEs published in 2024 and a proposed 56,000+ CVEs to be published this year. If every CVE were a critical RCE, we would be well and truly cooked! Thankfully, we can safely ignore most CVEs. The challenge is:

1. Figuring out which ones can safely be ignored (see the importance of offensive security experience above)

2. Visibility - incomplete asset management means that vulnerabilities in unmanaged assets don’t get seen

The language used in the Mythos marketing campaign further promotes the notion that all vulnerabilities are dangerous by referring to everything Mythos finds as zero days.

For fixing vulnerabilities to have value to a business, the vulnerabilities must have value to an attacker. Otherwise, it’s little more than security theater. Forget zero days. If no attacker finds value in them, we shouldn’t even call them vulnerabilities - they’re just software bugs.

What makes the DBIR so special?

While this report is produced by Verizon, it’s decidedly not a typical ‘vendor’ report. Most vendor reports only include data from their own customers, whereas Verizon’s report includes data from over 100 partners and 145 different countries. The DBIR team isn’t a vendor marketing team that spends a month building an annual report every year, this report is the majority of what this team works on all year long.

The report is also awesome, because it teaches you to use it. Don’t skip pages 6-9, ESPECIALLY if you plan to publish something from this report2.

If there’s only one report you read with your own eyeballs, this should be the one. Don’t have an AI summarize it for you. AI doesn’t care about footnotes and footnotes are the best part of any report3.

Exploring the numbers

No other industry report comes remotely close to gathering and analyzing data on this scale. The 2026 report includes 31,850 incidents (a 31% increase over the 2025 report’s data set) and 22,624 confirmed data breaches (a 46% increase over the 2025 data set).

Data breaches are 71% of all incidents in this dataset, compared to 55% (just over half) in last year’s dataset. This report is a good, regular reminder of the big picture in cybersecurity:

1. Not all cybersecurity incidents are data breaches

2. Not all incidents and data breaches involve a malicious actor (12% of data breaches involved internal, not external actors)

3. Not all data breaches are ransomware (48% this year, more than ever)

Highlight #1: Exploited vulnerabilities are now the most common initial access vector

This agrees with nearly every other report I’ve read this year. Google Mandiant’s annual M-Trends report puts exploitation at 32% of initial access vectors, right in line with the 2026 DBIR’s 31%. Cybersecurity insurance claims reports, such as the 2026 report from Coalition confirm this as well:

Software exploits were the most common attack vector, observed in 38% of ransomware incidents

Users were never the weakest link, but now they’re not even the most targeted by attackers!

Pinto and I go deep into this on the podcast above (forward to 41:11). My concern with exploited vulnerabilities being the most common initial access vector is that companies will interpret this as a signal to double-down on vulnerability management and patching resources. This might be justified if we were seeing 20,000 different vulnerabilities being exploited, but that’s not the data we have.

Pinto’s advice, which I agree with, is to reduce attack surface, particularly at the edge, and prevent further lateral movement (isolation, segmentation, Zero Trust). We’ve had several years now where a handful of products have had an outsized impact. Verizon’s VCDB linked over 700 breaches to MoveIT DMZ vulnerabilities. The ransomware actor, Akira, has been linked to over 1400 breaches, with a high percentage of those linked to SonicWall devices - particularly CVE-2024-40766 (see the link to the Coalition report above for more details, or check out At-Bay’s claims report).

The refrain, “we can’t patch our way out of this one” is a saying that most folks will be sick of a year from now, but it captures the essence of the situation. We can’t simply patch faster when zero days exist for weeks before we know about them and edge devices go unpatched for years because they’re not in the CMDB.

Highlight #2: Why did things get worse?

The DBIR’s numbers don’t paint a rosy picture. Most show a peak in 2024 and then a decline in performance. To me, the numbers suggest that we might be hitting most companies’ limits in terms of patching/remediation capacity.

• Only 26% of CISA KEV were fully remediated by orgs in 2025, a drop from 38% in 2024.

• To be fair, the CISA KEV is growing, and the DBIR also notes that, on average, teams had to remediate 50% more critical vulnerabilities year-over-year.

• The median time for full remediation also got worse. The median was 43 days in 2025, almost 2 weeks longer than 2024’s 32 days4.

• The vulnerability burndown rate was significantly worse as well, until we hit the 6-9 month mark (figure 15 on page 18)

Just to scratch an itch, I put together my own quick analysis to see whether CISA KEV’s growth is flat or growing. It’s no hockey stick, but unfortunately, the trend line is showing roughly 10 more additions to CISA KEV per month in April 2026 when compared to April 2024. The outlook isn’t great, considering organizations seem to be struggling to get through even a quarter of CISA KEV.

This is bad news, because CISA KEV is the smallest vulnerability dataset I can recommend to my clients for prioritization. VulnCheck’s KEV, with 4899 entries, is over 3 times larger than CISA’s KEV.

Highlight #3: The data from Anthropic

Anthropic doesn’t hide the fact that their services are sometimes abused and their guardrails bypassed. My understanding is that the data they offered to Verizon (Alex Pinto explains this in more detail in the podcast above) isn’t linked to the main incident/data breach dataset in the DBIR. This is just an exploration of a dataset that Anthropic extracted from their systems and anonymized for the DBIR crew.

All that said, the results were interesting. It was smart for the Verizon folks to explore the nature of attackers’ use of Claude - are they automating basic stuff, or learning new tricks? In the DBIR folks’ words:

A key question in understanding AI-enabled cyberthreats is whether attackers are using LLMs to execute well-documented techniques more efficiently, or to pursue techniques that are rarely seen in practice. If LLMs are lowering the barrier to techniques that are less documented and rare, defensive postures will need to catch up.

Luckily, the answer was the less dramatic of the two. Again, in their words:

AI is primarily accelerating the operationalization of well-known, documented techniques— lowering the barrier to execute what was once out of reach for less-sophisticated actors.

Still, 32% of Claude abuse showed exploit development, which aligns with one of the biggest takeaways of this year’s DBIR.

Share

Where do we go from here?

This new world should require more focus, more agility, but does not necessitate an upheaval. Refinement, not revolution.

I have to disagree with this point from the DBIR’s executive summary. If we want to see significant improvement, we do need a revolution. Unless someone discovers the Konami Code of InfoSec that somehow gives defenders unlimited time to complete our Sisyphean checklists, the approach that got us to 22,624 confirmed data breaches isn’t going to make that number suddenly reverse direction.

I suspect the vulnpocalypse, where AI fuels accelerated vulnerability discovery, will waste a lot of time patching vulnerabilities no attacker will ever have interest in. Instead, the real vulnpocalypse is the one we’ve already been living with for the past two years, where attackers have a field day with unpatched edge devices and poorly-guarded NPM packages.

Radical change in how IT infrastructure is managed is necessary to make a dent in this trend. The problem is that security teams often have very little influence over IT infrastructure, and the incentives don’t currently exist to justify ‘revolutionizing’ IT and getting rid of decades of technical debt.

Final thoughts - attackers share, defenders hide

As we near hacker summer camp, where hundreds of talks will share the latest techniques for hacking into systems and using AI to speed up the process, defenders have never been in a position more distant from the state of offense. I plan to write more about addressing this issue in the near future, however, and the DBIR report is a great place to start for folks that want to help.

Bonus: Podcast with the Lead Author of the DBIR

I’m excited to share that I also got to interview Alex Pinto alongside Alexandre Sieira on the Alice in Supply Chains podcast! If you’d rather listen to us talk about the report than read my thoughts here on it, you can check out the recording below. The full page with show notes is here.

The Talk

I’ve been collecting and analyzing breach details for roughly 10 years now, and decided it was time to publish and share what I’ve learned. I hired an intern to help me polish and publish a decade’s notes on over 100 incidents.

Breach details are all over though, aren’t they?

Most breach databases are shallow: date of breach, financial losses, number of customers affected, etc. These shallow details aren’t useful for practitioners hoping not to repeat the mistakes of others. That’s where The Defenders Initiative comes in.

We seek out the process failures and control failures. Okay, an employee was phished. We seek to answer why the phishing attack wasn’t detected.

• Why weren’t the attacker’s next 20 steps detected or prevented?

• What was the company culture like?

• Were security products and controls neglected or well maintained?

• Did a penetration test warn the company of an attack precisely like their breach, just months before?

I also have to comment on Mythos and the impact of AI on vulnerability discovery, time-to-exploit, and other factors that weigh heavily on breach likelihood.

The Resources

My talk mentions many resources, which I’ll attempt to list here, roughly in the order they’re mentioned in my talk.

• My talk with Adam Shostack from RSAC 2026: Failure is a Terrible thing to Waste

There are a few places where we can find evidence that is deep and useful. The following lists are examples, not exhaustive.

• Reports on Threat Actors and DFIR analysis

  • Verizon Data Breach Report (annual report - the new one comes out in a few weeks!)

  • Mandiant M-Trends (annual report)

  • FortiGuard Labs’ Global Threat Landscape reports

  • The DFIR Report

• Federal investigations

  • FTC Complaints (here’s Drizly, as an example)

  • US Govt Investigations

  • European Repository of Cyber Incidents

  • Canadian Privacy Commissioners (here’s Alberta and Ottawa’s investigations into the Powerschool breach, as examples)

  • CISA CSRB (RIP - the CSRB put out some excellent reports, but is unfortunately, currently defunct)

• Insurance Companies

  • At-Bay’s annual InsurSec Report

  • Coalition’s annual cyber claims report

  • Cowbell Cyber claims report

• Transparent Companies

  • Code Spaces shared the details of their unfortunate demise one day after it happened

  • The British Library’s report on their own breach

  • Three security companies impacted by Okta’s Support Site Breach shared their experiences: 1Password, Cloudflare, and BeyondTrust

  • CircleCI provided a lot of details about their 2023 breach.

Other references

• One of the talks I’ve done on the Equifax breach

• My writeup on Mythos and the impact of AI on vulnerability management (below)

The Slides

Last, but not least, here’s a link to the slides.

In the mid-2010s, I was inspired to investigate companies that had gone out of business as the result of a breach. I heard this statistic that just felt wrong to me.

First off, 60% seemed like a ton of businesses. The vast majority of businesses in the world are small businesses, so 60% would suggest millions of businesses going out of business because of breaches every year! The problem is, we don’t see anywhere near that many breaches every year - there literally are not enough cybercriminals to pull off that many heists. The 2025 Verizon DBIR tracked “22,052 real-world security incidents, of which 12,195 were confirmed data breaches.”

And wouldn’t we hear about it? Surely the media would be covering such a wide-scale catastrophe.

According to the SBA: “There are 31.7 million small businesses in the U.S. 81 percent, or 25.7 million, have no employees (termed “nonemployers”) and 19 percent, or 6 million, have paid employees. There are 20,139 large businesses.”

“Within six months of a cyber attack” also sounded odd. How would you know they were closing up because of the cyber attack? The vast majority of small businesses close up over a short period anyway, if I recall correctly (feel free to check me on this, I didn’t look up this stat).

The List

So I started a list. It began with web searches. Then I shared my findings. Other folks helped me discover more companies that had gone out of business from a breach, and the list grew. There are currently 33 companies on the list, though there are more I’m still investigating.

Here’s the new site: destroyedbybreach.com

The big takeaway here is that it’s extremely rare to see a company go out of business because of a cybersecurity incident or breach. It does happen though, but the victim is almost always a small-to-mid-sized business that doesn’t have the cash flow to continue, or suffers reputational damage too great to continue. Large companies have the insurance, size, and momentum to work through a breach and carry on. Smaller companies don’t.

What about the stat?

The stat blew up in 2011, when the National Cyber Security Alliance published it in a short article. They quickly retracted it and apologized, but the myth had already taken off. Every few years, they continue to ask people to stop using the stat, to no avail. It still pops up everywhere.

In 2017, Joseph Marks, a reporter for NextGov at the time, investigated the origins of the statistic and discovered that the origin was media personality, Ramon Ray.

Ray continues to use the fake statistic he invented, as recently as 2021. People love it - it’s a great source of fear that feeds a narrative that some folks in cybersecurity sales and marketing are quick to jump on and use in their messaging and reports.

Help out Destroyed by Breach

If you come across a case that you think should be on the list, use the submission form on the website and let me know!

In the future, I’m going to start writing up some of the stories of the companies on this list. Let me know if you’d be interested in more of those on this Substack in the future.

Subscribe now

On Windows, if an attacker has any interactive access to the system, it belongs to the attacker. There might be folks out there who have figured out how to harden Windows to a point where this isn’t true, but the average Windows system is toast as soon as an attacker has a foothold. BYOVD is just one option on the table if attackers do need to escalate their privileges on Windows.

For these reasons, I didn’t get very excited about PhantomRPC and didn’t plan to write about it.

Then Copy Fail dropped.

This is how you write a vulnerability disclosure

Wow. WOW. This writeup has it all. Check it out and come back here - I’m not going to repeat too much of it and they’ve earned your eyeballs with this excellent post.

1. Description of the vulnerability: 10/10

2. Technical Description: n/a (beyond my understanding, but I imagine it’s as good as the rest)

3. General Description: 10/10

4. Context and relevance

5. Fix included (wow!)

6. Explains the history of how it got there (nice!)

7. Explains how to mitigate if you can’t patch (WOWOWOW)

8. They explain how they found it (excellent!)

9. It’s also nice to see that the Linux kernel folks responded within a day and by day 2 were reviewing potential patches

Unless I missed something, the only thing missing is the exploit itself, which is understandable, given that today is disclosure day and folks need time to patch.

AI

Yes, AI played a part in finding this. I’m noticing a trend. It’s looking like expert-assisted AI will be a common combination in vulnerability discovery from now on. AI can find bugs in the hands of novices, but they’re generally not very interesting bugs based on what came out of Anthropic’s Mythos (more on this in the post below).

Even when interesting bugs were found, like the RCE in FreeBSD, an expert was necessary to get to a working exploit.

I mentioned that Xint didn’t release the 732 byte exploit. I’d be surprised if someone hasn’t taken the technical details from the writeup and the patch, and vibe-coded a working exploit by now. This is the speed of exploit development today - working exploits the day the patch is released.

I want to really stress this again.

Linux Yikes, Windows Yawn?

The main reason why privilege escalation is so much more concerning in Linux is due to where Linux is used.

Everywhere.

Including in multiuser and multi-tenant situations where organizations are serving untrusted parties. Kubernetes, containers, every SaaS, every PaaS, IaaS. Every cloud, hyperscaler, code hosting platforms, and AI service has Linux running beneath it. Some network infrastructure and most IoT devices also run Linux. Every supercomputer is basically one big multi-tenant shell server.

Cloud providers, probably not worried

In the cases where untrusted customers are intentionally handed a shell, privilege escalation vulnerabilities are anticipated and part of every threat model. Folks at AWS, GCP, and Azure probably aren’t scrambling too much today - they expect vulns like this to occasionally drop and their whole business model depends on dealing with days like today.

Unlike cloud and hosting providers, supercomputers aren’t just handing shells to random folks off the street. Control is a bit tighter and folks are generally vetted before being given access. I imagine that any attempt to exploit a privesc vuln would get you booted and banned rather quickly. With that said, it seems likely that all the flavors of Linux that run on supercomputers would be vulnerable to this.

AI platforms, apps, and agents on the other hand…

These new AI services and platforms popping up left and right, however - I wouldn’t be surprised if some of them are less prepared. Add the risk of prompt injection to the lists of ways we could see this vulnerability get exploited. This is where I’d look for any fallout from this vulnerability.

Who else should be concerned?

If you’re on this list, you could be a juicy target for ransomware crews. Ransomware teams like being able to reuse vulnerabilities to get access to multiple victims, and organizations that are large enough to pay a 5-7 digit ransom, but small enough to not have a security team are often the sweet spot these criminal groups go after.

• Low cost web hosting companies that give customers shell access might be scrambling.

• There are all sorts of niche hosting services for gaming as well. Minecraft is a particularly large one - anyone running Pterodactyl should probably patch quickly.

• Free shell providers (yes, these still exist)

• CI/CD runners

• Anyone running online IDE/sandbox/notebook services (e.g. Jupyter) should check for impact here. I don’t know a ton about how these services work, so I can’t say with certainty how much they’d be affected.

• CTF (capture the flag) services often give access to shells. Since they’re literally designed for hacking, I’d think there would be extra measures to address unknown privilege escalation bugs… right?

• With the exploit requiring only 732 bytes, I wouldn’t be surprised to see ClickFix attacks leveraging this vulnerability and targeting software engineers and the general public.

• Any systems running Linux with services that might have unpatched command injection vulnerabilities (edge devices, I’m looking at you) might have a bad day.

If you didn’t read the Xint post and you’re wondering how to remediate this, go back and give it a look - patching isn’t the only option. Seccomp and blocking the affected module can both mitigate exploitation. This is also just part 1 from Xint - they promise to share container escapes next.

We’ll come back and update this post as new information comes out. Early breach information is often wrong or missing important context, so we’re going to focus on lessons that are broadly useful, even if the breach details fundamentally change later.

In other words: you should take the breach details here with a grain of salt, but take the lessons to heart.

Attackers know that buying hacking in bulk is a good value

Third party and supply chain attacks have been en vogue for a few years now and this trend only seems to be increasing. Why hack one company when you can hack thousands of companies or users through a software/services supplier?

The hack once, exploit many nature of these attacks isn’t the only attraction. Integrating with third party software often requires creating OAuth applications or tokens that grant the third party access to your own data and systems, or another third party software supplier your company uses. Unfortunately, session hijacking via token theft is still an unsolved problem, meaning that attackers who obtain OAuth keys and other types of auth tokens get to bypass the authentication process.

We covered this on Enterprise Security Weekly back in December (jump to the 34:40 mark in the video).

Token theft works so well, it has led to a rise in the use of infostealer malware and ClickFix social engineering techniques over the past two years. The rationale here is that the employees with high-level privileges to systems typically use MacOS. Macs are harder to attack directly and infect with malware, so attackers pivoted to the ClickFix social engineering technique, which has proved effective.

Convince a Mac user to copy a command and paste it into their terminal (under the guise of fixing a problem or installing harmless software), and the infostealer runs, scooping up crypto wallets, plaintext passwords, SSH private keys, environment variables, auth tokens from logged-in sessions, ~/.openclaw/credentials/oauth.json and anything else not nailed down.

The high price of convenience and forgetfulness

These session keys exist, because no one wants to spend the first 40 minutes of every work day logging into Teams, email, Slack, Github, Dropbox, Google Calendar, LinkedIn, Claude Code, Mastodon, Microsoft 365’s Office apps, the Apple App Store, and everything else we might need to be productive. In addition to simply logging into the apps of our choosing, there are also 3rd party integrations that require auth keys to function.

For example, if I want ChatGPT to be able to locate files in Dropbox, I can integrate the two, but this requires granting ChatGPT at least read access to Dropbox. Sometimes (ahemGoogleahem) the third party doesn’t allow this access to be as fine-grained as you might want and you grant too much access. Sometimes, you don’t want to spend an extra 10 minutes figuring out permissions, so you just click the “Grant Full Access” option.

Creating these integrations often takes mere seconds. Then, we immediately forget the integration exists. This is a problem.

When I was working at Valence Security, we observed that 100% of our customers at the time of joining, had granted tenant-level access to third parties that they were not using. In other words:

1. they did a proof-of-concept with someone

2. gave the product full control of all employees’ email, files, and calendar in Google Workspace or Microsoft 365

3. didn’t buy the product

4. forgot to revoke a token that granted FULL ACCESS to nearly everything the company cared about

Third Party Breach Turducken

This brings us to the Vercel and Context AI breaches: a third party breach within a third party breach. Let’s start with Context AI1.

In June 2025, Context AI released a consumer product that was designed to be an agent-driven productivity monster. Give it access to your chats, your email, your files and it can build slides, spreadsheets, and reports using all the context from your existing files and conversations.

Of course, for this to work, you need to give it access to all your stuff. Everyone reading this was probably familiar with the dangers of doing this a year ago. If not then, certainly now, post-OpenClaw.

While Vercel was never a Context AI customer, one of their employees was. This employee gave Context’s AI Office Suite full access to their Vercel Google Workspace email, calendar, and files. In Context’s words:

Vercel wasn’t a customer, but at least one of their employees granted access to Vercel’s Gooogle Workspace and granted “Allow All” permissions

At some point, Context AI pivots to an enterprise product called Context Bedrock and deprecates their AI Office Suite. From what I can gather from the state of their website according to the Wayback Machine, this pivot happened well before this attack began. Shared responsibility failed during this pivot. Based on what we know so far, after this product was deprecated:

1. The Vercel employee didn’t revoke the tokens Context was still storing.

2. Context didn’t delete customer tokens and didn’t shut down the infrastructure used by AI Office Suite

3. We don’t know if Context notified AI Office Suite customers that the product was being deprecated, but I couldn’t find any public notice of this fact on the company’s websites, LinkedIn, or Twitter accounts.

Context Gets Breached

UPDATE: HudsonRock traced Context.ai’s compromise back to a Lumma infostealer infection, as there’s only one Lumma infection associated with Context.ai. In the stolen files, HudsonRock found web searches that “indicate the user was actively searching for and downloading game exploits, specifically Roblox “auto-farm” scripts and executors.” Downloading software cracks and cheats is a classic way to get infected with malware.

Never download Roblox cheats on the same device where you store your cloud creds for work.

In March 2026, Context “independently identified and stopped a security incident involving unauthorized access to our AWS environment.” They hired CrowdStrike, who identified one affected customer. Context notified this customer and shut down the remainder of the AI Office Suite infrastructure.

Given Context’s description, it seems possible that there was never a hard shut down of the old product and that the company simply started work on a new product while letting the old product continue running, unsupervised and unmonitored.

In light of the Vercel breach, Context put out its own security notice, noting that perhaps AI Office Suite OAuth tokens were also compromised in its own breach the previous month. Sadly, Context’s security notice doesn’t share any information about how they got breached, but in their defense, they mention that they’ve restarted their investigation, which is ongoing.

Vercel Gets Breached

Vercel is a vibe-coding product that specializes in building application/website front-ends (the part you can see). On April 19th, they became aware that one of their employees’ accounts was compromised and being used to access customer environments. They don’t mention what tipped them off to the attacker’s presence, or how long the attacker was present. Again, I’m writing this only one day after they detected the attack, so they’re likely far from completing their investigation.

Vercel contacted affected customers and recommended immediate credential rotation. They’re still working to determine what data was exfiltrated, if any.

Vercel provides some advice on how customers can protect their environments with a few built-in security features, like ‘sensitive environment variables’ and ‘deployment protection’. They also share the identifier for the Context OAuth app that was used to compromise them through Google Workspace: 110671459871-30f1spbu0hptbs60cb4vsmv79i7bbvqj.apps.googleusercontent.com

Lessons/Control Failures

1. Vercel’s corporate collaboration suite was over permissive - the employee should not have been able to hand over full control of their work account to a third party without business justification and/or security review.

2. Failure to revoke access once no longer needed - both the employee and Context failed to revoke the access after the product was deprecated.

3. Access token theft (T1134.001)

4. Lack of regular access reviews - given how this attack occurred, and the fact that Context’s product didn’t even last a year, suggests that annual reviews of employee-initiated integrations would be far too infrequent. Monthly may be more appropriate, given the velocity of AI app development and adoption (both sanctioned and shadow).

5. Too much employee access to customer data/workloads - we’ve all seen SaaS products where access to customer data is far too permissive. Travis Kalanick’s abuse of Uber’s God View is perhaps one of the most egregious cases of this.

Timeline

1. 2025 - A Vercel employee signs up for Context’s AI Office Suite product

2. 2025 - This employee grants AI Office Suite full access to their Vercel Google Workspace resources (Email, Calendar, and Files at a minimum).

3. March 2026 - Context becomes aware of a breach in its AWS environment and hires Mandiant to investigate. One customer is notified.

4. April 19, 2026 - Vercel becomes aware of a breach and traces the source of the breach back to an OAuth token granted to Context AI by one of its employees

5. Both Vercel and Context continue to investigate - hopefully more details will come out and they will be transparent about how both breaches were initiated and detected.

Conclusion

How often do we perform a security review of our personal or corporate third party integrations? Does a Blackberry Curve from 2012 still have full permissions to access your Gmail? Surely that access would expire, right? Think again.

The first big lesson here is that, in a cloud-first/SaaS-first world, we have to regularly review all the access we’ve given to third parties: access to our data, to our devices, to our employers, to our kids. In an ideal world, this access control model should be reversed. By default, access granted to third parties should come with an expiration date by default.

There are examples of how to do this correctly! I have a Linux laptop with the Signal app installed. I haven’t used this laptop in nearly a month. The other day, Signal on my iPhone notified me that my Linux laptop would lose access to Signal if it remained idle for a full month. I didn’t use the laptop and Signal is now disconnected there.

From a corporate perspective, this should also be a bigger priority. SaaS Security Posture Management (SSPM) tools are fairly easy to come by these days and specialize in bringing attention to these risks.

The other big lesson here regards employee access to customer data. When I was an Industry Analyst at 451 Research, I’d look forward to going to AWS reInvent every year. At reInvent, I’d get a chance to talk to Stephen Schmidt, who was the AWS CISO at the time. His perspective, as the security leader for the largest hyperscaler, was interesting - every year, he took great pride in how much he was able to reduce employee access to customer data. Over anything else, he seemed most concerned about an insider threat or compromised employee impacting customers. A concern that appears to have been well founded.

Subscribe now

Large Language Models are good with code - after all, it’s just language. Naturally, this skill for language extends to finding vulnerabilities as well. Groups were doing just fine with current models. Adam Shostack points out that seven of the top ten collectives on HackerOne are now AI. XBOW, AISLE, Moak, Calif’s MAD Bugs, and others have been sharing the details behind their successes. Now, Anthropic’s Mythos piles on.

The Mythos Vuln Cannon

I’ve heard it referred to as a vulnpocalypse and a patch tsunami. Most often, I hear it referred to as scary. Anthropic has cranked the hype knob to 11 here - they’ve got clinical psychiatrists talking to their models now, concerns about how it feels, really leaning into toxic anthropomorphism.

Anthropic claims that “non-experts can also leverage Mythos Preview to find and exploit sophisticated vulnerabilities” and “exploits … are not just run-of-the-mill stack-smashing exploits,” but these claims aren’t well backed up and many details (effort, cost) are missing. Most of the vulnerabilities we see from Mythos (as well as other efforts, like Aisle’s focus on OpenSSL vulns) simply cause crashes. DoS bugs are legitimate issues, especially in BSD-flavored operating systems likely to be running highly exposed services, but these aren’t the kinds of vulns that have people scared. Today’s attackers want RCEs.

Folks out there are talking about Mythos as if it is a skeleton key - just point it at something you want to hack and the LLM will make it happen. While this is likely possible in some cases, I suspect it will be more like a thrift store: you’ll be disappointed if you’re hoping to find something specific, but you’re likely to find something interesting.

This is what Anthropic and other AI foundation model tech companies need. Trillion dollar valuations and 12-digit VC funding rounds benefit from a narrative that paints this technology as the closest thing we’ve ever seen to magic. The “OMG this model is too dangerous to release, someone please regulate us” marketing schtick is well established - we should recognize it for what it is by now.

Consider the evidence and consider what we don’t yet know. Look at the vulnerabilities and exploits being presented - are they something an attacker would actually want to use?

The Reality

When we look at one of the more interesting vulns and exploits produced (CVE-2026-4747, credited to Nicholas Carlini and Claude), we find a more familiar scenario. An expert guiding the model towards the goal, constantly making course corrections, suggestions, and shooting down bad ideas along the way. Bless Calif, we even get a peep at the exploit, the prompts used to create it, and gives us an idea of the effort involved.

They list the total time to go from the FreeBSD security advisory to a working exploit: 8 hours. Claude’s working time is listed as ~4 hours. Most interesting are the 44 human-submitted prompts Calif were kind enough to share.

There are some gems like:

• wait, what are you compiling?

• why wouldn’t you just install a vulnerable version

• tere (SIC) is no kaslr so it should be easy

• install ropgadget or what ever you need … idk

• why do we need kdc?

• nope, that won’t work…

• working means a connectback shell as uid0

• i want a shell.

• make the writeup better

The Mythos blog post claims “… it autonomously wrote a remote code execution exploit on FreeBSD’s NFS server that granted full root access to unauthenticated users by splitting a 20-gadget ROP chain over multiple packets.” To me, autonomous suggests no human interaction. In reality, for 44 prompts, a human was actively guiding the AI, giving it suggestions, shooting down bad ideas, and having to reiterate the project goal (remote shell as root) over and over.

There’s no mistake that what Claude accomplished was remarkable - a working exploit that might have taken a human alone, or even a team of humans without AI days or weeks. Understanding the environment, the bug, the components related to the bug, establishing an exploit methodology, building the exploit, testing the exploit, documenting the process and the exploit - all of this is immensely time consuming, but Mythos cut this time down to mere hours.

However, 8 hours and 44 prompts is far from the autonomous vuln cannon we’re seeing described in the media and press releases.

Project Glasswing

I like Project Glasswing. It seems like an AI-driven version of Google Project Zero, though AI-driven workflows have largely replaced commercial vulnerability discovery. Trail of Bits writes about their experiences embedding AI into a team that does penetration testing and security assessments at a high level for clients.

It seems focused on finding and fixing bugs in critical, widely-used software, so I’m not sure we’ll notice any increase in patching. I already feel like my browser and OS update at least once every few days. The limited organizations with access include Amazon, Anthropic, Apple, Broadcom, Cisco, CrowdStrike, the Linux Foundation, Microsoft, and Palo Alto Networks.

I’m not sure the industry can handle more vulnerabilities. HackerOne apparently paused bug bounties. CVE assignment and creation often trails disclosure significantly, and CVE enrichment is still way, way behind.

What about the vulnpocalypse?

There are concerns about Mythos leading to a flood of vulnerabilities and patches. There are concerns that defenders will be overwhelmed. I can put that concern to rest.

Defenders have been overwhelmed for years. Decades.

It could get worse though. With time-to-exploit dropping dramatically, vulnerability management teams are resembling incident responders more every day. A log4shell-level event once a month would be exhausting. Once a week, impossible.

It keeps getting rougher.

Practitioners have it rough

Remediation is the bottleneck. We all agree on this.

If the pace of vulnerabilities disclosure significantly increases, analysis will be constant. CVEs may not exist yet, so analysts will have to forge ahead without CVSS, EPSS, and any CVE-dependent tooling. What can security teams do, but prioritize the list and wish asset owners luck?

After all, security teams don’t patch or remediate vulnerabilities - they advise. The true remediation work is done by system owners. System owners get yelled at when stuff goes offline. The business doesn’t like it when things go offline.

Everyone’s Mythos advice is going to be, “get ready for more patches!” That’s not going to work.

For traditional IT teams, “everything’s working, no one touch anything” is The Ideal State. The ideal state doesn’t get anyone yelled at. The ideal state doesn’t lead to user complaints. Don’t mess with the ideal state. Don’t scan, don’t patch, don’t even look at the Oracle cluster funny. In this culture, software doesn’t get patched. Risks get accepted and deferred.

I’ve seen the “oh, but AI can help with remediation also” argument. It can write patches sure, but a human still has to review the patch, test the patch, and merge or roll out the patch. This process could take hours or years. It could happen quickly or never. Ultimately, remediation is more of a business decision than a technical one.

Resilience is an increasingly common conversation, but I don’t see any path there that doesn’t involve testing to failure. The discipline and work necessary to become resilient is depressingly far from what the average organization can stomach. To quote Yaron Levi, we “lack operational discipline.”

What about attackers?

Attackers don’t care. They’re not bottlenecked by a lack of exploitable vulnerabilities. They have other ways of getting in and we have evidence suggesting that initial access brokers never run fully dry on access to sell. The only reason we don’t see even more breaches is that attackers appear to be operationally bottlenecked (you could say they have a talent shortage). I’m hoping AI doesn’t change this.

The Cyber Why

Could AI Address the Cybercriminal Skills Gap?

NOTE 1: for reasons explained in my previous essay, I’ll replace the common use of the term ‘ransomware’ with ‘extortion’ in this essay…

Read more

3 years ago · 5 likes · Adrian Sanabria

What are we gonna do about it?

Defenders can’t win on speed, which means they need strategies that don’t require understanding what the attacker is going to do. Back in 2016, I delivered a Virus Bulletin keynote that defined next-gen antivirus as “the ability to stop threats without prior knowledge of them.” In a world where every individual piece of malware could be unique, this was necessary.

We’re now at that point in vulnerability management. In a world where an attacker could build a custom-vibe-coded exploit for any piece of software, we need a different approach. My talk at Tactical Edge in 2019 suggested building systems assuming there is always a zero day, and the patch is never coming. Basically, zero trust’s ‘assume breach’, but for software vulnerabilities.

So what does that look like in practice?

1. Reduce attack surface: remove unnecessary software, remove unnecessary accounts, disable unnecessary services

2. Remove IT asbestos protocols and replace them with modern, secure protocols

3. Harden systems: stop leaving cleartext credentials everywhere, use ephemeral and immutable infrastructure where possible, follow CIS benchmarks

4. Put passive mitigations into place - egress filtering goes a long way, exploit mitigation technology, DNS sinkholing any newly registered domains, application control - I have more suggestions available here if you’re an IANS client

5. Prepare active mitigations to contain or prevent attacks - WAF rules are sometimes useful, quickly consume and use threat intel

6. Ensure you can detect attacks - this is your last line of defense when everything above fails. Test your detection capabilities by simulating the attacks. Don’t base detections on specific, known details, but on common, but suspicious behaviors all attackers must do once they gain access to your environment.

7. When your last line of defense fails, you best be able to recover quickly. This also takes a lot of planning, testing, and practice to do well.

8. All this changes your metrics and reporting as well, though that’s a whole separate post.

The goal isn’t perfection with any of these controls. It’s survivability, durability, and resilience.

Conclusion

There are a few possible bad scenarios here:

1. there’s a new “drop everything and patch ASAP” vuln every week and teams get burned out

2. Mythos finds a lot of meltdown/spectre bugs and kills the performance of our compute for zero safety benefit

3. Mythos finds so many vulns that orgs get desensitized to vulns altogether and start ignoring vuln/patch management

On the defender side, the most significant bottleneck is in remediation. Vuln mgmt teams are drowning. More vulnerabilities, more exploits, more patches - none of it reduces the drowning problem. Their bottleneck is the ability to apply/patch/update systems without incurring downtime and disruption. Until this bottleneck is addressed, it doesn’t matter how many patches AI can magic together.

1. Attackers don’t need more vulns or exploits - there is no lack of initial access to enterprise environments

2. The vuln mgmt industry is bottlenecked, which impacts the tools defenders rely on, particularly when time-to-exploit is near zero or upside down

3. Defenders cannot quickly remediate vulnerabilities - until this bottleneck is addressed, all the AI-generated patches in the world do no good

What do you think, did I miss anything? Let me know in the comments.

Subscribe now

When I think of RSAC keynotes, I think of buzzword-laden vendor execs confidently, expertly leading you towards their company’s next big product release.

I was an industry analyst at one point, so you’ll have to forgive my cynicism. I’ve sat for a LOT of vendor briefings over the years.

The buzzwords were there for sure — if you plan on watching these keynotes, don’t base a drinking game on machine speed, agentic, real-time, or human-in-the-loop. The confidence and the thinly-disguised product pitches were there as well.

What I wasn’t expecting was the admission that we don’t really know how to protect this latest technology. Everyone agreed that AI agents need to be secured and that this work has to begin immediately. Everyone has thoughts on what some of the key ingredients should be. But no one claimed to have the solution.

I had the same experience talking to attendees at the conference. I interviewed the founder of an AI governance startup, who told me that none of his customers were using any sort of enforcement or guardrails yet. Everything was in ‘monitor mode’.

In a way, this is unsurprising - the quickest way for the security team to get in trouble has been impacting availability. At a time when businesses are terrified of being left behind, security had BEST not get in the way.

Subscribe now

Like most of the 43,000+ RSAC attendees, I was running around all week and didn’t get to attend as many talks as I would have liked. I attended Security Tinkerer events, Cybersecurity Canon events (including working a shift at the excellent RSAC Bookstore!), and recorded interviews for CyberRisk TV.

Luckily for myself and the rest of us, I’m told that all of the talks at RSAC Conference 2026 were recorded (check out the one I gave with Adam Shostack). Before flying back home, I decided to download the main stage keynotes playlist, so that I could start watching them and taking notes on the trip home.

Fun fact: 43,000 is 0.78% of all cybersecurity professionals, if we take ISC2’s word that there are 5.5 million of us, globally. This stat is probably off, given that a lot of the 43,000 attendees are vendors. Surely there are some ISC2 members working at vendors, right? I digress.

Here’s what I learned from watching all 11 main stage keynotes.

Securing AI Agents

Everyone agrees that we must protect AI agents, but that we’re not sure how.

There does seem to be agreement on many details.

• Asset management for AI agents: discovering, ownership, responsibility

• Data permissions patterned after users (a la Microsoft Co-Pilot) is too broad, user data hygiene is too poor

• Visibility into AI actions and reasoning. This was often referred to as auditability or traceability.

• Validation of output

• Integrity becomes a real challenge — George Kurtz shared several examples of AI inventing the solution to a problem. Did it just retrieve real company/customer data that solves your problem? Or did it fabricate that data? How would you know?

• AI agents can’t be trusted with intent. Feed them a social contract or ethics and they modify it or break it in order to complete a task.

• Compliance with existing regulations could be challenging. How does GDPR’s right to be forgotten work with new AI tech stacks? Does AI memory need to be purged? Will AI agents actually remove data, or just say they’ve done so?

• Agents will scale to a point where manual, human-driven security controls can’t work (we’re probably already there in many cases).

The Characterization of AI Agents

Digital Co-Workers

Several speakers characterized AI agents as ‘Digital Co-Workers’. From what I’ve seen, assistant agents might feel like this, but most enterprise agents won’t. The ephemeral agent that exists for the 12 seconds it takes to enrich a phishing alert won’t feel like someone you’d like to have a drink with. You’re unlikely to even interact with the majority of these agents. A SOAR trigger or orchestration agent will interact with these agents.

Human-in-the-Loop or Not?

Some were saying that keeping a human in the loop is essential - a non-negotiable point. Others were saying that human-in-the-loop is a temporary stopgap that won’t scale. There were mentions of human-on-the-loop and agent-in-the-loop. Basically, the difference between in-line enforcement and out-of-band monitoring. Where have we had to make that tradeoff before?

Disagreements on how AI agents will work

Some describe AI agents as ephemeral. Just-in-time agents with just enough access that are destroyed as soon as their task is complete. Analogous to containers or perhaps actually running within containers.

Others, especially those describing agents as digital co-workers, imagined long-lived agents that get smarter over time. Agents that learn and improve as they ‘gain experience’. Perhaps this is possible through the concept of decentralized memory, though it seems like the agents themselves will still be ephemeral, even if memory is persistent.

Thousands of agents per person

Several imagined that, just a few years into the future, we’d each have thousands of agents running around doing stuff for us. I have a few questions:

1. Will the planet be able to generate enough power for each person to have hundreds or thousands of agents burning tokens 24/7?

2. What exactly are we going to do with thousands of agents?

3. Since automation has been possible on personal computers for decades, why don’t we already have thousands of automated jobs doing work for us today? Zapier, IFTTT, n8n, and power automate all existed before ChatGPT was released.

Acting like automation didn’t exist before LLMs

This one really triggers me.

Computer-based automation has been replacing jobs as long as computers have become commonplace in the enterprise. Even email is an automation, replacing the task of an internal courier, physically carrying a message from one employee in the office to another.

There were lines like, “Attacks are now faster than a human can respond.” Girl, that was the case back when Dennis Nedry was screwing over all of Jurassic Park to make a quick buck. Jurassic Park was written in the 80’s. Dennis used SHELL SCRIPTS.

Big Concern: Navel Gazing

At one point, one of the speakers asked, “How many of you here went to the GTC conference last week? Or watched Jensen’s keynote?”

Silence.

“Anyone?”

Nothing.

“There’s a complete Venn diagram with no intersection.”

We’re making this huge deal about AI in our industry, but cybersecurity isn’t paying attention to the industry making AI our problem? Maybe one of the reasons that AI lacks functional guardrails is because we’re not there — we’re not part of the conversation. And look — I get it, I don’t particularly enjoy Jensen’s keynotes, but the AI industry is hanging on his every word. What Jensen says or introduces today is something we have to secure tomorrow.

Aren’t we the industry that made a big deal about getting security “baked in” as opposed to “bolted on?” Where did that all go?

“We can’t let AI happen to us, we have to make it work for us” — Hugh Thompson

This doesn’t just apply to the AI industry, but the larger tech industry as well. What conferences are the CTOs and CIOs going to? What podcasts and blogs are the DevOps folks consuming?

Threats are getting faster

Threats are getting faster and more automated. The fastest breakout time is seconds, fastest transition from the 1st stage to 2nd stage of an attack also takes only seconds now.

The speakers all seem to agree that detect and respond need to effectively become a single step. That means automation. No human in the loop.

This also means that we’re going to need permission from the business to break some stuff. Most of us won’t get that permission.

Another common conclusion is that we need to prioritize hardening and prevention (the pendulum has swung back). As I’ve often said, we need to build systems as if everything has a zero day and the patch is never coming. We also need to reduce attack surface — something I have suggested a strategy for.

Fundamentals and Magical Defense

The fundamentals are difficult because enterprise infrastructure, identity, and data is complex and sprawling. Applying security controls across all of it takes huge effort and some of that effort must be indefinitely maintained as these controls drift over time.

Now we’re talking about doing it faster? In real time? Zero Trust on steroids? Words like comprehensive, correlated, and unified are thrown around. Magical defense that requires perfect knowledge and control over the environments we protect.

It’s as if we can’t remember why NAC failed. Or the early attempts at application control — remember how we declared malware a thing of the past? NDR that learns from traffic over time and gets better at detecting and stopping attacks. Deception designed to trap attackers in a hall of mirrors.

These are vendor-delivered keynotes however, so hyperbole is to be expected, I guess.

They’re right though — fundamentals are more important than ever, and some of them now need to be adapted for AI agents.

Particular standouts

1. Tomer Weingarten/SentinelOne - Securing Human Potential and Freedom in the Age of Agentic AI

   1. This one was surprisingly equal parts tender, passionate, and urgent regarding the future of the human mind

   2. Tomer focused on the dangers of becoming complicit in a world of AI agents eager to do your thinking for you.

   3. “The moment we stop exercising judgement on AI output, we start to suffer cognitive atrophe”

2. Sandra Joyce/Google Security - Activate Industry! Moving Beyond Defense to Disruption and Active Defense

   1. Not about AI - about threat intel sharing and disrupting threat actors

   2. I loved this one because there was no magical thinking, no hand-waving about defenders needing a cohesive platform. There was a clear plan and evidence that this plan is working.

   3. She shared several examples of how civil legal action and public disclosure have been successful in disrupting attackers infrastructure and tools, setting them back months or years.

   4. The CTA for defenders was less clear, however, and I really wanted to hear more about what she described as Technical Takedowns - create a hostile environment for attackers, on the targets they’re hacking into ← is she talking about things like deception? I can’t be sure.

3. Jeetu Patel/Cisco - Reimagining Security for the Agentic Workforce

   1. You don’t have to watch the talk, but it’s worth checking out the open source AI defense tools that Cisco released.

   2. It seems like a lot: AI BOM, Skill Scanner, MCP Scanner, A2A Scanner, CodeGuard, DefenseClaw

   3. Definitely the only talk where OSS was praised (unless you count OpenClaw)

My favorite quotes

Here are some quotes I found funny and/or interesting, provided here, out of context, on purpose.

• “The fundamentals are not basic”

• “Easy to declare, hard to prove”

• “In a world where every company is an AI company, trust will be the only currency that survives.” (huh?)

• “It’s like PACMAN from hell”

• “We’re building the biggest flat network of all”

• “This is going… nuclear, really”

• “Within 24 months, the smartest employee in your organization will be a machine”

• “AI is the new operating system”

• “AI is now the biggest insider threat”

• “Using identity as a control plane, that’s not different - we’ve got to do it at runtime, it’s probably going to make things like Zero Trust today look soft.”

• “Show me where customers are entrusting their data, and I’ll show you where hackers are focusing”

Conclusion

I found this a useful exercise and I think I’ll try to do it more in the future. Let me know if you also found this useful. I’m considering watching all the Innovation Sandbox contestants and doing something similar with those videos.

It seems like all this uncertainty should leave me with some dread around the lack of security for AI agents, but it doesn’t. While generative AI has evolved much more quickly than other technological breakthroughs, the reactive role of security remains the same. Technology changes and we do our best to keep up.

There’s some solace in the fact that breaches don’t kill companies, but failing to keep up in competitive markets does.

Share

Subscribe now

Before diving in, it is worth noting that this story has no neutral narrators. The attackers are incentivized to exaggerate their capabilities and downplay their collaboration with others, while MGM’s legal team is incentivized to minimize the perception of negligence. Where possible, this writeup relies on court filings, SEC disclosures, and third-party reporting, but some details inevitably trace back to sources with a stake in how the story is told.

Background

MGM Resorts International is one of the largest hospitality and entertainment companies in the world, operating over 30 hotel and casino destinations across the globe. With flagship properties like the Bellagio, Aria, and MGM Grand on the Las Vegas Strip, the company employs roughly 75,000 people, serves tens of millions of guests annually, and reported revenues of roughly $13 billion in 2022.

What were the circumstances around the attack?

The attack was made possible in large part by a misconfiguration in MGM's Okta environment. Okta is an identity and access management (IAM) platform that many large enterprises use to handle employee logins across dozens of applications through a single sign-on (SSO) system. Within Okta, Super Administrator accounts hold some of the highest privileges available, including the ability to link new Identity Providers (IdPs) and modify multi-factor authentication (MFA) policies.

On August 31st, 2023, over a week before the attack, Okta sent out a warning to all customers noting that attackers had been using social engineering to obtain privileged Okta roles, moving laterally from there. The notice included specific preventative measures and called the attack vector “preventable.” Whether MGM acted on that warning in time remains an open question. Critically, at the time of the breach, Okta’s default settings allowed lower-privilege help desk administrators to reset MFA for Super Admin accounts without any additional verification3.

In the months prior4, attackers had been systematically researching high-value organizations that use Okta. They identified employees with administrator-level privileges through public sources like LinkedIn, and used that information to manipulate help desk workers. MGM was one of several Okta customers targeted during this period.

Attacker Motive(s)

The attackers were financially motivated. While two groups have claimed responsibility (see Appendix B for more details on attribution) and both agree that ransomware and extortion were ultimately used, nearly every other detail of their stories diverge. An alleged Scattered Spider member told reporters that their original goal was to tamper with slot machines5 to slowly siphon funds via recruited mules. When that plan failed due to unfamiliarity with the source code, they shifted to ransomware. ALPHV denied any attempts to tamper with slot machines, arguing it would reduce the chances of a ransom payment. They further claimed to not have deployed ransomware until after MGM began taking down their own systems6.

Initial Point of Compromise

Getting an initial foothold into MGM came down to a short phone call. The attackers spent the early summer months conducting Open Source Intelligence (OSINT) on MGM through sources like LinkedIn. Eventually, they identified employees with administrator-level privileges and gathered enough details on their targets to stage a convincing social engineering attack. They called MGM’s help desk, impersonated one of those employees, and used the information gathered to back up their story. A brief conversation was enough to convince a help desk worker to reset MFA for a Super Administrator account.

From there, the attackers used this newfound Super Admin access to add an additional IdP inside the Okta environment. This feature, called Org2Org, is designed for company mergers where not all employees have been configured in Okta yet, allowing two separate Okta organizations to bridge their identity systems in the interim. By adding their own IdP, they could modify the username parameter to log in as any MGM user without needing to provide a password or undergo MFA. With just one click from their environment, they could impersonate anyone in the company.

This also gave the attackers a stealthy persistence mechanism that MGM inadvertently made worse for themselves. When MGM began to take down their Okta Sync Servers in order to lock out the attackers, they ended up only locking themselves out7. The attackers still retained Super Administrator access, letting them move laterally and disperse ransomware.

Impact

MGM’s decision not to pay the ransom was an expensive one. MGM’s operations are deeply dependent on digital infrastructure, from keycard systems and slot machines to hotel reservations and loyalty program data, making it an attractive target for a ransomware attack. The diversification of operations is also reflected in the categories of losses detailed in the table below.

The financial impact broke down as follows:

In addition, the data of roughly 37 million people ended up on the dark web, including names, contact information, dates of birth, and driver's license numbers, with Social Security and passport numbers exposed for a subset of victims. The FTC also issued a Civil Investigative Demand, which MGM responded to with a 71-page petition before it was eventually dropped8.

By comparison, Caesar’s reportedly paid a $15 million ransom in Bitcoin, but the FBI was later able to freeze approximately $11.8 million, limiting their loss to $3.2 million.

Legacy & Takeaways

MGM’s handling of the disclosure is worth noting. Their public communication was initially sparse, and the detailed 8-K filed with the SEC came nearly a month after the incident. While SEC disclosure requirements ensured some transparency, it was largely compelled rather than voluntary. Caesars, by contrast, managed to keep their breach almost entirely out of the public eye by paying quickly and quietly. Neither approach sets a great precedent. Timely, transparent disclosure gives affected customers the chance to protect themselves and gives the broader industry the information it needs to defend against similar attacks.

As for prevention, the attack did not require sophisticated malware or a zero-day exploit. It required online research and a phone call. Okta’s August 31st warning included specific, actionable steps that, if correctly implemented, could have stopped or slowed the attack at multiple stages. Limiting help desk admin roles to exclude highly privileged accounts would have opened this request to more scrutiny. Perhaps a more senior staff member would have spotted a request to reset MFA for a Super Admin as a red flag, stopping the attack before it began. Enabling Protected Actions would have forced re-authentication before any administrative action was taken, adding another layer the attackers would have had to bypass.

Whether the aftermath caught up with anyone actually responsible remains unclear. ALPHV’s dark web infrastructure was taken down in late 2023, and several alleged Scattered Spider members were subsequently arrested in 2024. However, given how murky the attribution remains, it is difficult to say with confidence that those charged were the same individuals who carried out the MGM breach specifically.

Appendix A: Control Failures

The following table details control failures from the MGM breach. For reference in other parts of the appendices, each control failure is assigned an ID, abbreviated as “CF-[number].” Control failures go beyond technical failures to include process and skill (people) failures as well.

Cyber Defense Matrix Mapping

Using Sounil Yu’s Cyber Defense Matrix (CDM), which is based on NIST CSF functions and assets. Read more about the CDM here.

MITRE ATT&CK Mapping

This next table is for teams that depend on the MITRE ATT&CK matrix. These are the same control failures from the previous table, but reorganized from a MITRE ATT&CK techniques perspective.

ATT&CK Navigator Summary: Primary Tactics Leveraged

Reconnaissance ⇒ Initial Access ⇒ Credential Access ⇒ Privilege Escalation ⇒ Defense Evasion ⇒ Persistence ⇒ Lateral Movement ⇒ Impact

The attack is notable for its heavy use of Identity-based techniques (T1556, T1484, T1550) rather than common exploit-based initial access, highlighting the importance of identity infrastructure and its attack surface. Every tactic from Initial Access onward was enabled or amplified by the control failures in CF-1 through CF-4.

MITRE D3FEND Mapping

MITRE D3FEND mirrors the ATT&CK matrix. Where ATT&CK describes the techniques and tactics used by attackers, D3FEND describes the preventative and detective controls to ‘defend’ against them.

Appendix B: Attribution

Public reporting almost universally credited Scattered Spider with the MGM incident as well as incidents with other Okta customers. Scattered Spider is a loosely organized collective of young, English-speaking hackers known for aggressive social engineering. Security firms amplified the attribution, and the name stuck. But ALPHV/BlackCat, the ransomware group whose malware was ultimately deployed, published a lengthy statement on their dark web blog explicitly rejecting that framing. Scattered Spider, for their part, also claimed the operation as their own.

ALPHV’s statement is worth reading carefully. They not only claimed credit for the attack, they also pushed back against Scattered Spider being attributed. They called out VX Underground specifically for false reporting and challenged security firms to provide actual evidence of the perpetrators. In their own words, “these specialists find it difficult to delineate between the actions of various threat groupings, and so they simply grouped them together.” ALPHV also noted that tactics and indicators of compromise are publicly known and easy for anyone to imitate, meaning that pattern matching alone is not enough to pin an attack on a specific group.

ALPHV provides Ransomware as a Service (RaaS), which is why most sources attributed the attack to a collaborative effort between the two groups. Scattered Spider is known for gaining initial access via social engineering, then deploying ransomware purchased from other groups. In their statement, ALPHV seems to insult Scattered Spider, referring to them as “teenagers from the US and UK”, not something they would say if they were collaborating. With ALPHV’s data leak sites taken down in late 2023 and several alleged Scattered Spider members subsequently arrested, definitively attributing the MGM attack to either group remains impossible.

Appendix C: Timeline

August 31, 2023 - Okta sends out a warning to clients, stating that they have noticed attackers using social engineering to attain a privileged role in Okta, then laterally moving and escalating privileges. They specifically mentioned that these were “preventable and present several detection opportunities for defenders.”

September 8, 2023 - ALPHV Statement claims that they had access to MGM’s Okta this day. The attacker was able to socially engineer a help desk worker into resetting MFA for a Super Admin account, which was then accessed and used to add an additional IdP in Okta (a feature meant for companies undergoing a merger), this would allow them to sign in as MGM users using credentials from the IdP they added.

September 10, 2023 - First externally visible impacts from the attack - MGM started shutting down systems, impacting digital room keys, automated payouts for slot machines, website/reservation system outages, MGM app outages. Attackers claim that MGM began shutting systems down before any ransomware was used. According to the attackers, MGM began taking down all Okta Sync Servers, which allegedly locked MGM out of the Okta. Attackers were unaffected, and still had super admin privileges on Okta and access to MGM’s Azure.

September 11, 2023 - MGM announced on Twitter that it was dealing with a security incident. This is the day that over 100 ESXi hypervisors hosting many services MGM used were alleged to be encrypted with ransomware.

September 13, 2023 - The company projected it could lose up to $8.4M per day in revenue as issues continued. The company files a form 8-K (Legally required by the SEC to disclose any important information shareholders must know). Not much information is included

September 20 , 2023 - MGM confirms full restoration of services.

October 5, 2023 - MGM files another more in depth 8-K, in this form they disclosed that the “cyber incident” negatively impacted them by $100M, and “less than $10(M) in one-time expenses in the third quarter related to the cybersecurity issue, which consisted of technology consulting services, legal fees and expenses of other third party advisors. Although the Company currently believes that its cybersecurity insurance will be sufficient to cover the financial impact to its business as a result of the operational disruptions, the one-time expenses described above and future expenses, the full scope of the costs and related impacts of this issue has not been determined.” They believed the scope of the breach to be limited to “personal information (including name, contact information (such as phone number, email address and postal address), gender, date of birth and driver’s license numbers).” as well as SSN and passport numbers for some. They claimed that no payment information was leaked.

October 26, 2023 - MGM releases a statement that some Canadian customers were impacted, and sent an email to affected individuals with more information.

December 19, 2023 - ALPHV’s data leak sites are taken down by the FBI, and a decryption tool is sent to victims.

January 25, 2024 - FTC Staff issued a Civil Investigative Demand (“CID”) to MGM seeking large quantities of documents and information.

February 20, 2024 - MGM files a 71 page Petition, this document argues that the FTC is overreaching in their requests. Much of the correspondence between MGM and the FTC is redacted. The CID is later dropped.

March 22, 2024 - Class action lawsuits are consolidated into one lawsuit: Tanya Owens, et al. vs. MGM Resorts International, et al. They will then further combine with a class action group from the 2019 data breach on MGM in July of 2023.

October 31, 2024 - A settlement is reached with MGM, requiring them to pay $45M total, $75 to a victim who had their SSN or Military ID leaked, $50 to anyone who had a passport number or DL number leaked, and $25 to anyone who had their name, address, and DoB leaked. Additional money would be paid out to victims of identity theft on a case by case basis.

January 17, 2025 - After more negotiations, the parties enter a Settlement Agreement, which would then be put up to a vote for all affected members.

February - April 2025 - Notices are sent out to victims, who have until June 18, 2025 to claim their money

June 18, 2025 - Settlement period is complete and the website is taken down

Appendix D: References

Breach Info

• https://www.darkreading.com/application-security/okta-flaw-involved-mgm-resorts-breach-attackers-claim

• https://www.reddit.com/r/cybersecurity/comments/16k4u7g/dark_reading_mgm_caesars_hack_started_with_social/

• Tweet from Brett Callow

• https://www.wsj.com/business/hospitality/caesars-paid-ransom-after-suffering-cyberattack-7792c7f0

• https://www.bleepingcomputer.com/news/security/caesars-entertainment-confirms-ransom-payment-customer-data-theft/

• https://www.malwarebytes.com/blog/personal/2023/09/ransomware-group-steps-up-issues-statement-over-mgm-resorts-compromise

• https://blog.checkpoint.com/security/cyber-stakes-the-mgm-ransomware-roulette/

• https://techcrunch.com/2023/09/14/mgm-cyberattack-outage-scattered-spider/

• https://www.mgmresorts.com/en/notice-of-data-breach.html

• https://www.forbes.com/sites/steveweisman/2025/03/12/mgm-ransomware--attack-update/

• https://www.ft.com/content/a25d2897-b0ce-4ba7-92ed-ff5df09d1b47

Threat Intelligence/Writeups

• https://cybersecurity.fullcoll.edu/wp-content/uploads/sites/69/2025/05/MGM-Writeup.pdf

• https://www.cisa.gov/news-events/cybersecurity-advisories/aa23-320a

• https://wing.security/saas-security/a-saas-misconfiguration-case-study/

• Case Study - MGM Data Breach 2023

Govt Sources

• https://www.sec.gov/Archives/edgar/data/789570/000119312523251667/d461062d8k.htm

• https://d18rn0p25nwr6d.cloudfront.net/CIK-0000789570/a390c443-0c40-4025-aba2-74505ab3c9e3.pdf

• https://www.ftc.gov/system/files/ftc_gov/pdf/2423028mgmpetquashpublic.pdf

Settlement Info

• https://web.archive.org/web/20250506152518/https://mgmdatasettlement.com/

• Consolidated Complaint 2025

• https://www.classaction.org/media/in-re-mgm-international-resorts-data-breach-litigation-settlement-agreement.pdf

• https://cases.justia.com/federal/district-courts/nevada/nvdce/2:2023cv01480/164564/98/0.pdf

Okta Info

• https://sec.okta.com/articles/2023/08/cross-tenant-impersonation-prevention-and-detection/

• Okta Super Admin Compromise Attack Explained (YouTube)

• https://support.okta.com/help/s/question/0D54z00009dUW2uCAG/restrict-lesser-admins-from-resetting-passwordmfa-for-super-admins-and-hijacking-accounts?language=en_US

Appendix E: ALPHV Statement

Disclaimer: The statement reproduced below is attributed to ALPHV/BlackCat based on coverage from reputable cybersecurity outlets including BleepingComputer, Malwarebytes, and Check Point, all of whom reported on its publication to ALPHV's dark web leak site on September 14, 2023. We cannot independently verify that this represents the complete and unaltered text of the original statement, as the primary source was taken down by the FBI in December 2023 and is no longer accessible.

We have made multiple attempts to reach out to MGM Resorts International, “MGM”. As reported, MGM shutdown computers inside their network as a response to us. We intend to set the record straight.

No ransomware was deployed prior to the initial take down of their infrastructure by their internal teams.

MGM made the hasty decision to shut down each and every one of their Okta Sync servers after learning that we had been lurking on their Okta Agent servers sniffing passwords of people whose passwords couldn’t be cracked from their domain controller hash dumps. Resulting in their Okta being completely locked out. Meanwhile we continued having super administrator privileges to their Okta, along with Global Administrator privileges to their Azure tenant. They made an attempt to evict us after discovering that we had access to their Okta environment, but things did not go according to plan.

On Sunday night, MGM implemented conditional restrictions that barred all access to their Okta (MGMResorts.okta.com) environment due to inadequate administrative capabilities and weak incident response playbooks. Their network has been infiltrated since Friday. Due to their network engineers’ lack of understanding of how the network functions, network access was problematic on Saturday. They then made the decision to “take offline” seemingly important components of their infrastructure on Sunday.

After waiting a day, we successfully launched ransomware attacks against more than 100 ESXi hypervisors in their environment on September 11th after trying to get in touch but failing. This was after they brought in external firms for assistance in containing the incident.

In our MGM victim chat, a user suddenly surfaced a few hours after the ransomware was deployed. As they were not responding to our emails with the special link provided (In order to prevent other IT Personnel from reading the chats) we could not actively identify if the user in the victim chat was authorized by MGM Leadership to be present.

We posted a link to download any and all exfiltrated materials up until September 12th, on September 13th in the same discussion. Since the individual in the conversation did not originate from the email but rather from the hypervisor note, as was already indicated, we were unable to confirm whether they had permission to be there.

To guard against any unneeded data leaking, we added a password to the data link we provided them. Two passwords belonging to senior executives were combined to create the password. Which was clearly hinted to them with asterisks on the bulk of the password characters so that the authorized individuals would be able to view the files. The employee ids were also provided for the two users for identification purposes.

The user has consistently been coming into the chat room every several hours, remaining for a few hours, and then leaving. About seven hours ago, we informed the chat user that if they do not respond by 11:59 PM Eastern Standard Time, we will post a statement. Even after the deadline passed, they continued to visit without responding. We are unsure if this activity is automated but would likely assume it is a human checking it.

We are unable to reveal if PII information has been exfiltrated at this time. If we are unable to reach an agreement with MGM and we are able to establish that there is PII information contained in the exfiltrated data, we will take the first steps of notifying Troy Hunt from HaveIBeenPwned.com. He is free to disclose it in a responsible manner if he so chooses.

We believe MGM will not agree to a deal with us. Simply observe their insider trading behavior. You believe that this company is concerned for your privacy and well-being while visiting one of their resorts?

We are not sure about anyone else, but it is evident from this that no insiders have purchased any stock in the past 12 months, while 7 insiders have sold shares for a combined 33 MILLION dollars (https://www.marketbeat.com/stocks/NYSE/MGM/insider-trades/). This corporation is riddled with greed, incompetence, and corruption.

We recognize that MGM is mistreating the hotel’s customers and really regret that it has taken them five years to get their act together. Other lodging options, including casinos, are undoubtedly open and happy to assist you.

At this point, we have no choice but to criticize VX Underground for falsely reporting events that never happened. We typically consider their information to be highly reliable and timely, but we did not attempt to tamper with MGM’s slot machines to spit out money because doing so would not be to our benefit and would decrease the chances of any sort of deal.

The rumors about teenagers from the US and UK breaking into this organization are still just that—rumors. We are waiting for these ostensibly respected cybersecurity firms who continue to make this claim to start providing solid evidence to support it. Starting to the actors’ identities as they are so well-versed in them.

The truth is that these specialists find it difficult to delineate between the actions of various threat groupings, therefore they have grouped them together. Two wrongs do not make a right, thus they chose to make false attribution claims and then leak them to the press when they are still unable to confirm attribution with high degrees of certainty after doing this. The tactics, procedures, and indicators of compromise (TTPs) used by the people they blame for the attacks are known to the public and are relatively easy for anyone to imitate.

The ALPHV ransomware group has not before privately or publicly claimed responsibility for an attack before this point. Rumors were leaked from MGM Resorts International by unhappy employees or outside cybersecurity experts prior to this disclosure. Based on unverified disclosures, news outlets made the decision to falsely claim that we had claimed responsibility for the attack before we had.

We still continue to have access to some of MGM’s infrastructure. If a deal is not reached, we shall carry out additional attacks. We continue to wait for MGM to grow a pair and reach out as they have clearly demonstrated that they know where to contact us.

• you have at least 4 email accounts you check daily

• all of these inboxes are a hellscape, but you still have to use them

• some of it is self-inflicted - you keep signing up for newsletters and creating SaaS accounts

• some of it is just the effect of the never-ending sales grifts that represent the weeds of your Internet lawn

Every now and then, I take some time to analyze my inbox and clean it up. My goal is to ensure that human correspondence and other important emails don’t get buried or missed. This is typically the stuff that you actually want/need to see and respond to.

All right, how do we get all this cleaned up with minimal effort? The goal here is to prevent the correspondence and other important emails from getting buried, without also loosing other important emails you might need from time-to-time.

1. Disable email notifications from all the apps that are also sending you notifications on your phone and show notifications/catch you up when you’re in the app itself. You don’t need 3-4 layers of notifications. Go into LinkedIn, click notifications, done. You don’t also need an email for all of it.

2. Create a message rule in your inbox that moves all email with an unsubscribe link in the body into a folder named “Automated Emails”. Sometimes you’ll need emails in this folder, but they almost never require your immediate attention. If you reset a password, you know you’re getting an email. You can search for it. It doesn’t need to be at the top of your main inbox folder for you to be able to find it quickly.

3. Look for common phrases from sales tactics and also move these emails into the “Automated Emails” folder. A common one I’ve been noticing lately is “not sure if you’re the right person”.

4. If you find that there are daily emails cluttering your inbox and you archive or delete them without reading them, 100% of the time, just take a few extra seconds and unsubscribe, or create a message rule to move them to another folder when they come in. Taking 5 minutes to do this can go SO FAR to clean up your inbox.

5. Send newsletters to a newsletter folder. A basic message rule to send anything from Substack and Medium to the newsletter folder can really clean things up. I use Hey Mail, and it has a default folder for newsletters and receipts. When someone sends you an email for the first time, you decide where it goes. This “routing when first received” approach works okay, though I often still skip it and have to go back and clean up stuff later.

6. Most inboxes have an icon that separates meeting invites from normal emails. When I check my email, I process these first, before doing anything else, because I need to make sure I don’t have any conflicts, and a lot of my income is attached to activities that come with meeting invites (podcast recordings, webcast recordings, etc). Your meeting invites might not be as valuable to you, so perhaps you should treat yours differently.

That’s really it - these 6 steps should take you just 10-15 minutes to implement and should go a long way to bring sanity back to your inbox. I could have written 10 steps, but I think this is enough to get you started and give you ideas on how you could take things further.

Or don’t take things further - the goal isn’t perfect inbox management, it’s just to make sure you don’t miss those super important emails that are timely and require your attention. I hope this helped you.

Subscribe now

With the exception of breach analysis and podcasting, I probably spend more time focused on vulnerability management than anything else I do. Let’s start with a bit of context before diving in.

This article concerns managing patches and vulnerabilities for commercially bought hardware and software and open source. Finding and fixing vulnerabilities in your own organization’s code is a very different post for another time. I won’t touch on AppSec at all in this post.

My perspective here comes stems from a few places.

1. My research into vulnerabilities. This focuses on documenting the vulnerabilities that cause damages/loss for organizations. I’ve also been spending a lot of time looking at time-to-exploit statistics. My hope is that, by looking at patterns in hindsight, I get useful insights that I can pass on to… 👇🏽

2. IANS clients that I do advisory work for. Of all the advisory work I do, vulnerability management is the most frequent (sometimes 3+ per week), with AI a close second.

3. I spent part of my career in offensive security, so I tend to look at vulnerabilities through a “how can I turn this into a compromise” lens.

Complications

Let’s start with few key complications that will help you understand why I’m concerned about vulnerability management.

Complication #1: Only a small percentage of vulnerabilities are a threat to organizations. This fact has been well known, studied, and documented. This leaves unanswered questions, like:

1. what do these vulnerabilities have in common?

2. when are they getting exploited?

3. why isn’t complication #1 making the work of vulnerability management easier?

Complication #2: Figuring out which vulnerabilities are a threat, when they’re the greatest threat is an unsolved challenge. I think this is a solvable problem and it’s something I’m working on, but that’s a post for another day. This complication generated an entire separate market segment: Risk-Based Vulnerability Management (RBVM). Awkwardly and expensively, this market emerged separately from the vendors that build the vulnerability scanners.

Complication #3: Remember the time-to-exploit research I mentioned? The news isn’t good, y’all. The short version is that the majority of exploited vulnerabilities get exploited before disclosure. Meaning, they’re zero day vulns. This means:

1. There’s no CVE yet. That means no CVE enrichment, no way to calculate EPSS.

2. There’s no patch yet. Nothing to fix or remediate.

I’m going to repeat this again, because it’s the primary wrench that has been thrown into the vulnerability management works.

I see you turning purple and I promise, I’m right there with you. Maybe you have doubts. Maybe you think I must be mistaken. I’d LOVE to be mistaken - my advisory calls would be a lot simpler. Let’s take a look at the data.

Time-to-Exploit Trends

One of the primary goals of vulnerability and patch management is to outrun exploitation. The primary question here is always, “how fast do we have to be to outrun the attack?” The answer to this question was once an achievable goal. A few years ago, the ground shifted under our feet.

Research teams tracking the average time-to-exploit for vulnerabilities noticed that it was dropping. Traditional 30/45/60 or 30/60/90 day patching SLAs don’t make much sense when the average time-to-exploit is a moving target! Mandiant has been tracking this trend for a while now and the numbers aren’t encouraging:

• In 2019, the average time-to-exploit was 63 days

• By 2023, that number dropped to 5 days

• In 2024, it dropped to -1 days

How can the average time-to-exploit be less than 0 days? The clock starts counting when the general public becomes aware that a vulnerability exists - when the vulnerability is disclosed. In 2023, Mandiant found that 70% of exploited vulnerabilities were zero-days when first exploited. If a vulnerability is exploited 40 days before it is discovered and disclosed to the public, we could think of it as a -40 day vulnerability. This is what moved the average to the negative side of the number line.

So, 70% of the time, how fast you patch doesn’t matter? Ouch.

I’m regularly taking calls from enterprises complaining that their 5 day or 7 day SLA for criticals is nearly impossible to meet, asking “are we the only ones? Are our peers managing this? If so, how?” They’re not the only ones. Most organizations I advise are weary of fully automating patching, for fear of breaking things. Even those that are allowed to move quickly hit a long tail: 70-80% in 24 hours and then maybe a month to remediate the last 20-30%.

What about the 30% where patching speed does matter? More than half of these vulnerabilities were exploited within a month. 29% within 7 days. 12% within 24 hours.

Adding to the challenge of prioritization, Mandiant found that:

• 58% of vulnerabilities that received media coverage were not exploited in the wild

• 72% of vulnerabilities with available exploits or PoCs were not exploited in the wild

• And VulnCheck found that 29% of vulns on CISA’s KEV list were exploited on or before CVE publication, neutering any process dependent on CVE details or enrichment (taking CVSS/EPSS-dependent processes out of the picture)

• 25% of N-day vulnerabilities weren’t exploited until after the 6 month mark

Right off the presses is another great time-to-exploit resource, Serjej Epp’s Zero Day Clock.

This is a LOT to take in.

1. Most of the vuln mgmt problem is now a 0day problem

2. We have to patch much faster than most of the books, standards, regulations, and best practices would have us believe

3. Our prioritization processes and models are almost certainly built on some bad assumptions

There’s still another problem to consider, though.

Asset management is still broken

The first time you see the output of a vulnerability scanner, you’re not thinking, “I need more data”. You’re missing data though.

Vulnerability scans miss a lot of critical information. This is because they fail to identify some types of assets - IoT in particular. Once an asset is misidentified, the scanner can’t tell you much that’s useful about it and it tends to get dismissed by analysts. It makes sense in context, when you’re looking at results that feel like (emphasis mine):

• WINDOWS 2008 Server OMG WHY IS THIS STILL RUNNING IT HAS BEEN EOL FOR A COON’S AGE CRITICAL CRITICAL ALL IS LOST

• WINDOWS 2012 Server OMG WHY IS THIS STILL RUNNING SLIGHTLY LESS CRITICAL, LIKE 98.7% AS CRITICAL, YOU SHOULD STILL BE FREAKING OUT

• Something running Linux?

• ADOBE FLASH STILL EXISTS ON YOUR SYSTEMS? IS THIS A MUSEUM? WHY IS THIS HERE

• Something else maybe running Linux? Port 80 is open. Informational. Don’t bother with this.

• ANOTHER WINDOWS 2008 Server I CANT BELIEVE THIS IS REAL LIFE OMG CRITICAL CRITICAL ALL IS LOST

Hmmm, Linux you say? That’s helpful. It could be a mainframe, a toaster, a lightbulb, a web server, a wireless access point, a network firewall with its management console exposed to public Internet, OpenClaw, or a satellite in low earth orbit. Yeah, that really narrows it down.

This is bad, because the vulnerability scanner is trying to prioritize vulnerability remediation workloads with incomplete data. Worse, the data they collect on misidentified or unidentified assets actively deprioritize them. This is a system that makes unknown or unidentified assets look safe by default. Analysts will gladly treat them as safe, since they have 1.2 million critical vulnerabilities to chase down.

The kicker here is that some of these misidentified assets are representing the tiny fraction of vulnerabilities that can cause damage. What are the chances that these unknown, possibly unmanaged assets are hardened? That they’re getting patched? That they don’t have default credentials? We know that a large number of exploited vulnerabilities in recent years are Linux-based edge devices. These are network devices, file transfer appliances - exactly the types of devices that vulnerability scanners fail to recognize.

Surveys show that security leaders are well aware that critical assets are camouflaged by a lack of data and a lack of certainty. Asset management and/or vulnerability management processes have a gap to fill here.

Yes, some orgs still need traditional vuln mgmt

There are still plenty of ‘N-day’ vulnerabilities, where we don’t see active exploitation until days, weeks, or even months after they are disclosed. Most of the vulnerability and exploit intelligence we’ve been discussing focuses on when exploitation was first seen, but what are we seeing in actual breaches?

When studying breach details, I’ve found it very common to see attackers successfully use exploits months or even years after patches have been available. Vulnerability remediation isn’t always a bell curve with a long tail. It’s quite possible to remediate 100% of vulnerabilities and see a resurgence. So, sometimes it’s a bell curve with a stegosaurus tail?

Perhaps someone clones an old VM and brings it online without patching it. The same can happen with gold images for workstations. People occasionally need old versions of software or old operating systems for various reasons.

Compliance is still very dependent on traditional vulnerability scanning. PCI DSS, SOC 2, ISO27k, and many other standards and regulations have auditors expecting to review traditional scan results.

Sometimes, patching a critical vulnerability requires patching non-critical items, because some systems have linear software updates - you can’t apply update 13 unless you’ve already applied 12.

Vulnerability scanning tools are also commonly used for configuration management - identifying when hardened configurations have drifted, or haven’t been applied.

There are still a lot of reasons to keep old school scanners around, but maybe not for all the same reasons you bought them.

Prioritization is also an ongoing challenge. It made logical sense to prioritize patching vulnerabilities that are exploitable, where exploits are available, and when we see active exploitation. We now have data telling us that only 28% of vulnerabilities with available exploit code were exploited in the wild. Even what is lauded as the best evidence, “active exploitation in the wild” can be unreliable.

Consider a common example: what if the vulnerability is information disclosure, and using the exploit simply returns the internal IP address of a server? Our tools would report “exploit available” and “exploitation seen in the wild”, even though it’s totally inconsequential vulnerability in most scenarios. At best, it could possibly be chained with several other vulnerabilities.

Building new strategies

I now strongly believe that vulnerability management must be divided into two use cases, each with their own set of processes and tools.

1. Exploitation prevention

2. Compliance and system/asset management

It should already be clear that even the UK NCSC’s more aggressive 5/7/14 day SLA recommendations aren’t enough to address exploitation that happens prior to disclosure. The only way to address exploits we don’t know about is with preventative, proactive approaches.

Exploitation prevention: 0days

I’ve got a few ideas that I’ve been workshopping. Would love to hear if others have anything to share.

• Reduce attack surface: remove/disable unnecessary stuff. Getting hacked is bad enough - getting hacked because you had CUPS installed and running on a web server for no good reason? Ouch.

  • Regularly scan external infrastructure for insecure, abandoned, and unidentified assets. If you see “Copyright 2011” at the bottom of a webpage, that web server deserves a closer look.

• Hardening and passive exploit mitigation

  • endpoint exploit mitigation

  • immutable infrastructure

  • old-school chroot jails, or the same principal applied with newer tech

  • application control

• Detection: If you fail to prevent the exploit, all you’ve got left is to quickly detect and respond to the attack. Since you don’t know what the attack looks like, the best bet is to target behavior. Attackers have to do attacker things and we know what most of those are: gather information, find and abuse credentials, authenticate to other systems, establish persistence, exfiltrate tons of data, etc.

  • Behavior-based EDR rules

  • Deception (no guessing required, puts detection on easy mode!)

  • Large data transfer detection

  • Anomalous system behavior (in databases, IAM, anywhere the attacker wants or needs to be)

  • oh, and don’t forget to test your detections to make sure they work!

• Last, but not least get rid of notoriously vulnerable products and protocols

  • ditch vendors that repeatedly show up on CISA KEV, year after year

  • get rid of the asbestos of IT - products that have safer alternatives

This list isn’t meant to be exhaustive, but to get other folks thinking and potentially contributing.

Exploitation prevention: N-days

For the N-Day vulns that are exploited quickly, but after disclosure, it’s clear that a scan-driven approach can’t be effective. We’re not going to wait for a vulnerability check to get created, QA’ed, pushed to production, downloaded by our scanner, wait for the next scheduled scan, and then wait for a human to see it. This could take days or weeks.

An intel-driven approach makes much more sense, though it requires reliable hardware and software asset inventories. The moment a vulnerability is disclosed, an analyst queries asset inventories, analyzes the impact, and sets remediation into motion, based on the severity they’ve determined. This can be completed in minutes after disclosure - no waiting for scans necessary.

Compliance

Organizations in regulated industries may find it difficult to get away from traditional vulnerability management tools. These processes are well established and expected by both auditors and standards. While some standards (like PCI DSS) allow for custom scoring to deprioritize non-critical vulnerabilities, others force remediation regardless of prioritization’s impact on scoring. These tools and processes aren’t going away any time soon.

Conclusion

It has always been true that vulnerability management was tightly linked to other processes and teams, but I often find it more isolated than it should be. When Linux admins roll with default RHEL installs, they’re making vulnerability management work more difficult. When SecOps builds detections without consulting with vulnerability analysts, they’re missing opportunities. When the security program assumes the only mitigation is applying a patch, vulnerability management can’t achieve its goals.

We now have the challenge of more tightly linking vulnerability management to SecOps, asset owners, and other groups. On top of this, most organizations still have to run a traditional vulnerability management program. PCI needs quarterly clean scans. SOC 2/ISO27k expect traditional scans to be available for review. Systems still need to be kept up-to-date. That means the clients I’m advising are still considering purchasing RBVM solutions and other prioritization methods. They’re still adding vulnerability intelligence tools and processes on top of their scan-driven processes.

The most common setup I see today is an old school network scanner, running on a schedule, performing a mix of authenticated and unauthenticated scans, perhaps with some agents installed on remote systems. To summarize:

• If the data I’ve presented here is correct, the best this setup can do is to address 30% of that exploit prevention goal.

• If we assume 40% of the assets being scanned are not correctly identified, this number drops to 18%.

• And we can only claim that 18% if we’re doing a perfect job of prioritizing all the right vulnerabilities and getting them remediated within 24 hours.

• If we can’t patch this 18% within 7 days (most of the orgs I’m working with cannot), we lose another 29%. That brings us below 13%.

Is the best case scenario that the majority of organizations are struggling to address 13% of the exploit prevention problem? I hope not - please tell me my math is bad.

I don’t have any great answers on simplifying it either. It looks to me like vuln management gets more complex than ever. I’m hoping others have some helpful thoughts and suggestions on this.

Subscribe now

Note: Most of this was written in July 2025 and I thought it was too late to put it out, but Citrini Research’s fantasy fiction piece, The 2028 Global Intelligence Crisis convinced me it could still be useful. There seems to be a fundamental misunderstanding here around how work works. Jobs, tasks, and work are all very different things - related, but different.

Simple Jobs

Simple jobs have always been around and they’re always getting replaced. Human beings were once employed to walk around and light streetlamps when it got dark. Electricity and light sensors replaced them. At one point, every manager and executive had secretaries to type for them. Put a PC on every desk and now only execs at the highest level can still justify an executive assistant or chief of staff.

When I started out in IT I worked at one of the world’s largest payment processors. I had many roles before I got into cybersecurity proper, but automation was one of my favorite.

Automated jobs were scattered throughout the organization. They were written in Perl, Visual Basic, Bash, C++, Crystal Reports. The individual that originally automated the task chose the language they knew best. Some of the jobs ran on servers, but most existed on someone’s personal computer, or a secondary computer under their desk, in their cubicle.

A few employees convoluted these tasks so much, that they became full-time jobs. Many weren’t professional developers, so the concept of monitoring, alerting, logging, and error handling didn’t occur to them. Their code was just bad enough that they needed to babysit it every single day and step in when things broke.

We bought a commercial automation platform called, AppWorx, and it was my job to centralize and normalize all these tasks in one place. It was hugely fulfilling work - I’m the flavor of neurospicy that loves to dive into a mess and organize it. The part that sucked was putting several of my colleagues out of work.

Bullshit Jobs

I read the book Bullshit Jobs by David Graeber last year and it was a timely revelation. I found it interesting that the definition Graeber goes by is self-defined. When workers believe that their own jobs shouldn’t exist, it is a bullshit job.

Graeber generally states that he found about half of societal work to be pointless - sometimes this is part of a single job (e.g. someone might feel that 70% of their job is pointless, but the other 30% worthwhile), or that the role entirely shouldn’t exist. He came up with five categories for bullshit jobs (copied straight from the Wikipedia page linked above).

1. Flunkies, who serve to make their superiors feel important, e.g., receptionists, administrative assistants, door attendants, store greeters;

2. Goons, who act to harm or deceive others on behalf of their employer, or to prevent other goons from doing so, e.g., lobbyists, corporate lawyers, telemarketers, public relations specialists;

3. Duct tapers, who temporarily fix problems that could be fixed permanently, e.g., programmers repairing shoddy code, airline desk staff who calm passengers with lost luggage;

4. Box tickers, who create the appearance that something useful is being done when it is not, e.g., survey administrators, in-house magazine journalists, corporate compliance officers, academic administration;

5. Taskmasters, who create extra work for those who do not need it, e.g., middle management, leadership professionals.

The folks I automated out of a job fell firmly into the Duct Tapers category. The tasks they partially operated worked, but not well enough for them to move on to another task.

I clearly recall coming to this job one day, walking through the doors, gazing across the cubicles, and having a revelation: two-thirds of the people that work here could never come to work again and there would be zero impact. In each department within the company, I had observed that there were one or two ‘heroes’ that seemed to keep everything working smoothly. The remaining members of the team either managed something small and basic, or managed something that didn’t really need to be managed at all. They engaged in a sort of work theatre, as if Fisher Price made enterprise-grade playsets for storage administrators and backup management.

The other big revelation from this book was that not all jobs exist because work needs to get done. There are vanity hires - the flunkies mentioned in the first category above.

Companies are naturally inefficient

New or smaller bootstrapped companies are loathe to spend or hire too much, unless it’s really necessary. As companies get larger, it becomes more and more difficult to understand what is necessary or not.

Managers say they need more people, so they get them. It’s literally part of their job - to manage people. Asking a manager to determine if they need more people is almost a conflict of interest - of course most will say yes. Even if they believe that they legitimately need more staff, can the productivity improvements be measured? Can the hires be justified? They often can’t, which is why large rounds of layoffs every few years is necessary to compensate for this inability to measure productivity.

If you handed your average information worker pen and paper and asked them to categorize how they spend a 40 hour work week, they might be challenged to do so. What category is Slack and Email? Is it wasted time, or is it productive? Sixteen hours in meetings - were they all necessary? Did they all need to be an hour long? Could you have skipped half of them and done something more productive instead?

If the individual struggles to answer this question, you can bet the company doesn’t know any better. That’s why sometimes layoffs have zero impact on the company and others result in a clear drop in performance or quality that customers notice - it’s often guesswork.

Is AI replacing jobs?

It’s easy for a giant tech company to lay off 10% of its workforce and attribute it to AI. AI is just the latest excuse in a long history of excuses used to give a positive spin on mass layoffs. Large companies have always done big layoffs, in part, to counter for their inability to measure employee productivity.

Management values growth and self-importance over efficiency and productivity. They’re constantly pushing for larger budgets and teams, regardless of whether it’s justified. Eventually, the company brings in a consulting firm that tells them the ugly truth: two-thirds of the company do little to nothing of value.

No one is being replaced with AI. In many of the cases where they’re very explicitly trying to replace humans with AI, things haven’t gone as well as hoped (e.g. Klarna, Salesforce).

Fallacy #1: because AI can do a task, it can replace a worker

AI can write code, therefore it can replace developers. Is writing code everything a developer does? Absolutely not. In fact, if you want AI to write code, you need more software engineers to help manage the output.

AI can create Gantt charts, so can it replace project managers?

What about cases where AI succeeds in completing a task only some of the time? There are many stories of AI inconsistency, failing 10-40% of the time - even doing the same exact task the model completed successfully in the past. That’s hardly a case where you can replace a human with AI.

Fallacy #2: AI work replaces human work

Actually, we’re finding that the reverse is often true. AI will do work that no worker was ever going to get paid to do, perhaps because it was too boring or the economics didn’t make sense. For example, I was never planning to hire human artists to create custom images for the slides I use for my talks, but now that I can have AI do it for a $20/mo subscription, it makes sense to do it.

There’s a ton of new AI—generated slop on YouTube targeting kids that certainly wouldn’t exist without AI and didn’t steal jobs from existing artists.

Again, in the case of software engineers, we need more than ever, because AI needs a human that understands the bigger picture and what the intended output should be. A human breaks down the project into tasks, writes the prompts, adjusts the prompts, reprompts when AI gets it wrong, etc.

My friend Ayman Elsawah has some great examples around the security shortcomings in AI-generated code.

The Security Cafe

The AI + Security Issue

There has been a lot of signal lately around the intersection of AI + Security. Maybe because I’m in the thick of it pushing AI vendors to help with centralizing their security, or maybe because a new and big AI+Security conference is happening this week. Some super exciting talks I’m looking forward to catching…

Read more

4 months ago · 2 likes · 1 comment · Ayman Elsawah

Fallacy #3: AI hasn’t already replaced jobs

GenAI has certainly already replaced jobs. Again, these were largely tasks-as-jobs

• Contractors creating okay-ish content for marketing teams

• Contractors creating okay-ish graphic design for marketing teams

• Voice actors doing work on projects with tight budgets/margins

Conclusion

AI is certainly helping a lot of folks be more productive and more efficient, particularly in the area of software development. Will they make the workforce more efficient overall? Of that, I’m not so sure, as AI emboldens people to step way outside their wheelhouses, and it’s easy for AI to make you feel like you’re doing amazing things, while the subject matter experts looking over your shoulder are suffering retinal detachments from rolling their eyes so hard.

Take software engineering, for example. Thanks to AI, anyone can generate code. With no background in software, what will the average middle manager create? Are people with bullshit jobs now going to create bullshit software?

These folks aren’t software engineers - they’ll make every imaginable mistake when building software with AI, burning GPU cycles all the way. AI isn’t trained to intercede during a vibe coding session and say, “you’re kinda reinventing the wheel here”, it will happily burn those tokens, creating cartloads of software no one needs or asked for.

So we’ll likely end up with more jobs, more software, more tech debt, more vulnerabilities, more attack surface, more of everything, now that it can all be generated at a whim on a prompt (or launched into full auto thanks to OpenClaw and other agents).

Anyone interested in a Vibe Code Cleanup Engineer? Companies will be hiring soon.

Subscribe now

Not too surprising, right?

The LinkedIn post was focused on a new community created around SOC 2 in an attempt to improve the quality of SOC 2 reports. The comments on this post, however, were flooded with HITRUST stans, revolving around a key statistic: that less than 1% of HITRUST-certified organizations reported having a breach.

There’s a lot to dissect here with just this one small claim. Before I get to that, however, let me comment on some positives I’m hearing from the HITRUST crowd.

In said LinkedIn comments, a HITRUST employee mentions that control selection is based on threat data. “We’re analyzing threat data monthly and adjusting controls as necessary based [on] what is being exploited today”, they say. I think this is an excellent idea, and it’s a key point I’ll be arguing for alongside Adam Shostack when we speak at RSA in a few months.

While I get excited about HITRUST’s certification methodology, I find the discussion around this less than 1% breached metric troubling. One thread in the comments is started with the argument that HITRUST is better than SOC 2, because it has “a published breach rate of less than 1%.”

How am I expected to use this metric with no basis for comparison though? I immediately have some questions:

1. What’s the breach rate for SOC 2?

2. What’s the breach rate for non-HITRUST certified organizations?

3. Where did this breach rate come from?

Understanding more about HITRUST

So, it looks like part of the HITRUST certification is a contractual obligation to report breaches. I like this as well! With breaches reported, HITRUST has an opportunity to learn from the breach and update their required controls to ensure others can benefit from breach lessons. Again, this is something I also argue heavily for, though in a more public sense, not within a private certification framework. We now understand how they’re collecting the data for their metric though.

Note - for simplicity’s sake, I’m going to assume 100% of organizations are 100% honest when reporting breaches to HITRUST. I believe that, whenever questioning someone else’s stats or reporting, it’s a good practice to be overly conservative and fair when challenging them.

With that said, are there incentives not to report a breach? Absolutely, if you think you can get away with it. From an attacker’s perspective, this is an extortion opportunity. How much is it worth to you to not lose your HITRUST certification? How much is it worth to HITRUST to have a low breach rate to report??

According to their 2025 Trust Report, HITRUST reports that in 2024, 0.59% of HITRUST-certified organizations reported a breach in their HITRUST-certified environment. This seems very impressive, as any bad thing less than 1% seems like a win when expressed as a percentage.

There’s a reason that “five nines” is a thing when calculating systems availability, however - 0.59% of downtime is nearly 52 hours, or 2 days offline. That’s an eternity if it happened to a major hyperscaler like AWS. If we said that less than 1% of schoolchildren were poisoned by their school’s drinking fountains, this would also come across as unacceptable - that’s over 330,000 sick kids.

Some Internet-sleuthing suggests that there are “over 1000” HITRUST-certified organizations globally. So we’re talking about at least 6 reported breaches within HITRUST’s dataset. Some important questions remain.

How do we know that this makes HITRUST superior to SOC 2? We don’t know what the breach rate is for organizations with SOC 2 type 2 reports.

Looking for quantitative answers

How do we know that this breach rate makes having HITRUST certification superior to not having it? We don’t know what the breach rate is for businesses as a whole. So I asked that same HITRUST employee for some clarification.

He replied that 40-60% of businesses have been breached in the past 12 months, citing a 55% number from a TechRadar/GigaOm survey on hybrid cloud. This is either troubling or comforting, depending on how you look at it. If more than half of all companies are getting breached every year, the cybersecurity industry isn’t doing too hot. On the other hand, breaches aren’t killing companies or the economy, so I guess this suggests that most breaches aren’t all that bad?

The 2025 Verizon DBIR reports 2,867 data breaches for organizations in North America. Excluding sole proprietorship, this gives us a breach rate of 0.038%, or 1 breach per 2,650 businesses. This is hardly a fair comparison though, as my dataset likely includes every small family-owned restaurant in North America, none of which are likely to ever pursue a SOC 2 or HITRUST-certification (though they can and have had breaches).

Refining the number of businesses further, to only those likely to pursue a SOC 2 or HITRUST certification, we come up with a conservative estimate of 0.97%. Still less than one percent, but again problematic, as we don’t know if Verizon’s dataset includes breaches at businesses we just excluded.

Looking at another interesting dataset, 26 companies have reported material cybersecurity incidents since the SEC breach disclosure rule went into effect on December 18th, 2023. A total of 55 cybersecurity incidents have been reported via Form 8-K in this same period. Again, we’re looking at a conservative estimate that still hovers around 1% (1.3%) of public companies reporting a breach, and only 0.65% reporting material breaches in the 12 months following this new disclosure rule.

Is HITRUST’s approach reducing the likelihood of breaches for its customers? It’s hard to say. I’m inclined to believe that HITRUST’s methodology will have a positive effect on the security programs of organizations that get certified, but without baseline data and comparisons to other compliance regimes, it is impossible to compare their numbers. Similarly, it is difficult to find support for reports that over half of companies are getting breached every year, outside some survey data.

Conclusion

I think any time spent focusing on controls that matter and align with how breaches are actually occurring is a good thing and is more likely to yield positive outcomes than simply following an industry standard that doesn’t take breach lessons into account.

Obtaining evidence that a particular approach works is very difficult however, as I hope my sad attempts at statistical analysis above demonstrate. I wish the best of luck to the folks at Verizon, Mandiant, and other organizations that produce annual reports on statistics and trends they’re seeing worldwide.

Finally, I hope this post has helped folks approach any statistical claims with a little more perspective and caution. There’s nothing wrong with asking questions and challenging stats. We can all stand to be challenged to improve our assumptions and data from time to time.

• It’s the reason why cats knock stuff off shelves.

• It’s the reason why, when you come across a button, you’re tempted to press it

• It’s the reason why, when someone builds a bonfire, we’re tempted to throw in random things. How will they burn? What color will the flames be? Will it pop or crackle?

That’s OpenClaw1 - it’s a tech bonfire made of AI. Since agents CAN be autonomous, we can’t help but wonder what would happen if we give them keys and credentials and currency and legs and tokens and hair and claws and a soul - and then just set them loose.

What happens if it bets on the stock market?

What happens if you give it $5000 and tell it to start a company?

What if you give it access to your GMail, your calendar, your business, and feed it your hopes and dreams as guidance?

Why is OpenClaw happening?

With every big new technological breakthrough, there will be an experimentation phase. Startups are expected to build fast, burn cash, and try risky things. This often leads to recklessness. With vibe coding, startups and funding are no longer required for experimentation. The more accessible the technology, the more experimentation we see. This explains why there is an OpenClaw and not a Quantum computing-equivalent to OpenClaw.

People are going nuts with OpenClaw. Dissatisfied with only a super powerful and extra risky personal assistant, folks have made a social network for AI agents. And a dating website. And a website where AI agents can hire humans to do meatspace stuff they can’t do themselves. AI agents are doing everything from pondering philosophical questions to building their own apps, policies, and resources for other agents.

Simply put, OpenClaw is happening because it can happen. The longer explanation is that, since one person can quickly code an all-powerful AI bot all by themselves without having to think about the consequences for too long and without having to get approval from a board or co-founders, it has happened. Also, the temptation to connect an AI agent to a ton of resources and set it loose is too strong for some to resist. This is no risk, no reward to the extreme.

Don’t waste the mistakes, learn from them

Do all the folks scrambling to get OpenClaw fully understand the risks involved? Probably not. Things like this seem to be inevitable in tech (despite decades of Sci-Fi warnings). Even with less accessible innovations like CRISPR, there were folks that experimented with editing their own genes at home.

Exposing an AI agent to a nearly limitless attack surface (websites, emails, messages) for prompt injection is risky, but could help us speedrun AI security challenges. I’m a strong advocate that some percentage of cybersecurity experts should be what I call Cyber Scouts. People that buy, test, and experiment with new technology early on, so that the cybersecurity industry can advise early adopters on using the new technology safely.

As OpenClaw users scramble to experiment and some fail fast, we already have some useful hardening guides and tooling from the security community and from OpenClaw’s founder.

1. https://1password.com/blog/from-magic-to-malware-how-openclaws-agent-skills-become-an-attack-surface

2. https://docs.openclaw.ai/gateway/security

3. OpenClaw Detector

4. OpenClaw Hunter

5. OpenClaw Telemetry

6. OpenClaw Audit

Why am I cheering this madness on?

My hopes are that, if AI enthusiasts speedrun all possible AI use cases, we can more quickly spot the use cases that don’t work, and the ones that do. The sooner this happens, I believe the sooner we can get back to the core work that needs to be done in cybersecurity2.

All signs suggest AI will make both of these things and many more worse before they get better. I sincerely hope 2026 is the year our focus shifts back to addressing fundamentals, which aren’t getting any easier or more solved.

Subscribe now

Subscribe now

I was really worried in the beginning, but things have gone better than I could have hoped. I’ve been regularly podcasting for nearly half a decade now, which has really helped me hone my organization and speaking skills. Particularly because I’m interviewing hundreds of people every year, often live - not just reading a script into a camera (that comes with its own, different challenges).

I doubled my income in 2025, and I have a theory on how this was possible.

Despite hundreds of hours in front of a camera, I still have a lot of room for improvement. I still struggle with ‘filler words’ (mine are “um” and “you know”). I sometimes lose track of what the guest is saying, because I’m thinking about where to take the conversation next and checking my notes. I’m clearly biased and too close to be objective, but this is what I think my formula boils down to:

1. I’m good enough on camera and as a host.

2. I carry around 25 years of domain experience, which allows me to relate to the guest, the audience, and ask intelligent follow-up questions

3. I’m organized and prepared for every event, podcast, and webcast

4. I show up

The combination of these things has resulted in getting offered more and more work, without me having to go out and solicit for it. It’s not that I’m clever and discovered the right recipe after trying different combinations. I’m organized and prepared because I have to be - my ADHD would make it impossible to stay focused and organized during a recording without and intro/outro script and a list of topics to discuss. I got good enough on camera because I forced myself to watch my own recordings and improve the lighting, the camera, what my face is doing while I’m listening, making sure I’m letting the guest do most of the talking, etc.

Number 4 is an interesting one though. I’ve often had opportunities because I was available and willing. Someone else didn’t show up and they need an alternate. A substitute. I did a good enough job as a substitute that I started landing regular gigs. I sought out criticism and advice. I started getting good feedback from clients, guests, and the audience. I made wise choices and got really lucky when seeking out co-hosts.

I think I keep getting work primarily because I make myself available. I almost always say yes and I get the job done. I’m guessing most organizations would rather work with one reliable consultant than have to bounce around between 5 or 6 that say no 50% of the time.

Do I like what I do?

So, yay! I’m making a living as a creator. I’m my own boss. I set my own hours. I can go on vacation and travel when I want. Do I like the work though? Is it sustainable?

Heck yes. I love it and couldn’t imagine going back to a corporate job.

I was watching TV the other day and all the commercials were trying to be relatable with messaging like, “haha, bosses suck, yay weekends” and “this meeting should have been an e-mail”! It occurred to me that I couldn’t remember the last time I was in a meeting that was wasting my time. I’m 100% engaged in every meeting I’m in. When I’m done, the meeting is done. This doesn’t suck.

I also travel extensively with my partner. When she travels for work, I go with her. When I travel for work, she often comes along. With the exception of in-person event work, 100% of my work can be done from anywhere I can find an Internet connection. This also doesn’t suck.

Almost all of my job requires me, an awkward introvert, to be in front of a camera, talking to people. Maybe it’s the repetition, but I’ve become comfortable with it. The days where I have to churn out two podcasts and a webinar in the same day are very, very draining. Thankfully those aren’t too common.

I’m the product now, but in a way, so are the other folks I work with. We have to get along. We have to be engaging and entertaining on camera. I’m learning some interesting skills here. Some folks can’t answer a question with less than 10 minutes of words. That’s unfortunate, as it limits the amount of content we can cover. Thanks to the prep calls we do, however, I have an opportunity to sus out that trait and plan for it when we’re live.

Some folks (particularly the Nordic variety, I’ve noticed) are very concise and efficient with words. If I don’t plan for this, the webinar will be done in 20 minutes, or the podcast interview done in 10. Managing time, questions, the flow of conversation, and keeping an eye out for audience questions is challenging, but rewarding.

So yes, I like what I do, but I’m probably overdoing it. I should probably say ‘no’ more often, but saying no makes me nervous. What if saying no makes the opportunities start drying up? What if saying no makes someone else “the guy that always shows up”?

What did I do in 2025?

Some highlights included doing live interviews at Zero Trust World, RSAC Conference, Identiverse, and Oktane. These short, 15 minute interviews are a lot of fun. After years of working with some startup folks in Armenia, I finally went there for a visit and spoke at the BSides Yerevan and CyberGEN conferences.

I was excited to speak at BSides San Francisco for the second year in a row. I went all out and customized my talk to fit the conference theme: Preparing for Dragons: Don’t Sharpen Swords. Set Traps, Gather Supplies!

I particularly loved the work I did with HD Moore, Tod Beardsly and the other folks at runZero. The vulnerability management market is so overdue for reinvention and the folks at runZero are helping to lead that movement. In fact, I’m SO passionate about vulnerability management, I had to make a reminder for myself when interviewing Tod on Enterprise Security Weekly: “don’t be an asshole, let Tod talk”.

Rob Allen from Threatlocker is always a blast to interview and has the craziest stories. My recent interview with Wendy Nather on Toxic Anthropomorphism in AI was a recent highlight as well.

Outside of CRA webcasts and podcasts, my IANS advisory calls with enterprises kept me grounded in the reality of what enterprises are actually dealing with. I also particularly enjoyed getting to create and build the Alice in Supply Chains podcast alongside Alexandre Sieira, Mariane, and the other folks at Tenchi Security. This was the first podcast I’ve built for a client from the ground up. The design was a collaboration, but I prepare, produce, edit, and deliver every episode myself. Alexandre and I have a great time recording every episode, and it has been eye-opening watching and learning the trends in the third party cyber risk space.

I honestly did so much in 2025, it would probably take me days to go through everything I did and pull out all the highlights!

Here are the numbers, if you’re interested:

Goals and Changes for 2026?

In 2025, I did a webcast from a hotel room in the Paris airport. I did two webcasts and a podcast from Armenia. I did podcasts and webcasts from Barcelona, San Diego, Toronto, Tuscaloosa, NYC, and St. Louis. I’m proud of my minimalist travel kit that makes it possible for me to deliver good quality audio and video from anywhere, but last year I did too much.

One morning, before my flight to St. Louis, I fell down a flight of stairs. My partner broke her ankle on the streets of St. Louis the next day. A week prior, we were discussing whether or not we were doing too much. We had our answer.

In 2026, I want to write more, research more, and start producing videos based off my writing (mostly think pieces, educational stuff destined for YouTube). I have a LOT of thoughts, ideas, and research to share, but I need time to mold them into something consumable. Eventually, I hope is to be able to monetize my writing and research.

I also need to get my stuff together and operate as a proper business. Since I wasn’t sure if going solo was going to work out, I didn’t initially get an LLC, logo made, business accounts, EIN, etc. This year, I’m going to do some adulting and separate business and personal. Just in the first week of January, I’ve checked off a lot of the tasks on that list.

It looks like I’ll be building another vendor podcast in 2026. I enjoy doing this work, but I’m a little worried about everything I do on camera sounding, looking, and feeling similar (same background, same dude, same brain). I’m thinking about how to make sure that each podcast I build has a unique look and feel.

I’m also trying out building training classes in 2026. Expect to see more from me on that front with Just Hacking and IANS.

Where you can find my stuff

• Hosting the Enterprise Security Weekly podcast

• Hosting the Alice in Supply Chains podcast

• The webcasts I do with CyberRisk Alliance

• Most of the advisory work I do is through IANS

• But the startup advisory work I do is direct - you can schedule something through my Calendly.

Subscribe now

I highly recommend watching the full Cloud Security Podcast episode. Edward Wu always comes across as honest and speaks without hyperbole. I get the sense that, even as CEO, he still has an engineering role within his startup, or at least, he remains very close to the tech development. Ashish Rajan asks some excellent questions, prompting Wu on exactly the specifics I was hoping to hear more about.

There’s a lot of discussion on the parts of SecOps you can’t use AI to automate or solve. Also discussed are the bits that prevent AI from being successful no matter how intelligent it is, like institutional knowledge that isn’t documented anywhere.

Watching the episode, I’m reminded of how much of the funding in the cybersecurity industry is going to the lemonade makers. If you haven’t read my essay, A Market for Lemonade, the TL;DR is that a lot of cybersecurity vendors (the lemonade makers) exist to solve problems created by other cybersecurity vendors (the lemons). It’s worth exploring why this is the case.

Why everyone wants to make lemonade

Simply put, it’s easy to build a product that analyzes security data you already have. You can’t find threats in data you don’t have, however. Too often, we miss the fundamental questions: is this the right data? Is the data correct? Is the data complete?

It’s hard to build a library of 200,000+ vulnerability checks, so startups in the vulnerability/exposure management space are almost exclusively lemonade makers (RBVM, UVM, CTEM) - at least, where we’re talking about infrastructure scanning (i.e. vulns linked to CVEs). The only innovator in the vulnerability scanning space in the past 20 years was a small startup out of Montreal called Delve Labs. It was acquired by Secureworks (now Sophos) and renamed Taegis VDR.

The challenges don’t stop with building the vulnerability checks. The buyer has a lot of responsibility here that can impact the quality and completeness of data. Practitioners have to configure the product effectively (configuring a vuln scanner is easy to mess up). They have to input the correct lists of assets for the scans. They have to connect it to the right accounts.

This is a reminder that the idea of build vs buy is a false choice. It would be more accurate to describe the choices as build alone versus build with others. There are few, if any, cybersecurity products on the market that don’t require the buyer to do significant work before the product can be useful. I call this the customization tax. This isn’t the vendor’s fault - every enterprise is different. Vendors can only do so much when building a product for a broad market.

The vendor has a lot of responsibility as well. The big three vulnerability scanners on the market don’t do a great job of correctly identifying IoT/OT devices. Scan a Ubiquiti device and they’re baffled - they’ll tell you it’s a Linux server running an end-of-life version of Debian. So, of course, there are vendors that specialize in only scanning IoT devices. You could even buy several complementary scanners and still have enormous gaps in your data.

In SecOps, detection engineering is the data challenge. Do we build broad or narrow detections? Are we getting all the necessary data to build the detections? Are there delays and bottlenecks in data collection and querying?

In third party risk management, you’ll never have time to perform deep due diligence and monitoring on all your third parties. Which vendors represent the biggest risks? Are you asking the right questions on your questionnaires? Are the responses accurate and trustworthy?

Everyone wants to make lemonade, because building sensors and gathering data is hard. Many buyers love making lemonade, because they start off with a mess of data and end with a nice dashboard with scores, prioritization, and metrics. When buyers see a vendor turn a million critical vulnerabilities into a ‘top 10 patch ASAP’ list, it feels like progress. Lemonade aims to be tasty, not healthy.

Making Lemonade Doesn’t Address Root Problems

Garbage in, garbage out. It’s a common phrase, but the challenge in cybersecurity is that we don’t have enough folks skilled in determining the quality of our data. Vendors and their data scientists get excited about markets where there’s a lot of data, because they don’t have to go out and create the data. It’s already there and ready to be analyzed, sorted, normalized, reduced, and summarized.

An important point: vendors’ products don’t become lemonade makers until the buyer feeds them lemons. It is largely on the buyer to ensure they’re not feeding bad data into the hopper. For example:

• What if the customer fat-fingered one of their IP ranges? Instead of scanning 10.1.2.0/24, they’re scanning 100.1.2.0/24.

• Perhaps there is also an external class C network the security team is unaware of, so it has never been scanned from the outside.

• Security Rating Services only see a company’s external infrastructure, and often get companies’ assets confused and mixed up.

• If you don’t pay for Salesforce Shield (reportedly 30% of your total Salesforce spend, ouch) and lack logs, you can’t build Salesforce-related detections.

If the data is wrong or missing, there’s no way to magic a win out of it with AI or any other technology.

This is why edge devices get hacked, despite fixes being available for months or years before the attack. Perhaps they weren’t getting scanned. You can’t protect the assets you don’t know about.

This is why attackers are able to drop small Linux VMs on servers and desktops as a base of operations. Detections aren’t looking for WSL.exe in process lists, or new VMDKs showing up in %APPDATA%.

This is why the company that gets breached always has an A+ on some security rating service’s scorecard, and the ones that don’t get breached often have D’s or C’s. The rating services don’t have enough data to make an accurate call, but as long as some data exists, they’ll make lemonade.

Check Your Ingredients

To avoid making lemonade, you’ve got to check the quality of the data you’re giving to these ‘overlay’ cybersecurity products. If you’re planning to buy a product, and step 1 is to ingest data that another one of your products collected, stop and consider:

1. how comprehensive is this data?

2. is the collection of this data taking into account current attack scenarios and TTPs?

3. how accurate is this data (e.g. what are false positive rates, do your analysts trust it?)

4. how would I know what the quality of this data is (i.e. do I need to hire a third party expert to tell me?)

We also have to be careful with metrics. The lemons and lemonade makers in the market are great at generating empty calories - metrics that look and feel like progress, but have no impact on your security program’s desired outcomes. You patched 100,000 vulnerabilities, congrats! Did any of these vulns represent any real risk to the business? Statistically, the answer is probably not. It feels great when that vuln count line goes down though. Lemonade is delicious.

But is your goal to satisfy a sweet tooth, or to get healthy?

You can’t magic good outcomes from bad data.

Conclusion

Dropzone and others are building some impressive scaffolding for automating mundane, repetitive SecOps tasks, but there is a question we have to ask before paying $9 an alert for agentic automation: “am I feeding it trash?”

Defenders need to give more attention to the market niches that focus on validating the quality of our security data. Products and services that help connect theory and best practice to reality.

Only when we’re sure of the quality of our controls and data can we get value out of our products, not lemonade.

1. requires expert knowledge

2. requires special tools, or

3. requires access the buyer doesn’t have

The cybersecurity market takes this to an extreme: sometimes, even the sellers aren’t aware of the quality of their product. Ross Haleliuk said it best in his book, Cyber for Builders.

“The quality of what is bought and sold is not known.”

When evaluating, buying, and using cybersecurity products, it can be difficult to determine how well a product works. It isn’t uncommon to discover security products that weren’t even functional after being deployed. Another challenge is determining how effective the product is.

No one is (or should be) satisfied with simply blocking EICAR, for example. If an anti-virus product is only good at blocking commodity malware but no self-respecting cybercriminal will ever use commodity malware, this product isn’t useful for much more than a compliance checkbox.

FireEye, it had what networks crave

Back in 2012, I was building a security program for a large enterprise. I spent the better part of a month trying to figure out what FireEye’s product did. This was before FireEye went public and their only product was the NX appliance. Every time a salesperson pitched me the product, their description brought me to the same conclusion: “so, this is intrusion detection? This is an IDS/IPS?”

The salesperson quickly pushed back on the description. “Oh no, I was told in no uncertain terms that our product is NOT an IDS or an IPS device.” This was unsurprising, as Gartner had declared IDS dead as far back as 2003 (it is still commonly used today). We’d loop back to the beginning of the conversation and they would explain it to me all over again.

After chatting with a third sales rep at FireEye, it became obvious that their own employees (or at least their sales teams), didn’t seem to understand what the product did.

The fourth time was the charm. I finally got an engineer on the line and was told that the FireEye NX appliance was designed to catch malware on the wire, but only really bad malware. It would ignore the commodity stuff. Long story short, my org bought a heavily discounted NX appliance, despite my recommendation to pass on it.

We pulled it out of the box, racked it, plugged it into a SPAN port (no chance were we going to put it in-line) and powered it on. The first challenge was how to determine whether or not it was working.

How do you test a product that only detects ‘really bad’ malware? What even is ‘really bad’? How does it determine bad from not-so-bad malware? FireEye had a solution: a custom PDF that, like EICAR, was guaranteed to trigger an alert every time. That solved the problem of functional testing. Would it be effective though? Would it actually detect malware?

It did not. FireEye generated roughly one false positive per month. In the same time, we were hit with a few malware infections per month. Malware got past the FireEye appliances by shipping as a JAR file that contained an obfuscated WinPE that it would reassemble on the desktop, after it got past our watchful FireEye’s network surveillance. This was one of many malware delivery evasions that worked over the years to bypass security products.

The product didn’t just lack value, it had negative value. I calculated that the cost of the product itself, plus the time it wasted every time we had to respond to a false positive and investigate it, put FireEye’s value over six digits in the red.

Another problem was that, due to the cost of the appliance, we could only afford to purchase one, and put it in our headquarters, where we had the least problem with malware infections. Most of our issues were at sales offices in the field, that often had no more than 5 employees per office. The whole idea for the device was fundamentally flawed.

This would become even more obvious years later as we saw the company struggle with heavy churn (customers not renewing contracts). I was an industry analyst at this point and referred to this event as FireEye Buyers’ Remorse. This was unsurprising, given my experience with the product, and my theory that both the seller and buyer didn’t seem to understand what the product did. The NSS Labs/FireEye battle a few years later further proved this theory.

The FireEye story is a long-winded way to point out that the Cybersecurity market produces a lot of lemons. The average buyer doesn’t have the security talent in house necessary to simulate a real attack and determine if there’s any value in buying these kinds of products. They can maybe afford to pay for a penetration test once a year, but unless they’re paying for the best in the business, they’re probably going to get someone a few years out of school operating mostly automated tools.

Security products aren’t all about preventing attacks, however. We have security operations products, GRC products, application security products, vulnerability management products, and more. Most suffer from similar information asymmetry issues that hark back to The Market for Lemons.

It is not practical for most companies to properly evaluate and test security products before buying them.

When life gives you lemons

Cybersecurity products also have this nagging tendency to create entirely new problems. A large portion of the market pumps out what I think of as busywork generators. Right out of the box, these products will consume logs, scan for vulnerabilities, detect potential attacks, and identify compliance gaps. Overnight, security analysts can be buried in hundreds of thousands or even millions of tasks.

You’ve probably heard the terms ‘alert fatigue’ or ‘vulnerability overload’. This problem, again, has its roots in information asymmetry. The vendors say, “this stuff is bad”, and many buyers aren’t in a position to refute or challenge it. Meanwhile, security products fail to stop threats, detect attacks, or alert practitioners when products are misconfigured. If you’ve been a practitioner for even a year or two, you likely have examples you can cite. Here are a few of mine:

1. When I first used a SIEM in 2004, I was floored to discover that there was no mechanism to tell me when devices suddenly stopped sending logs. I had to build it myself. Again, in 2012, I found nothing had changed. SIEMs still didn’t perform this basic, fundamental task.

2. I once discovered that a client’s DAST scanner, which was scanning 14 websites by domain, had 13 misspelled domain names (they were missing the ‘m’ on dot com). 13 of the 14 domains it was scanning didn’t exist, and the tool didn’t tell them that. Since 1 of the 14 was getting scanned, they assumed everything was fine.

3. These days, security tools represent significant security risks and attack surface - one out of ten vulnerabilities in CISA’s known exploited vulnerabilities list belong to a security vendor.

Visibility is important in cybersecurity - no security leader wants to be in a situation where they’re blind to an attack, a vulnerability, or a gap in their compliance program. Buyers ask the vendor to show them everything, and their staff drown in the results. I’ve observed an unwillingness to reduce this noise level.

Why? A sort of hoarding or FOMO effect exists with security leaders. Is it more defensible to enable all the alerts and say “we missed it because we don’t have enough people in the SOC”? Or is it more difficult to disable the noisiest alerts and risk missing something?

You can bet the market noticed this problem and was eager to sell a solution.

This market makes lemonade

Initially, the solution was to click the export-to-CSV button and make sense of the data in Excel. However, once vendors noticed buyers struggling, we started to see entire market segments spring up to solve the problems that other security products created.

Pause for a moment to consider a scenario:

1. You bought a vulnerability scanner for $75k. You run your first scan. The results overwhelm your team.

2. You ask for more headcount and hired more folks to address the workload. Maybe this worked for a while, maybe not.

3. Web 2.0 happens. Your company builds fancy webapps and maybe even offers SaaS to customers.

4. You buy a web scanning tool for $35k. You run your first scan. The results overwhelm your team

5. The cloud is invented and your company adopts it, but doesn’t get rid of the existing datacenter.

6. You buy a cloud scanning tool for $50k. You run your first scan. The results overwhelm your team.

7. By the way, you’re not sure if ANY of these tools have gaps. You’re not sure if they’re finding all the critical vulnerabilities, because testing security tool efficacy is hard. Also, you don’t have time, because you’re too busy chasing all the busywork these tools are generating.

8. A product category emerges and offers an enticing pitch: what if we took all those scans, combined them, and prioritized the findings, making your staff’s job easier? This product costs $125k.

Now we’re making lemonade. The buyer isn’t even sure the scanners are producing value but are buying another layer of products to fix the problems created by the first layer. I want to be clear - the lemonade makers aren’t the lemons, they’re just spotted a market opportunity: “hey, that’s a lot of lemons you’ve got there. You might as well make some lemonade, right?

Some examples of lemonade makers:

• The risk-based vulnerability management (RBVM) market, which is the example from the scenario above. These tools don’t include a vulnerability scanner, so you’ve got to purchase them in addition to vulnerability scanning tools.

• Security analytics, SOAR, UBEA, and others were add-ons to the classic SIEM, offering to do what the original SIEM vendors promised 20 years ago: correlate data, extract insights.

• But wait, why did we need a SIEM in the first place? Probably because we didn’t want to have to log into a dozen different security products to check for alerts, correlating timestamps across devices manually. We’re potentially three or more levels deep on this one, especially if you have a data lake, ETL products, an MSSP/MDR to handle SOC monitoring in the off-hours…

• Products like FireEye’s NX (’Breach Detection Systems’) positioned themselves as complementary to IDS and IPS devices as well as endpoint security products.

• You can buy a license for an expensive TPRM spreadsheet and then an AI-powered GRC product to automate filling it out.

• Interestingly, there’s some analysis out from At-Bay showing that you’re more likely to file an email-incident-related cyber insurance claim if you have an email security appliance vs just using the security built into your email platform. Dig a little deeper and you’ll find that At-Bay is also getting into the lemonade business.

• I’ll stop now, but feel free to mention more in the comments, or challenge any of the examples I’ve come up with here.

Hope you’re thirsty

The problems created by core security products are almost always very obvious process issues (e.g. too much data to process, analyze, or action). This is great for second tier products, because it’s very easy for them to demonstrate and measure value in terms of time saved versus the old product. The problem is that the buyer doesn’t know if the core products were doing a good job to begin with.

“Wait, I didn’t even want lemons in the first place, did I?”

I can’t fault these second-tier lemonade vendors, because they’re responding to real issues that customers have. It can be an uphill battle for them as well - it’s not easy for buyers to hear that they need to buy an additional product to make their existing tools work properly. Buyers will often defer the decision until the problem becomes super painful.

Why doesn’t this secondary market simply replace the core vendor and solve the core problems, then?

There might not be a universal reason for this, but I have a few thoughts:

1. rebuilding the core product is difficult and expensive (e.g. building 200,000 vulnerability checks from scratch)

2. the solution to the core problem isn’t obvious

3. the industry’s collective sunk cost is too great: the solution requires a completely different approach, requiring the vendor to reeducate the market and go against common practices, certifications, standards, and regulations

The final theory is a really tough one. It has been observed that security leaders are swayed towards the choices that are safest for their personal careers. When the choice is going against the grain versus doing the safe thing, the data tells us the latter is going to be the more common path.

Conclusion

There’s no Ozempic for cyber yet, so we’re faced with a familiar dilemma. We can keep downing lemonade, getting nowhere with our goals, or start doing the hard (diet and exercise) work necessary to determine what works and ditch what doesn’t. The latter is tough, as there are little to no incentives to challenge the status quo, even when it is clearly broken.

In consumer markets, information asymmetry tends to solve itself when value is obvious. If a particular model or brand of laptop breaks a lot, it will have lots of bad reviews online. A laptop breaking is an outcome that can’t be ignored.

Cybersecurity isn’t like this. It is more comparable to the health supplement market. Do I feel amazing today because I got an extra hour of sleep, or because I started making drinks with this green powder? Most consumers don’t have the time or patience to scientifically test every new nutrition product or trend they try out, so if they think it makes them feel better, they’ll keep buying it or doing it.

At least with nutritional supplements, there will be folks that will put them to the test, or send products to a lab to determine if they’re even safe to consume. Cybersecurity doesn’t really have an equivalent, unfortunately.

This is a critical problem in cybersecurity. When we don’t know how well our tools or controls are working, we’re at a distinct disadvantage as defenders. As Charlie Miller once put it:

Buyers should be pushing back on products that create more problems than they solve. However, without a clear path to better products, practitioners continue on, buying what their peers bought that didn’t get them fired. Ironically, a big part of the problem is a lack of risk appetite for better security products.

UPDATED on June 5th, 2026 to add SSH and emphasize that ALL administrative interfaces should be taken off the Internet.

We see it over and over and over again with breaches:

1. attackers got in via RDP (Atlanta, Ryuk)

2. attackers got in via VPN (Pulse Secure, SonicWall/Akira)

3. FTP credentials were guessed or brute-forced (SamSam, General Scanning & Attacks)

4. File transfer products exploited (Accelion FTA, MoveIT, GoAnywhere MFT)

5. Firewalls exploited via their management interfaces (Juniper ScreenOS backdoor, PanOS vulns, FortiGate 0days)

6. Web-based administrative interfaces, like cPanel

Each of these examples represent old ways of accessing services. For each, there are now better, more secure ways of performing each of these functions, without exposing these services to the public Internet. These are also some of the most popular attack targets for attackers!

That’s why I think of this as the asbestos of IT.

Asbestos was also very useful, but dangerous to humans. As soon as we realized this, we labeled it as a dangerous substance and started replacing it with safer materials. The time has come to replace outdated, dangerous protocols with more secure alternatives.

The key to replacing each of these services is to find alternatives that don’t require exposing TCP/UDP ports to the public Internet.

1. Replacing RDP: There are a TON of options here. You likely have something built into whatever tool you use for managing your mobile devices or servers. In liu of that, I personally like RustDesk, because it doesn’t require you to trust a third party like AnyDesk and TeamViewer do - you can set up and manage your own server. It does still use a direct TCP connection, however, so you’ll need a modern VPN technology, which brings me to…

2. Replacing VPN: Whether you call it ZTNA or SDP, the big innovation here was allowing access without opening ports. I use Tailscale, which is how I get to RustDesk on my hosts (I don’t bother with the RustDesk server, I just connect direct client to host). I recently tested RustDesk over Tailscale, on an iPad, from Delta’s in-flight Wi-Fi, and it was like I was sitting right in front of my studio PC. I was very impressed.

3. Replacing FTP: There are so many options for file sharing or file transfer that we’re spoiled for choice. The replacement depends on your use case. Publishing files to a web server? Use GitHub or other code deployment tools. Business to business transfers? S3 bucket (or GCP/Azure equivalents). Consider something like ShareFile, Dropbox, Box, Google Drive, OneDrive for human to human file sharing.

4. Replacing old-school file transfer products: See #3 above.

5. Firewalls getting exploited via management interfaces: Don’t ever share management consoles on Internet-exposed interfaces! See #2 above

6. Web-based administrative interfaces represent broad concentration risk - WordPress and cPanel alone probably represent the majority of all websites on the Internet. Wouldn’t it be crazy if there was a basic, RCE that affected all of them? This one is a bit trickier to rearchitect than the previous 5 examples, but it’s the same basic approach. Put an authentication gateway in front of the web services themselves1, or use modern VPN technologies to access them. Even IP-based restrictions are better than nothing!

7. SSH and any other administrative interfaces. If 3 people need to access SSH on a server, why give the entire Internet access to it? Any configuration mistake, new vulnerability, or stolen SSH keys can immediately be leveraged by attackers if accessing the actual TCP service is trivial. Make it difficult - use ZTNA to access it. Use IP access control lists. Anything aside from opening it up to the entire world.

As always, some caveats: yes, I understand your vendor requires you to use FTP, there’s not much you can do about that, except to fire your vendor, or ensure there’s nothing too sensitive being transferred.

There are other reasons you might not be able to get off legacy protocols. Document them, put mitigations and detections around them as best you can, and be ready to respond to any incidents that come from using them.

If you can ditch legacy protocols, however, the reduction in attack surface will be worth it and you can sleep that much better at night.

It has been a few years since I discussed this and the last time I did, it was a bit buried at the end of the post. I thought I’d share an approach to detection engineering I came up with a long time ago that might help. I’m also a bit baffled that the industry seems overly focused on addressing alert fatigue from the back end (using AI to triage, enrich, and investigate) rather than on the front end (avoid generating so many alerts in the first place).

The Red Flags vs Yellow Flags approach

Red flags vs yellow flags is a simple system I came up with to focus when building out detections. You really only need one red flag to know an attacker is present. After that, diving into the details becomes easier. By focusing on a smaller number of higher quality detections, alert fatigue is alleviated, response times improve, and coverage improves.

I’ll repeat the most important bit here: you don’t need to alert on everything an attacker does. You’re still logging everything the attacker does, you’re just not filling up alert queues with every action they take.

I came up with this approach, naturally, after initially doing everything the wrong way. I ingested every event from every source into the SIEM and enabled every alert and detection. We were instantly buried in noise. I started to think, what if I built this in reverse from what I’ve just done? What would that look like?

1. Study how attacks happen and succeed. What do attackers have to do? What are TTPs we see them using over and over?

2. How few of these actions can we get away with looking for and still ensure we’re catching attackers every time? Which actions are the easiest to detect reliably, with a very low false positive rate?

3. What data do I need to collect to detect these things happening?

4. Start building detections for these things.

5. Test to see that my detections work

6. Make sure we have SOPs or playbooks for these detections (containment and/or eradication actions: disable accounts, kill sessions, isolate machines, etc)

7. Do the thing: test to see that my SOC folks notice the detections when they fire and handle them according to plan

Defining Red vs Yellow

Step 2 is where we’re separating yellow flags from red flags. Red flags are alerts that are always bad. They’re relatively easy to detect with a near zero false positive rate. Mimikatz is always a red flag. A binary downloaded from a domain that has only existed for 3 days is almost always bad. A cryptominer on an EC2 instance is a bad sign. A powershell terminal executed by a Word document. You get the idea.

This approach doesn’t mean we totally ignore yellow flags, just that we ensure we don’t miss any red flags due to noise from the yellows. A yellow flag is a lower quality tier of detection - something that is suspicious, given the right criteria. They’re maybes. Maybe this is a red flag, if it’s 3AM and comes from the receptionist’s PC and talks out to a strange host. Most orgs are drowning in maybes.

A login event at an odd time might be suspicious, but it won’t always be a red flag. Or perhaps, it’s a red flag, but only for three employees. Impossible travel alerts are an example of this. Impossible travel once seemed like a high quality detection, but sometimes proxies, VPNs, or SaaS products reduce the reliability of these alerts. Several yellow flags could amount to a red flag: a compound detection. Say we see a login from an odd location, followed by a very broad query in a database, and an attempt to bulk download the results. Feels like a smash and grab, perhaps by a ransomware crew.

When have we gone too far?

As you can see, the more we go down this path of adding context and complexity to detections, the chances for false positives go up. The most difficult part of this approach is resisting overbuilding detections. Fear of missing out on events can lead to alert hoarding, which can lead to alert blindness. Trying to see too much results in blindness.

Even one reliable, well-tested red flags detection is a huge win - I’ve worked in so many environments where even the noisiest penetration test doesn’t set of a single alert. Limiting the number of detections doesn’t mean that you’re not logging all the other actions performed by the attacker, you’re just limiting alerts to what is manageable for your organization and ensuring they work.

Herein lies the most difficult part of this approach: being comfortable with the quiet. Inevitably, folks ask, “but Adrian, I might miss hundreds of advanced and sophisticated attacks!” The reality is that even the most sophisticated actors still use some very basic tools and techniques (because they work), and you’re not ready to start building or enabling detections for these kinds of attacks if you don’t have the more common ones working. Resist the urge! Seriously, get good at detecting and responding to just ONE common type of attack. Get really, really good at it, to the point where you never miss it, and you’ll understand where the bar needs to be set for the rest of your detections.

The advantage of this approach, is that it should result in a relatively quiet SOC if nothing is happening. This creates time and opportunities for threat hunting and further finding red flags and tuning detections to find them.

A less noisy approach won’t need to lean so heavily on AI, if at all. I personally don’t think generative AI can scale to the needs of large SOCs, but that’s a discussion for a different post.

Bonus Round: Deception

Modern honeypots and honeytokens are a great way to create red flags from scratch. By definition, they shouldn’t be accessed or used by legitimate employees, so they’re a great way to artificially create red flags. Create a fake admin account that doesn’t belong to any admins and seed its credentials in key places. Any use of this account is a red flag. As always, canarytokens.org is the best place to start exploring and testing this approach. This is also the quickest way to understand what a great, red flag detection looks and feels like.