Yearly security testing doesn't work anymore

Most businesses run a security audit once a year. A penetration tester comes in, writes a report, the critical items get fixed, the PDF gets filed. Until recently, that was reasonable. Finding vulnerabilities was slow, expensive, manual work.

That just changed. Last week, Anthropic announced they wouldn't be releasing their latest model, Claude Mythos, to the public. Instead, they've restricted it to a small group of security partners under something called Project Glasswing.

It's too good at finding security vulnerabilities. Their red team blog describes it chaining four separate vulnerabilities into a working browser exploit that escapes both the renderer and OS sandboxes. It escalated from ordinary user to full control of a Linux kernel box by finding and combining several vulnerabilities on its own. It wrote a remote code execution exploit on FreeBSD that gave full root access to unauthenticated users.

It found a bug in OpenBSD's TCP stack that had been sitting there for 27 years. A 16-year-old vulnerability in FFmpeg - the video encoding library used by virtually every piece of software that handles video - in a line of code that automated testing tools had hit five million times without catching the problem.

Nicholas Carlini from Anthropic's team: "I've found more bugs in the last couple of weeks than I found in the rest of my life combined."

During internal testing, Anthropic's Sam Bowman received an unexpected email from Mythos while eating a sandwich in a park. The model had been placed in a secured sandbox that wasn't supposed to have internet access. It found a way out and emailed him to let him know. He called it "an uneasy surprise."

Open source maintainers are drowning in real reports

It's not just Anthropic. The people who maintain the software everything runs on are seeing the same shift.

Greg Kroah-Hartman, one of the Linux kernel's lead maintainers, said something changed about a month ago. Reports went from obvious AI slop to real discoveries. "Now we have real reports. All open source projects have real reports that are made with AI, but they're good, and they're real."

Daniel Stenberg, who maintains curl (a tool installed on virtually every server and device you've ever used), is spending hours per day on AI-generated reports. Not junk. Real bugs.

Thomas Ptacek, a well-known security researcher, published a post titled "Vulnerability Research Is Cooked" - meaning the old manual approach to finding bugs is being overtaken by AI at a pace nobody expected.

What we actually do about it

We run AI security audits across our codebases every day. AI agents do rotating deep dives across servers and code - exposed credentials one day, package dependencies the next, network configurations the day after. Code changes get reviewed for vulnerability patterns as they're made, so problems get caught when they're introduced.

If your developers aren't running automated security scanning on every code change, start there. AI-assisted code review costs very little on top. And if you're relying on dozens of open source packages (most modern software is), tools like Dependabot and Snyk flag known vulnerabilities automatically.

Anthropic is putting $100M in credits and $4M in direct donations into this through Glasswing. Launch partners include AWS, Apple, Broadcom, Cisco, CrowdStrike, Google, JPMorganChase, Microsoft, NVIDIA, Palo Alto Networks and the Linux Foundation, with over 40 additional organisations given access.

Anthropic has committed to publicly reporting findings within 90 days.

The OpenBSD bug got patched. The Linux escalation bugs got fixed. Defenders found them first this time. Whether that keeps happening depends on how quickly everyone else catches up.