Skip to main content
Success
BACK TO BLOG
Phil Webb

Security testing needs to follow the code

AI-assisted research is another reason to check systems between annual assessments.

Security testing needs to follow the code

An annual penetration test gives a business an independent assessment at a point in time. It needs to sit alongside checks as code, dependencies and infrastructure change.

On 7 April 2026, Anthropic announced Project Glasswing, giving selected partners access to Claude Mythos Preview for defensive security work.

Its technical report describes a browser exploit chaining four vulnerabilities to escape the renderer and operating-system sandboxes. These are Anthropic's reported results, rather than an independent comparison of every model or researcher.

Anthropic also reported finding a 27-year-old OpenBSD vulnerability and a 16-year-old FFmpeg flaw missed by millions of automated test executions. It said those disclosed examples had been patched.

The useful question for a development team is what it can learn from these methods and verify in its own software.

A capable security model also needs restrictions. Test tools against systems you are authorised to assess, and limit the credentials and network access available during the work.

A long-undetected bug is a reason to revisit familiar code.

Reports still need investigation

More findings create work for maintainers. Someone has to reproduce the issue, establish its impact and check that a fix doesn't break something else.

An AI-generated report can describe a real vulnerability. It can also misunderstand the code. Treat the report as a lead to investigate.

A useful report includes a reproducible example, affected versions and a clear explanation of impact. That gives the maintainer something to test.

The same standard should apply to a report from a person or a model.

Testing needs to follow changes

Software that passed an assessment can still contain an undiscovered flaw. New research may expose it without the application itself having changed.

AI gives defenders another way to examine code. It also gives attackers assistance, so teams need a process for responding to credible findings.

Keep the independent assessment, and check what changed between assessments.

What we actually do about it

We run recurring AI-assisted checks across our code and infrastructure, rotating through areas such as dependencies, exposed credentials and configuration. Findings need verification and any fix needs testing. These checks can miss problems, so they complement other security work.

Automated dependency and code checks are a useful starting point. Add AI-assisted review where it helps your team investigate, and make sure someone owns the findings.

At launch, Anthropic committed up to $100 million in model credits and $4 million in direct donations, with access for its launch partners and more than 40 additional organisations.

It also said it would report on the programme's findings and lessons within 90 days of launch.

Finding a flaw is only the first step. The practical result comes when the affected systems receive a tested fix.

If you want to talk about how AI fits into your development or security workflow, get in touch.

Message us on WhatsApp