You can get a taste of this today yourself with Codex Security. I turned it on just as an experiment and in less than a week it has now become essential to all of us. I was shocked how accurate it is, how many security issues it found in existing code, how it continually finds them as we commit, and how NO ONE is immune from making these mistakes.
I'd say it is about 90% accurate for us. Often even the "Low" findings lead us to dig and realize it is actually exploitable. Everyone makes these mistakes, from the most junior to the most senior. They are just a class of bugs after all.
I expect tools like this to be a regular part of the development lifecycle from here on. We code with AI, we review with AI, we search for vulns with AI. Even if it isn't perfect, it is easily worth the cost IMHO. Highly recommend you get something enabled for your own repos ASAP
> I expect tools like this to be a regular part of the development lifecycle from here on. We code with AI, we review with AI, we search for vulns with AI. Even if it isn't perfect, it is easily worth the cost IMHO.
So, how is that supposed to work? Claude Code generates security bugs, then Claude Security finds them, then Claude Code generate fix, spend tokens, profit?
One issue I've seen with LLM's is adding superfluous code in the name of "safety" and confidently generating a bunch of stuff that was useful in years gone by, but now handled correctly by the standard lib. I'm of the opinion that less is more when it comes to code, and find the trend this is introducing quite frustrating.
I’ve had the same experience. The ui is a little unclear about this, because it says you have 5 scans, but 1 scan is just the continuous monitoring of the default branch of a repo.
The high impact findings have almost all been bang on for me.
I was especially surprised by the high-quality documentation it produces as well as how narrow the proposed fixes are.
I’m used to codex producing quite a but more code than it needs to, but the security model proposed fixes that are frequently <10 loc, targeting exactly the correct place.
It’s really quite good. I’m assuming it’ll be pretty expensive once out of beta, but as a business I’d be jumping on this.
I would recommend you to try out the setup with gpt-5.5-cyber as the orchestrator and deepseek-v4-flash or some other fast cheap model as its workers. Getting pretty good results using this setup.
This got me thinking, so what happens in two years?
every tom, dick and harry who can type english has the tools to attack any software that isn't patched.
tools that were accessible to specialized groups, now made available to anybody with a grudge and a few dollars for tokens.
and what does anthropic and openai do? They form an inner ring to make the latest models available first to Enterprises. Enterprises will cough up the prices that anthropic and openai set, they have no choice here. e
Eventually everybody pays. This does not sound good
This is what I did. Using a loop skill to dig problems and bugs in each step on development from design to coding to make sure the output software works properly and on purpose.
I help maintain a project that is used as a dependency by a lot of security tools to handle PE files.
It’s disappointing that Anthropic and OpenAI never responded to the applications to their respective programs for open source maintainers. From my perspective it seems like their offers are primarily for the shiny well-known projects, rather than ones that get only a few million monthly installs but aren’t able to get thousands of stars due to being “hidden” as a dependency of popular tool.
"get a taste of this". The real thing is, GPT-5.5 is better than Opus 4.7, so if Anthropic doesn't release Mythos soon, other people are going to notice and switch off Claude.
It seems to me like either your architecture is fucked up or you’re using the wrong language/tooling for the type of software you are making if you’re introducing security vulnerabilities that frequently.
> I was shocked how accurate it is, how many security issues it found in existing code, how it continually finds them as we commit, and how NO ONE is immune from making these mistakes.
Dude is flexing that he's pushing unsecure code every day, that's a skill!
I’m not sure how to reconcile anthropic’s update / some of the exuberant comments here with recent feedback like the following from curl maintainer Daniel Steinberg:
“I see no evidence that this setup [Mythos] finds issues to any particular higher or more advanced degree than the other tools have done before Mythos. Maybe this model is a little bit better, but even if it is, it is not better to a degree that seems to make a significant dent in code analyzing.”
You’re right, it’s a valid data point. But the U.K. government report is also a data point, and the Firefox report is a data point, and they suggest that it is, indeed, significantly better than current generation models. Maybe curl is significantly better hardened than most projects?
In any event, it barely matters. As Anthropic acknowledges, next level models are comings, theirs is only one of them. Current generation models are already good at things like tracing data flow through complex systems and there’s no reason to think that capability has topped out. So within a year it seems very likely we’ll have more than one commercially available model able to find vulnerabilities cheaply.
On the other hand, it seems that they’ve made much less progress on getting it to design solutions to these issues.
I think people sometimes misunderstand Daniel's point here, though it's clearer when taken in context of the rest of his article. The tools in general are getting a lot better at finding security bugs, it was unclear to Daniel based on his usage whether Mythos in particular is a huge step, but the Mythos generation of LLMs definitely are. Note though that Daniel was using Mythos somewhat indirectly. One thing I've taken away from the whole Mythos debate is that a) I suspect that Anthropic's GPU crunch meant that they felt they had to ration Mythos access anyway, so the calculus of whether they would release it generally was probably influenced by that, and b) finding bugs with Mythos or a similar model is still expensive -- a $20K or $100K Mythos run on Curl might have shown the same level of issues as other projects like Firefox, but Daniel didn't get that kind of access.
He posted a general update today on LinkedIn which I think gives the wider context:
> Not even half-way through this hashtag#curl release cycle we are already at 11 confirmed vulnerabilities - and there are three left in the queue to assess and new reports keep arriving at a pace of more than one/day.
> 11 CVEs announced in a single release is our record from 2016 after the first-ever security audit (by Cure 53).
> This is the most intense period in hashtag#curl that I can remember ever been through.
Curl has more eyes on it, and has had more tools thrown at it, and is better tested (and developed?) than 99% of software, it's very much not the norm. I wouldn't be surprised if that has something to do with it, if there is any kind of bias there (not sure if there is, it's also possible he's just right).
Daniel has been posting for months (years?) about how much scrutiny he gets from security researchers and various automated tools. I wouldn't expect curl to be the average case for mythos.
Curl, according to the authors own admission, is the most heavily tested and fuzzed open source library out there. So I think for him it's a different situation
It's a weird accident of fate that curl has somehow become the reference target for LLM bugfinding. Curl is not an especially interesting project. What seems to have happened is that Stenberg made waves (legitimately) complaining about LLM slop submissions, then more waves when LLM bug reports got good, and so now everyone seems to think a good measure of a vuln researcher is how many curl findings they generate. No. Curl is a straightforward CLI HTTP client.
The Linux kernel is the right reference target, if you need one.
He already scanned the codebase with Codex Security and a whole bunch of other AI tools, and fixed 200-300 bugs and CVEs. On top of that Mythos found 1 more bug and 1 more CVE is already impressive.
I'll say it. From the language of his post it doesn't seem like he was using Mythos with the correct harness / the way you're supposed to. A friend lent (?) it to him.
Yes, moving the goalposts, holding it wrong, yes that's what I believe
What I think based on the various things I've read is that Mythos is a standard advance in raw capability that was heavily trained on the process of being a security researcher. If you already had the skills to find and exploit bugs then Mythos is not a game changer, if you're an ordinary programmer it is a game changer because it's been so well tuned to wear the security researcher hat you don't have to give it much feedback at all.
> I’m not sure how to reconcile anthropic’s update ...
Why not? TFA says 23 000 findings "of all severities" and then, in the end, only 88 security advisories published.
What we'd really need is how many security advisories not related to Mythos findings have been published in the same time. If it's, say, 500 security advisories (just making a number up), wouldn't Anthropic's update in TFA and Daniel Steinberg's comments reconcile?
Like, yup, we've got a new tool to find exploits. It's a tool. It's new. We already had tools. Let's make the software world a bit more secure.
Now if you tell me that 100 security advisories have been published in that timespan and that 88 were due to Anthropic's Mythos: now I'd have to say that it's hard to reconcile Daniel Steinberg's position with TFA.
There has been a lot of cynicism around mythos, that it's just the usual public models without guardrails, etc. etc. but this:
> 1,752 of those high- or critical-rated vulnerabilities have now been carefully assessed by one of six independent security research firms, or in a small number of cases by ourselves. Of these, 90.6% (1,587) have proved to be valid true positives, and 62.4% (1,094) were confirmed as either high- or critical-severity.
for anybody who has applied opus, codex or oss models for vuln scanning - the true positive rate and discovery volume are a clear step change[0]. The ~50 partners in Glasswing have largely all previously run harnesses with other models and many of them have come out and said - essentially - "ye, wow"
Question now is what a second and third phases of access looks like - deciding which class of systems to secure. Routers, firewalls, SaaS, ERP systems, factory controllers, SCADA systems, zero-trust VPN gateways, telecoms gear and networks, medical devices - there's just so much to do
This is why I believe mythos will remain private for the foreseeable future. There's such a large surface that needs to be secured and so much to triage, fix, deploy.
That may suit Anthropic as private models can't be distilled. There's also a runaway effect of model improvement from the discovery, triage and fix data. This is likely already the most potent corpus of curated offensive data ever assembled and will only get better.
I don't see how Chinese companies are given access soon, or ever. We're likely going to see a world soon of CISA mandated audits, and where to buy a mythos-proof VPN gateway or home router - you'll have to buy American[1].
> There's also a runaway effect of model improvement from the discovery, triage and fix data. This is likely already the most potent corpus of curated offensive data ever assembled and will only get better.
But that corpus of data is accessible to all competitors, American or not.
I don't believe that this can't be replicated. I'd posit that there's enough annotated data out there (CVE+patch), only increasing thanks to Mythos, that if you specifically RL for this scenario, you can improve your models performance on finding vulnerabilities without access to Mythos.
> This is why I believe mythos will remain private for the foreseeable future. There's such a large surface that needs to be secured and so much to triage, fix, deploy.
sigh I remember the GPT-2 days - when it was the first time OpenAI restricted access to the models citing "humanity is not ready for it". The model was good at writing poetry or something.
Since then, I don't remember a single model announcement from OAI/ANT that didn't use similar wording.
The so-called leak of model announcement was marketing, it being dangerous is marketing, the world not being ready for it is marketing. And yes, the ones that were given access to saying "oh wow", believe or not, is also marketing.
It's all marketing. You can get the same results from any of the top-5/10 models that are generally available already.
Mythos is Anthropic's way to sell the new idea, because the previous one has democratized.
If you're not already applying static analysis and linters to your codebase (and I know many of you aren't), ask yourself why you would bother to apply an expensive LLM tool?
Not to say these things won't catch vulnerabilities static tools cannot, I think they can, it's just we already have the capability to automatically catch a large surface area of common vulns, and have chosen not to, often for expense reasons.
If you're a team that does already apply several layers of analysis and linting, and wants to add this on top, all power to you.
> Software developers should shorten their patch cycles and make security fixes available as quickly as possible. [...]
> Network defenders should shorten their patch testing and deployment timelines.
Shortening patch cycles will only help so much. It's funny that whenever an NPM supply chain attack is published, people recommend a cooldown before installing new versions, and then when a vulnerability is discovered, everybody jumps to patch. Clearly these two strategies collide at some point.
> The critical controls laid out by organizations like the National Institute of Standards and Technology and the UK’s National Cyber Security Centre are now all the more important, since they improve security without depending on any single patch landing in time. These include steps like hardening networks’ default configurations, enforcing multi-factor authentication, and keeping comprehensive logs for detection and response.
Most of these proposed controls are not new at all, but they are often costly to implemented and harm velocity in other ways, which is why they aren't widely in place.
For example, a super effective control is filtering outgoing network traffic. Many exploits rely on loading second and maybe third stages from the Internet, and if you block outgoing requests by default, it won't work.
But, blocking outgoing requests by default is super hard, and you risk blocking security updates etc. It can kinda work for a deployed application, but for an employee workstation? Basically impossible.
I wonder if we're approaching an era where we have to go back to saying "you cannot do this, because security" much more often than we'd like.
> Shortening patch cycles will only help so much. It's funny that whenever an NPM supply chain attack is published, people recommend a cooldown before installing new versions, and then when a vulnerability is discovered, everybody jumps to patch. Clearly these two strategies collide at some point
It’s a good point. As things speed up it will be harder to tell which patches are actually urgent and need to skip the cool-off period.
I think the more robust way of doing this is to have code audits on each published release. Agents can do some of this (eg Github could offer this scanning service, and let external parties fund the scanning on trusted compute).
I think of this more as a “proof of work” problem than provable security; if I see that Mythos has run for N hours on the patch release I am considering upgrading to, then this might suffice.
The key thing here is you need a way to crowdsource the funding of scans, and make them shareable so that the cost can be shared across the community. The package owner obviously can’t control the prompt. And can Mythos-class models be hardened enough to scan hostile code?
To your point on blocking requests, there are programming models that make this easier, like capability-based programming, where code that doesn’t need internet cannot get it; this doesn’t solve things fully, but my general prediction is that adding new architectural patterns is now a lot cheaper and easier to reliably apply across a codebase, so we may see more of this too.
Right now the only codebase I care about them fixing vulnerabilities in are the 3800 repositories that got stolen from GitHub.
"Vulnerabilities in the software that makes the internet" is honestly lower priority than "The platform that the software that makes the internet uses to make releases" If buyers of those internal repos find ways to break into GitHub such that they can cut software releases, or poison github actions from a distance, then we're all in a very ugly mess.
Don't forget that in those 3800 repos is likely also npmjs.org itself.
We have been working with the consumer-grade frontier models to develop what we call "lexploits" in legaltech, and they are insanely good at finding bugs across integrated pipelines. They're also surprisingly good at mitigating them!
Security vulnerabilities are one thing, but in legal we offer up a concept of "knowledge security" which goes to protecting the fidelity of the agent's legal context. Software bugs seem much more tractable because they're managed by software engineers, as opposed to the pipeline "vulnerabilities" we're finding. We wrote a little about one vector here where legal documents aren't quite what they seem: https://tritium.legal/blog/noroboto
No doubt there are many such knowledge domains exposed today. These are more concerning because they're understaffed and managed by non-technical people for the most part. No Mythos required.
> The bottleneck in fixing bugs like these is the human capacity to triage, report, and design and deploy patches for them. Finding them in the first place has become vastly more straightforward with Mythos Preview.
This has always been the bottleneck. Automated tools love to flag vulnerabilities, but almost all are false positives. These need to be triaged and evaluated by humans.
This is okay. I’d rather close a false positive after a careful review than miss it altogether.
I don’t think it’s appropriate for calling out humans as a bottleneck. They are an essential part of the process, I’m sure Mythos will also become a catalyst in the process.
It is definitely not the case that human remediation was the bottleneck for most vulnerability eradication 10 years ago. Proving out vulnerabilities was much harder than resolving them.
Is there a reason why they appear to conflate vulnerabilities and bugs? It's not clear where they are defining their terms, eg
"After one month, most partners have each found hundreds of critical- or high-severity vulnerabilities in their software. Collectively, they’ve found more than ten thousand. Several have told us that their rate of bug-finding has increased by more than a factor of ten. For instance, Cloudflare has found 2,000 bugs (400 of which are high- or critical-severity) across their critical-path systems, with a false positive rate that Cloudflare’s team considers better than human testers." (emphasis mine)
I wholeheartedly believe it's 100% intentional of Anthropic to use "vulnerability" to describe something that ranges from "serious attack vector" to "you forgot to add this variable to the useEffect dependency array".
I had a fun day today where I had deepseek-v4-flash subagents work out patch for dirty frag for systems with AF_ALG disabled and nscd turned on, to gain root access. The original published exploit wasn't working but the patched one worked like a charm.
I am still a believer that a 100 subagents with good-enough intelligence can get same results as mythos, I am ready for this opinion to be shattered when I eventually try mythos and I believe others here must have tried mythos out too.
That's probably true, but when you're talking about 100 subagents you're talking about something that costs $100/hour to run, and Mythos takes $20k to find a vulnerability, so the question isn't "can dumber models conceivably do this?" It's, if running inference with Mythos to find an exploit costs 5000 GPU-hours per exploit, how many GPU-hours does it cost with a dumber model?
My understanding so far is that that Mythos (and any model in general) can produce candidate reasoning but you really need a system around that reasoning that is capable of producing auditable security findings.
So, success is coming not just from the model but also from the harnesses they built around it. The Cloudflare post was more detailed on that front and I wish the rest would share more about it.
The vulnerabilities found continues to impress, and make legacy media, Twitter and Youtube go nuts. But we still have no data to prove this wasn't doable with the same initiative backed by Opus 4.7, and there is no GA for Mythos access.
There is independent research out there on frontier model security capability. AI Security Institute (UK) put out their paper comparing Mythos to other frontier models in early April. They've been tracking frontier model security capability since early 2023, so it's a decent dataset. https://www.aisi.gov.uk/blog/our-evaluation-of-claude-mythos...
. Mozilla found and fixed 271 vulnerabilities in Firefox 150 while testing Mythos Preview—over ten times more than they found in Firefox 148 with Claude Opus 4.6;
The era where you could reputably believe things published by anyone on this front is over. If you want this information, you’re going to have to attempt it yourself with the Opus API. It is entirely possible that any released model access will be heavily guardrailed against hacking attempts and Mythos is just an unrailed model. It is entirely possible that Mythos is a different architecture or size. We can’t know from the outside.
There is also a pretty big risk that anyone who is not you would leak the answer to the test. We are close to n=1 epistemics here. You’re going to have to do the research yourself.
Makes me wonder if Anthropic is really having issues with allocating compute (see recent deals with xAI and SpaceX). From available benchmarks, it seems like similar results should be possible with GPT 5.5 Pro or Opus 4.7 (with specific cybersecurity trained models).
This report is far more positive with a far lower false positive rate than I was expecting based on reports from the curl team and a few others. I guess I have just been hearing about the ten percent misses. Can anyone not employed by Anthropic who has used it vouch that it is equal to general human testers and do you need xbow to make it that way.
> Mozilla found and fixed 271 vulnerabilities in Firefox 150 while testing Mythos Preview—over ten times more than they found in Firefox 148 with Claude Opus 4.6
you would likely be quite interested in the more quantitative writeup from a real research team ! it’s linked about midway in to the article - similar functionally can be reached, yes, but not always and never with fewer tokens than what mythos requires.
I don't buy it. A lot of stuff this finds is also just simply wrong, benignly reported as true, despite upper/lower layers in the code burying the possibility of a vulnerability actually being exploited.
It's a performance/security trade-off too, it always has been. Additional checks and other measures do in fact need to be performed for security purposes.
Great marketing as always, but the rose-tinted view many have seems vicariously misplaced.
Do we have a sense that projects like OpenBSD/OpenSSH, FreeBSD, ISC[1] and Apache were included in the "blessed" initial participants in Project Glasswing ?
Or is it big name tech companies, banks and fashionable languages and package managers ?
People predict that in 50 years, no human will be driving a car, and people will be shocked that we let humans drive cars manually. Coding may be the same. So many vulnerabilities in code written by very competent programmers. Manually building large, complex systems without major bugs or security vulnerabilities seems to be a nearly impossible challenge.
And to consider AI agents are still mostly entirely limited to generating code in token-heavy programming languages designed to be written, tested and debugged by humans.
I hope this will never be the case. As long as we have personal vehicles they should be personally controlled. Self driving cars is such a waste of everyone's money.
Cities should all have better public transport and out in the middle of nowhere you don't need self driving anyway. (And yes, personal cars should be entirely banned from cities)
I reckon that in 50 years the very idea of code existing will be esoteric knowledge, a bit like binary. We simply won't care to think at that level of abstraction anymore.
Musk has been predicting self driving cars next year for fifteen years. Fifty years ago, everyone was going to be flying supersonic all the time. Flying cars were just around the corner. Interplanetary travel. Everyone forgets the technology that fails.
> Next, we will work with critical partners—including US and allied governments—to expand Project Glasswing to additional partners. And in the near future, once we’ve developed the far stronger safeguards we need, we look forward to making Mythos-class models available through a general release.
I wonder how long "near future" is in Anthropic time. I think they have incentives to delay the release of Mythos as long as possible both to save compute and delay distillation by rival labs.
Regardless, what they have been doing with Glasswing is very cool. It's clear that the world has been spared from a massive security nightmare that would have happened in any alternative timeline where the model is publicly released with weak safeguards.
IMO the talk about safeguards is utter nonsense. The model will either find vulnerabilities for you or it won't. If it will then you can broadly use the findings as you see fit.
As I see it the primary issue is giving time for the ecosystem to adapt. Once models of a given level of capability have been applied to the majority of the common software in daily use it becomes reasonably safe to release such models publicly regardless of how they are used.
> For instance, Cloudflare has found 2,000 bugs (400 of which are high- or critical-severity) across their critical-path systems, with a false positive rate that Cloudflare’s team considers better than human testers.
> For example, at one of our Glasswing partner banks, Mythos Preview helped to detect and prevent a fraudulent $1.5 million wire transfer after a threat actor compromised a customer’s email account and made spoof phone calls.
For some reason I am not able to relate to the concreteness of either of these.
First half of the page was occupied with a image, not sure if it was relevant in any ways other than setting up security scare. The size of code base, number of tokens, $ involved seem to be out of scope of the update for some reason. Personally I am getting skeptical about all these optics at this point, just some money printing scheme at high level.
Claude Mythos Preview will be available to participants at $25/$125 per million input/output tokens
...
Anthropic is committing up to $100M in usage credits for Mythos Preview
Although I'd expect reduced prices for cached tokens, which is not mentioned on their website at this point in time.
It would be informative to publish not only vulnerability numbers, but also vulnerability type statistics (as available here for example: https://cvedb.github.io/years.html), such that programmers can understand which types of exploits popular systems and languages commonly allow, and thereby encourage fundamental changes to fix or transition away from them.
I was made (2 months ago) a script that finds bugs in a github repo. I tested it with claude opus 4.1 and without reasoning and it resulted with high hallucinations. e.g. : "current latest next.js version is v15. v16 doesnt shipped yet. this project fails". i added context7 mcp but hallucaniton rate decreased only a small bit. if anyone wants to test it with other models, here is the link:
Does anyone know how we can get access to this? We're a financial institution, pretty well known locally but not internationally. Can we request access directly or maybe via bedrock/azure?
I have the feeling posts like that should be 1/4 the size, at max. At this point I don't care if it is AI-slop or human-slop: they are surprisingly alike. Information must be more dense, each sentence must carry some truth.
The math doesn't add up. They say they found more than 20k vulnerabilities, then it decreases to 1700 high or critical, then this number becomes 175 (when Claude didn't reassess the CVE severity) and over 500 later on. Then they say they confirmed 800 vulnerabilities... what happened to the 20k figure?
Plus, they also mention they check if fixes are available for the bugs they found. What are the chances they are re-reporting old bugs just to inflate their numbers? Bugs that were already fixed?
And how can we be sure their reassessment is not artifically increasing the severity of the CVEs found just to create FUD and sell their product?
Code contains deviations from assumed behaviour, and some behaviours might manifest themselves as failures. Some failures might be exploitable by attackers.
I worry that cybersecurity as target is all fine and good, but it’s looking for your keys under the streetlight. We are all familiar with computers. The problem is likely to be humans, especially in automated programmatic manipulation. The risk is that the next level of AI is going to make Fox News and other mass manipulation efforts look like kindergarten.
> that's just thousands of vulnerabilities being discovered by our trillion parameter model
> thousands of vulnerabilities and trillions of parameters?! At current energy prices, in this economic climate, isolated entirely within your datacenter?
> After one month, most partners have each found hundreds of critical- or high-severity vulnerabilities in their software.
And at the moment we have reports from like around 5(?) companies. Btw, Palo Alto Networks has found only 26 vulnerabilities [1]. I'm interested what those partners are and why they have such big amount of vulnerabilities.
> For instance, Cloudflare has found 2,000 bugs (400 of which are high- or critical-severity) across their critical-path systems, with a false positive rate that Cloudflare’s team considers better than human testers.
Yet decided not to share that number. I wonder why.
> Mozilla found and fixed 271 vulnerabilities in Firefox 150 while testing Mythos Preview—over ten times more than they found in Firefox 148 with Claude Opus 4.6;
Mozilla tested Opus 4.6 in a very limited setting (i.e. without proper harness and integration into their workflow; likely without large-scale codebase scanning). It's an incorrect comparison.
> The latest Palo Alto Networks release included over five times as many patches as usual.
Yeah, it's better to say "five times as many..." rather than "26 bugs". Btw, they also used GPT-5.5 and Opus 4.7, so the contribution from Mythos there is unclear.
> Microsoft has reported that the number of new patches they’ll release will “continue trending larger for some time.” And Oracle is finding and fixing vulnerabilities across its products and cloud multiple times faster than before.
Both Oracle and Microsoft are talking about "AI and cybersecurity" in general, not about Mythos.
> For the last few months, Anthropic has used Mythos Preview to scan more than 1,000 open-source projects, which collectively underpin much of the internet—and much of our own infrastructure.
> So far, Mythos Preview has found what it estimates are 6,202 high- or critical-severity vulnerabilities in these projects (out of 23,019 in total, including those it estimates as medium- or low-severity).
So, ~6 high- and critical- severity bugs per open-source project v.s. hundreds of high- and critical- severity bugs per partner projects. It looks like the math ain't mathing.
> One example of an open-source vulnerability that Mythos Preview detected was in wolfSSL, an open-source cryptography library that’s known for its security and is used by billions of devices worldwide. Mythos Preview constructed an exploit that would let an attacker forge certificates that would (for instance) allow them to host a fake website for a bank or email provider. The website would look perfectly legitimate to an end user, despite being controlled by the attacker. We’ll release our full technical analysis of this now-patched vulnerability (assigned CVE-2026-5194) in the coming weeks.
Of course, they didn't say that Mythos found only 8 bugs in wolfSSL vs 22 CVE fixed in wolfSSL 5.9.1.
Overall, it feels like yet another marketing stunt.
> And at the moment we have reports from like around 5(?) companies.
Which is not bad this early in the 90+45 day responsible disclosure window.
> Yet decided not to share that number. I wonder why.
It is bizarre to expect a company to disclose the false-positive rate of their security engineers, publicly. That does not happen.
> So, ~6 high- and critical- severity bugs per open-source project v.s. hundreds of high- and critical- severity bugs per partner projects. It looks like the math ain't mathing.
It is pretty obvious they're spending more compute on commercial partners. Why is this surprising?
> Of course, they didn't say that Mythos found only 8 bugs in wolfSSL vs 22 CVE fixed in wolfSSL 5.9.1.
WolfSSL is not the only software project in the world. Mozilla also came out with results that paint it as very effective. I don't think Mythos ever claimed to find all bugs anyways.
I wonder if it coincidentally becomes safe to release when compute capacity bought from SpaceX will provide enough headroom to let a lot more people run it.
It seems like Mythos is often (or typically?) costing $20k per vulnerability, so I don't think there will be enough compute capacity in the world any time soon to let a lot more people use it the way Glasswing is using it. That is not to say I think they are exaggerating its capabilities. That $20k is presumably the rough cost of renting the GPUs, and there are not enough GPUs in the world.
Total speculation: As the software world shakes out the many hidden vulns in their software, big AI will try to limit the access while it gets ironed out. Once the big projects/systems are reasonably patched after being vetted by SOTA models, the models will be released to the public. I don't think there's a scenario where Mythos-level or better models stay closed permanently.
> So far, Mythos Preview has found what it estimates are 6,202 high- or critical-severity vulnerabilities in these projects (out of 23,019 in total, including those it estimates as medium- or low-severity).
> 1,752 of those high- or critical-rated vulnerabilities have now been carefully assessed by one of six independent security research firms, or in a small number of cases by ourselves. Of these, 90.6% (1,587) have proved to be valid true positives, and 62.4% (1,094) were confirmed as either high- or critical-severity. That means that even if Mythos Preview finds no further vulnerabilities, at our current post-triage true-positive rates, it’s on track to have surfaced nearly 3,900 high- or critical-severity vulnerabilities in open-source code
> Since then, we and our approximately 50 partners have used Claude Mythos Preview to find more than ten thousand high- or critical-severity vulnerabilities across the most systemically important software in the world. Progress on software security used to be limited by how quickly we could find new vulnerabilities. Now it’s limited by how quickly we can verify, disclose, and patch the large numbers of vulnerabilities found by AI.
I guess they forgot to scan Visual Studio Code plugins and their endless npm dependencies.
You can get a taste of this today yourself with Codex Security. I turned it on just as an experiment and in less than a week it has now become essential to all of us. I was shocked how accurate it is, how many security issues it found in existing code, how it continually finds them as we commit, and how NO ONE is immune from making these mistakes.
I'd say it is about 90% accurate for us. Often even the "Low" findings lead us to dig and realize it is actually exploitable. Everyone makes these mistakes, from the most junior to the most senior. They are just a class of bugs after all.
I expect tools like this to be a regular part of the development lifecycle from here on. We code with AI, we review with AI, we search for vulns with AI. Even if it isn't perfect, it is easily worth the cost IMHO. Highly recommend you get something enabled for your own repos ASAP
> I expect tools like this to be a regular part of the development lifecycle from here on. We code with AI, we review with AI, we search for vulns with AI. Even if it isn't perfect, it is easily worth the cost IMHO.
So, how is that supposed to work? Claude Code generates security bugs, then Claude Security finds them, then Claude Code generate fix, spend tokens, profit?
One issue I've seen with LLM's is adding superfluous code in the name of "safety" and confidently generating a bunch of stuff that was useful in years gone by, but now handled correctly by the standard lib. I'm of the opinion that less is more when it comes to code, and find the trend this is introducing quite frustrating.
How do you avoid this pitfall?
I’ve had the same experience. The ui is a little unclear about this, because it says you have 5 scans, but 1 scan is just the continuous monitoring of the default branch of a repo.
The high impact findings have almost all been bang on for me. I was especially surprised by the high-quality documentation it produces as well as how narrow the proposed fixes are.
I’m used to codex producing quite a but more code than it needs to, but the security model proposed fixes that are frequently <10 loc, targeting exactly the correct place.
It’s really quite good. I’m assuming it’ll be pretty expensive once out of beta, but as a business I’d be jumping on this.
I would recommend you to try out the setup with gpt-5.5-cyber as the orchestrator and deepseek-v4-flash or some other fast cheap model as its workers. Getting pretty good results using this setup.
This got me thinking, so what happens in two years?
every tom, dick and harry who can type english has the tools to attack any software that isn't patched.
tools that were accessible to specialized groups, now made available to anybody with a grudge and a few dollars for tokens.
and what does anthropic and openai do? They form an inner ring to make the latest models available first to Enterprises. Enterprises will cough up the prices that anthropic and openai set, they have no choice here. e
Eventually everybody pays. This does not sound good
https://blog.chuanxilu.net/en/posts/2026/05/dual-pass-review...
This is what I did. Using a loop skill to dig problems and bugs in each step on development from design to coding to make sure the output software works properly and on purpose.
What kind of application are you developing?
Did you need to do anything special to get access to Codex Security?
I help maintain a project that is used as a dependency by a lot of security tools to handle PE files.
It’s disappointing that Anthropic and OpenAI never responded to the applications to their respective programs for open source maintainers. From my perspective it seems like their offers are primarily for the shiny well-known projects, rather than ones that get only a few million monthly installs but aren’t able to get thousands of stars due to being “hidden” as a dependency of popular tool.
"get a taste of this". The real thing is, GPT-5.5 is better than Opus 4.7, so if Anthropic doesn't release Mythos soon, other people are going to notice and switch off Claude.
It seems to me like either your architecture is fucked up or you’re using the wrong language/tooling for the type of software you are making if you’re introducing security vulnerabilities that frequently.
> I was shocked how accurate it is, how many security issues it found in existing code, how it continually finds them as we commit, and how NO ONE is immune from making these mistakes.
Dude is flexing that he's pushing unsecure code every day, that's a skill!
I’m not sure how to reconcile anthropic’s update / some of the exuberant comments here with recent feedback like the following from curl maintainer Daniel Steinberg:
“I see no evidence that this setup [Mythos] finds issues to any particular higher or more advanced degree than the other tools have done before Mythos. Maybe this model is a little bit better, but even if it is, it is not better to a degree that seems to make a significant dent in code analyzing.”
https://daniel.haxx.se/blog/2026/05/11/mythos-finds-a-curl-v...
You’re right, it’s a valid data point. But the U.K. government report is also a data point, and the Firefox report is a data point, and they suggest that it is, indeed, significantly better than current generation models. Maybe curl is significantly better hardened than most projects?
In any event, it barely matters. As Anthropic acknowledges, next level models are comings, theirs is only one of them. Current generation models are already good at things like tracing data flow through complex systems and there’s no reason to think that capability has topped out. So within a year it seems very likely we’ll have more than one commercially available model able to find vulnerabilities cheaply.
On the other hand, it seems that they’ve made much less progress on getting it to design solutions to these issues.
I think people sometimes misunderstand Daniel's point here, though it's clearer when taken in context of the rest of his article. The tools in general are getting a lot better at finding security bugs, it was unclear to Daniel based on his usage whether Mythos in particular is a huge step, but the Mythos generation of LLMs definitely are. Note though that Daniel was using Mythos somewhat indirectly. One thing I've taken away from the whole Mythos debate is that a) I suspect that Anthropic's GPU crunch meant that they felt they had to ration Mythos access anyway, so the calculus of whether they would release it generally was probably influenced by that, and b) finding bugs with Mythos or a similar model is still expensive -- a $20K or $100K Mythos run on Curl might have shown the same level of issues as other projects like Firefox, but Daniel didn't get that kind of access.
He posted a general update today on LinkedIn which I think gives the wider context:
https://www.linkedin.com/feed/update/urn:li:activity:7463481...
> Not even half-way through this hashtag#curl release cycle we are already at 11 confirmed vulnerabilities - and there are three left in the queue to assess and new reports keep arriving at a pace of more than one/day.
> 11 CVEs announced in a single release is our record from 2016 after the first-ever security audit (by Cure 53).
> This is the most intense period in hashtag#curl that I can remember ever been through.
Curl has more eyes on it, and has had more tools thrown at it, and is better tested (and developed?) than 99% of software, it's very much not the norm. I wouldn't be surprised if that has something to do with it, if there is any kind of bias there (not sure if there is, it's also possible he's just right).
Different people can have different experiences without contradiction. Maybe the curl source code was pretty clean to begin with?
Daniel has been posting for months (years?) about how much scrutiny he gets from security researchers and various automated tools. I wouldn't expect curl to be the average case for mythos.
Fortunately, this is just a press release for their new product 'Claude Security'. Just contact sales to find out more https://claude.com/product/claude-security
Curl, according to the authors own admission, is the most heavily tested and fuzzed open source library out there. So I think for him it's a different situation
It's a weird accident of fate that curl has somehow become the reference target for LLM bugfinding. Curl is not an especially interesting project. What seems to have happened is that Stenberg made waves (legitimately) complaining about LLM slop submissions, then more waves when LLM bug reports got good, and so now everyone seems to think a good measure of a vuln researcher is how many curl findings they generate. No. Curl is a straightforward CLI HTTP client.
The Linux kernel is the right reference target, if you need one.
If I said what I think, dang would tell me to read the site's guidelines.
He already scanned the codebase with Codex Security and a whole bunch of other AI tools, and fixed 200-300 bugs and CVEs. On top of that Mythos found 1 more bug and 1 more CVE is already impressive.
I'll say it. From the language of his post it doesn't seem like he was using Mythos with the correct harness / the way you're supposed to. A friend lent (?) it to him.
Yes, moving the goalposts, holding it wrong, yes that's what I believe
I believe that the real difference is the token burning to analyze entire code bases.
What I think based on the various things I've read is that Mythos is a standard advance in raw capability that was heavily trained on the process of being a security researcher. If you already had the skills to find and exploit bugs then Mythos is not a game changer, if you're an ordinary programmer it is a game changer because it's been so well tuned to wear the security researcher hat you don't have to give it much feedback at all.
> I’m not sure how to reconcile anthropic’s update ...
Why not? TFA says 23 000 findings "of all severities" and then, in the end, only 88 security advisories published.
What we'd really need is how many security advisories not related to Mythos findings have been published in the same time. If it's, say, 500 security advisories (just making a number up), wouldn't Anthropic's update in TFA and Daniel Steinberg's comments reconcile?
Like, yup, we've got a new tool to find exploits. It's a tool. It's new. We already had tools. Let's make the software world a bit more secure.
Now if you tell me that 100 security advisories have been published in that timespan and that 88 were due to Anthropic's Mythos: now I'd have to say that it's hard to reconcile Daniel Steinberg's position with TFA.
There has been a lot of cynicism around mythos, that it's just the usual public models without guardrails, etc. etc. but this:
> 1,752 of those high- or critical-rated vulnerabilities have now been carefully assessed by one of six independent security research firms, or in a small number of cases by ourselves. Of these, 90.6% (1,587) have proved to be valid true positives, and 62.4% (1,094) were confirmed as either high- or critical-severity.
for anybody who has applied opus, codex or oss models for vuln scanning - the true positive rate and discovery volume are a clear step change[0]. The ~50 partners in Glasswing have largely all previously run harnesses with other models and many of them have come out and said - essentially - "ye, wow"
Question now is what a second and third phases of access looks like - deciding which class of systems to secure. Routers, firewalls, SaaS, ERP systems, factory controllers, SCADA systems, zero-trust VPN gateways, telecoms gear and networks, medical devices - there's just so much to do
This is why I believe mythos will remain private for the foreseeable future. There's such a large surface that needs to be secured and so much to triage, fix, deploy.
That may suit Anthropic as private models can't be distilled. There's also a runaway effect of model improvement from the discovery, triage and fix data. This is likely already the most potent corpus of curated offensive data ever assembled and will only get better.
I don't see how Chinese companies are given access soon, or ever. We're likely going to see a world soon of CISA mandated audits, and where to buy a mythos-proof VPN gateway or home router - you'll have to buy American[1].
[0] vs ~30% or so in regular audit tools
[1] or allied
> There's also a runaway effect of model improvement from the discovery, triage and fix data. This is likely already the most potent corpus of curated offensive data ever assembled and will only get better.
But that corpus of data is accessible to all competitors, American or not. I don't believe that this can't be replicated. I'd posit that there's enough annotated data out there (CVE+patch), only increasing thanks to Mythos, that if you specifically RL for this scenario, you can improve your models performance on finding vulnerabilities without access to Mythos.
I don't see why they couldn't contract out to an American security firm that has access?
> This is why I believe mythos will remain private for the foreseeable future. There's such a large surface that needs to be secured and so much to triage, fix, deploy.
sigh I remember the GPT-2 days - when it was the first time OpenAI restricted access to the models citing "humanity is not ready for it". The model was good at writing poetry or something.
Since then, I don't remember a single model announcement from OAI/ANT that didn't use similar wording.
The so-called leak of model announcement was marketing, it being dangerous is marketing, the world not being ready for it is marketing. And yes, the ones that were given access to saying "oh wow", believe or not, is also marketing.
It's all marketing. You can get the same results from any of the top-5/10 models that are generally available already.
Mythos is Anthropic's way to sell the new idea, because the previous one has democratized.
> That may suit Anthropic as private models can't be distilled
They can be distilled internally… expect great things from Sonnet 4.8
If you're not already applying static analysis and linters to your codebase (and I know many of you aren't), ask yourself why you would bother to apply an expensive LLM tool?
Not to say these things won't catch vulnerabilities static tools cannot, I think they can, it's just we already have the capability to automatically catch a large surface area of common vulns, and have chosen not to, often for expense reasons.
If you're a team that does already apply several layers of analysis and linting, and wants to add this on top, all power to you.
> If you're not already applying static analysis and linters to your codebase
Because most issues are in business logic that static analyzers aren't going to catch.
Static analysis won't develop a one click exploit that works end to end for you.
I'm at a FAANG and even our static analysis tools are not great at identifying how many issues are actually reachable.
Ideally you use both. An AI model that has static analysis as part of the harness, so it can evaluate each potential finding.
Static analysis often shows many false positives. A more intelligent tool can help not to waste limited engineering time.
I quite like that the most honest answer for the majority of devs was downvoted then flagged to death.
Most people doing this now didn't use static analysis tools because they were seen as an unnecessary extra.
> Software developers should shorten their patch cycles and make security fixes available as quickly as possible. [...]
> Network defenders should shorten their patch testing and deployment timelines.
Shortening patch cycles will only help so much. It's funny that whenever an NPM supply chain attack is published, people recommend a cooldown before installing new versions, and then when a vulnerability is discovered, everybody jumps to patch. Clearly these two strategies collide at some point.
> The critical controls laid out by organizations like the National Institute of Standards and Technology and the UK’s National Cyber Security Centre are now all the more important, since they improve security without depending on any single patch landing in time. These include steps like hardening networks’ default configurations, enforcing multi-factor authentication, and keeping comprehensive logs for detection and response.
Most of these proposed controls are not new at all, but they are often costly to implemented and harm velocity in other ways, which is why they aren't widely in place.
For example, a super effective control is filtering outgoing network traffic. Many exploits rely on loading second and maybe third stages from the Internet, and if you block outgoing requests by default, it won't work.
But, blocking outgoing requests by default is super hard, and you risk blocking security updates etc. It can kinda work for a deployed application, but for an employee workstation? Basically impossible.
I wonder if we're approaching an era where we have to go back to saying "you cannot do this, because security" much more often than we'd like.
> Shortening patch cycles will only help so much. It's funny that whenever an NPM supply chain attack is published, people recommend a cooldown before installing new versions, and then when a vulnerability is discovered, everybody jumps to patch. Clearly these two strategies collide at some point
It’s a good point. As things speed up it will be harder to tell which patches are actually urgent and need to skip the cool-off period.
I think the more robust way of doing this is to have code audits on each published release. Agents can do some of this (eg Github could offer this scanning service, and let external parties fund the scanning on trusted compute).
I think of this more as a “proof of work” problem than provable security; if I see that Mythos has run for N hours on the patch release I am considering upgrading to, then this might suffice.
The key thing here is you need a way to crowdsource the funding of scans, and make them shareable so that the cost can be shared across the community. The package owner obviously can’t control the prompt. And can Mythos-class models be hardened enough to scan hostile code?
To your point on blocking requests, there are programming models that make this easier, like capability-based programming, where code that doesn’t need internet cannot get it; this doesn’t solve things fully, but my general prediction is that adding new architectural patterns is now a lot cheaper and easier to reliably apply across a codebase, so we may see more of this too.
Right now the only codebase I care about them fixing vulnerabilities in are the 3800 repositories that got stolen from GitHub.
"Vulnerabilities in the software that makes the internet" is honestly lower priority than "The platform that the software that makes the internet uses to make releases" If buyers of those internal repos find ways to break into GitHub such that they can cut software releases, or poison github actions from a distance, then we're all in a very ugly mess.
Don't forget that in those 3800 repos is likely also npmjs.org itself.
We have been working with the consumer-grade frontier models to develop what we call "lexploits" in legaltech, and they are insanely good at finding bugs across integrated pipelines. They're also surprisingly good at mitigating them!
Security vulnerabilities are one thing, but in legal we offer up a concept of "knowledge security" which goes to protecting the fidelity of the agent's legal context. Software bugs seem much more tractable because they're managed by software engineers, as opposed to the pipeline "vulnerabilities" we're finding. We wrote a little about one vector here where legal documents aren't quite what they seem: https://tritium.legal/blog/noroboto
No doubt there are many such knowledge domains exposed today. These are more concerning because they're understaffed and managed by non-technical people for the most part. No Mythos required.
>> Next, we will work with critical partners—including US and allied governments—to expand Project Glasswing to additional partners.
That means, they intend to make a load of money before a general release. It is a good strategy.
> The bottleneck in fixing bugs like these is the human capacity to triage, report, and design and deploy patches for them. Finding them in the first place has become vastly more straightforward with Mythos Preview.
This has always been the bottleneck. Automated tools love to flag vulnerabilities, but almost all are false positives. These need to be triaged and evaluated by humans. This is okay. I’d rather close a false positive after a careful review than miss it altogether.
I don’t think it’s appropriate for calling out humans as a bottleneck. They are an essential part of the process, I’m sure Mythos will also become a catalyst in the process.
It is definitely not the case that human remediation was the bottleneck for most vulnerability eradication 10 years ago. Proving out vulnerabilities was much harder than resolving them.
Is there a reason why they appear to conflate vulnerabilities and bugs? It's not clear where they are defining their terms, eg
"After one month, most partners have each found hundreds of critical- or high-severity vulnerabilities in their software. Collectively, they’ve found more than ten thousand. Several have told us that their rate of bug-finding has increased by more than a factor of ten. For instance, Cloudflare has found 2,000 bugs (400 of which are high- or critical-severity) across their critical-path systems, with a false positive rate that Cloudflare’s team considers better than human testers." (emphasis mine)
I wholeheartedly believe it's 100% intentional of Anthropic to use "vulnerability" to describe something that ranges from "serious attack vector" to "you forgot to add this variable to the useEffect dependency array".
I had a fun day today where I had deepseek-v4-flash subagents work out patch for dirty frag for systems with AF_ALG disabled and nscd turned on, to gain root access. The original published exploit wasn't working but the patched one worked like a charm.
I am still a believer that a 100 subagents with good-enough intelligence can get same results as mythos, I am ready for this opinion to be shattered when I eventually try mythos and I believe others here must have tried mythos out too.
That's probably true, but when you're talking about 100 subagents you're talking about something that costs $100/hour to run, and Mythos takes $20k to find a vulnerability, so the question isn't "can dumber models conceivably do this?" It's, if running inference with Mythos to find an exploit costs 5000 GPU-hours per exploit, how many GPU-hours does it cost with a dumber model?
So they gain access to these companies' proprietry software, right?
My understanding so far is that that Mythos (and any model in general) can produce candidate reasoning but you really need a system around that reasoning that is capable of producing auditable security findings.
So, success is coming not just from the model but also from the harnesses they built around it. The Cloudflare post was more detailed on that front and I wish the rest would share more about it.
The Cisco spec is interesting too, it pretty much describes an architecture of a harness: https://github.com/CiscoDevNet/foundry-security-spec
The vulnerabilities found continues to impress, and make legacy media, Twitter and Youtube go nuts. But we still have no data to prove this wasn't doable with the same initiative backed by Opus 4.7, and there is no GA for Mythos access.
There is independent research out there on frontier model security capability. AI Security Institute (UK) put out their paper comparing Mythos to other frontier models in early April. They've been tracking frontier model security capability since early 2023, so it's a decent dataset. https://www.aisi.gov.uk/blog/our-evaluation-of-claude-mythos...
. Mozilla found and fixed 271 vulnerabilities in Firefox 150 while testing Mythos Preview—over ten times more than they found in Firefox 148 with Claude Opus 4.6;
The era where you could reputably believe things published by anyone on this front is over. If you want this information, you’re going to have to attempt it yourself with the Opus API. It is entirely possible that any released model access will be heavily guardrailed against hacking attempts and Mythos is just an unrailed model. It is entirely possible that Mythos is a different architecture or size. We can’t know from the outside.
There is also a pretty big risk that anyone who is not you would leak the answer to the test. We are close to n=1 epistemics here. You’re going to have to do the research yourself.
Makes me wonder if Anthropic is really having issues with allocating compute (see recent deals with xAI and SpaceX). From available benchmarks, it seems like similar results should be possible with GPT 5.5 Pro or Opus 4.7 (with specific cybersecurity trained models).
This report is far more positive with a far lower false positive rate than I was expecting based on reports from the curl team and a few others. I guess I have just been hearing about the ten percent misses. Can anyone not employed by Anthropic who has used it vouch that it is equal to general human testers and do you need xbow to make it that way.
Training for Mythos finished in February, 2026 while training for Opus 4.7 finished around that same time.
If I understand correctly, Opus 4.7 was launched as nerfed Mythos with some improvements from 4.6.
Anthropic launches major bumps (like 4.6 to 4.7) every 4 - 5 months. So by all accounts, Mythos should be released by July.
The problem reduces to: How quickly can competing models surpass Opus 4.7 and start taking over Anthropic's market share?
I've seen a blog post by a security researcher saying that he was able to find the same vulnerabilities (for Firefox IIRC) with a ~30B params LLM...
So yeah, huge marketing as always.
> Mozilla found and fixed 271 vulnerabilities in Firefox 150 while testing Mythos Preview—over ten times more than they found in Firefox 148 with Claude Opus 4.6
4.6 but close.
you would likely be quite interested in the more quantitative writeup from a real research team ! it’s linked about midway in to the article - similar functionally can be reached, yes, but not always and never with fewer tokens than what mythos requires.
https://xbow.com/blog/mythos-offensive-security-xbow-evaluat...
Is this the God model that no one else can build? Unbelievable.
I don't buy it. A lot of stuff this finds is also just simply wrong, benignly reported as true, despite upper/lower layers in the code burying the possibility of a vulnerability actually being exploited. It's a performance/security trade-off too, it always has been. Additional checks and other measures do in fact need to be performed for security purposes.
Great marketing as always, but the rose-tinted view many have seems vicariously misplaced.
In the article they describe how all the vulns are actually exploitable end to end and >1000 have been independently verified as critical.
These aren't unreachable vulns.
I guess you could look at https://red.anthropic.com/2026/cvd/ to see exactly what was discovered.
Specially when this has been OAI/Anthropic's MO for years at this point.
I asked in a different thread:
Do we have a sense that projects like OpenBSD/OpenSSH, FreeBSD, ISC[1] and Apache were included in the "blessed" initial participants in Project Glasswing ?
Or is it big name tech companies, banks and fashionable languages and package managers ?
[1] Bind, DHCP
Probably? FreeBSD has had a large increase in security advisories the past couple months. More in the last two months than all of 2025 combined.
“Oi, you got a loicense to make secure software there?”
I joke but that is the world we are moving towards. I don’t think many on HN have thought through the second and third order implications.
The 'ethical' AI company creating a 2 tier access world. Who decides who is allowed to check their own codebase for vulnerabilities and who isn't?
People predict that in 50 years, no human will be driving a car, and people will be shocked that we let humans drive cars manually. Coding may be the same. So many vulnerabilities in code written by very competent programmers. Manually building large, complex systems without major bugs or security vulnerabilities seems to be a nearly impossible challenge.
And to consider AI agents are still mostly entirely limited to generating code in token-heavy programming languages designed to be written, tested and debugged by humans.
Here are two experimental exceptions:
https://github.com/vercel-labs/zerolang
https://github.com/sbhooley/ainativelang
I just wonder how many of those 1451 acknowledged findings were introduced by LLMs ...
I hope this will never be the case. As long as we have personal vehicles they should be personally controlled. Self driving cars is such a waste of everyone's money.
Cities should all have better public transport and out in the middle of nowhere you don't need self driving anyway. (And yes, personal cars should be entirely banned from cities)
I reckon that in 50 years the very idea of code existing will be esoteric knowledge, a bit like binary. We simply won't care to think at that level of abstraction anymore.
there is little evidence for this prediction.
Musk has been predicting self driving cars next year for fifteen years. Fifty years ago, everyone was going to be flying supersonic all the time. Flying cars were just around the corner. Interplanetary travel. Everyone forgets the technology that fails.
This is the MoviePass era of language models
> Next, we will work with critical partners—including US and allied governments—to expand Project Glasswing to additional partners. And in the near future, once we’ve developed the far stronger safeguards we need, we look forward to making Mythos-class models available through a general release.
I wonder how long "near future" is in Anthropic time. I think they have incentives to delay the release of Mythos as long as possible both to save compute and delay distillation by rival labs.
Regardless, what they have been doing with Glasswing is very cool. It's clear that the world has been spared from a massive security nightmare that would have happened in any alternative timeline where the model is publicly released with weak safeguards.
IMO the talk about safeguards is utter nonsense. The model will either find vulnerabilities for you or it won't. If it will then you can broadly use the findings as you see fit.
As I see it the primary issue is giving time for the ecosystem to adapt. Once models of a given level of capability have been applied to the majority of the common software in daily use it becomes reasonably safe to release such models publicly regardless of how they are used.
How much of this is RL’ing a good coding model on every CVE ever?
most it this comes from the pretrain imo. just scale + some RL = mythos
> For instance, Cloudflare has found 2,000 bugs (400 of which are high- or critical-severity) across their critical-path systems, with a false positive rate that Cloudflare’s team considers better than human testers.
> For example, at one of our Glasswing partner banks, Mythos Preview helped to detect and prevent a fraudulent $1.5 million wire transfer after a threat actor compromised a customer’s email account and made spoof phone calls.
For some reason I am not able to relate to the concreteness of either of these.
First half of the page was occupied with a image, not sure if it was relevant in any ways other than setting up security scare. The size of code base, number of tokens, $ involved seem to be out of scope of the update for some reason. Personally I am getting skeptical about all these optics at this point, just some money printing scheme at high level.
The report on findings is very interesting: 1451 acknowledged findings out of 23k candidates(~6%, not high but neither low).
But I didn't find the most important information (or maybe I missed it): how much did it cost to find 1451 security bugs?
We can at least put an upper limit on it. From https://www.anthropic.com/glasswing
Although I'd expect reduced prices for cached tokens, which is not mentioned on their website at this point in time.> Progress on software security used to be limited by how quickly we could find new vulnerabilities
And so was malicious vulnerability research.
the asymmetry stays the same though — defenders must find everything, attackers need one. LLMs accelerate both sides equally but that gap doesnt close
It would be informative to publish not only vulnerability numbers, but also vulnerability type statistics (as available here for example: https://cvedb.github.io/years.html), such that programmers can understand which types of exploits popular systems and languages commonly allow, and thereby encourage fundamental changes to fix or transition away from them.
I was made (2 months ago) a script that finds bugs in a github repo. I tested it with claude opus 4.1 and without reasoning and it resulted with high hallucinations. e.g. : "current latest next.js version is v15. v16 doesnt shipped yet. this project fails". i added context7 mcp but hallucaniton rate decreased only a small bit. if anyone wants to test it with other models, here is the link:
https://github.com/ErenayDev/instantbugs
Does anyone know how we can get access to this? We're a financial institution, pretty well known locally but not internationally. Can we request access directly or maybe via bedrock/azure?
Not from Anthropic, but OpenAI's Codex Security is. https://developers.openai.com/codex/security/setup
How is using Mythos different than existing static analysis, dynamic analysis, fuzzing, or DAST tooling?
Mythos is likely a harness around those things.
Stop with this nonsense. At the most basic level, it’s different because you just point it at a thing and it does everything else.
Aisle has hundreds of CVEs with publicly available models: https://aisle.com/wall-of-fame
I have the feeling posts like that should be 1/4 the size, at max. At this point I don't care if it is AI-slop or human-slop: they are surprisingly alike. Information must be more dense, each sentence must carry some truth.
The math doesn't add up. They say they found more than 20k vulnerabilities, then it decreases to 1700 high or critical, then this number becomes 175 (when Claude didn't reassess the CVE severity) and over 500 later on. Then they say they confirmed 800 vulnerabilities... what happened to the 20k figure?
Plus, they also mention they check if fixes are available for the bugs they found. What are the chances they are re-reporting old bugs just to inflate their numbers? Bugs that were already fixed?
And how can we be sure their reassessment is not artifically increasing the severity of the CVEs found just to create FUD and sell their product?
Code contains deviations from assumed behaviour, and some behaviours might manifest themselves as failures. Some failures might be exploitable by attackers.
Mythos couldn’t find the “tens thousand” typo in this post?
Is there a single source separating the harness from the model here? I would love to see a controlled experiment.
I wonder if Apple took part in the project
Yes, we did. I am the engineer leading Project Glasswing efforts at Apple.
I'm going to code myself up a new minivan.
You can get a taste of this today yourself with Swival /audit command and the security scanner is going to get even closer soon: https://medium.com/@swival/ai-vulnerability-scanning-needs-a...
I wonder how many minivans per second can ClaudeCode generate.
I worry that cybersecurity as target is all fine and good, but it’s looking for your keys under the streetlight. We are all familiar with computers. The problem is likely to be humans, especially in automated programmatic manipulation. The risk is that the next level of AI is going to make Fox News and other mass manipulation efforts look like kindergarten.
> good lord what is happening in there?!
> that's just thousands of vulnerabilities being discovered by our trillion parameter model
> thousands of vulnerabilities and trillions of parameters?! At current energy prices, in this economic climate, isolated entirely within your datacenter?
> yes
> may we see it?
> no
I built a missile that can blow you up.
>ya right.
Here's a demonstration of it blowing something up.
>can I have one.
No.
I believe them to some degree but this trend of posting stuff when it can’t be verified actually needs to end. I’m so tired of this bs marketing.
go
this is INSANEEE
> After one month, most partners have each found hundreds of critical- or high-severity vulnerabilities in their software.
And at the moment we have reports from like around 5(?) companies. Btw, Palo Alto Networks has found only 26 vulnerabilities [1]. I'm interested what those partners are and why they have such big amount of vulnerabilities.
> For instance, Cloudflare has found 2,000 bugs (400 of which are high- or critical-severity) across their critical-path systems, with a false positive rate that Cloudflare’s team considers better than human testers.
Yet decided not to share that number. I wonder why.
> Mozilla found and fixed 271 vulnerabilities in Firefox 150 while testing Mythos Preview—over ten times more than they found in Firefox 148 with Claude Opus 4.6;
Mozilla tested Opus 4.6 in a very limited setting (i.e. without proper harness and integration into their workflow; likely without large-scale codebase scanning). It's an incorrect comparison.
> The latest Palo Alto Networks release included over five times as many patches as usual.
Yeah, it's better to say "five times as many..." rather than "26 bugs". Btw, they also used GPT-5.5 and Opus 4.7, so the contribution from Mythos there is unclear.
> Microsoft has reported that the number of new patches they’ll release will “continue trending larger for some time.” And Oracle is finding and fixing vulnerabilities across its products and cloud multiple times faster than before.
Both Oracle and Microsoft are talking about "AI and cybersecurity" in general, not about Mythos.
> For the last few months, Anthropic has used Mythos Preview to scan more than 1,000 open-source projects, which collectively underpin much of the internet—and much of our own infrastructure. > So far, Mythos Preview has found what it estimates are 6,202 high- or critical-severity vulnerabilities in these projects (out of 23,019 in total, including those it estimates as medium- or low-severity).
So, ~6 high- and critical- severity bugs per open-source project v.s. hundreds of high- and critical- severity bugs per partner projects. It looks like the math ain't mathing.
> One example of an open-source vulnerability that Mythos Preview detected was in wolfSSL, an open-source cryptography library that’s known for its security and is used by billions of devices worldwide. Mythos Preview constructed an exploit that would let an attacker forge certificates that would (for instance) allow them to host a fake website for a bank or email provider. The website would look perfectly legitimate to an end user, despite being controlled by the attacker. We’ll release our full technical analysis of this now-patched vulnerability (assigned CVE-2026-5194) in the coming weeks.
Of course, they didn't say that Mythos found only 8 bugs in wolfSSL vs 22 CVE fixed in wolfSSL 5.9.1.
Overall, it feels like yet another marketing stunt.
[1] https://www.paloaltonetworks.com/blog/2026/05/defenders-guid...
> And at the moment we have reports from like around 5(?) companies.
Which is not bad this early in the 90+45 day responsible disclosure window.
> Yet decided not to share that number. I wonder why.
It is bizarre to expect a company to disclose the false-positive rate of their security engineers, publicly. That does not happen.
> So, ~6 high- and critical- severity bugs per open-source project v.s. hundreds of high- and critical- severity bugs per partner projects. It looks like the math ain't mathing.
It is pretty obvious they're spending more compute on commercial partners. Why is this surprising?
> Of course, they didn't say that Mythos found only 8 bugs in wolfSSL vs 22 CVE fixed in wolfSSL 5.9.1.
WolfSSL is not the only software project in the world. Mozilla also came out with results that paint it as very effective. I don't think Mythos ever claimed to find all bugs anyways.
Benefit of AI: it works fast
Drawback of AI: it works fast
BOOO RELEASE THE MODEL ALREADY GAWD
after IPO
I wonder if it coincidentally becomes safe to release when compute capacity bought from SpaceX will provide enough headroom to let a lot more people run it.
It seems like Mythos is often (or typically?) costing $20k per vulnerability, so I don't think there will be enough compute capacity in the world any time soon to let a lot more people use it the way Glasswing is using it. That is not to say I think they are exaggerating its capabilities. That $20k is presumably the rough cost of renting the GPUs, and there are not enough GPUs in the world.
"available to qualifying customers’ security teams on request." Seems they're already expanding access.
Total speculation: As the software world shakes out the many hidden vulns in their software, big AI will try to limit the access while it gets ironed out. Once the big projects/systems are reasonably patched after being vetted by SOTA models, the models will be released to the public. I don't think there's a scenario where Mythos-level or better models stay closed permanently.
stop noticing things, chud.
[edit: TFA addresses this, though I still find crazy 90% accuracy overall vs 20% accuracy for curl]
Is this suspected vulns or actual vulns? If I recall correctly, it produced 5 for curl but only 1 was legit
> So far, Mythos Preview has found what it estimates are 6,202 high- or critical-severity vulnerabilities in these projects (out of 23,019 in total, including those it estimates as medium- or low-severity).
> 1,752 of those high- or critical-rated vulnerabilities have now been carefully assessed by one of six independent security research firms, or in a small number of cases by ourselves. Of these, 90.6% (1,587) have proved to be valid true positives, and 62.4% (1,094) were confirmed as either high- or critical-severity. That means that even if Mythos Preview finds no further vulnerabilities, at our current post-triage true-positive rates, it’s on track to have surfaced nearly 3,900 high- or critical-severity vulnerabilities in open-source code
Did you RTFA?
I don't know why you're getting downvoted. This is exactly what was reported by curl's creator under the section "Five findings became one": https://daniel.haxx.se/blog/2026/05/11/mythos-finds-a-curl-v...
This is marketing. So probably suspected. Or somewhere in between.
What percentage of these are variations of the good old fashioned buffer overflow that would be impossible with Rust?
> Since then, we and our approximately 50 partners have used Claude Mythos Preview to find more than ten thousand high- or critical-severity vulnerabilities across the most systemically important software in the world. Progress on software security used to be limited by how quickly we could find new vulnerabilities. Now it’s limited by how quickly we can verify, disclose, and patch the large numbers of vulnerabilities found by AI.
I guess they forgot to scan Visual Studio Code plugins and their endless npm dependencies.
I mean that's really a different issue.