Mozilla says it used Anthropic’s Mythos model, together with a custom agent harness, to uncover 271 security issues in Firefox over roughly two months. The disclosure, supported by a small set of public Bugzilla reports, highlights a workflow that pairs large language models with deterministic tooling and verification to reduce hallucinations and produce actionable test cases — but it has also reignited debates about hype, disclosure practices, and the shifting economics and risks of AI-assisted security research.
What Mozilla actually did
Mozilla’s security team ran Mythos (and, to a lesser extent, Claude Opus 4.6) against Firefox source files while wrapping the model in a bespoke harness the engineers built. Rather than simply asking an LLM to scan code and generate prose bug reports, the harness gave the model programmatic access to the same build and test tools developers use. That let Mythos iterate: identify a suspect code path, craft a test case or HTML trigger, run it in a sanitizer build, and see whether the process crashed. When the build produced a reproducible failure, the output moved on to additional verification steps before being treated as a real vulnerability.
The harness and the verification pipeline
The key engineering work was the harness: code that instructs the model, exposes read/write access to files, and calls existing Mozilla tools and fuzzers to validate hypotheses. Mozilla’s approach also used a second LLM to grade outputs from the first, creating an automated verification signal that mimics how human reviewers confirm issues. For memory-safety bugs, a clear success signal exists (the sanitizer build crashes), which makes automated verification straightforward. That deterministic signal, combined with tooling that can run and evaluate test cases at scale, is what Mozilla credits for the sharply reduced rate of false positives compared with earlier experiments that produced plausible but hallucinated bug reports.
The results and how they were categorized
Over the two-month run, Mythos and the harness produced 271 internal security findings. Mozilla says 180 were designated sec-high (vulnerabilities exploitable through normal user behavior), 80 were sec-moderate, and 11 were sec-low. Rather than filing separate CVEs for each internally discovered bug, Mozilla bundles many internal fixes into rollup patches and typically keeps full Bugzilla reports private for a period after fixes land to avoid releasing ready-made exploit information. Mozilla has published a dozen of the Bugzilla reports from this set to provide additional transparency.
Why some experts remain skeptical
Even with published test cases, critics warn of cherry-picking and PR-friendly narratives. Early AI-assisted bug scans produced many human-only triage workloads because models hallucinated details; Mozilla’s harness attempts to address that, but skeptics ask whether the publicized examples are the best-case subset. There are also broader concerns: once tools that can find vulnerabilities at scale exist, malicious actors may use similar techniques. The debate has shifted from “can models find bugs?” to “how cost-effective, reliable, and safely usable are these systems in real-world defensive and offensive workflows?”
Economics, access, and the future of tooling
Practical adoption hinges on cost and accessibility. Running frontier models like Mythos at the token volumes needed to scan large codebases can be expensive; organizations will weigh those cloud and model costs against human engineering time. Some researchers are experimenting with local LLMs or hybrid approaches that may offer more affordable long-term options. Another real question is whether Mozilla will open-source the harness or share implementation details: doing so would accelerate defensive adoption but could also help attackers if release timing and mitigations aren’t handled carefully.
What this means for defenders and attackers
Mozilla’s effort shows a path forward where AI models become productive parts of security toolchains when paired with deterministic verification and engineering around success signals. For defenders, that suggests investments in tooling that makes model outputs testable and automatable are likely to pay off. For attackers, the concern is inevitable: improved tooling lowers the barrier to finding exploitable code at scale. The net effect will depend on how broadly defensive teams can adopt similar verification infrastructures and how responsibly disclosures and tool releases are managed.
Mozilla’s motives and the open questions that remain
Mozilla frames the work as an engineering breakthrough and a public-good demonstration rather than marketing: their stated goal is to spur conversation and adoption of robust, harnessed approaches to AI-assisted vulnerability discovery. But open questions persist: how many false negatives or missed classes of bugs exist, how reproducible the results are across other projects and languages, and whether Mozilla will share the harness code or only further controlled examples. The conversation is likely to continue as other organizations try similar techniques and as attackers respond in kind.
Licensed under CC BY-ND 4.0 International
