Six curl CVEs after OpenAI and Anthropic came back with zero
TL;DR Highlight
AISLE reported 29 curl issues and 6 became CVEs, after frontier tools reported zero
Who Should Read
A security or backend developer weighing AI code-security scanners
Core Mechanics
- Frontier tools found zero in curl
- On Aug 24, 2026, Daniel Stenberg wrote that Mythos can't find more and Codex Security shows an empty list.
- Three CVEs were pending for curl at that time.
- AISLE filed 29 reports
- The next day Stenberg posted "Mythos: 0 Aisle: 29".
- Within days, 6 of the 29 were reviewed and became public CVEs.
- All six CVEs are Low severity
- They include an OpenSSL provider use-after-free, an OpenSSL pinning bypass, and native CA store connection reuse.
- They also include a secure-attribute bypass with tab, a wolfSSL CA-cache callback issue, and a domain-scoped public-suffix cookie.
- All were fixed in curl 8.22.0 and credit Stanislav Fort as reporter.
- Real production code, not a benchmark
- It was not a CTF or a benchmark with known answers.
- curl's maintainers decided whether each finding was real and merited a CVE.
Community Reactions
- Opinion that it is content marketing without technical detail
- Commenters noted there is no explanation of how it was done.
- Some said the underlying models were not disclosed and might come from Anthropic or OpenAI.
- Opinion that a model and a harness are not comparable
- The view is that comparing a base model with base model plus harness is expected to favor the latter.
- Doubts about fairness of the comparison
- Only 6 of 29 became CVEs, all Low, so one commenter wondered about a higher false-positive tuning.
- Another noted OpenAI and Anthropic had studied curl for a while and suggested testing on a 2023 release.
- Experience from a maintainer who received AISLE reports
- One commenter said the signal-to-noise was reasonably good on libnbd.
- They said the tool chained two exploits and proposed a sensible patch.
How to Apply
- Run a small AI scan comparison on your repo
- Paste a security review prompt into your coding agent for one module today.
- Count reproducible findings versus false positives.
- Judge tools by acceptance, not report count
- Run candidate tools on the same code.
- Compare maintainer-accepted findings and severity, not raw counts.
- Require repro steps for every finding
- Force the prompt to include reproduction and impact.
- Drop findings that cannot be reproduced from the review queue.
Try It
# Paste into Claude Code or ChatGPT (simplified version of the post's approach: AISLE's system is not public, so this only imitates it with a prompt)
You are a security code reviewer. Look only for vulnerabilities related to memory safety, certificate/pinning validation, cookie and header handling, and connection reuse in the code below.
{{code or file path}}
For each finding, write: 1) location, 2) the condition (config/input) that makes it vulnerable, 3) reproduction steps, 4) estimated severity (Low/Medium/High) with reasons, 5) a proposed patch. If you cannot write reproduction steps, leave it out.
Next step: pick one finding, actually reproduce it, and record your false-positive rate.Terminology
Related Papers
The Judge Can Debias Itself: Awareness and Self-Debiasing for Reliable LLM-as-a-Judge
Warning an LLM judge about bias or letting it self-critique improved 20 of 21 model–bias combinations.
90% of US working hours are spent on tasks not covered well by AI benchmarks
Mapping 29 benchmarks to job tasks shows 90% of US working hours lack good coverage
Postgres 19 delayed after AI reviewers find more bugs than humans can fix
LLM code review surfaced so many bugs that Postgres 19 slipped to November and 8-10 features were reverted
Show HN: Gemini 4 Argon plays StarCraft
LLMs wrote StarCraft bots in one hour; GPT-6 Astra scored highest.
A Plausible Explanation Is Enough to Mislead Multimodal Judges
Adding plausible explanations to fixed answers cut MLLM judges' wrong-answer rejection by 12.7–13.8 points.
Neural networks are clean
Looking at neurons directly shows most have one main job, challenging superposition.