The corpus: 26 apps, three cohorts
The corpus is 26 real applications built with AI coding tools, audited between June and July 2026: 21 built by other people and 5 of my own production apps.
The 21 third-party apps split into two groups. The first 11, a pilot plus a 10-repo corpus, were public vibe-coded apps audited exhaustively across all 12 readiness pillars; that is where the pattern library was built. The next 10 were held-out validation: disjoint third-party apps the engine had never seen, audited blind to test the method. The last 5 are my own production apps, run through the same audit, because I wanted the method pointed at me before I published numbers about anyone else.
Across all 26, scores ranged from 29 to 81 out of 100 (mean 52.1, median 51). 22 apps came out red, 4 amber, 0 green. The 22 red apps are exactly the 22 with at least one confirmed-critical finding; the four ambers had none. The audits produced 958 confirmed findings across the 21 third-party apps, roughly 46 per app, and my own 5 added 11 criticals and 100 mediums. A distilled ledger of the 420 notable findings is what most published pattern counts are computed from.
How an audit runs
The audit engine is automated; the findings are human-verified. The engine reads the application's code across 12 readiness pillars and produces candidate findings. I then verify each one against the code, not pattern-matched: every finding is pinned to a file and line, with commit hashes for git-history findings, and the mechanism spelled out. Nothing is reported on vibes.
A finding only counts once it is confirmed. Confirmed-critical means an exploitable ship-stopper. Confirmed-medium means a real defect with a bounded blast radius. Below those sit smells and hygiene items, tracked but never counted as criticals. "Confirmed" means adversarially checked.
Pillars that do not apply to an app (no payments, no AI surface, no server) are excluded from scoring rather than zeroed. Bands are a risk posture, not a certificate: any confirmed critical forces a red band regardless of score, while the score itself measures how much remediation is left. A tidy app with one critical can score 76 and still be red, its fix a one-line version bump; a 59 amber app has no landmines but 21 things to finish. Both readings are intentional.
The verification protocol
Every candidate finding carries an explicit refutation check: before I try to confirm it, I try to disprove it. Candidates that could not be proven were dropped from the record entirely. The discipline runs in both directions. In 3 of the 21 third-party apps, dependency scanners flagged 33 to 44 known CVEs; I traced every one through the code and exactly zero were reachable. Those three reports say so in writing, and the apps were reported clean on that axis.
The held-out validation run is why "0 false positives" is a measurable claim rather than a slogan. The 10 validation apps were never used to build the pattern library, so the engine's numbers on them are unbiased. Audited blind, every finding the engine reported survived human verification: zero false positives, recall 1.0 against human-verified ground truth, and correct not-applicable gating on all 10.
The deep-audit corpus was also re-checked after the fact: 36 independent re-verification runs across the 12 pillars found zero regressions and zero new false positives.
How the stats are counted
Every published stat keeps its honest denominator. "Of 26" includes my own five apps. "Of 21" means the third-party apps only. "Of 14" means the third-party apps with an AI or LLM surface, because a denial-of-wallet stat is meaningless for an app with no AI endpoint. A cohort stat is never rounded up to "of 26" to sound bigger.
I write "9 of 21", never "43%". At this sample size a percentage borrows precision the data does not have, and it hides the denominator a reader needs to judge the claim. Counts with denominators are checkable.
Pattern counts computed from the 420-finding distilled ledger are lower bounds, since the ledger keeps the notable findings rather than all 958; where that applies, the published stat says "at least". Audited apps are referred to only by an anonymized label (Repo 01 through Repo 26) and a generic category such as "food-delivery app". No names, no identifying details, nothing that lets a reader trace a finding back to a specific team.
Honest limitations
26 apps is a real corpus and a small sample. It supports counts and mechanisms. It cannot support population claims about all AI-built apps, so I do not make them.
The sample is not random. The third-party apps are public vibe-coded projects, plus my own five, so there is selection bias toward builders who shipped in public. The window is June and July 2026: AI coding tools and their default output change quickly, and these numbers are a snapshot of that window, refreshed only as new audits enter the corpus.
Some claims the corpus cannot make yet, so the blog does not make them either. There are no per-platform percentages: the corpus spans several AI builders, and no per-builder cohort is large enough to stand alone, so the stats read as mechanism rates in AI-generated apps rather than "X% of apps from tool Y". Averages on thin pillars are avoided too; the revenue-and-billing pillar average rests on 3 apps, so I never cite it. And one internal caveat belongs in public: the recall 1.0 figure is confirmed-findings recall against curated ground truth, not recall against deliberately seeded defects.
Corrections and updates
Posts that cite corpus numbers link their first stat to this page, so every claim has one canonical source. When a fact changes (the corpus grows, a finding is reclassified, a cited number stops holding up) the post's substance is updated and its updated date moves to the day the facts did. Cosmetic date bumps never happen: a date on this site only moves when the content underneath it moved.
If a number here or in a post looks wrong to you, email bilal@axonbuild.com and I will check it against the ledger. If it is wrong, it gets corrected in the post, in public.