You asked the assistant that wrote your app whether the code was alright, and it said yes. Maybe it listed three small things, fixed them, and said yes again. Now you are trying to work out whether that answer is worth anything before you spend money on a person to look.
The split is knowable before you ask. Source inspection can settle what is written and wrong, and it can establish that a required control is absent when the reviewer knows to look for it. Runtime behavior still needs runtime evidence, and that boundary does not move as models get better.
What follows is read from published vendor documentation and one company’s published production numbers, set against the 26 AI-built applications AxonBuild audited in June and July 2026. Nothing was bought and no tool was run for this piece.
An AI code review catches problems that are written down and wrong: a key in git history, a framework version with a known auth bypass, an endpoint doing privileged work with no auth check. It can also find a missing control when the review checks a known requirement against the repository. It cannot settle whether a control that exists actually runs. In 26 audited apps, the three most common failures were absences.
What an AI code review actually catches
Start with the fair yes, because it is a real yes. GitHub’s own resource page on AI code reviews, published 9 April 2025, answers the question directly: “Not only can AI perform code reviews, but it’s getting better at it every day.” The same page sets the boundary in its own words: AI code review “is best used as a complement to human review, not a replacement.”
The best public numbers come from Cloudflare, which is not selling code review to you and has no reason to undersell its own system. In 30 days of production data published in April 2026, Cloudflare’s engineering team reported 131,246 AI review runs across 48,095 merge requests in 5,169 repositories between 10 March and 9 April 2026. Median completion was 3 minutes 39 seconds. Median cost was $0.98 per review including trivial scans. The system produced 159,103 findings, about 1.2 per review, and the security reviewer alone produced 11,985 findings, of which 4% were critical severity. Figures read from the post on 16 August 2026.
That is what the yes buys: minutes, a dollar, and roughly one finding per change, at a volume no human team could match.
In the 26 apps AxonBuild audited, the findings an AI reviewer is best at were real and expensive. Three of the 21 third-party apps had a real secret permanently in git history, one of them a live AI-provider key. Nine of the 26 ran a framework version with a publicly known, reachable remote-code-execution or auth-bypass path, and the repair for most of them was a one-line version bump. Both classes sit in the text of the code, and a reader that knows what a leaked key looks like will find a leaked key.
The awkward part of that good news is what the corpus scores say about how often those failures happen. Secrets and Credentials scored best of the twelve pillars in that 21-app cohort, at 84.4 out of 100. Reliability and Correctness scored worst, at 31.4. The stereotype failure, the one an automated read is best at, is also the one AI-built apps commit least often.
And the error runs in both directions. In 3 of those 21 apps a dependency scanner listed 33 to 44 known vulnerabilities and none of them could be reached from the running application. Those three apps were written up as clean on that pillar.
Anyone who tells you automated review only misses things has not watched it over-report.
Most of what is published on this question comes from the companies selling the tools. Two are worth reading and stay unlinked here because they sell code review or security services: greptile.com/blog/ai-code-reviews-conflict and aikido.dev/blog/ai-for-code-review, the more two-sided of them and still written for a security team choosing a scanner.
What an AI code review cannot see
Sort the findings from those 26 apps by the kind of evidence that settles each one and three classes fall out. The sort is the useful part, because it tells you in advance which of your questions a read can answer.
| Kind of finding | What settles it | Example from the 26 audited apps | Can reading the code settle it? |
|---|---|---|---|
| Present and wrong | The text of the code | A live AI-provider key in git history; a framework version with a publicly known, reachable auth bypass; an endpoint doing privileged work with no auth check in front of it | Yes, and this is where an AI reviewer earns its keep |
| Present but not doing anything | Running the app | Rate-limit buckets never attached; error boundaries never mounted; a safety rulebook no running code reads; a schema whose column comment claimed values were stored encrypted over plaintext | No. Reading it says the control is there |
| Not there at all | Knowing what this kind of app needs | No tests, no error recording, nothing checking a change before it reaches production, no limit on the most expensive endpoint, no metering on a paid AI call | Yes, when the review starts from a known requirement |
A reviewer that reads code, human or machine, can establish an absence only when it already knows what should have been there for this kind of app. That is a different task from reviewing the code that is present.
Controls that exist but never run
Built-but-not-wired controls recur across the audited corpus. Rate-limit buckets defined and never attached to a route. Error boundaries written and never mounted. In one app, a safety rulebook sat in the repository with no running code reading it. Two apps stored tokens and storage keys in plaintext in the database while the schema’s own column comments claimed the values were stored encrypted. A read of that schema reports encryption at rest. The rows say otherwise.
The sharpest version in the corpus was a retail point-of-sale app whose checkout “test suite” never executed the actual checkout code. The tests were green and the checkout was untested. A test suite reporting green while customers report the opposite deserves its own page, so it gets one line here, but it is the cleanest illustration of the class: every artifact a reviewer looks for was present.
Cloudflare says the same thing from the other side, and from a party with every reason to claim otherwise. Its post states plainly that the system “isn’t a replacement for human code review, at least not yet with today’s models,” and names three weak spots: architectural awareness beyond the visible diff, cross-system impact on downstream consumers of a changed API, and subtle concurrency bugs that static analysis cannot see. All three are questions about behavior, not about text.
Error recording is the everyday case. When nothing is watching, a request that returns 200 and still failed leaves no trace at all, and the reviewer reading the code sees a handler that looks fine.
Things that were never written
Three absences dominate the corpus, and each carries its own denominator because the cohorts differ.
At least 23 of the 26 audited apps had zero working automated tests. Among the 21 third-party apps, 17 recorded errors nowhere, so when a customer hit a bug it simply disappeared. At least 17 of those 21 had nothing checking a change before it reached production, and several disabled their own type and lint checks at build time so that broken code would still compile and ship. Those counts come from the cross-repo index over all 26 audits, computed from the findings ledger rather than estimated, and the cohort is fixed at June to July 2026.
An absence has no line number. Nothing flags it, no rule matches it, no diff contains it. A reviewer notices that an app has no error recording only by arriving with a list of what an app of this kind needs and checking it against what is there. Ask a model to review your code and it reviews your code. Ask what is missing and you get a different, better answer, which is the whole argument of the prompt section below.
This is also where reading and understanding come apart over time. What 26 audits found about maintainability covers the slower version of the same problem: the code stays readable while the thing it adds up to stops being anything anybody chose.
Whether it is the right thing at all
GitHub’s tutorial for reviewing AI-generated code tells reviewers to ask a question no model can answer about its own output: “Does this code solve the right problem? Does it follow our conventions?” The same page says to “Look for hallucinated APIs, ignored constraints, or incorrect logic” and to “Watch for tests that are deleted or skipped, instead of fixed.” As of 16 August 2026 that page carries no publication date, so treat it as current-as-read.
Solving the right problem is the class that owners feel first, usually as a pattern rather than a bug. One owner building a large platform on Base44 described it this way: “We’ve had instances where fixing one issue introduced two or three new ones somewhere else.”
Nothing in that description is a code defect. Each individual fix was probably correct in the file it touched. What is failing is a judgement nobody is making about the system as a whole, and there is no line to point at. It is the same wall you hit when the AI cannot fix the bug it created, seen from the judging side rather than the repair side.
Should the AI that wrote it be the one that reads it?
A model reviewing its own output scores its own patterns as correct by definition, so its confidence stops carrying information. In one experiment the same model gave its own function 10 out of 10 and gave code the author described as the same quality 6 out of 10 once it believed another tool wrote it. Use a different model, or a separate session at minimum.
A June 2026 paper formalizes the mechanism, and a dev.to experiment shows what it looks like at one desk. The formal version is “When AI Reviews Its Own Code: Recursive Self-Training Collapse in Code LLMs” by Xinyuan Song, Zekun Cai and Liang Zhao, submitted 26 June 2026.
Read what it measured before borrowing it. The paper compares three recursive fine-tuning regimes, feeding AI-written code back as training data with no review, then with model-independent filters such as compilation and static checks, then with the model’s own signals as the filter. It studies training loops rather than one person asking a chatbot to check one file. Within that setting the authors report that the binary self-gate “enters a rubber-stamp regime where acceptance scores rise while benchmark correctness falls,” and conclude that stable training “requires exogenous verification rather than model-coupled self-review.”
Exogenous verification means a check the model does not control. That idea travels beyond the training loop even though the experiment does not.
The desk-level version is an April dev.to post, I Asked AI to Review Its Own Code. It Gave Itself 10/10, by an author writing as Harsh. The experiment was to ask a model to write a function that averaged an array, then ask the same model to review and rate it. The verdict came back “This code is clean, efficient, and well-structured. I’d give it a 10/10.” The function crashed on an empty array. Once that was pointed out, the rating went to 11/10. Then code the author described as the same quality was pasted in, attributed to a different tool. That time the model found seven issues and gave it 6/10.
Code the author described as the same quality drew seven issues and a 6 out of 10 once the model stopped believing it was the author.
Whether a fresh session is enough to fix that is genuinely unsettled. In the Hacker News thread on the question, AI code review: Should the author be the reviewer? from 1 May 2025, the commenter black3r argued that “…the reviews are typically done with a fresh context…” and concluded “…in my eyes ‘the author is not the same as the reviewer here.’” The commenter JonChesterfield replied: “I wonder how persuasive that line of reasoning is. It’s nonsense in a few dimensions but that doesn’t appear to be a blocker to accepting a claim.” Both positions are still live.
The practical rule sits below the disagreement and does not depend on settling it. Use a different model where you can, a separate session at minimum, and phrase the request so that a negative answer is allowed and cheap to give. One owner who had worked through a stack of free tools described what happens without that permission: “…they tend to generate fixes that are not there or not accurate…” A reviewer that cannot say “I do not know” will invent something rather than return nothing.
Can ChatGPT do a code review?
ChatGPT can do a code review through a documented flow, with two prerequisites many app owners lack. OpenAI’s documentation covers reviewing changes across ChatGPT and Codex clients: a /review command, a Git repository on your machine, and the GitHub CLI authenticated before pull-request context works.
The documented code review flow works like this, read from OpenAI’s page on 16 August 2026. You type /review in the composer and choose what to review: against a base branch, uncommitted changes including staged, unstaged and untracked files, a specific commit, or a custom set of review instructions. Findings come back as inline comments in the review pane, lined up with the code they refer to, and the review does not modify the working tree. You can ask for evidence behind a finding or a narrower follow-up in chat.
The prerequisites decide whether this reaches you. The doc requires a Git repository, plus GitHub CLI (gh) installed and authenticated with gh auth login before pull-request context and review comments work. If your whole application lives inside Lovable, Base44 or Replit and you have never exported it, neither is met.
You can still paste files into a chat window and ask for a read, which is free and worth doing, though that is a review of whatever you pasted rather than of the application. As of 16 August 2026, OpenAI’s code review documentation does not describe any route that reads an application hosted inside a third-party builder without a Git repository. Do not go looking for a setting that would enable it.
What is a good prompt for code review?
A good prompt for a code review names the target, names the criteria, and makes “I cannot tell from this code” an allowed answer. Three additions do most of the work: ask what is missing by category, require a file and a line per finding, and ask which findings would change if the app were running.
OpenAI’s own guidance is one sentence: “In your prompt, identify the pull request, branch, commit, files, and review criteria.” The criteria half is where an owner adds the most value, because the default criteria are whatever the model assumes.
Each rule below aims at a class the default review misses.
- 01 Ask what is missing as well as what is wrong. Name the categories out loud, because a model will not invent a category you did not mention: tests, error recording, a check before deploy, a limit on the most expensive endpoint, metering on any paid AI call
- 02 Require a file and a line for every finding, and make "I cannot tell from this code" an allowed answer. Without that permission the model fills the gap, which is exactly how a review turns into a list of fixes for problems that are not there
- 03 Ask which findings would change if the app were running. That question separates the present-and-wrong class from the present-but-not-wired class out loud, and it tells you which items you still have to check by hand
Put together, the prompt is short:
Review this repository for the owner of the app, who does not read code.
1. List what is MISSING as well as what is wrong. Check specifically for:
automated tests, error recording or alerting, anything that checks a change
before it reaches production, a rate limit on the most expensive endpoint,
and metering on any paid AI call.
2. Give a file and a line for every finding. If the code alone cannot settle a
question, answer "I cannot tell from this code" and say what would settle it.
3. At the end, list which of your findings would change if you could run the
app, and name the exact check you would run for each one.The findings from step 3 are the ones a read cannot close, and some of them you can settle yourself. The security version of this question walks through three running checks that do exactly that, so they are not repeated here.
Manual code review and AI review: what settles what
A person reading code hits the same wall on the middle class, because reading is reading. What a manual code review adds is the two classes that need knowledge from outside the file: what should have been there, and whether the thing built was the right thing.
| The question | An AI reading the code | A person reading the code | A check against the running app |
|---|---|---|---|
| Is this line wrong? | Settles it | Settles it | Not needed |
| Is this dependency version exploitable? | Settles it, and faster | Settles it | Not needed |
| Does this control actually run? | Cannot settle it | Cannot settle it | Settles it |
| Should this control exist at all? | Rarely settles it | Settles it, if they know this kind of app | Cannot settle it |
| Is this the right thing to have built? | Cannot settle it | Settles it, given the business context | Cannot settle it |
Read the middle row twice. It surprises people who assume a human reviewer is a strictly better version of the machine. Nobody reading a repository can tell you a defined rate-limit bucket is attached to nothing, short of tracing every call site by hand, and nobody reading a schema can tell you the column marked encrypted is holding plaintext. That is what a run settles, and only a run.
What a person is worth, in this specific job, is the bottom two rows. What actually happens in a code review, step by step, is a separate question with its own page. For the decision in front of you, the difference that matters is that a person arrives with a list of what an app like yours needs and a view about whether the feature was worth building, and the model arrives with your code.
If you are weighing paying a person to read an AI-built app against running another free pass, the useful comparison is which class of question each one can close, not whose accuracy is higher.
When an AI review is enough, and when it is not
Decide by consequence rather than by code. The question is what happens if something in the app is wrong and nobody notices for a month.
An AI pass on its own is proportionate when nobody’s money moves through the app, nobody else’s personal data is in it, you are the only user or the users are people you know, and losing the data would be annoying rather than serious. Internal tools, prototypes, and side projects mostly sit here. Run the review, act on the findings, spend nothing.
Get a person involved when a card gets charged, a second customer can log in and see their own data, a paid AI endpoint is reachable from the internet, or data would be gone for good if a change went wrong. Every one of those is a question about behavior under conditions a reviewer never sees, and the corpus says that is where AI-built apps fail: the invisible engineering nobody prompted for, rather than the leaked keys of the stereotype.
The middle case is the honest one. If your app has customers but nothing that would be catastrophic, run the AI pass first, use the “what is missing” prompt above, and treat the list of things it says it cannot tell from the code as the shopping list for whatever you buy next. That list is short, specific, and worth more than a general opinion about whether the app is good. It also tells you whether your next question is which AI code review tool to pick or something else entirely.
Common questions about AI code reviews
Can AI review a whole app, not just a change I made?
Yes, and a whole-app pass is more useful to an owner than a diff review is. Tools built for teams review one change against a codebase the reviewer already understands, which is the wrong shape when nobody on your side understands it, including an app somebody else built for you with AI. Point the model at the repository, ask what is missing by category, and expect a longer and vaguer answer than a diff review gives. You gain coverage and lose precision, because with more code in context a model reports more things it cannot confirm.
Can AI update its own code?
Yes, and it is worth separating that from reviewing. Every AI coding tool edits its own output on request, and most will apply a fix as soon as it names one. Judging is the part that does not work on its own output, so a model that writes a fix and then declares the fix good has told you nothing new. Ask for the change, then check the result somewhere else: a different session, a different model, or a test that fails before and passes after.
Is an AI code review the same as a security scan?
No. A scanner matches known conditions, such as a dependency version with a published vulnerability or a pattern that looks like a hard-coded credential, and reports every match. An AI review reads the code and reasons about it, including things no rule covers. Both over-report, and neither settles whether a control that exists actually runs.
What does an AI code review cost?
Almost nothing per run. Cloudflare published a median cost of $0.98 per review run, trivial scans included, across 131,246 runs between 10 March and 9 April 2026, and that is a company paying model rates directly at production volume. A review you ask for inside a chat subscription you already pay for costs nothing extra beyond your usage allowance. The expensive part of a code review has never been the reading. Prices checked 16 August 2026.
Should I use a different model to review than the one that wrote it?
Yes where you can, and at minimum a separate session. A model asked to grade its own output recognizes its own patterns as correct, which is what the dev.to experiment above shows in one person’s hands and what the June 2026 paper on recursive self-training formalizes for training loops. A different model has no stake in the choices made in the file. Whether a fresh session with the same model is enough is still openly argued.
What are the etiquette rules for code reviews?
The etiquette rules for code reviews are a team question: how quickly to respond, how to phrase a criticism, whether to approve with comments, how large a change should be before you split it. If you are the only person on your project, none of it applies to you. The question worth your attention is which of your questions a read of the code can settle, which is what the rest of this page covers.
What do I do with the findings if I cannot read code?
Sort them before you act. Ask the model to mark each finding with the file and line, the consequence in plain words, and whether it could confirm the finding from the code alone. Findings it could confirm are actionable now. Findings it could not are the list you take to a person, and they are usually the short list. Anything with no file and no line attached is a guess, and worth removing before you pay anyone to work through the rest.
Will an AI review tell me my app is ready for customers?
No, and it should not try. Readiness is a judgement about consequences under conditions a reviewer never observes: what happens at a hundred users, whether the second customer can see the first customer’s data, whether a failed payment leaves an account in a state nobody handles. A review of the text can rule out some ways of being unready and cannot rule them all out. Of the 26 apps AxonBuild audited, not one came out clean.
Need this fixed in your own app?
New clients can start once with one agreed blocker for $99. We fix it within three business days once access works, and you pay after seeing it work.
