Technote

Working with AI Practical

What AI code review catches, and what it cannot

Missing escaping and missing capability checks are caught reliably. What has to be executed to be known — runtime behaviour, hook ordering across plugins, whether the change matches the intent — remains a job for a human or a test.

Use AI code review for a while and you accumulate two reactions in parallel: “good catch” and “how did it miss that?”. The split is not random. It follows a fairly sharp boundary, and once you can see the boundary you know where in the process to put the tool — and you stop being disappointed by expectations it was never going to meet.

What it catches — defects visible in the code itself

AI review is reliable on defects that are self-contained within the changed code. Three come up constantly.

First, missing output escaping. A variable going straight into echo is visible without any wider context. A human on their twentieth review of the day skims past those lines; the model does not get tired.

Second, a missing capability or nonce check. A save handler with no current_user_can() or check_admin_referer() gets flagged. Again, one function is enough to judge it.

Third, obvious logic slips: an inverted condition, an initialisation that belongs outside the loop, a return value quietly discarded.

What these share is that they can be decided without executing anything and without opening another file.

What it cannot — anything you have to run

The boundary is not difficulty but execution — decidable by reading, or not

Runtime behaviour is the first blind spot. Which values actually arrive, whether they are ever empty, what the external API really returns — none of that is written in the file. The model can say “this could be null here”, which is a statement about possibility. Whether it is, in your system, requires running it.

Hook ordering bites particularly often in WordPress. When three plugins attach to the same filter at different priorities, what your callback receives and what it passes on is decided by the installed combination. A review that sees only the changed file cannot see that combination.

Matching the intent is the deepest blind spot. AI explains what code does perfectly well; it has no basis for answering whether that is what you meant to build, because the requirement lives outside the code. Immaculate code can implement the wrong feature, and only a person can make that call.

Where to put it, then

Ordering the review — AI goes first in order to spend human attention better

Putting AI review ahead of the human is the practical arrangement. With the mechanical defects already filtered, the reviewer is not spending attention writing “escaping missing on line 40” and can concentrate on intent and design. The bottleneck in review has always been human attention.

The layer after that is execution. Hook order and runtime values only reveal themselves when the code runs, so the change needs one pass somewhere close to the real environment. Automated tests are ideal; failing that, walking the affected path by hand on staging surfaces most of the blind spots above.

Finally, how you word the review request changes the output. “Review this” produces less than “the intent of this change is A, and it must not break B — review it from that angle”. Supplying the intent from outside the code partially fills one of the gaps described here.

The deployment and verification steps that follow review are covered in the From local to production series, and more on AI in development sits in the AI in practice archive. How this sequence runs on real client sites is published step by step on our process page.

More on this topic

All technotes

Working with AI Advanced

Security review for AI-written code: four recurring gaps

Generated code usually looks idiomatic. The trouble is that it can look idiomatic while omitting precisely the escaping, the prepared statement, the capability check or the path validation…

Developers 11 min read

Working with AI Advanced

A project context file, so the same mistake is not repeated

Every session starts blank, but a project's decisions and incident history must not. Collect conventions, reasoning, traps already hit and regression checks in one document, and the same…

Developers 9 min read

Working with AI Practical

Writing WordPress code with AI: context decides the result

The same question gives wildly different answers, and the variable is usually not the model but what you handed over. Supply your conventions, your version constraints and the…

Developers 10 min read

₩270,000 · Join the program