Coding agent verification tools, compared
We make ShipSure, so weigh our opinion accordingly. This is still the most accurate map we can draw of the category — including the cases where you shouldn't buy anything at all.
What this category is
A distinct problem appeared once coding agents got good enough to work unattended: the agent reports that it finished, and that report is generated from what it intended to do rather than from what is now in your repository. It isn’t deception. It’s self-assessment, and self-assessment contains no independent check.
The failure looks mundane and costs a weekend. A test file picks up a syntax error, the runner skips the whole suite, and the agent writes “all tests passing” because nothing printed a failure. That shape — a claim that sounds true and cannot be checked from the transcript — is what every tool below is reacting to.
The tools
| Tool | Approach | Deterministic verdict | Source stays local |
|---|---|---|---|
| ShipSure | Checks repo state against a task contract and project policy — tests, build, types, lint, diff scope, dependencies | Yes | Yes |
| Critique | Independent finish pass — reconstructs changes, tests hypotheses, returns evidence as JSON and exit codes | Yes | Not published |
| agentwatch | Observes agent actions and flags gaps between claimed behaviour and recorded activity | Yes | Not published |
| XOR | Re-runs known failing security conditions in isolated environments | Yes | Not published |
| DIY GitHub Action | Your own workflow running tests and diffing output; open-source options exist | Yes | Runs in CI |
| AI reviewers | An LLM reads the diff and comments on it (CodeRabbit, Qodo, Greptile) | Output varies run to run | Varies |
Rows are limited to what each vendor publishes about itself. “Not published” means exactly that — we didn’t find a public answer and we won’t guess at a competitor’s architecture. Check their docs before deciding.
When you need none of this
If you review every diff before merging, your agents aren’t running unattended, and CI already gates merges, then a workflow that runs your test suite catches most of this for free. Do that first, whether or not you ever pay us. Verification tools earn their place once you’ve stopped reading every diff — not before. See ShipSure vs a DIY GitHub Action for the honest version of that argument.
Where AI code reviewers fit
They solve a different problem and they’re good at it. An LLM reading a diff can tell you the logic looks wrong in a way no test would catch. What it can’t be is a gate — that kind of feedback varies between runs, which is precisely what a hard compliance gate can’t tolerate. Run both if it helps. They barely overlap.
Where ShipSure differs
No AI computes the verdict. Verdicts come from git diffs, test runs, builds, type checks, linters and policy rules. Put a model in that path and it inherits the first model’s blind spots one level up, and you lose reproducibility — the same commit has to return the same verdict every time or it can’t gate anything.
Verification runs where the code lives. Your source doesn’t leave the machine; only evidence and metadata do.
A contract that gates nothing doesn’t pass. If a task declares no meaningful gate — just a title, or nothing but inherited defaults — then “verified” is a meaningless answer, and returning it is worse than returning nothing because it manufactures confidence. ShipSure treats that as a problem with the contract.
Honest limitations
ShipSure won’t stop an agent writing bad code — it tells you before you merge instead of after. It won’t judge whether the feature is what you meant; a pass means the gates you declared held. JavaScript and TypeScript adapters are the mature ones, with other stacks running through a generic shell adapter today. And a check that can’t execute returns inconclusive, never a pass — quietly converting “I couldn’t tell” into “fine” is how people learn to ignore a verifier.
Competitor details reflect published material as of August 2026 and will drift. Corrections welcome at support@shipsure.space — we’d rather fix this page than win an argument on it.