CodeVetter
Download
← CodeVetter

Verification field guide

Comparing local desktop verification with cloud PR review

Compare CodeVetter, CodeRabbit, and Greptile on where your code is processed and what each publishes as reproducible evidence, sourced from each vendor's own docs.

The short answer

CodeVetter is a local-first desktop application that verifies AI-written code by executing it on your machine. CodeRabbit and Greptile are cloud pull-request bots that analyse your diff after you push it. The difference that decides most of these evaluations is where your code is processed. This page compares the three on two axes we can verify from public evidence: where the code goes, and whether the tool publishes a reproducible benchmark you can run yourself. Every statement about another vendor is quoted from that vendor's own public documentation, with the date it was checked.

Where your code is processed

This is the one hard architectural difference rather than a feature gap. All three send code to a large language model provider. What differs is what sits between your repository and that provider.

CodeVetter. Reviews execute on your machine. From codevetter.com/privacy (last updated 2026-09-01):

"CodeVetter is a native macOS app. Reviews run on your machine. The repo you point it at, the diff being reviewed, your notes, and the review history all live in a local SQLite database in the app data directory. None of that goes to a CodeVetter-owned server — there isn't one."

The same page is equally explicit about what does leave: "When you run a review, CodeVetter sends your code + the review prompt to whichever provider (Anthropic, OpenAI, OpenRouter, your own gateway) you've picked. Their privacy policy applies." There is no CodeVetter server in the path. There is still a model provider in the path, and it is one you choose and contract with directly.

CodeRabbit. The default product is cloud. Self-hosting exists and is documented, with two qualifications worth reading before assuming it applies to you. From docs.coderabbit.ai/self-hosted/overview, checked 2026-09-05:

"Self-hosted CodeRabbit runs the CodeRabbit review agent inside your own infrastructure instead of CodeRabbit's cloud."

"The self-hosted option is available for CodeRabbit Enterprise customers with 500 or more user seats."

"Pull request orchestration and review results stay within your environment; code and prompt data leave it only to reach the LLM provider."

That last sentence is the honest version of the whole axis, and it applies to CodeVetter too: self-hosting relocates the orchestration, not the inference.

Greptile. Cloud by default, with a self-hosted deployment that the docs describe in stronger terms than CodeRabbit's. From greptile.com/docs/security/selfhost, checked 2026-09-05:

"The entirety of Greptile's service can be self-hosted in an air-gapped environment."

And from greptile.com/docs/deployment-options, checked 2026-09-05:

"Use Cloud if you want Greptile running in minutes with zero infrastructure management. Use Self-Hosted if you need data sovereignty, air-gapped environments, or custom LLM providers."

The same page documents Docker Compose for teams up to 100 developers and Kubernetes above that — so unlike CodeRabbit's 500-seat Enterprise floor, the self-hosted path is documented for small teams.

The axis, side by side

CodeVetter CodeRabbit Greptile
Default deployment Local desktop app Vendor cloud Vendor cloud
Vendor server in the path None — "there isn't one" (/privacy) Yes, unless self-hosted Yes, unless self-hosted
Self-hosted option N/A — already local Enterprise, documented at 500+ seats Documented from small-team Docker Compose upward, incl. air-gapped
Model provider Yours: Anthropic, OpenAI, OpenRouter, or your own gateway Yours, under self-host ("bring your own model") Yours: OpenAI, Anthropic, Bedrock, Azure OpenAI, Vertex AI
Code reaches an LLM provider Yes Yes Yes

All competitor rows quoted or summarised from the vendor documentation cited above, checked 2026-09-05.

What each tool publishes as evidence

The second axis is not "who publishes a benchmark." All three do. The question a reader actually needs answered is narrower: can you re-run it on your own code?

  • CodeVetter publishes a benchmark with its cases, scorer and outputs at /benchmark. It is not restated here.
  • Greptile publishes a five-tool, 50-PR, five-repository comparison at greptile.com/benchmarks with a written protocol, and says of it: "All PRs come from public, verifiable repositories, so you can inspect the sources and reproduce the runs on your own." The same page dates the work: "this evaluation was conducted in July 2025, and these tools evolve quickly, so performance may change over time." Checked 2026-09-05.
  • CodeRabbit publishes benchmark and evaluation write-ups on its blog and points at a third-party benchmark, Martian's Code Review Bench, describing Martian's "methodology and code" as "fully open source." Checked 2026-09-05.

The third-party harness is real, and it is worth being precise about what it lets you do. Martian's Code Review Bench is published in full at github.com/withmartian/code-review-benchmark under an MIT licence. Its README is unambiguous: "We open-source everything: the PRs, the golden comments, the LLM judge prompts, the evaluation pipeline, and a continuously-updated online benchmark that avoids training data leakage." Anyone can run it — "Running a tool that isn't on the leaderboard is open to anyone."

But read what running it means: "Fork the 50 benchmark PRs into a GitHub org where your tool is installed." The harness runs on Martian's public pull-request corpus, not on your repository. That is the distinction this section exists to draw. A public benchmark tells you how a tool scored on fifty pull requests that someone else chose. It does not tell you how it behaves on yours, and no vendor on this page — CodeVetter included — ships something that does.

A caution that belongs on this page more than any performance figure. Both CodeRabbit and Greptile publish a post claiming the #1 position on the same benchmark, on the same metric — F1, which is the metric Martian's leaderboard is ordered by. CodeRabbit's post reports a January–February 2026 sampling window. Greptile's is dated 30 July 2026, and reproduces the leaderboard table it is citing.

Both posts are accurate. Code Review Bench samples pull requests continuously and Martian says so: "Martian updates the leaderboard continuously as it analyzes new pull requests." Neither vendor is misrepresenting anything, and this page is not suggesting otherwise. The lesson is structural: a "#1" on a live leaderboard is a timestamp, not a property of the product. A comparison page that quoted either ranking as current would be presenting a six-month-old snapshot as a fact — which is precisely why there are no rankings or scores below.

This page therefore publishes no catch-rate, precision, recall, latency or cost figure for any of the three. That is a deliberate omission, not a gap.

When a cloud PR bot is the better choice

Plainly: often.

  • Review has to cover everyone's pull requests, not yours. A cloud bot reviews every PR in a repository with no per-developer install. CodeVetter is a desktop app someone has to run.
  • You want review where review already happens. CodeRabbit attaches to the pull request, IDE and CLI, with incremental follow-ups on new commits.
  • You need whole-repository context. Greptile indexes the codebase to review a diff against the rest of the system, which a local pre-push check on one change set does not attempt.
  • Nobody has a Mac. CodeVetter publishes an Apple silicon build only (/download); the cloud tools are platform-independent.

If your constraint is coverage across a team, a cloud PR bot is the better tool and this page is not going to argue otherwise.

Head-to-head detail

The full comparisons live on their own pages and are not duplicated here.

  • CodeVetter vs. CodeRabbit — verification evidence against automated PR review, and where the two overlap in a workflow.
  • CodeVetter vs. Greptile — local verification against repo-aware review, including how Greptile's TREX layer executes code during review, so Greptile is not a static-only reviewer.

How this comparison was built

Sources. CodeVetter claims come from /privacy and /benchmark. CodeRabbit claims come from docs.coderabbit.ai (/self-hosted/overview) and coderabbit.ai/blog (/coderabbit-tops-martian-code-review-benchmark). Greptile claims come from greptile.com/docs (/security/selfhost, /deployment-options), greptile.com/benchmarks, and greptile.com/content-library/greptile-martian-code-review-benchmark. The third-party benchmark is github.com/withmartian/code-review-benchmark. All pages checked 2026-09-05.

What we did not test. We did not run CodeRabbit or Greptile. We did not run any of the three against a shared case set. We did not verify vendor claims about their own internals beyond quoting what they publish, and we did not evaluate any self-hosted deployment. No figure on this page is a measurement we made.

What would make a real head-to-head. The same immutable cases, disclosed configuration, repeated runs, and one scorer applied to all three. Until that exists, treat every vendor comparison table — including the one above — as a map of stated capabilities, not measured performance.

How to decide in your own repository

  1. Pick five representative agent tasks and freeze their acceptance criteria.
  2. Run each product in its normal supported configuration.
  3. Preserve every finding, executed check, skipped check, and provider setting.
  4. Score task completion and regressions with repository-owned checks.
  5. Report latency, cost, false positives, and missing evidence separately.

Where to go next