code-gauntlet

Code Gauntlet

Adversarial multi-agent code review for Claude Code. Your PR runs a gauntlet: parallel concern-specialized reviewers find issues, then every finding must survive deterministic verification, skeptical validation, and a blind challenge before it reaches you. Formerly published as deep-review.

How it reviews

Seven discovery agents examine your changes in parallel, each through a different lens:

Agent	Model	Focus
bug-detector	Sonnet	Logic errors, edge cases, error handling, resource leaks
security-reviewer	Opus	OWASP top 10, injection, auth, SSRF, deserialization
cross-file-impact	Sonnet	How changes affect callers and dependents across the codebase
test-analyzer	Sonnet	Test coverage gaps, test quality, missing edge cases
conventions-and-intent	Sonnet	CLAUDE.md compliance, spec alignment, comment accuracy
type-design-analyzer	Sonnet	Type encapsulation and invariant design
code-simplifier	Sonnet	Simplification opportunities

Security always runs on Opus because models have complementary vulnerability-class detection profiles.

Discovery is only the entrance. Each finding then runs the gauntlet:

1. Merge — finding-ID deduplication across channels, schema validation, agent attribution
2. Verify — deterministic git-blame classification (new vs. surfaced code) and factual verification against the actual source, trusted only through a nonce + SHA + count receipt
3. Validate — an independent validator attempts to disprove each finding with its full content and codebase access, adjusting confidence either way
4. Filter — confidence and severity thresholds, prompt-injection filtering, cross-agent semantic dedup, consensus boosts, disagreement suppression, routing to the main report or improvement suggestions
5. Blind challenge — a fresh agent attempts to disprove each finding without the original reasoning or evidence; claims it cannot verify from the code are removed
6. Deliver — deterministic ranking and selection, posted verbatim

Only findings that survive every stage get delivered. Anything that fails is eliminated, downgraded, or routed to a lower tier — and every degradation (a failed agent, an untrusted verification) is recorded as an explicit gap in the report rather than silently absorbed.

Benchmark results

Development is gated by a judged benchmark (bench/): golden PRs from real open-source repos with independently established findings, scored by a pinned judge using symmetric three-bucket adjudication (golden-matched / valid-extra / noise). Published tools are re-scored under the same judge as anchors.

Release	Run	PRs	Golden recall	Noise rate	Tokens
v3.1 (current)	paired mini-subset	6	0.633	0.223	6.4M
v3.0	gate subset	15	0.695	0.180	20.9M
v3.0	holdout	10	0.741	0.209	17.8M
v2 (previous architecture)	gate subset	15	0.492	0.164	41.1M
CodeRabbit (anchor)	gate subset	15	0.627	0.566†	—
Claude CLI review (anchor)	gate subset	15	0.339	0.481†	—
claude-code review (anchor)	gate subset	15	0.271	0.542†	—

Every release ships behind its own measurement under the same pinned judge (claude-opus-4-5-20251101), so the table reads as a progression: v2 → v3.0 (the workflow rewrite, measured at its July 2026 release gate — runs subset-20260721-015119 and holdout-20260721-085348) → v3.1 (schema-fidelity hardening, light scope, and orchestrator-model pinning — measured on its paired mini-subset, run custom-20260723-102149: the 6 highest-golden-density gate PRs, pre-registered before any run). The v3.1 recall clears both release hard bars (> 0.627 CodeRabbit anchor, > 0.492 v2) with the small-N caveat stated in PR #20, and its union smoke matched the v3.0 comparator's recall at 17% lower cost. Code Gauntlet tops every anchor on golden recall while holding noise under its pre-registered 24% ceiling. The interactive report — per-PR buckets, per-dimension recall, cost, judge drift — is generated by bench/report.py; see the published benchmark report.