Agentic code review, stabilized
Approve too easily and defects ship. Review too hard and the loop never ends. Fredrik's severity-gated design finds the narrow middle without a crude max-iterations cutoff — this page shows how it works, where it can still spin, and four upgrades that make convergence a property instead of a hope.
Part 1 · As designed
Three sequential roles, three sequential reviewers, and one termination rule. Plans stay in context, never in the repo. The commit hook checks form, not truth — attribution belongs to reviewers who sign verdicts.
"The work is wrong, or carries a severe flaw. An incremental fix is unlikely to reach the right answer." Graded by wrongness, not fix size — a one-line behaviour change can't hide here as a nit.
"The outcome is right, the execution is not." Usually a standards violation. Needs rework and another look.
"Fixable without another look." Recorded, fixed, no restart. The pressure valve that keeps nits from fueling the loop.
Part 2 · The stress test
The termination proof rests entirely on reviewer calibration. Three forces can push the loop back out of the narrow middle — the doc names severity inflation as a risk and leaves oscillation unaddressed.
A reviewer that can always surface a MODERATE recreates the forever-loop with extra steps. Zero is only reachable if blocking findings are scarce by construction — nothing in the base design makes them scarce.
The main loop fuel. A confident, articulate, incorrect MODERATE costs a full rework round and a full re-review — and fresh-context reviewers regenerate them independently, round after round.
The fix for the Standards reviewer's finding trips the Annotations reviewer, whose fix trips Standards again. A-fixes-break-B cycles can hold blocking count constant forever while everyone works hard.
Part 3 · The upgrades
None of these is an iteration cap. Each one removes a specific source of non-termination while keeping the severity system intact — battle-tested in production agent fleets.
A MAJOR or MODERATE must carry a concrete failure scenario — an input/state that produces a wrong outcome, or the exact clause violated. No scenario, it demotes to MINOR.
Why it terminates: vague unease can no longer block. Blockers become scarce by construction, so zero becomes reachable. Kills failure mode 01.
Before a finding blocks, a fresh-context skeptic is prompted to kill it. Survivors block; refuted findings are logged with the refutation.
Why it terminates: plausible-but-wrong findings are filtered before they cost a rework round. Kills failure mode 02, cheaply.
Once a reviewer passes the work, it may only raise new blockers on lines touched since its verdict. Everything else it already signed.
Why it terminates: the reviewable surface shrinks monotonically. A-fixes-break-B cycles lose their fuel. Kills failure mode 03.
If the blocking count fails to strictly decrease for two consecutive rounds, stop iterating — escalate to replan or human. Rate-based, not count-based.
Why it terminates: it never fires on a healthy loop (those strictly decrease), but bounds every unhealthy one. The safety net max-iters pretends to be.
Part 4 · Paste-ready
The upgrades as contract language you can drop straight into CONTRIBUTING.md. Each block is the reviewer- or maintainer-facing wording that implements the rule — copy, paste, adjust the voice. This is the actual lever for each idea; the sliders in the Loop Lab below are just its shadow.
A MAJOR or MODERATE verdict must state a failure scenario: the input or state that produces the wrong outcome, or the exact standard clause violated. A finding that cannot name one is recorded as MINOR and does not block.
Goes in the reviewer instructions. The commit hook can check for the scenario's form, exactly like the existing attestation checks.
Before rework begins, each blocking finding is handed to a fresh context with only the diff and the claim: "Attempt to refute this finding. If you cannot reproduce the failure or locate the violated clause, mark it refuted." Refuted findings are logged with the refuter's name, under the same attribution rules as verdicts.
A new stage between review and rework. Costs one cheap read per blocker; saves a full rework round per false positive.
You previously passed revision N. New blocking findings are in scope only on lines changed since your verdict. Re-raising a finding on an unchanged line requires naming what your earlier pass missed.
Goes in the re-review prompt. Keeps fresh context intact — it narrows scope, not memory.
The maintainer records the blocking count each round. If it fails to strictly decrease for two consecutive rounds, stop iterating and escalate with the round log attached.
Goes in the maintainer loop. Bounded without max-iters; never fires on a healthy run.
Part 5 · See the dynamics
A seeded Monte-Carlo simulation of the review loop — 300 trials per run. Set how hostile the environment is, flip upgrades on and off, and watch the open-blocker curve bend. Baseline (no upgrades) always runs alongside your configuration for comparison.
| Round | Baseline | Your config |
|---|
Part 6 · The receipts
Commit 2c37e1c of annotated-tree — the --githook-guide feature, 5 files, +112/−17 — staged in a worktree exactly as a maintainer leaves work. Judged by fresh-context agents handed the repo's own review-prompt.md gate blocks verbatim, against its own repo-standards.md, with the real binary running the machine gates. U1, U2, and U4 layered on top.
"…enforces the first-line annotation format (strict-check), and teaches it…"
"maps a workspace an agent can route from without opening files, and keeps it honest"
The MAJOR flagged a real charter/product drift that the repo's own history later resolved — via the other of the two remedies the finding named. No causal claim; the drift was real, and it got fixed.
Follow-up challenge: scan repo-standards.md for a gray area and try to induce oscillation. Three candidate tensions were identified (annotation bound vs forced file-split, DbC vs Fail Fast, charter concreteness vs altitude) and the sharpest was probed live on real code.
Part 7 · Bring your own data
The sim shows dynamics; this shows your repo. Verdicts already carry names, so the loop emits everything needed. Paste per-round logs from a real run and get the empirical view: actual convergence, divergence rounds flagged, and a per-reviewer calibration ledger — the severity-inflation detector, running on data instead of priors.
2c37e1c of fredrikolis/annotated-tree — the --githook-guide feature, 5 files, +112/−17 — reviewed by fresh-context agents running the repo's own docs/review-prompt.md gates against docs/repo-standards.md, with U1/U2/U4 layered on. Round 1: the machine gate went red on two annotation bounds, and Gate A raised one MAJOR (the embedded guide's Gates A/C exceed the root charter — an AUTO-REJECT the adversarial refuter upheld with the standard's own clause). Neutral re-plan, rework, round 2: MAJOR + MODERATE = 0, tests 19/19. Two rounds, strictly decreasing, the stop rule never fired. "Load real run" below charts it.{
"rounds": [
{ "findings": [
{ "reviewer": "standards",
"severity": "MODERATE",
"overturned": false }
]}
]
}
severity: MAJOR | MODERATE | MINOR. Set overturned: true when the finding was later refuted, withdrawn, or shown wrong. One entry per finding, one object per round, in order.
| Reviewer | Blockers raised | Overturned | Overturn rate | Read |
|---|