Cut model spend: fewer and smaller reviews, cached prefixes, resumed retries, capped fix rounds - #477

Summary

Cuts what CoderHelm spends on model calls. The audit of 17 metered days put the run rate at about $1,350 a month: reviews about $638, runs about $577, cache writes 48% of the bill. This PR takes the levers that don't change which model does the work: fewer reviews of the same code, smaller re-reviews, a cache layout that is read instead of re-written, retries that pick up where the earlier run stopped, a hard cap on CoderHelm's own fix rounds, lighter plan reviews, and a cost view that shows the metered total.

Model routing is unchanged on purpose. Review writing, implement at every complexity and every fix pass keep their current models, including a team's saved heavy model. No per-route overrides were added.

Monthly savings below are the audit's estimates from 17 days of data; they overlap, so they don't add up exactly.

Changes

1. No PR-reviewer pass on CoderHelm's own pushes while its run is still fixing the PR (~$100/month).

• review_scope::own_loop_holds_push: a push to CoderHelm's own PR, whose head commit is CoderHelm's, while the run that owns the PR is running or awaiting_ci, is not reviewed (review_pr::own_loop_hold). The run asks the reviewer for its verdict itself when CI is green (review_for_run), as before.
• A person's push to the same PR is reviewed as always; so is any push once the run has ended.
• review_pr::review_on_handoff: a run that completes asks for a review of its PR's head (trigger handoff), so a PR handed over without a verdict on its last head still gets one. A head already reviewed returns its verdict with no model call; a draft waits for ready-for-review; the handoff takes the per-commit claim so it and a ready-for-review event never both run.

2. Cache-friendly prompts (~$130/month).

• anthropic::SHARED_PREFIX_END: a caller can mark where its system prompt's shared prefix ends. The prompt is then sent as two blocks: the shared one with cache_control: {"type": "ephemeral", "ttl": "1h"}, the rest with the 5-minute cache, and the last tool definition with the 1-hour cache too (tools render first). The automatic tail breakpoint stays at 5 minutes, so 1-hour entries always come before 5-minute ones, as the API requires. Four breakpoints at most. The marker never reaches the model. Callers without it are unchanged.
• The coder (implement::run, which every pass of a run goes through: first implement, plan rounds, review, proof and CI fixes) marks its rules and the repository's and team's instructions as the shared prefix; the tools' sections and the pass's task come after it. The 1-hour TTL is for passes that start after a CI wait, which is often longer than five minutes.
• Server-side clearing of old tool results now triggers at 200k tokens and clears at least 100k (was 120k and 50k): each clear re-writes everything after it, and 16% of measured cache writes came from clears.
• History stays append-only on the models in use; the tool-loop request is built in one function (tool_loop_request) so the prefix is the same bytes every turn.
• 1-hour writes are already metered at their own price (cache_write_1h).

3. Retries take up the earlier run's plan and branch (~$45/month).

• run_status::retry_resumes: a person's Retry carries resume_from (the earlier run) unless the plan was what failed (the run stopped in triage or planning, or its error is about the plan). Start over still starts over.
• The worker loads that run's own plan, makes it this run's (mark_plan_run, triage saved), keeps the commits on the branch instead of resetting it, and tells the coder it is continuing. A run that kept no plan has nothing to take up and plans as usual.

4. Reviews: debounce, hunks-only re-reviews, and the unchanged-code skip (~$120 to $180/month).

• Push reviews wait 3 minutes (was 1) for the PR to go quiet; every later push's review supersedes the earlier one, so a burst gets one review.
• review_scope::reviewed_base: a push after a review that asked for changes (or left questions) is reviewed from that review's commit: only the hunks changed since, plus the earlier findings' status (the open threads, as before). On CoderHelm's own PR, earlier blockers still open stay listed in "Open review findings".
• A hunks-only re-review never approves: when it would, it queues a full review of the whole PR (full_review) and posts nothing. Ready-for-review, label, a person's ask and a run's own ask always get a full review.
• Why the unchanged-code skip fired 3 times in 17 days: it applied only after an approval, and only when the push was ahead of the approved commit. A rebase onto the base branch left the approved commit off the branch, so the PR was reviewed in full even when its own change was identical; and after a request for changes nothing was ever skipped. Now the PR's own change is compared whether the branch was merged or rebased, and the skip applies after any verdict: the earlier verdict stands with no model call when the PR's own change is the one reviewed.

5. Own-review fix rounds: hard cap of 2, with carried context (~$40/month plus the $10 to $20 tails).

• own_review::next stops after MAX_FIX_ROUNDS (2) fix rounds whatever the progress, and hands the PR to a person with the open findings listed (the run waits on a person, the PR stays a draft). The rule of two rounds in a row without progress still applies before that. The feedback path's self-review cap follows the same constant.
• Each fix round records what it was asked to fix, each lookup it made with what came back, and its own account (own_review::round_note, kept on the run, newest 12 KB). The next round's prompt carries it after the review, with "don't run a lookup again unless its answer may have changed".

8. Lighter plan review (part of the audit's ~$30 to $40/month; that estimate also counted skipping plan review on simple tickets, which this PR doesn't do).

• The team agents read a draft plan at effort low (skills::review_effort), and the planner's revision turn after their review runs at the same effort (was high). Plan review still runs for every ticket.

9. Cost view shows the metered total.

• /api/analytics: each period's cost_usd is the per-call meter's total (its DAY# items): runs, including work on runs started on earlier days, reviews, follow-ups, chat. cost_by_feature says what it was for; run_cost_usd keeps what the run records add up to, and the cost per run and per merged PR stay on it. Each day's cost_usd and tokens are metered too. cost_basis says which one is shown (runs when one repository is picked, since the meter isn't kept per repository).
• The feedback pass's run-record update added tokens with tokens_in = tokens_in + :ti, which fails on a record without the attribute and so lost all four token kinds and the cost; it uses if_not_exists like every other write.
• Dashboard: Analytics: say whether tokens per day count every model call or only runs coderhelm#316 titles the tokens chart by what it counts. selfhost/versions.json pins it.

10. Cost regression checks.

• A hunks-only re-review of a typical fix push sends under a tenth of the full review's diff (review_scope test).
• The shared prefix is the same bytes across passes (anthropic::shared_prefix_tests).

Verification

• cargo fmt --check, cargo clippy --workspace --all-targets -D warnings, cargo test --workspace: pass.
• New tests: review_scope (own pushes held only during the run's loop, a person's push reviewed; reviewed base after a change request; approval needs a full review; hunks-only budget; wiring), own_review (two-round cap, carried round notes, bounded), resume::own_round_tests, anthropic::shared_prefix_tests (identical prefix bytes across passes, 1-hour then 5-minute ordering, at most four breakpoints, clearing thresholds), run_status::retry_resumes_tests, mod::retry_resume_tests, skills::review_effort_tests, analytics::metered_tests (the $78 against $168 day).
• The recorded review and policy eval cases run in cargo test and pass. Model routing is unchanged, so the route evals were not re-run.
• Dashboard: lint, typecheck, OpenAPI check, build, and e2e/analytics.spec.ts pass locally; CI green on Hybrid: abandoned broker sign-ins expire after a day #316.