Repository navigation
Admin teams: count runs and PRs reviewed once each, from where they are written - #474
Merged
Merged
Conversation
…re written The admin Teams page read each team's runs and reviews from its analytics items. Reviews were never really counted there: the `reviews` counter was added only when a review job's model usage reached the worker, counted answers to questions and jobs that stopped before posting, skipped carried approvals, and shows 0 for every team. Plugin runs were listed with the team's runs but never added to `total_runs`, and a redelivered run message counted its run twice. common::team_counts now adds each count exactly once: one transaction writes a once-only marker (`COUNTED#<team>` partition of the analytics table) and the ADD on the month and ALL_TIME items. - Runs: agent runs when their record is created, plugin runs on any write of their record (and `completed` / `failed` once they end). - PRs reviewed (`prs_reviewed`): every stored review (store_review_record) counts its pull request once per month and once in all, when it was posted and isn't an answer to a question: the rule the weekly email and the analytics page now share (counts_as_review). - The admin page reads `total_runs` and `prs_reviewed` (Activity), and the dashboard labels say "in all" and "this month" (pinned here). - scripts/team_counts_backfill.py adds the PRs reviewed and plugin runs from before the deploy through the same markers. Dry run by default; not run.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The admin Teams page showed "46 runs · 0 reviews" for an organisation with many reviewed pull requests, and 0 reviews for every team. It read
reviewsfrom the analytics items, and that counter was never a count of reviews: it was added only when a review job's model usage reached the worker (inrecord_review_usage, next to the cost write), counted answers to questions and jobs that stopped before posting, and skipped carried approvals. Plugin runs were listed with the team's runs but never added tototal_runs, and a redelivered run message counted its run twice.Each count is now added exactly once, where its record is written, and the page reads those counts. A "review" on the page is a pull request the reviewer posted a review on, counted once in all and once per month: the rule the weekly email counts by ("PRs reviewed"). Paired with CoderHelm/coderhelm#315 (labels "in all" and "this month"), pinned here.
Changes
common::team_counts(new):add_oncewrites a once-only marker (COUNTED#<team>partition of the analytics table, never seen by period reads) and theADDon the month andALL_TIMEitems in one transaction; a marker already there cancels it, so replays and the backfill never add twice.counts_as_reviewis the one rule (posted, not an answer to a question), now also used by the weekly email and the analytics page.total_runs): agent runs whencreate_run_recordwrites the record (once per run id); plugin runs on any write of their record throughRunsTable(gateway and worker), pluscompleted/failedonce they end.prs_reviewed):store_review_record, which every stored review goes through (full reviews, replies, carried approvals), counts the PR. The oldADD reviewsand its calls on paths that never posted are gone;record_review_usageonly stores the cost on the review record.routes/admin.rs):Activity::from_itemsreadstotal_runsandprs_reviewedfromALL_TIMEand this month's item: twoGetItems per team, as before. API field names are unchanged (reviews_total/reviews_monthnow mean PRs reviewed).scripts/team_counts_backfill.py(not run): adds the PRs reviewed (fromRSTAT#/REVIEW#records) and plugin runs from before the deploy through the same markers, so whatever the services already counted is skipped. Dry run by default;--applywrites.selfhost/versions.json: dashboard pinned to dbfbd17 (Admin teams: runs and PRs reviewed, in all and this month coderhelm#315).admin::tests::the_counts_match_the_team_s_runs_and_reviews(a fixture team's runs and reviews, including a redelivered run, a plugin run written three times, a question, an unposted re-review and the same PR in two months, go through the counting code; the page's counts equal what the records say);team_counts::tests::every_run_kind_and_every_stored_review_is_counted(fails if a runs-table writer, the plugin run log or a stored review stops counting, or anything else adds to these counters; checked by removing each call); once-only and per-month tests for runs, plugin runs and PRs.Verification
cargo fmt --all --check,cargo clippy -p common -p gateway -p worker --all-targets -D warnings: clean.cargo test -p common -p gateway -p worker: 1,418 + 724 + 1,192 passed, 0 failed.--helpruns. Not run against any table.python3 scripts/team_counts_backfill.py --stage prod(dry run), then with--apply.