Skip to content

skills(checklist-test): one strict qa-run title shape, and a parser that holds it - #10941

Merged
huangyiirene merged 3 commits into
mainfrom
claude/issue-9617-qa-run-title-contract
Aug 22, 2026
Merged

huangyiirene merged 3 commits into
mainfrom
claude/issue-9617-qa-run-title-contract

Conversation

@huangyiirene

@huangyiirene huangyiirene commented Aug 21, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #9617

Implements the maintainer ruling recorded on the card 2026-08-20 — option 2, tighten the
qa-run record title to ONE strict shape, new records only. Provenance quoted verbatim and
untranslated: 「其他接受你的建议。」

Files

  • .claude/skills/checklist-test/SKILL.md — the binding rule and a pointer (governed surface)
  • docs/qa/platform-checklist/RUNNER.md — the field-by-field grammar
  • scripts/qa/qa-rollup.mjs — the strict parser and its embedded --self-test

The contract

QA run · SELECTOR (JUDGED/TOTAL) · FRAMEWORK-SHA8 · DATE · COUNTS

Exactly five middle-dot fields. (JUDGED/TOTAL) is mandatory and bare-digits-only, with
judged not exceeding total. The sha is exactly 8 lowercase hex. The date is a real calendar
date. COUNTS is /-separated N VERDICT segments — uppercase, in the report order
PASS / PARTIAL / FAIL / BLOCKED / NOT-RUN, each bucket at most once. An omitted bucket still
means "not declared", never zero, so an author who means zero writes 0 FAIL.

Retired, all of them: (FULL area), (N items), (N of M items), (N/M items consulted),
omitting the parenthetical entirely, and the trailing (11 not-run) spelling of NOT-RUN.
(FULL area) is why the ruling exists — it declares no total, and a record wearing it judged
13 of its area's 33 items while reading as complete coverage.

Where the contract is written

The grammar lives in docs/qa/platform-checklist/RUNNER.md, as a new
"The record title — one canonical shape" subsection under its existing "## Run records"
section: the shape, a field-by-field table, the declared-vs-zero rule, the retired phrasings
with the reason each died, and the new-records-only note.

SKILL.md keeps the binding rule — one shape, strict parsing, judged/total mandatory — plus a
pointer. That split is the skill line ratchet's own stated remedy ("principles in SKILL.md,
on-demand detail elsewhere"): the file was at its ceiling of 232 with zero headroom, so
documenting the contract inside it overflowed by 9 lines. It is back to exactly 232, with no
ceiling exception and nothing maintainer-authored deleted.

The parser is strict, not tolerant

scripts/qa/qa-rollup.mjs (landed by PR #9616) matches that shape and nothing else. The
multi-phrasing branches are gone, and so are judgedCell's three inferred denominators — they
existed only because the retired phrasings declared no total, and inferring a denominator the
record never claimed is the confident nonsense the contract removes.

Every rejection names the field that failed, because that reason is what the "Not parsed"
section prints and that section is the enforcement surface.

Existing records are not migrated, and the enforcement surface stays legible

The ruling is new-records-only. Measured against the live corpus on 2026-08-21 (34 records,
open and closed) the strict parser gives 3 parsed / 31 predating the contract / 0 deviating.
An undifferentiated "Not parsed" list would therefore be 31 rows deep on day one, and the first
genuine deviation would be invisible inside it — defeating the surface the ruling relies on.

So the render partitions the unparsed set by a CANONICAL_FROM constant, using the GitHub API's
created_at. This is not tolerant multi-shape parsing: nothing is extracted from a legacy
title, no legacy shape is recognised, and every record is equally unparsed. Only the explanation
differs — records predating the contract are labelled expected and not defects; records written
after it are the signal. Every record is still printed, never dropped.

Verification

  • node scripts/qa/qa-rollup.mjs --self-test — green, 78 assertions over 5 canonical shapes and
    16 retired ones. Both halves are load-bearing: a parser re-widened to accept a retired phrasing
    passes every happy-path assertion, so the rejections are the half that actually holds the ruling.
  • Ablation — widening the parenthetical regex by one character class ([^)]*) to re-accept
    (6/14 items consulted) turns the self-test red with exactly 1 failure, naming that fixture
    as having parsed. Mutation confirmed on disk before the run (injected form present 1, strict form
    present 0, plus the diff of the line); the restore leg ran under an EXIT INT TERM trap and was
    confirmed absent afterwards, tree clean, self-test green again.
  • Gate families re-derived by node scripts/pm/dispatch-gates.mjs (no path args) on a clean tree
    at e249bff62 — it read the 3-path change set itself. 10 named, all 10 green, plus
    check-platform-checklist run as an adjacent extra, also green. check:pm-skill-ratchet now
    reports: "✓ check-skill-line-ratchet: .claude/skills/checklist-test/SKILL.md is 232 lines
    (ceiling 232; headroom 0)."

Governed surface

.claude/skills/** — draft-only, human merge (Prime Directive #14). Not queued, not armed, never
flipped ready. RUNNER.md is not a governed face; that edit is confined to the grammar section.
No changeset: this PR publishes nothing, matching 12 of 12 recent .claude/skills/** merges, so it
carries skip-changeset.


Generated by Claude Code

claude added 2 commits August 21, 2026 19:35
…hat holds it

Maintainer ruling 2026-08-20 (option 2 on the drift card): tighten the
run-record title convention to exactly ONE shape, new records only.

  QA run · <selector> (<judged>/<total>) · <sha8> · <YYYY-MM-DD> · <counts>

SKILL.md now documents that as the contract rather than a shape no record
wrote. `(<judged>/<total>)` is mandatory and bare-digits-only, which retires
all five phrasings the corpus grew -- `(FULL area)`, `(77 items)`,
`(5 of 10 items)`, `(6/14 items consulted)`, and omitting the parenthetical.
`(FULL area)` is why the ruling exists: it declares no total, and a record
wearing it judged 13 of its area's 33 items while reading as full coverage.
NOT-RUN keeps one spelling -- a `<n> NOT-RUN` counts segment; the trailing
`(11 not-run)` parenthetical is retired.

qa-rollup.mjs is strict, not tolerant: five fields, mandatory bare
judged/total with judged <= total, exactly 8 lowercase hex, a real calendar
date, and counts as ` / `-separated `<n> <VERDICT>` segments, uppercase, in
report order, each bucket at most once. The multi-phrasing branches are gone,
and so are judgedCell's three inferred denominators -- they existed only
because the retired phrasings declared no total, and inferring a denominator
the record never claimed is the confident nonsense the contract removes. An
omitted bucket still means "not declared", never zero.

Existing records are not migrated, so the render splits the unparsed set by
`CANONICAL_FROM` using the API's created_at -- never anything read out of a
title. Nothing is extracted from a legacy title and no legacy shape is
recognised; only the explanation differs. Measured over the live corpus
(34 records, 2026-08-21): 3 parse, 31 predate the contract, 0 deviate. Without
the split the enforcement surface would be 34 rows deep and the first genuine
deviation invisible inside it.

--self-test carries both halves: 5 canonical shapes that must parse and 16
retired ones that must be rejected, each asserting the reason names the field
that failed. The rejections are the half that holds the ruling -- a parser
re-widened to accept one of them passes every happy-path assertion.

Fixes #9617

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ApyDuQY2fkunMCqXiqvBhR
…sentials

The ratchet ceiling for this file is 232 with zero headroom, so the first
draft's 21 added lines were 21 over. Compressed to the binding elements only:
the shape, the five field rules, the retired phrasings as a bare list, and the
enforcement surface. Provenance is one line and the (FULL area) story is one
clause, per the ratchet's own remedy text.

Still 241 — 9 over. The remaining 9 cannot be reclaimed without either editing
the ceiling ledger (scripts/pm/check-skill-line-ratchet.mjs, outside this
card's declared file surface) or deleting unrelated content from the file.
Reported for a maintainer decision rather than resolved by either.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ApyDuQY2fkunMCqXiqvBhR
@github-actions github-actions Bot added size/l documentation Improvements or additions to documentation labels Aug 21, 2026
@huangyiirene huangyiirene added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Aug 21, 2026 — with Claude
…n SKILL.md

PM ruling on the ratchet question (option C): .claude/skills/checklist-test/SKILL.md
was at its line ceiling of 232 with zero headroom, so documenting the contract in it
overflowed by 9 lines. Rather than raise the ceiling (needs a maintainer ruling the
2026-08-20 title ruling is not) or delete maintainer-authored content, the field-by-field
grammar moves to the operational doc this skill already cites throughout.

docs/qa/platform-checklist/RUNNER.md gains 'The record title — one canonical shape' under
its existing '## Run records' section: the shape, a field-by-field table, the
declared-vs-zero rule, the retired phrasings with why (FULL area) died, and the
new-records-only note. SKILL.md keeps the binding rule — one shape, strict parsing,
judged/total mandatory — plus a pointer, and is back to exactly 232.

This is the ratchet's own remedy: principles in SKILL.md, on-demand detail elsewhere.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ApyDuQY2fkunMCqXiqvBhR
@github-actions

Copy link
Copy Markdown
Contributor

⛔ merge queue 构建失败 — 先分诊,再决定要不要重排

队列构建 32543326482 红了。队列跑的是全量套件(PR 侧 CI 只跑 affected 子集),
所以失败的测试可能在本 PR 没碰过的包里 —— 那不是重排能修的。每次盲目重排都会让排在后面的所有 PR 重建一轮。

失败的 job(日志抽取,best effort):

  • Console Pin Gate — 失败步骤: Build the Console SPA at the pinned objectui SHA

    ✗ Build failed in 5.83s
    

↳ 失败原因 是判读的关键:超时(Test timed out in … / Hook timed out in …)多半是负载/时序,不是本 PR 的回归;
断言(AssertionError: …)才指向真实的行为改变。两者的 FAIL 行长得一模一样,只有这一行能区分。

跨 PR 相同签名(24h,按失败测试文件聚合):

  • ⚠️ 本次没有可用的聚合签名(日志里没有能解析出测试文件名的 FAIL 行)—— 这不是「没有同签名的其他 PR」,是这一轮没测到。跨 PR 聚合本次不可用,请手工比对其他 PR 的同类评论。
  • ⚠️ 24h 评论账本没读完(超过 5 页仍未读到窗口尽头),所以上面的「不同 PR 数」是下界,不是全量。

历史信号:

  • 本 PR 过去 24h 无队列失败记录(首次)。
  • 过去 24h 队列共有 57 个失败构建(不含本次)。

分诊清单:

  1. 失败测试在本 PR 改动的包里 → 真回归,修 PR。
  2. 失败测试与本 PR 无关 → 看上面的「跨 PR 相同签名」;已有汇总 issue ⇒ flaky/环境问题实锤,去那张 issue 上谈,修好前重排只会再烧一轮全队列。
  3. 两者都不是 → 可能与同组 PR 语义冲突;等前面的 PR 落地或失败出队后再重排一次即可,不要连续重排。

Generated by Claude Code · merge-queue-triage workflow (#4859)

Merged via the queue into main with commit 81845c6 Aug 22, 2026
32 checks passed
@huangyiirene
huangyiirene deleted the claude/issue-9617-qa-run-title-contract branch August 22, 2026 01:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/l skip-changeset PR has no user-facing published change; bypasses the changeset gate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

skills(checklist-test): the documented QA-run title convention matches 0 of 23 real records

2 participants