Repository navigation
perf(service-messaging): an idle dispatcher tick reads the database twice, not 18 and 10 times - #28403
Conversation
…of probing every partition The reap reads before it writes, and one environment-wide due-work read gates the per-partition claim and digest probes. Claude-Session: https://claude.ai/code/session_01QUs3Bvkfh86Jgvi7gewhBD Co-authored-by: Claude <noreply@anthropic.com>
…ments Counts at the client (knex query events) with a global afterUpdate hook, the kernel shape cloud measured 18 and 10 at. Claude-Session: https://claude.ai/code/session_01QUs3Bvkfh86Jgvi7gewhBD Co-authored-by: Claude <noreply@anthropic.com>
…onal reap or a per-partition probe Claude-Session: https://claude.ai/code/session_01QUs3Bvkfh86Jgvi7gewhBD Co-authored-by: Claude <noreply@anthropic.com>
…tick Claude-Session: https://claude.ai/code/session_01QUs3Bvkfh86Jgvi7gewhBD Co-authored-by: Claude <noreply@anthropic.com>
📓 Docs Drift CheckThis PR changes 1 package(s): 1 hand-written doc(s) NAME something this change touched and may need an implementation-accuracy re-verification:
⛔ 2 release-owned page(s) also name something this change touched. These are read-only:
What this run could not see
Coarse fallback — 6 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin dbc31404398cbcc414248981e4a70877b8331e72 && git checkout dbc31404398cbcc414248981e4a70877b8331e72
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 63a9bcb773045ccffe336d0ee4876c748f5adbfb 570a250c1096c8565086de20b28b0792de780773 && git checkout -B drift-repro 63a9bcb773045ccffe336d0ee4876c748f5adbfb && git merge --no-ff 570a250c1096c8565086de20b28b0792de780773
node scripts/docs-audit/affected-docs.mjs --json 63a9bcb773045ccffe336d0ee4876c748f5adbfb
|
…the changeset Claude-Session: https://claude.ai/code/session_01QUs3Bvkfh86Jgvi7gewhBD Co-authored-by: Claude <noreply@anthropic.com>
|
domain:server#4 · os-zhuang · session_01QUs3BtyH9zgxKzYrJiH18J Contract reviewServed-tier: Face: the declaration face alone ( ① Derived judgments
② Semver level
③ Boundary flags
Implemented-by: VERDICT: PASS |
|
domain:server#4 · os-zhuang · session_01QUs3BtyH9zgxKzYrJiH18J ACCEPT · PR #28403 · Head-sha 维护者速读
席位意见:空闲开销从 28 条降到 4 条, 席位核对
|
Requested by Sam · project thread
Fixes #28383
Clause-②: no
Metadata-protocol: none
维护者速读
hasDueWork是可选方法,自定义 outbox 不实现就保持旧行为。回滚即 revert 本 PR。packages/services/service-messaging/src/dispatcher.ts(runTick里的门)、src/sql-outbox.ts(reapExpired与hasDueWork)、src/dispatcher-idle-cost.integration.test.ts(语句数钉子)。pnpm --filter @objectstack/service-messaging exec vitest run --maxWorkers=2 src/dispatcher-idle-cost.integration.test.ts src/http-dispatcher-idle-cost.integration.test.ts,预期 13 个用例全绿,空闲 tick 钉在 2 条 SQL。What changes
An idle tick of each delivery dispatcher now costs two SQL statements, whatever
partitionCountis. Measured at the client (knexqueryevents) on a harness with one globalafterUpdatehook, the kernel shape cloud measured:ac09e7b85NotificationDispatcherHttpDispatcherThe card's bar is ≤ 6 per dispatcher.
reapExpiredinsql-outbox.tsandsql-http-outbox.tsfirst reads one expiredin_flightid (fields: ['id'], limit: 1). If it finds none, it returns. If it finds one, it runs the predicateUPDATE, which is unchanged, so every claim expired by then still goes back topending, not only the one the read saw. On a kernel the old unconditionalUPDATEwas two statements (the engine reads the matched rows for the per-rowafterUpdatedispatch, then writes); it is now one read.INotificationOutboxandIHttpOutboxgain an optionalhasDueWork({ now }). It asks, in oneSELECT id … LIMIT 1across every partition and organization, whether anypendingrow is due. The predicate is the claim's own with the partition term dropped, and for notifications with thedigest_keyarm dropped, so it covers immediate and batched rows alike.runTickasks it after the reap. If nothing is due, the tick ends there: no partition lock is taken and no claim or digest probe runs. If something is due, the tick runs the full per-partition pass exactly as before, for one extra read. Both built-in stores (SQL and memory) implement it. A store without it keeps probing every partition.DueWorkOptionsandHttpDueWorkOptionsare exported.partitionCountand the storedpartition_keyare unchanged.Why
An idle hosted tenant kernel read its database 56 times a minute with nothing queued: 2 ticks × (18 + 10) at the framework's 30 s idle ceiling, plus a 140-statement backoff ramp in the first minute. Those reads count against Turso's rows-read quota, and cloud#2177 once had a staging account suspended for reads. With this PR an idle minute costs 8 statements at the 30 s ceiling.
Tests
dispatcher-idle-cost.integration.test.tsandhttp-dispatcher-idle-cost.integration.test.tsnow count SQL statements on the delivery table at the client as well as engine calls, with a global no-opafterUpdatehook registered so a predicateUPDATEis the read-then-write pair it is on a kernel. On this harness, main measures 18 and 10 statements per tick, the same figures cloud measured.{ update: 0, find: 2 × TICKS }engine calls, and exactly2 × TICKSSQL statements, allSELECT. This replaces the old≤ TICKS × (1 + 2 × partitionCount)and≤ TICKS × (1 + partitionCount)ceilings.notBefore; an HTTP retry acked withnextRetryAt) keeps the tick at two reads and goes out on the first tick after it comes due.UPDATEran exactly once when a claim had expired. This guards against a reap that never writes.reap, and so nohasDueWork) are unchanged and still see2 × partitionCountandpartitionCountclaims a tick.sql-outbox-audit-columns.test.ts: its fake engine's scriptedfindqueue now opens with a reap read that finds a stale row, so the reapUPDATEpayload it checks forupdated_atis still produced.system-context.pin.test.ts: the population counts include the reap reads and onehasDueWorkcall, so the new reads are also held to the explicit system opt-in.http-dispatcher-idle-backoff.test.ts: it timestamped a tick atclaim(), which an idle tick no longer calls. It now timestamps atreap(), which every tick calls once first, as the notification side's file already did.Acceptance notes
acquire, and on the common single-node or memory lock every partition is held anyway. A read that moves nothing needs no lock, as with the reap. Keeping the per-partition claims under their locks leaves the busy path, the batch size per partition and the digest-under-lock rule exactly as they were. On a multi-node cluster, a node whose partitions are all held elsewhere pays the same lock attempts as before whenever something is due anywhere, and two reads when nothing is.packages/core/src/dispatch-loop.ts).ORDER BY id(the engine's default when alimitis set) over alimit 1. That is one row from the primary key ordering, not a scan cost this card measured.Verification
All readings at head
95f4da30aunless stated.Statement counts (before / after)
A throwaway measurement on main
ac09e7b85(not committed), with and without a globalafterUpdatehook, one tick each over an empty outbox:select * … where status = ? and claimed_at < ?then theupdate, followed by one or twoselect id … partition_key = ?per partition.select id … where status = ? and claimed_at < ? … limit ?, thenselect id … where status = ? and ((next_… is null) or (next_… <= ?)) … limit ?.Package tests and typecheck
pnpm --filter @objectstack/service-messaging test: Test Files 62 passed (62), Tests 759 passed (759). Lock VERDICT command-exit 0 at18e1c546b. The later commit only adds the changeset and oneas HttpClaimCredentialcast in a test.pnpm --filter @objectstack/service-messaging typecheck: exit 0 at95f4da30a. At18e1c546bthe first run caught one TS2345 in the new HTTP retry leg (anHttpDeliverypassed as the claim credential), fixed with the cast the dispatcher itself uses.95f4da30a: 13 passed (13).Ablation (scripts/ablation-replace.mjs, restored, git diff HEAD empty, marker grep 0)
Tests import
src/relatively, so no dist rebuild sits between mutation and run.hasDueWorkgate inNotificationDispatcher.runTickreplaced by a marker. Notification idle-cost file: 2 failed / 5 passed (the idle pin:find: 170vs20, 17 statements vs 2; the deferred-row pin).SqlHttpOutbox.reapExpired's early return replaced by a marker. HTTP idle-cost file: 2 failed / 4 passed (the idle pin:update: 10vs0; the retry pin: 4 statements vs 2).Dispatch gates and lint
node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack: 62 families derived from 15 paths vs merge baseac09e7b85(473 changed lines, under the 3000 threshold). Every command was run with stdin closed, and each exit code was captured to disk before any pipe.--ranreconciliation: 62 derived, 62 run, 0 NOT-MEASURED, 0 UNRUN; it exits 0.pnpm check:dual-build-cjs-loadsandpnpm check:i18nafter a fullpnpm build(lock VERDICT command-exit 0).node scripts/check-plugin-teardown-shape.mjs --self-testaftergit fetch --unshallow origin main(48 cases pass).pnpm lint(repo-wideeslint . --no-inline-config): lock VERDICT command-exit 0.Human Minimum
packages/specholds no runtime logic. (Not touched.)pnpm lintpasses. (See Verification.)main, one pull request.packages/spectouched: n/a, not touched.content/docs/references/,packages/*/CHANGELOG.mdorcontent/docs/releases/.Generated by Claude Code