Skip to content

Canonicalise driver-materialised timestamps at the metadata adapter boundaries - #14040

Merged
zhuangjianguo merged 2 commits into
mainfrom
claude/issue-13997-authored-at-canonicalisation
Sep 1, 2026
Merged

zhuangjianguo merged 2 commits into
mainfrom
claude/issue-13997-authored-at-canonicalisation

Conversation

@zhuangjianguo

Copy link
Copy Markdown
Collaborator

Fixes #13997

MetadataItem.authoredAt is declared z.string().describe('ISO-8601 timestamp') and MetadataItem is a z.infer, so the field is string to every consumer. MetadataStats.mtime is declared z.string().datetime(). Three producers adapted a driver row into those declared types without converting the value, and on Postgres and MySQL a JS Date landed in each of them.

The mechanism

created_at / updated_at are builtin audit columns and recorded_at is a declared Field.datetime. SqlDriver#formatOutput repairs the audit columns (repairNaiveUtcAuditTimestamp) and folds declared datetime columns (normalizeSqliteDatetimeOutput) only inside its if (this.isSqlite) arm, and withPostgresCalendarDayAsText leaves the instant types alone on purpose: "timestamptz / timestamp are deliberately untouched: those are instants, a Date is the right materialisation for them, and Field.datetime depends on it." Pinned live in packages/drivers/driver-sql/src/sql-driver-13567-audit-stamp-materialisation.test.ts.

Two independent reasons nothing reported it: the row is any, so tsc saw a string assignment that never happened; and the runtime validator that would have caught it never runs on these paths (see the observation at the end).

What changed

Canonicalised at the producer — the adapter boundary that asserts the declared type — matching the spelling auditMetaItem already applies to sys_metadata_audit.occurred_at in protocol.ts. Not a third spelling, and not a tolerant fallback: it converts the one per-dialect materialisation the driver genuinely produces into the single declared spelling. Values that were already canonical (SQLite) pass through byte-identically.

site field was
sys-metadata-repository.ts getByHash() MetadataItem.authoredAt (row as any).recorded_at ?? …
sys-metadata-repository.ts rowToItem() (via get()) MetadataItem.authoredAt row.updated_at ?? row.created_at ?? …
database-loader.ts stat() MetadataStats.mtime record.updatedAt ?? record.createdAt ?? …

Route B — normalising at the driver's read door — is deliberately not taken: it reverses a stated driver decision and belongs to the whole census, not to this card.

⚠️ The card's classification of one site was wrong, and this PR repairs it too

The issue and its triage both list sys-metadata-repository.ts:417 (getByHash) among the five sibling producers that already spell it canonically, on the stated ground that recorded_at is a declared Field.datetime rather than a builtin audit column. Measured, that ground does not hold: the declared-Field.datetime coercion is SQLite-gated too. In packages/drivers/driver-sql/src/sql-driver.ts the datetimeFields normalisation loop sits at brace depth 2 inside the isSqlite arm opened at line 15879 and closed at 15968 — and withPostgresCalendarDayAsText says in as many words that Field.datetime depends on the Date materialisation.

So getByHash was a third straight-through site, not a canonical sibling. It is in one of the two files the card names and carries the identical defect on the identical declared field, so it is repaired here rather than deferred. Everything else measured in this sweep is filed separately (below).

The card's central contrast survives and is stronger than stated — it is 9 canonical producers against 3 straight-through ones, and the straight-through ones are still exactly the sites that pass a driver row through unconverted:

  • canonical (unchanged): sys-metadata-repository.ts:606, in-memory-repository.ts:120, metadata-fs/src/repository.ts:271, :300, :374, :421 (all from a JSONL log field written as this.now().toISOString()), memory-loader.ts:72, remote-loader.ts:108, filesystem-loader.ts:227
  • straight-through (all three repaired here): sys-metadata-repository.ts:417, :1752, database-loader.ts:917

The pin

packages/metadata-protocol/src/sys-metadata-repository-13997-authored-at-canonicalisation.test.ts, plus two cases appended to database-loader.test.ts.

The existing schema test proves nothing about this seam because it feeds a hand-made string — the assertion and the input share an identity. Every case here drives a hand-made Date instead, the one shape the live dialects produce and no existing fixture ever did, and each carries a non-vacuity guard asserting the input really is a Date before the output is read. There is no driver dependency: metadata-protocol has none and must not grow one, so the Date is hand-made for the same reason the #13567 pin states for the OCC seam next door. The authoredAt cases assert through MetadataItemSchema.safeParse itself rather than a hand-rolled regex.

Ablation, against the committed tree, restored under a trap on absolute paths: reverting the three call sites turns the pins red — 2 of 3 in the new file, 1 in database-loader.test.ts — with AssertionError: expected 'object' to be 'string', which is the defect verbatim. The idempotence cases stayed green under the ablation, which is what shows the pins discriminate rather than being globally sensitive. Restore verified by an empty git diff HEAD and blob hashes identical to HEAD.

Gates

Run at 9caf55e082, exit codes captured before any pipe. node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack named 30 families, re-derived after the final commit (the ledger edit pulled in 7 more, all run and green).

Two went red and both were mine, each repaired rather than baselined:

  • check:objectql-double-limit — the new fake find ignored opts.limit. Now applies the caller's bound after the filter and by presence; the gate's graded population moves from 98 to 99 applying the bound, and no baseline grew.
  • check:engine-double-contract — the new double's three verbs are pinned to ObjectQL's dispatch predicates, but the RETAINED ledger did not know the file, so nothing protected it. Regenerated with --write: 3 rows added or grown, 0 lost. They are pinned rows, not baseline entries.

check:type-check-debt --re-measure passes against the built workspace closure: 29 ledger entries re-measured, none above its recorded number, "surplus: none — every entry sits exactly at its measurement", so the new test file adds zero type errors. Both affected packages carry type-check DEBT ledger entries and declare no typecheck script, so a per-package typecheck is not available here and this ratchet is the real type evidence.

Tests: @objectstack/metadata 627/627, @objectstack/metadata-protocol 2063 passed with 10 pre-existing skips.

Not measured, and reported as such rather than as passes: check-test-completeness (exit 3 — it parses a CI test-run summary that does not exist locally). check:dual-build-cjs-loads initially exit 3 and green once the closure was built.

⚠️ One observation, stated because it should not pass silently — deliberately not fixed here

MetadataItemSchema is a declared runtime validator that exists, is trusted, and has zero production coverage: its only .parse call sites in the repo are its own unit test (packages/metadata-core/test/types.test.ts:70,74,78), which feeds a hand-made sample, so it evaluates only against inputs that were never near a driver — which is exactly why it did not catch this. Whether it should be parsed somewhere on a real path, and whether other declared schemas are in the same state, is a larger question than this card and is not answered here.

That zero is measured with a firing positive control, in the same file: the sibling MetadataEventSchema, declared beside it in packages/metadata-core/src/types.ts and exported from the same barrel, is parsed on production paths (packages/metadata/src/metadata-manager.ts:693, packages/client/src/realtime-api.ts:107), and the same expression family finds 289 hits across packages/. So the zero is a fact about this schema, not an artefact of the expression or the search scope.

Filed separately, not addressed here

#14037 — five further adapter-boundary sites in these same two files cast a driver Date into a declared ISO-string timestamp (MetadataEvent.ts, MetadataHistoryRecord.recordedAt, MetadataRecord.createdAt / updatedAt). Three of them are declared z.string().datetime(), stricter than the field this card dealt with. Scope for that card, not this one.

#13973 (the census) and #13382 (the OCC seam) remain open and are not touched by this change.


Generated by Claude Code

…pter boundaries

`MetadataItem.authoredAt` is declared `z.string()` ('ISO-8601 timestamp') and
`MetadataStats.mtime` is declared `z.string().datetime()`. Three producers
adapted a driver row into those declared types without converting the value:

  - `SysMetadataRepository#getByHash` — `recorded_at`, a declared
    `Field.datetime` on `sys_metadata_history`
  - `SysMetadataRepository#rowToItem` (via `#get`) — the builtin audit columns
  - `DatabaseLoader#stat` — the same audit columns, through `rowToRecord`

`SqlDriver#formatOutput` repairs the audit columns and folds declared datetime
columns only inside its `if (this.isSqlite)` arm, and
`withPostgresCalendarDayAsText` leaves `timestamptz` / `timestamp` deliberately
untouched — so on Postgres and MySQL a JS `Date` landed in a field every
consumer reads as a `string`. Nothing reported it: `row` is `any`, and
`MetadataItemSchema` is parsed nowhere on a production path.

Canonicalised at the producer, matching the spelling `auditMetaItem` already
applies to `sys_metadata_audit.occurred_at`. Already-canonical SQLite text
passes through byte-identically. Pinned by driving a hand-made `Date` through
each adapter, asserted through `MetadataItemSchema` itself.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F3jdziLbAPGeceVNmSox5L
… double, and record its pins

Two gates named this file, both correctly:

  - `check:objectql-double-limit` — the fake `find` ignored `opts.limit`, so it
    would answer more rows than the real engine. Now applies the caller's bound
    AFTER the filter and BY PRESENCE, which is the conforming shape (the gate's
    graded population moves 98 -> 99 applying the bound; no baseline grows).
  - `check:engine-double-contract` — the new double's three write/read verbs
    are pinned to ObjectQL's dispatch predicates but the RETAINED ledger did not
    know the file yet, so nothing protected it. Regenerated with `--write`:
    3 rows added or grown, 0 lost.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F3jdziLbAPGeceVNmSox5L
@github-actions github-actions Bot added the size/m label Sep 1, 2026
@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

3 anchor(s) derived from 2 changed package(s); no hand-written page names any of them, so this run has nothing to list — not a clean bill of health. This check sees only pages that NAME a derived anchor: one that documents this change in prose, or enumerates it in an authoring dialect, names none and stays invisible to it on every run.

What this run could not see
  • 1 name(s) were too generic to anchor anything (single lowercase words)
  • the SDK route bridge reached 47 of 219 client-bound route-ledger rows — the other 172 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 172: 14 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 56 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 102 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.

Coarse fallback — 12 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 5e2c04da7db38c4db0b138fad8b9be5b4ef308fc → packageMentionDocs.

Which tree this was computed on

This run read content/docs from 28683db0f84523bed42e36287b3608ceb55cef41 — the merge of head 9caf55e082f59e9bfdb686dafd93e783f6bd6bc5 into base 5e2c04da7db38c4db0b138fad8b9be5b4ef308fc, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 28683db0f84523bed42e36287b3608ceb55cef41 && git checkout 28683db0f84523bed42e36287b3608ceb55cef41
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 5e2c04da7db38c4db0b138fad8b9be5b4ef308fc 9caf55e082f59e9bfdb686dafd93e783f6bd6bc5 && git checkout -B drift-repro 5e2c04da7db38c4db0b138fad8b9be5b4ef308fc && git merge --no-ff 9caf55e082f59e9bfdb686dafd93e783f6bd6bc5

node scripts/docs-audit/affected-docs.mjs --json 5e2c04da7db38c4db0b138fad8b9be5b4ef308fc

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/m tests tooling

Projects

None yet

2 participants