Repository navigation
fix(spec): $like / $ilike _ is one Unicode code point on every JS face - #20222
Conversation
…int on the JS faces likePatternToRegexSource returned a source that every JS face compiled without flags, so `[\s\S]` read one UTF-16 code unit: `_` missed a stored emoji that every SQLite face (GLOB `?`) matched. likePatternToRegExp now owns the compilation with `u`; matchesLikePattern and both driver-memory mingo doors take it. LIKE_DESCRIPTION and the docblocks say which "character" is meant: one Unicode code point. Claude-Session: https://claude.ai/code/session_01Rjy9MeetSfq34PKn81CRiN Co-authored-by: Claude <noreply@anthropic.com>
The family cells (`_`, `__`, `a_b`, `a__b`, `___`, and the $ilike twin) over values outside the BMP, asserted as literal code-point rows on the spec, formula, all three driver-memory doors, driver-sql (better-sqlite3), driver-sqlite-wasm (sql.js) and TursoDriver local plus remote (stub and a real libSQL engine); each SQL face is also held to the JS answer. The spec suite adds the u-flag legality pin over every ASCII punctuation mark. Claude-Session: https://claude.ai/code/session_01Rjy9MeetSfq34PKn81CRiN Co-authored-by: Claude <noreply@anthropic.com>
Clause-②: yes (narrowing), with the ADR-0087 not-required disposition. Claude-Session: https://claude.ai/code/session_01Rjy9MeetSfq34PKn81CRiN Co-authored-by: Claude <noreply@anthropic.com>
+ likePatternToRegExp (function) on ./data (0 breaking, 1 added), and the $like description's code-point sentence in the generated filter reference. Claude-Session: https://claude.ai/code/session_01Rjy9MeetSfq34PKn81CRiN Co-authored-by: Claude <noreply@anthropic.com>
…ke-underscore-code-point
📓 Docs Drift Check5 anchor(s) derived from 2 changed package(s); no hand-written page names any of them. What this run could not see
Coarse fallback — 137 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin ef8deacbdac297ee94797aea951836bfc11c69c5 && git checkout ef8deacbdac297ee94797aea951836bfc11c69c5
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 805af4f290565955d6e6b56ee46fed45f721d955 713d1504d6e9c710c94dae8efaeda953e478b932 && git checkout -B drift-repro 805af4f290565955d6e6b56ee46fed45f721d955 && git merge --no-ff 713d1504d6e9c710c94dae8efaeda953e478b932
node scripts/docs-audit/affected-docs.mjs --json 805af4f290565955d6e6b56ee46fed45f721d955 |
Contract reviewServed-tier: ① Derived judgments
② Semver level
③ Boundary flags
Implemented-by: VERDICT: PASS |
Fixes #20143
Clause-②: yes
What was wrong
$like/$ilikeanswered two different row sets for one filter, depending on the driver behind the object. The SQLite faces (driver-sqlon better-sqlite3,driver-sqlite-wasm,driver-tursolocal and remote) compile_to GLOB's?throughlikePatternToGlobPattern, and?counts code points. The JavaScript faces compiled the spec'slikePatternToRegexSourcewith no regex flags, so_→[\s\S]read one UTF-16 code unit, which is half of an emoji. The spec's text said "_matches exactly one character" without saying which unit.Triage set the direction:
_is one code point. The stop condition (a real semantic reason for the UTF-16 reading) was measured and does not hold; see below.The fix
@objectstack/spec(packages/spec/src/data/filter.zod.ts)likePatternToRegExp(pattern, foldAscii = false): RegExp. It compileslikePatternToRegexSourcewith theuflag and nothing else, so the flag lives in the one shared definition instead of in each consumer.matchesLikePatternnow evaluateslikePatternToRegExp.likePatternToRegexSourceis unchanged and still exported. Its docblock gains a section saying the source means one code point per_only underu.LIKE_DESCRIPTIONand the GLOB table now say which "character" is meant.@objectstack/driver-memory(src/memory-driver.ts): both mingo doors (the$likefilter atnormalizeFilterCondition, and the ASTlike/ilikenode atconvertConditionToMongo) built their ownnew RegExp(likePatternToRegexSource(...))with no flags. They now takelikePatternToRegExp. This is a source change outside the claim's expected surface. Measurement put the flag's choice in these two call sites, not in the spec, so a spec-only change would have left both doors on UTF-16. This is the producer of theRegExpmoving into the spec, not consumer-side tolerance.@objectstack/formula: no source change.matchesFilterConditionanswers throughmatchesLikePattern.driver-sql/driver-turso/driver-sqlite-wasmsource change. Those faces are the oracle; they gain test files only, per the card's "pin the family cells on every face".Changed
describe()text, verbatim (LIKE_DESCRIPTION, both copies)Before:
After:
No refusal text changed.
likePatternToRegExpthrows the same plainErroras the source translation on a dangling escape (the backstop behindhasDanglingLikeEscape), and that message is pinned.Measured: every face, before and after
A probe (18 patterns over 9 values holding U+1F600 and U+1D49C) ran each face from its built
dist/against a code-point oracle (the spec regex compiled withu):matchesLikePatternformula$likedoore7f69dbb713d1504The driver-memory reference matcher is not exported from
dist/, so it is pinned in the unit suite instead.The family cells, before → after on the JS faces (after = what every SQLite face already answered)
Rows:
x,😀,😀😀,a😀b,a😀😀b,A𝒜B,ab,axb,axyb.$like_x😀,x$like__ab,😀ab,😀😀$likea_baxba😀b,axb$likea__ba😀b,axyba😀😀b,axyb$like___axba😀b,axb,A𝒜B$like😀_/_😀😀😀$ilike_x😀,x$ilikeA_Baxba😀b,axb,A𝒜B$ilikeA__Ba😀b,axyb,A𝒜Ba😀😀b,axybThe answer set moves both ways, so the changeset's declaration names the measured direction arm, narrowing (BREAKING, graded
minorin the launch window), with an ADR-0087not-required (no-migration-prescription)disposition. No pattern is newly refused, no refusal is lifted, and values made only of BMP characters answer exactly as before.Every consumer of the translation (Zone 2 item 2), before → after
specmatchesLikePatternnew RegExp: UTF-16likePatternToRegExp: code pointformulamatchesFilterConditionmatchesLikePattern: UTF-16driver-memory$like/$ilikedoor (mingo)new RegExp: UTF-16likePatternToRegExp: code pointdriver-memoryASTlike/ilikedoor (mingo)new RegExp: UTF-16likePatternToRegExp: code pointdriver-memoryreference matchermatchesLikePattern: UTF-16driver-tursoturso-20024-like-stored-nulgrid oraclematchesLikePattern; its rows are BMP-onlyservice-analytics,@objectstack/lint, objectqlhaving,driver-mongodbgit grepof both symbols)git grepof both symbols: 0 hits at HEAD5c61e5241and at the pinned.objectui-shaf8a9d0fb(exit 1 on both; a control grep in the same tree hits)No other LIKE-to-regex translation exists in
packages/**/src.u-flag side effects (Zone 2 items 3 and 4)\ ^ $ . * + ? ( ) [ ] { } |and/. That is the regex syntax characters plus/, exactly the identity escapesuaccepts. Every other ASCII punctuation mark is emitted bare, whereureads it as itself. This is pinned over all 33 printable ASCII non-alphanumerics, bare and backslash-escaped, withfoldAsciioff and on: each compiles underuand matches only itself. The escaped set is pinned to equal that set. A control provesureally rejects\-and\:.[\s\S], never.. Both wildcards use[\s\S], so underua_is one code point including a newline (pinned: source^[\s\S]$, anda_bmatchesanewlineb).😀is^😀$), andureads the pair as one code point. Underu,%never splits a pair:%followed by a lone low surrogate does not match😀(pinned).$ilike/foldAscii. The fold stays in the source as[Aa]classes, unaffected byu.likePatternToRegExp(...).flagsis pinned to exactlyu, with noi. The$ilikefamily cells are pinned on every face.The stop condition: nothing needs the UTF-16 reading (Zone 2 item 5)
@objectstack/formulalowers CEL to$like:cel-to-filteremits no$like, andrls-predicateleaves SQLLIKEunparseable. The only$likeevaluation ismatches-filter.ts, via the spec.size()informulaalready counts code points:size('😀')is 1 andsize('a😀b')is 3 (pinned in the formula suite). The old$likeanswer disagreed with the same package's string length.Pins (every face, literal code-point rows, not parity alone)
packages/spec/src/data/filter-like-code-point.test.tspackages/formula/src/matches-filter-like-code-point.test.tspackages/drivers/driver-memory/src/memory-20143-like-code-point.test.tspackages/drivers/driver-sql/src/sql-driver-20143-like-code-point.test.tspackages/drivers/driver-sqlite-wasm/src/sqlite-wasm-20143-like-code-point.test.tspackages/drivers/driver-turso/src/turso-20143-like-code-point.test.tsEach SQLite face also holds itself to the JS answer (
formula, or the spec'smatchesLikePatternwhereformulais not a dependency), which is the cross-face half.Ablation (committed fix first; one-shot, no permanent test file)
The mutation replaced the
'u'flag argument with an expression that evaluates to the empty flag string and carries the markerABLATION20143. It went throughscripts/ablation-replace.mjsunder its restore trap, inside a script with its owntraprestoring fromHEAD.pnpm --filter @objectstack/spec buildexited 0, andablation-dist-preflight.mjs @objectstack/spec ABLATION20143exited 0, so the marker is indist/. The expected direction was written down before running: JS faces red on the family cells; SQLite faces green on their literal cells and red only on the cross-face leg. Observed: spec 13 failed / 23; formula 11 / 12 (CELsize()stays green); driver-memory 10 / 11; driver-sql 1 / 12, driver-sqlite-wasm 1 / 12, driver-turso 1 / 12 (each the cross-face leg only). That is the predicted direction.git hash-objectequals theHEADblob838a80720b3d,git diff HEADis empty, andgit status --porcelainis clean. Rebuild exited 0,ablation-dist-preflight.mjs ... --absentexited 0, and all six suites are green again.Verification (at
713d1504, after mergingorigin/mainat585c9af5)Build of the six-package closure: turbo 10/10.
pnpm --filter @objectstack/spec check:generatedpassed after the merge. Regenerated here:api-surface/data.json(0 breaking, 1 added:likePatternToRegExp),export-origins/data.json,content/docs/references/data/filter.mdx.Full suites: spec 542 files / 15,943 passed (2 todo); formula 38 / 1,039; driver-memory 55 / 1,339; driver-sqlite-wasm 35 / 656; driver-turso 73 / 1,921 (16 skipped); driver-sql 189 files (+11 skipped) / 3,111 passed (170 skipped).
Typecheck, exit 0 on all six: spec, formula, driver-memory, driver-sql, driver-sqlite-wasm, driver-turso. Each new test file is counted inside a compiled program with
--listFiles:tsconfig.jsonfor the four drivers, andtsconfig.test.json(viacheck:test-typecheck) for spec and formula.Gates: 108 derived at
713d1504(dispatch-gates.mjs --commands --repo objectstack-ai/objectstack), reconciled with--ran: 101 exit 0, 0 unrun, 7 NOT MEASURED. All seven are exit 3, PREREQUISITE NOT MET:check:doc-formula-expressions,check:doc-security-postureandcheck:docs-transcript-driftneed@objectstack/lintbuilt.check:lean-entry-closureneedsobjectqlbuilt, andcheck:skill-examplesneedsclient/client-reactbuilt.check:dual-build-cjs-loadsandcheck:type-check-debtneed the whole workspace built.The targeted prerequisite build never acquired the shared verify-lock (two 9-minute queue-timeouts behind other seats' batches), so those seven are declared to CI's Lint & Repo Gates. The seven roster families whose roster sits in a touched directory were run as well, all exit 0:
check:meta-url-spelling,check:spec-changes,check:authz-resolver,check:error-code-casing,check:filter-alias-parity,check:object-def-param-keys,check:tenant-chokepoint.check-adr-0087-registrationreads the changeset as[clause-②-narrowing] not-required (no-migration-prescription).Acceptance notes
_is exactly one character":content/docs/api/client-sdk.mdx:553,content/docs/deployment/troubleshooting.mdx:148, and thepackages/client/src/query-builder.ts:88JSDoc. They stay true under the reading the spec now defines, and are not edited here. Carrier: none.likePatternToRegexSourcestays exported, so a future consumer could still compile it withoutu. Its docblock carries the prohibition, and after this PR nothing outside the spec calls it. Retiring the export would be a removal and is not this card.utreat it as its own code point._arms are NOT MEASURED, as on the card. SQL_is one character there.Generated by Claude Code