Skip to content

Installer update: check every model the release calls before deploying - #319

Merged
nambok merged 1 commit into
mainfrom
fix/cli-update-model-check
Oct 26, 2026
Merged

nambok merged 1 commit into
mainfrom
fix/cli-update-model-check

Conversation

@nambok

@nambok nambok commented Oct 26, 2026

Copy link
Copy Markdown
Collaborator

Summary

coderhelm-selfhost update, run from someone's machine, now does the same model check as the updater before it applies a release. It reads the release's models.json only after the signed SHA256SUMS check from #310 passes. It then tests each model through the install's provider and stops with the same one-sentence fix and console link. --skip-model-check passes only a model the release doesn't call. Follow-up to #318.

Changes

  • installer/lib/release.mjs: releaseModels() runs inside fetchRelease, after verifySignature.
    • It reads models.json only when the signed SHA256SUMS lists it, and only when the file matches its checksum.
    • When the release was verified with gh instead of cosign, models.json must also carry its own build attestation.
    • It refuses a list made for another release, and throws if it is called without a completed signature check.
    • A signed release that lists no models returns null. This also applies to cached releases, which are checked again.
  • installer/lib/commands.mjs: update runs updateModelCheck after the release is fetched and verified, and before migration notes, the diff or deploy.
    • The install's <prefix>-worker gets a model_check direct invoke with the release's models plus the config's own, using the release's Bedrock ids per prefix. The probe team comes from the SYSTEM/MODELS record.
    • Each model prints PASS, FAIL or SKIP. A FAIL shows the fix and the console link.
    • A refused model the release calls (or the config names) stops the update, and --skip-model-check can't override it.
    • --skip-model-check passes only failures on models the release doesn't call, and records "model check skipped for …" in the result file.
    • A worker that doesn't answer also stops the update.
    • Rollback, --source checkouts and dry runs skip the check.
    • Nothing accepts terms or subscribes.
  • installer/lib/cli.mjs: help for --skip-model-check.
  • infra/lib/updates-stack.ts: the update build runs coderhelm-selfhost update, so its role may now invoke <prefix>-worker and read the settings table, limited to the SYSTEM partition by a LeadingKeys condition. The same check therefore runs inside the build too.
  • Docs: updates.md and model-providers.md describe the CLI check and the skip rule.

Verification

  • Installer npm test: 97 pass. The new tests cover:
    • models.json is fetched only after verify-blob passes, and never when the signature fails.
    • A file changed after signing is refused.
    • With gh, models.json needs its own attestation.
    • Nothing is read without a completed signature check, and another release's list is refused.
    • A refused model the release calls stops the update before deploy, with the fix and link, even with --skip-model-check.
    • A pass deploys.
    • Skip works only for a model the release doesn't call, and the result note records it.
    • A model the config names is required too.
    • A worker that doesn't answer stops the update.
    • The worker payload carries the release's Bedrock ids and the probe team.
    • No marketplace, agreement or use-case command ever runs.
    • A signed release without a model list, and a checkout, skip the check with no AWS calls.
  • infra npm test: 64 pass, including the build role's new worker invoke and SYSTEM-only GetItem.
  • python3 -m unittest discover -s lambda/tests and node --test selfhost/launch/*.test.mjs pass. No Rust changes.

coderhelm-selfhost update reads the release's models.json only after the
signature check passes (and, verified with gh, only with its own build
attestation), only when the signed SHA256SUMS lists it and its checksum
matches. The install's worker then tests each model the release calls,
plus the config's own, through the install's provider. A refused model
stops the update before anything deploys, with the fix and the console
link. --skip-model-check goes on only when the refused model is one the
release doesn't call. A rollback and a checkout skip the check. The update
build's role may ask the worker and read the SYSTEM setting so the same
check runs there.
@nambok
nambok force-pushed the fix/cli-update-model-check branch from d66d450 to 3414626 Compare October 26, 2026 23:42
@nambok
nambok merged commit 252e1cb into main Oct 26, 2026
8 checks passed
@nambok
nambok deleted the fix/cli-update-model-check branch October 26, 2026 23:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant