feat(agent-org): add ordered revision events and incremental run sync #634

Summary

Add a durable, ordered Revision Event layer for Agent Org Runs so the existing shared Run View can apply bounded incremental changes and recover missed updates by replay.

This issue builds on #272, #373, #419, and #424. It does not replace the current recovery, finality, or Snapshot architecture.

Current develop baseline

As of the 2026-08-01 architecture review, Agent Org is already event-assisted:

• canonical Run, Task, Inbox, Approval, Session, and Recovery facts live in SQLite;
• committed mutations publish the lightweight agent_org:run_changed invalidation;
• Root, Member, Kanban, and Monitor consumers share one agentOrgRunViewStore per Run;
• live changes are coalesced before one compact Snapshot refresh;
• visible non-terminal Runs use a 60-second Snapshot fallback for missed pushes;
• zero-subscriber cache entries are evicted after 30 seconds.

The missing capability is not basic event-driven invalidation. The missing capability is a durable, per-Run order and replay boundary. The current push is ephemeral, contains no exact change order, and cannot replay notifications missed during disconnect or restart.

The existing work_revision remains finality evidence only. The in-memory per-session SessionEvent store remains the rendered transcript projection only. Neither should be overloaded as the Agent Org domain revision stream.

Proposed architecture

Canonical tables remain authoritative. Revision Events are small, durable descriptions of committed changes.

flowchart LR
    M["Canonical mutation"] --> TX["One SQLite transaction"]
    TX --> STATE["Update Run / Task / Inbox / Approval / Session fact"]
    TX --> EVENT["Append bounded event at revision N"]
    STATE --> COMMIT["Commit"]
    EVENT --> COMMIT
    COMMIT --> HINT["Publish latest revision hint"]
    SNAP["Snapshot @ revision R"] --> STORE["Existing shared Run View Store"]
    HINT --> STORE
    STORE -->|"Gap"| REPLAY["Replay after last applied revision"]
    REPLAY --> STORE

Persistence

• Add agent_org_runs.state_revision INTEGER NOT NULL DEFAULT 0.
• Add agent_org_run_events keyed by (org_run_id, revision).
• Include event_id, schema_version, typed kind, entity identity, optional causation/actor identity, bounded payload, and timestamp.
• Allocate the next revision and append events with the caller's existing connection and business transaction.
• If event insertion fails, the canonical mutation rolls back.
• Publish the highest committed revision only after commit.
• Existing Runs start at revision zero and stay in Snapshot compatibility mode until the declared stream version has complete mutation coverage.

The Run row owns the cursor; do not add a second cursor table that can diverge from Run lifecycle.

Initial event kinds

• run.status_changed
• run.summary_changed
• task.created
• task.updated
• task.deleted
• inbox.inserted
• inbox.read
• inbox.resolved
• approval.changed
• member.runtime_changed
• intervention.changed

Polling ticks, Snapshot reads, Watchdog inspection, coalesced Wake requests, and no-op turns do not create events. Backpressure is a transport/resync condition, not a fake business mutation.

Snapshot, hints, and replay

1. Load one consistent compact Snapshot with snapshotRevision and eventStreamVersion.
2. Attach the existing shared store to the Run's live hint subscription.
3. Request bounded replay after snapshotRevision.
4. Deduplicate replay/live overlap by (runId, revision).
5. Apply only a continuous sequence.
6. On a gap, replay the missing range before later revisions.
7. On unsupported schema, unavailable history, or malformed payload, preserve current UI and reload a Snapshot.

Replay must use keyset pagination with both row and byte limits. Large Plan, TaskOutput, message, transcript, and artifact bodies remain behind existing detail APIs.

The current agent_org:run_changed push remains best effort and should carry only { orgRunId, latestRevision }. SQLite replay, not an unbounded live queue, is the recovery mechanism.

Frontend ownership and lifecycle

Extend the existing agentOrgRunViewStore; do not create a competing Agent Org store.

• one subscription and one replay pipeline per Run, not per component;
• duplicate/old revisions are no-ops;
• gaps pause incremental application and trigger bounded replay;
• one malformed Run is isolated from other Runs;
• hidden windows do not perform high-frequency Snapshot scans;
• terminal Runs stop recurring fallback polling after their final revision is observed;
• zero-subscriber entries retain the existing 30-second grace, then release Snapshot, replay state, subscriptions, and timers;
• reconnect keeps only a scalar watermark and replays once, with bounded backoff.

No implementation may buffer an unbounded Rust or TypeScript event queue, hydrate the complete event lifetime, or place large durable content in event payloads.

Delivery plan

Phase 1 — Durable contract in shadow mode

• Add migration, event envelope, validation, and connection-scoped append helper.
• Instrument all covered Run, Task, Inbox, Approval, Intervention, and member-runtime mutations.
• Keep UI behavior Snapshot-based until mutation coverage is complete.

Phase 2 — Replay and revision hints

• Add Snapshot revision/capability fields and bounded replay API.
• Add the highest committed revision to the existing invalidation push.
• Verify crash-after-commit, disconnect, overflow, restart, and deletion.

Phase 3 — Incremental shared Run Store

• Add ordered application, gap recovery, schema fallback, and per-Run isolation to the existing store.
• Enable only for a complete eventStreamVersion.
• Retain the 60-second Snapshot verifier.

Phase 4 — Measured polling reduction

• Compare incremental projection state with fresh canonical Snapshots.
• Measure replay gaps, fallback rate, divergence, idle CPU, memory, and multi-window behavior.
• Reduce fallback polling only after production evidence meets an agreed threshold.

Non-goals

• Do not reopen terminal Runs.
• Do not implement post-completion Conversation or Follow-up Runs here; see feat(agent-org): support post-completion chat and follow-up runs #635.
• Do not replace canonical tables with event sourcing.
• Do not reuse work_revision or the session timeline EventStore as the domain stream.
• Do not add large payloads or an arbitrary retention TTL.
• Do not remove Snapshot fallback before measured rollout evidence.
• Do not introduce a complete generic Scheduler Outbox in the first implementation.

Acceptance criteria

• Canonical mutation and corresponding events commit atomically.
• Concurrent mutations produce unique, continuous per-Run revisions.
• Snapshot plus replay equals a fresh authoritative Snapshot.
• Duplicate, old, and out-of-order delivery is handled idempotently.
• A crash after commit but before live hint is recovered by replay.
• Replay enforces both row and serialized-byte limits.
• Large content never enters event payloads.
• Unknown schema and malformed payload preserve valid UI and fall back safely.
• Zero-subscriber, hidden, terminal, offline, reconnect, delete, and multi-window lifecycle tests prove bounded CPU, memory, I/O, subscriptions, and timers.
• Pause, resume, approval, Task, Inbox, intervention, member runtime, and finality order remains canonical.
• One malformed Run cannot disrupt another Run or block recovery/mutations.
• Explicit Run deletion cascades the event history.
• Rust/database tests, frontend store/reducer tests, production HTTP E2E, rendered WebDriver E2E, typecheck, lint, circular checks, formatting, and scoped Clippy pass.

Design document

Updated design: docs/architecture-audit-2026-08-01/AgentOrgRevisionEventArchitecture.md.