soul.demarkus.io:6309/plans/watch-agent-messaging.md/v3 active reader meta

WATCH streams and agent-to-agent communication

An active plan: a WATCH verb that holds a QUIC stream open and pushes change events, so agents, caches, and the library react to writes instead of polling. Audience: demarkus maintainers implementing it. Drafted 2026-09-15 as a future plan; made active 2026-09-25 with the gates resolved below. No code yet.

Why now

A review of the transport on 2026-09-25 found that QUIC's original reasons (DESIGN.md: TLS 1.3, migration, multiplexing, 0-RTT, no HTTP) mostly hold over TLS on TCP too or are unused; 0-RTT was never built. The only planned feature that needs QUIC itself is WATCH: one stream per subscription, many on one connection without head-of-line blocking, cancel by stream reset, migration keeping a laptop's subscriptions alive, and a client-opened stream so subscribers need no public address. Shipping WATCH is what makes the transport choice pay. If it stalls, the fallback is a TLS on TCP binding for direct clients.

The WebSub-style SUBSCRIBE with a callback URL in docs/DESIGN.md stays removed (backlog): agents are not servers. SPEC §13 lists Subscriptions as a planned extension; the SPEC section written in PR 2 replaces that entry.

Gates, resolved

Gate Resolution
1. ADR superseding "Verb Set: Complete" PR 2 writes docs/adr/0025-watch-verb.md, mirrored to /adr/
2. Protocol version on the wire Waived for WATCH. A server without it answers bad-request (protocol/request.go:102, handler.go:148), never a hang, so WATCH is additive. Fortification 1 stays a separate 1.0 item. The ADR records the waiver.
3. A consumer where polling hurts Three named consumers, below. Each phase names the one it serves.
4. Connection vs stream error classification PR 1, before any watch code
5. MCP feature freeze Not triggered: no new MCP tool. The library relay is an SSE endpoint on the broker, not a tool.

Consumers

  1. Agent mailbox. Agents publish to /agents/<id>/inbox/<ulid>.md; a harness runs demarkus watch in the background and wakes the agent on a line. Claim by expected version, reply by APPEND, close by ARCHIVE, discover by LOOKUP; WATCH is the only missing piece.
  2. Cache and graph freshness. Long-lived processes (demarkus-mcp per session, the broker's world pool, the library) cache reads and graph nodes. A watch on / invalidates by path instead of by TTL; resync drops the whole cache. Complements ADR 0013.
  3. Library live view. The reading room refreshes an open document when it changes. Browsers cannot speak QUIC, so this runs through a broker SSE relay (PR 9).

Wire shape

Request: WATCH /prefix/ (or an exact document path), request metadata since: <cursor> optional, the usual token.

Response: a stream of frontmatter blocks, no bodies.

  • First block: status: ok, cursor (current head).
  • Event block: cursor, path, version, hash, op (publish, append, archive). Events are hints; clients fetch by hash for content.
  • Heartbeat block: status: ok, cursor, every 20s while idle. Lets the client detect a stalled stream and keeps the cursor fresh; below the default 25s keepalive and any sane idle timeout.
  • Terminal blocks end the stream: resync (cursor unknown or too old), unauthorized (token revoked or expired), closing (server drain).

Cursor: opaque <epoch>:<seq>. At least once, ordered within an epoch. On resync the client rebuilds from LIST and VERSIONS and resumes from the returned cursor.

The existing ParseResponse reads one response per stream; WATCH needs an incremental block reader in protocol/ with the same size limits per block. The codec lives in protocol/ (ADR 0014: the module is wire and stored format).

Implementation order

One PR per row. Each lists its proof; a PR is not done until the proof test exists and bash pre-commit.sh passes.

PR Scope Proof
1 Client error classification. Split isTransientError (client/fetch/fetch.go:484) into connection-level (idle timeout, application close, handshake, dial) which evicts, and stream-level (stream reset, one request's deadline) which retries on a new stream and keeps the connection. evict's drain path stays for connection failures. A FETCH whose stream is reset leaves the pooled connection in place; a second request on it succeeds. A connection close still evicts.
2 Protocol. ADR 0025, SPEC section, VerbWatch, status values resync and closing, streaming block codec, cursor type. Update /architecture/protocol.md verb table. Codec round trip with fuzzing; oversize block rejected; an old server answers bad-request.
3 Change hub server/internal/changefeed, no store deps. Publish, subscribe by prefix and cursor, ring buffer for resume, bounded queue per subscriber; a full queue ends that subscriber with resync. The write path never blocks. Contiguous sequence under concurrent publishers; slow subscriber gets resync while a fast one keeps receiving; ring overflow on resume gives resync.
4 Single-server serving. A backend.Store decorator (the writepolicy.Enforce pattern, backend.go:94) emits after a committed Publish, Append, or SetArchived. File store epoch is random per process start. worldruntime.serveStream (runtime.go:137): release the concurrency slot once subscribed, exempt the stream from the request deadline, charge the rate limiter once at subscribe, add per-world and per-connection watch limits below MaxIncomingStreams, send closing on drain. Auth: read on the prefix, each event rechecked against the token's paths, token reload rechecks live subscriptions. Watch then publish: event within 100 ms. Drain sends closing and the wait group finishes. A token scoped to /a/** watching / sees no /b/ events. Watches at the limit get an error status, ordinary requests still served.
5 Client and CLI. fetch.Client.Watch(ctx, WatchRequest) returns an event iterator; each watch read in its own goroutine so unread data cannot stall the shared flow-control window; reconnects with the last cursor; surfaces resync. demarkus watch mark://host/prefix/ prints one tab-separated line per event. Consumer 1 end to end on a local soul: session A publishes to an inbox, session B's demarkus watch prints it within 100 ms; kill the server, restart, client gets resync and continues. A FETCH stream reset while a watch is open leaves the watch delivering.
6 Cache invalidation. demarkus-mcp and the broker world pool hold one watch per world while their cache is non-empty; events invalidate by path, resync clears. Shared in the fetch client cache, not per surface. Publish through one session, a read through another session's MCP returns the new version with no TTL wait.
7 Knowledge server. Head poller per world while it has subscribers; epoch is the world ID, sequence the head sequence, so cursors survive restarts. Head receipts carry no paths (bucketstore/model.go:97), so the poller diffs old and new root snapshots per path and emits in sequence order, coalescing several commits to the latest version per path. A replica emits its own commits immediately; the hub drops sequences it already emitted. Two replicas: a write through A reaches a watcher on B within one poll interval; restart B, the watcher resumes without resync; a cursor older than the ring gets resync.
8 Mailbox skill. Plugin skill for the inbox layout, claim by expected version, ack by archive, demarkus watch under a harness monitor. No core change. Two agents split a task list with no double claims.
9 Broker SSE relay and library live view. The broker exposes an authenticated SSE endpoint per world and prefix, fanning one upstream watch out to many browser sessions; the library subscribes for the open document. An edit in one browser session updates another's open document; broker holds one upstream watch per world and prefix regardless of browser count.

Risks

  • Dumb-server drift. The server now holds subscriptions. Kept small: events are hints, no durable subscriber state, overload degrades to resync.
  • Store as a message bus. Every mailbox message is a stored version. The skill says documents are for durable coordination; chatter belongs elsewhere.
  • Bucket poll cost. Object reads per world per interval, paid only while subscribers exist. PR 6 makes long-lived watches common, so measure the poll cost before PR 7 ships with cache invalidation on.
  • Silent misses. A dropped event looks like quiet. Cursors make gaps detectable; tests assert contiguous sequences; heartbeats carry the cursor.
  • UDP reachability. Direct watchers inherit QUIC egress limits. Browsers and blocked networks go through the broker relay.

Open questions

  • Should a subscription accept LOOKUP filter predicates (tag=inbox) so agents watch a subject rather than a path?
  • Should events carry changed section anchors, so a watcher fetches one section instead of the whole document?
  • Should the file store persist its sequence so cursors survive restarts? Without it, every local server restart forces a cache-wide resync in PR 6.
  • Heartbeat interval: fixed 20s, or advertised by the server in the first block?
trail
  1. soul.demarkus.io:6309 v3