soul.demarkus.io:6309/plans/engine-not-wire.md/v2 draft reader meta

Plan: The engine, not the wire

Audience: demarkus maintainers. Repositions demarkus as a memory and knowledge engine for agents, demotes QUIC from "the protocol" to one binding of the store contract, freezes transport work that has no shipping consumer, and names the fortification work the engine needs. Drafted 2026-10-08 in an idea session with Fritz; proposed, no code. Revises On the Protocol as the Permanent Layer and extends Fortify the durable core.

Premise

The word "protocol" has been covering three layers that now pull in different directions.

Layer What it is Who uses it Verdict
Product memory for agents (soul), shared knowledge for orgs, promotion edge, write-time gates, recall every user, through MCP plugins this is what the project became
Contract eight verbs, frontmatter metadata, immutable versions with a hash chain, LOOKUP catalog, WATCH cursors, grants every binding load-bearing; it is the memory model
Transport QUIC on 6309, bearer listener on UDP 443, SNI routing, planned h3, WebTransport, 0-RTT, HTTPS adapter server to server, replicas, CLI, souls thin consumers, high tax

Since ADR 0032 the MCP gateway dispatches to local worlds in process with no stream and no bytes. Every human and every LLM host reaches the knowledge system through MCP. The wire is left with machine edges: cross-cluster reads through the pool, replica hints, the CLI, souls on demarkus-server, and the bearer listener of ADR 0033, built for CLI, SDK and live views that do not ship against it yet.

What the engine offers that a plain MCP memory server does not is the contract, not the socket: versioned immutable markdown with declared metadata, two tiers with a promotion edge and gates, federation across worlds and servers, a catalog that never reads bodies, and change hints for sync without a search engine.

Decision rules

  • The durable-core rule stays: would this still matter if every product on top were thrown away? It applies to the contract.
  • New rule for transport: no transport work without a named consumer that ships in the same quarter. "Live views will want it" is not a consumer.
  • The existing revisit clause in Verb set and wire format stands: if the live layer stalls or QUIC shows no gain on most hops, the fallback is a TLS on TCP binding, by ADR.

Workstreams

1. Reposition in the soul and the repo

  • ADR 0037: the contract is the protocol; QUIC, MCP and the in-process exchange are bindings of it. Supersedes the "Transport is QUIC by design" framing; keeps ADR 0014 (the protocol module is the wire and stored format) as the contract's home.
  • Rewrite the core principle in Architecture: the librarian stays, the bookshelf is an engine with three doors.
  • Rewrite the transport section of Verb set and wire format as "Bindings": in-process, QUIC, MCP, with what each owns.
  • README and the pages branch lead with memory and knowledge, not mark://. Positioning line from 2026-08-29 holds: storage engine for agent memory, brokers are the access tier, MCP is the driver.
  • Proof: a new reader of the README can say what demarkus is without the word QUIC.

2. Freeze transport

  • Lean into QUIC: workstreams 2 to 8, 10 and 11 move to parked; 1 (measure) stays only if the baseline is cheap. Workstream 9 is done by ADR 0032.
  • Bearer listener (ADR 0033): merge with the branch, config gated and default off in the chart until a consumer ships. Not removed; the code is written and the SNI and verifier reuse is small.
  • demarkus-sdk: keep, narrowed to the license fix for demarkus-library and a Go driver. Other languages stay unscheduled.
  • Realtime demarkus: level 1 and the fan-out are shipped; levels above wait for the live-views decision in workstream 5.
  • Protocol freeze after WATCH and ALPN mark/1 (declared 2026-09-27) stands. No ninth verb.
  • Proof: the active plans hub carries no transport plan without a consumer line.

3. Fortify the memory tier (soul)

  • Recall quality: measure before improving. Reuse the leaderboard adapter shape as a local harness: a fixed set of recall questions against a production-shaped soul, hit rate on mark_lookup top 3, tokens spent to a correct fetch. Lives on benchmarks; results in /research/.
  • Promotion edge: the promote cascade's dedup and refresh legs get the same failure-line treatment as the skills convention demands; measure promoted versus staged ratio.
  • Retention and purge: Version retention and Purge are engine work and stay first class.
  • Plugin parity: Claude Code, Cursor, OpenCode, Cowork expose the same recall and record behaviour; one prompt source, one gate set.
  • Proof: a recall harness number that moves release to release.

4. Fortify the knowledge tier

  • Store correctness under load: Bucket store commit log, its GCS gate and production rollout, the compactor off the writers, the memory wall. These are the engine; they keep their order.
  • Provenance and supersession: durable-core fortifications 2, 3 and 4 (client-verifiable history, superseded-by on read, valid-from and valid-until) are the properties that make the curated layer worth more than raw sources. Pull them forward.
  • Ingestion: Knowledge ingestion through one curation gate, with dedup across worlds.
  • Self-host without GCS: the appliance story is the next lever named by ADR 0032; a file-backed knowledge server with the in-process broker.
  • Proof: durable-core items 2 to 4 shipped; one self-hosted knowledge system on a single VPS with SSO.

5. Decide live views

The one product feature that needs the wire to reach a browser is live views (reading room, realtime library). Decide: is it a 2026 goal or not? If yes, the bearer listener gets its consumer and WebTransport returns to scope. If no, it stays dark and the library reads through MCP or the SDK. This decision gates workstream 2's parked list.

Sequencing

  1. Workstream 1 (reposition) and the live-views decision first; both are writing and one conversation.
  2. Workstream 2 (freeze) follows the ADR; mostly status lines and a chart flag.
  3. Workstreams 3 and 4 run as the ordinary roadmap, with the recall harness as the first new item.

Non-goals

  • Ripping QUIC out or adding an HTTP verb mapping. The rejected bindings stay rejected; a TLS on TCP binding is the recorded fallback only.
  • A ninth verb or a wire change.
  • Dropping the SDK; the library license conflict is real.

Risks

  • Live views lose momentum if the decision in workstream 5 slips; name a date.
  • Identity on the wire regresses to capability tokens for CLI users until the bearer listener has a consumer. Accepted; the CLI is an operator tool today.
  • Sunk cost in ADR 0033 argues for shipping it live. The counter is that a dark door costs nothing and an open one costs a verifier call per request on an internet-facing UDP port.
  • "Engine" positioning can read as a database pitch. Keep the agent-memory framing in front; the engine is the how.

Open questions

  • Bearer listener: dark by default, or live because the chart already serves UDP 443?
  • Is demarkus-library a product or a demo? It decides the SDK and live-views priority.
  • Public word: engine, store, or memory system? The 1.0 concept model settled soul, memory and knowledge; the thing underneath still has no one name.
trail
  1. soul.demarkus.io:6309 v2