Plan: The engine, not the wire
Audience: demarkus maintainers. Repositions demarkus as a memory and knowledge engine for agents, demotes QUIC from "the protocol" to one binding of the store contract, settles the identity door as MCP over HTTPS, freezes transport work that has no shipping consumer, and names the fortification work the engine needs. Drafted 2026-10-08 in an idea session with Fritz; proposed, no code. Revises On the Protocol as the Permanent Layer and extends Fortify the durable core.
Premise
The word "protocol" has been covering three layers that now pull in different directions.
| Layer | What it is | Who uses it | Verdict |
|---|---|---|---|
| Product | memory for agents (soul), shared knowledge for orgs, promotion edge, write-time gates, recall | every user, through MCP plugins | this is what the project became |
| Contract | eight verbs, frontmatter metadata, immutable versions with a hash chain, LOOKUP catalog, WATCH cursors, grants | every binding | load-bearing; it is the memory model |
| Transport | QUIC on 6309, bearer listener on UDP 443, SNI routing, planned h3, WebTransport, 0-RTT, HTTPS adapter | server to server, replicas, CLI, souls | thin consumers, high tax |
Since ADR 0032 the MCP gateway dispatches to local worlds in process with no stream and no bytes. Every human and every LLM host reaches the knowledge system through MCP. The wire is left with machine edges: cross-cluster reads through the pool, replica hints, the CLI, souls on demarkus-server, and the bearer listener of ADR 0033, built for CLI, SDK and live views that do not ship against it yet.
What the engine offers that a plain MCP memory server does not is the contract, not the socket: versioned immutable markdown with declared metadata, two tiers with a promotion edge and gates, federation across worlds and servers, a catalog that never reads bodies, and change hints for sync without a search engine.
Two doors, by credential kind
Settled 2026-10-08. One credential per protocol, not one per port.
| Door | Protocol | Credential | Who | Port |
|---|---|---|---|---|
| Machines | raw mark:// over QUIC |
capability token | server to server, replicas, souls, operators, the plugin binary | 6309, inside the cluster or over a private network |
| People and agents | MCP over HTTPS | OAuth identity (Google login, broker token) | every plugin host, browsers, SDK apps that want identity | 443 at the load balancer |
Consequences:
- HTTP/3 is a load balancer setting, not server work. The GKE load balancer negotiates h3 with clients and speaks h1 or h2 to the pod. MCP on UDP 443 costs no code.
- WATCH reaches browsers and hosts through MCP's streamable HTTP as server-sent events. Hints are small; WebTransport is not needed. This is the roadmap entry Reaching MCP clients.
- Identity on the raw
markwire is unnecessary. The bearer listener of ADR 0033 is removed from the branch before merge, not shipped dark: dark code with no path to live is debt. ADR 0033 moves to superseded by ADR 0037. - The CLI as yourself: if ever needed, the CLI speaks MCP to the gateway with the device flow, like every other identity client. Not scheduled; the CLI is an operator tool with capability tokens. A broker-issued short-lived capability token (the old Remote token issuance idea) is the fallback if MCP in the CLI proves too odd; it costs re-adding a mint path ADR 0032 deleted.
- Owning the wire matters inside the cluster. Outside, HTTP wins, and that is fine.
Decision rules
- The durable-core rule stays: would this still matter if every product on top were thrown away? It applies to the contract.
- New rule for transport: no transport work without a named consumer that ships in the same quarter. "Live views will want it" is not a consumer.
- The existing revisit clause in Verb set and wire format stands: if the live layer stalls or QUIC shows no gain on most hops, the fallback is a TLS on TCP binding, by ADR.
Workstreams
1. Reposition in the soul and the repo
- ADR 0037: the contract is the protocol; QUIC, MCP and the in-process exchange are bindings of it; two doors by credential kind. Supersedes the "Transport is QUIC by design" framing and ADR 0033; keeps ADR 0014 (the
protocolmodule is the wire and stored format) as the contract's home. - Rewrite the core principle in Architecture: the librarian stays, the bookshelf is an engine with three bindings and two doors.
- Rewrite the transport section of Verb set and wire format as "Bindings": in-process, QUIC, MCP, with what each owns.
- README and the pages branch lead with memory and knowledge, not
mark://. Positioning line from 2026-08-29 holds: storage engine for agent memory, brokers are the access tier, MCP is the driver. - Proof: a new reader of the README can say what demarkus is without the word QUIC.
2. Freeze transport
- Lean into QUIC: workstreams 2 (UDP 443 for raw mark), 7a (h3 on the broker) and 7b (WebTransport) are closed by the two-doors decision; 3 to 6, 8, 10 and 11 move to parked; 1 (measure) stays only if the baseline is cheap. Workstream 9 is done by ADR 0032.
- Bearer listener (ADR 0033): removed from
feat/broker-in-processbefore merge. The SNI routing and verifier reuse it added stay where the gateways already use them. - demarkus-sdk: keep, narrowed to the license fix for demarkus-library and a Go driver for capability-token clients. Identity clients use MCP. Other languages stay unscheduled.
- Realtime demarkus: level 1 and the fan-out are shipped; the next level is WATCH hints over MCP streamable HTTP, gated by the live-views decision in workstream 5.
- Protocol freeze after WATCH and ALPN
mark/1(declared 2026-09-27) stands. No ninth verb. - Proof: the active plans hub carries no transport plan without a consumer line.
3. Fortify the memory tier (soul)
- Recall quality: measure before improving. Reuse the leaderboard adapter shape as a local harness: a fixed set of recall questions against a production-shaped soul, hit rate on
mark_lookuptop 3, tokens spent to a correct fetch. Lives onbenchmarks; results in/research/. - Promotion edge: the
promotecascade's dedup and refresh legs get the same failure-line treatment as the skills convention demands; measure promoted versus staged ratio. - Retention and purge: Version retention and Purge are engine work and stay first class.
- Plugin parity: Claude Code, Cursor, OpenCode, Cowork expose the same recall and record behaviour; one prompt source, one gate set.
- Proof: a recall harness number that moves release to release.
4. Fortify the knowledge tier
- Store correctness under load: Bucket store commit log, its GCS gate and production rollout, the compactor off the writers, the memory wall. These are the engine; they keep their order.
- Provenance and supersession: durable-core fortifications 2, 3 and 4 (client-verifiable history,
superseded-byon read,valid-fromandvalid-until) are the properties that make the curated layer worth more than raw sources. Pull them forward. - Ingestion: Knowledge ingestion through one curation gate, with dedup across worlds.
- Self-host without GCS: the appliance story is the next lever named by ADR 0032; a file-backed knowledge server with the in-process broker.
- Proof: durable-core items 2 to 4 shipped; one self-hosted knowledge system on a single VPS with SSO.
5. Decide live views
Live views (reading room, realtime library) are the one product feature that needs change hints in a browser. With the two-doors decision they ride MCP streamable HTTP, so the question is priority, not transport. Decide: is it a 2026 goal or not? If yes, WATCH over MCP is the next realtime level. If no, the library reads through MCP without hints.
Sequencing
- Workstream 1 (reposition) and the live-views decision first; both are writing and one conversation.
- Workstream 2 (freeze) follows the ADR: remove the bearer listener from the branch, status lines on the parked plans.
- Workstreams 3 and 4 run as the ordinary roadmap, with the recall harness as the first new item.
Non-goals
- Ripping QUIC out or adding an HTTP verb mapping. The rejected bindings stay rejected; MCP is the HTTP-shaped door and a TLS on TCP binding for raw mark is the recorded fallback only.
- A ninth verb or a wire change.
- Dropping the SDK; the library license conflict is real.
Risks
- Live views lose momentum if the decision in workstream 5 slips; name a date.
- Removing the bearer listener discards a built step of ADR 0032's branch. Accepted: a verifier call per request on an internet-facing UDP port with no dialer is worse than deleted code in history.
- The CLI has no identity path. Accepted while it is an operator tool; the MCP device flow is the answer if that changes.
- "Engine" positioning can read as a database pitch. Keep the agent-memory framing in front; the engine is the how.
Open questions
- Is demarkus-library a product or a demo? It decides the SDK and live-views priority.
- Public word: engine, store, or memory system? The 1.0 concept model settled soul, memory and knowledge; the thing underneath still has no one name.