Plan: The engine, not the wire

Audience: demarkus maintainers. Repositions demarkus as a memory and knowledge engine for agents, demotes QUIC from "the protocol" to one binding of the store contract, settles the identity door as MCP over HTTPS, says what becomes of the soul server, freezes transport work that has no shipping consumer, and names the fortification work the engine needs. Drafted 2026-10-08 in an idea session with Fritz; proposed, no code. Souls section amended 2026-10-19 with the shape of the merge. Revises On the Protocol as the Permanent Layer and extends Fortify the durable core.

Premise

The word "protocol" has been covering three layers that now pull in different directions.

Since ADR 0032 the MCP gateway dispatches to local worlds in process with no stream and no bytes. Every human and every LLM host reaches the knowledge system through MCP. The wire is left with machine edges: cross-cluster reads through the pool, replica hints, the CLI, souls on demarkus-server, and the bearer listener of ADR 0033, built for CLI, SDK and live views that do not ship against it yet.

What the engine offers that a plain MCP memory server does not is the contract, not the socket: versioned immutable markdown with declared metadata, two tiers with a promotion edge and gates, federation across worlds and servers, a catalog that never reads bodies, and change hints for sync without a search engine.

Two doors, by credential kind

Settled 2026-10-08. One credential per protocol, not one per port.

Consequences:

• HTTP/3 is a load balancer setting, not server work. The GKE load balancer negotiates h3 with clients and speaks h1 or h2 to the pod. MCP on UDP 443 costs no code.
• WATCH reaches browsers and hosts through MCP's streamable HTTP as server-sent events. Hints are small; WebTransport is not needed. This is the roadmap entry Reaching MCP clients.
• Identity on the raw mark wire is unnecessary. The bearer listener of ADR 0033 is removed from the branch before merge, not shipped dark: dark code with no path to live is debt. ADR 0033 moves to superseded by ADR 0037.
• The CLI as yourself: if ever needed, the CLI speaks MCP to the gateway with the device flow, like every other identity client. Not scheduled; the CLI is an operator tool with capability tokens. A broker-issued short-lived capability token (the old Remote token issuance idea) is the fallback if MCP in the CLI proves too odd; it costs re-adding a mint path ADR 0032 deleted.
• Owning the wire matters inside the cluster. Outside, HTTP wins, and that is fine.

Souls

The soul server (demarkus-server: file store, raw mark on 6309, capability tokens, no broker, no bucket) is the engine in its simplest form. For a soul the MCP door is client side: the plugin binary is the MCP server and dials raw mark behind it, so the engine only ever sees a machine with a token. That fits the two doors. One shape does not:

Decisions:

• Hosted souls become memory-broker tenants: a tenant world with Google login on 443. That is what the memory broker exists for. soul.demarkus.io stays a personal instance or retires; demarkus-server never faces the public internet as a product. Nothing forces a personal remote demarkus-server to move; the tenant path is for souls hosted for others.
• The local soul stays demarkus-server today. Nothing changes for the plugin.
• Constraint for the merge: a soul on a laptop outlives the hosted product. The local profile runs with no broker, no bucket, no account and no network. In ADR 0037.

The merge, shaped 2026-10-19 (ADR 0037 decision 6): one runtime, two thin mains, not one binary.

The runtime is already one codepath since ADR 0032 and the memory-model carve-out; the soul main is a thick duplicate of knowledgeserver's wiring, and the knowledge path has no file backend. The merge closes both:

• knowledgeserver gains the file-store opener (the self-host without GCS item in workstream 4), so a knowledge system runs on a VPS without a bucket.
• demarkus-server becomes a local main over knowledgeserver: file backend, one world, one token file, static TLS; it imports none of the right column, and the layer guard in pre-commit enforces that. A Go binary carries what its main imports, so the local binary stays near today's 8 MB stripped against 70 MB for the full binary, with no cloud, OAuth or telemetry init in a laptop process. Build tags only for the GCS opener.
• demarkus-knowledge-server and demarkus-knowledge collapse into the one full binary. Three mains become two.

Plugin compatibility is the gate for the merge. The plugin contract stays byte for byte:

• The MCP tool names and schemas in the plugin binary do not change.
• mark://localhost:6309 stays the local soul address; the default port stays 6309; ALPN stays mark.
• Token discovery stays ~/.mark/tokens.d; an existing token file keeps working.
• The managed-binary launcher keeps launching a binary named demarkus-server; with the thin main there is no rename and no transition release.
• The soul's files on disk are read by the merged binary without migration; the file store format is the stored format of ADR 0014.
• Proof: the plugin's eval suite and /soul-status pass against the old binary and the new demarkus-server with the same config and the same soul directory; the upgrade capture under testdata/upgrade is the same guard in CI.

Decision rules

• The durable-core rule stays: would this still matter if every product on top were thrown away? It applies to the contract.
• New rule for transport: no transport work without a named consumer that ships in the same quarter. "Live views will want it" is not a consumer.
• The existing revisit clause in Verb set and wire format stands: if the live layer stalls or QUIC shows no gain on most hops, the fallback is a TLS on TCP binding, by ADR.

Workstreams

1. Reposition in the soul and the repo

• ADR 0037: the contract is the protocol; QUIC, MCP and the in-process exchange are bindings of it; two doors by credential kind; the local profile constraint. Supersedes the "Transport is QUIC by design" framing and ADR 0033; keeps ADR 0014 as the contract's home.
• Rewrite the core principle in Architecture: the librarian stays, the bookshelf is an engine with three bindings and two doors.
• Rewrite the transport section of Verb set and wire format as "Bindings": in-process, QUIC, MCP, with what each owns.
• README and the pages branch lead with memory and knowledge, not mark://. Positioning line from 2026-08-29 holds: storage engine for agent memory, brokers are the access tier, MCP is the driver.
• Proof: a new reader of the README can say what demarkus is without the word QUIC.

2. Freeze transport

• Lean into QUIC: workstreams 2 (UDP 443 for raw mark), 7a (h3 on the broker) and 7b (WebTransport) are closed by the two-doors decision; 3 to 6, 8, 10 and 11 move to parked; 1 (measure) stays only if the baseline is cheap. Workstream 9 is done by ADR 0032.
• Bearer listener (ADR 0033): removed from feat/broker-in-process before merge. The SNI routing and verifier reuse it added stay where the gateways already use them.
• demarkus-sdk: keep, narrowed to the license fix for demarkus-library and a Go driver for capability-token clients. Identity clients use MCP. Other languages stay unscheduled.
• Realtime demarkus: level 1 and the fan-out are shipped; the next level is WATCH hints over MCP streamable HTTP, gated by the live-views decision in workstream 5.
• Protocol freeze after WATCH and ALPN mark/1 (declared 2026-09-27) stands. No ninth verb.
• Proof: the active plans hub carries no transport plan without a consumer line.

3. Fortify the memory tier (soul)

• Recall quality: measure before improving. Reuse the leaderboard adapter shape as a local harness: a fixed set of recall questions against a production-shaped soul, hit rate on mark_lookup top 3, tokens spent to a correct fetch. Lives on benchmarks; results in /research/.
• Promotion edge: the promote cascade's dedup and refresh legs get the same failure-line treatment as the skills convention demands; measure promoted versus staged ratio.
• Retention and purge: Version retention and Purge are engine work and stay first class.
• Plugin parity: Claude Code, Cursor, OpenCode, Cowork expose the same recall and record behaviour; one prompt source, one gate set.
• Hosted souls onto the memory broker: a migration path from a soul.demarkus.io token to a tenant world, documents and versions intact.
• Proof: a recall harness number that moves release to release; one hosted soul migrated with its history.

4. Fortify the knowledge tier

• Store correctness under load: Bucket store commit log, its GCS gate and production rollout, the compactor off the writers, the memory wall. These are the engine; they keep their order.
• Provenance and supersession: durable-core fortifications 2, 3 and 4 (client-verifiable history, superseded-by on read, valid-from and valid-until) are the properties that make the curated layer worth more than raw sources. Pull them forward.
• Ingestion: Knowledge ingestion through one curation gate, with dedup across worlds.
• Self-host without GCS: a file-store backend for demarkus-knowledge with the in-process broker. This is also the first step of the soul-server merge in the Souls section.
• Proof: durable-core items 2 to 4 shipped; one self-hosted knowledge system on a single VPS with SSO.

5. Decide live views

Live views (reading room, realtime library) are the one product feature that needs change hints in a browser. With the two-doors decision they ride MCP streamable HTTP, so the question is priority, not transport. Decide: is it a 2026 goal or not? If yes, WATCH over MCP is the next realtime level. If no, the library reads through MCP without hints.

Sequencing

1. Workstream 1 (reposition) and the live-views decision first; both are writing and one conversation.
2. Workstream 2 (freeze) follows the ADR: remove the bearer listener from the branch, status lines on the parked plans.
3. Workstreams 3 and 4 run as the ordinary roadmap, with the recall harness as the first new item.
4. The soul-server merge waits for the file-store backend in knowledgeserver, then lands as the thin local main behind the plugin compatibility gate.

Non-goals

• Ripping QUIC out or adding an HTTP verb mapping. The rejected bindings stay rejected; MCP is the HTTP-shaped door and a TLS on TCP binding for raw mark is the recorded fallback only.
• A ninth verb or a wire change.
• Dropping the SDK; the library license conflict is real.
• Changing the plugin's tool surface, address, token discovery or on-disk soul format for the merge.
• One binary for both profiles. Two mains over one runtime is the merge; a single file would only save one release artifact and cost the local profile its size and its audit surface.

Risks

• Live views lose momentum if the decision in workstream 5 slips; name a date.
• Removing the bearer listener discards a built step of ADR 0032's branch. Accepted: a verifier call per request on an internet-facing UDP port with no dialer is worse than deleted code in history.
• The CLI has no identity path. Accepted while it is an operator tool; the MCP device flow is the answer if that changes.
• The thin main drifts back into a thick one. Mitigation: the layer guard forbids the right-column packages in cmd/demarkus-server, and the stripped size of the local binary is the metric to watch.
• "Engine" positioning can read as a database pitch. Keep the agent-memory framing in front; the engine is the how.

Open questions

• Is demarkus-library a product or a demo? It decides the SDK and live-views priority.
• Public word: engine, store, or memory system? The 1.0 concept model settled soul, memory and knowledge; the thing underneath still has no one name.
• Hosted soul migration: does a tenant world accept a bulk import with versions intact, or does the move flatten history to one version per document?