Completed Plans

Archive of implementation plans that have been executed and shipped.

Content Addressing — COMPLETED ✓

Completed: March 2026 (Phase 3)

Hash-based fetch, in-memory index, mirror foundation. All five implementation steps completed:

1. content-hash added to FETCH responses — SHA-256 of stripped body in metadata
2. Hash index in Store — BuildHashIndex(), LookupHash(), UpdateHashIndex(), RemoveHashEntry()
3. Index updates on writes — Write(), SetArchived() keep index synchronized
4. Hash-based FETCH — isHashPath() validates /sha256-<64hex>, handleFetchByHash() retrieves by hash
5. Startup initialization — BuildHashIndex() called on server startup

Key details: Content-hash is separate from etag (stripped body vs full doc). Current versions only indexed. Read auth checked after hash resolves to real path. In-memory index rebuilt on startup.

Foundation for: Federation, content-addressed mirroring, distributed caching

Federation — COMPLETED ✓

Completed: March 2026 (Phase 3)

Agent-driven hash discovery via MCP tools. Zero server changes, zero new verbs.

Shipped:

• mark_index MCP tool — crawls source server, collects hashes, publishes index to hub
• mark_resolve MCP tool — resolves content by hash using hub index
• client/internal/index package — Parse, Build, Merge for markdown hash index documents
• Manifest check enforced by tool, force override, dry_run mode, 1000 doc cap

Persistent Graph — COMPLETED ✓

Completed: March 2026 (Phase 4)

Disk-backed graph store, incremental crawl, backlinks.

Shipped:

• client/internal/graphstore package — nodes, edges, etags, timestamps, atomic writes, schema versioning
• CrawlAndPersist — unified crawl + merge + save, nil-safe, shared across CLI/TUI/MCP
• mark_backlinks MCP tool — reverse link lookup
• Graph export — Store.Export() renders as publishable markdown, ParseExport() parses back
• TUI graph view — Links (BFS), Backlinks, Topology sub-views
• Graph seeding — TUI loads instantly from stored graph while crawl runs in background

Read Auth (Server-Side) — COMPLETED ✓

Completed: 2026-03-14 (Phase 5)

Per-path read token enforcement on the server. Fully backwards compatible — no read tokens = everything public.

Shipped:

• RequiresReadAuth(path) on TokenStore — pre-computed readPaths at load time
• authorizeRead handler helper — checks token store, exempts /.well-known/agent-manifest.md
• Integrated into FETCH, LIST, VERSIONS handlers
• Content-addressed fetch respects read auth (hash resolves to path first, then checks auth)
• Versioned path auth — /doc.md/v2 checks auth on base path /doc.md
• Directory path normalization — /private and /private/ both match /private/** patterns

Remaining (Phase 5): Client-side read auth — fetch.Client read methods (Fetch, List, Versions) need a token parameter, then CLI, TUI, and MCP need to pass it through. The server enforces correctly; the clients just can't send a token on reads yet.

Conflict-Aware Merge in mark_publish — COMPLETED ✓

Completed: 2026-05-05 (PR #101 in client/v0.12.25 — diff3 + merge-candidate response; v0.12.26 — default flipped from "fail" to "merge")

Tool-level diff3 merge that reduces content loss under concurrent writes. The MCP tool produces a structurally-merged candidate body when mark_publish hits a version conflict; the agent semantically verifies the candidate and republishes. No wire-protocol changes; merge logic lives entirely in the Go client.

Shipped:

• client/internal/merge/ package — Diff3(base, ours, theirs) Result (in-package implementation, ~250 lines, line-based LCS + hunk walk, no external dependency) + Candidate(client, path, ours, expectedVersion, meta) orchestrating publish-or-merge in one shot.
• on_conflict parameter on mark_publish: "merge" (default since v0.12.26) returns a merge candidate on conflict; "fail" opts out to strict optimistic-concurrency semantics.
• Git-style conflict markers in body (<<<<<<< / ======= / >>>>>>>) — agents handle natively from training; format lives in client Go code and can be swapped without breaking the wire protocol.
• LCS dp table capped at 2M cells (~16 MB) — pathological inputs (1 MiB body of 1-byte lines) fall through to a single-hunk merge rather than allocating gigabytes.
• Candidate rejects responses where latest.Version <= 0 to prevent silent "create-only" semantics from re-publishing into nothing.

Key decisions:

• Tool never auto-publishes a diff3 result. Always returns the candidate to the agent for semantic verification. Line-disjoint changes are not semantically-disjoint (duplicate bullets, contradictions, list reorder collisions can pass diff3 but corrupt the document).
• No internal retry loop in the tool. Each mark_publish call is one-shot; iteration lives in the agent's natural fetch-modify-publish loop.
• mark_append deliberately out of scope. Append-as-stream is the right primitive there; auto-resolve handles its tiny race window because there is nothing to merge.
• Default flipped from "fail" to "merge" in v0.12.26 because the "fail"-default left naive callers exposed to silent content loss — the exact failure mode the feature exists to prevent. The shape change (merge-candidate vs conflict) is loud, not silent.

Wire-level impact: zero. on_conflict is an MCP tool parameter handled in the Go client.

Claude Code Plugin — COMPLETED ✓

Completed: 2026-04-23 plan landed; demarkus-memory plugin v0.1.1 shipped on the marketplace; SessionStart hook + lazy-spawned server + auto-generated token in active use across every demarkus development session since.

One-click marketplace install gives Claude Code users a local, versioned memory layer backed by a spawned demarkus-server. Zero core code changes — the plugin is built entirely on existing primitives (DEMARKUS_AUTH env var, demarkus-token generate, ALPN-tagged /health, tokens.Resolve).

Shipped:

• plugins/claude-code/ plugin tree: .claude-plugin/plugin.json, hooks/postinstall.sh + hooks/session-start.sh, .mcp.json, slash commands (/soul, /soul-journal, /soul-status, /soul-init, /soul-context, /soul-memory), skills/memory/SKILL.md triggering on remember/save/recall intents, seed/index.md template.
• .claude-plugin/marketplace.json at repo root for /plugin marketplace add latebit-io/demarkus.
• postinstall.sh — detects platform via uname -sm (darwin/arm64, darwin/amd64, linux/amd64, linux/arm64), downloads demarkus-server + demarkus-mcp + demarkus-token from GitHub releases, verifies SHA256 checksums against bundled file, installs to ${CLAUDE_PLUGIN_ROOT}/bin/.
• session-start.sh — probes existing demarkus server on default port 6310 via ALPN-tagged /health (a non-demarkus process can't pass ALPN negotiation, so port-reuse is safe), spawns server detached when absent, generates a /*-scoped publish+archive token on first run, writes raw token to ~/.demarkus/soul/.token mode 600, exports DEMARKUS_AUTH before MCP client launches, copies seed index.md when content root is empty.
• Plugin config at ~/.demarkus/plugin-memory.conf records the chosen MODE (default | isolated | reuse), SOUL_DIR, PORT.

Key decisions:

• Lazy spawn, no service manager (no launchd/systemd in v1). Server persists across sessions; next session reuses it via the ALPN-tagged probe.
• No core code changes. Every primitive (DEMARKUS_AUTH, /health, tokens.Resolve, the demarkus-token generate CLI redirection pattern) already existed. The plugin is pure plumbing.
• Content root at ~/.demarkus/soul/ by default. All versioning, graph data, tokens, logs live under it.
• Token mode 600 on ~/.demarkus/soul/.token.

In active use: the demarkus development workflow itself uses the plugin as the agent's memory layer — every session this conversation runs against the lazy-spawned server, every journal entry is a mark_append via the plugin's MCP wiring, the universe-deployment plan is fetched/republished through it.

Phase 6 — Universe Deployment (mostly complete)

Phase 6 (/plans/universe-deployment.md) is ~90% complete as of 2026-05-13: Helm charts for server / broker / agent, OIDC token broker binary + chart, kind-tested upgrade-wipe regression, per-service runtime images, OCI Helm chart publish via release pipeline, structured-slog observability with operator-facing schema doc. Remaining: §6.4 (topology examples) and §6.6 (ops-runbook docs), both under reframing per the "ops polish vs knowledge universe" cost/value pushback. Active plan stays at /plans/universe-deployment.md until §6.4 + §6.6 are resolved or formally deferred.

Broker MCP Gateway — COMPLETED ✓

Completed: 2026-05-23 (Slice 8 / PR #152 merged; full plan v7 closed)

Enterprise-facing MCP gateway on the demarkus-broker. Single /mcp endpoint exposing the 13-tool demarkus surface to plugin-style agents over JSON-RPC over Streamable HTTP, authenticated by the company SSO (id_token bearer through the PR4 compositeVerifier), with world access-tokens cached broker-side per session and never persisted. Solves the "20 dev teams × N worlds plugin-config grind" + corporate-network UDP-blocking problems in one. Plan archive: /plans/broker-https-gateway.md (v7 changelog at the top traces every load-bearing pivot from v1 REST → v7 complete).

Shape:

• HTTPS at the org boundary (/mcp over Streamable HTTP per current MCP spec) terminates at the broker; corporate proxies, Ingress, DPI appliances pass it through unchanged. QUIC stays inside the cluster, where firewalls don't see it.
• Identity is canonical verified email (trim + lowercase, email_verified=true required) — same key the broker already uses for /me/install, /tokens, AllowConfig, audit logs. ONE identity dimension across every surface.
• World access-tokens minted lazily per (canonical-email, world) via Issuer.MintFiltered, cached in-memory with LRU + idle eviction, never persisted. Broker restart drops the cache; next tool call re-mints. Singleflight on concurrent first-call bursts. Retry-on-401-after-mint with exponential backoff absorbs the kubelet→world-Secret propagation lag.
• Tool URLs carry the worldName as host: mark://{worldName}/{path}. Broker resolves to cluster-internal Service DNS (<name>.<namespace>.svc.cluster.local:6309 default, overridable via worlds[].internalAddress).
• Byte-for-byte markdown proxy. mark_fetch through the broker returns the same body, version, etag, content-hash, and metadata keys as mark_fetch against the same world via stdio/direct-QUIC. Pinned by a formatToolResult/formatResult byte-equal reference test that catches drift before shipping.

Shipped (in order):

• Pre-Flight 0 (#142, 2026-05-20): hoisted client/internal/fetch + client/internal/merge to public so tools/demarkus-broker could import them across module boundaries. 5 files relocated, 9 consumers re-imported.
• Pre-Flight 1 (2026-05-20): mark3labs/mcp-go v0.44+ ships NewStreamableHTTPServer as a production http.Handler; Stream Resumability is the only missing feature and it's already out-of-scope.
• Slice 1 (#143, 2026-05-20, b90cda6): foundation — listener + JSON-RPC dispatcher + initialize + tools/list (13 tool definitions, placeholder handlers) + OAuth metadata (/.well-known/oauth-protected-resource RFC 9728 + /.well-known/oauth-authorization-server RFC 8414, the latter aliasing the OIDC Discovery handler) + gatewayAuth middleware with RFC 6750+9728 WWW-Authenticate challenge.
• Slice 2 (#146, 2026-05-21, 7c529d4): read tools mark_fetch / mark_list / mark_versions + email-keyed sessionCache + worldPool + singleflight on (email, world) mints + retry-on-401 with exponential backoff (OQ#9 resolution). Proxy-fidelity test pins byte-for-byte parity with local demarkus-mcp.
• Slice 3 (#147, 2026-05-21, 5387701): write tools mark_publish / mark_append / mark_archive. dispatchWithAuth refactor (closure-based, shared retry loop across read + write). Conflict + not-permitted forward verbatim (not as tool errors).
• Slice 4a (#148, 2026-05-22, 115a09b): federation reads mark_discover + mark_resolve. Cross-org candidate skip semantics. Hoisted client/internal/index → client/index.
• Slice 4b+5 (#149, 2026-05-22, fa9f86d): graph-store federation tools mark_backlinks + mark_graph + mark_index + mark_graph_export + mark_graph_publish backed by an ephemeral in-memory graph store (pod-scoped lifetime, re-crawl after restart). Hoisted client/graphstore + client/graph + client/links. Brought all 13 tools to real handlers. Bucket-store-backed persistence parked for the post-broker design window (/thoughts.md § "On Bucket Stores").
• Slice 6 (#150, 2026-05-22, 80a4008): conflict-aware merge in mark_publish. on_conflict="merge" candidate flow + default flipped from "fail" to "merge" to match local demarkus-mcp. brokerMergeAdapter captures ctx in the struct (only viable shape against merge.Client's ctx-free interface).
• Slice 7 (#151, 2026-05-22, 277f83f): chart wiring (server.mcp.* block, worlds[].internalAddress, deployment second containerPort + optional TLS volume, service mcp port, networkpolicy ingress port, ingress.mcp.host separate-hostname topology, parallel cert-manager Certificate). New chart README "MCP gateway" section + new tools/demarkus-broker/MCP-API.md 13-tool spec. Kind harness --with-mcp-smoke stage (builds broker locally, sideloads, installs LOCAL chart, runs RFC 9728 + 8414 metadata + 401-challenge checks). 99 helm-unittest cases green.
• Slice 8 (#152, 2026-05-23, bc0d5cc): /knowledge-join plugin slash command + URL-validation/slug-derivation script + first shell-test suite under plugins/claude-code/tests/ (11 cases, python3-mocked broker). Plugin v0.1.2 → v0.2.0. CLIENT pin 0.12.33 → 0.12.36; TOOLS pin 0.1.10 → 0.1.16 (drift catch-up). CI workflow permissions: block for dorny/paths-filter@v3 shipped same PR.

Key decisions worth remembering:

• Single identity dimension across the broker (canonical verified email). Pinned in v5 after the alternative (subject-hash-keyed sessions) would have given the broker two identity dimensions to keep in sync. Same canonical email across MCP, /me/install, /tokens, AllowConfig, audit logs.
• Gateway is always on, not a feature flag. Pinned in v4 after Slice 1 — MCPConfig.Addr defaults to :8081 so pre-gateway YAMLs upgrade silently. The "OAuth-only deployment" shape was never a real product.
• Byte-for-byte markdown proxy. Pinned as a Non-Negotiable in v4. The broker is a wire-shape adapter (HTTP/JSON-RPC ↔ QUIC/demarkus), not a content transformation layer. Federation, content-addressing, and downstream hash-equality all depend on this.
• Separate hostname for the MCP gateway (ingress.mcp.host parallel to ingress.host). Avoids .well-known/* collisions between OIDC discovery (management API) and OAuth resource metadata (MCP). Independent cert rotation too.
• Retry-on-401-after-mint (OQ#9 resolution, Slice 2). The kubelet→world-Secret propagation lag means a freshly-minted token can briefly 401 at the world. Broker-side bounded retry with exponential backoff (6 attempts, 250ms → 8s) absorbs the race; the alternatives (sync-wait on world health endpoint, or prewarm during MCP initialize) were rejected — the former would have violated the "no demarkus-server changes" Non-Negotiable, the latter would front-load SIGHUPs for worlds the user may never touch.
• Ephemeral graph store, not persistent. Slice 4b decision — broker is a wire-shape adapter, persistent state belongs elsewhere. Re-crawl after broker restart is the documented operator expectation. Bucket-store-backed persistence parked for post-broker (/thoughts.md § "On Bucket Stores").
• No in-line PEM mode for the chart's MCP TLS — existingSecretRef only (Slice 7). Cleartext private keys in helm release history are never acceptable, even for dev.
• Slug heuristic = first DNS label of the broker hostname (Slice 8), not "strip-broker-and-com → org". Matches typical enterprise broker URL shapes (mcp.broker.acme.com → mcp). For IP literals the heuristic degrades to a number — the slash command tells the user to rename via claude mcp if it's bad.

Worth pinning generally:

• Sprig's int is a best-effort cast (cast.ToInt, not cast.ToIntE) — {{ "abc" | int }} returns 0 with no error. Any chart helper piping a user-controlled string through int MUST pre-validate (regexMatch "^[0-9]+$") and range-check before the cast. Otherwise typos render containerPort: 0 and crash the pod with no breadcrumb. (Caught by CodeRabbit on Slice 7.)
• String-compare on host:port is wrong for collision detection. :8080, 0.0.0.0:8080, 127.0.0.1:8080, [::]:8080 all bind the same port but compare as different strings. Extract the port number and compare numerically. (Caught by CodeRabbit on Slice 7.)
• Backgrounded processes inside $(...) command substitution inherit stdout fd — $() waits for stdout to close, so the subshell never completes until you kill the backgrounded process explicitly AND wait for its stdout to drain. Always redirect background process stdout to /dev/null inside command substitution. (Hit in Slice 8 test runner with a python mock HTTP server.)

Plan archive: /plans/broker-https-gateway.md stays in place as the architectural reference and decision trail. The v1-v7 changelog at the top traces every load-bearing pivot.

Universe-onboarding rewire: the original PR6/PR7/PR8 of that plan were absorbed here — PR6 (tools/demarkus-join binary) canceled in favor of Slice 8's slash command; PR7/PR8 (docs + plugin slash commands) folded into Slices 7-8.

RFC 7591 DCR Follow-on (Broker MCP Gateway) — COMPLETED ✓

Completed: 2026-05-26 (PR #153, 14728a7)

Small but load-bearing post-merge follow-on to the Broker MCP Gateway. The kind/k8s deploy testing surfaced that Claude Code's native MCP authorization client refuses to proceed against an authorization server whose discovery doc omits registration_endpoint — the MCP authorization spec mandates RFC 7591 Dynamic Client Registration. /knowledge-join device flow was fine (static client_id=demarkus-cli), but the spec-driven OAuth dance was blocked.

Shipped:

• tools/demarkus-broker/internal/broker/register.go — RFC 7591 /register handler. Rubber-stamp by design (no persistence): mints a 128-bit random base64url client_id, echoes request metadata, force-pins token_endpoint_auth_method: "none" and grant_types: ["urn:ietf:params:oauth:grant-type:device_code"], returns 201 with Cache-Control: no-store.
• discovery.go — applyDiscoveryOverrides now rewrites registration_endpoint to <brokerURL>/register. Field appears on both /.well-known/openid-configuration and /.well-known/oauth-authorization-server (shared Discovery.Handler()).
• server.go — routed under the same ipRateLimit middleware as the device-flow POSTs.
• 9 subtests covering shape, echo, broker-field override, distinct IDs, invalid JSON → 400, oversized → 413, GET → 405, Cache-Control header, forced grant_types.

Key decisions worth remembering:

• No client database. Broker has zero client_id enforcement on the device-flow path (device.go:108 accepts any non-empty string). Real authz lives at id_token verification + per-world Allow list. Persistence would be vestigial state until per-client policy (audit, redirect URI checks, per-client rate buckets) becomes a real requirement. The register.go doc comment names the exact moment to flip rubber-stamp into a real handler.
• Force-pin grant_types, not reject. RFC 7591 §3.2.2 says the AS MUST respond invalid_client_metadata for disallowed grants. We normalize because the MCP authorization spec leans authorization_code-first and the broker can't do auth-code anyway (no authorization_endpoint). Force-pinning tells the client what they got rather than failing with "incompatible" at registration time.
• Open registration, no initial_access_token gate. IP rate limiter is the only protection. Acceptable for the demarkus deployment posture (broker behind ingress); if a customer broker grows abuse exposure, gated mode keyed off the /me/install bundle is the natural escalation — same shape, no new infra.

CodeRabbit nits caught + addressed pre-merge:

• Missing Cache-Control: no-store on the 201 — added.
• grant_types echoed caller input unrestricted — force-pinned.
• Empty-body comment overclaimed RFC 7591 §3.1 — tightened.

Verification: broker test suite green (4.677s), pre-commit.sh clean across protocol/server/client/tools, end-to-end Claude Code → cluster broker auth dance confirmed in cluster.