# Completed Plans Archive of implementation plans that have been executed and shipped. ## Content Addressing — COMPLETED ✓ **Completed:** March 2026 (Phase 3) Hash-based fetch, in-memory index, mirror foundation. All five implementation steps completed: 1. `content-hash` added to FETCH responses — SHA-256 of stripped body in metadata 2. Hash index in Store — `BuildHashIndex()`, `LookupHash()`, `UpdateHashIndex()`, `RemoveHashEntry()` 3. Index updates on writes — `Write()`, `SetArchived()` keep index synchronized 4. Hash-based FETCH — `isHashPath()` validates `/sha256-<64hex>`, `handleFetchByHash()` retrieves by hash 5. Startup initialization — `BuildHashIndex()` called on server startup **Key details:** Content-hash is separate from etag (stripped body vs full doc). Current versions only indexed. Read auth checked after hash resolves to real path. In-memory index rebuilt on startup. **Foundation for:** Federation, content-addressed mirroring, distributed caching --- ## Federation — COMPLETED ✓ **Completed:** March 2026 (Phase 3) Agent-driven hash discovery via MCP tools. Zero server changes, zero new verbs. **Shipped:** - `mark_index` MCP tool — crawls source server, collects hashes, publishes index to hub - `mark_resolve` MCP tool — resolves content by hash using hub index - `client/internal/index` package — Parse, Build, Merge for markdown hash index documents - Manifest check enforced by tool, `force` override, `dry_run` mode, 1000 doc cap --- ## Persistent Graph — COMPLETED ✓ **Completed:** March 2026 (Phase 4) Disk-backed graph store, incremental crawl, backlinks. **Shipped:** - `client/internal/graphstore` package — nodes, edges, etags, timestamps, atomic writes, schema versioning - `CrawlAndPersist` — unified crawl + merge + save, nil-safe, shared across CLI/TUI/MCP - `mark_backlinks` MCP tool — reverse link lookup - Graph export — `Store.Export()` renders as publishable markdown, `ParseExport()` parses back - TUI graph view — Links (BFS), Backlinks, Topology sub-views - Graph seeding — TUI loads instantly from stored graph while crawl runs in background --- ## Read Auth (Server-Side) — COMPLETED ✓ **Completed:** 2026-03-14 (Phase 5) Per-path read token enforcement on the server. Fully backwards compatible — no read tokens = everything public. **Shipped:** - `RequiresReadAuth(path)` on `TokenStore` — pre-computed `readPaths` at load time - `authorizeRead` handler helper — checks token store, exempts `/.well-known/agent-manifest.md` - Integrated into FETCH, LIST, VERSIONS handlers - Content-addressed fetch respects read auth (hash resolves to path first, then checks auth) - Versioned path auth — `/doc.md/v2` checks auth on base path `/doc.md` - Directory path normalization — `/private` and `/private/` both match `/private/**` patterns **Remaining (Phase 5):** Client-side read auth — `fetch.Client` read methods (`Fetch`, `List`, `Versions`) need a token parameter, then CLI, TUI, and MCP need to pass it through. The server enforces correctly; the clients just can't send a token on reads yet. --- ## Conflict-Aware Merge in `mark_publish` — COMPLETED ✓ **Completed:** 2026-05-05 (PR #101 in client/v0.12.25 — diff3 + merge-candidate response; v0.12.26 — default flipped from `"fail"` to `"merge"`) Tool-level diff3 merge that reduces content loss under concurrent writes. The MCP tool produces a structurally-merged candidate body when `mark_publish` hits a version conflict; the agent semantically verifies the candidate and republishes. No wire-protocol changes; merge logic lives entirely in the Go client. **Shipped:** - `client/internal/merge/` package — `Diff3(base, ours, theirs) Result` (in-package implementation, ~250 lines, line-based LCS + hunk walk, no external dependency) + `Candidate(client, path, ours, expectedVersion, meta)` orchestrating publish-or-merge in one shot. - `on_conflict` parameter on `mark_publish`: `"merge"` (default since v0.12.26) returns a merge candidate on conflict; `"fail"` opts out to strict optimistic-concurrency semantics. - Git-style conflict markers in body (`<<<<<<<` / `=======` / `>>>>>>>`) — agents handle natively from training; format lives in client Go code and can be swapped without breaking the wire protocol. - LCS dp table capped at 2M cells (~16 MB) — pathological inputs (1 MiB body of 1-byte lines) fall through to a single-hunk merge rather than allocating gigabytes. - `Candidate` rejects responses where `latest.Version <= 0` to prevent silent "create-only" semantics from re-publishing into nothing. **Key decisions:** - **Tool never auto-publishes a diff3 result.** Always returns the candidate to the agent for semantic verification. Line-disjoint changes are not semantically-disjoint (duplicate bullets, contradictions, list reorder collisions can pass diff3 but corrupt the document). - **No internal retry loop in the tool.** Each `mark_publish` call is one-shot; iteration lives in the agent's natural fetch-modify-publish loop. - **`mark_append` deliberately out of scope.** Append-as-stream is the right primitive there; auto-resolve handles its tiny race window because there is nothing to merge. - **Default flipped from `"fail"` to `"merge"` in v0.12.26** because the `"fail"`-default left naive callers exposed to silent content loss — the exact failure mode the feature exists to prevent. The shape change (`merge-candidate` vs `conflict`) is loud, not silent. **Wire-level impact:** zero. `on_conflict` is an MCP tool parameter handled in the Go client. --- ## Claude Code Plugin — COMPLETED ✓ **Completed:** 2026-04-23 plan landed; demarkus-memory plugin v0.1.1 shipped on the marketplace; SessionStart hook + lazy-spawned server + auto-generated token in active use across every demarkus development session since. One-click marketplace install gives Claude Code users a local, versioned memory layer backed by a spawned `demarkus-server`. Zero core code changes — the plugin is built entirely on existing primitives (DEMARKUS_AUTH env var, `demarkus-token generate`, ALPN-tagged `/health`, `tokens.Resolve`). **Shipped:** - `plugins/claude-code/` plugin tree: `.claude-plugin/plugin.json`, `hooks/postinstall.sh` + `hooks/session-start.sh`, `.mcp.json`, slash commands (`/soul`, `/soul-journal`, `/soul-status`, `/soul-init`, `/soul-context`, `/soul-memory`), `skills/memory/SKILL.md` triggering on remember/save/recall intents, `seed/index.md` template. - `.claude-plugin/marketplace.json` at repo root for `/plugin marketplace add latebit-io/demarkus`. - `postinstall.sh` — detects platform via `uname -sm` (darwin/arm64, darwin/amd64, linux/amd64, linux/arm64), downloads `demarkus-server` + `demarkus-mcp` + `demarkus-token` from GitHub releases, verifies SHA256 checksums against bundled file, installs to `${CLAUDE_PLUGIN_ROOT}/bin/`. - `session-start.sh` — probes existing demarkus server on default port 6310 via ALPN-tagged `/health` (a non-demarkus process can't pass ALPN negotiation, so port-reuse is safe), spawns server detached when absent, generates a `/*`-scoped publish+archive token on first run, writes raw token to `~/.demarkus/soul/.token` mode 600, exports `DEMARKUS_AUTH` before MCP client launches, copies seed `index.md` when content root is empty. - Plugin config at `~/.demarkus/plugin-memory.conf` records the chosen `MODE` (default | isolated | reuse), `SOUL_DIR`, `PORT`. **Key decisions:** - **Lazy spawn, no service manager** (no launchd/systemd in v1). Server persists across sessions; next session reuses it via the ALPN-tagged probe. - **No core code changes.** Every primitive (`DEMARKUS_AUTH`, `/health`, `tokens.Resolve`, the `demarkus-token generate` CLI redirection pattern) already existed. The plugin is pure plumbing. - **Content root at `~/.demarkus/soul/`** by default. All versioning, graph data, tokens, logs live under it. - **Token mode `600`** on `~/.demarkus/soul/.token`. **In active use:** the demarkus development workflow itself uses the plugin as the agent's memory layer — every session this conversation runs against the lazy-spawned server, every journal entry is a `mark_append` via the plugin's MCP wiring, the universe-deployment plan is fetched/republished through it. --- ## Phase 6 — Universe Deployment (mostly complete) Phase 6 (`/plans/universe-deployment.md`) is ~90% complete as of 2026-05-13: Helm charts for server / broker / agent, OIDC token broker binary + chart, kind-tested upgrade-wipe regression, per-service runtime images, OCI Helm chart publish via release pipeline, structured-slog observability with operator-facing schema doc. Remaining: §6.4 (topology examples) and §6.6 (ops-runbook docs), both under reframing per the "ops polish vs knowledge universe" cost/value pushback. Active plan stays at `/plans/universe-deployment.md` until §6.4 + §6.6 are resolved or formally deferred. --- ## Broker MCP Gateway — COMPLETED ✓ **Completed:** 2026-05-23 (Slice 8 / PR #152 merged; full plan v7 closed) Enterprise-facing MCP gateway on the demarkus-broker. Single `/mcp` endpoint exposing the 13-tool demarkus surface to plugin-style agents over JSON-RPC over Streamable HTTP, authenticated by the company SSO (id_token bearer through the PR4 compositeVerifier), with world access-tokens cached broker-side per session and never persisted. Solves the "20 dev teams × N worlds plugin-config grind" + corporate-network UDP-blocking problems in one. Plan archive: `/plans/broker-https-gateway.md` (v7 changelog at the top traces every load-bearing pivot from v1 REST → v7 complete). **Shape:** - HTTPS at the org boundary (`/mcp` over Streamable HTTP per current MCP spec) terminates at the broker; corporate proxies, Ingress, DPI appliances pass it through unchanged. QUIC stays *inside* the cluster, where firewalls don't see it. - Identity is canonical verified email (trim + lowercase, `email_verified=true` required) — same key the broker already uses for `/me/install`, `/tokens`, `AllowConfig`, audit logs. ONE identity dimension across every surface. - World access-tokens minted lazily per (canonical-email, world) via `Issuer.MintFiltered`, cached in-memory with LRU + idle eviction, never persisted. Broker restart drops the cache; next tool call re-mints. Singleflight on concurrent first-call bursts. Retry-on-401-after-mint with exponential backoff absorbs the kubelet→world-Secret propagation lag. - Tool URLs carry the worldName as host: `mark://{worldName}/{path}`. Broker resolves to cluster-internal Service DNS (`..svc.cluster.local:6309` default, overridable via `worlds[].internalAddress`). - **Byte-for-byte markdown proxy.** `mark_fetch` through the broker returns the same body, version, etag, content-hash, and metadata keys as `mark_fetch` against the same world via stdio/direct-QUIC. Pinned by a `formatToolResult`/`formatResult` byte-equal reference test that catches drift before shipping. **Shipped (in order):** - **Pre-Flight 0** (#142, 2026-05-20): hoisted `client/internal/fetch` + `client/internal/merge` to public so `tools/demarkus-broker` could import them across module boundaries. 5 files relocated, 9 consumers re-imported. - **Pre-Flight 1** (2026-05-20): `mark3labs/mcp-go v0.44+` ships `NewStreamableHTTPServer` as a production `http.Handler`; Stream Resumability is the only missing feature and it's already out-of-scope. - **Slice 1** (#143, 2026-05-20, `b90cda6`): foundation — listener + JSON-RPC dispatcher + `initialize` + `tools/list` (13 tool definitions, placeholder handlers) + OAuth metadata (`/.well-known/oauth-protected-resource` RFC 9728 + `/.well-known/oauth-authorization-server` RFC 8414, the latter aliasing the OIDC Discovery handler) + `gatewayAuth` middleware with RFC 6750+9728 `WWW-Authenticate` challenge. - **Slice 2** (#146, 2026-05-21, `7c529d4`): read tools `mark_fetch` / `mark_list` / `mark_versions` + email-keyed `sessionCache` + `worldPool` + singleflight on (email, world) mints + retry-on-401 with exponential backoff (OQ#9 resolution). Proxy-fidelity test pins byte-for-byte parity with local demarkus-mcp. - **Slice 3** (#147, 2026-05-21, `5387701`): write tools `mark_publish` / `mark_append` / `mark_archive`. `dispatchWithAuth` refactor (closure-based, shared retry loop across read + write). Conflict + not-permitted forward verbatim (not as tool errors). - **Slice 4a** (#148, 2026-05-22, `115a09b`): federation reads `mark_discover` + `mark_resolve`. Cross-org candidate skip semantics. Hoisted `client/internal/index` → `client/index`. - **Slice 4b+5** (#149, 2026-05-22, `fa9f86d`): graph-store federation tools `mark_backlinks` + `mark_graph` + `mark_index` + `mark_graph_export` + `mark_graph_publish` backed by an **ephemeral in-memory graph store** (pod-scoped lifetime, re-crawl after restart). Hoisted `client/graphstore` + `client/graph` + `client/links`. Brought all 13 tools to real handlers. Bucket-store-backed persistence parked for the post-broker design window (`/thoughts.md` § "On Bucket Stores"). - **Slice 6** (#150, 2026-05-22, `80a4008`): conflict-aware merge in `mark_publish`. `on_conflict="merge"` candidate flow + default flipped from `"fail"` to `"merge"` to match local demarkus-mcp. `brokerMergeAdapter` captures ctx in the struct (only viable shape against `merge.Client`'s ctx-free interface). - **Slice 7** (#151, 2026-05-22, `277f83f`): chart wiring (server.mcp.* block, worlds[].internalAddress, deployment second containerPort + optional TLS volume, service `mcp` port, networkpolicy ingress port, ingress.mcp.host separate-hostname topology, parallel cert-manager Certificate). New chart README "MCP gateway" section + new `tools/demarkus-broker/MCP-API.md` 13-tool spec. Kind harness `--with-mcp-smoke` stage (builds broker locally, sideloads, installs LOCAL chart, runs RFC 9728 + 8414 metadata + 401-challenge checks). 99 helm-unittest cases green. - **Slice 8** (#152, 2026-05-23, `bc0d5cc`): `/knowledge-join` plugin slash command + URL-validation/slug-derivation script + first shell-test suite under `plugins/claude-code/tests/` (11 cases, python3-mocked broker). Plugin v0.1.2 → v0.2.0. CLIENT pin 0.12.33 → 0.12.36; TOOLS pin 0.1.10 → 0.1.16 (drift catch-up). CI workflow `permissions:` block for `dorny/paths-filter@v3` shipped same PR. **Key decisions worth remembering:** - **Single identity dimension across the broker (canonical verified email).** Pinned in v5 after the alternative (subject-hash-keyed sessions) would have given the broker two identity dimensions to keep in sync. Same canonical email across MCP, `/me/install`, `/tokens`, `AllowConfig`, audit logs. - **Gateway is always on, not a feature flag.** Pinned in v4 after Slice 1 — `MCPConfig.Addr` defaults to `:8081` so pre-gateway YAMLs upgrade silently. The "OAuth-only deployment" shape was never a real product. - **Byte-for-byte markdown proxy.** Pinned as a Non-Negotiable in v4. The broker is a *wire-shape adapter* (HTTP/JSON-RPC ↔ QUIC/demarkus), not a content transformation layer. Federation, content-addressing, and downstream hash-equality all depend on this. - **Separate hostname for the MCP gateway** (`ingress.mcp.host` parallel to `ingress.host`). Avoids `.well-known/*` collisions between OIDC discovery (management API) and OAuth resource metadata (MCP). Independent cert rotation too. - **Retry-on-401-after-mint** (OQ#9 resolution, Slice 2). The kubelet→world-Secret propagation lag means a freshly-minted token can briefly 401 at the world. Broker-side bounded retry with exponential backoff (6 attempts, 250ms → 8s) absorbs the race; the alternatives (sync-wait on world health endpoint, or prewarm during MCP `initialize`) were rejected — the former would have violated the "no demarkus-server changes" Non-Negotiable, the latter would front-load SIGHUPs for worlds the user may never touch. - **Ephemeral graph store, not persistent.** Slice 4b decision — broker is a wire-shape adapter, persistent state belongs elsewhere. Re-crawl after broker restart is the documented operator expectation. Bucket-store-backed persistence parked for post-broker (`/thoughts.md` § "On Bucket Stores"). - **No in-line PEM mode for the chart's MCP TLS** — `existingSecretRef` only (Slice 7). Cleartext private keys in helm release history are never acceptable, even for dev. - **Slug heuristic = first DNS label** of the broker hostname (Slice 8), not "strip-broker-and-com → org". Matches typical enterprise broker URL shapes (`mcp.broker.acme.com` → `mcp`). For IP literals the heuristic degrades to a number — the slash command tells the user to rename via `claude mcp` if it's bad. **Worth pinning generally:** - **Sprig's `int` is a best-effort cast** (`cast.ToInt`, not `cast.ToIntE`) — `{{ "abc" | int }}` returns 0 with no error. Any chart helper piping a user-controlled string through `int` MUST pre-validate (`regexMatch "^[0-9]+$"`) and range-check before the cast. Otherwise typos render `containerPort: 0` and crash the pod with no breadcrumb. (Caught by CodeRabbit on Slice 7.) - **String-compare on `host:port` is wrong for collision detection.** `:8080`, `0.0.0.0:8080`, `127.0.0.1:8080`, `[::]:8080` all bind the same port but compare as different strings. Extract the port number and compare numerically. (Caught by CodeRabbit on Slice 7.) - **Backgrounded processes inside `$(...)` command substitution inherit stdout fd** — `$()` waits for stdout to close, so the subshell never completes until you kill the backgrounded process explicitly AND wait for its stdout to drain. Always redirect background process stdout to `/dev/null` inside command substitution. (Hit in Slice 8 test runner with a python mock HTTP server.) **Plan archive:** `/plans/broker-https-gateway.md` stays in place as the architectural reference and decision trail. The v1-v7 changelog at the top traces every load-bearing pivot. **Universe-onboarding rewire:** the original PR6/PR7/PR8 of that plan were absorbed here — PR6 (`tools/demarkus-join` binary) canceled in favor of Slice 8's slash command; PR7/PR8 (docs + plugin slash commands) folded into Slices 7-8.