# 2026-05-23

## Broker MCP Gateway — plan complete

Slice 8 (`/knowledge-join` plugin slash command) merged today as PR #152 (`bc0d5cc`). That closes out the Broker MCP Gateway plan — all eight slices + Pre-Flight 0/1 shipped across roughly two weeks. Plan archived to `/completed-plans.md`; the source document at `/plans/broker-https-gateway.md` stays in place as the architectural reference (v7 changelog at the top traces every load-bearing pivot from v1 REST → v7 complete).

### What landed in Slice 8

- `plugins/claude-code/commands/knowledge-join.md` — prompt-shaped slash command; takes a broker URL, invokes the script, parses kv output, runs `claude mcp add --transport http {slug} {mcp-url}`.
- `plugins/claude-code/scripts/knowledge-join.sh` — validates HTTPS scheme (case-insensitive per RFC 3986), fetches `/.well-known/oauth-protected-resource` (RFC 9728) to confirm the URL points at a Slice 7+ broker, derives a slug from the first DNS label of the host (lowercased + sanitized to `[a-z0-9-]+`), emits machine-readable key=value output. Test-only `DEMARKUS_KNOWLEDGE_JOIN_ALLOW_HTTP=1` escape hatch lets the shell tests drive a local http:// mock without TLS plumbing.
- `plugins/claude-code/tests/knowledge-join_test.sh` — first shell-test suite under `plugins/claude-code/tests/` (new directory). 11 test cases: happy path, http:// rejection (with and without escape hatch), missing-scheme rejection, missing-arg rejection, 404 + 500 broker responses, broker unreachable, trailing-slash normalization, slug derivation, port stripping, uppercase scheme accepted + normalized. Mock broker is a tiny python3 `BaseHTTPRequestHandler` configured to return a specific status code.
- `plugins/claude-code/scripts/lib.sh` — pin bumps: `CLIENT_VERSION` 0.12.33 → 0.12.36, `TOOLS_VERSION` 0.1.10 → 0.1.16. `SERVER_VERSION` already at 0.17.10 latest. Catch-up drift, not Slice-8-specific.
- `plugins/claude-code/.claude-plugin/plugin.json` — plugin v0.1.2 → v0.2.0 (new top-level slash command warrants a minor bump).

### CodeRabbit round-1 fixes (3 valid, all addressed)

- **Case-sensitive URL scheme match.** `case "${URL}" in https://*) ;; http://*) ...` rejected `HTTPS://...`. Per RFC 3986 scheme is case-insensitive. Fixed to `[Hh][Tt][Tt][Pp][Ss]://*` / `[Hh][Tt][Tt][Pp]://*` patterns. Used bracketed character classes instead of `shopt -s nocasematch` to keep the case-insensitivity surgical (no surprise effect on later case statements). Plus added scheme-portion lowercasing to the emitted `url=` / `mcp-url=` so downstream `claude mcp list` stays clean. New test pins both the accept path and the lowercase-normalization invariant.
- **MD fences missing language identifier.** Three fences in `knowledge-join.md` got `bash` tags.
- **`test_slug_first_label_lowercased` didn't actually test lowercasing.** Original test invocation used `http://localhost:...` (already lowercase) but had a long meta-comment justifying why it wasn't really testing the lowercasing. Sloppy. Rewrote: invocation now uses `http://LOCALHOST:${port}`, assertion is `^slug=localhost$`. The `tr [:upper:] [:lower:]` step is now actually exercised end-to-end.

### CI workflow GITHUB_TOKEN failure

`dorny/paths-filter@v3` errored with "Bad credentials" on PR #152's checks. The action calls `pulls.listFiles` on `pull_request` events and needs `pull-requests: read`; the repo's default `GITHUB_TOKEN` permissions were restricted (Settings → Actions tightening), so the action couldn't authenticate. Surgical fix: added a job-level `permissions: contents:read pull-requests:read` block to the `detect-changes` job in `.github/workflows/ci.yml`. Other jobs (test-protocol, test-server, test-client, test-broker, test-charts) don't call the GitHub API and inherit the default. Shipped as part of PR #152 since the CI failure was blocking the very PR it appeared in. `release.yml` doesn't have the same issue — it runs on `push: branches: [main]` where paths-filter does a local diff against the previous commit instead of hitting the API.

### Decisions made this session that weren't in the plan

- **Slug heuristic = first DNS label, NOT "strip-broker.-and-.com → org".** Plan's example was "`acme` from `broker.acme.com`" — ambiguous. Chose first-label because typical enterprise broker URLs are `mcp.broker.<org>.com` or `broker.<org>.com` where the meaningful per-deployment identifier IS the first label. IP literals degrade to "127" — ugly but the user-facing output tells the operator to rename via `claude mcp` if the heuristic is bad.
- **Script writes errors to BOTH stdout (as `FAIL: ...`) AND stderr.** Stdout-formatted FAIL lets the slash command surface a clean error to the user; stderr (with `[demarkus-memory] error:` prefix matching `lib.sh::die`) makes the script useful when run directly by a developer. Two surfaces, one canonical message.
- **Test-only HTTPS escape hatch is an env var, not a CLI flag.** `DEMARKUS_KNOWLEDGE_JOIN_ALLOW_HTTP=1` is deliberately long + namespaced + not a CLI flag so a user can't accidentally enable it via shell history or `--help` discovery.
- **Pin bumps include catch-up drift, not just Slice-8 changes.** `feedback_plugin_version_pins.md` says pins move with every plugin change. CLIENT was 3 behind and TOOLS 6 behind — caught up to latest in this slice rather than leaving stale pins for future work to inherit.

### Bugs worth pinning

- **Stdout-fd inheritance deadlock in the test runner.** First version backgrounded the python mock as `python3 - <<'PY' &` inside a test function whose output was captured via `out=$( "test_${name}" )`. The python child inherited the parent's stdout fd; `$()` waits for that fd to close, so the substitution never completed. Cleanup trap fired correctly but `$()` had already captured the output and was waiting. Fix: `>/dev/null 2>&1 &` on the python invocation. The deadlock CANNOT occur in the production plugin path because the script has no `&` anywhere — every operation is synchronous (verified by `grep`). Production curl has `--connect-timeout 5 --max-time 15` so worst-case the script blocks 15s on a hung TLS handshake; bounded latency, no deadlock.
- **`curl -w "%{http_code}" || echo "000"` doubles the "000".** Curl writes "000" to stdout via `-w` on connection-level failure AND returns non-zero exit, so `|| echo "000"` appends another "000" giving "000000". Fixed by dropping the OR and using `${http_code:-000}` parameter-expansion fallback for the rare case where curl produces no output at all.
- **Test expectation for IP-literal slug.** I wrote tests expecting `slug=127-0-0-1` (assuming dots → hyphens via sanitize). Actual: first-label rule gives `slug=127`. Fixed tests to match the script's documented heuristic.
- **Sprig `int` is a footgun (already in 2026-05-22 journal but reinforced).** Slice 7's CodeRabbit round caught this in chart helpers; same lesson applies to any best-effort string-to-int coercion. Always pre-validate.

### Scope outcome vs plan estimate

- Production code: ~125 LOC (script) + ~60 LOC (command markdown). Plan estimate ~50 LOC. Over because the script handles 5 distinct error classifications with operator-readable typed errors per case, and the command markdown spells out per-error suggested fixes.
- Test code: ~200 LOC across 11 cases. Plan estimate ~80 LOC. Same overshoot rationale as Slice 7's tests.
- `bash plugins/claude-code/tests/knowledge-join_test.sh` → 11/11 green. `bash pre-commit.sh` → fmt + vet + golangci-lint × 4 modules, all green (no Go changes; pre-commit verifies nothing else broke).

### What this means

The whole Broker MCP Gateway plan is shipped. From the v1 REST framing through v7 plan-complete, the architectural pivots were:
- v1 → v2: pivot from REST to MCP server (single endpoint, JSON-RPC, agent-as-client is the only first-pass consumer)
- v3: URL shape resolved (`mark://{worldName}/{path}`); join flow becomes a slash command, not a binary
- v4: gateway is always on (not a feature flag); byte-for-byte proxy contract pinned
- v5: session cache pinned to canonical verified email (single identity dimension across the broker)
- v6: plan-vs-code drift documented (WorldTokenTTL + WorldPool never landed); ingress topology pinned to separate hostnames
- v7: plan complete

Documented gaps left as deliberate post-merge work (each parked in Out of Scope with a pointer to its future home):
- **Bucket-store-backed persistent graph store** — `/thoughts.md` § "On Bucket Stores". The graph-store ephemerality is a documented operator expectation, not debt.
- **Full id_token / initialize / tools/call E2E kind-harness stage** — Slice 7's `--with-mcp-smoke` proves chart wiring; a full E2E would exercise `claude mcp add` against a mock OIDC + broker, which requires the slash-command flow in-loop. Worth as a future kind-stage if a customer flags the gap.
- **mTLS broker → world.** The worldPool can grow it without changing the tool surface.
- **Browsable REST surface.** Additive on the same listener (sharing OAuth auth + session cache).

### Worth pinning as a methodology note

Three things worked well across this plan:
1. **Slice-per-PR** with each slice independently mergeable + reviewable. The eight slices averaged ~250 LOC production + ~350 LOC tests each. Reviewer cognitive budget stayed bounded; the proxy-fidelity test (Slice 2) caught one byte-equal drift before Slice 6 even started.
2. **Living plan document with a top-of-file changelog.** Every pivot got a version bump and a changelog paragraph at the top. Reading the v1→v7 changelog now is the fastest way to understand WHY the architecture is what it is, not just WHAT it is.
3. **CodeRabbit round per PR, addressed before merge.** Slices 7 and 8 each had a CodeRabbit round with 3-6 actionable comments; both surfaced real bugs (silent Sprig `int` coercion, missing case-insensitive scheme match) plus minor polish (doc fence languages, comment accuracy). Worth keeping the discipline of addressing them BEFORE merge so the merge commit doesn't carry follow-up debt.

### Next steps

The MCP gateway plan is the second-largest enterprise-shaped piece (after Universe Deployment Phase 6) shipped this year. The natural next architectural design window — per `/thoughts.md` and Fritz's framing — is **bucket-store-backed persistence** as a k8s-native alternative to PVC-backed filesystems. It applies broader than just the broker's graph store: demarkus-server's versioned document store, the broker's issuances Secret, and any future stateful piece all benefit from the same shape. Not urgent now; very interesting later.