# Journal — 2026-05-31

## demarkus-memory plugin v0.3.0 — self-documenting sessions + LOOKUP recall (#168, merged)

Shipped a plugin update that started as a security-pin bump and grew into a real feature. Merged to main as squash commit `c54927e`.

### What landed
- **Security pins bumped** to the fix-containing releases from yesterday's dep bump: `SERVER 0.17.10→0.17.13`, `CLIENT 0.12.36→0.12.38`, `TOOLS 0.1.16→0.1.28`. The existing version-sentinel drift logic in `ensure_binaries` propagates these to existing installs on next session start. Plugin version `0.2.0→0.3.0`; reconciled the stale `marketplace.json` (was wrongly `0.1.1`).
- **Always-on self-documentation.** New `context/session-guidance.md` is injected every session via a `SessionStart` `hookSpecificOutput.additionalContext` payload. `session-start.sh` now folds all setup output to stderr (`{ … } 1>&2`) so stdout carries only the JSON, and emits it last. Two new `lib.sh` helpers: `json_escape` (portable awk JSON-string encoder, no jq) and `server_health_warning`.
- **Server-down warning.** When the configured memory server isn't reachable, the injected context leads with a ⚠️ so the agent surfaces it instead of silently failing memory calls. Health signal is PID-liveness (managed modes) / adopted-process scan (reuse) — deliberately NOT the port probe, which degrades to permissive without lsof/ss and would false-warn.
- **LOOKUP integration.** `mark_lookup` is now the first recall step in the skill ("what do I know about X") with the catalog-not-full-text caveat. Agent-owned tagging + importance is now instructed across all three write paths — free-form publishes (skill), proactive self-documentation (session guidance), and `/soul-journal` (creation-time `tags`/`importance`, with a re-publish note since `mark_append` carries no metadata). `knowledge-join.md` corrected 13- → 14-tool surface.

### Review + fixes
- CodeRabbit flagged one real issue: `server_health_warning` passed raw `.pid` contents to `kill -0`, which misreads empty/option-like (`-1`)/garbage content as alive. Fixed with a `^[0-9]+$` guard before probing; tested across no-file / empty / option-like / garbage / dead-pid / live-pid.
- The "docstring coverage 50% < 80%" pre-merge warning was a soft check, skipped with reason (the bash helpers already carry header comments; not contorting code for a metric).

### .gitignore bug caught in passing
Fritz noticed `tools/demarkus-token` showed as ignored. Root cause: the unanchored stray-binary entry `tools/demarkus-token` matched the **source package directory** (tools layout puts main packages at `tools/<name>/`, unlike client/server which use `cmd/`), silently ignoring any new untracked file in the package. `main.go` survived only because it predated the rule. First instinct was to delete the line, but Fritz pushed back ("it should build on change no?") — empirically confirmed `cd tools/demarkus-token && go build` does drop a `tools/demarkus-token/demarkus-token` binary, so the protection was real. Correct fix: anchor to the binary path. Extended to `demarkus-publish` and `demarkus-broker`, which had the same latent footgun and were never covered.

### Lesson
"Delete the redundant ignore" was wrong — the artifact it guarded against does get built (just via a different invocation than I first reasoned about). Test the actual build behavior before declaring an ignore rule pointless. Anchoring beats deleting when a stray-binary name collides with a source directory.

### Note
This entry couldn't be tag/importance-stamped: the demarkus-soul MCP `mark_publish` gateway here predates the `metadata` param that #168 (and the earlier LOOKUP work) added to the local `demarkus-mcp`. So the very recall improvement shipped today isn't yet available on this soul's gateway — worth a follow-up to bump the soul server/client to a build that exposes publisher metadata over MCP.


## Soul-doc reconciliation + plugin-store divergence check (later session)

Fritz asked "are we using the new plugin." Surfaced two things worth recording.

### Soul docs lagged the plugin release
`/index.md` (was v40, modified 05-30) and `/roadmap.md` (v37) both predated #168's merge, so they still narrated the plugin as **v0.2.0**. The plugin code itself was consistent at **v0.3.0** (repo `plugin.json` + `marketplace.json` + installed cache all agree) — drift was docs-only. Updated both: index Plugins bullet + Claude Code Plugin version-history block in roadmap now reflect v0.3.0 (#168) with the lookup-wire / self-doc-guidance / pin-bump summary. index → v41, roadmap → v38.

### The session was NOT routing through the plugin's store
Four mark-protocol MCP servers are wired in this Claude Code install:
- `demarkus-soul` → `demarkus-mcp -host mark://soul.demarkus.io` — the remote production soul. **This is what the preflight + all work used** (correct, per CLAUDE.md).
- `plugin:demarkus-memory:demarkus-memory` → wrapper at `…/cache/demarkus/demarkus-memory/0.3.0/…`, points at `mark://localhost:6309` — the plugin's lazy-spawned **local** store.
- `demarkus-hub`, `knowledge` — also connected.

The plugin's local `localhost:6309` store is stale and unrelated: its `/index.md` is v7, modified **2026-05-01**, and it's a hub for **nib / Junto** projects, not demarkus. So the v0.3.0 plugin is installed and live in-session (it injected the SessionStart self-documentation guidance), but its backing store is a different, month-old personal soul — divergent from the demarkus soul where project knowledge actually lives. Two distinct backends; no conflict, but worth knowing they aren't the same memory.

### Resolved a prior-entry worry
The earlier entry noted the soul's MCP gateway predated the `metadata` param so this journal "couldn't be tag/importance-stamped." That's no longer true — today's index/roadmap publishes carried `metadata` (tags + importance) and succeeded, and this journal already shows `tags`/`importance` in frontmatter. The soul gateway now exposes publisher metadata over MCP. Follow-up closed.


## Plan reconciliation — closed 5 stale plans against verified ground truth

Fritz: "I think some of these are done, can we compare and close the ones that are." Listed all 24 `/plans/` docs, then verified the live-looking candidates against actual git/PRs/code (one subagent per plan, read-only). The soul plan docs had badly lagged the late-May broker sprint (#154-#165) and the GKE deploy — several plans the docs described as "in flight / starting / blocked" were in fact fully merged.

Closed (appended a verified Status block to each plan doc + moved Active→Completed in `/index.md` v42):
- **lookup-verb** — #166 merged; only stale `draft` frontmatter.
- **broker-stable-mint** — #158/#159/#163/#164/#165. Lazy per-world provisioning, cache-stable retries (mint cascade structurally impossible), dead `DefaultToken` knobs gone. Design simplified to one long-lived per-world write token (beyond plan, deliberate).
- **broker-deadcode-cleanup** — #159 (unblocked) + #164 (`f9a24e9`). Every targeted symbol (`mcp_session.go`/sessionCache, `/tokens` API, `issuer.go`, sweeper trim) gone; `go build` clean.
- **universe-onboarding** — CLOSED. PR6 (`tools/demarkus-join`) canceled, PR7/8 absorbed into the MCP Gateway (#151/#152). The plan body's "all 8 PRs unstarted" was frozen at v1; the index was the correct record. Only doc debt remains.
- **knowledge-system-gke-deploy** — COMPLETE except the deferred launch blog. Verified against the live deploy repo (template, pushed 2026-05-30) *and* a live RFC 8414 response from `knowledge.demarkus.io` — the actual Phase 7 acceptance test passing. Backups pivoted restic→GCS to CSI VolumeSnapshots (documented).

Left OPEN with a status correction (not closed):
- **broker-auth-code-grant** — core RFC 6749 + PKCE grant SHIPPED and tested (#155/#156), but the plan's PR3 (auth-code kind-smoke stage + a live `/knowledge-join` journal verification) was never done. `deploy/kind/up.sh` still drives only the device-flow path. Decision pending: complete PR3 or descope.

Also added **versions-sharding** to the index Active list — a fully-specced, unstarted server storage change (per-doc `versions/<doc>/vN` + lazy migration) that had been missing from the index entirely.

### Lesson
The index/roadmap drift in only ~4 days because plan docs are written before merge and never re-touched after. The reliable signal is `gh pr list --state merged` + grepping for the symbols a plan promised to add/remove — not the plan's own prose. Worth a periodic "reconcile plans vs PRs" pass.


## broker-auth-code-grant PR3 — closed the kind-smoke gap

Fritz: "close the gap with the kind smoke test." Added an auth-code + PKCE end-to-end stage to `deploy/kind/up.sh`'s `--with-mcp-smoke` path. Read the actual broker handlers first (`oauth_authorize.go`, `device.go` `deviceTokenAuthCode`, `discovery.go`, `server.go` routes) rather than trusting the plan prose — confirmed the OAuth surface is on `:8080` (not the `:8081` MCP gateway), token endpoint is `POST /device/token` with the `grant_type` switch, and the callback dispatches to `authCodeCallback` on a non-empty `AuthCodeID`.

New `authcode-smoke` curl pod: `/oauth/authorize` (S256) → mock IdP auto-approve → `/auth/callback` (state cookie replayed) → 302 to loopback `redirect_uri` with broker code (assert state echoed + `iss`) → negative wrong-verifier `400 invalid_grant` → positive `200` Bearer → replay `400`. The negative-before-positive ordering is deliberate: `Redeem` preserves the one-shot code on a verifier mismatch, so the wrong-verifier check proves PKCE enforcement without burning the code.

Two gotchas the code-read caught:
- `values-broker.yaml` (the `--with-mcp-smoke` values) has no `insecureCookies`, unlike the Stage-4 `values-broker-argo.yaml`. The state cookie is `Secure` by default, so it would not replay over the pod's plain HTTP and the callback would fail at state validation. Set `server.insecureCookies=true` via `--set` (scoped to the smoke install, leaving plain Stage 2 alone).
- PKCE challenge must be `base64url-nopad(sha256(verifier))`; computed host-side with openssl (the curl pod has no openssl) and cross-checked the pipeline against a Python reference — exact match.

Live read-only probe of `broker.knowledge.demarkus.io` confirmed the prod surface: discovery advertises the grant, missing `client_id` → `400 invalid_request`, `response_type=token` → 302 `error=unsupported_response_type` (the new redirecting handler, not the old direct stub), valid PKCE → 302 to Google. The original stub is gone in prod.

`bash -n` clean, `pre-commit.sh` green (shell-only). Not executed in a live kind cluster (needs image build + spin-up) — left for Fritz. Change uncommitted per the no-commit-on-his-behalf rule. Plan doc updated; index flip to Completed deferred until commit.

### Lesson
Reading the handlers before writing the harness paid for itself twice (the `:8080`/`:8081` split and the Secure-cookie trap). A smoke test written against the plan's description instead of the routes would have failed in the cluster with a confusing state-validation error.


## auth-code kind smoke — ran green in a real cluster

Fritz committed the `up.sh` change (`dc7e50e` on `broker-auth-code-kind-smoke`) and asked me to run it. `deploy/kind/up.sh --with-broker --with-mcp-smoke` reused the existing `knowledge-system` kind cluster, rebuilt the broker from the committed checkout, sideloaded, installed the local chart with `insecureCookies`, and the new auth-code + PKCE stage passed all four assertions (authorize→code, negative PKCE, positive mint, one-shot replay) plus the three MCP-gateway checks. Exit 0 in ~57s (cluster already up; image build was the bulk).

Confirmed two things the design depended on but I couldn't verify statically: (1) the Secure-cookie fix works — the state cookie replayed over plain HTTP on the callback leg; (2) `refreshStore.Issue` is wired in Stage 2, so the auth-code grant mints a real Bearer without the Stage-4 argo/world fan-out. The negative-before-positive ordering also held in practice — the wrong-verifier `400` didn't consume the one-shot code.

The plan's PR3 kind-smoke deliverable is now implemented and verified live; ready to close on merge.