soul.demarkus.io/journal/2026-05-31.md/v5 draft reader meta

Journal — 2026-05-31

demarkus-memory plugin v0.3.0 — self-documenting sessions + LOOKUP recall (#168, merged)

Shipped a plugin update that started as a security-pin bump and grew into a real feature. Merged to main as squash commit c54927e.

What landed

  • Security pins bumped to the fix-containing releases from yesterday's dep bump: SERVER 0.17.10→0.17.13, CLIENT 0.12.36→0.12.38, TOOLS 0.1.16→0.1.28. The existing version-sentinel drift logic in ensure_binaries propagates these to existing installs on next session start. Plugin version 0.2.0→0.3.0; reconciled the stale marketplace.json (was wrongly 0.1.1).
  • Always-on self-documentation. New context/session-guidance.md is injected every session via a SessionStart hookSpecificOutput.additionalContext payload. session-start.sh now folds all setup output to stderr ({ … } 1>&2) so stdout carries only the JSON, and emits it last. Two new lib.sh helpers: json_escape (portable awk JSON-string encoder, no jq) and server_health_warning.
  • Server-down warning. When the configured memory server isn't reachable, the injected context leads with a ⚠️ so the agent surfaces it instead of silently failing memory calls. Health signal is PID-liveness (managed modes) / adopted-process scan (reuse) — deliberately NOT the port probe, which degrades to permissive without lsof/ss and would false-warn.
  • LOOKUP integration. mark_lookup is now the first recall step in the skill ("what do I know about X") with the catalog-not-full-text caveat. Agent-owned tagging + importance is now instructed across all three write paths — free-form publishes (skill), proactive self-documentation (session guidance), and /soul-journal (creation-time tags/importance, with a re-publish note since mark_append carries no metadata). knowledge-join.md corrected 13- → 14-tool surface.

Review + fixes

  • CodeRabbit flagged one real issue: server_health_warning passed raw .pid contents to kill -0, which misreads empty/option-like (-1)/garbage content as alive. Fixed with a ^[0-9]+$ guard before probing; tested across no-file / empty / option-like / garbage / dead-pid / live-pid.
  • The "docstring coverage 50% < 80%" pre-merge warning was a soft check, skipped with reason (the bash helpers already carry header comments; not contorting code for a metric).

.gitignore bug caught in passing

Fritz noticed tools/demarkus-token showed as ignored. Root cause: the unanchored stray-binary entry tools/demarkus-token matched the source package directory (tools layout puts main packages at tools/<name>/, unlike client/server which use cmd/), silently ignoring any new untracked file in the package. main.go survived only because it predated the rule. First instinct was to delete the line, but Fritz pushed back ("it should build on change no?") — empirically confirmed cd tools/demarkus-token && go build does drop a tools/demarkus-token/demarkus-token binary, so the protection was real. Correct fix: anchor to the binary path. Extended to demarkus-publish and demarkus-broker, which had the same latent footgun and were never covered.

Lesson

"Delete the redundant ignore" was wrong — the artifact it guarded against does get built (just via a different invocation than I first reasoned about). Test the actual build behavior before declaring an ignore rule pointless. Anchoring beats deleting when a stray-binary name collides with a source directory.

Note

This entry couldn't be tag/importance-stamped: the demarkus-soul MCP mark_publish gateway here predates the metadata param that #168 (and the earlier LOOKUP work) added to the local demarkus-mcp. So the very recall improvement shipped today isn't yet available on this soul's gateway — worth a follow-up to bump the soul server/client to a build that exposes publisher metadata over MCP.

Soul-doc reconciliation + plugin-store divergence check (later session)

Fritz asked "are we using the new plugin." Surfaced two things worth recording.

Soul docs lagged the plugin release

/index.md (was v40, modified 05-30) and /roadmap.md (v37) both predated #168's merge, so they still narrated the plugin as v0.2.0. The plugin code itself was consistent at v0.3.0 (repo plugin.json + marketplace.json + installed cache all agree) — drift was docs-only. Updated both: index Plugins bullet + Claude Code Plugin version-history block in roadmap now reflect v0.3.0 (#168) with the lookup-wire / self-doc-guidance / pin-bump summary. index → v41, roadmap → v38.

The session was NOT routing through the plugin's store

Four mark-protocol MCP servers are wired in this Claude Code install:

  • demarkus-soul → demarkus-mcp -host mark://soul.demarkus.io — the remote production soul. This is what the preflight + all work used (correct, per CLAUDE.md).
  • plugin:demarkus-memory:demarkus-memory → wrapper at …/cache/demarkus/demarkus-memory/0.3.0/…, points at mark://localhost:6309 — the plugin's lazy-spawned local store.
  • demarkus-hub, knowledge — also connected.

The plugin's local localhost:6309 store is stale and unrelated: its /index.md is v7, modified 2026-05-01, and it's a hub for nib / Junto projects, not demarkus. So the v0.3.0 plugin is installed and live in-session (it injected the SessionStart self-documentation guidance), but its backing store is a different, month-old personal soul — divergent from the demarkus soul where project knowledge actually lives. Two distinct backends; no conflict, but worth knowing they aren't the same memory.

Resolved a prior-entry worry

The earlier entry noted the soul's MCP gateway predated the metadata param so this journal "couldn't be tag/importance-stamped." That's no longer true — today's index/roadmap publishes carried metadata (tags + importance) and succeeded, and this journal already shows tags/importance in frontmatter. The soul gateway now exposes publisher metadata over MCP. Follow-up closed.

Plan reconciliation — closed 5 stale plans against verified ground truth

Fritz: "I think some of these are done, can we compare and close the ones that are." Listed all 24 /plans/ docs, then verified the live-looking candidates against actual git/PRs/code (one subagent per plan, read-only). The soul plan docs had badly lagged the late-May broker sprint (#154-#165) and the GKE deploy — several plans the docs described as "in flight / starting / blocked" were in fact fully merged.

Closed (appended a verified Status block to each plan doc + moved Active→Completed in /index.md v42):

  • lookup-verb — #166 merged; only stale draft frontmatter.
  • broker-stable-mint — #158/#159/#163/#164/#165. Lazy per-world provisioning, cache-stable retries (mint cascade structurally impossible), dead DefaultToken knobs gone. Design simplified to one long-lived per-world write token (beyond plan, deliberate).
  • broker-deadcode-cleanup — #159 (unblocked) + #164 (f9a24e9). Every targeted symbol (mcp_session.go/sessionCache, /tokens API, issuer.go, sweeper trim) gone; go build clean.
  • universe-onboarding — CLOSED. PR6 (tools/demarkus-join) canceled, PR7/8 absorbed into the MCP Gateway (#151/#152). The plan body's "all 8 PRs unstarted" was frozen at v1; the index was the correct record. Only doc debt remains.
  • knowledge-system-gke-deploy — COMPLETE except the deferred launch blog. Verified against the live deploy repo (template, pushed 2026-05-30) and a live RFC 8414 response from knowledge.demarkus.io — the actual Phase 7 acceptance test passing. Backups pivoted restic→GCS to CSI VolumeSnapshots (documented).

Left OPEN with a status correction (not closed):

  • broker-auth-code-grant — core RFC 6749 + PKCE grant SHIPPED and tested (#155/#156), but the plan's PR3 (auth-code kind-smoke stage + a live /knowledge-join journal verification) was never done. deploy/kind/up.sh still drives only the device-flow path. Decision pending: complete PR3 or descope.

Also added versions-sharding to the index Active list — a fully-specced, unstarted server storage change (per-doc versions/<doc>/vN + lazy migration) that had been missing from the index entirely.

Lesson

The index/roadmap drift in only ~4 days because plan docs are written before merge and never re-touched after. The reliable signal is gh pr list --state merged + grepping for the symbols a plan promised to add/remove — not the plan's own prose. Worth a periodic "reconcile plans vs PRs" pass.

broker-auth-code-grant PR3 — closed the kind-smoke gap

Fritz: "close the gap with the kind smoke test." Added an auth-code + PKCE end-to-end stage to deploy/kind/up.sh's --with-mcp-smoke path. Read the actual broker handlers first (oauth_authorize.go, device.go deviceTokenAuthCode, discovery.go, server.go routes) rather than trusting the plan prose — confirmed the OAuth surface is on :8080 (not the :8081 MCP gateway), token endpoint is POST /device/token with the grant_type switch, and the callback dispatches to authCodeCallback on a non-empty AuthCodeID.

New authcode-smoke curl pod: /oauth/authorize (S256) → mock IdP auto-approve → /auth/callback (state cookie replayed) → 302 to loopback redirect_uri with broker code (assert state echoed + iss) → negative wrong-verifier 400 invalid_grant → positive 200 Bearer → replay 400. The negative-before-positive ordering is deliberate: Redeem preserves the one-shot code on a verifier mismatch, so the wrong-verifier check proves PKCE enforcement without burning the code.

Two gotchas the code-read caught:

  • values-broker.yaml (the --with-mcp-smoke values) has no insecureCookies, unlike the Stage-4 values-broker-argo.yaml. The state cookie is Secure by default, so it would not replay over the pod's plain HTTP and the callback would fail at state validation. Set server.insecureCookies=true via --set (scoped to the smoke install, leaving plain Stage 2 alone).
  • PKCE challenge must be base64url-nopad(sha256(verifier)); computed host-side with openssl (the curl pod has no openssl) and cross-checked the pipeline against a Python reference — exact match.

Live read-only probe of broker.knowledge.demarkus.io confirmed the prod surface: discovery advertises the grant, missing client_id → 400 invalid_request, response_type=token → 302 error=unsupported_response_type (the new redirecting handler, not the old direct stub), valid PKCE → 302 to Google. The original stub is gone in prod.

bash -n clean, pre-commit.sh green (shell-only). Not executed in a live kind cluster (needs image build + spin-up) — left for Fritz. Change uncommitted per the no-commit-on-his-behalf rule. Plan doc updated; index flip to Completed deferred until commit.

Lesson

Reading the handlers before writing the harness paid for itself twice (the :8080/:8081 split and the Secure-cookie trap). A smoke test written against the plan's description instead of the routes would have failed in the cluster with a confusing state-validation error.

trail
  1. soul.demarkus.io v5