soul.demarkus.io:6309/demarkus-knowledge-system-deploy/journal/2026-06-12.md/v5 draft reader meta

2026-06-12 — demarkus-library in-cluster deploy

Added the Universe Library to the cluster (PR #70, merged to main):

  • apps/demarkus-library/applicationset.yaml — chart demarkus-library 0.1.0 from ghcr.io/latebit-io/charts, sync-wave 2, broker transport, ingress library.<domain> (letsencrypt-prod), home world root, replica 1 (in-memory sessions).
  • apps/demarkus-library/external-secret.yaml — ESO library-oauth-client from OpenBao secret/library/oauth-client (property client_secret), reusing the cluster-scoped openbao ClusterSecretStore.
  • deployment.yaml — library-web redirectURI swapped from local-e2e library.dev.test:8443 to https://library.knowledge.demarkus.io/auth/callback.
  • docs/runbook-library.md — manual OpenBao steps: widen external-secrets policy with secret/data/library/*, seed the plaintext client secret (sha256 must equal deployment.yaml's clientSecretHash).

Key gotcha: the library's broker.url must be the MCP gateway origin (https://knowledge.demarkus.io, i.e. https://<domain>), NOT broker.<domain> — RFC 8414 metadata is served only on the gateway host (verified live; broker.<domain>/.well-known/oauth-authorization-server 404s), and the metadata routes authorize/token to broker.<domain> itself.

Validated: helm template render with interpolated prod values (env/secretKeyRef/ingress/Certificate correct), scripts/smoke-broker-web-clients.sh green, image+chart 0.1.0 confirmed on ghcr.

OpenBao seed + secret rotation (post-merge)

Original library-web plaintext was unrecoverable locally (hash-scanned shell history, Claude transcripts, ~/latebit — zero matches against the registered clientSecretHash), so took the runbook's rotation path instead of waiting on the password manager:

  • Widened external-secrets policy with secret/data/library/* (now 4 paths).
  • Generated fresh secret, seeded secret/library/oauth-client v1.
  • Updated deployment.yaml clientSecretHash to e44adb5a… (merged as a0a02bc; broker config re-rendered + pod rolled within ~15s of ArgoCD pickup).
  • ESO synced, pod Running 1/1, TLS cert Ready.

Verified live ✅

  • Front-channel: /login/start → broker /oauth/authorize (client_id=library-web, PKCE S256, registered redirect) → 302 to Google. Broker accepts the registration.
  • Back-channel: user logged in through a real browser — token exchange with the rotated secret succeeded, reading room renders. demarkus-library is in production at https://library.knowledge.demarkus.io.

Security note for cleanup: the OpenBao root token is in plain shell history (~/.zsh_history, export BAO_TOKEN=… — one live with root policy). Runbook says password-manager only; should be rotated/scrubbed.

Side finding — local soul (demarkus-memory plugin, localhost:6309) is broken: server binary predates LOOKUP verb, and plugin token (sha256-d5002dbb…) isn't in the server's tokens.toml (expects sha256-a905094e…); writes failing since ~June 10. The repo .mcp.json demarkus-soul server (soul.demarkus.io) is the working write path.

Related documents

trail
  1. soul.demarkus.io:6309 v5