# 2026-06-12 — demarkus-library in-cluster deploy

Added the Universe Library to the cluster (PR #70, merged to main):

- `apps/demarkus-library/applicationset.yaml` — chart `demarkus-library` 0.1.0 from ghcr.io/latebit-io/charts, sync-wave 2, broker transport, ingress `library.<domain>` (letsencrypt-prod), home world `root`, replica 1 (in-memory sessions).
- `apps/demarkus-library/external-secret.yaml` — ESO `library-oauth-client` from OpenBao `secret/library/oauth-client` (property `client_secret`), reusing the cluster-scoped `openbao` ClusterSecretStore.
- `deployment.yaml` — `library-web` redirectURI swapped from local-e2e `library.dev.test:8443` to `https://library.knowledge.demarkus.io/auth/callback`.
- `docs/runbook-library.md` — manual OpenBao steps: widen `external-secrets` policy with `secret/data/library/*`, seed the plaintext client secret (sha256 must equal deployment.yaml's `clientSecretHash`).

Key gotcha: the library's `broker.url` must be the **MCP gateway origin** (`https://knowledge.demarkus.io`, i.e. `https://<domain>`), NOT `broker.<domain>` — RFC 8414 metadata is served only on the gateway host (verified live; `broker.<domain>/.well-known/oauth-authorization-server` 404s), and the metadata routes authorize/token to `broker.<domain>` itself.

Validated: helm template render with interpolated prod values (env/secretKeyRef/ingress/Certificate correct), `scripts/smoke-broker-web-clients.sh` green, image+chart 0.1.0 confirmed on ghcr.

## OpenBao seed + secret rotation (post-merge)

Original `library-web` plaintext was unrecoverable locally (hash-scanned shell history, Claude transcripts, ~/latebit — zero matches against the registered clientSecretHash), so took the runbook's rotation path instead of waiting on the password manager:

- Widened `external-secrets` policy with `secret/data/library/*` (now 4 paths).
- Generated fresh secret, seeded `secret/library/oauth-client` v1.
- Updated `deployment.yaml` clientSecretHash to `e44adb5a…` (merged as a0a02bc; broker config re-rendered + pod rolled within ~15s of ArgoCD pickup).
- ESO synced, pod Running 1/1, TLS cert Ready.

## Verified live ✅

- Front-channel: `/login/start` → broker `/oauth/authorize` (client_id=library-web, PKCE S256, registered redirect) → 302 to Google. Broker accepts the registration.
- Back-channel: user logged in through a real browser — token exchange with the rotated secret succeeded, reading room renders. **demarkus-library is in production at https://library.knowledge.demarkus.io.**

Security note for cleanup: the OpenBao **root token is in plain shell history** (`~/.zsh_history`, `export BAO_TOKEN=…` — one live with root policy). Runbook says password-manager only; should be rotated/scrubbed.

Side finding — local soul (demarkus-memory plugin, localhost:6309) is broken: server binary predates LOOKUP verb, and plugin token (sha256-d5002dbb…) isn't in the server's tokens.toml (expects sha256-a905094e…); writes failing since ~June 10. The repo `.mcp.json` demarkus-soul server (soul.demarkus.io) is the working write path.

## Related documents

- [Knowledge system GKE deploy](/plans/knowledge-system-gke-deploy.md): cluster deploy plan this library rollout extends
- [Universe library](/plans/universe-library.md): the library product deployed here
- [Universe deployment](/plans/universe-deployment.md): broker and gateway topology the library binds to
- [ADR 0001 broker confidential web clients](/adr/0001-broker-confidential-web-clients.md): library-web client secret model used