soul.demarkus.io:6309/demarkus-library/journal/2026-06-12.md/v16 draft reader meta

2026-06-12 — demarkus-library deploy manifests + CI/release

Roadmap item 1 (deploy manifests), branch feature/deploy-manifests, awaiting Fritz commit/PR.

Shipped (working tree)

  • .goreleaser.yml (root): single-module → plain v tags, NO monorepo block (unlike demarkus). Used archives.ids not archives.builds — deprecated since goreleaser 2.8 (demarkus repo still on old syntax; goreleaser check fails locally on it). Added var version = "dev" to main.go for the -X main.version ldflag.
  • Dockerfile (root): COPY-only over dist/docker/<arch>/ staged binaries (no QEMU compile), distroless/static-debian12:nonroot — CA certs needed for outbound HTTPS to broker. EXPOSE 8080/tcp.
  • deploy/helm/demarkus-library/: mirrors broker chart conventions. Key divergences:
    • replicaCount: 1 + strategy: Recreate — sessions in-memory; rolling update = login flapping. PDB default-off (minAvailable 1 would block drains at 1 replica).
    • Env-based config (no config.yaml Secret like broker). Chart-managed -oauth Secret only for cleartext library.broker.clientSecret; existingSecretRef mode for production. checksum/secret annotation only in cleartext mode.
    • demarkus-library.validate helper mirrors NewAppConfig startup validation at render time (transport enum, broker-mode required fields, secret mutual exclusion, quic host).
    • NetworkPolicy egress: DNS + TCP 443 (broker) + UDP 6309 (networkPolicy.quicEgressPort, federation/quic dials).
    • Probes both hit /health (only endpoint; broker has /healthz+/readyz).
    • automountServiceAccountToken: false (library never talks to kube-apiserver, unlike broker).
    • 23 helm-unittest tests (deployment/secret/ingress suites) pass.
  • .github/workflows/ci.yml: paths-filter (app/charts), golangci-lint v2.9.0 action, go test/vet/build, goreleaser check, helm lint+unittest.
  • .github/workflows/release.yml: test → semver (PaulHatch, tag_prefix v, no change_path — single module, chart-only commits must bump too) → tag + goreleaser-pro + stage binaries + buildx multi-arch (amd64/arm64/armv7) → ghcr.io/latebit-io/demarkus-library + helm chart to oci://ghcr.io/latebit-io/charts, version==appVersion==image tag.
  • Lint cleanup: 13 errcheck issues fixed with explicit _ = discards (demarkus convention) in broker.go, oauth.go, 2 test files. Default golangci-lint now 0 issues.

Deferred / decisions

  • demarkus's strict .golangci.yml (gocritic/revive/gocyclo/modernize) NOT adopted — 18 issues would need fixing (mostly gocritic style in tests). CI uses lint defaults.
  • Repo secret needed before first release: GORELEASER_KEY (goreleaser pro). GITHUB_TOKEN covers ghcr.
  • Verified locally: go test ✓, golangci-lint 0 issues ✓, goreleaser check ✓, snapshot build + version stamp ✓, helm lint ✓, helm template (7 resources) ✓, 23 unittest ✓.

Next

  • Fritz: commit, PR, add GORELEASER_KEY secret.
  • Roadmap item 2: drop broker rateLimit.tokens to ~30/min burst ~15 (demarkus repo AppSet values).
  • Then rendering upgrades (chroma first).

Correction: GORELEASER_KEY is an org-level (global) secret — already available to this repo, no per-repo setup needed.

Module-path migration to latebit-io (later same day)

demarkus merged the rename (15cc18b fix(package): latebit → latebit-io, tags client/v0.12.40 + protocol/v0.7.13, proxy-resolvable — verified). Library follow-up (working tree on top of 712511f "feat(CI): added deploy manifests"):

  • Imports rewritten github.com/latebit/demarkus/ → latebit-io; go.mod replaces DROPPED, now requires client v0.12.40 + protocol v0.7.13 (tagged, fetchable — no sibling checkout needed anywhere).
  • Library's own module renamed too: module github.com/latebit-io/demarkus-library (same latent mismatch; go mod edit -module + import sweep). Gotcha: renaming internal imports before the module line makes go mod tidy fetch the repo from the proxy as an external dep — fix module line first.
  • CI/release sibling-clone steps removed from both workflows.
  • Verified: build ✓, tests ✓, lint 0 ✓, vet ✓, goreleaser check + snapshot ✓.

The 2026-06-12 sibling-clone entry in debugging.md is now historical.

PR #10 review response (CodeRabbit, 11 findings — all addressed)

Workflows: actions pinned to commit SHAs (tag comments kept) · ci.yml top-level permissions: contents: read · release.yml write scopes moved to release job only · non-cancelling concurrency: release-${ref} (semver/tag race) · release test job now also runs helm lint+unittest (chart-only push can't ship broken chart) · if: failure() rollback step deletes release+tag so a failed publish doesn't burn the version (tag must still exist BEFORE goreleaser — GORELEASER_CURRENT_TAG).

Chart: validate helper trims all presence checks (matches NewAppConfig trim; whitespace-only fails at render not CrashLoop) + new cleartextSecret helper shared by secret.yaml/deployment.yaml · automountServiceAccountToken: false on POD spec (held SA-only before; broke when serviceAccount.create=false) · NetworkPolicy ingress adds same-namespace podSelector: {} (default install was unreachable — ingress.enabled defaults false) · NOTES insecure warning gated on quic OR federation — NOT the reviewer's exact ask (move under quic branch): DEMARKUS_INSECURE also governs broker-mode federation QUIC dials, so quic-branch-only would hide a real warning.

Code: session_test.go setup Save now t.Fatalf on error.

24 chart tests (added whitespace-trim regression), lint 0, go tests pass.

PR #10 outside-diff comments (2, both fixed)

  • release job gated github.ref == 'refs/heads/main' — workflow_dispatch from a feature branch could otherwise tag + publish unmerged code.
  • goreleaser-action version: latest → "~> v2" in both workflows (bounded drift).

Roadmap item 2 — broker rate limit back down

demarkus-knowledge-system-deploy branch fix/broker-rate-limit-down (uncommitted): apps/demarkus-broker/applicationset.yaml rateLimit.tokens 60/30 → 30/15, comment rewritten (session pool made reads 1 request; the old comment's "revisit down once the library reuses its MCP session" condition is met). YAML validated. ArgoCD auto-syncs on merge (automated + selfHeal). Roadmap item 2 done pending Fritz commit/PR; next is in-cluster library install (item 1 wrap-up), then rendering upgrades (chroma).

Rendering upgrades (a)+(b) — branch feature/rendering-chroma

Debt.md items: chroma highlighting + GFM alerts + emoji, all server-side zero JS, on one branch (uncommitted).

  • Chroma (goldmark-highlighting/v2): CLASSES mode, never inline styles (bluemonday strips style; widening = CSS-injection surface). Policy admits chromaClasses regex on pre/code/span + tabindex="0" on pre (chroma a11y). static/chroma.css GENERATED by internal/adapter/inbound/web/gen_chroma_css.go (go:build ignore, go generate ./internal/adapter/inbound/web) — github light + github-dark in @media (prefers-color-scheme: dark) (site declares color-scheme: light dark). Generator lives OUTSIDE static/ — the go:embed static/* glob would embed+serve it.
  • GFM alerts (zmtcreative/gm-alert-callouts v0.8.0): emits div.callout.callout-<kind) + .callout-title/.callout-title-text/.callout-body. Default inline-SVG icons REPLACED with emoji map (WithIcons) — svg through bluemonday would mean a dozen attr allowances. calloutClasses regex bounds kind (author-controlled via [!KIND]); data-callout deliberately stripped. CSS in page.html uses light-dark() (GitHub palette both modes). Unknown kinds render neutral; plain blockquotes untouched (test-pinned).
  • Emoji (yuin/goldmark-emoji): :shortcode: → unicode chars, sanitizer-inert.
  • Live smoke (quic vs soul): chroma classes + css verified on roadmap doc's sh fence. goldmark bumped 1.7.8→1.7.13 by deps.
  • Suite green -race, lint 0. Remaining rendering debt: mermaid + KaTeX islands (ADR 0003 concessions, later).

PR #11 review fix

chromaClasses shape pattern ([a-z]{1,3}[0-9]?) admitted non-chroma tokens (nav/btn). Replaced with an allowlist BUILT FROM chroma.StandardTypes at init — exact by construction, upgrade-proof (formatter reads class names from the same table). Longest-first alternation (prefix shadowing). Tests cover nav/btn/foo1 and mixed bg nav (whole attr value must match). Suite green.

Rendering upgrades merged + released

PR #11 → main (86d490a). Release pipeline auto-shipped v0.1.1 (fix: → patch bump; image + chart at 0.1.1). Pipeline working as designed on its first real post-v0.1.0 release. Roadmap item 3 (a)+(b) DONE; (c) mermaid + KaTeX islands remain in debt.md. Next queued: universe view (mark_worlds broker tool → floor page) or Phase 2 restricted collections.

Rendering (c) — mermaid + KaTeX islands (branch feature/rendering-islands)

ADR 0003 concessions recorded (ADR updated v3). Design:

  • Loader static/islands.js (~4KB, only always-loaded piece): lazy-loads heavy vendored libs ONLY when page content needs them; re-scans on htmx:afterSettle (fragment swaps). Mermaid: keys on pre>code.language-mermaid, swaps to pre.mermaid, mermaid.run, render failure restores source block. KaTeX: textContent probe for delimiters → auto-render with trust:false, throwOnError:false.
  • Vendored (static/vendor/, ~3.7MB embedded): mermaid 11.15.0, katex 0.17.0 (js+auto-render+css+woff2 only).
  • Server side: hugo-goldmark-extensions/passthrough shields TeX from markdown (delimiters: inline \(..\), display $$..$$ + \[..\]; single-$ rejected — dollar-amount false positives, test-pinned). Policy admits language-mermaid on code (chroma has no mermaid lexer → plain fallback keeps the class; bluemonday was the stripper). Sanitizer remains backstop (script-in-math test).
  • goldmark bumped 1.7.13→1.8.2 (passthrough dep). Suite green -race, lint 0.
  • Gotcha: smoke-test 404s were a STALE server pid from the earlier chroma smoke still bound to :8099 serving the old binary — kill %1 doesn't cross Bash tool invocations; use lsof -ti :PORT | xargs kill.

Islands merged + released

PR #12 → main (1f68dac), v0.1.2 auto-released. Review fixes before merge: loadScript AND loadCSS failure-aware + retryable (evict rejected promise from cache; explicit terminal catches → console.warn, degradation preserved); CodeRabbit path_filters exclude static/vendor/** + *.min.js (vendored payloads, stylelint noise).

Rendering roadmap item fully DONE (chroma, alerts, emoji, mermaid, KaTeX). debt.md rendering section shipped end to end.

Found: semver quirk — PaulHatch minor_pattern: "feat:" is a SUBSTRING match; feat(scope): doesn't contain feat: → scoped feats bump PATCH not minor (v0.1.2 should arguably have been v0.2.0). Same quirk in demarkus repo workflows. Fix if desired: regex form /^feat(\(.+\))?:/ (+ same for BREAKING CHANGE which is body-matched, that one's fine). Flagged to Fritz.

Scratch demo page /demarkus-library/scratch/rendering-demo.md left in place as a rendering test card (importance 0.1); archive if unwanted.

Remaining queue: universe view (mark_worlds → floor page) · Phase 2 restricted collections.

Reading Room design session (end of day)

Fritz brought full design notes ("The Reading Room" — context-optimized reading, digital universe library, not digital garden). Hashed out + locked:

  • ADR 0005 (v2, 15 decisions): spatial trails (stack-in-URL, truncate-and-append, scroll-not-resize, spines), Tufte column + working margin, status trust badge, catalog/graph/universe-as-pane, no search box, friendly frontmatter, rendered-HTML cache requirement, desktop-only, + agent symmetry amendment (projection-never-source, escape to protocol, authoring contract in-universe, constructible trail format, librarian consumes trail).
  • plans/reading-room.md (v3): full design, feasibility (htmx-hard VIABLE — trail is server state in URL; only new JS = scroll-into-view snippet), rate-limit math forcing the render cache, phasing R1 (Tufte pane) → R2 (trail engine) → R3 (previews/backlinks/graph pane), universe-view research alongside.
  • Roadmap v12 re-queued accordingly. R1 is next up and fully specified.

Session ended here; next session starts R1.

Late addition — inclusivity rule (ADR 0005 v3, decision 16): the library must ALWAYS work both as a single-world app (direct QUIC, e.g. soul.demarkus.io, no broker/login) and against a full knowledge system (broker). Single-world mode: the universe IS the world (floor = world root, graph pane = world graph, catalog = direct lookup). Broker-only features are incomplete, not shipped. Every R-phase acceptance-tested in both transports.

Evening: Reading Room R1 — the Tufte pane BUILT (Fable, branch feature/reading-room-r1)

R1 per ADR 0005 / plans/reading-room.md, all on the branch, staged, awaiting Fritz's commit. Suite green -race, vet clean, gofmt clean.

  • Domain/ports: domain.Document gains Status/Tags/Properties/Modified/ Version/Agent; domain.Rendered{HTML, Properties} is the Renderer port's new return (frontmatter parsed, not discarded); ReadingService gains Tag (tag pages) + Raw (protocol escape); WorldGateway.Lookup gains a filter arg (QUIC → fetch.LookupOptions.Filter; broker → filter tool arg, omitted when empty; federated passes through).
  • Markdown adapter: splitFrontmatter (same strict shape as the old strip, now returns the fence) + parseFrontmatter (yaml.v3 Node walk — ordered top-level scalars/scalar-lists only, garbage → nil, never fails a render). New sidenote.go goldmark extension: extension.Footnote + AST transformer at priority 1000 (after footnote's 999) converts single-paragraph, referenced-once footnotes into inline <sup class="sidenote-number"></sup><span class="sidenote">…</span>; multi-paragraph/multi-ref notes stay classic footnotes (sanitizer admits fn-shaped ids/hrefs/classes only — DOM clobbering stays closed). CSS counters number both ends; no ids for sidenotes.
  • Service: status authority order (metadata status: tag axis → frontmatter status: → "draft"; values bounded ^[a-z0-9][a-z0-9-]{0,31}$, invalid → draft). Synthetic views (Browse/History/Search/Tag) carry no margin. Tag sends the tag as query AND tag= filter (exact narrowing + sane ranking).
  • Web: Tufte pane — grid minmax(0,60ch) 250px, gap 2.5rem, Charter/ Georgia serif, off-white/off-black via light-dark(), booktabs tables (heavy top/bottom, light header rule, no verticals). Pinned .doc-meta (sticky, paper background + z-index so sidenotes slide under): status badge, tag links (status: axis excluded — the badge carries it), provenance dl (modified/version→editions link/agent), properties dl, escape block (mark:// address + raw + editions). Sidenotes float right with negative margin into the margin column. New routes /w/:world/tags/:tag + /w/:world/raw/* (text/plain, body verbatim). Nav search box removed (decision 5); /search route survives for R2's catalog pane. New reading_handlers_test.go covers margin rendering, axis-tag exclusion, tag/raw routes, error mapping, no-search-box.
  • Authoring contract (decision 13) published in-universe: mark://soul.demarkus.io/.well-known/library/authoring.md (status:accepted, importance .85); repo source docs/authoring-contract.md.
  • Acceptance, quic mode (live against soul.demarkus.io): VERIFIED. rendering-demo.md (republished v2 with frontmatter + 4-footnote matrix): 2 sidenotes converted with inline markup intact, notes 3+4 correctly remain footnotes (fn:3,fn:4), badge status-wip (metadata axis beat frontmatter status: draft), properties block rendered (list joined ", "), mark:// + raw escape (text/plain, frontmatter intact), tag page /tags/sidenotes lists the doc, no search box, versions/browse margin-free, authoring contract renders status-accepted.
  • Acceptance, broker mode (live, headless limit): boots against knowledge.demarkus.io with ~/.config/demarkus-library/dev.env + mkcert TLS; turnstile guards everything including the NEW routes (/, /w/root/ tags/adr, /w/root/raw/index.md all 302 → /login with return_to; /login 200, no search box). Post-login read path not exercised — web SSO needs a browser; Claude Code's knowledge-system OAuth tokens aren't reachable headless. Unit level: broker gateway filter pass-through + parseToolResult margin metadata tested; broker tool surface is byte-for-byte demarkus-mcp whose mark_lookup declares filter. Fritz: one click-through wanted — source ~/.config/demarkus-library/ dev.env && ./demarkus-library, log in at https://library.dev.test:8443, eyeball a root-world doc's margin + a tag click. Residual risk: deployed broker build predating LOOKUP filter would make tag pages return unfiltered results (cosmetic, not erroring).
  • Visual polish pass (spacing, dark mode, sticky-meta overlap behavior on long sidenote docs) deliberately left for Fritz's eyeball round.

R1 broker-mode acceptance CLOSED

The dev.env credentials had been stranded by the 2026-06-12 in-cluster rotation (library-web re-registered with cluster-only redirect + new secret hash). Fix: registered a SECOND confidential client library-web-dev in demarkus-knowledge-system-deploy deployment.yaml (hash of the existing local secret, redirect https://library.dev.test:8443/auth/callback) so prod rotations can never strand local e2e again; dev.env CLIENT_ID switched to library-web-dev. Fritz merged; ArgoCD synced; render smoke green (empty/single/multi). Purely additive — prod library-web untouched, no refresh tokens invalidated (RefreshStore is k8s-Secret-backed anyway).

Fritz logged in through the live broker and verified the Tufte pane against root-world content. R1 acceptance complete in BOTH transports (decision 16). Probe gotcha for next time: OAuth endpoints live on broker.knowledge.demarkus.io (discovery at knowledge.demarkus.io works; /oauth/authorize on the bare domain 404s).

Remaining R1 tail: Fritz commits/PRs feature/reading-room-r1 (staged); visual polish round by eyeball.

Night: Reading Room R2 — the trail engine BUILT (Fable, branch feature/reading-room-r2)

R1 merged as PR #13 (CodeRabbit round: frontmatter strip now validates the fence is a YAML mapping before stripping — ---\nintro\n--- is content). R2 built on top, staged for Fritz. Suite green -race. Build decisions recorded in plans/reading-room.md §"R2 build decisions" BEFORE building.

  • Trail URL (decisions 1/14): /t/<pane>/~/<pane>?focus=i — pane chunk = /w/ route tail (<world>/d/<path>, <world>/tags/<tag>, trailing slash = listing). ~ reserved separator segment; raw slashes (no %2F for ingress); focus 0-based default-last clamped; cap 10 (parse 400s past it, click drops oldest). Codec + click algebra are pure functions in web/trail.go: truncate-to-N+append, dedup→focus, spine→focus-only.
  • Cache (decision 9): port.DocumentCache + LRU adapter (adapter/outbound/cache, 512 entries default, no TTL). Service: Read/Browse/Tag now cachePut; ReadCached/BrowseCached/TagCached read through (nil cache ⇒ live). Focused-live policy: trail handler reads the focused pane live, everything else cached ⇒ every click = exactly 1 world read; staleness only where attention isn't.
  • Canvas: /t/* renders paneVMs — spine (vertical title + status badge) / body (immediate parent, full body, sidenotes hidden — their float would bleed into the next pane) / focused (R1 Tufte pane + margin; only focused gets doc-meta). Unfocused pane error → tombstone spine ("gone", dashed) so stale shared trails survive; focused error → real 404/401. trailizeLinks post-process (trail_links.go): /w/ doc+tag hrefs → their post-click trail URLs; the link to the pane open at i+1 gets class="active-link" (highlighted links ARE the breadcrumb). versions/raw stay /w/ (deliberate escapes). overscroll-behavior-x: contain; scroll-focused-pane-into-view rides islands.js (DOMContentLoaded + htmx:afterSettle). / 302s to the default one-pane trail; /w/ stays the permalink space. Templates refactored: shared head/nav/doc-meta defines; new canvas.html.
  • Trail format spec (decision 14) published: mark://soul.demarkus.io/.well-known/library/trails.md (status:accepted)
    • repo docs/trail-format.md; cross-links the authoring contract.
  • Acceptance, quic (live, soul.demarkus.io): VERIFIED. 3-pane trail: spine/body/focused modes, exactly one doc-meta, wip badge + 2 sidenotes inside focused pane, trailized + active links; tag pane's catalog rows continue the trail (lookup results live on the path); listing pane; tombstone trail 200 with gone-spine; focused-missing 404; malformed 400; / → default trail redirect.
  • Acceptance, broker (headless limit): R2 binary live behind turnstile; /t/* 302→login with return_to. Gotcha that cost a few minutes: the R1-era process still held :8443 — new server logged bind failure and curl was hitting the OLD binary (404 on /t/). Kill by lsof -ti :8443, not the pid file, before re-verifying. Post-login trail walk = Fritz click-through (login, click links to stack panes, spine click, tag pane).
  • Call-pattern pinned in tests: ReadCached/ReadCached/Read for a 3-pane trail; BrowseCached/Tag/ReadCached for mixed kinds with focus=1.
trail
  1. soul.demarkus.io:6309 v16