soul.demarkus.io:6309/demarkus-library/roadmap.md/v18 draft reader meta

demarkus-library — Roadmap & Resume

Pick-up-where-we-left-off doc. Authoritative plan is /plans/universe-library.md; the Reading Room redesign lives in /plans/reading-room.md (invariants locked in ADR 0005). Build model: Opus 4.8 default, Fable 5 for load-bearing design.

Where we are (2026-06-13 — Reading Room LIVE in the cluster)

main HEAD a1b539d (library). The whole Reading Room is deployed and working in prod: R1 (Tufte pane) → R2 (trail engine) → the floor (universe view, pane zero), both transports (decision 16). Cluster versions: library 0.1.6, broker 0.1.36, worlds (root + world-a) server 0.17.18 — all Synced/Healthy. Next build phase is R3. Recent detail in journal 2026-06-13 + journal 2026-06-12.

  • Phase 0 foundation spike · Phase 1a reading room (stacks, card catalog, editions, link rewriting, htmx fragment-vs-page).
  • Phase 1b — library card: DONE. Broker confidential web clients (demarkus 117e2a6) · oauth client (PKCE S256, Basic auth §2.3.1 escaping) · server-side sessions + single-flight refresh · ctx threading on ports · broker MCP WorldGateway with per-reader pooled sessions (1 request per read; fixed the live 429s) · login turnstile · composition-root swap DEMARKUS_TRANSPORT=quic|broker · native TLS for local e2e (env at ~/.config/demarkus-library/dev.env, client id library-web-dev — dedicated dev broker client; prod is library-web). E2e verified live.
  • Multi-world floor + federation (fc4277c): (world, path) addressing, /w/<world>/... routes, federated composite gateway — KS names via broker, host[:port] via direct QUIC (DEMARKUS_FEDERATION default true). Read token never leaves home host (test-pinned). Cross-world mark:// links clickable.
  • Deploy + CI/release — DONE (PR #10, v0.1.0): goreleaser-pro, distroless Dockerfile, helm chart (replicaCount 1 + Recreate — in-memory sessions), hardened workflows. Module paths on latebit-io. Releases auto-cut on merge to main (PaulHatch per-module semver). Deploy = ArgoCD watching latebit-io/demarkus-knowledge-system-deploy; image+chart pins in apps/<app>/applicationset.yaml, world list + webClients in deployment.yaml.
  • Rendering — DONE (PRs #11/#12): chroma classes-mode · GFM alerts · emoji · mermaid + KaTeX islands (lazy, degrade to source). Scratch card: /demarkus-library/scratch/rendering-demo.md.
  • Reading Room R1 — Tufte pane: MERGED (PR #13), LIVE. 60ch column + 250px margin · pinned margin trust signals (status badge: metadata status: tag axis → frontmatter fallback → draft; tags; provenance; parsed-frontmatter properties; mark:// + raw-source escape) · sidenote goldmark extension (footnote→margin; multi-para/multi-ref degrade) · lookup-backed tag pages + WorldGateway.Lookup filter · /w/:world/raw/* · search box removed · authoring contract published (mark://soul.demarkus.io/.well-known/library/authoring.md).
  • Reading Room R2 — trail engine: MERGED (PR #14), LIVE (build decisions in plans/reading-room.md §"R2 build decisions"):
    • Trail URL /t/<pane>/~/<pane>?focus=i; ~ = pane separator; pure codec
      • click algebra (truncate/append, dedup→focus, spine→focus). Cap 10, raw slashes (no %2F). Pane kinds: <world>/d/<path> (doc; trailing slash or empty = listing), <world>/tags/<tag>, bare u = floor.
    • Rendered-document cache (decision 9): port.DocumentCache + LRU adapter. Focused-live policy: every click costs exactly 1 world read.
    • Canvas: spines / body-only parent / focused Tufte pane; tombstone spine for unreadable unfocused panes; trailizeLinks rewrites /w/ hrefs to post-click trail URLs + active-link breadcrumb. Trail format spec published (mark://soul.demarkus.io/.well-known/library/trails.md).
    • Codec note: paneAddrFromParts(world,kind,value) (trail.go) is the single source of truth for pane semantics; parsePaneChunk (/t/ chunks) and paneAddrFromRoute (/w/ hrefs) both call it. Extracted after the two diverged (PR #16). Don't re-duplicate — share semantics, not syntax.
  • The floor — universe view (pane zero): MERGED (PR #15), LIVE (design in plans/reading-room.md §"Universe-view research" + 2 addenda):
    • demarkus prereqs shipped: mark_worlds broker tool (#182, 02e52ef) lists authorized worlds; LOOKUP match-all query * (b8aedb7) returns the whole catalog under scope in importance order — the floor's node source. A world must run server ≥ b8aedb7 (0.17.18) or it tombstones.
    • WorldGateway.Worlds(): broker parses mark_worlds' table; QUIC returns the home world (extensional universe); federated prefers Names→Hosts.
    • service.Floor/FloorCached: worlds × Lookup("*"), parsed from the LOOKUP markdown table (status from the tag axis); failed world → tombstone (dimmed, never dropped); ErrUnauthorized → re-login. Single-slot cache.
    • Web: pane kind u; / → /t/u; SSR SVG floorSVG (worlds as circles r∝satellite count, top-10 docs on orbit ring r∝importance, stroke=status, every node a plain with its post-click trail URL). ADR 0003's canvas island stays UNSPENT — SSR SVG was enough at universe scale.
    • PR #16 fix: trail parser rejected the world-root listing chunk (<world>/d/, empty value); the floor's world-node link hit it. Fixed + consolidated the two parsers (see codec note above). Library 0.1.6.

How to resume

cd ~/latebit/demarkus-library && go test ./... && go build -o /tmp/dl ./cmd/demarkus-library
# quic mode (no auth, soul world): then open /t/u (the floor)
PORT=8080 DEMARKUS_INSECURE=true DEMARKUS_HOST=soul.demarkus.io \
  DEMARKUS_AUTH=<token from .mcp.json> /tmp/dl
# broker mode: source ~/.config/demarkus-library/dev.env && /tmp/dl
#   OAuth endpoints live on broker.knowledge.demarkus.io (bare domain 404s
#   /oauth/authorize). If bind fails, an old run holds the port: lsof -ti :8443.
# local floor world (match-all build, tagged docs): recipe in journal
#   2026-06-12 "The floor BUILT" — token paths need "/**" not "/*".
# cluster (GKE): kubectl ctx gke_knowledge-49722_...; creds expire mid-session
#   → ! gcloud auth login. ArgoCD apps in ns argocd; check operationState,
#   not just sync.status (it reads stale on a retrying op).

File map (hexagonal — ADR 0002)

  • internal/core/{domain,port,service} — entities (Document, Floor, WorldInfo), ports; ReadingService = Read/Browse/History/Search/Tag/Raw + *Cached + Floor/FloorCached; WorldGateway adds Worlds(); DocumentCache port; service/floor.go (catalog-table parse, status from tag axis)
  • internal/adapter/inbound/web — Echo routes (/t/* canvas incl. u floor, /w/:world/... permalinks incl. tags + raw), trail.go (codec + click algebra + paneAddrFromParts), trail_links.go (trailize pass), trail_handlers.go (+ floorPaneView), floor.go (SSR SVG), auth turnstile, session pkg, templates (page/content/canvas/login; shared head/nav/doc-meta), static/ (htmx, islands.js incl. focused-pane scroll)
  • internal/adapter/outbound/{world,broker,federated,oauth,markdown,cache} — direct QUIC (token→home only; Worlds=home world), MCP pool (Worlds parses mark_worlds), shape-router, broker OAuth, goldmark, LRU document cache
  • cmd/demarkus-library — composition root, AppConfig, brokerAuth shim
  • docs/ — authoring-contract.md + trail-format.md (sources of the in-universe /.well-known/library/ docs; keep both copies in sync)

Next sessions — queued work

1. Reading Room R3 — previews + backlinks + graph pane (NEXT)

  • Hover preview cards: htmx fragment (hx-trigger="mouseenter delay:300ms once" → server fragment), one component serving both outbound body links and backlinks (build once, both directions — ADR 0005 §margin).
  • Backlinks margin block: the one margin item with no data source yet. Decide the edge source: broker graph store (ephemeral, per-pod, crawl- populated) vs library-side crawl vs the render-time observed-links map (plans/reading-room.md addendum 1 — the library already parses every link in rewriteLinks; accumulate cross-doc targets free). Must work in BOTH transports (decision 16) — quic has no broker graph store.
  • Graph neighborhood pane: one doc + its links/backlinks as a pane that continues the trail (decision 4). New pane kind <world>/g/<path> is reserved in the research notes — add to trail codec + trails.md spec (decision 14). Likely SSR SVG again (same approach as the floor).
  • Record build decisions in plans/reading-room.md before building (process).

2. Floor enrichment (pairs with R3's edge-source decision)

Hub-sourced topology (plans/reading-room.md addendum 2 — hubs are the primary map: durable, versioned, transport-symmetric); portal nodes for externally-linked hosts in quic mode; world-view zoom level (one world's catalog with dir clusters). Hub discovery convention still open (manifest self-declaration vs DEMARKUS_HUB env — lean manifest + override).

3. Phase 2 — restricted collections (per /plans/universe-library.md)

Per-world/per-path authz surfaced in the reading room. Depends on broker-side authz semantics — re-read the plan + broker authz.go first.

Maybe / observed

  • Deploy CI papercut: apps-immutable-check runs the guard via ... | tee, so the pipe masks the script's exit (2 on render-fail, 1 on flag) — the check can NEVER fail the build, only posts a PR comment. Add set -o pipefail / drop the tee so it can actually gate. (This let the worlds immutable-change merge un-blocked on 2026-06-13.)
  • Semver quirk: PaulHatch minor_pattern: "feat:" is substring — feat(scope): bumps PATCH not minor. Fix via regex /^feat(\(.+\))?:/.
  • Broker rate-limit drop (60/30 → 30/15) on KS-deploy branch fix/broker-rate-limit-down — confirm merged/synced.
  • Visual polish round by eyeball (pane widths/gaps, spine legibility, dark mode, canvas scroll, sticky-meta vs long sidenotes; floor at many-worlds scale + satellite label crowding).

Standing deferred

  • Find + fix publishers that wrote metadata fences into bodies (R1's friendly frontmatter reduces harm; source fix still open).
  • Shared session store (or sticky sessions) → replicaCount > 1 + RollingUpdate + PDB — the R2 doc cache + single-slot floor cache are also per-pod (fine at replicaCount 1; revisit together).
  • Adopt demarkus's strict .golangci.yml (18 issues to clean first).
  • Phase 3 cataloging desk (writes, CodeMirror island) · Phase 4 AI librarian (SSE; joins the canvas as a pane, trail = its context) · Phase 5 public face.

History (condensed — details in journals + ADRs)

  • 2026-06-10: Phases 0 + 1a built + merged (PRs #1–2).
  • 2026-06-11: Phase 1b planned (plan, ADR 0004) + shipped (PRs #3–#9); floor + federation; frontmatter strip. token = /device/token, authorize = /oauth/authorize, /mcp always bearer-gated, no refresh rotation. Gotchas in debugging.md.
  • 2026-06-12: v0.1.0 deploy/CI (#10) · latebit-io migration · in-cluster install · v0.1.1/v0.1.2 rendering (#11/#12) · ADR 0005 · R1 (#13) · R2 (#14) (journal).
  • 2026-06-13: universe-view research · demarkus mark_worlds (#182) + LOOKUP match-all (b8aedb7) · floor merged (#15) · cluster rollout of R1+R2+floor (server 0.17.18 / broker 0.1.36 / library 0.1.5 → 0.1.6), incl. the one-time world StatefulSet orphan-delete recreate + the chart fix (#184: VCT uses selectorLabels, so future server bumps are clean) · trail root-listing fix + parser consolidation (#16, library 0.1.6) — the Reading Room is now fully live (journal).

Git / process notes

  • Work on feature/* / fix/* branches. Never auto-commit — Fritz commits and opens PRs; CodeRabbit reviews via .coderabbit.yaml (static/vendor/** excluded — vendored payloads). Releases auto-cut on merge.
  • Deploy = ArgoCD on latebit-io/demarkus-knowledge-system-deploy. Bump chart targetRevision + image.tag together in apps/<app>/applicationset.yaml. Library/broker are Deployments (clean roll); worlds are StatefulSets (VCT immutable — now stable post-#184).
  • PR scopes: domain, port, service, web, world, markdown, cmd, view, deps, ci, broker, auth, session, oauth, federated, deploy, docker, helm.
trail
  1. soul.demarkus.io:6309 v18