soul.demarkus.io:6309/demarkus-library/roadmap.md/v17 draft reader meta

demarkus-library — Roadmap & Resume

Pick-up-where-we-left-off doc. Authoritative plan is /plans/universe-library.md; the Reading Room redesign lives in /plans/reading-room.md (invariants locked in ADR 0005). Build model: Opus 4.8 default, Fable 5 for load-bearing design.

Where we are (2026-06-13 — R1 + R2 + the floor MERGED)

main HEAD 21a513d (floor merged, PR #15; R2 #14, R1 #13). Library is in the cluster (running pre-R1 v0.1.2 until the next release/rollout). Released: v0.1.0 (deploy/CI), v0.1.1 (chroma/alerts/emoji), v0.1.2 (mermaid/KaTeX islands); R1+R2+floor on main, not yet released. All accepted live in BOTH transports (decision 16). Details in journal 2026-06-12.

  • Phase 0 foundation spike · Phase 1a reading room (stacks, card catalog, editions, link rewriting, htmx fragment-vs-page).
  • Phase 1b — library card: DONE. Broker confidential web clients (demarkus 117e2a6) · oauth client (PKCE S256, Basic auth with §2.3.1 escaping) · server-side sessions + single-flight refresh · ctx threading on ports · broker MCP WorldGateway with per-reader pooled sessions (1 request per read; fixed the live 429s) · login turnstile (/login, /login/start, /auth/callback, /logout; HX-Redirect for htmx) · composition-root swap DEMARKUS_TRANSPORT=quic|broker · native TLS serving for local e2e (mkcert + hosts-file recipe in journal 2026-06-11; env saved at ~/.config/demarkus-library/dev.env, client id library-web-dev — dedicated dev client registered 2026-06-12 after the prod rotation stranded the old creds). E2e verified live.
  • Multi-world floor + federation (fc4277c): (world, path) addressing, /w/<world>/... routes, federated composite gateway — knowledge-system names via broker, host[:port] via direct QUIC (DEMARKUS_FEDERATION, default true, strict parse). Read token never leaves home host (test-pinned). Cross-world mark:// links clickable; root hub = the floor.
  • Deploy manifests + CI/release — DONE, v0.1.0 (PR #10): goreleaser-pro, COPY-only distroless Dockerfile, helm chart (replicaCount 1 + Recreate — in-memory sessions), hardened workflows (SHA-pinned actions, scoped tokens, concurrency, main-ref guard, tag rollback). Module paths on latebit-io (client v0.12.40 + protocol v0.7.13, no replaces). Library installed in-cluster; library-web client secret was rotated (hash-only storage — plaintext lost as designed).
  • Rendering — DONE end to end (PRs #11 v0.1.1 + #12 v0.1.2): chroma classes-mode highlighting · GFM alerts · goldmark-emoji · mermaid + KaTeX islands (lazy failure-aware islands.js; degrades to source). Scratch test card: /demarkus-library/scratch/rendering-demo.md (v2 exercises sidenotes
    • frontmatter properties + status axis).
  • Reading Room R1 — the Tufte pane: MERGED (PR #13), accepted both transports. 60ch column + 250px margin · pinned margin trust signals (status badge: metadata status: tag axis → frontmatter fallback → draft; tags; provenance; parsed-frontmatter properties; mark:// + raw-source escape) · sidenote goldmark extension · lookup-backed tag pages + WorldGateway.Lookup filter · /w/:world/raw/* · search box removed · authoring contract published (mark://soul.demarkus.io/.well-known/library/authoring.md).
  • Reading Room R2 — the trail engine: MERGED (PR #14), accepted both transports (build decisions in plans/reading-room.md §"R2 build decisions"):
    • Trail URL /t/<pane>/~/<pane>?focus=i; pure codec + click algebra (truncate/append, dedup→focus, spine→focus). Cap 10, raw slashes.
    • Rendered-document cache (decision 9): port.DocumentCache + LRU adapter. Focused-live policy: every click costs exactly 1 world read.
    • Canvas: spines / body-only parent / focused Tufte pane; tombstone spine for unreadable unfocused panes; trailizeLinks rewrites /w/ hrefs to post-click trail URLs + active-link breadcrumb. scroll-into-view rides islands.js. Trail format spec published (mark://soul.demarkus.io/.well-known/library/trails.md).
  • The floor — universe view (pane zero): MERGED (PR #15) (design in plans/reading-room.md §"Universe-view research" + 2 addenda):
    • demarkus prereq shipped: mark_worlds broker tool (#182, 02e52ef) lists authorized worlds; LOOKUP match-all query * (#…, b8aedb7) returns the whole catalog under scope in importance order — the floor's node source. A world must run ≥ b8aedb7 or it tombstones.
    • WorldGateway.Worlds(): broker parses mark_worlds' table; QUIC returns the home world (extensional universe); federated prefers Names→Hosts.
    • service.Floor/FloorCached: worlds × Lookup("*"), parsed from the LOOKUP markdown table (status from the tag axis); failed world → tombstone (dimmed, never dropped); ErrUnauthorized → re-login. Single-slot floor cache, focused-live.
    • Web: pane kind u (bare chunk — the floor IS the universe); / → /t/u; SSR SVG floorSVG (worlds as circles r∝satellite count, top-10 docs on orbit ring r∝importance, stroke=status, every node a plain carrying its post-click trail URL). ADR 0003's canvas island stays UNSPENT — SSR SVG was enough at universe scale, as researched.
    • Acceptance: unit -race; live quic against a local match-all server (full floor, status strokes, importance sizing, satellite-click → trail); old-server tombstone confirmed; broker /t/u turnstiled.

How to resume

cd ~/latebit/demarkus-library && go test ./... && go build -o /tmp/dl ./cmd/demarkus-library
# quic mode (no auth, soul world):
PORT=8080 DEMARKUS_INSECURE=true DEMARKUS_HOST=soul.demarkus.io \
  DEMARKUS_AUTH=<token from .mcp.json> /tmp/dl    # then open /t/u (the floor)
# broker mode: source ~/.config/demarkus-library/dev.env && /tmp/dl
# (full recipe in journal 2026-06-11; OAuth endpoints live on
#  broker.knowledge.demarkus.io — the bare domain 404s /oauth/authorize.
#  If the bind fails: an older run may hold :8443 — lsof -ti :8443.)
# local floor world (4 tagged docs, match-all build): recipe in journal
# 2026-06-12 "The floor BUILT" — needs server >= b8aedb7.
# chart checks:
helm lint deploy/helm/demarkus-library && helm unittest deploy/helm/demarkus-library

File map (hexagonal — ADR 0002)

  • internal/core/{domain,port,service} — entities (Document, Floor, WorldInfo), ports (ctx+world on every method), Read/Browse/History/Search/Tag/Raw + *Cached + Floor/FloorCached; WorldGateway adds Worlds(); DocumentCache port; service/floor.go
  • internal/adapter/inbound/web — Echo handlers/routes (/t/* canvas incl. the u floor pane, /w/:world/... permalinks incl. tags + raw), trail codec/click algebra (trail.go) + trailize pass (trail_links.go) + canvas handlers + floor SVG (floor.go), auth turnstile, session pkg, templates (page/content/canvas/login; shared head/nav/doc-meta), static/ (htmx, islands.js incl. focused-pane scroll, chroma.css, vendor/)
  • internal/adapter/outbound/{world,broker,federated,oauth,markdown,cache} — direct QUIC (token→home only; Worlds=home world), MCP pool (Worlds parses mark_worlds), shape-router, broker OAuth, goldmark, LRU document cache
  • internal/adapter/bearer — ctx token contract between middleware & gateway
  • cmd/demarkus-library — composition root, AppConfig, brokerAuth shim
  • deploy/helm/demarkus-library — chart · .goreleaser.yml + Dockerfile at root · .github/workflows/{ci,release}.yml
  • docs/ — authoring-contract.md + trail-format.md (sources of the in-universe /.well-known/library/ docs)

Next sessions — queued work

0. Release + rollout (now blocks visible floors)

  • Cut a demarkus server release + roll the world pods (soul + cluster): until a world runs ≥ b8aedb7 the floor tombstones it. This is the gate on the floor actually showing satellites in prod.
  • Cut a library release with R1+R2+floor and roll the cluster (mind the PaulHatch semver quirk — feat(scope): bumps PATCH; decide minor bump).
  • Visual polish round by eyeball (pane widths/gaps, spine legibility, dark mode, canvas scroll feel, sticky-meta vs long sidenotes; floor layout at many-worlds scale + satellite label crowding).

1. Reading Room R3 — previews + backlinks + graph pane

Hover preview cards (htmx fragments, one component both directions), backlinks margin block (data source decision: broker graph persistence vs library-side crawl — note the render-time observed-links map idea in plans/reading-room.md addendum 1), graph neighborhood pane (continues the trail — decision 4; new pane kind <world>/g/<path> reserved in the research notes). Librarian-consumes-trail (decision 15) lands with Phase 4.

2. Floor enrichment (after R3's edge-source decision)

Hub-sourced topology (plans/reading-room.md addendum 2: hubs are the primary map — durable + versioned + transport-symmetric); portal nodes for externally-linked hosts in quic mode; world-view zoom level (one world's full catalog with dir clusters). Hub discovery convention still open (manifest self-declaration vs DEMARKUS_HUB env — lean manifest + override).

3. Phase 2 — restricted collections (per /plans/universe-library.md)

Per-world/per-path authorization surfaced in the reading room. Depends on broker-side authz semantics — re-read the plan + broker authz.go first.

Maybe / observed

  • Semver quirk: PaulHatch minor_pattern: "feat:" is substring — feat(scope): bumps PATCH not minor. Fix via regex /^feat(\(.+\))?:/ in library + demarkus workflows if desired.
  • Broker rate-limit drop (60/30 → 30/15) on demarkus-knowledge-system-deploy branch fix/broker-rate-limit-down — status unknown, confirm merged/synced.

Standing deferred

  • Find + fix publishers that wrote metadata fences into bodies (R1's friendly-frontmatter rendering reduces the harm; source fix still open).
  • Shared session store (or sticky sessions) → unlock replicaCount > 1 + RollingUpdate + PDB — note the R2 document cache + the single-slot floor cache are also per-pod (fine at replicaCount 1; revisit together).
  • Adopt demarkus's strict .golangci.yml (18 issues to clean first).
  • Phase 3 cataloging desk (writes, CodeMirror island) · Phase 4 AI librarian (SSE; joins the canvas as a pane, trail = its context) · Phase 5 public face.

History (condensed — details in journals + ADRs)

  • 2026-06-10: Phases 0 + 1a built and merged (PRs #1–2).
  • 2026-06-11: Phase 1b planned (plan, ADR 0004) and fully shipped (PRs #3–#9); floor + federation; frontmatter strip. Key endpoint facts: token = /device/token, authorize = /oauth/authorize, /mcp always bearer-gated, no refresh rotation. Gotchas in debugging.md.
  • 2026-06-12: v0.1.0 deploy/CI (PR #10) · latebit-io module migration · in-cluster install + client-secret rotation · v0.1.1 chroma/alerts/emoji (PR #11) · v0.1.2 mermaid/KaTeX islands (PR #12) · Reading Room designed
    • ADR 0005 · R1 Tufte pane merged (PR #13) · R2 trail engine merged (PR #14).
  • 2026-06-13: universe-view research recorded · demarkus mark_worlds (#182) + LOOKUP match-all (b8aedb7) · the floor / universe view merged (library PR #15) — SSR SVG pane zero, both transports (journal).

Git / process notes

  • Work on feature/* / fix/* branches. Never auto-commit — Fritz commits and opens PRs; CodeRabbit reviews via .coderabbit.yaml (static/vendor/** excluded from review — vendored payloads).
  • PR scopes: domain, port, service, web, world, markdown, cmd, view, deps, ci, broker, auth, session, oauth, federated, deploy, docker, helm.
trail
  1. soul.demarkus.io:6309 v17