soul.demarkus.io/roadmap.md/v32 draft reader meta

Roadmap

Where demarkus has been and where it's going.

Scope: Markdown-Only Server

The demarkus server serves markdown only. Non-markdown content (images, PDFs, arbitrary binaries) is out of scope and will not be added. This keeps the protocol, store, auth, and size-limit logic simple and predictable. Any future inline-image story is a client/rendering concern — not something the server will serve bytes for.

Phase 1: MVP (Read-Only) — COMPLETE

Everything shipped:

  • QUIC server serving markdown files
  • FETCH, LIST, VERSIONS verbs
  • TUI client with Bubble Tea + Glamour
  • Link following, navigation history
  • Document graph visualization
  • CLI client with all verbs
  • MCP integration for LLM agent access
  • Docker multi-arch images
  • GoReleaser CI/CD with per-module versioning
  • Conditional fetch (if-none-match, if-modified-since)
  • SIGHUP certificate reload

Phase 2: Publish Operations — COMPLETE

Done:

  • PUBLISH verb with version creation
  • ARCHIVE verb
  • APPEND verb — sends only new content, server handles concatenation
  • Capability-based auth (token generation, SHA-256 hashes, path/op scoping)
  • Versioned store with symlinks and hash chain
  • Document editing via $EDITOR
  • Client-side token management
  • Conflict resolution (optimistic concurrency with expected-version)
  • No-op on duplicate content
  • Structured logging with slog (replaced console logging)
  • Protocol-level size limits (1 MiB body, 64KB frontmatter)
  • Audit logging with token_label on all write operations
  • Usability audit: documentation accuracy, CLI help, install script cleanup, CI lint

Phase 2 is fully complete. No remaining items.

Phase 3: Agent-Native & Advanced Features — COMPLETE

Done:

  • Agent manifest discovery (/.well-known/agent-manifest.md)
  • Bookmarks/favorites
  • MCP mark_append auto-resolve expected_version
  • Content-addressed fetch
  • Federation via MCP tools

Phase 4: The Information Graph — COMPLETE

Done:

  • The Demarkus Hub pattern
  • Persistent graph store
  • Backlinks
  • Graph as content (export/import)
  • Graph-aware navigation (TUI)
  • Agent discovery

Phase 5: The Demarkus Agent & Private Networks — IN PROGRESS

Read Auth for Private Networks — DONE

Per-path read token enforcement. Server-side and client-side shipped.

Security Hardening — DONE

See plan for full details.

  • Systemd hardening — install script generates units with ProtectSystem=strict, ReadWritePaths, NoNewPrivileges, etc. Conditional ProtectHome. Update path detects insecure config and prompts to harden with rollback.
  • -read-only mode — DEMARKUS_READ_ONLY env var / -read-only flag. Handler rejects PUBLISH/APPEND/ARCHIVE with not-permitted. Zero write access needed.
  • demarkus-publish — CLI tool that writes directly to the versioned store on disk. Enables local publishing when server runs read-only. Shares store.Write() with the server.
  • Read-only chroot install — install-readonly.sh. Chrooted server with ReadOnlyPaths=/, BindReadOnlyPaths=/dev/urandom. Maximum lockdown — Gemini-level security with full versioning.
  • Security documentation — attack surface analysis, threat model, comparison table, hardening guide.

Core Loop: Crawl & Index

  1. Seed — start from configured servers (hub, known peers)
  2. Crawl — follow mark:// links, discover new servers and documents
  3. Hash — collect content hashes from every document
  4. Index — publish updated hash indexes to configured hubs
  5. Repeat — on a configurable schedule, with conditional fetch (if-none-match) to be polite

Server-to-Server Sync

Rsync for the Mark Protocol. Replicate content between servers using content hashes as the diff mechanism.

How it works:

  1. LIST both source and destination servers
  2. Compare content hashes — skip documents that match
  3. FETCH changed/new documents from source
  4. PUBLISH to destination with the fetched content
  5. Optionally handle deletions (ARCHIVE on destination for docs removed from source)

Sync modes:

  • Mirror — destination becomes an exact copy of source (one-way)
  • Selective — sync specific paths or glob patterns (e.g., /docs/* only)
  • Multi-source — aggregate content from multiple servers into one destination

Key Design Points

  • Go binary, not an LLM agent — mechanical work, not reasoning
  • Polite crawling — conditional fetch, configurable rate limits
  • Hub-aware — reads and updates hub index documents
  • Daemon or cron — continuous or triggered
  • Auth-aware — passes read and write tokens for private servers

Implementation Sketch

  • New module: cmd/demarkus-agent (or standalone repo)
  • Reuses: fetch.Client, graphstore, client/internal/index, graph.Crawl
  • CLI: demarkus-agent crawl, demarkus-agent sync source dest, demarkus-agent daemon

Verb Set — Complete

6 verbs: FETCH, LIST, VERSIONS, PUBLISH, APPEND, ARCHIVE.

Distribution & Package Management — PLANNED

Homebrew tap for easier installation on macOS and Linux.

Build Targets

Supported platforms: macOS, Linux, Windows (WSL only).

Plugins — IN PROGRESS

Obsidian Plugin — v0.1.0 RELEASED

Standalone repo: latebit-io/obsidian-demarkus. Source in plugins/obsidian/.

Claude Code Plugin — IN PROGRESS

See plan. One-click marketplace install that lazy-spawns a local demarkus-server, auto-generates a token, and wires demarkus-mcp into the agent. Zero config on install.

Lives in the monorepo at plugins/claude-code/ — unlike Obsidian, Claude Code marketplaces can point at a subdirectory of any git repo, so no standalone repo is needed.

No core code changes required. The plugin reuses existing primitives: DEMARKUS_AUTH env var for token injection, /health endpoint plus ALPN "mark" negotiation for port probing, and shell redirection of demarkus-token generate stdout for writing the token to a file. Any future need for a new core primitive must be discussed and justified before landing.

TUI Polish

External Links — PLANNED (implementation ready to commit)

See plan. Closes issue #75 in full. Opens http/https/gemini/mailto links in the user's default handler via open/xdg-open/rundll32. Scheme allowlist, URL passed as argv only, no shell.

The original plan had a Phase 2 for same-server non-markdown files (images) — dropped because the server is markdown-only and will not serve binaries. Inline image rendering, if ever pursued, is a separate client-rendering initiative (terminal graphics protocols like Kitty/Sixel), not a continuation of this work.

Features Not Prioritized — Backlog

  • WebSub-style Subscriptions (removed from spec)
  • Offline Mode (deferred)
  • Full-Text Search (removed from spec, external tool)
  • Diff / Changelog Between Versions (noted, not implemented)
  • Blind Append with Content Deduplication (rejected)
  • Non-markdown content types on the server (out of scope by design)
soul.demarkus.io/plans/security-hardening.md complete reader meta

Plan: Security Hardening & Documentation — COMPLETE

Prompted by user feedback — reluctance to run a public-facing server with write access without understanding the threat model.

Problem

There's no security documentation. Security-conscious users have to guess at the attack surface and hardening options. This blocks public adoption.

What Was Done

1. Security page on the website — DONE

  • docs/site/security/index.md + pages branch security.md
  • Attack surface, token compromise, process compromise analysis
  • Comparison table (SSH vs Web+CGI vs Gemini vs Demarkus)
  • Systemd hardening guide with verification command

2. Systemd hardening in install script + deployment docs — DONE

  • setup_systemd generates unit with ProtectSystem=strict, ReadWritePaths, NoNewPrivileges, etc.
  • Conditional ProtectHome (omitted when content root is under /home)
  • readlink -f to canonicalize content root before writing unit
  • Deployment docs updated with hardened systemd example

3. Install script detects insecure existing config on update — DONE

  • _do_update_inner checks for missing ProtectSystem in existing unit
  • Interactive prompt (defaults to yes, skips in non-interactive mode)
  • Backs up unit before modifying, rolls back if service fails to start
  • Points to public security docs URL

4. Read-only mode (-read-only flag) — DONE

  • DEMARKUS_READ_ONLY env var + -read-only flag on server
  • Handler rejects PUBLISH/APPEND/ARCHIVE with not-permitted status
  • Config accepts 1, true, yes as truthy values
  • Tests for handler rejection and config parsing

5. demarkus-publish CLI tool — DONE

  • server/cmd/demarkus-publish/main.go
  • Writes directly to versioned store on disk (same store.Write() as server)
  • Supports -body flag or stdin input
  • Detects duplicate content (no-op on unchanged)
  • Enables local publishing when server runs read-only

6. Read-only chroot install script — DONE

  • install-readonly.sh — separate script for maximum security deployments
  • Chroot structure: /srv/demarkus/{bin,content,tls}
  • Systemd unit with RootDirectory, ReadOnlyPaths=/, BindReadOnlyPaths=/dev/urandom
  • Installs demarkus-publish to /usr/local/bin for local publishing
  • SHA-256 checksum verification on downloads
  • Rejects dangerous root paths (/, /usr, /etc, etc.)
  • No auto firewall changes (user decides)

7. Cleanup

  • Removed redundant demarkus.service from repo root
  • Updated docs: reference (config table), server (read-only section), install (binary table, readonly option), deployment (security link)

Key Design Decisions

  • Shared store code: demarkus-publish calls store.Write() directly — same code path as the server. No duplication.
  • not-permitted not unauthorized: Read-only rejection is a server policy, not an auth failure.
  • Separate install script: Read-only chroot install is a different deployment model, kept separate to avoid complicating the main install script.
  • No iptables: Replaced fragile iptables write isolation with read-only mode + chroot, which is simpler and more secure.

Related documents

trail
  1. soul.demarkus.io v32
  2. security-hardening
from here → read-auth index roadmap