Architecture

Demarkus is a Go monorepo with four modules connected by local replace directives.

Core Principle: The Agent as the Librarian

The server is a bookshelf — it holds documents and hands them back when asked. That's it. The agent is the librarian — it knows where things are, builds indexes, discovers content across servers, routes around failures, and helps you find what you need.

This is a strict requirement, not a suggestion. The server and protocol must stay dumb and fast. All intelligence — federation, discovery, indexing, routing, caching strategies — lives in the client layer (MCP tools, agents, CLI). When considering where to put new functionality, the answer is almost always "in the agent/client, not the server."

Federation is the proof: zero server changes, two MCP tools, all intelligence at the edge. The server doesn't even know federation exists.

Module Boundaries

• protocol/ — Wire format types, parsing, serialization. No network code. The foundation everything else depends on.
• server/ — QUIC server. Accepts Mark Protocol requests over QUIC streams, routes to handlers. Depends on protocol.
• client/ — CLI, TUI (Bubble Tea), and MCP server. All three share the same protocol client. Depends on protocol.
• tools/ — Dev utilities like demarkus-token for generating auth tokens.

Key Design Decisions

Stream Abstraction

handler.Stream is just io.ReadWriteCloser. This means handler tests don't need QUIC — they use mock streams (bytes.Buffer or similar). The entire handler layer is testable without network setup.

Frontmatter as map[string]string

Parsed frontmatter uses map[string]string rather than typed structs. This avoids YAML auto-typing issues (e.g., yes becoming true) and keeps the parsing layer simple. Type conversion happens at the handler level where context is known.

Versioned Store with Symlinks

Documents are stored as versioned files with symlinks pointing to current:

content-root/
  doc.md -> versions/doc.md.v3
  versions/
    doc.md.v1
    doc.md.v2
    doc.md.v3

Each version is immutable once written. SHA-256 hash chain links versions together for integrity verification.

Capability-Based Auth

Tokens grant capabilities, not identities. Server stores SHA-256 hashes in TOML, never raw tokens. Each token is scoped to path globs and specific operations. The server doesn't know or care who you are — only what you're allowed to do.

Read Auth for Private Networks

The auth system supports both "publish" and "read" operations on tokens. Write auth is always enforced — if a tokens file is configured, all writes require a valid token. Read auth is opt-in and per-path: if any token has "read" in its operations with a path pattern matching the request, that path requires read auth. Paths without read tokens remain public.

Key implementation details:

• readPaths []string on TokenStore — pre-computed at load time from tokens with "read" operations, avoiding per-request iteration
• RequiresReadAuth(path) checks both the path as-is and with a trailing slash to prevent bypass via /private vs /private/
• authorizeRead handler helper gates FETCH, LIST, and VERSIONS
• /.well-known/agent-manifest.md is always public so agents can discover capabilities before authenticating
• Content-addressed fetch by hash resolves to a real path first, then checks auth on that path — knowing a hash doesn't bypass access control
• Versioned paths (/doc.md/v2) check auth on the base path (/doc.md)
• Hot-reloadable via SIGHUP along with the rest of the token store

Path Traversal Defense

filepath.Clean + .. check. Returns not-found (not forbidden) to avoid leaking information about server structure.

Status as Text

Human-readable status strings (ok, not-found, conflict) instead of numeric codes. Fits the markdown-native philosophy — everything should be readable.

Size Limits

All protocol-level size constants live in protocol/request.go:

ParseRequest enforces these at the parsing layer using io.LimitReader to prevent unbounded memory allocation. The store adds a small allowance (maxStoreFrontmatter = 256) for its own version/hash-chain frontmatter when checking on-disk file sizes.

Verb Set — Complete

The protocol verb set is finalized at 6 verbs. No more verbs planned.

APPEND reuses the same auth (publish capability), versioning, hash chain, and optimistic concurrency infrastructure as PUBLISH. The key difference: APPEND sends only new content, the server concatenates it with the existing body. Document must exist (returns not-found otherwise), and combined size is checked against MaxBodyLength.

SEARCH was considered and removed — full-text search is better handled as an external tool built on LIST + FETCH rather than as a protocol-level concern.

Wire Format

Request: VERB /path\n followed by optional YAML frontmatter and body. Response: YAML frontmatter (always includes status) + markdown body.

Protocol constants: port 6309, ALPN "mark", scheme mark://.

What I've Internalized

The architecture rewards simplicity. When I'm tempted to add abstraction, the right move is usually to keep the function short and the flow obvious. Fritz and I have a shared instinct here: if a solution feels clever, it's probably wrong. The best code in this project reads like prose.

Audit Logging

All write operations (PUBLISH, APPEND, ARCHIVE, UNARCHIVE) emit structured slog records tagged with "audit", true. Each audit line includes: operation, path, version, success/failure, and token_label (the TOML key identifying which token made the write, e.g., "fritz-laptop").

Key design constraint: identity information stays in server-side logs only. Token labels are never stored in version files or exposed through the protocol. This preserves capability-based auth — tokens grant what you can do, not who you are. The operator controls log retention and persistence through their infrastructure (systemd journal, logrotate, etc.).

Token labels are sanitized before logging to prevent control-character injection, consistent with how request paths are handled.

Content-Addressed Fetch

FETCH responses include content-hash: sha256-<64hex> — the SHA-256 of the response body (after stripping store frontmatter). This is distinct from etag, which hashes the raw on-disk content including store frontmatter.

Clients can fetch by hash instead of path: FETCH /sha256-<64hex>. The server maintains an in-memory map[string]string (hash → path) protected by sync.RWMutex. The index is:

• Built on startup by walking symlinks (current versions only, skips archived)
• Updated incrementally on Write() and Archive()/unarchive
• Not persisted — the content directory is the source of truth

Key implementation details:

• isHashPath() in handler detects the path pattern (exactly 72 chars: / + sha256- + 64 hex)
• handleFetchByHash() resolves hash → path via the index, then serves normally through serveDocument()
• Hash detection runs before parseVersionPath() in the FETCH dispatch chain
• computeContentHash() hashes the stripped body, not doc.Content (which includes store frontmatter)

Agent-Driven Hash Discovery

Servers don't crawl or discover content — agents do. The librarian pattern in action:

1. Agent fetches documents from servers, collecting content-hash values
2. Agent builds hash → location mappings
3. Agent publishes the mapping as a document to a hub

When content goes missing, agents check hub indexes for the hash, find alternative servers, and fetch by hash directly. Servers stay simple (serve content, answer hash lookups). Agents own discovery, routing, and indexing. Hubs are just servers hosting index documents — no special protocol support needed.

This is documented in SPEC.md Section 12.1.