Content-Addressed Fetch — Implementation Plan

Context

Demarkus already has the building blocks for content addressing: SHA-256 hash chains in the versioned store, content hashes computed for etags, and a client cache at ~/.mark/cache/. The hub pattern just shipped for discovery. Content addressing adds the "fetch from anywhere" primitive — any server that has the content can serve it, verified by hash.

Approach

No new verbs. FETCH handles it via path detection. Five incremental steps, each independently testable.

Step 1: Add content-hash to FETCH responses

File: server/internal/handler/handler.go

• Add computeContentHash(body string) string — SHA-256 of the stripped body (not doc.Content which includes store frontmatter)
• Call it in serveDocument() after stripFrontmatter(), set meta["content-hash"] = "sha256-<hex>"
• Also add to handleFetchVersion() for historical version responses
• Tests: verify content-hash appears in FETCH responses, matches expected hash of body

Step 2: Add hash index to the Store

File: server/internal/store/store.go

• Add fields to Store struct: hashIndex map[string]string, hashMu sync.RWMutex
• BuildHashIndex() — walks content root following symlinks (current versions only), reads each, strips frontmatter, hashes body, populates index. Skips versions/ dirs and archived docs. Returns error only on walk failure, logs individual file errors.
• LookupHash(hash string) (string, bool) — read-locked lookup
• UpdateHashIndex(reqPath string, body []byte) — write-locked upsert
• RemoveHashEntry(reqPath string) — write-locked removal by path (iterates to find matching path)
• Tests: table-driven for build, lookup, update, remove

Step 3: Update hash index on writes

File: server/internal/store/store.go

• In Write() after successful symlink rename (line ~576), call s.UpdateHashIndex(reqPath, content) — content here is the raw body parameter, not stored bytes
• In SetArchived() — call RemoveHashEntry when archiving, UpdateHashIndex when unarchiving
• Append() flows through Write() so it's handled automatically
• Tests: write a doc, verify hash index updated; archive, verify removed

Step 4: Handle hash-based FETCH

File: server/internal/handler/handler.go

• isHashPath(path string) (string, bool) — validates path matches /sha256-<64 hex chars> (73 chars total: / + sha256- + 64 hex)
• handleFetchByHash(w, req, hash) — calls Store.LookupHash(hash), if not found → not-found, otherwise Store.Get(path, 0) → serveDocument()
• Insert hash detection at the top of handleFetch(), before parseVersionPath
• Tests: fetch by valid hash, fetch unknown hash, malformed hash path

Step 5: Wire up on startup

File: server/cmd/demarkus-server/main.go

• After s := store.New(cfg.ContentDir) (line 102), call s.BuildHashIndex()
• Log entry count: logger.Info("content hash index built", "entries", s.HashIndexSize())
• Non-fatal on error (log warning, continue)

Key Design Decisions

• content-hash is separate from etag: etag hashes doc.Content (includes store frontmatter), content-hash hashes the stripped body (what the client receives). Different values.
• Current versions only: historical versions get content-hash in their response but aren't indexed for lookup. Keeps the index small.
• Archived docs excluded: archiving removes from index, unarchiving re-adds.
• In-memory index, no persistence: the content directory is the source of truth. Rebuilt on startup.
• Path syntax FETCH /sha256-<hex>: detected by pattern match, no protocol changes needed.

Files Modified

Verification

1. cd server && go test ./... after each step
2. bash pre-commit.sh at the end
3. Manual: start server, FETCH a doc, note content-hash in response, FETCH by that hash

Related documents

• Journal 2026-03-08: session in which this plan was written
• Federation: fetch-from-anywhere primitive this hash addressing enables