# Content-Addressed Fetch — Implementation Plan ## Context Demarkus already has the building blocks for content addressing: SHA-256 hash chains in the versioned store, content hashes computed for etags, and a client cache at `~/.mark/cache/`. The hub pattern just shipped for discovery. Content addressing adds the "fetch from anywhere" primitive — any server that has the content can serve it, verified by hash. ## Approach No new verbs. FETCH handles it via path detection. Five incremental steps, each independently testable. ### Step 1: Add `content-hash` to FETCH responses **File:** `server/internal/handler/handler.go` - Add `computeContentHash(body string) string` — SHA-256 of the stripped body (not `doc.Content` which includes store frontmatter) - Call it in `serveDocument()` after `stripFrontmatter()`, set `meta["content-hash"] = "sha256-"` - Also add to `handleFetchVersion()` for historical version responses - Tests: verify `content-hash` appears in FETCH responses, matches expected hash of body ### Step 2: Add hash index to the Store **File:** `server/internal/store/store.go` - Add fields to `Store` struct: `hashIndex map[string]string`, `hashMu sync.RWMutex` - `BuildHashIndex()` — walks content root following symlinks (current versions only), reads each, strips frontmatter, hashes body, populates index. Skips `versions/` dirs and archived docs. Returns error only on walk failure, logs individual file errors. - `LookupHash(hash string) (string, bool)` — read-locked lookup - `UpdateHashIndex(reqPath string, body []byte)` — write-locked upsert - `RemoveHashEntry(reqPath string)` — write-locked removal by path (iterates to find matching path) - Tests: table-driven for build, lookup, update, remove ### Step 3: Update hash index on writes **File:** `server/internal/store/store.go` - In `Write()` after successful symlink rename (line ~576), call `s.UpdateHashIndex(reqPath, content)` — `content` here is the raw body parameter, not stored bytes - In `SetArchived()` — call `RemoveHashEntry` when archiving, `UpdateHashIndex` when unarchiving - `Append()` flows through `Write()` so it's handled automatically - Tests: write a doc, verify hash index updated; archive, verify removed ### Step 4: Handle hash-based FETCH **File:** `server/internal/handler/handler.go` - `isHashPath(path string) (string, bool)` — validates path matches `/sha256-<64 hex chars>` (73 chars total: `/` + `sha256-` + 64 hex) - `handleFetchByHash(w, req, hash)` — calls `Store.LookupHash(hash)`, if not found → not-found, otherwise `Store.Get(path, 0)` → `serveDocument()` - Insert hash detection at the top of `handleFetch()`, before `parseVersionPath` - Tests: fetch by valid hash, fetch unknown hash, malformed hash path ### Step 5: Wire up on startup **File:** `server/cmd/demarkus-server/main.go` - After `s := store.New(cfg.ContentDir)` (line 102), call `s.BuildHashIndex()` - Log entry count: `logger.Info("content hash index built", "entries", s.HashIndexSize())` - Non-fatal on error (log warning, continue) ## Key Design Decisions - **`content-hash` is separate from `etag`**: etag hashes `doc.Content` (includes store frontmatter), content-hash hashes the stripped body (what the client receives). Different values. - **Current versions only**: historical versions get `content-hash` in their response but aren't indexed for lookup. Keeps the index small. - **Archived docs excluded**: archiving removes from index, unarchiving re-adds. - **In-memory index, no persistence**: the content directory is the source of truth. Rebuilt on startup. - **Path syntax `FETCH /sha256-`**: detected by pattern match, no protocol changes needed. ## Files Modified | File | Change | |------|--------| | `server/internal/handler/handler.go` | `content-hash` in responses, hash path detection, `handleFetchByHash` | | `server/internal/store/store.go` | Hash index struct fields, Build/Lookup/Update/Remove methods, index updates in Write/SetArchived | | `server/cmd/demarkus-server/main.go` | One-line startup call to `BuildHashIndex` | | `server/internal/handler/handler_test.go` | Tests for content-hash, hash fetch, malformed paths | | `server/internal/store/store_test.go` | Tests for hash index operations | ## Verification 1. `cd server && go test ./...` after each step 2. `bash pre-commit.sh` at the end 3. Manual: start server, FETCH a doc, note `content-hash` in response, FETCH by that hash