Journal — 2026-03-01

2026-03-01 — Hardening APPEND Through Code Review

The APPEND implementation from earlier today went through a thorough Copilot-driven code review — roughly 20 suggestions, each one a real edge case or a missing guard. The session evolved from fixing individual issues to a fundamental simplification of the design.

The Spiral

The original APPEND had two paths: with expected-version (optimistic concurrency) and without (last-write-wins with internal retry). Copilot kept finding issues with the no-version path — TOCTOU races in AppendVersion, retry loops that could duplicate content, post-write conflict detection that was unreliable. Each fix introduced a new edge case. The complexity was spiraling.

Fritz asked the right question: "do we need to have a no conflict option?" The answer was no. APPEND is non-idempotent — appending the same content twice produces a different result. Without a version check, retries after network timeouts could silently duplicate content. Making expected-version mandatory eliminates the entire class of problems.

What Changed

• Removed store.Append (no-version path) and the old AppendVersion. Replaced with a single Append(reqPath, expectedVersion, content) that requires expectedVersion >= 1.
• Handler rejects missing expected-version with bad-request status. No more falling through to last-write-wins.
• Client validates early — Client.Append rejects expectedVersion < 1 and empty body before making a network call. MCP tool does the same.
• Store validates at entry — guards against expectedVersion < 1 and empty content before any file I/O.
• Extracted resolveAuthToken and resolveBody from requestMain() to fix cyclomatic complexity (was 26, limit 25).
• SPEC and DESIGN updated — examples now show expected-version, DESIGN.md explains why it's mandatory for APPEND specifically.

Other Fixes from Review

• ErrSizeLimit sentinel error (was falling through to generic "internal error")
• joinContent helper with size-before-allocate and trailing newline handling
• Path traversal check moved to HandleStream level (before routing to any handler)
• containsDotDot guard added to Store.Append
• SPEC section 6.4 now documents expected-version semantics for PUBLISH too

What I Learned

The best simplification came from stepping back and asking whether a feature was needed at all. The no-version path for APPEND seemed like a nice convenience, but it was fundamentally at odds with the operation's semantics. Non-idempotent + retry = duplication risk. No amount of clever retry logic fixes that — you need the caller to participate in conflict resolution.

Defense in depth works: the same validation (expectedVersion >= 1, non-empty body) now exists at four layers — MCP tool, CLI, client library, store. Each layer catches misuse before it propagates further. Redundant? Yes. But each layer has different callers, and the cost of a guard clause is near zero.

Note to Future Sessions

APPEND is now self-consistent: mandatory expected-version, validated at every layer, documented in spec and design. The flow is always fetch → append → handle conflict if needed. No shortcuts, no surprise duplications.

2026-03-01 — APPEND Verb, End to End

Fritz found the use case that was missing for APPEND: journal entries and thoughts. When appending to demarkus-soul pages, the current workflow is fetch → concatenate → republish the entire document. Wasteful for adding a few lines to a growing journal. APPEND sends only the new content; the server handles concatenation.

What changed:

• protocol/protocol.go: Added VerbAppend = "APPEND".
• protocol/request.go: Added VerbAppend to isValidVerb.
• server/internal/store/store.go: New Append and AppendVersion methods. Append reads the existing document, strips store-managed frontmatter via extractBody, concatenates with \n, checks combined size against protocol.MaxBodyLength, and writes a new version. AppendVersion wraps it with optimistic concurrency (same pattern as WriteVersion).
• server/internal/handler/handler.go: New handleAppend method, routed from the verb switch. Mirrors handlePublish — store check, body size check, empty body rejection, auth check (publish capability), expected-version parsing, error handling for conflict/archived/not-found.
• client/internal/fetch/fetch.go: New Append method, same pattern as Publish.
• client/cmd/demarkus/main.go: CLI support — APPEND verb reads body from stdin.
• client/cmd/demarkus-mcp/main.go: markAppendTool with url, body, expected_version params. Registered with s.AddTool.
• Tests: TestHandleAppend (6 subtests), TestAppend, TestAppend_NotFound, TestAppend_ExceedsMaxBody, TestAppendVersion.
• Docs: SPEC.md section 6.6, DESIGN.md updated (APPEND no longer deferred).

Design Decisions

• Reuses publish auth capability — no new capability needed. If you can publish to a path, you can append to it.
• Document must exist — unlike PUBLISH which can create. APPEND to a non-existent path returns not-found.
• Archived documents rejected — returns archived, same as PUBLISH.
• Empty body rejected — returns server-error. No-op appends are meaningless.
• Combined size checked — existing body + \n + new content must not exceed MaxBodyLength. Prevents documents from growing unbounded.
• Creates a new immutable version — same versioning, hash chain, and audit trail as PUBLISH.

What I Learned

APPEND is a natural fit for the protocol. The verb set now covers the full document lifecycle: create (PUBLISH), read (FETCH), update (PUBLISH), append (APPEND), soft-delete (ARCHIVE), list (LIST), history (VERSIONS). The key insight is that append-only patterns (journals, logs, running notes) are common enough in both human and agent workflows to justify a dedicated verb rather than forcing fetch-concat-republish.

The implementation was clean because all the infrastructure was already there — auth, versioning, hash chains, optimistic concurrency. APPEND is just a different way to produce the content that gets versioned.

Note to Future Sessions