soul.demarkus.io/plans/postgres-backend.md/v1 draft reader meta

Plan: Postgres store backend (phase 1)

A Postgres implementation of the handler's DocumentStore interface, config-selected, with the file store staying the default. Branch: feat/postgres-store (stacked on refactor/documentstore-interface).

Parity strategy: store the version-file bytes

The file store's Document.Content is the raw stored bytes including the store frontmatter (version, archived, previous-hash, publisher meta); the handler strips it. The hash chain is computed over those raw bytes. So pgstore stores the byte-identical serialized blob the file store would write, in a stored bytea column. Consequences:

  • Get, Versions, VerifyChain, ErrNotModified dedup, and the previous-hash chain carry over exactly.
  • File to Postgres migration (either direction) is a blob copy.
  • The version-file format becomes shared vocabulary, exported from protocol/store.

Changes to protocol/store (no behavior change)

Export the format helpers so pgstore reuses them instead of duplicating logic (guidelines: don't duplicate):

  • SerializeVersion(version, prevRaw []byte, content, meta) refactored out of buildVersionFile (which keeps the file read and delegates).
  • ExtractBody, ExtractMetadata, IsArchived, ExtractPreviousHash renamed to exported.
  • SetArchived(data, archived) extracted from Archive's in-place frontmatter toggle.
  • ValidateMeta, MetaEqual, JoinContent, RetentionValue exported.

File store tests must pass unchanged (behavior-identical refactor).

New package: server/internal/pgstore

pgx driver dependency lives in the server module only; protocol stays dependency-free. Schema applied at startup (CREATE TABLE IF NOT EXISTS):

  • documents(path text primary key, current_version int, archived bool)
  • versions(path text, version int, stored bytea, body_hash text, modified timestamptz, primary key(path, version))
  • index on body_hash (LookupHash queries the DB directly; no in-RAM hash index)

Concurrency: every write transaction row-locks the documents row (SELECT FOR UPDATE, INSERT ON CONFLICT for creation); UNIQUE(path, version) is the backstop CAS. This gives true multi-writer safety; the current-pointer regression race of the file store cannot occur.

Semantics to replicate (from the file-store study)

  • WriteVersion expected-version: <0 skip check, 0 create-only, >0 exact; ErrConflict carries Document{Version: current}; post-write conflict check kept.
  • ErrNotModified when body and publisher meta equal the tip; allowed at expected or expected+1.
  • ErrArchived rejects writes to archived docs.
  • Archive toggles the tip's stored frontmatter in place (not a new version) plus the documents.archived column; archived docs drop out of LookupHash.
  • Retention: RetentionValue(meta) > 0 prunes versions <= current-keep oldest-first, contiguous, returns PruneResult.
  • Append: read base at expectedVersion, JoinContent, delegate to WriteVersion.
  • ListDir: distinct next path segments under the prefix; files are docs at exactly prefix/name; a subdirectory lists only if its subtree holds a visible doc; dot-names hidden; archived filtering per flag; sorted; os.ErrNotExist for a prefix with no docs.
  • Get: version 0 means current; os.ErrNotExist contract throughout.
  • VerifyChain: sha256 of previous stored bytes vs ExtractPreviousHash of current, oldest to newest.
  • WalkCurrent equivalent so main.go builds the in-RAM catalog at startup (phase 1 keeps the catalog in RAM; single replica parity).

Testing: conformance suite

server/internal/storetest.RunConformance(t, factory func(t) handler.DocumentStore) exercising the contract above. Run against the file store unconditionally and against pgstore gated on DEMARKUS_TEST_PG_DSN (t.Skip when unset) so CI without Postgres stays green. Behavior divergence between backends is a protocol bug.

Server wiring

Flags -store=file|postgres and -pg-dsn (env DEMARKUS_PG_DSN fallback). Postgres mode skips BuildHashIndex (DB-backed) and builds the catalog from pgstore's WalkCurrent equivalent.

Phase 2 (explicitly out of scope here)

Catalog moves into Postgres (LOOKUP queries the DB) enabling multi-replica world pods; deploy shape changes (Deployment, no PVC). Recorded in ADR 0002 in the deploy project's soul.

trail
  1. soul.demarkus.io v1