soul.demarkus.io:6309/index/decisions.md/v39 draft reader meta

Decisions

Architecture decision records for the demarkus core, for maintainers and agents finding what was settled and where the canonical copy lives.

Part of the demarkus-soul hub. Canonical here since 2026-09-28; docs/adr/ in git is a frozen archive of 0001 to 0027 (see where decisions live). Sub-project decisions live in their own series, for example /demarkus-library/adr/. Status, dates and supersession live in each ADR.

  • Decisions 0001 to 0020: the first twenty, from the broker web-client registry to the reconcile owner

  • ADR 0021: world registry announces drops, DNS label world names, one tenant door

  • ADR 0022: the MCP gateway takes its dependencies from Run, not the Server

  • ADR 0023: the broker is core, oauthsrv, gateway and storage behind one Run

  • ADR 0024: plugin binary packages by concern, a harness is a host.Host value

  • ADR 0025: WATCH is the eighth verb, the store emits hints only, protocol 1.0 freezes after it

  • ADR 0026: a client SDK under MIT with its own semver, the server never imports it

  • ADR 0027: world token files are optional at open and split into runtime and static files

  • ADR 0028: every store owns a durable change epoch and sequence behind backend.ChangeSource

  • ADR 0029: the broker issues the federation agent's hub token from agentTokens

  • ADR 0030: the broker generates and persists its state cookie key

  • ADR 0031: the bucket store seals receipts into change blocks the hub reads as a backlog

  • ADR 0032: the knowledge and memory brokers run inside the knowledge server process

  • ADR 0033: the knowledge server's bearer listener takes an identity bearer in the auth field

  • ADR 0034: the knowledge server derives the federation graph in process into the hub's checkpoints

  • ADR 0035: drop content-addressed fetch and the hash index tools before protocol 1.0

  • ADR 0036: the bucket store commits through create-only log slots with group commit

  • ADR 0037: the contract is the protocol, transports are bindings, two doors by credential kind

  • ADR 0038: a broker write carries the verified email as user and the MCP client name as agent

  • ADR 0039: drop the standalone federation agent before 1.0; the knowledge server is the only graph producer

  • ADR 0040: memory mechanics (gate, doctor, journal, context, curate, promote staging) live in the server; skills keep intent and the gate

  • ADR 0041: the knowledge gateway serves the hub's four branding documents without a bearer, so the library brands a cold pod and its sign-in page

  • ADR 0042: soul entries are agent-written; software supplies facts, never entries

  • ADR 0043: the memory model lives in client/memory/, a directory of the client module, not its own module

  • ADR 0044: client/mdoutline splits into protocol/mdoutline, the model's client/memory/outline and marktools

  • ADR 0045: write gates (secrets, personal data, policy) run in the store in both tiers, warns reach the writer under a reserved warning key, the hook mirrors the scan pre-write

soul.demarkus.io:6309/adr/0034-federation-graph-derived-in-process.md draft reader meta

ADR 0034: The federation graph is derived in the knowledge server

Status: proposed (2026-10-02). PR 536, branch feat/incremental-federation. Supersedes ADR 0029 and amends ADR 0027. Follows ADR 0032. Plans: incremental federation and cutover. Part of the decisions hub.

Context

The federation agent crawled every world every five minutes, whether or not anything had changed. It then published the hub's graph, snapshot and hash indexes with a publish token the broker minted for it (ADR 0029). The token's source changed three times in four days. Production depended on a Job-minted Secret that External Secrets copied and nothing recreated. About 730 reads and 8 writes a cycle went to output that was mostly the agent's own. Every world store now emits a durable change feed (ADR 0028), and the knowledge server and broker share one process (ADR 0032).

Decision

  • The deriver. One knowledge server replica, under the Lease demarkus-federation, derives every static knowledge world's graph from its change feed through an in-process WATCH. It checkpoints the graph into the hub as per-world, version-pinned shards and a CAS manifest (/graph/worlds/<world>/), writing only shards whose bytes changed. It also renders the hub's /graph.md from those checkpoints, for the library. All of it runs in process, under protocol.Grant for /graph/worlds/** and /graph.md, with no token and no Secret.
  • Readers. The knowledge gateway seeds those worlds from the checkpoints. mark_resolve asks the caller's worlds directly instead of a published hash index.
  • The universe map. It covers the org's worlds. Servers outside them (production's soul.demarkus.io) appear as link targets, not crawled nodes, until registered remote worlds are derived too.
  • Agent tokens. The broker issues none: agentTokens, its reconciler, its records and the agent-token Role rule are removed. So is the broker world config's tokensSecret, which only that reconciler used.
  • The world's runtime tokens file. It keeps its mount, but no broker writes it. The bootstrap Job or the operator owns it. This amends ADR 0027, under which the broker owned that file.
  • Charts. broker.federation.hub defaults to the one world marked hub: true. The umbrella chart drops its agent subchart. The demarkus-agent chart and image stay published standalone.

Consequences

  • Production. It needs no hub token, no External Secret and no agent Deployment. An idle system does no federation work beyond the backstop poll.
  • Cost of an edit. Three hub writes (shard, manifest, /graph.md), about 4.5 s at the bucket store's commit interval, and /graph.md is rewritten at most once per interval. The 1 MiB body limit on /graph.md stays: past it the export is skipped and logged, so the library must move to the world manifests before the org outgrows it.
  • Library. It loses the outside servers the agent used to crawl.
  • Stale configs. A config still carrying agentTokens or tokensSecret fails at load, naming the field.
  • Deploy order. The deploy removes the agent app in the same sync that turns federation on, so /graph.md never has two writers. The agent's snapshot and hash indexes in the hub are archived afterwards.
trail
  1. soul.demarkus.io:6309 v39
  2. 0034-federation-graph-derived-in-process
from here → 0027-world-token-files-optional-and-split 0028-stores-own-a-durable-change-sequence 0029-broker-issues-agent-hub-tokens 0032-broker-in-the-knowledge-server-process decisions agent-in-process