# ADR 0034: The federation graph is derived in the knowledge server

Status: proposed (2026-10-02). PR 536, branch `feat/incremental-federation`. Supersedes [ADR 0029](/adr/0029-broker-issues-agent-hub-tokens.md) and amends [ADR 0027](/adr/0027-world-token-files-optional-and-split.md). Follows [ADR 0032](/adr/0032-broker-in-the-knowledge-server-process.md). Plans: [incremental federation](/plans/agent-in-process.md) and [cutover](/plans/federation-cutover.md). Part of the [decisions hub](/index/decisions.md).

## Context

The federation agent crawled every world every five minutes, whether or not anything had changed. It then published the hub's graph, snapshot and hash indexes with a publish token the broker minted for it (ADR 0029). The token's source changed three times in four days. Production depended on a Job-minted Secret that External Secrets copied and nothing recreated. About 730 reads and 8 writes a cycle went to output that was mostly the agent's own. Every world store now emits a durable change feed (ADR 0028), and the knowledge server and broker share one process (ADR 0032).

## Decision

- **The deriver.** One knowledge server replica, under the Lease `demarkus-federation`, derives every static knowledge world's graph from its change feed through an in-process WATCH. It checkpoints the graph into the hub as per-world, version-pinned shards and a CAS manifest (`/graph/worlds/<world>/`), writing only shards whose bytes changed. It also renders the hub's `/graph.md` from those checkpoints, for the library. All of it runs in process, under `protocol.Grant` for `/graph/worlds/**` and `/graph.md`, with no token and no Secret.
- **Readers.** The knowledge gateway seeds those worlds from the checkpoints. `mark_resolve` asks the caller's worlds directly instead of a published hash index.
- **The universe map.** It covers the org's worlds. Servers outside them (production's `soul.demarkus.io`) appear as link targets, not crawled nodes, until registered remote worlds are derived too.
- **Agent tokens.** The broker issues none: `agentTokens`, its reconciler, its records and the agent-token Role rule are removed. So is the broker world config's `tokensSecret`, which only that reconciler used.
- **The world's runtime tokens file.** It keeps its mount, but no broker writes it. The bootstrap Job or the operator owns it. This amends ADR 0027, under which the broker owned that file.
- **Charts.** `broker.federation.hub` defaults to the one world marked `hub: true`. The umbrella chart drops its agent subchart. The `demarkus-agent` chart and image stay published standalone.

## Consequences

- **Production.** It needs no hub token, no External Secret and no agent Deployment. An idle system does no federation work beyond the backstop poll.
- **Cost of an edit.** Three hub writes (shard, manifest, `/graph.md`), about 4.5 s at the bucket store's commit interval, and `/graph.md` is rewritten at most once per interval. The 1 MiB body limit on `/graph.md` stays: past it the export is skipped and logged, so the library must move to the world manifests before the org outgrows it.
- **Library.** It loses the outside servers the agent used to crawl.
- **Stale configs.** A config still carrying `agentTokens` or `tokensSecret` fails at load, naming the field.
- **Deploy order.** The deploy removes the agent app in the same sync that turns federation on, so `/graph.md` never has two writers. The agent's snapshot and hash indexes in the hub are archived afterwards.