Federation: Agent-Driven Hash Discovery

Context

Content-addressed fetch shipped. Every FETCH response includes content-hash, servers maintain in-memory hash indexes, and FETCH /sha256-<64hex> works. The hub pattern is live at mark://hub.demarkus.io. This plan adds two MCP tools implementing the agent-driven discovery pattern from SPEC.md Section 12.1.

What Shipped

Two new MCP tools + one new package. Zero server changes. Zero new verbs.

client/internal/index package

Parse, Build, and Merge functions for markdown hash index documents. Format:

# Content Index

> Source: mark://docs.example.com
> Indexed: 2026-03-07T15:30:00Z
> Documents: 42

| Hash | Server | Path |
|------|--------|------|
| sha256-<64hex> | mark://docs.example.com | /guide.md |

mark_resolve MCP tool

Resolves content by hash using a hub index document. Validates hash format, fetches the index, finds matching servers, tries each one, verifies content-hash in response matches.

mark_index MCP tool

Crawls a source server (LIST + FETCH), collects content-hash from every document, publishes a hash index to a target hub.

Safety features:

• Manifest check enforced by tool — target must have /.well-known/agent-manifest.md or tool refuses to publish
• force=true — explicit override, visible in MCP approval
• dry_run — returns index without publishing
• Source manifest warning — warns if source has no manifest (but proceeds)
• 1000 doc cap — prevents runaway crawls

Files

Key Design Decisions

• No server changes — federation is agent-layer concern
• Index is a markdown document — published via PUBLISH, fetched via FETCH
• Manifest check is tool-enforced — agent can't bypass without explicit force
• Hash verification on resolve — defense in depth against untrusted servers
• Merge preserves multi-source indexes — single hub can aggregate many servers

Content Addressing: Semantic Identity Across Servers

Every document has a SHA-256 hash based on its content. Same hash = same document, regardless of path or server.

This simple fact enables:

Mirroring

A document exists at /faq.md on server A and /docs/faq.md on server B. Both produce the same hash. The index lists both locations. Agents fetch from any server — deduplication happens automatically.

Deduplication

Server C copies the FAQ (maybe for resilience, maybe for local archival). Same hash. The index now has three entries. Agents discover all three without coordination between servers.

Verification

You resolve a hash and receive a document. The content-hash in the response is cryptographic proof the content matches. No trust in paths, server behavior, or transport — the hash is the contract.

Location-Independent Discovery

Agents ask "what document has this hash?" instead of "where is /faq.md?". Documents can move between servers, be renamed, be republished — the hash stays the same. Clients and agents stay decoupled from paths.

Federation Without Coordination

Server A publishes docs on authentication. Server B independently publishes docs on the same topic (maybe a translation, maybe a different perspective). Both get indexed under the same hash. Hubs aggregate them. No central registration, no pre-coordination. Agents discover both automatically.

Convergence

When many independent teams document the same knowledge domain, their hashes converge. The index naturally groups equivalent content. Agents can present "here are 5 servers with the same authentication guide" without anyone asking them to coordinate.

This is the foundation of distributed knowledge networks — documents have stable identities across servers and time.