# Demarkus / Knowledge System FAQ

For the RFC review session. Sources: demarkus repo, demarkus-knowledge-system-deploy repo. Status: WIP.

## How do you interact with the knowledge system?

Two access paths: MCP over HTTPS for agents, web or CLI for humans.

- Agents: `mark_*` MCP tools against the broker over HTTPS, URLs as `mark://<world>/<path>`.
- Auth: MCP OAuth device flow at the broker.
- Humans: the library web app (OAuth web client of the broker), or `demarkus` CLI / `demarkus-tui` direct to a world over QUIC.
- Navigation anchors on the `root` hub; policy and templates under `mark://root/.well-known/demarkus/`.

## What are a world, a hub, root, and an index?

- **World**: one demarkus server in a knowledge system, addressed by logical name. Own store, tokens, lifecycle; describes itself in `world.md`.
- **Hub**: a server aggregating cross-server state: content-hash indexes from `mark_index` and the `/graph.md` export. `mark_resolve` and backlink seeding read from it.
- **Root**: the guaranteed world every system has, acting as its hub. Holds the global entry point and org conventions under `/.well-known/demarkus/`.
- **Index**: `index.md` is a curated entry-point document, the discovery backstop for what lookup can't surface. The crawler separately publishes machine hash indexes (`/index/<host>.md`) to the hub.

## What automated cues push agents to record memory?

Five hooks in the memory plugin.

- SessionStart: injects the routing table (decisions to `/adr/`, gotchas to `debugging.md`, progress to `journal/`).
- Stop: journal nudge when files changed but nothing was written to the soul.
- PostToolUse on `mark_publish`: promote nudge when a new ADR lands.
- UserPromptSubmit: recall-first reminder on "did we decide" questions.
- Pre/PostToolUse tag-gate: enforces `tags` and `importance` on every write.

## How does content get into the knowledge system?

Through the promote bridge: `/promote <soul-path>` runs the `knowledge-promote` cascade.

- Triage → distill for a shared audience (strip personal framing, secrets, PII).
- Dedup against the catalog → tag to the system taxonomy → route to a writable world.
- Human gate → `mark_publish` with provenance → back-stamp the soul source.
- Direct `mark_publish` to a joined system also works; it passes the same tag-gate and policy.

## How does finding knowledge work?

Primary: `mark_lookup` catalog queries. Secondary: graph traversal and hub pages.

- Lookup matches a query against declared tags and titles; returns an importance-ranked table (path, importance, title, tags).
- Filters: `tag=`, `modified-after=`. Not full-text search; untagged documents are invisible to it.
- From a hit: `mark_explore` to orient, `mark_fetch url#anchor` for the sections needed.
- Backstops: `index.md` hubs, `mark_backlinks` / `mark_graph` for link traversal, `mark_discover` for a server's manifest.

## How is relevance decided?

Deterministic scoring, no embeddings (`server/internal/catalog/catalog.go`).

- Score = count of distinct query terms matching tags (exact, case-insensitive) or title (substring, case-insensitive).
- Sort: score desc, then declared `importance`, then modification time, then path.
- Filters apply before ranking.

## How is importance stamped on a document?

The publisher declares it.

- `metadata.importance` on `mark_publish`, float in [0,1], stored out of band, indexed into the catalog.
- Absent, unparseable, or out-of-range values default to 0.5.
- `mark_append` carries no metadata; the value holds until the next `mark_publish`.

## How does the agent decide importance?

Judgment steered by injected guidance, not computed.

- SessionStart context and the `soul-memory` / `knowledge-promote` skills instruct: reserve 0.8+ for hubs, architecture, key decisions; routine notes lower.
- The server never infers it; the gate only validates the range.

## How does the document graph work?

`mark_graph` crawls outbound `mark://` links (depth default 2, max 5).

- Edges carry provenance: link label, source section anchor, occurrence count.
- Typed relations come from `rel-<predicate>` publisher metadata, e.g. `rel-supersedes`.
- Crawls persist to a graph store that answers `mark_backlinks`; seeded from a published `/graph.md`, local crawls take precedence.
- `mark_graph_publish` republishes the store as a crawlable `/graph.md` (generated doc, default retention 20).

## How does the graph work world to world?

Cross-world links are ordinary `mark://{worldName}/{path}` links.

- The broker resolves the logical name to an internal address; crawls cross world boundaries, clients never see internal topology.
- The broker's graph store is in-memory and pod-scoped.
- It seeds on demand from each world's published `/graph.md`, so cold pods answer backlinks without a crawl.
- A world without `/graph.md` falls back to unseeded behavior.

## Is edge information there to guide agents to relevant information?

Yes (ADR 0004).

- Label: what the linker calls the target.
- Anchor: the exact source section, a direct `mark_fetch url#anchor` jump.
- Count: link strength signal.
- `rel-` types: "what superseded this" rather than bare "mentions".
- Principle: the agent owns judgment, the server owns accumulation.

## What does the agent use to decide whether to read a document?

Catalog and graph evidence before bodies.

- The `mark_lookup` row: path, importance, title, tags, modified time.
- Backlink provenance: label, anchor, count, `rel-` type.
- `mark_explore`: outline, links, backlinks, siblings in one call.
- Staged reading: `mark_fetch` returns an outline above 8KB, so the agent targets a `#section` anchor.
- `mark_discover` manifests and hub `index.md` pages set context first.

## Why deploy a knowledge system on k8s?

A knowledge system is distributed: many independent servers behind one broker.

- Each world is its own server with its own store and lifecycle.
- Worlds run and fail independently; one world going down does not take the catalog with it.
- Worlds are added or upgraded without touching the others.
- The whole universe is declared in one place and converges to that declaration.
- Smaller needs: the reference deployment itself recommends one or two plain deploys at a fraction of the cost.

## How does the system keep data fresh?

- Reads dispatch to the world's server; repeat fetches of an unchanged document return an "unchanged" notice.
- Every write creates a new hash-chained version.
- Optimistic concurrency (`expected_version` plus merge-on-conflict) stops stale writes clobbering newer ones.
- The federation crawler re-crawls hourly, indexing every world's content hashes and graph into `root`.
- `/soul-refresh` pulls promoted soul copies forward from the authoritative knowledge copy; local edits re-enter only through `/promote`.

## How do diffs and merges work? When does the agent merge documents?

Two layers.

- Same document, concurrent edits: a conflicting `mark_publish` returns a diff3 merge candidate (base, ours, theirs, git-style markers). The machine merges structure; the agent reviews semantics and republishes.
- Across documents: the promote cascade's dedup step looks up the subject, fetches close matches, and prefers a gated update to an existing doc over a near-duplicate. Conflicts go to the human gate, never a silent overwrite.
- Backstop: `/knowledge-doctor` sweeps find content-hash duplicates across paths and worlds.

## Why does federation matter, and why a server per world or memory?

No central authority or registry.

- Anyone can run a server; content mirrors freely.
- The hash chain lets agents on different mirrors verify they hold identical versions.
- A server per world makes the ownership boundary physical: own store, token file, writer allowlist, lifecycle. Teams evolve independently; blast radius stays contained.
- A soul is the same server at personal scale.
- Cross-server discovery sits on top: the crawler indexes content hashes into `root`; `mark_resolve` fetches by hash across servers.

## What tools does the memory plugin expose?

Fifteen `mark_*` MCP tools.

| Tool | Description |
|------|-------------|
| `mark_fetch` | Fetch a document or `#section`; bodies over 8KB return an outline |
| `mark_list` | List documents and subdirectories; archived hidden by default |
| `mark_explore` | One doc's outline, outbound links, backlinks, and siblings in one call |
| `mark_lookup` | Catalog lookup: importance-ranked matches on tags and titles |
| `mark_publish` | Create or update a document; metadata, optimistic concurrency, diff3 on conflict |
| `mark_append` | Append to an existing document; no metadata, auto-resolves version |
| `mark_archive` | Archive a document; hidden from listings, history preserved |
| `mark_versions` | Version history with hash-chain validation |
| `mark_graph` | Crawl outbound `mark://` links; persists edges to the graph store |
| `mark_backlinks` | What links here, with edge provenance |
| `mark_graph_export` | Export the graph store as publishable markdown |
| `mark_graph_publish` | Export and publish the graph as `/graph.md` (retention default 20) |
| `mark_discover` | Fetch a server's agent manifest |
| `mark_resolve` | Resolve content by SHA-256 hash via a hub index |
| `mark_index` | Crawl a server, publish its content-hash index to a hub |

## What tools does the knowledge plugin expose?

Sixteen, via the broker MCP gateway.

- The same 15 as the memory plugin: deliberate parity, one vocabulary regardless of transport.
- Plus `mark_worlds`: enumerates the system's worlds with their `world.md` descriptors. Broker-only, because the local MCP's universe is its one world.
- URLs address worlds by logical name instead of host:port.
- Writes pass a per-world writer allowlist with one shared per-world token; SSO is the org gate.

## How is knowledge formatted and held to a standard?

The standard is itself published on `root` under `.well-known/demarkus/`.

- `policy.md`: strictness, required tag axes such as `category:`, optional required OKF fields.
- `template.md`: per-world layout.
- `style.md`: H1 as name, one-sentence summary under it, unique headings (headings are anchors), no em dashes, no frontmatter fences.
- Joined agents mirror the policy into local write-time gates: tags, axes, importance range, mechanically checkable style rules, at the declared severity (warn / block / ask).
- `/knowledge-doctor` audits the corpus after the fact.

## Why QUIC?

Encryption is mandatory: QUIC (RFC 9000) carries TLS 1.3 with no plaintext fallback.

- Multiplexed streams: one bidirectional stream per exchange.
- Connection migration; 0-RTT resumption restricted to idempotent verbs.
- None of HTTP's accumulated surface: no cookies, session identifiers, Referer or User-Agent headers, query strings, tracking parameters, or content negotiation.
- Seven text verbs with frontmatter parameters instead.
- The broker fronts QUIC worlds over HTTPS, so enterprise clients need no direct QUIC reachability.

## What is the auth and security model?

Capability tokens on the server; SSO at the broker.

- Writes are denied unless a token store is configured.
- Reads are public unless a token grants `read` on a path pattern, which makes matching paths require one (protect `/**` for a private intranet).
- Knowledge system: OIDC SSO is the org gate. Reads open to any authenticated identity; writes pass a per-world writer allowlist, dispatched with one shared long-lived per-world token.
- Policy forbids publishing secrets, credentials, or PII.

## How are versions and integrity handled?

- Every write creates an immutable new version linked by a SHA-256 hash chain.
- `mark_versions` validates the chain; mirrors verify identical content by version and hash.
- Nothing is deleted by default: `mark_archive` hides a document and keeps history.
- The one destructive path is `metadata.retention`, which permanently prunes old versions. It exists for generated documents; clients warn before applying it elsewhere.

## Why no full-text or semantic search?

Deliberate (SPEC §6.7).

- LOOKUP is a per-world catalog over author-declared tags and importance. It never reads bodies; there is no centralized index.
- Full-text and semantic search stay out of core as an opt-in sidecar reading demarkus over LIST/FETCH.
- Keeps the server simple and deterministic; ranking judgment stays in the agent.

## What are the known limitations?

- The graph is not yet optimal: the broker's store is in-memory and pod-scoped, crawls are on-demand, edge semantics are recent (ADR 0004).
- Full-text or semantic search requires an added component; until then recall depends on tagging discipline.
- Behavior at large scale is unproven: catalog size, crawl cost, and cross-world graph growth have not been tested at enterprise corpus sizes.