Demarkus / Knowledge System FAQ

For the RFC review session. Sources: demarkus repo, demarkus-knowledge-system-deploy repo. Status: WIP.

How do you interact with the knowledge system?

Agents: mark_* MCP tools against the broker over HTTPS, addressing documents as mark://<world>/<path>. Auth is MCP OAuth device flow at the broker. Humans: the library web app (an OAuth web client of the broker), or demarkus CLI / demarkus-tui direct to a world over QUIC where reachable. Navigation anchors on the root hub; policy and templates live under mark://root/.well-known/demarkus/.

What are a world, a hub, root, and an index?

• World: one demarkus server inside a knowledge system, addressed by logical name (mark://<world>/<path>). It has its own store, tokens, and lifecycle, and describes itself in a world.md descriptor.
• Hub: a server that aggregates cross-server state: content-hash indexes published by mark_index and the /graph.md link-graph export. mark_resolve and backlink seeding read from it.
• Root: the guaranteed world every knowledge system has, acting as its hub. It holds the global entry point and the org conventions under /.well-known/demarkus/ (policy, template, style); the federation crawler indexes every world's content hashes into it.
• Index: index.md is a world's or directory's curated entry-point document, the discovery backstop for anything lookup can't surface. Separately, the crawler publishes machine-generated hash indexes (/index/<host>.md) to the hub for content-addressed resolution.

What automated cues push agents to record memory (journal, ADRs, etc.)?

Five hooks in the memory plugin:

• SessionStart: injects the routing table (decisions to /adr/, gotchas to debugging.md, progress to journal/).
• Stop: journal nudge when files changed but nothing was written to the soul.
• PostToolUse on mark_publish: promote nudge when a new ADR lands.
• UserPromptSubmit: recall-first reminder on "did we decide" questions.
• Pre/PostToolUse tag-gate: enforces tags and importance on every write.

How does content get into the knowledge system?

Through the promote bridge. /promote <soul-path> runs the knowledge-promote cascade: triage → distill for a shared audience (strip personal framing, secrets, PII) → dedup against the catalog → tag to the system taxonomy → route to a writable world → human gate → mark_publish with provenance → back-stamp the soul source. Direct mark_publish to a joined system also works and passes the same tag-gate and policy.

How does finding knowledge work?

mark_lookup is the card catalog: it matches a query against declared tags and titles and returns an importance-ranked table (path, importance, title, tags), with tag= and modified-after= filters. It is not full-text search; untagged documents are invisible to it. From a hit: mark_explore to orient, mark_fetch url#anchor for the sections needed. Backstops: index.md hubs, mark_backlinks and mark_graph for link traversal, mark_discover for a server's agent manifest.

How is relevance decided?

Deterministic scoring, no embeddings. Score = count of distinct query terms matching tags (exact, case-insensitive) or title (substring, case-insensitive). Sort order: score desc, then declared importance, then modification time, then path. Filters apply before ranking. (server/internal/catalog/catalog.go)

How is importance stamped on a document?

The publisher declares it: metadata.importance on mark_publish, float in [0,1], stored out of band and indexed into the catalog. Absent, unparseable, or out-of-range values default to 0.5. mark_append carries no metadata, so the value holds until the next mark_publish.

How does the agent decide importance?

Judgment steered by injected guidance, not computed. SessionStart context and the soul-memory / knowledge-promote skills instruct: reserve 0.8+ for hubs, architecture, and key decisions; routine notes lower. The server never infers it; the gate only validates the range.

How does the document graph work?

mark_graph crawls outbound mark:// links (depth default 2, max 5). Edges carry provenance (link label, source section anchor, occurrence count) plus typed relations from rel-<predicate> publisher metadata, e.g. rel-supersedes. Crawls persist to a graph store that answers mark_backlinks, seeded from a published /graph.md when available; local crawls take precedence. mark_graph_publish republishes the store as a crawlable /graph.md (generated doc, default retention 20).

How does the graph work world to world in a knowledge system?

Cross-world links are ordinary mark://{worldName}/{path} links; the broker resolves the logical name to an internal address, so crawls cross world boundaries and clients never see internal topology. The broker's graph store is in-memory and pod-scoped. It seeds on demand from each configured world's published /graph.md, so cold pods answer backlinks without a crawl; a world without /graph.md falls back to unseeded behavior.

Is the information on graph edges there to guide agents to relevant information?

Yes (ADR 0004). The label says what the linker calls the target, the anchor points at the exact source section for a mark_fetch url#anchor jump, the count signals link strength, and rel- types answer "what superseded this" rather than bare "mentions". Principle from the ADR: the agent owns judgment, the server owns accumulation.

What does the agent use to decide whether to read a document?

Catalog and graph evidence before bodies: the mark_lookup row (path, importance, title, tags, modified time), backlink provenance (label, anchor, count, rel- type), and mark_explore's bundle of outline, links, backlinks, and siblings. Reading is staged: mark_fetch returns an outline instead of the body above 8KB, so the agent targets a #section anchor. mark_discover manifests and hub index.md pages set context first.

Why deploy a knowledge system on a k8s cluster?

Because a knowledge system is inherently distributed: many worlds, each its own server with its own store and lifecycle, composed behind one broker. k8s matches that shape. Each world runs and fails independently, so one team's world going down does not take the catalog with it. Worlds are added or upgraded without touching the others, and the whole universe is declared in one place and converges to that declaration. For smaller needs the reference deployment itself recommends one or two plain deploys at a fraction of the cost.

How does the system keep data fresh?

Reads dispatch to the world's server; repeat fetches of an unchanged document return an "unchanged" notice rather than stale content. Each write creates a new hash-chained version. Optimistic concurrency (expected_version plus merge-on-conflict) stops stale writes clobbering newer ones. The federation crawler re-crawls hourly, indexing every world's content hashes and graph into the root hub. /soul-refresh pulls promoted soul copies forward from the authoritative knowledge copy; local edits re-enter only through /promote.

How do knowledge diffs and merges work? When does the agent merge documents?

Two layers. Concurrent edits to one document: a conflicting mark_publish returns a diff3 merge candidate (base, ours, theirs, git-style markers); the machine merges structure, the agent reviews semantics and republishes. Cohesion across documents: the promote cascade's dedup step looks up the subject, fetches close matches, and prefers a gated update to an existing doc over a near-duplicate; conflicts go to the human gate, never a silent overwrite. /knowledge-doctor sweeps find content-hash duplicates across paths and worlds after the fact.

Why does federation matter, and why a server per world or memory?

No central authority or registry: anyone can run a server, content mirrors freely, and the hash chain lets agents on different mirrors verify they hold identical versions. A server per world makes the ownership boundary physical: each world has its own store, token file, writer allowlist, and lifecycle, so teams evolve independently and blast radius stays contained. A soul is the same server at personal scale. Cross-server discovery sits on top: the federation crawler indexes content hashes into the root hub, and mark_resolve fetches by hash across servers.

What tools does the memory plugin expose?

Fifteen mark_* MCP tools:

What tools does the knowledge plugin expose?

Sixteen, via the broker MCP gateway: the same 15 as the memory plugin (deliberate parity, one vocabulary regardless of transport) plus mark_worlds, which enumerates the system's worlds with their world.md descriptors. mark_worlds is broker-only because the local MCP's universe is its one world. URLs address worlds by logical name instead of host:port. Writes pass a per-world writer allowlist with one shared per-world token; SSO is the org gate.

How is knowledge formatted and held to a standard?

The standard is itself published on root under .well-known/demarkus/: policy.md (strictness, required tag axes such as category:, optional required OKF fields), template.md (per-world layout), style.md (H1 as name, one-sentence summary under it, unique headings because headings are anchors, no em dashes, no frontmatter fences). Joined agents mirror the policy into local write-time gates that check tags, axes, importance range, and the mechanically checkable style rules, at the declared severity (warn / block / ask). /knowledge-doctor audits the corpus after the fact.

Why QUIC?

Encryption is mandatory: QUIC (RFC 9000) carries TLS 1.3 with no plaintext fallback. Also: multiplexed streams (one bidirectional stream per exchange), connection migration, 0-RTT resumption restricted to idempotent verbs, and none of HTTP's accumulated surface: no cookies, session identifiers, Referer or User-Agent headers, query strings, tracking parameters, or content negotiation. Seven text verbs with frontmatter parameters instead. The broker fronts QUIC worlds over HTTPS, so enterprise clients need no direct QUIC reachability.

What is the auth and security model?

Capability tokens on the server: writes are denied unless a token store is configured; reads are public unless a token grants read on a path pattern, which makes matching paths require one (protect /** for a private intranet). In a knowledge system, OIDC SSO is the org gate: reads are open to any authenticated identity, writes pass a per-world writer allowlist at the broker, dispatched with one shared long-lived per-world token. Policy forbids publishing secrets, credentials, or PII.

How are versions and integrity handled?

Every write creates an immutable new version linked by a SHA-256 hash chain; mark_versions validates the chain, and mirrors can verify identical content by version and hash. Nothing is deleted by default: mark_archive hides a document and keeps history. The one destructive path is metadata.retention, which permanently prunes old versions; it exists for generated documents, and clients warn before applying it elsewhere.

Why no full-text or semantic search?

Deliberate (SPEC §6.7). LOOKUP is a per-world catalog over author-declared tags and importance; it never reads bodies and there is no centralized index. Full-text and semantic search stay out of core as an opt-in sidecar reading demarkus over LIST/FETCH. This keeps the server simple and deterministic and puts ranking judgment in the agent.

What are the known limitations?

The graph is not yet optimal: the broker's store is in-memory and pod-scoped, crawls are on-demand, and edge semantics are recent (ADR 0004). Full-text or semantic search requires an added component; until then recall depends on tagging discipline. Behavior at large scale is unproven: catalog size, crawl cost, and cross-world graph growth have not been tested at enterprise corpus sizes.