Demarkus / Knowledge System FAQ

For the RFC review session. Sources: demarkus repo, demarkus-knowledge-system-deploy repo. Status: WIP.

How do you interact with the knowledge system?

Agents and humans use different doors to the same catalog.

• Agents: mark_* MCP tools against the broker over HTTPS, URLs as mark://<world>/<path>.
• Auth: MCP OAuth device flow at the broker.
• Humans: the library web app (OAuth web client of the broker), or demarkus CLI / demarkus-tui direct to a world over QUIC.
• Navigation anchors on the root hub; policy and templates under mark://root/.well-known/demarkus/.

What are a world, a hub, root, and an index?

• World: one demarkus server in a knowledge system, addressed by logical name. Own store, tokens, lifecycle; describes itself in world.md.
• Hub: a server aggregating cross-server state: content-hash indexes from mark_index and the /graph.md export. mark_resolve and backlink seeding read from it.
• Root: the guaranteed world every system has, acting as its hub. Holds the global entry point and org conventions under /.well-known/demarkus/.
• Index: index.md is a curated entry-point document, the discovery backstop for what lookup can't surface. The crawler separately publishes machine hash indexes (/index/<host>.md) to the hub.

What automated cues push agents to record memory?

Five hooks in the memory plugin.

• SessionStart: injects the routing table (decisions to /adr/, gotchas to debugging.md, progress to journal/).
• Stop: journal nudge when files changed but nothing was written to the soul.
• PostToolUse on mark_publish: promote nudge when a new ADR lands.
• UserPromptSubmit: recall-first reminder on "did we decide" questions.
• Pre/PostToolUse tag-gate: enforces tags and importance on every write.

How does content get into the knowledge system?

Through the promote bridge: /promote <soul-path> runs the knowledge-promote cascade.

• Triage → distill for a shared audience (strip personal framing, secrets, PII).
• Dedup against the catalog → tag to the system taxonomy → route to a writable world.
• Human gate → mark_publish with provenance → back-stamp the soul source.
• Direct mark_publish to a joined system also works; it passes the same tag-gate and policy.

How does finding knowledge work?

mark_lookup is the card catalog; the graph and hubs back it up.

• Lookup matches a query against declared tags and titles; returns an importance-ranked table (path, importance, title, tags).
• Filters: tag=, modified-after=. Not full-text search; untagged documents are invisible to it.
• From a hit: mark_explore to orient, mark_fetch url#anchor for the sections needed.
• Backstops: index.md hubs, mark_backlinks / mark_graph for link traversal, mark_discover for a server's manifest.

How is relevance decided?

Deterministic scoring, no embeddings (server/internal/catalog/catalog.go).

• Score = count of distinct query terms matching tags (exact, case-insensitive) or title (substring, case-insensitive).
• Sort: score desc, then declared importance, then modification time, then path.
• Filters apply before ranking.

How is importance stamped on a document?

The publisher declares it.

• metadata.importance on mark_publish, float in [0,1], stored out of band, indexed into the catalog.
• Absent, unparseable, or out-of-range values default to 0.5.
• mark_append carries no metadata; the value holds until the next mark_publish.

How does the agent decide importance?

Judgment steered by injected guidance, not computed.

• SessionStart context and the soul-memory / knowledge-promote skills instruct: reserve 0.8+ for hubs, architecture, key decisions; routine notes lower.
• The server never infers it; the gate only validates the range.

How does the document graph work?

mark_graph crawls outbound mark:// links (depth default 2, max 5).

• Edges carry provenance: link label, source section anchor, occurrence count.
• Typed relations come from rel-<predicate> publisher metadata, e.g. rel-supersedes.
• Crawls persist to a graph store that answers mark_backlinks; seeded from a published /graph.md, local crawls take precedence.
• mark_graph_publish republishes the store as a crawlable /graph.md (generated doc, default retention 20).

How does the graph work world to world?

Cross-world links are ordinary mark://{worldName}/{path} links.

• The broker resolves the logical name to an internal address; crawls cross world boundaries, clients never see internal topology.
• The broker's graph store is in-memory and pod-scoped.
• It seeds on demand from each world's published /graph.md, so cold pods answer backlinks without a crawl.
• A world without /graph.md falls back to unseeded behavior.

Is edge information there to guide agents to relevant information?

Yes (ADR 0004).

• Label: what the linker calls the target.
• Anchor: the exact source section, a direct mark_fetch url#anchor jump.
• Count: link strength signal.
• rel- types: "what superseded this" rather than bare "mentions".
• Principle: the agent owns judgment, the server owns accumulation.

What does the agent use to decide whether to read a document?

Catalog and graph evidence before bodies.

• The mark_lookup row: path, importance, title, tags, modified time.
• Backlink provenance: label, anchor, count, rel- type.
• mark_explore: outline, links, backlinks, siblings in one call.
• Staged reading: mark_fetch returns an outline above 8KB, so the agent targets a #section anchor.
• mark_discover manifests and hub index.md pages set context first.

Why deploy a knowledge system on k8s?

A knowledge system is inherently distributed; k8s matches that shape.

• Many worlds, each its own server with its own store and lifecycle, composed behind one broker.
• Worlds run and fail independently; one world going down does not take the catalog with it.
• Worlds are added or upgraded without touching the others.
• The whole universe is declared in one place and converges to that declaration.
• Smaller needs: the reference deployment itself recommends one or two plain deploys at a fraction of the cost.

How does the system keep data fresh?

• Reads dispatch to the world's server; repeat fetches of an unchanged document return an "unchanged" notice.
• Every write creates a new hash-chained version.
• Optimistic concurrency (expected_version plus merge-on-conflict) stops stale writes clobbering newer ones.
• The federation crawler re-crawls hourly, indexing every world's content hashes and graph into root.
• /soul-refresh pulls promoted soul copies forward from the authoritative knowledge copy; local edits re-enter only through /promote.

How do diffs and merges work? When does the agent merge documents?

Two layers.

• Same document, concurrent edits: a conflicting mark_publish returns a diff3 merge candidate (base, ours, theirs, git-style markers). The machine merges structure; the agent reviews semantics and republishes.
• Across documents: the promote cascade's dedup step looks up the subject, fetches close matches, and prefers a gated update to an existing doc over a near-duplicate. Conflicts go to the human gate, never a silent overwrite.
• Backstop: /knowledge-doctor sweeps find content-hash duplicates across paths and worlds.

Why does federation matter, and why a server per world or memory?

No central authority or registry.

• Anyone can run a server; content mirrors freely.
• The hash chain lets agents on different mirrors verify they hold identical versions.
• A server per world makes the ownership boundary physical: own store, token file, writer allowlist, lifecycle. Teams evolve independently; blast radius stays contained.
• A soul is the same server at personal scale.
• Cross-server discovery sits on top: the crawler indexes content hashes into root; mark_resolve fetches by hash across servers.

What tools does the memory plugin expose?

Fifteen mark_* MCP tools.

What tools does the knowledge plugin expose?

Sixteen, via the broker MCP gateway.

• The same 15 as the memory plugin: deliberate parity, one vocabulary regardless of transport.
• Plus mark_worlds: enumerates the system's worlds with their world.md descriptors. Broker-only, because the local MCP's universe is its one world.
• URLs address worlds by logical name instead of host:port.
• Writes pass a per-world writer allowlist with one shared per-world token; SSO is the org gate.

How is knowledge formatted and held to a standard?

The standard is itself published on root under .well-known/demarkus/.

• policy.md: strictness, required tag axes such as category:, optional required OKF fields.
• template.md: per-world layout.
• style.md: H1 as name, one-sentence summary under it, unique headings (headings are anchors), no em dashes, no frontmatter fences.
• Joined agents mirror the policy into local write-time gates: tags, axes, importance range, mechanically checkable style rules, at the declared severity (warn / block / ask).
• /knowledge-doctor audits the corpus after the fact.

Why QUIC?

Encryption is mandatory: QUIC (RFC 9000) carries TLS 1.3 with no plaintext fallback.

• Multiplexed streams: one bidirectional stream per exchange.
• Connection migration; 0-RTT resumption restricted to idempotent verbs.
• None of HTTP's accumulated surface: no cookies, session identifiers, Referer or User-Agent headers, query strings, tracking parameters, or content negotiation.
• Seven text verbs with frontmatter parameters instead.
• The broker fronts QUIC worlds over HTTPS, so enterprise clients need no direct QUIC reachability.

What is the auth and security model?

Capability tokens on the server; SSO at the broker.

• Writes are denied unless a token store is configured.
• Reads are public unless a token grants read on a path pattern, which makes matching paths require one (protect /** for a private intranet).
• Knowledge system: OIDC SSO is the org gate. Reads open to any authenticated identity; writes pass a per-world writer allowlist, dispatched with one shared long-lived per-world token.
• Policy forbids publishing secrets, credentials, or PII.

How are versions and integrity handled?

• Every write creates an immutable new version linked by a SHA-256 hash chain.
• mark_versions validates the chain; mirrors verify identical content by version and hash.
• Nothing is deleted by default: mark_archive hides a document and keeps history.
• The one destructive path is metadata.retention, which permanently prunes old versions. It exists for generated documents; clients warn before applying it elsewhere.

Why no full-text or semantic search?

Deliberate (SPEC §6.7).

• LOOKUP is a per-world catalog over author-declared tags and importance. It never reads bodies; there is no centralized index.
• Full-text and semantic search stay out of core as an opt-in sidecar reading demarkus over LIST/FETCH.
• Keeps the server simple and deterministic; ranking judgment stays in the agent.

What are the known limitations?

• The graph is not yet optimal: the broker's store is in-memory and pod-scoped, crawls are on-demand, edge semantics are recent (ADR 0004).
• Full-text or semantic search requires an added component; until then recall depends on tagging discipline.
• Behavior at large scale is unproven: catalog size, crawl cost, and cross-world graph growth have not been tested at enterprise corpus sizes.