Demarkus / Knowledge System FAQ

For the RFC review session. Sources: demarkus repo, demarkus-knowledge-system-deploy repo. Status: WIP.

How do you interact with the knowledge system?

Two access paths: MCP over HTTPS for agents, web or CLI for humans.

• Agents: mark_* MCP tools against the broker over HTTPS, URLs as mark://<world>/<path>.
• Auth: MCP OAuth device flow at the broker.
• Humans: the library web app (OAuth web client of the broker), or demarkus CLI / demarkus-tui direct to a world over QUIC.
• Navigation anchors on the root hub; policy and templates under mark://root/.well-known/demarkus/.

What are a world, a hub, root, and an index?

• World: one demarkus server in a knowledge system, addressed by logical name. Own store, tokens, lifecycle; describes itself in world.md.
• Hub: a server aggregating cross-server state: content-hash indexes from mark_index and the /graph.md export. mark_resolve and backlink seeding read from it.
• Root: the guaranteed world every system has, acting as its hub. Holds the global entry point and org conventions under /.well-known/demarkus/.
• Index: index.md is a curated entry-point document, the discovery backstop for what lookup can't surface. The crawler separately publishes machine hash indexes (/index/<host>.md) to the hub.

What automated cues push agents to record memory?

Five hooks in the memory plugin.

• SessionStart: injects the routing table (decisions to /adr/, gotchas to debugging.md, progress to journal/).
• Stop: journal nudge when files changed but nothing was written to the soul.
• PostToolUse on mark_publish: promote nudge when a new ADR lands.
• UserPromptSubmit: recall-first reminder on "did we decide" questions.
• Pre/PostToolUse tag-gate: enforces tags and importance on every write.

How does content get into the knowledge system?

Through the promote bridge: /promote <soul-path> runs the knowledge-promote cascade.

• Triage → distill for a shared audience (strip personal framing, secrets, PII).
• Dedup against the catalog → tag to the system taxonomy → route to a writable world.
• Human gate → mark_publish with provenance → back-stamp the soul source.
• Direct mark_publish to a joined system also works; it passes the same tag-gate and policy.

How does finding knowledge work?

Primary: mark_lookup catalog queries. Secondary: graph traversal and hub pages.

• Lookup matches a query against declared tags and titles; returns an importance-ranked table (path, importance, title, tags).
• Filters: tag=, modified-after=. Not full-text search; untagged documents are invisible to it.
• From a hit: mark_explore to orient, mark_fetch url#anchor for the sections needed.
• Backstops: index.md hubs, mark_backlinks / mark_graph for link traversal, mark_discover for a server's manifest.

How is relevance decided?

Deterministic scoring, no embeddings (server/internal/catalog/catalog.go).

• Score = count of distinct query terms matching tags (exact, case-insensitive) or title (substring, case-insensitive).
• Sort: score desc, then declared importance, then modification time, then path.
• Filters apply before ranking.

How is importance stamped on a document?

The publisher declares it.

• metadata.importance on mark_publish, float in [0,1], stored out of band, indexed into the catalog.
• Absent, unparseable, or out-of-range values default to 0.5.
• mark_append carries no metadata; the value holds until the next mark_publish.

How does the agent decide importance?

Judgment steered by injected guidance, not computed.

• SessionStart context and the soul-memory / knowledge-promote skills instruct: reserve 0.8+ for hubs, architecture, key decisions; routine notes lower.
• The server never infers it; the gate only validates the range.

How does the document graph work?

mark_graph crawls outbound mark:// links (depth default 2, max 5).

• Edges carry provenance: link label, source section anchor, occurrence count.
• Typed relations come from rel-<predicate> publisher metadata, e.g. rel-supersedes.
• Crawls persist to a graph store that answers mark_backlinks; seeded from a published /graph.md, local crawls take precedence.
• mark_graph_publish republishes the store as a crawlable /graph.md (generated doc, default retention 20).

How does the graph work world to world?

Cross-world links are ordinary mark://{worldName}/{path} links.

• The broker resolves the logical name to an internal address; crawls cross world boundaries, clients never see internal topology.
• The broker's graph store is in-memory and pod-scoped.
• It seeds on demand from each world's published /graph.md, so cold pods answer backlinks without a crawl.
• A world without /graph.md falls back to unseeded behavior.

Is edge information there to guide agents to relevant information?

Yes (ADR 0004).

• Label: what the linker calls the target.
• Anchor: the exact source section, a direct mark_fetch url#anchor jump.
• Count: link strength signal.
• rel- types: "what superseded this" rather than bare "mentions".
• Principle: the agent owns judgment, the server owns accumulation.

What does the agent use to decide whether to read a document?

Catalog and graph evidence before bodies.

• The mark_lookup row: path, importance, title, tags, modified time.
• Backlink provenance: label, anchor, count, rel- type.
• mark_explore: outline, links, backlinks, siblings in one call.
• Staged reading: mark_fetch returns an outline above 8KB, so the agent targets a #section anchor.
• mark_discover manifests and hub index.md pages set context first.

Why deploy a knowledge system on k8s?

A knowledge system is distributed: many independent servers behind one broker.

• Each world is its own server with its own store and lifecycle.
• Worlds run and fail independently; one world going down does not take the catalog with it.
• Worlds are added or upgraded without touching the others.
• The whole universe is declared in one place and converges to that declaration.
• Smaller needs: the reference deployment itself recommends one or two plain deploys at a fraction of the cost.

How does the system keep data fresh?

• Reads dispatch to the world's server; repeat fetches of an unchanged document return an "unchanged" notice.
• Every write creates a new hash-chained version.
• Optimistic concurrency (expected_version plus merge-on-conflict) stops stale writes clobbering newer ones.
• The federation crawler re-crawls hourly, indexing every world's content hashes and graph into root.
• /soul-refresh pulls promoted soul copies forward from the authoritative knowledge copy; local edits re-enter only through /promote.

How do diffs and merges work? When does the agent merge documents?

Two layers.

• Same document, concurrent edits: a conflicting mark_publish returns a diff3 merge candidate (base, ours, theirs, git-style markers). The machine merges structure; the agent reviews semantics and republishes.
• Across documents: the promote cascade's dedup step looks up the subject, fetches close matches, and prefers a gated update to an existing doc over a near-duplicate. Conflicts go to the human gate, never a silent overwrite.
• Backstop: /knowledge-doctor sweeps find content-hash duplicates across paths and worlds.

Why does federation matter, and why a server per world or memory?

No central authority or registry.

• Anyone can run a server; content mirrors freely.
• The hash chain lets agents on different mirrors verify they hold identical versions.
• A server per world makes the ownership boundary physical: own store, token file, writer allowlist, lifecycle. Teams evolve independently; blast radius stays contained.
• A soul is the same server at personal scale.
• Cross-server discovery sits on top: the crawler indexes content hashes into root; mark_resolve fetches by hash across servers.

What tools does the memory plugin expose?

Fifteen mark_* MCP tools.

What tools does the knowledge plugin expose?

The same 15 MCP tools as the memory plugin (deliberate parity, one vocabulary regardless of transport), with URLs addressing worlds by logical name instead of host:port. On top of those:

Writes pass a per-world writer allowlist with one shared per-world token; SSO is the org gate.

How is knowledge formatted and held to a standard?

The standard is itself published on root under .well-known/demarkus/.

• policy.md: strictness, required tag axes such as category:, optional required OKF fields.
• template.md: per-world layout.
• style.md: H1 as name, one-sentence summary under it, unique headings (headings are anchors), no em dashes, no frontmatter fences.
• Joined agents mirror the policy into local write-time gates: tags, axes, importance range, mechanically checkable style rules, at the declared severity (warn / block / ask).
• /knowledge-doctor audits the corpus after the fact.

Why QUIC?

• Encryption is mandatory: TLS 1.3 built in, no plaintext fallback.
• Fast multiplexed streams, one per request.
• No HTTP layer: no cookies, tracking headers, or query strings. Seven text verbs instead.
• The broker fronts worlds over HTTPS, so clients need no direct QUIC access.

What is the auth and security model?

Capability tokens on the server; SSO at the broker.

• Writes are denied unless a token store is configured.
• Reads are public unless a token grants read on a path pattern, which makes matching paths require one (protect /** for a private intranet).
• Knowledge system: OIDC SSO is the org gate. Reads open to any authenticated identity; writes pass a per-world writer allowlist, dispatched with one shared long-lived per-world token.
• Policy forbids publishing secrets, credentials, or PII.

How are versions and integrity handled?

• Every write creates an immutable new version linked by a SHA-256 hash chain.
• mark_versions validates the chain; mirrors verify identical content by version and hash.
• Nothing is deleted by default: mark_archive hides a document and keeps history.
• The one destructive path is metadata.retention, which permanently prunes old versions. It exists for generated documents; clients warn before applying it elsewhere.

Why no full-text or semantic search?

Deliberate (SPEC §6.7).

• LOOKUP is a per-world catalog over author-declared tags and importance. It never reads bodies; there is no centralized index.
• Full-text and semantic search stay out of core as an opt-in sidecar reading demarkus over LIST/FETCH.
• Keeps the server simple and deterministic; ranking judgment stays in the agent.

What are the known limitations?

• The graph is not yet optimal: the broker's store is in-memory and pod-scoped, crawls are on-demand, edge semantics are recent (ADR 0004).
• Full-text or semantic search requires an added component; until then recall depends on tagging discipline.
• Behavior at large scale is unproven: catalog size, crawl cost, and cross-world graph growth have not been tested at enterprise corpus sizes.