soul.demarkus.io/plans/plugin-knowledge-quality.md/v3 draft reader meta

Plan: Plugin Knowledge-Quality Enforcement

Status: active (opened 2026-06-01). Raises the demarkus-memory Claude Code plugin from "advisory knowledge system" to "enforced knowledge system," and aligns the per-project template to the proven demarkus-soul structure.

Owner: Fritz. Driver: agent.

Problem

The plugin splits into two layers at very different maturity:

  • Infra layer (binary install, server spawn, token mint, version pins, health checks): solid, deterministic, harness-executed.
  • Knowledge-quality layer (context/session-guidance.md, the soul-memory skill, routing + tagging rules): 100% advisory prose executed by the model on best effort. Nothing verifies any of it happened.

Every load-bearing quality assumption — tag every publish, recall before answering, route content to the right file, journal significant sessions — is hope, not enforcement. The server "infers nothing." So quality degrades silently and invisibly: a tagless publish is accepted happily and becomes permanently invisible to mark_lookup. That is the single most damaging hole, because the entire recall half of the system depends on tags the agent must remember to set by hand while focused on content.

Strategy

Move enforcement off advisory prose and onto hooks, which the harness executes deterministically. Pure composition of existing surface — no core changes (consistent with the core-vs-overlay layering rule: the server stays permissive; the agent-driven client gets the guardrail).

Severity is configurable but the check always runs — this is a strictness level, not an opt-in capability (no "validation off" deployment exists):

  • warn (default) — PostToolUse injects a system-reminder after a tagless publish, nudging a re-publish with tags. Non-blocking.
  • block — PreToolUse denies a tagless publish until tags are added.
  • ask — PreToolUse escalates to the user.

Hook mechanics (verified against code.claude.com/docs/en/hooks)

  • PreToolUse: hookSpecificOutput.permissionDecision in allow|deny|ask|defer + permissionDecisionReason. Carries block/ask.
  • PostToolUse: hookSpecificOutput.additionalContext injects a system-reminder the model sees next turn. Carries warn. (Plain stdout is debug-only, not model-visible.)
  • Matchers are JS regex against the full mcp__server__tool name. Scope to mcp__.*demarkus-memory__mark_publish so it hits the plugin's own server, not the user's other demarkus servers (soul/hub/knowledge).
  • One dispatcher script handles both events; branches on hook_event_name, reads STRICTNESS. Zero runtime dependencies — the payload is parsed by a pure-awk character-level JSON scanner (no jq, no python; awk is already the engine behind json_escape). A naive grep would false-match the arbitrary body value, so the scanner tracks string boundaries + the object-key path and reads only the exact fields it needs. Fails open — any parse hiccup defers, never blocks.

Work items

1. Publish tag/importance gate — DONE 2026-06-01 (PR #171, pending merge)

hooks/publish-gate.sh (one dispatcher on both PreToolUse + PostToolUse) + matcher wiring in plugin.json. Tagless publish (tool_input.metadata.tags absent/empty/whitespace, or a non-string tags value) or importance outside [0,1] → warn/block/ask per strictness. Strictness read by configured_strictness in lib.sh: DEMARKUS_MEMORY_STRICTNESS env override → ~/.demarkus/plugin-memory.strictness file → warn default (kept in its own file, orthogonal to plugin-memory.conf, so setup.sh rewrites never clobber it). Parser is the pure-awk publish_metadata_check in lib.sh. Pins bumped SERVER 0.17.13→0.17.14 (client/tools already latest). Plugin version 0.3.0→0.4.0.

CodeRabbit review (PR #171) surfaced four real correctness holes in the awk parser, all fixed + regression-tested (suite now 21 cases, green):

  • escaped-whitespace bypass — {"tags":"\n"} was stored raw (backslash+n) and passed; added a pure-awk json_unescape (decodes \n \t \r \b \f \/ \" \\ and \uXXXX; whitespace codepoints → space, others → non-space marker) applied before trim/range checks.
  • non-string tags bypass — {"tags":false|null|0} passed; recordValue now tracks whether the value came from a JSON string (tagsStr) and tags_ok requires it.
  • exponent importance — 1e-1 was wrongly rejected; regex now allows ([eE][+-]?[0-9]+)?.
  • escaped-whitespace test coverage added.

Follow-up noted: the plugin's shell tests (tests/*.sh) are not run by any CI workflow today — knowledge-join_test.sh is manual too. Wiring them into CI is a separate, small task (touches .github/workflows/, mind the YAML ${{ }}-in-comments rule).

2. Relocate decaying guidance to the moment of action — DONE 2026-06-01 (PR #171)

session-guidance.md tagging block tightened: the gate now carries the tagging rule at write time, so the startup prose states it tersely, points at the gate, and makes the enforced-vs-advisory split explicit (tagging is backstopped; recall / journaling / routing are not). Lighter startup payload = less to summarize away in long sessions.

3. Session-end journal nudge (Stop hook, guarded)

Stop hook checks: did the session do real work but write nothing to the soul? If so, nudge to journal. Guard against re-prompt loops (only fire once per session; respect a sentinel). Next up.

4. Recall-before-answering (deferred until 1-3 land)

Hardest, most false-positive-prone. Good capture is the precondition for good recall — fix tags first. Cheap version: UserPromptSubmit pattern-match ("did we / what did / last time / decide") → one-line "check the soul" nudge.

5. Catalog hygiene tooling (/soul-doctor)

Surfaces existing rot: orphan docs (no inbound links, not in any index), broken [[links]], duplicate content, stale index entries, ADR numbering collisions. Built on mark_backlinks / mark_graph. Tooling, not enforcement.

6. Align local per-project template to the proven demarkus-soul structure

The current template (plan/tasks.md, architecture.md, patterns.md, roadmap.md, adr/NNNN, journal/) is thin, defined three times with drift (session-guidance.md, SKILL.md, seed/index.md), and poorer than the structure demarkus-soul actually evolved. Adopt the proven layout: a maintained per-project index.md hub, architecture.md, patterns.md, guidelines.md, debugging.md (highest recall value — lessons from bugs), roadmap.md, debt.md, adr/NNNN-*.md, plans/ as a first-class lifecycle (active/completed/archived), journal/<YYYY-MM-DD>.md, thoughts.md. Define the canonical template once; the other docs reference it (kills the three-way drift). Confirmed by Fritz 2026-06-01: lift the proven soul structure wholesale. Likely its own branch/PR (separate from the gate PR).

7. Knowledge-system template + policy, anchored at the root hub

Every knowledge system gets a guaranteed hub world named root that tracks things globally (Fritz's call 2026-06-01). The org template + policy live there as ordinary versioned docs:

  • mark://root/.well-known/demarkus/template.md — required structure (same canonical layout as #6).
  • mark://root/.well-known/demarkus/policy.md — strictness level + required tags / rules.

Read path is constrained by a hard fact: hooks cannot authenticate to the broker (the OAuth token lives in Claude Code's MCP machinery, not a file). So:

  • The tag-presence gate works unchanged on knowledge-system publishes — tags are in the tool-call input the hook receives; no broker access needed. Matcher extends to mcp__<slug>__mark_publish (the per-slug names /knowledge-join registers).
  • The template/policy are agent-read (only the agent can reach the broker via the MCP OAuth session). Session guidance: on a knowledge system, fetch root policy/template first and follow it. Advisory by necessity, but it is structure convention, not a security boundary.
  • For enforced strictness, the agent is courier: reads policy.md's declared level → writes plugin-memory.strictness.<slug> locally → the credential-less hook reads it off disk and enforces. Org declares once on root; agent mirrors; hook enforces.

Customization = the org edits the two root docs. Versioned like any demarkus doc; propagates to every member on next session. No plugin release, no per-user setup. Open question: how the plugin learns which world is root — prefer the broker advertising it in /.well-known metadata (the join script already hits that endpoint) over a guessed/fixed name.

Sequencing

1 → 2 → 6 (canonical template, used by 7) → 3 → 7 → 5 → 4. Items 1 + 2 done (PR #171). Next substantive in-branch piece is 3 (another enforcement hook); 6 + 7 are separate efforts.

Constraints honored

  • No core changes — pure plugin composition.
  • Zero runtime dependencies — pure awk/bash, no jq/python (Fritz 2026-06-01: do not add external deps that may not be installed).
  • Version pins bumped on every plugin change (scripts/lib.sh).
  • No opt-in capability knob — strictness is a severity level on an always-on check.
  • Plans live on the soul, not in a vendor folder.
trail
  1. soul.demarkus.io v3