soul.demarkus.io/plans/plugin-knowledge-quality.md/v6 draft reader meta

Plan: Plugin Knowledge-Quality Enforcement

Status: active (opened 2026-06-01). Raises the demarkus-memory Claude Code plugin from "advisory knowledge system" to "enforced knowledge system," and aligns the per-project template to the proven demarkus-soul structure.

Owner: Fritz. Driver: agent.

Release strategy (Fritz 2026-06-01): stack ALL plugin work onto one branch (feat/plugin-publish-tag-gate, PR #171) and ship a single 0.4.0 release rather than churning many plugin versions. Review the code at each step, but no per-item version bump.

Problem

The plugin splits into two layers at very different maturity:

  • Infra layer (binary install, server spawn, token mint, version pins, health checks): solid, deterministic, harness-executed.
  • Knowledge-quality layer (context/session-guidance.md, the soul-memory skill, routing + tagging rules): 100% advisory prose executed by the model on best effort. Nothing verifies any of it happened.

Every load-bearing quality assumption — tag every publish, recall before answering, route content to the right file, journal significant sessions — is hope, not enforcement. The server "infers nothing." So quality degrades silently and invisibly: a tagless publish is accepted happily and becomes permanently invisible to mark_lookup. That is the single most damaging hole, because the entire recall half of the system depends on tags the agent must remember to set by hand while focused on content.

Strategy

Move enforcement off advisory prose and onto hooks, which the harness executes deterministically. Pure composition of existing surface — no core changes (consistent with the core-vs-overlay layering rule: the server stays permissive; the agent-driven client gets the guardrail).

Severity is configurable but the check always runs — this is a strictness level, not an opt-in capability (no "validation off" deployment exists):

  • warn (default) — PostToolUse injects a system-reminder after a tagless publish, nudging a re-publish with tags. Non-blocking.
  • block — PreToolUse denies a tagless publish until tags are added.
  • ask — PreToolUse escalates to the user.

Hook mechanics (verified against code.claude.com/docs/en/hooks)

  • PreToolUse: hookSpecificOutput.permissionDecision in allow|deny|ask|defer + permissionDecisionReason. Carries block/ask.
  • PostToolUse: hookSpecificOutput.additionalContext injects a system-reminder the model sees next turn. Carries warn. (Plain stdout is debug-only, not model-visible.)
  • Stop: only top-level {"decision":"block","reason":...} surfaces anything (no confirmed additionalContext). So a Stop nudge necessarily blocks one extra turn. Input carries session_id, transcript_path, stop_hook_active; loop-guard on stop_hook_active plus a per-session sentinel.
  • Matchers are JS regex against the full mcp__server__tool name. The publish gate uses a broad mcp__.*__mark_publish matcher and scopes precisely in-script (local soul + registered knowledge systems only). Stop takes no matcher.
  • One dispatcher script handles both publish events; branches on hook_event_name, reads STRICTNESS. Zero runtime dependencies — payloads parsed by pure-awk character-level JSON scanners (no jq, no python; awk is already the engine behind json_escape). A naive grep would false-match the arbitrary body value, so the scanner tracks string boundaries + the object-key path and reads only the exact fields it needs. Fails open — any parse hiccup defers, never blocks.

Work items

1. Publish tag/importance gate — DONE 2026-06-01 (PR #171)

hooks/publish-gate.sh (one dispatcher on both PreToolUse + PostToolUse) + matcher wiring in plugin.json. Tagless publish (tool_input.metadata.tags absent/empty/whitespace, or a non-string tags value) or importance outside [0,1] → warn/block/ask per strictness. Strictness read by configured_strictness in lib.sh: DEMARKUS_MEMORY_STRICTNESS env override → ~/.demarkus/plugin-memory.strictness file → warn default (kept in its own file, orthogonal to plugin-memory.conf, so setup.sh rewrites never clobber it). Parser is the pure-awk publish_metadata_check in lib.sh. Pins bumped SERVER 0.17.13→0.17.14 (client/tools already latest). Plugin version 0.3.0→0.4.0.

CodeRabbit review surfaced four real correctness holes in the awk parser, all fixed + regression-tested:

  • escaped-whitespace bypass — {"tags":"\n"} was stored raw (backslash+n) and passed; added a pure-awk json_unescape (decodes \n \t \r \b \f \/ \" \\ and \uXXXX; whitespace codepoints → space, others → non-space marker) applied before trim/range checks.
  • non-string tags bypass — {"tags":false|null|0} and array/object values passed; recordValue now tracks whether the value came from a JSON string (tagsStr) and tags_ok requires it.
  • exponent importance1e-1 was wrongly rejected; regex now allows ([eE][+-]?[0-9]+)?.
  • escaped-whitespace + array/object test coverage added. Publish suite now 22 cases, green.

Follow-up noted: the plugin's shell tests (tests/*.sh) are not run by any CI workflow today — knowledge-join_test.sh is manual too. Wiring them into CI is a separate, small task (touches .github/workflows/, mind the YAML ${{ }}-in-comments rule).

2. Relocate decaying guidance to the moment of action — DONE 2026-06-01 (PR #171)

session-guidance.md tagging block tightened: the gate now carries the tagging rule at write time, so the startup prose states it tersely, points at the gate, and makes the enforced-vs-advisory split explicit. (Reconciled in item 3 to note journaling now has a soft session-end backstop; in item 6 to point routing at the canonical template.)

3. Session-end journal nudge (Stop hook, guarded) — DONE 2026-06-01 (PR #171)

hooks/session-journal-nudge.sh (Stop hook, no matcher). Nudges once, at session end, when the session changed files (Edit/Write/NotebookEdit tool_use in the transcript — a low-false-positive "real work" signal) but recorded nothing to any demarkus memory store (no mark_publish/mark_append via MCP attribution or tool name). Since a Stop hook can only surface a message by blocking, it emits {"decision":"block","reason":...} once with an explicit "if it's routine, just stop" out. Two loop guards: stop_hook_active==true (also tolerates a quoted "true") early-exits, and a per-session sentinel fires at most once per session. New parser stop_hook_fields in lib.sh (pure awk). Tests: tests/session-journal-nudge_test.sh, 11 cases.

Open question for review: the blocking Stop nudge forces one extra turn per qualifying session. Acceptable + bounded (once per session, easy out), but if it proves annoying, add a disable knob (e.g. DEMARKUS_MEMORY_JOURNAL_NUDGE=off / strictness file) — deferred until there's signal it's needed.

4. Recall-before-answering (deferred until 1-3 land)

Hardest, most false-positive-prone. Good capture is the precondition for good recall — fix tags first. Cheap version: UserPromptSubmit pattern-match ("did we / what did / last time / decide") → one-line "check the soul" nudge.

5. Catalog hygiene tooling (/soul-doctor)

Surfaces existing rot: orphan docs (no inbound links, not in any index), broken [[links]], duplicate content, stale index entries, ADR numbering collisions. Built on mark_backlinks / mark_graph. Tooling, not enforcement.

6. Align local per-project template to the proven demarkus-soul structure — DONE 2026-06-01 (PR #171)

Single source of truth: seed/project-template.md, seeded to the soul root as /project-template.md (via the generalized seed_doc in lib.sh, called from session-start.sh). It never clobbers, so a user-customized /project-template.md persists across sessions. The proven layout: maintained per-project index.md hub, architecture.md, patterns.md, guidelines.md, debugging.md, roadmap.md, debt.md, thoughts.md, adr/<NNNN>-*.md, plans/<name>.md, journal/<YYYY-MM-DD>.md. The four previously-drifting consumers now defer to it: seed/index.md (pointer), skills/soul-memory/SKILL.md (aligned routing + fetch-the-template instruction), context/session-guidance.md (aligned routes + pointer), and commands/soul-context.md (was pulling the now-removed plan/tasks.mdroadmap.md). New tests: tests/seed_doc_test.sh (5 cases). No plugin-version bump (one 0.4.0 release).

Note: this establishes the "template lives in the store" pattern that #7 reuses for the knowledge system (org template at mark://root/...).

7. Knowledge-system template + policy, anchored at the root hub — DONE (plugin side) 2026-06-01 (PR #171)

Every knowledge system gets a guaranteed hub world named root that tracks things globally. root discovery resolved as a fixed-name convention (Fritz's call); broker-advertised discovery deferred as a future enhancement.

Plugin side built + tested:

  • Gate extends to joined knowledge systems. The publish-gate matcher is broadened to mcp__.*__mark_publish, and publish_gate_scope (lib.sh) classifies each call: local (server contains demarkus-memory → global strictness), ks:<slug> (a registered knowledge system → per-slug strictness), or empty (a demarkus server the plugin doesn't manage → not gated). So the tag check now enforces on org writes too, but never on an unrelated personal soul.
  • Registry. /knowledge-join runs scripts/register-knowledge.sh <slug> after claude mcp add; register_knowledge_system / is_registered_knowledge_system (lib.sh) maintain ~/.demarkus/knowledge-systems (one slug per line, idempotent).
  • Per-slug strictness. configured_strictness [slug] precedence: env → plugin-memory.strictness.<slug> → global plugin-memory.strictness → warn. The agent mirrors the org's declared level here (courier).
  • Convention + reference docs. examples/knowledge-system/ (README + policy.md) documents the root well-known paths (mark://root/.well-known/demarkus/{policy,template}.md), the policy format (a strictness: line + required tags), and the agent-courier flow. commands/knowledge-join.md updated: register the slug, then fetch+follow the root template/policy and mirror strictness.
  • Tests: tests/knowledge-gate_test.sh, 8 cases (registry idempotency, local still gated, foreign/unregistered ignored, registered gated, per-slug strictness + isolation, tagged defers).

Deferred / out of plugin scope (org + deployment side): actually creating the root world and publishing policy.md/template.md on a live knowledge system (e.g. knowledge.demarkus.io); broker advertising the hub world in /.well-known metadata; de-registration on claude mcp remove (stale entry is harmless — never matched). The agent-read of root docs is advisory by necessity (only the agent can reach the broker; a hook cannot).

Sequencing

1 → 2 → 3 → 6 → 7 → 5 → 4, all stacked on one branch (PR #171) toward a single 0.4.0 release. Items 1 + 2 + 3 + 6 + 7 (plugin side) done. Remaining: 5 (/soul-doctor hygiene tooling), 4 (recall nudge). Org-side root setup tracked separately from the plugin PR.

Constraints honored

  • No core changes — pure plugin composition.
  • Zero runtime dependencies — pure awk/bash, no jq/python (Fritz 2026-06-01: do not add external deps that may not be installed).
  • One stacked branch, single 0.4.0 release — no plugin-version churn (Fritz 2026-06-01).
  • Version pins bumped on every plugin change (scripts/lib.sh).
  • No opt-in capability knob — strictness is a severity level on an always-on check.
  • Plans live on the soul, not in a vendor folder.
trail
  1. soul.demarkus.io v6