2026-06-23 — soul-doctor dangling-reference check

Drafted a new dangling & unlinked references deep-check into plugins/claude-code/commands/soul-doctor.md.

Why (the gotcha that motivated it)

Tracing how an agent reaches a referenced doc, hit a real failure: ADR 0006 in the demarkus-library project soul says "supersedes ADR 0005", but ADR 0005 isn't in the store — adr/ holds only 0006. Neither retrieval path reaches it:

Graph traversal can't — the "supersedes ADR 0005" reference is prose, not a [text](url) link. links.Extract (client/links/links.go) only walks *ast.Link nodes, so the ADR crawled with LinkCount: 0. Latent relationships written in prose / inline code are invisible to mark_graph / mark_backlinks.
Catalog lookup proves absence — mark_lookup query="*" scoped to the project returns the complete subtree census (6 docs, only ADR 0006). The catalog is authoritative for absence: * is a full census, so "no match" means "not here," not "didn't crawl it." That certainty is the catalog's real value, distinct from finding things.

The existing soul-doctor broken-links check missed it because a broken link requires a link; this reference was never an edge.

The check (design decisions)

Deep tier (per-doc fetch) — it's a body-text scan; core checks stay fetch-free. Inherits the cap/disclose discipline.
Zero extra calls — resolves each prose mention against the inventory (mark_list, existence) + edges (mark_graph, already-linked?) already gathered.
Two verdicts, different fixes: unlinked (target exists, no edge → convert prose to a link) vs dangling (resolves to nothing → restore or remove; confirm via mark_lookup since catalog is authoritative for absence).
Tight pattern set to bound false positives: ADR[ -]?#?\d{3,4} + bare/inline mark:// URLs not inside a link. Exclude self-references and code-fence mentions (same fence-skip firstH1/the catalog use). Left out *.go mentions (point at repo, not soul) and title fuzzy-matching (too noisy).
Distinct from broken-links: that follows an edge to a missing target; this finds references that were never edges.

Broader frame (not yet acted on)

Two improvement axes identified. (1) prose-reference gap → this doctor check + optional publish-time link suggest-mode. (2) semantic-recall gap (catalog is tags/title only) → a client/broker-side embedding index, not a SEARCH verb — ADR 0006 D7 closed full-text search by design ("never will be"). Respecting that, semantic recall belongs in the gateway where mark_index already reads bodies, mirroring the ephemeral/bucket-store pattern of the graph store.

Not yet done: dry-run the new check against the demarkus-library project to confirm it flags the 0005 dangling ref + 0006 unlinked refs.

Update — dry-ran the check against demarkus-library

Ran the new deep-check over all 6 docs. Confirmed the predicted findings: ADR 0005 dangling (cited from 5/6 docs), ADR 0003 dangling (from the ADR itself); architecture.md → ADR 0006 unlinked (cited ~4× in prose, no in-body link). index.md self-annotates 0005's absence ("no md file exists") → report as known, not actionable.

The dry-run exposed two bugs in the v1 draft (now fixed in soul-doctor.md):

already-linked? must parse the citing doc's OWN markdown links, not the crawl edge store. The crawl seeds from the hub, so orphan docs are never visited and their outbound links never enter the edge store. journal/2026-06-19.md does link [ADR 0006](…) but is an orphan → the edge-store rule would falsely flag it unlinked. Fix: the deep tier already holds every body; resolve from parsed body links.
bare/inline mark:// pattern too noisy → deferred to v2. False-positives on cross-world refs (can't existence-check in a single-world audit) and inline-code placeholders (`mark://host/...`). v1 is ADR-pattern only.

Lesson: a graph-derived "is it linked?" signal silently degrades for orphans (unreachable from the crawl seed) — same failure shape as the world-map orphan bug in this project's own 2026-06-20 journal ("not proven orphan ≠ linked"). Prefer the in-hand body over a derived graph signal when you already paid for the fetch.

Update — /knowledge-doctor drafted + dry-run against the live broker

Added plugins/claude-code-knowledge/commands/knowledge-doctor.md (the broker/multi-world counterpart to soul-doctor). Bumped demarkus-knowledge 0.2.0 → 0.3.0. The substrate needed zero server work — the broker already exposes the full mark_* surface incl. mark_worlds; a doctor is purely a command.

Dry-ran it against the joined knowledge system (worlds root + world-a). Real findings: ~28 broken nib/plans/*.md links (dir never published), entire junto/* missing, test-cruft orphans, nib/index.md (the hub) untagged → violates require_tags: category AND has an in-body --- frontmatter block (stored v1 but body says version:47 — exported doc pasted back).

Three broker-specific bugs the dry-run exposed in the v1 draft (all fixed):

A [not-found] crawl node is already-confirmed broken — the crawler fetched it. Re-confirming via mark_fetch (as the draft said) = N wasted calls. Only targets absent from Nodes (possible depth artifact) need inventory confirmation.
[error] ≠ [not-found]; mark_worlds membership disambiguates. world-a → mark://soul.demarkus.io/index.md crawled [error] because the host isn't a member world = external/cross-system pointer, not a broken doc. A member-world [error] = transient dispatch flake (retry once).
Untitled check must scope to [ok] nodes — (no title) on the 28 not-found nodes is the broken-link finding, not untitled; unqualified it double-reports. (Same latent bug in soul-doctor — fixed there too.)

Bonus check both doctors gained: in-body frontmatter block (body opens with --- carrying version/previous-hash/meta.* = exported-doc-pasted-back; renders garbled, metadata stale). Found live on nib/index.md.

Meta-lesson: dry-running a check against real data is worth more than reasoning about it — both the soul-doctor and knowledge-doctor dry-runs found design bugs the prose review missed. Crawl-node status (ok/not-found/error) carries information the v1 drafts treated as uniform.