# Knowledge Ingestion Pipeline

promoted: mark://world-a/test/knowledge-ingestion-pipeline.md@v1

> Promoted to the demarkus knowledge system (`world-a`) on 2026-06-18 — the dogfood run, the pipeline's first real exercise of itself. This soul copy is the living draft; the published copy above is authoritative. Reconciliation is directional: local edits re-enter upward through `/promote`'s gate, and `/soul-refresh` pulls knowledge changes back down. Marker-only back-stamp (this is a living plan, not a stub).

A narrative and design for how organizational knowledge flows into a demarkus knowledge destination, and how the two Claude Code plugins (demarkus-memory / the soul, and demarkus-knowledge / the shared catalog) compose into a single curation pipeline. Status: **draft, unstarted.** All design forks are resolved (see Decisions); what remains is build prerequisites — the promote primitive, access discovery, and the coherence edge — not open questions.

## The thesis: soul = write-ahead log, knowledge destination = published read-model

The soul and the knowledge destination are not two stores that happen to coexist. They are two stages of one refinement pipeline.

- **Soul** is the capture tier: cheap, continuous, private, high-volume, low bar to write. Agents already record here as they work (journals, debugging lessons, ADRs, patterns). Noisy but rich. It is meant to be a firehose, and that is correct.
- **Knowledge destination** is the curated index: high bar, shared, authoritative, deduped, tagged to the org taxonomy, low-volume relative to the soul.
- **The promotion from soul to the knowledge destination IS the curation pipeline.** The model-routing and gate decisions below apply to this internal flow exactly as they apply to external sources.

This is CQRS / write-ahead-log thinking applied to knowledge: append-heavy cheap capture, periodic compaction into a queryable authoritative read-model. We already built the front of the pipeline. It is the soul. What is missing is the promotion lane on top of it.

## The unifying move: the soul is the universal staging tier

External sources (Confluence, Slack, Jira, meeting notes) do not land in the knowledge destination directly. They land in a soul (an ingestion soul) first. Then the same promotion gate that curates agent-authored soul content also curates externally-ingested content. One gate, many inflows.

```text
INFLOWS                          STAGING (soul tier)        GATE                       CATALOG (knowledge)
agents recording        ───────► personal souls ──┐
Confluence ETL          ───────► ingestion soul ───┼─► curation gate ───────────────► knowledge endpoint
Slack/Jira/mtg distill  ───────► ingestion soul ───┘   triage→distill→dedup→tag→        authoritative,
                                                       human approve                     deduped, provenance
```

Soul is the universal write-buffer. The knowledge destination is the published read-model. The gate is built once and reused for every inflow. The reverse edge — keeping the soul fresh as the published copy evolves — is the Coherence section.

## The bridge: detection-gated capability tiers

Promote is not owned by either plugin. It is the bridge that mutual detection creates between two roles, and it activates only when both roles are present.

**Two roles the bridge connects.**

1. **Memory** (demarkus-memory) — the personal knowledge base. Its own backing store is transport-flexible: local or a remote demarkus server. This is the baseline and is fully usable on its own.
2. **Knowledge destination** (demarkus-knowledge) — the shared/team target. This is not only the broker. It is an abstraction over two transports, and promote must work against either:
   - a **broker-fronted knowledge system** — multi-world, OIDC/OAuth, the org-wide hub; and
   - a **plain remote demarkus server** — a single remote world, direct QUIC, no broker.

So "knowledge system / base" means any demarkus write target that is not your personal soul. The promote primitive targets an **endpoint**, not specifically the broker. That generalization is load-bearing: everything below is written against endpoints, not against the broker.

**The unifying abstraction: a demarkus endpoint.** An endpoint is "a demarkus server you can read or write," reached either directly (QUIC, single world) or through a broker (HTTPS to QUIC, multi-world system). What varies across endpoints is transport, auth, and single- versus multi-world. Memory, Knowledge, and the Library (the demarkus-library web reading room, which must read across all of them) all speak to endpoints. Promote simply targets one that is not your soul.

**Capability tiers, gated on mutual detection.** Detection is mutual and read-only: memory recognizes a knowledge plugin, and the knowledge plugin recognizes memory (a conf-peek in both directions, mirroring the one-way peek that exists today). Each adapts to the other's presence.

| Installed | Pattern | Behavior |
|---|---|---|
| Memory only | Personal knowledge base | Memory, local or remote. Promote is dormant — there is no destination. |
| Knowledge only | Direct publisher / consumer | Read (lookup/fetch) and deliberate high-bar authoring straight into the catalog, subject to the policy gate. No staging tier, so no promotion pipeline and no triggers — you author at the high bar on purpose, you do not lift low-bar drafts. |
| Memory + Knowledge | Promotion-enabled | Full bridge: staged low-bar capture, triggers, and the promote cascade that lifts soul content up to the catalog's high bar. |

**Triggers are inherently memory-side.** The "this seems important — promote it?" prompts (manual, signal-based on recall-count or doc type, and the batch sweep) all observe the staging tier. With no soul there is nothing to observe, so the knowledge-only tier has no triggers by construction: it is deliberate manual authoring, not staged promotion. The cascade exists to lift low-bar soul content up to the high bar; when you author directly into the catalog you are already at the high bar and skip it.

**Where the code lives follows the bridge, not one plugin.** The operation splits along its natural seam.

- **Memory side (intent / source):** detect a knowledge endpoint, mark a candidate, read the source doc, apply the back-stamp. Memory stays broker-unaware throughout — it only detects that an endpoint exists and hands off; it never talks to the broker itself.
- **Knowledge side (execution):** triage, distill, dedup against the catalog, taxonomy translation, the policy gate, the endpoint write, and destination selection (which world, filtered to the caller's writer allow-set — readable is not writable).
- **Back-stamp stays one-directional:** the knowledge side reports the resulting `mark://` back, and memory applies its own `promoted` stamp and refresh. The knowledge plugin never writes the soul.

The batch sweep (below) is orthogonal: it automates the memory-side "mark a candidate" step at volume, and remains a separate agent/daemon rather than plugin code.

## The narrative: life of one fact

1. **Capture.** An agent fixes a gnarly bug and records the root cause to its local soul `debugging.md`. Immediate, private, cheap. This already works.
2. **Accumulate.** Days pass; the soul gathers journals, patterns, ADRs. Some trivia, some durable and broadly useful. Uncurated.
3. **Trigger.** The fact gets recalled repeatedly, or it is a high-signal type (an ADR), or a periodic sweep flags it. It becomes a promotion candidate.
4. **Triage** (cheap model, firehose volume): durable? broadly useful? not already in the catalog? Most soul content fails here and correctly stays personal.
5. **Distill and dedup** (capable model, low volume): rewrite for a shared audience, strip personal and local framing, check overlap and conflict against existing catalog docs.
6. **Tag** (cheap model): assign `category:` plus org tags from the closed taxonomy.
7. **Gate** (human): approve. The authority check.
8. **Publish** to the chosen knowledge endpoint, with provenance linking back to the soul origin (and the external source if any). The soul doc is stamped *promoted* and refreshed to the settled version.
9. **Consume.** Any agent now finds it via `mark_lookup`, authoritative and shared. The originating soul links to it rather than keeping a stale fork; the Coherence edge keeps that link live as the published copy evolves.

External sources enter the same lifecycle at step 1, with an ETL/distill in place of an agent.

## Model routing: cascade, do not pick one model

Curation is not one task. Decompose by difficulty and volume, then route each sub-task to the cheapest model that clears the bar. Cost tracks volume, and volume is concentrated at the cheap stages.

| Sub-task | Volume | Difficulty | Model |
|---|---|---|---|
| Route: doc- vs conversation-shaped | High | Trivial | Rules or Haiku |
| Triage: does this contain a durable decision? | Very high (firehose) | Low | Haiku |
| Extract structured fields | High | Low-med | Haiku |
| Tag to policy (closed taxonomy) | Medium | Low | Haiku |
| Distill conversation to authoritative paragraph | Low (survivors) | High (faithfulness) | Sonnet/Opus |
| Dedup / conflict-resolve vs catalog | Low | High (judgment) | Sonnet/Opus |
| Authority gate | Lowest | Highest | Human |

Run the cheap model on the firehose as the triage gate (discards ~95% of Slack/Jira noise cheaply). Run the capable model only on the survivors. Never run the expensive model over every message.

The lever that lets Haiku go further is gate strength. A strong human gate makes Haiku-drafted distillations safe (a human polishes an 80%-good draft). An optional Haiku refute-check (a second cheap call that tests the distillation against its source) pushes strong-model usage lower still. Auto-publish with no gate is the only mode that forces the strong model, and it should not be used into an authoritative store.

## The connective tissue the two plugins are missing

Today the plugins compose only by convention ("draft in soul, promote when ready"). There is no mechanism. The build list is this glue.

1. **A promote primitive.** Mark a soul doc as a candidate; a `promote` operation re-tags to the shared taxonomy, dedups against the catalog, and publishes with provenance to a chosen endpoint. It is a detection-gated bridge, not owned by either plugin (see "The bridge: detection-gated capability tiers"): memory contributes detect, candidate-mark, source-read, and back-stamp; knowledge contributes the gate, dedup, endpoint write, and destination selection.
2. **Two-way provenance.** The knowledge doc links to its soul/external origin. The soul doc is stamped `promoted → mark://world/path` so it is not re-promoted and is not duplicated. This enforces the existing "do not copy authoritative knowledge into the soul" rule mechanically.
3. **Trigger policy.** Memory-side only (there is no staging tier to watch without a soul). Manual ("ready for others") plus signal-based (recall-count, doc type) plus a batch sweep (a cron agent scans souls, runs the cascade, queues candidates for approval).
4. **Dedup, staged.** First dedup against the catalog (cheap, do now). Defer dedup across souls (entity resolution, hard) until the catalog flow is proven.
5. **Taxonomy translation.** Soul tags are loose and personal; the knowledge destination requires a `category:` axis and policy compliance. Promotion re-tags. Haiku does this (closed-set classification is its sweet spot).
6. **Access discovery (read and write).** Destination-selection needs the caller's `{readable, writable}` sets, but `mark_worlds` lists *readable* worlds only (the #189 distinction) and says nothing about write. Discovery is topology-dependent: against a **brokered system** the broker knows your identity and can enumerate both sets (extend `mark_worlds` with a writable surface); against a **plain remote server** there is no directory — the token you send IS your access, so discovery is introspecting that token's operation/path grants. Both resolve to the same `{readable, writable}` the router needs. Prerequisite for routing.
7. **The coherence edge.** The downward counterpart to promote — keeping the soul fresh as the published copy evolves. See "Coherence".

## Autonomy: human-in-the-loop by default, capped by the destination

Promotion autonomy is a graded knob, not a local boolean. Authority over a shared authoritative store belongs to its owner, so auto-publish cannot be a purely client-side toggle — otherwise any client flips to auto and floods the catalog.

Model it as two levels:

- **Destination ceiling**, set by the world owner in the world's well-known descriptor (see "Partitioning"): `human-only` / `verify-then-auto` / `auto`. The org root caps at `human-only`; a personal remote endpoint or a low-stakes scratch world may allow `auto`.
- **Local preference**, applied as `min(local preference, destination ceiling)`. Default is human-in-the-loop.

Autonomy is coupled to model tier (see Model routing): `auto` requires the strong model plus the refute-check; human-in-the-loop is exactly what makes a cheaper Haiku draft safe. Start everything at `human-only`; relax per-destination only once the gate and verification are proven.

## Partitioning: discovered, not designed

Where knowledge lands is not a partition scheme the pipeline invents. It is already determined by the access topology and by each world's self-description. The pipeline conforms to it; it does not define it. Two layers decide the destination.

1. **Access (hard constraint).** Access is read *and* write, both token-encoded. You can typically read broadly across a knowledge system but write only to your writer allow-set (readable is not writable — the #189 distinction). Write access filters the candidate worlds for any promotion; read access governs what the Library and lookups can see. How you *discover* your access is topology-dependent — ask the broker (identity-based) or introspect your token's grants (capability-based, on a plain remote server); see access discovery in the connective tissue.
2. **Well-known hints (soft routing).** Each world self-describes via a per-world well-known descriptor, alongside the existing `policy.md` and `template.md` — e.g. `mark://<world>/.well-known/demarkus/world.md`: owning team, subject domain, partition role, and its autonomy ceiling (the same ceiling the Autonomy section needs). Destination-selection reads these to route a candidate to the right world among the ones you can write.

Source of truth is the world owner — each world declares its own scope, self-sovereign. The root hub may aggregate descriptors into a discovery index, but it must be **derived** (from the per-world descriptors or the federation crawler), never hand-maintained, or the central map drifts from what worlds actually hold.

**Destination-selection algorithm.** Enumerate writable worlds (needs access discovery above) → read each one's descriptor → either auto-route (match the candidate's subject/team to a world's declared domain) or present a labeled, write-filtered list for the human to pick.

## Coherence: keeping the soul fresh after promotion

Promotion is an upward edge (soul → knowledge). It needs a downward counterpart, or the soul's promoted docs rot while the authoritative copy moves on. The rule that makes this safe: **reconciliation is directional, never a symmetric merge.** Knowledge is the base of truth; the soul refreshes *from* it; soul changes re-enter *as gated updates*. A naive two-way merge that let local soul edits silently land in the authoritative doc would corrupt the authority model.

Three mechanisms, each leaning on primitives demarkus already has.

1. **Default: link-not-copy, so there is nothing to merge.** Once promoted, the soul doc becomes a stub — title plus `promoted → mark://world/path` — and the body of truth lives in knowledge. Recall resolves the link and fetches fresh. The identity-case staleness class disappears because the soul defers instead of holding a rival copy.
2. **If the soul keeps a cached snapshot (offline, speed): invalidate by version, do not merge.** Stamp the cache with the promoted version and etag/content-hash (`promoted → mark://world/path@v3`). On pull or recall, compare the stamped version to the live one; if they differ the cache is stale, so refresh the body down — knowledge wins. Deterministic, no model; demarkus versions, etags, and content-hashes already exist for exactly this.
3. **Local edits after promotion re-promote as a gated update, not a silent overwrite.** If the soul refines its copy, that edit becomes a new promotion candidate — an *update* to the existing knowledge doc — back through the gate. The actual content reconciliation uses `mark_publish`'s existing conflict-aware merge (`expected_version` plus diff3 with conflict markers). The soul proposes; the gate disposes.

The back-stamp does double duty here: it marks promoted (anti-re-promote) **and** performs the first downward refresh, syncing the soul stub to the *settled* post-gate version rather than the pre-promotion draft.

**Related-but-distinct staleness is the hard case, deferred.** A separate soul note that merely overlaps a refined knowledge doc (not the same doc) needs semantic relatedness to catch. Defer it; when added, lean on the link graph first (`mark_backlinks` / `mark_graph` — if the soul note links the knowledge doc, a version bump propagates a "your linked source moved" notice) before reaching for embeddings or an LLM similarity pass. This is the dedup problem in reverse and should not block phase 0.

## Source taxonomy (external inflows)

The external sources are not the same shape, and they need different handling before they reach the gate.

| Source | Shape | Signal density | Freshness | Strategy |
|---|---|---|---|---|
| Confluence | Document | Medium-high | Slow | Near-direct ETL, deterministic, no LLM |
| Jira | Issue + workflow | Low (process noise) | Live | Extract outcomes only (decisions, resolved-bug root cause) |
| Slack | Conversation | Very low | Append-only | Distill resolved threads only; most never crosses |
| Meeting notes | Semi-structured transcript | Medium | One-shot | Distill to decisions, action items, context |

Confluence is doc-shaped and converts deterministically (own sub-plan; a `tools/demarkus-import` Go binary using `client/fetch.Client.Publish`, pure-Go or pandoc converter, two-pass link rewrite, idempotency manifest). Slack/Jira/meetings are conversation-shaped: worthless raw, value locked in threads that distill to a paragraph. They are the reason the distillation stage exists.

## Phasing

Sequence by fit, not by which API is easiest.

1. **Confluence first.** Doc-shaped, deterministic, highest signal. Proves the publish path, tagging, and provenance at lowest risk.
2. **Meeting notes.** Bounded volume, decision-dense. First real test of the distillation plus human-gate loop.
3. **Jira.** Outcomes-only extractor. Teaches entity-resolution against the existing Confluence docs.
4. **Slack last.** Lowest fit, highest volume, hardest privacy story. By now the gate is mature enough to be ruthless.

Underlying all of this: the soul-to-knowledge promotion lane is phase 0, because it is the gate every external source reuses. Build the gate against soul content first (it already exists and is safe to experiment on), then point the external ETLs at it.

## Risks and things to watch

- **Most soul content must never promote.** If more than ~10–20% crosses the gate, the bar is too low and the catalog becomes a soul mirror, destroying the curation value. Ruthlessness is the feature.
- **Fork staleness.** Promotion forks knowledge. Solve with link-not-copy, a `promoted` marker, and version-stamped cache invalidation; the downward refresh edge is specced in "Coherence". Without it the soul accumulates stale copies of authoritative docs.
- **Merge directionality.** The coherence edge must be directional — knowledge authoritative, soul refreshes from it, soul edits re-enter as gated updates. A symmetric merge that silently lands local edits in the authoritative doc corrupts authority. Guard this explicitly in the implementation.
- **Authority needs an owner.** The human gate per world maps to `policy.md`'s `owner:`. Decide who approves before wiring triggers.
- **Signal-based triggers need telemetry that may not exist yet** (recall counts). Start with manual plus doc-type triggers; add signal-based once recall is instrumented.
- **ACL to world boundary.** Source resources have access controls; worlds have read/write grants. A private channel must not flow into a world everyone reads. This is governance, not code. The per-world descriptor plus writer allow-set is the enforcement surface.
- **Privacy/PII.** Never publish PII or secrets to a shared world (policy already forbids it). Distillation must strip, not merely summarize.

## Decisions (all resolved)

1. **Is the knowledge destination the curated tier only, or also the raw retrieval tier?** **Curated-only**, to keep it useful. The soul (and external ingestion souls) hold the raw/retrieval tier.
2. **Human-in-the-loop or auto-publish for distillations?** **Human-in-the-loop by default; autonomy is a graded knob capped by the destination, not chosen freely by the client** (see "Autonomy"). Destination owner sets a ceiling (`human-only` / `verify-then-auto` / `auto`) in the world's well-known descriptor; local config picks at or below it; default `human-only`. Coupled to model tier.
3. **How are worlds partitioned for ingestion?** **Discovered, not designed** (see "Partitioning"). It falls out of the access topology (read and write, token-encoded; broad read, narrow write) plus per-world well-known descriptors declaring team, domain, partition role, and autonomy ceiling. The pipeline enumerates writable worlds and routes by their declared domain.
4. **Where does the promote primitive live?** **A detection-gated bridge capability, owned by neither plugin unilaterally** (see "The bridge"). Memory contributes the intent/source half and stays broker-unaware; the knowledge plugin contributes the execution half (gate, dedup, endpoint write, destination selection). Mutual read-only detection lights it up. The destination is a parameter — a team world, the org system root, or a plain remote demarkus server — filtered to the caller's writer allow-set. The batch sweep remains a separate agent. **Back-stamp: one-directional** — the knowledge side reports the published `mark://`, and memory writes the `promoted` stamp and the first downward refresh onto its own soul doc (syncing it to the settled post-gate version, see "Coherence"); the knowledge plugin never writes the soul, preserving the v0.5.0 partition.

## Build prerequisites (surfaced by the decisions)

Not design forks — concrete things that must exist before phase 0.

- **The promote primitive** (the bridge): candidate-mark, source-read, hand-off, execution cascade, one-directional back-stamp.
- **Access discovery** (read and write): a writable-worlds surface on the broker, plus token-grant introspection for plain remote servers (connective tissue #6).
- **The per-world well-known descriptor** (`world.md`): team, domain, partition role, autonomy ceiling — the routing and autonomy substrate.
- **The coherence edge** (knowledge → soul): link-not-copy stubs, version-stamped cache invalidation by etag/content-hash, and re-promotion of local edits as gated updates via `mark_publish` conflict-merge (see "Coherence").

## Dogfood

With the forks resolved, promote this very plan from the soul to the knowledge destination as the first exercise of the pipeline it describes. **Done 2026-06-18** — published to `mark://world-a/test/knowledge-ingestion-pipeline.md` (v1); this soul doc carries the marker-only back-stamp above.

## Build progress

### 2026-06-17 — prerequisites, brokered half (PRs pending)

Started phase 0 with the **prerequisites-first** path (Fritz's call over a minimal cascade slice). Two of the four build prerequisites landed against real code in the monorepo:

- **Access discovery — brokered half: DONE.** `mark_worlds` now emits a `writable` column. The broker already computed the writer set (`authorizedWorlds` / `worldAllows(&w.Allow, claims)`) for `/auth/callback` and `/me/install`; it just was not exposed through the read-discovery tool (#189 deliberately listed *readable* worlds only). The handler now captures the caller's claims and renders `writable: yes/no` per row via `worldAllows`, so a caller learns its `{readable, writable}` sets in one call — the seam destination-selection routes promotions on. Append-only column, so the library's `mark_worlds` table parser is unaffected. Files: `tools/demarkus-broker/internal/broker/mcp_tools_worlds.go` (+`yesNo` helper), `mcp_tools_list.go` (description), `mcp_tools_worlds_test.go` (4-column asserts + writable distinction: team-a writable, secret-b readable-not-writable for alice). Full broker suite + pre-commit green. **Merged #191.**
- **Per-world `world.md` descriptor: DONE (spec + seed).** Convention defined and an example shipped at `plugins/claude-code-knowledge/examples/knowledge-system/world.md`, wired into that dir's README. Structured core (`descriptor_version`, `team`, `domain`, `partition_role` ∈ hub|team|project|scratch, `autonomy_ceiling` ∈ human-only|verify-then-auto|auto) + convention prose. Per-world (published to `mark://<world>/.well-known/demarkus/world.md`), unlike root-global `policy.md`/`template.md`. A world with no descriptor still works: writable-but-unlabeled, manual pick-list only, defaults `human-only`. Reader is plain `mark_fetch` — no code. Derived root discovery index deferred.

**Deferred within access discovery — A2, plain-remote token-grant introspection.** The live knowledge target is brokered, so the `mark_worlds` writable column unblocks routing today. A2 (a demarkus-server surface to introspect a token's `{operations, path-globs}` so a plain remote endpoint resolves the same `{readable, writable}`) is scoped after the brokered path is proven.

**Remaining before/with phase 0:** the promote primitive (the cascade) and the coherence edge. A1+B are the routing/autonomy substrate they sit on.


### 2026-06-17 (cont.) — the promote primitive

Built the promote primitive as the detection-gated bridge of Decision 4 — split across both plugins, model-driven (slash command + skill), bash only for detection + back-stamp glue. Sits on the A1+B substrate from earlier today.

- **Mutual detection (memory side).** Memory now peeks the demarkus-knowledge registry (`~/.demarkus/knowledge-systems`) read-only — the reverse of knowledge's existing peek at `plugin-memory.conf`. New `knowledge_endpoints()` / `knowledge_present()` in the memory `lib.sh` + a `scripts/detect-knowledge.sh` (mirrors `detect-soul.sh`): `NO_KNOWLEDGE` → promote dormant; `KNOWLEDGE` + slugs otherwise. Memory stays broker-unaware (file-only). 5 new shell tests, all plugin suites green.
- **`/promote` command (memory side, entry + back-stamp).** `plugins/claude-code/commands/promote.md`. Flow: detection gate → read source soul doc → already-promoted check (anti-re-promote) → hand to the knowledge cascade → capture published `mark://` → **back-stamp the soul doc** (one-directional; knowledge never writes the soul). Back-stamp offers two modes: *stub* (link-not-copy, default for reference docs) and *marker-only* (keep body + `promoted → mark://…@vN`, for living docs like an active plan). Directional reconciliation enforced in the instructions.
- **`knowledge-promote` cascade skill (knowledge side, execution).** `plugins/claude-code-knowledge/skills/knowledge-promote/SKILL.md` (first skill in that plugin; auto-discovered). The full cascade: triage (ruthless — most content stays personal) → distill (strip personal framing + secrets/PII) → dedup/conflict vs catalog (`mark_lookup`) → tag to taxonomy (policy `require_tags`/`category:`) → destination-select (**`mark_worlds` writable column** + per-world **`world.md` `domain`/`autonomy_ceiling`** — the A1+B payoff) → human gate capped by `min(local, ceiling)` → publish with provenance → return `mark://`. Model-routing note included for the future batch sweep.

**Capability tiers honored:** memory-only → `/promote` dormant (no endpoint); knowledge-only → no `/promote` (it's a memory command), matching "no staging tier, no promotion"; both → full bridge.

**Not yet built / open:** the **coherence edge's** active half (version-stamped cache invalidation + re-promotion-as-gated-update is described in the command but not mechanized beyond the `@vN` stamp); signal-based + batch-sweep triggers (manual `/promote` is the only trigger so far); A2 (plain-remote token introspection). Plugin version bumps (memory feature, knowledge feature) left for Fritz's commit. **Dogfood now executable: `/promote /plans/knowledge-ingestion.md`** once a destination world exists.


### 2026-06-17 (cont.) — the coherence edge

Built the active downward half of the coherence edge (knowledge → soul), folded onto the promote-primitive branch. All plugin work; the broker-reaching steps are agent-driven commands (hooks can't reach the broker), per the plan.

- **Standardized promoted marker (mechanism 1+2 substrate).** `/promote` now emits a single greppable marker line `promoted: mark://<world>/<path>@v<N>` in both back-stamp modes (stub + marker-only), and adds a `promoted` metadata tag so promoted docs are discoverable via `mark_lookup filter=tag=promoted`. The `@v<N>` stamp is the version-invalidation key.
- **`/soul-refresh` command (mechanism 2+3, downward sync).** `plugins/claude-code/commands/soul-refresh.md`. Detection-gated (reuses detect-knowledge.sh). Finds promoted docs by tag → parses `@v<N>` → reads live knowledge version `<M>` → in-sync (skip) / dangling (surface, don't delete) / moved (`M>N`). Refresh is **directional by mode**: a *stub* refreshes down automatically (no rival body to lose); a *marker-only* doc surfaces for a human choice — pull-down (knowledge wins) or re-promote local edits **upward** through `/promote`'s gate (mechanism 3). Never a silent two-way merge. Related-but-distinct staleness explicitly deferred per plan.
- Discoverability: session-guidance now mentions `/promote` + `/soul-refresh` activate only when a knowledge system is joined.

**Coherence edge status:** mechanism 1 (link-not-copy stub) + mechanism 2 (version-stamped invalidation/refresh) + mechanism 3 (re-promotion as gated update) are all mechanized. The `mark_publish` diff3 conflict-merge the plan cites for content reconciliation is reached via `/promote`'s update path, not re-implemented.

**Phase-0 build prerequisites — now all but A2 done:** promote primitive ✓, access discovery brokered half ✓ (A2 plain-remote deferred), world.md descriptor ✓, coherence edge ✓. Remaining phase-0 surface: signal/batch triggers (manual `/promote` + `/soul-refresh` are the only triggers), then the dogfood `/promote /plans/knowledge-ingestion.md`. Plugin version bumps left for Fritz. **Merged #192 (memory v0.6.0 / knowledge v0.2.0).**

### 2026-06-18 — dogfood run

Ran the pipeline on itself. `/promote /plans/knowledge-ingestion.md` end-to-end against the live brokered knowledge system (`knowledge.demarkus.io`): triage passed (durable, broadly useful, no catalog dup via `mark_lookup`); distilled to a shared-audience copy (the three build-progress session logs collapsed to a single "Implementation status" section; "draft" status line dropped; provenance added); policy `block`/`require_tags: category` satisfied with `category:reference`; destination chosen at the human gate. The live broker predates #191 so `mark_worlds` had no writable column — destination resolved by the gate instead. The system has no demarkus-project world (only `root` hub + `world-a` = nib + e2e test docs), so it landed as a test/reference doc: `mark://world-a/test/knowledge-ingestion-pipeline.md@v1`. This soul doc carries the marker-only back-stamp (top). First real exercise of the pipeline; the mechanism works end-to-end.


### 2026-06-18 — promotion triggers (doc-type + on-demand sweep)

Started moving past manual-only promotion. Per the plan's trigger policy (memory-side; manual + doc-type now, signal-based deferred until recall telemetry exists, batch sweep a separate agent), built the two buildable pieces:

- **Doc-type trigger — `promote-nudge.sh` hook (memory).** PostToolUse on `mark_publish`. Fires a discreet "/promote this?" nudge only when all hold: publish targets this plugin's soul (`publish_gate_scope == local`), a knowledge endpoint is joined (`knowledge_present`), the path is high-signal (an ADR — `*/adr/*.md`), and the body carries no `promoted: mark://` marker (so a back-stamp re-publish never re-nudges). Tight like recall-nudge; fails open; additionalContext, never block. Registered as a second PostToolUse hook alongside publish-gate. 7 shell tests, green.
- **On-demand batch sweep — `/promote-scan` command (memory).** Detection-gated. Excludes the already-promoted set (`mark_lookup filter=tag=promoted`), surfaces high-signal candidates (ADRs, architecture, high-importance/shared-tagged docs; skips journals/thoughts/debt), lightly triages, and presents a ranked shortlist for the user to `/promote`. Never promotes itself (each goes through `/promote`'s gate). The autonomous scheduled batch agent stays out of plugin code, per the plan.
- Plumbing: memory `0.6.0 → 0.7.0` (plugin.json + marketplace.json + description), session-guidance updated to name `/promote-scan` + the ADR nudge.

**Signal-based triggers (recall-count) remain deferred** — they need recall telemetry that doesn't exist yet. Doc-type is the safe start the plan prescribed. Branch: `feat/promotion-triggers` (uncommitted, Fritz commits).


### 2026-06-18 — Confluence (phase 1) deferred

Scoped Confluence ingestion against the codebase, then deferred to a later phase (Fritz's call). Captured so it isn't re-derived when resumed:

- **Pipeline shape:** `tools/demarkus-import` (new tools/ Go binary) — fetch Confluence pages (REST, storage-format XHTML) → convert XHTML→markdown → two-pass link rewrite (page-ID → mark-path map, then rewrite) → idempotency manifest (page-ID → {path, content-hash}, republish only changed via `expected_version`) → publish.
- **Publish path:** `client/fetch.Client.Publish(host, path, body, token, …)` is QUIC-to-a-demarkus-server, NOT the broker HTTPS gateway. Cluster worlds aren't externally reachable over QUIC, so the natural target is an **ingestion soul** (the plan's staging tier), then the existing `/promote` gate lifts curated pages to knowledge worlds. Reuses the gate; matches the universal-staging-tier move.
- **Note:** `goldmark` (in the tree) is md→AST, not HTML→md — the converter needs a real HTML→markdown path (pure-Go x/net/html walker / small dep, vs pandoc shell-out).
- **Decision made:** when resumed, build **fixtures-first** — the deterministic core against checked-in Confluence storage-format XHTML fixtures, with live REST fetch behind an interface. Testable without external creds; no fabricated source. (Converter choice + destination — ingestion-soul vs near-direct — left open; revisit with the real demo universe + whether a staging server exists.)


### 2026-06-18 — A2: plain-remote access discovery (last prerequisite)

Closed the final phase-0 prerequisite — `{readable, writable}` discovery against a **plain** remote demarkus server (no broker directory; the token IS the access). The brokered half (A1, `mark_worlds` writable) shipped earlier; this is its plain-server analogue.

**Constraint honored:** the 7-verb set is finalized, so NO new verb. Instead a **synthesized well-known doc served via FETCH**: `/.well-known/demarkus/access.md`, computed per-request from the presented token, dispatched ahead of read-auth and never gated (it reflects only the caller's own grants → no leak of other tokens or structure; respects the not-found-not-forbidden principle). The client reads it via plain `mark_fetch` — no new MCP tool either. It lives under the same `.well-known/demarkus/` prefix as policy/template/world.

**Server (Go):**
- `auth.GrantsFor(token) → Grants{Label, Recognized, Operations, Paths}` — resolves a presented token to its grants; empty/unknown/expired all collapse to anonymous (never reveals which), returns copies. (`server/internal/auth/auth.go`)
- `server/internal/handler/access.go` — `AccessManifestPath` const, `handleAccessManifest` (synthesizes, honors if-none-match, emits etag+content-hash), pure `buildAccessManifest` rendering the grants table + writable/readable summary. Three cases: no-tokens-file (public read, writes denied), anonymous (public baseline), recognized (its own grants).
- Dispatched at the top of `handleFetch` so a stored file can't shadow it.
- Tests: `TestGrantsFor` (auth), `TestBuildAccessManifest` + `TestHandleAccessManifest` (handler — incl. "stored file does not shadow", "anonymous leaks no paths", "served with no tokens file"). Full server suite + pre-commit green.

**Client (skill):** `knowledge-promote` destination-select now branches by topology — brokered → `mark_worlds` writable + `world.md`; plain remote → `mark_fetch /.well-known/demarkus/access.md`, choose a path under a writable pattern, default ceiling `human-only`. Same `{writable}` either way.

**Docs:** `docs/SPEC.md` §11.8 documents the access-discovery well-known doc.

**Follow-up (not blocking):** the server feature releases as a new `server/vX` (CI auto-bumps from the `feat(server)` commit); the demarkus-memory plugin's `SERVER_VERSION` pin in `lib.sh` should bump to that once released, so a plain-remote soul/endpoint gets the access-manifest-capable server.

**Phase-0 prerequisites: ALL DONE** — promote primitive ✓, access discovery (brokered ✓ + plain-remote ✓), world.md ✓, coherence edge ✓. Plus triggers (doc-type + scan) ✓ and the dogfood ✓. Remaining epic work is later-phase: signal-based triggers (need telemetry) and external inflows (Confluence first, deferred).


### 2026-06-18 — A2 reverted, deferred again (correction)

The previous entry's "A2 DONE / ALL prerequisites done" is **superseded**: A2 was built and reviewed, then **reverted on Fritz's call** before any commit. Reasoning (the conflict that tipped it to defer):

- The live system **and the upcoming demo are brokered** → A1 (`mark_worlds` writable) already covers destination discovery there.
- The plugin's **own managed soul already knows its grants** (it mints the token), so it needs no introspection.
- A2's server-side access manifest therefore served only a *third-party plain-remote promote destination* — a case with **no current instance**, and one the plan had originally parked as deferred.
- Decisive cost: shipping it would bake a **permanent public contract** (a well-known path + SPEC §11.8 clause) to honor and version forever, for an unused case. Carrying that against the dumb-bookshelf principle wasn't worth it yet.

The design is preserved in the prior v13 entry (synthesized `/.well-known/demarkus/access.md` via FETCH, `auth.GrantsFor`, never-gated, reflects only the caller's token) — ~20-minute resurrect when a real plain-remote destination or a first-class "what can my token do here?" need appears. Branch `feat/plain-remote-access-discovery` discarded; main untouched.

**Accurate phase-0 status:** promote primitive ✓, **access discovery — brokered ✓ (A2 plain-remote DEFERRED)**, world.md ✓, coherence edge ✓, triggers (doc-type + scan) ✓, dogfood ✓. Phase 0 functional for all brokered use; A2 waits on a real plain-remote need. Remaining epic: signal-based triggers (telemetry) + external inflows (Confluence, deferred).


### 2026-06-18 — plain-remote promote, all client-side (no spec change)

Fritz's steer: promote SHOULD work against a plain demarkus server, but **without** the server/SPEC change A2 proposed. Resolved by replacing server-side grant *discovery* with client-side *declaration*: for a plain endpoint you register, you already know where you may write, so declare it at registration. The server stays a dumb bookshelf; nothing in the protocol changes.

Key reframe: the promote bridge still needs both plugins (knowledge plugin owns the cascade); what widens is **what a destination can be** — a brokered knowledge system OR a registered plain remote demarkus server.

Built (all in demarkus-memory + the knowledge-promote skill, no Go/server/SPEC):
- **Plain-target registry** `~/.demarkus/promote-targets` (`<mcp-slug> <write-path> [label]`). lib.sh: `promote_targets()`, `promote_destination_present()` (brokered OR plain). Memory stays broker-unaware.
- **`scripts/promote-target.sh`** (`list` / `add <slug> <path> [label]`) — validates path starts with `/`, idempotent on slug+path. Inline registration UX: `/promote` offers to register when no destination exists.
- **`scripts/detect-promote.sh`** replaces `detect-knowledge.sh` — unified detector: `NONE` | `knowledge <slug>` | `target <slug> <path> [label]`. (Old script + test removed.)
- **`promote-nudge.sh`** gate widened `knowledge_present` → `promote_destination_present` (ADR nudge fires for a plain target too).
- **Commands** `/promote`, `/promote-scan`, `/soul-refresh` consume `detect-promote.sh`; `/promote` routes brokered (mark_worlds/world.md) vs plain (declared path, ceiling human-only, deny surfaces at publish).
- **knowledge-promote skill** step 5 re-adds a plain-remote branch — caller-supplied path (NOT the reverted access.md), policy.md honored only if present.
- Tests: `detect-promote_test` (5), `promote-target_test` (6), `promote-nudge_test` (+plain-target case, 8). Full plugin regression green; shellcheck only the standard SC1091.
- memory `0.7.0 → 0.8.0`; marketplace + session-guidance reworded to "a joined knowledge system or a registered plain remote server."

This is what A2 was *for*, done the client-side way — so A2 (server access manifest) stays correctly dead. Discovery-by-declaration > discovery-by-protocol here.

**Capability note:** plain-remote promote still requires the knowledge plugin installed (cascade lives there) + a registered target. Memory-only is still dormant by design.