# 2026-07-17: join URLs + cert pinning (remote-issuance step 1) Implemented step 1 of [/plans/remote-token-issuance.md](/plans/remote-token-issuance.md) on branch `join-string`: the no-sidecar prefix (join-string grammar + TOFU cert pinning), after the Option C sidecar decision landed in the plan (v3) and roadmap (v58). ## What shipped - `client/joinurl` (new package): Build/Parse of `mark://host#token=...&fp=sha256:` join URLs, cert fingerprint helpers (DER, PEM), `VerifyPin` TLS verifier. Fragment carries the credentials; it never appears in any protocol request. - `client/fetch`: `Options.Pins` (host to fingerprint map). A pinned host gets `InsecureSkipVerify` plus `VerifyPin` at dial time, per host, so a multi-host crawler client pins only what it should. Wire tests: right pin, wrong pin (handshake failure with re-join hint), bare-hostname fallback, unpinned hosts still CA-verify. - `client/internal/tokens`: entry gains `pin` field (tokens.toml, additive), `SetPin`/`Pins`/`ResolvePin` (flag > `DEMARKUS_PIN` > stored). - All three clients wire stored pins; `demarkus-mcp` gains `-pin`; new `demarkus join ''` stores token + pin in one paste. - `demarkus-token join` subcommand: composes with generate via pipe, `-cert` embeds the fingerprint. - `install.sh`: summary now prints a ready-to-paste `/soul-join ` and `demarkus join` line. - Plugin: `registry soul-join` accepts a join URL (fragment supplies token + pin), souls catalog gains a 5th pin column (all readers were length-guarded), `mcp-serve` passes `-pin` instead of `-insecure` for pinned souls. `commands/soul-join.md` updated; plugin 0.12.16 to 0.13.0. - Docs: `docs/site/tools/index.md` join-URL section. ## Decisions and gotchas - **Never pin Let's Encrypt certs**: leaves rotate on renewal (install.sh runs certbot twice daily) and would break every pinned client. install.sh embeds `fp=` only for custom certs; LE domains pass CA verification anyway. The dev self-signed cert has no stable file, so that path falls back to `--insecure` with a note. - **Go url fragment gotcha**: `u.Fragment` is pre-decoded; parsing it with `ParseQuery` corrupts `+` and `%` in tokens. Use `u.EscapedFragment()`. Conversely `url.Values.Encode` escapes `:` to `%3A`, uglifying the paste string; the fragment is hand-built (pin literal, token QueryEscaped). - Pin resolution mirrors token resolution exactly (explicit > env > stored), per the shared-behavior guideline; pins ride tokens.toml instead of a new file. - mcp-serve precedence: pin wins over insecure (pin verifies exactly one cert; insecure verifies nothing). ## Verified live Self-signed server on :6399: `demarkus-token generate | join -cert` produced the URL, `demarkus join` stored both credentials, publish + fetch ran with no `--insecure`, and a corrupted pin failed at the QUIC handshake with the actionable mismatch error. Plugin path: `registry soul-join ` wrote the 0600 token file, `pinned=1`, catalog row carries the pin. Full `make test` + `bash pre-commit.sh` clean. Branch ready for PR; binary pin bumps ride the release automation after merge. ## Follow-up: self-signed to CA migration Question surfaced the scenario: server moves from a pinned self-signed cert to a CA cert. Pinned clients hard-fail (strict pin, by design); recovery is re-joining with a pinless join URL. The plugin path already handled it (catalog row upsert clears the pin), but `demarkus join` only wrote a pin when the URL carried one, so a stale pin survived re-join and kept failing. Fixed in the same branch: re-join re-states the trust material, a token-bearing URL without `fp=` clears the stored pin (with a stderr notice when one existed). Verified live against the e2e server. Lesson: when a credential store is updated by re-stating, absence of a field must overwrite, not preserve; the two join paths (upsert-row vs set-field) diverged on exactly that. ## Cut: pinning removed, branch ships token-only Fritz was uneasy about the direction; the complexity audit agreed with him. The join URL half is ~100 lines of formatting with zero trust semantics; the pinning half carried a third trust state, rotation coupling, a migration bug (caught pre-merge), mixed-version edges, and hand-rolled VerifyPeerCertificate. Every follow-up question in review prep was a pinning question. HPKP precedent (browsers added then removed pinning in 2018) and the narrow constituency (only domainless hosts; LE covers the rest) settled it. Cut executed on the branch: joinurl is token-only (`mark://host#token=...`, unknown fragment keys still fail loudly so future keys cannot silently drop), fetch/tokens/CLI/TUI/MCP pin code removed, souls catalog back to 4 columns, `demarkus-token join` drops `-cert`, install.sh join line token-only with an `--insecure` note for self-signed, both plugin docs and site docs updated. Full test suite + pre-commit clean; token-only e2e re-verified live (CLI join, publish, plugin soul-join with fragment token). Decision recorded in the plan (v4): pinning deferred; if verified joins for domainless hosts return, they ride the sidecar redeem handshake (fingerprint out of band in the redeem response), not user-visible trust state. Plan sequencing also revised: sidecar redeem-only demoted; authenticated create is the sidecar MVP. Lesson: when a feature generates a stream of "what happens when..." questions before it ships, that is the complexity bill arriving early; the severable half that causes them is the half to cut. ## PR #261 review round (CodeRabbit) Eight findings fixed, two declined: - Fixed: joinurl.Parse rejects URL state Join cannot represent (userinfo, query, duplicate token keys) and bracketed IPv6 literals (downstream slug derivation and default-port append would misparse them); Build rejects `@` and `[` in hosts. Negative tests for all, plus a SoulJoin regression for bracketed IPv6. - Fixed: install.sh no longer swallows demarkus-token join failures with `2>/dev/null || true`; a failure warns with the manual command. - Fixed: RemoteSoulRow returns an error for a catalog row with no host instead of ok=true (mcp-serve would have launched with an empty -host); mcp-serve surfaces catalog I/O errors separately from "not in the catalog". - Fixed: demarkus-token join bounds stdin with a LimitReader (4KB) so piping a wrong file cannot become a "token". - Fixed: joinurl package comment tightened to 3 lines; new test's MkdirAll error handled. - Fixed: pi-memory soul-join doc no longer shows a token-bearing join URL as slash-command input (contradicted pi's own token-in-transcript rule); credentialed joins go in the user-run terminal command. - Declined: checking fmt.Fprintf errors on stderr usage output; every existing subcommand's usage block uses the same unchecked idiom and each is followed by os.Exit(1). - Declined: repo-wide IPv6 support in fetch.ParseMarkURL / TLS SNI; pre-existing behavior shared by every caller, not introduced by this PR. The join paths now fail loudly on IPv6 literals so nothing broken can be stored; full IPv6 support filed as follow-up. ## PR #261 review round 2 (outside-diff + late inline) Two fixed, one declined: - Fixed: port-only authorities rejected in joinurl (Build refuses a leading-colon host, Parse requires a non-empty hostname); regression cases both paths. - Fixed: SoulJoin now routes every non-HTTPS host through joinurl.Parse, not only fragment-bearing ones, so malformed legacy inputs (paths, userinfo, query, IPv6 literals) fail loudly instead of half-parsing into a broken catalog row. Behavior change accepted deliberately: inputs like mark://host/path previously "succeeded" by storing a corrupt host. Regression cases added. - Declined: unifying server/tools version resolution in install.sh into one pinned manifest. Independent per-component latest-release resolution is the installer's pre-existing pattern for every component; the skew window for the join subcommand is the gap between merging and the tools release publishing, and the failure mode after round 1's fix is a visible warning with the manual command. A single release manifest is a real installer improvement but a separate effort, not this PR. ## PR #261 review round 3 One finding, fixed: joinurl gains a shared validateHostPort rule (non-empty hostname, port numeric 1-65535, max 5 digits) enforced by both Build and Parse. Previously Build emitted h:bad and h:65536, and Parse accepted numeric out-of-range ports (Go's net/url validates digits only, not range). Subsumes the round-2 port-only-authority check. Negative tests: h:bad, h:65536, h:0, port-only, both paths. ## PR #261 review round 4 One finding, fixed: validateHostPort's hostname check flipped from a rune blocklist (which kept missing cases: spaces, stray brackets) to a DNS-charset allowlist ([a-zA-Z0-9._-]). Subsumes the previous ContainsAny("/#?@[") check entirely; one rule now covers every malformed-host class from rounds 1-4. Tests: "h x" and "h]x" rejected. Lesson: for input validation on a credential-bearing string, start with an allowlist; every blocklist round-trip with a reviewer is a missed-rune bug you shipped. ## Merged PR #261 merged to main 2026-07-17 as commit 4c5f90d (squash) after four CodeRabbit rounds. Local branch deleted; main pulled. Plan doc updated to v5 (sequencing step 1 DONE with merge refs). The release chain takes it from here: client/tools releases, then the automated pin-bump PR repoints the plugins at binaries that carry the join subcommand and join-URL SoulJoin (until that lands, a new install.sh with an old tools release warns and falls back to the manual command). Next up per the plan: the sidecar, redeem + authenticated create together. ## Parity decision: the sidecar is the broker in single-host mode Fritz set a parity goal: personal, team, and non-k8s deployments get the same library + broker experience as the k8s universe, including invites. Analysis showed the broker's only real k8s bindings are the Secrets writer (becomes a tokens.toml writer; the server already file-watch reloads) and packaging (becomes systemd/compose); OIDC works fine off-cluster via a Google/GitHub OAuth app, and the library only knows a broker URL. Consequence: no new sidecar binary; the remote-token-issuance plan (v6) now sequences broker single-host mode first (k8s audit, file backend, packaging), then invites in the broker minting core for both backends. The one open design question is identity for the zero-external-dependency personal tier; leaning capability-token front door before any built-in IdP. Convergence moved from the last step to the first.