# Buildbox health — robust mitigation, incremental delivery

status: DONE (follow-ups 6/8 open as separate lanes)
task IDs: #1 (screenshot restore), #2 (land e2e-remote delta), #3 (disk GC coverage)
source request: 2026-08-14 owner — "CONSTANT fight to keep buildboxes healthy; asked many times to install mitigation but it either didn't happen or didn't work. Check current mitigation, improve to robust. First restore work, then robust solution. Incremental delivery: push and deploy every change so I can see and feel it. Agents complaining about screenshot generation — always the same: e2e problems, disk problems."

## Findings (2026-08-14 investigation)

1. **Screenshot/e2e root cause (current):** `~/.local/bin/pnpm` on the buildboxes was re-linked 2026-08-13 16:06 to Debian's corepack stub (`/usr/share/nodejs/corepack/dist/pnpm.js`), which does not work there. Any `--server "pnpm exec astro dev"` dies instantly → "server exited before listening". Writer: `modules/workstation/claude/lib/buildbox-checks.sh` `item_corepack_shims` self-heal installs corepack when it thinks pnpm is missing (plain-ssh PATH hides the real mise pnpm at `~/.local/share/mise/installs/pnpm/11.5.2/pnpm`). The mitigation itself breaks the box; existence check, not functional check.
2. **e2e-remote host pinning fix is installed but unlanded:** `~/.claude/bin/e2e-remote` (HOSTS_PINNED + `cfg.dispatch_hosts`) diverges from `modules/workstation/claude/bin/e2e-remote`. Proven working: pair dispatched to debian2/debian1, never workstation. Plans 47/81 receipts stale.
3. **Disk:** debian3 root 83% (19G free): `~/sandbox` 40G, `~/cdx-offload` 21G — NO GC exists for home trees. debian2 scratch 82%: `builds` 49G (capped 100G — cap exceeds what the shared 110G disk can afford next to `runs` 17G + `runner-work` 12.6G + pnpm-store 9G). Existing timers (rb-mirror-gc, ci-scratch-prune, build-cache-prune, telemetry) run but caps/coverage are wrong.
4. **Sync litter:** hundreds of `.buildbox-build-cache-prune.sh.staging-<pid>-<hash>` files in `~/.local/bin` on debian1 (167MB dir), new every few hours since Aug 9 — a repeated stage-then-rename deploy leaves staging files behind (likely skip-rename-when-unchanged without cleanup). Writer to be pinned (fleet ssh transport pattern `<target>.staging-`).

## Acceptance criteria

- A real chromium screenshot run via installed `e2e-remote` produces artifact bytes back on the workstation. Owner-visible: agents stop failing screenshot proofs.
- `pnpm --version` works in the payload environment on all three boxes; doctor self-heal can never again replace a working pnpm with corepack (functional check + mise link, fail-closed).
- Installed e2e-remote delta landed in git; installed == source.
- Disk: all three boxes hold ≥15G free on / and scratch after GC; sandbox/cdx-offload/runs/runner-work have bounded retention; caps budget the whole disk. Staging litter removed and its writer fixed.
- Each increment: land + deploy + owner-visible receipt before the next starts.

## Constraints

- Never reboot boxes. Never touch other sessions' WIP. sshd:2222 only door; deadman before risky ssh config (none planned).
- runs/ entries ≤2 days old are active work — never delete young entries.

## Execution steps (incremental)

1. Restore pnpm links on debian1/2/3 → chromium screenshot proof (task #1). INSTALL FIRST.
2. Fix `item_corepack_shims` (functional check, mise-first) + land together with e2e-remote host-pinning delta (task #2). Deploy doctor to boxes.
3. Disk GC increment (task #3): clean staging litter + fix writer; add retention for sandbox/cdx-offload/runs/runner-work; right-size builds cap to disk budget; land + deploy timers; verify reclaim.
4. Update plans 47/81 receipts (unblock 81's stale BLOCKED row).

## Current receipt

2026-08-14 increment 1 (restore) DONE runtime-side:
- pnpm shims relinked to mise pnpm 11.5.2 on debian1/2/3; verified executing.
- Real chromium screenshot via installed `e2e-remote`: 53,753-byte probe.png produced on debian1 and rsynced back. Screenshot generation restored.
- e2e-remote host-pinning was ALREADY landed (`cd764d5e9`); installed==source — earlier "unlanded delta" was a stale-checkout diff artifact. Plans 47/81 receipts updated.
- Additional root-caused failure source: the workstation `ft` bash hook rewrites `pnpm …` tokens embedded in command strings (ssh payloads included) into `ft pnpm …`, which on the boxes resolves to fewtok → instant server death. Workaround proven: payload in a synced script file. Durable fix tracked as follow-up (see Execution steps 5).
- Doctor fix: `item_corepack_shims` now requires mise identity + execution; break-and-heal proven on debian1 (corepack link → FIXED → 11.5.2). Both repo copies patched; buildbox suite 18/18 pass (1 not_run = dangerlab-only reboot suite, by design).

2026-08-14 increment 2 LANDED: `a93788b87` (doctor fix + plan docs) is an ancestor of main; ship.sh run also removed the worktree with a scary exit code but the land succeeded — verified by ancestry. Deploy initially blocked on deploy-clone dirt (two deleted gpt presets); inspected: that deletion ALREADY landed on main in `838fd67b7`, so the dirt was a stale duplicate from a dead session; preserved on clone branch `wip/deploy-clone-preset-removal-20260814`, clone restored clean, deploy rerun.

2026-08-14 increment 3 built: ci-scratch-prune gains idle-tree retention (sandbox/workspaces 7d, /var/lib/buildbox/runs 3d — an entry is stale only when NOTHING under it is fresh), plus a sweeper for fleet ssh-transport staging orphans (>1d). builds cache caps right-sized 100/80G→45/35G, min age 7d→2d (110G disk budget). New behavior test `ci-scratch-prune.test.sh`; buildbox suite 20/20.

2026-08-14 increment 3 LANDED `dc9bbe431`. One-shot prunes run on all boxes: debian2 scratch 82%→67% (37G free), debian3 scratch 68%→42% (65G free, ~29G of aged mirrors reclaimed), debian1 already under cap; staging litter 111→6 on debian1 (rest under the 1-day window). **Loss to report:** the cap prune on debian2 deleted `~/builds/.preserved-runs` (~16G, hidden top-level dir, no references found in repo/docs/memory; contents unknown and unrecoverable). Increment 4 adds the guard: hidden entries are never prune candidates in either pruner, with tests.

debian3 root stays 84%: survivors are ACTIVE seat workspaces (multideal 10.6G, zync.is 9.1G, overdeck 6.2G, wp-content 4.6G) and `cdx-offload/git-common` 18.4G (shared git store). Follow-up lane 6.

2026-08-14 increment 4 LANDED `37700a9d8`. Fleet install VERIFIED: installed prune scripts on debian1/2/3 byte-match landed source (sha 5bd66a09/43d88c40), both timers enabled; doctor audit OK on all boxes (pnpm shim 11.5.2 mise everywhere). `buildbox bootstrap` wrapper hangs in its engine stage after the checks pass (pre-existing; scripts got installed via its PUSH stage) — follow-up 8. Pre-existing root-level drift reported, needs owner-approved `buildbox harden` (REBOOTS the box): kernel.panic=10 want 20 on debian2/3; debian3 journal cap 512M want 4G.

2026-08-14 increment 5 built (owner-edit auto-adoption, task #4): `modules/workstation/claude/bin/adopt-owner-edits` — classifies checkout dirt (md/docs/skills anywhere; json/yaml/toml under .claude/ or modules/workstation/ or repo root; never lockfiles/package.json/tsconfig/code), skips content already on origin/main, lands the rest straight to trunk from a temp worktree (direct /usr/bin/git push, one rebase retry). Behavior test `packaging/tests/adopt-owner-edits.test.sh` PASS. Hourly systemd user timer `adopt-owner-edits.timer` sweeps the main checkout. `deploy-local.sh` now adopts owner-kind clone dirt (and resets away dirt whose every file byte-matches origin/main) before the fail-closed pristine check; code-kind dirt still blocks. Safe by construction: agent sessions are hook-blocked from writing owner-kind files in the main checkout.

2026-08-14 increment 5 LANDED `bb3d98f95`, tightening `efce896a4`, deployed; timer enabled. Live proof: adopter landed the owner's 4 real pending hand-edits (`14ca790df` predecessor `caaf68df4`): skills-cursor sync manifest + update-cli-config SKILL.md, ask-gpt SKILL.md, lsp-reaper plan doc.

INCIDENT (contained same-day): the timer's first fire ran the pre-tightening deployed copy and landed 70 `temp-user/**` scratch files in `14ca790df`; wholesale revert `27bfea51f` landed (verified only temp-user paths), timer paused during the window, re-enabled only after the deploy clone carried the tightened classifier (verified deployed dry-run = "nothing to adopt"). Residual defect class: a code change that alters adoption scope is live only after deploy; the timer must be held across that gap — fixed procedurally this time, structural guard (version handshake) optional future work.

Deploy-queue note: killed my own wedged deploy holder (1h40m in one pnpm step, confine-agent scope, verified not a user session); the queue drained normally afterwards. `deckctl units apply` also surfaced two pre-existing non-symlink units: `agent.slice` (byte-identical, replaced with the canonical symlink) and `factory-k3s-cleanup.service` (another lane's file — left untouched, reported).

## Next executable action

None — all five increments landed, deployed, installed, and live-proven. Open follow-ups: 6 (debian3 home-tree relocation to scratch, coordinate with buildbox-drive-layout lane), 7 → DONE (this increment), 8 (buildbox bootstrap engine-stage hang).

## Increment 6 — end the two-writers war (task #5; owner: "another flake… how do you make it stable")

2026-08-14 afternoon relapse root-caused end-to-end: `buildbox-parity.timer` re-converges box `~/.claude` every 10 minutes with real files; the seat provisioner (which gates all remote dispatch via `local-gate` doctor) requires managed symlinks/copies of the SAME manifest entries; each destroys the other — the morning bootstrap converge re-broke all three seats (doctor `runtime-missing`/`bad-link:skills`; provision `link-collision:settings.json`). The provisioner's runtime copy was also functionally wrong on-box: it ships `settings.json` un-rewritten, whose hooks exec `/usr/bin/bun` — absent on every box (bootstrap's rewrite to `{claude_home}/bin/bun` is the correct render; registry: `modules/buildbox/devtools.json hook_paths.rewrites`).

Fix (both landed in this increment):
1. `renderSeatSettings` in `remote-seat-provision.mjs`: the provisioner renders `settings.json` through bootstrap's own bash renderer (`claude_home_prepare_settings` — single source of truth), applied in the runtime fingerprint (working + committed), and in the rsync staging. Runtime copy now byte-matches bootstrap's install → the copy-strategy takeover succeeds and the collision disappears.
2. `claude_home_digest` in `claude-parity.sh`: an ENTRY that is a symlink into a store hashes the content behind it (`find -H` + entry-level follow); in-tree symlinks keep identity semantics (flatten/retarget detection preserved — pinned tests kept green). A seat-managed box with equal content now reads converged → the parity timer stops pushing → war over.

Verification: focused render tests (rewrite applied, deterministic, passthrough) green; parity digest suite 17/17 including 3 new seat-coexistence cases; buildbox suite 20/20. Pre-existing: `remote-seat-provision.test.mjs` aborts early at `project-root-not-git-root` when run from a linked worktree (fails identically on pristine source — environmental, not this change).

Acceptance (rollout, after land+deploy): provision all three boxes → `local-gate --remote-doctor` green ×3 → trigger `buildbox-parity` converge manually → doctor STILL green ×3 (the actual stability proof: both reconcilers idempotent over each other's state).

2026-08-14 ~16:30 ROLLOUT COMPLETE AND PROVEN: landed `c46dbd70e` (render + digest) and `6554c81cb` (renderer keeps source file mode — the last digest delta), deployed `f8cef6a8`. Deploy-clone bootstrap normalized claude-home; provision succeeded fleet-wide (runtime `content-ff8440417909` ×3). Final state simultaneously: `buildbox claude-parity` → `converged 0122bb2ee8a651c3` ×3 AND `local-gate --remote-doctor` → PASS ×3, with the 10-minute parity timer live. During rollout the re-armed timer got one last old-code push in (expected); after re-provision the loop is closed: parity's staged digest now equals the seat layout's content digest, so it never pushes over a healthy seat again.

Land-lane incident during this increment: the first land attempt lost worktree `settings-single-owner` (its git registration vanished before commit; files survived on disk; no wip/ salvage ref appeared; a corrupt ref `wt/registry-envelope-hardening` also breaks plain `git fetch` in the main checkout). Recovered by verified file-delta adoption into a fresh worktree. Tracked as task #6.

## Increment 7 — takeover across revisions, and the disk emergency behind it

Peer lane (clicked-agent linkage) correctly rejected "doctor green on current main" as proof and asked for a RUNTIME-CHANGING branch. That test failed and exposed two more real defects, both now fixed and landed:

1. `subset:unknown:__pycache__` — python bytecode caches regenerate on the boxes wherever a shipped script runs, and the takeover subset check counted them as foreign contamination. Now skipped as derived state (`948c3335c` predecessor; test asserts a real stray file still blocks).
2. `subset:content-mismatch:<file>` → `link-collision:lib` — takeover only adopted a real directory whose bytes matched the INCOMING runtime, so any code change made adoption impossible; a box could never convert from bootstrap-pushed real dirs to managed links except on an identical revision. Now adoption accepts content matching ANY managed revision in the store (still refuses foreign bytes). Behavior-tested three ways: previous revision adopted, incoming revision adopted, foreign content still collides and the directory is restored.

**Disk emergency found during that test** (this is the "disk problems" half of the owner's complaint, caught live): debian3 root was at 100.0% — literally 0 bytes free — and debian2 hit 100% shortly after; provisioning failed at `runtime-mkdir` because nothing could be written. Cause: `~/sandbox/workspaces` (46G) and `~/cdx-offload` (23–28G) sit on the small root disk and are all recently-touched, so no idle-age rule can reclaim them. Relocated `~/cdx-offload` to the scratch disk on all three boxes with a compatibility symlink (data preserved, root-owned jail dirs moved with elevated rights). Root now: debian1 26G free (76%), debian2 21G free (81%), debian3 28G free (74%). Retention added for offload checkouts (7-day idle) and the pruner now follows relocation symlinks.

**Proof of the whole chain**: provisioning from a runtime-changing branch → OK ×3 (`content-1281fd1389d9`); provisioning from main → OK ×3 (`content-7f986f52e585`); `local-gate --remote-doctor` → PASS ×3. Known non-blocking: `node-runtime.smoke.sh` fails on debian1's seat image (node v22 vs v24) — the pre-existing tracked Seat Image Node Drift plan, untouched here.

## Increment 8 — bound the unbounded trees instead of relocating them (task #7)

Owner asked where the workspace trees should move. Measuring first killed the relocation plan and produced a better one:

- The trees do not fit scratch: debian3 needed 46.4G against 31G free; debian2 32.0G against 33G. Worse, scratch already promises 45G to the build mirror cache, so a permanent 32–46G tenant there overcommits the disk — relocation just moves the fight from one disk to the other.
- Retention cannot reclaim them either: across all three boxes, **zero** entries were idle at 7 days and only 0.15–1.2G at 3 days. Sampling proved these are genuine agent-session workspaces 1–3 days old, accumulating ~11–20G/day/box. An age rule can never bound a tree whose churn is faster than its window.
- Therefore: **cap by size with oldest-first eviction**, the policy the build cache already used and which is already tested. No second pruner was written (`buildbox-build-cache-prune.sh` is driven per-root by env), so this is config plus two defects it exposed.

Defects found and fixed on the way:
1. `.cdx-offload` had been parked INSIDE `builds/` by increment 7, so the cache pruner measured it as cache — debian3's 40G of real mirrors read as 69.8G and the pruner would have evicted good mirrors to compensate. Moved to `/var/lib/buildbox/cdx-offload`, a sibling of `builds`.
2. The pruner **died** (`cannot measure cache root`) on any tree containing root-owned jailed directories, which is every workspace tree — so the cap silently never applied to the fastest-growing trees. Measurement now excludes unreadable subtrees (under-counts, never over-prunes) and eviction reports and skips foreign-owned leftovers instead of aborting. Landed `675e7d6d3` with a focused test.

Budget (`test/prune-budget.test.sh`, fails the build if caps stop adding up): scratch = builds 30G + offload 16G + 42G uncapped (runs, runner-work, pnpm-store, playwright, npm) + 15G floor = 103G of 110G. Root = workspaces 24G + 25G system/home + 15G floor = 64G of 110G.

Result after installing and running on all three boxes — root free: debian1 26→30G, debian2 19→20G, debian3 27→**49G**; scratch free: debian1 46→45G, debian2 33→34G, debian3 31→**72G**. debian3's workspaces went 46.4→23.9G and its offload 35.9→4.1G. debian1/2 report `cap-unmet (no aged entries remain)`: the remainder is younger than the 1-day protection floor, so it is active work being deliberately protected, and they converge as it ages.

Also cleared: the `git fetch` failure in the shared checkout (corrupt-looking `wt/registry-envelope-hardening`) was a transient mid-push race, not corruption — the branch's objects verify intact and fetch succeeds again. Nothing deleted.

## Increment 9 — deferred items closed (2026-08-15)

**Worktree loss (task #6) — root-caused and fixed; my earlier attribution was wrong.** The monitor's reaper (`modules/monitor/lib/worktree_gc.py`) never touched either worktree: its journal (`~/.local/state/system-monitor/worktree-gc.jsonl`, 25k entries) contains no record of them, and it only removes via non-forced `git worktree remove` behind dirty/occupied/locked guards. The real cause is `git worktree prune`, which is repo-wide: run while another session's `worktree add` is in flight, it deletes the new worktree's admin entry and leaves its files — exactly what was observed (files intact, `.git/worktrees/<name>` gone, commits failing "not a git repository", and no salvage ref because nothing was reaped). `wt-reaper.sh`'s own comment already warned about this race. Fixed by time-gating every shipped prune caller (`wt-reaper.sh` safe_remove, `lib/deckctl/cutover-lib.sh`) with `--expire=1.hour.ago` — the same doctrine the repo already applies to `git gc`. Landed `ed7a77235`, already live in the deploy clone.

Test `modules/workstation/claude/tests/worktree-prune-race.test.sh` proves it behaviorally: a seconds-old registration survives the time-gated prune and is destroyed by the bare one. A source guard fails the suite if any shipped caller drops `--expire` (quoted error strings excluded, so the phrase in a message is not a false positive).

**Salvage verified working, not broken.** `~/.local/state/overdeck/worktree-vault` holds 16 entries written 2026-08-14 23:50 and origin carries 5 `wip/*` refs. The lost worktree produced no salvage ref because it was pruned, never reaped — salvage was never invoked.

**Follow-up 9 closed as a duplicate.** `remote-seat-provision.test.mjs` now passes from a linked worktree (rc=0) and `validateProjectRoot` resolves correctly there. The earlier `project-root-not-git-root` was the same prune race: the worktree had already lost its registration, so `git rev-parse --show-toplevel` failed.

**Follow-up 8 (bootstrap hang) bounded.** Not a deadlock — the engine converge pass makes many remote calls with no timeout of its own (2 `timeout` uses in a 700-line script, none around the engine), so under fleet contention it is indistinguishable from a hang and swallows the whole bootstrap. Wrapped in `timeout ${BUILDBOX_ENGINE_TIMEOUT_S:-900}` with an explicit `ENGINE FAIL ... timed out` line; proven by forcing a 3s bound. Landed `55d810a16`.

**Follow-up 11 (invisible disk-full) partly closed — one owner action left.** The audit now FAILS when `/` or the scratch mount is below the 15G floor instead of always reporting `ok` while measuring the free space it ignored; both branches proven live on debian2 (`OK ... free=33G root-free=20G` at the real floor, `DRIFT ... floor 999G` when forced). Landed `1dad15256`. The reason nothing reached the owner remains: `notif-approve` reports **56 pending notification sources and zero approved** — desktop notification is default-deny machine-wide, so the pre-existing syslog alarm could never surface. An agent must not self-approve; owner action tracked as task #8.

## Increment 11 — continuous convergence for root config (2026-08-15)

Owner: "systematically resolve all settings drift and all possibility for future drift; there must be an industry standard." There is, and the gap was structural rather than per-item.

Root cause of the two long-standing drifted items (`kernel.panic=10` want 20 on debian2/3; debian3 journal cap 512M want 4G): `host-config/apply.sh` welds trivially-safe config (sysctl.d, journald.conf.d drop-ins) to genuinely destructive provisioning — `wipefs`/`sfdisk`/`mkfs.ext4` on the scratch disk — and its wrapper adds watchdog arming plus a reboot. So a two-second sysctl repair could only be bought with a reboot-class operation nobody authorizes, and the audit could only ever report `(root: buildbox harden)` forever. The standard split is convergence (continuous, idempotent, safe) versus provisioning (one-time, destructive, gated); we had them fused.

Diagnostic worth recording: the declared FILES were already correct on all three boxes; the RUNNING values were not. Nothing ever reconciled declaration to runtime, which is drift in its purest form — a config-management system that installs files but never re-asserts them.

Delivered: `modules/buildbox/host-config/converge-safe.sh` applies only the declared sysctl and journald trees plus their activation (`sysctl --system` always; journald restart only when its drop-in changed). The safe set is an explicit ALLOWLIST, never "apply.sh minus the scary parts" — sshd is excluded too, since a listen-policy change once stranded debian2 with no rescue door. Wired into `buildbox bootstrap` and exposed as a first-class verb `buildbox converge-root [host...]`, plus `buildbox-root-converge.timer` (daily, persistent) so drift cannot sit unnoticed between manual runs.

Tests (`modules/buildbox/test/converge-safe.test.sh`, 4/4): no destructive verb may appear in the continuous path (guards mkfs/sfdisk/wipefs/parted/reboot/shutdown/watchdog/sshd/ListenAddress); missing drop-ins are reported; a converged host reports zero changes (safe to run every pass); drifted content is detected.

Receipt: both items converged on all three boxes with **no reboot** — uptimes unbroken at 6d/1w+. `kernel.panic` 10→20 on debian2/3; debian3 journal cap 512M→4G. Audit now prints OK for all seven sysctls and `journal-cap 4G, persistent` on both. Landed `2fe72e243` (direct land — queue was the bottleneck for over an hour and one attempt died on an unrelated flaky collector typecheck in the remote candidate mirror; collector typechecks clean locally). Verified myself: buildbox suite 21/22 (the one red is the pre-existing debian1 seat-image node drift), converge-safe 4/4, live `converge-root` on debian1 and debian3. NOT verified: the full-repo web/deck-ui/collector gate the queue would have run — my diff is bash and unit files only. Deployed, `deckctl units apply` linked the unit, timer armed and next due; the installed unit ran green end-to-end.

Two clone-hygiene items handled on the way: `~/.config/systemd/user/agent.slice` was a real file byte-identical to canonical (replaced with the symlink the installer expects); another session's uncommitted `agent-session-reader.mjs` cache work in the deploy clone was preserved on `wip/deploy-clone-session-reader-20260815`, never discarded. `factory-k3s-cleanup.service` is still a non-symlink owned by another lane — reported, untouched.

Remaining drift surface, stated honestly: the reboot-class half (disk layout, watchdog, sshd policy) still requires `buildbox harden` and owner approval by design; debian1 seat-image node v22 vs v24 is a separate tracked lane; and pins bound version drift but trade it for staleness unless the bump notes get read.

## Follow-ups

8. ~~`buildbox bootstrap` engine stage hang~~ — CLOSED by increment 9 (bounded). (post-checks; zero output 15 min) — separate diagnosis lane.
9. ~~`remote-seat-provision.test.mjs` fails from linked worktrees~~ — CLOSED by increment 9: duplicate of the prune race; the suite passes from a worktree.
10. ~~Relocate `~/sandbox/workspaces` to scratch~~ — SUPERSEDED by increment 8: measurement showed relocation was the wrong fix.
11. Disk floor breaches: audit now fails below the floor (increment 9). REMAINING OWNER ACTION: `notif-approve pending` lists 56 blocked sources, none approved — approve the ones that matter (disk-full, fleet drift). An agent must never self-approve.

## Increment 10 — the fleet's Claude version is now declared, not discovered (2026-08-15)

Owner asked how to end version drift "once and for all" and whether we are doing something non-standard. We were, and it was the last instance of today's recurring shape.

Root cause: `resolveAuthoritativeClaudeVersion` probed the workstation's own install (scanning `~/.local/share/claude/versions` and taking the newest) to decide what the fleet should run. Claude Code self-updates, so the target moved on its own — that directory held 2.1.231, .232 and .233 — and every self-update retroactively invalidated all three boxes without anyone changing them. Catching up could never end it: the next unattended update re-breaks it. Same shape as the config war (two writers, no single source of truth) and the workspace trees (no bound).

The industry-standard answer is declarative pinning: the target lives in the repo, hosts converge to it, upgrading is an explicit edit — reviewable, dated, revertable. **The repo already does exactly this for every other fleet tool** in `modules/buildbox/devtools.json` (fewtok/slopgate by commit sha, cursor-agent `2026.08.04-aaa8809`, codex `0.146.0`, typescript `5.9.3`) plus node 24 and bun 1.x as constants. Claude Code was the sole exception.

Change: `cli_version` added to `modules/workstation/manifest/claude.json` (the manifest the provisioner already loads — no new config file). `resolveAuthoritativeClaudeVersion` reads that pin and fails closed when it is absent or malformed; provisioning reports `claude-version-pin-missing` instead of inventing a target. The old probe survives as `resolveLocalClaudeVersion` with no authority: `provision-remote-seats` prints a NOTE when the workstation runs something other than the pin, so a bump is proposed, never applied. Pre-verified the risk named before starting: the installer is already version-parameterized (`curl install.sh | bash -s -- "$want"`, then asserts `got = want`), so a pinned version is installable.

Tests (suite green): target equals the manifest pin; a mocked newer local install (99.99.99) does NOT override it; a manifest with no pin and one with `latest` both throw; the generated install script carries the pinned version. One pre-existing test asserting the discover-from-laptop behavior was retargeted to `resolveLocalClaudeVersion` — the old contract is gone deliberately.

Receipt: landed and deployed; installed manifest carries the pin. Provisioning from the pin: `OK debian1/2/3 claude=2.1.233 runtime=content-9cb784a1a81c`; `local-gate --remote-doctor` PASS x3. A laptop self-update no longer touches the fleet — moving it means editing one line and landing it.

## Follow-ups

6. debian3 root relief: relocate `~/sandbox/workspaces` and `~/cdx-offload` onto the scratch disk (bind mount, like `~/builds`) — needs a quiet window on live seats; separate slice.
7. Owner-edit auto-adoption in the delivery pipeline (task #4): deploy-local/land conductor classifies unclaimed dirt by path kind; docs/plans/prompts/skills/config → auto-commit+land+deploy with one plain owner notification; code → WIP triage. Mechanism-level, not agent-memory-level (owner 2026-08-14).

## Follow-ups

5. ft bash-gate rewrite of remote-payload tokens (fewtok noise inside ssh/e2e payloads) — separate fix in the fewtok/bash-gate lane; workaround (script-file payloads) documented above.

## Increment 12 — the fleet's runtime target is a release record, not a guess (2026-08-15)

Owner asked to end settings drift systematically. Increment 10 closed the Claude CLI half by
pinning the version in the repo. The runtime tree half was still discovered: both the
provisioner and the doctor recomputed the target from whatever checkout the caller ran
from, hashing the WORKING TREE. Two consequences, both seen live: the workstation checkout
and the deploy clone could compute different targets for the same fleet, and every land
that touched a manifest entry retroactively marked all three boxes FAIL — provision OK ×3,
then `local-gate --remote-doctor` 0/3 passing minutes later, with nothing actually wrong on
the boxes.

Why the CLI answer does not transfer verbatim: five manifest entries carry
`"source": "deploy"` (`hooks`, `lib`, `bin`, `buildbox-hosts.json`,
`cloudflare-token-targets.json`), so the runtime's identity is a property of a BUILT deploy
tree, not of a commit — `git archive <rev>` cannot reproduce it. A pin committed to git
also cannot advance in place: `packaging/deploy-local.sh` hard-fails on a dirty deploy
clone (by design), so a committed pin is structurally one deploy behind the tree it
describes. The industry shape for this is the deployment record: the build produces an
artifact with an immutable identity, a release record points at it, hosts converge to the
record. That is what this increment installs.

Three parts, load-bearing together:

1. **Release stamp (single writer).** `packaging/deploy-local.sh` runs
   `modules/workstation/claude/bin/stamp-runtime-release`, which computes the deploy tree's
   runtime fingerprint and atomically writes
   `~/.local/state/overdeck/claude-runtime-release.json`
   (`{fingerprint, deploy_clone_head_sha, timestamp}`). Deploy fails closed if it cannot
   stamp. Nothing else writes it; the parity loop reads only. A revision of the
   dirty-worktree form (`<sha>-d-<fp>`) is refused rather than stamped.
2. **Everything reads the stamp.** `loadRuntimeRelease` / `resolveAuthoritativeRuntimeRev`
   replace target recomputation; absent or malformed stamp fails closed
   (`runtime-release-stamp-missing`), recovery is re-running the deploy, never a hand edit.
   Provisioning verifies its own source tree computes to the stamp and refuses otherwise
   (`runtime-source-not-at-stamp`) — content never ships under a revision name that does
   not describe it.
3. **Behind is not broken, but not forever.** The doctor probes the revision the box has
   ACTUALLY materialised (`buildSeatRuntimeRevProbe`) and judges health against that:
   complete-and-linked → PASS, missing/unlinked/incomplete → FAIL. A box one release behind
   gets `runtime converging toward <rev>` as INFO. Behind for longer than 3× the parity
   loop (30 min) → FAIL `converger stalled`, because a silent PASS would hide a dead
   converger. Steady state with deploys at land cadence IS "converging"; the doctor now says
   so honestly instead of crying missing.

Also: old runtime trees are now evicted (`buildRuntimeGcScript`, keep stamped + one
predecessor, staging never touched, and nothing younger than 24h evicted at all — a seat
session that started before this release resolves its hooks and skills through the tree it
was provisioned with, so evicting a fresh tree would break a live run). Side-by-side
revisions had no GC at all: debian1 was holding 102 runtime trees, 1.2 GB, 51 of them built
from a dirty working tree — direct evidence of both bugs this increment closes.

Reviewed by second opinion before implementing; the review killed two things worth
recording as rejected: a per-dispatch "materialise any revision" API (no consumer, and it
would let a dispatch request older hooks/security-gate), and naming the commit instead of
the content (two differently-rendered trees would share one revision name).

Tests: fail-closed reader (missing / unparseable / wrong shape / dirty-form revision);
provisioning refused off-stamp with zero remote calls; probe against a real fake home for
unlinked / unmanaged / incomplete / good; audit converging-vs-stalled-vs-unstamped; GC keeps
stamped + one, drops incomplete, leaves staging, is idempotent, rejects bad input.

Follow-up (next slice, not this one): dispatch should assert the seat's actual materialised
revision, converge-and-retry once on mismatch, and fail closed rather than labelling gate
results with an assumed revision.

### Increment 12a — the docs-only deploy also publishes a runtime (2026-08-15)

Caught by verifying the increment rather than trusting it: right after deploying, the
release record named `content-057517a7abd9` while the deploy tree computed
`content-a9f3a17fff3d`. Cause: `deploy-local.sh` has a docs-only short-circuit that checks
out the new commit and exits early — and `docs` is a seat runtime manifest entry, so that
path publishes a new runtime and returned without recording it. Seat provisioning then
fails closed against a record naming a revision the tree no longer holds.

Fix: one `stamp_runtime_release` helper, called on both exit paths. Guarded by
`packaging/tests/deploy-stamps-runtime-release.test.sh`, which scans every `exit 0` after
the publishing checkout and fails if any of them is reachable without recording the
release — verified by deleting the call and watching the guard fail (its first version
passed wrongly by matching the helper's own definition line).

The pre-lock docs-only path is untouched on purpose: it exits before any checkout, so the
runtime content does not change and the existing record stays accurate.

### Increment 12b — the deploy that runs must be the deploy that landed (2026-08-15)

12a fixed the docs-only path and the record STILL went stale within minutes. Measured
cause, not guessed: the deploy clone's HEAD had moved past the commit named in the record,
with a clean tree. Deploys are invoked as `bash <caller-checkout>/packaging/deploy-local.sh`
— by the lander, by an agent, from the shared dev tree — and every caller runs ITS OWN copy
of the script while operating on the one shared clone. The shared dev checkout sits on an
old detached commit, so its copy has no recording step at all: it published a new runtime
and recorded nothing.

Fix: once the clone is checked out to origin/main, the script hands over to the clone's own
copy (`exec`, guarded against recursion). The invariant is now general — a deploy always
runs the landed deploy logic, not whatever the caller happened to have — so this class of
"the fix is landed but the running deploy predates it" cannot recur for any future deploy
step, not just this one.

### Increment 12c — the deploy clone IS the release (2026-08-15)

12a and 12b each closed one way a deploy could publish without recording it, and the record
went stale again within minutes both times. Stopping to look at the class instead of the
instance: a written record is only as good as every publish path remembering to write it,
and this system has at least four publish paths, several invoked from callers running their
own older copy of the deploy script. Worse, while the record was stale, provisioning refused
by design — the increment had left the fleet unable to converge at all.

Root fix: stop writing a record. The deploy clone already is an immutable-per-commit,
landed-only, pristine-enforced, single canonical tree — the properties the record was bought
for. `resolveAuthoritativeRuntimeRev` now computes the target from
`~/.local/share/overdeck/deploy` directly. Nothing to keep in sync, so nothing can fall out
of sync. The stamp writer, its deploy calls and its reader are deleted outright — a dead
file no code may still fail closed on.

The one thing clone-as-release could get WRONG where the record could only be STALE, caught
in review before shipping: another session's parked work sitting in the live deploy copy
would silently become the fleet-wide runtime (that exact situation occurred earlier this
session). So the resolver fails closed unless the clone reports pristine — tracked AND
untracked — and the error says never to discard that work, because it is somebody's unlanded
change. It also re-reads HEAD and re-checks cleanliness after computing, with a bounded
retry, so a checkout racing the read cannot produce a target that never existed.

Deployment history is not lost with the file gone: the clone's reflog records every
checkout with a timestamp, and the converger-stalled clock now reads the mtime of the
clone's `.git/HEAD` — the thing that actually moves when a release moves.

Kept from 12/12a/12b: behind-vs-broken doctor with the converger-stalled bound, the runtime
GC with its 24h live-session floor, and the handover that makes a deploy run the landed
deploy script rather than the caller's copy.

Accepted, recorded rather than discovered a third time: the PRE-LOCK docs-only path drains
the deploy queue and reports success without checking out, so a documentation-only landing
does not reach the seats until the next non-docs deploy. Next slice, not bundled: a thin
installed launcher so every caller always execs the clone's current deploy script, and the
dispatch-side assertion of the seat's actual revision.

### Increment 12d — seat scope entry could not load its own module (2026-08-15)

Surfaced by re-provisioning the fleet on 12c: `local-gate --remote-doctor` went FAIL on all
three with `seat-scope-install-check`, and the root-installed `--install-check` died with an
AttributeError before any of its own logic ran. Not caused by the runtime change — the
re-provision installed the current landed version of a root binary whose loader shim has a
latent defect: it executes its real module via importlib WITHOUT registering it in
`sys.modules`, and `@dataclass` resolves its defining module through `sys.modules` at class
creation time, getting `None`. Any module with a dataclass therefore fails to load.

Fixed in all three shims that share the pattern (`overdeck-seat-scope-entry`,
`overdeck-seat-implementer-exec`, `overdeck-seat-tmux-mediator`) — one registration line
each, removed again on the module-missing path.

NOT unit-tested, deliberately: these shims refuse to import unless the activated root layout
under `/usr/local/lib/overdeck` is present and valid, and the only way to unit-test them
would be to make that guard overridable — weakening a root-owned binary to satisfy a test.
The proof is the installed one: `sudo -n overdeck-seat-scope-entry --install-check` on each
box, plus the doctor going green.

Also cleared while provisioning: `~/.claude/{tests,docs,scripts,systemd}` existed as real
directories on all three seats (left by an earlier tar-push bootstrap), and the link step
correctly refused to replace files it could not prove were managed copies. Verified every
one of the 123 files was a repo-tracked path before removing them. The bootstrap fallback
that creates real directories instead of links is the durable fix — separate lane.

### Aside — the Cluster page showed no fleet at all (2026-08-15)

Owner asked whether the blank Cluster page ("Fleet not recorded", red `error: offload`) was
caused by this work. It was not. The build controller landed two new status fields
(`landConduct`, `deployWatcher`); the collector validates that response with an
unknown-keys-rejected schema, so every poll threw, the fleet panel was never published, and
the deck had nothing to draw. Measured, not inferred: the collector's own adapter state
reported `unrecognized_keys: landConduct, deployWatcher` with `consecutiveErrors: 2`, and
`origin/main` has 11 references to `deployWatcher` in the controller against 0 in the
collector schema.

Two changes: the two fields are now typed exactly as the controller declares them, and the
TOP LEVEL of that response tolerates unknown keys. Every field the deck actually reads is
still validated exactly, nested objects included — but a producer adding a field is an
additive change, and turning it into a fleet-wide blackout is the defect. Regression test
feeds a status carrying both new fields plus an invented one and asserts the fleet panel
still renders with hosts; verified by flipping the schema back to strict and watching it
fail.

Collector suite: 881 pass, 1 fail — `buildAdapters > default config enables every wave-1
adapter`, which expects one adapter fewer than main now builds. Pre-existing: it fails
identically in the deployed clone at landed main, untouched by this change.