# Execution-plane insulation — laptop is a terminal, not a compute node

audience: AI coding agents first. Status: PROPOSAL — direction for user review; supersedes nothing yet.

## Problem, measured

- Complaint registry (`0 DOCS/COMPLAINT_REGISTRY.jsonl`, 207 rows): cpu-load 36, system-crash 13, disk-io-load 7, memory-load 5, runaway-process 5, graphics-crash 2 — 68 complaints are machine-takeover, the #1 cluster.
- Live snapshot 2026-08-06 09:39 while user complained: laptop load 9.44; 4 `confine-agent-*.scope` + 2 `confine-build-*.scope` + 1 harness `v2-child-dispatch-*.scope` running locally; npm installs at 120–200% CPU.
- Failure shape: harness/factory runs execute ON the laptop (harnessd, `~/wt-factory`, cursor/codex seats in bwrap+scopes). Killing the visible agent is useless — the daemon respawns work; user lacks one kill switch and one live inventory.

## Rule

**No agent compute on the laptop.** Insulation by placement, not by wrapping. The only model process allowed locally is the user's interactive main session. Every dispatched seat — harness, factory, cursor, codex, workflow subagent fleets — runs on a buildbox inside a named cgroup scope. Protection layers on the laptop (tmpjail, mem-guard alerts, per-scope ceilings) become telemetry, not the boundary.

Why placement beats a laptop jail: cgroups/bwrap/podman share the laptop kernel — OOM storms, IO saturation, and fork floods still stall the desktop and crash the graphics env (13 system-crash + 2 graphics-crash rows). A buildbox bounds the blast radius to a disposable machine; worst case is `ssh reboot`, laptop unaffected. Both boxes exist (31 GiB / 8 CPU), with transport, health, GC, and doctor infra already landed.

## Verdict on existing plans

| Plan | Insulation verdict |
|------|--------------------|
| `2026-08-05-agent-seats-on-buildboxes.md` (crash-plan Phase 2) | RIGHT primitives (per-seat scope+slice, per-seat UID, netns, OOM group kill, stop verb, doctor), WRONG scope: interactive seats only; non-goals explicitly exclude harness scheduling — the thing that burns the laptop. STATUS 2026-08-06: substantially IMPLEMENTED on main (`seat-remote`, `seat-launcher`, `provision-remote-seats`, seat guard/proxy/scope tooling, 26 lib files); final hardening in flight in `.worktrees/remote-agent-seats` — P3 below rides on landed machinery, not future work |
| `2026-08-03-software-factory-adoption.md` + factory execution plan + SSSF v3 | Reliability/observability contracts — necessary, but zero insulation: dispatches still run in `~/wt-factory` on the laptop |
| harness confine (bwrap tmpjail + `confine-*.scope`, landed 2026-08-05) | Babysitter class: bounds mistakes on a shared kernel; does not stop load/IO/crash takeover; keep as defense in depth on the boxes |
| ca.sh transient-unit caps (in flight, `.worktrees/remote-agent-seats`) | Same babysitter class when the dispatch host is the laptop: MemoryMax/CPUQuota/TasksMax on a shared kernel; correct as depth once dispatches run on the boxes |
| `2026-08-05-buildbox-scratch-gc.md` | Sweeper class: cleans mess after the fact; I8 ephemerality makes the swept mess impossible instead — retire once runs are ephemeral |

Gap: nothing moves the harness execution plane off the laptop. That is the restructuring.

## Target architecture

Laptop keeps: deck web UI, systray, collector, terminal attach (ssh+tmux), thin submit clients (`runplan`, `factory-dispatch`, `claudex --seat`), the ONE interactive main session.

Buildboxes keep everything else:

1. **harnessd on the boxes.** System service under `agent-run.slice` (bounded like `agent-seat.slice`: MemoryMax, CPUQuota, CPUWeight=1, TasksMax, OOMPolicy=kill). Laptop `runplan` becomes submit/attach over the existing remote-build transport. Run worktrees live under `~/runs/<run-id>` on the box (outside remote-build GC roots, same rule as `~/seats`).
2. **Factory/harness seats = seat primitives.** Every dispatch (cursor, codex, claude wrapper) starts inside `agent-run-<run-id>.scope` under `agent-run.slice`. Reuse Phase 2 machinery: scope creation, cgroup readback verification, confine bwrap INSIDE the scope as depth, kill by unit name.
3. **Fail-closed local-spawn deny.** `ca.sh`, `cdx`, `factory-dispatch.sh`, harness dispatch refuse to start a seat when hostname is the laptop — exit 97 naming the remote path, exactly the proven `install-headless-guard` pattern. An agent that "forgets" cannot comply wrongly: the naive form fails. Not a monitor — an inability.
4. **One kill switch.** `deckctl agents` lists every `agent-*`/`confine-*`/`v2-*` scope across laptop+boxes with cgroup CPU/mem and owning run/seat id. `deckctl agents stop --all | --host X | --run R | --seat S` = `systemctl stop` on the named scopes, over SSH, independent of any agent's cooperation; harnessd stop included (`--all` stops the service before the scopes so nothing respawns). Ultimate fallback: reboot the box; laptop keeps working.
5. **Observability where the user already looks.** Collector gains a scope-inventory feed per host; deck page + systray badge show `local: 0 | debian1: N | debian2: M`. `local > 0` renders red — the invariant is visible at a glance. Edge-triggered only.

## Fail-stopped invariants — the box cannot become the laptop nightmare

User objection (2026-08-06, registry C208): a buildbox with full disk, 100% memory, or an overheating loop, unattended and unreachable, is the SAME nightmare minus the power button. Placement alone is NOT the answer. These invariants are — each enforced by kernel or hardware, never by a monitor watching agents:

| # | Invariant | Mechanism (deterministic, not a reaper) |
|---|-----------|------------------------------------------|
| I1 | SSH and systemd always respond, at any agent load | Aggregate agent slices capped strictly below capacity (e.g. MemoryMax 24G of 31G, CPUQuota 600% of 800%); `system.slice`/sshd get `MemoryMin` + high `CPUWeight`. Kernel arbitration, no daemon |
| I2 | Agents cannot fill the root filesystem | All agent-writable trees (`~/runs`, `~/seats`, scratch, caches) on a dedicated volume or project quota. Runaway write hits quota → that run's write fails → run fails; box healthy, SSH fine |
| I3 | No run outlives its budget | `RuntimeMaxSec=` on every run scope: the kernel kills the cgroup at the deadline. A loop dies by construction — nothing needs to notice it |
| I4 | A hard-wedged box reboots itself | systemd hardware watchdog (`RuntimeWatchdogSec=`, softdog fallback): kernel fails to pet → automatic reboot, no human touch required |
| I5 | Reboot lands in a safe state, never back in the loop | Boot-to-drain: after any reboot harnessd starts with runs PARKED (no auto-resume); run scopes are `Restart=no` and their budget is spent. One edge-triggered notification names the box and the parked runs |
| I6 | The user can always cut power from anywhere | Remote power-cycle path per box (smart plug / Wake-on-LAN pair, or IPMI where hardware allows) reachable from laptop and phone — the power-button equivalent, without sacrificing the laptop |
| I7 | Thermal ceiling is the CPU quota | Sustained draw bounded by I1's CPUQuota; if a box proves thermally marginal, the quota drops — a config value, not a fan-watching daemon |
| I8 | Mess cannot outlive the run | CI-runner semantics: every run executes in an ephemeral workspace (rootless podman `--rm`, or btrfs subvolume deleted by unit teardown) whose writable layer is destroyed with the run scope — finish, deadline, or kill alike. The ONLY persistence channels are declared outputs: the run's git branch (pushed) and a quota-bounded artifacts dir. Agents WILL make mess and will NOT clean it up — so nothing they leave behind has anywhere persistent to live. No GC sweeper exists because there is nothing to sweep |

I8 corollary — **salvage-push on teardown**: unsaved agent WIP MUST NOT depend on agent diligence. The run lifecycle (deterministic teardown code, executed after the agent process is dead and before the workspace is destroyed) runs `git add -A && git commit && git push` to `wip/<run-id>` whenever the tree is dirty. Everything disappears from the box; nothing disappears from git. In-run checkpoints stay the harness engine's job (it already commits+pushes at stage boundaries), so worst-case loss is bounded by one stage, and even a crashed run's half-finished tree is inspectable and resumable from the pushed branch.

Distinction that governs every future addition here: a *limit* is enforced by the kernel and cannot be exceeded; a *reaper* observes and reacts and can miss. Only limits count toward these invariants; reapers are telemetry.

Provisioning for I1/I2/I4 is part of P3 box setup (doctor-verified readback, like the seat slice); I5 is harnessd boot semantics; I6 is one-time hardware setup per box.

## Isolation layer decision — rootless podman

Threat model: agents are klutzes, not kernel attackers — they loop, eat RAM, fill disks, leave mess. Containment needs cgroup ceilings, namespace isolation, and an ephemeral writable world. Constraint: boxes are memory/disk-tight; cold start must be cheap.

**Decision: rootless podman `--rm` per run**, cgroup-parented under `agent-run.slice`.

| Layer | Mem overhead | Disk | Cold start | Verdict |
|-------|-------------|------|-----------|---------|
| rootless podman `--rm` | ~0 (just the processes) | shared image layers (~1–2 GiB toolchain image, once); per-run overlay = only what the run writes | <1 s | **CHOSEN** |
| full VM (KVM/qemu) | GiB reserved per VM | image per VM | 10 s–min | overkill for accident containment on 31 GiB boxes |
| firecracker/kata micro-VM | ~100–200 MiB/VM | kernel+rootfs management | ~1 s | only if the shared kernel becomes untrusted — see upgrade path |
| systemd-nspawn | ~0 | manual rootfs trees | <1 s | podman minus OCI images/`--rm` ergonomics; no advantage |
| bwrap (current confine) | ~0 | shares host fs | instant | no ephemerality, no image; demote to depth inside the container if still useful |
| k8s / nomad | control plane eats the box | — | — | rejected (standing decision) |

Why podman is sufficient, not optimistic:

1. **Slice above, container below.** The run container's cgroup lives INSIDE `agent-run.slice`, so I1's kernel arbitration bounds even a fully wild container; sshd and the box survive regardless. Podman provides workspace isolation + ephemerality (I8); the slice provides box unkillability (I1). Both kernel-enforced, zero daemons.
2. **Upgrade path is config, not re-architecture.** Same OCI image + same `podman run` switch to the kata runtime (micro-VM per container) via a runtime flag if the threat model ever escalates. Nothing is thrown away.
3. **Userns replaces per-run UID provisioning.** `--userns=auto` gives each run an unprivileged UID range with no `useradd` machinery; interactive seats keep the Phase 2 `ods-*` design (they need persistent checkouts + tmux attach).

Run shape: base toolchain image (node, bun, pnpm, git, agent CLIs) built once per box and layer-cached; `podman run --rm --userns=auto --memory/--pids-limit/--cpus <caps> --network <per-run netns>` with the workspace as a quota-bounded volume; persistence only via pushed branch + artifacts mount (I8).

## Phasing (each exit measurable)

- **P2 (exists, land first):** agent-seats-on-buildboxes — builds slice/scope/UID/netns/stop/doctor primitives. Exit: its Task 5 acceptance.
- **P3 — harness plane migration:** harnessd + factory dispatch target boxes; run worktrees remote; laptop submit/attach only. Exit: a full factory phase runs green with zero non-interactive model/build processes on the laptop (`deckctl agents` shows local: 0 during the run).
- **P4 — deny + kill + eyes:** local-spawn deny in the four dispatchers; `deckctl agents` + stop; systray/deck inventory. Exit: naive local dispatch exits 97; `deckctl agents stop --all` empties both boxes in one command; badge shows the invariant.
- **P5 — demote babysitters:** laptop mem-guard agent-process alerts and laptop tmpjail become box-side telemetry; delete laptop-side special cases that exist only to survive local agent load. Exit: complaint registry machine-takeover categories flat for 30 days.

## Non-goals

- No laptop VM/podman compute jail — dominated by placement (shared kernel, still consumes the user's RAM/CPU).
- No cloud execution — buildboxes are owned, free, and inside the trust boundary.
- No change to interactive main-session UX; terminal attach stays instant via tmux.