# Containerized agent execution as the default

audience: AI coding agents first. Contract-level — seams and decisions, no implementation bodies.
Task #3. Precedes #96 (k3s).

Revised 2026-08-08 after an adversarial codex review that read the referenced code. Every claim
below about existing behavior is quoted from the tree; corrections from that review are folded in.

## Why

A fork bomb in an agent process took the owner's laptop down and cost a full working day. Agent
processes run directly on the host, so a runaway agent reaches the whole machine. Containment must
move from "cgroup limits on a host process" to "the agent process runs inside a container that is
destroyed atomically".

Second driver: #96 selects k3s as the single cluster scheduler. k3s schedules pods. The unit this
spec produces — one agent session, one container, declared resources — is that pod. Build it so it
is portable to k3s, not podman-only.

## Owner decisions (settled 2026-08-08 — do NOT relitigate)

1. **The whole agent CLI session runs inside the container**, not just its builds. A fork bomb dies
   with the container.
2. **Dispatched runs first; the owner's interactive terminal sessions later.** Bugs must be found by
   runs that are cheap to relaunch.
3. **Credentials arrive from the host, never baked into an image.** See "Credentials" — read-only is
   the *source*, not the mount the runtime writes to.
4. **The `-unsafe` hatches are the mandatory failsafe.** A containerization bug must never leave the
   owner unable to start an agent.

## What already exists — extend it, never build a second one

| Path | What it already does |
|---|---|
| `modules/sandbox/host/bin/sandbox-run` | One ephemeral rootless container: `--cap-drop=ALL`, `--security-opt=no-new-privileges`, `--userns=keep-id`, `--pids-limit`, `--memory`/`--memory-swap`, `--cpus`, `--cpu-shares`, `--rm`. **Creates** the workspace (`mkdir -p "$WORKSPACE"`) and mounts the whole root (`--volume "$SANDBOX_ROOT:/sandbox:rw"`). |
| `modules/workstation/claude/bin/agent-sandbox` (deployed to `~/.claude/bin/`) | **The launcher already exists.** Picks a registry host with the `agent-seat` role, ssh's to it, invokes `sandbox-run` with `--id/--memory/--pids/--cpus/--tty`, passes a command after `--`, rsyncs toolgap state back, returns the remote exit code. |
| `modules/sandbox/image/Containerfile` | Digest-pinned debian + node/pnpm/bun/uv/gh/rust/python, `agent` uid 1000, `HOME=/sandbox/home`, tini. `COPY bin/ /usr/local/bin/` only — **no agent CLI is installed**. |
| `modules/sandbox/host/lib/receiver.mjs` | Forced-command ssh receiver for the E2E pair; host identity pinned to `/etc/machine-id`; path containment under `$HOME/sandbox/workspaces`. |
| `modules/systray/remote_dispatch.py` | Placement claims, rsync mirror, remote git mirror under `$HOME/cdx-offload/git-common`, ssh exec. |
| `modules/systray/command_router.py` | The dispatch *caller*. `_run_captured()` is the only path into `remote_dispatch`. |
| `~/.claude/bin/_agent-unsafe.sh` | Shared body for `cld-unsafe`/`cdx-unsafe`/`ca-unsafe`/`opencode-unsafe`/`kiro-cli-unsafe`/`agy-unsafe`. Host execution in `unsafe.slice`. Rejects agent markers, containers, and a missing TTY *before* launching. |

**This work is wiring, not greenfield.** The single largest correction to an earlier draft: there is
no new launcher. `agent-sandbox` is extended.

## Architecture

```dot
digraph {
  rankdir=LR;
  router  [label="command_router.py\n_run_captured (+ Claude, Cursor callers)"];
  dispatch[label="remote_dispatch.py\nplacement, rsync, git mirror"];
  launch  [label="agent-sandbox (EXTEND)\nhost pick, ssh, exit code"];
  runner  [label="sandbox-run (EXTEND)\npodman flags, caps, limits"];
  pod     [label="container = one agent session"];
  router -> dispatch -> launch -> runner -> pod;
}
```

One container per agent session: the CLI process, its builds, its tests. `--rm`, so destroying it
reclaims everything the session spawned.

The `-unsafe` hatches are **not** on this graph. They are a separate, human-only entry point (see
"Failsafe").

## Scope of "dispatch" — name every caller

`remote_dispatch` is reached only via `command_router.py::_run_captured()`, which is enabled only by
`CodexAdapter.capture_log_path()`, and only for `exec`. Claude inherits `capture_log_path() -> None`;
cursor-agent goes through the harness/remote-seat path. **Changing `remote_dispatch.py` alone
containerizes `cdx exec` and nothing else.**

The implementation MUST enumerate every dispatch caller in `command_router.py` and the harness
remote-seat path, and state per caller: containerized now, or explicitly out of scope with a reason.
A silent "codex only" outcome is a failed implementation of this spec.

## Fail-closed — the existing fail-open must die

`command_router.py` currently prints `RUNNING ON THE WORKSTATION` and proceeds on the host when
`open_session()` fails, unless `CDX_OFFLOAD_STRICT=1`. **That fallback is the exact failure this
work exists to prevent.** Strict becomes the default: an unavailable node, an unbuilt image, an
unresolvable credential, or a failed container start aborts with a distinct non-zero exit and a
message naming the cause. There is no env var an agent can set that turns host execution back on.

Rollback is by reverting the change or by a human running an `-unsafe` hatch — not by a variable.

## Credentials — read-only source, ephemeral writable copy

Measured on this machine (all `600`):

| Runtime | Credential file | Auth kind |
|---|---|---|
| codex | `~/.codex/auth.json` | ChatGPT token |
| claude | `~/.claude/.credentials.json` | OAuth |
| cursor-agent | `~/.config/cursor/auth.json` | OAuth (resolved as `XDG_CONFIG_HOME \|\| ~/.config` + `/auth.json`) |

The comment in `modules/harness/wrappers/ca.sh` claiming cursor auth lives in `~/.cursor` is wrong —
`~/.cursor/cli-config.json` is configuration. Fix that comment as part of this work.

**All three runtimes WRITE these files on OAuth refresh** (cursor's installed code calls
`writeFile(authFilePath, …)`). A bare `:ro` mount at the live pathname therefore breaks
authentication the first time a token refreshes.

Contract:
- Host credential file → mounted `:ro` into a staging path under `/sandbox-secrets/`.
- Container start copies staging → the runtime's expected pathname inside the container's own
  writable `HOME`. The copy is ephemeral and dies with the container.
- **Refreshed credentials are discarded, not written back to the host.** A container must never be
  able to modify host auth state. Consequence to accept and state in the implementation: a refresh
  that happens only inside a container is lost, and the host copy remains authoritative.
- Table-driven per runtime (one place adds a runtime), fail-closed on an unresolvable path.
- Secrets never enter an image layer, never an env var visible in `podman inspect`, never a log line.

**Account routing is currently dropped at the ssh boundary.** `remote_command()` forwards only
`OD_LOCAL_DISPATCH_OK=1 OD_REMOTE_EXEC=1`, so a remote codex run uses the *remote* user's default
`~/.codex/auth.json`, not the locally selected account. The launcher MUST carry the resolved
credential **identity/path**, never token values, from `open_session()` through to the container.

## Workspace, git mirror, and path mapping

Three existing constraints collide here. All three must be satisfied.

1. `sandbox-run` mounts `$SANDBOX_ROOT:/sandbox:rw` — the whole shared root, all workspaces and the
   shared sandbox home. The spec's containment claim is only true once a **permitted-root +
   `realpath` symlink-safe validation seam** exists. Pin the permitted remote root explicitly; a
   workspace resolving outside it aborts.
2. `remote_dispatch` rewrites a linked worktree's `.git` to an absolute remote path under
   `$HOME/cdx-offload/git-common` (`printf "gitdir: %s\n"`). Mounting only the worktree leaves that
   gitdir outside the container and **git breaks again** — the exact defect #94 just fixed. The
   launcher MUST mount `GitMirror.remote_common` and `remote_git_dir` at the **same absolute paths**
   inside the container, read-write.
3. The E2E channel requires the workspace to sit under `/sandbox/workspaces` in the container and
   the corresponding `$HOME/sandbox/workspaces` on the host (`e2e-remote` + `receiver.mjs` enforce
   this pairing). A tree mirrored under `$HOME/cdx-offload` breaks that mapping. Either place
   dispatched workspaces under the workspaces root, or extend the receiver's mapping — **decide and
   pin one**; do not leave both live.

## systemd-run — scoped precisely, not blanket

Rootless podman needs setuid-root `newuidmap`/`newgidmap`. Under `NoNewPrivs=1` setuid cannot
elevate, so podman fails:

```
Error: fatal error, invalid internal status, unable to create a new pause process:
cannot re-exec process to join the existing user namespace
```

Measured: `podman images` fails in an agent shell, succeeds via
`systemd-run --user --pipe --quiet --wait -- podman images`. `podman system migrate` is NOT a fix.
Each failed attempt leaks a `podman-pause-*.scope`.

**`NoNewPrivs` does NOT cross ssh** — sshd starts a fresh process tree on the node. The dispatched
path therefore does not need the re-exec, and the existing denial/ceiling proofs already ssh in and
call podman directly.

Contract: re-exec through `systemd-run --user` **only when the launcher's own
`/proc/self/status` reports `NoNewPrivs: 1`**. Condition it on the measurement, never on a
hardcoded assumption about where the code runs. This matters for the later interactive phase
(decision 2), where the launcher *is* inside an agent tree.

When the re-exec is taken, pin the handoff explicitly — `deck-sudo` and `_agent-unsafe.sh` each
already encode half of it:
- `--expand-environment=no` (or equivalent escaping) — `systemd-run` expands `$`.
- explicit `PATH` and `HOME`; the manager environment is not the caller's.
- `--wait --pipe --collect`, signal propagation, and exact exit-code mapping (a `systemd-run`
  failure must be distinguishable from a non-zero exit of the contained process).
- recursion detection: a launcher already inside its own transient scope must not re-exec again.

**User-manager preflight.** The re-exec needs a live user manager. `sandbox-provision` touches only
the target home; `modules/buildbox/lib/buildbox-checks.sh` is what runs `loginctl enable-linger`.
The launcher MUST preflight `Linger=yes`, `XDG_RUNTIME_DIR=/run/user/<uid>`, and `/run/user/<uid>/bus`
before relying on `systemd-run`, and fail closed with a message naming `loginctl enable-linger` as
the remedy. Verified on the workstation: manager `loaded/active`, `Linger=yes`, bus present. The
three registry nodes are NOT yet attested — attest them as part of implementation.

## The agent CLI inside the image

The image installs no agent CLI. Pin one of:
- baked into the image (reproducible, but a pinned CLI version per image rebuild), or
- bind-mounted from the host **with every dependency it needs**, or
- provisioned into the sandbox root at container start.

State the choice and why. Bind-mounting a node-based CLI without its runtime/node_modules is the
predictable failure — verify by running the real CLI, not by asserting the mount exists.

## Failsafe — the `-unsafe` hatches

They stay exactly as they are, and are reached **only by a human at a TTY**. `_agent-unsafe.sh`
already rejects agent markers, container environments, and a missing TTY before launching anything.

**An agent-settable environment variable MUST NOT route to a hatch.** Any design in which a
containerization failure automatically falls back to an uncontained host run is rejected — that is
the fork bomb path. The hatch is a human typing `cdx-unsafe`/`cld-unsafe`/`ca-unsafe`.

Tests for the hatches use the existing stub-runtime + real-PTY pattern.

## Ledger and resource declarations

Neither seam exists yet:
- `~/.claude/lib/agent-session-ledger.sh` has no containment-mode or resource fields, and
  `cdx exec` reserves stdout for the bare log path. Pin the schema fields, the writer, and the
  output channel — the launcher cannot simply print JSON to stdout.
- `--rm` deletes the container before an OOM/pids kill can be attributed. Promising "surface the
  real reason" requires a **cidfile + `podman events`/`inspect` capture before removal**. Specify it
  or drop the promise; do not ship a bare non-zero dressed as an explanation.

`sandbox-run` defaults today: memory 12g, pids 512, cpus 4, cpu-shares 256. Those numbers must be
**measured against a real containerized agent run** (peak RSS, peak CPU, peak pid count) and the
defaults set from the measurement, with the measurement stated. #96 will schedule pods on them.

## Error handling

| Condition | Behaviour |
|---|---|
| node unreachable / image absent / container start fails | abort, distinct exit code; **never** host fallback |
| `systemd-run` needed but no user manager / no bus / `Linger=no` | abort naming `loginctl enable-linger` |
| credential path unresolvable for the runtime | abort before starting an agent that would fail mid-task |
| workspace resolves outside the permitted root (post-`realpath`) | abort |
| git mirror paths not mountable at the same absolute path | abort — a run whose git is broken is worse than no run (#94) |
| container OOM / pids kill | attribute via cidfile + inspect captured before removal |

## Testing

- Re-exec is taken **iff** `/proc/self/status` says `NoNewPrivs: 1` — both branches tested.
- Fail-closed on each error row above, one test per row asserting the distinct exit code.
- No host-fallback path exists: assert that a forced `open_session()` failure aborts, and that no
  env var restores workstation execution.
- Credential staging: mounted `:ro`, copied to a writable in-container path, a simulated refresh
  writes only inside the container, host file byte-identical after the run, and the secret appears
  in no log line.
- Git: a real `git commit` succeeds inside the container in a dispatched linked worktree (the #94
  regression, re-proved under containment).
- A real agent task runs end-to-end inside the container and its edits reach the host worktree. A
  design that has never executed an agent proves nothing; this test is mandatory.
- `-unsafe` hatches still start an agent (stub runtime + real PTY) when the containerized path is
  deliberately broken.
- `modules/sandbox/test/denial-proofs.sh` **must be updated, not merely kept green**: it asserts an
  exact mount count (`grep -cE " /sandbox | /sandbox-secrets/" … '3'`), which every credential mount
  breaks, and its "only workspace" claim is already false because `/sandbox` exposes all workspaces.
  Replace the count with an exact destination allowlist plus per-mount read-only assertions.
  `ceiling-proofs.sh` should stay compatible (it passes explicit memory/PID limits) — verify, do not
  assume.

Never run browser/E2E work locally; route through `~/.claude/bin/e2e-remote`. Development and
verification happen on a registry-reachable buildbox. Never touch node sshd/network/boot config.

## Out of scope

- Interactive owner sessions (decision 2 — a later pass; the `NoNewPrivs` branch is built now so
  that pass is wiring).
- Per-seat credentials (#64).
- k3s itself (#96) — this spec only guarantees the unit it will schedule.
- Network egress policy inside the container.

## Architecture decisions

- **Extend `agent-sandbox`; do NOT add a second launcher.** It already resolves the host from the
  registry, ssh's, invokes `sandbox-run`, and maps the exit code. A parallel `agent-sandbox-launch`
  would be exactly the duplication that produced two independent schedulers (#96).
- **Credential table separate from the launcher body.** It is the part that changes when a runtime
  is added (#64 adds two) and the part most likely to leak a secret if written ad hoc at a call site.
- **`sandbox-run` stays the podman mechanism only** — additive flags for extra mounts and an
  external workspace; no policy. It must remain replaceable by a k3s pod spec without touching a
  caller.
- **No new sandbox module.** `modules/sandbox` already encodes containment.
- **Rejected: `OD_SANDBOX=0` as a kill switch.** An agent-settable variable that restores
  uncontained host execution recreates the failure mode this work exists to remove. Rollback is a
  revert or a human-typed `-unsafe` hatch.
