# Self-improving incident automation — owner-endorsed design

audience: AI coding agents first. Source: owner, verbatim, 2026-08-16 (session 6c0d0325).
Board row: #manual-1314d953a1e69e99. Implementer: codex
(`-m gpt-5.6-terra -c model_reasoning_effort=medium`); tiers 1–2 buildable now.
Tier 3's ONLY gate is the technical condition the owner actually named, verbatim: "the
agent needs to launch locally, at least for now, until we complete all the overdeck ctl
toolkit." A named lifting condition AUTO-LIFTS the moment it is verifiably met (owner
doctrine, 2026-08-16) — build tier 3 the day the toolkit is complete, no re-ask.
NEVER gate development the owner asked for on owner clearance; the only clearance gate
in this system is reboots. (An earlier revision of this spec added a "deny-gate
clearance" condition here — that was an agent generalization from the pi_spawn:remote
hold, which the owner cleared with the 2026-08-16 GO. It was never an owner statement
about tier 3 and is removed.)

**Owner ruling (this conversation):** escalating incidents to the owner is pointless —
they'd just relay back to an agent. Automate that hop. Build an "emergency seat agent"
system, launched **locally only** for now (owner's named condition above). Start with
**deployment incidents** as the first class.

## The hierarchy (three tiers)

1. **Fire consumer (standing unit, script-sized)** — edge-triggered on new
   `agent-incident` rows landing in state `asked` on the requests board (a
   `.path`/`.timer` watching the requests store, shaped like the existing deploy drain).
   Does one thing: exact-key match the fire signature (`gh-runner:<name>:offline`,
   `deploy-clone:*-dirty`, etc.) against the playbook table, then either run the script
   or launch the agent. Every outcome written back to the row — a dead consumer must
   leave the row in `asked`, never silently consumed.

2. **Playbook table (deterministic scripts, keyed by signature)** — script hit =
   seconds, zero tokens. The runner-revival (`systemctl --user start` + verify listener +
   GH online status + ship with proof) and clone-equivalence-restore (diff dirty files vs
   `origin/main`; byte-identical → lossless restore + redeploy; divergent → escalate,
   never discard) procedures from this session are the first two playbooks, already
   field-tested by hand.

3. **Emergency seat agent (spawn-on-fire, NOT resident)** — launched locally when no
   playbook matches. One seat harness with **per-type briefs and type-scoped toolsets**
   (deploy incidents get deploy powers only: deploy-local, equivalence-check, restore,
   redeploy — never reboots, kill-by-name, or clone discards), not N standing agents
   idle-burning quota.

## The self-improving mechanism (the owner's key addition)

**The agent is not the remediation — it is the playbook compiler.** Its standing
instructions: fix the live incident, then identify the failure class and **write the
deterministic script that makes calling an agent unnecessary next time**. Each signature
gets an agent visit once; every later crossing runs the generated script for free. The
registry grows itself.

Constraints baked into the agent's instructions (from today's field evidence):

- **Script encodes the decision boundary, not just the happy path** — the fail-closed arm
  (escalate/stop) is written first; a success-only script will eventually discard
  someone's work.
- **Script ships with a RED proof** — demonstrated refusing on a synthetic non-remediable
  case before the signature is registered as script-handled; otherwise the table
  accumulates untested automation (fake-green class).
- **Scripts land through the normal path** — repo module, landed, deployed; never dropped
  loose on a box or in `~/.claude`.
- **"Not scriptable" is a valid output** — genuinely judgment-dependent classes (e.g.
  divergent-clone: depends on whose work it is) get marked agent-permanent in the
  registry rather than forced into a brittle script.

## Lifecycle announcements (owner addition, 2026-08-16)

Owned by the CONSUMER, not the agent, so they survive agent death:

1. **On spawn:** one Telegram message via botmaster — which signature fired, which row,
   what the agent is being asked to do (owner language, one message per spawn).
2. **On close:** one Telegram message announcing the case is closed and deploying
   continues, with the agent's full transcript attached as a file:
   `botmaster --attachment <transcript-path> "<close text>"` (the outbound-attachment
   lane shipped 2026-08-16). The consumer reads the transcript from the seat's
   transcript path after the agent exits — an agent that died mid-run still gets its
   partial transcript attached, with the close text saying so honestly.
3. No progress-ping stream in between — the spawn and close messages are the two
   crossings; anything else is the agent's own escalation path (which is already
   bounded to one message).

## Safety floor (unchanged from doctrine)

One remediation attempt per crossing (same failure twice = stop and report, never retry
loops); exact-service scope; outcome always written back (`shipped` with proof, or
`asked` with escalation noted).

## Standing state to know

- Deployment class has a complete failure corpus from 2026-08-16: `deploy-clone-dirty`,
  `tool-shims-stale`, `sandbox-image-drift`, `reset-failed`, `smoke-failed-rolled-back` —
  each with a mechanical check, bounded remediation, provable outcome.
- The edge-triggered deploy-refusal notification (landed `aba9b1b8f`) is the consumer's
  clean wake signal for this class.
- Deploy incidents are workstation-side (clone, queue, shims, state all on the laptop),
  so local-first launch costs nothing; buildbox-side classes stay agent-over-SSH until
  the ctl toolkit is complete (the owner's named condition — auto-lifts when met).