audience: AI coding agents first.

# Factory supervision and resolution

## Result

Make every current-factory run durably owned, visibly terminal, resumable from verified checkpoints, and bounded in automatic repair. Route protected-machinery failures to an independent incident resolver. NEVER let a run edit machinery judging that run. NEVER let automatic recovery loop without finite attempt, time, token, and cost limits.

Plan slug: `factory-supervision-and-resolution`.

This contract targets `modules/harness/factory/**`, not archived harness v1/v2. Reuse concepts from `docs/specs/2026-08-07-failure-routing-and-escalation-design.md`; NEVER port its `modules/harness/v2/**` implementation.

## Existing truth — preserve, do not re-invent

Shipped current-factory pieces:

- `adw_modules/tracer.py` persists sessions, phases, attempts, process identity, host, model, account, tokens, cost, prompts, tool calls, gates, and diffs.
- `adw_modules/session.py` reopens an `adw_id`, records current PID/start ticks/argv, and closes traces on handled signals.
- `adw_modules/control.py` validates process identity for stop and implements manual `factory reconcile` for provably dead sessions.
- `runner.Run.phase()` records phase start, terminal state, error, and diff.
- `adw_plan_build_test_quality.py` has a three-pass quality/fix loop.
- `agents.execute()` persists a parsed agent envelope before rejecting non-success status.
- Incident state already includes `filed | dispatching | running | needs-attention | resolved`.
- `IncidentService` supports idempotent filing, dispatch-state writes, comments, and artifact-backed `incident.resolve`.
- Factory presets provide per-run seat/model/account composition through isolated `PI_CODING_AGENT_DIR`.

Partial, not complete:

- `--adw-id` joins trace identity and Pi context; it is NOT workflow resume. Relaunch executes the ADW from its entrypoint.
- Manual `factory reconcile` marks dead sessions failed; no durable supervisor runs it automatically.
- Incident dispatch records `starting`; no shipped coordinator launches, heartbeats, escalates, or ingests terminal resolver results.
- Incident selector WIP in `.worktrees/incident-dispatch-selector-options` establishes authoritative CLI/model/effort/account choices. Preserve and finish it; do not create another option authority.
- Incident type taxonomy contributes dispatch brief text only. No global or per-type execution defaults exist.

Absent:

- durable run owner lease and owner heartbeat;
- detached run supervisor;
- phase program counter and verified resume checkpoints;
- resolver chains in current factory;
- independent protected-machinery remediation;
- owner-gated task-contract rewrite flow;
- `/incidents` Defaults persistence and precedence.

Obsolete artifacts:

- `docs/specs/2026-08-07-failure-routing-and-escalation-design.md` and `docs/plans/2026-08-07-failure-routing-and-escalation*` target harness v2. Treat as design input only.
- v2 supervision branches and blocked integration branches are NOT WIP for this feature.

## Scope

MUST deliver:

1. Durable supervisor ownership for every mutating factory run.
2. Automatic orphan detection and explicit terminal state.
3. Same-`adw_id` resume from last verified phase checkpoint.
4. Configurable ordered resolver chains.
5. Finite automatic resolution budgets and progress circuit breakers.
6. Failure classification separating target-code, request-contract, provider, environment, and protected-machinery failures.
7. Automatic typed incident filing for protected-machinery failures.
8. Independent isolated resolver execution for approved incident types.
9. `/incidents` Defaults for global and per-type harness/model/effort/account resolver policy.
10. Owner-gated task-contract rewrite proposals with unified and side-by-side diffs.
11. Full trace/UI visibility for ownership, leases, checkpoints, attempts, decisions, budgets, incidents, and terminal outcomes.

MUST NOT:

- restore harness v1/v2;
- make collector polling or page loads execution triggers;
- let target run mutate `modules/harness/factory/**`, gate code, factory prompt code, auth routing, cluster control, incident coordinator, or data-safety machinery judging that run;
- weaken, skip, baseline, rewrite, or reinterpret a red gate merely to continue;
- alter tests solely to make a task pass;
- infer incident defaults from previous incidents;
- silently substitute harness, model, effort, account, or permission mode;
- start nested automatic resolver recursion;
- auto-rewrite task intent or acceptance criteria;
- report a dead, orphaned, budget-exhausted, or unresolved run as success.

## Invariants

### R-OWN-1 — one fenced owner generation

Every non-terminal run MUST have at most one active supervisor generation. Lease expiry during machine/service outage is allowed; no worker may advance state after its generation expires. Supervisor restart MUST reconcile each run within 30 seconds of service readiness and either acquire a higher generation or terminalize it. CLI, browser, launcher agent, terminal, and SSE connection are clients, never owners.

### R-TERM-1 — every run becomes visibly terminal

Every run MUST reach `succeeded | failed | cancelled | needs-attention`. Process disappearance, wrapper timeout, provider exit, supervisor restart, and machine reboot MUST NOT leave `running` indefinitely.

### R-RESUME-1 — resume only verified state

Resume MUST start from last checkpoint whose worktree commit/tree hash, phase output digest, and gate evidence still match. Mismatch MUST stop as `checkpoint-diverged`; NEVER guess or replay over unknown state.

### R-JUDGE-1 — run cannot edit its judge

Target resolver may edit only target-project paths authorized by request contract. Failure involving protected machinery MUST stop target resolution and create an independent incident.

### R-BOUND-1 — automatic recovery is finite

Each resolution epoch MUST carry non-null limits for attempts, wall time, input/output tokens, and cost. First exhausted limit stops automatic work. `0`, missing, negative, and unlimited values MUST fail policy validation.

### R-PROGRESS-1 — retry requires measurable progress

An automatic attempt may continue only when at least one authoritative signal improves: red gate set shrinks, failure class narrows, failure fingerprint changes after a relevant diff, or required artifact appears. Narrative confidence is not progress.

### R-INTENT-1 — semantic task changes require owner approval

Resolver may propose a complete revised task contract. It MUST NOT activate that contract. Only owner approval may replace request or acceptance criteria and start a new resolution epoch.

### R-AUTH-1 — billed identity is observed truth

Trace and incident metadata MUST record harness, model, effort, account, and credential identity from executing process/runtime authority. Requested values remain separately visible. Path or requested profile alone is not billed-account evidence.

## Architecture

```text
factory CLI / UI client
        |
        v
FactorySupervisor (durable user service; sole run owner)
        |
        +--> RunWorker (current ADW, one leased child)
        |       |
        |       +--> phase checkpoints + attempts + gates + diffs
        |       +--> target-code ResolutionController
        |
        +--> FailureClassifier
                |
                +--> target code/provider/environment -> bounded in-run resolver chain
                +--> request contract -> owner proposal decision
                +--> protected machinery -> typed incident outbox
                                                |
                                                v
                                  IncidentDispatchCoordinator
                                                |
                                                v
                                  isolated independent resolver
                                                |
                         repair + canaries + install + land + resume original run
```

Keep three deep boundaries:

1. `FactorySupervisor`: ownership, leases, worker lifecycle, reconciliation, resume admission.
2. `ResolutionController`: immutable policy snapshot, checkpoints, attempts, progress, budget, escalation.
3. `IncidentDispatchCoordinator`: incident claim/lease, independent workspace, resolver launch, heartbeat, result ingestion.

Do not add separate modules for trivial state formatting or one-call adapters.

## Durable run ownership

### Runtime model

Install one user service: `overdeck-factory-supervisor.service`. It MUST:

1. start with normal local deployment;
2. acquire one SQLite supervisor lease using compare-and-set semantics;
3. accept run submissions through a local Unix socket or equivalent mode-`0600` local IPC;
4. spawn each `RunWorker` as a foreground child in stable `overdeck-factory-run-<adw-id>.service` user unit;
5. require worker to persist generation-checked heartbeats, stdout/stderr artifact references, phase state, and terminal receipt directly;
6. renew run leases only while generation-matched worker heartbeats arrive;
7. reconcile expired leases, stable unit state, and dead process identities after service restart;
8. write terminal state or launch resolution; and
9. expose no public network listener.

Worker identity MUST include host boot ID, PID, process start ticks, argv digest, and stable unit name. A worker whose lease generation expires MUST stop before its next state transition. Supervisor restart may inspect stable unit state and durable receipts; it MUST NOT depend on inherited pipes or a parent `wait()` result.

`factory <adw> ...` becomes submit-and-attach:

1. validate request, repo, branch, config, preset, and account authority;
2. submit immutable run envelope;
3. receive `{ adwId, leaseGeneration, acceptedAt }` only after durable commit;
4. attach to trace stream;
5. on client disconnect, leave worker and supervisor ownership unchanged.

A compatibility foreground mode may exist only for deterministic tests. Production CLI MUST fail closed when supervisor is unavailable; it MUST NOT silently fall back to unsupervised execution.

### Persisted seams

Add additive SQLite migrations; NEVER edit shipped table definitions.

```text
RunState = queued | running | resolving | awaiting-owner | needs-attention | succeeded | failed | cancelled
RunStopReason = checkpoint-diverged | lease-conflict | worker-lost | budget-exhausted | policy-invalid | string

RunLease {
  adwId, generation, supervisorId, workerUnit, hostBootId,
  workerPid, workerStartTicks, workerArgvSha256,
  acquiredAt, heartbeatAt, expiresAt
}

RunCheckpointIndex {
  adwId, sequence, phaseId, workflowRevisionSha256,
  checkpointArtifactRef, status, createdAt
}

RunControlEvent {
  adwId, sequence, kind, payloadJson, createdAt
}
```

Lease generation MUST increase on every ownership change. Writers MUST include expected generation. Stale owners MUST receive a conflict and stop writing.

Migration projection maps shipped `sessions.status`: `success -> succeeded`, `fail -> failed`, `running -> running`. Keep legacy values readable; new writes use canonical `RunState`. SQLite state transition is authoritative. JSONL/console/UI emission occurs from committed transition records and is best-effort projection; never write projection first.

### Reconciliation ladder

Stop at first matching rung:

1. Terminal run: no action.
2. Fresh lease + matching live process identity: continue observation.
3. Expired lease + matching live worker: acquire next generation; send re-fence request through stable unit control channel; worker MUST pause, persist acknowledgment bound to new generation, then resume writes. No acknowledgment within bounded grace -> terminate verified old identity and resume from checkpoint in a new worker.
4. Expired lease + dead worker + valid checkpoint + resumable failure: start bounded resolution/resume.
5. Expired lease + dead worker + no valid checkpoint: mark `needs-attention`, persist diagnosis, file typed incident when protected machinery may be responsible.
6. Conflicting live owners or unverifiable process identity: mark `needs-attention`; NEVER kill by PID alone.

Run `factory reconcile` through this same service contract. Manual reconcile remains a control command, not a separate implementation.

## Checkpoints and resume

### Versioned workflow contract

Current imperative ADW functions cannot resume safely: downstream phases depend on Python locals and loop counters. Convert each resumable ADW to a versioned definition before enabling resume.

```text
WorkflowDefinitionV1 {
  workflowId, revisionSha256, phases: PhaseDefinitionV1[]
}

PhaseDefinitionV1 {
  phaseId, inputSchema, outputSchema, allowedTransitions,
  contractDependencies: TaskContractField[]
}

PhaseCheckpointV1 {
  adwId, workflowId, workflowRevisionSha256, phaseId, sequence,
  taskContractSha256, contractDependencyDigests,
  outputEnvelope, outputSchemaSha256, loopState, claimedPaths,
  artifactRefs, worktreeHead, worktreeTree, worktreePatchRef,
  gateEvidenceSha256, createdAt
}
```

Stable `phaseId` values form explicit program counter. Persist complete typed output envelopes and loop state; hashes alone are insufficient. Before downstream execution, hydrate typed inputs from checkpoint envelopes and validate schema digests.

Each phase declares exact `TaskContractV1` fields its output depends on. Checkpoint stores canonical digest for each declared field plus whole contract revision digest. After owner-approved revision, reuse a checkpoint only when every declared dependency digest still matches; invalidate first incompatible phase and all downstream checkpoints. Changed `fullRequest` normally invalidates request/plan onward. Changed acceptance, authorized paths, or gates invalidates every phase declaring those fields. If dependency declaration is missing or ambiguous, restart from immutable admission baseline; NEVER reuse old semantic output under new intent.

Pin workflow revision at admission. Changed workflow definition makes old checkpoint non-resumable unless an explicit tested migration converts every affected envelope and transition.

Checkpoint materialization MUST use an owned run-worktree snapshot/patch reference and exact expected tree. NEVER use broad `reset --hard`, `clean`, or stash. A run worktree containing unknown edits becomes `checkpoint-diverged`.

### Checkpoint write order

For every phase:

1. persist phase start with lease generation;
2. execute phase;
3. capture worktree HEAD and tree hash;
4. persist structured output and gate evidence;
5. persist checkpoint atomically with phase terminal state;
6. only then advance program counter.

A failed or interrupted phase has attempt evidence but no successful checkpoint.

### Resume contract

```text
resume_run(adw_id, checkpoint_sequence, resolution_epoch) -> RunResumeReceipt
```

Resume MUST:

- reuse same `adw_id`;
- preserve previous attempts and agent conversations as immutable history;
- reload pinned task-contract revision, workflow-definition revision, preset snapshot, resolver policy snapshot, and account authority;
- verify checkpoint workflow/schema digests, typed output envelopes, loop state, claimed paths, artifact references, and worktree materialization;
- hydrate every downstream input from validated checkpoint output; no downstream phase may depend on an unreconstructed Python local;
- skip completed phases only when checkpoint verification passes;
- re-enter failed phase from last verified checkpoint;
- emit `run.resumed` with old/new lease generation and checkpoint sequence.

Resume MUST NOT recompute policy from mutable current defaults. Explicit owner-approved task-contract revision creates a new pinned policy snapshot and resolution epoch.

## Failure classification and trust boundary

```text
FailureClass =
  target-code
  | target-acceptance
  | provider-unavailable
  | provider-invalid-result
  | account-unavailable
  | environment
  | request-contract-defective
  | gate-defective
  | factory-defective
  | harness-defective
  | auth-defective
  | cluster-defective
  | data-safety
  | budget-exhausted
  | cancelled
```

Classifier MUST use structured evidence: phase owner, exception type, gate id, protected path, process exit, provider status, failure fingerprint, and baseline canary result. LLM may advise classification but MUST NOT override deterministic protected-path or data-safety classification.

Routing:

| Class | Route |
|---|---|
| `target-code`, `target-acceptance` | bounded in-run resolver chain |
| `provider-unavailable`, `account-unavailable` | registered fallback within same rung; no capability escalation |
| `provider-invalid-result` | next resolver rung if budget remains |
| `environment` | one dedicated environment-repair rung, then incident |
| `request-contract-defective` | owner-gated proposal flow |
| `gate-defective`, `factory-defective`, `harness-defective`, `auth-defective`, `cluster-defective`, `data-safety` | stop target resolver; typed independent incident |
| `budget-exhausted` | `needs-attention`; no automatic incident resolver recursion |
| `cancelled` | terminal cancelled |

Protected path registry MUST be centralized, versioned, tested, and visible in trace. It includes current factory engine, gates, prompt system, permissions, resolver policy, incident coordinator, account/auth routing, cluster control, deployment safety, and data-safety controls. A protected failure may require editing those paths, but only an independent incident resolver may do so.

## Bounded resolution controller

### Policy snapshot

```text
ResolverSeat {
  harness: string
  model: string
  effort: string
  account: string
}

ResolutionBudget {
  maxAttempts: positive integer
  maxWallSeconds: positive integer
  maxInputTokens: positive integer
  maxOutputTokens: positive integer
  maxCostUsd: positive number
  sameFingerprintLimit: positive integer
}

ResolutionPolicy {
  resolver: ResolverSeat[]
  environmentResolver?: ResolverSeat
  budget: ResolutionBudget
  unsafe: boolean
}
```

Resolver array order is weakest/cheapest to strongest. Every seat MUST validate against deployed capability and account authority before run admission. Pin resolved wrapper model and observed identity separately.

Target policy precedence, lowest to highest:

1. centrally enforced safety ceilings;
2. shipped `sssf.config.yaml` defaults;
3. project `.factory/sssf.config.yaml`;
4. named preset;
5. permitted per-run override;
6. immutable admitted policy snapshot.

Add `resolution.resolver[]`, `resolution.environment_resolver`, and `resolution.budget` to `SSSFConfig` validation. A higher layer may lower ceilings or select an allowed seat; it MUST NOT exceed central ceilings. Persist source revision and capability-manifest digest with resolved values.

Canonical execution field is `harness`; incident HTTP compatibility adapter maps it to existing `cli`. Canonical `effort` maps to existing `reasoningEffort`. Canonical permission is `unsafe:boolean`; capability adapter resolves it to registered permission mode. Do not fork these shapes across factory and incident policy.

Ship finite initial profiles:

| Profile | Attempts | Wall | Input tokens | Output tokens | Cost | Same fingerprint |
|---|---:|---:|---:|---:|---:|---:|
| `fast` | 2 | 1,200 s | 500,000 | 100,000 | USD 2 | 2 |
| `deep` | 3 | 2,700 s | 1,500,000 | 300,000 | USD 5 | 2 |
| protected incident | 3 | 3,600 s | 1,500,000 | 300,000 | USD 10 | 2 |

All shipped profiles set `unsafe:false`. Owner may enable `unsafe` only through explicit validated incident/run policy; escalation MUST NOT turn it on.

Initial resolver chain follows registered `builder` seats from `fast` then `deep`, with one stronger registered builder seat only when capability authority exposes it. Never use reviewer-only seats as coding resolvers. Installer resolves account from explicit deployed preset configuration and persists it; incident history is not authority. Missing valid account leaves policy visibly invalid and blocks auto-repair.

Automatic policy MUST inherit finite run/preset budgets when present. Preset or run override may lower limits. It MUST NOT increase a centrally enforced safety ceiling. One attempt per resolver rung is default; `maxAttempts` may truncate chain or permit explicitly configured repeat attempts. Truncation MUST be visible before launch.

Use a durable per-epoch budget ledger. Reserve estimated dispatch budget transactionally before launch; reconcile actual provider usage after return. Pass provider-supported token/output caps. Enforce wall time by terminating only verified child identity. Provider accounting may arrive after call completion; one call may overshoot reservation, but ledger MUST block every later dispatch once any actual ceiling is crossed. Record reservation, actual usage, and overshoot explicitly.

### Attempt algorithm

For each attempt:

1. restore last verified checkpoint;
2. pin attempt id, rung, remaining budget, and starting tree hash;
3. dispatch resolver with exact failure evidence and protected-path restrictions;
4. persist envelope even when resolver reports `status=fail`;
5. inspect actual diff, gates, failure set, and artifact output;
6. restore unauthorized paths immediately;
7. calculate progress;
8. green gates -> checkpoint and resume run;
9. progress -> next permitted attempt from improved tree;
10. no progress, regression, repeated fingerprint, or exhausted limit -> restore checkpoint and escalate or stop.

A resolver `status=fail` MUST NOT erase useful evidence or automatically abort remaining rungs. Deterministic gates and progress decide continuation.

### Progress and circuit breakers

Normalize failure fingerprint from failure class, phase, gate ids, normalized error signature, checkpoint tree, and relevant tool exit codes. Exclude timestamps, PIDs, random ids, and log ordering noise.

Stop or escalate when any holds:

- same normalized fingerprint reaches `sameFingerprintLimit` without gate-set reduction;
- attempt produces no relevant diff and no required artifact;
- red gate set grows;
- protected path is touched;
- worktree diverges from expected checkpoint;
- account/model identity differs from pinned authority;
- any attempt, wall, token, or cost limit is exhausted;
- resolver requests nested resolver/incident dispatch.

On red-set growth, discard attempt. On unchanged red set, discard unless a named required intermediate artifact appeared. On shrink, keep improved tree and create an intermediate resolution checkpoint.

Automatic resolution epochs are never unbounded. Owner proposal iterations are not automatic epochs: each owner action launches exactly one bounded proposal attempt.

## Independent protected-machinery remediation

### Execution ledger and Kanboard projection

Kanboard's multi-call metadata/column/comment updates cannot own exactly-once execution. Add local durable incident execution ledger:

```text
IncidentExecutionV1 {
  incidentId, dispatchId, leaseGeneration, leaseOwner,
  leaseAcquiredAt, leaseHeartbeatAt, leaseExpiresAt,
  source, policySnapshot, state, launchIntent,
  stableUnit, hostBootId, workerPid, workerStartTicks,
  workerArgvSha256, admissionReceipt, heartbeatAt, resultArtifactRef,
  createdAt, updatedAt
}

IncidentCheckpointV1 {
  incidentId, dispatchId, leaseGeneration, attempt,
  worktreeHead, worktreeTree, worktreePatchRef,
  policySnapshotSha256, budgetLedgerRef, failureEvidenceSha256,
  gateEvidenceSha256, canaryEvidenceSha256, createdAt
}

IncidentOutboxV1 {
  sequence, eventId, incidentId, dispatchId, leaseGeneration,
  projectionKind, payloadArtifactRef, payloadSha256, deliveredAt
}
```

Ledger is execution authority; Kanboard remains durable incident/user record and eventually consistent projection. Claim dispatch by compare-and-set before launch. Persist launch intent before process admission; persist admission receipt bound to stable unit, boot identity, process identity, and lease generation before reporting `running`. Project every transition through idempotent outbox records carrying replayable payload artifact and digest; reconcile partial Kanboard writes after restart. One lease generation may ingest terminal result exactly once.

Takeover ladder mirrors factory worker fencing: expired lease is detected from persisted owner/acquisition/heartbeat/expiry; valid live resolver must acknowledge higher generation before further writes; absent acknowledgment terminates verified identity and resumes only from latest validated `IncidentCheckpointV1`. Missing/diverged checkpoint stops `needs-attention`. A crash after launch but before admission receipt MUST reconcile stable unit identity before any new resolver starts.

`source="factory-supervisor"` is immutable and server-authored. Browser request cannot set it.

### Incident creation

FactorySupervisor MUST file one idempotent incident when protected failure is confirmed. Idempotency key:

```text
sha256(adwId + resolutionEpoch + failureClass + normalizedFingerprint)
```

Incident evidence MUST include:

- original `adw_id`, repo, branch/worktree identity, phase, checkpoint, and request revision;
- failure class and normalized fingerprint;
- exact failed gates and bounded log excerpts with trace references;
- requested and observed harness/model/effort/account/host;
- protected paths implicated;
- automatic attempts already consumed and budget outcome;
- resume command/control token reference, never credentials.

Do not paste secrets, full environment, auth files, or uncontrolled raw logs into Kanboard metadata. Store large evidence in factory trace storage and link by stable local trace id.

### Independent resolver

Incident resolver MUST:

1. run outside target ADW and outside target worktree;
2. create a dedicated repo worktree from current trusted base;
3. use incident Defaults resolved and pinned at dispatch time;
4. acquire one incident dispatch lease;
5. run ordered resolver seats under finite incident budget;
6. enforce protected scope for this incident type;
7. run immutable type-specific canaries plus full touched-module gates;
8. for local factory/harness infrastructure, install live and verify real entrypoint before landing;
9. land only after deterministic gates and canaries pass;
10. publish signed resolution artifact containing commit, deployed bytes/version, gates, canaries, and resume disposition;
11. resume original run from its last verified checkpoint;
12. keep incident open until resumed run crosses the original failed gate.

Independent resolver MAY edit broken gate/factory/harness/auth/cluster/data-safety machinery only when incident type authorizes those paths. It MUST NOT change incident coordinator, resolver policy, required canary definitions, or its own scope in the same attempt. A defect in those second-order controls stops at `needs-attention` for owner action; NEVER recurse.

If machinery canaries pass but original resumed gate remains red, continue within same incident epoch only when remaining budget and progress rules permit. Otherwise stop original run and incident at `needs-attention`; NEVER file another incident.

### Verified resolution artifact

Resolver returns evidence; coordinator verifies and signs final artifact with deployment-generated Ed25519 key held only by coordinator service. Public verification key and key id are server configuration; browser never receives private key.

```text
ResolutionArtifactV1 {
  version: "incident-resolution-artifact/v1"
  incidentId, dispatchId, leaseGeneration,
  commit, deployedVersion, deployedBytesSha256,
  authorizedScopeSha256, gateEvidenceSha256, canaryEvidenceSha256,
  signerKeyId, signedAt, resumeDisposition, signature
}
```

Signature covers canonical encoding of every field except `signature`. `incident.resolve` MUST verify schema, signature, incident/dispatch/generation binding, authorized scope, and deployed-byte evidence. Resolution remains provisional until original run crosses original failed gate; only then move Kanboard to Resolved. Key rotation keeps old public keys for artifact verification and never re-signs history.

### Narrow supersession of incidents v1

This design supersedes `docs/specs/2026-08-08-incidents-page-design.md` no-auto-retry/no-relaunch clauses only for typed factory-generated incidents carrying `source="factory-supervisor"` and a validated automatic-repair policy. Manual incidents retain existing one-attempt behavior unless an owner explicitly dispatches again. Collector polling, browser loads, SSE reconnects, and Kanboard observation remain non-authoritative and MUST NOT launch work.

## Incident Defaults

### Authority and persistence

Add a versioned server-owned settings document. Browser MUST use typed collector routes; never edit workstation files directly.

```text
IncidentResolverDefaultsV1 {
  version: "incident-resolver-defaults/v1"
  global: IncidentResolutionDefaults
  byType: Record<IncidentTypeId, PartialIncidentResolutionDefaults>
}

IncidentResolutionDefaults {
  resolver: ResolverSeat[]
  budget: ResolutionBudget
  unsafe: boolean
  autoRepair: boolean
}
```

Every resolver seat contains harness, model, effort, and account. UI presents those four fields together per rung. `unsafe` defaults false. `autoRepair` defaults false for manual/user-defined types.

Initial protected-type policy enables `autoRepair:true` for `gate-defective`, `factory-defective`, `harness-defective`, `auth-defective`, and `cluster-defective` after their immutable canaries ship. `data-safety` enables automatic diagnosis and non-destructive repair only; destructive migration, data deletion, restore, or irreversible mutation requires owner decision. Incident coordinator/policy/canary/signing failures always use `autoRepair:false` to prevent recursion.

Precedence, lowest to highest:

1. global defaults;
2. per-incident-type override;
3. explicit manual filing selection;
4. immutable policy pinned to an existing dispatch.

Explicit filing selection does not mutate defaults. Historical incidents never influence defaults. Removed/invalid capability makes affected default visibly invalid and blocks dispatch; NEVER silently substitute.

Routes:

```text
GET /incidents/defaults
PUT /incidents/defaults
```

`PUT` requires same-origin mutation protection, bearer auth, bounded body, runtime schema validation, optimistic revision/ETag, and atomic source-preserving persistence. Reject stale writes with `409 settings-conflict` and return current revision.

### UI

Add `Defaults` within `/incidents`, not general `/settings`. Compose existing shared controls and `DataTable`; do not create new primitives.

UI MUST support:

- global resolver chain rows with Harness, Model, Effort, Account;
- add/remove/reorder rung actions;
- global finite budget fields;
- per-type inherit/override controls;
- visible invalid/stale capability state;
- unsaved-change and save-error state;
- read-only resolved-policy preview for selected incident type;
- keyboard-complete editing and both themes.

Use authoritative `/incidents/options` capability data from incident selector work. Merge that WIP before implementing Defaults or rebase this work onto it. Do not duplicate manifest/account parsing.

## Owner-gated task-contract rewrites

### Canonical task contract

Admission MUST persist complete immutable contract revisions; truncated session request text is not contract authority.

```text
AcceptanceCriterionV1 {
  id: string
  statement: string
  verificationCommand?: string
  expectedOutcome: string
}

TaskContractV1 {
  version: "factory-task-contract/v1"
  fullRequest: string
  acceptanceCriteria: AcceptanceCriterionV1[]
  authorizedTargetPaths: string[]
  requiredGateIds: string[]
  baselineRevision: string
  repoIdentity: string
  worktreeIdentity: string
  workflowId: string
  policyRefs: string[]
}

TaskContractRevisionV1 {
  revision, contract: TaskContractV1, canonicalSha256, createdAt, createdBy
}
```

Gate IDs and authorized paths are trust-boundary fields. Resolver may propose changes but only owner approval can activate them. Persist every revision without truncation; trace may store a bounded preview plus artifact reference. Canonical encoding is UTF-8 RFC 8785 JSON Canonicalization Scheme over `TaskContractV1`; `canonicalSha256` is lowercase SHA-256 of those bytes and lives in revision envelope, never inside hashed contract.

### Proposal seam

When classifier returns `request-contract-defective`, automatic target resolution stops. Resolver may create:

```text
TaskContractProposal {
  proposalId, adwId, contractRevision, baseContractSha256,
  proposedContract: TaskContractV1, unifiedDiff, rationale,
  expectedImpact, createdAt, resolverIdentity
}
```

Proposal MUST contain complete replacement contract, not patch instructions. Server recomputes diff and digest; resolver-supplied diff is display evidence only.

```text
ProposalExecutionV1 {
  proposalId, iteration, ownerDecisionRevision, launchIntent,
  stableUnit, hostBootId, workerPid, workerStartTicks,
  admissionReceipt, state, resultProposalId, createdAt, updatedAt
}
```

`Reject with feedback` transaction MUST commit owner decision plus one idempotent proposal launch intent keyed by `{proposalId, iteration}`. Coordinator claims intent by compare-and-set, persists stable-unit admission receipt, and reconciles intent/admission/result after crash. Replaying same intent launches at most one active worker; terminal result binds exactly one next proposal. Browser/API thread never launches resolver directly.

Run state becomes `awaiting-owner`. Automatic budgets stop accruing while waiting. At admission, pin authorized owner principal/capability from authenticated local Overdeck session authority. Resolver, worker, supervisor, and coordinator identities MUST NOT satisfy owner decision authorization. Browser decision mutation requires authenticated owner, same-origin protection, optimistic proposal revision, and bounded body.

One durable decision offers exactly:

- `Approve and continue`;
- `Reject and fail run`;
- `Reject with feedback`.

Behavior:

- Approve: compare `baseContractSha256`; atomically persist owner principal, timestamp, proposal revision, base digest, decision, and new `TaskContractV1` revision; preserve old revision; clone prior pinned resolution policy into a new epoch with fresh counters; recompute phase dependency digests and resume same `adw_id` from last contract-compatible verified checkpoint, or immutable admission baseline when none survives. Mutable current defaults MUST NOT be recomputed.
- Reject and fail: atomically persist same decision evidence, mark run failed, cancel pending proposal, and launch no agent.
- Reject with feedback: atomically persist same decision evidence plus owner text; launch exactly one proposal call with non-null per-call wall, token, and cost limits; remain `awaiting-owner` when new proposal arrives. Owner may repeat without count limit because every iteration requires an explicit owner action. No automatic proposal loop exists.

### Diff UI

Render proposal through approved `DiffView` with:

- `Unified` and `Side by side` modes;
- before and after panes visible together;
- line numbers, additions, deletions, and unchanged context;
- formatted Markdown preview beside raw contract diff;
- sticky decision actions;
- resolver identity, model, effort, account, timing, and source failure;
- no fabricated summary.

Expose proposal from factory transcript always. Expose it from incident detail only when proposal has an explicit linked incident id; request-contract defects do not create incidents by default. Owner action must use optimistic proposal revision; stale actions return conflict and refresh current proposal.

## Observability

Every transition MUST write structured trace before UI projection:

- supervisor lease acquire/renew/expire/takeover;
- worker spawn/admission/exit;
- checkpoint write/verify/diverge;
- classification with deterministic evidence;
- policy snapshot and chain truncation;
- resolution attempt start/end, budgets before/after, model/account truth;
- failure fingerprint and progress decision;
- tree keep/restore and exact diff reference;
- incident file/dispatch/heartbeat/result;
- proposal create/approve/reject/feedback;
- resume admission and original-gate result;
- terminal outcome.

Factory transcript rows MUST be clickable and open existing Astryx drawers. Show commands, bounded stdout/stderr, files, diffs, gates, timing, host, requested/observed identity, account, and raw trace. `needs-attention` MUST show why automatic work stopped and the exact owner action available.

Desktop/systray notifications are edge-triggered only: notify on transition into `awaiting-owner`, `needs-attention`, or terminal failure. Never pollute repeated refreshes with duplicate notifications.

## Security and safety

- Validate every execution choice against deployed capability authority at submission and immediately before launch.
- Use argv arrays; incident/task text is data, never shell.
- Keep IPC local and authenticated by filesystem ownership or dedicated service credential.
- Redact secrets before trace persistence and incident filing.
- Use process PID plus start ticks/identity; never signal PID alone.
- Use compare-and-set leases and monotonic revisions for every coordinator write.
- Make worktree restore target explicit; never run broad clean/reset/stash operations.
- Keep original gates authoritative. Independent canaries may add evidence, never replace original gate rerun.
- Never allow resolver-generated edits to policy, scope, canary, or lease records.
- Data-safety incident repair requires dedicated immutable canaries and cannot auto-land destructive migrations.
- Owner cancellation wins at every boundary and prevents new dispatches.

## Testing

### Factory unit/integration

- launcher client exits while worker continues; supervisor retains ownership and terminalizes run;
- supervisor dies/restarts during phase; next lease generation reconciles exactly once;
- worker dies before first checkpoint -> visible `needs-attention`;
- worker dies after checkpoint -> same `adw_id` resumes at failed phase, completed phases do not rerun;
- checkpoint tree mismatch -> `checkpoint-diverged`, no resume;
- duplicate/stale lease writer rejected;
- resolver envelope `status=fail` with useful diff persists evidence and advances according to deterministic progress;
- same fingerprint, no diff, red-set growth, protected-path touch, and every budget dimension each stop correctly;
- red-set shrink keeps intermediate checkpoint;
- provider/account unavailability uses registered fallback without consuming capability rung;
- machinery failure files one idempotent incident and target resolver stops;
- independent resolver cannot edit its own policy/canaries/coordinator;
- repaired machinery passes canaries, installs live, resumes original run, and original gate turns green before incident resolves;
- second-order coordinator defect stops without nested incident recursion;
- fault injection after lease claim, worker admission, checkpoint commit, outbox enqueue/delivery, live install, artifact verification, and resume transition converges idempotently.

Run full mandatory suite:

```text
python3 -m pytest modules/harness/factory/tests/ -q
```

### Collector/incidents

- defaults GET/PUT round-trip, stale revision conflict, malformed policy, unknown type, invalid capability/account, source preservation, and atomic persistence;
- global/per-type/explicit/pinned precedence;
- factory incident idempotency and evidence redaction;
- one incident lease, heartbeat, terminal result, stale lease reconciliation;
- manual incident retains one-attempt behavior;
- factory automatic incident obeys pinned chain and finite budget;
- valid resolution artifact required before `incident.resolve`;
- incident remains open until original factory gate passes.

Run full collector suite with no warnings.

### Web

- Defaults global and per-type editing, inheritance, reorder, invalid capability, conflict, error retention, both themes, and keyboard operation;
- proposal unified/side-by-side rendering from real before/after contracts;
- approve, reject-fail, reject-feedback, stale decision conflict;
- transcript ownership/checkpoint/resolver/incident rows expose real trace fields;
- no fabricated running, resolved, budget, identity, or summary state.

Run:

```text
pnpm --filter @overdeck/deck-ui test
pnpm --filter @overdeck/deck-ui typecheck
pnpm --filter web test
pnpm --filter web typecheck
pnpm --filter web build
```

Browser/dev-server verification MUST run through `e2e-remote`.

### Deterministic end-to-end canary

Use provider-free fixtures. Prove:

1. submit run and disconnect launcher;
2. worker hits deterministic target-code failure;
3. rung 1 makes no progress, rung 2 fixes it;
4. run resumes and succeeds;
5. separate run hits deterministic gate-defective failure;
6. one incident files and independent resolver repairs fixture machinery;
7. live install verification passes;
8. original run resumes across original gate;
9. incident resolves;
10. task-contract defect displays a real diff and remains paused until owner action.

No exhaustion, OOM, fork, disk-fill, or stress test is permitted on workstation.

## Delivery decomposition

Do not implement this cross-trust-boundary design as one factory run.

1. **Producer track — trusted independent agent:** FactorySupervisor, current-factory workflow definitions/checkpoints/resume, resolver controller, failure classifier, protected-path registry, execution-ledger migrations, factory CLI. Factory agents cannot edit `modules/harness/factory/**`.
2. **Incident trust track — trusted independent agent:** incident dispatch coordinator, execution ledger/outbox, resolver policy enforcement, protected scope, immutable canaries, signing/verification, and factory-incident ingestion. This machinery judges incident repair and cannot be built by a run it launches.
3. **Incident app track — factory:** Defaults persistence/API, ordinary Kanboard projection/API changes, and non-authoritative settings surfaces. Start only after rebasing/landing incident selector WIP; MUST NOT edit coordinator/policy/canary/signing machinery.
4. **UI track — factory:** `/incidents` Defaults, proposal diff/decision surfaces, Astryx transcript projection. Prerequisites: approved `DiffView` implementation from Astryx worktree lands with unified + side-by-side modes, barrel export, gallery registration, accessibility, and both-theme tests; required backend seams land first.
5. **Integration track — trusted independent agent:** protected-machinery canaries, live install, original-run resume proof, end-to-end failure canary.

Each track gets its own implementation plan and acceptance receipt. Shared schema contracts from this design MUST be copied verbatim into dependent plans.

## Rollout

1. Land trace migrations and supervisor in observe-only mode; compare automatic reconciliation with manual `factory reconcile`.
2. Enable supervised submission; keep resolver disabled; prove launcher disconnect and reboot reconciliation.
3. Enable bounded target-code resolver for deterministic canary preset only.
4. Enable incident filing without auto-repair; verify idempotency and evidence.
5. Enable independent auto-repair per incident type, default off except explicitly approved factory machinery types.
6. Enable owner task-contract proposals.
7. Enable general presets only after canary and live UI show complete evidence.

Every stage has rollback to previous control path without schema rollback. Existing traces remain readable.

## Architecture decisions

- Keep durable supervisor separate from collector. Collector serves app data; supervisor owns run execution. Page/SSE observation never gains launch authority.
- Keep incident coordinator separate from factory target resolver. It crosses trust domain and may edit judge machinery.
- Reuse incident selector capability authority and factory preset account isolation. Do not build a third model/account registry.
- Reuse `DiffView` for task-contract proposals. No new diff primitive.
- Keep one resolver chain shape (`ResolverSeat[]`) across factory and incident defaults. Different policy snapshots and budgets preserve trust boundaries.
- Stamp policies and checkpoints. Never recompute mutable defaults on resume.
- Treat `status=fail` as evidence, not an unconditional controller abort.
- Permit unlimited owner-driven proposal revisions, but exactly one bounded resolver call per owner feedback action.
- Preserve manual incident one-attempt semantics. Automatic remediation applies only to validated factory-generated incident types.
- Reject daemonless periodic reconciliation as primary ownership. A cron/poll can detect eventually but cannot guarantee one owner or capture terminal child state.