# Incidents and Resolvers Domain

**Status:** Approved target-state baseline — amended by pre-run reuse audit  
**Failure primitives appear earlier; generic product activation:** I8.

## Target State

AWP treats recurring/unresolved software-delivery failures as durable `Incident` state with deterministic resolver recipes first, agentic resolution second, and verified fixes returning through normal change/review workflows.

```text
failure/health signal
  -> Incident
       -> known deterministic ResolutionRecipe?
            yes -> ResolutionAttempt -> verify
            no/still failing
              -> Resolver / agentic ResolutionAttempt
              -> proposed fix/change/script
              -> normal ChangeSet -> Review -> Verification
              -> verified resolution
              -> optional reusable ResolutionRecipe proposal
```

## Core Primitives

### Incident

Stable record of a significant lifecycle/operational failure, scope, severity, impact, related primitive IDs, detection source, status, timeline and resolution.

### Resolver

Role/configuration capable of diagnosing/remediating an Incident within policy. May be deterministic mechanical resolver or agent/model-backed resolver.

### ResolutionAttempt

Immutable attempt provenance: resolver/recipe/model/account/workspace/input state, actions, outputs, evidence, result.

### ResolutionRecipe

Reviewed reusable deterministic remediation procedure/script for a known class of failure. A generated or harvested recipe does not become trusted automatically; it passes normal qualification/review/verification before reusable execution.

## Existing Overdeck Recovery Knowledge

AWP must **not** begin resolver implementation by rewriting recovery logic from memory.

Before CI/deployment/self-healing resolver code is authored, inspect the relevant Overdeck sources under the binding capability-harvest plan, especially:

```text
deployment-untangling scripts
CI/CD helper/failure scripts
cleanup/retry/orphan recovery helpers
auth/config drift recovery helpers
other deterministic operational recipes proven by real use
```

Treat these as capability/knowledge sources, not trusted code by default.

Qualification flow:

```text
existing helper/script
  -> identify failure class + preconditions
  -> identify workstation/global-state assumptions
  -> isolate deterministic action
  -> map capabilities/credentials to AWP policy
  -> define idempotency/retry/rollback/reconciliation
  -> wrap/adapt/port as candidate ResolutionRecipe
  -> representative failure fixture
  -> Review + VerificationEvidence
  -> trusted reusable recipe
```

If an accepted FOSS/native mechanism deletes the old helper entirely, harvest its failure classification/tests/operational knowledge and use `replace-with-foss` rather than porting unnecessary code.

Do not migrate scripts that require broad workstation sudo, hidden mutable global configuration, shared checkouts or reusable publication credentials without removing those assumptions.

## Invariants From Day 1

- failure cannot disappear because a retry succeeded; incident/retry history remains observable where material;
- deterministic known resolver preferred before expensive/agentic resolver;
- existing proven deterministic recovery capability is audited before equivalent custom recipe code is written;
- resolver has only configured capabilities and never broadens its own authority;
- code/config changes produced by resolver use normal ChangeSet/Review/Verification path;
- production/destructive remediation observes Approval policy;
- repeated failure after resolution can reopen/link rather than duplicate context blindly;
- resolution evidence is explicit;
- successful one-off agent output does not automatically become a trusted recipe.

## Incident Sources

Factory/Agent workspace failure, CI blocker, deployment failure, cluster/resource issue, provider outage/drift, configuration/auth issue and self-healing detector can all create/link Incidents.

Not every transient retryable error becomes an Incident; domain policy defines escalation threshold/impact.

## Lifecycle

```text
open -> triaged -> resolving -> monitoring/verification -> resolved
 -> reopened
 -> cancelled/not-applicable
```

Blocked/waiting reason can be human approval, provider recovery, missing connection or unresolved root cause.

## Resolver Selection

Resolver selection prefers the least risky proven mechanism that can falsifiably resolve the incident:

```text
known safe deterministic recipe
  -> provider/native recovery action
  -> bounded diagnostic action
  -> agentic resolver
```

Selection records why a recipe/provider/agent was chosen, required authority, and verification expected after the attempt.

## Increment Realization

| Increment | Incident realization |
|---|---|
| I1 | explicit failure reasons/retries and durable WIP; significant failures can be recorded minimally; harvest I1-relevant cleanup/recovery knowledge. |
| I5 | CI resolver-specific behavior; qualify existing CI helpers before replacements. |
| I6 | deployment resolver-specific behavior; qualify deployment-untangling scripts before replacements. |
| I8 | generic Incident/Resolver/Recipe product and UI activates; consolidate proven recipes. |
| I9 | enterprise on-call/export/support integration if required. |

## Current Implementation State

Target spec; generic self-healing is not an I1 blocker. Legacy source paths are not invented here; the harvest audit records exact repositories/paths/commits and dispositions when inspected.

## Deferred Realization

Automated root-cause correlation/ML or large incident-management suites are not baseline. Integrate externally if later needed.

## Temporary Dogfood Behavior

Before I8, failures can surface as structured Needs Attention/run failure with explicit retry/resolution history. Do not create invisible shell-script fix paths that bypass future Incident semantics.

Vetted deterministic helpers may be invoked through a temporary adapter before rich Incident UI exists, but their attempt/result/evidence must still be attributable.

## Failure / Recovery

Resolver failure creates another ResolutionAttempt, not loss of Incident. Recipe execution is idempotent/reconcilable and records exactly what changed. A partially executed legacy-derived recipe must expose reconciliation/rollback rather than retrying blindly.

## UI Implications

Incident page shows impact, affected lifecycle, timeline, attempted resolutions, evidence, next action and reusable recipe history—not raw log dumps. Recipe origin may show `harvested`, `AWP-authored`, `provider-native`, or `agent-proposed` as provenance, not as trust level by itself.

## FOSS / Harvest Links

- `../../plans/AWP-OVERDECK-CAPABILITY-HARVEST-PLAN.md`
- `../architecture/platform-reuse.md`
- CI/deployment provider specs for mechanics under resolver actions.

## Decisions / ADRs

Self-healing authority remains bounded by Policy/Approval and normal code-review gates. Replacing a proven deterministic recovery path with custom generic machinery requires an explicit reuse/harvest disposition.