---
name: od-testing
description: Use when planning, running, fixing, optimizing, or reporting tests, checks, gates, CI verification, flaky or slow suites, test temporary directories, remote test execution, or pre-deployment verification in Overdeck.
---

# Overdeck testing

audience: AI coding agents first.

## Core rule

Run cheapest high-signal checks first. Run broad suites once, after focused + static checks pass. Preserve exact evidence; NEVER pay twice for unchanged verification.

## Feature acceptance: correctness only

Feature verification MUST exercise requested behavior through real owner user flow. Prove output and interaction correctness, not route, component, selector, process, or file existence.

1. Create or select authoritative data that reaches feature.
2. Enter feature through same navigation owner uses.
3. Exercise every requested interaction and visible state.
4. Assert requested outputs and forbidden/default-hidden outputs.
5. Verify installed runtime from `origin/main`.

```text
DO NOT: /agents returns 200 + page has <main> → feature passed
DO NOT: no clickable rows + unit tests pass → clicked-agent flow passed
TARGET: click real agent → story renders → groups/diff/diagnostics work → raw detail hidden by default
```

Smoke checks may diagnose availability. Smoke checks MUST NOT satisfy feature acceptance or completion. Missing data needed to enter user flow = failed/incomplete verification; create authoritative fixture through supported runtime seam or keep task open.

Completion = owner can SEE + USE requested behavior in installed runtime AND exact code is on `origin/main`. Built, tested, landed, deployed, or route-loaded alone = incomplete.

## Incremental owner-visible delivery

Split multi-stage features at usable behavior boundaries. Each checkpoint MUST reach `origin/main`, deploy, and pass real owner-flow acceptance before next checkpoint starts.

```text
DO NOT: build layout + grouping + diagnostics + diffs + linkage behind one final gate
TARGET: layout → land/deploy/prove → grouping → land/deploy/prove → diagnostics → land/deploy/prove
```

A passing local checkpoint does not authorize accumulating later checkpoints behind it. If work was already bundled, state process failure honestly; do not fabricate retroactive increments. Ship smallest independently useful safe prefix immediately, then continue.

## Required sequence

1. **Focused:** run changed seam's smallest deterministic tests.
2. **Static:** compile/typecheck, parse config/YAML, lint changed scope, `git diff --check`.
3. **Broad:** run mandatory full suite once only after 1–2 pass.
4. **Installed:** after deployment, invoke real installed entrypoint and assert changed behavior.

The land conductor has two gate backends. `OVERDECK_LAND_GATE_MODE=local` (the default) runs the
captured command through `local-gate`; dark `checks` mode verifies the submitted head SHA through
GitHub Checks and never runs that command locally. Tests for the Checks backend must use a fake API,
assert local mode is unchanged, and prove API/credential failure refuses the land rather than
falling back. Runtime settings live in `~/.config/overdeck/land-gate.env`; environment wins.

```text
DO NOT: full suite → discover syntax failure → full suite again
TARGET: focused test → static checks → one full gate → installed proof
```

## Temporary files

Keep every test temp root outside repository and all ancestor Git worktrees. Repo ancestry changes Git discovery and creates false results.

- Default root: `${XDG_CACHE_HOME:-$HOME/.cache}/overdeck/tests/<suite>/<unique-run>`.
- Set suite-supported temp variables before process start (`TMPDIR`, pytest `--basetemp`, tool cache flags).
- Give parallel workers separate mutable roots.
- NEVER place temp roots under `/home/user/Projects`, checkout, or worktree.
- NEVER broadly delete shared temp/cache roots. Remove only exact run-owned path when cleanup is required.

Deploy-clone fixtures must model the release-tree permission boundary: the successful
path ends read-only, the next serialized deploy reopens it, and cleanup explicitly restores
owner write permission before removing the fixture. Generated installer output belongs under
XDG state/cache roots; tests must assert any release-tree-facing entry is only a symlink.

## Dependencies

Reuse locked environments and package caches. Install only when declared dependency state is absent or changed.

- Preserve lockfile enforcement.
- Share immutable download caches; NEVER share mutable test state, SQLite databases, coverage files, or active virtual environments across workers.
- A warning is not cache success. Resolve every warning.

## Slow full suites

Use repository gate wrapper when naive command is wrong or repeated. One wrapper owns:

- safe external temp root;
- focused/static/full subcommands;
- deterministic shard manifest;
- per-shard exit + timing capture;
- aggregate fail-closed verdict;
- exact-revision receipt lookup/write.

Parallelize only proven-independent groups. Route remote workloads through registered reachable buildboxes and existing offload wrappers. Missing, skipped, interrupted, or duplicate shard result = failure. Never invent ad hoc splits that change fixture/order semantics.

## Timing + clocks

Enable per-test duration output for slow-suite runs. Optimize measured leaders first.

- Inject fake monotonic clock, sleeper, and process seams for retries, leases, deadlines, backoff, and timeout boundaries.
- Keep one bounded real-process integration test for wiring, signaling, and output drain.
- NEVER shorten production timeout or change host/node clock to make tests fast.

## Exact-revision receipts

Reuse broad-suite success only when receipt proves exact tested tree, command, dependency state, toolchain, shard coverage, and clean exit.

Committed tree: commit ID + clean status. Dirty tree: commit ID + deterministic diff hash; receipt invalidates on any byte change.

Receipt MUST record exact commands, focused/static/full results, start/end/elapsed time, dependency hashes, tool versions, worker identities, shard map, temp isolation, logs, and exit codes. Write atomically after all shards pass. Deployment MUST NOT rerun an unchanged broad gate; installed seam proof still runs.

## Kubernetes admission image refreshes

For an admitted Factory image refresh, a landed digest and local tests do not
prove the cluster has accepted the new image. The refresh receipt must show the
returned immutable digest, all source pin sites updated together, the landed
revision, and `kubectl apply -f modules/harness/factory/kubernetes/runtime.yaml`
before resubmitting a Job. A landed-but-not-applied admission policy continues
to deny the newly pinned workload containers.

## Red flags

- Broad suite is first command.
- Temp path sits in repository ancestry.
- Same full suite runs twice for unchanged bytes.
- Suggested receipt command/tool does not exist.
- Dependency install is unrelated to changed stack.
- Parallel split lacks an equivalence contract.
- Final checklist reruns a gate already covered by valid receipt.
- Sleep replaces an injectable clock.
- A scale fixture omits the production shape that causes the expensive work (for example, null
  multiplexer fields while claiming to bound per-socket process churn).

Stop and correct workflow before continuing.

## Self-update — MANDATORY

This doc is the fleet's memory. You changed the architecture, wiring, commands, or
doctrine this skill describes — or a live incident just proved a rule here wrong or
missing → EDIT THIS FILE in the same landing as the change. Adopted AND rejected
decisions both go in, with the measured why. An owner reminder to record a lesson is
a failure of this rule, not the trigger for it.

Post-deploy observability steps that touch a developer checkout must be fail-open and
prove refusal behavior with real Git fixtures; deployment success cannot depend on a
clean browsing checkout.