# Harness v2 Phase 3 Durable Queue and Run Windows Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use /ship (recommended) or /executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

audience: AI coding agents first.

**Goal:** Add a durable linear plan queue with restart-safe run windows, strict failure stops, and honest queue visibility in CLI, TUI, and Overdeck.

**Architecture:** Keep current v2 per-run supervision authoritative. Store ordered queue state in one atomically rewritten JSONL file; invoke one short systemd worker per transition to reconcile a running entry or launch one eligible entry, then exit. Derive completion from existing append-only run journals and supervision receipts; expose a read-only queue projection through the v2 control API.

**Tech Stack:** Node.js CommonJS, systemd user services/timers, atomic JSONL state, append-only NDJSON journals, Rust/ratatui TUI, Bun/TypeScript collector, React/Astro, `node:test`, Vitest.

---

## Wave Plan

| Wave | Tasks | Files touched | Safe to parallelize? |
|---|---|---|---|
| 1 | Task 1, Task 2 | `modules/harness/v2/queue.js`, `modules/harness/v2/test/queue.test.js`; `modules/harness/systemd/harness-queue.service`, `modules/harness/systemd/harness-queue.timer`, `modules/harness/v2/test/systemd-phase3.test.js` | ✅ no overlap |
| 2 | Task 3, Task 4 | `modules/harness/v2/bin/runplan.js`, `modules/harness/v2/bin/queue-worker.js`, `modules/harness/spec/events.schema.json`, `modules/harness/v2/test/queue-worker.test.js`; `modules/harness/v2/control-api.js`, `modules/harness/v2/test/control-api.test.js` | ✅ no overlap |
| 3 | Task 5, Task 6 | `collector/src/adapters/harness.ts`, `collector/src/adapters/harness.test.ts`; five distinct `modules/harness/tui/src/*` files and four distinct `modules/harness/tui/tests/*` files | ✅ no overlap |
| 4 | Task 7 | `apps/web/src/lib/panel-data.ts`, `apps/web/src/components/plans/PlansContent.tsx`, `apps/web/src/components/plans/PlansContent.test.tsx` | single task |
| 5 | Task 8 | `modules/harness/v2/test/index.js` | single task |

## File Structure

- `modules/harness/v2/queue.js` — queue schema, lock, atomic persistence, mutation rules, terminal projection, and local-time window evaluation.
- `modules/harness/v2/bin/queue-worker.js` — one idempotent reconcile-or-launch transition, then exit.
- `modules/harness/v2/bin/runplan.js` — `queue` CLI routing and supervised launch reuse.
- `modules/harness/systemd/harness-queue.service` — one short queue transition.
- `modules/harness/systemd/harness-queue.timer` — persistent eligibility/reconciliation trigger.
- `modules/harness/spec/events.schema.json` — queue launch and window-bypass journal events.
- `modules/harness/v2/control-api.js` — authenticated read-only `GET /queue`.
- `collector/src/adapters/harness.ts` — queue projection into plans panel data.
- `modules/harness/tui/src/{control,state,plans,lib,main}.rs` — queue transport, state, and read-only plans-view rendering.
- `apps/web/src/lib/panel-data.ts` — queue panel contract.
- `apps/web/src/components/plans/PlansContent.tsx` — read-only queue section composed from existing deck-ui exports.

### Task 1: Durable Queue Store and Run-Window Projection

**Wave:** 1

**Blocks:** Task 3, Task 4

**Blocked by:** —

**Files:**
- Create: `modules/harness/v2/queue.js` — queue persistence and scheduling rules.
- Create: `modules/harness/v2/test/queue.test.js` — schema, concurrency, recovery, and fake-clock window tests.

**Contract (pin EXACTLY — this is the divergence-prone surface):**
- Queue file: `${HARNESS_HOME:-$HOME/.harness}/v2/queue.jsonl`; one current entry per line, preserved in linear order.
- Entry shape: `{ id, repo, slug, preset?, account?, added_at, status, attempt, run_id?, supervision_token?, window?, next_eligible_at?, bypass_requested_at?, terminal_at?, failure? }`.
- Status enum: `pending | running | done | failed | skipped`. `repo` is an absolute canonical repository path. `id` is opaque and stable. Reject duplicate nonterminal `{repo,slug}` entries.
- `window` shape: `{ source, start, end, time_zone }`; `start` and `end` are strict `HH:MM`, `time_zone` is an IANA zone accepted by `Intl.DateTimeFormat`, and `source` is the CLI literal or named window key.
- Named windows resolve from `${HARNESS_HOME:-$HOME/.harness}/v2/windows.json`, an object mapping names to `HH:MM-HH:MM@IANA` strings. Explicit windows use that same literal syntax. Reject equal endpoints, malformed times, invalid zones, and unknown names; support same-day and overnight windows.
- Export queue load/list/add/skip/remove/start-intent/update operations, terminal projection from `readJournal(repo, slug, runId)`, and `nextEligibleAt(window, nowMs)`.
- Serialize every read-modify-write under `${queueFile}.lock`. Lock identity includes boot ID, PID, and `/proc/<pid>/stat` start time; recover only a provably stale lock. Live or ambiguous ownership fails closed.
- Persist with same-directory temporary file, `fsync`, rename, and directory `fsync`. Malformed JSON, unknown fields/status, duplicate IDs, partial final lines, impossible transitions, or corrupt terminal receipts fail closed without rewriting the source file.

**Behavior:**
- `add` appends pending work without disturbing completed history. `skip` accepts only earliest matching `pending|failed` entry in the caller repository. `rm` accepts pending entries only.
- Terminal success requires `plan-end`, all task statuses `succeeded|skipped-done`, non-pending push/lock state, and land status `none|merged|pr-open|pr-merged`. `plan-error`, `supervision.restart-refused`, incomplete land, or any other task outcome projects failed.
- Failed entry blocks every later entry until explicit `queue start` retry intent or deliberate skip. Retry increments `attempt`, assigns a new deterministic run identity, and preserves prior failure evidence.
- Window evaluation is pure and fake-clockable. Persisted `next_eligible_at` is an absolute UTC ISO timestamp; reboot does not shift eligibility. Running work ignores window end.

**Acceptance:**
- [ ] `node --test modules/harness/v2/test/queue.test.js`
- [ ] Fake-clock cases cover before/inside/after, overnight, spring-forward, fall-back, invalid zone, and reboot from persisted `next_eligible_at`.
- [ ] Concurrent add/update cases preserve every entry; crash before rename leaves last valid queue intact; live/ambiguous lock refuses mutation.

### Task 2: Stateless Queue systemd Service and Timer

**Wave:** 1

**Blocks:** Task 3

**Blocked by:** —

**Files:**
- Create: `modules/harness/systemd/harness-queue.service` — one queue transition.
- Create: `modules/harness/systemd/harness-queue.timer` — persistent queue wake-up.
- Create: `modules/harness/v2/test/systemd-phase3.test.js` — unit syntax and no-daemon contract tests.

**Contract (pin EXACTLY — this is the divergence-prone surface):**
- `harness-queue.service` uses `Type=oneshot` and invokes released `v2/bin/queue-worker.js` once.
- `harness-queue.timer` uses `Persistent=true`, targets `harness-queue.service`, and wakes at least once per minute without spawning overlapping workers.
- Unit state contains no queue entries, plan paths, mutable argv, or retry counters. Queue file and run journals remain sole durable authority.
- Service has no `Restart=always`, resident loop, or daemon process. One invocation performs at most one launch or one terminal-state transition and exits.

**Behavior:**
- Reboot re-arms eligibility checks without inventing launches or terminal receipts.
- Repeated service starts against unchanged state are no-ops.
- A failed entry keeps timer invocations inert until explicit retry/skip mutation.

**Acceptance:**
- [ ] `node --test modules/harness/v2/test/systemd-phase3.test.js`
- [ ] `systemd-analyze --user verify modules/harness/systemd/harness-queue.service modules/harness/systemd/harness-queue.timer`

### Task 3: Queue CLI and Idempotent Worker

**Wave:** 2

**Blocks:** Task 5, Task 6

**Blocked by:** Task 1, Task 2

**Files:**
- Modify: `modules/harness/v2/bin/runplan.js` — queue subcommand routing and reusable supervised launch seams.
- Create: `modules/harness/v2/bin/queue-worker.js` — one reconcile-or-launch transition.
- Modify: `modules/harness/spec/events.schema.json` — `queue.launch` and `queue.window-bypassed`.
- Create: `modules/harness/v2/test/queue-worker.test.js` — CLI, failure-stop, resume, crash-gap, and exactly-once integration tests.

**Contract (pin EXACTLY — this is the divergence-prone surface):**
- CLI:
  - `runplan queue add <slug> [--repo-root <path>] [--preset <name>] [--account <slug>] [--window <name|HH:MM-HH:MM@IANA>]`
  - `runplan queue`
  - `runplan queue start [--now]`
  - `runplan queue skip <slug> [--repo-root <path>]`
  - `runplan queue rm <slug> [--repo-root <path>]`
- `runplan queue` prints ordered rows with ID, repository, slug, status, attempt, window, next eligibility, run ID, and failure; missing optional values render `-`, never fabricated values.
- `queue start` records retry intent for the first failed entry when present, persists `bypass_requested_at` on the target when `--now` is supplied, enables/starts `harness-queue.timer`, triggers `harness-queue.service`, and returns. It never waits in a resident coordinator loop.
- Worker algorithm under queue lock: reconcile sole running entry from durable journal/supervision receipts; if terminal, persist `done|failed` and exit; otherwise select first pending eligible entry, persist `running` with incremented attempt plus deterministic `run_id` and `supervision_token`, launch through existing `launchSupervised(...)`, append `queue.launch`, then exit.
- Crash gap is idempotent: if queue says running but supervision registry is absent, reuse persisted identities and launch once; if exact registry/unit exists, adopt it; mismatched registry identity fails closed. Never create a second live run for one queue attempt.
- Failed retry passes existing `--retry-blocked` semantics. Live/crashed-recoverable runs remain running for Phase 2 reconcile; only durable terminal/restart-refused receipts settle them.
- `--now` bypasses only target entry's current window, persists audit timestamp before launch, and appends `queue.window-bypassed` to that run journal after registration.

**Behavior:**
- Three entries run strictly one at a time. Failure at entry two produces `done,failed,pending`; timer ticks do not retry or launch entry three.
- After plan two is fixed, explicit `queue start` retries entry two, then later timer/service invocations launch entry three exactly once.
- Direct `runplan <plan>` remains unchanged and may run independently in parallel.

**Acceptance:**
- [ ] `node --test modules/harness/v2/test/queue-worker.test.js`
- [ ] Integration fixture proves `done,failed,pending`, explicit retry, then `done,done,done`, with one live unit and one successful terminal receipt per attempt.
- [ ] Kill tests at pre-persist, post-persist/pre-launch, and post-launch boundaries recover without lost state or duplicate launch.

### Task 4: Queue Read API

**Wave:** 2

**Blocks:** Task 5, Task 6

**Blocked by:** Task 1

**Files:**
- Modify: `modules/harness/v2/control-api.js` — authenticated queue projection.
- Modify: `modules/harness/v2/test/control-api.test.js` — queue endpoint contract and corruption isolation.

**Contract (pin EXACTLY — this is the divergence-prone surface):**
- Add authenticated `GET /queue`.
- Response: `{ queue: [{ id, repo, slug, preset, account, addedAt, status, attempt, runId, window, nextEligibleAt, bypassRequestedAt, terminalAt, failure }] }`; optional values are explicit `null`.
- Preserve file order. Return no internal lock path, supervision token, PID identity, credentials, or mutable filesystem handle.
- Queue read failure returns non-200 with exact error; it never returns an empty queue for unreadable/corrupt state.
- Existing `/runs` response and capabilities remain backward compatible.

**Behavior:**
- API is read-only; all queue mutation stays in `runplan queue`.
- Pending entries are visible before any run registry record exists.

**Acceptance:**
- [ ] `node --test --test-name-pattern='queue' modules/harness/v2/test/control-api.test.js`
- [ ] Tests cover empty, pending/window-held, running, done, failed, skipped, null optionals, auth rejection, and corrupt-file failure.

### Task 5: Collector Queue Projection

**Wave:** 3

**Blocks:** Task 7

**Blocked by:** Task 3, Task 4

**Files:**
- Modify: `collector/src/adapters/harness.ts` — poll and validate `GET /queue`, then include it in plans panel data.
- Modify: `collector/src/adapters/harness.test.ts` — lossless queue projection and failure retention tests.

**Contract (pin EXACTLY — this is the divergence-prone surface):**
- Add exported `HarnessQueueEntry` matching Task 4 response and make `HarnessPlansPanelData` emission `{ runs, queue }`.
- Poll `/runs` and `/queue` as mandatory state for a complete plans panel. Invalid queue shape or request failure rejects the poll so collector retains prior panel state; never substitute `[]`.
- Forward queue values losslessly. Preserve `null` optionals and source status/window strings; do not synthesize run IDs, timestamps, task counts, or status colors.
- Keep queue entries separate from run summaries; correlation uses explicit `runId` only when non-null.

**Behavior:**
- Pending entries appear in collector state before launch.
- A running entry and its run may both appear without being merged into fabricated data.

**Acceptance:**
- [ ] `bun test collector/src/adapters/harness.test.ts`
- [ ] Tests cover exact ordered projection, null preservation, failed request retaining prior state, and no fabricated run fields.

### Task 6: TUI Queue Visibility

**Wave:** 3

**Blocks:** Task 8

**Blocked by:** Task 3, Task 4

**Files:**
- Modify: `modules/harness/tui/src/control.rs` — `GET /queue` client and strict decoder.
- Modify: `modules/harness/tui/src/state.rs` — durable queue snapshot state.
- Modify: `modules/harness/tui/src/plans.rs` — read-only queue rows above run rows.
- Modify: `modules/harness/tui/src/lib.rs` — queue model, refresh application, and projection.
- Modify: `modules/harness/tui/src/main.rs` — fetch queue with plans refresh.
- Modify: `modules/harness/tui/tests/plans.rs` — queue row projection.
- Modify: `modules/harness/tui/tests/state.rs` — refresh and stale-state behavior.
- Modify: `modules/harness/tui/tests/model.rs` — queue model fixtures.
- Modify: `modules/harness/tui/tests/interaction.rs` — queue rows remain non-actionable.

**Contract (pin EXACTLY — this is the divergence-prone surface):**
- Add deserializable `QueueEntry` matching Task 4 and `ControlClient::queue() -> Result<Vec<QueueEntry>, ControlError>`.
- Plans projection exposes queue as a distinct read-only section ordered by queue file order; never convert queue entries into `RunSummary`.
- Row fields: ordinal, repository basename plus full path in detail, slug, status, attempt, window or `-`, next eligible or `-`, and linked run ID only when present.
- Queue decode/transport failure preserves last valid snapshot and surfaces transport notice; it never clears rows.
- Existing run selection, attach, lifecycle, steer, and decision actions do not target queue rows.

**Behavior:**
- Operator can distinguish window-held, running, failed-stop, skipped, and completed entries.
- Missing values render `-`; timestamps come only from API.

**Acceptance:**
- [ ] `cargo test --manifest-path modules/harness/tui/Cargo.toml queue`
- [ ] Tests prove ordered projection, honest missing values, stale snapshot retention, and zero run actions from queue-row input.

### Task 7: Overdeck Plans Queue Visibility

**Wave:** 4

**Blocks:** Task 8

**Blocked by:** Task 5

**Files:**
- Modify: `apps/web/src/lib/panel-data.ts` — structural `HarnessQueueEntry` and `{ runs, queue }` plans data.
- Modify: `apps/web/src/components/plans/PlansContent.tsx` — read-only queue section.
- Modify: `apps/web/src/components/plans/PlansContent.test.tsx` — queue states, missing values, and no-action tests.

**Contract (pin EXACTLY — this is the divergence-prone surface):**
- Render queue in a distinct `SectionCard` before run groups, using existing `DeckTable`, `StatusChip`, timestamp formatting, and tooltip exports from `@overdeck/deck-ui`; create no new deck-ui primitive.
- Columns: position, repository, plan, status, attempt, window, next eligible, run. Use queue order; no client-side resort.
- Status text remains source truth. Use existing `planStatusCategory` only where it already maps the queue status; unknown status stays neutral and visible.
- Repository displays basename with full absolute path in existing tooltip treatment. Null time/window/run values display em dash. Do not invent timestamps, run links, task totals, or progress.
- Queue rows are read-only in Phase 3. Link to run detail only when `runId` is non-null; no skip/remove/start controls.
- Preserve existing empty-run behavior: queued entries still render when `runs` is empty.

**Behavior:**
- Pending work is visible before launch; failed queue stop is distinct from a failed run.
- Both themes, keyboard navigation, focus visibility, and viewport-contained scrolling follow `od-ui-dev`.

**Acceptance:**
- [ ] `pnpm --filter web test -- src/components/plans/PlansContent.test.tsx`
- [ ] Tests cover queue-only panel, all five statuses, window-held entry, null optionals, linked running entry, and no fabricated data/actions.

### Task 8: Register and Run Full Phase 3 Regression Suite

**Wave:** 5

**Blocks:** —

**Blocked by:** Task 1, Task 2, Task 3, Task 4, Task 5, Task 6, Task 7

**Files:**
- Modify: `modules/harness/v2/test/index.js` — register Phase 3 Node suites exactly once.

**Contract (pin EXACTLY — this is the divergence-prone surface):**
- Register `queue.test.js`, `queue-worker.test.js`, and `systemd-phase3.test.js` exactly once; preserve every existing registration.
- Run focused queue/window tests before full v2, preset, collector, TUI, deck-ui, and web gates.
- No test may start a resident queue process, use wall-clock sleeps for eligibility, bypass corrupt state, or suppress warnings.

**Behavior:**
- Full regression proves Phase 1 trust controls and Phase 2 supervision/recovery remain unchanged.
- Queue, timer, collector, TUI, and web surfaces agree on one API contract and preserve honest missing data.

**Acceptance:**
- [ ] `node --check modules/harness/v2/queue.js`
- [ ] `node --check modules/harness/v2/bin/queue-worker.js`
- [ ] `node --check modules/harness/v2/bin/runplan.js`
- [ ] `node --test modules/harness/v2/test/queue.test.js modules/harness/v2/test/queue-worker.test.js modules/harness/v2/test/systemd-phase3.test.js modules/harness/v2/test/control-api.test.js`
- [ ] `node modules/harness/v2/test/index.js`
- [ ] `node modules/harness/presets/_validate.mjs`
- [ ] `bun test collector/src/adapters/harness.test.ts`
- [ ] `cargo test --manifest-path modules/harness/tui/Cargo.toml`
- [ ] `pnpm --filter @overdeck/deck-ui test && pnpm --filter @overdeck/deck-ui typecheck`
- [ ] `pnpm --filter web test -- src/components/plans/PlansContent.test.tsx && pnpm --filter web build && pnpm --filter web typecheck`