# harness-tier-backlog — Design audience: AI coding agents first. Contract-level: seams + decisions, NOT code bodies. slug: `harness-tier-backlog` · date: 2026-07-01 ## Purpose Today `resolve-seat.sh` silently falls back to a preset's flat/default binding whenever a role has no tier layer for the requested tier (rung 4 of the resolution ladder in `lib/resolve-seat.sh:238-254`) — a task authored as `tier:"high"` can execute on whatever model the preset happens to bind that role to, with zero signal. This spec adds: (1) detection + surfacing of that silent downgrade in the web UI, (2) a durable, triageable **backlog** of tasks worth re-running on a stronger binding — populated both by that auto-detection and by a manual user flag — scoped per-plan / per-project / globally, and (3) a manual re-run action that requeues a backlog item against an explicit tier/binding override. This binds three already-specced/landed surfaces: `lib/resolve-seat.sh` + `src/runner.js` (harness core, Plan A), the `runstate/v1` journal (`lib/journal.sh`), and the Web UI gateway (`docs/specs/2026-06-30-harness-web-ui-design.md`). It adds nothing to those contracts except new record types and gateway routes that read/write them. ## Ground truth (do not re-derive) - **Resolution ladder** (`lib/resolve-seat.sh:238-254`): overrides["seat.tier"] → overrides["seat"] → preset tiered binding → preset flat binding → BLOCK. Rung 4 fires whenever `preset.seats[seat]` has no tier layer at all (`has("wrapper")` true on the seat object itself) — this is indistinguishable, today, from "this preset was never given tiers for this role" vs. "operator meant to route high-tier here and forgot to." - **Partial tier coverage still hard-BLOCKs today — this spec closes that gap at lint time, not runtime.** When a seat *has* a tier layer but not the *requested* tier (e.g. `seats.coder` defines `low`/`medium` but the task asks `high`), rung 3 fails (no matching key) and rung 4 also fails (a tiered seat object has no top-level `wrapper`) → today's ladder hard BLOCKs. The project's "LLM mismatch must never halt a run" rule was already resolved for the analogous *role* problem via authoring-time lint (`task.seat` must exist in the preset), not a runtime fallback — this spec applies the same mechanism to tiers instead of adding a new resolver rung: see Architecture Decisions. - **`task.tier` is defaulted before backlog detection can see whether it was omitted.** `runner.js` normalizes `task.tier || "regular"` at multiple use sites (`src/runner.js:389,394,521`) with no separate flag for "plan author wrote this explicitly" vs. "plan omitted it." The auto-downgrade detector cannot use the post-default value as its trigger — see Architecture Decisions for the fix. - **`binding` on the journal record already carries the actually-resolved seat/tier** (`runstate/*.jsonl`, e.g. `{"binding":{"tier":"regular"}}` — `src/runner.js:182,231`), written post-fallback by `dispatchWithFallback` (`src/runner.js:188-231`). The journal is the only place both "what the task asked for" (`task.tier`, from the plan JSONL) and "what actually ran" (`binding`) already coexist — no new capture point is needed for the auto detector, only a comparison. - **Global vs. per-repo split already exists and is the established pattern to extend, not invent.** Per-run truth lives at `/runstate/` (git-adjacent, durable); the **global** registry `~/.harness/runs/.json` is a **pointer only** (`{runId, slug, repoRoot, project, repo, branch, worktree, runstateDir, created}`), reaped on read if `runstateDir` is gone (`docs/specs/2026-06-30-harness-control-api-design.md:242-246`). This spec's global backlog view follows the identical shape: pointer-only global index, full record stays in the repo. - **Tier taxonomy is being renamed** `critical/regular/simple` → `low/medium/high` (companion change, out of scope here but assumed landed first — see Non-goals). This spec's records use the new names. - **Priority is a separate axis from tier**, per user direction: tier (`low/medium/high`) says how capable a model the task *needs*; backlog priority says how *urgently* a flagged item should be re-run. Only ~3 roles exist today so most flagged items cluster at `tier:"high"` — priority is what actually ranks them against each other. ## Architecture Decisions **Global backlog storage: per-repo pointer *registry*, not a per-record pointer index.** Three options were weighed. (a) One canonical `~/.harness/backlog.jsonl` holding full entries — simplest to query, but a second copy of task-binding truth outside the git-adjacent journal, diverging from this project's truth-lives-in-repo rule and risking drift if a repo is deleted/moved without updating the global copy. (b, this spec's original draft) a global index with one pointer row *per backlog record* — avoids (a)'s duplication, but still creates a second mutable store that must track every record's lifecycle; a repo-local status rewrite (e.g. `open`→`resolved`) has no relationship to its pointer row, so the pointer is stale the instant it's written and "reap-on-read" only catches *deleted repos*, not *stale status*. (c) `~/.harness/backlog-repos.jsonl` holding one row **per repo**, `{repoRoot, runstateDir, project, lastSeen}` — a pure locate-a-repo cache, never a locate-a-record index. **(c) accepted.** The global Backlog view enumerates repos from this cache, then reads each repo's own `runstate/backlog.jsonl` directly for live status/priority/order — there is exactly one place any record's state lives, and the global file only ever needs a write when a *new repo* starts using backlog (not on every record append/status change), which eliminates the per-record drift Codex/advisor both flagged. This mirrors the intended shape of the (currently spec-only, not yet built — verified: no `~/.harness/runs/*.json` writer exists in `src/supervisor.js`/`bin/ensure-web.sh`/the web gateway as of this writing) global run registry; once that registry lands, `backlog-repos.jsonl` can likely be dropped in favor of reusing it, but this spec does not depend on that landing first. **Detection lives in `runner.js`, not `resolve-seat.sh`.** `resolve-seat.sh` is a pure function (plan+runconfig+node → binding or fail-closed) and must stay side-effect-free — it has no journal-append capability today and should not gain one. The comparison ("`task.tier` requested vs. preset had no tier layer for this seat") is knowable entirely from data `runner.js` already holds after calling `resolve-seat.sh` (the resolved binding) and the preset it already loaded — so detection is a post-call check in `runner.js`, not a change to the resolver's contract. **Partial tier coverage is closed by a preset-schema lint rule, not a new resolver rung.** Considered adding a "nearest-available-tier" rung to `resolve-seat.sh` before BLOCK; rejected. It has three coupled problems: (1) it puts the same "LLM assigned something the preset can't resolve" problem behind two different mechanisms — lint for roles, runtime silent-fallback for tiers — with no principled reason for the split; (2) "nearest lower tier" has no defined answer when the requested tier is *below every tier the seat defines* (seat has `{medium,high}`, task asks `low`); (3) it conflates two opposite-direction mismatches — asked-high-got-medium is a real downgrade (backlog-worthy), asked-low-got-high is a wasteful *upgrade* (not a correctness risk, not backlog-worthy) — under one rung and one detection rule. **Accepted instead:** `presets/_validate.mjs` (or the schema it drives) requires that any seat defining a tier layer at all define **all three tiers** (`low`/`medium`/`high`) — partial coverage becomes a preset-authoring error caught at validate/lint time, before any run starts, exactly like the seat-name lint already planned for roles. `resolve-seat.sh`'s ladder is unchanged: rung 3 (tiered) always has a key to find once presets pass lint; rung 4 (flat) is unchanged; BLOCK remains reserved for a seat name absent from the preset entirely. This keeps resolve-seat.sh's contract exactly as it is today — only the `rung` field (below) is new. **`task.tier` explicitness is captured before defaulting, not inferred after.** Plan JSONL parsing must preserve whether `tier` was present on the raw task record (`tierExplicit: boolean`) alongside the existing default-to-`"regular"`-equivalent normalization at `src/runner.js:389,394,521` — the default value itself is never sufficient to distinguish "omitted" from "author wrote the default on purpose." The auto-downgrade detector reads `tierExplicit`, not the defaulted `task.tier`. **Backlog persistence gets its own helper, `lib/backlog.sh`, not a reuse of `lib/journal.sh`.** `lib/journal.sh`'s `_journal_validate_record` hard-requires `v == "runstate/v1"` plus the runstate-specific field set (`slug/task/wave/state/base/head/rc/ts`) — it will reject a `backlog/v1` record outright. `lib/backlog.sh` provides its own `append` (validates the `backlog/v1` shape, below) and `set-status` (in-place line rewrite by `id`) — same locking discipline as `journal.sh`, independent schema. **Backlog record `id` is deterministic, not random, to make auto-downgrade append idempotent.** `id = sha1(slug, task, requested.seat, requested.tier, resolved.wrapper, resolved.model)` (or equivalent stable hash over those fields). Re-dispatch/resume of the same task under the same downgrade re-computes the same `id`. Idempotency scope is explicit and status-dependent: if a record with that `id` already exists with `status:"open"` or `status:"requeued"`, `append` is a no-op (refreshes nothing, avoids duplicate rows across retries of a run still in flight). If the existing record has `status:"resolved"` or `"dismissed"`, that is a **genuine recurrence** — the same downgrade happening again after the user believed it was handled — and `append` writes a **new** record (a fresh `id` disambiguated by appending `created_ts` to the hash input, so it never collides with the closed one) rather than silently reopening or dropping it. A random ID (e.g. ULID) would prevent neither the in-flight-retry duplication nor let recurrence-after-close be distinguished from it. ## Data model ### Backlog record (per-repo, canonical) — `runstate/backlog.jsonl`, append-only, one JSON object per line Same file-per-repo, append-only-with-in-place-status-rewrite convention as `runstate/*.jsonl` and the brainstorm skill's own session JSONL — status changes rewrite that record's line in place (found by `id`), everything else is append-only. ``` { "v": "backlog/v1", "id": "", "slug": "", "task": "", "wave": , "reason": "auto-downgrade" | "manual-flag", "requested": {"seat": "", "tier": "low|medium|high"}, "resolved": {"wrapper": "...", "model": "...", "tier": "low|medium|high|null", "rung": 4}, "note": "", "priority": null | "", "order": , "status": "open" | "requeued" | "resolved" | "dismissed", "created_ts": "", "requeue": null | {"runId": "", "task": "", "ts": ""} } ``` `priority` and `order` are **manual-only fields** (per accepted design decision) — nothing in this system ever writes them except a user action in the web UI. No heuristic priority scoring. Vocabulary is open (any string), not constrained to `P0/P1`, to match "priority can be finer than just critical" without the system inventing a taxonomy the user didn't ask for. ### Global repo registry — `~/.harness/backlog-repos.jsonl`, append-only, reap-on-read ``` {"repoRoot": "...", "runstateDir": "...", "project": "...", "lastSeen": ""} ``` One row per repo, not per backlog record (see Architecture Decisions). `lib/backlog.sh append` upserts this row (rewrite `lastSeen` in place if `repoRoot` already present, append if not) whenever it writes a repo-local backlog record. This file is shared across repos, so `journal.sh`'s per-repo lock does not cover it — the upsert is an unlocked read-modify-write. This is accepted, not an oversight: the field is a `lastSeen` discovery cache, not a status source of truth, so a lost race (two upserts overlapping, one `lastSeen` value clobbering another) is benign — worst case the Global tab shows a slightly stale timestamp for one repo, never a wrong or missing record. The global Backlog view reads this file to find candidate repos, then reads each repo's own `runstate/backlog.jsonl` directly — this file never carries record-level status, so it can never go stale at the record level, only at the repo level (handled by the existing reap-on-read: drop rows whose `runstateDir` no longer exists). ## Detection & population ### 1. Auto-downgrade (in `runner.js`, alongside existing `resolveTaskSeat`/`dispatchWithFallback` calls) After a binding resolves (`src/runner.js:188-231`), compare `task.tierExplicit` (captured at plan-parse time, before any defaulting — see Architecture Decisions) against the resolved binding's `rung`. If `tierExplicit` is true (the plan author wrote a tier, not the implicit default) **and** `rung` is `4` (flat fallback fired — the seat has no tier layer at all) → call `lib/backlog.sh append` with `reason:"auto-downgrade"`, `requested` = what the task asked for, `resolved` = the binding that actually ran (including `rung`), `priority: null`, `status: "open"`. (Partial tier coverage no longer reaches the resolver at all — it's a preset-lint failure, caught before any run starts; see Architecture Decisions.) This needs `resolve-seat.sh` to report *which rung fired* — today it only returns the resolved binding. Add a non-breaking field to `resolve-seat.sh`'s stdout binding object: `"rung": 1|2|3|4` (`lib/resolve-seat.sh: 244-254`, one value per branch already present). `runner.js` reads it; nothing else in the resolver's contract changes. ### 2. Manual flag (web UI) New gateway route on an already-loaded Agent/Job page (`docs/specs/2026-06-30-harness-web-ui-design.md` components 6/7): `POST /api/runs/:runId/tasks/:taskId/backlog {note}` → gateway resolves `runId` → `repoRoot` via the run registry (same lookup already used for every other per-run route), calls `lib/backlog.sh append` with `reason:"manual-flag"`, `requested`/`resolved` filled from that task's actual journal binding record, `note` required non-empty (distinguishes an intentional flag from noise). Both paths call the same `lib/backlog.sh append` helper (repo-local record write, then repo-registry upsert) — one control point, not an atomic transaction across two files (see Error handling for what happens when the second write fails). ## Web UI — Backlog page New page, `web/src/pages/backlog.astro` + `Backlog.tsx` island, added as component 9 to the existing Web UI spec's page set (Overview/Job/Agent/OKF/Adapter). Scope tabs: **Plan** (current run's `slug` only, reachable from the Job page) / **Project** (all slugs sharing the current run's `repoRoot`) / **Global** (every pointer in `~/.harness/backlog-index.jsonl`, dereferenced, reaping any whose `repoRoot`/`runstateDir` no longer exists — same reap-on-read rule the run registry already follows). List view: grouped by `priority` (nulls last, unprioritized), user-draggable `order` within a group, each row shows task/slug/requested-vs-resolved tier, reason badge (auto-downgrade vs manual-flag), and two actions: **set priority** (free-text input, writes `priority`) and **re-run at tier** (dropdown of tiers available for that role in the current preset, defaulting to the task's originally-requested tier). ### Re-run action `POST /api/backlog/:backlogId/requeue {tier}` → gateway resolves the backlog record's `repoRoot`/`slug`/`task`, then calls a new `runner.js` export, `runSingleTaskWithOverrides(task, overrides)`, which constructs the `runconfig.overrides["."]` (or `[""]`) entry per the existing runconfig override schema resolve-seat.sh already validates (`lib/resolve-seat.sh:104-207` — no new override shape) and dispatches via the existing synthetic-seat path (`resolveSyntheticSeat`, `src/runner.js:403-`). This wrapper exists so the gateway calls one named orchestration entry point instead of assembling `runconfig.overrides` itself and reaching into `resolveSyntheticSeat` directly — the override-construction logic stays in `runner.js`, where the rest of seat resolution already lives. On dispatch, `lib/backlog.sh set-status` rewrites the backlog record: `status: "requeued"`, `requeue: {runId, task, ts}`. The re-run's own journal record is the source of truth for whether it then succeeds — the backlog entry is not marked `resolved` automatically; a user marks it `resolved`/`dismissed` from the UI once satisfied (manual close, matching manual priority — no auto-inferred success signal here). ## Error handling Fail-closed, matching the rest of the harness. Backlog append is best-effort-but-never-silent: if the repo-local append fails (e.g. `runstate/` not writable), the auto-downgrade detector logs to the run's own event stream as an `error` event (existing `events/v1` shape) rather than crashing the task — a missed backlog entry must never fail a task that otherwise succeeded. **No cross-file atomicity claim.** `lib/backlog.sh append` writes the repo-local record first, then upserts the global repo-registry row — these are two separate file writes, not a transaction; there is no invariant that guarantees both always land together. If the registry upsert fails after a successful repo-local write, the repo record exists but the Global view's repo cache doesn't yet list that repo — the fix is not "make it atomic" but "make the miss self-correcting": the Project/Plan tabs (which read the repo directly, not the registry) are unaffected, and the Global tab is a cache that can be repaired by re-scanning known repos on demand — out of scope for v1 as an automatic background process, noted as a Non-goal. Requeue against a tier the active preset doesn't support for that role → `resolve-seat.sh` BLOCKs exactly as it does today (Unknown seat/tier) — the gateway surfaces that 4xx inline, same pattern as the existing steer-409 handling. This should now be rare: presets with partial tier coverage fail lint before a run ever starts (see Architecture Decisions), so a live BLOCK on tier is only reachable if lint was bypassed; BLOCK remains the correct outcome for a seat name that doesn't exist in the preset at all. ## Testing strategy - `resolve-seat.sh`: rung field asserted for all four ladder branches (1/2/3/4), confirming no existing passing case changes behavior. - `presets/_validate.mjs` (or the schema it drives): a preset with a seat defining `low`/`medium` but not `high` (or any other partial tier set) fails lint; a seat defining all three, or none, passes. - `lib/backlog.sh`: `append` rejects a non-`backlog/v1` record and a manual-flag with empty `note`; append is a no-op for the same deterministic `id` while `status` is `open`/`requeued`; append writes a new, distinct record when the same downgrade recurs after the prior record was `resolved`/`dismissed`; `set-status` rewrites only the targeted line. - `runner.js` detector: fixture preset with a flat-only role + a task with `tierExplicit:true, tier:"high"` → asserts exactly one `backlog/v1` record with `reason:"auto-downgrade"` and `resolved.rung:4`; fixture task with `tierExplicit:false` (tier omitted, defaulted) → asserts zero records even if rung 4 fired; fixture preset with full tier coverage → asserts zero records. - Gateway: fixture repo with a `runstate/backlog.jsonl` → `GET` for each scope tab returns the right filtered set; Global tab reads `backlog-repos.jsonl` then dereferences each repo directly (not per-record pointers); repo row reaped when `runstateDir` is gone; requeue route asserts `runSingleTaskWithOverrides` is invoked with the correct override, not that the gateway constructs `runconfig.overrides` itself. - Web UI: priority set + reorder persists (`order` field), re-run button disabled when the preset has no alternate tier to offer. ## Non-goals (v1) - Automatic backlog-item close on requeue success — closing is manual. - Automatic background repair of `backlog-repos.jsonl` (re-scanning the filesystem for repos it missed) — the registry is a best-effort cache; a missing row only affects the Global tab's *discovery* of a repo, never that repo's own Plan/Project tabs, which read `runstate/backlog.jsonl` directly. - Any numeric/heuristic priority scoring — priority is free-text, user-set, full stop. - The `critical/regular/simple` → `low/medium/high` tier rename and the preset-vs-plan role-enum validation (earlier conversation, items 3/4/5/6) — assumed landed as a prerequisite; this spec only consumes the renamed tier vocabulary, it does not perform the rename. **Exception:** the all-or-nothing tier-coverage lint rule in `presets/_validate.mjs` ships as part of *this* spec, not the rename — it is small, mechanical, reuses the existing lint mechanism already planned for role validation, and the backlog auto-detector's `rung`-based trigger depends on partial coverage being unreachable at runtime (see Architecture Decisions). - Reusing the global run registry (`~/.harness/runs/.json`) instead of the new `backlog-repos.jsonl` — not done because, as of this writing, no writer for that registry exists yet in `src/supervisor.js`, `bin/ensure-web.sh`, or the web gateway; it is spec-only. `backlog-repos.jsonl` is intentionally the same shape so a later migration is a rename, not a redesign. ## Architecture Decisions (self-review record) Size gate: 2 new components (Backlog page + gateway routes; detector inside existing `runner.js`) — below the 3-module depth-audit threshold, self-review Phase 2 skipped per skill gate.