# Agent and Cluster Observability

audience: AI coding agents first.

status: DONE
task IDs: #1 (DONE), #2 (DONE), #4 (DONE)
source request: Make Overdeck observability useful: unified activity logs, agent/build/session/node visibility, time-series charts, drill-down, and terminal attach. Never substitute notification-delivery metadata for actual event evidence. Improve `/logs` end to end from four screenshots: label chart time and numeric axes; add pointer-hover and keyboard-focus point tooltips with timestamp/value; connect honest authoritative log-volume, commit, bot, build, and incident sources where available; demote unsupported series; make requested-series coverage subordinate to useful metrics, latest value, source, and gaps; render cached data immediately while lazy refresh uses one stale-while-revalidate lifecycle and indexed collector reads; compact the activity table to viewport width with Account in detail, truncated Context using the approved tooltip, and preserve sorting, search, accessibility, and responsive behavior. Task #4 source request/screenshots report a confusing count mismatch: `/logs` “Active agents” peaks/current around 618–717, `/agents` reports 212 live, while `/sessions` host e14 reports Live 25, Idle 18, Reconnectable 98, In ledger 712. Determine whether hundreds currently run on the laptop; expose exact definitions, scopes, freshness, raw-record membership, and accurate reconciled counts. Never call historical records active agents. Make labels and adjacent explanations self-explanatory. Show calendar dates on chart X axes when ranges cross dates.

## Outcome

Overdeck shows what work ran, where, when, and with what result across Factory, harness, Claude, CDX, bots, builds, Git, incidents, and k3s/SSH nodes. Owner can filter, search, chart, drill into one run, inspect logs, and attach to its terminal.

## Acceptance criteria

- Inventory every authoritative event/log source before UI work.
- Normalize event identity, project, category, host/node, session/run, timestamps, lifecycle, result, and evidence link without fabricating missing fields.
- Use `@platform-modules/ui-primitives` `DataTable` for searchable/sortable activity rows and existing platform primitives elsewhere.
- Row click opens run/session detail with individual logs and lifecycle evidence.
- Charts support 1h, 1d, 3d, 7d, 30d, all-time; project/category filtering; active-agent, log-volume, commit, bot, build, and incident activity.
- Cluster view maps sessions/builds/agents to nodes and links to agent detail and tmux attach.
- Notification suppression is not an incident. Record underlying event; do not harass owner with “notification failed” noise.
- Both themes work; report and product surfaces do not regress to light-only.
- Production is browser-proven with honest live data.
- `TimeSeriesChart` renders meaningful labeled X time and Y numeric axes. Every point exposes a timestamp/value tooltip by pointer hover and keyboard focus.
- Requested activity makes useful metrics, latest value, source, and gaps primary; coverage is subordinate. Unsupported series stay honest and visually secondary.
- Cached `/logs` data renders immediately. One React Query stale-while-revalidate lifecycle refreshes lazily without duplicate timers or stale falsehood; collector source/index caches avoid repeat corpus scans.
- Activity events fit the viewport without horizontal body scrolling. Columns prioritize time, source, category, severity, compact context, duration, result, and actions. Account moves to detail. Existing approved tooltip exposes full truncated context. Sorting, search, keyboard access, and mobile behavior remain intact.
- Measure current API latency and bottleneck before caching changes; record deterministic before/after results.

## Preserved context

- Existing report: `/home/user/.local/share/overdeck/reports/2026-08-08-session-report.html`.
- Earlier activity/log/session UI tasks may contain reusable delivery; recover before rebuilding.
- Existing Prometheus/node_exporter/Grafana stack supplies host metrics but does not replace agentic event semantics.
- `live-report-sync` owns current status truth. This plan owns product observability and drill-down UX.

## Constraints

- UI MUST obey `.claude/skills/od-ui-dev/SKILL.md` and use platform primitives first.
- Do not dump raw logs as the primary UX.
- Do not claim observability from registration, suppression, or stale task labels.
- Keep SSH and k3s execution sources visible during incremental migration.
- This plan must not block emergency offload or Factory restoration.

## Execution steps

1. Recover prior observability brainstorm, shipped components, schemas, and task receipts.
2. Inventory authoritative event sources and retention/query limits.
3. Write normalized event/detail/time-series contract.
4. Implement ingestion and deterministic correlation before presentation.
5. Build DataTable activity view, detail drill-down, charts, node/session map, and terminal attach links from reusable components.
6. Verify source counts, lifecycle transitions, filters, theme behavior, and browser paths.
8. Measure current `/activity` latency and source-read cost; identify the bottleneck before changing cache behavior.
9. Inspect the four supplied screenshots and current chart, logs UI, query lifecycle, source descriptors, readers, and caches.
10. Add chart axes and accessible point tooltips; redesign requested-series hierarchy and unsupported states.
11. Connect existing authoritative sources where descriptors and retained evidence support them; leave unsupported metrics explicit and secondary.
12. Implement cached-first lazy refresh with one React Query lifecycle and reusable collector source/index caching.
13. Compact the activity `DataTable` using existing primitives and tooltip; move Account into detail while preserving table behavior.
14. Run deterministic latency checks, full collector/deck-ui/web gates, diff checks, configured slop gate, and both-theme remote browser proof with screenshots.
15. Commit locally, integrate `origin/main`, land through the canonical ship orchestrator, deploy, verify installed behavior, and record receipts.

## Task #4 acceptance delta — count semantics and date-safe charts

- Audit authoritative schemas, producers, installed API responses, and raw record IDs behind `/logs`, `/agents`, and `/sessions`; record units, host/fleet scope, freshness, historical/current meaning, lifecycle boundaries, duplicate/session aggregation, and open-ended interval semantics. Labels are untrusted until traced.
- Build one deterministic reconciliation fixture mapping raw record IDs to every displayed count. Assert set relations and identities across current running agent processes, live terminal sessions, idle/reconnectable tmux sessions, and historical ledger records.
- Fix calculation defects. Active intervals MUST use lifecycle evidence and half-open boundaries; unknown ends MUST NOT remain active indefinitely. Never count sessions as agents or call retained history “active agents.”
- Rename every count to its precise domain term. Add concise adjacent explanations covering what, scope, source, freshness/time window, and why counts differ. Link headline counts to exact filtered records when an existing route supports it.
- `TimeSeriesChart` X ticks MUST include calendar date plus time when the visible range crosses a local calendar day, use compact/date-only labels for longer ranges, state locale/timezone honestly, and expose absolute date/time in tooltips. Ambiguous same-time endpoints across dates are forbidden.
- Add tests for cross-midnight labels, current-versus-history semantics, raw-ID reconciliation, count invariants, lifecycle/open-end handling, and visible explanation text.
- Run collector, deck-ui, and web tests/typechecks/build with clean warnings. Run remote browser proof against real APIs in both themes, inspect screenshots, and reconcile displayed numbers to raw source IDs.
- Commit without co-author, integrate `origin/main`, land through canonical ship wrapper, deploy, verify installed responses/UI, update installed report, then mark task #4 and this plan DONE.
- Browser-proof correction delta (2026-08-14): match the proof explanation assertion to the actual visible lifecycle explanation rather than invented wording. The suspected `/logs?event=` hydration defect was disproved: the unchanged installed component opened the exact-record dialog in the fresh real-API run because the route supplies the selected event server-side. The earlier timeout was timing-related; do not add redundant client fallback state or an unrealistic direct-render regression.
- Delivery-gate correction delta (2026-08-14): retain the complete 830-test collector suite while bounding its runner to one file worker and one concurrent test. The full candidate gate reproduced a 5-second runner timeout in `malformed status after success retains last-good snapshot` under fleet contention even though the same deterministic case completes in milliseconds alone and completed in 10.39 seconds in the contended run. Restore the suite's established 20-second per-test ceiling; do not skip, isolate, or weaken the regression. Re-run the exact full candidate gate cleanly before landing.
- Web-gate correction delta (2026-08-15): keep every web test and constrain Vitest to one file worker. Exact-source focused rerun proves two deterministic `HooksContent.test.tsx` failures. Resetting mock implementations did not change the result: the initial successful controls query effect can run after a failed mutation and erase its visible error. Mutation start and success already own error clearing; remove the competing query-success effect, preserve the reset-based test isolation, then rerun focused, static, and exact integrated full gates. Do not skip files, retry assertions, or raise assertion timeouts.
- Post-rebase collector correction delta (2026-08-15): the exact full gate now has one deterministic failure in the limit-clamping test: expected 2,505 events but received 2,514. The fixture overrides nine older sources but omits the newly registered `tool-suggest` source, so the test reads nine real retained records from the build host. Isolate the fixture by overriding every registered source, retain the exact 2,000 clamp and ordering assertions, then rerun the focused activity test and exact full gate. Do not change the production limit or accept host-dependent totals.

## Preserved WIP

- Required worktree/ref: `/home/user/Projects/overdeck/.worktrees/observability-count-semantics`, branch `wt/observability-count-semantics` tracking `origin/main`.
- Preserve all unrelated shared-checkout and plan-registry WIP. Reuse landed observability contracts, `TimeSeriesChart`, routes, and tests; no new generic primitive.

## Task #4 execution steps

1. Capture installed API responses and raw source IDs before changes.
2. Trace schemas, producers, interval construction, classifiers, fleet counts, and ledger aggregation.
3. Add deterministic failing reconciliation/date-label tests.
4. Repair calculations and precise labels/explanations/drill-down with existing components.
5. Run focused then full clean gates.
6. Run `e2e-remote` real-API proof in both themes; inspect screenshots and numeric reconciliation.
7. Commit, integrate latest main, canonical ship, deploy, verify installed values, update report and receipts.

## Task #4 current receipt

2026-08-15: DONE. Collector limit fixture now overrides all ten registered sources, including `tool-suggest`, and the exhaustive TypeScript source map prevents future host-data leaks. Final source landed at `2eca4d14a81944c91b51991c32e25071d081b12c` and deployed from the exact landed revision. Installed web, collector, and watchdog services are active; `/logs`, `/agents`, `/sessions`, and `/cluster` return HTTP 200.

Exact final gates: collector 855 passed with 2 intentional skips; deck-ui 429/429; web 445/445 across 49 files; production build completed; integrated Factory suite 194/194. Astro reported two pre-existing deprecation hints in untouched `FileIncidentForm.tsx`; Rollup reported third-party Lexical annotation messages from generated package files. Neither signal originates in this change, and project diagnostics contain no error or warning.

Fresh real-API proof ran through installed `e2e-remote` on registry host `debian1`; remote Chromium exited 0 with no workstation fallback, console errors, request interception, or action requests. Both themes passed. Response-bound snapshots reconcile 193 current operating-system agent processes, 17 working terminals, 13 idle terminals, 144 reconnectable sessions, and 2,555 retained ledger records. These are distinct domains captured at their stated timestamps; retained ledger records are not active agents. `/logs` labels the 48-point historical series “Concurrent completed sessions”; unfinished ledger intervals are excluded. Long-range ticks show compact browser-local dates, timezone/locale disclosure is visible, and point labels expose absolute ISO timestamps.

All fresh artifacts were inspected under `apps/web/browser-proof/agent-observability-live/results/`. Cluster response status is 200: three ready nodes, zero workloads, four source states, and 2,463 records explicitly shown as unplaced rather than running agents. Sidebar process count 194 and `/agents` proof count 193 came from different response times; each page reconciles to its own response.

## Next executable action

None — task #4 is landed, deployed, installed, and browser-proven.

## Latest acceptance delta

2026-08-13 wire-contract acceptance delta: remove duplicated collector public wire definitions. Collector MUST import/re-export shared `@overdeck/activity-contract` response/event/coverage/series types directly; raw source-reader types may remain collector-internal. Re-open `collector/src/activity/wire-contract.ts`, `collector/src/activity/types.ts`, and `packages/activity-contract/src/index.ts`; run exact collector typecheck and record command/output. Standalone JSX/module-resolution diagnostics are non-authoritative only if web project typecheck remains clean. Do not spawn/delegate, commit, deploy, or run browser.

2026-08-13 final-review acceptance delta: fix all nine current defects. (1) Enforce half-open interval ordering when one session ends exactly as another starts; simultaneous boundary MUST NOT peak at 2, with exact regression test. (2) Derive series coverage from authoritative retained coverage, NEVER first observed event; distinguish quiet measured time from unknown gaps and test both. (3) Generic agent-session exit MUST end with unknown result unless producer supplies typed outcome. (4) Kubernetes lifecycle MUST use authoritative transition/phase fields only, NEVER resource/title text; successful job named `failed-migration` MUST remain success. (5) `TimeSeriesChart` MUST split SVG paths at declared gaps and test no connecting segment. (6) Chart interactive targets MUST share SVG coordinate space/layout under narrow scrolling and remain accessible, with test. (7) `/sessions?session=` MUST robustly consume identity on load/reload/history and select exact session, or route to an existing exact-record destination; test reload/history. (8) Browser proof MUST fail closed when required charts, facets, evidence, session, or terminal capability is absent; honest live absence MUST use explicit capability-aware unavailable assertions, NEVER silent conditional skip. (9) Shared `ActivitySourceCoverage` MUST own `storage` and `queryBounds`; collector and UI MUST use/render required source inventory and caveats through strict shared contract ownership. Run all full gates cleanly. Do not spawn/delegate, commit, deploy, or run browser.

2026-08-13 coordinator acceptance delta: complete approved reusable deck-ui `TimeSeriesChart` with actual points, truthful coverage/gaps, accessible states, token-only light/dark themes, colocated tests, barrel export, and `/design-system` gallery registration; use it in `LogsContent`; replace standalone query-cache last-good test with rendered `ClusterContent` failed-refresh coverage through production query path; complete real-proxy browser-proof script interactions for facets, evidence, cluster, sessions/tmux, themes, refresh cadence, and console cleanliness without interception, but MUST NOT run browser. Run full collector suite/typecheck, deck-ui tests/typecheck, focused web tests, web build/typecheck, and diff checks. Do not spawn/delegate, commit, deploy, or run browser.

2026-08-13 frozen-review corrections: canonical active intervals MUST be built independently of display filters and requested endpoint clipping; reusable source snapshots/indexes MUST serve changing sliding bounds and filters without rescanning; session finish reason MUST distinguish cancellation, failure, success, and unknown; action records with no authoritative outcome MUST remain unknown; `projectCounts`, `series`, and `errorCount` MUST remain optional for rolling deployment; URL hydration/history tests MUST cover category, severity, grouping, search, project, range, and selected record; aggregate and source drawers MUST reuse normalized detail rows; direct collector-state last-good coverage and promised rendered failed-refresh coverage MUST be truthful; real-proxy browser harness MUST cover all acceptance interactions, themes, request success, and console cleanliness without interception, but MUST NOT run yet. Synchronize plan/index next action after full gates.

2026-08-12 stable review: use one shared workspace activity wire contract; one source normalizer for aggregate and entries; select exact source records before evidence navigation; derive active-agent concurrency from canonical ledger intervals; expose commits, bots, builds, and incidents only from authoritative descriptor capabilities or mark unavailable; build proof from the real collector or a shared fixture builder; remove category/project facet duplication with a local generic `FacetButtonGroup`; preserve rolling-deploy compatibility; make URL state complete and evidence navigation same-origin and allowlisted; keep sliding windows fresh; remove the duplicate error request; add deterministic direct last-good fleet state coverage; remove sensitive and duplicate proof artifacts; run all requested gates without commit, deploy, or browser execution. Measured load acceptance: replace two `/activity` polls per 15 seconds with one response carrying the error total, and cache or index source reads so repeated consumers do not rescan the full corpus; deterministic tests MUST prove one activity poll and reuse of source reads. Baseline lower bound is about 168 MB parsed and 61,000 file opens per minute.

## Current receipt

2026-08-13 final-review repair complete in `/home/user/Projects/overdeck/.worktrees/agent-observability-final`; no browser, commit, or deploy. Active-agent sweep now orders ends before starts at equal timestamps and exact half-open regression peaks at 1. Count series require explicit authoritative `coverageFrom`; healthy retained quiet ranges emit measured zero while unknown retention is unavailable. Generic session exits complete with unknown result unless producer emits exact typed success. Kubernetes classification reads only authoritative `detail.phase/status/reason`; successful `failed-migration` title remains success. `TimeSeriesChart` splits paths across declared gaps and places accessible hit targets inside the same SVG coordinate system as visible marks. Persistent session terminal consumes `?session=` after client hydration, follows `popstate`, writes identity on open, and clears it on close; reload/history test passes. Browser proof has no silent capability skips: required charts/facets/evidence/session/terminal paths either execute or assert explicit response-backed unavailable/zero states. Shared `ActivitySourceCoverage` requires `storage` and `queryBounds`; raw source coverage remains internal, normalized aggregate/source-entry responses strictly own required metadata, and Logs renders both inventory caveats.

Landed observability candidate reached `origin/main`; automatic deployment stopped safely because runtime Pi authentication at `modules/workstation/pi/agent/auth.json` is preserved inside the deploy clone but was not excluded from source cleanliness checks. Follow-up worker uses clean landed worktree `/home/user/Projects/overdeck/.worktrees/disk-admission-land`; packaging fix adds exact private exclude without reading, moving, or deleting credentials. Deployment regression suite passes 30/30. Deployment, installed proof, report, DONE receipt, and final land remain pending.

Verification: exact collector regressions 20/20 pass; full collector suite and TypeScript pass. Chart regressions 4/4 pass; full deck-ui suite and TypeScript pass. Focused sessions/logs 40/40 pass; full web suite passes; production web build passes; web typecheck 221 files with 0 errors, 0 warnings, 0 hints. Browser-proof source parses without execution. `git diff --check` passes. Build initially caught server-side `window` access in session URL hydration; moved hydration into client effect and reran affected test/build/typecheck cleanly.

Test and land exact runtime-auth exclusion, deploy landed main, verify installed observability routes/endpoints, write installed report, then set and land `DONE`.

2026-08-13 wire-contract correction complete in `/home/user/Projects/overdeck/.worktrees/agent-observability-final`; no browser, commit, or deploy. `collector/src/activity/types.ts` now imports and re-exports public events, coverage, response, series, skipped-entry, and suppression types directly from `@overdeck/activity-contract`; only raw source-reader coverage, reader options, and collector query options remain local. `collector/src/activity/wire-contract.ts` directly re-exports shared response types instead of checking structural assignability. Exact collector command `cd /home/user/Projects/overdeck/.worktrees/agent-observability-final/collector && bun run typecheck` passed with output `$ tsc --noEmit`. Authoritative web command `cd /home/user/Projects/overdeck/.worktrees/agent-observability-final && pnpm --filter web typecheck` passed 221 files with 0 errors, 0 warnings, and 0 hints; runner additionally reported one package-install warning without warning text, outside Astro diagnostics. `git diff --check` passed with no output.

2026-08-13 follow-up acceptance delta: direct-re-export `ACTIVITY_SEVERITY_ORDER` and `ALL_ACTIVITY_CATEGORIES` from `@overdeck/activity-contract`; MUST NOT import these values into collector local scope. Preserve collector public API. Rerun exact collector typecheck and `git diff --check`. Do not spawn/delegate, commit, deploy, or run browser.

2026-08-13 follow-up complete: `collector/src/activity/types.ts` now direct-re-exports `ACTIVITY_SEVERITY_ORDER` and `ALL_ACTIVITY_CATEGORIES` from `@overdeck/activity-contract` without importing either into local scope, preserving the collector public API and eliminating both no-unused diagnostics. Exact command `cd /home/user/Projects/overdeck/.worktrees/agent-observability-final/collector && bun run typecheck` passed with output `$ tsc --noEmit`; `git diff --check` passed with no output. No spawn/delegation, commit, deploy, or browser run.

2026-08-13 delivery acceptance delta: audit candidate scope; create one local commit with short imperative message and no co-author; run prepared real-proxy browser proof through `~/.claude/bin/e2e-remote` with server and browser both remote, server bound to `127.0.0.1`, no interception; inspect every proof artifact/result; fix defects or warnings and use a new commit for follow-up changes. MUST NOT land or deploy.

2026-08-13 delivery complete through local commit and live proof; MUST NOT land or deploy. Base observability commit `8be2f6c9e`; browser-proof follow-up commits `2590dfbbd`, `f7e181299`, `28829269f`, `8378ba93a`, `68c760029`, `a7974b0eb`, `b4cd492b4`, `ed7127a70`, `371cdf3e6`, `7db05011c`, `732a8a841`, `0dcd27376`, `c5bf814d4`, `e4cec6cd8`, `7d799a9af`, `450e38b46`, and `9840712ab`. Final exact proof command used `~/.claude/bin/e2e-remote` with remote pnpm bootstrap, collector plus Astro server bound to `127.0.0.1:31337`, real server-side collector proxy, no interception, and remote Chromium; server ready after 3s on `debian1`, client exit 0. Inspected all seven artifacts under `apps/web/browser-proof/agent-observability-live/results/`: `logs.png`, `cluster-arrival.png`, `cluster-arrival.txt`, `cluster-response.json`, `cluster.png`, `sessions.png`, `receipt.json`. Receipt: 1h events 0; nine source inventories; active-agent series available with 48 points; remaining five series explicitly unavailable; exact request list includes filter/range polling and no action requests; cluster returned honest 503 `cluster-snapshot-unavailable` and rendered visible error; sessions rendered zero reconnectable with host coverage; only console error exactly matched cluster 503. Full follow-up gates passed: collector `bun run typecheck && bun test --timeout 20000`; web `pnpm --filter web typecheck && pnpm --filter web build` with 0 errors/warnings/hints; deck-ui `pnpm --filter @overdeck/deck-ui test && pnpm --filter @overdeck/deck-ui typecheck`; `git diff --check` clean.

2026-08-13 final-delivery acceptance delta: reproduce `SourceEntriesContent` implicit-any diagnostics only with authoritative project check; MUST NOT alter code for standalone JSX noise. Fetch and integrate `origin/main` safely. Rerun all affected clean gates and `git diff --check`. Land ONLY through `.claude/scripts/ship.sh`; then run `packaging/deploy-local.sh`. Verify installed `/logs`, `/cluster`, and collector endpoints. Use `e2e-remote` for browser+server pair. If live cluster remains unavailable, verify visible unavailable state and MUST NOT claim links proven. Write installed report `/home/user/.local/share/overdeck/reports/2026-08-10-agent-observability.md` with exact receipts. Set plan/index `DONE` only after installed proof; commit, land, and deploy final receipt updates. Mark task #1 complete only then.

2026-08-13 final-delivery receipt: authoritative web typecheck passed 221 files with 0 errors, 0 warnings, and 0 hints; no standalone JSX/module noise changed source. Observability and runtime-auth preservation landed through canonical `.claude/scripts/ship.sh`; served implementation release identifies commit `b8941c371b6520d795d72b1c2b569871e2e1db47`. Canonical deployment completed far enough to promote that release and restart installed services. `overdeck-web.service`, `overdeck-collector.service`, and `overdeck-web-watchdog.service` are active with successful results. Installed `http://127.0.0.1:31337/logs` and `/cluster` return HTTP 200. Installed `/api/collector/activity` returns HTTP 200 with 500 retained events and nine coverage sources. Installed `/api/collector/cluster` returns HTTP 200 with authoritative cluster, node, source, workload, build, and session fields. Earlier remote real-proxy browser proof remains valid and inspected all seven artifacts with no interception or action requests; its cluster request honestly returned 503 and visible unavailable state at proof time, so that run did not claim cluster-link navigation. Installed cluster data became available during final verification; endpoint availability is proven, but cluster-link browser navigation was not rerun or claimed. Installed report: `/home/user/.local/share/overdeck/reports/2026-08-10-agent-observability.md`.

Next executable action: none — task #1 is complete.

## 2026-08-13 `/logs` UX correction receipt

Task #2 is ACTIVE in preserved worktree `/home/user/Projects/overdeck/.worktrees/logs-observability-ux`. Source request and acceptance delta are captured above. Inspected the current `/logs` browser proof, mandatory UI contract, deck-ui barrel/gallery, `TimeSeriesChart` and tests, `LogsContent` and tests, React Query activity hook, and collector activity reader/registry/cache. Baseline installed `/api/collector/activity?range=1d`: HTTP 200, 485,201 bytes; first request 7,985.27 ms, with repeated requests still taking multiple seconds. The collector already retains normalized source snapshots for 20 seconds, but each request still filters, aggregates, builds series, and serializes the large event set. The UI also had two independent 15-second refresh mechanisms: a React Query poll and a component timer that changed `from`/`to`, producing fresh query keys.

Implemented chart plot margins, labeled numeric Y ticks, labeled time X ticks, and axis titles while preserving pointer-hover and keyboard-focus exact timestamp/value tooltips. Chart package tests pass. Reworked activity query ownership so one React Query 15-second stale-while-revalidate cycle keeps a stable cache key, renders cached data immediately, computes fresh sliding bounds only inside the fetch, and removes the duplicate component timer. Web typecheck passes with 0 errors, 0 warnings, and 0 hints; the remote runner separately reported one package-install warning without warning text, which must be rechecked in the final gate. No source availability, requested-series hierarchy, collector derived-index optimization, activity table redesign, browser proof, commit, land, deploy, or installed proof is complete yet. Unrelated WIP remains untouched.

Task #2 delivery candidate complete through post-merge gates and browser proof in `/home/user/Projects/overdeck/.worktrees/logs-observability-ux`. Source audit found no registered authoritative descriptors for commits, bots, builds, or incidents; these remain honest compact unavailable metrics. Agent sessions and retained Kubernetes snapshots support log volume where their coverage is healthy. Chart axes and pointer/keyboard point tooltips are visible and accessible. Requested metrics now make value, latest, source, and gaps primary responsive cards with coverage subordinate; unsupported metrics no longer dominate. Activity events remain the approved `DataTable`, compact to time/source/category/severity/context/duration/result/actions; Account remains searchable and appears in detail only, and Context uses the approved full-text tooltip.

React Query now owns one stable-key 15-second stale-while-revalidate lifecycle; cached data renders immediately and fresh sliding bounds are computed only during fetch. Collector snapshots sort and timestamp-index normalized source events once per deterministic 20-second cache generation, then binary-slice each requested window before filtering. Installed baseline `/api/collector/activity?range=1d`: 7,985.27 ms first response, 485,201 bytes, repeated requests still multi-second. Candidate direct-reader measurement: 1,488.34 ms cold; warm samples 180.97, 210.13, 159.66, 161.44, 122.63, 118.03, and 127.96 ms; warm median 159.66 ms.

Verification: collector full suite/typecheck pass; deck-ui 406 tests/typecheck pass; web 395 tests, 0-error/0-warning/0-hint typecheck, and production build pass; post-merge collector/deck-ui/web gates pass; `git diff --check` clean. Real collector + Astro + remote Chromium proof passes on Debian1 with no interception; final screenshot inspected after correcting two hierarchy defects. Commits: `8c5d13ef3`, `caa8f3b42`, `e7bd4cc13`, `3412d8afc`; latest `origin/main` integrated cleanly. Build fleet repair restored canonical runtime links on Debian1/2; both remote doctors pass fully. Debian3 remains low disk after the canonical build sweep retained 54 GiB, but no longer blocks supported capacity. Gate tooling emitted one existing warning that ast-grep uses an unpinned PATH binary and remote install metadata reported one warning without text; project diagnostics remained clean.

Task #2 DONE. `/logs` UX candidate and deployment-order repair landed and deployed at `2f2ae0e57780ca0d31aa6871061a5c04e940bf04`. Full deploy exited 0 from pristine alternate deploy clone with canonical global lock; unrelated canonical deploy-clone WIP remained untouched. `overdeck-web.service`, `overdeck-collector.service`, and `overdeck-web-watchdog.service` are active/running/successful. Installed `/logs` HTTP 200: 16,237 bytes, 24.73 ms cold, 0.85 ms warm median. Installed `/api/collector/activity?range=1d` HTTP 200: 500 events, nine coverage sources, 511,746 bytes; baseline cold 7,985.27 ms, final cold 5,829.71 ms, final seven-sample warm median 1,498.26 ms. Direct reader remains faster at 1,488.34 ms cold and 159.66 ms warm median; installed HTTP overhead is dominated by serializing the retained 500-event payload. Active agents is available. Log volume, commits, bots, builds, and incidents remain honestly unavailable because no complete authoritative source is connected. Installed report updated at `/home/user/.local/share/overdeck/reports/2026-08-10-agent-observability.md`. Next executable action: none.

## Recovered lost-session receipt — 2026-08-12

Ground truth transcript: `/home/user/.claude/projects/-home-user-Projects-overdeck/38549e85-6aeb-4b67-a9ce-8fe2e649b559.jsonl`. Session closed cleanly after substantial continuation not reflected in earlier handoff.

- Implementation commits `14cb92ee6` and `fcd1baa3d` are ancestors of `origin/main`; observability backend and UI were landed rather than lost.
- Current integration worktree: `/home/user/Projects/overdeck/.worktrees/agent-observability-final`.
- Final independent review replaced earlier low-priority normalization suggestion with two required corrections:
  1. `collector/src/activity/read.ts:127`: generated “Open evidence” URL uses dead deep-link state. Build functional evidence navigation and test it.
  2. Event normalization derives lifecycle/result only from exit code. Preserve explicit source-owned lifecycle/result for denied actions, Kubernetes transitions, and agent sessions without exit codes; add focused behavior tests.
- UI review additionally requires shared test setup and focused behavior tests. Never accept visual-only proof for evidence links or filters.
- Preserve already landed source. Fix only verified residual findings; do not rebuild completed observability work.


Browser verification requires a supported `e2e-remote` input transport for uncommitted worktrees, or a committed candidate ref. User explicitly prohibited commits. Current wrapper cannot verify this candidate as framed.

## Historical blocked receipt

Remote build infrastructure was fenced (`epoch-mismatch` / `runtime-missing` on all three buildboxes), so that earlier gate stopped after collector and deck-ui suites rather than silently falling back to the laptop. This blocker was later cleared; current receipt above supersedes it.

## 2026-08-13 correction receipt

- Fixed source-owned outcomes: denied actions now remain failed/failure even with exit code 0; agent start and finish records emit explicit lifecycle/result instead of becoming unknown. Focused source tests cover both paths.
- Fixed evidence navigation end to end: evidence URLs target `/logs/<source>?event=<record>`, source pages read the URL selection, query the exact record across all retained history, reset paging for exact selection, and open the existing detail drawer. Collector, client serialization, and UI initial-state tests cover the contract.
- Activated existing shared Vitest setup (`apps/web/src/test-setup.ts`) across web tests by removing redundant per-file matcher imports. Focused web result: 49 passed.
- Focused collector result: 26 passed. Full collector and deck-ui runs advanced cleanly until the fleet fence stopped the combined gate; no failure was ignored or treated as green.
- Full completion remains pending because the mandatory remote gate and remote browser proof cannot run while every registered buildbox fails seat audit with `runtime-missing`.