# Live-fire precision audit — 3 real repos, 24 enumerated findings (2026-06-19)

audience: AI coding agents first. BLUF. This is MEASURED precision on UNSEEN production code — not the controlled fix-commit corpus. Ground truth here = per-finding triage against the real code by an Opus fix-agent loaded with each repo's full context, with code citations.

## BLUF — the reliability answer
The LLM bands are NOT yet "deterministic, no-false-positive, agent-trustable." A fix-agent had to re-read the real code to separate true bugs from false alarms — the exact failure mode to avoid (agent reasons through false positives, re-scans anyway). BUT the gate caught 7 real, severe money bugs now fixed + tested. The deterministic oracle leg stayed correct (SILENT / UNRELIABLE, zero false positives) — validating the design split: **oracle = agent-trustable; LLM = triage-required.**

## Scope
Targets = the 6 files the sg-oracle-runs live scan flagged across trance-shop-israel, zync.is, platform. One Opus fix-agent per repo, each: load repo CLAUDE.md/AGENTS.md/.claude → triage every finding → fix only CONFIRMED on an isolated `sec-fix/gate-findings` worktree (no push/merge). Findings are k=3-roll LLM candidates (3/3 = flagged all 3 rolls, NOT proof).

## Results — 24 enumerated findings
| Repo | CONFIRMED-FIXED | FALSE-ALARM | NEEDS-HUMAN | branch / commits |
|------|----|----|----|----|
| trance-shop-israel | 3 | 2 | 2 | `sec-fix/gate-findings` (base 97e10590) — `3aa16c91`, `4b3e2b34`; astro check 0/594 |
| zync.is | 0 | 8 | 0 | none (0 fixes) |
| platform | 4 | 3 (+~7 1/3 tail) | 2 | `sec-fix/gate-findings` — `8e639d8`; typecheck clean, 52/52 |
| **total** | **7** | **13 (+tail)** | **4** | all isolated, none pushed |

Real-signal rate ≈ (7 fixed + 4 genuine-but-host-layer) / 24 ≈ 46%. False-alarm ≈ 54% — BUT 8 of 13 are ONE systematic gate defect (see #1). Exclude it → 5 false / 16 ≈ 31%.

### CONFIRMED-FIXED (7, real money/security bugs, now fixed+tested)
- trance refund.ts: malformed `request.json()` → silent full-scope refund (now 400). `3aa16c91`.
- trance webhook.ts: `subscription.renewed` resurrects a cancelled subscription by replay (now `ne(status,'cancelled')` scoped no-op). `4b3e2b34`.
- trance webhook.ts: hardcoded 30-day extension (now derived from row `billingCycle`, extends from `max(now,currentPeriodEnd)`). `4b3e2b34`.
- platform sumit.ts: `webhookSecret: undefined` → both-empty timing-safe compare fail-OPEN (now throws at construction). `8e639d8`.
- platform sumit.ts: negative `platformAmountMinor` → negative UnitPrice = live platform credit (now rejects). `8e639d8`.
- platform sumit.ts: partial-refund failure → retry double-refund (now `SumitRefundPartialFailureError` carries completed legs for resume). `8e639d8`.
- platform sumit.ts: platform vendor selected by POSITION not identity (now by `CompanyID`, position fallback). `8e639d8`.

### NEEDS-HUMAN (4, real but host-layer / contract decisions — NOT fixable in isolation)
- trance webhook.ts: body unauthenticated — orphaned receiver, NO sender contract exists on the wire (no HMAC to consume). Fabricating one = stub.
- trance webhook.ts: no idempotency on renewed — payload has no event-id/timestamp; needs a dedup table + a sender that emits a stable key.
- platform index.ts: non-atomic charge→ledger window (host-layer atomicity; module documents the DedupStore recovery model).
- platform stripe.ts: Connect transfer `destination` read from caller-supplied `metadata` — funds-redirect footgun; typed-field fix is a public-contract change.

## The 3 gate-improvement findings (the precision levers — MEASURED)
1. **Comment-vs-code blindness [#1 lever] — CLOSED (2026-06-19).** zync = 0/8 precision: ALL 8 findings sat inside `/* */` comment blocks in stub files; only the executable path returns a mock URL. The scanner read comments as live code. **RESOLVED by the band-1 prompt fix** (`baseline.prompt.txt`: scoped executes-only rule — "flag only code that EXECUTES; a defect SOLELY inside commented-out/illustrative code is NOT a finding" — paired with the mandatory `Code` anchor + the absence-class rule so absence bugs are NOT suppressed) **plus the navigable-emit citation-spine** (`gate.py`/`semantic_merge.py`: capture the quoted snippet, resolve the real line from the target file, emit a dedicated `code` field — d689fe5). Precision-hold MEASURED n≥3 on the two real zync stubs (`payment-gateways/{cardcom,stripe}.ts`, READ-ONLY): **0 comment-construct false positives** — the prompt did NOT flag the commented-out real-Cardcom call or the ACCEPTED-RISK future-work prose; every finding named an executing line (`docs/validation/2026-06-19-citation-prompt-recall.md` §Precision-hold). Recall NO-REGRESSION confirmed at the same record: 17/17 right-reason caught, every cell ≥ the bar floor, absence-class cells hold, controlled OLD==NEW at k=9 on the suspect cells (per-roll union), discriminator 17/17. The band-3 deterministic comment-stripper VERIFIER is **DEFERRED per delete-test** (the band-1 prompt fix recovers the catch the verifier would; building it now is YAGNI until a real residual proves the LLM cannot hold the line). OPEN, named non-goals: residual executable-stub over-flagging (the stub's own `log()`/`throw` lines are real executing code — a host-layer triage call, not a comment FP) and gate-defect #2 below (cross-file UNRELIABLE confidence).

   **Live-repo robust-fix branches (NEEDS-HUMAN / out-of-scope-real resolutions — isolated, UNPUSHED):** trance `36606dfe`/`c85d533f`; zync `04b1fb3`; platform `0587629`/`2179bde`/`542e980`.
2. **Cross-file UNRELIABLE, yet LLM still confident.** trance TOCTOU (3/3 critical) + itemIds IDOR (2/3) were FALSE-ALARM — the real guard (compare-and-swap `UPDATE … WHERE status IN(...)` + `payment_refunds.idempotency_key` UNIQUE; order fetched scoped by `userId` with items intersected) lived in an UNRESOLVED predicate import. The oracle correctly self-reported UNRELIABLE (no-false-clean WORKED) — but the LLM leg emitted confident criticals on code it could not see. The band-2 cross-file resolver (resolver #36) is the mechanism that would pull the guard into scope and suppress these. Until then, an LLM finding whose decisive predicate is in an unresolved import should be DOWNGRADED, not emitted at 3/3.
3. **Roll-count ≠ true-positive probability.** Several 3/3 findings were false alarms (all zync; trance TOCTOU). k=3 agreement measures model-consistency, NOT correctness. Confidence reporting must not imply truth-probability; right-reason verification is required.

## Design-split validation
The deterministic oracle leg produced ZERO false positives across all 6 files — SILENT with full resolution, UNRELIABLE on unresolved imports. This is the agent-trustable tier. The LLM leg is the value tier (caught 7 real bugs) but is triage-required, not block-on-emit. Confirms ARCHITECTURE §10 inv 8 (band-3 precision bar) and the no-false-clean contract.

## Out-of-scope reals surfaced (flagged, untouched)
- zync: stub payment gateways are wired into 2 LIVE routes (`payment-link.ts`, `portal/invoices.ts`) → production serves MOCK payment URLs. Documented `ACCEPTED-RISK (S9-i2-004)`.
- trance: webhook reads env via `process.env` (unreachable in CF Workers) vs refund.ts's correct `getRuntimeEnv(locals)`; add UNIQUE index on `refund_requests(order_id) WHERE status='pending'` for atomic queue dedup.