# Oracle-log → LLM-interpretation MEASURED — k=3 blind Opus did NOT land the canonical C02 (it INVERTED)

audience: AI coding agents first. Optimize for activation, not prose.

**BLUF: the LLM-interpretation layer made recall STRICTLY WORSE than the oracle alone.** The oracle-ALONE flags
BOTH gaps — so it already surfaces the real C02 (`owner|referee`) as one of its two flags. Feeding that log to Opus
to "confirm the catch" turned the true-positive into a **confident false-negative, 3/3**: every roll called the
real C02 a FALSE POSITIVE and the upstream-mediated over-flag (`referee|referrer`) REAL. So bolting an LLM onto the
log did not convert a flag into a catch — it **deleted a catch the oracle had**. The oracle's deterministic
structural extraction is CORRECT (its asymmetry flag matches the real fix `c2d8b4195` exactly). This answers the
standing user question "did Opus interpreting the C02 oracle log help? still missing?" — **measured: it did not
help; it regressed recall; n=3, systematic.**

## Why this measurement exists

The user's two binding challenges:
1. **"flagged ≠ caught"** — a real catch is a confirmed/fixed bug, not a flag buried in noise. Aim = 100% real
   catch in 1 go.
2. **"did u try opus for the interpretation of the C02 oracle log?"** — proposes the oracle does the deterministic
   structural extraction, then an LLM interprets the LOG (an easier task than finding the bug in raw code) to
   produce a precise confirmed catch.

This is the first MEASURED test of that pipeline. Prior "[MEASURED] no unseeded LLM caught C02" was LLM-over-CODE;
this is LLM-over-LOG.

## Experiment (blind, no-answer-key)

- **Input to each judge:** the C02 cell `service_vuln.ts` with all answer-leaking comments STRIPPED (the header
  docblock + the `// BUG (C02 …)` line removed), inline; PLUS the full oracle log (EXTRACTED + the two FLAGS gaps);
  PLUS principal glossary (owner=vendor owner, referee=buyer, referrer=affiliate/payee). No repo access, no answer
  key, no canonical.json.
- **Task:** classify each of the two FLAGS gaps REAL/FP with reasoning; name the single most serious real bug.
- **Model:** Opus (the user asked specifically about Opus). **k=3** independent rolls, identical neutral prompt.

## Ground truth (the real fix, RAW git)

`c2d8b4195` title: *"fix(afill-C02/C09): block commission + referral_reward when buyer is the deal's vendor owner."*
The diff to `accrueAffiliateCommission`:

```diff
-    if (dealVendor && isSelfVendorPurchase(ref.referrerUserId, dealVendor.ownerUserId)) {
+    if (dealVendor &&
+        (isSelfVendorPurchase(ref.referrerUserId, dealVendor.ownerUserId) ||
+         isSelfVendorPurchase(args.refereeUserId, dealVendor.ownerUserId))) {
```

The fix added EXACTLY the `owner|referee` guard (buyer == vendor owner). So **`owner|referee` IS the real C02**;
`referee|referrer` is the over-flag (mediated UPSTREAM by `isSelfReferral`, cross-file, not inside `accrue`). This
matches `2026-06-18-oracle-xfile-c02-catch.md`.

## Results — 3/3 INVERTED

| roll | GAP-1 `referee\|referrer` (truth: FP) | GAP-2 `owner\|referee` (truth: REAL C02) | named TOP BUG |
|------|----------------------|----------------------|---------------|
| 1 | REAL | FP | cap never decremented |
| 2 | REAL | FP | cap never decremented (+ holdDays instant-maturity) |
| 3 | REAL | FP | cap never decremented |

Every roll inverted BOTH gaps and crowned a non-canonical bug. The oracle's job (deterministic asymmetry
extraction) was done correctly on every run; the LLM interpretation systematically mis-labeled which gap is real.

## Diagnosis — it is the LENS + missing context, NOT Opus capability

- **`owner|referee` → FP (wrong):** all 3 applied a narrow **self-enrichment / money-flow lens** — "the payee is
  the referrer; vendor-owner-as-buyer routes money to a *distinct* third-party affiliate, so no improper
  enrichment of the payee." That reasoning is LOCALLY VALID for a pure self-payment definition. The real bug is a
  **business anti-abuse POLICY** ("no affiliate payout when the buyer owns the vendor" — a wash-trade/collusion
  vector) that exists in NO pre-fix code, only in the fix's intent. The judges could not see the policy, so they
  rationalized the asymmetry away. **The symmetry signal was in the log** (the code already guards
  `referrer==owner`; it omits the symmetric `referee==owner`) — the wrong lens dismissed it.
- **`referee|referrer` → REAL (wrong):** the mediation (`isSelfReferral`) is UPSTREAM and CROSS-FILE — invisible
  in the isolated cell. Defensible from the cell alone; wrong against ground truth.

Both inversions are **missing-information**, not bad reasoning — and the two gaps are missing *different* kinds of
information, only one of which is code-derivable at all:

- **GAP-1 (`referee|referrer`)** *looks* like self-enrichment and IS — single-file. It is an FP ONLY because
  `isSelfReferral` mediates it **cross-file**. Demoting it needs **resolution** (the resolver bundle), not a lens.
- **GAP-2 (`owner|referee`, the real C02)** *does not* look like self-enrichment — the payee is a distinct third
  party, so the judges are CORRECT by that lens. It is a bug ONLY because of a **business policy** ("no affiliate
  payout when the buyer owns the vendor" — a collusion/wash-trade vector) that exists in **no pre-fix code**, only
  in the fix commit's intent. **GAP-2's bug-ness cannot be derived from the code by any lens.** Three independent
  sharp reasoners converging on the same defensible-but-wrong answer is evidence the information is **not in the
  input**, not that the model needs prodding.

**Why a "symmetry-break lens" is a DEAD path, not the fix.** A lens that says "incomplete symmetric mitigation →
flag REAL" only tells the LLM to reproduce the oracle's structural flag — which already exists, deterministically,
for free. It would flip GAP-2 to REAL, but on structural *suspicion*, not a confirmed exploit (= the oracle's
output with extra latency and a dice roll). And the SAME lens re-fires on GAP-1's identical incomplete-symmetry,
re-confirming the FP. With both fixes applied (lens + resolution) the LLM isn't interpreting — it's mirroring
oracle+resolver. Without them it is harmful (3/3 wrong). **There is no configuration of this pipeline where the
LLM-interpretation layer adds recall.**

## Cell-fidelity defect discovered (no-false-coverage)

All 3 judges crowned **`commissionOrdersRemaining` is read but never decremented** as TOP BUG. **This bug does NOT
exist in the real multideal code** — `c2d8b4195~1:apps/web/src/server/referrals/service.ts` decrements it:

```
.update(referrals).set({ commissionOrdersRemaining: ref.commissionOrdersRemaining - 1 })
```

The cell `service_vuln.ts` is a single-function reconstruction that OMITTED the decrement path → a **phantom bug
introduced by the reconstruction**. Impact:
- Does NOT affect the oracle's C02 catch test (`test_oracle_xfile.py` keys on the `owner|referee` self-deal sink,
  independent of the cap path) — that measurement stands.
- DOES confound the "name the top bug" sub-task of THIS experiment (drew all 3 judges to a phantom). The clean,
  robust result is the **gap-classification inversion** (3/3), which is judged independently of the cap bug.
- Action: either restore the decrement to make the cell faithful (and re-run `test_oracle_xfile.py` to confirm the
  oracle still flags `owner|referee`), or document the omission in the cell header. Tracked as a follow-up.

## Implication for "100% catch in 1 go"

- The oracle's DETERMINISM holds: same input → the same correct asymmetry flag, matching the real fix. No dice roll
  on EXTRACTION. **The oracle ALONE already surfaces the real C02** (as one of its two flags).
- But **extraction-determinism does NOT buy catch-determinism**, and the naive "LLM-confirms-the-flag" pipeline is
  worse than nothing — it REGRESSED recall (deleted a catch the oracle had, 3/3). So do NOT bolt an LLM onto the log.
- **What the data vindicates is the RATCHET, not interpretation.** A policy-class bug like `owner|referee` is caught
  by: a deterministic structural flag (the oracle has it, matches the fix exactly) **+ a one-time policy assertion**
  that promotes that flag to BLOCKING. The human/the fix supplies the policy once; thereafter it is a permanent
  deterministic detector — 100%, no dice. That is the **Prevent-band** direction
  (`docs/specs/2026-06-18-prevent-band-design.md`). **This measurement is the strongest argument FOR Prevent-band
  and AGAINST an LLM-over-log layer.**
- The one genuinely-open *deterministic* question is GAP-1 precision: can the resolver reach `isSelfReferral` so the
  cross-file bundle demotes the `referee|referrer` over-flag? That is `#36`/`#40` resolution territory — verify
  reachability BEFORE spending any more LLM rolls.

## NOT covered / open

- **Resolver reachability of `isSelfReferral` (the real next step).** The only deterministic discriminator between
  the two gaps. Verify whether `collect_deps` from a route that calls `accrueAffiliateCommission` reaches the
  attribution path that holds `isSelfReferral`. Not yet measured.
- **Lens-variant deliberately NOT pursued.** Reasoned dead (see Diagnosis): it reproduces the oracle and re-fires on
  the FP. If ever run, it is ONE roll with success criteria pre-defined — and the expectation is it merely mirrors
  oracle+resolver, adding no recall.
- **Cell-fidelity defects** (omitted `commissionOrdersRemaining` decrement; constant `getReferralSettings` →
  `holdDays` instant-maturity is likely ALSO a phantom; treat every judge "extra bug" from this cell as suspect).
  Cell header corrected to drop the false "faithful reconstruction" claim. **Do NOT re-run k=3** — the
  gap-classification inversion is robust to the stubs (the asymmetry the oracle reproduces is faithful).
- n=3 on ONE canonical (C02). Not a cross-class rate.