# Executable-stub severity-calibration — CANDIDATE (item #15, STAGED, NOT APPLIED)

date: 2026-06-20 · status: **CANDIDATE — not in live `baseline.prompt.txt`** · parent: `docs/specs/2026-06-19-finding-citation-and-comment-precision-design.md` §Non-goals · gate to apply: k≥3 measurement (below)

## BLUF
The residual ~50% over-flag class left open after the comment-FP close is **executable-stub over-flagging**: the baseline raises LOW/defensive-depth findings on *unwired stub files* (a `console.info` PII-log, a `terminalNumber` interpolated into a **returned mock URL**) as if production-live. This is **severity-calibration, not comment-blindness** (`...comment-precision-design.md:60`) and not suppressible — the stub's own `log()`/`throw` lines ARE executing code (`docs/validation/2026-06-19-live-fire-precision-3repos.md:37`). The fix is a **falsifier-first** stub-context *severity NOTE* (`docs/notes/external-prior-art.md:16`), NOT a withhold.

**Why this doc and not a live prompt edit:** the rule's entire mechanism is an LLM behavior change with NO deterministic core to conformance-test. Its recall effect is knowable ONLY by rolling, and this session does not roll (project doctrine: "LLM bands measured STATISTICALLY, k≥3 — NEVER a single-run claim"). Shipping it to the always-on production default unmeasured would be exactly that forbidden single-run change, on the *deployed* gate, blind. So the honest completion is: mechanism fully specified + the A/B cells that measure it + the apply-gate. Same shape as every measurement-blocked item; mirrors the comment-FP precedent (design → apply → measured k=9, `docs/validation/2026-06-19-citation-prompt-recall.md`).

## The candidate rule (exact text — to be inserted after `baseline.prompt.txt:82`, NOT yet applied)
```
- Stub-context severity calibration (NOTE, never withhold): when the file is a PROVEN stub —
  it returns a hardcoded mock/sentinel, throws "not implemented", or is otherwise unwired
  scaffold with no real data/secret/money flow reachable — a LOW or MEDIUM defensive-depth
  finding whose impact REQUIRES that wiring (e.g. a PII field logged that holds only mock data,
  an identifier interpolated into a RETURNED MOCK url) is reported with its severity calibrated
  DOWN one tier and an explicit "stub-context" note in the Trigger field. NEVER suppress it —
  report it calibrated. This calibration NEVER applies to, and the following stay at full rubric
  severity regardless of stub context: authentication bypass, fail-open auth/crypto, injection
  (SQLi/XSS/SSRF/command), IDOR/BOLA/cross-tenant, signature/replay bypass, secret/credential
  exposure, and any absence-class guard whose absence is itself the defect. A fail-open stub IS
  the defect; stub context never lowers it.
```
Design properties that bound the recall risk:
- **NOTE, never withhold** — preserves no-false-clean (`baseline.prompt.txt:83` absence-class rule is untouched; nothing is removed, only one tier of severity moved with a visible reason).
- **Never-calibrate allowlist** — covers the ENTIRE critical+high security surface (`baseline.prompt.txt:64-65` rubric) plus absence-class. The worst case the rule can produce is a LOW/MEDIUM defensive-depth finding (`:67`) carrying a context note — it structurally cannot touch a critical.
- **Scoped to PROVEN stub** — the trigger is a positive, checkable predicate (returns mock/sentinel, throws not-implemented), not "looks unfinished".

## A/B fixture pairs — the measurement cells (build at neutral path, RAW-git provenance if from a real fix)
| id | shape | expected under candidate | role |
|----|-------|--------------------------|------|
| A1 | mock-URL payment stub: `console.info(user.email)` + `terminalNumber` in a RETURNED mock url | finding KEPT, severity calibrated DOWN one tier + `stub-context` note | precision target (the residual ~50%) |
| A2 | stub with `return true // TODO auth` (fail-open default before any real check) | **stays CRITICAL — NO calibration** | **recall-regression tripwire** (the rule must NOT downgrade this) |
| A3 | stub that interpolates a request param straight into a real outbound `fetch()` (SSRF) | **stays HIGH — NO calibration** (injection allowlisted) | tripwire |
| A4 | non-stub production file with the SAME PII-log | finding at FULL severity (no stub predicate) | discriminator (calibration must not leak to live code) |

A2/A3 are the load-bearing cells: if the candidate ever downgrades them, it is REJECTED. A1 proving calibrated + A4 proving un-calibrated is the precision win.

## Apply-gate (the one blocked step — do NOT apply the rule until ALL green)
1. **Recall no-regression** — k≥3 union over the 17 LLM corpus cells under OLD vs NEW prompt; NEW must not drop any right-reason catch (same bar as `2026-06-19-citation-prompt-recall.md`). A2/A3 stay at full severity every roll.
2. **Precision-hold** — A1 calibrated + A4 un-calibrated, n≥3, on the real zync/stub live-fire files (the 0/8 comment cluster's host files).
3. **Discriminator** — safe.ts 17/17 unchanged (no new trigger-happiness).
Apply to `baseline.prompt.txt` ONLY after 1+2+3 are measured green. Record as a validation doc; update CLAUDE.md "Measured state".

## Gate-impact note (why a downgrade is NOT cosmetic — measurement-gated, confirmed)
PREVENT block-eligibility has two paths (`prevent/runner.py:104-109`): a finding BLOCKs if `level=="error" AND block_authorized(d)` **OR** if `is_ratcheted(f)`. Severity (`level`) is one of the two block paths. A stub-downgrade that drops a finding off `error` removes it from the severity-block path — it then blocks ONLY if separately ratcheted. So calibration is **gate-affecting** (BLOCK→WARN for an unratcheted error finding), not a reporting cosmetic. This is a further reason the change must clear the apply-gate before touching the production default — the never-calibrate allowlist exists precisely so no block-eligible critical/high can be calibrated off the gate.

## Status / done-bar
Mechanism fully specified + allowlist guardrail + measurement cells defined + apply-gate + gate-impact understood. **RATE-PENDING:** recall-no-regression + precision-hold are UNMEASURED this session (no rolls) and are the sole remaining step. This is "real completion" under the locked done-bar (mechanism + documented measurement gap), NOT a deferral: there is no TODO here, only one gated measurement the session blocks — identical to the comment-FP precedent before its k=9 roll.