# security-gate — PRIOR ART & what to borrow

audience: AI coding agents first. Imperative, BLUF, caveman-compressed prose. Purpose: the external landscape (products that overlap our parts) + the **decided** borrow/don't-borrow list, each mapped to the band/contract/roadmap item it lands in. This is a REFERENCE doc — `docs/ARCHITECTURE.md` stays the canonical design; this records influences + open candidate work. Do NOT adopt anything here without the validate-before-adopt loop (§3).

---

## 1. The landscape — products that overlap our parts

| Product | Maps to our… | What it is | Recall-measured? |
|---|---|---|---|
| **Semgrep** (`semgrep/semgrep`) | whole engine: microkernel + rules-as-data + registry + autofix | deterministic core; `fix:` on the rule; rules ship with annotated test files; **Assistant** adds LLM triage | no |
| **Vulnhuntr** (`protectai/vulnhuntr`) | band-2 orchestrated-LLM cross-file | LLM traces call chain user-input → sink, **pulls function defs on demand**; found a real Ragflow 0-day | no |
| **claude-code-security-review** (Anthropic) | band-1 LLM pass + our claude-cli dispatch | Claude-backed semantic SAST GitHub Action over PR diffs | no |
| **IRIS** (`iris-sast/iris`, academic) | our hybrid thesis | neurosymbolic: CodeQL over-approximates, LLM **prunes** false positives (Java) | partial (research) |
| **Deepsec** | orchestrator/mapper dispatch | agent harness: resumable parallel runs, custom matchers, optional revalidation | no |
| **GitHub Code Scanning + Copilot Autofix** | DETECT + executable resolution | integrated detect→autofix; reports mean-time-to-fix (460k+ alerts fixed 2025) | no (fix-rate, not recall) |
| classic SAST: `github/codeql`, Bandit/Brakeman/gosec, Trivy/Grype, Gitleaks/TruffleHog, `pre-commit/pre-commit` | band-3 deterministic + the hook framework | per-language SAST, SCA, secrets, hook runner | n/a |

**Our differentiator (no OSS equivalent): coverage-is-the-product** — git-fix-pinned corpus + **statistically measured recall (k≥3)** + **no-false-clean** + deterministic-first ladder with delete-test. Every tool above is single-roll and **accepts false-cleans** for noise reduction. That acceptance is the anti-pattern we differentiate from (§2, exclusions row).

## 2. Borrow list — DECIDED takeaways, each mapped to where it lands

Status legend: `TODO` = adopt after validation · `SPIKE` = evaluate first · `WATCH` = bank for a later phase · `ANTI` = do NOT copy, differentiate from.

| # | Takeaway | Source | Lands in | Priority | Status |
|---|---|---|---|---|---|
| 1 | **`exploit_scenario` field** — a concrete attack narrative per finding. Right-reason FORCING FUNCTION (a model that must write a working exploit can't hand-wave); feeds Verdict (scenario cohering with canonical = stronger credit) | claude-code-security-review | finding contract (`prevent/contract.py`, optional field) + Verdict (§ARCHITECTURE 4) | high · cheap | TODO |
| 2 | **Iterative source→sink context-fetching** — LLM traces the call chain and pulls function defs **on demand** while tracing, not via one-shot pre-resolution | Vulnhuntr | **band-2 cross-file** — the #1 open problem (barrel resolver + proactive trigger, `orchestrator/resolver.py`) | high · critical-path | TODO |
| 3 | **Rule + test-file convention** — every rule ships a test file annotating expected pass/fail lines = our canary/anti-canary/discriminator conformance suite, proven format | Semgrep | detector conformance fixtures (admission gate, spec §4) | med | TODO |
| 4 | **LLM-as-triage-filter (neurosymbolic ordering)** — deterministic detector over-approximates (high recall/low precision), LLM **prunes** false positives. Inverse of LLM-as-detector; precision lever without losing recall | IRIS | any band-3 detector that over-flags | med | SPIKE |
| 5 | **Resumable runs + revalidation pass** — resumable parallel state + 2nd adversarial pass over findings = our k≥3 union + security-guard, made durable for scale | Deepsec | orchestrator run-state + DETECT revalidation stage | low (scale-phase) | WATCH |
| 6 | **Measure fix-applied, not just detection** — report time/outcome to *resolution*, reinforcing "a catch is a FIX, not a flag" | GitHub Autofix | Verdict verify-gate + registry resolution column | med | WATCH |
| 7 | **Custom-scan-instructions append slot** — base prompt + per-surface appendable categories. Confirms our per-surface routing design (#16/#22) | claude-code-security-review | band-1/2 per-surface prompt routing | — | confirms design |
| 8 | **Phased methodology** (recon → data-flow trace → assessment) as a reasoning scaffold | claude-code-security-review | band-1 prompt structure (flaky-tail lift candidate) | med | SPIKE |
| 9 | **Exclusion lists** (drop DOS / rate-limit / resource-exhaustion / secrets-on-disk / "non-security-critical" input-validation) | claude-code-security-review + industry default | — | — | **ANTI** — violates no-false-clean + coverage-is-product. Never copy; differentiate. |

## 3. The claude-code-security-review prompt — benchmark, do NOT swap

**Run it as a CANDIDATE through the band-1 harness; adopt only on ZERO regression vs 17/17** (`docs/handoffs/...` validate-before-adopt loop). Normalize THREE mismatches first or the measurement is apples-to-oranges:

1. **Diff-scoped vs our whole-file cells.** Its prompt says review ONLY PR-new issues ("Do not comment on existing security concerns"); our cells are git-fix-pinned whole vuln files. **Strip the diff-scoping** for the bench harness.
2. **Its exclusions score blind spots as misses.** It drops whole classes (row 9). **Map exclusions onto our taxonomy before scoring** — a cell in an excluded class is a scope exclusion, NOT a capability miss.
3. **Its >80%-confidence / minimize-false-positives bias trades recall for precision.** Our goal is 100% recall; that bias likely SUPPRESSES the flaky-tail cells (7×2/3) we want to lift. Judge on whether it regresses any of the 17, NOT on overall noise.

License: Anthropic's, permissive — learn freely. Verbatim text → check LICENSE + attribute in a comment. Cleaner: author our own, cite as reference.

Best realistic outcome: it **validates our baseline** or donates a few category phrasings — not a replacement.

## 4. Buy-vs-build — wrap FOSS for band-3, build only the semantic moat

**Band-3 is mostly WRAPPED FOSS behind our finding contract, NOT hand-written scanners.** SCA, secrets, and syntactic taint SAST are solved problems. Rebuilding them is waste; wrapping them ADVANCES the goal — a deterministic FOSS catch replaces an LLM dice-roll (~86% k=1) on the syntactic subset = "100% in one run, no dice". Each wrap = one `detector.json` (`kind:exec`) + a thin adapter mapping the tool's native output → the finding contract (§ARCHITECTURE 2). SoT for which-tool-when-why-how + the lane diagram: `.visual/data/toolflow.json` (rendered at `toolflow.html`).

**Do NOT rebuild — wrap these (TS/JS corpus targets). All permissive/free FOSS, usable in private/commercial repos with no restriction:**

| Class | Wrap | Replaces / why | Lane | License |
|---|---|---|---|---|
| SCA / deps | **Trivy** or **osv-scanner** | retires hand-rolled pnpm-audit `deps` detector; OSV DB, multi-ecosystem | both (fast) | Apache-2.0 |
| Secrets | **Gitleaks** | regex+entropy, pre-commit native; never hand-roll | PREVENT (fast) | MIT |
| Syntactic taint SAST (SQLi, cmd-injection, path-traversal, insecure config) | **Semgrep** (`p/security-audit`, `p/owasp-top-ten`) | thousands of rules + taint mode; deterministic, zero-dice, beats an LLM roll on what it catches | both (fast) | LGPL-2.1 (CLI + community rules), shelled out → no copyleft obligation |
| same, latency-critical | **ast-grep** (Rust) | Semgrep-lite, hot-path-friendly; ties to the Rust PREVENT thread | PREVENT (fastest) | MIT |

**EXCLUDED — not free-FOSS for our cross-repo use, dropped from plans:** CodeQL (LICENSE restricts use to OSS projects; private/commercial repos need paid GitHub Advanced Security — not FOSS for us) and TruffleHog (AGPL, and redundant with Gitleaks). No capability lost: band-2 LLM + oracle already carry the semantic dataflow CodeQL would have. **Candidate rule: a tool that is not free-FOSS for private/commercial repos is not a candidate** — that check is what flagged CodeQL.

**The wrap-decision gate (NOT the build delete-test).** The delete-test gates *building* band-3. Wrapping is ~zero build cost, so the bar drops:

> Wrap a FOSS tool IF (a) it passes our conformance gate (flags the canary, quiet on the safe cell) AND (b) it removes a dice-roll OR adds recall — even on classes the LLM sometimes catches.

- **deterministic ≠ complete.** A wrap removes the dice ONLY on what it catches; taint rules miss obfuscated/cross-file flows. A deterministic miss is still a miss. **Measure recall; never treat "tool ran" as "class covered"** (no-false-coverage-claim still binds). Run every wrapped rule through OUR conformance gate before trusting it — being Semgrep does not exempt it.
- **Free-FOSS is a candidate precondition** — the gate runs across private/commercial repos, so a tool whose license restricts commercial/private use (CodeQL) or imposes viral copyleft we cannot meet is NOT a candidate. The kept set is all permissive/free (Apache/MIT/LGPL-via-subprocess); nothing to verify per-adoption.

**What stays ours (the moat — no FOSS does it well, do NOT wrap):** semantic authz / IDOR / self-deal / complete-mediation (S2/S3/S9, the oracle + LLM bands). The answer is data-model-dependent reasoning ("is this the *right* tenant predicate?"), not pattern matching. This is why the LLM bands exist; don't dilute their budget trying to wrap it.

**Two integration points for the FOSS output (both valid):**
1. **standalone band-3 detector** — in the ladder, deterministic-first.
2. **context-priming the LLM / security-guard** (IRIS inverted, row 4 made concrete) — run Semgrep/Gitleaks/Trivy first, hand the deterministic hits to the LLM so it stops re-finding syntactic bugs and spends its whole budget on the semantic residue. **Higher-leverage for security-guard specifically.**

**Surface-aware:** PREVENT hot path = fast tools only (Semgrep/ast-grep/Gitleaks/Trivy). CodeQL's compile step → DETECT-only if ever (§ARCHITECTURE 7, the latency lever is cold-start).