# Assembled-orchestrator pilots — S4-001 + S6 + STEP2 (salvaged 2026-06-16)

audience: AI coding agents first. Salvaged pilot evidence; superseded for project decisions by the SoT design spec.

---

# STEP-2 zync-audit pilot — generalization beyond payments (2026-06-17)

audience: AI coding agents first. Measured log; tags [MEASURED] vs [INFERRED]. Do not prettify.

## BLUF
The fable zync.is security audit (11 classes S1–S11, ~6 P0 + ~36 P1 confirmed) is now the STEP-2
ground-truth corpus. Two blind pilots run with the **UNMODIFIED v2 "payments" prompt** (k=3 union)
both CAUGHT their non-payments canonical. The v2 prompt is a strong GENERAL adversarial code reviewer,
not a payments-only one. The step-2 gap is therefore NOT "payments blindness" — it is (1) precision /
semantic dedup, and (2) structurally-different classes that an LLM code-review cannot do by construction.

## Pilots [MEASURED] — blind, isolated single-file cells, /tmp/sg_cfg clean config
| cell | class | canonical | gate (v2@sonnet/MED, k=3) result |
|---|---|---|---|
| S4-001 | SQLi | `kb.ts` `sql.raw(\`ARRAY[…'${id}'…]\`)` injection | **CAUGHT 3/3, ranked #1 critical.** + kbPortalQuery IDOR (1/3) + many real bugs |
| S6-001 | SSRF | PATCH `/:id` updates url without `assertSafeWebhookUrl` (POST has it) | **CAUGHT (union)** — but as 3 separate "1/3" paraphrases |
| S6-004 | SSRF | in-file `assertSafeWebhookUrl` passes non-dotted/octal IP + metadata host | **CAUGHT (partial)** — "non-dotted-decimal/octal IPv4" 1/3 |

Reports: `S4-001_gate_v2.md`, `S6_gate_v2.md`. Pre-fix cells via `git show <fix>~1:<path>`
(S4: 9b0c485, S6: bec8bc8). Ground truth: `/home/user/Projects/zync.is/docs/plans/audit/security/`
(orchestrator-only — gate subagents never see it).

## Reframed gaps
- **GAP 1 — precision / semantic dedup [MEASURED, now #1 priority].** S6-001 surfaced as THREE separate
  `1/3` entries ("Missing SSRF guard on PATCH", "PATCH stores arbitrary URLs", "SSRF bypass via PATCH").
  Word-overlap dedup counts PHRASINGS, not bugs. Two harms: (a) noise (~17–19 findings/file : ~1 canonical);
  (b) it HIDES true recall — one bug found by all 3 rolls in different words reads as flaky `1/3`. The
  roll-count signal is corrupted until dedup is semantic. This is task #17, promoted from "precision nicety"
  to "the measurement instrument is distorted."
- **GAP 2 — structurally-different classes need DETERMINISTIC tool legs [INFERRED from class taxonomy].**
  S11 = dependency CVEs (pdfjs CVE-2024-4367, drizzle GHSA) + missing CSP/HSTS response headers in
  wrangler.toml. An LLM reading a `.ts` route file cannot find a CVE in a lockfile it wasn't given, nor run
  `pnpm audit`, nor probe live response headers. Fable's OWN audit used `pnpm audit` + live `curl` probes for
  exactly S11 — i.e. fable did NOT use an LLM for these. The principled mirror: add deterministic legs
  (pnpm-audit leg, header/config checker, optional live-probe) the way the oracle is a deterministic leg for
  complete-mediation. Do NOT expect the LLM leg to cover deps/headers. NOT yet measured (a run would only
  confirm the obvious miss); stated as a design fact, flagged for validation when the tool legs exist.

## What the pilots did NOT yet establish (honest)
- Recall on classes still untested: S1 auth-session, S2 RBAC (incidentally seen in S6 run), S5 XSS/email-HTML,
  S7 webhook-auth, S10 OAuth, S8 file-authz. Expect mostly catchable (code-logic) — UNMEASURED.
- "ALL bugs in 1 round" bar: NOT met/claimed on 2 cells. Need a recall sweep across classes after dedup fix.
- File coverage: the gate only catches a bug if RUN on the file containing it; the audit spans ~40 files →
  "1 round" = run the gate over the audited file set (file-selection is its own step).

## Recommendation (evidence-based, ordered)
1. **Fix semantic dedup (#17) FIRST** — highest leverage: restores the recall signal + cuts noise. Embedding
   or a small LLM merge pass over the unioned findings.
2. **Add deterministic tool legs for S11-class** — pnpm-audit leg + header/config checker. Mirrors fable.
3. **Light prompt generalization (#16)** — strip residual "payments" framing → "security/correctness",
   broaden the class checklist; re-validate it does NOT regress the 6 payments cells. Likely SMALL given the
   v2 prompt already generalizes.
4. **Recall sweep** across the 11 classes (1–2 cells each, after dedup) → measure true per-class recall →
   then push toward the "all bugs in 1 round" bar.

---

# Security-gate report

## Scope (files reviewed)
- `kb.ts` — target

## kb.ts
**Oracle (complete-mediation):** SILENT

**LLM leg (v2@sonnet/MED, 3-roll union):**
- [critical] SQL Injection via `sql.raw()` in `listArticlesBySpaceIds`  _(found by 3/3 rolls)_
- [high] Portal Queries Missing `isNull(deletedAt)` — Soft-Deleted PUBLISHED Articles Exposed to Customers  _(found by 3/3 rolls)_
- [high] `moveArticle` Depth Check Non-Atomic With Write — Concurrent Moves Can Exceed `MAX_TREE_DEPTH`  _(found by 3/3 rolls)_
- [high] `softDeleteArticle` subtree deletion is non-atomic — partial deletion on interruption  _(found by 2/3 rolls)_
- [high] Cycle creation in `moveArticle` causes infinite loop / stack overflow in `treeDepth` and `subtreeHeight`  _(found by 2/3 rolls)_
- [high] Self-Parenting Creates Permanent Cycle — Infinite Loops in `treeDepth`, `subtreeHeight`, and `softDeleteArticle` BFS  _(found by 1/3 rolls)_
- [high] `updateArticleMeta` Accepts `parentId` Without Depth Validation or Space Ownership Check  _(found by 1/3 rolls)_
- [high] `softDeleteArticle` subtree deletion is not atomic — partial deletes on Worker eviction  _(found by 1/3 rolls)_
- [medium] TOCTOU slug uniqueness race in `duplicateArticle`  _(found by 2/3 rolls)_
- [medium] `NaN` can be written to the position column via `parseFloat` on corrupt data  _(found by 2/3 rolls)_
- [medium] Slug uniqueness check in `duplicateArticle` omits `tenantId` filter  _(found by 2/3 rolls)_
- [medium] `updateArticle` and `incrementViewCount` Operate on Soft-Deleted Articles  _(found by 1/3 rolls)_
- [medium] `moveArticle` Does Not Validate That `parentId` Belongs to the Same Space — Cross-Space Reparenting Corrupts Tree  _(found by 1/3 rolls)_
- [medium] `getLastSiblingPosition` counts soft-deleted siblings, skewing new article positions  _(found by 1/3 rolls)_
- [medium] `kbPortalQuery` trusts `customerId` without verifying it belongs to `tenantId` (cross-file assumption / IDOR)  _(found by 1/3 rolls)_
- [low] Hard `deleteArticle` does not cascade to children — orphans become stale roots  _(found by 1/3 rolls)_
- [low] Published children of DRAFT/non-published parents surface as root nodes in portal tree  _(found by 1/3 rolls)_
- [low] `deleteArticle` hard-deletes without orphan handling, coexisting with `softDeleteArticle`  _(found by 1/3 rolls)_
- [low] `subtreeHeight` loads entire space into Worker memory — unbounded result set  _(found by 1/3 rolls)_

---

# Security-gate report

## Scope (files reviewed)
- `endpoints.ts` — target

## endpoints.ts
**Oracle (complete-mediation):** SILENT

**LLM leg (v2@sonnet/MED, 3-roll union):**
- [high] `GET /:id/deliveries/:did` ignores the `:id` path parameter — intra-tenant IDOR  _(found by 3/3 rolls)_
- [high] Missing SSRF guard on `PATCH /:id` — stored URL bypasses all IP-block rules  _(found by 1/3 rolls)_
- [high] PATCH endpoint stores arbitrary URLs without SSRF validation — bypass enables internal URL persistence  _(found by 1/3 rolls)_
- [high] Retry endpoint has no idempotency guard — concurrent retries produce duplicate outbound dispatches  _(found by 1/3 rolls)_
- [high] SSRF bypass via PATCH — `assertSafeWebhookUrl` never called on URL update  _(found by 1/3 rolls)_
- [medium] `JSON.parse(endpointRaw.secretEncrypted)` throws uncaught on malformed or null DB value  _(found by 3/3 rolls)_
- [medium] Outbound fetch precedes DB record insertion in both retry and test paths — Worker eviction leaves delivery unrecorded  _(found by 2/3 rolls)_
- [medium] `POST /:id/deliveries/:did/retry` has no status guard — already-delivered events re-dispatched unconditionally  _(found by 1/3 rolls)_
- [medium] `assertSafeWebhookUrl` does not block non-dotted-decimal or octal IPv4 representations  _(found by 1/3 rolls)_
- [medium] Rotate-secret has a TOCTOU race — one concurrent caller receives a non-functional plaintext secret  _(found by 1/3 rolls)_
- [medium] No rate limiting on test dispatch or manual retry — authenticated HTTP amplification / DoS proxy  _(found by 1/3 rolls)_
- [medium] No timeout on outbound `fetch` — Worker held indefinitely by slow target  _(found by 1/3 rolls)_
- [low] Role check for `rotate-secret` is case-sensitive only for two variants and casts session to `any`-shaped object  _(found by 1/3 rolls)_
- [low] Test deliveries can be retried as real deliveries — no status guard on retry  _(found by 1/3 rolls)_
- [low] Owner role check silently excludes title-cased `'Owner'` — unverified cross-file dependency  _(found by 1/3 rolls)_
- [low] Fragile role comparison using dual-case strings with an unsafe type cast  _(found by 1/3 rolls)_
- [low] DNS rebinding / TOCTOU window in SSRF guard  _(found by 1/3 rolls)_