# AWP Verification Enforcement Design **Date:** 2026-08-19 **Status:** Working product-design baseline; approved direction, not yet implementation-grade spec. **Purpose:** Define the enforcement layer that makes AWP's verification discipline apply consistently across coding agents, review agents, CI, merge and release without relying on prompts or duplicate testing. ## 1. Core rule Verification policy is enforced by AWP, not by individual agents. Agents may reason about what evidence is missing and request verification. They do not decide that a task/review/change is complete merely because they ran commands or wrote that tests passed. ```text QualityStrategy + CIPlan | v ChangeSet / Review / Release context | v VerificationAuthority <---- authoritative control-plane module | +--> VerificationPlan +--> VerificationAdmission +--> EvidenceLedger +--> GateEvaluator | v allowed progression or explicit block ``` `VerificationAuthority` is a logical AWP control-plane responsibility. It should begin as a module inside the modular monolith, not a separate service. ## 2. What the enforcement layer owns ```text VerificationAuthority ChangeClassifier VerificationPlanner VerificationAdmission EvidenceLedger EvidenceInvalidator GateEvaluator ``` Responsibilities: - determine which properties/risks are affected by a ChangeSet; - select the smallest sufficient checks required by QualityStrategy/CIPlan; - admit or deny expensive/broad verification requests; - execute or dispatch authoritative checks; - persist evidence bound to exact inputs; - reuse still-valid evidence across agent roles/stages; - invalidate only evidence whose relevant inputs changed; - decide whether Task, Review, ChangeSet, Merge or Release gates are satisfied; - record why verification broadened or why it stopped. ## 3. Agent contract ### Coding agent A coding agent receives: ```text Task ChangeSet/workspace identity applicable QualityRequirements current VerificationPlan existing valid evidence, if any ``` It may: - write/fix code; - add focused tests when required; - request a planned verification step; - request broader verification with a machine-readable reason; - run bounded ad-hoc debugging commands that do not count as authoritative evidence. It may not: - mark the Task verification gate complete; - substitute an arbitrary full-suite run for missing required evidence; - repeatedly rerun already-valid broad checks without an admitted reason; - self-report terminal verification evidence from prose/log text. The Task can advance to review only when `TaskVerificationGate` is satisfied by the GateEvaluator. ### Code review agent A review agent receives the diff plus the existing `EvidenceBundle` from implementation. It does **not** start from zero. Its verification job is: ```text inspect implementation + existing evidence -> identify uncovered review findings/risks -> request only the missing/delta verification -> reuse valid implementation evidence ``` It should not rerun implementation tests merely because review started. A review agent can request additional verification when, for example: ```text review found a new risk or defect implementation evidence does not cover a required property source changed after evidence was produced test/check definition changed relevant environment/toolchain changed prior evidence is flaky/untrusted/incomplete reviewer made a code change shared contract blast radius was underestimated ``` The Review can advance only when `ReviewVerificationGate` is satisfied. If a review agent changes code, AWP computes the delta, invalidates only affected evidence, and creates the smallest new VerificationPlan for that delta. ## 4. Stage gates Verification is stage-aware. Recommended gates: ```text TaskVerificationGate focused evidence for the WorkItem/Task implementation ReviewVerificationGate implementation evidence reused + review-specific delta evidence ChangeSetCompletionGate project-defined final qualification before publication/merge MergeGate required GitHub/CI invariants and policy satisfied ReleaseGate release-specific qualification/evidence DeploymentGate deployment preflight + applicable post-deploy verification ``` These are not identical suites executed repeatedly. Each gate declares which evidence it requires and may reuse evidence produced earlier when the source/check/environment identity is still valid. ## 5. Evidence is reusable across agents Authoritative evidence must be bound to enough identity to prove it still applies. Working evidence identity: ```text VerificationEvidence property / QualityRequirement / CIInvariant source tree / ChangeSet identity relevant file/package/capability scope check definition/version/hash test definition hash where relevant toolchain/lockfile inputs where relevant environment/runtime fingerprint where relevant fixture/schema/config inputs where relevant result producer / runner identity timestamp receipt/artifact references ``` The implementation agent and review agent therefore share evidence instead of each recreating it. Example: ```text Coding agent runs focused auth regression -> PASS evidence for tree T1 Review agent reviews tree T1 -> evidence still valid -> does not rerun auth regression Review agent changes auth code -> tree T2 -> auth evidence invalidated -> targeted auth verification planned again ``` ## 6. Authoritative evidence cannot come from agent claims An agent saying `tests pass` is not evidence. Authoritative evidence is produced by a trusted AWP verification execution path or imported from an accepted CI provider with verified identity/completeness. ```text Agent request | v VerificationAdmission | v VerificationRunner / CI provider | v machine-readable result + receipt | v EvidenceLedger ``` This prevents model behavior, hallucinated logs or incomplete command execution from satisfying lifecycle gates. ## 7. Verification admission The enforcement layer distinguishes targeted verification from broad/high-cost verification. Broad verification requires a recorded reason, for example: ```text shared-contract-change wide-dependency-impact targeted-failure review-discovered-risk completion-gate release-qualification environment-change manual-user-request ``` If valid evidence already proves the same property for the same relevant inputs and no invalidation/escalation trigger exists, AWP should reject or warn on a redundant broad request. Default behavior: ```text targeted planned check -> admit automatically recognized redundant broad check -> deny by default or require explicit override according to project policy required CompletionGate -> admit at its lifecycle point user explicitly requests extra verification -> admit and label as user-requested evidence ``` ## 8. Tool/runtime enforcement Prompt instructions are insufficient. AWP controls the agent execution environment and tool interface, so verification policy should also be enforced at tool admission. Recommended model: ```text AgentProvider tool call -> AWP Tool Gateway -> classify command/action -> match project CheckCatalog / VerificationPlan -> VerificationAdmission decision -> execute / deny / mark ad-hoc ``` `CheckCatalog` maps known project verification commands/scripts/workflows to semantic checks and cost/scope classes. Examples: ```text pnpm --filter @app/auth test authz.spec.ts -> targeted known check pnpm test -> broad repository test gate playwright test tests/cuj/publish.spec.ts -> focused CUJ E2E factory-smoke-all -> broad smoke suite ``` Known broad checks can therefore be denied when no permit/reason exists. AWP should not make the coding shell useless. Bounded ad-hoc commands remain available for debugging, but: - they do not satisfy authoritative gates unless admitted/recorded as a VerificationStep; - they remain subject to execution time/resource budgets; - recognized broad verification commands can be intercepted before execution; - unknown commands that happen to perform testing cannot be perfectly classified, so correctness enforcement must never depend solely on command interception. The hard guarantee is lifecycle-gate enforcement and authoritative evidence. Command admission is the additional mechanism that reduces wasted compute/time. ## 9. Reviewer duplication prevention Review should use an explicit evidence-delta protocol. ```text Review starts -> AWP supplies EvidenceBundle -> reviewer identifies findings and unproven properties -> GateEvaluator computes delta -> only missing/invalid evidence becomes runnable ``` The reviewer UI/agent context should show, concisely: ```text Already established Q-17 auth rejection regression ........ PASS / reusable Q-21 affected package typecheck ....... PASS / reusable Needs review evidence R-04 migration rollback path .......... not covered Invalidated by review edit Q-17 auth rejection regression ........ rerun required ``` The normal reviewer action is not `Run tests`. It is `Resolve missing evidence` only when missing evidence exists. ## 10. Evidence invalidation Evidence reuse is safe only when invalidation is precise. At minimum consider invalidation when relevant inputs change: ```text source files in affected scope test/check definitions public/shared contracts dependency lockfile or relevant dependency versions compiler/runtime configuration schema/migrations fixtures/reference data environment/runtime profile security/policy configuration ``` A change outside a piece of evidence's dependency scope should not invalidate it automatically. When dependency impact cannot be determined confidently, AWP broadens conservatively and records `impact-confidence-low` as the reason. ## 11. Role-specific verification intent Different roles have different verification responsibilities: ```text coding agent prove changed behavior enough to hand off review agent challenge implementation + evidence, add only missing proof CI provider execute required project checks at policy-defined lifecycle points AWP GateEvaluator decide whether evidence is sufficient ``` No role independently owns the definition of `done`. ## 12. Example flow ```text Task: fix authorization bypass AWP creates plan V1 focused auth regression V2 affected auth package typecheck Coding agent fixes code -> requests V1, V2 -> both PASS -> TaskVerificationGate satisfied Review agent starts -> receives V1/V2 receipts -> reviews diff -> finds no uncovered behavior -> requests no tests -> ReviewVerificationGate satisfied ChangeSet ready for merge -> CompletionGate requires repository lint + required CI invariants -> ARC/GitHub Actions executes once -> receipts imported -> MergeGate satisfied ``` If the reviewer changes a shared auth contract instead: ```text review edit changes contract -> affected graph expands -> V1/V2 partially invalidated -> contract integration check added -> authenticated CUJ subset becomes affected -> broader evidence is now justified ``` ## 13. UX consequences Agents and humans should see verification as evidence status, not a wall of commands. Task/review surfaces should answer: ```text What must be proven? What is already proven? What became stale? What is the next smallest missing check? Why would broader verification run? What gate blocks progression? ``` Normal actions: ```text Run required check Request broader verification Open evidence Explain why this is required ``` Avoid a generic primary `Run all tests` action. ## 14. Architecture decision direction Recommended architecture: ```text AWP control plane Verification module planner admission evidence ledger/invalidation gate evaluator Execution plane VerificationRunner in AgentRun workspace for focused/local checks ARC/GitHub Actions for project CI/completion gates ``` Do not introduce a separate external verification service unless measured scale or isolation later requires it. ## 15. Next spec questions ```text Q1 Exact VerificationEvidence identity/invalidation schema. Q2 Exact gate hierarchy and relationship to CIInvariant/Check/CIExecutionPlan. Q3 CheckCatalog discovery from existing repository scripts/workflows. Q4 Tool Gateway command classification and strict-vs-advisory admission modes. Q5 Runtime/compute budget model for ad-hoc verification. Q6 Review-agent evidence bundle and evidence-delta protocol. Q7 Import/trust rules for GitHub Actions evidence and expected-vs-executed completeness. Q8 Policy for user-requested redundant verification and administrative overrides.