# ThemeFactory Verification Engine

**Status:** implementation authority for deterministic theme completion
**Date:** 2026-08-22
**Purpose:** make `theme complete` a machine-verifiable state and prevent themes that do not satisfy their declared contracts from entering ThemeFactory `main` or `dist/`.

## 1. Completion is evidence, not an agent claim

An agent saying **theme complete** has no authority. ThemeFactory translates completion into one question:

```text
Can the required verification plan for this exact theme tree and exact distribution artifact produce PASS?
```

If not, the theme is **NOT RELEASEABLE** regardless of prose claims, screenshots, local logs, or a checked-in receipt. The authoritative receipt is regenerated by CI.

The uploaded WordPress Theme Engineer methodology remains the policy source: workflow routing, checkpoint IDs, quality baseline, UI defect invariants, packaging rules, and verification language. ThemeFactory adds an executable verification layer underneath it.

```text
Methodology / project contract
          ↓
CIInvariant / test registry
          ↓
profile-aware verification plan
          ↓
GitHub Actions provider
          ↓
ARC ephemeral K3s runner
          ↓
machine-readable evidence + completeness receipt
          ↓
release gate
```

This also aligns with AWP: GitHub Actions owns workflow/job execution, ARC owns ephemeral runner registration/scaling/pod lifecycle, and future AWP `VerificationAuthority` owns intent, evidence admission/reuse/invalidation, gate meaning, and operator UX. ARC never becomes CI identity.

## 2. One engine, composable profiles

Do **not** build one blanket suite and do **not** build separate CI systems for each builder. Every theme receives a small universal core plus only the profiles/features it declares.

### Mandatory core

```text
package integrity
WordPress install + activation
PHP / JS / JSON static integrity
security/static hygiene
frontend smoke
responsive hard invariants
accessibility baseline
navigation / keyboard
SEO semantics
performance budgets
admin/editor smoke appropriate to architecture
fresh install from the exact dist ZIP
receipt completeness
```

### Architecture profiles

#### `gutenberg-hybrid`

- block editor boot and network/REST health;
- `theme.json` contract;
- PHP page templates;
- patterns;
- custom block metadata and editor behavior;
- editor/frontend parity;
- editability round trips;
- template registration, assignment and composition.

#### `block-fse`

- `templates/*.html` and `parts/*.html` block grammar;
- `theme.json`/block support contracts;
- Site Editor boot;
- template-part registration;
- template editing + frontend round trip;
- patterns/style variations where declared.

#### `classic`

- PHP template hierarchy;
- registered menus/sidebars/widgets;
- Customizer behavior when declared;
- editor/content rendering without assuming block templates.

#### `elementor`

- pinned Elementor compatibility fixture;
- Elementor editor boot with no fatal/network errors;
- global kit/tokens;
- Theme Builder template existence/conditions;
- container/dynamic-data contract;
- Elementor edit → save → frontend round trip;
- dependency/lock-in declaration.

#### `woocommerce`

- pinned WooCommerce compatibility fixture;
- theme WooCommerce support;
- shop/product/cart/checkout/account surfaces;
- add-to-cart and cart-state journey;
- template override compatibility;
- WooCommerce-specific responsive/accessibility geometry.

### Workflow profiles

#### `new-theme`
There is no canonical source. Visual authority is an approved immutable screenshot/snapshot set or approved design-system contract. CI must never allow the same unapproved change to both alter implementation and silently bless a new baseline.

#### `mockup-to-theme`
Compare approved mockup/reference → WordPress output through structural, geometric and screenshot layers.

#### `source-preserving-conversion`
Compare original/reference → converted WordPress output for structure, component count, media/aspect ratios, geometry, responsive behavior and screenshots. Every allowed deviation is declared in a source-preservation manifest.

#### `enhancement`
Protect unchanged baseline surfaces and run focused tests for declared enhancements; visual regressions outside the enhancement scope fail.

### Feature profiles

Feature tests are added only when declared: `demo-import`, `custom-blocks`, `custom-post-types`, `forms`, `carousels`, `page-templates`, `source-preservation`, `customizer`, `widgets`, `multilingual`, `woocommerce`, `elementor`, `comments`, `embeds`, `breadcrumbs`.

A United Pets-style conversion therefore selects roughly:

```text
core
+ gutenberg-hybrid
+ source-preserving-conversion
+ demo-import
+ custom-blocks
+ custom-content-types
+ forms
+ carousels
+ page-templates
```

It does **not** run Elementor, FSE Site Editor or WooCommerce checks.

## 3. Every theme declares a machine-readable contract

Each theme carries `theme-test.yaml`. Example:

```yaml
schema: 1
workflow: existing-theme-conversion
architecture: gutenberg-hybrid

features:
  demo_import: true
  custom_blocks: true
  custom_post_types: true
  forms: true
  carousels: true
  page_templates: true

pages:
  home:
    path: /
    template: page-templates/home.php
    expected_blocks:
      - united-pets/source-section
      - united-pets/query-cards

  adoption-stories:
    path: /adoption-stories/
    template: page-templates/adoption-stories.php
    expected_blocks:
      - united-pets/source-section
      - united-pets/query-cards

demo_import:
  location: appearance
  activation_notice: true
  idempotent: true

components:
  service_cards:
    selector: ".services .service-card"
    equal_height: true
  adoption_cards:
    selector: ".adoption-card"
    equal_height: true

runtime:
  viewports: [320, 390, 480, 768, 1024, 1440, 1920]
  browser_shards: auto
```

Context matters. `equal_height: true` is a deterministic contract. ThemeFactory must not universally reject intentionally variable masonry cards.

The contract is schema-validated before any expensive job starts. Missing information required to select a test is a planning failure, not a reason to silently run everything.

## 4. The five United Pets escapes are permanent non-waivable regression gates

These failures are now test classes, not project folklore.

### `TF-EDITOR-001` — every imported Gutenberg page opens cleanly

For every imported page, CI opens the real block editor and asserts:

```text
0 × "Error loading block"
0 × "The response is not a valid JSON response"
0 × invalid/unexpected block content
0 × block-recovery prompts
0 × uncaught page exceptions
0 × theme-controlled console errors
0 × failed theme-controlled REST requests
```

Network responses are inspected. A theme block request returning non-2xx, HTML where JSON is expected, PHP warning/notice/fatal output, or malformed JSON fails.

### `TF-EDITABILITY-001` — real editability round trip

Previewing a block is insufficient. Representative content must be edited, saved, observed on the frontend and restored.

```text
open editor
→ locate declared editable field
→ change to unique CI sentinel
→ save
→ load frontend
→ assert sentinel is visible/represented
→ restore original
```

Run for representative text, image/link fields, and dynamic content records when declared. `looks editable != is editable`.

### `TF-INSTALL-001` — rewrite rules/permalinks work immediately after install

A fresh site must not require a human or CI workaround such as visiting **Settings → Permalinks** or running `wp rewrite flush` after theme activation/import. CI sets a pretty permalink structure before activation, installs/activates the exact candidate ZIP, runs the declared importer when applicable, and immediately requests declared page/post/CPT archive and singular URLs.

The gate asserts:

```text
activate exact candidate theme
→ no manual rewrite flush
→ run demo import when declared
→ no manual rewrite flush
→ declared pretty-permalink routes resolve
→ declared CPT rewrite bases exist in WordPress rewrite_rules
```

Implementation policy: themes that add rewrite rules/CPT bases flush them **once** at `after_switch_theme` and/or the import/migration boundary after registrations exist. Calling `flush_rewrite_rules()` on ordinary frontend/admin requests is a failure because it is expensive and masks lifecycle defects.

### `TF-DEMO-001` — activation/import UX + idempotence

On a clean site:

```text
activate theme
→ activation notice appears
→ notice offers Import Demo CTA
→ CTA resolves under Appearance
→ no top-level Import menu
→ action has capability + nonce protection
→ run import
→ snapshot state A
→ run import again
→ snapshot state B
→ A == B
```

Compare pages, posts, attachments, menu items, terms, CPTs and relevant settings, not merely row counts from one table.

### `TF-LAYOUT-001` — declared geometry invariants under content stress

For repeated cards with `equal_height: true`:

```text
max(height) - min(height) <= declared tolerance
```

Run with stress fixtures: short/long title, short/long/missing excerpt, missing/alternate image, long action label and translated-length strings, across the contract widths. Measure actual `getBoundingClientRect()`, not screenshots alone.

### `TF-TEMPLATE-001..003` — templates must be functional

1. Ask WordPress (`get_page_templates()`) and assert required templates are registered.
2. Assert imported pages carry the expected `_wp_page_template`, not `default`.
3. Create a new empty page, choose the template and assert its declared editable block composition is produced/available. A PHP file that exists but yields an empty editor is not implemented.

All five are blocking whenever applicable.

## 5. Editability coverage is measurable

Custom/source-preserving blocks declare stable editable field identities, for example:

```json
{
  "editableFields": [
    "headline",
    "description",
    "image",
    "button_text",
    "button_url"
  ]
}
```

The page/component contract declares required editable fields. CI compares required vs exposed fields and then round-trips a representative subset. A source section containing six headings, nine paragraphs, four images and five links but exposing only one heading/paragraph fails coverage even if it renders perfectly.

## 6. UI defect invariants become executable contracts

The uploaded `WordPress Theme UI Defect Invariants and Validation Workflow` is the browser invariant authority. ThemeFactory implements deterministic checks where geometry/state can be observed directly and contract-driven checks where intent is required.

Hard families include:

- required control unreachable/un-clickable;
- invisible element intercepting pointer input;
- stale overlay/backdrop;
- scroll lock without owner;
- keyboard trap;
- obscured focused control;
- irretrievably clipped content;
- zero/near-zero interactive geometry;
- overlap between independent controls/text/media;
- pathological character-level word fragmentation;
- unrecoverable truncation;
- unintended document horizontal overflow;
- nested/double scroll traps;
- elements outside viewport;
- unintended hidden required content/focusable descendants;
- clipped focus ring/dropdown/validation message;
- fixed/sticky collisions, including WP admin bar;
- modal escape/dismissal/focus restoration;
- undersized or inseparable controls;
- repeated component spacing/alignment drift;
- declared equal-height card violation;
- giant empty reserved regions after missing content/media;
- pathological line widths;
- breakpoint gaps where both/neither responsive variants appear;
- destructive image cropping/distortion;
- disabled-state semantic/state bugs;
- accidental duplicate components/IDs/submits/content;
- declared breadcrumb/header/footer contract loss;
- absolute-position/z-index interception bugs;
- hover-only functionality without focus/keyboard equivalent;
- Gutenberg wide/full/nested-block resilience;
- dynamic navigation/site identity/post titles/missing media/empty archives/pagination;
- form/comments/table/embed/caption states where declared;
- editor/frontend contract;
- mobile menu/viewport/touch/toolbar/input/table invariants.

Tests measure geometry/state. They do not ask an AI whether a page “looks broken.”

## 7. Visual verification is three layers, selected by workflow

Pixel diff alone is brittle and insufficient.

### Structural

Verify major sections, headings, components, navigation, repeated item count, shells and semantic roots.

### Geometric

Measure bounding rectangles, spacing, alignment, sizing and aspect ratios with declared tolerances.

### Screenshot

Use Playwright screenshots against an approved baseline/reference after fonts, animations, time-dependent data and uncontrolled third parties are made deterministic.

For a new theme, baseline changes require independent approval. For mockup work, compare approved mockup → WP. For conversion, compare original/reference → WP. Do not let an implementation change self-approve its own new visual baseline.

## 8. Deterministic environment

The release receipt records exact:

```text
WordPress
PHP
DB
Node
Playwright
Chromium
font fixture
locale
TZ
viewport
DPR
plugin fixture versions
container image digests
contract/check-definition hashes
```

CI disables/replaces sources of nondeterminism where possible:

- animations/transitions for geometry/screenshot assertions;
- random/demo-time content;
- uncontrolled analytics/ads/remote fonts;
- fixed sleeps (replace with awaited signals + finite deadlines);
- browser-context reuse between security/session cells.
- digest-pinned runtime images are explicitly inspected/acquired before container creation; registry latency uses a dedicated bounded acquisition timeout and is never coupled to `docker run` startup.

Themes declare minimum WP/PHP. PR CI verifies the pinned current compatibility target; scheduled compatibility verifies the declared minimum/current matrix so expensive compatibility does not multiply every iteration.

## 9. Performance policy: deterministic hard gates, CWV measurement separate

Core Web Vitals are useful but environment-sensitive. Hard-gate deterministic properties instead:

- JS/CSS byte budgets;
- request budget;
- DOM-node budget;
- image intrinsic dimensions/aspect ratio/lazy behavior;
- font count/weight budget;
- query/count budgets where measurable;
- avoid layout shift caused by omitted media dimensions;
- no duplicate asset loading.

Lighthouse/CWV may be informational or tolerance-band scheduled evidence. Do not make a flaky timing number the only release authority.

## 10. CI stages: cheapest reliable catcher, no redundant ceremony

This is derived from the `ci-cd-expert` skill and AWP Quality/CI policy.

### Stage A — planning/static (fast)

Run on every relevant PR and finish quickly:

- contract/ledger/schema/workflow topology;
- changed-file/theme detection;
- PHP syntax;
- JS/JSON/theme.json/block metadata;
- package hygiene/security/secret/local-path/symlink scans;
- static architecture checks that can be decided without WordPress;
- deterministic dist tree identity when applicable.

Never run a browser to answer a static question.

### Stage B — package/runtime qualification

For each changed theme, build/identify one candidate artifact, install it once into a fresh WordPress fixture, set the declared pretty-permalink structure, activate it, prove rewrite rules are usable without a manual flush, then run only the applicable architecture/workflow/feature checks against that fixture. Reuse the fixture inside one shard/job where isolation permits.

### Stage C — release artifact acceptance

When source/dist changes can ship:

```text
build deterministic ZIP
→ record digest
→ discard source-side assumptions
→ install exact ZIP into fresh WordPress
→ import twice if applicable
→ editor/admin/frontend/profile tests
→ visual/reference checks if applicable
→ archive/tree equivalence
→ aggregate expected-vs-executed receipts
```

The distributed bytes are the bytes that passed.

### Scheduled compatibility/measurement

Minimum/current WP/PHP compatibility, heavier performance measurements and broad maintenance suites live on schedule unless a change explicitly affects those contracts.

## 11. Change-aware selection without fake-green risk

The planner uses:

```text
changed themes
+ theme contract
+ architecture/workflow/features
+ test registry inputs
+ lifecycle stage
= exact required test IDs
```

Rules:

1. Core required tests always apply to a changed shippable theme.
2. Profile tests apply only to matching architecture/workflow/features.
3. A check ID appears at most once in a plan even if multiple profiles imply it.
4. No hand-maintained route list: route/cell manifests are generated from contract/imported WP state and checked for drift.
5. Browser shards contain only non-empty indivisible groups and are identified by shard ID, never hostname.
6. Aggregate validates shard IDs are complete, cell sets are disjoint, and `executed == expected` exactly.
7. A test skipped by explicit policy records `skipped-with-reason`; it is never translated to pass.
8. Missing required provider check blocks release.
9. Valid exact-input evidence can be reused; a prose claim or checked-in receipt cannot.
10. A required gate must be observed failing on a synthetic violation before it is trusted.

## 12. ARC / K3s execution topology

ThemeFactory CI runs through GitHub Actions on repo-scoped ARC ephemeral runners in the existing K3s cluster. Kubernetes may place jobs on eligible `debian1`, `debian2` or `debian3`; workflows **must not pin shards to hostnames**.

Two trust/capability scale sets are used:

```text
themefactory-ci
  unprivileged
  static/planner/aggregation
  no service-account token
  no publication credential

themefactory-runtime
  trusted internal only
  DIND capability for disposable WordPress/MariaDB/browser fixtures
  no reusable Git publication credential
  fork PRs rejected before checkout/runtime admission
```

This separation follows AWP’s runner boundary: privilege differences use separate runner profiles, ARC owns pod lifecycle only, project workflows remain the provider source definition, and future AWP consumes normalized receipts/check results.

The runner scale-set names are shared labels. GitHub/ARC/Kubernetes dispatch decides which buildbox runs a shard. This keeps capacity across all three nodes usable.

### ARC efficiency rules

- no `actions/cache` wrapping self-hosted runner-local stores;
- avoid repeated setup by keeping jobs coarse enough to reuse one WordPress fixture while still shardable;
- no workstation fallback;
- no host-pinned shards;
- `concurrency.cancel-in-progress` cancels superseded PR candidates before expensive stages;
- browser artifacts (screenshots/traces) upload on failure; receipts/timings upload always;
- browser shard count is planner-owned and only non-empty shards are dispatched;
- per-step and per-test timing JSON is produced from run 1;
- no `continue-on-error` on required gates;
- no fixed sleeps for readiness;
- no retries used to turn deterministic red into green;
- fork PRs never execute untrusted code on privileged runtime runners.
- the verified custom runner archive is persisted in each K3s node's air-gap image-import directory so node/service restarts can restore the `imagePullPolicy: Never` runner without registry fallback.
- runtime ARC pods reserve ephemeral storage for the fresh DIND image graph/workspace (DIND 8Gi request/20Gi limit; runner 2Gi request/8Gi limit), so scheduler/eviction decisions account for actual disk pressure.

## 13. AWP integration boundary

ThemeFactory is deliberately project-owned CI today. Its model is shaped so AWP I5 can consume it later without rewriting the tests.

Mappings:

```text
ThemeFactory test ID        -> AWP CIInvariant / CheckCatalog entry
verification plan           -> CIExecutionPlan subset
shard/job occurrence        -> CIExecution provider observation
machine receipt             -> VerificationEvidence
aggregate gate              -> GateEvaluator input
contract/profile selection  -> ChangeClassifier + CapabilityMap input
```

Receipt identities include exact source tree, test definition, contract, environment and artifact digest so future AWP EvidenceLedger can selectively reuse/invalidate evidence.

## 14. Methodology checkpoints remain policy authority

Do not rewrite the uploaded 96 checkpoint registry into implementation code. Maintain a separate mapping from checkpoint ID → executable test IDs. Some checkpoint criteria are subjective/product decisions; CI must not pretend those became deterministic by asking an AI.

Classify policy criteria as:

```text
AUTOMATED              machine falsifiable
APPROVAL_REQUIRED      requires approved baseline/owner/reviewer evidence
OPTIONAL_MEASUREMENT   informative environment-sensitive measurement
NOT_APPLICABLE         contract proves feature absent
```

Every required automated criterion must map to at least one reachable blocking test. Every approval criterion must have explicit admissible approval evidence. Omission is not pass.

## 15. CI-generated proof artifact

Each authoritative run emits `verification/themefactory-receipt.json`, for example:

```json
{
  "theme": "united-pets",
  "tree": "<40-char tree>",
  "profiles": ["core", "gutenberg-hybrid", "source-preserving-conversion"],
  "expected": 247,
  "executed": 247,
  "notStarted": 0,
  "passed": 247,
  "failed": 0,
  "artifact": {
    "sha256": "...",
    "freshInstall": true
  },
  "result": "PASS"
}
```

CI regenerates this file. A checked-in copy is never authoritative by itself. Aggregate cross-checks receipts against raw reporter output in the same artifact.

## 16. Theme completion UX

A releaseable theme can be summarized as:

```text
Build             PASS
Install           PASS
Demo import       PASS
Editor            PASS
Editability       PASS
Templates         PASS
Accessibility     PASS
Responsive        PASS
UI invariants     PASS
Visual fidelity   PASS
Packaging         PASS
Fresh ZIP install PASS

required expected == executed
required failures = 0
RELEASEABLE
```

A non-releaseable theme names exact gate/test IDs and affected page/component. `COMPLETE` from an agent never overrides this state.

## 17. Repository layout

```text
ThemeFactory/
├── themes/<theme>/theme-test.yaml
├── dist/
├── ci/
│   ├── policy.yaml
│   ├── profiles/*.yaml
│   ├── themefactory-tests.jsonl
│   ├── checkpoint-tests.yaml
│   ├── schemas/
│   ├── arc/
│   └── runtime/
├── tests/
│   ├── verification/
│   └── runtime/
├── tools/
│   ├── theme-ci
│   └── theme_ci.py
└── .github/workflows/themefactory-verify.yml
```

Canonical developer/agent command:

```bash
./tools/theme-ci verify <theme>
```

Exit `0` means the selected local gate passed. Exit nonzero means not releaseable. Only provider-produced CI evidence can satisfy the merge/release authority.

## 18. Durable test ledger

`ci/themefactory-tests.jsonl` is the implementation task ledger. One JSON object = one semantic test. It records:

- stable test ID;
- domain/title;
- execution phase;
- architecture/workflow/feature applicability;
- blocking/required classification;
- runner capability;
- input scope;
- assertions/evidence;
- implementation path;
- methodology checkpoint links;
- implementation status (`todo`, `in_progress`, `complete`, `deferred`);
- proof that the guard was observed failing where applicable.

A script validates unique IDs, valid dependencies, no unreachable required tests, and that every `complete` test names an existing implementation.

## 19. Existing tests at ledger creation

The only pre-existing deterministic ThemeFactory verification checks marked complete at initialization are the package/archive tests already present in `tests/test_theme_factory.py`:

- install → pack → audit round trip;
- reject unsafe ZIP traversal;
- reject multiple roots;
- require `style.css` + `index.php`;
- reject case-fold duplicate paths;
- reject symlinks;
- require generated ZIP to stay inside output;
- reject unexpected files in the packaging container.

Browser/editor/import/template/accessibility/profile checks were **not** considered complete merely because United Pets had ad-hoc release scripts/evidence. Durable CI requires reusable test implementations wired into the gate.

## 19A. Merge authority when native branch protection is unavailable

GitHub currently returns `403` for branch-protection configuration on this private repository/plan. ThemeFactory therefore carries a fail-closed provider-check merge authority at `./tools/theme-safe-merge`. This is not a replacement for CI; it is the admission point that refuses to mutate `main` unless provider evidence is already green.

The command requires all of the following at execution time:

- PR is open, non-draft, and targets `main`;
- local `HEAD` exactly equals the PR head SHA;
- GitHub's check-runs API reports `Full gate` on that exact SHA;
- the check is `completed` with `conclusion=success`;
- the provider app is `github-actions`;
- merge uses `gh pr merge --match-head-commit <sha>` with neither `--admin` nor `--auto`, so a head update between evidence validation and merge refuses rather than racing.

`--check-only` validates the same provider evidence without mutating the repository. When the GitHub plan supports required status checks, native branch protection should become the first line of enforcement; the safe-merge command remains defense in depth.

## 20. Failure-prevention doctrine

Before trusting the system:

1. every required test is reachable from the planner and workflow;
2. every new guard has a RED fixture/synthetic violation and GREEN proof;
3. expected/executed completeness is validated globally;
4. no required workflow job uses `continue-on-error`;
5. no fork PR can execute untrusted code in a privileged ARC runner;
6. no broad suite is run twice for the same exact evidence identity merely for reassurance;
7. no theme/profile causes irrelevant builder/plugin tests to run;
8. no baseline can be silently self-approved;
9. no receipt synthesizes tests it did not observe;
10. a run that suddenly becomes much faster/quieter is investigated as possible missing execution, not celebrated automatically.

That is the definition of a ThemeFactory Verification Engine: contracts and observable behavior, with the smallest sufficient tests selected per theme, and a release gate that agents cannot talk their way around.