--- name: creating-ui-matrix-tests description: Use when building granular per-page, per-role UI-element regression tests — asserting every button is clickable/disabled when it should be, every input/textarea/table/form renders per role, catching small UI regressions a flow test misses. Distinct from journey/flow tests. Ships a manifest-driven matrix harness template plus a deterministic validator that fails closed on rotted selectors, routes, and roles. --- # Creating UI Matrix Tests audience: AI coding agents first. Build a **granular per-page × per-role UI-element regression matrix**: assert each element's expected state on each page, per role. Catches small UI bugs — a button enabled when it MUST be disabled, a table that stopped rendering, a textarea that lost maxlength — that flow tests skip. **Distinct from `creating-journey-e2e-tests`.** Journeys = temporal, cross-page, one outcome, happy + failure branches (FLOW bugs). This = spatial, per-page, per-role element-state inventory (UI REGRESSIONS). Separate passes; never merge; never put element-state assertions in journey files. ## Iron law **Never assert against a selector, route, or role you did not read from source.** A manifest row pointing at a nonexistent testid, a deleted route, or an invented role is a **silent no-op that always passes** — worse than no test, because it reports coverage it does not have. Step 4's validator enforces this deterministically. Run it. Do not hand-check. ## Step 1 — Discover the harness; do NOT rebuild Most mature repos already own a sweep/matrix harness. Extend it. Standing up a parallel one splits the source of truth and both rot. Search before writing anything: | Look for | Where | Never claim missing until | |---|---|---| | Existing sweep/matrix runner | `**/*sweep*`, `**/*matrix*`, `**/e2e/**` | grepped those globs | | Auth/session mechanism | test configs + `**/helpers/**`, `**/fixtures/**` | grepped `storageState`, `addCookies`, `SignJWT`, `login` | | Runners + scripts | `playwright.config*`, package scripts | listed them | | Real roles | schema/RBAC enum | read the enum at its source | | Real routes | framework route source (pages dir, router table, manifest) | enumerated them | | Existing testids | `rg 'data-testid'` | counted them | Record what you found with `path:line`. **A claim of absence with no grep behind it is a false blocker** — it sends the reader to build what they already own. Found a harness → extend its manifest. Found none → scaffold from the template below. ## Step 2 — Three layers, strongest leverage first Stop at the layer that covers the need. **Layer 1 — Component-contract tests (do FIRST, highest leverage).** If the project mandates a shared component library (confirm the mandate exists), test each shared component's behavior contract ONCE at component level: Button (enabled/disabled/loading/click fires), Textarea (types/maxlength/required/error), Table (populated/empty/loading), Select, Dialog. Per-page work then collapses to "the RIGHT component is present in the RIGHT state" — not re-testing button mechanics on every page. **Layer 2 — Generic per-page × per-role invariant sweep.** Assert real invariants on every page × role: a11y (axe) pass, zero unallowlisted console errors, zero same-origin failed requests, pixel-stable masked snapshot. NEVER `body` visible or `toHaveURL(/.*/)`. **Layer 3 — Declarative element manifest (high-value pages only).** For pages where exact element state per role matters (checkout, admin tables, money forms), declare expected states as DATA; one generic runner asserts each row. Adding a page = a data entry, not a new test. Reserve for money/complex surfaces; Layers 1+2 cover the rest. ## Step 3 — Address elements by testid on matrix-covered surfaces **Add `data-testid` to every element the matrix covers.** Accessible-name anchors (`getByRole('button', {name})`) break on any multi-locale site — the name changes per locale, so a name-anchored row silently matches nothing in the other locales. Testids are locale-invariant. Rules: - Address by `testid`, or by explicit `role` + `name`. NEVER by DOM ordinal. - `name` REQUIRES an explicit `role`. An unscoped name cannot address inputs, textareas, tables, or regions. - Derive every testid from real component source. Never invent one. ## Step 4 — Gate ```text node /scripts/validate-matrix.mjs \ --routes --roles --src ``` Generate `routes-file` and `roles-file` from the project's real route source and role enum (step 1). Every flag is REQUIRED — each powers one check; omitting one would silently disable it. The validator is deterministic and free. It proves: every testid exists in source, every route is real, every role is real, every state is assertable, no duplicate rows, no vacuous rows, no `404`-as-denial, no range status. It does NOT prove semantics — that is the test run's job. `INVALID` → fix and rerun. Never report a matrix while the gate is red. **This gate is the anti-rot mechanism.** A matrix with no validator degrades silently as the app moves: routes get renamed, testids get dropped, roles get retired, and every stale row keeps reporting green. ## Quality bar Every assertion MUST fail on a real regression. Reject: ```ts // DO NOT — passes on a blank error page expect(resp?.status()).toBeLessThan(400); // DO NOT — tautology await expect(page.locator('body')).toBeVisible(); // DO NOT — silent skip if (await el.isVisible()) { await expect(el).toBeEnabled(); } ``` Require one exact status, one exact state per row. Multiple legit states → one row each. `log()` any route or element intentionally excluded — silent truncation reads as "covered" when it is not. **`404` is NEVER proof of authorization.** A deleted route returns `404` to everyone forever, and a `404`-accepting authz row passes permanently. Declare `redirect` or `403`. --- ## Portable harness template (scaffold when no harness exists) Framework-agnostic Playwright matrix runner. Project supplies manifest data + one auth adapter; the runner is generic. ### `ui-matrix/matrix.json` — the data ```json [ { "route": "/en/dashboard/releases", "roles": ["artist"], "deniedRoles": { "anonymous": "redirect", "customer": "403" }, "expectStatus": 200, "elements": [ { "testid": "release-table", "state": "populated" }, { "testid": "release-title", "state": "required" }, { "role": "textbox", "name": "Catalog number", "state": "editable" } ] } ] ``` `state` ∈ `present | absent | visible | enabled | disabled | editable | readonly | required | optional | empty | populated`. `count` applies to `present | absent` only. `deniedRoles` values ∈ `redirect | 403`. Routes, roles, and testids above are ILLUSTRATIVE — replace with values grepped from the project. ### `ui-matrix/matrix-auth.ts` — the seam (project implements) ```ts import type { BrowserContext } from '@playwright/test'; export type Role = string; /** Authenticate `context` as `role` against `baseURL`. `anonymous` = no-op. */ export async function applyAuth( context: BrowserContext, role: Role, baseURL: string, ): Promise { if (role === 'anonymous') return; throw new Error(`applyAuth: wire role "${role}" to the project session mechanism (step 1)`); } ``` ### `ui-matrix/matrix.spec.ts` — the generic runner (do not edit per project) ```ts import { expect, test, type Locator, type Page } from '@playwright/test'; import { readFileSync } from 'node:fs'; import { join } from 'node:path'; import { applyAuth, type Role } from './matrix-auth'; type State = | 'present' | 'absent' | 'visible' | 'enabled' | 'disabled' | 'editable' | 'readonly' | 'required' | 'optional' | 'empty' | 'populated'; type ElementSpec = { testid?: string; role?: string; name?: string; state: State; count?: number }; type Entry = { route: string; roles: Role[]; deniedRoles?: Record; expectStatus?: number; elements?: ElementSpec[]; }; const matrix = JSON.parse( readFileSync(join(process.cwd(), 'ui-matrix/matrix.json'), 'utf8'), ) as Entry[]; function locate(page: Page, el: ElementSpec): Locator { if (el.testid) return page.getByTestId(el.testid); if (el.role && el.name) { return page.getByRole(el.role as Parameters[0], { name: el.name, exact: true }); } throw new Error(`element spec needs testid, or role+name: ${JSON.stringify(el)}`); } // Radix/shadcn render `aria-required`, native inputs render `required`. Accept either; demand one. async function assertRequired(loc: Locator, expected: boolean) { const native = await loc.getAttribute('required'); const aria = await loc.getAttribute('aria-required'); const isRequired = native !== null || aria === 'true'; expect(isRequired, `required=${expected}`).toBe(expected); } async function assertState(loc: Locator, spec: ElementSpec) { switch (spec.state) { case 'absent': return expect(loc).toHaveCount(spec.count ?? 0); case 'present': return expect(loc).toHaveCount(spec.count ?? 1); case 'visible': return expect(loc).toBeVisible(); case 'enabled': return expect(loc).toBeEnabled(); case 'disabled': return expect(loc).toBeDisabled(); case 'editable': return expect(loc).toBeEditable(); case 'readonly': return expect(loc).not.toBeEditable(); case 'required': return assertRequired(loc, true); case 'optional': return assertRequired(loc, false); case 'empty': return expect(loc).toBeEmpty(); case 'populated': return expect(loc).not.toBeEmpty(); } } for (const entry of matrix) { for (const role of entry.roles) { test(`${entry.route} | ${role} | elements`, async ({ page, context, baseURL }) => { const base = baseURL ?? 'http://localhost:4321'; await applyAuth(context, role, base); const consoleErrors: string[] = []; const failedRequests: string[] = []; const origin = new URL(base).origin; page.on('console', (m) => { if (m.type() === 'error') consoleErrors.push(m.text()); }); page.on('pageerror', (e) => consoleErrors.push(e.message)); page.on('response', (r) => { const u = new URL(r.url()); if (u.origin === origin && r.status() >= 400) failedRequests.push(`${r.status()} ${u.pathname}`); }); const resp = await page.goto(`${base}${entry.route}`, { waitUntil: 'domcontentloaded' }); expect(resp?.status(), `${entry.route} status`).toBe(entry.expectStatus ?? 200); for (const el of entry.elements ?? []) await assertState(locate(page, el), el); expect(consoleErrors, `console errors on ${entry.route}`).toHaveLength(0); expect(failedRequests, `failed requests on ${entry.route}`).toHaveLength(0); }); } for (const [role, denial] of Object.entries(entry.deniedRoles ?? {})) { test(`${entry.route} | ${role} | denied`, async ({ page, context, baseURL }) => { const base = baseURL ?? 'http://localhost:4321'; await applyAuth(context, role as Role, base); const resp = await page.goto(`${base}${entry.route}`, { waitUntil: 'domcontentloaded' }); if (denial === '403') { expect(resp?.status(), `${role} must get 403 on ${entry.route}`).toBe(403); } else { expect(page.url(), `${role} must be redirected away from ${entry.route}`).not.toContain(entry.route); expect(resp?.status(), `${role} redirect must land on a real page`).toBe(200); } }); } } ``` Extend with axe (`@axe-core/playwright`) and masked `toHaveScreenshot` per page × role for the full Layer-2 sweep once the skeleton is green. At scale (hundreds of cells) aggregate findings to a JSONL log instead of failing per-test. ## Completion gate - [ ] Step 1 discovery ran; no claim of absence lacks a grep behind it. - [ ] Existing harness extended, or none exists and the template was scaffolded. - [ ] Every testid, route, and role traced to source. - [ ] Matrix-covered elements carry `data-testid`. - [ ] No range status, no `404`-as-denial, no tautology, no conditional skip. - [ ] `validate-matrix.mjs` prints `VALID`. - [ ] Matrix executed; failures triaged as real regression or wrong row.