---
name: creating-ui-matrix-tests
description: Use when building granular per-page, per-role UI-element regression tests — asserting every button is clickable/disabled when it should be, every input/textarea/table/form renders per role, catching small UI regressions a flow test misses. Distinct from journey/flow tests. Ships a manifest-driven matrix harness template plus a deterministic validator that fails closed on rotted selectors, routes, and roles.
---

# Creating UI Matrix Tests

audience: AI coding agents first.

Build a **granular per-page × per-role UI-element regression matrix**: assert each element's expected state on each page, per role. Catches small UI bugs — a button enabled when it MUST be disabled, a table that stopped rendering, a textarea that lost maxlength — that flow tests skip.

**Distinct from `creating-journey-e2e-tests`.** Journeys = temporal, cross-page, one outcome, happy + failure branches (FLOW bugs). This = spatial, per-page, per-role element-state inventory (UI REGRESSIONS). Separate passes; never merge; never put element-state assertions in journey files.

## Iron law

**Never assert against a selector, route, or role you did not read from source.** A manifest row pointing at a nonexistent testid, a deleted route, or an invented role is a **silent no-op that always passes** — worse than no test, because it reports coverage it does not have.

Step 4's validator enforces this deterministically. Run it. Do not hand-check.

## Step 1 — Discover the harness; do NOT rebuild

Most mature repos already own a sweep/matrix harness. Extend it. Standing up a parallel one splits the source of truth and both rot.

Search before writing anything:

| Look for | Where | Never claim missing until |
|---|---|---|
| Existing sweep/matrix runner | `**/*sweep*`, `**/*matrix*`, `**/e2e/**` | grepped those globs |
| Auth/session mechanism | test configs + `**/helpers/**`, `**/fixtures/**` | grepped `storageState`, `addCookies`, `SignJWT`, `login` |
| Runners + scripts | `playwright.config*`, package scripts | listed them |
| Real roles | schema/RBAC enum | read the enum at its source |
| Real routes | framework route source (pages dir, router table, manifest) | enumerated them |
| Existing testids | `rg 'data-testid'` | counted them |

Record what you found with `path:line`. **A claim of absence with no grep behind it is a false blocker** — it sends the reader to build what they already own.

Found a harness → extend its manifest. Found none → scaffold from the template below.

## Step 2 — Three layers, strongest leverage first

Stop at the layer that covers the need.

**Layer 1 — Component-contract tests (do FIRST, highest leverage).** If the project mandates a shared component library (confirm the mandate exists), test each shared component's behavior contract ONCE at component level: Button (enabled/disabled/loading/click fires), Textarea (types/maxlength/required/error), Table (populated/empty/loading), Select, Dialog. Per-page work then collapses to "the RIGHT component is present in the RIGHT state" — not re-testing button mechanics on every page.

**Layer 2 — Generic per-page × per-role invariant sweep.** Assert real invariants on every page × role: a11y (axe) pass, zero unallowlisted console errors, zero same-origin failed requests, pixel-stable masked snapshot. NEVER `body` visible or `toHaveURL(/.*/)`.

**Layer 3 — Declarative element manifest (high-value pages only).** For pages where exact element state per role matters (checkout, admin tables, money forms), declare expected states as DATA; one generic runner asserts each row. Adding a page = a data entry, not a new test. Reserve for money/complex surfaces; Layers 1+2 cover the rest.

## Step 3 — Address elements by testid on matrix-covered surfaces

**Add `data-testid` to every element the matrix covers.** Accessible-name anchors (`getByRole('button', {name})`) break on any multi-locale site — the name changes per locale, so a name-anchored row silently matches nothing in the other locales. Testids are locale-invariant.

Rules:

- Address by `testid`, or by explicit `role` + `name`. NEVER by DOM ordinal.
- `name` REQUIRES an explicit `role`. An unscoped name cannot address inputs, textareas, tables, or regions.
- Derive every testid from real component source. Never invent one.

## Step 4 — Gate

```text
node <skill-dir>/scripts/validate-matrix.mjs <matrix.json> \
  --routes <routes-file> --roles <roles-file> --src <source-dir>
```

Generate `routes-file` and `roles-file` from the project's real route source and role enum (step 1). Every flag is REQUIRED — each powers one check; omitting one would silently disable it.

The validator is deterministic and free. It proves: every testid exists in source, every route is real, every role is real, every state is assertable, no duplicate rows, no vacuous rows, no `404`-as-denial, no range status. It does NOT prove semantics — that is the test run's job.

`INVALID` → fix and rerun. Never report a matrix while the gate is red.

**This gate is the anti-rot mechanism.** A matrix with no validator degrades silently as the app moves: routes get renamed, testids get dropped, roles get retired, and every stale row keeps reporting green.

## Quality bar

Every assertion MUST fail on a real regression.

Reject:

```ts
// DO NOT — passes on a blank error page
expect(resp?.status()).toBeLessThan(400);
// DO NOT — tautology
await expect(page.locator('body')).toBeVisible();
// DO NOT — silent skip
if (await el.isVisible()) { await expect(el).toBeEnabled(); }
```

Require one exact status, one exact state per row. Multiple legit states → one row each. `log()` any route or element intentionally excluded — silent truncation reads as "covered" when it is not.

**`404` is NEVER proof of authorization.** A deleted route returns `404` to everyone forever, and a `404`-accepting authz row passes permanently. Declare `redirect` or `403`.

---

## Portable harness template (scaffold when no harness exists)

Framework-agnostic Playwright matrix runner. Project supplies manifest data + one auth adapter; the runner is generic.

### `ui-matrix/matrix.json` — the data

```json
[
  {
    "route": "/en/dashboard/releases",
    "roles": ["artist"],
    "deniedRoles": { "anonymous": "redirect", "customer": "403" },
    "expectStatus": 200,
    "elements": [
      { "testid": "release-table", "state": "populated" },
      { "testid": "release-title", "state": "required" },
      { "role": "textbox", "name": "Catalog number", "state": "editable" }
    ]
  }
]
```

`state` ∈ `present | absent | visible | enabled | disabled | editable | readonly | required | optional | empty | populated`.
`count` applies to `present | absent` only. `deniedRoles` values ∈ `redirect | 403`.
Routes, roles, and testids above are ILLUSTRATIVE — replace with values grepped from the project.

### `ui-matrix/matrix-auth.ts` — the seam (project implements)

```ts
import type { BrowserContext } from '@playwright/test';

export type Role = string;

/** Authenticate `context` as `role` against `baseURL`. `anonymous` = no-op. */
export async function applyAuth(
  context: BrowserContext,
  role: Role,
  baseURL: string,
): Promise<void> {
  if (role === 'anonymous') return;
  throw new Error(`applyAuth: wire role "${role}" to the project session mechanism (step 1)`);
}
```

### `ui-matrix/matrix.spec.ts` — the generic runner (do not edit per project)

```ts
import { expect, test, type Locator, type Page } from '@playwright/test';
import { readFileSync } from 'node:fs';
import { join } from 'node:path';
import { applyAuth, type Role } from './matrix-auth';

type State =
  | 'present' | 'absent' | 'visible' | 'enabled' | 'disabled'
  | 'editable' | 'readonly' | 'required' | 'optional' | 'empty' | 'populated';

type ElementSpec = { testid?: string; role?: string; name?: string; state: State; count?: number };
type Entry = {
  route: string;
  roles: Role[];
  deniedRoles?: Record<Role, 'redirect' | '403'>;
  expectStatus?: number;
  elements?: ElementSpec[];
};

const matrix = JSON.parse(
  readFileSync(join(process.cwd(), 'ui-matrix/matrix.json'), 'utf8'),
) as Entry[];

function locate(page: Page, el: ElementSpec): Locator {
  if (el.testid) return page.getByTestId(el.testid);
  if (el.role && el.name) {
    return page.getByRole(el.role as Parameters<Page['getByRole']>[0], { name: el.name, exact: true });
  }
  throw new Error(`element spec needs testid, or role+name: ${JSON.stringify(el)}`);
}

// Radix/shadcn render `aria-required`, native inputs render `required`. Accept either; demand one.
async function assertRequired(loc: Locator, expected: boolean) {
  const native = await loc.getAttribute('required');
  const aria = await loc.getAttribute('aria-required');
  const isRequired = native !== null || aria === 'true';
  expect(isRequired, `required=${expected}`).toBe(expected);
}

async function assertState(loc: Locator, spec: ElementSpec) {
  switch (spec.state) {
    case 'absent': return expect(loc).toHaveCount(spec.count ?? 0);
    case 'present': return expect(loc).toHaveCount(spec.count ?? 1);
    case 'visible': return expect(loc).toBeVisible();
    case 'enabled': return expect(loc).toBeEnabled();
    case 'disabled': return expect(loc).toBeDisabled();
    case 'editable': return expect(loc).toBeEditable();
    case 'readonly': return expect(loc).not.toBeEditable();
    case 'required': return assertRequired(loc, true);
    case 'optional': return assertRequired(loc, false);
    case 'empty': return expect(loc).toBeEmpty();
    case 'populated': return expect(loc).not.toBeEmpty();
  }
}

for (const entry of matrix) {
  for (const role of entry.roles) {
    test(`${entry.route} | ${role} | elements`, async ({ page, context, baseURL }) => {
      const base = baseURL ?? 'http://localhost:4321';
      await applyAuth(context, role, base);

      const consoleErrors: string[] = [];
      const failedRequests: string[] = [];
      const origin = new URL(base).origin;
      page.on('console', (m) => { if (m.type() === 'error') consoleErrors.push(m.text()); });
      page.on('pageerror', (e) => consoleErrors.push(e.message));
      page.on('response', (r) => {
        const u = new URL(r.url());
        if (u.origin === origin && r.status() >= 400) failedRequests.push(`${r.status()} ${u.pathname}`);
      });

      const resp = await page.goto(`${base}${entry.route}`, { waitUntil: 'domcontentloaded' });
      expect(resp?.status(), `${entry.route} status`).toBe(entry.expectStatus ?? 200);

      for (const el of entry.elements ?? []) await assertState(locate(page, el), el);

      expect(consoleErrors, `console errors on ${entry.route}`).toHaveLength(0);
      expect(failedRequests, `failed requests on ${entry.route}`).toHaveLength(0);
    });
  }

  for (const [role, denial] of Object.entries(entry.deniedRoles ?? {})) {
    test(`${entry.route} | ${role} | denied`, async ({ page, context, baseURL }) => {
      const base = baseURL ?? 'http://localhost:4321';
      await applyAuth(context, role as Role, base);
      const resp = await page.goto(`${base}${entry.route}`, { waitUntil: 'domcontentloaded' });

      if (denial === '403') {
        expect(resp?.status(), `${role} must get 403 on ${entry.route}`).toBe(403);
      } else {
        expect(page.url(), `${role} must be redirected away from ${entry.route}`).not.toContain(entry.route);
        expect(resp?.status(), `${role} redirect must land on a real page`).toBe(200);
      }
    });
  }
}
```

Extend with axe (`@axe-core/playwright`) and masked `toHaveScreenshot` per page × role for the full Layer-2 sweep once the skeleton is green. At scale (hundreds of cells) aggregate findings to a JSONL log instead of failing per-test.

## Completion gate

- [ ] Step 1 discovery ran; no claim of absence lacks a grep behind it.
- [ ] Existing harness extended, or none exists and the template was scaffolded.
- [ ] Every testid, route, and role traced to source.
- [ ] Matrix-covered elements carry `data-testid`.
- [ ] No range status, no `404`-as-denial, no tautology, no conditional skip.
- [ ] `validate-matrix.mjs` prints `VALID`.
- [ ] Matrix executed; failures triaged as real regression or wrong row.
