# AWP Dogfood Live Factory Testing Policy

**Date:** 2026-08-19  
**Status:** Accepted owner testing policy for live Factory/AgentRun dogfooding.  
**Scope:** Live Factory/AgentRun tests only. This does not define production/customer model policy or the general Account/Model product design.

## 1. Purpose

Live Factory tests need cheap/throwaway agent capacity so AWP can repeatedly exercise execution, retry, checkpoint, cleanup, provider/account selection, ChangeSet and review/merge mechanics without consuming valuable model quota.

The model used for a test run is therefore chosen for the purpose of the test, not because it is the best coding model.

## 2. Primary live-test ladder

### Tier 1 — throwaway lifecycle load

Use Codex with:

```text
model: gpt-5.3-codex-spark
effort: low
account: any authorized account with Spark access
```

Use Spark for tests whose main purpose is to exercise the Factory/AgentRun lifecycle rather than obtain a high-quality code solution. It is acceptable if the model loops or produces low-value work when the test is about dispatch, streaming, cancellation, retry, persistence, cleanup, account selection, rate exhaustion, failure visibility or ChangeSet plumbing.

When Spark capacity is exhausted, rotate to another authorized Spark-capable account.

Account/model identity must be recorded on the `Attempt` that used it.

### Tier 2 — free fallback when a plausible completion matters

Use OpenCode with OpenCode Zen:

```text
model: deepseek-v4-flash-free
```

This is the preferred free fallback when Spark is exhausted or the test needs a materially more useful response than a looping throwaway model.

Free-model availability is volatile. Resolve/probe availability immediately before live execution rather than treating the current model ID as permanently guaranteed.

### Tier 3 — alternate free coding fallback

Use OpenCode with OpenRouter:

```text
model: cohere/north-mini-code:free
```

Use when the Zen/DeepSeek fallback is unavailable or North is a better fit for the specific coding-agent test.

OpenRouter free-model availability/rate limits are volatile and must be probed immediately before execution.

## 3. OpenRouter direct API versus OpenCode

For **Factory/AgentRun end-to-end tests**, prefer North through the OpenCode agent harness rather than calling the raw OpenRouter model endpoint directly.

Reason:

```text
Factory E2E is testing
  agent harness
  + tool execution
  + workspace behavior
  + streaming/events
  + cancellation/retry
  + change production

Raw OpenRouter API tests only
  model/provider inference connectivity
```

A direct OpenRouter `cohere/north-mini-code:free` call is appropriate for provider/model connectivity, capability discovery or ModelGateway tests. It is not equivalent to a coding-agent Factory run unless AWP itself later owns an equivalent tool-using coding-agent harness.

## 4. Credential handling

Do not scrape or copy the OpenRouter API key out of OpenCode settings on every run.

If an existing local OpenCode configuration already contains the credential, bootstrap/import it once into the AWP connection/secret mechanism or explicitly provision the test environment from that owner-controlled source.

Target model:

```text
OpenRouter Connection
  -> CredentialReference
  -> SecretStore
```

The credential must not be copied into Plan, Task, FactoryRun, AgentRun, Attempt, CI or ordinary project configuration.

OpenCode may consume an attempt-scoped credential/environment projection; the canonical reusable secret remains behind `CredentialReference`.

## 5. Account rotation and Attempt provenance

Account/model selection is sticky for one `Attempt`.

Do not silently switch accounts/models halfway through the same Attempt because that destroys provenance and makes rate/failure behavior difficult to reason about.

When a selected account is exhausted or otherwise unavailable:

```text
Attempt N
  -> rate/capacity exhaustion
  -> terminal/retryable failure recorded
  -> checkpoint/WIP preserved
  -> select next eligible account/model
  -> Attempt N+1
```

The new Attempt records its own provider, account, model and reason for selection.

## 6. Test-purpose routing

```text
lifecycle/plumbing/chaos test
  -> Spark low effort first

need a real code response
  -> DeepSeek V4 Flash Free via OpenCode Zen

Zen unavailable / North preferred
  -> North Mini Code Free via OpenCode + OpenRouter

provider connectivity only
  -> direct provider API is acceptable
```

Do not spend a high-value model on a test whose pass condition does not depend on answer quality.

## 7. Live-test safety

- Normal AgentRuns execute on the K3s/build execution plane, not the user's workstation.
- Live Factory tests use isolated disposable/recoverable workspaces.
- No live test agent receives reusable Git publication or production credentials.
- Test repositories/branches must be disposable or explicitly test-scoped unless the acceptance journey intentionally exercises trusted publication.
- AWP must preserve WIP/checkpoints before destructive retry/cleanup.

## 8. Current provider facts are probes, not architecture

As of 2026-08-19, official provider documentation exposes:

```text
OpenCode Zen: deepseek-v4-flash-free
OpenRouter:    cohere/north-mini-code:free
```

Those are current test selections, not permanent AWP architecture. The product model remains Provider / Model / Account / AccountSelectionPolicy with provider-specific model IDs behind adapters.

If a free model disappears, replace the test alias with the next suitable zero-cost model without redesigning Factory semantics.

## 9. Stable test aliases

Implementation should prefer stable AWP test-purpose aliases over scattering vendor model IDs through fixtures:

```text
factory-throwaway
factory-free-useful
factory-free-fallback
```

Current resolution:

```text
factory-throwaway      -> Codex / gpt-5.3-codex-spark / low
factory-free-useful    -> OpenCode Zen / deepseek-v4-flash-free
factory-free-fallback  -> OpenCode + OpenRouter / cohere/north-mini-code:free
```

The resolved provider/account/model is persisted on each Attempt for audit/reproducibility.

## 10. Relationship to product policy

This document is a dogfood/testing policy only.

Later product specs may define richer account selection, cost/rate routing, health scoring and customer-configurable model policies. Those later mechanisms must be able to express this dogfood policy without hard-coding these model names into Factory domain logic.