# System: AI Infrastructure Layer — Implementation Plan **Spec:** docs/specs/2026-05-31-system-ai.md · **Slug:** system-ai · **Wave:** 2 **Depends on:** foundation-monorepo, foundation-auth-rbac ## Goal Build the provider-agnostic AI infrastructure layer that every Zync module consumes through a single `callAI()` function. It exposes an adapter interface for Anthropic / OpenAI / Google, a fallback-chain executor that silently retries retryable errors across an ordered model list, and a credit-accounting system that abstracts raw tokens into tenant-facing quota percentages and dollar spend. Zync admins configure models, pricing, quotas, and per-use-case system prompts centrally at `/admin/ai`; tenants manage personality, extra-usage, and purchased credits at `/settings/ai`. ## Architecture A new shared package `packages/ai` (published as `@zync/ai`) holds the adapters, executor, accounting middleware, system-prompt assembly, and the public `callAI()` entry point. It depends on `packages/db` (Drizzle schema + query helpers) and `packages/types` (enums/DTOs). Data flow for an AI call: caller → `callAI()` → load `ai_global_config` + `ai_tenant_settings` → assemble system prompt → `withCreditAccounting()` checks quota via foundation `usage_counters` (counter keys `ai_tokens_quota` / `ai_tokens_extra` / `ai_tokens_purchased`) → `executeWithFallback()` runs the main model then ordered backups → on success deduct tokens, compute USD cost from `ai_model_pricing`, write `ai_usage_log`. Admin and tenant HTTP routes live in `apps/zync-api` and reuse foundation `requireAdminSession()`, `requirePermission()`, `requireTier()`. Upstream tables consumed: `tenants(id)`, `users(id)`, `admin_users(id)` (all UUID PKs), `usage_counters(tenant_id, counter_key, period, count)`. Upstream exports consumed: `incrementCounter` / `checkCounterLimit` / `QuotaExceededError` from `packages/db/src/queries/usage.ts`, `meetsMinimumTier` / `TenantTier` from `packages/auth`, `tenantQuery` / `systemQuery` from `packages/db/src/queries`. ## Tech Stack - **Package:** `packages/ai` (`@zync/ai`) — TypeScript, edge-runtime safe (no Node built-ins). - **DB:** Neon Postgres via Hyperdrive (`DB` binding), Drizzle ORM; schema files in `packages/db/src/schema/ai.ts`, queries in `packages/db/src/queries/ai.ts`. - **API:** `apps/zync-api` (Hono), Zod-validated routes, foundation auth/tier middleware. - **Admin UI:** `apps/zync-app` route group under `/admin/ai` (system-admin only); **Tenant UI:** `/settings/ai`. React + TanStack Query v5 + `packages/ui` primitives. - **Adapters:** `@anthropic-ai/sdk`, `openai`, `@google/generative-ai` — invoked via `fetch`-compatible clients. - **Bindings/secrets:** `ANTHROPIC_API_KEY` (existing), new `OPENAI_API_KEY`, `GOOGLE_AI_API_KEY`; `DB`, `KV`. ## Wave Plan | Sub-wave | Tasks | Files touched | Parallelizable? | |----------|-------|---------------|-----------------| | A — Schema | 1 | `packages/db/src/schema/ai.ts`, migration | No (foundation of all) | | B — Package core | 2, 3, 4 | `packages/ai/src/{adapter,adapters/*,executor}.ts` | 3 after 2; 4 after 2 | | C — Queries + accounting + prompt | 5, 6, 7, 8 | `packages/db/src/queries/ai.ts`, `packages/ai/src/{accounting,prompt,index}.ts` | 5 parallel to B; 6,7,8 after 1,2,3,4,5 | | D — Admin API | 9, 10 | `apps/zync-api/src/routes/admin/ai.ts` | After 5; 10 after 9 | | E — Tenant API | 11 | `apps/zync-api/src/routes/ai.ts` | After 6,8 | | F — Admin UI | 12 | `apps/zync-app/src/modules/admin-ai/**` | After 9,10 | | G — Tenant UI | 13 | `apps/zync-app/src/modules/settings-ai/**` | After 11 | | H — Seed + bindings + tests | 14, 15, 16 | seed, `wrangler.toml`, test files | 14 after 1; 15 anytime; 16 last | ## Tasks ### Task 1: AI database schema & migration **Blocks:** 5, 9, 14 · **Blocked by:** — **Files:** - Create: `packages/db/src/schema/ai.ts` - Create: `packages/db/migrations/00XX_ai_infrastructure.sql` - Modify: `packages/db/src/schema/index.ts` (re-export ai schema) **Steps:** - [ ] Define all six AI tables as Drizzle pgTable definitions matching the DDL below verbatim. - [ ] Add the two `ai_usage_log` indexes and ensure `backup_model_ids`, `use_case_prompts`, `use_case_overrides` are `jsonb`. - [ ] Generate the SQL migration via drizzle-kit and hand-verify it matches the canonical DDL (UUID PKs, TIMESTAMPTZ, BOOLEAN, JSONB, NUMERIC, CHECK constraints, single-row `ai_global_config`). - [ ] Confirm FKs reference existing foundation tables: `tenants(id)`, `users(id)`, `admin_users(id)`. **Schema / Interfaces:** ```sql -- Per-tenant personality, preferences, extra-usage toggle CREATE TABLE ai_tenant_settings ( tenant_id UUID PRIMARY KEY REFERENCES tenants(id) ON DELETE CASCADE, personality_prompt TEXT, use_case_overrides JSONB NOT NULL DEFAULT '{}', extra_usage_enabled BOOLEAN NOT NULL DEFAULT false, extra_spend_limit_usd NUMERIC(10,2), auto_reload_enabled BOOLEAN NOT NULL DEFAULT false, auto_reload_amount_usd NUMERIC(10,2), updated_at TIMESTAMPTZ NOT NULL DEFAULT now() ); -- Per-tier monthly AI quota configuration (Zync admin managed) CREATE TABLE ai_tier_quotas ( tier TEXT NOT NULL CHECK (tier IN ('freelancer', 'business', 'enterprise', 'white_label')), model_id TEXT NOT NULL, monthly_tokens BIGINT NOT NULL, extra_allowed BOOLEAN NOT NULL DEFAULT false, extra_max_usd NUMERIC(10,2), effective_from DATE NOT NULL, effective_to DATE, PRIMARY KEY (tier, effective_from) ); -- Token pricing per model (admin-configurable) CREATE TABLE ai_model_pricing ( model_id TEXT PRIMARY KEY, provider TEXT NOT NULL CHECK (provider IN ('anthropic', 'openai', 'google')), label TEXT NOT NULL, input_cost_per_1m NUMERIC(12,6) NOT NULL, output_cost_per_1m NUMERIC(12,6) NOT NULL, is_vision_capable BOOLEAN NOT NULL DEFAULT false, active BOOLEAN NOT NULL DEFAULT true, created_at TIMESTAMPTZ NOT NULL DEFAULT now(), updated_at TIMESTAMPTZ NOT NULL DEFAULT now() ); -- Global AI configuration (single row enforced) CREATE TABLE ai_global_config ( id INT PRIMARY KEY DEFAULT 1 CHECK (id = 1), main_model_id TEXT NOT NULL REFERENCES ai_model_pricing(model_id), backup_model_ids JSONB NOT NULL DEFAULT '[]', use_case_prompts JSONB NOT NULL DEFAULT '{}', updated_at TIMESTAMPTZ NOT NULL DEFAULT now(), updated_by UUID REFERENCES admin_users(id) ); -- Per-AI-call usage and cost log CREATE TABLE ai_usage_log ( id UUID PRIMARY KEY DEFAULT gen_random_uuid(), tenant_id UUID NOT NULL REFERENCES tenants(id) ON DELETE CASCADE, user_id UUID REFERENCES users(id), use_case TEXT NOT NULL, model_id TEXT NOT NULL, provider TEXT NOT NULL CHECK (provider IN ('anthropic', 'openai', 'google')), input_tokens INT NOT NULL, output_tokens INT NOT NULL, total_tokens INT NOT NULL, cost_usd NUMERIC(12,6) NOT NULL, billed_from TEXT NOT NULL CHECK (billed_from IN ('quota', 'extra', 'purchased')), duration_ms INT NOT NULL, entity_type TEXT, entity_id UUID, error TEXT, created_at TIMESTAMPTZ NOT NULL DEFAULT now() ); CREATE INDEX ai_usage_log_tenant_period ON ai_usage_log(tenant_id, date_trunc('month', created_at)); CREATE INDEX ai_usage_log_use_case ON ai_usage_log(tenant_id, use_case, created_at); -- Purchased extra credits (beyond monthly quota) CREATE TABLE ai_credit_purchases ( id UUID PRIMARY KEY DEFAULT gen_random_uuid(), tenant_id UUID NOT NULL REFERENCES tenants(id) ON DELETE CASCADE, amount_usd NUMERIC(10,2) NOT NULL, tokens_granted BIGINT NOT NULL, tokens_remaining BIGINT NOT NULL, payment_reference TEXT, expires_at TIMESTAMPTZ, created_at TIMESTAMPTZ NOT NULL DEFAULT now() ); ``` Monthly quota consumption is tracked through the foundation `usage_counters(tenant_id, counter_key, period, count)` table — no new counter table. Counter keys: `ai_tokens_quota`, `ai_tokens_extra`, `ai_tokens_purchased`; `period` = `'YYYY-MM'`. **Acceptance:** - [ ] Migration applies cleanly to a Neon branch; all six tables exist with correct types. - [ ] Inserting a second `ai_global_config` row fails the `CHECK (id = 1)`. - [ ] All FKs are UUID→UUID and resolve against foundation tables. ### Task 2: Adapter contract & shared types **Blocks:** 3, 4, 6, 8 · **Blocked by:** — **Files:** - Create: `packages/ai/package.json` (`@zync/ai`, deps: `@anthropic-ai/sdk`, `openai`, `@google/generative-ai`, `@zync/db`, `@zync/types`) - Create: `packages/ai/tsconfig.json` - Create: `packages/ai/src/adapter.ts` - Create: `packages/ai/src/errors.ts` - Create: `packages/ai/src/use-cases.ts` **Steps:** - [ ] Scaffold the `@zync/ai` package with strict TS config extending `packages/config/tsconfig.base.json`. - [ ] Define `AIMessage`, `AIContentBlock`, `AIRequest`, `AIResponse`, `AIAdapter` interfaces verbatim. - [ ] Define `AIProviderError` with the five error codes and `retryable` flag in `errors.ts`. - [ ] Define `AIUseCase` union type and `AI_USE_CASES` const array with all seven keys. **Schema / Interfaces:** ```ts // packages/ai/src/adapter.ts export interface AIMessage { role: 'system' | 'user' | 'assistant'; content: string | AIContentBlock[] } export interface AIContentBlock { type: 'text' | 'image'; text?: string; image?: { mediaType: string; data: string } } export interface AIRequest { messages: AIMessage[]; model: string; maxTokens?: number; temperature?: number; stream?: boolean } export interface AIResponse { content: string; inputTokens: number; outputTokens: number; model: string; provider: string; durationMs: number } export interface AIAdapter { provider: 'anthropic' | 'openai' | 'google' call(request: AIRequest): Promise stream(request: AIRequest): ReadableStream } // packages/ai/src/errors.ts export class AIProviderError extends Error { constructor( public code: 'rate_limit' | 'timeout' | 'model_unavailable' | 'auth_error' | 'unknown', public provider: string, public retryable: boolean, message: string, ) { super(message) } } export class QuotaExceededError extends Error {} // re-exported from @zync/db usage helper for credit blocks export class ExtraSpendLimitError extends Error {} // maps to HTTP 402 // packages/ai/src/use-cases.ts export type AIUseCase = | 'expense_ocr' | 'expense_tax_eval' | 'task_autocreate' | 'ai_assistant' | 'telegram_assistant' | 'kb_suggest' | 'invoice_extract' export const AI_USE_CASES: AIUseCase[] = [ 'expense_ocr', 'expense_tax_eval', 'task_autocreate', 'ai_assistant', 'telegram_assistant', 'kb_suggest', 'invoice_extract', ] ``` **Acceptance:** - [ ] `@zync/ai` typechecks in isolation; exports resolve from `packages/ai/src/index.ts` stub. ### Task 3: Provider adapter implementations **Blocks:** 4 · **Blocked by:** 2 **Files:** - Create: `packages/ai/src/adapters/anthropic.ts` - Create: `packages/ai/src/adapters/openai.ts` - Create: `packages/ai/src/adapters/google.ts` - Create: `packages/ai/src/adapters/index.ts` (`getAdapter(provider)`) **Steps:** - [ ] Implement `AnthropicAdapter` using `@anthropic-ai/sdk`; map `AIContentBlock` images to Claude Vision image blocks (base64 + mediaType); read key from `env.ANTHROPIC_API_KEY`. - [ ] Implement `OpenAIAdapter` (`gpt-4o`, `o1`) reading `env.OPENAI_API_KEY`; map image blocks to OpenAI `image_url` data-URI form. - [ ] Implement `GoogleAdapter` (Gemini) reading `env.GOOGLE_AI_API_KEY`. - [ ] In each adapter, normalize provider errors into `AIProviderError`: 429 → `rate_limit` (retryable), network/abort → `timeout` (retryable), 404/model-not-found → `model_unavailable` (retryable), 401/403 → `auth_error` (NOT retryable), else `unknown` (NOT retryable). - [ ] Populate `AIResponse.inputTokens`/`outputTokens` from each provider's usage metadata; set `durationMs` from a monotonic timer around the call. - [ ] `getAdapter(provider, env)` returns the adapter instance for `'anthropic' | 'openai' | 'google'`. **Schema / Interfaces:** ```ts // packages/ai/src/adapters/index.ts import type { AIAdapter } from '../adapter' export function getAdapter(provider: 'anthropic' | 'openai' | 'google', env: AIEnv): AIAdapter export interface AIEnv { ANTHROPIC_API_KEY: string; OPENAI_API_KEY: string; GOOGLE_AI_API_KEY: string } ``` **Acceptance:** - [ ] Each adapter returns a populated `AIResponse` against a mocked provider client. - [ ] A 429 from any provider surfaces as `AIProviderError{ code:'rate_limit', retryable:true }`; a 401 as `retryable:false`. ### Task 4: Fallback chain executor **Blocks:** 8 · **Blocked by:** 2, 3 **Files:** - Create: `packages/ai/src/executor.ts` **Steps:** - [ ] Implement `withTimeout(promise, ms)` via `Promise.race` with a `setTimeout`-based rejection that throws `AIProviderError{ code:'timeout', retryable:true }`. - [ ] Implement `executeWithFallback(request, settings, opts)`: build chain `[mainModel, ...backupModels]`; for each, call `getAdapter(provider).call({ ...request, model })` wrapped in `withTimeout` (default 30_000ms). - [ ] On `AIProviderError` with `retryable === true`: store as `lastError`, continue to next model. - [ ] On non-retryable error: rethrow immediately (surface bad-prompt/auth failures). - [ ] If chain exhausts: throw `Error("All models failed. Last error: ...")`. **Schema / Interfaces:** ```ts // packages/ai/src/executor.ts export interface ModelConfig { provider: 'anthropic' | 'openai' | 'google'; model: string; label: string } export interface AISettings { mainModel: ModelConfig; backupModels: ModelConfig[] } export async function executeWithFallback( request: AIRequest, settings: AISettings, opts?: { timeoutMs?: number }, ): Promise ``` **Acceptance:** - [ ] Main model rate-limited → executor returns the first backup's response. - [ ] Main model auth_error → executor rethrows without trying backups. - [ ] All models timeout → executor throws "All models failed". ### Task 5: AI Drizzle query helpers **Blocks:** 6, 9, 11 · **Blocked by:** 1 **Files:** - Create: `packages/db/src/queries/ai.ts` **Steps:** - [ ] Implement `getGlobalConfig(db)` → the single `ai_global_config` row joined to resolve `main_model_id` and `backup_model_ids` into `ModelConfig[]` (label/provider from `ai_model_pricing`). - [ ] Implement `updateGlobalConfig(db, { mainModelId, backupModelIds, useCasePrompts, updatedBy })`. - [ ] Implement `listModelPricing(db, { activeOnly? })`, `getModelPricing(db, modelId)`, `addModel(db, row)`, `updateModelPricing(db, modelId, patch)`, `setModelActive(db, modelId, active)`. - [ ] Implement `getTierQuotas(db)` (current rows: `effective_to IS NULL`), `upsertTierQuota(db, tier, patch)` — sets prior current row's `effective_to = today`, inserts new row with `effective_from = today`. - [ ] Implement `getTenantSettings(db, tenantId)` (returns defaults if no row), `upsertTenantSettings(db, tenantId, patch)`. - [ ] Implement `insertUsageLog(db, row)`; `listUsageLog(db, filter, page)` (filters: tenant, useCase, model, date range); `tenantUsageBreakdown(db, tenantId, range)` (calls + % per use case). - [ ] Implement `getCreditPurchases(db, tenantId)`, `insertCreditPurchase(db, row)`, `deductPurchasedCredit(db, tenantId, tokens)` (FIFO over non-expired rows with `tokens_remaining > 0`). - [ ] Implement analytics aggregations: `tokensByUseCase(db, period)`, `costByTenant(db, period)`, `errorRateByModel(db, period)`, `costTrend(db, months)`. **Schema / Interfaces:** ```ts // packages/db/src/queries/ai.ts export function getGlobalConfig(db: Db): Promise<{ mainModel: ModelConfig; backupModels: ModelConfig[]; useCasePrompts: Record }> export function getModelPricing(db: Db, modelId: string): Promise<{ inputCostPer1m: number; outputCostPer1m: number; provider: string; isVisionCapable: boolean } | null> export function getTenantSettings(db: Db, tenantId: string): Promise export function insertUsageLog(db: Db, row: AIUsageLogInsert): Promise export function deductPurchasedCredit(db: Db, tenantId: string, tokens: number): Promise // tokens actually drawn ``` **Acceptance:** - [ ] `getGlobalConfig` returns resolved `ModelConfig` objects, not raw model-id strings. - [ ] `upsertTierQuota` closes the previous current row and the new row is the only one with `effective_to IS NULL`. ### Task 6: Credit accounting & cost conversion **Blocks:** 8, 11 · **Blocked by:** 2, 5 **Files:** - Create: `packages/ai/src/accounting.ts` - Create: `packages/ai/src/pricing.ts` **Steps:** - [ ] Implement `calculateCost(db, { inputTokens, outputTokens }, modelId)`: `(inputTokens/1e6)*inputCostPer1m + (outputTokens/1e6)*outputCostPer1m` from `ai_model_pricing`. - [ ] Implement `usdToTokens(db, usd, modelId)`: blended rate `(input+output)/2` per 1M → `floor((usd/blended)*1e6)`. Use main model's pricing for purchases. - [ ] Implement `getQuotaStatus(tenantId)`: resolve tier from membership, read tier quota `monthly_tokens` from `ai_tier_quotas`, read consumed from `usage_counters` key `ai_tokens_quota` period `YYYY-MM`; return `{ percentUsed, tokensUsed, tokensTotal, tokensRemaining, resetsAt }`. - [ ] Implement `isExtraUsageAllowed(tenantId)`: tier `extra_allowed` AND `ai_tenant_settings.extra_usage_enabled`. - [ ] Implement `checkExtraSpendLimit(tenantId)`: extra spend this month (sum `ai_usage_log.cost_usd` where `billed_from='extra'`) vs `min(tenant.extra_spend_limit_usd ?? tier.extra_max_usd, tier.extra_max_usd)`; throw `ExtraSpendLimitError` (HTTP 402) if exceeded. - [ ] Implement `deductAndLog(tenantId, info)`: in ONE DB transaction — increment the correct `usage_counters` key (`ai_tokens_quota`/`ai_tokens_extra`/`ai_tokens_purchased`), for purchased path also `deductPurchasedCredit`, then `insertUsageLog`. Determine `billed_from` from which bucket paid. - [ ] Implement `withCreditAccounting(tenantId, useCase, fn)` exactly per spec: pre-check quota → if exhausted check extra allowed + spend limit → run `fn()` → compute cost → `deductAndLog` → return response. On `fn()` throw, log a failed `ai_usage_log` row with `error` set and zero billing. **Schema / Interfaces:** ```ts // packages/ai/src/pricing.ts export function calculateCost(db: Db, t: { inputTokens: number; outputTokens: number }, modelId: string): Promise export function usdToTokens(db: Db, usd: number, modelId: string): Promise // packages/ai/src/accounting.ts export interface QuotaStatus { percentUsed: number; tokensUsed: number; tokensTotal: number; tokensRemaining: number; resetsAt: string } export function getQuotaStatus(db: Db, tenantId: string): Promise export function isExtraUsageAllowed(db: Db, tenantId: string): Promise export function checkExtraSpendLimit(db: Db, tenantId: string): Promise // throws ExtraSpendLimitError export function withCreditAccounting(ctx: AICtx, tenantId: string, useCase: AIUseCase, fn: () => Promise): Promise ``` **Acceptance:** - [ ] Quota-exhausted tenant with extra disabled → `withCreditAccounting` throws `QuotaExceededError` before calling `fn`. - [ ] Extra-enabled tenant past spend limit → throws `ExtraSpendLimitError` (402). - [ ] Successful call writes exactly one `ai_usage_log` row and increments one `usage_counters` key in the same transaction. - [ ] `calculateCost` for 1M input + 1M output at $3/$15 returns 18.000000. ### Task 7: System-prompt assembly **Blocks:** 8 · **Blocked by:** 5 **Files:** - Create: `packages/ai/src/prompt.ts` **Steps:** - [ ] Implement `assembleSystemPrompt(db, tenantId, useCase)`: base = `ai_global_config.use_case_prompts[useCase]`; append `ai_tenant_settings.personality_prompt` (if set); append `ai_tenant_settings.use_case_overrides[useCase]` (if set), joined by blank lines, in that exact order. - [ ] Tenant text only appends — it can never replace the admin base (enforce by concatenation order, never substitution). - [ ] Enforce caps at assembly read-time defensively: personality ≤ 500 chars, per-use-case override ≤ 300 chars (truncate + warn if a stored value somehow exceeds). **Schema / Interfaces:** ```ts // packages/ai/src/prompt.ts export function assembleSystemPrompt(db: Db, tenantId: string, useCase: AIUseCase): Promise ``` **Acceptance:** - [ ] With base + personality + override all set, output is base, then personality, then override, in order. - [ ] With only base set, output equals the base prompt. ### Task 8: Public `callAI()` entry point **Blocks:** 11 · **Blocked by:** 2, 4, 6, 7 **Files:** - Create: `packages/ai/src/index.ts` **Steps:** - [ ] Implement `callAI(ctx, opts)`: load `getGlobalConfig` → build `AISettings`; `assembleSystemPrompt`; prepend assembled system message to `opts.messages`; wrap `executeWithFallback` inside `withCreditAccounting(tenantId, useCase, ...)`; pass `entityType`/`entityId` through to the usage log. - [ ] Re-export public surface: `callAI`, `executeWithFallback`, `AIProviderError`, `QuotaExceededError`, `ExtraSpendLimitError`, types `AIMessage`/`AIRequest`/`AIResponse`/`AIUseCase`/`ModelConfig`/`AISettings`, `getAdapter`, `assembleSystemPrompt`, `calculateCost`, `usdToTokens`, `getQuotaStatus`. - [ ] Assert vision capability for `expense_ocr`/`invoice_extract` when image blocks are present: skip non-vision models in the fallback chain (filter chain by `ai_model_pricing.is_vision_capable`). **Schema / Interfaces:** ```ts // packages/ai/src/index.ts — public API for other Workers (@zync/ai) export async function callAI(ctx: AICtx, opts: { tenantId: string userId?: string useCase: AIUseCase messages: AIMessage[] maxTokens?: number entityType?: string entityId?: string }): Promise export interface AICtx { db: Db; env: AIEnv } ``` **Acceptance:** - [ ] `callAI` assembles system prompt, executes with fallback, accounts credit, and returns `AIResponse`. - [ ] An OCR call with an image block never routes to a model where `is_vision_capable = false`. ### Task 9: Admin AI config & model API routes **Blocks:** 10, 12 · **Blocked by:** 5 **Files:** - Create: `apps/zync-api/src/routes/admin/ai.ts` - Modify: `apps/zync-api/src/routes/admin/index.ts` (mount router) **Steps:** - [ ] Mount under `/admin/ai`, every route behind `requireAdminSession()`. - [ ] `GET /admin/ai/config` → `getGlobalConfig`; `PUT /admin/ai/config` → Zod-validate `{ mainModelId, backupModelIds: string[], useCasePrompts: Record }` → `updateGlobalConfig` with `updatedBy = session.sub`. - [ ] `GET /admin/ai/models` → `listModelPricing`; `POST /admin/ai/models` (Zod) → `addModel`; `PUT /admin/ai/models/:modelId` (Zod) → `updateModelPricing`; `PATCH /admin/ai/models/:modelId/active` (Zod `{ active: boolean }`) → `setModelActive` (soft delete preserves historical cost records). - [ ] `GET /admin/ai/quotas` → `getTierQuotas`; `PUT /admin/ai/quotas/:tier` (Zod) → `upsertTierQuota` (effective_from = today, closes prior row). - [ ] All bodies validated via Zod before business logic (`require-zod-validation-in-routes`). **Schema / Interfaces:** ``` GET /admin/ai/config PUT /admin/ai/config GET /admin/ai/models POST /admin/ai/models PUT /admin/ai/models/:modelId PATCH /admin/ai/models/:modelId/active GET /admin/ai/quotas PUT /admin/ai/quotas/:tier ``` **Acceptance:** - [ ] Non-admin session → 401/403 on every `/admin/ai/*` route. - [ ] Deactivating a model removes it from selection but its historical `ai_usage_log`/cost rows remain. ### Task 10: Admin usage log & analytics API routes **Blocks:** 12 · **Blocked by:** 9 **Files:** - Modify: `apps/zync-api/src/routes/admin/ai.ts` **Steps:** - [ ] `GET /admin/ai/usage` → paginated `listUsageLog` with filters `tenant`, `use_case`, `model`, date range; support `?format=csv` for CSV export. - [ ] `GET /admin/ai/analytics` → `{ tokensByUseCase, costByTenant, errorRateByModel, costTrend }` for the requested period (default current month; trend = past 6 months). - [ ] Validate query params via Zod; behind `requireAdminSession()`. **Schema / Interfaces:** ``` GET /admin/ai/usage ?tenant&use_case&model&from&to&page&format=csv GET /admin/ai/analytics ?period ``` **Acceptance:** - [ ] Usage endpoint paginates and filters correctly; CSV export returns `text/csv`. - [ ] Analytics returns all four aggregations. ### Task 11: Tenant AI settings, quota, usage & credits API routes **Blocks:** 13 · **Blocked by:** 5, 6, 8 **Files:** - Create: `apps/zync-api/src/routes/ai.ts` - Modify: `apps/zync-api/src/routes/index.ts` (mount `/api/ai`) **Steps:** - [ ] Mount under `/api/ai`; all routes pass auth middleware; mutating routes require `settings:write` via `requirePermission('settings:write')`; settings/personality/extra features require `requireTier('business')`. - [ ] `GET /api/ai/settings` → `getTenantSettings`; `PUT /api/ai/settings` (Zod: `{ personalityPrompt?: string(≤500), useCaseOverrides?: Record, extraUsageEnabled?: boolean, extraSpendLimitUsd?: number }`) → `upsertTenantSettings`; cap `extraSpendLimitUsd` at tier `extra_max_usd`. - [ ] `GET /api/ai/quota` → `getQuotaStatus` → `{ percentUsed, tokensUsed, tokensTotal, resetsAt }` (raw token counts available to UI for the bar fill but UI shows % only). - [ ] `GET /api/ai/usage` → `tenantUsageBreakdown` (calls + % per use case; features shown by label). - [ ] `GET /api/ai/credits` → `getCreditPurchases` balance + history; convert `tokens_remaining` → USD at current main-model rate. - [ ] `POST /api/ai/credits/purchase` (Zod `{ amountUsd: number }`, 5≤amount≤100 for custom; presets $5/$10/$25/$50) → `usdToTokens` at lock-in rate → create payment intent via payment adapter handoff → on confirmation `insertCreditPurchase` with `tokens_granted`/`tokens_remaining`; returns payment intent. Behind `requireTier('business')`. - [ ] Freelancer tier: settings/personality/credits routes return 402 (OCR-only tier; AI features require Business+). **Schema / Interfaces:** ``` GET /api/ai/settings PUT /api/ai/settings GET /api/ai/quota -> { percentUsed, tokensUsed, tokensTotal, resetsAt } GET /api/ai/usage GET /api/ai/credits POST /api/ai/credits/purchase -> { paymentIntent } ``` **Acceptance:** - [ ] Freelancer tenant → 402 on `/api/ai/settings` PUT and `/api/ai/credits/purchase`. - [ ] `extraSpendLimitUsd` above tier `extra_max_usd` is clamped to the tier max. - [ ] `/api/ai/quota` never returns the dollar-to-token rate. ### Task 12: Admin `/admin/ai` dashboard UI **Blocks:** — · **Blocked by:** 9, 10 **Files:** - Create: `apps/zync-app/src/modules/admin-ai/index.tsx` (lazy route) - Create: `apps/zync-app/src/modules/admin-ai/ModelConfigPanel.tsx` - Create: `apps/zync-app/src/modules/admin-ai/ModelPricingTable.tsx` - Create: `apps/zync-app/src/modules/admin-ai/UseCasePrompts.tsx` - Create: `apps/zync-app/src/modules/admin-ai/TierQuotaTable.tsx` - Create: `apps/zync-app/src/modules/admin-ai/UsageLog.tsx` - Create: `apps/zync-app/src/modules/admin-ai/AnalyticsDashboard.tsx` - Create: `apps/zync-app/src/modules/admin-ai/api.ts` (TanStack Query hooks) **Steps:** - [ ] Main model selector: dropdown of `active = true` models; backup list as drag-to-reorder ordered list showing `label (provider)`, add/remove; persist `backup_model_ids` order via `PUT /admin/ai/config`. - [ ] Model pricing table: columns label, provider, model ID, input cost/1M, output cost/1M, vision toggle, active toggle; Add (form), Edit (inline), Deactivate (soft). - [ ] Use-case prompts: one expandable row per `AI_USE_CASES` key with a textarea; save to `use_case_prompts`; "Test" button POSTs prompt + sample payload to main model and shows raw response. - [ ] Per-tier quota table: row per tier — monthly token quota (int), extra allowed (toggle), max extra spend/month USD (shown only if extra allowed); save via `PUT /admin/ai/quotas/:tier`. - [ ] Usage table: paginated, filter by tenant/use case/model/date range, CSV export button. - [ ] Analytics: tokens-by-use-case bar chart, cost-by-tenant ranked table, error-rate-by-model line chart, 6-month cost-trend area chart. - [ ] A11y: drag-reorder list keyboard-operable (move up/down buttons + `aria-grabbed`); tables use ``; charts have text-equivalent data tables. Respect `prefers-reduced-motion` for any chart transitions. RTL-safe layout via logical CSS properties. **Acceptance:** - [ ] Reordering backups persists and is reflected in fallback order. - [ ] Deactivated models disappear from the main/backup selectors. - [ ] Keyboard-only user can reorder the backup list and operate all toggles. ### Task 13: Tenant `/settings/ai` dashboard UI **Blocks:** — · **Blocked by:** 11 **Files:** - Create: `apps/zync-app/src/modules/settings-ai/index.tsx` (lazy route) - Create: `apps/zync-app/src/modules/settings-ai/QuotaBar.tsx` - Create: `apps/zync-app/src/modules/settings-ai/ExtraUsageSettings.tsx` - Create: `apps/zync-app/src/modules/settings-ai/PersonalitySettings.tsx` - Create: `apps/zync-app/src/modules/settings-ai/UsageAnalytics.tsx` - Create: `apps/zync-app/src/modules/settings-ai/PurchasedCredits.tsx` - Create: `apps/zync-app/src/modules/settings-ai/api.ts` (TanStack Query hooks) **Steps:** - [ ] Quota bar: percentage-only fill from `/api/ai/quota`; label "X% of your monthly allocation · Resets "; color green <60%, amber 60–85%, red >85%. Never display raw token counts. Bar has `role="progressbar"` with `aria-valuenow/min/max` and accessible text. - [ ] Extra usage settings: shown only when tier `extra_allowed`; "Enable extra usage" toggle → `extra_usage_enabled`; monthly spend limit number input in ILS (displayed), stored as USD, capped at tier `extra_max_usd`; hint "You won't be charged more than ₪X/month on extra AI usage". - [ ] Personality: AI personality textarea (≤500 chars, live counter); per-use-case instructions collapsible list, one textarea per accessible use case (≤300 chars each) → `use_case_overrides`. - [ ] Usage analytics: bar chart of token usage by feature (labels not keys); time-range selector (this month / last 3 / last 6); breakdown table (Feature, Calls, % of usage). - [ ] Purchased credits: shown only if purchases exist; current balance "$X.XX remaining"; auto-reload toggle with amount field (min $5, max $100); purchase history table (date, amount, remaining); Buy credits button → purchase modal ($5/$10/$25/$50/custom) → `POST /api/ai/credits/purchase` handoff to payment adapter. - [ ] Tier-gate the whole page for Freelancer: render upgrade prompt instead of settings (AI customization is Business+). Use `useTierGate('business')`. - [ ] A11y: charts have data-table equivalents; `prefers-reduced-motion` honored; RTL via logical properties; ILS shown via locale adapter, USD stored internally. **Acceptance:** - [ ] Quota bar shows percentage only and correct color thresholds; no token numbers visible. - [ ] Freelancer sees an upgrade prompt, not the settings form. - [ ] Spend-limit input is capped at the tier max and displayed in ILS. ### Task 14: Seed data (default models, pricing, prompts, quotas) **Blocks:** — · **Blocked by:** 1 **Files:** - Create: `packages/db/src/seed/ai.ts` - Modify: `packages/db/src/seed/index.ts` **Steps:** - [ ] Seed `ai_model_pricing` with at least the default main model (`claude-3-5-haiku-20241022`, anthropic, vision-capable) plus representative OpenAI (`gpt-4o`) and Google (`gemini-1.5-flash`) entries with provider-published costs. - [ ] Seed `ai_global_config` single row: `main_model_id` = default Anthropic model, `backup_model_ids` = ordered `[gpt-4o-id, gemini-id]`, and a default seed prompt for every `AI_USE_CASES` key in `use_case_prompts`. - [ ] Seed `ai_tier_quotas` with `effective_from = today`, `effective_to = NULL` for all four tiers: freelancer (OCR-only, low token cap), business/enterprise/white_label with configurable defaults and `extra_allowed = true` for Business+. **Acceptance:** - [ ] After seeding, `getGlobalConfig` resolves a valid main + backup chain and a non-empty prompt for each use case. - [ ] Each tier has exactly one current `ai_tier_quotas` row. ### Task 15: Worker bindings & secrets **Blocks:** — · **Blocked by:** — **Files:** - Modify: `apps/zync-api/wrangler.toml` - Modify: `apps/zync-api/src/env.d.ts` (Env type) - Modify: `.dev.vars.example` **Steps:** - [ ] Declare `OPENAI_API_KEY` and `GOOGLE_AI_API_KEY` secrets (in addition to existing `ANTHROPIC_API_KEY`); document `wrangler secret put` for each. - [ ] Extend the `Env` interface with `OPENAI_API_KEY` and `GOOGLE_AI_API_KEY: string`. - [ ] Verify CSP `connect-src` already permits the provider endpoints used server-side (adapters call providers from the Worker, not the browser — no CSP change needed; confirm no client-side provider calls exist). **Acceptance:** - [ ] `wrangler dev` boots with all three provider keys present in `Env`. ### Task 16: Package & accounting tests **Blocks:** — · **Blocked by:** 4, 6, 7, 8 **Files:** - Create: `packages/ai/test/executor.test.ts` - Create: `packages/ai/test/accounting.test.ts` - Create: `packages/ai/test/pricing.test.ts` - Create: `packages/ai/test/prompt.test.ts` **Steps:** - [ ] Executor: retryable error falls through to next model; non-retryable rethrows immediately; exhausted chain throws "All models failed"; timeout treated as retryable. - [ ] Pricing: `calculateCost` and `usdToTokens` blended-rate math against known fixtures. - [ ] Accounting: quota-exhausted + extra-disabled throws `QuotaExceededError`; over spend-limit throws `ExtraSpendLimitError`; success path writes one usage log + one counter increment in a transaction. - [ ] Prompt: assembly order base → personality → override; base-only case. **Acceptance:** - [ ] `pnpm turbo test --filter=@zync/ai` passes.