--- name: learn-from-mistakes description: Use when a session ends or user invokes /learn-from-mistakes --- # Learn From Mistakes Audience: AI coding agents. Extract agent mistakes from current session. Two defenses per mistake: (1) text learned rule into skills/agents owning that domain — self-trains agent in-context; (2) statically-detectable mistakes ALSO get a `slopgate` rule — fail-closed hard gate at commit. Step 8 triages + delegates rule authoring to `slopgate-improve`. ## When to Invoke - User types `/learn-from-mistakes` - `wrap-up` skill calls at session end - After session with visible corrections, retries, failures **Scope limit:** Current context window only. Compacted/archived history inaccessible — state this if user asks about older mistakes. ## Step 1: Read Project CLAUDE.md Read project `CLAUDE.md`, extract: - Specialized skills + file path ownership (e.g., `src/components/ui/**` → `md-ui-developer`) - Local agents under `.claude/agents/` Ground truth for eligible targets. Only skills/agents in CLAUDE.md writable. Generic global skills (`plan`, `brainstorm`, `ship`, `debug`, `tdd`, `verification-before-completion`, etc.) — **read-only. NEVER write learned rules there**. ## Step 2: Collect Mistake Signals **Use LLM-semantic analysis, not keyword grep.** Understand what agent assumed vs what reality was. Keyword patterns below are hints, not filters. ### Class A — Behavioral Mistakes Agent did wrong thing in code/process: - User: "no", "don't", "wrong", "revert", "that's not right", "never", "stop" - Same file edited 2+ times - Build/type/test failure → agent fix → success - **Filter:** count only if followed by actual edit/fix OR build/test failed OR user cited specific rule ### Class B — False Assumptions (no code edit required) Agent stated wrong fact about any service, API, framework, architecture, business logic, requirements: - LLM asserted non-existent capability ("I'll set up the sandbox", "X will send a webhook") - LLM asked user to do something impossible/unnecessary - LLM built something later removed because premise was wrong - User corrected claim: "X doesn't exist", "we don't have X", "that's not how Y works", "you're assuming" - Advisor called → read what assumption triggered it + what correction was - Agent self-reversal: "I was wrong", "actually", "I assumed incorrectly" - **No code edit required** — mistake = false belief, not downstream code ### Class C — Meta-Failures Agent violated rule already written in target skill → skill loaded but not followed. Log separately. Do NOT add new rule. **User changing mind mid-session ≠ mistake.** Skip unless Class A/B signal met. Cross-ref git diff: changed file matches correction signal → mistake. Changed file matches only user's stated feature request → intentional, skip. ## Step 3: Classify Each Mistake Per mistake: ``` file: signal: wrong: correct: ``` ## Step 4: Resolve Target Skill/Agent Match mistake file path against CLAUDE.md skill registry → find responsible skill/agent. No match → log unresolved. Suggest creating new skill/agent + registering in CLAUDE.md. ## Step 5: Check for Meta-Failure Before writing rule, read target skill's hand-written body. Proposed rule restates something already written → **meta-failure**: agent failed to load/follow skill, not missing rule. Log as: `meta-failure: skill not loaded/followed` — do NOT add rule. ## Step 6: Dedup Check Read full `## Learned Rules` section of target file. Semantically compare proposed rule against existing entries — judgment, not grep. - Covered → skip insert, increment `fired:N` on matching entry - Not covered → proceed ## Step 7: Write Rule Append to `## Learned Rules` at file bottom (create if missing). **Format — caveman, 3 lines max:** ```markdown ### {slug} | fired:1 | {YYYY-MM-DD} {wrong behavior} → wrong. {correct behavior}. {file/context if relevant}. Prevent: {concrete action to stop this recurring — check X first, read Y before Z, never assume W without verifying}. ``` **Quality gates:** - States wrong + correct behavior - Has concrete mitigation — not "be careful". Specific: "read skill before touching X", "verify API docs before assuming capability", "cast CF env var with `Number()` before use" - Max 3 lines, terse — readable under context pressure **Mitigation must answer: "what should agent do BEFORE making this mistake?"** - False assumption → "verify [source] before assuming [X]" - Behavioral mistake → "load [skill] / check [rule] before [action]" - Type/coercion error → "always cast [var] with [method] before use" ## Step 8: Programmatic Blocker (slopgate) Text rule self-trains agent in-context. Statically-detectable mistakes ALSO deserve a hard gate: `slopgate` fires fail-closed at commit, on uncommitted + non-skill paths, regardless of which agent/context. Both fire at different times → keep both; not redundant. Run per Class A/B mistake AFTER its text rule written. ### 8a — Gate precondition ```bash ROOT=$(git rev-parse --show-toplevel 2>/dev/null) [ -f "$ROOT/.slopgate/config.toml" ] && echo "gated" || echo "ungated" ``` `ungated` → skip programmatic blocker, text rule is sole defense. Do NOT auto-run `slopgate init` here (out of scope for a mistake pass). Note in report: `programmatic-blocker: repo ungated — run /slopgate-init to enable`. ### 8b — Static detectability triage Apply `slopgate-improve` bucket model (A regex / B ast / C semantic / skip) to each mistake. Do NOT copy bucket definitions — that skill is the single source of truth. - Bucket A/B (banned token, hardcoded value, missing required attr, import-boundary, file-shape) → **detectable**, proceed to 8c. - Bucket C/skip (judgment, intent, runtime behavior, needs type info) → **not detectable**, text rule is sole defense. Stop. ### 8c — Dedup vs existing rule IDs Build covered-ID set per `slopgate-improve` Phase 1 (config.toml `baseline`/`stack` pack names + project ast `.yml` ids under `.slopgate/rules/ast/`). Proposed rule ID already present → skip authoring, note `already-gated`. NEVER mint a duplicate rule ID. ### 8d — Tier + autonomy gate Assign lowest tier with no FPs (per `slopgate-improve` tier model). Autonomy by tier — learn-from-mistakes can run unattended at session end (`wrap-up` calls it), so: | Tier / severity | Action | |-----------------|--------| | project, non-critical | author autonomously — repo-local, reversible | | global (baseline/stack) OR critical severity | **propose only** — list in report for user confirm, do NOT self-apply | Default tier = **project** unless mistake is provably runtime/framework-universal. One session's evidence MUST NOT silently push a global rule. ### 8e — Author + verify + apply (delegate) Feed the single mistake as input to `slopgate-improve` authoring phases — Phase 4A/4B (draft regex/ast + canary) → Phase 5 (canary match + `slopgate --self-test`, exit 0) → Phase 6 (apply to correct tier). Do NOT re-run `slopgate-improve` Phase 2 (it re-mines all convention sources — wasteful for one known mistake). NEVER skip the canary. ### 8f — Tighten text rule when gated Mistake now hard-gated → its text rule can shorten: replace verbose `Prevent:` with pointer. Format: ```markdown ### {slug} | fired:1 | {YYYY-MM-DD} {wrong} → wrong. {correct}. Hard-gated: slopgate `{rule-id}`. ``` Ungated/bucket-C mistakes keep full `Prevent:` line (Step 7 format). ## Step 9: Report ``` ## Learning Report ### Mistakes Found: N | # | Mistake | Signal | Target | Rule Slug | Mitigation | |---|---------|--------|--------|-----------|------------| | 1 | SpacingKey raw number | type error | md-ui-developer | spacing-token-raw-number | check SpacingKey type before passing props | ### Changes Made - Text rules added: N - Text rules updated (fired count++): N - Meta-failures (skill not loaded): N - Unresolved (no target): N ### Programmatic Blockers (slopgate) - Authored (project, auto): `` (tier:sev) — - Proposed (global/critical, NEEDS USER CONFIRM): `` (tier:sev) — - Not detectable (bucket C/skip — text rule only): - Already gated (dup ID): `` - Repo ungated: run `/slopgate-init` to enable hard gates ### Unresolved - → no skill covers `src/X/**` — consider creating `md-X`, register in CLAUDE.md ```