# AutoRemaster — Mastering Methodology

Technical reference for the three-phase mastering engine (`tools/automaster_app/pipeline/`). Covers the full signal flow, every design choice and its reasoning, the constraints that shaped them, and the measured failures that forced revisions.

---

## 1. Problem Statement

AI-generated music (Suno, ACE-Step, etc.) fails as a finished master in specific, diagnosable ways:

1. **Bandwidth ceiling** — codec/model output rolls off at ~14–16 kHz. Commercial masters extend to ~20 kHz. This is the single biggest "sounds like AI" tell.
2. **Mix-level defects baked into a stereo file** — no stems exist, so reverb wash, dull transients, weak sub, and harsh 2–5 kHz buildup can't be fixed at the source.
3. **Tonal balance drift** — spectral tilt rarely matches genre conventions; no human mix engineer referenced anything.
4. **Loudness/dynamics mismatch** — wrong integrated loudness, wrong peak-to-loudness ratio for the genre.
5. **Stereo image errors** — width that doesn't follow genre norms per frequency band (e.g. wide bass, narrow highs).

A conventional mastering chain (EQ → comp → limiter on the stereo bus) cannot fix (1) or (2) at all, and fixes (3)–(5) only approximately. That is why the previous engine produced "better than raw Suno but still amateurish" results.

**Success criterion (chosen at design time):** the master must match measured metrics of commercial reference tracks in the same genre, verified by an automated gate. Not "sounds good to me" — numbers, against real masters.

---

## 2. Constraints

| Constraint | Consequence |
|---|---|
| Laptop, CPU only | demucs `htdemucs_ft` is the practical ceiling for separation quality; ~10–15 min/track fresh, ~8 GB peak RSS. Acceptable: quality was explicitly prioritized over time. |
| No ground-truth stems | Separation residual is real and must be measured, gated, and worked around — never assumed clean. |
| Per-genre targets must come from somewhere | Reference profiles are built from actual commercial masters placed in `tmp/refs/<genre>/`; synthetic/builtin DNA profiles exist only as fallback (they measured ~18 dB off real targets — see §9). |
| Fully automatic | Every threshold that depends on program material must be measured from the signal, never hardcoded (this was a recurring bug class — see §9). |
| Pipeline SR | Everything standardized to 44.1 kHz (`resample_poly` with rational factors) before processing. |

---

## 3. Architecture Overview

```
input.wav
   │
   ▼
┌─ PHASE A — stem repair ────────────────────────────────┐
│ demucs htdemucs_ft → 4 stems (vocals/drums/bass/other) │
│ residual quality gate (−15 dB) → full / light / bypass │
│ per-stem repair chains → LUFS-trimmed remix bus        │
└────────────────────────────────────────────────────────┘
   │
   ▼
HF bandwidth extension (SBR-style)        ← the "sounds like AI" fix
   │
   ▼
┌─ PHASE B — master bus ─────────────────────────────────┐
│ match-EQ (linear phase, vs reference profile)          │
│ multiband compression                                  │
│ glue saturation (oversampled arctan)                   │
│ 3-band width correction                                │
│ residual EQ (iterative, post-dynamics)                 │
│ ┌─ closed loop ────────────────────────────┐           │
│ │ residual width → true-peak limiter →     │           │
│ │ measure post-limiter width drift →       │           │
│ │ bias target, redo width, re-limit        │           │
│ └──────────────────────────────────────────┘           │
└────────────────────────────────────────────────────────┘
   │
   ▼
┌─ PHASE C — verification gate ──────────────────────────┐
│ spectral distance, TP, LUFS, PLR, width, HF sharpness, │
│ sibilance, whistle artifacts — vs reference profile    │
└────────────────────────────────────────────────────────┘
   │
   ▼
master.wav (24-bit, TPDF-dithered, −1.0 dBTP) + report.json
```

**Why three phases instead of a better stereo-bus chain:** defects (1) and (2) in §1 are unfixable on the stereo bus. Stem separation converts mix problems into stem problems, which DO have known fixes (de-essing a vocal stem works; de-essing a full mix collateral-damages cymbals). HF extension synthesizes the missing octave that no EQ boost can create (you cannot boost what isn't there). The reference engine turns "professional sounding" from an opinion into a measurable target.

Phase A **never hard-fails**: separation unavailable → bypass (master the original mix); separation degraded → "light mode" (skip the riskiest stem processing, halve nonlinear amounts). The master bus always runs.

---

## 4. Phase A — Stem Separation & Repair

### 4.1 Separation and the quality gate

- **Model:** `htdemucs_ft` (fine-tuned Hybrid Transformer Demucs, 4 stems). Best CPU-viable separator; stems are cached by input-file hash so demucs runs once per track ever.
- **Residual gate:** reconstruct `sum(stems)` and measure residual vs input. Real htdemucs residual on AI material measures **−21 to −22 dB** — the original spec's −40 dB threshold was unreachable and silently forced light mode on every track. Calibrated to **−15 dB**: below → "ok" (full processing), above → "degraded" (light mode).
- **Lesson encoded:** gate thresholds must be calibrated against measured reality, not aspiration.

### 4.2 Per-stem chains (`stem_chains.py`)

All chains preserve shape (stem in → stem out) and run whistle-artifact notching where separation artifacts concentrate (see §4.3).

**Vocals:** whistle notch → de-esser → spectral dereverb → presence EQ → level ride.
- De-esser = LR4 bandpass 5–9 kHz + compressor on the band only, recombined with the untouched rest. Threshold is **band RMS + 6 dB** — only sibilant peaks above the band's own average get compressed.
- Dereverb = soft spectral subtraction: per-bin 10th-percentile magnitude over time approximates the reverb tail floor; subtract 1.5× that floor, floor the reduction at −12 dB. Conservative on purpose — aggressive spectral subtraction produces musical noise worse than reverb. Skipped in light mode (separation bleed makes the tail estimate unreliable).
- Ride compressor threshold = **stem RMS − 3 dB** (adaptive — see §9.4).

**Drums:** lookahead transient shaper → parallel compression → oversampled HF exciter.
- Transient shaper: dual-envelope detector (1 ms fast / 80 ms slow), transient = normalized (fast−slow) excess; the *signal* is delayed by the 5 ms lookahead so the Hann-smoothed gain ramps up *before* the hit — no attack clipping, no detector lag.
- Parallel (NY) compression: 8:1 smashed copy mixed 30% under the dry 70%. Density without killing transients — the standard reason engineers use it.
- Exciter: `tanh(3x)` on the >6 kHz band, 4× oversampled, +30% (15% light). Adds the cymbal air AI drums lack.

**Bass:** mono below cutoff → oversampled saturation → phase-locked sub synthesis.
- Mono < 100 Hz (genre-profile cutoff): low-frequency S content wastes limiter headroom and breaks vinyl/club playback.
- Saturation `tanh(drive·x)` 4× oversampled, 40% wet: harmonics make bass audible on small speakers.
- **Phase-locked sub** (only when measured sub deficit > 12 dB between 25–60 Hz and 60–120 Hz bands): pYIN tracks the bass fundamental; a sine is synthesized at **half** the fundamental via cumulative phase (`phase = cumsum(2π·f0/2 / sr)`) — phase-continuous across pitch changes, which a per-note oscillator is not. Amplitude follows the bass envelope. The pYIN voiced/unvoiced gate is smoothed with a 20 ms Hann window — the raw 0/1 gate at hop boundaries clicks audibly (§9.5).

**Other (synths/guitars/keys):** spectral smoother → harshness band compression → decorrelation.
- Harshness control: LR4 band 2–5 kHz compressed at **band RMS + 6 dB** threshold, 3:1 — this is where AI mixes pile up fatiguing energy.
- Decorrelation only when channel correlation > 0.95 (fake stereo): complementary alternating ±2 dB bells at interleaved ⅓-octave centers 1–10 kHz. L boosts where R cuts → **sums flat in mono** — this is why it replaces Haas delay widening, which combs in mono.

### 4.3 Whistle suppression

Separation + band compression exposes narrow tonal residues ("whistles") that read instantly as artifacts. Detection mirrors the Phase C analyzer exactly (same STFT, same prominence logic — measurement parity, §8): peaks with prominence ≥ 10 dB over the smoothed spectrum get IIR notches (Q=30), max 3, strongest first. Applied in vocal and other chains, and inside the de-esser band before compression amplifies them.

### 4.4 Remix bus (`remixer.py`)

Stems are LUFS-measured and trimmed toward genre-profile stem balance before summation. This is the "improve the mix" step the old engine lacked entirely: a static remix with corrected stem levels.

---

## 5. HF Bandwidth Extension (`hf_extension.py`)

The most important module for de-AI-ing the sound, and the one with the most revision scars.

**Method (SBR-style, per channel):**
1. **Detect roll-off:** mean STFT magnitude, ⅙-octave smoothed; roll-off = first frequency above 4 kHz that drops 30 dB below the 1–4 kHz reference band. No-op if roll-off ≥ 0.85·Nyquist (already full-band).
2. **Copy the octave below roll-off up** by STFT bin shift (4096 FFT, 75% overlap). Harmonic structure of the 7–14 kHz octave is a perceptually plausible basis for 14–20 kHz.
3. **Suppress tonal peaks first:** cap source bins at **6 dB above the local (5-bin) median magnitude** before shifting. Without this, any tonal component near the splice gets *replicated an octave up* as a synthetic whistle — measured as +15 dB narrowband images in early runs.
4. **Per-frame splice gain:** ratio of splice-region magnitude to source magnitude computed **per STFT frame**, smoothed ~200 ms (`uniform_filter1d(size=9)`). The original implementation used one track-mean ratio — which laid a *constant* synthetic HF bed under quiet sections. A steady hiss floor that ignores the music is precisely the "sounds like AI" artifact this module exists to remove. Energy must follow the program.
5. **Spectral decay envelope:** −3 dB/octave from the splice. Natural instrument spectra decay; a flat copy reads as synthetic brightness.
6. **Noise blend 45%:** shaped complex noise (same magnitude envelope, ÷√2) replaces 45% of the copied content. Pure copied harmonics sound metallic/ringing; pure noise sounds like hiss. 45% was tuned on the evaluation corpus.
7. **Hard cap, then transient boost — in that order:** every extended bin is capped at the per-frame splice level × decay envelope (prevents any overshoot), *then* transient frames get +3 dB in the first extended octave only. Cap-after-boost would erase the boost.
8. **Transient detection on the HF source band only** (`Z[i_src_lo:i_roll]` flux, top-decile threshold). Full-spectrum flux meant *kick drums* triggered HF boosts — bass energy pumping the synthetic treble.
9. **±1 kHz crossfade** across the splice point — no spectral seam.

---

## 6. Phase B — Master Bus

Stage order is deliberate: **corrective EQ → dynamics → color → image → residual corrections → limiter**. Dynamics and saturation re-tilt the spectrum, which is why residual correction stages exist after them (§6.5).

### 6.1 Match-EQ (`match_eq.py`)

- Correction = reference ⅓-octave curve − source curve, both measured at −14 LUFS normalization (loudness-independent comparison).
- Gaussian-smoothed in octave domain (σ=1.5 bands) — match the *tonal balance*, not every notch; chasing narrow features creates ringing and overfits one section of the song.
- **±4 dB cap**: larger corrections mean the source is broken in a way EQ shouldn't fix alone (and Phase A should have); also keeps the FIR well-conditioned.
- **Zero below 30 Hz** (don't boost rumble), **cut-only above the pre-extension roll-off** — above the original roll-off, content is synthetic (ours); boosting it would amplify synthesis artifacts to chase a reference curve. HF extension owns that region; match-EQ may only tame its excess.
- Applied as **4097-tap linear-phase windowed-sinc FIR** (`firwin2` + `fftconvolve`), group delay compensated exactly. Linear phase because at mastering, phase rotation between corrected/uncorrected regions smears transients; latency is irrelevant offline.

### 6.2 Multiband compression (`mb_compressor.py`)

- LR4 crossover tree (flat-summing by construction — Linkwitz-Riley 4th order is *the* standard mastering crossover because |H_lp|²+|H_hp|² sums to allpass, no amplitude ripple at the seams).
- Per-band threshold derived from the band's own RMS: `T = rms + 6 − GR·r/(r−1)` — algebraically places the threshold so a signal at (rms+6) dB receives exactly the design gain reduction GR at ratio r. Self-calibrating to program level.

### 6.3 Glue saturation (`saturation.py`)

- Single shared `oversampled(fn, factor=4)` core: polyphase upsample → nonlinearity → polyphase downsample. **All** nonlinear stages in the engine (glue, drum exciter, bass saturation) route through it — a `tanh` at base rate aliases its harmonics straight back into the audible band; 4× oversampling puts the harmonics above the fold-back region. One implementation, one correctness proof.
- Master glue: arctan blend, 5–10% wet from genre profile. Subliminal harmonic density — the "expensive console" effect, not distortion.

### 6.4 Width (`width.py`)

- 3 bands (LR4 splits at 250 Hz / 4 kHz), per-band target = reference profile's S/(M+S) energy ratio.
- Exact solve: given target t and mid energy `em`, new side energy `es' = t/(1−t)·em` → side gain `√(es'/es)`.
- Correction capped at **±50% per pass** (raised from ±20%: real-corpus deviations up to 0.41 made the smaller cap structurally unable to converge within iteration limits).
- Mono-bass below genre cutoff is preserved (re-applied after width).

### 6.5 Residual EQ (post-dynamics)

Single-pass match-EQ measured pre-dynamics landed **4–9 dB off** in individual bands after MB compression and saturation re-tilted the spectrum. So: re-measure on the actual bus signal and correct again, **iteratively** (max 2), with `gaussian_sigma=0` — at this stage we are nulling specific measured band errors, not shaping tone; smoothing would defeat the point. Converged when max band deviation < 4.5 dB; same ±4 dB / 30 Hz / cut-only-above-rolloff rules.

### 6.6 The width ↔ limiter closed loop — key finding

Three architectures were tried; the physics decides:

1. **Width after the limiter** (tried, rejected, caused a measured regression): widening raises true peaks (~3.5 dB measured) → mandatory TP re-trim → **−3.5 LU loudness loss and +4 dB PLR error**. The acceptance test that introduced this checked width and TP but not LUFS — the gate caught it one full run later.
2. **Width before the limiter, open-loop** (insufficient): linked limiting applies *equal per-sample gain to both channels*, so it cannot change instantaneous S/M — but it **time-reweights**: gain reduction lands disproportionately on mono-heavy kick transients, shifting the *integrated* S/(M+S) toward S-heavy passages. Estimated ~0.01; **measured 0.064** (high band) on hard-limited psytrance at −10 LUFS. The estimate failed because at EDM loudness targets the limiter takes 8–10 dB GR on transients — time-reweighting scales with GR depth.
3. **Closed loop (current):** up to 3 rounds of:
   ```
   width-correct pre-limit signal toward target → limit →
   measure post-limiter band width vs true target →
   all |drift| < 0.03 → done
   else → bias the width target by −drift, redo width on the SAVED
          pre-limit signal, re-limit
   ```
   Aim past the target by exactly the amount the limiter pushes back. Measured convergence: round 0 high-band drift +0.064 → round 1 **−0.012**. Each round re-limits the clean pre-limit signal, so dither is never applied twice. Acceptance tolerance 0.03 sits under the 0.05 verification-gate floor with margin.

### 6.7 True-peak limiter (`tp_limiter.py`)

ITU-R BS.1770-4-style detection, ceiling −1.0 dBTP:

1. Gain-stage to the genre target LUFS (from the reference profile — never one default for all genres).
2. **4× oversample** (polyphase) — inter-sample peaks are invisible at base rate; a sample-peak limiter at −1.0 dBFS routinely lets through −0.3 dBTP overs that clip DACs and lossy codecs.
3. Required gain per OS sample = `min(1, ceiling/|x|)`; **sliding-window minimum** over the 5 ms lookahead (O(n) `minimum_filter1d`) → **Hann-smoothed attack** (clamped to never exceed the required curve — smoothing must not overshoot) → exponential release computed at a **16× decimated control rate** (release time constants ≫ control period, so decimation is free CPU).
4. **Program-dependent release:** 50–200 ms scaled by crest factor — transient material gets fast release (transparency), dense material slow (no pumping distortion).
5. Apply gain in the OS domain, downsample, safety clamp.
6. Up to two corrective passes if output LUFS misses target by > 0.5 LU (limiting itself eats loudness; one pass can't know how much in advance). Reported `max_gain_reduction_db` is the **max across passes** (was: silently overwritten by the last, smallest pass).
7. Final TP re-trim to ceiling, then **TPDF dither** at 24-bit — the actual quantization step, applied exactly once, last.

---

## 7. Phase C — Verification Gate (`verify.py`)

Every master is measured and gated against its genre profile. A track that fails any check is reported FAIL with per-check values — no silent shipping.

| Check | Threshold | Why this threshold |
|---|---|---|
| Spectral distance | mean ≤ 2.5 dB, max band ≤ 5.0 dB, over 50 Hz–16 kHz | Bands outside this range are dominated by synthesis (HF) and rumble (LF), not mix decisions |
| True peak | ≤ −0.9 dBTP | −1.0 target + 0.1 measurement tolerance |
| LUFS | within ±1 LU of profile | Streaming-normalization era: loudness identity, not war |
| PLR | within ±2 dB of profile | Catches both over-limiting (crushed) and under-limiting (weak) |
| Band width deviation | ≤ max(0.25·ref, 0.05) per band | Relative tolerance for wide bands, absolute floor for narrow (mono bass ref ≈ 0 would otherwise demand exact zero) |
| HF sharpness 10–16 kHz | ±3 dB vs profile | Specifically polices HF-extension over/under-shoot |
| Sibilance ratio | ≤ ref × 1.2 | 4–8 k / 1–4 k energy ratio — harshness guard |
| Whistle count | master ≤ input | The pipeline must never *add* tonal artifacts; input may already have them |

**Measurement parity is enforced:** the gate, the match-EQ correction, and the residual stages all call the *same* spectrum function and the same whistle detector. A correction measured by analyzer A and judged by analyzer B fails on the difference between analyzers, not on the audio.

---

## 8. Reference Engine (`reference_engine.py`)

Profiles per genre, built from commercial masters (≥2–3 tracks averaged):

- **⅓-octave spectrum via Welch** (16384-sample Hann segments, 50% overlap), each ref loudness-normalized to −14 LUFS first. Original implementation windowed the *entire 5-minute track* with one Hann — tapering the first and last ~quarter toward zero and biasing every spectral target toward the middle of the arrangement (typically the loudest drop). Every EQ decision inherited that bias. Welch averages uniformly across the whole song.
- Plus: integrated LUFS, LRA (3 s short-term loudness, 95th−10th percentile), PLR, true peak, 3-band S/(M+S) width ratios.
- Builtin DNA-derived profiles exist as fallback only — measured ~18 dB off real commercial targets in places; real refs are the source of truth.
- Profiles are JSON, rebuilt whenever the measurement code changes (a Welch profile and a single-window profile are not comparable).

---

## 9. Failure Log — what was wrong and what it taught

Every revision below was driven by a measured failure, not taste.

1. **Synthetic corpus invalidated the gate** (0/3 fail): demucs degrades synthetic test audio in unrepresentative ways, and builtin profiles were ~18 dB off. → Evaluation corpus must be *real AI tracks*; targets must come from *real commercial masters*.
2. **Unreachable separation gate** (−40 dB) silently forced light mode on every track. → Measure the real residual (−21/−22 dB), calibrate the gate (−15 dB) from data.
3. **Single-window spectrum** biased every tonal target to the song's middle. → Welch averaging (§8).
4. **Hardcoded compressor thresholds** (−24/−30/−28 dBFS) on demucs stems whose absolute level is arbitrary: quiet stems escaped compression entirely, hot stems got smashed. → Every threshold derived from the measured RMS of what it compresses (vocal/drums: rms−3 dB; de-ess/harshness bands: band-rms+6 dB; MB comp: analytic rms-based formula).
5. **Hard pYIN voicing gate clicked** at hop boundaries. → 20 ms Hann smoothing on the gate.
6. **Static HF splice gain** = constant hiss bed in quiet sections; **full-spectrum flux** = kicks pumping synthetic treble; **boost-then-cap** = caps erasing boosts. → Per-frame gain, HF-band flux, cap-before-boost (§5).
7. **Post-limiter width** destroyed LUFS/PLR via TP re-trim; its acceptance check verified only width+TP. → Width pre-limiter in a closed loop (§6.6); *and a process rule: any change to gain staging must re-verify every gate metric, not just the one being fixed.*
8. **Limiter report overwrote max GR per pass** — under-reported actual limiting. → max across passes.
9. **Whistles added by processing** (1 input → 3 in master): band compression amplified separation residues. → Analyzer-parity notching at the sources (§4.3) + the gate's "never more whistles than input" rule.

---

## 10. Performance Envelope

| Item | Cost |
|---|---|
| demucs htdemucs_ft (fresh) | ~10–15 min/track CPU, ~8 GB peak RSS; cached by file hash thereafter |
| Full pipeline with cached stems | ~1.5–5 min/track |
| Limiter closed-loop extra round | a few seconds (limiter + width only; demucs/EQ untouched) |
| Evaluation run | `PYTHONPATH=tools venv310/bin/python3 tools/evaluate_library.py --library tmp/library --refs tmp/refs --profile-dir tmp/evaluation/profiles` |

Current status: **3/3 evaluation tracks PASS** (melodic techno −9.3 LUFS, psytrance −10.4, tech house −10.0; all −1.00 dBTP; spectral distance ≤ 1.2 dB vs commercial references) at commit `be77f1f`.