LlamaCasty/TASKS/task-08-audio-milestone.md

TASK 8 — Audio milestone: real stream audio + voice filters + TRAX music (2026-08-14)

Catalog: TASKS.md — status and requirements live here.

Goal: every audio issue done and tested in one branch — real mic/game audio reaches the encoder (replacing the anullsrc silence), MicVolume/GameAudioVolume/mute become real pre-AAC gains, TASK 10's voice filters land, auto-duck keeps the creator's voice over game + music, and a free background-music source ("TRAX") plays into the sound bar and the stream.

Status: ✅ SHIPPED 2026-08-14 — 196 tests passing, 0 warnings (ONE integration test: AudioPipelineTests.Mix_WithFiltersDuckAndGain_Lands_On_AudioPipe)

1. ✅ Real audio into the encoder — the mixer's loopback samples were dropped today (AudioMixer.OnLoopbackSample metered only) and MicVolume was meter-only; FfmpegArgs.cs:28 ran -f lavfi -i anullsrc (silence). Transport: ffmpeg reads a Windows named pipe — -f f32le -ar 48000 -ac 2 -i \\.\pipe\ytllive_audio replaces the anullsrc block, plus explicit -map 0:v -map 1:a; pipe name via EncoderOptions.AudioPipeName (DefaultAudioPipeName = "ytllive_audio"). Mixer stays two inputs (mic + loopback) — no N-source abstraction, no StereoMixer class. The VM owns the pipe lifecycle (BeginGoLive → _audioMixer.StartLive(pipeName), StopStream → _audioMixer.StopLive() before the pump stops); FfmpegEncoder.cs is untouched.
2. ✅ Honest gains reach the stream — MicVolume scales the mic channel; GameAudioVolume + mute scale the loopback channel (the "mute in preview but stream still plays" footgun dies with it). One knob per input = KISS. Implemented as Func<double> gain seams on AudioMixer (micGain/loopbackGain), wired by MainViewModel.
3. ✅ Voice filters (TASK 10, before the meter AND the mix) — VoiceFilterChain: bass (LowShelfFilter 120 Hz +4 dB) → treble (HighShelfFilter 8 kHz +3 dB) → noise gate (pure C#, creator's choice) → compressor (threshold 0.5, 4:1). Pure stateful DSP (TDF2), sine-in unit tests. Meter stays post-filter.
4. ✅ Auto-duck — pure envelope (AutoDucker): mic RMS > 0.02 → game+music dip ×0.25 (~12 dB), attack 0.05 / release 0.005, recover on release. Always-on, no knobs. Strong form of the creator's "voice always over the game volume" idea.
5. ✅ TRAX — background music, FREE (keeps the one paid line = Alerts + flash removal). MusicPlayer: NAudio MediaFoundationReader (mp3/wav/m4a) → VolumeWaveProvider16 at the hardcoded 0.20 (fixed, not changeable) → WaveOutEvent on the default device (added the sibling NAudio.WinMM 2.2.1 package — WaveOutEvent isn't in NAudio.Wasapi), loop on end. No 3rd mixer input and no MusicVolume — music plays on the desktop, the existing loopback picks it up: heard in headphones, the sound-bar meter bounces, and the stream carries it through the loopback channel (ducked with game when the voice is hot). Known wrinkle (out of scope): YouTube mutes VODs carrying copyrighted music — future "music on live, off VOD".
6. ✅ TRAX footer control (final spec, agreed 2026-08-14) — YtButtonSecondary, status dot + "TRAX" text, in the center footer stack beside MIC:
   • Status dot (same 8px Ellipse pattern as MIC/Socials): red no track loaded · yellow loaded not playing · green playing.
   • Left-click → toggle play/pause; no track loaded → opens the picker instead.
   • Right-click → always opens the in-app track picker (TraxButton_PreviewMouseRightButtonUp, code-behind pattern like GameSpeaker_MouseLeftButtonUp, e.Handled = true so no context menu).
   • Tooltip → playing/loaded track name; "No track — right-click to choose background music" when empty.
   • No slider — volume is the 0.20 constant. Picker = plain OS OpenFileDialog filtered to mp3/wav/m4a/aac/flac/ogg.
7. ✅ Sound bar shows music — IsGameAudioBarVisible = gameDetectorProducingSound || isMusicPlaying; relabelled "Game Audio Capture" → "Desktop Audio" (TRAX rides the same channel). Follow-up (2026-08-15): the gating is gone — the game audio bar is ALWAYS visible (the IGameAudioDetector/GameAudioDetector/GameAudioHysteresis stack + the VM's 250ms poll timer and their tests were deleted; the bar kept hiding the desktop meter when no full-screen game with sound was up, and the creator wants it pinned).
8. ✅ Capture hardening (design changed at build time) — the plan's "force IEEE-float 48 kHz on both WASAPI sources" was dropped: NAudio 2.2.1's WasapiCapture exposes no overridable GetDefaultMixFormat, so both sources capture the device's own mix format and the mixer's pure TinyResampler normalizes any rate/channel count to 48 kHz stereo (the resampler IS the design, not a fallback). Stop path disposes the audio pipe (EOF) in MainViewModel.StopStream before the pump stops, so both ffmpeg inputs end in order.
9. ✅ Schema v9 — single-row Music (TrackPath, IsEnabled; volume is the 0.20 constant) + migration in LayoutStore.cs (user_version 9; Save gained an optional Music? music = null param so existing 3-arg callers still compile; load query + null reset).
10. ✅ Docs in the same commit — ai.md audio section rewritten ("one knob per input", 2-in mix, auto-duck, TRAX free, VOD-mute wrinkle), TASK 10's status, this task's status, HANDOFF.

Files (new): Services/Audio/VoiceFilterChain.cs (+ LowShelfFilter/HighShelfFilter/NoiseGate/Compressor), Services/Audio/AudioRingBuffer.cs (async-arrival source buffers), Services/Audio/AutoDucker.cs, Services/Audio/TinyResampler.cs, Services/Audio/MusicPlayer.cs, Services/Audio/NamedPipeAudioWriter.cs (+ IAudioPipeWriter seam), Models/Music.cs.

Files (changed): AudioMixer.cs (filter chain before meter+mix, loopback into the mix, gains via Func<double> seams, ducker, silence-filler so the pipe never stalls, StartLive/StopLive), WasapiMicAudioSource.cs (device mix format — resampler normalizes; no forced format), FfmpegArgs.cs (named-pipe input + -map, name via EncoderOptions.AudioPipeName), EncoderOptions.cs (AudioPipeName), ytLive.csproj (NAudio.WinMM 2.2.1 for WaveOutEvent), LayoutStore.cs (schema v9), MainViewModel.cs/MainWindow.xaml/MainWindow.xaml.cs (footer TRAX group, sound-bar visibility OR music, go-live starts the pipe, end closes it first).

Testing (Good Dog Rule — ONE integration test): AudioPipelineTests.Mix_WithFiltersDuckAndGain_Lands_On_AudioPipe — real filter chain + mixer + ducker + NamedPipeAudioWriter, fake IAudioSources, test-side NamedPipeClientStream reads the bytes and asserts post-filter/post-gain/post-duck mixed stereo (ring buffers pre-filled so the first pipe tick already carries real audio — avoids a start-of-stream silence race). Units: each DSP stage (known sine-in → expected gain), ring buffer wrap/underflow/overwrite, ducker envelope, resampler (down/up/stateful/identity), FfmpegArgs pipe+map, schema v9 roundtrip + no-music-row. Existing tests stay green.

Out of scope (this branch): IP webcam (video-only when it lands — never an audio input), chat box source (TASK 3 item 18), alt-key crop, credits, background removal, music-off-VOD, PremiumUrl (TASK 10 seam).