Files
LlamaCasty/docs/ai/output.md
T
gramps 24077e58cf fix(stepdown): restart the audio pipe around encoder changes
A quality step-down now closes the named audio pipe before stopping/flushing the
pump, then opens a fresh server before restarting the pump. This prevents ffmpeg
from waiting on the old audio input and avoids reconnecting to a broken pipe.

EncoderRestartTests pins the audio-stop → video-stop → audio-start → video-start
order. 616 tests green, clean build with 0 warnings.
2026-11-11 09:52:27 -08:00

345 lines
29 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Output — the ffmpeg bundle, the encoder, recording, and audio
> Split out of [`ai.md`](../../ai.md) (2026-10-26) so a session reads only what its task touches;
> `ai.md` keeps a short invariant stub for this file. Directory map: [`index.md`](index.md).
Everything that leaves the app: locating and shipping `ffmpeg.exe`, how the encoder subprocess is
driven (video stdin + audio named pipe), local recording as a first-class mode, and the WASAPI capture
/ filter / mix chain that feeds it. Content is the decisions plus the failures that produced them;
superseded designs are marked inline.
---
## The encoder
### FFmpeg locator (TASK 4 ship step 2 — shipped 2026-08-10; **rewritten to BUNDLED 2026-09-28**, plan in TASKS.md)
The encoder's one external dependency is `ffmpeg.exe`. **It is BUNDLED — never downloaded, never
fetched at runtime.** `IFfmpegLocator` resolves an absolute path on demand: **the bundle first**
(`<AppContext.BaseDirectory>\ffmpeg\ffmpeg.exe`), then **PATH** as the escape hatch for someone who
deliberately installed their own. The order is the opposite of the original ("the user's own install
wins") on purpose: a PATH `ffmpeg` may be a different major version or a **GPL** build, and the
bundle is the one that was pinned, license-vetted and tested. If neither hits, it throws a loud
`IOException` naming the missing folder and the fetch script — the old "found something on disk"
cache path is gone deliberately, because silently satisfying a missing bundle with a stale copy is
exactly the silent drift a certification-gated dependency must not have.
**The bundle is not in git** (135 MB of incompressible DLLs would tax every commit on this
9p-bridged tree). `scripts/fetch-ffmpeg.sh` populates `./ffmpeg` from a pinned month-end BtbN
LGPL-**shared** win64 artifact, **verifies SHA-256 before extracting**, and is idempotent; set
`FFMPEG_ZIP=<path>` to reuse a pre-downloaded archive. `ytLive.csproj` copies `ffmpeg\**` to the
output (`CopyToPublishDirectory="Always"` for the MSIX package) and the `GuardBundledFfmpeg` target
**fails the build** if the folder is absent, so a fresh clone finds out before the creator presses
Record rather than during a take. `ffmpeg.exe` + `ffprobe.exe` + the `libav*.dll` family must stay in
one folder — Windows resolves a shared build's DLLs from the executable's own directory.
`ffplay.exe` (18 MB) is deliberately **not** shipped; nothing calls it.
BtbN LGPL-shared (not gyan.dev, not static): it drops GPL-only libx264/x265 while keeping
NVENC/QSV/AMF + libopenh264 + native AAC, and dynamic linking means LGPL compliance is "license text +
source offer" with no static-relink (§6) material — see the Licensing guardrails below. libopenh264
is **compiled into `avcodec-63.dll`** (verified 2026-09-28: the encoder is registered there, and there
is no separate OpenH264 binary to ship), so Cisco's BSD terms + patent grant ride along inside item 1
of `THIRD-PARTY-NOTICES.txt`. Pin: `autobuild-2026-08-31-13-27` (month-end on purpose — BtbN keeps
~14 dailies but month-end builds ~2 years; a daily pin is what 404'd, see below).
**⛔ Why this was rewritten (2026-09-27 → shipped 2026-09-28).** The pinned-download design was a
defect, not a cold-start edge case. It **failed in production on 2026-09-01** — the previous daily
pin aged out of BtbN's 14-day retention and the download 404'd on the creator's first real recording
attempt. Two facts made "the pin is a const, just bump it" wrong:
- **Microsoft Store policy 10.2.2 forbids dynamic code inclusion.** Downloading an executable
from the internet and running it is the exact pattern the policy names. An MSIX submission
is rejected over it.
- Bumping the pin is whack-a-mole against a retention window we do not control.
Bundling deletes the startup network dependency entirely. The LGPL-**shared** build choice is
**unaffected** — dynamic linking was chosen for LGPL §6 compliance ("license text + source offer"),
not to enable downloading. Analysis: `TASKS/research-store-certification.md` §4.
**Guardrail:** `FfmpegLocatorTests.TheLocatorCarriesNoNetworkCapability` asserts the locator's
source contains no `HttpClient`/URL/downloader at all, so the policy cannot rot back.
**Every other third-party binary already ships via NuGet — ffmpeg was the only download-and-execute.**
Verified 2026-09-28: `e_sqlite3.dll` and `WebView2Loader.dll` land in `runtimes\win-x64\native\`
(SQLitePCLRaw + WebView2 packages) and NAudio/Notification.Wpf/Microsoft.Data.Sqlite/ProtectedData
are managed. Two gaps remain and are **not** fixed by bundling: the **WebView2 Evergreen runtime** is
a machine prerequisite (a manifest framework-dependency + graceful-failure path, TASK 48 item 2), and
**Velopack's updater downloads and executes** — its teardown is a compliance action, not just cleanup.
### Live encoder + RTMP push (TASK 4 ship step 3 — shipped 2026-08-12, plan in TASKS.md)
The encoder is a **thin orchestrator over `ffmpeg.exe`** — no H.264/AAC code in the app. It spawns the
subprocess (path from `IFfmpegLocator`), feeds raw BGRA compositor-crop frames into stdin, lets FFmpeg
scale them to the selected output tier, and parses `-stats`
stderr lines into `StreamHealth` (bitrate/FPS/duration, dropped-from-frame-count). `FfmpegEncoder`
(`IFfmpegEncoder` seam) holds: `StartAsync` (locate → probe `-encoders` → spawn → stderr loop →
drain loop), `SubmitFrameAsync` (slice 10: bounded-queue ENQUEUE — never a pipe write; the drain task
owns stdin writes), `StopAsync` (flush queue → stdin EOF → ffmpeg
finalizes + exits by itself; a 10s watchdog kills it), `Dispose` (force-kill + wait), and the
`HealthUpdated`/`ProcessFailed` events. **Pattern:** the encoder never touches `Process` — it drives the
`IEncoderProcess` seam (`FfmpegEncoderProcess` wraps the real `Process`, redirected stdin/stdout/stderr
+ exit control); a `Func<IEncoderProcess>` factory + the locator are constructor-injected, so the
integration test fakes the whole subprocess (probe + encoder) with a Channel-backed `TextReader` whose
`Complete()` is EOF (`null`), never a `ChannelClosedException`.
**Decisions (locked):** args are pure (`FfmpegArgs.Build`, no string building in the encoder):
`-f rawvideo -pix_fmt bgra -video_size inputWxH -framerate FPS -i pipe:0` (**NO `-re`** — the FramePump
is the pacer since slice 9, 2026-09-10; `-re` added a second, fighting clock on the rawvideo demux).
When input and output dimensions differ, each output block adds `-vf scale=outputW:outputH:flags=bilinear,setsar=1`;
otherwise frames pass through at their existing dimensions.
+ a **real audio input — the
mixer writes IEEE-float stereo to a Windows named pipe** (`-f f32le -ar 48000 -ac 2
-i \\.\pipe\ytllive_audio`, name via `EncoderOptions.AudioPipeName`; replaced the old `-f lavfi -i
anullsrc` silence in the TASK 8 audio milestone) + explicit `-map 0:v -map 1:a` + `-c:v <enc> -b:v K
-maxrate K -bufsize 2K` + **`-g fps×4 -keyint_min fps×4 -sc_threshold 0 -bf 0 -pix_fmt yuv420p`**
(≤4s keyframes, closed GOP, H.264 compliance) + `-c:a aac -ar 48000 -ac 2 -f flv <rtmpUrl>`.
**Record output (TASK 18):** `FfmpegArgs.Build` emits one self-contained block per output, each its own
`-map 0:v -map 1:a` + codec tags (`AddVideoTags` helper). Stream block = `-f flv <rtmpUrl>`; record block =
`-f mp4 <RecordPath>`. `EncoderOptions.StreamEnabled`/`RecordEnabled`/`RecordPath` gate each block, so
the engine runs record-only (no RTMP) or stream-only — **stream+record simultaneously is OUT by creator
ruling (2026-09-01): the VOD is already the copy, and dual-encoding drags mid-range chassis and degrades
BOTH outputs ("we're not them"). The old "cheap on NVENC" claim was an unverified assumption; the
UI constraint is landed (2026-09-20): the REC/ON-AIR pills are radio-exclusive — arming one clears the
other — while the dual-block machinery stays generic.**
`FfmpegEncoder.StartAsync` throws unless at least one output is enabled.
**Encoder choice is probed from the binary's `-encoders` listing** (`FfmpegEncoderPicker`, pure):
NVENC → QSV → AMF → OpenH264 fallback, **never libx264** (GPL; see Licensing). `EncoderOptions.VideoEncoder`
forces one and skips the probe. `EncoderOptions` carries raw input W×H from the compositor crop, final
output W×H/FPS/bitrate from the quality tier, and
`GopSize` = FPS×4. Constructed by `MainViewModel` since ship step 5 (`new FfmpegEncoder(new FfmpegLocator())`),
driven by the `FramePump` below.
---
## Recording
### Local recording (TASK 18 — shipped 2026-08-29, VM + top-bar UX; plan in TASKS.md)
Recording is **independent from streaming** and needs no YouTube sign-in. A single segmented
**REC|ON-AIR switch** in the top bar picks the mode (redesigned 2026-09-22 from the old two pills):
**REC** (local file, works signed-out) / **ON-AIR** (streaming; **armable signed-out** — intent may
light while offline, 2026-09-20: a greyed pill was the dead end the guard shipped to forbid). The switch
selection is intent; `IsRecording`/`IsLive` are reality — the reality line (dot + word + elapsed) only
lights when a session is actually running.
- **State model (`MainViewModel`):** pills `RecordPillOn`/`OnAirPillOn` stay the **routing model**
(radio-exclusive — arming one clears the other, record-OR-live 2026-09-01; the ON-AIR setter stays
armable signed-out (2026-09-20: intent must be able to light while offline)). **Top bar redesign
(2026-09-22):** the bar is ONE surface rendered by the FIRST decision — Record or Stream — and each
world is the whole bar (`ShowIdleWorld`/`ShowRecordAction`/`ShowStreamActions`, wired by
`RaiseTopBarModes()`). The old sign-lights + two pill toggles + status dot + PRIVATE/TEST chips are
gone; the mode is one **segmented REC|ON-AIR switch** (`SegmentToggle`/`SegmentLabel` styles), and
"going back" is one tap on the other segment. Idle worlds: Record = switch + "Start Recording"
(`PrimaryStartButtonLabel` names the armed world); Stream = switch
+ "Go Live" + Test (Test is a **child of Stream** — visible only stream-armed AND signed-in) plus the
account zone on the right. **The account zone is WORLD-INDEPENDENT (2026-09-23):**
`ShowSignInButton` = signed-out and `ShowAvatarButton` = the signed-in avatar, in Record and Stream
worlds alike — it's app-level identity, and keeping it put pins the bar height (it no longer pops in
and out between worlds). Right-click the avatar = Change Account/Logout. Running retires the switch
and worlds; one reality line
`● word elapsed` + End replace them (`RunningVisible`/`RunningText`/`RunningDotBrush` — TEST/LIVE/REC,
gold/red/green). The account **status dot is gone** — the avatar/Sign In pair carries connectedness.
`ShowPrimaryStartButton` remains the always-idle face. `StopStream` clears both pills (see "Stop ends
everything; failures roll back" below).
- **Filenames** (pure `Services/RecordingFile`): auto-name `ty-<yyyyMMdd>-<HHmm start>-<fff>-0000.mp4` at
start; **rename-on-stop** to `ty-…-<hh2mm2 actual length>.mp4` (`FinalizeRecordingAsync` after the pump
stops & the file closes), numeric `-2`/`-3` suffix on collision (`UniquePath`). Length comes from
`_liveElapsed`, which the session timer walks while `IsLive || IsRecording`. The **`<fff>` millisecond
segment is load-bearing, not decoration** (2026-09-27): both names derive from `_recordStartTime`, so it
survives the rename, and without it two instances started in the same minute compute the *same* start
path and both ffmpeg processes write one file — which is exactly the dev multi-instance case
(`Helpers/InstanceProfile`, one instance recording the other).
- **Private broadcasts end the same way (creator ruling 2026-10-09).** After `StopStream`'s
transition(complete), `OfferToKeepPrivateBroadcastAsync` reads the broadcast's privacy **from YouTube**
(`GetBroadcastPrivacyAsync` — visibility can change mid-stream, local state isn't trusted) and, only if
it is `private`, shows `KeepBroadcastDialog` (repo root): **Enter = Keep** (`IsDefault`), **Escape /
Delete / the X = delete** via `DeleteFinishedBroadcastAsync` — `videos.delete?id=<broadcastId>` (a live
broadcast's id IS its video id; 204), falling back to `liveBroadcasts.delete` on 404. An unreadable
status asks nothing and deletes nothing; a failed delete toasts that the video is still on YouTube.
Decision logic = `MainViewModel.ClosePrivateBroadcastAsync` (internal static, tested in
`PrivateBroadcastCloseOutTests`); `KeepBroadcastPromptOverride` is the dialog seam.
- **Save dialog: Cancel DISCARDS (creator ruling 2026-09-26).** The `RenameRecordingDialog` result is a
two-outcome decision, and it lives in the extracted `MainViewModel.CompleteRecordingSave(startPath, dir,
autoStem, chosenStem, creatorSaved)` (internal, so `ytLive.Tests` can drive it against real files without
a frame pump — see `ytLive.Tests/RecordingSaveDialogTests.cs`):
- `creatorSaved == false` (**Cancel / Escape / the X**) → **DELETE the temp file.** Nothing is moved,
nothing is left in the videos folder. A failed delete returns `DiscardFailed` and the toast NAMES the
file + folder, because a "discarded" recording still on disk is worse than no report.
- `creatorSaved == true` (Save, or **Enter on the pre-filled default**) → `File.Move` to the final name.
A blank/whitespace box still means "keep the auto name" — but only on an explicit Save.
- **The bug this fixed:** the caller used to `File.Move` UNCONDITIONALLY and only overwrite `stem` when
the dialog returned true, so `ShowDialog() == false` fell straight through into the save — Cancel
silently kept the recording under the default name. The modal itself was always correct; the
side effect was in the wrong place. **Any modal whose result gates a side effect must have every
branch handled explicitly — a falsy result must never fall through into the effect.**
- **Folder:** default `%APPDATA%\ytLlive\recordings\`, user-overridable via `ChooseRecordFolderCommand`
(`OpenFolderDialog`), persisted through `LayoutStore.Load/SaveRecordFolder` (`RecordFolder` key).
- **Explicit sign-out only:** `StopStream` no longer clears the session/token. Sign out via **Logout
only** — Change Account re-authenticates *in place* and deliberately does not sign out first
(2026-09-27; signing out first deleted `ytLlive.auth` before the browser even opened, so a
cancelled switch lost a working session). Stopping a recording leaves the creator signed in.
- **Stop ends everything; failures roll back (2026-09-01):** `StopStream` also clears both intent
pills (`RecordPillOn`/`OnAirPillOn`) — a lit pill with no session behind it is a lie. Every
frame-pump death and every go-live prep failure now runs the full `StopStream` teardown instead of
leaving a limbo (`StreamStatus.Error` with `IsRecording=true` over a dead encoder — the first-launch
zombie recording). Toast copy distinguishes "Recording stopped" from "stream pipeline stopped";
`AudioMixer.StopLive` is idempotent-safe for rollbacks that never reached StartLive. Seams:
`OnFramePumpFailed` + `IsRecording` setter made internal (test-only, InternalsVisibleTo).
Test: `SessionTeardownTests`.
- `EncoderOptions.StreamEnabled/RecordEnabled/RecordPath` gate outputs; record+stream is one ffmpeg with two
output blocks (the dual-block machinery is unreachable from the UI since the pills are radio-exclusive).
Rest of the engine (FramePump/encoder) is unchanged — `BuildEncoderOptions` always returns an options
(flags from the pills + `_activeRecordPath`), so record-only reaches the pump.
---
## Audio
### Live audio capture (TASK 4 ship step 4 — shipped 2026-08-12; game audio bar 2026-08-13; **TASK 8 audio milestone: real stream audio + filters + duck + TRAX, shipped 2026-08-14**, plan in TASKS.md)
**Capture runs for the app's lifetime and is KISS by rule**: desktop/game audio is automatic (WASAPI
loopback), the mic is the creator's only audio control (meter/mute/volume already shipped). The whole
layer sits behind an **`IAudioSource` seam** (`Services/Audio/`: `Start`/`Stop`/`Started`/`SampleReady`/
`Failed`, IDisposable) so the app never touches NAudio directly and tests inject hermetic fakes (no
devices, no timers).
- **Sources (NAudio 2.2.1 — `NAudio.Wasapi` + `NAudio.WinMM` (for `WaveOutEvent`), MIT — item 9 in
`THIRD-PARTY-NOTICES.txt`):** `WasapiLoopbackAudioSource` = `WasapiLoopbackCapture` on the default
render device; `WasapiMicAudioSource` = `WasapiCapture` with the device resolved by `FriendlyName`
matching `MicSourceName` (the app only persists the **DisplayName**), falling back to the default
capture endpoint. Both run at the **device's own mix format** — NAudio 2.2.1 exposes no overridable
`GetDefaultMixFormat` on `WasapiCapture`, so there is no forced 48 kHz path; the mixer's
`TinyResampler` normalizes any rate/channel layout to 48 kHz stereo (this replaced the earlier
"force IEEE-float 48k" plan — the resampler IS the design). Mic device resolution re-reads the name
provider `Func<string?>` at each `Start`, so a mic picked mid-session takes effect **immediately**
(the mixer restarts the mic on pick). Both sources raise `Started` once their capture loop actually
begins — the mixer turns that into `MicConnected`.
- **`AudioMixer`** owns both sources; **`StartMicCaptureAsync` starts the mixer once at startup** and
`Shutdown` disposes it — NOT go-live — so the meters preview live. Mic samples: downmix to mono →
`TinyResampler` → **`VoiceFilterChain` per sample** (bass `LowShelfFilter` 120 Hz +4 dB → treble
`HighShelfFilter` 8 kHz +3 dB → `NoiseGate` (threshold 0.005, hysteresis 0.5, attack 0.5, release
0.0005) → **`SpeechLeveler`** (2026-10-09: automatic gain steering speech RMS to −20 dBFS over a
0.3 s window, gain −6..+20 dB, rises ≤10 dB/s / falls ≤30 dB/s, learns only while the gate is open
and above −54 dBFS so pauses/hum are never pumped) → `Compressor` (threshold 0.5, ratio 4:1); all
pure DSP) → ring buffer → the
**post-filter** `AudioLevelMeter` (RMS, 0.2 exponential smoothing) → `MicLevelChanged` →
marshalled to the UI thread → `AudioLevel`. Loopback samples: resample → ring buffer → second meter
→ `LoopbackLevelChanged` → the game bar's `GameAudioLevel`. The raw linear level maps to the meter's
display scale by `AudioLevelMeter.ToDisplay` (−60..0 dBFS → 0..1, **honest** — the +10 dB display boost
was removed 2026-10-09; the mic meter is `ToDisplay(AudioLevel × MicVolume)`, true dB math, no ×1.2).
**Why (2026-10-09):** a creator's YouTube upload measured −28.5 LUFS / −8.9 dBFS peak against YouTube's
−14 LUFS reference (YouTube only turns loud videos down, never quiet ones up) while the boosted meter
looked "dialed in". Nothing in the chain added gain — the compressor folds only above −6 dBFS and the
limiter only caps. Re-running that same audio through the new chain (mic volume 0.8): **−17.9 LUFS,
−0.7 dBFS peak**. Recording and streaming share this one mix, so both were (and now are) the same.
Measure with the bundled ffmpeg: `ffmpeg -i file -af ebur128=peak=true:framelog=quiet -f null -`.
**Mic connection state surfaces as events:** `MicConnected` (source `Started`), `MicFailed` (source
`Failed`), and `RestartMic()` re-resolves + restarts just the mic (loopback keeps running). Failures
log via `AppLog`; a mic failure zeroes the meter, a loopback failure never kills the mic. Meter `Push`
is unconditional (a `?.` on the event would skip the meter update when nothing is subscribed yet).
- **Go-live audio (TASK 8):** `BeginGoLive` → `StartLive(pipeName)` (idempotent) creates the named pipe
server (`NamedPipeAudioWriter`, behind the `IAudioPipeWriter` seam) and a live loop task; every
`mixInterval` (default 10 ms) `FillAndMix` reads a chunk from each ring buffer (silence-fills
underruns), computes the post-filter mic RMS → `AutoDucker` (threshold 0.02, duck ×0.25 / −12 dB,
attack 0.05, release 0.005, always on), applies the **honest gains** via `Func<double>` seams
(`micGain = MicMuted ? 0 : MicVolume`; `loopbackGain = GameMuted ? 0 : GameAudioVolume`, × duck),
mixes mono mic + stereo loopback into interleaved stereo, runs the
**`MasterLimiter`** (a **−1 dBFS ceiling** so the honest gains can never sum past 0 dBFS and clip the
AAC encode — instant attack per frame, smoothed release toward unity), and writes the floats to the pipe. **The gains are wired through `AudioGainProvider`**
(`Services/Audio/AudioGainProvider.cs`) — the VM hands its `MicGain`/`LoopbackGain` Funcs to the
mixer at construction, read live each tick, so the volume sliders and mute buttons are stream-honest.
(Before this wiring the seams defaulted to unity and the controls were decorative.) `StopStream` →
`StopLive()` (closes the pipe → ffmpeg audio EOF) **BEFORE** stopping the frame pump (video EOF) —
the reverse order stalls on pipe backpressure. Encoder step-down follows the same order: close the
audio pipe, flush/stop the pump, create a fresh server on the same name, then restart the pump so ffmpeg
connects to a live server. `FfmpegEncoder` itself is untouched; the VM owns the pipe lifecycle.
**Recordings are fragmented MP4 (TASK 32·4, 2026-11-05):** the record block carries
`-movflags +frag_keyframe+empty_moov+default_base_moof`, so the index is written first and a playable
fragment is flushed at every keyframe. A plain MP4 puts its index at the end, which is why a power cut
or a killed process previously left the whole take unreadable. The FLV stream block never gets it.
**Two clip-audio rings join that mix (2026-11-05):** `EnqueueAlertAudio` (TASK 47) is pre-scaled by the
alert layer and mixed at **unity, never ducked** — an alert must be heard over the mic.
`EnqueueDesktopMediaAudio` (TASK 21's audio slice) is the opposite on purpose: a video layer's
soundtrack joins the **desktop/game ring**, so the desktop fader and the auto-ducker apply exactly as
they do for game audio and TRAX. Both are fed by the shared `Services/VideoClipPipe.cs`; the layer's
own Mute/Level are applied upstream by `MediaVideoSourceManager` (level/mute are per **session**, since
one decoder is shared per file path).
- **Audio sync offset (TASK 22):** after the limiter, `FillAndMix` routes the whole interleaved stereo
mix through a **`AudioSyncDelay`** (`Services/Audio/AudioSyncDelay.cs`) — a pure delay line whose
offset comes from a `Func<int> syncOffsetMs` seam (the VM's global `AudioSyncOffsetMs`, **−500..+500
ms**, persisted as `Audio.SyncOffsetMs`). This is OBS's documented fix for lip-sync: **positive**
delays audio when it runs ahead of video; **negative** advances audio when it runs behind by eating
the first `|N|` ms of the live stream head (see `AudioMixer.StartLive`; the delay line stays
clamped 0..500 internally — negative bypasses it entirely). The advance is armed once at `StartLive`
(eating the head mid-stream is impossible); positive is live-reactive (re-read per tick). A slider
on the mic bar (`"AUDIO SYNC"`, locked while live/recording via `IsEditMode`) surfaces it
(a one-time status dot next to it was removed 2026-09-23 per creator feedback — the slider +
numeric tooltip carry the state). Changing the delay flushes the line (a live change clicks
rather than smears).
- **TRAX — free background music (TASK 8):** `MusicPlayer` = NAudio `MediaFoundationReader`
(mp3/wav/m4a) → `VolumeWaveProvider16` at the hardcoded **0.20** bed (no slider) → `WaveOutEvent` on
the default device, **looping on any clean natural end** (`PlaybackStopped` with `e.Exception == null`
→ rewind + replay; the earlier `Position >= Length` check was unreliable for `MediaFoundationReader`
and let tracks play once then stop). The desktop/game bar's mute toggles `LocalGain` (0 or 1;
`UpdateTraxLocalGain` runs on every `GameMuted` change and on TRAX load), so muting kills TRAX
in the headphones — matching the stream, where it rides the loopback channel. The volume slider
drives system volume (`PushSystemVolume`) — which scales the HEADPHONES; the loopback capture is
pre-endpoint-volume (disproven-then-fixed 2026-09-01), so the STREAM follows the knob via
`AudioGainProvider.LoopbackGain = GameMuted ? 0 : GameAudioVolume`, and the game meter multiplies
its display by the same. One knob, three honest paths. No third mixer input. The footer
**TRAX** button lives **inside the game audio bar** (in the preview overlay), **left of the
"Desktop Audio" label** (the creator's pick — it rides the desktop channel, so the music control
sits with the desktop audio controls): status dot (**red** no track / **yellow** loaded stopped /
**green** playing) + "TRAX"; **left-click** toggles play/pause (opens the in-app picker via `OpenFileDialog` when no track is loaded); **right-click** always opens
the picker (`TraxButton_PreviewMouseRightButtonUp`, `e.Handled = true`, code-behind pattern); tooltip
shows the loaded/playing track name or "No track — right-click to choose background music". The picked
track persists via **schema v9** single-row `Music` (`TrackPath`/`IsEnabled`) in `LayoutStore`. Known
wrinkle (out of scope, future feature): YouTube mutes VODs carrying copyrighted music — "music on
live, off VOD" is tabled.
- **⛔ The load is behind a bounded probe, because a share that is gone must never
freeze startup (2026-10-04).** `LoadLayout` → `LoadTrax` used to call `MusicPlayer.Load`
synchronously on the UI thread, and `new MediaFoundationReader(path)` does **not** fail
fast on a dead SMB session — it burns the whole ARP/TCP/SMB timeout (~40s, measured)
and *then* throws `0x80070035`. Measured: `OnStartup` 08:42:59 → `MainWindow loaded`
08:43:43, with 42 of those 44s inside `LoadLayout` on that one track. The rule now:
**a probe may time out; only a bind is allowed to be on the creator's path.**
`IsTrackReachableAsync(path, budget, File.Exists)` runs the probe on a thread-pool
thread under a `Task.WhenAny` ceiling (`TraxProbeBudget` = **2s**) and reports
offline/missing/slow as a normal state, not an error. Only a probe that passed gets
the real bind, and **that bind stays on the UI thread** with every other
`MusicPlayer` mutation, so no new cross-thread race is introduced. The orphan probe
thread stays parked until SMB gives up — deliberate: it is not the thread anyone
waits on. Same principle covers the webcam: a device held by another app fails fast
(sharing violation) but a virtual-camera driver can stall enumeration, so device
probes belong off the startup path too, not behind a long timeout.
- **Mic status dot (`Models/MicStatus.cs`)** on the preview-bottom MIC button: green = `MicConnected`,
yellow = `MicFailed` (in use/unplugged), red = no mic device at startup (the mixer is never started,
so loopback and the game bar can't run either — no capture devices at all). **The picked mic persists:
`LayoutStore`'s `Settings` key/value table stores `MicSourceName` (saved on pick via
`SaveMicSourceName`), and the VM restores it at construction BEFORE the mixer's first `Start` — so a
restart reconnects the same already-vetted device (green) or reports it missing (yellow), instead of
silently falling back to the default endpoint.**
- **Game audio bar** (desktop/game, **always visible** — TASK 4's `IGameAudioDetector` show/hide gating was **removed 2026-08-15**: the bar used to appear only while a full-screen game with sound was up, which kept hiding the creator's desktop meter; the detector stack is gone, desktop audio is just automatic WASAPI loopback): overlaid **at the bottom of the preview window** (bottom-center, dark translucent
chip, a mirror of the mic bar: meter + mute + volume slider). It's monitoring UI **in the preview
grid** — the meter is display-only (fill = `ToDisplay(GameAudioLevel) × GameAudioVolume` — the ×volume
term was missing until 2026-09-01: loopback capture does NOT shrink with the endpoint, so an
unscaled meter left it pegged at 20% volume; `GameMeterHonestyTests` guards it), but its **volume
slider drives system volume** (`PushSystemVolume` — live headphones follow the knob; the capture
side is scaled in the mix, see TRAX note above) **and mute kills TRAX** (`MusicPlayer.LocalGain` = 0). Relabelled
**"Desktop Audio"** (TASK 8) since TRAX rides the same channel.
- **`WaveToFloat`** (pure, shared): WASAPI mix formats → interleaved float — IEEE float 32-bit direct,
PCM 16-bit normalized to -1..1, `WaveFormatExtensible` with the IEEE-float subformat GUID
(`NAudio.Dmo.AudioMediaSubtypes.MEDIASUBTYPE_IEEE_FLOAT`), trailing partial samples ignored.
- Build **0 warnings**; **197 passing** (DSP/ring-buffer/ducker/resampler/pipe + mixer/hysteresis/
game-detector/meter-scale unit tests + the TASK 8 integration test
`AudioPipelineTests.Mix_WithFiltersDuckAndGain_Lands_On_AudioPipe` + the polish-batch integration
test `Mix_HonorsProviderGains_AndGameMute_KillsTheLoopback` proving the provider gains reach the
pipe and mute silences the loopback).
---
## Related files
- [`frame-pipeline.md`](frame-pipeline.md) — the `FramePump` that feeds this encoder (pacing, backpressure, teardown order).
- [`ui-surface.md`](ui-surface.md) — resolution tiers (W×H/FPS/bitrate) and toast notifications.
- [`youtube-api.md`](youtube-api.md) — go-live/prep ordering and the broadcast lifecycle.
- [`product.md`](product.md) — licensing guardrails (the ffmpeg pin, LGPL obligations).
- [`TASKS/research-store-certification.md`](../../TASKS/research-store-certification.md) — §4, the ffmpeg bundling analysis.