Files

T

gramps 47f409f0e4 fix(audio): drive the mix from the clock — the audio track was 2.1% longer than the show

Measured on the creator's recording (ty-20261112-1118-423-0000-2.mp4), which drifts
out of sync over ten seconds:

  wall clock (pump start -> audio stop)   9.875 s
  video: 594 frames, uniform 1/60         9.888 s   correct
  audio: 485,376 samples @ 48 kHz        10.112 s   +2.1%, 224 ms long

So it was not VFR and not wall-clock PTS: the video is CFR and matches the clock (the
only PTS anomaly in the file is one 38 ms gap from two lost frames). The audio track is
what was too long, and a static offset cannot fix a rate error — which is why the opening
looked aligned (the 175 ms mic delay cancels the camera's constant pipeline latency) and
the end did not.

Cause: LiveLoopAsync wrote a fixed 480 frames per iteration and trusted that each
iteration was 10 ms. Under a raised timer resolution they land near 9.8 ms — 102 ticks
per second where 100 are expected, i.e. 102% of an audio second per wall second. FFmpeg
stamps audio by sample count, so every second of show pushed the sound ~2% later against
a correct picture. Video never had the bug because the pump paces off a stopwatch.

Fix: Services/Audio/AudioClock asks the clock how many frames are owed
(expected - produced, clamped to one 200 ms chunk) and the loop produces exactly that,
padding on underrun. A stalled loop pays its debt over the following ticks instead of
dumping one huge write. FillAndMix takes a frame count, and the master limiter/meter take
spans, so a partial chunk is processed without touching stale buffer tail.

674 tests (one log-rotation flake under full-suite load while the app itself is running
and writing startup.log; passes alone), 0 warnings. Ear check pending.

2026-11-12 11:28:28 -08:00

32 KiB

Raw Blame History

Output — the ffmpeg bundle, the encoder, recording, and audio

Split out of ai.md (2026-10-26) so a session reads only what its task touches; ai.md keeps a short invariant stub for this file. Directory map: index.md.

Everything that leaves the app: locating and shipping ffmpeg.exe, how the encoder subprocess is driven (video stdin + audio named pipe), local recording as a first-class mode, and the WASAPI capture / filter / mix chain that feeds it. Content is the decisions plus the failures that produced them; superseded designs are marked inline.

The encoder

FFmpeg locator (TASK 4 ship step 2 — shipped 2026-08-10; rewritten to BUNDLED 2026-09-28, plan in TASKS.md)

The encoder's one external dependency is ffmpeg.exe. It is BUNDLED — never downloaded, never fetched at runtime. IFfmpegLocator resolves an absolute path on demand: the bundle first (<AppContext.BaseDirectory>\ffmpeg\ffmpeg.exe), then PATH as the escape hatch for someone who deliberately installed their own. The order is the opposite of the original ("the user's own install wins") on purpose: a PATH ffmpeg may be a different major version or a GPL build, and the bundle is the one that was pinned, license-vetted and tested. If neither hits, it throws a loud IOException naming the missing folder and the fetch script — the old "found something on disk" cache path is gone deliberately, because silently satisfying a missing bundle with a stale copy is exactly the silent drift a certification-gated dependency must not have.

The bundle is not in git (135 MB of incompressible DLLs would tax every commit on this 9p-bridged tree). scripts/fetch-ffmpeg.sh populates ./ffmpeg from a pinned month-end BtbN LGPL-shared win64 artifact, verifies SHA-256 before extracting, and is idempotent; set FFMPEG_ZIP=<path> to reuse a pre-downloaded archive. ytLive.csproj copies ffmpeg\** to the output (CopyToPublishDirectory="Always" for the MSIX package) and the GuardBundledFfmpeg target fails the build if the folder is absent, so a fresh clone finds out before the creator presses Record rather than during a take. ffmpeg.exe + ffprobe.exe + the libav*.dll family must stay in one folder — Windows resolves a shared build's DLLs from the executable's own directory. ffplay.exe (18 MB) is deliberately not shipped; nothing calls it.

BtbN LGPL-shared (not gyan.dev, not static): it drops GPL-only libx264/x265 while keeping NVENC/QSV/AMF + libopenh264 + native AAC, and dynamic linking means LGPL compliance is "license text + source offer" with no static-relink (§6) material — see the Licensing guardrails below. libopenh264 is compiled into avcodec-63.dll (verified 2026-09-28: the encoder is registered there, and there is no separate OpenH264 binary to ship), so Cisco's BSD terms + patent grant ride along inside item 1 of THIRD-PARTY-NOTICES.txt. Pin: autobuild-2026-08-31-13-27 (month-end on purpose — BtbN keeps ~14 dailies but month-end builds ~2 years; a daily pin is what 404'd, see below).

⛔ Why this was rewritten (2026-09-27 → shipped 2026-09-28). The pinned-download design was a defect, not a cold-start edge case. It failed in production on 2026-09-01 — the previous daily pin aged out of BtbN's 14-day retention and the download 404'd on the creator's first real recording attempt. Two facts made "the pin is a const, just bump it" wrong:

Microsoft Store policy 10.2.2 forbids dynamic code inclusion. Downloading an executable from the internet and running it is the exact pattern the policy names. An MSIX submission is rejected over it.
Bumping the pin is whack-a-mole against a retention window we do not control.

Bundling deletes the startup network dependency entirely. The LGPL-shared build choice is unaffected — dynamic linking was chosen for LGPL §6 compliance ("license text + source offer"), not to enable downloading. Analysis: TASKS/research-store-certification.md §4. Guardrail: FfmpegLocatorTests.TheLocatorCarriesNoNetworkCapability asserts the locator's source contains no HttpClient/URL/downloader at all, so the policy cannot rot back.

Every other third-party binary already ships via NuGet — ffmpeg was the only download-and-execute. Verified 2026-09-28: e_sqlite3.dll and WebView2Loader.dll land in runtimes\win-x64\native\ (SQLitePCLRaw + WebView2 packages) and NAudio/Notification.Wpf/Microsoft.Data.Sqlite/ProtectedData are managed. Two gaps remain and are not fixed by bundling: the WebView2 Evergreen runtime is a machine prerequisite (a manifest framework-dependency + graceful-failure path, TASK 48 item 2), and Velopack's updater downloads and executes — its teardown is a compliance action, not just cleanup.

Live encoder + RTMP push (TASK 4 ship step 3 — shipped 2026-08-12, plan in TASKS.md)

The encoder is a thin orchestrator over ffmpeg.exe — no H.264/AAC code in the app. It spawns the subprocess (path from IFfmpegLocator), feeds raw BGRA compositor-crop frames into stdin, lets FFmpeg scale them to the selected output tier, and parses -stats stderr lines into StreamHealth (bitrate/FPS/duration, dropped-from-frame-count). FfmpegEncoder (IFfmpegEncoder seam) holds: StartAsync (locate → probe -encoders → spawn → stderr loop → drain loop), SubmitFrameAsync (slice 10: bounded-queue ENQUEUE — never a pipe write; the drain task owns stdin writes), StopAsync (flush queue → stdin EOF → ffmpeg finalizes + exits by itself; a 10s watchdog kills it), Dispose (force-kill + wait), and the HealthUpdated/ProcessFailed events. Pattern: the encoder never touches Process — it drives the IEncoderProcess seam (FfmpegEncoderProcess wraps the real Process, redirected stdin/stdout/stderr

exit control); a Func<IEncoderProcess> factory + the locator are constructor-injected, so the integration test fakes the whole subprocess (probe + encoder) with a Channel-backed TextReader whose Complete() is EOF (null), never a ChannelClosedException.

Decisions (locked): args are pure (FfmpegArgs.Build, no string building in the encoder): -f rawvideo -pix_fmt bgra -video_size inputWxH -framerate FPS -i pipe:0 (NO -re — the FramePump is the pacer since slice 9, 2026-09-10; -re added a second, fighting clock on the rawvideo demux). When input and output dimensions differ, each output block adds -vf scale=outputW:outputH:flags=bilinear,setsar=1; otherwise frames pass through at their existing dimensions.

a real audio input — the mixer writes IEEE-float stereo to a Windows named pipe (-f f32le -ar 48000 -ac 2 -i \\.\pipe\ytllive_audio, name via EncoderOptions.AudioPipeName; replaced the old -f lavfi -i anullsrc silence in the TASK 8 audio milestone) + explicit -map 0:v -map 1:a + -c:v <enc> -b:v K -maxrate K -bufsize 2K + -g fps×4 -keyint_min fps×4 -sc_threshold 0 -bf 0 -pix_fmt yuv420p (≤4s keyframes, closed GOP, H.264 compliance) + -c:a aac -ar 48000 -ac 2 -f flv <rtmpUrl>. Rate control is CBR and it is per-encoder (FfmpegArgs.AddRateControl). The -b:v/-maxrate/-bufsize triad is capped VBR, not CBR: measured 2026-11-12 on the bundled BtbN build, a static frame at -b:v 2000k produced 9 kbps under NVENC's default and 1951 kbps with -rc cbr — the cause of an @8 stream that averaged 5.1 Mbps. NVENC and AMF are passed -rc cbr; QSV has no -rc and selects CBR itself when bitrate == maxrate (libavcodec/qsvenc.c, select_rc_mode), so the triad is its whole ask; OpenH264's -rc_mode bitrate changed nothing measurable here and is not passed. YouTube's encoder spec requires CBR. Never assume -maxrate equals CBR — it does not.

The starting rate is learned, never imposed (Models/RateHistory, ViewModels/MainViewModel.Rate.cs). Each finished broadcast is filed locally — tier, probed encoder, rated, settled, steps spent — and the only thing that may change a starting rate is a proposal the creator clicks. A high-water mark is the highest rate that finished a session with zero steps, never the highest number seen, so one easy session cannot set a target the machine cannot hold. Raising needs 10% above the current rate across five clean sessions of the same tier and encoder (history against NVENC is not evidence about QSV); lowering needs three step-downs in five, because fixing damage should not wait. "Keep mine" is remembered against that specific number, so the same question is not asked again until the evidence moves. Nothing is uploaded. Record output (TASK 18): FfmpegArgs.Build emits one self-contained block per output, each its own -map 0:v -map 1:a + codec tags (AddVideoTags helper). Stream block = -f flv <rtmpUrl>; record block = -f mp4 <RecordPath>. EncoderOptions.StreamEnabled/RecordEnabled/RecordPath gate each block, so the engine runs record-only (no RTMP) or stream-only — stream+record simultaneously is OUT by creator ruling (2026-09-01): the VOD is already the copy, and dual-encoding drags mid-range chassis and degrades BOTH outputs ("we're not them"). The old "cheap on NVENC" claim was an unverified assumption; the UI constraint is landed (2026-09-20): the REC/ON-AIR pills are radio-exclusive — arming one clears the other — while the dual-block machinery stays generic. FfmpegEncoder.StartAsync throws unless at least one output is enabled. Encoder choice is probed from the binary's -encoders listing (FfmpegEncoderPicker, pure): NVENC → QSV → AMF → OpenH264 fallback, never libx264 (GPL; see Licensing). EncoderOptions.VideoEncoder forces one and skips the probe. EncoderOptions carries raw input W×H from the compositor crop, final output W×H/FPS/bitrate from the quality tier, and GopSize = FPS×4. Constructed by MainViewModel since ship step 5 (new FfmpegEncoder(new FfmpegLocator())), driven by the FramePump below.

Recording

Local recording (TASK 18 — shipped 2026-08-29, VM + top-bar UX; plan in TASKS.md)

Recording is independent from streaming and needs no YouTube sign-in. A single segmented REC|ON-AIR switch in the top bar picks the mode (redesigned 2026-09-22 from the old two pills): REC (local file, works signed-out) / ON-AIR (streaming; armable signed-out — intent may light while offline, 2026-09-20: a greyed pill was the dead end the guard shipped to forbid). The switch selection is intent; IsRecording/IsLive are reality — the reality line (dot + word + elapsed) only lights when a session is actually running.

State model (MainViewModel): pills RecordPillOn/OnAirPillOn stay the routing model (radio-exclusive — arming one clears the other, record-OR-live 2026-09-01; the ON-AIR setter stays armable signed-out (2026-09-20: intent must be able to light while offline)). Top bar redesign (2026-09-22): the bar is ONE surface rendered by the FIRST decision — Record or Stream — and each world is the whole bar (ShowIdleWorld/ShowRecordAction/ShowStreamActions, wired by RaiseTopBarModes()). The old sign-lights + two pill toggles + status dot + PRIVATE/TEST chips are gone; the mode is one segmented REC|ON-AIR switch (SegmentToggle/SegmentLabel styles), and "going back" is one tap on the other segment. Idle worlds: Record = switch + "Start Recording" (PrimaryStartButtonLabel names the armed world); Stream = switch
- "Go Live" + Test (Test is a child of Stream — visible only stream-armed AND signed-in) plus the account zone on the right. The account zone is WORLD-INDEPENDENT (2026-09-23): ShowSignInButton = signed-out and ShowAvatarButton = the signed-in avatar, in Record and Stream worlds alike — it's app-level identity, and keeping it put pins the bar height (it no longer pops in and out between worlds). Right-click the avatar = Change Account/Logout. Running retires the switch and worlds; one reality line ● word elapsed + End replace them (RunningVisible/RunningText/RunningDotBrush — TEST/LIVE/REC, gold/red/green). The account status dot is gone — the avatar/Sign In pair carries connectedness. ShowPrimaryStartButton remains the always-idle face. StopStream clears both pills (see "Stop ends everything; failures roll back" below).
Filenames (pure Services/RecordingFile): auto-name ty-<yyyyMMdd>-<HHmm start>-<fff>-0000.mp4 at start; rename-on-stop to ty-…-<hh2mm2 actual length>.mp4 (FinalizeRecordingAsync after the pump stops & the file closes), numeric -2/-3 suffix on collision (UniquePath). Length comes from _liveElapsed, which the session timer walks while IsLive || IsRecording. The <fff> millisecond segment is load-bearing, not decoration (2026-09-27): both names derive from _recordStartTime, so it survives the rename, and without it two instances started in the same minute compute the same start path and both ffmpeg processes write one file — which is exactly the dev multi-instance case (Helpers/InstanceProfile, one instance recording the other).
Private broadcasts end the same way (creator ruling 2026-10-09). After StopStream's transition(complete), OfferToKeepPrivateBroadcastAsync reads the broadcast's privacy from YouTube (GetBroadcastPrivacyAsync — visibility can change mid-stream, local state isn't trusted) and, only if it is private, shows KeepBroadcastDialog (repo root): Enter = Keep (IsDefault), Escape / Delete / the X = delete via DeleteFinishedBroadcastAsync — videos.delete?id=<broadcastId> (a live broadcast's id IS its video id; 204), falling back to liveBroadcasts.delete on 404. An unreadable status asks nothing and deletes nothing; a failed delete toasts that the video is still on YouTube. Decision logic = MainViewModel.ClosePrivateBroadcastAsync (internal static, tested in PrivateBroadcastCloseOutTests); KeepBroadcastPromptOverride is the dialog seam.
Save dialog: Cancel DISCARDS (creator ruling 2026-09-26). The RenameRecordingDialog result is a two-outcome decision, and it lives in the extracted MainViewModel.CompleteRecordingSave(startPath, dir, autoStem, chosenStem, creatorSaved) (internal, so ytLive.Tests can drive it against real files without a frame pump — see ytLive.Tests/RecordingSaveDialogTests.cs):
- creatorSaved == false (Cancel / Escape / the X) → DELETE the temp file. Nothing is moved, nothing is left in the videos folder. A failed delete returns DiscardFailed and the toast NAMES the file + folder, because a "discarded" recording still on disk is worse than no report.
- creatorSaved == true (Save, or Enter on the pre-filled default) → File.Move to the final name. A blank/whitespace box still means "keep the auto name" — but only on an explicit Save.
- The bug this fixed: the caller used to File.Move UNCONDITIONALLY and only overwrite stem when the dialog returned true, so ShowDialog() == false fell straight through into the save — Cancel silently kept the recording under the default name. The modal itself was always correct; the side effect was in the wrong place. Any modal whose result gates a side effect must have every branch handled explicitly — a falsy result must never fall through into the effect.
Folder: default %APPDATA%\ytLlive\recordings\, user-overridable via ChooseRecordFolderCommand (OpenFolderDialog), persisted through LayoutStore.Load/SaveRecordFolder (RecordFolder key).
Explicit sign-out only: StopStream no longer clears the session/token. Sign out via Logout only — Change Account re-authenticates in place and deliberately does not sign out first (2026-09-27; signing out first deleted ytLlive.auth before the browser even opened, so a cancelled switch lost a working session). Stopping a recording leaves the creator signed in.
Stop ends everything; failures roll back (2026-09-01): StopStream also clears both intent pills (RecordPillOn/OnAirPillOn) — a lit pill with no session behind it is a lie. Every frame-pump death and every go-live prep failure now runs the full StopStream teardown instead of leaving a limbo (StreamStatus.Error with IsRecording=true over a dead encoder — the first-launch zombie recording). Toast copy distinguishes "Recording stopped" from "stream pipeline stopped"; AudioMixer.StopLive is idempotent-safe for rollbacks that never reached StartLive. Seams: OnFramePumpFailed + IsRecording setter made internal (test-only, InternalsVisibleTo). Test: SessionTeardownTests.
EncoderOptions.StreamEnabled/RecordEnabled/RecordPath gate outputs; record+stream is one ffmpeg with two output blocks (the dual-block machinery is unreachable from the UI since the pills are radio-exclusive). Rest of the engine (FramePump/encoder) is unchanged — BuildEncoderOptions always returns an options (flags from the pills + _activeRecordPath), so record-only reaches the pump.

Audio

Live audio capture (TASK 4 ship step 4 — shipped 2026-08-12; game audio bar 2026-08-13; TASK 8 audio milestone: real stream audio + filters + duck + TRAX, shipped 2026-08-14, plan in TASKS.md)

Capture runs for the app's lifetime and is KISS by rule: desktop/game audio is automatic (WASAPI loopback), the mic is the creator's only audio control (meter/mute/volume already shipped). The whole layer sits behind an IAudioSource seam (Services/Audio/: Start/Stop/Started/SampleReady/ Failed, IDisposable) so the app never touches NAudio directly and tests inject hermetic fakes (no devices, no timers).

Sources (NAudio 2.2.1 — NAudio.Wasapi + NAudio.WinMM (for WaveOutEvent), MIT — item 9 in THIRD-PARTY-NOTICES.txt): WasapiLoopbackAudioSource = WasapiLoopbackCapture on the default render device; WasapiMicAudioSource = WasapiCapture with the device resolved by FriendlyName matching MicSourceName (the app only persists the DisplayName), falling back to the default capture endpoint. Both run at the device's own mix format — NAudio 2.2.1 exposes no overridable GetDefaultMixFormat on WasapiCapture, so there is no forced 48 kHz path; the mixer's TinyResampler normalizes any rate/channel layout to 48 kHz stereo (this replaced the earlier "force IEEE-float 48k" plan — the resampler IS the design). Mic device resolution re-reads the name provider Func<string?> at each Start, so a mic picked mid-session takes effect immediately (the mixer restarts the mic on pick). Both sources raise Started once their capture loop actually begins — the mixer turns that into MicConnected.
AudioMixer owns both sources; StartMicCaptureAsync starts the mixer once at startup and Shutdown disposes it — NOT go-live — so the meters preview live. Mic samples: downmix to mono → TinyResampler → VoiceFilterChain per sample (bass LowShelfFilter 120 Hz +4 dB → treble HighShelfFilter 8 kHz +3 dB → NoiseGate (threshold 0.005, hysteresis 0.5, attack 0.5, release 0.0005) → SpeechLeveler (2026-10-09: automatic gain steering speech RMS to −20 dBFS over a 0.3 s window, gain −6..+20 dB, rises ≤10 dB/s / falls ≤30 dB/s, learns only while the gate is open and above −54 dBFS so pauses/hum are never pumped) → Compressor (threshold 0.5, ratio 4:1); all pure DSP) → ring buffer → the post-filter AudioLevelMeter (RMS, 0.2 exponential smoothing) → MicLevelChanged → marshalled to the UI thread → AudioLevel. Loopback samples: resample → ring buffer → second meter → LoopbackLevelChanged → the game bar's GameAudioLevel. The raw linear level maps to the meter's display scale by AudioLevelMeter.ToDisplay (−60..0 dBFS → 0..1, honest — the +10 dB display boost was removed 2026-10-09; the mic meter is ToDisplay(AudioLevel × MicVolume), true dB math, no ×1.2). Why (2026-10-09): a creator's YouTube upload measured −28.5 LUFS / −8.9 dBFS peak against YouTube's −14 LUFS reference (YouTube only turns loud videos down, never quiet ones up) while the boosted meter looked "dialed in". Nothing in the chain added gain — the compressor folds only above −6 dBFS and the limiter only caps. Re-running that same audio through the new chain (mic volume 0.8): −17.9 LUFS, −0.7 dBFS peak. Recording and streaming share this one mix, so both were (and now are) the same. Measure with the bundled ffmpeg: ffmpeg -i file -af ebur128=peak=true:framelog=quiet -f null -. Mic connection state surfaces as events: MicConnected (source Started), MicFailed (source Failed), and RestartMic() re-resolves + restarts just the mic (loopback keeps running). Failures log via AppLog; a mic failure zeroes the meter, a loopback failure never kills the mic. Meter Push is unconditional (a ?. on the event would skip the meter update when nothing is subscribed yet).
Go-live audio (TASK 8): BeginGoLive → StartLive(pipeName) (idempotent) creates the named pipe server (NamedPipeAudioWriter, behind the IAudioPipeWriter seam) and a live loop task; every mixInterval (default 10 ms) FillAndMix reads a chunk from each ring buffer (silence-fills underruns), computes the post-filter mic RMS → AutoDucker (threshold 0.02, duck ×0.25 / −12 dB, attack 0.05, release 0.005, always on), applies the honest gains via Func<double> seams (micGain = MicMuted ? 0 : MicVolume; loopbackGain = GameMuted ? 0 : GameAudioVolume, × duck), mixes mono mic + stereo loopback into interleaved stereo, runs the MasterLimiter (a −1 dBFS ceiling so the honest gains can never sum past 0 dBFS and clip the AAC encode — instant attack per frame, smoothed release toward unity), and writes the floats to the pipe. The gains are wired through AudioGainProvider (Services/Audio/AudioGainProvider.cs) — the VM hands its MicGain/LoopbackGain Funcs to the mixer at construction, read live each tick, so the volume sliders and mute buttons are stream-honest. (Before this wiring the seams defaulted to unity and the controls were decorative.) StopStream → StopLive() (closes the pipe → ffmpeg audio EOF) BEFORE stopping the frame pump (video EOF) — the reverse order stalls on pipe backpressure. Encoder step-down follows the same order: close the audio pipe, flush/stop the pump, create a fresh server on the same name, then restart the pump so ffmpeg connects to a live server. FfmpegEncoder itself is untouched; the VM owns the pipe lifecycle. Recordings are fragmented MP4 (TASK 32·4, 2026-11-05): the record block carries -movflags +frag_keyframe+empty_moov+default_base_moof, so the index is written first and a playable fragment is flushed at every keyframe. A plain MP4 puts its index at the end, which is why a power cut or a killed process previously left the whole take unreadable. The FLV stream block never gets it. Two clip-audio rings join that mix (2026-11-05): EnqueueAlertAudio (TASK 47) is pre-scaled by the alert layer and mixed at unity, never ducked — an alert must be heard over the mic. EnqueueDesktopMediaAudio (TASK 21's audio slice) is the opposite on purpose: a video layer's soundtrack joins the desktop/game ring, so the desktop fader and the auto-ducker apply exactly as they do for game audio and TRAX. Both are fed by the shared Services/VideoClipPipe.cs; the layer's own Mute/Level are applied upstream by MediaVideoSourceManager (level/mute are per session, since one decoder is shared per file path).
Audio sync offset (TASK 22): after the limiter, FillAndMix routes the whole interleaved stereo mix through a AudioSyncDelay (Services/Audio/AudioSyncDelay.cs) — a pure delay line whose offset comes from a Func<int> syncOffsetMs seam (the VM's global AudioSyncOffsetMs, −500..+500 ms, persisted as Audio.SyncOffsetMs). This is OBS's documented fix for lip-sync: positive delays audio when it runs ahead of video; negative advances audio when it runs behind by eating the first |N| ms of the live stream head (see AudioMixer.StartLive; the delay line stays clamped 0..500 internally — negative bypasses it entirely). The advance is armed once at StartLive (eating the head mid-stream is impossible); positive is live-reactive (re-read per tick). A slider on the mic bar ("AUDIO SYNC", locked while live/recording via IsEditMode) surfaces it (a one-time status dot next to it was removed 2026-09-23 per creator feedback — the slider + numeric tooltip carry the state). Changing the delay flushes the line (a live change clicks rather than smears).
Microphone-only delay (2026-11-12): a camera's frames reach the compositor later than the mic's samples reach the pipe, so the voice runs ahead of the lips. AudioSyncDelay now takes a channel count, and FillAndMix runs a second line (mono) over micChunk before the mix — loopback and alert audio are not tied to a camera frame and must not move with it. Seam: Func<int> micSyncOffsetMs (VM MicSyncOffsetMs, 0..500 ms, persisted Audio.MicSyncOffsetMs, shipped default 175 — the creator's own C920 measurement, not a constant: the right number belongs to the machine). Presets off / 150 / 200 sit in the LIP-SYNC bay under the whole-mix slider. Both lines are reconfigured every tick, so both carry the 2026-09-12 warning: a Configure that reallocates discards the buffered audio every 10 ms.
The audio clock is the wall clock, not the tick count (2026-11-12). LiveLoopAsync asks Services/Audio/AudioClock.FramesDue(elapsed, produced, rate, maxChunk) how many frames are owed and produces exactly that, instead of writing a fixed 480 frames per iteration. The fixed version turned a scheduler error into a permanent A/V rate error: iterations land near 9.8 ms rather than 10, which is 102 ticks/s where 100 are expected — 2.1% more audio than wall time. Measured on a 9.9 s recording: video 9.888 s (correct, 594 frames at 60 fps), audio 10.112 s, so the sound slid ~200 ms behind the lips across ten seconds and no static delay could fix it. Buffers size to the largest chunk the planner may return (OutputSampleRate / 5), and a stalled loop pays its debt over the following ticks rather than dumping one huge write. Video never had this bug — the pump paces off a stopwatch.
TRAX — free background music (TASK 8): MusicPlayer = NAudio MediaFoundationReader (mp3/wav/m4a) → VolumeWaveProvider16 at the hardcoded 0.20 bed (no slider) → WaveOutEvent on the default device, looping on any clean natural end (PlaybackStopped with e.Exception == null → rewind + replay; the earlier Position >= Length check was unreliable for MediaFoundationReader and let tracks play once then stop). The desktop/game bar's mute toggles LocalGain (0 or 1; UpdateTraxLocalGain runs on every GameMuted change and on TRAX load), so muting kills TRAX in the headphones — matching the stream, where it rides the loopback channel. The volume slider drives system volume (PushSystemVolume) — which scales the HEADPHONES; the loopback capture is pre-endpoint-volume (disproven-then-fixed 2026-09-01), so the STREAM follows the knob via AudioGainProvider.LoopbackGain = GameMuted ? 0 : GameAudioVolume, and the game meter multiplies its display by the same. One knob, three honest paths. No third mixer input. The footer TRAX button lives inside the game audio bar (in the preview overlay), left of the "Desktop Audio" label (the creator's pick — it rides the desktop channel, so the music control sits with the desktop audio controls): status dot (red no track / yellow loaded stopped / green playing) + "TRAX"; left-click toggles play/pause (opens the in-app picker via OpenFileDialog when no track is loaded); right-click always opens the picker (TraxButton_PreviewMouseRightButtonUp, e.Handled = true, code-behind pattern); tooltip shows the loaded/playing track name or "No track — right-click to choose background music". The picked track persists via schema v9 single-row Music (TrackPath/IsEnabled) in LayoutStore. Known wrinkle (out of scope, future feature): YouTube mutes VODs carrying copyrighted music — "music on live, off VOD" is tabled.
- ⛔ The load is behind a bounded probe, because a share that is gone must never freeze startup (2026-10-04). LoadLayout → LoadTrax used to call MusicPlayer.Load synchronously on the UI thread, and new MediaFoundationReader(path) does not fail fast on a dead SMB session — it burns the whole ARP/TCP/SMB timeout (~40s, measured) and then throws 0x80070035. Measured: OnStartup 08:42:59 → MainWindow loaded 08:43:43, with 42 of those 44s inside LoadLayout on that one track. The rule now: a probe may time out; only a bind is allowed to be on the creator's path. IsTrackReachableAsync(path, budget, File.Exists) runs the probe on a thread-pool thread under a Task.WhenAny ceiling (TraxProbeBudget = 2s) and reports offline/missing/slow as a normal state, not an error. Only a probe that passed gets the real bind, and that bind stays on the UI thread with every other MusicPlayer mutation, so no new cross-thread race is introduced. The orphan probe thread stays parked until SMB gives up — deliberate: it is not the thread anyone waits on. Same principle covers the webcam: a device held by another app fails fast (sharing violation) but a virtual-camera driver can stall enumeration, so device probes belong off the startup path too, not behind a long timeout.
Mic status dot (Models/MicStatus.cs) on the preview-bottom MIC button: green = MicConnected, yellow = MicFailed (in use/unplugged), red = no mic device at startup (the mixer is never started, so loopback and the game bar can't run either — no capture devices at all). The picked mic persists: LayoutStore's Settings key/value table stores MicSourceName (saved on pick via SaveMicSourceName), and the VM restores it at construction BEFORE the mixer's first Start — so a restart reconnects the same already-vetted device (green) or reports it missing (yellow), instead of silently falling back to the default endpoint.
Game audio bar (desktop/game, always visible — TASK 4's IGameAudioDetector show/hide gating was removed 2026-08-15: the bar used to appear only while a full-screen game with sound was up, which kept hiding the creator's desktop meter; the detector stack is gone, desktop audio is just automatic WASAPI loopback): overlaid at the bottom of the preview window (bottom-center, dark translucent chip, a mirror of the mic bar: meter + mute + volume slider). It's monitoring UI in the preview grid — the meter is display-only (fill = ToDisplay(GameAudioLevel) × GameAudioVolume — the ×volume term was missing until 2026-09-01: loopback capture does NOT shrink with the endpoint, so an unscaled meter left it pegged at 20% volume; GameMeterHonestyTests guards it), but its volume slider drives system volume (PushSystemVolume — live headphones follow the knob; the capture side is scaled in the mix, see TRAX note above) and mute kills TRAX (MusicPlayer.LocalGain = 0). Relabelled "Desktop Audio" (TASK 8) since TRAX rides the same channel.
WaveToFloat (pure, shared): WASAPI mix formats → interleaved float — IEEE float 32-bit direct, PCM 16-bit normalized to -1..1, WaveFormatExtensible with the IEEE-float subformat GUID (NAudio.Dmo.AudioMediaSubtypes.MEDIASUBTYPE_IEEE_FLOAT), trailing partial samples ignored.
Build 0 warnings; 197 passing (DSP/ring-buffer/ducker/resampler/pipe + mixer/hysteresis/ game-detector/meter-scale unit tests + the TASK 8 integration test AudioPipelineTests.Mix_WithFiltersDuckAndGain_Lands_On_AudioPipe + the polish-batch integration test Mix_HonorsProviderGains_AndGameMute_KillsTheLoopback proving the provider gains reach the pipe and mute silences the loopback).

frame-pipeline.md — the FramePump that feeds this encoder (pacing, backpressure, teardown order).
ui-surface.md — resolution tiers (W×H/FPS/bitrate) and toast notifications.
youtube-api.md — go-live/prep ordering and the broadcast lifecycle.
product.md — licensing guardrails (the ffmpeg pin, LGPL obligations).
TASKS/research-store-certification.md — §4, the ffmpeg bundling analysis.

32 KiB Raw Blame History Unescape Escape