fix: clear pre-live capture ring backlog at StartLive (2.2s A/V sync lag)

AudioMixer.Start() begins mic/loopback capture at app startup to feed the
level meters, so both 2s ring buffers fill with pre-live audio. StartLive()
drained from the oldest tail sample, putting every recorded event ~2.2s late
in the audio track (confirmed by clap analysis + cross-correlation on two
takes: +2.11 to +2.22s).

Fix: AudioMixer.StartLive() now runs _micBuffer.Clear() + _loopbackBuffer.Clear()
immediately after the pipe starts, before the drain task runs. Recording now
begins at go-live; the <=10ms in-flight chunk evicted by Clear() is imperceptible.

Regression test: StartLive_DiscardsPreLiveBacklog_SoFirstAudioIsCurrent
saturates the loopback ring with stale 0.8 pre-live audio, then asserts the
wire carries fresh post-live 0.2 (max < 0.3).

Reference (external, per derivative-work rule): OBS 'Audio mixer' keeps its
buffers fed continuously and syncs the stream start timestamp at record time
rather than replaying pre-live capture; a go-live flush of the capture buffer
is the accepted pattern for live tools restarting a stream.

Docs: HANDOFF.md (fix shipped), MyMistakes.md (A/V sync measurement recipe).
This commit is contained in:
2026-09-14 11:24:12 -07:00
parent 5ba4d709ae
commit 11a7af2dc0
4 changed files with 230 additions and 148 deletions
+41
View File
@@ -174,6 +174,47 @@ packages, high-quality downscale via `TransformedBitmap`. Screenshots compress
---
### Measuring audio-video A/V sync from a recording (no ears needed) — RECIPE
Derived 2026-09-12/14 (the "audio cut off at the end" / "audio delayed" reports). You
cannot "listen" to a take — measure it. Proven on two independent takes (talking+claps,
clean-clap test) with a tight, matching result each time.
**Method (FFmpeg probe + numpy, run from WSL):**
1. **Click/hiss-SILENT test takes are the gold standard.** Have the creator do a
loud, single-frame-syncable event (one clap after ~10s of near-silence) — the
video position of the hands-meet peak and the audio position of the transient
are both unambiguous. That single event replaced counting words forever.
2. **Extract audio as raw mono PCM** (`ffmpeg -vn -ac 1 -ar 48000 -f s16le`) and
compute a short window RMS envelope (5ms windows) in numpy. The clap is the global
max (`argmax`); also print a coarse 100ms table — it shows the pre-clap artifacts
(faint blips) vs the event (100× larger) vs true digital silence.
3. **Video motion per frame** — `-vf "tblend=all_mode=difference,signalstats,metadata=print"`
captured to stdout (NOT `file=` inside the filter — the `\\` path escapes mangle;
have PowerShell-style quoting bite; `metadata=print` writes to ffmpeg's log, so
redirect stdout to a file). Parse `YAVG` values; the clap is the frame with the
motion spike (2.75 vs a ~0.1–0.5 animated-widget background).
4. **Offset = audio_event_time − video_event_time**; every positive second means audio
is BEHIND video by that much. Cross-correlate the two full envelopes (60fps-resampled
RMS vs YAVG, normalized) to double-check the single-event peak — the autocorr-style
peak must be sharp (unique max at +133 frames, second-best ≈0.26).
5. **Sanity checks baked in:** count frames (`nb_frames` vs wall log) — if video is
real-time you've ruled out the truncation bug as the cause; compare stream
durations (audio > video by the lag is the SIGNATURE of a delayed audio tail, not
truncation); check `dropped writes` — dropped audio moves events EARLIER (opposite
sign), so it can never explain a lag. **A constant offset ≠ drift:** fixed lag =
buffering/backlog, growing offset = clock mismatch.
**The ROOT CAUSE the recipe led to (record it so it's never re-derived):** the mixer's
capture rings are fed from APP STARTUP (for the meters); `StartLive` never clears them,
so the drained audio trail begins ~a full ring-depth (2.0s) behind go-live. **Rule: any
"live preview/drain" sink fed by a continuously-capturing buffer MUST clear the buffer
at go-live, or early output is stale backlog.** The 2s ring + configured 300ms sync
delay + ~100ms pipeline = exactly the measured 2.11–2.22s. The lag ALSO scales with
"time since app launch" up to the ring depth — a take right after a restart reads as
~370ms while a fully-warmed app reads ~2.2s. Same code, wildly different numbers.
### Feeding a rawvideo pipe at 60fps: deadline pacing + row-blit budget
Derived 2026-09-03 (take-3 diagnosis — the stats seam from `97ffc42` named the stage