fix: clear pre-live capture ring backlog at StartLive (2.2s A/V sync lag)
AudioMixer.Start() begins mic/loopback capture at app startup to feed the level meters, so both 2s ring buffers fill with pre-live audio. StartLive() drained from the oldest tail sample, putting every recorded event ~2.2s late in the audio track (confirmed by clap analysis + cross-correlation on two takes: +2.11 to +2.22s). Fix: AudioMixer.StartLive() now runs _micBuffer.Clear() + _loopbackBuffer.Clear() immediately after the pipe starts, before the drain task runs. Recording now begins at go-live; the <=10ms in-flight chunk evicted by Clear() is imperceptible. Regression test: StartLive_DiscardsPreLiveBacklog_SoFirstAudioIsCurrent saturates the loopback ring with stale 0.8 pre-live audio, then asserts the wire carries fresh post-live 0.2 (max < 0.3). Reference (external, per derivative-work rule): OBS 'Audio mixer' keeps its buffers fed continuously and syncs the stream start timestamp at record time rather than replaying pre-live capture; a go-live flush of the capture buffer is the accepted pattern for live tools restarting a stream. Docs: HANDOFF.md (fix shipped), MyMistakes.md (A/V sync measurement recipe).
This commit is contained in:
@@ -174,6 +174,47 @@ packages, high-quality downscale via `TransformedBitmap`. Screenshots compress
|
||||
|
||||
---
|
||||
|
||||
### Measuring audio-video A/V sync from a recording (no ears needed) — RECIPE
|
||||
|
||||
Derived 2026-09-12/14 (the "audio cut off at the end" / "audio delayed" reports). You
|
||||
cannot "listen" to a take — measure it. Proven on two independent takes (talking+claps,
|
||||
clean-clap test) with a tight, matching result each time.
|
||||
|
||||
**Method (FFmpeg probe + numpy, run from WSL):**
|
||||
|
||||
1. **Click/hiss-SILENT test takes are the gold standard.** Have the creator do a
|
||||
loud, single-frame-syncable event (one clap after ~10s of near-silence) — the
|
||||
video position of the hands-meet peak and the audio position of the transient
|
||||
are both unambiguous. That single event replaced counting words forever.
|
||||
2. **Extract audio as raw mono PCM** (`ffmpeg -vn -ac 1 -ar 48000 -f s16le`) and
|
||||
compute a short window RMS envelope (5ms windows) in numpy. The clap is the global
|
||||
max (`argmax`); also print a coarse 100ms table — it shows the pre-clap artifacts
|
||||
(faint blips) vs the event (100× larger) vs true digital silence.
|
||||
3. **Video motion per frame** — `-vf "tblend=all_mode=difference,signalstats,metadata=print"`
|
||||
captured to stdout (NOT `file=` inside the filter — the `\\` path escapes mangle;
|
||||
have PowerShell-style quoting bite; `metadata=print` writes to ffmpeg's log, so
|
||||
redirect stdout to a file). Parse `YAVG` values; the clap is the frame with the
|
||||
motion spike (2.75 vs a ~0.1–0.5 animated-widget background).
|
||||
4. **Offset = audio_event_time − video_event_time**; every positive second means audio
|
||||
is BEHIND video by that much. Cross-correlate the two full envelopes (60fps-resampled
|
||||
RMS vs YAVG, normalized) to double-check the single-event peak — the autocorr-style
|
||||
peak must be sharp (unique max at +133 frames, second-best ≈0.26).
|
||||
5. **Sanity checks baked in:** count frames (`nb_frames` vs wall log) — if video is
|
||||
real-time you've ruled out the truncation bug as the cause; compare stream
|
||||
durations (audio > video by the lag is the SIGNATURE of a delayed audio tail, not
|
||||
truncation); check `dropped writes` — dropped audio moves events EARLIER (opposite
|
||||
sign), so it can never explain a lag. **A constant offset ≠ drift:** fixed lag =
|
||||
buffering/backlog, growing offset = clock mismatch.
|
||||
|
||||
**The ROOT CAUSE the recipe led to (record it so it's never re-derived):** the mixer's
|
||||
capture rings are fed from APP STARTUP (for the meters); `StartLive` never clears them,
|
||||
so the drained audio trail begins ~a full ring-depth (2.0s) behind go-live. **Rule: any
|
||||
"live preview/drain" sink fed by a continuously-capturing buffer MUST clear the buffer
|
||||
at go-live, or early output is stale backlog.** The 2s ring + configured 300ms sync
|
||||
delay + ~100ms pipeline = exactly the measured 2.11–2.22s. The lag ALSO scales with
|
||||
"time since app launch" up to the ring depth — a take right after a restart reads as
|
||||
~370ms while a fully-warmed app reads ~2.2s. Same code, wildly different numbers.
|
||||
|
||||
### Feeding a rawvideo pipe at 60fps: deadline pacing + row-blit budget
|
||||
|
||||
Derived 2026-09-03 (take-3 diagnosis — the stats seam from `97ffc42` named the stage
|
||||
|
||||
Reference in New Issue
Block a user