fix(media): pace clip audio to a deadline too, and prove the keyed layer in the OUTPUT

Two follow-ups from the creator's lip-sync observation.

1. Audio still slept a fixed 50 ms per chunk — the same "sleep on top of the work" fault fixed for
   video an hour ago, left in place. Every chunk's read cost was added, so a clip's audio fell
   progressively behind its picture: small per chunk, unbounded over a long clip. It now uses the
   same deadline pacer, so audio and video advance on the same wall clock.

2. The preview pane is NOT the compositor. "Preview looks great" proves the decode and the preview
   bitmap; the recording and stream are blitted by SceneCompositor. `MediaCompositeOutputTests`
   renders a keyed video layer over a background through the real compositor and asserts the two
   things that matter: keyed (alpha 0) pixels let the background through, and the subject half
   covers it. So "green screen removed" is now proven in the output path, not just the pane.

Also: a round-clipped video layer had no preview trigger (only webcam+Round existed), so it would
render in the output and be invisible in the pane. Same unit-circle path now covers it.

Tests: 16 scoped (pacer, real-clip rate + keying, composite output, decode pacing, alert pacing)
green. FULL SUITE NOT RUN — the machine is in use: two runs under load aborted in the WPF test host
(HWND subclass crash) and one hit the 15-minute cap. Re-run scripts/verify.sh in a quiet window.
This commit is contained in:
2026-11-05 10:40:25 -08:00
parent 5577aaaf91
commit 3a56807ee3
3 changed files with 99 additions and 5 deletions
+10 -5
View File
@@ -226,6 +226,8 @@ public sealed class VideoClipPipe : IDisposable
});
var stream = process.StandardOutput;
var passStart = Stopwatch.GetTimestamp();
var pacer = new FramePacer();
var raw = new byte[65536];
var carry = Array.Empty<byte>();
var floatScratch = new float[AudioChunkFloats];
@@ -254,11 +256,14 @@ public sealed class VideoClipPipe : IDisposable
{
chunks++;
AudioAvailable?.Invoke(new AudioSample(floatScratch, AudioSampleRate, AudioChannels));
// Pace the NEXT chunk so the clip's audio lands at real-time cadence,
// matching how the video pipe is paced by frame rate.
await _delay(
TimeSpan.FromSeconds(AudioChunkFloats / (double)(AudioSampleRate * AudioChannels)), ct)
.ConfigureAwait(false);
// Deadline pacing, exactly like the video pipe: sleep only what is LEFT of this
// chunk's 50 ms. A fixed sleep added the read cost every chunk, so audio fell
// progressively behind the picture on a long clip (2026-11-05).
var wait = pacer.DelayFor(
TimeSpan.FromSeconds(AudioChunkFloats / (double)(AudioSampleRate * AudioChannels)),
Stopwatch.GetElapsedTime(passStart));
if (wait > TimeSpan.Zero)
await _delay(wait, ct).ConfigureAwait(false);
floatScratch = new float[AudioChunkFloats];
floatIndex = 0;
}