fix(media): play at the clip's real rate — the pipe was copying a frame per read

Reported as "the default video and the green screen video are playing at a hideously slow
rate… I thought the playback timing issues were resolved?!" They were not: two independent
faults, both in the shared clip pipe.

1. QUADRATIC COPY IN THE FRAME READER. `RawVideoFrameReader` re-concatenated the whole pending
   buffer on every `Feed`, and the pipe feeds it 64 KB at a time. A 1080p BGRA frame is 8 MB, so
   one frame cost ~127 copies of a growing buffer — roughly half a gigabyte of memcpy per frame.
   Measured on a 30 fps clip: 6 frames in 3 seconds (2 fps). The reader now fills one preallocated
   frame buffer and hands that array to the VideoFrame, so a frame is copied exactly once. Pipe
   reads are 256 KB. Same clip: 29 fps.

2. PACING SLEPT ON TOP OF THE WORK. A fixed 1/fps sleep after each frame adds the frame's own
   cost, so 20 ms of work on a 30 fps clip yields 1/(33+20) ≈ 19 fps — measured 20.9. Pacing is
   now deadline-based (`Helpers/FramePacer.cs`): wait only what is left of the frame's interval,
   resync when a frame overruns instead of bursting, never sleep longer than one interval.

The alert clip shares this pipe, so it gained both fixes.

Tests: `FramePacerTests` (subtract work, no drift, late → resync, zero duration, reset);
`ChromaKeyRealClipTests.RealClip_PlaysAtItsNativeRate` (throughput floor of 15 fps on the real
30 fps clip — the old code sat at 2); the two pacing assertions now check "at most one interval"
rather than an exact sleep, which is what deadline pacing means. Full suite: 574 passed,
0 failed, clean build.
This commit is contained in:
2026-11-05 09:57:07 -08:00
parent 69e595da25
commit 5577aaaf91
8 changed files with 203 additions and 35 deletions
+15 -2
View File
@@ -25,6 +25,10 @@ namespace ytLive.Services;
/// </summary>
public sealed class VideoClipPipe : IDisposable
{
/// <summary>Pipe read size. A 1080p BGRA frame is ~8 MB, so 64 KB reads meant 127 awaits and
/// 127 reader calls per frame; 256 KB keeps the syscall count sane without a large allocation.</summary>
private const int ReadBufferBytes = 256 * 1024;
public const int AudioSampleRate = 48000;
public const int AudioChannels = 2;
private const int AudioChunkFloats = 4800; // 50 ms of stereo
@@ -45,6 +49,7 @@ public sealed class VideoClipPipe : IDisposable
private readonly Func<TimeSpan, CancellationToken, Task> _delay;
private readonly bool _audio;
private readonly ChromaKeyOptions? _key;
private readonly FramePacer _pacer = new();
private readonly CancellationTokenSource _cts = new();
private bool _running;
private IDecodeProcess? _activeVideo;
@@ -158,6 +163,7 @@ public sealed class VideoClipPipe : IDisposable
private async Task<int> RunVideoAsync(string ffmpegPath, TimeSpan frameDuration, CancellationToken ct)
{
var passStart = Stopwatch.GetTimestamp();
var args =
$"-hide_banner -loglevel error -i \"{_path}\" " +
$"-f rawvideo -pix_fmt bgra -vf {VideoFilterChain.BuildVideo(_width, _height, _key)} -an -";
@@ -174,7 +180,8 @@ public sealed class VideoClipPipe : IDisposable
});
var reader = new RawVideoFrameReader(_width, _height);
var buffer = new byte[65536];
var buffer = new byte[ReadBufferBytes];
_pacer.Reset();
var stream = process.StandardOutput;
var frames = 0;
while (!ct.IsCancellationRequested && !process.HasExited)
@@ -188,7 +195,13 @@ public sealed class VideoClipPipe : IDisposable
frames++;
FrameAvailable?.Invoke(frame);
if (frameDuration > TimeSpan.Zero)
await _delay(frameDuration, ct).ConfigureAwait(false);
{
// Deadline pacing: sleep what is LEFT of this frame's interval, so the frame's
// own work is not added on top (see FramePacer).
var wait = _pacer.DelayFor(frameDuration, Stopwatch.GetElapsedTime(passStart));
if (wait > TimeSpan.Zero)
await _delay(wait, ct).ConfigureAwait(false);
}
}
}
_activeVideo = null;