perf(video): move tier scaling to ffmpeg

The frame pump now submits crop-sized BGRA frames; FFmpeg scales them to final tier dimensions. Bundled scaler benchmark: 300 608x1080 to 1080x1920 frames in 0.435s (scaler-only).

Reference: https://ffmpeg.org/ffmpeg-filters.html#scale-1
This commit is contained in:
2026-11-04 10:52:35 -08:00
parent 76e8c22815
commit 6310f50c27
11 changed files with 93 additions and 47 deletions
+8 -4
View File
@@ -67,7 +67,8 @@ a machine prerequisite (a manifest framework-dependency + graceful-failure path,
### Live encoder + RTMP push (TASK 4 ship step 3 — shipped 2026-08-12, plan in TASKS.md)
The encoder is a **thin orchestrator over `ffmpeg.exe`** — no H.264/AAC code in the app. It spawns the
subprocess (path from `IFfmpegLocator`), feeds raw BGRA master frames into stdin, and parses `-stats`
subprocess (path from `IFfmpegLocator`), feeds raw BGRA compositor-crop frames into stdin, lets FFmpeg
scale them to the selected output tier, and parses `-stats`
stderr lines into `StreamHealth` (bitrate/FPS/duration, dropped-from-frame-count). `FfmpegEncoder`
(`IFfmpegEncoder` seam) holds: `StartAsync` (locate → probe `-encoders` → spawn → stderr loop →
drain loop), `SubmitFrameAsync` (slice 10: bounded-queue ENQUEUE — never a pipe write; the drain task
@@ -80,8 +81,10 @@ integration test fakes the whole subprocess (probe + encoder) with a Channel-bac
`Complete()` is EOF (`null`), never a `ChannelClosedException`.
**Decisions (locked):** args are pure (`FfmpegArgs.Build`, no string building in the encoder):
`-f rawvideo -pix_fmt bgra -video_size WxH -framerate FPS -i pipe:0` (**NO `-re`** — the FramePump
is the pacer since slice 9, 2026-09-10; `-re` added a second, fighting clock on the rawvideo demux)
`-f rawvideo -pix_fmt bgra -video_size inputWxH -framerate FPS -i pipe:0` (**NO `-re`** — the FramePump
is the pacer since slice 9, 2026-09-10; `-re` added a second, fighting clock on the rawvideo demux).
When input and output dimensions differ, each output block adds `-vf scale=outputW:outputH:flags=bilinear,setsar=1`;
otherwise frames pass through at their existing dimensions.
+ a **real audio input — the
mixer writes IEEE-float stereo to a Windows named pipe** (`-f f32le -ar 48000 -ac 2
-i \\.\pipe\ytllive_audio`, name via `EncoderOptions.AudioPipeName`; replaced the old `-f lavfi -i
@@ -99,7 +102,8 @@ other — while the dual-block machinery stays generic.**
`FfmpegEncoder.StartAsync` throws unless at least one output is enabled.
**Encoder choice is probed from the binary's `-encoders` listing** (`FfmpegEncoderPicker`, pure):
NVENC → QSV → AMF → OpenH264 fallback, **never libx264** (GPL; see Licensing). `EncoderOptions.VideoEncoder`
forces one and skips the probe. `EncoderOptions` also carries W×H/FPS/bitrate from the quality tier and
forces one and skips the probe. `EncoderOptions` carries raw input W×H from the compositor crop, final
output W×H/FPS/bitrate from the quality tier, and
`GopSize` = FPS×4. Constructed by `MainViewModel` since ship step 5 (`new FfmpegEncoder(new FfmpegLocator())`),
driven by the `FramePump` below.