perf(video): move tier scaling to ffmpeg
The frame pump now submits crop-sized BGRA frames; FFmpeg scales them to final tier dimensions. Bundled scaler benchmark: 300 608x1080 to 1080x1920 frames in 0.435s (scaler-only). Reference: https://ffmpeg.org/ffmpeg-filters.html#scale-1
This commit is contained in:
@@ -254,9 +254,10 @@ stays XAML (editing view); the compositor is the output view.
|
||||
`VisualBrush` samples all of `CanvasGrid`, and if the dims were inside it the box would show the
|
||||
dim shading — especially after the box is dragged away and the fixed capture slides under the dims.
|
||||
|
||||
- **Render the active tier's output rect directly** (`CompositorOptions {SourceRectX/Y/W/H,
|
||||
OutputWidth, OutputHeight}`, fed from `MainViewModel.OutputRect*`): 16:9 = full 1920×1080 1:1; the
|
||||
vertical 9:16 tier = composite the centered 607×1080 crop then bilinear-upscale to 1080×1920.
|
||||
- **Composite the active tier's crop at crop size** (`CompositorOptions {SourceRectX/Y/W/H,
|
||||
OutputWidth, OutputHeight}`, fed from `MainViewModel.OutputRect*`): 16:9 = full 1920×1080; the
|
||||
vertical 9:16 tier = centered 608×1080 crop. FFmpeg's `scale` filter resizes the raw BGRA frame
|
||||
to the selected output tier after it leaves the frame pump; `setsar=1` keeps square pixels.
|
||||
- **CPU posture:** with FFmpeg as a subprocess the master crosses a CPU readback to the pipe every frame
|
||||
anyway, so GPU compositing buys little at this layer count (2-3 live layers; static layers
|
||||
pre-composite once into a cached base). GPU effort belongs to **NVENC** (the encoder), not composition;
|
||||
@@ -363,9 +364,11 @@ seam:** `Func<Scene?>`, `Func<SceneElement, VideoFrame?>` resolver, `Func<Compos
|
||||
pin every frame's content, repeated backing-array identity proves the pool recycles),
|
||||
`Composite_OpaqueFullCover_Backdrop_CopiesEveryPixel_Into_Scratch` (memcpy branch + scratch
|
||||
sentinel). `FakeEncoder.SubmitFrameAsync` now snapshots bytes like the real stdin write — holding the
|
||||
reference would race legitimate recycling. Take 5 must show `avg render ≤ ~10ms, ≈300/300`; the
|
||||
vertical tier's final 1080×1920 `BilinearScale` still allocates fresh per frame (same GC lesson when
|
||||
someone streams vertical — recorded as a follow-up, not silently "done").
|
||||
reference would race legitimate recycling. Take 5 must show `avg render ≤ ~10ms, ≈300/300`.
|
||||
**Vertical/output scaling moved to bundled FFmpeg (2026-11-04):** the C# compositor no longer
|
||||
enlarges the 608×1080 crop to 1080×1920 on every frame. The bundled FFmpeg `scale` filter processed
|
||||
300 synthetic BGRA frames in 0.447s on this machine; that is a scaler-only microbenchmark, not a
|
||||
live-stream result.
|
||||
- **Slice 3 — chat raster-on-change (2026-09-04, take 5):** take 5 measured `138/300, avg render
|
||||
25.5ms` — the per-tick blits were cheap now, but `ResolveOutputFrame` → `RenderChatBox` ran a FULL
|
||||
WPF raster (`ChatBoxRenderer`: FormattedText + `RenderTargetBitmap` + CopyPixels + channel swap)
|
||||
|
||||
Reference in New Issue
Block a user