perf(capture): slice 8 — buffer ring + paste-cache Epoch; gen2 visibility (take-11 spikes)

Take 11 (c10ce06c) validated the off-UI architecture: typical frames land
work ~10ms + wait ~6.8ms = 16.7 exactly on the deadline; 212/300 best yet.
The ENTIRE remaining gap is periodic 35-65ms render spikes that WORSENED
across the take (189 -> 147) — the signature of gen2 GC pauses. Biggest
churner is structural: the screen capture minted a fresh ~8.3MB byte[] per
DWM frame (~500MB/s of LOH), a producer OBS never does (it owns fixed
surface pools).

- ScreenCaptureFrameSource: 4-deep buffer ring with size-matched slots (a
  <=17ms consumer cannot be lapped at 60Hz) + reused downscale row scratch.
- VideoFrame.Epoch: monotonic per producer frame. The paste cache keys on
  array IDENTITY, so recycled arrays MUST be distinguished — epoch joins the
  PasteKey. Producers handing fresh arrays leave it 0 (key unchanged effect).
- Stats print 'gen2 +N' per 5s window: next take acquits or convicts GC
  without another guess (rule: prove the stage).
- Test (the ONE): PasteCache_RecycledArrayWithNewEpoch_ReRasterizes_NotStaleHits
  — same array, new content, bumped epoch; fails on the old key by
  construction. 37/37 compositor/pump, clean build.
- Next suspect if gen2 stays hot: the 10Hz WebView2 capture (full-canvas PNG
  decode + fresh arrays on the UI thread) — recorded, untouched.

Creator audio ask queued in the same working session (+40% post-mix master
gain before the -1dBFS limiter) lands as its own commit next.
This commit is contained in:
2026-09-04 12:36:14 -07:00
parent eb4c379b91
commit fbc8562cf4
7 changed files with 128 additions and 20 deletions
+18 -6
View File
@@ -60,12 +60,24 @@ RealApp boot-smoke. Scope-check passed.
`avg wait` so render+submit+wait ≈ period (accounting closed — nothing can hide). Webcam now `avg wait` so render+submit+wait ≈ period (accounting closed — nothing can hide). Webcam now
routes through the paste cache too (bypass re-sampled 156k px even between identical device routes through the paste cache too (bypass re-sampled 156k px even between identical device
frames). 52/52 per-class green, clean build 0 warnings, committed this slice. frames). 52/52 per-class green, clean build 0 warnings, committed this slice.
4. **Take 11 (user, ~30s record-only):** read the superscript; expect `≈300/300 frames, wait ≈ the true remainder, worst render` now visible. If ~300: playback must be honest 1x — saga CLOSED, Unit B starts. If still ~250-280 with worst-render spikes: raster-miss spikes (web capture ~ every second) — next slice is pre-rasterizing on content change rather than on first-tick-after-change (cache the miss behind a swap-in). If wait is STILL large: the context theory was wrong and I have egg to eat — re-instrument, don't guess. 4. **Take 11 ran + slice 8 (2026-09-04, committed with the audio unit below):** off-UI loop
~22, wait ~0-3` — honest 60fps IF render+submit ≤ ~16.7. If wait is near zero and n/300 sits at WORKED — typical frames land exactly on the deadline (work ~10 + wait ~6.8 = 16.7; 212/300).
~200, the remaining gap is pure render 22ms → next slice = per-phase compositor timing (the stats Remaining gap = periodic 35-65ms spikes growing across the take = gen2 GC pauses; the capture
can split blit phases the same way they split resolve; that is the honest path, not a guess). path minted a fresh ~8.3MB byte[] per DWM frame (~500MB/s LOH). Slice 8: 4-deep buffer ring
If take 10 lands at ~300/300 → recording saga CLOSED. in ScreenCaptureFrameSource (size-matched slots, reused row scratch) + `VideoFrame.Epoch`
5. **Unit B — the top bar + session logic (user spec 2026-09-04 re-sent twice + decisions settled in Q&A):** joins the paste-cache key so recycled arrays can never false-hit + `gen2 +N` per stats window
(suspect-or-acquit — never guess at the tail again). Test `PasteCache_RecycledArrayWithNewEpoch_
ReRasterizes_NotStaleHits` fails on the old key by construction. 37/37 compositor/pump green.
4b. **AUDIO (creator ask 2026-09-04):** post-mix volume up ~40% — constant master gain applied
in AudioMixer.FillAndMix BEFORE the −1 dBFS limiter (limiter still owns the ceiling: the
gain can't add clipping, it just makes the limiter bind sooner on hot material). If the
creator reports pumping on loud game audio → the constant is the tuning knob (or per-input
defaults need raising instead — discuss with data).
5. **Take 12 (user, ~30s record + talk):** stats: `gen2 +N` should be ~0-1/window, `worst render`
→ ~16-20ms, n/300 → 300; playback honest speed; file audio ~40% hotter than before. If gen2
still >2/window: next suspect is the 10Hz WebView2 capture (full-canvas PNG decode + fresh
arrays on the UI thread) — throttle or move it, cited pattern first.
6. **Unit B — the top bar + session logic (user spec 2026-09-04 re-sent twice + decisions settled in Q&A):**
- Two-line top bar. Line 1: center = REC + **LIVE** pills (text renamed from ON-AIR; pills become - Two-line top bar. Line 1: center = REC + **LIVE** pills (text renamed from ON-AIR; pills become
mutually-exclusive RADIOS — record-OR-stream ruling), right = avatar + **Login/Logout** button mutually-exclusive RADIOS — record-OR-stream ruling), right = avatar + **Login/Logout** button
(no account status light). Line 2: centered primary **Start** (grayed while NO pill armed — (no account status light). Line 2: centered primary **Start** (grayed while NO pill armed —
+6 -5
View File
@@ -19,10 +19,11 @@ public sealed class SceneCompositor
// chat/web/image layers are identical frame after frame. Same shape as OBS (the // chat/web/image layers are identical frame after frame. Same shape as OBS (the
// source surface is cached, the compositor blits): a scaled/round-clipped/mirrored // source surface is cached, the compositor blits): a scaled/round-clipped/mirrored
// layer rasterizes once and is pasted (row-copy/blend) on every later tick. // layer rasterizes once and is pasted (row-copy/blend) on every later tick.
// Keyed by the SOURCE FRAME IDENTITY — every producer (capture, webcam, chat // Keyed by the SOURCE FRAME IDENTITY (array + Epoch) — producers hand out fresh
// renderer, web capture, static cache) hands out immutable byte[], so "same pixels // immutable byte[], or recycled-ring arrays whose monotonic Epoch distinguishes
// array + same target rect" ⇒ same raster. Bounds-checked eviction clears wholesale. // each generation (screen capture, take-11 fix), so "same array+epoch + same
private sealed record PasteKey(byte[] Pixels, int SrcW, int SrcH, int DstW, int DstH, bool Round, bool Mirror); // target rect" ⇒ same raster. Bounds-checked eviction clears wholesale.
private sealed record PasteKey(byte[] Pixels, long Epoch, int SrcW, int SrcH, int DstW, int DstH, bool Round, bool Mirror);
private static readonly Dictionary<PasteKey, VideoFrame> _pasteCache = new(); private static readonly Dictionary<PasteKey, VideoFrame> _pasteCache = new();
private static readonly object _pasteGate = new(); private static readonly object _pasteGate = new();
private const int MaxPasteEntries = 48; private const int MaxPasteEntries = 48;
@@ -284,7 +285,7 @@ public sealed class SceneCompositor
VideoFrame? raster = null; VideoFrame? raster = null;
lock (_pasteGate) lock (_pasteGate)
{ {
key = new PasteKey(frame.BgraPixels, frame.Width, frame.Height, dw, dh, isRound, mirror); key = new PasteKey(frame.BgraPixels, frame.Epoch, frame.Width, frame.Height, dw, dh, isRound, mirror);
_pasteCache.TryGetValue(key, out raster); _pasteCache.TryGetValue(key, out raster);
} }
+41 -8
View File
@@ -30,6 +30,18 @@ public sealed class ScreenCaptureFrameSource : IScreenCaptureSource
private bool _framePending; private bool _framePending;
private DateTime _lastErrorLog = DateTime.MinValue; private DateTime _lastErrorLog = DateTime.MinValue;
// Buffer recycling (take-11 spikes, 2026-09-04): a fresh ~8.3MB byte[] per DWM
// frame ≈ 500MB/s of LOH churn — the gen2 pauses it forces surfaced as the
// "worst render 35-65ms" spikes that capped fps at ~42 long after the compositor
// itself was fast. A 4-deep ring rotated round-robin is never lapped by a
// ≤17ms consumer at 60Hz; each hand-out carries an Epoch so identity-keyed
// consumers (the compositor's paste cache) cannot false-hit a recycled array.
private readonly byte[]?[] _frameRing = new byte[4][];
private int _ringNext;
private long _epoch;
private byte[]? _row0;
private byte[]? _row1;
// The composition master frame (see ai.md "Resolution tiers"): the background // The composition master frame (see ai.md "Resolution tiers"): the background
// is an input layer, so we never hold a CPU frame bigger than the master. // is an input layer, so we never hold a CPU frame bigger than the master.
private const int MaxBackgroundWidth = 1920; private const int MaxBackgroundWidth = 1920;
@@ -157,7 +169,26 @@ public sealed class ScreenCaptureFrameSource : IScreenCaptureSource
} }
} }
private static VideoFrame CopyToVideoFrame(SoftwareBitmap bitmap) private byte[] RentRingBuffer(int size)
{
for (var tries = 0; tries < _frameRing.Length; tries++)
{
var idx = (_ringNext + tries) % _frameRing.Length;
var buf = _frameRing[idx];
if (buf is { Length: var len } && len == size)
{
_ringNext = (idx + 1) % _frameRing.Length;
return buf;
}
}
var slot = _ringNext;
_ringNext = (slot + 1) % _frameRing.Length;
var fresh = new byte[size];
_frameRing[slot] = fresh;
return fresh;
}
private VideoFrame CopyToVideoFrame(SoftwareBitmap bitmap)
{ {
var sw = bitmap.PixelWidth; var sw = bitmap.PixelWidth;
var sh = bitmap.PixelHeight; var sh = bitmap.PixelHeight;
@@ -178,21 +209,23 @@ public sealed class ScreenCaptureFrameSource : IScreenCaptureSource
var dw = Math.Max(1, (int)(sw * scale)); var dw = Math.Max(1, (int)(sw * scale));
var dh = Math.Max(1, (int)(sh * scale)); var dh = Math.Max(1, (int)(sh * scale));
// DWM delivers an opaque surface (alpha 255); bilinear keeps it 255. // DWM delivers an opaque surface (alpha 255); bilinear keeps it 255.
return new VideoFrame(dw, dh, DownscaleBgra(data, sw, sh, srcStride, dw, dh)) { IsOpaque = true }; var scaled = RentRingBuffer(dw * dh * 4);
return new VideoFrame(dw, dh, DownscaleBgra(data, sw, sh, srcStride, dw, dh, scaled))
{ IsOpaque = true, Epoch = ++_epoch };
} }
var pixels = new byte[count]; var pixels = RentRingBuffer(count);
Marshal.Copy(data, pixels, 0, pixels.Length); Marshal.Copy(data, pixels, 0, pixels.Length);
return new VideoFrame(sw, sh, pixels) { IsOpaque = true }; return new VideoFrame(sw, sh, pixels) { IsOpaque = true, Epoch = ++_epoch };
} }
// Bilinear downscale to the master frame. Reads each source row pair through // Bilinear downscale to the master frame. Reads each source row pair through
// Marshal.Copy (no unsafe), writing tightly packed BGRA output. // Marshal.Copy (no unsafe), writing tightly packed BGRA output.
private static byte[] DownscaleBgra(IntPtr src, int sw, int sh, int srcStride, int dw, int dh) private byte[] DownscaleBgra(IntPtr src, int sw, int sh, int srcStride, int dw, int dh, byte[] dst)
{ {
var row0 = new byte[srcStride]; // Row scratch is per-capture-thread and reused across frames (same churn lesson).
var row1 = new byte[srcStride]; var row0 = _row0 != null && _row0.Length >= srcStride ? _row0 : (_row0 = new byte[srcStride]);
var dst = new byte[dw * dh * 4]; var row1 = _row1 != null && _row1.Length >= srcStride ? _row1 : (_row1 = new byte[srcStride]);
var xs = sw / (double)dw; var xs = sw / (double)dw;
var ys = sh / (double)dh; var ys = sh / (double)dh;
+7
View File
@@ -13,6 +13,13 @@ public sealed class VideoFrame
public byte[] BgraPixels { get; } public byte[] BgraPixels { get; }
public int Stride => Width * 4; public int Stride => Width * 4;
/// <summary>Producer frame epoch — a monotonically increasing number stamped by
/// sources that RECYCLE their pixel buffers (the screen-capture ring, take-11 spike
/// fix). Consumers that key caches on buffer identity (the compositor's paste cache)
/// MUST include it, or a recycled array false-hits with stale content. Producers
/// that hand out fresh arrays per frame leave it 0 — identity alone is then enough.</summary>
public long Epoch { get; init; }
/// <summary>Producer contract: every alpha byte in <see cref="BgraPixels"/> is 255 /// <summary>Producer contract: every alpha byte in <see cref="BgraPixels"/> is 255
/// (DWM capture surfaces and MediaCapture video carry no alpha — the OS fills 255). /// (DWM capture surfaces and MediaCapture video carry no alpha — the OS fills 255).
/// Lets the compositor take a straight-copy fast path for a full-canvas opaque /// Lets the compositor take a straight-copy fast path for a full-canvas opaque
+1 -1
View File
File diff suppressed because one or more lines are too long
+12
View File
@@ -794,6 +794,18 @@ seam:** `Func<Scene?>`, `Func<SceneElement, VideoFrame?>` resolver, `Func<Compos
(the ONE — inline-pumping SyncContext proves continuations never return to the starting thread) + (the ONE — inline-pumping SyncContext proves continuations never return to the starting thread) +
70/70. Take 9 verdict: wait ≈ true remainder (period → 16.7, n → ~300); `worst render` names any 70/70. Take 9 verdict: wait ≈ true remainder (period → 16.7, n → ~300); `worst render` names any
remaining raster-miss spikes; render+submit+wait still == period — the accounting holds. remaining raster-miss spikes; render+submit+wait still == period — the accounting holds.
- **Slice 8 — capture buffer ring + paste-cache Epoch (take 11, 2026-09-04):** take 11 validated
the architecture — typical frames now land `work ~10ms + wait ~6.8ms = 16.7`, exactly the deadline
(212/300). The whole remaining gap is PERIODIC 35-65ms worst-render spikes that worsened across
the take (189→147 frames/5s) — the signature of gen2 GC pauses, now provable via `gen2 +N` in
every stats window. Biggest churn was structural: `ScreenCaptureFrameSource` minted a fresh
~8.3MB `byte[]` per DWM frame (~500MB/s LOH). It now rotates a 4-deep ring (a ≤17ms consumer can
never be lapped at 60Hz), size-matched per slot, with reused downscale row scratch. Recycled
arrays would poison the paste cache (it keys on array IDENTITY), so `VideoFrame.Epoch` —
monotonic per producer frame, 0 for fresh-array producers — joins the key. Regression test
`PasteCache_RecycledArrayWithNewEpoch_ReRasterizes_NotStaleHits` fails on the old key by
construction. Next suspect if gen2 stays hot: the 10Hz WebView2 capture loop (full-canvas PNG
decode + fresh arrays on the UI thread) — recorded as follow-up, untouched this slice.
- **Stop ordering matters:** `StopAsync` stops the encoder (closes stdin → EOF → ffmpeg finalizes+exits) - **Stop ordering matters:** `StopAsync` stops the encoder (closes stdin → EOF → ffmpeg finalizes+exits)
**before** awaiting the loop, because closing stdin unblocks a write stuck on pipe backpressure — the **before** awaiting the loop, because closing stdin unblocks a write stuck on pipe backpressure — the
reverse order would deadlock. `ProcessFailed` self-stops the pump. `Failed` while live flips reverse order would deadlock. `ProcessFailed` self-stops the pump. `Failed` while live flips
+43
View File
@@ -211,6 +211,49 @@ public class SceneCompositorTests
AssertColor(third, 101, 101, 255, 0, 0); AssertColor(third, 101, 101, 255, 0, 0);
} }
/// <summary>The ONE integration test for the capture buffer ring + paste-cache epoch
/// (take-11 spike fix): a recycled capture buffer hands out the SAME byte[] with
/// different content; only VideoFrame.Epoch can keep the paste cache honest. Without
/// the epoch in the key, frame 2 false-hits frame 1's raster and the recording shows
/// stale pixels for as long as the array keeps cycling — the exact bug this pins.</summary>
[Fact]
public void PasteCache_RecycledArrayWithNewEpoch_ReRasterizes_NotStaleHits()
{
var pixels = new byte[64 * 32 * 4];
for (var i = 0; i < pixels.Length; i += 4)
{
pixels[i] = 0; pixels[i + 1] = 0; pixels[i + 2] = 255; pixels[i + 3] = 255; // red
}
var gen1 = new VideoFrame(64, 32, pixels) { IsOpaque = true, Epoch = 1 };
var webcam = new WebcamSceneConfig { X = 4, Y = 4, Width = 32, Height = 16 };
var scene = new Scene { Name = "Live" };
scene.Elements.Add(webcam);
var options = new CompositorOptions
{
SourceRectX = 0, SourceRectY = 0, SourceRectWidth = 128, SourceRectHeight = 64,
OutputWidth = 128, OutputHeight = 64,
};
var compositor = new SceneCompositor();
var out1 = compositor.Render(scene, _ => gen1, null, options);
AssertColor(out1, 20, 12, 255, 0, 0);
// Generation 2: the SAME array, new content, bumped epoch (what the capture
// ring does 60 times a second).
for (var i = 0; i < pixels.Length; i += 4)
{
pixels[i] = 0; pixels[i + 1] = 255; pixels[i + 2] = 0; pixels[i + 3] = 255; // green
}
var gen2 = new VideoFrame(64, 32, pixels) { IsOpaque = true, Epoch = 2 };
var out2 = compositor.Render(scene, _ => gen2, null, options);
AssertColor(out2, 20, 12, 0, 255, 0); // a stale hit would still be red
// And the same epoch twice still hits the cache (idempotent paste).
var out3 = compositor.Render(scene, _ => gen2, null, options);
Assert.Equal(out2.BgraPixels, out3.BgraPixels);
}
[Fact] [Fact]
public void Render_VerticalTier_Outputs_1080x1920_From_The_Center_Crop() public void Render_VerticalTier_Outputs_1080x1920_From_The_Center_Crop()
{ {