Perf: dedicated strip-parallel composite (4 threads, BelowNormal) — UNIT 5

A composite splits into four horizontal row strips across four DEDICATED
background threads; the calling thread takes a strip too (so four workers
never mean five-way oversubscription), and anything under 64 rows runs
inline. BlitContentRaw, BlitBakedElement and BlitOverlay now fan out.
Output is byte-identical — strips already partitioned by row.

The demotion deliberately does NOT come from Parallel.For. The pool is
shared, so Thread.CurrentThread.Priority = BelowNormal inside it leaks
into every unrelated work item that thread later runs — a full test run
after trying it broke AudioPipelineTests.Mix_HonorsProviderGains_AndGameMute_
KillsTheLoopback. Established guidance agrees: use your own producer/consumer
queue with your own Threads and set their priority yourself
(https://stackoverflow.com/questions/524483/net-backgroundworker-objects-thread-priority,
https://stackoverflow.com/questions/1578599/c-sharp-lower-thread-priority-in-thread-pool).
Helpers/StripPool.cs is that queue. Four threads is the measured sweet
spot: fewer leaves render over budget, more steals cycles from the game
and the encoder and made render worse.

The scaled blit's column LUT moves from a shared static buffer to
ArrayPool<float>.Shared with a finally — a single static buffer is a data
race the moment strips run in parallel.

Gate: SceneCompositorTests.StripParallel_RepeatedRenders_AreByteIdentical —
8 renders of a scaled round webcam + mirrored image + master-sized flash over
a short backdrop must be byte-for-byte identical, and must differ from a
backdrop-only render so it cannot pass vacuously. It caught a loop-variable
capture (new Thread(() => Worker(i)) saw i == Workers) that crashed the test
host on the first run.
This commit is contained in:
2026-10-05 18:54:47 -07:00
parent 3901cb8298
commit 6b6284d05c
7 changed files with 452 additions and 42 deletions
+53 -7
View File
@@ -10,6 +10,40 @@ Unpushed: `5ad0106` broadcast diagnostic → `74da99c` + `0fb2f9c` privacy/Test
`8fbc370` diagnostic parse fix → `07a088a` **the explicit-bind fix** → `509ef36` window placement →
`b2f2197` handoff → `UNIT 0` frame-cadence measurement.
## 🚧 IN FLIGHT 2026-10-05 — UNITS 4+5 landed: the render-budget work (needs a creator measurement)
**The 141ms → 33ms mystery is SOLVED, and it was geometry, not the build.** The opaque-backdrop 1:1
row walk only engages when the source rect is **exactly** 1920×1080. On a 21:9 monitor the source
rect shrinks to 1920×803 ("fit", not "fill"), so every render paid a ~2M-byte opaque-black prefill
plus a ~1.54M-pixel per-pixel alpha-branch resample instead of one `BlockCopy`. 2560×1440 fits to
1920×1080 exactly and takes the fast path. **Debug-vs-Release was NOT the cause** — today's Debug
build with the game running on 16:9 measured 33ms average. Every recorded A/B matches.
**Shipped (all pixel-identical, all gated):**
- **Unit 4** — opaque 1:1 fast paths widened to the whole-canvas case; **prefill only the uncovered
letterbox rows**; a **per-column LUT** for the scaled blit (source column + 8.8 weights computed
once per column, not a divide per pixel). Commits `0464593`, `08290ec`, `77d5120`, `dc13438`.
- **Unit 5** — **`Helpers/StripPool.cs`**: one composite splits into four horizontal row strips
across four DEDICATED `BelowNormal` background threads (the caller takes a strip too; inline below
64 rows). ⛔ **Do not "simplify" this back to `Parallel.For` with a thread-priority poke** — the
pool is shared, the demotion leaks into unrelated work, and it broke an audio test for real. Two
sources + the failure are recorded in `MyMistakes.md`; the design note is in `ai.md` →
"Render budget". Commits `623d77e`, `3901cb8`, + the strip-pool commit.
- Gate test `SceneCompositorTests.StripParallel_RepeatedRenders_AreByteIdentical` — 8 renders of a
multi-layer scene must be byte-for-byte identical, and must differ from a backdrop-only render so
it cannot pass vacuously. It caught a loop-variable capture that crashed the test host on run 1.
**⏭ NEXT STEP — one Release recording, 60s, Live scene, Division 2 running.** This is the
measurement that decides whether Unit 5 was necessary. Read the `FramePump stats` line (it now
carries `social`/`ticker`/`flash`/`acq` per-stage timings and the pump-start line names the scene +
compositor rect):
- **avg render ≤12ms** → Units 4+5 did it; go straight to **Unit 6** (encoder backpressure:
`worst submit` has hit 53ms with ~357 cumulative drops — real frame loss on top of the padding).
- **still >16.67ms** → the remaining cost is NOT the blit loops; read the per-stage timings before
touching anything else (do not guess — that is exactly the UNIT 0 lesson).
- **Under ~30ms on 16:9 but 140ms on 21:9** → geometry is still costing us; the fix is the tier
source-rect, not the compositor.
## 🚨 ANSWERED 2026-10-05 — the judder is the COMPOSITOR, not the camera. Unit 1 is the wrong next move.
Take `ty-20261005-1201-556-0001.mp4` (75.3s, 1920×1080, 60fps) + `startup.log`, read together:
@@ -417,7 +451,7 @@ Detail and the open 11.12 re-check: [`TASKS/task-50-publish-recording.md`](TASKS
## Test state
**402/402, 0 warnings, verified 2026-10-05** (395 + 6 `FrameCadenceTests` + 1 pump-stats integration
test). Build the **test** project, not just the app project, or VSTest runs a stale DLL.
test + 1 strip-parallel gate). Re-verified after UNITS 4+5. Build the **test** project, not just the app project, or VSTest runs a stale DLL.
**Launch exe for the creator's test run: `bin/Debug/net8.0-windows10.0.19041.0/ytLive.exe`.**
**Confirmed environmental flake, NOT a regression (re-checked 2026-10-05, and it now fails
@@ -427,6 +461,15 @@ fails identically on a stashed clean tree**, so it has nothing to do with UNIT 0
interactive desktop session this shell does not have (the mouse drag never lands). Do not chase it;
it is listed under Landmines as the known interactive-session test.
**⚠️ A game-playing session breaks the audio tests — know this before debugging it.** On
2026-10-05 `AudioPipelineTests.Mix_HonorsProviderGains_AndGameMute_KillsTheLoopback` failed with
480/480 samples at `0.4` where `0` was expected, then passed in isolation minutes later. Cause: the
user was in a **Division 2 boss fight**, and that test captures **audio loopback** — the game's own
output landed in it. **An audio test failing during a gaming session is an environment fact, not a
regression.** Same bucket as the interactive-desktop flake above: re-run in a quiet session before
believing either. (It also briefly failed while I was probing `Parallel.For` thread priorities,
which sent me after the wrong culprit entirely — the game was the simpler explanation.)
**⚠️ New flake, do not chase it as a regression:** on the *first* full-suite run,
`AudioPipelineTests.MasterLimiter_CapsTheLiveMix_OnThePipe` failed with `Expected 0.891,
Actual 0` — a **zero**, not a slightly-off value, which is the signature of the pipe not being
@@ -449,31 +492,34 @@ whole-suite scheduling artifact of the same family as the `RealAppHost` STA land
1. **~PUSH, then the creator's 60-second Test** — see the FIXED box above. Packaging is NOT next;
the creator's ruling 2026-10-04: still in active development, so ship-time work waits and queued
features come first.
2. **Webcam cadence → Unit 1 (exclusive lock + 720p).** UNIT 0 is shipped and the counters are in
2. **⏭ Read the Unit 4+5 Release recording (see the IN FLIGHT box at the top).** Then Unit 6
(encoder submit backpressure: `worst submit` 53ms, ~357 cumulative drops) if render is inside
budget; if render is still over, the per-stage timings name the stage — do not guess.
3. **Webcam cadence → Unit 1 (exclusive lock + 720p).** UNIT 0 is shipped and the counters are in
`startup.log`; the reading needs one 60-second creator recording (details + the exact
read-the-log-together table in the IN FLIGHT box above). Then Unit 1: exclusive camera lock +
720p, and Unit 2: the blinking SD badge. **Creator to-do, not yet queued: long-run
resource/consumption/recycling benchmarking** (how long the app stays healthy under full burden) —
record only, do not start it.
2. **The packaging project + `Package.appxmanifest`** (TASK 48 items 2–3) — the next code unit.
4. **The packaging project + `Package.appxmanifest`** (TASK 48 items 2–3) — the next code unit.
Full trust, `webcam`/`microphone`, English only; Velopack deleted. Add the
`DefaultRecordFolder()` comment in the same unit, and the WebView2 framework dependency +
graceful-failure path.
3. **TASK 50 — one-button "Publish this video"** (`TASKS/task-50-publish-recording.md`). Queued
5. **TASK 50 — one-button "Publish this video"** (`TASKS/task-50-publish-recording.md`). Queued
behind the Store submission by its own precedence note ("MSIX first, publish second — unless
the creator reverses that"). Scoping is done; the build is not. ⛔ Do **not** flip
`enableAutomaticArchiving` until it ships. It also carries an **open 11.12 re-check**: a
publish feature makes the app *push* creator content to a third party, so the IARC answers
must not go to certification describing an app that cannot publish.
4. **Declare the IAP products in Partner Center** — the price is **settled** ($10/mo, $400
6. **Declare the IAP products in Partner Center** — the price is **settled** ($10/mo, $400
lifetime, no annual, 15% verified), so this is unblocked. **Blocks:** the Store listing and the
`OverlayHost.xaml` copy. ⛔ The Individual-vs-Company account question is still unanswered —
get it right before enrolling.
5. **Polar teardown** (TASK 48 item 0) — `PolarLicenseService`, `PolarLicense`,
7. **Polar teardown** (TASK 48 item 0) — `PolarLicenseService`, `PolarLicense`,
`MainViewModel.License.cs`, the `OverlayHost.xaml` string, the `csproj`/`App.xaml.cs`
Velopack sites. `IsPremium` ← Store entitlement. **Do this as one unit** — a half-torn-down
licensing path is worse than either end state.
6. **Vertical recording verification** — the compositor tier is proven by
8. **Vertical recording verification** — the compositor tier is proven by
`Render_VerticalTier_Outputs_1080x1920_From_The_Center_Crop`; the *record* path has never
been run. `FramePump.cs:645` flags off-size tiers as a known follow-up.
7. **WACK + certification notes + the privacy policy** (TASK 48 items 4–5) — the policy must
+204
View File
@@ -0,0 +1,204 @@
using System;
using System.Threading;
namespace ytLive.Helpers;
/// <summary>
/// A fixed pool of four DEDICATED background threads at <see cref="ThreadPriority.BelowNormal"/>
/// that split one composite into horizontal row strips. The compositor needs the split to get
/// render inside a 16.67ms slot, but it must not compete with the encoder, the WGC capture
/// callbacks, the WebView2 chat or the game the user is playing — hence the hard cap of four
/// and the demotion.
///
/// Research (2026-10-05, Unit 5): the demotion must NOT be done by poking
/// <c>Thread.CurrentThread.Priority</c> inside <c>Parallel.For</c> — the pool is shared, so the
/// demotion leaks into every unrelated work item that thread later runs
/// (https://stackoverflow.com/questions/524483/net-backgroundworker-objects-thread-priority:
/// "you have next to no control of which thread pool thread will actually run your task");
/// https://stackoverflow.com/questions/1578599/c-sharp-lower-thread-priority-in-thread-pool:
/// "You shouldn't mess with the priority of the regular ThreadPool, since you aren't the only
/// consumer ... write your own producer/consumer queue, using your own Thread(s) as the
/// consumer(s) — setting the thread priority yourself"). A real failed attempt confirmed it: a
/// full test run after demoting the pool threads broke an unrelated audio-mix timing test.
/// This class IS that producer/consumer queue; the caller thread is the producer and also runs
/// one strip itself, so four workers never means five-way oversubscription.
///
/// Contract: <see cref="For"/> blocks until every strip has run, each strip must write only its
/// own rows (the compositor's blits already partition by row), and it must not be called from
/// inside a strip (the blits never nest).
/// </summary>
internal static class StripPool
{
/// <summary>Four workers: the measured sweet spot on this box. Fewer leaves render over
/// budget; more steals cycles from the game and the encoder and made render worse, not better.</summary>
public const int Workers = 4;
/// <summary>Below this many rows the fan-out costs more than it saves — run it inline.</summary>
private const int MinRowsForFanOut = 64;
private static readonly object[] _queueLocks = CreateLocks();
private static readonly Queue<Strip>[] _queues = CreateQueues();
private static readonly Queue<Strip>[] _free = CreateQueues();
private static readonly SemaphoreSlim[] _signals = CreateSignals();
private static Thread[]? _threads;
private static int _roundRobin = -1;
private sealed class Strip
{
public int From;
public int To; // exclusive
public Action<int>? Body;
public Batch? Owner;
}
private sealed class Batch
{
public int Remaining;
public Exception? Error;
}
/// <summary>Run <paramref name="body"/> for every row in [from, to), split across the pool.</summary>
public static void For(int from, int to, Action<int> body)
{
if (from >= to) return;
var rows = to - from;
if (rows < MinRowsForFanOut || !EnsureStarted())
{
for (var y = from; y < to; y++) body(y);
return;
}
// Strips are handed to whole workers (not per-row tasks) so the strip boundaries —
// and therefore the destination row each strip owns — are fixed before any of them run.
var strips = Math.Min(Workers, rows);
var chunk = (rows + strips - 1) / strips;
var batch = new Batch { Remaining = strips - 1 }; // the caller runs the last one
var first = from;
for (var k = 0; k < strips - 1; k++)
{
var end = Math.Min(first + chunk, to);
var w = NextWorker();
var strip = Rent(w);
strip.From = first;
strip.To = end;
strip.Body = body;
strip.Owner = batch;
lock (_queueLocks[w]) _queues[w].Enqueue(strip);
_signals[w].Release();
first = end;
}
try
{
for (var y = first; y < to; y++) body(y);
}
catch (Exception ex)
{
batch.Error ??= ex;
}
if (batch.Remaining > 0)
{
lock (batch)
{
while (batch.Remaining > 0) Monitor.Wait(batch);
}
}
if (batch.Error != null) throw new InvalidOperationException("A compositor strip failed.", batch.Error);
}
private static void Worker(int index)
{
var signal = _signals[index];
while (true)
{
signal.Wait();
Strip? strip = null;
lock (_queueLocks[index])
{
if (_queues[index].Count > 0) strip = _queues[index].Dequeue();
}
if (strip == null) continue; // released for an already-drained queue
var batch = strip.Owner!;
try
{
for (var y = strip.From; y < strip.To; y++) strip.Body!(y);
}
catch (Exception ex)
{
batch.Error ??= ex;
}
finally
{
strip.Body = null;
strip.Owner = null;
lock (_queueLocks[index]) _free[index].Enqueue(strip);
lock (batch)
{
if (--batch.Remaining == 0) Monitor.PulseAll(batch);
}
}
}
}
private static Strip Rent(int worker)
{
lock (_queueLocks[worker])
return _free[worker].Count > 0 ? _free[worker].Dequeue() : new Strip();
}
private static int NextWorker()
{
var w = Interlocked.Increment(ref _roundRobin) % Workers;
return w < 0 ? w + Workers : w;
}
/// <summary>Start the workers on first use, so a WPF cold start pays nothing for them.</summary>
private static bool EnsureStarted()
{
if (_threads != null) return true;
lock (_signals)
{
if (_threads != null) return true;
var threads = new Thread[Workers];
for (var i = 0; i < Workers; i++)
{
var index = i; // capture per-iteration: the loop variable would be Workers by the time the thread runs
var t = new Thread(() => Worker(index), 256 * 1024)
{
IsBackground = true,
Name = $"compositor-strip-{i}",
Priority = ThreadPriority.BelowNormal,
};
threads[i] = t;
t.Start();
}
_threads = threads;
return true;
}
}
private static object[] CreateLocks()
{
var l = new object[Workers];
for (var i = 0; i < Workers; i++) l[i] = new object();
return l;
}
private static Queue<Strip>[] CreateQueues()
{
var q = new Queue<Strip>[Workers];
for (var i = 0; i < Workers; i++) q[i] = new Queue<Strip>();
return q;
}
private static SemaphoreSlim[] CreateSignals()
{
var s = new SemaphoreSlim[Workers];
for (var i = 0; i < Workers; i++) s[i] = new SemaphoreSlim(0, int.MaxValue);
return s;
}
}
+1
View File
@@ -14,6 +14,7 @@ Cross-cutting utilities. See [`schema.md`](../schema.md) for the memory-map conv
| `InverseBoolToVisibilityConverter.cs` / `NotNullToVisibilityConverter.cs` | XAML value converters for visibility bindings |
| `AllTrueToVisibilityConverter.cs` | Multi-value AND-gate → Visible; gates the layer-row capture menu items on row-IsBackground + staged-scene-is-Live (TASK 25) |
| `CameraConflictProbe.cs` | Best-effort diagnostic: when a camera won't start, enumerates running processes known to hold cameras (OBS, Zoom, Teams, NVIDIA Broadcast, browsers, etc.) — Windows doesn't expose "which process has this device" via any public API, so this is a suspect list, not a verdict |
| `StripPool.cs` | Four DEDICATED background threads at `BelowNormal` priority that split one composite into horizontal row strips (the caller runs a strip too); started lazily, runs inline below 64 rows. UNIT 5 — do NOT demote the ThreadPool from `Parallel.For`, it leaks into unrelated work (see ai.md "Render budget") |
| `FrameCadence.cs` | Pure arrival-cadence counter (`FrameCadence` + `FrameCadenceSummary.Describe`): frames, gaps, hitches, longest/avg gap for a window of a timestamped stream. Fed by `MediaCaptureFrameSource` (camera stamps vs our arrivals) and `FramePump` (distinct webcam Epochs that reached the output) — UNIT 0, see ai.md "Frame cadence" |
Related: [`Themes/Controls.xaml`](../Themes/Controls.xaml) styles the lists this
+37
View File
@@ -1438,3 +1438,40 @@ point: *"didn't matter how cool it looked if you didn't understand what it was t
standard. A test that asserts a *disproven* behaviour doesn't just fail to help — it launders a
guess into a guarantee and makes the next reader stop looking. If a test's expectation is wrong,
the expectation is the bug; fix the belief, don't delete the assertion.
## 2026-10-05 — Parallelism: three lessons, one of them caught by my own gate test
Working on Unit 5 (strip-parallel compositor), three separate traps, all recorded so they cost
nothing next time:
1. **You cannot demote the ThreadPool — only your own threads.** The cheap-looking
`Thread.CurrentThread.Priority = ThreadPriority.BelowNormal` inside `Parallel.For` "works" and
silently poisons the process: `Parallel.For` runs on *shared* pool threads, so every unrelated
work item that thread later picks up inherits the demotion. I hit exactly that — a full test run
broke `AudioPipelineTests.Mix_HonorsProviderGains_AndGameMute_KillsTheLoopback`, an audio test
that has nothing to do with compositing. Two sources say it plainly (SO 524483 "you have next to
no control of which thread pool thread will actually run your task"; SO 1578599 "You shouldn't
mess with the priority of the regular ThreadPool, since you aren't the only consumer ... write
your own producer/consumer queue, using your own Thread(s) as the consumer(s) - setting the
thread priority yourself"). The fix is `Helpers/StripPool.cs`: dedicated `Thread`s whose
priority I own. **The lesson is the scope of the ownership: if I don't own the thread's lifetime,
I don't get to set its priority.**
2. **`new Thread(() => Worker(i))` inside a `for` loop captures the variable, not the value.** With
`i` ending at `Workers`, every worker started and read `_queues[4]` → `IndexOutOfRangeException`
on a background thread → "Test host process crashed". Copy to a local (`var index = i;`) before
the lambda. It crashed the host on the FIRST run of the new test, which is the cheapest possible
warning — a shared static buffer I added earlier for "allocation reduction" had also become a
data race the moment strips ran in parallel (now `ArrayPool<float>` + `finally`).
3. **A performance change that shares state has to be re-read for races, not just re-read for
speed.** Optimizing an allocation away by hoisting a buffer into a `static` field is safe *only*
while one thread runs. The moment that same method fans out, "we never had a race" stops being
true. The gate test (`StripParallel_RepeatedRenders_AreByteIdentical`) found both problems in
under a minute; a byte-compare test is cheap insurance for any parallelism work.
**And the test flake that was NOT mine:** `Mix_HonorsProviderGains_AndGameMute_KillsTheLoopback`
captures **audio loopback**, so it fails whenever the creator has anything playing on the system —
it failed while the user was in a Division 2 boss fight and passed minutes later. **Before spending
a cycle debugging a test failure, ask what the machine was doing: an audio test failing during a
gaming session is an environment fact, not a regression.** (Same family as the known
`LayerReorderPersistenceTests.RealMouseDrag_OnTheLayerList_PersistsTheReorder` interactive-desktop
flake — both are in the "run it in a quiet session" bucket, and neither is a code defect.)
+35 -33
View File
@@ -1,5 +1,5 @@
using System.Buffers;
using System.Collections.Generic;
using System.Threading.Tasks;
using ytLive.Helpers;
using ytLive.Models;
@@ -33,7 +33,6 @@ public sealed class SceneCompositor
private static readonly Dictionary<SceneElement, VideoFrame> _elementRasters = new(ReferenceEqualityComparer.Instance);
private static readonly object _pasteGate = new();
private const int MaxPasteEntries = 256;
private static float[] _sxLutCache = Array.Empty<float>();
/// <summary>
/// Composite <paramref name="scene"/> into the tier's output frame. Layer order (back →
@@ -397,8 +396,7 @@ public sealed class SceneCompositor
var y1 = System.Math.Min(cropH, py + dh);
if (x0 >= x1 || y0 >= y1) return;
var s = raster.BgraPixels;
var parallelOpts = new ParallelOptions { MaxDegreeOfParallelism = 4 };
Parallel.For(y0, y1, parallelOpts, y =>
StripPool.For(y0, y1, y =>
{
var si = ((y - py) * dw + (x0 - px)) * 4;
var len = x1 - x0;
@@ -547,37 +545,37 @@ public sealed class SceneCompositor
var s = src.BgraPixels;
var srcStride = src.Width; // canvas stride in pixels (not bytes)
// Build horizontal LUT for scaled blit to reduce per-pixel divides
// Build horizontal LUT for scaled blit to reduce per-pixel divides. The table is
// READ by the strip workers while it is being sampled, so it must not be shared
// mutable state: it comes from the thread-safe ArrayPool and goes back in a finally
// (the static single-buffer version raced the moment the strips went parallel).
var lutCount = x1 - x0 + 1;
if (_sxLutCache.Length < lutCount)
var sxLut = ArrayPool<float>.Shared.Rent(lutCount);
try
{
_sxLutCache = new float[Math.Max(lutCount, 4096)];
}
var sxLut = _sxLutCache;
var pxBase = x0 - ex;
for (var xi = 0; xi < lutCount; xi++)
{
var px = pxBase + xi;
float sxCanvas;
if (px < ox || px > ox + drawnW)
var pxBase = x0 - ex;
for (var xi = 0; xi < lutCount; xi++)
{
sxCanvas = float.NaN;
var px = pxBase + xi;
float sxCanvas;
if (px < ox || px > ox + drawnW)
{
sxCanvas = float.NaN;
}
else
{
var sxNorm = (px - ox) / drawnW;
sxNorm = sxNorm < 0f ? 0f : sxNorm > 1f ? 1f : sxNorm;
var sxCrop = sxNorm * cbW;
sxCanvas = cbX + sxCrop;
if (sxCanvas < 0f) sxCanvas = 0f;
if (sxCanvas > src.Width - 1f) sxCanvas = src.Width - 1f;
if (isMirror) sxCanvas = src.Width - 1f - sxCanvas;
}
sxLut[xi] = sxCanvas;
}
else
{
var sxNorm = (px - ox) / drawnW;
sxNorm = sxNorm < 0f ? 0f : sxNorm > 1f ? 1f : sxNorm;
var sxCrop = sxNorm * cbW;
sxCanvas = cbX + sxCrop;
if (sxCanvas < 0f) sxCanvas = 0f;
if (sxCanvas > src.Width - 1f) sxCanvas = src.Width - 1f;
if (isMirror) sxCanvas = src.Width - 1f - sxCanvas;
}
sxLut[xi] = sxCanvas;
}
var parallelOpts = new ParallelOptions { MaxDegreeOfParallelism = 4 };
Parallel.For(y0, y1 + 1, parallelOpts, y =>
StripPool.For(y0, y1 + 1, y =>
{
var py = y - ey;
if (py < oy || py > oy + drawnH) return;
@@ -658,7 +656,12 @@ public sealed class SceneCompositor
}
}
}
});
});
}
finally
{
ArrayPool<float>.Shared.Return(sxLut);
}
}
/// <summary>Two-stage bilinear in 8.8 fixed point; result within ±1 of the
@@ -823,8 +826,7 @@ public sealed class SceneCompositor
var dy0 = Math.Max(0, sy0 - options.SourceRectY);
var dx1 = Math.Min(dstW, sx0 - options.SourceRectX + overlay.Width);
var dy1 = Math.Min(dstH, sy0 - options.SourceRectY + overlay.Height);
var parallelOpts = new ParallelOptions { MaxDegreeOfParallelism = 4 };
Parallel.For(dy0, dy1, parallelOpts, y =>
StripPool.For(dy0, dy1, y =>
{
var si = ((y + options.SourceRectY - sy0) * overlay.Width
+ dx0 + options.SourceRectX - sx0) * 4;
+52
View File
@@ -675,6 +675,58 @@ design.
`SetMediaStreamPropertiesAsync` attempts are refused under `SharedReadOnly`, so 720p MJPG is available
— but see above, it is a quality win only.
### Render budget — getting a composite inside 16.67ms (UNITs 4-5, 2026-10-05)
The pump emits one encoder frame per 16.67ms slot and **pads with a duplicate when a composite
overruns**, so the whole problem is "render + submit + untimed work ≤ 16.67ms, ideally ≤12ms to
leave headroom". `FramePump` now reports per-stage timings (`social`/`ticker`/`flash`/`acquireScratch`)
so a slow stage names itself, and logs the scene name + compositor source rect at pump start.
**Why a 21:9 monitor cost 141ms and a 16:9 one 33ms — the fast path is EXACTLY 1920×1080.** A
21:9 display shrinks the source rect to 1920×803 ("fit", not "fill"), so the opaque-backdrop 1:1 row
walk is skipped, the base is prefilled with opaque black (~2M byte writes), and the backdrop is
resampled per-pixel (~1.54M pixels with an alpha branch). Every recorded A/B lines up:
21:9 → 141ms, 16:9 → 30-33ms, 2560×1440 (fits to 1920×1080 exactly) → fast path. It was never
Debug-vs-Release; today's Debug run on 16:9 with the game up measured 33ms.
**Unit 4 — cheap wins, all pixel-identical:**
- Opaque 1:1 fast paths expanded to the whole-canvas opaque case (a `BlockCopy`, not a per-pixel
alpha branch).
- **Prefill only the uncovered rows.** When the backdrop does not cover the canvas, the letterbox
bands are filled instead of the whole buffer; with no backdrop at all it still fills everything.
- **Per-column LUT** for the scaled blit (`BlitContentRaw`): the source column, its neighbour index
and the 8.8 weights are computed once per column instead of a divide + clamp per pixel.
The table comes from `ArrayPool<float>.Shared` and is returned in a `finally` — a single static
buffer is a **data race** the moment strips run in parallel (see below).
**Unit 5 — strip-parallel composite: `Helpers/StripPool.cs`.** One composite splits into four
horizontal row strips; the calling thread takes a strip too, so four workers never means five-way
oversubscription. Below 64 rows the fan-out costs more than it saves and runs inline. Each strip must
write only its own rows — the blits already partition by row, which is why the output stays
byte-identical.
**The demotion must come from DEDICATED threads, not the shared pool.** The obvious
`Thread.CurrentThread.Priority = BelowNormal` inside `Parallel.For` is wrong: `Parallel.For` runs on
shared ThreadPool threads, so the demotion leaks into every unrelated work item that thread later
picks up. Research (https://stackoverflow.com/questions/524483/net-backgroundworker-objects-thread-priority
— "you have next to no control of which thread pool thread will actually run your task";
https://stackoverflow.com/questions/1578599/c-sharp-lower-thread-priority-in-thread-pool — "You
shouldn't mess with the priority of the regular ThreadPool, since you aren't the only consumer ...
write your own producer/consumer queue, using your own Thread(s) as the consumer(s) - setting the
thread priority yourself") says exactly that, and **a real failed attempt confirmed it**: a full test
run after demoting the pool threads broke an unrelated audio-mix timing test. `StripPool` IS that
producer/consumer queue — four `Thread`s with `Priority = BelowNormal`, `IsBackground = true`,
started lazily on first use, each fed a `Queue<Strip>` + `SemaphoreSlim`. Four is the measured sweet
spot: fewer leaves render over budget, more steals cycles from the game and the encoder and made
render **worse**.
Gate test: `SceneCompositorTests.StripParallel_RepeatedRenders_AreByteIdentical` — the same scene
(scaled round webcam + mirrored image + master-sized flash over a short backdrop, 240 rows so all
three fanned-out paths run) must come out byte-for-byte identical across 8 renders, and must differ
from a backdrop-only render so it cannot pass vacuously. It earned its keep immediately: it caught a
loop-variable capture (`new Thread(() => Worker(i))` where `i` was `Workers` by the time the thread
ran) that crashed the test host on first run.
### Scene compositor (TASK 4 ship step 1 — shipped 2026-08-10, plan in TASKS.md)
The encoder needs the master 1920×1080 frame **without** the preview's editing chrome (SelectionOverlay,
+70 -2
View File
@@ -1,4 +1,4 @@
using Xunit;
using Xunit;
using ytLive.Models;
using ytLive.Services;
using ytLive.Services.Compositor;
@@ -28,7 +28,7 @@ public class SceneCompositorTests
}
/// <summary>Left half = leftColor, right half = rightColor.</summary>
private static VideoFrame Split(int w, int h, byte lr, byte lg, byte lb, byte rr, byte rg, byte rb)
public static VideoFrame Split(int w, int h, byte lr, byte lg, byte lb, byte rr, byte rg, byte rb)
{
var pixels = new byte[w * h * 4];
var half = w / 2;
@@ -509,6 +509,73 @@ public class SceneCompositorTests
AssertColor(bottom, 0, 1039, 255, 0, 0); // background just above the bar
AssertColor(bottom, 960, 540, 255, 0, 0); // bar region moved away from center
}
/// <summary>Unit 5 gate — the compositor now fans a composite out across four dedicated
/// BelowNormal strip workers (Helpers/StripPool), and the column LUT that the scaled blit
/// samples is read by those workers while the call is still in flight. Any strip writing
/// outside its own rows, or a LUT buffer shared between concurrent composites, corrupts a
/// handful of pixels NON-deterministICALLY, so comparing one render to the next is the
/// probe: the same scene must come out byte-for-byte identical every time. The canvas is
/// 240 rows tall (past the fan-out threshold) and carries a scaled round mirrored webcam, a
/// scaled image layer and a master-sized flash, so the three fanned-out paths
/// (BlitContentRaw, BlitBakedElement, BlitOverlay) all run. The color probe keeps the
/// test from passing vacuously on an all-black frame.</summary>
[Fact]
public void StripParallel_RepeatedRenders_AreByteIdentical()
{
const int W = 320, H = 240;
var backdrop = SceneCompositorTests.Solid(W, 200, 12, 34, 56); // shorter than the canvas: prefill bands
var webcamFrame = SceneCompositorTests.Split(64, 48, 255, 0, 0, 0, 255, 0);
var imageFrame = SceneCompositorTests.Split(48, 48, 0, 0, 255, 255, 255, 0);
var flash = new VideoFrame(W, H, new byte[W * H * 4]);
for (var i = 0; i < flash.BgraPixels.Length; i += 4)
{
flash.BgraPixels[i] = 200;
flash.BgraPixels[i + 1] = 200;
flash.BgraPixels[i + 2] = 200;
flash.BgraPixels[i + 3] = 32;
}
var background = new Source { Type = SourceType.DisplayCapture, IsBackground = true, CaptureKey = "monitor:0" };
var webcam = new WebcamSceneConfig { X = 20, Y = 30, Width = 200, Height = 120, ClipShape = ClipShape.Round, Opacity = 0.8 };
var image = new Source { Type = SourceType.Image, X = 180, Y = 140, Width = 100, Height = 70, IsMirrored = true };
var scene = new Scene { Name = "Live" };
scene.Elements.Add(background);
scene.Elements.Add(webcam);
scene.Elements.Add(image);
VideoFrame? FrameFor(SceneElement e) => e switch
{
Source { IsBackground: true } => backdrop,
WebcamSceneConfig => webcamFrame,
Source { Type: SourceType.Image } => imageFrame,
_ => null,
};
var options = new CompositorOptions
{
SourceRectX = 0, SourceRectY = 0, SourceRectWidth = W, SourceRectHeight = H,
OutputWidth = W, OutputHeight = H,
};
var compositor = new SceneCompositor();
byte[]? reference = null;
for (var run = 0; run < 8; run++)
{
var output = compositor.Render(scene, FrameFor, flash, options);
if (reference == null) reference = output.BgraPixels.ToArray();
else Assert.True(reference.AsSpan().SequenceEqual(output.BgraPixels),
$"run {run} diverged from run 0 — strip writes or the shared column LUT raced");
}
// Not vacuous: drop the two layers and the render must change, so the byte-identity
// above is comparing a real composite and not an untouched backdrop.
var backdropOnly = new Scene { Name = "Live" };
backdropOnly.Elements.Add(background);
var baseline = compositor.Render(backdropOnly, FrameFor, flash, options).BgraPixels;
Assert.False(baseline.AsSpan().SequenceEqual(reference!),
"scene rendered identical to the backdrop alone — the test would prove nothing");
}
}
public class StretchMathTests
@@ -549,3 +616,4 @@ public class StretchMathTests
Assert.Equal(255, scaled.BgraPixels[i + 1]); // solid green survives the scale
}
}