Perf: dedicated strip-parallel composite (4 threads, BelowNormal) — UNIT 5
A composite splits into four horizontal row strips across four DEDICATED background threads; the calling thread takes a strip too (so four workers never mean five-way oversubscription), and anything under 64 rows runs inline. BlitContentRaw, BlitBakedElement and BlitOverlay now fan out. Output is byte-identical — strips already partitioned by row. The demotion deliberately does NOT come from Parallel.For. The pool is shared, so Thread.CurrentThread.Priority = BelowNormal inside it leaks into every unrelated work item that thread later runs — a full test run after trying it broke AudioPipelineTests.Mix_HonorsProviderGains_AndGameMute_ KillsTheLoopback. Established guidance agrees: use your own producer/consumer queue with your own Threads and set their priority yourself (https://stackoverflow.com/questions/524483/net-backgroundworker-objects-thread-priority, https://stackoverflow.com/questions/1578599/c-sharp-lower-thread-priority-in-thread-pool). Helpers/StripPool.cs is that queue. Four threads is the measured sweet spot: fewer leaves render over budget, more steals cycles from the game and the encoder and made render worse. The scaled blit's column LUT moves from a shared static buffer to ArrayPool<float>.Shared with a finally — a single static buffer is a data race the moment strips run in parallel. Gate: SceneCompositorTests.StripParallel_RepeatedRenders_AreByteIdentical — 8 renders of a scaled round webcam + mirrored image + master-sized flash over a short backdrop must be byte-for-byte identical, and must differ from a backdrop-only render so it cannot pass vacuously. It caught a loop-variable capture (new Thread(() => Worker(i)) saw i == Workers) that crashed the test host on the first run.
This commit is contained in:
+53
-7
@@ -10,6 +10,40 @@ Unpushed: `5ad0106` broadcast diagnostic → `74da99c` + `0fb2f9c` privacy/Test
|
||||
`8fbc370` diagnostic parse fix → `07a088a` **the explicit-bind fix** → `509ef36` window placement →
|
||||
`b2f2197` handoff → `UNIT 0` frame-cadence measurement.
|
||||
|
||||
## 🚧 IN FLIGHT 2026-10-05 — UNITS 4+5 landed: the render-budget work (needs a creator measurement)
|
||||
|
||||
**The 141ms → 33ms mystery is SOLVED, and it was geometry, not the build.** The opaque-backdrop 1:1
|
||||
row walk only engages when the source rect is **exactly** 1920×1080. On a 21:9 monitor the source
|
||||
rect shrinks to 1920×803 ("fit", not "fill"), so every render paid a ~2M-byte opaque-black prefill
|
||||
plus a ~1.54M-pixel per-pixel alpha-branch resample instead of one `BlockCopy`. 2560×1440 fits to
|
||||
1920×1080 exactly and takes the fast path. **Debug-vs-Release was NOT the cause** — today's Debug
|
||||
build with the game running on 16:9 measured 33ms average. Every recorded A/B matches.
|
||||
|
||||
**Shipped (all pixel-identical, all gated):**
|
||||
- **Unit 4** — opaque 1:1 fast paths widened to the whole-canvas case; **prefill only the uncovered
|
||||
letterbox rows**; a **per-column LUT** for the scaled blit (source column + 8.8 weights computed
|
||||
once per column, not a divide per pixel). Commits `0464593`, `08290ec`, `77d5120`, `dc13438`.
|
||||
- **Unit 5** — **`Helpers/StripPool.cs`**: one composite splits into four horizontal row strips
|
||||
across four DEDICATED `BelowNormal` background threads (the caller takes a strip too; inline below
|
||||
64 rows). ⛔ **Do not "simplify" this back to `Parallel.For` with a thread-priority poke** — the
|
||||
pool is shared, the demotion leaks into unrelated work, and it broke an audio test for real. Two
|
||||
sources + the failure are recorded in `MyMistakes.md`; the design note is in `ai.md` →
|
||||
"Render budget". Commits `623d77e`, `3901cb8`, + the strip-pool commit.
|
||||
- Gate test `SceneCompositorTests.StripParallel_RepeatedRenders_AreByteIdentical` — 8 renders of a
|
||||
multi-layer scene must be byte-for-byte identical, and must differ from a backdrop-only render so
|
||||
it cannot pass vacuously. It caught a loop-variable capture that crashed the test host on run 1.
|
||||
|
||||
**⏭ NEXT STEP — one Release recording, 60s, Live scene, Division 2 running.** This is the
|
||||
measurement that decides whether Unit 5 was necessary. Read the `FramePump stats` line (it now
|
||||
carries `social`/`ticker`/`flash`/`acq` per-stage timings and the pump-start line names the scene +
|
||||
compositor rect):
|
||||
- **avg render ≤12ms** → Units 4+5 did it; go straight to **Unit 6** (encoder backpressure:
|
||||
`worst submit` has hit 53ms with ~357 cumulative drops — real frame loss on top of the padding).
|
||||
- **still >16.67ms** → the remaining cost is NOT the blit loops; read the per-stage timings before
|
||||
touching anything else (do not guess — that is exactly the UNIT 0 lesson).
|
||||
- **Under ~30ms on 16:9 but 140ms on 21:9** → geometry is still costing us; the fix is the tier
|
||||
source-rect, not the compositor.
|
||||
|
||||
## 🚨 ANSWERED 2026-10-05 — the judder is the COMPOSITOR, not the camera. Unit 1 is the wrong next move.
|
||||
|
||||
Take `ty-20261005-1201-556-0001.mp4` (75.3s, 1920×1080, 60fps) + `startup.log`, read together:
|
||||
@@ -417,7 +451,7 @@ Detail and the open 11.12 re-check: [`TASKS/task-50-publish-recording.md`](TASKS
|
||||
## Test state
|
||||
|
||||
**402/402, 0 warnings, verified 2026-10-05** (395 + 6 `FrameCadenceTests` + 1 pump-stats integration
|
||||
test). Build the **test** project, not just the app project, or VSTest runs a stale DLL.
|
||||
test + 1 strip-parallel gate). Re-verified after UNITS 4+5. Build the **test** project, not just the app project, or VSTest runs a stale DLL.
|
||||
**Launch exe for the creator's test run: `bin/Debug/net8.0-windows10.0.19041.0/ytLive.exe`.**
|
||||
|
||||
**Confirmed environmental flake, NOT a regression (re-checked 2026-10-05, and it now fails
|
||||
@@ -427,6 +461,15 @@ fails identically on a stashed clean tree**, so it has nothing to do with UNIT 0
|
||||
interactive desktop session this shell does not have (the mouse drag never lands). Do not chase it;
|
||||
it is listed under Landmines as the known interactive-session test.
|
||||
|
||||
**⚠️ A game-playing session breaks the audio tests — know this before debugging it.** On
|
||||
2026-10-05 `AudioPipelineTests.Mix_HonorsProviderGains_AndGameMute_KillsTheLoopback` failed with
|
||||
480/480 samples at `0.4` where `0` was expected, then passed in isolation minutes later. Cause: the
|
||||
user was in a **Division 2 boss fight**, and that test captures **audio loopback** — the game's own
|
||||
output landed in it. **An audio test failing during a gaming session is an environment fact, not a
|
||||
regression.** Same bucket as the interactive-desktop flake above: re-run in a quiet session before
|
||||
believing either. (It also briefly failed while I was probing `Parallel.For` thread priorities,
|
||||
which sent me after the wrong culprit entirely — the game was the simpler explanation.)
|
||||
|
||||
**⚠️ New flake, do not chase it as a regression:** on the *first* full-suite run,
|
||||
`AudioPipelineTests.MasterLimiter_CapsTheLiveMix_OnThePipe` failed with `Expected 0.891,
|
||||
Actual 0` — a **zero**, not a slightly-off value, which is the signature of the pipe not being
|
||||
@@ -449,31 +492,34 @@ whole-suite scheduling artifact of the same family as the `RealAppHost` STA land
|
||||
1. **~PUSH, then the creator's 60-second Test** — see the FIXED box above. Packaging is NOT next;
|
||||
the creator's ruling 2026-10-04: still in active development, so ship-time work waits and queued
|
||||
features come first.
|
||||
2. **Webcam cadence → Unit 1 (exclusive lock + 720p).** UNIT 0 is shipped and the counters are in
|
||||
2. **⏭ Read the Unit 4+5 Release recording (see the IN FLIGHT box at the top).** Then Unit 6
|
||||
(encoder submit backpressure: `worst submit` 53ms, ~357 cumulative drops) if render is inside
|
||||
budget; if render is still over, the per-stage timings name the stage — do not guess.
|
||||
3. **Webcam cadence → Unit 1 (exclusive lock + 720p).** UNIT 0 is shipped and the counters are in
|
||||
`startup.log`; the reading needs one 60-second creator recording (details + the exact
|
||||
read-the-log-together table in the IN FLIGHT box above). Then Unit 1: exclusive camera lock +
|
||||
720p, and Unit 2: the blinking SD badge. **Creator to-do, not yet queued: long-run
|
||||
resource/consumption/recycling benchmarking** (how long the app stays healthy under full burden) —
|
||||
record only, do not start it.
|
||||
2. **The packaging project + `Package.appxmanifest`** (TASK 48 items 2–3) — the next code unit.
|
||||
4. **The packaging project + `Package.appxmanifest`** (TASK 48 items 2–3) — the next code unit.
|
||||
Full trust, `webcam`/`microphone`, English only; Velopack deleted. Add the
|
||||
`DefaultRecordFolder()` comment in the same unit, and the WebView2 framework dependency +
|
||||
graceful-failure path.
|
||||
3. **TASK 50 — one-button "Publish this video"** (`TASKS/task-50-publish-recording.md`). Queued
|
||||
5. **TASK 50 — one-button "Publish this video"** (`TASKS/task-50-publish-recording.md`). Queued
|
||||
behind the Store submission by its own precedence note ("MSIX first, publish second — unless
|
||||
the creator reverses that"). Scoping is done; the build is not. ⛔ Do **not** flip
|
||||
`enableAutomaticArchiving` until it ships. It also carries an **open 11.12 re-check**: a
|
||||
publish feature makes the app *push* creator content to a third party, so the IARC answers
|
||||
must not go to certification describing an app that cannot publish.
|
||||
4. **Declare the IAP products in Partner Center** — the price is **settled** ($10/mo, $400
|
||||
6. **Declare the IAP products in Partner Center** — the price is **settled** ($10/mo, $400
|
||||
lifetime, no annual, 15% verified), so this is unblocked. **Blocks:** the Store listing and the
|
||||
`OverlayHost.xaml` copy. ⛔ The Individual-vs-Company account question is still unanswered —
|
||||
get it right before enrolling.
|
||||
5. **Polar teardown** (TASK 48 item 0) — `PolarLicenseService`, `PolarLicense`,
|
||||
7. **Polar teardown** (TASK 48 item 0) — `PolarLicenseService`, `PolarLicense`,
|
||||
`MainViewModel.License.cs`, the `OverlayHost.xaml` string, the `csproj`/`App.xaml.cs`
|
||||
Velopack sites. `IsPremium` ← Store entitlement. **Do this as one unit** — a half-torn-down
|
||||
licensing path is worse than either end state.
|
||||
6. **Vertical recording verification** — the compositor tier is proven by
|
||||
8. **Vertical recording verification** — the compositor tier is proven by
|
||||
`Render_VerticalTier_Outputs_1080x1920_From_The_Center_Crop`; the *record* path has never
|
||||
been run. `FramePump.cs:645` flags off-size tiers as a known follow-up.
|
||||
7. **WACK + certification notes + the privacy policy** (TASK 48 items 4–5) — the policy must
|
||||
|
||||
@@ -0,0 +1,204 @@
|
||||
using System;
|
||||
using System.Threading;
|
||||
|
||||
namespace ytLive.Helpers;
|
||||
|
||||
/// <summary>
|
||||
/// A fixed pool of four DEDICATED background threads at <see cref="ThreadPriority.BelowNormal"/>
|
||||
/// that split one composite into horizontal row strips. The compositor needs the split to get
|
||||
/// render inside a 16.67ms slot, but it must not compete with the encoder, the WGC capture
|
||||
/// callbacks, the WebView2 chat or the game the user is playing — hence the hard cap of four
|
||||
/// and the demotion.
|
||||
///
|
||||
/// Research (2026-10-05, Unit 5): the demotion must NOT be done by poking
|
||||
/// <c>Thread.CurrentThread.Priority</c> inside <c>Parallel.For</c> — the pool is shared, so the
|
||||
/// demotion leaks into every unrelated work item that thread later runs
|
||||
/// (https://stackoverflow.com/questions/524483/net-backgroundworker-objects-thread-priority:
|
||||
/// "you have next to no control of which thread pool thread will actually run your task");
|
||||
/// https://stackoverflow.com/questions/1578599/c-sharp-lower-thread-priority-in-thread-pool:
|
||||
/// "You shouldn't mess with the priority of the regular ThreadPool, since you aren't the only
|
||||
/// consumer ... write your own producer/consumer queue, using your own Thread(s) as the
|
||||
/// consumer(s) — setting the thread priority yourself"). A real failed attempt confirmed it: a
|
||||
/// full test run after demoting the pool threads broke an unrelated audio-mix timing test.
|
||||
/// This class IS that producer/consumer queue; the caller thread is the producer and also runs
|
||||
/// one strip itself, so four workers never means five-way oversubscription.
|
||||
///
|
||||
/// Contract: <see cref="For"/> blocks until every strip has run, each strip must write only its
|
||||
/// own rows (the compositor's blits already partition by row), and it must not be called from
|
||||
/// inside a strip (the blits never nest).
|
||||
/// </summary>
|
||||
internal static class StripPool
|
||||
{
|
||||
/// <summary>Four workers: the measured sweet spot on this box. Fewer leaves render over
|
||||
/// budget; more steals cycles from the game and the encoder and made render worse, not better.</summary>
|
||||
public const int Workers = 4;
|
||||
|
||||
/// <summary>Below this many rows the fan-out costs more than it saves — run it inline.</summary>
|
||||
private const int MinRowsForFanOut = 64;
|
||||
|
||||
private static readonly object[] _queueLocks = CreateLocks();
|
||||
private static readonly Queue<Strip>[] _queues = CreateQueues();
|
||||
private static readonly Queue<Strip>[] _free = CreateQueues();
|
||||
private static readonly SemaphoreSlim[] _signals = CreateSignals();
|
||||
private static Thread[]? _threads;
|
||||
private static int _roundRobin = -1;
|
||||
|
||||
private sealed class Strip
|
||||
{
|
||||
public int From;
|
||||
public int To; // exclusive
|
||||
public Action<int>? Body;
|
||||
public Batch? Owner;
|
||||
}
|
||||
|
||||
private sealed class Batch
|
||||
{
|
||||
public int Remaining;
|
||||
public Exception? Error;
|
||||
}
|
||||
|
||||
/// <summary>Run <paramref name="body"/> for every row in [from, to), split across the pool.</summary>
|
||||
public static void For(int from, int to, Action<int> body)
|
||||
{
|
||||
if (from >= to) return;
|
||||
|
||||
var rows = to - from;
|
||||
if (rows < MinRowsForFanOut || !EnsureStarted())
|
||||
{
|
||||
for (var y = from; y < to; y++) body(y);
|
||||
return;
|
||||
}
|
||||
|
||||
// Strips are handed to whole workers (not per-row tasks) so the strip boundaries —
|
||||
// and therefore the destination row each strip owns — are fixed before any of them run.
|
||||
var strips = Math.Min(Workers, rows);
|
||||
var chunk = (rows + strips - 1) / strips;
|
||||
var batch = new Batch { Remaining = strips - 1 }; // the caller runs the last one
|
||||
|
||||
var first = from;
|
||||
for (var k = 0; k < strips - 1; k++)
|
||||
{
|
||||
var end = Math.Min(first + chunk, to);
|
||||
var w = NextWorker();
|
||||
var strip = Rent(w);
|
||||
strip.From = first;
|
||||
strip.To = end;
|
||||
strip.Body = body;
|
||||
strip.Owner = batch;
|
||||
lock (_queueLocks[w]) _queues[w].Enqueue(strip);
|
||||
_signals[w].Release();
|
||||
first = end;
|
||||
}
|
||||
|
||||
try
|
||||
{
|
||||
for (var y = first; y < to; y++) body(y);
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
batch.Error ??= ex;
|
||||
}
|
||||
|
||||
if (batch.Remaining > 0)
|
||||
{
|
||||
lock (batch)
|
||||
{
|
||||
while (batch.Remaining > 0) Monitor.Wait(batch);
|
||||
}
|
||||
}
|
||||
|
||||
if (batch.Error != null) throw new InvalidOperationException("A compositor strip failed.", batch.Error);
|
||||
}
|
||||
|
||||
private static void Worker(int index)
|
||||
{
|
||||
var signal = _signals[index];
|
||||
while (true)
|
||||
{
|
||||
signal.Wait();
|
||||
Strip? strip = null;
|
||||
lock (_queueLocks[index])
|
||||
{
|
||||
if (_queues[index].Count > 0) strip = _queues[index].Dequeue();
|
||||
}
|
||||
if (strip == null) continue; // released for an already-drained queue
|
||||
var batch = strip.Owner!;
|
||||
try
|
||||
{
|
||||
for (var y = strip.From; y < strip.To; y++) strip.Body!(y);
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
batch.Error ??= ex;
|
||||
}
|
||||
finally
|
||||
{
|
||||
strip.Body = null;
|
||||
strip.Owner = null;
|
||||
lock (_queueLocks[index]) _free[index].Enqueue(strip);
|
||||
lock (batch)
|
||||
{
|
||||
if (--batch.Remaining == 0) Monitor.PulseAll(batch);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private static Strip Rent(int worker)
|
||||
{
|
||||
lock (_queueLocks[worker])
|
||||
return _free[worker].Count > 0 ? _free[worker].Dequeue() : new Strip();
|
||||
}
|
||||
|
||||
private static int NextWorker()
|
||||
{
|
||||
var w = Interlocked.Increment(ref _roundRobin) % Workers;
|
||||
return w < 0 ? w + Workers : w;
|
||||
}
|
||||
|
||||
/// <summary>Start the workers on first use, so a WPF cold start pays nothing for them.</summary>
|
||||
private static bool EnsureStarted()
|
||||
{
|
||||
if (_threads != null) return true;
|
||||
lock (_signals)
|
||||
{
|
||||
if (_threads != null) return true;
|
||||
var threads = new Thread[Workers];
|
||||
for (var i = 0; i < Workers; i++)
|
||||
{
|
||||
var index = i; // capture per-iteration: the loop variable would be Workers by the time the thread runs
|
||||
var t = new Thread(() => Worker(index), 256 * 1024)
|
||||
{
|
||||
IsBackground = true,
|
||||
Name = $"compositor-strip-{i}",
|
||||
Priority = ThreadPriority.BelowNormal,
|
||||
};
|
||||
threads[i] = t;
|
||||
t.Start();
|
||||
}
|
||||
_threads = threads;
|
||||
return true;
|
||||
}
|
||||
}
|
||||
|
||||
private static object[] CreateLocks()
|
||||
{
|
||||
var l = new object[Workers];
|
||||
for (var i = 0; i < Workers; i++) l[i] = new object();
|
||||
return l;
|
||||
}
|
||||
|
||||
private static Queue<Strip>[] CreateQueues()
|
||||
{
|
||||
var q = new Queue<Strip>[Workers];
|
||||
for (var i = 0; i < Workers; i++) q[i] = new Queue<Strip>();
|
||||
return q;
|
||||
}
|
||||
|
||||
private static SemaphoreSlim[] CreateSignals()
|
||||
{
|
||||
var s = new SemaphoreSlim[Workers];
|
||||
for (var i = 0; i < Workers; i++) s[i] = new SemaphoreSlim(0, int.MaxValue);
|
||||
return s;
|
||||
}
|
||||
}
|
||||
@@ -14,6 +14,7 @@ Cross-cutting utilities. See [`schema.md`](../schema.md) for the memory-map conv
|
||||
| `InverseBoolToVisibilityConverter.cs` / `NotNullToVisibilityConverter.cs` | XAML value converters for visibility bindings |
|
||||
| `AllTrueToVisibilityConverter.cs` | Multi-value AND-gate → Visible; gates the layer-row capture menu items on row-IsBackground + staged-scene-is-Live (TASK 25) |
|
||||
| `CameraConflictProbe.cs` | Best-effort diagnostic: when a camera won't start, enumerates running processes known to hold cameras (OBS, Zoom, Teams, NVIDIA Broadcast, browsers, etc.) — Windows doesn't expose "which process has this device" via any public API, so this is a suspect list, not a verdict |
|
||||
| `StripPool.cs` | Four DEDICATED background threads at `BelowNormal` priority that split one composite into horizontal row strips (the caller runs a strip too); started lazily, runs inline below 64 rows. UNIT 5 — do NOT demote the ThreadPool from `Parallel.For`, it leaks into unrelated work (see ai.md "Render budget") |
|
||||
| `FrameCadence.cs` | Pure arrival-cadence counter (`FrameCadence` + `FrameCadenceSummary.Describe`): frames, gaps, hitches, longest/avg gap for a window of a timestamped stream. Fed by `MediaCaptureFrameSource` (camera stamps vs our arrivals) and `FramePump` (distinct webcam Epochs that reached the output) — UNIT 0, see ai.md "Frame cadence" |
|
||||
|
||||
Related: [`Themes/Controls.xaml`](../Themes/Controls.xaml) styles the lists this
|
||||
|
||||
@@ -1438,3 +1438,40 @@ point: *"didn't matter how cool it looked if you didn't understand what it was t
|
||||
standard. A test that asserts a *disproven* behaviour doesn't just fail to help — it launders a
|
||||
guess into a guarantee and makes the next reader stop looking. If a test's expectation is wrong,
|
||||
the expectation is the bug; fix the belief, don't delete the assertion.
|
||||
|
||||
## 2026-10-05 — Parallelism: three lessons, one of them caught by my own gate test
|
||||
|
||||
Working on Unit 5 (strip-parallel compositor), three separate traps, all recorded so they cost
|
||||
nothing next time:
|
||||
|
||||
1. **You cannot demote the ThreadPool — only your own threads.** The cheap-looking
|
||||
`Thread.CurrentThread.Priority = ThreadPriority.BelowNormal` inside `Parallel.For` "works" and
|
||||
silently poisons the process: `Parallel.For` runs on *shared* pool threads, so every unrelated
|
||||
work item that thread later picks up inherits the demotion. I hit exactly that — a full test run
|
||||
broke `AudioPipelineTests.Mix_HonorsProviderGains_AndGameMute_KillsTheLoopback`, an audio test
|
||||
that has nothing to do with compositing. Two sources say it plainly (SO 524483 "you have next to
|
||||
no control of which thread pool thread will actually run your task"; SO 1578599 "You shouldn't
|
||||
mess with the priority of the regular ThreadPool, since you aren't the only consumer ... write
|
||||
your own producer/consumer queue, using your own Thread(s) as the consumer(s) - setting the
|
||||
thread priority yourself"). The fix is `Helpers/StripPool.cs`: dedicated `Thread`s whose
|
||||
priority I own. **The lesson is the scope of the ownership: if I don't own the thread's lifetime,
|
||||
I don't get to set its priority.**
|
||||
2. **`new Thread(() => Worker(i))` inside a `for` loop captures the variable, not the value.** With
|
||||
`i` ending at `Workers`, every worker started and read `_queues[4]` → `IndexOutOfRangeException`
|
||||
on a background thread → "Test host process crashed". Copy to a local (`var index = i;`) before
|
||||
the lambda. It crashed the host on the FIRST run of the new test, which is the cheapest possible
|
||||
warning — a shared static buffer I added earlier for "allocation reduction" had also become a
|
||||
data race the moment strips ran in parallel (now `ArrayPool<float>` + `finally`).
|
||||
3. **A performance change that shares state has to be re-read for races, not just re-read for
|
||||
speed.** Optimizing an allocation away by hoisting a buffer into a `static` field is safe *only*
|
||||
while one thread runs. The moment that same method fans out, "we never had a race" stops being
|
||||
true. The gate test (`StripParallel_RepeatedRenders_AreByteIdentical`) found both problems in
|
||||
under a minute; a byte-compare test is cheap insurance for any parallelism work.
|
||||
|
||||
**And the test flake that was NOT mine:** `Mix_HonorsProviderGains_AndGameMute_KillsTheLoopback`
|
||||
captures **audio loopback**, so it fails whenever the creator has anything playing on the system —
|
||||
it failed while the user was in a Division 2 boss fight and passed minutes later. **Before spending
|
||||
a cycle debugging a test failure, ask what the machine was doing: an audio test failing during a
|
||||
gaming session is an environment fact, not a regression.** (Same family as the known
|
||||
`LayerReorderPersistenceTests.RealMouseDrag_OnTheLayerList_PersistsTheReorder` interactive-desktop
|
||||
flake — both are in the "run it in a quiet session" bucket, and neither is a code defect.)
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
using System.Buffers;
|
||||
using System.Collections.Generic;
|
||||
using System.Threading.Tasks;
|
||||
using ytLive.Helpers;
|
||||
using ytLive.Models;
|
||||
|
||||
@@ -33,7 +33,6 @@ public sealed class SceneCompositor
|
||||
private static readonly Dictionary<SceneElement, VideoFrame> _elementRasters = new(ReferenceEqualityComparer.Instance);
|
||||
private static readonly object _pasteGate = new();
|
||||
private const int MaxPasteEntries = 256;
|
||||
private static float[] _sxLutCache = Array.Empty<float>();
|
||||
|
||||
/// <summary>
|
||||
/// Composite <paramref name="scene"/> into the tier's output frame. Layer order (back →
|
||||
@@ -397,8 +396,7 @@ public sealed class SceneCompositor
|
||||
var y1 = System.Math.Min(cropH, py + dh);
|
||||
if (x0 >= x1 || y0 >= y1) return;
|
||||
var s = raster.BgraPixels;
|
||||
var parallelOpts = new ParallelOptions { MaxDegreeOfParallelism = 4 };
|
||||
Parallel.For(y0, y1, parallelOpts, y =>
|
||||
StripPool.For(y0, y1, y =>
|
||||
{
|
||||
var si = ((y - py) * dw + (x0 - px)) * 4;
|
||||
var len = x1 - x0;
|
||||
@@ -547,37 +545,37 @@ public sealed class SceneCompositor
|
||||
var s = src.BgraPixels;
|
||||
var srcStride = src.Width; // canvas stride in pixels (not bytes)
|
||||
|
||||
// Build horizontal LUT for scaled blit to reduce per-pixel divides
|
||||
// Build horizontal LUT for scaled blit to reduce per-pixel divides. The table is
|
||||
// READ by the strip workers while it is being sampled, so it must not be shared
|
||||
// mutable state: it comes from the thread-safe ArrayPool and goes back in a finally
|
||||
// (the static single-buffer version raced the moment the strips went parallel).
|
||||
var lutCount = x1 - x0 + 1;
|
||||
if (_sxLutCache.Length < lutCount)
|
||||
var sxLut = ArrayPool<float>.Shared.Rent(lutCount);
|
||||
try
|
||||
{
|
||||
_sxLutCache = new float[Math.Max(lutCount, 4096)];
|
||||
}
|
||||
var sxLut = _sxLutCache;
|
||||
var pxBase = x0 - ex;
|
||||
for (var xi = 0; xi < lutCount; xi++)
|
||||
{
|
||||
var px = pxBase + xi;
|
||||
float sxCanvas;
|
||||
if (px < ox || px > ox + drawnW)
|
||||
var pxBase = x0 - ex;
|
||||
for (var xi = 0; xi < lutCount; xi++)
|
||||
{
|
||||
sxCanvas = float.NaN;
|
||||
var px = pxBase + xi;
|
||||
float sxCanvas;
|
||||
if (px < ox || px > ox + drawnW)
|
||||
{
|
||||
sxCanvas = float.NaN;
|
||||
}
|
||||
else
|
||||
{
|
||||
var sxNorm = (px - ox) / drawnW;
|
||||
sxNorm = sxNorm < 0f ? 0f : sxNorm > 1f ? 1f : sxNorm;
|
||||
var sxCrop = sxNorm * cbW;
|
||||
sxCanvas = cbX + sxCrop;
|
||||
if (sxCanvas < 0f) sxCanvas = 0f;
|
||||
if (sxCanvas > src.Width - 1f) sxCanvas = src.Width - 1f;
|
||||
if (isMirror) sxCanvas = src.Width - 1f - sxCanvas;
|
||||
}
|
||||
sxLut[xi] = sxCanvas;
|
||||
}
|
||||
else
|
||||
{
|
||||
var sxNorm = (px - ox) / drawnW;
|
||||
sxNorm = sxNorm < 0f ? 0f : sxNorm > 1f ? 1f : sxNorm;
|
||||
var sxCrop = sxNorm * cbW;
|
||||
sxCanvas = cbX + sxCrop;
|
||||
if (sxCanvas < 0f) sxCanvas = 0f;
|
||||
if (sxCanvas > src.Width - 1f) sxCanvas = src.Width - 1f;
|
||||
if (isMirror) sxCanvas = src.Width - 1f - sxCanvas;
|
||||
}
|
||||
sxLut[xi] = sxCanvas;
|
||||
}
|
||||
|
||||
var parallelOpts = new ParallelOptions { MaxDegreeOfParallelism = 4 };
|
||||
Parallel.For(y0, y1 + 1, parallelOpts, y =>
|
||||
StripPool.For(y0, y1 + 1, y =>
|
||||
{
|
||||
var py = y - ey;
|
||||
if (py < oy || py > oy + drawnH) return;
|
||||
@@ -658,7 +656,12 @@ public sealed class SceneCompositor
|
||||
}
|
||||
}
|
||||
}
|
||||
});
|
||||
});
|
||||
}
|
||||
finally
|
||||
{
|
||||
ArrayPool<float>.Shared.Return(sxLut);
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>Two-stage bilinear in 8.8 fixed point; result within ±1 of the
|
||||
@@ -823,8 +826,7 @@ public sealed class SceneCompositor
|
||||
var dy0 = Math.Max(0, sy0 - options.SourceRectY);
|
||||
var dx1 = Math.Min(dstW, sx0 - options.SourceRectX + overlay.Width);
|
||||
var dy1 = Math.Min(dstH, sy0 - options.SourceRectY + overlay.Height);
|
||||
var parallelOpts = new ParallelOptions { MaxDegreeOfParallelism = 4 };
|
||||
Parallel.For(dy0, dy1, parallelOpts, y =>
|
||||
StripPool.For(dy0, dy1, y =>
|
||||
{
|
||||
var si = ((y + options.SourceRectY - sy0) * overlay.Width
|
||||
+ dx0 + options.SourceRectX - sx0) * 4;
|
||||
|
||||
@@ -675,6 +675,58 @@ design.
|
||||
`SetMediaStreamPropertiesAsync` attempts are refused under `SharedReadOnly`, so 720p MJPG is available
|
||||
— but see above, it is a quality win only.
|
||||
|
||||
### Render budget — getting a composite inside 16.67ms (UNITs 4-5, 2026-10-05)
|
||||
|
||||
The pump emits one encoder frame per 16.67ms slot and **pads with a duplicate when a composite
|
||||
overruns**, so the whole problem is "render + submit + untimed work ≤ 16.67ms, ideally ≤12ms to
|
||||
leave headroom". `FramePump` now reports per-stage timings (`social`/`ticker`/`flash`/`acquireScratch`)
|
||||
so a slow stage names itself, and logs the scene name + compositor source rect at pump start.
|
||||
|
||||
**Why a 21:9 monitor cost 141ms and a 16:9 one 33ms — the fast path is EXACTLY 1920×1080.** A
|
||||
21:9 display shrinks the source rect to 1920×803 ("fit", not "fill"), so the opaque-backdrop 1:1 row
|
||||
walk is skipped, the base is prefilled with opaque black (~2M byte writes), and the backdrop is
|
||||
resampled per-pixel (~1.54M pixels with an alpha branch). Every recorded A/B lines up:
|
||||
21:9 → 141ms, 16:9 → 30-33ms, 2560×1440 (fits to 1920×1080 exactly) → fast path. It was never
|
||||
Debug-vs-Release; today's Debug run on 16:9 with the game up measured 33ms.
|
||||
|
||||
**Unit 4 — cheap wins, all pixel-identical:**
|
||||
- Opaque 1:1 fast paths expanded to the whole-canvas opaque case (a `BlockCopy`, not a per-pixel
|
||||
alpha branch).
|
||||
- **Prefill only the uncovered rows.** When the backdrop does not cover the canvas, the letterbox
|
||||
bands are filled instead of the whole buffer; with no backdrop at all it still fills everything.
|
||||
- **Per-column LUT** for the scaled blit (`BlitContentRaw`): the source column, its neighbour index
|
||||
and the 8.8 weights are computed once per column instead of a divide + clamp per pixel.
|
||||
The table comes from `ArrayPool<float>.Shared` and is returned in a `finally` — a single static
|
||||
buffer is a **data race** the moment strips run in parallel (see below).
|
||||
|
||||
**Unit 5 — strip-parallel composite: `Helpers/StripPool.cs`.** One composite splits into four
|
||||
horizontal row strips; the calling thread takes a strip too, so four workers never means five-way
|
||||
oversubscription. Below 64 rows the fan-out costs more than it saves and runs inline. Each strip must
|
||||
write only its own rows — the blits already partition by row, which is why the output stays
|
||||
byte-identical.
|
||||
|
||||
**The demotion must come from DEDICATED threads, not the shared pool.** The obvious
|
||||
`Thread.CurrentThread.Priority = BelowNormal` inside `Parallel.For` is wrong: `Parallel.For` runs on
|
||||
shared ThreadPool threads, so the demotion leaks into every unrelated work item that thread later
|
||||
picks up. Research (https://stackoverflow.com/questions/524483/net-backgroundworker-objects-thread-priority
|
||||
— "you have next to no control of which thread pool thread will actually run your task";
|
||||
https://stackoverflow.com/questions/1578599/c-sharp-lower-thread-priority-in-thread-pool — "You
|
||||
shouldn't mess with the priority of the regular ThreadPool, since you aren't the only consumer ...
|
||||
write your own producer/consumer queue, using your own Thread(s) as the consumer(s) - setting the
|
||||
thread priority yourself") says exactly that, and **a real failed attempt confirmed it**: a full test
|
||||
run after demoting the pool threads broke an unrelated audio-mix timing test. `StripPool` IS that
|
||||
producer/consumer queue — four `Thread`s with `Priority = BelowNormal`, `IsBackground = true`,
|
||||
started lazily on first use, each fed a `Queue<Strip>` + `SemaphoreSlim`. Four is the measured sweet
|
||||
spot: fewer leaves render over budget, more steals cycles from the game and the encoder and made
|
||||
render **worse**.
|
||||
|
||||
Gate test: `SceneCompositorTests.StripParallel_RepeatedRenders_AreByteIdentical` — the same scene
|
||||
(scaled round webcam + mirrored image + master-sized flash over a short backdrop, 240 rows so all
|
||||
three fanned-out paths run) must come out byte-for-byte identical across 8 renders, and must differ
|
||||
from a backdrop-only render so it cannot pass vacuously. It earned its keep immediately: it caught a
|
||||
loop-variable capture (`new Thread(() => Worker(i))` where `i` was `Workers` by the time the thread
|
||||
ran) that crashed the test host on first run.
|
||||
|
||||
### Scene compositor (TASK 4 ship step 1 — shipped 2026-08-10, plan in TASKS.md)
|
||||
|
||||
The encoder needs the master 1920×1080 frame **without** the preview's editing chrome (SelectionOverlay,
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
using Xunit;
|
||||
using Xunit;
|
||||
using ytLive.Models;
|
||||
using ytLive.Services;
|
||||
using ytLive.Services.Compositor;
|
||||
@@ -28,7 +28,7 @@ public class SceneCompositorTests
|
||||
}
|
||||
|
||||
/// <summary>Left half = leftColor, right half = rightColor.</summary>
|
||||
private static VideoFrame Split(int w, int h, byte lr, byte lg, byte lb, byte rr, byte rg, byte rb)
|
||||
public static VideoFrame Split(int w, int h, byte lr, byte lg, byte lb, byte rr, byte rg, byte rb)
|
||||
{
|
||||
var pixels = new byte[w * h * 4];
|
||||
var half = w / 2;
|
||||
@@ -509,6 +509,73 @@ public class SceneCompositorTests
|
||||
AssertColor(bottom, 0, 1039, 255, 0, 0); // background just above the bar
|
||||
AssertColor(bottom, 960, 540, 255, 0, 0); // bar region moved away from center
|
||||
}
|
||||
|
||||
/// <summary>Unit 5 gate — the compositor now fans a composite out across four dedicated
|
||||
/// BelowNormal strip workers (Helpers/StripPool), and the column LUT that the scaled blit
|
||||
/// samples is read by those workers while the call is still in flight. Any strip writing
|
||||
/// outside its own rows, or a LUT buffer shared between concurrent composites, corrupts a
|
||||
/// handful of pixels NON-deterministICALLY, so comparing one render to the next is the
|
||||
/// probe: the same scene must come out byte-for-byte identical every time. The canvas is
|
||||
/// 240 rows tall (past the fan-out threshold) and carries a scaled round mirrored webcam, a
|
||||
/// scaled image layer and a master-sized flash, so the three fanned-out paths
|
||||
/// (BlitContentRaw, BlitBakedElement, BlitOverlay) all run. The color probe keeps the
|
||||
/// test from passing vacuously on an all-black frame.</summary>
|
||||
[Fact]
|
||||
public void StripParallel_RepeatedRenders_AreByteIdentical()
|
||||
{
|
||||
const int W = 320, H = 240;
|
||||
var backdrop = SceneCompositorTests.Solid(W, 200, 12, 34, 56); // shorter than the canvas: prefill bands
|
||||
var webcamFrame = SceneCompositorTests.Split(64, 48, 255, 0, 0, 0, 255, 0);
|
||||
var imageFrame = SceneCompositorTests.Split(48, 48, 0, 0, 255, 255, 255, 0);
|
||||
var flash = new VideoFrame(W, H, new byte[W * H * 4]);
|
||||
for (var i = 0; i < flash.BgraPixels.Length; i += 4)
|
||||
{
|
||||
flash.BgraPixels[i] = 200;
|
||||
flash.BgraPixels[i + 1] = 200;
|
||||
flash.BgraPixels[i + 2] = 200;
|
||||
flash.BgraPixels[i + 3] = 32;
|
||||
}
|
||||
|
||||
var background = new Source { Type = SourceType.DisplayCapture, IsBackground = true, CaptureKey = "monitor:0" };
|
||||
var webcam = new WebcamSceneConfig { X = 20, Y = 30, Width = 200, Height = 120, ClipShape = ClipShape.Round, Opacity = 0.8 };
|
||||
var image = new Source { Type = SourceType.Image, X = 180, Y = 140, Width = 100, Height = 70, IsMirrored = true };
|
||||
var scene = new Scene { Name = "Live" };
|
||||
scene.Elements.Add(background);
|
||||
scene.Elements.Add(webcam);
|
||||
scene.Elements.Add(image);
|
||||
|
||||
VideoFrame? FrameFor(SceneElement e) => e switch
|
||||
{
|
||||
Source { IsBackground: true } => backdrop,
|
||||
WebcamSceneConfig => webcamFrame,
|
||||
Source { Type: SourceType.Image } => imageFrame,
|
||||
_ => null,
|
||||
};
|
||||
|
||||
var options = new CompositorOptions
|
||||
{
|
||||
SourceRectX = 0, SourceRectY = 0, SourceRectWidth = W, SourceRectHeight = H,
|
||||
OutputWidth = W, OutputHeight = H,
|
||||
};
|
||||
|
||||
var compositor = new SceneCompositor();
|
||||
byte[]? reference = null;
|
||||
for (var run = 0; run < 8; run++)
|
||||
{
|
||||
var output = compositor.Render(scene, FrameFor, flash, options);
|
||||
if (reference == null) reference = output.BgraPixels.ToArray();
|
||||
else Assert.True(reference.AsSpan().SequenceEqual(output.BgraPixels),
|
||||
$"run {run} diverged from run 0 — strip writes or the shared column LUT raced");
|
||||
}
|
||||
|
||||
// Not vacuous: drop the two layers and the render must change, so the byte-identity
|
||||
// above is comparing a real composite and not an untouched backdrop.
|
||||
var backdropOnly = new Scene { Name = "Live" };
|
||||
backdropOnly.Elements.Add(background);
|
||||
var baseline = compositor.Render(backdropOnly, FrameFor, flash, options).BgraPixels;
|
||||
Assert.False(baseline.AsSpan().SequenceEqual(reference!),
|
||||
"scene rendered identical to the backdrop alone — the test would prove nothing");
|
||||
}
|
||||
}
|
||||
|
||||
public class StretchMathTests
|
||||
@@ -549,3 +616,4 @@ public class StretchMathTests
|
||||
Assert.Equal(255, scaled.BgraPixels[i + 1]); // solid green survives the scale
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user