Files
LlamaCasty/docs/ai/frame-pipeline.md
T
gramps b39439abdc docs: the vertical take is clean — 29.8 distinct webcam images/s
Creator's Release take `ty-20261109-1142-036-0005.mp4` (1080×1920 @ 30, 300 s) measured by cropping the
webcam tile and diffing frames: 29.8 distinct images/s overall, median 30 s window 30.0/s, worst 24/s,
one 4-frame repeat run in 9026 frames. The app's own lines agree (`webcam 28.4–30.0/s` against camera
`arrivals 29.9–30.1/s`).

- The 25.6/s failure recorded on the 2026-10-05 16:9 take does **not** reproduce in the vertical tier,
  so the vertical take no longer blocks the cadence units.
- Recorded the measurement trap in `frame-pipeline.md`: a threshold of 1.0 on a downscaled tile
  reports long "0.0/s" stretches during low motion — duplicates sit below 0.02, a still face at
  0.3–0.7. Read the histogram before believing a count.
- Vertical-tier camera check reported 1–6 pass; both creator-hands items in TASKS.md are closed.

(The mic being off for the first 2:42 does not affect video cadence.)
2026-11-09 11:53:58 -08:00

862 lines
75 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Frame pipeline — compositor, render budget, measurement, and the live pump
> Split out of [`ai.md`](../../ai.md) (2026-10-26) so a session reads only what its task touches;
> `ai.md` keeps a short invariant stub for this file. Directory map: [`index.md`](index.md).
This is the subsystem that turns scenes into the encoder's frames: the CPU compositor that renders
the master, the budget it must fit inside, the instrumentation that measures it, and the
`FramePump` that paces frames into ffmpeg. Everything here is current architecture plus the
measured history that produced it; superseded designs are marked inline.
---
## Budget, measurement & the compositor
### Frame cadence — how a dropped frame became measurable (UNIT 0, 2026-10-05)
**The problem this exists to solve.** The recorded file cannot distinguish a dropped camera frame
from a padded repeat. `-f rawvideo` carries no per-frame timestamps — ffmpeg assigns pts from frame
order at the DECLARED fps — so a webcam delivering 25 of its 30 frames still muxes a full-length,
correctly-timed 60fps file. Measured on take `ty-20261005-1001-821-0001.mp4` (85.6s, 1080p60): the
lower-right webcam region held 25.6 distinct images/s against the camera's 30fps, with 1,153 normal
33ms intervals, 281 gaps of 50–100ms, and 96 bunched 17ms gaps — roughly one hitch in five, which is
what makes a face judder while the file looks flawless. (The "webcam renders at ~18fps" figure that
used to sit in this map was WRONG: it came from a different session and counted every newly rendered
layer, not webcam arrivals. Corrected 2026-10-05.)
**Vertical-tier take (2026-11-09).** `ty-20261109-1142-036-0005.mp4`, Release, 1080×1920 @ 30 fps,
300 s, webcam in its default bottom-right tile. Cropped the tile, downscaled to 160×88 grey and
diffed frame to frame: **29.8 distinct images/s overall, median 30-second window 30.0/s, worst 24/s,
one 4-frame repeat run in 9026 frames.** The app's own lines agree — `webcam 28.4–30.0/s` against
`arrivals 29.9–30.1/s`. So the 25.6/s failure above does **not** reproduce in the vertical tier, and
the vertical take therefore does not block the cadence units.
⚠️ Threshold matters. A naive `mean abs diff > 1.0` on a downscaled tile reports long "0.0/s"
stretches during low motion — that is the measurement failing, not the camera. Duplicates encode to
near-zero (<0.02); a still face with sensor noise sits around 0.3–0.7. Use ~0.05 and read the
histogram before believing a count.
**`Helpers/FrameCadence.cs`** — one pure counter, three feeds. `Add(timestamp)` on any shared
monotonic clock; `Summarize()` → frames, rate (gaps ÷ span, NOT frames ÷ span — N frames span N-1
periods), gaps, hitches (default threshold 50ms ≈ 1.5× a 30fps period), longest and average gap;
`Describe(label)` gives every producer the same one-line shape so two lines can be read against each
other. An out-of-order stamp counts as a frame but produces no gap AND does not become the reference
for the next interval — otherwise one bad stamp invents a multi-second "hitch" twice.
**Three measurement points, and what each separates:**
| Line | Fed by | Answers |
|---|---|---|
| `Camera cadence WxH: arrivals … \| camera …` | `MediaCaptureFrameSource` | device/USB/driver cadence (`MediaFrame.SystemRelativeTime`, QPC domain, nullable) vs what the reader handed us. `TryAcquireLatestFrame` keeps only the LATEST frame, so falling behind is silent by design — this is where that shows |
| `webcam …` on the pump's stats line | `FramePump.TimedResolver` | distinct webcam `VideoFrame.Epoch` values that reached a composite — what actually got into the output |
| `frames per 5s from R renders` | `FramePump` | submits vs REAL composites — the duplicate-on-lag ratio, the only honest reading of how much of the stream is a repeat |
**Why the stats-line denominators changed (the actual bug).** Every average was divided by
`statFrames`, the number of frames SUBMITTED — which deadline padding always fills to the full 60fps
target. A pump fed one webcam frame per 200ms logged "300/300 frames per 5s, avg render 12.4ms"
while producing 60 distinct composites: the render average was diluted ~5× and the loss was
invisible. Render/resolve/wait are now divided by `statRenders`; submit stays per-frame (it really
happens per submit). Regression test:
`FramePumpTests.Stats_ReportRealRendersAndDistinctWebcamFrames_NotJustPaddedSubmits`.
**Reading the three together:** camera 30/s steady + arrivals 25/s = WE dropped it (reader or
compositor). Ragged in both = the device, the USB link, or the driver. Pump `webcam` below
`arrivals` = the compositor missed frames the reader had already delivered.
**✅ WHAT THE FIRST REAL READING SAID (2026-10-05) — the camera was never the problem.**
`ty-20261005-1201-556-0001.mp4` + its log: the C920 delivers **30.0/s with ZERO gaps ≥50ms**, and the
device's own stamps are identical, so the camera, USB link, and driver are all clean. The pump did
**34 composites per 5s (6.8/s) at ~141ms each** (worst 294ms), and the file holds **6.8 distinct
webcam images/s at 88.7% duplicates with no arrival hitches at all**. The judder is entirely us
holding one face for 140ms and padding the rest of a 60fps container. `resolve` is 0.1ms, so it is
pure pixel work in the composite, not frame lookup. Two consequences, both load-bearing:
- **720p is a sharpness win, NOT a smoothness win** — it triples the bytes stretched per render on a
path already 4.4× over budget. Schedule the exclusive lock AFTER the compositor work or it makes
the judder measurably worse.
- **The 10:01 take hit 25.6 images/s and the 12:01 take 6.8, same scene shape
(`full-render split=0 elements=6 dynamic=5`), same 640×480 camera.** Composites went 29–34ms →
143–155ms between sessions with nothing logged explaining it. That comparison was made by EYE on
two takes because the log carried no per-element breakdown and no scene dump — both now exist
(see "Per-element composite timing + scene dump" below); naming the culprit is a reading, not a
guess, and guessing is explicitly out of bounds. **Superseded 2026-10-16:** "same scene shape" was
an eye comparison of takes whose geometry was never logged, and the next bad take still on disk is
the *vertical* tier (below). A Release 1920×1080 take now measures 1.0ms with no element over
0.4ms — see "Per-element composite timing + scene dump" → what the first instrumented take said.
**Counter subtlety, so it is not re-derived:** the pump's `webcam` line read 13/s while the file held
6.8/s. Not a bug — the counter stamps a frame when the resolver FIRST sees it, and one composite takes
141ms, so ~4 camera frames arrive during it (signature pass stamps one epoch, the actual draw uses a
later one). The file is the truth; the pump counter is a pipeline-health signal and overcounts by
design.
**Still unmeasured:** the camera is stuck at the C920 default 640×480 YUY2 @30 because all 185
`SetMediaStreamPropertiesAsync` attempts are refused under `SharedReadOnly`, so 720p MJPG is available
— but see above, it is a quality win only.
### Per-element composite timing + scene dump (UNIT 0 follow-up, 2026-10-16 — MEASUREMENT ONLY)
The gap the cadence work left open is now closed by instrumentation, and closed by nothing else: the
141ms take was never compared to the 30ms take because **no line in the log said which element cost
what, and no line said what the scene WAS.** Both now exist. No fix has been attempted, and none is
allowed until a real reading names the culprit.
- **`Services/Compositor/CompositeTimings.cs`** — a collector owned by the pump, threaded through
`SceneCompositor.Render` / `CompositeLayers` / `BakeStaticBase` / `BlitBakedElement` /
`BlitCachedLayer` (and `SceneGraph.GetBakedBase` → bake path) as an optional last parameter, so
every other caller (thumbnails, tests, transition preview) is untouched and costs nothing.
`Add(stage, ticks)` accumulates; `Describe(renders, minMs = 0.1)` renders the averaged breakdown
slowest-first, dropping sub-0.1ms noise.
- **Stage names** (stable, greppable): `backdrop` (prefill + live capture + background art + their
resolves), `base` (the cached-bake `BlockCopy`), `flash`, `social`, `ticker`, `zoom`, `scale`,
`overlay` (the fully-static path's per-frame overlay copies), `bake backdrop`, `bake overlays`, and
**one entry per element as `#<z-index> <name>`** — the index IS the z-order, so the log names the
layer by both position and name. Bake-pass elements are prefixed `bake #…` so a one-off rebake
cannot masquerade as a per-composite cost.
- **`#<i> <name> raster` nests inside its element's total** and is the paste-cache rasterize miss
(`BlitCachedLayer`'s `raster == null` branch) — the direct answer to "is one element re-sampling
every tick, or is the paste itself slow". A stage's time includes that stage's frame resolve;
`resolve` is separately totalled in the existing stats line.
- **`FramePump parts (N renders/5s): …`** — logged once per 5s window (same window as
`FramePump stats`), averaged per render so the numbers compare directly with `avg render`, then the
collector is reset. **No parts line, or a short one, is a reading too:** the fully-static shortcut
and C4 cache hits composite no elements (`cache 92R/130H` beside a missing parts line is expected).
- **`FramePump scene dump: '<name>' elements=N split=S`** followed by one line per element
(`[i] name kind=… type=… vis=… rect=x,y WxH op=… clip=…`), logged when the pump first runs and
whenever the active scene changes. This is the record the 10:01/12:01 comparison never had.
- Gate test: `ytLive.Tests/CompositeTimingTests.cs` (one `[Fact]`) — a REAL composite of a full-canvas
element plus a 64×64 one must rank the full-canvas element first, carry the `raster` sub-stage and
the fixed stages, and the dump must name both elements with z-index/type/rect and echo the split.
- **`scripts/webcam-cadence.py`** — the other half of a reading: what the FILE holds (distinct webcam
images/s + hold histogram), beside what the pump says it did. Recipe and its three traps (H.264
re-encoding, the round-clip crop, scene switches): `MyMistakes.md` → "Measuring webcam CAMERA cadence
in a recording".
**Read takes in Release.** The creator launches `dotnet run -c Release`; Debug pixel paths are
several× slower, so a Debug number is not a reading of this pipeline (see
[`environment.md` → Run](environment.md#run)).
**Derivative check (AGENTS.md):** this is not invented — OBS's own answer to "which source is slow" is
the **Source Profiler**, which reports the *average execution time of each source's tick and render
functions* ([docs.obsproject.com → Source Profiler](https://docs.obsproject.com/reference-libobs-util-source-profiler),
per-source timers via `util/profiler.h`). Same shape here: one timer per element per composite, reported
as an average, plus the stage split. The `#<i> <name> raster` sub-stage is the extra this case needs —
it separates "re-sampled every tick" from "slow paste" without another session.
### ✅ What the first instrumented take said (2026-10-16, Release) — the judder DID NOT reproduce
Take `ty-20261016-1841-559-0002.mp4` (174.5s, 1920×1080@60, Release build `e95154d9` — a **recording**,
not preview; scene `Live`, switched to `Chat` at 28s, back to `Live` at 44s) and its log window:
| Reading | This take |
|---|---|
| composites | **301 per 5s = 60/s**, every slot; 0 dropped, 0 stalls after warm-up |
| avg render | **1.0–1.1ms**, worst 2.5ms — ~7% of the 16.67ms budget |
| `parts` | `#5 HD Pro Webcam C920 0.4ms, #5 … raster 0.2ms, backdrop 0.2ms` — the other four elements are under the 0.1ms print floor |
| C4 full-composite cache | **151R/150H** per window — half the ticks were a whole-frame `BlockCopy` |
| webcam at the resolver | **30.0/s**, longest gap **50.0ms** (one extra 60Hz frame = sampling phase, not a drop) |
| the file itself | **29.6 distinct webcam images/s, 82% of images held exactly 2 frames**, longest hold 267ms (`scripts/webcam-cadence.py`) |
Nothing in this take is over budget and **no element is expensive — there is no culprit here to fix.**
The two stalls in the take are not per-composite cost: 124ms on the very first render (warm-up) and
53ms on the switch into `Chat` (a rebake).
**Scale of the disagreement:** the historic slow readings are 141ms (Oct 5 12:01) and 45ms (Oct 9
19:48) — 40–140× the ~1ms this take needed per composite. Three facts about those takes, all read off
the same log:
- **Oct 9 19:48 (`ty-20261009-1948-121-0000-2.mp4`) was the VERTICAL tier**, not a 16:9 one:
`FramePump started (1080×1920 @ 60 fps) scene='Live' comp=608x1080` — the 607×1080 centre crop
bilinear-upscaled to 1080×1920 (2M px), 92 renders/5s, avg render 45.0ms, worst 278ms, 92 stalls,
`cache 92R/0H`. The file on disk is 11.6s and every frame is identical (a static desktop), so it
cannot be read for cadence — only its cost.
- **Oct 5 12:01 (`…1201-556-0001.mp4`, no longer on disk) logged `FramePump started (1920×1080 @ 60
fps)` with NO `scene=/comp=` suffix** — that suffix did not exist yet, so its source rect was never
recorded and the map's "21:9 → 1920×803" explanation stays a hypothesis.
- **A take can latch mid-take.** Oct 5 12:29 goes 1.5ms (avg render) → 3.0ms → **45.2ms within 15
seconds and stays there** (`cache 10R/0H`, then `98R/0H`). So the slow state is not a fixed property
of a scene geometry. The consistent signature of every slow window is `cache xR/**0H**`: the C4
whole-frame cache never reused once, so every tick composited. The healthy take shows the other
stable state (composite 1ms → every second tick is byte-identical → whole-frame reuse). **The latch
is the thing to test on a reproducing take** — it is a hypothesis, not a reading; the `parts` line
is what will decide it.
**What this does NOT say.** This take is the 1:1 master tier and its scene logs
`full-render split=0 elements=6 dynamic=4` (the bad-period takes logged `dynamic=5`), so it never
exercises the failing geometry. "The healthy path is healthy" is not "the judder is gone". Reproducing
it needs a Release take in the failing configuration (vertical tier first, then the 21:9 display
config) — that take's `parts` line names the stage, and only then is a fix unit allowed.
**Reading the `parts` line's denominators:** like `avg render`, it is divided by `statRenders`, which
counts C4 REUSES as well as real composites (the 301 above = 151 real + 150 whole-frame copies). So in
a healthy 30fps-camera take both averages are ~2× optimistic; the `cache NR/NH` pair on the same line
says how much of the window was real composite work. `CompositeTimings.Describe` drops stages under
0.1ms by default, and **a missing or short parts line is itself a reading** — a fully-static window
(every tick a C4 hit) composites no elements at all.
### Render budget — getting a composite inside 16.67ms (UNITs 4-5, 2026-10-05)
The pump emits one encoder frame per 16.67ms slot and **pads with a duplicate when a composite
overruns**, so the whole problem is "render + submit + untimed work ≤ 16.67ms, ideally ≤12ms to
leave headroom". `FramePump` now reports per-stage timings (`social`/`ticker`/`flash`/`acquireScratch`)
so a slow stage names itself, and logs the scene name + compositor source rect at pump start.
**Why a 21:9 monitor cost 141ms and a 16:9 one 33ms — the fast path is EXACTLY 1920×1080.** A
21:9 display shrinks the source rect to 1920×803 ("fit", not "fill"), so the opaque-backdrop 1:1 row
walk is skipped, the base is prefilled with opaque black (~2M byte writes), and the backdrop is
resampled per-pixel (~1.54M pixels with an alpha branch). Every recorded A/B lines up:
21:9 → 141ms, 16:9 → 30-33ms, 2560×1440 (fits to 1920×1080 exactly) → fast path.
**Corrected 2026-10-16 — the "never Debug-vs-Release" half of that is withdrawn.** A Release
1920×1080 take measures **1.0–1.1ms** where an Oct 9 take of the same tier measured 10.4ms, and
`Build <hash>` carries no build configuration, so the config of every pre-2026-10-16 take is
UNKNOWN: no historic render number may be compared against a Release take. The 21:9 explanation
itself stays a hypothesis — the 141ms take's start line predates the `scene=/comp=` suffix, so its
source rect was never recorded. A take can also latch mid-take (see the instrumentation section).
**Unit 4 — cheap wins, all pixel-identical:**
- Opaque 1:1 fast paths expanded to the whole-canvas opaque case (a `BlockCopy`, not a per-pixel
alpha branch).
- **Prefill only the uncovered rows.** When the backdrop does not cover the canvas, the letterbox
bands are filled instead of the whole buffer; with no backdrop at all it still fills everything.
- **Per-column LUT** for the scaled blit (`BlitContentRaw`): the source column, its neighbour index
and the 8.8 weights are computed once per column instead of a divide + clamp per pixel.
The table comes from `ArrayPool<float>.Shared` and is returned in a `finally` — a single static
buffer is a **data race** the moment strips run in parallel (see below).
**Unit 5 — strip-parallel composite: `Helpers/StripPool.cs`.** One composite splits into four
horizontal row strips; the calling thread takes a strip too, so four workers never means five-way
oversubscription. Below 64 rows the fan-out costs more than it saves and runs inline. Each strip must
write only its own rows — the blits already partition by row, which is why the output stays
byte-identical.
**The demotion must come from DEDICATED threads, not the shared pool.** The obvious
`Thread.CurrentThread.Priority = BelowNormal` inside `Parallel.For` is wrong: `Parallel.For` runs on
shared ThreadPool threads, so the demotion leaks into every unrelated work item that thread later
picks up. Research (https://stackoverflow.com/questions/524483/net-backgroundworker-objects-thread-priority
— "you have next to no control of which thread pool thread will actually run your task";
https://stackoverflow.com/questions/1578599/c-sharp-lower-thread-priority-in-thread-pool — "You
shouldn't mess with the priority of the regular ThreadPool, since you aren't the only consumer ...
write your own producer/consumer queue, using your own Thread(s) as the consumer(s) - setting the
thread priority yourself") says exactly that, and **a real failed attempt confirmed it**: a full test
run after demoting the pool threads broke an unrelated audio-mix timing test. `StripPool` IS that
producer/consumer queue — four `Thread`s with `Priority = BelowNormal`, `IsBackground = true`,
started lazily on first use, each fed a `Queue<Strip>` + `SemaphoreSlim`. Four is the measured sweet
spot: fewer leaves render over budget, more steals cycles from the game and the encoder and made
render **worse**.
Gate test: `SceneCompositorTests.StripParallel_RepeatedRenders_AreByteIdentical` — the same scene
(scaled round webcam + mirrored image + master-sized flash over a short backdrop, 240 rows so all
three fanned-out paths run) must come out byte-for-byte identical across 8 renders, and must differ
from a backdrop-only render so it cannot pass vacuously. It earned its keep immediately: it caught a
loop-variable capture (`new Thread(() => Worker(i))` where `i` was `Workers` by the time the thread
ran) that crashed the test host on first run.
### Scene compositor (TASK 4 ship step 1 — shipped 2026-08-10, plan in TASKS.md)
The encoder needs the master 1920×1080 frame **without** the preview's editing chrome (SelectionOverlay,
DimRects, output-rect outline, badge, placeholder). WPF's `RenderTargetBitmap` is software-rendered and
captures the visual tree *including* chrome, so the preview can't be captured — the output is a **second,
parallel software compositor** over the `VideoFrame` (BGRA8) seam, and the XAML preview
(`MainWindow.xaml` CanvasGrid + element DataTemplate) is the rendering contract it replicates. Two
renderers must agree: geometry, `UniformToFill` cover-crop, round clip, mirror, border, z-order. Preview
stays XAML (editing view); the compositor is the output view.
- **The `DimRects` shading is itself preview chrome, so it renders as a SIBLING of `CanvasGrid`
(above it, below the `ZoomBox`), never a descendant** (2026-10-08, creator ruling): the zoom box's
`VisualBrush` samples all of `CanvasGrid`, and if the dims were inside it the box would show the
dim shading — especially after the box is dragged away and the fixed capture slides under the dims.
- **Composite the active tier's crop at crop size** (`CompositorOptions {SourceRectX/Y/W/H,
OutputWidth, OutputHeight}`, fed from `MainViewModel.OutputRect*`): 16:9 = full 1920×1080; the
vertical 9:16 tier = centered 608×1080 crop. FFmpeg's `scale` filter resizes the raw BGRA frame
to the selected output tier after it leaves the frame pump; `setsar=1` keeps square pixels.
- **CPU posture:** with FFmpeg as a subprocess the master crosses a CPU readback to the pipe every frame
anyway, so GPU compositing buys little at this layer count (2-3 live layers; static layers
pre-composite once into a cached base). GPU effort belongs to **NVENC** (the encoder), not composition;
if composition grows (wipes, filters, many layers), a D3D11 compositor can replace this one **behind
the same seam** — the CPU master buffer stays the contract.
- **Every global overlay must be composited on EVERY render path (bug found 2026-09-26).** The
frame pump has three exits: `RenderFull` (no graph / no static base), `CompositeLayers` (static base
+ dynamic tail), and the **fully-static shortcut**, which stretches the cached bake and returned it
directly. That third path silently **discarded every per-frame overlay** — the social bar, the alert
ticker and now the brand flash. A creator whose scene was one static background saw none of them,
and the flash looked "preview-only forever" for a reason unrelated to the tier. `SceneCompositor.Overlay`
is therefore `internal static` (it copies before blitting, so the cached bake is never mutated) and
the shortcut now composites overlays onto a copy before stretching. **When adding an overlay, grep
for all three paths** — a fourth exit that forgets it fails the same silent way.
- **`VideoFrame` buffers are recycled by some producers, and `Epoch` is not optional.** A producer that
hands out the same array across frames MUST bump `Epoch`, or `FramePump`'s buffer-identity paste cache
false-hits and freezes stale content (the take-11 class of bug). The `C4` cache signature mixes
`Epoch`, so a per-frame overlay naturally invalidates it. `BrandFlashPresenter` reuses one 8MB
master buffer and stamps `Epoch` per read for exactly this reason — allocating a fresh 8MB frame per
tick is not an option at 1080p30.
- **Dev-only: two instances side by side (`Helpers/InstanceProfile.cs`).** Pre-1.0 the creator runs one
instance to stream and another to screen-capture it. Nothing prevented a second instance (no mutex, no
port); what broke it was shared state — the whole-scene read/write layout DB (clobber), Chromium's
**exclusive** lock on the WebView2 user data folder (second instance won't start), the auth token store
(a test instance overwrites the real YouTube sign-in), and `startup.log`. Set
`YTLIVE_INSTANCE=<id>` and that process gets a private root at `%APPDATA%\ytLlive\instances\<id>\`
for the DB, the auth file, the log and the WebView2 folder. Recording folder and the ffmpeg `tools\`
cache stay shared on purpose; global hotkeys stay un-namespaced (Windows refusing the second
`RegisterHotKey` is the correct answer). The **entire implementation is inside `#if DEBUG`** — a
Release build compiles to `DataRoot => DefaultRoot` and `WebViewDataFolder => null`, and the call
sites are unconditional so Release cannot drift. Proof + harness: `ytLive.Tests/InstanceIsolationTests.cs`.
⚠️ Grepping the binary for `YTLIVE_INSTANCE` proves nothing — a `const` is inlined and appears in
neither build; check the `InstanceVariable` **field** in metadata instead.
- **Branding flash is composited by the output path too** (it's on the live output, per Monetization),
passed in as a pre-rendered `VideoFrame?` — the compositor core stays pure byte-math, no WPF. Likely a
bundled asset rather than runtime text rendering (deterministic, no font/layout risk).
- **Frame sources are injected** via a `Func<SceneElement, VideoFrame?>` resolver
(`SceneCompositor.Render(scene, frameFor, flashFrame, options)`) — the caller maps each element to
its frame (webcam → `DeviceId`, image → `AssetId` via `StaticPixelCache`, backdrop → `CaptureKey`),
so the compositor is pure, WPF-free, and hermetic to test. The capture managers wire into that
resolver in the encoder step, not the compositor step. The master buffer (the compositor's return
value) is the seam a future D3D11 compositor would honor identically.
---
## The live pump — frame producer to encoder
### Live frame pipeline (TASK 4 ship step 5 — shipped 2026-08-12, health stats 2026-08-13, plan in TASKS.md)
The **`FramePump`** (`Services/Encoder/`) is the live frame producer: while live it snapshots the active
scene each tick, resolves every element to its latest frame, composites it into the tier's output frame,
and paces frames into the encoder at the tier's FPS. **rawvideo is stamped by ARRIVAL:** ffmpeg assigns
pts from frame order at the declared fps — supply rate = output speed. A starved producer (pump < fps)
ships a time-lapse, truncated file with NO error (take-2 lesson, 2026-09-01); the pump therefore logs
`FramePump stats: n/target frames per 5s from R renders, avg render Xms (resolve R2), avg submit Yms` so a slow stage names itself (the `frames … from R renders` pair and the render-count denominators arrived with UNIT 0 — see "Frame cadence" above; before that every average was divided by the SUBMITTED count, which deadline padding always fills to the full target, so a pump fed one webcam frame per 200ms still logged "300/300 frames" and its render average was diluted ~an order of magnitude) — and since 2026-09-04 so does the BUILD: every build gets a GUID stamped
into `Helpers/BuildStamp` (csproj `GenerateBuildStamp` target, fresh per compile — no lying incremental build), logged at startup
("Build xxxxxxxx (compiled ...)"). **Take 6 (stamp-less, ambiguous): render 35-41ms with slice 3 built or not unknown — attribution is why the stamp exists; takes 7+ are
readable.** If the breakdown says resolve dominates, the suspects are pre-cached chat/config thrash or the web frame; if blit dominates, the general-path element sizes. **Pattern — everything is a constructor-injected
seam:** `Func<Scene?>`, `Func<SceneElement, VideoFrame?>` resolver, `Func<CompositorOptions>`,
`Func<EncoderOptions?>`, `Func<IFfmpegEncoder>`, `Action<string>` log, and an injectable pacing delay
(default `Task.Delay`; tests inject `Task.Yield`). The pump is free of WPF and of the capture managers.
- **`StartAsync` never throws** — the VM fires-and-forgets it from the sync command handler; failures log
+ surface via the `Failed` event. `BuildEncoderOptions` now always returns options (TASK 18) with
`StreamEnabled`/`RecordEnabled`/`RecordPath` baked from the pills + `_activeRecordPath`; the pump skips the
encoder only when `options == null`, i.e. "no output configured". `MainViewModel._rtmpUrlProvider` is that
seam — a `Func<string?>` that now yields the reusable stream's ingest URL (TASK 9, shipped 2026-08-16):
loaded from the `LayoutStore` cache at startup and set fresh by `PrepareAndStartLiveAsync` before the pump
starts. **The pump reads the URL once at startup**, which is why go-live ensures the stream BEFORE
`StartAsync`.
- **Deadline pacing + row-blit render (take-3 starvation fix, 2026-09-03):** take 3 recorded 30s into a
2.1s file and the 5s stats line named the cause in one number — `17/300 frames per 5s, avg render
258.1ms, avg submit 1.5ms`. Two defects, both fixed: (1) the pump slept the FULL interval after each
render, so period = render + interval — OBS's `video_thread` (libobs/media-io/video-io.c) pattern
replaces it: absolute `nextTick += intervalTicks` deadline, sleep only the remainder, and on overrun
skip the wait. (NOTE, slice 9: the original "skip the missed ticks (rebase)" half of this fix was
wrong — see the slice 9 correction below.) (2) the compositor did
per-pixel float sampling + `Math.Round` blending over all 2.07M master pixels (backdrop) and scanned
the whole destination per overlay (a 64px social-bar strip cost 2M iterations). `SceneCompositor` now
follows the libyuv pattern (BSD-3, chromium.googlesource.com/libyuv/libyuv — cited per the
derivative-work rule): 1:1 aligned blits take a row-walk fast path (bilinear at scale 1 + integer
offset is the identity) with per-pixel alpha branch and integer fixed-point blend; `BlitOverlay`
clips to the intersection rect; a full-cover live backdrop skips the opaque-black pre-fill. The
general per-pixel path handled scaled/round/mirrored elements. Test:
`Pump_Paces_To_The_Deadline_Compensating_Render_Cost` records the requested wait
via the pacing seam (a seam fake must genuinely await — a synchronous completed task runs the whole
pump loop on `StartAsync`'s continuation and hangs the run; MyMistakes).
- **Slice 2 of the render fix (2026-09-04, take 4):** the pacing held (file no longer truncated at
loop level) but render stayed at 58.9ms — the per-pixel row walk was still 2M managed iterations and
every tick allocated a fresh 8.3MB master buffer. Three changes: (1) **`VideoFrame.IsOpaque`** — a
producer-contract flag (only the screen-capture and webcam paths set it; DWM/MediaCapture fill alpha
255 by contract); a full-canvas, aligned, opacity-1 blit of an opaque frame is now ONE
`Buffer.BlockCopy` (~1.5ms) instead of the loop, and the black pre-fill is skipped when the backdrop
covers. (2) **integer fixed-point bilinear** in the general `BlitContent` path (row-hoisted invariants,
no divisions, no `Math.Round`) — within ±1 of the float reference, inside the ±2 test tolerance.
(3) **scratch pool in the pump** — `AcquireScratch`/`ReleaseScratch` recycle the master buffer
(max 4, keyed by length, owned-by-reference so bake-cache/social-bar/static-art arrays can never be
captured); release happens strictly AFTER `SubmitFrameAsync` returns (the write to stdin copies),
and the free-list Contains guard makes the transition Cut path safe (`BlendFrame` returns the
to-frame itself, aliasing the scratch). The per-tick `fromScene` render in the transition branch was
**dead weight** (BlendFrame uses `TransitionService.FromFrame` captured at `Start`, never the pump's) —
removed, and the pump's `fromSceneProvider` seam + `MainViewModel` call site went with it. Tests:
`Pump_Pools_ScratchBuffers_Across_Frames_Without_Stale_Pixels` (the ONE: alternating backdrop colors
pin every frame's content, repeated backing-array identity proves the pool recycles),
`Composite_OpaqueFullCover_Backdrop_CopiesEveryPixel_Into_Scratch` (memcpy branch + scratch
sentinel). `FakeEncoder.SubmitFrameAsync` now snapshots bytes like the real stdin write — holding the
reference would race legitimate recycling. Take 5 must show `avg render ≤ ~10ms, ≈300/300`.
**Vertical/output scaling moved to bundled FFmpeg (2026-11-04):** the C# compositor no longer
enlarges the 608×1080 crop to 1080×1920 on every frame. The bundled FFmpeg `scale` filter processed
300 synthetic BGRA frames in 0.447s on this machine; that is a scaler-only microbenchmark, not a
live-stream result.
- **Slice 3 — chat raster-on-change (2026-09-04, take 5):** take 5 measured `138/300, avg render
25.5ms` — the per-tick blits were cheap now, but `ResolveOutputFrame` → `RenderChatBox` ran a FULL
WPF raster (`ChatBoxRenderer`: FormattedText + `RenderTargetBitmap` + CopyPixels + channel swap)
EVERY tick whenever the message buffer was non-empty — and the buffer survives between sessions,
so even a signed-out record-only take paid it. Established answer, same as OBS text sources:
**raster on change, blit the cache every tick.** `ChatOverlayLayer` now keeps a content version
(`Messages.CollectionChanged` → `_contentVersion++`) plus a config key (box size + all Chat*
props); `RenderFrame` returns the cached `VideoFrame` by identity until either changes (the
compositor only ever reads a cached frame). Test: `ChatOverlayLayerCacheTests` (RealApp, real
renderer — `Same()` for unchanged inputs, `NotSame()` on message/config change, null on empty).
Accepted cost pending take 6: one slow tick (~15-25ms) per arriving message; if chat-burst frame
loss shows up, the next slice moves the re-render off-tick (debounced, dispatcher-side).
- **TASK 3.17 — static TextOverlay raster-on-change:** `TextOverlayRenderer.UpdatePreview` runs WPF
`FormattedText` + `RenderTargetBitmap` only on the dispatcher when text, formatting, or box size
changes. It publishes one immutable straight-alpha BGRA `VideoFrame` and a `WriteableBitmap`
copied from those same pixels; preview and output therefore share the raster. `FramePump` calls
only `RenderFrame` (a concurrent cache read), never WPF. Text is a static `Source`, so its raster
participates in the static bake or above-split path; property changes invalidate the bake, and the
full-render signature includes the resolved text frame. Its premultiplied WPF readback is
unpremultiplied before entering `VideoFrame`, whose compositor contract is straight-alpha BGRA.
- **Slice 5 — paste cache for non-opaque layers (2026-09-04, take 7/8 data):** the render/resolve
split (added in the stamp commit) finally named the last thing honestly: `resolve ≈ 0` but
`render 26-27ms` (124-135/300, ≈2.2x) — the chat fix HAD worked; the remaining cost was the
compositor re-rasterizing EVERY layer per tick even when its pixels never changed (this creator's
Live scene: chat 159k + web widget 271k + image 95k + webcam 156k px ≈ 680k samples @ ~38ns each).
OBS's actual shape: sources cache their surface, the compositor pastes. `SceneCompositor` now
routes non-opaque layers through `BlitCachedLayer`: a layer rasterizes ONCE into an element-space,
TRANSPARENT-based frame keyed by (source-array identity, source W×H, ceil'd dst rect, round,
mirror), then every later tick PASTES it (integer position, row alpha-blend, opacity applied at
paste). **Correction (2026-09-12, the web-widget "black box"):** the raster builds on a
TRANSPARENT base, but the sampler's PARTIAL-alpha branch used the opaque-dst blend — it
premultiplied the color into RGB and forced `alpha=255`. A translucent widget pixel then pasted
as opaque darkened ink (the box in recordings) while the raw-bitmap preview stayed correct.
`BlitContentRaw` now takes `transparentDst` (the raster call passes `true` and writes straight
color + straight alpha; the paste rows do the source-over). Master paths are bit-identical.
Guard: `PasteCache_SemiTransparentLayer_RevealsBackdrop_NotOpaqueInk`. Producers hand out fresh immutable arrays, so array-identity keys can never serve stale
content; dict bounded at 48, cleared wholesale on overflow. Only changing content (webcam device
frames, web capture ticks, chat messages) resamples; the opaque backdrop keeps its memcpy path.
Position/opacity drags are now near-free (no resample — paste params, not cache keys).
ONE integration test: `PasteCache_RepeatRender_IsByteIdentical_And_ContentChangePropagates`
(raster-vs-paste byte equality incl. round-clip margins + new-array propagation); the whole
existing pixel suite guards sampler semantics. Take 9 must show `≈300/300, avg render ≤ ~8ms`;
if it lands there, the recording saga closes. Follow-ups unchanged: vertical-tier scale alloc,
debounced chat re-render under message bursts.
- **Slice 6 — the sleep quantum WAS the ceiling (take 9, 2026-09-04):** period measured ~37ms while
work (render+submit) was ~25 — the missing ~12ms is `Task.Delay` rounding EVERY request up to the
Windows clock tick (documented ~15.6ms default; learn.microsoft.com/en-us/dotnet/api/system.threading.tasks.task.delay).
A frame finishing 3ms early requests a 3ms wait and sleeps a full 15.6 — capping the producer at
~27fps NO MATTER how fast the compositor ran. This is why takes 7→9 showed zero playback change
despite real render wins: the sleep floor dominated everything above it. Established fix (game-loop
canon — stackoverflow.com/questions/5441464; timeBeginPeriod — learn.microsoft.com/en-us/windows/win32/api/timeapi/nf-timeapi-timebeginperiod):
`timeBeginPeriod(1)` for the pump's lifetime (paired `timeEndPeriod` in the finally), sleep only the
BULK of the remainder (request minus 2ms), `Thread.SpinWait` the last ~2ms across the deadline;
blown deadlines still rebase. (SUPERSEDED in part: slice 9 deleted the rebase — deadlines only
advance; slice 10 replaced the burst re-write with one fresh composite per iteration.) Stats gained
`avg wait Xms` so render+submit+wait must ≈ period —
the accounting is closed, no stage can hide again. Same slice: the webcam's `IsOpaque` paste-cache
bypass was removed (it re-sampled ~156k px every tick even between identical device frames; the
cached paste beats the sampler on hits and costs the same on misses). Take 10 verdict: `≈300/300`
honest fps — if short, the wait/render split names the remaining term with no ambiguity left.
- **Slice 7 — the loop was ON the UI thread the whole time (take-8 numbers, 2026-09-04):** with
the wait finally measured, take 8 (59a02a5b) showed the impossible pair — `render 22ms, wait 10ms`
against a 16.7ms deadline: a blown deadline rebases with NO wait. The wait was the producer
**queued behind the live preview on the dispatcher**: `PumpAsync` starts from a UI command handler
and its `await` continuations inherit the UI `SynchronizationContext`, so the "WPF-free" compositor
actually rendered on the UI thread and ran only when WPF let it. OBS runs its media loops on
dedicated threads for exactly this reason. Fix: `_pumpTask = Task.Run(() => PumpAsync(...))`
(no sync context inside → continuations stay on the pool). Side effects handled, not ignored:
`StaticPixelCache.Get` now locks (pool miss-decodes race UI callers); `ChatOverlayLayer.RenderFrame`
checks its cache off-thread but MARSHALS the rare raster miss to the dispatcher (DrawingVisual/
RenderTargetBitmap are UI-thread objects) and re-validates there; pump events already marshalled.
Also: `GCLatencyMode.SustainedLowLatency` for the pump's life, stats gained `worst render Xms`
(spike visibility — bimodal averages hid them), webcam now routes through the paste cache (bypass
re-sampled 156k px even between identical device frames). Tests: `Pump_Produces_OffTheStartingContext`
(the ONE — inline-pumping SyncContext proves continuations never return to the starting thread) +
70/70. Take 9 verdict: wait ≈ true remainder (period → 16.7, n → ~300); `worst render` names any
remaining raster-miss spikes; render+submit+wait still == period — the accounting holds.
- **Slice 8 — capture buffer ring + paste-cache Epoch (take 11, 2026-09-04):** take 11 validated
the architecture — typical frames now land `work ~10ms + wait ~6.8ms = 16.7`, exactly the deadline
(212/300). The whole remaining gap is PERIODIC 35-65ms worst-render spikes that worsened across
the take (189→147 frames/5s) — the signature of gen2 GC pauses, now provable via `gen2 +N` in
every stats window. Biggest churn was structural: `ScreenCaptureFrameSource` minted a fresh
~8.3MB `byte[]` per DWM frame (~500MB/s LOH). It now rotates a shared-frame ring (4-deep at slice 8,
deepened to 8-deep after the "flash" finding — consumer holds must never outlive depth × source
period), size-matched per slot, with reused downscale row scratch. Recycled
arrays would poison the paste cache (it keys on array IDENTITY), so `VideoFrame.Epoch` —
monotonic per producer frame, 0 for fresh-array producers — joins the key. Regression test
`PasteCache_RecycledArrayWithNewEpoch_ReRasterizes_NotStaleHits` fails on the old key by
construction. The camera producer carried the same fresh-array churn (110-220MB/s at 30-60fps):
it now rotates its own 8-deep ring + Epoch (same identity rule), and the WebView2 capture
reuses a canvas scratch + 8-deep output ring + a reused WriteableBitmap instead of minting two
fresh arrays + a new bitmap per 10Hz tick. Next suspect if gen2 stays hot: the WPF preview load
itself driving gen2 — recorded as follow-up, untouched this slice.
- **Slice 9 — the "rebase" WAS the recording time-lapse (2026-09-10, the recording playback-timing
take):** slice 6's "blown deadlines still rebase" was the recording-timing bug all along. In a real
take the pump emitted `215-219/300 per 5s` (~43fps) while `render+wait == period` still looked closed —
the rebase ERASED every missed slot (deadline = wall-now again), so the missed ticks never showed in
the stats. rawvideo has no per-frame timestamps: ffmpeg muxes by frame count at `-framerate 60`, so a
43fps reality was authored into a 60fps container and **every recording played ~1.4x fast** (takes
confirmed with a WSL ticker visible in the recording: file duration 11.44s vs ~15.5s wall). Two
defects, both fixed the OBS way (`libobs/media-io/video-io.c` — the video thread NEVER resets its
deadline; every interval tick outputs ONE frame, and a late render means repeated content — judder —
never a skipped timestamp):
(1) **count-based CFR emission:** `while (GetTimestamp() >= nextTick) { SubmitFrame(frame); nextTick += intervalTicks; statFrames++; }`
submits exactly one frame per crossed slot, re-writing the current composite when the render overruns
(duplicated footage = correct duration, not a time-lapse), and `nextTick` is NEVER reset to wall-now;
(2) **`-re` removed from `FfmpegArgs`** — it was a second, fighting pacer on the rawvideo demux
(its "Resumed reading … after a lag" grew 0.79s→4.82s across the take, leaving the pump behind its own
honest-stats count). One fix covers the game background AND the webcam — both flow through this one
pump. Muxed duration is now frame-count ÷ fps == wall time by construction. Test: the pump suite stays
green unchanged — the pacing test's `<interval` wait assertion still holds because the frame emits at
the next slot boundary, not immediately. (Unrelated pre-existing failure found the same day:
`Composite_FullScene_MasterPixels` pixel (1380,700) cyan-vs-magenta — reproduces with this fix stashed,
untouched by it, recorded as a follow-up.)
- **Slice 10 — bounded encoder queue + drop policy + burned-in frame counter (2026-09-10, the
playback-pacing take):** slice 9 made the DURATION right but the CREATOR still read the recorded
ticker as "1...23...4...56..." — variable pacing. The aggregates (301/300, uniform PTS, 15.6s wall vs
15.74s file) could not see it, and the cause was finally measured in `FfmpegEncoder.SubmitFrameAsync`:
it BLOCKED on `WriteAsync(8.3MB) + FlushAsync` whenever ffmpeg lagged the pipe, and slice 9's
burst `while` loop then re-wrote that SAME composite for every slot that ticked past — frozen
content runs (the "smeared ticker"). Fixed the OBS way (the encoder queue in `libobs/obs-encoder.c`:
the encoder's thread NEVER couples back into the video thread; overload = dropped content, never a
frozen video thread):
(1) **`FfmpegEncoder.SubmitFrameAsync` is now an ENQUEUE** into a bounded `Channel<byte[]>` (cap 120 ≈
2s at 60fps) owned by a dedicated drain task with the stdin write + flush; the caller NEVER blocks on
the pipe. Pixels are copied into an `ArrayPool` buffer before enqueue (the write now happens later on
the drain thread, so the pump's scratch reusable the moment submit returns — same contract as before).
(2) **drop-on-overflow (creator's choice: freshness over coverage):** when the queue is full the NEWEST
frame is dropped and counted (`IFfmpegEncoder.DroppedFrames`, `Interlocked`); the session never freezes
or smears — it drops. Stop flushes the whole queue then EOF (`Channel.TryComplete` → drain writes
leftovers → closes stdin → ffmpeg finalizes+exits), so no accepted frame is ever lost at stop.
(3) **pump loops ONE submit per iteration** (replacing slice 9's burst): every due slot gets a FRESH
composite — no catch-up slot ever repeats frozen content; missed slots vanish as a count-based gap.
The deadline counter is still never reset to wall-now (slice 9 ruling).
(4) **burned-in frame counter (the new judge):** `_outputIndex` is burned into a 6-digit dot-matrix
strip (white box + black 5×7 glyphs, bottom-right corner) of EVERY composite before submit. The WSL
ticker is demoted because its own timers smear under Windows host load; decoding the recording and
reading the strip is clock-independent: the number advances +1 per frame and jumps by exactly the
counted drops (queue overflow OR skipped catch-up slots). Stats gained `worst submit Xms`, `dropped N`,
`stalls K`; a stall logger names any iteration > 2× interval with its render/submit split — with the
queue, submit is ~1ms, so a stall means render/resolve. The strip sits above the social bar (bar is
composited, then burned over) — a debug judge, tiny at 1080p. Test:
`Backpressure_QueueOverflow_DropsFrames_AndNeverBlocks` (the ONE: a 40ms-per-frame sink fake makes
every enqueue fill the queue; submit must return instantly, drops are counted, stop flushes exactly
submitted − dropped bytes). Full suite 289/290 (the pre-existing compositor pixel failure unchanged).
Audio untouched (follow-up). WSL ticker-under-load reliability check still to run (informational — the
burned counter is the judge).
- **Slice 11 — web-layer animation ran at ~1/6 speed; capture cadence 10Hz→~30Hz + de-throttle
(2026-09-10, the "widget animates too slow" take):** the recording is 60fps but the WebView2
capture loop was a blind 100ms `DispatcherTimer` = 10Hz — a 60fps-designed widget was sampled 6×
under its native rate (repeated-footage slow-mo). Two stacked throttles:
> Superseded by **Slice 14** (below): `CaptureScheduler` and the cadence hooks were the polling
> design; frame-driven composition capture replaced them — nothing here is current architecture.
(1) **the 10Hz cap** (unconditional, code-proven) and
(2) **Chromium hidden-page throttling** (`requestAnimationFrame` parked, JS timers clamped to ~1s
per WebView2Feedback#1172/#3070 + Chrome-88 timer throttling) whenever the app window is unfocused
or covered — the page is off-screen at (-5000,-5000), so it is a hidden page the moment the host
window loses occlusion.
Fixed:
(1) **`CaptureScheduler` (new, `Services/CaptureScheduler.cs`)** replaces the per-session
`DispatcherTimer`: a dispatcher timer that DROPS ticks while a capture is in flight (latest-wins,
never queues) — the in-flight drop is what made raising the cadence safe (concurrent full-HD PNG
`CapturePreviewAsync` calls would stack ~10-30ms encodes and publish stale-after-fresh). Interval is
owner-configurable: recording 33ms (~30Hz), idle 200ms. Unit-tested without a WebView2 runtime
(the ONE test: `CaptureScheduler_Drops_Ticks_While_Capture_InFlight_And_Resumes` — overlapping
ticks dropped, capture resumes when idle, driven by a TCS so it is deterministic).
(2) **de-throttle browser args**: the shared `CoreWebView2Environment` is created with
`--disable-backgrounding-occluded-windows --disable-renderer-backgrounding
--disable-features=CalculateNativeWinOcclusion` (the Electron/Streamlabs-class embedder answer for
occluded-window animation throttling) BEFORE `EnsureCoreWebView2Async`.
(3) **cadence hook**: `MainViewModel` sets `SetCaptureInterval(33)` on record/stream start and
`(200)` on stop (`MainViewModel.Streaming.Operations.cs`).
(4) **capture-cost telemetry**: first 30 captures per session log elapsed ms to startup.log —
PNG-encode+decode cost decides whether ~30Hz stays or drops to ~20Hz; the FramePump drops frames
(never time-lapses, slice 10) if UI-thread GC churn starves it, so the cost is observable.
Full suite 290/291 passing, the sole failure the pre-existing compositor pixel test. The web-overlay
transparency verification (the first-capture diagnostic bound to about:blank) and the audio-silence
item remain open push-gate items — both untouched by this slice.
- **Slice 12 (audio diagnostics, 2026-09-10):** the recorded audio is full-length silent AAC
(−91dB, 1124 frames / 23.95s in the latest take) — the named-pipe delivered ~9.1MB of zeros to
ffmpeg for the entire recording, so the loop ran and connected, but both WASAPI capture sources
delivered nothing. Sources started fine (`Audio: using system default mic...` logged), no failure
callbacks fired, and the mixer's existing integration test (`Mix_WithFiltersDuckAndGain_Lands_On_AudioPipe`)
proves the loop→pipe math is sound. The fault is capture-side: either the default render endpoint
carried nothing (audio played on a non-default device — common), or both endpoints were held
exclusive, or genuinely nothing played. Added permanent per-5s live-loop telemetry to startup.log
(`Audio live: pipe connected=, dropped writes=, micLevel=, loopLevel=, drained, peakMix`) and
`NamedPipeAudioWriter.DroppedWrites` — names the exact stage on the next take without new
code. `AudioMixer.FillAndMix` now returns `(MicRms, MicDrained, LoopDrained)` for the telemetry
accumulation. Full suite 290/291, same pre-existing sole failure.
- **Slice 13 (web transparency diagnostic re-point, 2026-09-10):** the "?" in MyMistakes'
"RESOLVED (? VERIFY)" is being closed. The one-shot widget dump + alpha stats bound to the
FIRST capture ever = the initial about:blank document — a blind instrument. NavigationCompleted
for the REAL widget URL now arms `WidgetDumpRemaining = 5`; those 5 post-paint captures dump
`%TEMP%\ytLive-web-<id>-w1..5.png` + the shared `AlphaStats(...)` line (alpha[min/max/mean/zero%])
+ FindContentBounds result to startup.log. That line alone decides the fix branch: zero% ⇒ the
pre-parse injection did not hold for this widget (opaque capture → chroma-key / stronger DOM
injection); large zero% + tight contentBounds ⇒ the capture IS transparent and the recording's
black block lives downstream (compositor blend/underlay). `AlphaStats` extracted as the shared
sampler. No new tests (WebView2 runtime not instantiable in tests; the re-point is log-only).
- **Slice 14 — web frames are now COMPOSITION-CAPTURED; the PNG poll + `CaptureScheduler` are gone
(2026-09-14, the "still not 60fps" take):** slice 11's ~30Hz was still a `CapturePreviewAsync` PNG
poll — every 1920×1080 full-HD encode+decode cost 35–165ms, so the session budget capped real
cadence at ~20Hz for a 60fps-designed widget. Root fix = frame-driven capture, not faster polling:
the WebView2 renderer now feeds `Windows.Graphics.Capture` directly through a
`CoreWebView2CompositionController` (the mechanism `WebView2CompositionControl` and Flutter's
`webview_windows` use — see `graphics_context.cc` `CreateGraphicsCaptureItemFromVisual`, which
captures the root `surface_` visual).
- **Pipeline (per session):** `CreateCoreWebView2CompositionControllerAsync(WindowHandle)` — the
parent HWND must exist, so `MainWindow.InitWebView2()` moved from the ctor to **Loaded** (was
`InitWebView2(WebViewHostPanel)`; the hidden XAML `WebViewHostPanel` overlay is deleted).
`controller.RootVisualTarget` = a child `ContainerVisual` (`RelativeSizeAdjustment = 1,1`) under a
root `ContainerVisual` (1920×1080, `IsVisible=true`) built on ONE `Windows.UI.Composition.Compositor`
(created once on the UI thread); `Bounds = 0,0,1920,1080`, `BoundsMode=UseRawPixels`,
`ShouldDetectMonitorScaleChanges=false`, `RasterizationScale=1.0`, `IsVisible=true`,
`DefaultBackgroundColor=Transparent`; then `GraphicsCaptureItem.CreateFromVisual(root)`.
Frames pull from a free-threaded `Direct3D11CaptureFramePool` (2 buffers) →
`SoftwareBitmap.CreateCopyFromSurfaceAsync` (BGRA, `BitmapAlphaMode.Straight` — the compositor
blends STRAIGHT alpha, so premultiplied readback would wreck corner anti-aliasing) → per-frame
`FindContentBounds` alpha-bbox on the capture worker → `VideoFrame.CropBounds` stamped → ring up to
2 fresh + Epoch (reuse; GC lesson slice 8) → dispatcher-coalesced copy (Render priority) into ONE
crop-sized shared `WriteableBitmap`; `PreviewBitmapChanged` raised only on bitmap (re)creation.
Also unpumped: the OBS-side de-throttle flags (slice 11) stay — they fix the hidden-page JS
throttling; the capture path itself no longer polls.
- **Implemented in:** new `Services/WebCaptureFrameSource.cs` (owns item+pool+session — mirrors
`ScreenCaptureFrameSource`, which servers as the copy template) + reworked `Services/WebView2Manager.cs`
(owns controllers, the visual tree, nav/script wiring, preview; internal seam ctor
`(Dispatcher, Func<string, IScreenCaptureSource>?)` so the tests instantiate zero WinRT; public
ctor `(Dispatcher)`).
- **CoreMessaging DQ recipe (record-once):** the 19041 CsWinRT projection has NO
`DispatcherQueueController.CreateOnCurrentThread()` (CS0117 — only `CreateOnDedicatedThread` +
`FromAbi(IntPtr)`). P/Invoke `coreMessaging.dll!CreateDispatcherQueueController` with struct
`DispatcherQueueOptions { DwSize, ThreadType = 2 (DQTYPE_THREAD_CURRENT), ApartmentType = 2 (DQTAT_COM_STA) }`,
wrap via `DispatcherQueueController.FromAbi(ptr)` (mirror of `CaptureInterop`), only then
`new Compositor()` — all once on the WPF UI thread (the app dispatches Render there).
- **Compile notes (each cost a build cycle):** `Compositor` collides with the repo's OWN
`ytLive.Services.Compositor` namespace → fully-qualify `Windows.UI.Composition.Compositor`;
`CoreWebView2CompositionController` exposes **`Close()`**, not `Dispose()`; `Color` is ambiguous
(`System.Drawing` vs `System.Windows.Media`) → `System.Drawing.Color.Transparent`.
- **Dropped:** `Services/CaptureScheduler.cs` (DELETED — no cadence to schedule), all three
`SetCaptureInterval` hooks in `MainViewModel.Streaming.Operations.cs`, the XAML `WebViewHostPanel`.
`TransparentBackgroundScript` const is unchanged (a test pins it).
- **Test model (Good Dog):** reworked `ytLive.Tests/WebView2ManagerTests.cs` — dropped the 4
control-size tests + the `CaptureScheduler_Drops…` test; `FindContentBounds` tests moved to
`WebCaptureFrameSource.FindContentBounds`; the ONE integration test
`Frames_PublishCroppedPreview_And_CoalesceToLatest_CarryingCropBounds` drives the internal seam with
`FakeWebSource : IScreenCaptureSource` + a real background-STA `DispatcherPump` (copied from
`ScreenCaptureManagerTests`): register → one crop-sized preview bitmap published → `CropBounds` +
`IsOpaque=false` on `GetLatestFrame` → back-to-back pumps coalesce to latest. Byte assertion compares
the DENSE 2×2 crop (`CropBytes` helper — slices source rows with stride gaps, a contiguous range
spans rows wrongly).
- Full suite **293/293 green, 0 warnings**. Audio untouched. **NOT YET VERIFIED ON DEVICE** — the
composition path needs a real 60fps-widget run (open item; see HANDOFF). Web work commits stay
LOCAL (no push) until the user greenlights.
- **Slice 15 — one frame per deadline SLOT: duplicate-on-lag, never skip (2026-09-14, device takes
ty-1723/1726):** slice 10's freshness choice — a render overrun VANISHES the missed slots from the
file — authored ACCELERATED playback: ~35ms render cost vs the 16.6ms slot, one fresh frame per
35ms (log wall-1726: `FramePump stall: iteration 33ms (> 2× the 17ms interval): worst render
33-46ms`), and a 60fps container muxed on arrival → 1723: 697 frames = 11.62s video vs 11.84s
audio; 1726: 163 frames = 2.72s vs 2.93s. The video also ended 0.21–0.24s before the audio ("audio
speeds up, cuts off at the end"). OBS's answer is duplicate-on-lag: `libobs/media-io/video-io.c`
emits one frame per tick and a lagger DUPLICATES the newest frame, counted as "lagged frames due
to rendering lag/stalls" (obs-output.c) — a wall-time hole never exists (docs.obsproject.com/
backend-design: "If the video frame queue is full, it will duplicate the last frame"). The pump's
submit is now a bounded catch-up: `while (now >= nextTick) { submit; nextTick += intervalTicks; }`
over a `deadlineNow` captured once per iteration — the LATEST composite, fresh on the first missed
slot, repeated (OBS's duplication) for the rest. Duration == wall under any render load, at the
cost of a short judder during a stall. The burst is safe because `Channel.TryWrite`
never blocks (slice 10's queue — the take-9 smear was the BLOCKING pipe-write re-copying a stale buffer
during a long freeze; here each emit is nanoseconds). The burned `_outputIndex` (slice 10 judge)
moved INSIDE the submit loop so every emitted slot carries its own +1 (and the old unconditional
pre-gate bump no longer gaps the sequence on non-submitting fast-render iterations). **Good Dog
test** `Pump_Overrun_Renders_EmitsEverySlot_NotSkipped`: 60fps, resolver sleeps 35ms, asserts ≥0.65
of the wall slots are emitted (the skip-pump wrote ~1/35ms ≈ 200 in 7s; the slot pump ~415). Full
suite **294/294 green, 0 warnings**. The +0.6s audio-late clap reading on 1726 was confounded by
the 1.1x acceleration — re-measure on device; if a real residual remains it is the audio pipeline.
No push (web/A/V work stays local).
- **Slice 16 — the desktop capture conversion was the bottleneck: fast downscale +
cadence throttle + reuse-distance ring (2026-09-14, device take ty-1742):** slice 15
fixed pacing but the 1742 desktop layer was still jerky/frozen with a horizontal
tear. Decoded to raw frames and audited: the **desktop band was frozen 21s of 23.35s
(90%)** — ~6.1 content updates/s, freeze runs up to 2.28-2.78s, dup-run max 90
frames (1.5s). The render stat (`worst render 33-36ms`) was real but MOOT: the
capture CONVERSION was the wall. The monitor delivers at the **240Hz DWM cadence**
(~4.2ms); with one-in-flight conversions and each 2560×1440→1920×1080
`DownscaleBgra` at ~150ms under load, `LatestFrame` updated a handful of times/s —
the desktop feed inside a 60fps file read ~90% frozen. (Webcam + audio were their
own paths — fine, matching the report.) Three changes, all in
`Services/ScreenCaptureFrameSource.cs` (+ `Services/FrameRingBuffer.cs`):
(1) `DownscaleBgra` is now **integer 8.8 fixed-point, "shift only at the end"** —
the exact two-stage math of `SceneCompositor.Bilinear` (MyMistakes take-4 rule),
dropping double-per-pixel to a row-walk of integer ops (the capture downscale had
stayed the naive float twin of the 258ms disaster); (2) a **10ms conversion floor**
(`MinConvertInterval`) so the 240Hz tail stops queuing ~150ms of serialized
conversion per second — the open edge sits just above the ~60/s the pump can use;
(3) the ring is a **reuse-distance pool** (`FrameRingBuffer`, depth 8, redLine 4):
a buffer is only rewritten ≥4 rents after its last hand-out else a fresh allocation,
replacing the blind round-robin — since `ScreenCaptureManager` keeps
`session.LatestFrame` across conversions and the dispatcher preview copy lags,
"who released the buffer" needs a consumer API that doesn't exist; a reuse
DISTANCE needs no cooperation (the 1742 new-top/old-bottom tear is that
read-under-write, structural now). Telemetry added: a startup.log line every 2s
(`frames/s, conv avg/max ms, skip busy/cadence, ring allocs`) so the device take
can be judged numerically. **Good Dog test**
`Ring_NoLap_ReusesOnlyAfterRedLineRents` (depth 3/redLine 4 exercises the red-line
skip; the 8/4 config settles at 8 buffers and never grows). Full suite **295/295
green, 0 warnings**. C4 (compositor Epoch-cached downscale / blit-on-change) was
DEFERRED: the slice-15 approved plan assumed a native-res relocation; the
measurement recast it — relocating a ~30ms float downscale to the render thread
just moves the same cost into the slot budget. Re-measure on device; if render
still >16.6ms slots after capture feeds ≤60 real updates/s, add C4. No push.
- **Slice 17 — the OS readback was the real wall: overlapping conversions + monotonic
publish + deeper pool (2026-09-14, device take ty-1824):** slice 16's downscale fix
landed but the desktop was still ~90% frozen on the 1824 take (band 6.8/s updates, max
freeze 4.85s). The new 2s telemetry was decisive: `conv avg 46-50ms max ~61ms` with
`skip busy 37-83` — the **GPU→CPU readback (`CreateCopyFromSurfaceAsync`), not
`DownscaleBgra`**, is the ~47ms wall (240Hz HDR compositing + encoder + 2 capture
devices share the GPU); at one-in-flight that caps the desktop feed at ~17-20
updates/s, which is the file's whole-frame ~7 content-moments/s. Two facts reshaped
the fix: **(a)** shrinking the pool size does NOT scale the desktop (Microsoft docs:
"If content is larger than the frame, the contents are **clipped**") — readback stays
at native 2560×1440; **(b)** delivery is healthy (60-100 arrivals/s), so the lever is
conversion throughput, not the pool size. Changes in `Services/ScreenCaptureFrameSource.cs`:
conversions overlap up to **MaxConcurrentConversions = 3** (pool deepened to 5 buffers
so in-flight frames fit), each completion publishes ONLY if its Epoch is strictly
newer than the last published (`MonotonicGate` — a slow older completion must never
overwrite a newer LatestFrame), Epoch increments via `Interlocked`, and the
DownscaleBgra row-scratch became per-conversion locals (concurrent callers). The
render side (33-42ms → ~30 unique composites/s) is the NEXT cap after capture speeds
up — that's the C4 slice, queued right after this re-measure. **Good Dog test**
`PublishGate_TryPublish_OnlyStrictlyNewerWins`. Full suite **296/296 green, 0
warnings**. NOT YET DEVICE-VERIFIED; target: telemetry frames/s jumps ≥ ~30-40 and the
band audit drops below ~50% frozen. No push.
- **Slice 18 — C4: the FramePump blit-on-change composite cache (2026-09-15, take ty-1841
verdict):** the 1841 take proved capture fixed (band ~20 fresh updates/s, no tears, pacing
clean) but logged a FramePump **stall on EVERY iteration** (`totalMs 21-44`,
`render=full-render split=0 elements=6 dynamic=4`, worst render 166ms startup spike) →
~22-28 composites/s hard-caps the desktop inside the 60fps file. The SceneGraph split
CANNOT fix this wired: `GetSplitPoint` returns **0** because the live-capture backdrop is
element 0 and must stay dynamic — a baked capture goes stale — so every tick runs the full
re-composite even when no input changed (the compositor's per-layer paste cache already makes
identical frames identical per-element; the waste is re-COMPOSITING the whole 8.3MB frame).
Same shape as OBS (sources cache their surface, the compositor renders on update only —
the pattern this file has cited since the 2026-09-04 paste-cache slice) at the FRAME level.
`FramePump.RenderFull` now wraps both full-render call sites (no-SceneGraph + split=0
fallback): it computes `BuildFullRenderSignature` (a deterministic hash of the crop/output
dims, the social bar identity + top, and — per element, MIRRORING the compositor's own
resolution — the element ref, layout/visual bits, and the frame each element would resolve
through the SAME resolver seam, using buffer-array identity + Epoch + CropBounds; a changed
frame forces a re-composite, an unchanged one never serves stale bytes) and, when it matches
the last composite, reuses the cache with ONE `Buffer.BlockCopy` (~3ms) instead. The cache
buffer is a **separate long-lived array** — never the scratch pool — because the caller burns
the frame counter and recycles the scratch AFTER `RenderScene` returns; the cache is written
from the rendered scratch BEFORE returning (pre-burn, pre-recycle). Engagement is gated on
the 1:1 config (`SourceRect == Output`, the only deployed tier); the vertical tier's cached
fresh-scale-buffer follow-up stays exactly where it was. Telemetry: internal `CacheHits`/
`CacheRenders` counters surface as `cache {renders}R/{hits}H` on the 5s stats line, and the
ticks' key is an internal `OutputIndex` (burned counter, stable across one tick's resolver
passes). **Good Dog test**
`FullRenderCache_StaticInputs_RenderOnce_Then_Reuse_UntilInputChanges`: static scene →
exactly ONE composite (renders==1, hits>0, byte-identical content above the burn strip),
then a new frame (new array + monotonic Epoch) invalidates and propagates. Existing pool test
now passes a STABLE scene (production hands `StagedScene`; a fresh scene per tick churns the
element refs inside the signature and hides the cache — the tests that care about cache
behavior must mirror production) and keys its red/blue alternation on `OutputIndex`/frame
count so the tick's two resolver passes (signature + render) don't double-advance a call-count
flip. Full suite **297/297 green, 0 warnings**; verify.sh scope-locked to FramePump.cs +
FramePumpTests.cs + docs. Target next take: `cache` hits dominate on TV holds, `avg render`
drops toward the BlockCopy, stalls vanish, then the band audit + clap re-measure decide push.
No push.
- **Stop ordering matters:** `StopAsync` stops the encoder — since slice 10 it FLUSHES the pending
queue (`Channel.TryComplete` → drain writes the leftovers, closes stdin → EOF → ffmpeg finalizes+exits;
an accepted frame is never lost) — **before** awaiting the loop. The old reverse-order deadlock was
a submit stuck on pipe backpressure; the drain thread owns that wait now, and stopping the encoder
first keeps the pump's own (non-blocking) submits from racing the completed queue as a spurious drop.
`ProcessFailed` self-stops the pump. `Failed` while live flips
`StreamStatus.Error` (minimal).
- **Health stats (ship step 6, shipped 2026-08-13):** `HealthUpdated` is bound to the bottom bar —
`MainViewModel.OnFramePumpHealthUpdated` marshals to the UI thread (the encoder's stderr loop raises on
a background thread) and copies into `CurrentHealth` (the bottom bar's existing `CurrentHealth.*`
bindings). `ResetHealth(status)` zeroes dropped/duration on go-live and on End so stats never linger
from a previous session; bitrate/FPS stay on the tier's targets (`ApplyStreamQuality`). The bar shows
real encoder values since TASK 9 (2026-08-16) wired the reusable stream URL into `_rtmpUrlProvider`.
- **`MainViewModel` owns the resolver** (`ResolveOutputFrame`): `WebcamSceneConfig` →
`CameraManager.GetLatestFrame(DeviceKeyForWebcam(WebcamId))` — **device-keyed** (configs carry the
identity GUID; sessions key on the Windows device id — the old direct-GUID lookup returned null
forever and the webcam was structurally absent from output; take-2 fix 2026-09-01,
`WebcamOutputKeyTests`), `Source { IsLiveCapture, CaptureKey }` →
`ScreenCaptureManager.GetLatestFrame(CaptureKey)` (the new accessor mirroring `CameraManager`), image/
background → `StaticPixelCache.Get(AssetId)`. `BuildCompositorOptions` rounds the VM's `OutputRect*`
doubles to ints — the vertical 607.5 half-pixel crop rounds to a perfectly-centered **608** (`Math.Round`,
ToEven); `BuildEncoderOptions` fills W×H/FPS/bitrate from the tier once the URL provider yields one.
- **Media source decoder (TASK 21, Increment B — 2026-08-31):** `MediaVideoSource` (`Services/MediaVideoSource.cs`)
decodes a local file to `VideoFrame`s by spawning ffmpeg with
`-f rawvideo -pix_fmt bgra -vf scale=WxH -an` and draining the raw BGRA stdout pipe through the pure
`RawVideoFrameReader` (`Services/RawVideoFrameReader.cs`), raising `FrameAvailable` per frame. Implements
`IMediaFrameSource` (`Services/IMediaFrameSource.cs`: `Key`, `FrameAvailable`, `Completed`, `StartAsync`/`StopAsync`).
The subprocess sits behind `IDecodeProcess`/`FfmpegDecodeProcess` (binary-stdout mirror of the encoder's
`IEncoderProcess`), and the path comes from `IFfmpegLocator` — codec-agnostic, uses the ffmpeg already shipped,
testable with a fake process. Decode sessions are owned app-wide by `MediaVideoSourceManager`
(`Services/MediaVideoSourceManager.cs`): refcounted by `MediaPath`, `Func<string, IMediaFrameSource?>` factory
seam (mirror of `ScreenCaptureManager`), `AcquireAsync`/`ReleaseAsync`/`ReleaseAllAsync`/`GetLatestFrame`,
coalescing each file's frames onto the UI dispatcher onto one shared `WriteableBitmap`. Frames emit as fast as
the pipe produces them. Wired into the live pipeline (slice 1 step 4): `ResolveOutputFrame` reads
`_mediaManager.GetLatestFrame(MediaPath)` for a `MediaSource`, and `Source.DisplaySource`
(`Models/Source.cs`) routes `VideoImageSource` for `MediaSource` so the preview/canvas show decoded video.
Slice 2a shipped the probe seam: `FfmpegFrameRateParser` (pure, `avg_frame_rate=` then
`r_frame_rate=` rational N/N/M, unknown→null) + `IFrameRateProbe`/`FfmpegFrameRateProbe` (derives sibling `ffprobe.exe`
from the located ffmpeg dir, reuses the `IDecodeProcess` seam for the ffprobe subprocess text; null if
ffprobe absent) + `FfmpegLocator.ProbeFileName` now also extracts `ffprobe.exe` from the pinned archive
(conditional — old caches without it just get no pacing). Slice 2b wired pacing: `MediaVideoSource`
takes optional `IFrameRateProbe?` + `Func<TimeSpan,CancellationToken,Task>? delay` (default `Task.Delay`)
seams, probes FPS once in `RunAsync`, and delays by 1/fps after each emitted frame; no probe/unknown → no
pacing (ffmpeg's own pipe backpressure already throttles the decode). Slice 3 added loop control:
`IMediaFrameSource.Looping` (bool); `MediaVideoSource` takes a `Func<IDecodeProcess>` process factory (a single
`Process` can't be re-`Start()`ed, so each loop pass creates a fresh decoder) and wraps the decode in a
`do…while (Looping)` — restart on natural EOF instead of raising `Completed`. Production wiring (MainViewModel
media factory): passes `() => new FfmpegDecodeProcess()` as the factory, `FfmpegFrameRateProbe(new FfmpegLocator(),
() => new FfmpegDecodeProcess())` as the probe.
Still open: session acquisition on add/remove (UI picker) + wiring `Source.MediaIsLooping` into
`IMediaFrameSource.Looping` (needs a manager-level per-path loop provider, comes with the picker slice).
**Acquisition shipped 2026-11-05:** `AddMediaCommand` (`MainViewModel.Media.cs`) opens a video-filtered
`OpenFileDialog` and `AddMediaSource(path)` adds a `MediaSource` layer through the ordinary `AddSource`
path, defaulting to a full-canvas 1920x1080 rect, then `RegisterMediaSource` subscribes the layer and
`EnsureMediaSession` starts one decoder per **distinct path** (`HasSession` guard keeps refcounts at 0/1).
Remove (`RemoveElement`) unsubscribes and releases only when no scene references the file any more;
`ReacquireMediaSources` runs after `LoadLayout` so persisted layers get sessions and stale files are
released (`GetActiveKeys`). Loop is a manager-level per-path answer: the ctor takes
`Func<string,bool>? loopProvider` (production: "any live media reference loops"), applied before
`StartAsync`, and `SetLooping(key, bool)` applies a live toggle. A file swap releases the old path after
the layer no longer references it. **Sound ships with it (2026-11-05):** both `MediaVideoSource` and
`AlertClipDecoder` are now thin wrappers over one `Services/VideoClipPipe.cs` (video pipe + `f32le`
audio pipe, both real-time paced, loop restarts both), so a clip is a clip. The manager applies the
layer's Level/Mute per session and raises `AudioAvailable`, and `MainViewModel` feeds it to
`AudioMixer.EnqueueDesktopMediaAudio` — the DESKTOP/GAME ring, so the desktop fader and the voice
ducker apply. The alert clip keeps its own unity-gain ring (`EnqueueAlertAudio`, never ducked).
**Green/blue screen (TASK 64)** rides the same pipe as a decode option: `VideoFilterChain.BuildVideo`
appends `colorkey=…` + `despill=…` + `format=bgra` after the scale, so the key is computed by ffmpeg
and both consumers (shared preview bitmap, compositor frame) carry the same alpha — no app-side
per-pixel pass, no second render path. A media session is therefore keyed by **path + look**
(`MainViewModel.MediaSessionKey`), so two layers with identical settings share one decoder and two
different looks get two. The offer comes from `ChromaKeyDetector` over one probed frame
(`FfmpegChromaProbe`); settings persist on the `Source` row.
- **Social bar on the output (bar bug-fix branch):** the `FramePump` takes an optional
`socialBar: Func<(VideoFrame? Frame, SocialBarPosition Position)>?` seam, re-read **every frame** (so a
mid-stream position flip applies immediately). The strip is pre-rasterized by `Compositor/SocialBarRenderer.cs`
(WPF glue — WPF glue here is fine because the strip is rendered once per config change on the UI thread,
the resulting immutable frame is then composited pure-CPU by `SceneCompositor`), and `SceneCompositor.Render`
blits it **last — above the branding flash** at `socialBarTop` (0 = top, `SourceRectHeight − barHeight` =
bottom) in master space. `MainViewModel` owns the frame (`_socialBarFrame`, rebuilt by `RenderSocialBarFrame`
on load/save/`NotifySocialsChanged`).
- **Privacy blur layer (2026-11-01):** `SourceType.Blur` is a Live-scene element that blurs the
composed frame inside a rect or ellipse. **It is universal**: `SceneCompositor` applies the blur
group AFTER every layer has been composited (before the flash/social/ticker overlays), so it can
cover the webcam, chat, alerts — anything — which is the creator's "moment of privacy" case. Its
position in the layer list is cosmetic for that reason (it is added at the top), and the earlier
"never above the brand" rule is superseded for blurs: a blur over the logo does obscure it. `SceneCompositor` treats all visible blurs as ONE group the first time one is
reached in the z-order (`ApplyBlurLayers`), sorts them **strongest first** and passes a shared
coverage map to `RegionBlur.ApplyInPlace`, so an overlap resolves to the strongest single blur
rather than a blurred blur. `RegionBlur` shares the webcam's radius curve and constants
(`Downscale 4`, `MinRadius 1`, `MaxRadius 6`, `RadiusFor(strength)`), so a strength means the
same thing in both features; the webcam's `BackgroundEffectProcessor` delegates to it.
Blur is `ElementKind.Dynamic`, so it is never baked into the static base and every render path
(pump, preview, thumbnails, scene dump) gets it from the one compositor.
- Known consideration: the pump reads the active scene on a background thread while the UI can still edit
it; a concurrent-mutation exception is contained (logged + `Failed` + the pump stops) rather than
crashing. The background thread + video pipeline is the new reality since this step.
---
## Related files
This file owns the compositor and the pump. The neighbours live elsewhere:
- [`output.md`](output.md) — **live encoder + RTMP push**, `IFfmpegEncoder`/`FfmpegEncoder`, `FfmpegArgs`, locator and bundle.
- [`TASKS/task-21-media-source.md`](../../TASKS/task-21-media-source.md) — **media source**, `MediaVideoSource` (ffmpeg `rawvideo` decode) feeding the pump's resolver.
- [`overlays.md`](overlays.md) — **social bar** and **native events & alerts**, the overlay strips the compositor blits last.
- [`environment.md`](environment.md) — **Run** and **Tests**, how to launch in Release and how the suite is run.