# Frame pipeline — compositor, render budget, measurement, and the live pump > Split out of [`ai.md`](../../ai.md) (2026-10-26) so a session reads only what its task touches; > `ai.md` keeps a short invariant stub for this file. Directory map: [`index.md`](index.md). This is the subsystem that turns scenes into the encoder's frames: the CPU compositor that renders the master, the budget it must fit inside, the instrumentation that measures it, and the `FramePump` that paces frames into ffmpeg. Everything here is current architecture plus the measured history that produced it; superseded designs are marked inline. --- ## Budget, measurement & the compositor ### Frame cadence — how a dropped frame became measurable (UNIT 0, 2026-10-05) **The problem this exists to solve.** The recorded file cannot distinguish a dropped camera frame from a padded repeat. `-f rawvideo` carries no per-frame timestamps — ffmpeg assigns pts from frame order at the DECLARED fps — so a webcam delivering 25 of its 30 frames still muxes a full-length, correctly-timed 60fps file. Measured on take `ty-20261005-1001-821-0001.mp4` (85.6s, 1080p60): the lower-right webcam region held 25.6 distinct images/s against the camera's 30fps, with 1,153 normal 33ms intervals, 281 gaps of 50–100ms, and 96 bunched 17ms gaps — roughly one hitch in five, which is what makes a face judder while the file looks flawless. (The "webcam renders at ~18fps" figure that used to sit in this map was WRONG: it came from a different session and counted every newly rendered layer, not webcam arrivals. Corrected 2026-10-05.) **`Helpers/FrameCadence.cs`** — one pure counter, three feeds. `Add(timestamp)` on any shared monotonic clock; `Summarize()` → frames, rate (gaps ÷ span, NOT frames ÷ span — N frames span N-1 periods), gaps, hitches (default threshold 50ms ≈ 1.5× a 30fps period), longest and average gap; `Describe(label)` gives every producer the same one-line shape so two lines can be read against each other. An out-of-order stamp counts as a frame but produces no gap AND does not become the reference for the next interval — otherwise one bad stamp invents a multi-second "hitch" twice. **Three measurement points, and what each separates:** | Line | Fed by | Answers | |---|---|---| | `Camera cadence WxH: arrivals … \| camera …` | `MediaCaptureFrameSource` | device/USB/driver cadence (`MediaFrame.SystemRelativeTime`, QPC domain, nullable) vs what the reader handed us. `TryAcquireLatestFrame` keeps only the LATEST frame, so falling behind is silent by design — this is where that shows | | `webcam …` on the pump's stats line | `FramePump.TimedResolver` | distinct webcam `VideoFrame.Epoch` values that reached a composite — what actually got into the output | | `frames per 5s from R renders` | `FramePump` | submits vs REAL composites — the duplicate-on-lag ratio, the only honest reading of how much of the stream is a repeat | **Why the stats-line denominators changed (the actual bug).** Every average was divided by `statFrames`, the number of frames SUBMITTED — which deadline padding always fills to the full 60fps target. A pump fed one webcam frame per 200ms logged "300/300 frames per 5s, avg render 12.4ms" while producing 60 distinct composites: the render average was diluted ~5× and the loss was invisible. Render/resolve/wait are now divided by `statRenders`; submit stays per-frame (it really happens per submit). Regression test: `FramePumpTests.Stats_ReportRealRendersAndDistinctWebcamFrames_NotJustPaddedSubmits`. **Reading the three together:** camera 30/s steady + arrivals 25/s = WE dropped it (reader or compositor). Ragged in both = the device, the USB link, or the driver. Pump `webcam` below `arrivals` = the compositor missed frames the reader had already delivered. **✅ WHAT THE FIRST REAL READING SAID (2026-10-05) — the camera was never the problem.** `ty-20261005-1201-556-0001.mp4` + its log: the C920 delivers **30.0/s with ZERO gaps ≥50ms**, and the device's own stamps are identical, so the camera, USB link, and driver are all clean. The pump did **34 composites per 5s (6.8/s) at ~141ms each** (worst 294ms), and the file holds **6.8 distinct webcam images/s at 88.7% duplicates with no arrival hitches at all**. The judder is entirely us holding one face for 140ms and padding the rest of a 60fps container. `resolve` is 0.1ms, so it is pure pixel work in the composite, not frame lookup. Two consequences, both load-bearing: - **720p is a sharpness win, NOT a smoothness win** — it triples the bytes stretched per render on a path already 4.4× over budget. Schedule the exclusive lock AFTER the compositor work or it makes the judder measurably worse. - **The 10:01 take hit 25.6 images/s and the 12:01 take 6.8, same scene shape (`full-render split=0 elements=6 dynamic=5`), same 640×480 camera.** Composites went 29–34ms → 143–155ms between sessions with nothing logged explaining it. That comparison was made by EYE on two takes because the log carried no per-element breakdown and no scene dump — both now exist (see "Per-element composite timing + scene dump" below); naming the culprit is a reading, not a guess, and guessing is explicitly out of bounds. **Superseded 2026-10-16:** "same scene shape" was an eye comparison of takes whose geometry was never logged, and the next bad take still on disk is the *vertical* tier (below). A Release 1920×1080 take now measures 1.0ms with no element over 0.4ms — see "Per-element composite timing + scene dump" → what the first instrumented take said. **Counter subtlety, so it is not re-derived:** the pump's `webcam` line read 13/s while the file held 6.8/s. Not a bug — the counter stamps a frame when the resolver FIRST sees it, and one composite takes 141ms, so ~4 camera frames arrive during it (signature pass stamps one epoch, the actual draw uses a later one). The file is the truth; the pump counter is a pipeline-health signal and overcounts by design. **Still unmeasured:** the camera is stuck at the C920 default 640×480 YUY2 @30 because all 185 `SetMediaStreamPropertiesAsync` attempts are refused under `SharedReadOnly`, so 720p MJPG is available — but see above, it is a quality win only. ### Per-element composite timing + scene dump (UNIT 0 follow-up, 2026-10-16 — MEASUREMENT ONLY) The gap the cadence work left open is now closed by instrumentation, and closed by nothing else: the 141ms take was never compared to the 30ms take because **no line in the log said which element cost what, and no line said what the scene WAS.** Both now exist. No fix has been attempted, and none is allowed until a real reading names the culprit. - **`Services/Compositor/CompositeTimings.cs`** — a collector owned by the pump, threaded through `SceneCompositor.Render` / `CompositeLayers` / `BakeStaticBase` / `BlitBakedElement` / `BlitCachedLayer` (and `SceneGraph.GetBakedBase` → bake path) as an optional last parameter, so every other caller (thumbnails, tests, transition preview) is untouched and costs nothing. `Add(stage, ticks)` accumulates; `Describe(renders, minMs = 0.1)` renders the averaged breakdown slowest-first, dropping sub-0.1ms noise. - **Stage names** (stable, greppable): `backdrop` (prefill + live capture + background art + their resolves), `base` (the cached-bake `BlockCopy`), `flash`, `social`, `ticker`, `zoom`, `scale`, `overlay` (the fully-static path's per-frame overlay copies), `bake backdrop`, `bake overlays`, and **one entry per element as `# `** — the index IS the z-order, so the log names the layer by both position and name. Bake-pass elements are prefixed `bake #…` so a one-off rebake cannot masquerade as a per-composite cost. - **`# raster` nests inside its element's total** and is the paste-cache rasterize miss (`BlitCachedLayer`'s `raster == null` branch) — the direct answer to "is one element re-sampling every tick, or is the paste itself slow". A stage's time includes that stage's frame resolve; `resolve` is separately totalled in the existing stats line. - **`FramePump parts (N renders/5s): …`** — logged once per 5s window (same window as `FramePump stats`), averaged per render so the numbers compare directly with `avg render`, then the collector is reset. **No parts line, or a short one, is a reading too:** the fully-static shortcut and C4 cache hits composite no elements (`cache 92R/130H` beside a missing parts line is expected). - **`FramePump scene dump: '' elements=N split=S`** followed by one line per element (`[i] name kind=… type=… vis=… rect=x,y WxH op=… clip=…`), logged when the pump first runs and whenever the active scene changes. This is the record the 10:01/12:01 comparison never had. - Gate test: `ytLive.Tests/CompositeTimingTests.cs` (one `[Fact]`) — a REAL composite of a full-canvas element plus a 64×64 one must rank the full-canvas element first, carry the `raster` sub-stage and the fixed stages, and the dump must name both elements with z-index/type/rect and echo the split. - **`scripts/webcam-cadence.py`** — the other half of a reading: what the FILE holds (distinct webcam images/s + hold histogram), beside what the pump says it did. Recipe and its three traps (H.264 re-encoding, the round-clip crop, scene switches): `MyMistakes.md` → "Measuring webcam CAMERA cadence in a recording". **Read takes in Release.** The creator launches `dotnet run -c Release`; Debug pixel paths are several× slower, so a Debug number is not a reading of this pipeline (see [`environment.md` → Run](environment.md#run)). **Derivative check (AGENTS.md):** this is not invented — OBS's own answer to "which source is slow" is the **Source Profiler**, which reports the *average execution time of each source's tick and render functions* ([docs.obsproject.com → Source Profiler](https://docs.obsproject.com/reference-libobs-util-source-profiler), per-source timers via `util/profiler.h`). Same shape here: one timer per element per composite, reported as an average, plus the stage split. The `# raster` sub-stage is the extra this case needs — it separates "re-sampled every tick" from "slow paste" without another session. ### ✅ What the first instrumented take said (2026-10-16, Release) — the judder DID NOT reproduce Take `ty-20261016-1841-559-0002.mp4` (174.5s, 1920×1080@60, Release build `e95154d9` — a **recording**, not preview; scene `Live`, switched to `Chat` at 28s, back to `Live` at 44s) and its log window: | Reading | This take | |---|---| | composites | **301 per 5s = 60/s**, every slot; 0 dropped, 0 stalls after warm-up | | avg render | **1.0–1.1ms**, worst 2.5ms — ~7% of the 16.67ms budget | | `parts` | `#5 HD Pro Webcam C920 0.4ms, #5 … raster 0.2ms, backdrop 0.2ms` — the other four elements are under the 0.1ms print floor | | C4 full-composite cache | **151R/150H** per window — half the ticks were a whole-frame `BlockCopy` | | webcam at the resolver | **30.0/s**, longest gap **50.0ms** (one extra 60Hz frame = sampling phase, not a drop) | | the file itself | **29.6 distinct webcam images/s, 82% of images held exactly 2 frames**, longest hold 267ms (`scripts/webcam-cadence.py`) | Nothing in this take is over budget and **no element is expensive — there is no culprit here to fix.** The two stalls in the take are not per-composite cost: 124ms on the very first render (warm-up) and 53ms on the switch into `Chat` (a rebake). **Scale of the disagreement:** the historic slow readings are 141ms (Oct 5 12:01) and 45ms (Oct 9 19:48) — 40–140× the ~1ms this take needed per composite. Three facts about those takes, all read off the same log: - **Oct 9 19:48 (`ty-20261009-1948-121-0000-2.mp4`) was the VERTICAL tier**, not a 16:9 one: `FramePump started (1080×1920 @ 60 fps) scene='Live' comp=608x1080` — the 607×1080 centre crop bilinear-upscaled to 1080×1920 (2M px), 92 renders/5s, avg render 45.0ms, worst 278ms, 92 stalls, `cache 92R/0H`. The file on disk is 11.6s and every frame is identical (a static desktop), so it cannot be read for cadence — only its cost. - **Oct 5 12:01 (`…1201-556-0001.mp4`, no longer on disk) logged `FramePump started (1920×1080 @ 60 fps)` with NO `scene=/comp=` suffix** — that suffix did not exist yet, so its source rect was never recorded and the map's "21:9 → 1920×803" explanation stays a hypothesis. - **A take can latch mid-take.** Oct 5 12:29 goes 1.5ms (avg render) → 3.0ms → **45.2ms within 15 seconds and stays there** (`cache 10R/0H`, then `98R/0H`). So the slow state is not a fixed property of a scene geometry. The consistent signature of every slow window is `cache xR/**0H**`: the C4 whole-frame cache never reused once, so every tick composited. The healthy take shows the other stable state (composite 1ms → every second tick is byte-identical → whole-frame reuse). **The latch is the thing to test on a reproducing take** — it is a hypothesis, not a reading; the `parts` line is what will decide it. **What this does NOT say.** This take is the 1:1 master tier and its scene logs `full-render split=0 elements=6 dynamic=4` (the bad-period takes logged `dynamic=5`), so it never exercises the failing geometry. "The healthy path is healthy" is not "the judder is gone". Reproducing it needs a Release take in the failing configuration (vertical tier first, then the 21:9 display config) — that take's `parts` line names the stage, and only then is a fix unit allowed. **Reading the `parts` line's denominators:** like `avg render`, it is divided by `statRenders`, which counts C4 REUSES as well as real composites (the 301 above = 151 real + 150 whole-frame copies). So in a healthy 30fps-camera take both averages are ~2× optimistic; the `cache NR/NH` pair on the same line says how much of the window was real composite work. `CompositeTimings.Describe` drops stages under 0.1ms by default, and **a missing or short parts line is itself a reading** — a fully-static window (every tick a C4 hit) composites no elements at all. ### Render budget — getting a composite inside 16.67ms (UNITs 4-5, 2026-10-05) The pump emits one encoder frame per 16.67ms slot and **pads with a duplicate when a composite overruns**, so the whole problem is "render + submit + untimed work ≤ 16.67ms, ideally ≤12ms to leave headroom". `FramePump` now reports per-stage timings (`social`/`ticker`/`flash`/`acquireScratch`) so a slow stage names itself, and logs the scene name + compositor source rect at pump start. **Why a 21:9 monitor cost 141ms and a 16:9 one 33ms — the fast path is EXACTLY 1920×1080.** A 21:9 display shrinks the source rect to 1920×803 ("fit", not "fill"), so the opaque-backdrop 1:1 row walk is skipped, the base is prefilled with opaque black (~2M byte writes), and the backdrop is resampled per-pixel (~1.54M pixels with an alpha branch). Every recorded A/B lines up: 21:9 → 141ms, 16:9 → 30-33ms, 2560×1440 (fits to 1920×1080 exactly) → fast path. **Corrected 2026-10-16 — the "never Debug-vs-Release" half of that is withdrawn.** A Release 1920×1080 take measures **1.0–1.1ms** where an Oct 9 take of the same tier measured 10.4ms, and `Build ` carries no build configuration, so the config of every pre-2026-10-16 take is UNKNOWN: no historic render number may be compared against a Release take. The 21:9 explanation itself stays a hypothesis — the 141ms take's start line predates the `scene=/comp=` suffix, so its source rect was never recorded. A take can also latch mid-take (see the instrumentation section). **Unit 4 — cheap wins, all pixel-identical:** - Opaque 1:1 fast paths expanded to the whole-canvas opaque case (a `BlockCopy`, not a per-pixel alpha branch). - **Prefill only the uncovered rows.** When the backdrop does not cover the canvas, the letterbox bands are filled instead of the whole buffer; with no backdrop at all it still fills everything. - **Per-column LUT** for the scaled blit (`BlitContentRaw`): the source column, its neighbour index and the 8.8 weights are computed once per column instead of a divide + clamp per pixel. The table comes from `ArrayPool.Shared` and is returned in a `finally` — a single static buffer is a **data race** the moment strips run in parallel (see below). **Unit 5 — strip-parallel composite: `Helpers/StripPool.cs`.** One composite splits into four horizontal row strips; the calling thread takes a strip too, so four workers never means five-way oversubscription. Below 64 rows the fan-out costs more than it saves and runs inline. Each strip must write only its own rows — the blits already partition by row, which is why the output stays byte-identical. **The demotion must come from DEDICATED threads, not the shared pool.** The obvious `Thread.CurrentThread.Priority = BelowNormal` inside `Parallel.For` is wrong: `Parallel.For` runs on shared ThreadPool threads, so the demotion leaks into every unrelated work item that thread later picks up. Research (https://stackoverflow.com/questions/524483/net-backgroundworker-objects-thread-priority — "you have next to no control of which thread pool thread will actually run your task"; https://stackoverflow.com/questions/1578599/c-sharp-lower-thread-priority-in-thread-pool — "You shouldn't mess with the priority of the regular ThreadPool, since you aren't the only consumer ... write your own producer/consumer queue, using your own Thread(s) as the consumer(s) - setting the thread priority yourself") says exactly that, and **a real failed attempt confirmed it**: a full test run after demoting the pool threads broke an unrelated audio-mix timing test. `StripPool` IS that producer/consumer queue — four `Thread`s with `Priority = BelowNormal`, `IsBackground = true`, started lazily on first use, each fed a `Queue` + `SemaphoreSlim`. Four is the measured sweet spot: fewer leaves render over budget, more steals cycles from the game and the encoder and made render **worse**. Gate test: `SceneCompositorTests.StripParallel_RepeatedRenders_AreByteIdentical` — the same scene (scaled round webcam + mirrored image + master-sized flash over a short backdrop, 240 rows so all three fanned-out paths run) must come out byte-for-byte identical across 8 renders, and must differ from a backdrop-only render so it cannot pass vacuously. It earned its keep immediately: it caught a loop-variable capture (`new Thread(() => Worker(i))` where `i` was `Workers` by the time the thread ran) that crashed the test host on first run. ### Scene compositor (TASK 4 ship step 1 — shipped 2026-08-10, plan in TASKS.md) The encoder needs the master 1920×1080 frame **without** the preview's editing chrome (SelectionOverlay, DimRects, output-rect outline, badge, placeholder). WPF's `RenderTargetBitmap` is software-rendered and captures the visual tree *including* chrome, so the preview can't be captured — the output is a **second, parallel software compositor** over the `VideoFrame` (BGRA8) seam, and the XAML preview (`MainWindow.xaml` CanvasGrid + element DataTemplate) is the rendering contract it replicates. Two renderers must agree: geometry, `UniformToFill` cover-crop, round clip, mirror, border, z-order. Preview stays XAML (editing view); the compositor is the output view. - **The `DimRects` shading is itself preview chrome, so it renders as a SIBLING of `CanvasGrid` (above it, below the `ZoomBox`), never a descendant** (2026-10-08, creator ruling): the zoom box's `VisualBrush` samples all of `CanvasGrid`, and if the dims were inside it the box would show the dim shading — especially after the box is dragged away and the fixed capture slides under the dims. - **Composite the active tier's crop at crop size** (`CompositorOptions {SourceRectX/Y/W/H, OutputWidth, OutputHeight}`, fed from `MainViewModel.OutputRect*`): 16:9 = full 1920×1080; the vertical 9:16 tier = centered 608×1080 crop. FFmpeg's `scale` filter resizes the raw BGRA frame to the selected output tier after it leaves the frame pump; `setsar=1` keeps square pixels. - **CPU posture:** with FFmpeg as a subprocess the master crosses a CPU readback to the pipe every frame anyway, so GPU compositing buys little at this layer count (2-3 live layers; static layers pre-composite once into a cached base). GPU effort belongs to **NVENC** (the encoder), not composition; if composition grows (wipes, filters, many layers), a D3D11 compositor can replace this one **behind the same seam** — the CPU master buffer stays the contract. - **Every global overlay must be composited on EVERY render path (bug found 2026-09-26).** The frame pump has three exits: `RenderFull` (no graph / no static base), `CompositeLayers` (static base + dynamic tail), and the **fully-static shortcut**, which stretches the cached bake and returned it directly. That third path silently **discarded every per-frame overlay** — the social bar, the alert ticker and now the brand flash. A creator whose scene was one static background saw none of them, and the flash looked "preview-only forever" for a reason unrelated to the tier. `SceneCompositor.Overlay` is therefore `internal static` (it copies before blitting, so the cached bake is never mutated) and the shortcut now composites overlays onto a copy before stretching. **When adding an overlay, grep for all three paths** — a fourth exit that forgets it fails the same silent way. - **`VideoFrame` buffers are recycled by some producers, and `Epoch` is not optional.** A producer that hands out the same array across frames MUST bump `Epoch`, or `FramePump`'s buffer-identity paste cache false-hits and freezes stale content (the take-11 class of bug). The `C4` cache signature mixes `Epoch`, so a per-frame overlay naturally invalidates it. `BrandFlashPresenter` reuses one 8MB master buffer and stamps `Epoch` per read for exactly this reason — allocating a fresh 8MB frame per tick is not an option at 1080p30. - **Dev-only: two instances side by side (`Helpers/InstanceProfile.cs`).** Pre-1.0 the creator runs one instance to stream and another to screen-capture it. Nothing prevented a second instance (no mutex, no port); what broke it was shared state — the whole-scene read/write layout DB (clobber), Chromium's **exclusive** lock on the WebView2 user data folder (second instance won't start), the auth token store (a test instance overwrites the real YouTube sign-in), and `startup.log`. Set `YTLIVE_INSTANCE=` and that process gets a private root at `%APPDATA%\ytLlive\instances\\` for the DB, the auth file, the log and the WebView2 folder. Recording folder and the ffmpeg `tools\` cache stay shared on purpose; global hotkeys stay un-namespaced (Windows refusing the second `RegisterHotKey` is the correct answer). The **entire implementation is inside `#if DEBUG`** — a Release build compiles to `DataRoot => DefaultRoot` and `WebViewDataFolder => null`, and the call sites are unconditional so Release cannot drift. Proof + harness: `ytLive.Tests/InstanceIsolationTests.cs`. ⚠️ Grepping the binary for `YTLIVE_INSTANCE` proves nothing — a `const` is inlined and appears in neither build; check the `InstanceVariable` **field** in metadata instead. - **Branding flash is composited by the output path too** (it's on the live output, per Monetization), passed in as a pre-rendered `VideoFrame?` — the compositor core stays pure byte-math, no WPF. Likely a bundled asset rather than runtime text rendering (deterministic, no font/layout risk). - **Frame sources are injected** via a `Func` resolver (`SceneCompositor.Render(scene, frameFor, flashFrame, options)`) — the caller maps each element to its frame (webcam → `DeviceId`, image → `AssetId` via `StaticPixelCache`, backdrop → `CaptureKey`), so the compositor is pure, WPF-free, and hermetic to test. The capture managers wire into that resolver in the encoder step, not the compositor step. The master buffer (the compositor's return value) is the seam a future D3D11 compositor would honor identically. --- ## The live pump — frame producer to encoder ### Live frame pipeline (TASK 4 ship step 5 — shipped 2026-08-12, health stats 2026-08-13, plan in TASKS.md) The **`FramePump`** (`Services/Encoder/`) is the live frame producer: while live it snapshots the active scene each tick, resolves every element to its latest frame, composites it into the tier's output frame, and paces frames into the encoder at the tier's FPS. **rawvideo is stamped by ARRIVAL:** ffmpeg assigns pts from frame order at the declared fps — supply rate = output speed. A starved producer (pump < fps) ships a time-lapse, truncated file with NO error (take-2 lesson, 2026-09-01); the pump therefore logs `FramePump stats: n/target frames per 5s from R renders, avg render Xms (resolve R2), avg submit Yms` so a slow stage names itself (the `frames … from R renders` pair and the render-count denominators arrived with UNIT 0 — see "Frame cadence" above; before that every average was divided by the SUBMITTED count, which deadline padding always fills to the full target, so a pump fed one webcam frame per 200ms still logged "300/300 frames" and its render average was diluted ~an order of magnitude) — and since 2026-09-04 so does the BUILD: every build gets a GUID stamped into `Helpers/BuildStamp` (csproj `GenerateBuildStamp` target, fresh per compile — no lying incremental build), logged at startup ("Build xxxxxxxx (compiled ...)"). **Take 6 (stamp-less, ambiguous): render 35-41ms with slice 3 built or not unknown — attribution is why the stamp exists; takes 7+ are readable.** If the breakdown says resolve dominates, the suspects are pre-cached chat/config thrash or the web frame; if blit dominates, the general-path element sizes. **Pattern — everything is a constructor-injected seam:** `Func`, `Func` resolver, `Func`, `Func`, `Func`, `Action` log, and an injectable pacing delay (default `Task.Delay`; tests inject `Task.Yield`). The pump is free of WPF and of the capture managers. - **`StartAsync` never throws** — the VM fires-and-forgets it from the sync command handler; failures log + surface via the `Failed` event. `BuildEncoderOptions` now always returns options (TASK 18) with `StreamEnabled`/`RecordEnabled`/`RecordPath` baked from the pills + `_activeRecordPath`; the pump skips the encoder only when `options == null`, i.e. "no output configured". `MainViewModel._rtmpUrlProvider` is that seam — a `Func` that now yields the reusable stream's ingest URL (TASK 9, shipped 2026-08-16): loaded from the `LayoutStore` cache at startup and set fresh by `PrepareAndStartLiveAsync` before the pump starts. **The pump reads the URL once at startup**, which is why go-live ensures the stream BEFORE `StartAsync`. - **Deadline pacing + row-blit render (take-3 starvation fix, 2026-09-03):** take 3 recorded 30s into a 2.1s file and the 5s stats line named the cause in one number — `17/300 frames per 5s, avg render 258.1ms, avg submit 1.5ms`. Two defects, both fixed: (1) the pump slept the FULL interval after each render, so period = render + interval — OBS's `video_thread` (libobs/media-io/video-io.c) pattern replaces it: absolute `nextTick += intervalTicks` deadline, sleep only the remainder, and on overrun skip the wait. (NOTE, slice 9: the original "skip the missed ticks (rebase)" half of this fix was wrong — see the slice 9 correction below.) (2) the compositor did per-pixel float sampling + `Math.Round` blending over all 2.07M master pixels (backdrop) and scanned the whole destination per overlay (a 64px social-bar strip cost 2M iterations). `SceneCompositor` now follows the libyuv pattern (BSD-3, chromium.googlesource.com/libyuv/libyuv — cited per the derivative-work rule): 1:1 aligned blits take a row-walk fast path (bilinear at scale 1 + integer offset is the identity) with per-pixel alpha branch and integer fixed-point blend; `BlitOverlay` clips to the intersection rect; a full-cover live backdrop skips the opaque-black pre-fill. The general per-pixel path handled scaled/round/mirrored elements. Test: `Pump_Paces_To_The_Deadline_Compensating_Render_Cost` records the requested wait via the pacing seam (a seam fake must genuinely await — a synchronous completed task runs the whole pump loop on `StartAsync`'s continuation and hangs the run; MyMistakes). - **Slice 2 of the render fix (2026-09-04, take 4):** the pacing held (file no longer truncated at loop level) but render stayed at 58.9ms — the per-pixel row walk was still 2M managed iterations and every tick allocated a fresh 8.3MB master buffer. Three changes: (1) **`VideoFrame.IsOpaque`** — a producer-contract flag (only the screen-capture and webcam paths set it; DWM/MediaCapture fill alpha 255 by contract); a full-canvas, aligned, opacity-1 blit of an opaque frame is now ONE `Buffer.BlockCopy` (~1.5ms) instead of the loop, and the black pre-fill is skipped when the backdrop covers. (2) **integer fixed-point bilinear** in the general `BlitContent` path (row-hoisted invariants, no divisions, no `Math.Round`) — within ±1 of the float reference, inside the ±2 test tolerance. (3) **scratch pool in the pump** — `AcquireScratch`/`ReleaseScratch` recycle the master buffer (max 4, keyed by length, owned-by-reference so bake-cache/social-bar/static-art arrays can never be captured); release happens strictly AFTER `SubmitFrameAsync` returns (the write to stdin copies), and the free-list Contains guard makes the transition Cut path safe (`BlendFrame` returns the to-frame itself, aliasing the scratch). The per-tick `fromScene` render in the transition branch was **dead weight** (BlendFrame uses `TransitionService.FromFrame` captured at `Start`, never the pump's) — removed, and the pump's `fromSceneProvider` seam + `MainViewModel` call site went with it. Tests: `Pump_Pools_ScratchBuffers_Across_Frames_Without_Stale_Pixels` (the ONE: alternating backdrop colors pin every frame's content, repeated backing-array identity proves the pool recycles), `Composite_OpaqueFullCover_Backdrop_CopiesEveryPixel_Into_Scratch` (memcpy branch + scratch sentinel). `FakeEncoder.SubmitFrameAsync` now snapshots bytes like the real stdin write — holding the reference would race legitimate recycling. Take 5 must show `avg render ≤ ~10ms, ≈300/300`. **Vertical/output scaling moved to bundled FFmpeg (2026-11-04):** the C# compositor no longer enlarges the 608×1080 crop to 1080×1920 on every frame. The bundled FFmpeg `scale` filter processed 300 synthetic BGRA frames in 0.447s on this machine; that is a scaler-only microbenchmark, not a live-stream result. - **Slice 3 — chat raster-on-change (2026-09-04, take 5):** take 5 measured `138/300, avg render 25.5ms` — the per-tick blits were cheap now, but `ResolveOutputFrame` → `RenderChatBox` ran a FULL WPF raster (`ChatBoxRenderer`: FormattedText + `RenderTargetBitmap` + CopyPixels + channel swap) EVERY tick whenever the message buffer was non-empty — and the buffer survives between sessions, so even a signed-out record-only take paid it. Established answer, same as OBS text sources: **raster on change, blit the cache every tick.** `ChatOverlayLayer` now keeps a content version (`Messages.CollectionChanged` → `_contentVersion++`) plus a config key (box size + all Chat* props); `RenderFrame` returns the cached `VideoFrame` by identity until either changes (the compositor only ever reads a cached frame). Test: `ChatOverlayLayerCacheTests` (RealApp, real renderer — `Same()` for unchanged inputs, `NotSame()` on message/config change, null on empty). Accepted cost pending take 6: one slow tick (~15-25ms) per arriving message; if chat-burst frame loss shows up, the next slice moves the re-render off-tick (debounced, dispatcher-side). - **Slice 5 — paste cache for non-opaque layers (2026-09-04, take 7/8 data):** the render/resolve split (added in the stamp commit) finally named the last thing honestly: `resolve ≈ 0` but `render 26-27ms` (124-135/300, ≈2.2x) — the chat fix HAD worked; the remaining cost was the compositor re-rasterizing EVERY layer per tick even when its pixels never changed (this creator's Live scene: chat 159k + web widget 271k + image 95k + webcam 156k px ≈ 680k samples @ ~38ns each). OBS's actual shape: sources cache their surface, the compositor pastes. `SceneCompositor` now routes non-opaque layers through `BlitCachedLayer`: a layer rasterizes ONCE into an element-space, TRANSPARENT-based frame keyed by (source-array identity, source W×H, ceil'd dst rect, round, mirror), then every later tick PASTES it (integer position, row alpha-blend, opacity applied at paste). **Correction (2026-09-12, the web-widget "black box"):** the raster builds on a TRANSPARENT base, but the sampler's PARTIAL-alpha branch used the opaque-dst blend — it premultiplied the color into RGB and forced `alpha=255`. A translucent widget pixel then pasted as opaque darkened ink (the box in recordings) while the raw-bitmap preview stayed correct. `BlitContentRaw` now takes `transparentDst` (the raster call passes `true` and writes straight color + straight alpha; the paste rows do the source-over). Master paths are bit-identical. Guard: `PasteCache_SemiTransparentLayer_RevealsBackdrop_NotOpaqueInk`. Producers hand out fresh immutable arrays, so array-identity keys can never serve stale content; dict bounded at 48, cleared wholesale on overflow. Only changing content (webcam device frames, web capture ticks, chat messages) resamples; the opaque backdrop keeps its memcpy path. Position/opacity drags are now near-free (no resample — paste params, not cache keys). ONE integration test: `PasteCache_RepeatRender_IsByteIdentical_And_ContentChangePropagates` (raster-vs-paste byte equality incl. round-clip margins + new-array propagation); the whole existing pixel suite guards sampler semantics. Take 9 must show `≈300/300, avg render ≤ ~8ms`; if it lands there, the recording saga closes. Follow-ups unchanged: vertical-tier scale alloc, debounced chat re-render under message bursts. - **Slice 6 — the sleep quantum WAS the ceiling (take 9, 2026-09-04):** period measured ~37ms while work (render+submit) was ~25 — the missing ~12ms is `Task.Delay` rounding EVERY request up to the Windows clock tick (documented ~15.6ms default; learn.microsoft.com/en-us/dotnet/api/system.threading.tasks.task.delay). A frame finishing 3ms early requests a 3ms wait and sleeps a full 15.6 — capping the producer at ~27fps NO MATTER how fast the compositor ran. This is why takes 7→9 showed zero playback change despite real render wins: the sleep floor dominated everything above it. Established fix (game-loop canon — stackoverflow.com/questions/5441464; timeBeginPeriod — learn.microsoft.com/en-us/windows/win32/api/timeapi/nf-timeapi-timebeginperiod): `timeBeginPeriod(1)` for the pump's lifetime (paired `timeEndPeriod` in the finally), sleep only the BULK of the remainder (request minus 2ms), `Thread.SpinWait` the last ~2ms across the deadline; blown deadlines still rebase. (SUPERSEDED in part: slice 9 deleted the rebase — deadlines only advance; slice 10 replaced the burst re-write with one fresh composite per iteration.) Stats gained `avg wait Xms` so render+submit+wait must ≈ period — the accounting is closed, no stage can hide again. Same slice: the webcam's `IsOpaque` paste-cache bypass was removed (it re-sampled ~156k px every tick even between identical device frames; the cached paste beats the sampler on hits and costs the same on misses). Take 10 verdict: `≈300/300` honest fps — if short, the wait/render split names the remaining term with no ambiguity left. - **Slice 7 — the loop was ON the UI thread the whole time (take-8 numbers, 2026-09-04):** with the wait finally measured, take 8 (59a02a5b) showed the impossible pair — `render 22ms, wait 10ms` against a 16.7ms deadline: a blown deadline rebases with NO wait. The wait was the producer **queued behind the live preview on the dispatcher**: `PumpAsync` starts from a UI command handler and its `await` continuations inherit the UI `SynchronizationContext`, so the "WPF-free" compositor actually rendered on the UI thread and ran only when WPF let it. OBS runs its media loops on dedicated threads for exactly this reason. Fix: `_pumpTask = Task.Run(() => PumpAsync(...))` (no sync context inside → continuations stay on the pool). Side effects handled, not ignored: `StaticPixelCache.Get` now locks (pool miss-decodes race UI callers); `ChatOverlayLayer.RenderFrame` checks its cache off-thread but MARSHALS the rare raster miss to the dispatcher (DrawingVisual/ RenderTargetBitmap are UI-thread objects) and re-validates there; pump events already marshalled. Also: `GCLatencyMode.SustainedLowLatency` for the pump's life, stats gained `worst render Xms` (spike visibility — bimodal averages hid them), webcam now routes through the paste cache (bypass re-sampled 156k px even between identical device frames). Tests: `Pump_Produces_OffTheStartingContext` (the ONE — inline-pumping SyncContext proves continuations never return to the starting thread) + 70/70. Take 9 verdict: wait ≈ true remainder (period → 16.7, n → ~300); `worst render` names any remaining raster-miss spikes; render+submit+wait still == period — the accounting holds. - **Slice 8 — capture buffer ring + paste-cache Epoch (take 11, 2026-09-04):** take 11 validated the architecture — typical frames now land `work ~10ms + wait ~6.8ms = 16.7`, exactly the deadline (212/300). The whole remaining gap is PERIODIC 35-65ms worst-render spikes that worsened across the take (189→147 frames/5s) — the signature of gen2 GC pauses, now provable via `gen2 +N` in every stats window. Biggest churn was structural: `ScreenCaptureFrameSource` minted a fresh ~8.3MB `byte[]` per DWM frame (~500MB/s LOH). It now rotates a shared-frame ring (4-deep at slice 8, deepened to 8-deep after the "flash" finding — consumer holds must never outlive depth × source period), size-matched per slot, with reused downscale row scratch. Recycled arrays would poison the paste cache (it keys on array IDENTITY), so `VideoFrame.Epoch` — monotonic per producer frame, 0 for fresh-array producers — joins the key. Regression test `PasteCache_RecycledArrayWithNewEpoch_ReRasterizes_NotStaleHits` fails on the old key by construction. The camera producer carried the same fresh-array churn (110-220MB/s at 30-60fps): it now rotates its own 8-deep ring + Epoch (same identity rule), and the WebView2 capture reuses a canvas scratch + 8-deep output ring + a reused WriteableBitmap instead of minting two fresh arrays + a new bitmap per 10Hz tick. Next suspect if gen2 stays hot: the WPF preview load itself driving gen2 — recorded as follow-up, untouched this slice. - **Slice 9 — the "rebase" WAS the recording time-lapse (2026-09-10, the recording playback-timing take):** slice 6's "blown deadlines still rebase" was the recording-timing bug all along. In a real take the pump emitted `215-219/300 per 5s` (~43fps) while `render+wait == period` still looked closed — the rebase ERASED every missed slot (deadline = wall-now again), so the missed ticks never showed in the stats. rawvideo has no per-frame timestamps: ffmpeg muxes by frame count at `-framerate 60`, so a 43fps reality was authored into a 60fps container and **every recording played ~1.4x fast** (takes confirmed with a WSL ticker visible in the recording: file duration 11.44s vs ~15.5s wall). Two defects, both fixed the OBS way (`libobs/media-io/video-io.c` — the video thread NEVER resets its deadline; every interval tick outputs ONE frame, and a late render means repeated content — judder — never a skipped timestamp): (1) **count-based CFR emission:** `while (GetTimestamp() >= nextTick) { SubmitFrame(frame); nextTick += intervalTicks; statFrames++; }` submits exactly one frame per crossed slot, re-writing the current composite when the render overruns (duplicated footage = correct duration, not a time-lapse), and `nextTick` is NEVER reset to wall-now; (2) **`-re` removed from `FfmpegArgs`** — it was a second, fighting pacer on the rawvideo demux (its "Resumed reading … after a lag" grew 0.79s→4.82s across the take, leaving the pump behind its own honest-stats count). One fix covers the game background AND the webcam — both flow through this one pump. Muxed duration is now frame-count ÷ fps == wall time by construction. Test: the pump suite stays green unchanged — the pacing test's `` (cap 120 ≈ 2s at 60fps) owned by a dedicated drain task with the stdin write + flush; the caller NEVER blocks on the pipe. Pixels are copied into an `ArrayPool` buffer before enqueue (the write now happens later on the drain thread, so the pump's scratch reusable the moment submit returns — same contract as before). (2) **drop-on-overflow (creator's choice: freshness over coverage):** when the queue is full the NEWEST frame is dropped and counted (`IFfmpegEncoder.DroppedFrames`, `Interlocked`); the session never freezes or smears — it drops. Stop flushes the whole queue then EOF (`Channel.TryComplete` → drain writes leftovers → closes stdin → ffmpeg finalizes+exits), so no accepted frame is ever lost at stop. (3) **pump loops ONE submit per iteration** (replacing slice 9's burst): every due slot gets a FRESH composite — no catch-up slot ever repeats frozen content; missed slots vanish as a count-based gap. The deadline counter is still never reset to wall-now (slice 9 ruling). (4) **burned-in frame counter (the new judge):** `_outputIndex` is burned into a 6-digit dot-matrix strip (white box + black 5×7 glyphs, bottom-right corner) of EVERY composite before submit. The WSL ticker is demoted because its own timers smear under Windows host load; decoding the recording and reading the strip is clock-independent: the number advances +1 per frame and jumps by exactly the counted drops (queue overflow OR skipped catch-up slots). Stats gained `worst submit Xms`, `dropped N`, `stalls K`; a stall logger names any iteration > 2× interval with its render/submit split — with the queue, submit is ~1ms, so a stall means render/resolve. The strip sits above the social bar (bar is composited, then burned over) — a debug judge, tiny at 1080p. Test: `Backpressure_QueueOverflow_DropsFrames_AndNeverBlocks` (the ONE: a 40ms-per-frame sink fake makes every enqueue fill the queue; submit must return instantly, drops are counted, stop flushes exactly submitted − dropped bytes). Full suite 289/290 (the pre-existing compositor pixel failure unchanged). Audio untouched (follow-up). WSL ticker-under-load reliability check still to run (informational — the burned counter is the judge). - **Slice 11 — web-layer animation ran at ~1/6 speed; capture cadence 10Hz→~30Hz + de-throttle (2026-09-10, the "widget animates too slow" take):** the recording is 60fps but the WebView2 capture loop was a blind 100ms `DispatcherTimer` = 10Hz — a 60fps-designed widget was sampled 6× under its native rate (repeated-footage slow-mo). Two stacked throttles: > Superseded by **Slice 14** (below): `CaptureScheduler` and the cadence hooks were the polling > design; frame-driven composition capture replaced them — nothing here is current architecture. (1) **the 10Hz cap** (unconditional, code-proven) and (2) **Chromium hidden-page throttling** (`requestAnimationFrame` parked, JS timers clamped to ~1s per WebView2Feedback#1172/#3070 + Chrome-88 timer throttling) whenever the app window is unfocused or covered — the page is off-screen at (-5000,-5000), so it is a hidden page the moment the host window loses occlusion. Fixed: (1) **`CaptureScheduler` (new, `Services/CaptureScheduler.cs`)** replaces the per-session `DispatcherTimer`: a dispatcher timer that DROPS ticks while a capture is in flight (latest-wins, never queues) — the in-flight drop is what made raising the cadence safe (concurrent full-HD PNG `CapturePreviewAsync` calls would stack ~10-30ms encodes and publish stale-after-fresh). Interval is owner-configurable: recording 33ms (~30Hz), idle 200ms. Unit-tested without a WebView2 runtime (the ONE test: `CaptureScheduler_Drops_Ticks_While_Capture_InFlight_And_Resumes` — overlapping ticks dropped, capture resumes when idle, driven by a TCS so it is deterministic). (2) **de-throttle browser args**: the shared `CoreWebView2Environment` is created with `--disable-backgrounding-occluded-windows --disable-renderer-backgrounding --disable-features=CalculateNativeWinOcclusion` (the Electron/Streamlabs-class embedder answer for occluded-window animation throttling) BEFORE `EnsureCoreWebView2Async`. (3) **cadence hook**: `MainViewModel` sets `SetCaptureInterval(33)` on record/stream start and `(200)` on stop (`MainViewModel.Streaming.Operations.cs`). (4) **capture-cost telemetry**: first 30 captures per session log elapsed ms to startup.log — PNG-encode+decode cost decides whether ~30Hz stays or drops to ~20Hz; the FramePump drops frames (never time-lapses, slice 10) if UI-thread GC churn starves it, so the cost is observable. Full suite 290/291 passing, the sole failure the pre-existing compositor pixel test. The web-overlay transparency verification (the first-capture diagnostic bound to about:blank) and the audio-silence item remain open push-gate items — both untouched by this slice. - **Slice 12 (audio diagnostics, 2026-09-10):** the recorded audio is full-length silent AAC (−91dB, 1124 frames / 23.95s in the latest take) — the named-pipe delivered ~9.1MB of zeros to ffmpeg for the entire recording, so the loop ran and connected, but both WASAPI capture sources delivered nothing. Sources started fine (`Audio: using system default mic...` logged), no failure callbacks fired, and the mixer's existing integration test (`Mix_WithFiltersDuckAndGain_Lands_On_AudioPipe`) proves the loop→pipe math is sound. The fault is capture-side: either the default render endpoint carried nothing (audio played on a non-default device — common), or both endpoints were held exclusive, or genuinely nothing played. Added permanent per-5s live-loop telemetry to startup.log (`Audio live: pipe connected=, dropped writes=, micLevel=, loopLevel=, drained, peakMix`) and `NamedPipeAudioWriter.DroppedWrites` — names the exact stage on the next take without new code. `AudioMixer.FillAndMix` now returns `(MicRms, MicDrained, LoopDrained)` for the telemetry accumulation. Full suite 290/291, same pre-existing sole failure. - **Slice 13 (web transparency diagnostic re-point, 2026-09-10):** the "?" in MyMistakes' "RESOLVED (? VERIFY)" is being closed. The one-shot widget dump + alpha stats bound to the FIRST capture ever = the initial about:blank document — a blind instrument. NavigationCompleted for the REAL widget URL now arms `WidgetDumpRemaining = 5`; those 5 post-paint captures dump `%TEMP%\ytLive-web--w1..5.png` + the shared `AlphaStats(...)` line (alpha[min/max/mean/zero%]) + FindContentBounds result to startup.log. That line alone decides the fix branch: zero% ⇒ the pre-parse injection did not hold for this widget (opaque capture → chroma-key / stronger DOM injection); large zero% + tight contentBounds ⇒ the capture IS transparent and the recording's black block lives downstream (compositor blend/underlay). `AlphaStats` extracted as the shared sampler. No new tests (WebView2 runtime not instantiable in tests; the re-point is log-only). - **Slice 14 — web frames are now COMPOSITION-CAPTURED; the PNG poll + `CaptureScheduler` are gone (2026-09-14, the "still not 60fps" take):** slice 11's ~30Hz was still a `CapturePreviewAsync` PNG poll — every 1920×1080 full-HD encode+decode cost 35–165ms, so the session budget capped real cadence at ~20Hz for a 60fps-designed widget. Root fix = frame-driven capture, not faster polling: the WebView2 renderer now feeds `Windows.Graphics.Capture` directly through a `CoreWebView2CompositionController` (the mechanism `WebView2CompositionControl` and Flutter's `webview_windows` use — see `graphics_context.cc` `CreateGraphicsCaptureItemFromVisual`, which captures the root `surface_` visual). - **Pipeline (per session):** `CreateCoreWebView2CompositionControllerAsync(WindowHandle)` — the parent HWND must exist, so `MainWindow.InitWebView2()` moved from the ctor to **Loaded** (was `InitWebView2(WebViewHostPanel)`; the hidden XAML `WebViewHostPanel` overlay is deleted). `controller.RootVisualTarget` = a child `ContainerVisual` (`RelativeSizeAdjustment = 1,1`) under a root `ContainerVisual` (1920×1080, `IsVisible=true`) built on ONE `Windows.UI.Composition.Compositor` (created once on the UI thread); `Bounds = 0,0,1920,1080`, `BoundsMode=UseRawPixels`, `ShouldDetectMonitorScaleChanges=false`, `RasterizationScale=1.0`, `IsVisible=true`, `DefaultBackgroundColor=Transparent`; then `GraphicsCaptureItem.CreateFromVisual(root)`. Frames pull from a free-threaded `Direct3D11CaptureFramePool` (2 buffers) → `SoftwareBitmap.CreateCopyFromSurfaceAsync` (BGRA, `BitmapAlphaMode.Straight` — the compositor blends STRAIGHT alpha, so premultiplied readback would wreck corner anti-aliasing) → per-frame `FindContentBounds` alpha-bbox on the capture worker → `VideoFrame.CropBounds` stamped → ring up to 2 fresh + Epoch (reuse; GC lesson slice 8) → dispatcher-coalesced copy (Render priority) into ONE crop-sized shared `WriteableBitmap`; `PreviewBitmapChanged` raised only on bitmap (re)creation. Also unpumped: the OBS-side de-throttle flags (slice 11) stay — they fix the hidden-page JS throttling; the capture path itself no longer polls. - **Implemented in:** new `Services/WebCaptureFrameSource.cs` (owns item+pool+session — mirrors `ScreenCaptureFrameSource`, which servers as the copy template) + reworked `Services/WebView2Manager.cs` (owns controllers, the visual tree, nav/script wiring, preview; internal seam ctor `(Dispatcher, Func?)` so the tests instantiate zero WinRT; public ctor `(Dispatcher)`). - **CoreMessaging DQ recipe (record-once):** the 19041 CsWinRT projection has NO `DispatcherQueueController.CreateOnCurrentThread()` (CS0117 — only `CreateOnDedicatedThread` + `FromAbi(IntPtr)`). P/Invoke `coreMessaging.dll!CreateDispatcherQueueController` with struct `DispatcherQueueOptions { DwSize, ThreadType = 2 (DQTYPE_THREAD_CURRENT), ApartmentType = 2 (DQTAT_COM_STA) }`, wrap via `DispatcherQueueController.FromAbi(ptr)` (mirror of `CaptureInterop`), only then `new Compositor()` — all once on the WPF UI thread (the app dispatches Render there). - **Compile notes (each cost a build cycle):** `Compositor` collides with the repo's OWN `ytLive.Services.Compositor` namespace → fully-qualify `Windows.UI.Composition.Compositor`; `CoreWebView2CompositionController` exposes **`Close()`**, not `Dispose()`; `Color` is ambiguous (`System.Drawing` vs `System.Windows.Media`) → `System.Drawing.Color.Transparent`. - **Dropped:** `Services/CaptureScheduler.cs` (DELETED — no cadence to schedule), all three `SetCaptureInterval` hooks in `MainViewModel.Streaming.Operations.cs`, the XAML `WebViewHostPanel`. `TransparentBackgroundScript` const is unchanged (a test pins it). - **Test model (Good Dog):** reworked `ytLive.Tests/WebView2ManagerTests.cs` — dropped the 4 control-size tests + the `CaptureScheduler_Drops…` test; `FindContentBounds` tests moved to `WebCaptureFrameSource.FindContentBounds`; the ONE integration test `Frames_PublishCroppedPreview_And_CoalesceToLatest_CarryingCropBounds` drives the internal seam with `FakeWebSource : IScreenCaptureSource` + a real background-STA `DispatcherPump` (copied from `ScreenCaptureManagerTests`): register → one crop-sized preview bitmap published → `CropBounds` + `IsOpaque=false` on `GetLatestFrame` → back-to-back pumps coalesce to latest. Byte assertion compares the DENSE 2×2 crop (`CropBytes` helper — slices source rows with stride gaps, a contiguous range spans rows wrongly). - Full suite **293/293 green, 0 warnings**. Audio untouched. **NOT YET VERIFIED ON DEVICE** — the composition path needs a real 60fps-widget run (open item; see HANDOFF). Web work commits stay LOCAL (no push) until the user greenlights. - **Slice 15 — one frame per deadline SLOT: duplicate-on-lag, never skip (2026-09-14, device takes ty-1723/1726):** slice 10's freshness choice — a render overrun VANISHES the missed slots from the file — authored ACCELERATED playback: ~35ms render cost vs the 16.6ms slot, one fresh frame per 35ms (log wall-1726: `FramePump stall: iteration 33ms (> 2× the 17ms interval): worst render 33-46ms`), and a 60fps container muxed on arrival → 1723: 697 frames = 11.62s video vs 11.84s audio; 1726: 163 frames = 2.72s vs 2.93s. The video also ended 0.21–0.24s before the audio ("audio speeds up, cuts off at the end"). OBS's answer is duplicate-on-lag: `libobs/media-io/video-io.c` emits one frame per tick and a lagger DUPLICATES the newest frame, counted as "lagged frames due to rendering lag/stalls" (obs-output.c) — a wall-time hole never exists (docs.obsproject.com/ backend-design: "If the video frame queue is full, it will duplicate the last frame"). The pump's submit is now a bounded catch-up: `while (now >= nextTick) { submit; nextTick += intervalTicks; }` over a `deadlineNow` captured once per iteration — the LATEST composite, fresh on the first missed slot, repeated (OBS's duplication) for the rest. Duration == wall under any render load, at the cost of a short judder during a stall. The burst is safe because `Channel.TryWrite` never blocks (slice 10's queue — the take-9 smear was the BLOCKING pipe-write re-copying a stale buffer during a long freeze; here each emit is nanoseconds). The burned `_outputIndex` (slice 10 judge) moved INSIDE the submit loop so every emitted slot carries its own +1 (and the old unconditional pre-gate bump no longer gaps the sequence on non-submitting fast-render iterations). **Good Dog test** `Pump_Overrun_Renders_EmitsEverySlot_NotSkipped`: 60fps, resolver sleeps 35ms, asserts ≥0.65 of the wall slots are emitted (the skip-pump wrote ~1/35ms ≈ 200 in 7s; the slot pump ~415). Full suite **294/294 green, 0 warnings**. The +0.6s audio-late clap reading on 1726 was confounded by the 1.1x acceleration — re-measure on device; if a real residual remains it is the audio pipeline. No push (web/A/V work stays local). - **Slice 16 — the desktop capture conversion was the bottleneck: fast downscale + cadence throttle + reuse-distance ring (2026-09-14, device take ty-1742):** slice 15 fixed pacing but the 1742 desktop layer was still jerky/frozen with a horizontal tear. Decoded to raw frames and audited: the **desktop band was frozen 21s of 23.35s (90%)** — ~6.1 content updates/s, freeze runs up to 2.28-2.78s, dup-run max 90 frames (1.5s). The render stat (`worst render 33-36ms`) was real but MOOT: the capture CONVERSION was the wall. The monitor delivers at the **240Hz DWM cadence** (~4.2ms); with one-in-flight conversions and each 2560×1440→1920×1080 `DownscaleBgra` at ~150ms under load, `LatestFrame` updated a handful of times/s — the desktop feed inside a 60fps file read ~90% frozen. (Webcam + audio were their own paths — fine, matching the report.) Three changes, all in `Services/ScreenCaptureFrameSource.cs` (+ `Services/FrameRingBuffer.cs`): (1) `DownscaleBgra` is now **integer 8.8 fixed-point, "shift only at the end"** — the exact two-stage math of `SceneCompositor.Bilinear` (MyMistakes take-4 rule), dropping double-per-pixel to a row-walk of integer ops (the capture downscale had stayed the naive float twin of the 258ms disaster); (2) a **10ms conversion floor** (`MinConvertInterval`) so the 240Hz tail stops queuing ~150ms of serialized conversion per second — the open edge sits just above the ~60/s the pump can use; (3) the ring is a **reuse-distance pool** (`FrameRingBuffer`, depth 8, redLine 4): a buffer is only rewritten ≥4 rents after its last hand-out else a fresh allocation, replacing the blind round-robin — since `ScreenCaptureManager` keeps `session.LatestFrame` across conversions and the dispatcher preview copy lags, "who released the buffer" needs a consumer API that doesn't exist; a reuse DISTANCE needs no cooperation (the 1742 new-top/old-bottom tear is that read-under-write, structural now). Telemetry added: a startup.log line every 2s (`frames/s, conv avg/max ms, skip busy/cadence, ring allocs`) so the device take can be judged numerically. **Good Dog test** `Ring_NoLap_ReusesOnlyAfterRedLineRents` (depth 3/redLine 4 exercises the red-line skip; the 8/4 config settles at 8 buffers and never grows). Full suite **295/295 green, 0 warnings**. C4 (compositor Epoch-cached downscale / blit-on-change) was DEFERRED: the slice-15 approved plan assumed a native-res relocation; the measurement recast it — relocating a ~30ms float downscale to the render thread just moves the same cost into the slot budget. Re-measure on device; if render still >16.6ms slots after capture feeds ≤60 real updates/s, add C4. No push. - **Slice 17 — the OS readback was the real wall: overlapping conversions + monotonic publish + deeper pool (2026-09-14, device take ty-1824):** slice 16's downscale fix landed but the desktop was still ~90% frozen on the 1824 take (band 6.8/s updates, max freeze 4.85s). The new 2s telemetry was decisive: `conv avg 46-50ms max ~61ms` with `skip busy 37-83` — the **GPU→CPU readback (`CreateCopyFromSurfaceAsync`), not `DownscaleBgra`**, is the ~47ms wall (240Hz HDR compositing + encoder + 2 capture devices share the GPU); at one-in-flight that caps the desktop feed at ~17-20 updates/s, which is the file's whole-frame ~7 content-moments/s. Two facts reshaped the fix: **(a)** shrinking the pool size does NOT scale the desktop (Microsoft docs: "If content is larger than the frame, the contents are **clipped**") — readback stays at native 2560×1440; **(b)** delivery is healthy (60-100 arrivals/s), so the lever is conversion throughput, not the pool size. Changes in `Services/ScreenCaptureFrameSource.cs`: conversions overlap up to **MaxConcurrentConversions = 3** (pool deepened to 5 buffers so in-flight frames fit), each completion publishes ONLY if its Epoch is strictly newer than the last published (`MonotonicGate` — a slow older completion must never overwrite a newer LatestFrame), Epoch increments via `Interlocked`, and the DownscaleBgra row-scratch became per-conversion locals (concurrent callers). The render side (33-42ms → ~30 unique composites/s) is the NEXT cap after capture speeds up — that's the C4 slice, queued right after this re-measure. **Good Dog test** `PublishGate_TryPublish_OnlyStrictlyNewerWins`. Full suite **296/296 green, 0 warnings**. NOT YET DEVICE-VERIFIED; target: telemetry frames/s jumps ≥ ~30-40 and the band audit drops below ~50% frozen. No push. - **Slice 18 — C4: the FramePump blit-on-change composite cache (2026-09-15, take ty-1841 verdict):** the 1841 take proved capture fixed (band ~20 fresh updates/s, no tears, pacing clean) but logged a FramePump **stall on EVERY iteration** (`totalMs 21-44`, `render=full-render split=0 elements=6 dynamic=4`, worst render 166ms startup spike) → ~22-28 composites/s hard-caps the desktop inside the 60fps file. The SceneGraph split CANNOT fix this wired: `GetSplitPoint` returns **0** because the live-capture backdrop is element 0 and must stay dynamic — a baked capture goes stale — so every tick runs the full re-composite even when no input changed (the compositor's per-layer paste cache already makes identical frames identical per-element; the waste is re-COMPOSITING the whole 8.3MB frame). Same shape as OBS (sources cache their surface, the compositor renders on update only — the pattern this file has cited since the 2026-09-04 paste-cache slice) at the FRAME level. `FramePump.RenderFull` now wraps both full-render call sites (no-SceneGraph + split=0 fallback): it computes `BuildFullRenderSignature` (a deterministic hash of the crop/output dims, the social bar identity + top, and — per element, MIRRORING the compositor's own resolution — the element ref, layout/visual bits, and the frame each element would resolve through the SAME resolver seam, using buffer-array identity + Epoch + CropBounds; a changed frame forces a re-composite, an unchanged one never serves stale bytes) and, when it matches the last composite, reuses the cache with ONE `Buffer.BlockCopy` (~3ms) instead. The cache buffer is a **separate long-lived array** — never the scratch pool — because the caller burns the frame counter and recycles the scratch AFTER `RenderScene` returns; the cache is written from the rendered scratch BEFORE returning (pre-burn, pre-recycle). Engagement is gated on the 1:1 config (`SourceRect == Output`, the only deployed tier); the vertical tier's cached fresh-scale-buffer follow-up stays exactly where it was. Telemetry: internal `CacheHits`/ `CacheRenders` counters surface as `cache {renders}R/{hits}H` on the 5s stats line, and the ticks' key is an internal `OutputIndex` (burned counter, stable across one tick's resolver passes). **Good Dog test** `FullRenderCache_StaticInputs_RenderOnce_Then_Reuse_UntilInputChanges`: static scene → exactly ONE composite (renders==1, hits>0, byte-identical content above the burn strip), then a new frame (new array + monotonic Epoch) invalidates and propagates. Existing pool test now passes a STABLE scene (production hands `StagedScene`; a fresh scene per tick churns the element refs inside the signature and hides the cache — the tests that care about cache behavior must mirror production) and keys its red/blue alternation on `OutputIndex`/frame count so the tick's two resolver passes (signature + render) don't double-advance a call-count flip. Full suite **297/297 green, 0 warnings**; verify.sh scope-locked to FramePump.cs + FramePumpTests.cs + docs. Target next take: `cache` hits dominate on TV holds, `avg render` drops toward the BlockCopy, stalls vanish, then the band audit + clap re-measure decide push. No push. - **Stop ordering matters:** `StopAsync` stops the encoder — since slice 10 it FLUSHES the pending queue (`Channel.TryComplete` → drain writes the leftovers, closes stdin → EOF → ffmpeg finalizes+exits; an accepted frame is never lost) — **before** awaiting the loop. The old reverse-order deadlock was a submit stuck on pipe backpressure; the drain thread owns that wait now, and stopping the encoder first keeps the pump's own (non-blocking) submits from racing the completed queue as a spurious drop. `ProcessFailed` self-stops the pump. `Failed` while live flips `StreamStatus.Error` (minimal). - **Health stats (ship step 6, shipped 2026-08-13):** `HealthUpdated` is bound to the bottom bar — `MainViewModel.OnFramePumpHealthUpdated` marshals to the UI thread (the encoder's stderr loop raises on a background thread) and copies into `CurrentHealth` (the bottom bar's existing `CurrentHealth.*` bindings). `ResetHealth(status)` zeroes dropped/duration on go-live and on End so stats never linger from a previous session; bitrate/FPS stay on the tier's targets (`ApplyStreamQuality`). The bar shows real encoder values since TASK 9 (2026-08-16) wired the reusable stream URL into `_rtmpUrlProvider`. - **`MainViewModel` owns the resolver** (`ResolveOutputFrame`): `WebcamSceneConfig` → `CameraManager.GetLatestFrame(DeviceKeyForWebcam(WebcamId))` — **device-keyed** (configs carry the identity GUID; sessions key on the Windows device id — the old direct-GUID lookup returned null forever and the webcam was structurally absent from output; take-2 fix 2026-09-01, `WebcamOutputKeyTests`), `Source { IsLiveCapture, CaptureKey }` → `ScreenCaptureManager.GetLatestFrame(CaptureKey)` (the new accessor mirroring `CameraManager`), image/ background → `StaticPixelCache.Get(AssetId)`. `BuildCompositorOptions` rounds the VM's `OutputRect*` doubles to ints — the vertical 607.5 half-pixel crop rounds to a perfectly-centered **608** (`Math.Round`, ToEven); `BuildEncoderOptions` fills W×H/FPS/bitrate from the tier once the URL provider yields one. - **Media source decoder (TASK 21, Increment B — 2026-08-31):** `MediaVideoSource` (`Services/MediaVideoSource.cs`) decodes a local file to `VideoFrame`s by spawning ffmpeg with `-f rawvideo -pix_fmt bgra -vf scale=WxH -an` and draining the raw BGRA stdout pipe through the pure `RawVideoFrameReader` (`Services/RawVideoFrameReader.cs`), raising `FrameAvailable` per frame. Implements `IMediaFrameSource` (`Services/IMediaFrameSource.cs`: `Key`, `FrameAvailable`, `Completed`, `StartAsync`/`StopAsync`). The subprocess sits behind `IDecodeProcess`/`FfmpegDecodeProcess` (binary-stdout mirror of the encoder's `IEncoderProcess`), and the path comes from `IFfmpegLocator` — codec-agnostic, uses the ffmpeg already shipped, testable with a fake process. Decode sessions are owned app-wide by `MediaVideoSourceManager` (`Services/MediaVideoSourceManager.cs`): refcounted by `MediaPath`, `Func` factory seam (mirror of `ScreenCaptureManager`), `AcquireAsync`/`ReleaseAsync`/`ReleaseAllAsync`/`GetLatestFrame`, coalescing each file's frames onto the UI dispatcher onto one shared `WriteableBitmap`. Frames emit as fast as the pipe produces them. Wired into the live pipeline (slice 1 step 4): `ResolveOutputFrame` reads `_mediaManager.GetLatestFrame(MediaPath)` for a `MediaSource`, and `Source.DisplaySource` (`Models/Source.cs`) routes `VideoImageSource` for `MediaSource` so the preview/canvas show decoded video. Slice 2a shipped the probe seam: `FfmpegFrameRateParser` (pure, `avg_frame_rate=` then `r_frame_rate=` rational N/N/M, unknown→null) + `IFrameRateProbe`/`FfmpegFrameRateProbe` (derives sibling `ffprobe.exe` from the located ffmpeg dir, reuses the `IDecodeProcess` seam for the ffprobe subprocess text; null if ffprobe absent) + `FfmpegLocator.ProbeFileName` now also extracts `ffprobe.exe` from the pinned archive (conditional — old caches without it just get no pacing). Slice 2b wired pacing: `MediaVideoSource` takes optional `IFrameRateProbe?` + `Func? delay` (default `Task.Delay`) seams, probes FPS once in `RunAsync`, and delays by 1/fps after each emitted frame; no probe/unknown → no pacing (ffmpeg's own pipe backpressure already throttles the decode). Slice 3 added loop control: `IMediaFrameSource.Looping` (bool); `MediaVideoSource` takes a `Func` process factory (a single `Process` can't be re-`Start()`ed, so each loop pass creates a fresh decoder) and wraps the decode in a `do…while (Looping)` — restart on natural EOF instead of raising `Completed`. Production wiring (MainViewModel media factory): passes `() => new FfmpegDecodeProcess()` as the factory, `FfmpegFrameRateProbe(new FfmpegLocator(), () => new FfmpegDecodeProcess())` as the probe. Still open: session acquisition on add/remove (UI picker) + wiring `Source.MediaIsLooping` into `IMediaFrameSource.Looping` (needs a manager-level per-path loop provider, comes with the picker slice). - **Social bar on the output (bar bug-fix branch):** the `FramePump` takes an optional `socialBar: Func<(VideoFrame? Frame, SocialBarPosition Position)>?` seam, re-read **every frame** (so a mid-stream position flip applies immediately). The strip is pre-rasterized by `Compositor/SocialBarRenderer.cs` (WPF glue — WPF glue here is fine because the strip is rendered once per config change on the UI thread, the resulting immutable frame is then composited pure-CPU by `SceneCompositor`), and `SceneCompositor.Render` blits it **last — above the branding flash** at `socialBarTop` (0 = top, `SourceRectHeight − barHeight` = bottom) in master space. `MainViewModel` owns the frame (`_socialBarFrame`, rebuilt by `RenderSocialBarFrame` on load/save/`NotifySocialsChanged`). - **Privacy blur layer (2026-11-01):** `SourceType.Blur` is a Live-scene element that blurs the composed frame inside a rect or ellipse. **It is universal**: `SceneCompositor` applies the blur group AFTER every layer has been composited (before the flash/social/ticker overlays), so it can cover the webcam, chat, alerts — anything — which is the creator's "moment of privacy" case. Its position in the layer list is cosmetic for that reason (it is added at the top), and the earlier "never above the brand" rule is superseded for blurs: a blur over the logo does obscure it. `SceneCompositor` treats all visible blurs as ONE group the first time one is reached in the z-order (`ApplyBlurLayers`), sorts them **strongest first** and passes a shared coverage map to `RegionBlur.ApplyInPlace`, so an overlap resolves to the strongest single blur rather than a blurred blur. `RegionBlur` shares the webcam's radius curve and constants (`Downscale 4`, `MinRadius 1`, `MaxRadius 6`, `RadiusFor(strength)`), so a strength means the same thing in both features; the webcam's `BackgroundEffectProcessor` delegates to it. Blur is `ElementKind.Dynamic`, so it is never baked into the static base and every render path (pump, preview, thumbnails, scene dump) gets it from the one compositor. - Known consideration: the pump reads the active scene on a background thread while the UI can still edit it; a concurrent-mutation exception is contained (logged + `Failed` + the pump stops) rather than crashing. The background thread + video pipeline is the new reality since this step. --- ## Related files This file owns the compositor and the pump. The neighbours live elsewhere: - [`output.md`](output.md) — **live encoder + RTMP push**, `IFfmpegEncoder`/`FfmpegEncoder`, `FfmpegArgs`, locator and bundle. - [`TASKS/task-21-media-source.md`](../../TASKS/task-21-media-source.md) — **media source**, `MediaVideoSource` (ffmpeg `rawvideo` decode) feeding the pump's resolver. - [`overlays.md`](overlays.md) — **social bar** and **native events & alerts**, the overlay strips the compositor blits last. - [`environment.md`](environment.md) — **Run** and **Tests**, how to launch in Release and how the suite is run.