Slice 10 from today-slices (d1126dd): the paste cache minted a fresh raster
array per new source frame (webcam ~30 keys/s ≈ 18MB/s LOH churn →
gen2 pauses that ate camera frames). Fix: _elementRasters — re-rasterize
INTO the element's existing array when geometry holds, so steady-state
allocation ≈ 0. The paste cache keys on (array+epoch) still protect correctness;
_elementRasters[element] tracks the element's current raster so stale keys can
never paste a mid-update array. MaxPasteEntries 48→256 (was a leak guard,
not an allocation stream). Regression test:
PasteCache_RingRecycledSources_ReRasterInPlace_WithoutAllocationChurn.
Build 0 warnings, 289 tests.
Take 11 (c10ce06c) validated the off-UI architecture: typical frames land
work ~10ms + wait ~6.8ms = 16.7 exactly on the deadline; 212/300 best yet.
The ENTIRE remaining gap is periodic 35-65ms render spikes that WORSENED
across the take (189 -> 147) — the signature of gen2 GC pauses. Biggest
churner is structural: the screen capture minted a fresh ~8.3MB byte[] per
DWM frame (~500MB/s of LOH), a producer OBS never does (it owns fixed
surface pools).
- ScreenCaptureFrameSource: 4-deep buffer ring with size-matched slots (a
<=17ms consumer cannot be lapped at 60Hz) + reused downscale row scratch.
- VideoFrame.Epoch: monotonic per producer frame. The paste cache keys on
array IDENTITY, so recycled arrays MUST be distinguished — epoch joins the
PasteKey. Producers handing fresh arrays leave it 0 (key unchanged effect).
- Stats print 'gen2 +N' per 5s window: next take acquits or convicts GC
without another guess (rule: prove the stage).
- Test (the ONE): PasteCache_RecycledArrayWithNewEpoch_ReRasterizes_NotStaleHits
— same array, new content, bumped epoch; fails on the old key by
construction. 37/37 compositor/pump, clean build.
- Next suspect if gen2 stays hot: the 10Hz WebView2 capture (full-canvas PNG
decode + fresh arrays on the UI thread) — recorded, untouched.
Creator audio ask queued in the same working session (+40% post-mix master
gain before the -1dBFS limiter) lands as its own commit next.
Take 9's numbers were decisive: paste cache moved work to ~25ms/frame but the
period stayed ~37ms. The missing ~12ms per tick is Task.Delay rounding every
sub-tick request up to the Windows system-clock tick (~15.6ms default —
documented: learn.microsoft.com/en-us/dotnet/api/system.threading.tasks.task.delay).
A frame finishing 3ms early requested 3ms and slept 15.6. Producer capped at
~27fps no matter how fast the compositor got — which is why two real render
fixes read as 'zero change' in playback. Game-loop/OBS canon for this
(stackoverflow.com/questions/5441464; learn.microsoft.com/en-us/windows/win32/
api/timeapi/nf-timeapi-timebeginperiod): raise the timer resolution for the
session, sleep only the bulk of the remainder, and SPIN the last ~2ms across
the deadline.
- FramePump: timeBeginPeriod(1) on entering the pump loop, timeEndPeriod(1) in
the finally; pacing = bulk _pacingDelay(ahead - 2ms) + bounded Thread.SpinWait
tail; blown deadlines rebase unchanged (never burst).
- Stats now report avg wait: render+submit+wait must equal the period — the
accounting is closed, no stage can hide in an unmeasured gap again.
- Webcam dropped its IsOpaque paste-cache bypass: it re-sampled ~156k px every
tick even between identical device frames; cached paste beats the sampler on
hits, costs the same on misses.
52/52 per-class green (pacing + pixel suites unchanged — output byte-stable),
clean build 0 warnings. Docs same commit (ai.md slice 6, TASKS.md take-10
gate, MyMistakes #3 + renumber, HANDOFF). User's top-bar spec remains next in
queue (Unit B) — re-sent many times, captured, no open questions.
The stamped build settled what slices 3-4 could not: chat cache works (resolve
~0.0ms) but render stayed 26-27ms -> 124-135/300. The cost was the compositor
re-rasterizing EVERY layer every tick: this Live scene re-samples chat (159k) +
web widget (271k) + image (95k) + cam (156k) ~ 680k px @ ~38ns — for layers
whose pixels do not change between chat/web/cam updates.
OBS shape: cache the surface, paste per tick. BlitCachedLayer rasterizes a
non-opaque layer ONCE into an element-space, transparent-based frame keyed by
(source-array identity, src W/H, ceil'd dst rect, round, mirror), then pastes:
integer position, row alpha-blend, opacity applied at paste. Producers hand out
fresh immutable arrays -> array-identity keys cannot serve stale content; dict
bounded (48, clears whole). Drag/opacity live in paste params, not keys, so
editing stops triggering resamples too. Opaque backdrop keeps the memcpy path;
the webcam keeps the direct path via its IsOpaque flag (revisit if take 9 is
borderline).
ONE integration test: PasteCache_RepeatRender_IsByteIdentical_And_ContentChange-
Propagates (byte-exact raster-vs-paste incl. round-clip margins, new-array
propagation); existing pixel suite guards sampler semantics. 85/85 across
compositor/pump/chat/capture/session classes, clean build 0 warnings. Also:
BuildStampTests.cs was written last commit but never staged — its own scope-check
slip, added here (the run had used the on-disk file; tracked now).
Docs same commit: ai.md slice 5 + stale 'general path only 130k' claim corrected,
TASKS.md take-9 gate, MyMistakes recipe (prove the stage; a fix that doesn't move
the stat wasn't the bottleneck), HANDOFF. take 9 expectation: 300/300, render
<= ~8ms -> saga closes, Unit B starts.
Two defects made the producer 17x slow (37s record -> 2.1s/127-frame file,
rawvideo stamps by arrival): FramePump slept the FULL interval after each
render (period = render+submit+interval) and SceneCompositor did per-pixel
float sampling + Math.Round blends over all 2.07M master pixels, scanning the
whole destination per overlay (258ms avg render vs 1.5ms submit).
Both solutions are established, not invented — researched before coding per
the derivative-work rule:
- deadline pacing: OBS libobs/media-io/video-io.c video_thread (nextTick +=
intervalTicks, sleep only the remainder, rebase on overrun, never burst)
- row blits: libyuv pattern (BSD-3, chromium.googlesource.com/libyuv/libyuv)
— 1:1 aligned identity fast path, per-pixel alpha branch, integer
fixed-point blend, overlay clipped to the intersection rect, skip the dead
black pre-fill when the backdrop covers
ONE integration test: Pump_Paces_To_The_Deadline_Compensating_Render_Cost
(lands after a fake-seam lesson: pacing fakes must await, not complete
synchronously, or the pump loop runs inline on StartAsync and hangs vstest).
Clean build 0 warnings; FramePumpTests 10/10, SceneCompositor/SceneGraph/
SocialBar/StretchMath 20/20. Docs same commit: ai.md pipeline section,
TASKS.md TASK 18 (webcam-in-output + rename modal verified from take 3),
MyMistakes recipe, HANDOFF rewritten. Take 4 pending on the user's machine.
- Add ElementKind (Static/Dynamic) to SceneElement base; Source/WebcamSceneConfig classify
- Services/SceneGraph.cs: owns the Scenes collection (ViewModel's Scenes delegates to it),
the element mutation surface (Add/Insert/Remove/Move, each invalidating the bake), and the
queries that were scattered LINQ (GetBackground/GetWebcam/GetChatBoxes/GetSplitPoint/IsStatic)
- SceneCompositor: split-aware BakeStaticBase + CompositeLayers + Render(.., staticBase, split);
builds/caches the static base below the split point in source-rect space
- FramePump: optional SceneGraph -> optimized bake+composite path; falls back to full render
- MainViewModel: routes element mutations through the graph; invalidates the bake on static
layout/opacity/visibility/useDefaultBackground changes and after background heal/Ensure
- Integration test SceneGraphTests.BakedStaticBase_WithDynamicLayer_CompositesCorrectly
Derivative survey (mandated): tried before writing — OBS does per-source opacity/visibility
caching and static-scene baking; this mirrors OBS's 'cached static source' optimization.
3 documented defensive deviations from the spec (ChatOverlayLayer stays decoupled;
background helpers stay VM-static for direct testability; full facade peel deferred post-1.0)
in TASKS.md + ai.md. Scope check passed; clean build 0 warnings.
- TRAX: LocalGain now applies 0.2× factor so music is quieter relative to desktop audio slider
- TRAX tooltip: switched to ToolTipService with Placement=Top to fix z-order in Viewbox overlay
- Mic meter: MeterLevel multiplied by 1.2× before clamping for ~20% higher visual readout
- Volume sliders: click anywhere on track to jump thumb to that position before dragging
- Backdrop: Image Visibility now binds to BackdropVisible; compositor checks IsVisible before blit
- Renamed 'Backdrop' to 'Game Capture' in EnsureBackdrop + test assertions
Software compositor that renders a scene into the encoder's master VideoFrame,
mirroring the XAML preview minus editing chrome: backdrop -> background ->
elements (UniformToFill cover-crop, round clip, mirror, opacity, border) ->
branding flash.
NOTE FOR USERS: this change shows NO difference in the app's UI — it is pure
backend scaffolding laying the groundwork for live video capture/streaming.
The preview you see is unchanged.
- Services/Compositor/: SceneCompositor (Render(scene, frameFor resolver,
flashFrame, CompositorOptions)), CompositorOptions (source rect + output
size; 16:9 full master, vertical 607x1080 -> 1080x1920), StretchMath (pure
UniformToFill + bilinear), StaticPixelCache (asset bytes -> BGRA8 frame)
- Frame sources injected via Func<SceneElement, VideoFrame?> resolver, so the
compositor is pure, WPF-free, and hermetic to test (D3D11 upgrade behind the
same seam later)
- SceneElement.TryGetBorderColor public (shared hex parse), stale
MainViewModel comment fixed, pre-existing CS1998 in YouTubeAuthServiceTests
cleaned up
- Tests: SceneCompositorTests integration (full scene + vertical tier + flash)
+ StretchMath units, docs updated (72 tests passing, 0 warnings)
gramps