feat: web sources frame-captured via composition (CoreWebView2CompositionController → Windows.Graphics.Capture); PNG poll + CaptureScheduler deleted

The ~30Hz CapturePreviewAsync PNG poll capped real cadence at ~20Hz
(35–165ms full-HD encode+decode), so a 60fps widget still juddered at ~1/6
speed. Replaced polling with frame-driven capture of the composition
controller's root visual — the mechanism WebView2CompositionControl and
Flutter's webview_windows use (graphics_context.cc captures the root
surface_ visual via CreateGraphicsCaptureItemFromVisual; reference:
github.com/microsoft/Windows.UI.Composition.WinUI / flutter-internal
webview_windows). Frames now arrive at the renderer's own pace; capture
memory is epoch'd ring reuse + one crop-sized shared WriteableBitmap.

New Services/WebCaptureFrameSource.cs owns GraphicsCaptureItem + free-
threaded Direct3D11CaptureFramePool + session (Straight alpha readback,
per-frame FindContentBounds → CropBounds). WebView2Manager reworked around
per-session composition controllers + one UI-thread Compositor created via
the CoreMessaging CreateDispatcherQueueController P/Invoke (the 19041
projection lacks CreateOnCurrentThread); internal seam ctor
(Dispatcher, Func<string,IScreenCaptureSource>?) for hermetic tests.
CaptureScheduler.cs deleted; the three SetCaptureInterval cadence hooks
removed; InitWebView2() moved from MainWindow ctor to Loaded (a parent
HWND must exist for the composition controller); the hidden WebViewHostPanel
overlay deleted. TransparentBackgroundScript unchanged.

Tests: WebView2ManagerTests reworked — 4 control-size + scheduler tests
dropped, FindContentBounds tests moved to WebCaptureFrameSource, ONE
integration test (Frames_PublishCroppedPreview_And_CoalesceToLatest_CarryingCropBounds)
drives the seam with a FakeWebSource + background-STA DispatcherPump.
Suite 293/293, 0 warnings.

NOTE: composition path NOT yet verified on a device — the take is the
next step. Web work committed locally only (no push per standing rule).

Docs same-commit: ai.md Slice 14 + supersede marker on Slice 11, HANDOFF,
MyMistakes (CoreMessaging DQ + namespace-landmine recipe), TASK 17,
Controls/ViewModels/Services indexes.
This commit is contained in:
2026-09-14 16:14:00 -07:00
parent 40c453805d
commit c01206fb8a
16 changed files with 890 additions and 581 deletions
+38 -1
View File
@@ -418,7 +418,8 @@ CLR→real-ffmpeg run to the native Windows suite.
"not a regression" for weeks without ever recording WHY.)
**The trap:** `VisualTreeHelper.HitTest(window, pt)` returned the window's
`WebViewHostPanel` overlay (`IsHitTestVisible="False"`, `Opacity=0`, ZERO children) for
`WebViewHostPanel` overlay (`IsHitTestVisible="False"`, `Opacity=0`, ZERO children — since the
2026-09-14 composition-capture redesign the overlay is DELETED, but the API lesson stands) for
EVERY point in the window — so a "corner is grabbable" assertion could never pass, and
it looked like a real interaction bug. The actual input pipeline (`UIElement.InputHitTest`,
what Mouse routing uses) correctly returned the element's Grid at elem-center/corner-in/
@@ -508,6 +509,42 @@ while live (`IsEditMode`) since a mid-stream advance flip is meaningless. Test r
`StartLive_NegativeOffset_AdvancesAudio_ByDroppingTheStreamHead` (emit 6×0.9 into an 8-tick budget,
long 0.2 bed, wire must show only 0.2).
## Composition video capture of WebView2: the CoreMessaging DQ recipe + the build gotchas
(2026-09-14, the ~20→60fps web-capture redesign — record-once so the next GPU-path work never
re-derives it. Everything that follows was already settled by OBS/Flutter `webview_windows`; the
cost this session was the 19041 projection's gaps.)
1. **No `DispatcherQueueController.CreateOnCurrentThread()` on the 10.0.19041 projection**
(CS0117 — only `CreateOnDedicatedThread` + `FromAbi(IntPtr)` exist). P/Invoke
`coreMessaging.dll!CreateDispatcherQueueController` with a sequential
`DispatcherQueueOptions { DwSize, ThreadType=2 (DQTYPE_THREAD_CURRENT), ApartmentType=2 (DQTAT_COM_STA) }`,
wrap via `DispatcherQueueController.FromAbi(ptr)` (mirror of `CaptureInterop`), THEN `new Compositor()`.
One controller + one compositor per UI thread, created once.
2. **`CoreWebView2CompositionController` needs a real parent HWND** — there is no window in the
MainWindow ctor, so `InitWebView2()` moved from the ctor to `MainWindow_Loaded` (source
registration is guarded by `ContainsKey`, so the layout-load path is safe).
3. **Capture the ROOT visual, not the control.** `GraphicsItem` for a composition controller comes
from `GraphicsCaptureItem.CreateFromVisual(root)` where `root` is your own 1920×1080
`ContainerVisual` (`IsVisible=true`) holding the controller's `RootVisualTarget` as a
`RelativeSizeAdjustment=1,1` child — exactly what `webview_windows`'s `graphics_context.cc`
does (`CreateGraphicsCaptureItemFromVisual` on the root `surface_` visual). Hide nothing,
move nothing, poll nothing — the capture is frame-driven at the renderer's pace.
4. **Straight alpha, never premultiplied.** Read back with `BitmapAlphaMode.Straight`; the
compositor's blend is straight-alpha, so `Premultiplied` readback wrecks corner anti-aliasing.
5. **Namespace landmines that each cost a build cycle in `Services/`:** `Compositor` resolves to the
repo's OWN `ytLive.Services.Compositor` namespace — fully-qualify `Windows.UI.Composition.Compositor`;
`CoreWebView2CompositionController.Close()` is the disposal call (no `Dispose()`); `Color` is
ambiguous (`System.Drawing` vs `System.Windows.Media`) — qualify `System.Drawing.Color.Transparent`.
6. **Testing a compositor-built bitmap without a runtime:** the internal seam ctor
`(Dispatcher, Func<string, IScreenCaptureSource>?)` + a `FakeWebSource` + a real background-STA
`DispatcherPump` (borrowed from ScreenCaptureManagerTests) drives the whole session path hermetic.
When byte-comparing a crop, slice the source with stride gaps (helper `CropBytes`) — a contiguous
range silently spans rows.
Recipe verified on green suite + 0-warning build; the composition path itself still needs a device
take (open item, HANDOFF).
## A pipe-read test that samples one tick is a timing flake by construction
(2026-09-14, re-discovered) `Mix_HonorsProviderGains_AndGameMute_KillsTheLoopback` fails