feat: web sources frame-captured via composition (CoreWebView2CompositionController → Windows.Graphics.Capture); PNG poll + CaptureScheduler deleted
The ~30Hz CapturePreviewAsync PNG poll capped real cadence at ~20Hz (35–165ms full-HD encode+decode), so a 60fps widget still juddered at ~1/6 speed. Replaced polling with frame-driven capture of the composition controller's root visual — the mechanism WebView2CompositionControl and Flutter's webview_windows use (graphics_context.cc captures the root surface_ visual via CreateGraphicsCaptureItemFromVisual; reference: github.com/microsoft/Windows.UI.Composition.WinUI / flutter-internal webview_windows). Frames now arrive at the renderer's own pace; capture memory is epoch'd ring reuse + one crop-sized shared WriteableBitmap. New Services/WebCaptureFrameSource.cs owns GraphicsCaptureItem + free- threaded Direct3D11CaptureFramePool + session (Straight alpha readback, per-frame FindContentBounds → CropBounds). WebView2Manager reworked around per-session composition controllers + one UI-thread Compositor created via the CoreMessaging CreateDispatcherQueueController P/Invoke (the 19041 projection lacks CreateOnCurrentThread); internal seam ctor (Dispatcher, Func<string,IScreenCaptureSource>?) for hermetic tests. CaptureScheduler.cs deleted; the three SetCaptureInterval cadence hooks removed; InitWebView2() moved from MainWindow ctor to Loaded (a parent HWND must exist for the composition controller); the hidden WebViewHostPanel overlay deleted. TransparentBackgroundScript unchanged. Tests: WebView2ManagerTests reworked — 4 control-size + scheduler tests dropped, FindContentBounds tests moved to WebCaptureFrameSource, ONE integration test (Frames_PublishCroppedPreview_And_CoalesceToLatest_CarryingCropBounds) drives the seam with a FakeWebSource + background-STA DispatcherPump. Suite 293/293, 0 warnings. NOTE: composition path NOT yet verified on a device — the take is the next step. Web work committed locally only (no push per standing rule). Docs same-commit: ai.md Slice 14 + supersede marker on Slice 11, HANDOFF, MyMistakes (CoreMessaging DQ + namespace-landmine recipe), TASK 17, Controls/ViewModels/Services indexes.
This commit is contained in:
@@ -11,7 +11,10 @@
|
||||
2. ✅ Schema v10: `WebUri TEXT` column on `Source` table + migration in `LayoutStore.cs`
|
||||
3. ✅ Persist `Source.WebUri` on save/load (currently in-memory only — lost on restart)
|
||||
4. ✅ Hidden off-screen `WebView2` control per web source — navigates to `WebUri`, renders in-app
|
||||
5. ✅ Frame capture from WebView2 (`CoreWebView2.CapturePreviewAsync`) → `WriteableBitmap` (BGRA8)
|
||||
5. ✅ Frame capture from WebView2 (**superseded 2026-09-14:** was `CoreWebView2.CapturePreviewAsync`
|
||||
→ `WriteableBitmap` (BGRA8); now composition-capture via
|
||||
`CoreWebView2CompositionController` → `Windows.Graphics.Capture` — see the rendering-model
|
||||
update below. PNG polling is deleted)
|
||||
6. ✅ Wire into `FramePump` resolver — `Source { Type: WebSource }` → latest WebView2 frame
|
||||
7. ✅ Wire into `SceneCompositor` — render web source as an image element at its position/size
|
||||
8. ✅ Preview shows live web content (not just a blank rectangle)
|
||||
@@ -33,6 +36,18 @@ measure DOM stuff on NavigationCompleted (unsettled layout broke the image — `
|
||||
sizing the container to the crop (`ab29ec8` reverted — made it worse); a widget glow effect can
|
||||
fake a gap — verify with a plain widget first; measure rendered pixels instead.
|
||||
|
||||
**Rendering model update (2026-09-14, COMPOSITION capture — the PNG poll is gone):** frame capture
|
||||
is no longer `CapturePreviewAsync` → PNG (35–165ms per full-HD encode+decode capped real cadence at
|
||||
~20Hz; item 5 below is superseded). Each session now renders through a
|
||||
`CoreWebView2CompositionController` visual into `Windows.Graphics.Capture`
|
||||
(`GraphicsCaptureItem.CreateFromVisual` of the root 1920×1080 `ContainerVisual`), pulled frame-driven
|
||||
from a free-threaded `Direct3D11CaptureFramePool` at the renderer's own rate (~60fps, parity with the
|
||||
browser). Straight-alpha readback, alpha-bbox crop + `CropBounds` per frame, epoch'd ring reuse, one
|
||||
shared crop-sized preview bitmap. The old `SetCaptureInterval` cadence hooks and `CaptureScheduler`
|
||||
are deleted; the hidden XAML `WebViewHostPanel` overlay and `CapturePreviewAsync` telemetry are gone.
|
||||
`TransparentBackgroundScript` is unchanged (a test pins it). Full design + the CoreMessaging DQ
|
||||
recipe in `../ai.md` → Slice 14. **Open: device verification of ~60fps web animation.**
|
||||
|
||||
**Properties panel (2026-08-28):** web URI ✓/✕ icon buttons are `IsTabStop="False"` so Tab flows
|
||||
X→Y→W→H→URI; the ✕ button now clears the URI textbox (was reverting to the pre-accept snapshot).
|
||||
Slider style gained `IsMoveToPointEnabled="True"` — click-anywhere-on-bar jumps the thumb to the
|
||||
@@ -42,6 +57,7 @@ click (volume sliders keep their manual `SetSliderValueFromClick`, harmless dupl
|
||||
|
||||
- WebView2 is the only option for Windows — it's pre-installed on Windows 10 20H2+ and Windows 11
|
||||
- The web source is a standard element — positioned/sized/opacitied like any image source
|
||||
- Frame capture rate can be lower than video FPS (5-10 fps for web content is fine)
|
||||
- Frame capture is drive-by-renderer (composition capture, ~60fps parity with the browser since
|
||||
2026-09-14; the old "5-10 fps is fine" poll model is deleted)
|
||||
- This enables Streamlabs/StreamElements overlays via web URLs
|
||||
|
||||
|
||||
Reference in New Issue
Block a user