gramps
c01206fb8a
The ~30Hz CapturePreviewAsync PNG poll capped real cadence at ~20Hz (35–165ms full-HD encode+decode), so a 60fps widget still juddered at ~1/6 speed. Replaced polling with frame-driven capture of the composition controller's root visual — the mechanism WebView2CompositionControl and Flutter's webview_windows use (graphics_context.cc captures the root surface_ visual via CreateGraphicsCaptureItemFromVisual; reference: github.com/microsoft/Windows.UI.Composition.WinUI / flutter-internal webview_windows). Frames now arrive at the renderer's own pace; capture memory is epoch'd ring reuse + one crop-sized shared WriteableBitmap. New Services/WebCaptureFrameSource.cs owns GraphicsCaptureItem + free- threaded Direct3D11CaptureFramePool + session (Straight alpha readback, per-frame FindContentBounds → CropBounds). WebView2Manager reworked around per-session composition controllers + one UI-thread Compositor created via the CoreMessaging CreateDispatcherQueueController P/Invoke (the 19041 projection lacks CreateOnCurrentThread); internal seam ctor (Dispatcher, Func<string,IScreenCaptureSource>?) for hermetic tests. CaptureScheduler.cs deleted; the three SetCaptureInterval cadence hooks removed; InitWebView2() moved from MainWindow ctor to Loaded (a parent HWND must exist for the composition controller); the hidden WebViewHostPanel overlay deleted. TransparentBackgroundScript unchanged. Tests: WebView2ManagerTests reworked — 4 control-size + scheduler tests dropped, FindContentBounds tests moved to WebCaptureFrameSource, ONE integration test (Frames_PublishCroppedPreview_And_CoalesceToLatest_CarryingCropBounds) drives the seam with a FakeWebSource + background-STA DispatcherPump. Suite 293/293, 0 warnings. NOTE: composition path NOT yet verified on a device — the take is the next step. Web work committed locally only (no push per standing rule). Docs same-commit: ai.md Slice 14 + supersede marker on Slice 11, HANDOFF, MyMistakes (CoreMessaging DQ + namespace-landmine recipe), TASK 17, Controls/ViewModels/Services indexes.
4.5 KiB
TASK 17 — Web source rendering (WebView2)
Catalog:
TASKS.md— status and requirements live here.
Goal: make the web source actually render URLs into the preview and stream output.
Status: ✅ Done — required for v1 (2026-08-28)
- ✅ Add
Microsoft.Web.WebView2NuGet package - ✅ Schema v10:
WebUri TEXTcolumn onSourcetable + migration inLayoutStore.cs - ✅ Persist
Source.WebUrion save/load (currently in-memory only — lost on restart) - ✅ Hidden off-screen
WebView2control per web source — navigates toWebUri, renders in-app - ✅ Frame capture from WebView2 (superseded 2026-09-14: was
CoreWebView2.CapturePreviewAsync→WriteableBitmap(BGRA8); now composition-capture viaCoreWebView2CompositionController→Windows.Graphics.Capture— see the rendering-model update below. PNG polling is deleted) - ✅ Wire into
FramePumpresolver —Source { Type: WebSource }→ latest WebView2 frame - ✅ Wire into
SceneCompositor— render web source as an image element at its position/size - ✅ Preview shows live web content (not just a blank rectangle)
- ☐ Handle navigation errors, invalid URIs, timeout gracefully
Rendering model (2026-08-28, canvas-size viewport + ALPHA-BBOX crop — DONE, creator-verified):
the page renders at the MASTER CANVAS size (1920×1080), stable, never tracked (no reflow/truncation;
scrollbars suppressed via overflow:hidden). FindContentBounds scans the Bgra32 capture and crops
to the bounding box of non-transparent pixels — the widget's true rendered extent, measured from the
frame itself (NO DOM query, immune to layout timing; can never truncate content; full-canvas widgets
fall through to the full frame = prior verified-good image). Stretch="Fill" maps the cropped frame
flush under the selection box → box hugs the widget on all four sides. QueryContentBoundsAsync +
the JS content-bounds script are DELETED. RESULT: bounding verified PERFECT by the creator with
two widgets. Root-cause note on the earlier "remaining defect/gap": that was a WIDGET GLOW EFFECT
(the widget's own CSS glow pushes out its perceived borders), not a code bug — the alpha-bbox was
already hugging the glow halo. Lesson: test with a second, plain widget before changing code.
⚠️ Lesson bank: never return JSON.stringify from ExecuteScriptAsync (double-encodes); never
measure DOM stuff on NavigationCompleted (unsettled layout broke the image — a5b9952); avoid
sizing the container to the crop (ab29ec8 reverted — made it worse); a widget glow effect can
fake a gap — verify with a plain widget first; measure rendered pixels instead.
Rendering model update (2026-09-14, COMPOSITION capture — the PNG poll is gone): frame capture
is no longer CapturePreviewAsync → PNG (35–165ms per full-HD encode+decode capped real cadence at
~20Hz; item 5 below is superseded). Each session now renders through a
CoreWebView2CompositionController visual into Windows.Graphics.Capture
(GraphicsCaptureItem.CreateFromVisual of the root 1920×1080 ContainerVisual), pulled frame-driven
from a free-threaded Direct3D11CaptureFramePool at the renderer's own rate (~60fps, parity with the
browser). Straight-alpha readback, alpha-bbox crop + CropBounds per frame, epoch'd ring reuse, one
shared crop-sized preview bitmap. The old SetCaptureInterval cadence hooks and CaptureScheduler
are deleted; the hidden XAML WebViewHostPanel overlay and CapturePreviewAsync telemetry are gone.
TransparentBackgroundScript is unchanged (a test pins it). Full design + the CoreMessaging DQ
recipe in ../ai.md → Slice 14. Open: device verification of ~60fps web animation.
Properties panel (2026-08-28): web URI ✓/✕ icon buttons are IsTabStop="False" so Tab flows
X→Y→W→H→URI; the ✕ button now clears the URI textbox (was reverting to the pre-accept snapshot).
Slider style gained IsMoveToPointEnabled="True" — click-anywhere-on-bar jumps the thumb to the
click (volume sliders keep their manual SetSliderValueFromClick, harmless duplication).
Design decisions
- WebView2 is the only option for Windows — it's pre-installed on Windows 10 20H2+ and Windows 11
- The web source is a standard element — positioned/sized/opacitied like any image source
- Frame capture is drive-by-renderer (composition capture, ~60fps parity with the browser since 2026-09-14; the old "5-10 fps is fine" poll model is deleted)
- This enables Streamlabs/StreamElements overlays via web URLs