Web source: geometry-agnostic viewport==box model — viewport locked to element box, Stretch=Fill, no dead space

2026-08-28 10:48:53 -07:00
parent 9d4103c44b
commit 8d793af91a
5 changed files with 121 additions and 82 deletions
@@ -6,28 +6,34 @@
 working). **NOT pushed — user rule: never push without explicit instruction (2026-08-24).** No feature
 branches pre-1.0: all work lands on `main` per work unit.

-## The web-source rendering fix (2026-08-28) — USER-CONFIRMED WORKING
+## The web-source rendering work (2026-08-28) — CURRENT MODEL: viewport == element box

-**The bug:** preloaded web resources rendered truncated (upper-left only) and misbehaved as the box
-resized. The WebView2 control was being resized to the element box as it changed.
+**Background:** the fixed-viewport fix (1920×1080 control + `Stretch="Uniform"`) made the page
+render without truncation but left dead space between the selection box and the content (box
+aspect ≠ capture aspect → letterbox; box is sized by the parent layout, capture is a separate
+hardcoded viewport — they bind to different rects). A CSS-reset experiment made the gap worse
+and was shelved by the creator. He stressed the systems must be GEOMETRY-AGNOSTIC: no 16:9,
+no fixed render size, no per-widget assumptions.

-**The fix — one idea, three-line change, the OBS/CEF pattern:**
- `Services/WebView2Manager.cs` — viewport is FIXED at the master canvas size (1920×1080), never
-  tracks the box. Control lives off-screen at (-5000,-5000). Page renders full at its designed size;
-  the preview scales the captured frame independently via `<Image Stretch="Uniform">`. This is what
-  OBS obs-browser does (fixed-viewport texture + independent scene-item scale). Removed the
-  PropertyChanged resize hook. Added an immediate `CaptureFrame` on `NavigationCompleted` so content
-  appears instantly on load (was waiting for the next 100ms tick).
- `ViewModels/MainViewModel.cs` — `InitWebView2` now calls `RegisterLoadedWebSources()` itself;
-  before, registration only happened when `LoadLayout` ran after manager init, so preloaded sources
-  never registered at startup.
- `MainWindow.xaml` — scene-element `<Image>` had a LOCAL `Stretch="UniformToFill"` that outranked
-  the Style DataTrigger (WPF precedence). Removed the local attribute; Style default is
-  `UniformToFill`; `IsWebSource` trigger overrides to `Stretch="Uniform"` (no crop/truncation).
- `ytLive.Tests/WebView2ManagerTests.cs` — 3 scale-sync tests tested the removed behavior; replaced
-  with 2 tests asserting the fixed 1920×1080 viewport and that it does not track source dimensions.
+**The model that delivers it (implemented, needs visual verification):**
+- `Services/WebView2Manager.cs` — the WebView2 viewport IS the element box
+  (`Source.Width × Source.Height`). The page renders at exactly the box size; a `PropertyChanged`
+  hook keeps the viewport locked to the box (re-triggers capture on resize, sub-0.5px drift is
+  ignored). `CaptureFrame` is unchanged (aspect-preserving Physical-size capture).
+- `MainWindow.xaml` — web `<Image>` uses `Stretch="Fill"` (capture aspect == box aspect by
+  construction, so Fill == Uniform with zero dead space; during a transient stale frame it just
+  holds aspect briefly). Everything else still `UniformToFill`.
+- Selection rect == content rect by construction → flush edges, no padding, works for any widget
+  geometry the creator throws at it. FramePump still reads `GetLatestFrame` via `ResolveOutputFrame`.
+- Transparent-background JS injection retained (background only — no layout/CSS-reset meddling).

-Build: 0 warnings, 0 errors. User verified it renders correctly.
+Build: 0 warnings; WebView2Manager tests 4/4 (size tracks source, sub-pixel drift ignored,
+minimum 1, transparent bg).
+
+**Landmines (earlier attempts — do not repeat):** fixed-viewport (letterbox dead space),
+CSS reset/margin juggling (enlarged the gap), control-tracking-box combined with
+`UniformToFill` display (truncation), ZoomFactor meddling. The working invariant is:
+**viewport rect == selection rect, display = Fill.**

 ## Key facts / landmines

@@ -44,7 +50,7 @@ Build: 0 warnings, 0 errors. User verified it renders correctly.
 - No `CODING.md`; conventions live in `AGENTS.md` + `ai.md`.

 ## Next steps
-1. Await creator comments on the committed rendering fix; address what he raises; also testing
-   the box-vs-content padding fix (CSS reset + (0,0) anchor) just committed.
+1. AWAIT creator's verdict on viewport==box (geometry-agnostic) rendering — visual check with a
+   widget URL at several box sizes; the selection frame must snap flush with no padding.
 2. Follow-up only if the creator asks: graceful handling of navigation errors / invalid URIs (TASK 17 ×9).
 3. Continue queue: TASK 18 local recording, TASK 21 media source, TASK 22 audio sync offset.
@@ -856,9 +856,7 @@
                                                                     </DataTrigger>
                                                                     <DataTrigger Binding="{Binding IsWebSource}" Value="True">
                                                                         <Setter Property="Visibility" Value="Visible"/>
-                                                                         <Setter Property="Stretch" Value="Uniform"/>
-                                                                         <Setter Property="HorizontalAlignment" Value="Left"/>
-                                                                         <Setter Property="VerticalAlignment" Value="Top"/>
+                                                                         <Setter Property="Stretch" Value="Fill"/>
                                                                     </DataTrigger>
                                                                     <MultiDataTrigger>
                                                                        <MultiDataTrigger.Conditions>
@@ -15,20 +15,6 @@ namespace ytLive.Services;

 public sealed class WebView2Manager : IDisposable
 {
-    // Eliminates the wrapper padding/whitespace between the box and the page:
-    // clears UA margins/paddings, forces html/body to fill the viewport and
-    // anchors content at (0,0) so the rendered frame matches the widget's size.
-    private const string CssResetInjection =
-        "(function(){" +
-        "var s=document.createElement('style');" +
-        "s.textContent='html,body{margin:0!important;padding:0!important;" +
-        "width:100%!important;height:100%!important;overflow:hidden!important;" +
-        "background:transparent!important}';" +
-        "(document.head||document.documentElement).appendChild(s);" +
-        "if(document.documentElement)document.documentElement.style.background='transparent';" +
-        "if(document.body)document.body.style.background='transparent';" +
-        "})();";
-
    private readonly Panel _hostPanel;
    private readonly Dispatcher _dispatcher;
    private readonly Dictionary<string, WebSourceSession> _sessions = new();
@@ -54,13 +40,14 @@ public sealed class WebView2Manager : IDisposable
    {
        if (_sessions.ContainsKey(source.Id)) return;

-        // OBS pattern: the browser viewport is the canvas size (1920×1080),
-        // never the element box. The page renders full at its designed size;
-        // the preview scales the captured frame independently (Stretch=Uniform).
+        // Geometry-agnostic OBS model: the browser viewport IS the element box.
+        // The page renders at exactly Source.Width × Source.Height, so the
+        // captured frame and the selection rect are always the same rectangle —
+        // no fixed render size, no aspect assumptions, no dead space or padding.
        var webView = new WebView2
        {
-            Width = 1920,
-            Height = 1080,
+            Width = Math.Max(1, source.Width),
+            Height = Math.Max(1, source.Height),
            DefaultBackgroundColor = System.Drawing.Color.Transparent,
            Visibility = System.Windows.Visibility.Visible,
        };
@@ -82,6 +69,22 @@ public sealed class WebView2Manager : IDisposable

        _sessions[source.Id] = session;

+        // Interop sync: keep the viewport locked to the element box so the
+        // selection rect always bounds the actual rendered content.
+        source.PropertyChanged += (_, e) =>
+        {
+            if (e.PropertyName is not (nameof(Source.Width) or nameof(Source.Height))) return;
+            if (session.Disposed) return;
+            if (Math.Abs(session.Control.Width - source.Width) > 0.5 ||
+                Math.Abs(session.Control.Height - source.Height) > 0.5)
+            {
+                session.Control.Width = Math.Max(1, source.Width);
+                session.Control.Height = Math.Max(1, source.Height);
+                if (session.Initialized)
+                    _ = CaptureFrame(source.Id);
+            }
+        };
+
        timer.Tick += (_, _) =>
        {
            if (!session.Disposed && session.Initialized)
@@ -122,11 +125,18 @@ public sealed class WebView2Manager : IDisposable

            var cws = session.Control.CoreWebView2!;

-            await cws.AddScriptToExecuteOnDocumentCreatedAsync(CssResetInjection);
+            cws.NavigationStarting += (_, _) =>
+            {
+                _ = cws.ExecuteScriptAsync(
+                    "document.documentElement.style.background='transparent';" +
+                    "document.body.style.background='transparent';");
+            };

            cws.NavigationCompleted += (_, _) =>
            {
-                _ = cws.ExecuteScriptAsync(CssResetInjection);
+                _ = cws.ExecuteScriptAsync(
+                    "document.documentElement.style.background='transparent';" +
+                    "document.body.style.background='transparent';");
                _ = CaptureFrame(source.Id);
            };

@@ -917,18 +917,15 @@ capturing the primary display (which causes the infinity mirror effect).
 8. ✅ Preview shows live web content (not just a blank rectangle)
 9. ☐ Handle navigation errors, invalid URIs, timeout gracefully

-**Rendering model (2026-08-28, the fix that works):** WebView2 viewport is FIXED at the master
-canvas size (1920×1080) in `Register`, never tracking the element box — the OBS/CEF pattern
-(fixed render viewport, display box scaled independently via `Stretch="Uniform"`). Capturing the
-box-sized viewport was the truncation bug. Display `Stretch`: web → `Uniform` (no crop),
-everything else → `UniformToFill`. Graceful-error handling (item 9) is a follow-up.
-
-**Box-vs-content padding (2026-08-28):** whitespace gaps between the bounding box and the web
-content were the page's own UA margins/padding + body sizing to content instead of the viewport.
-Fixed by injecting a CSS reset into every document via
-`CoreWebView2.AddScriptToExecuteOnDocumentCreatedAsync` (margin/padding 0, html/body 100% of
-viewport, overflow hidden, transparent bg) and anchoring the web `<Image>` at (0,0)
-(`HorizontalAlignment/VerticalAlignment=Left/Top`) inside the element box.
+**Rendering model (2026-08-28, geometry-agnostic — supersedes the earlier fixed-viewport model):**
+the WebView2 viewport IS the element box (`Source.Width × Source.Height`); the page renders at
+exactly the box size and the captured frame is displayed with `Stretch="Fill"`, so the selection
+rect always bounds the actual rendered content — no fixed render size, no aspect assumptions,
+no dead space/padding for any widget geometry. A `PropertyChanged` hook keeps the viewport
+locked to the box; capture is re-triggered on resize. Graceful-error handling (item 9) is a
+follow-up. Earlier attempts for the record: (a) fixed 1920×1080 viewport + `Stretch="Uniform"`
+fixed the truncation but left box-vs-content gaps (box aspect ≠ capture aspect); (b) CSS reset
+injection made gaps larger (belayed by creator).

 ### Design decisions

@@ -20,29 +20,7 @@ public sealed class WebView2ManagerTests
    }

    [Fact]
-    public void WebView2_Control_Size_Is_Fixed_At_Canvas_Resolution()
-    {
-        _app.Run(() =>
-        {
-            var panel = new Canvas();
-            var source = new Source
-            {
-                Type = SourceType.WebSource,
-                Width = 640,
-                Height = 480,
-            };
-
-            using var manager = new WebView2Manager(panel, Dispatcher.CurrentDispatcher);
-            manager.Register(source);
-
-            var webView = panel.Children.OfType<WebView2>().First();
-            Assert.Equal(1920.0, webView.Width);
-            Assert.Equal(1080.0, webView.Height);
-        });
-    }
-
-    [Fact]
-    public void WebView2_Control_Size_Does_Not_Track_Source_Dimensions()
+    public void WebView2_Control_Size_Tracks_Source_Dimensions()
    {
        _app.Run(() =>
        {
@@ -58,12 +36,62 @@ public sealed class WebView2ManagerTests
            manager.Register(source);

            var webView = panel.Children.OfType<WebView2>().First();
+            Assert.Equal(640.0, webView.Width);
+            Assert.Equal(480.0, webView.Height);

            source.Width = 1024;
            source.Height = 768;

-            Assert.Equal(1920.0, webView.Width);
-            Assert.Equal(1080.0, webView.Height);
+            Assert.Equal(1024.0, webView.Width);
+            Assert.Equal(768.0, webView.Height);
+        });
+    }
+
+    [Fact]
+    public void WebView2_Control_Size_Ignores_Sub_One_Pixel_Drift()
+    {
+        _app.Run(() =>
+        {
+            var panel = new Canvas();
+            var source = new Source
+            {
+                Type = SourceType.WebSource,
+                Width = 640,
+                Height = 480,
+            };
+
+            using var manager = new WebView2Manager(panel, Dispatcher.CurrentDispatcher);
+            manager.Register(source);
+
+            var webView = panel.Children.OfType<WebView2>().First();
+
+            source.Width = 640.4;
+            Assert.Equal(640.0, webView.Width);
+
+            source.Height = 480.4;
+            Assert.Equal(480.0, webView.Height);
+        });
+    }
+
+    [Fact]
+    public void WebView2_Control_Size_Minimum_Is_One()
+    {
+        _app.Run(() =>
+        {
+            var panel = new Canvas();
+            var source = new Source
+            {
+                Type = SourceType.WebSource,
+                Width = 0,
+                Height = 0,
+            };
+
+            using var manager = new WebView2Manager(panel, Dispatcher.CurrentDispatcher);
+            manager.Register(source);
+
+            var webView = panel.Children.OfType<WebView2>().First();
+            Assert.Equal(1.0, webView.Width);
+            Assert.Equal(1.0, webView.Height);
        });
    }