Web source: container == content — element grid + selection box sized to the cropped frame's canvas extent

- FindContentBounds already trimmed the bitmap; the grid/box still rendered at the uncropped user
  size, so Fill stretched the tight widget into a bigger frame = dead-between-content-and-border.
- ContentCanvasSize maps the crop rect to canvas units (1920/pixW per axis, DPI-correct) and the
  preview event now carries (bitmap, canvasW, canvasH).
- OnWebView2PreviewBitmapChanged sets the element Width/Height to that size; the grid and the
  SelectionOverlay (both bound to the element's Width/Height) now equal the widget exactly.
- IsPreviewDragging (set in MainWindow drag/resize) suppresses auto-fit mid-gesture.
- Tests 10/10 (alpha bounds, canvas-size mapping incl. DPI, control size); 0 warnings.
This commit is contained in:
2026-08-28 12:08:04 -07:00
parent bccdb4892c
commit ab29ec801d
7 changed files with 103 additions and 51 deletions
+26 -28
View File
@@ -6,36 +6,34 @@
working). **NOT pushed — user rule: never push without explicit instruction (2026-08-24).** No feature working). **NOT pushed — user rule: never push without explicit instruction (2026-08-24).** No feature
branches pre-1.0: all work lands on `main` per work unit. branches pre-1.0: all work lands on `main` per work unit.
## The web-source rendering work (2026-08-28) — CURRENT MODEL: canvas-size viewport + ALPHA-BBOX crop ## The web-source rendering work (2026-08-28) — CURRENT MODEL: canvas-size viewport + alpha crop + container==content
**History of failures (see MyMistakes.md for the full log):** box-size rendering clipped full-bleed **History of failures (see MyMistakes.md for the full log):**
widgets to their upper-left chunk (fixed — page renders at the master canvas 1920×1080, stable, the 1. box-sized viewport clipped full-bleed widgets to their upper-left chunk → render at the master
one render the creator confirmed solid). Then four crop attempts failed in different ways: (1) the canvas 1920×1080 (stable), the only render the creator confirmed solid.
`JSON.stringify` return double-encoded under `ExecuteScriptAsync`, silently disabling the crop; 2. `JSON.stringify` return double-encoded under `ExecuteScriptAsync` → crop silently disabled.
(2) `scrollWidth/scrollHeight` at canvas size returns the whole viewport for full-canvas wrapper 3. `scrollWidth/scrollHeight` at canvas size = whole viewport for full-canvas wrapper bodies → no trim.
bodies → no trim (ghost boundary); (3) union-of-DOM-elements measured an unsettled layout on 4. DOM-element-union crop measured an unsettled layout on NavigationCompleted → BROKE the image.
NavigationCompleted and BROKE the image (creator's "don't break the rendering" boundary crossed). 5. Alpha-bbox crop (from the frame itself) fixed the BITMAP but the grid/box stayed at the uncropped
(4) Reverting to f39ebd9's scroll-crop was another dead end (#2 again). user size → Fill stretched the tight widget into a big frame → dead space between content and border.
**The CURRENT MODEL (this commit):** **The CURRENT MODEL (this change):**
- `Services/WebView2Manager.cs` — page renders at 1920×1080 (stable, never tracked); `overflow:hidden` - `Services/WebView2Manager.cs` — page renders at 1920×1080 (stable, never tracked); `overflow:hidden`
+ transparent-bg injection. `FindContentBounds` scans the Bgra32 capture per frame and crops to the + transparent-bg injection. `FindContentBounds` crops the Bgra32 capture to the bounding box of
bounding box of NON-TRANSPARENT pixels — the widget's true rendered extent, measured from the frame non-transparent pixels each frame (immune to timing, can never truncate content). `ContentCanvasSize`
itself. No DOM query, no layout timing: it can never truncate visible content (worst case = full maps the crop rect to CANVAS UNITS (DPI-correct). Preview event now carries (bitmap, canvasW, canvasH).
frame = the confirmed-good image). `Stretch="Fill"` maps the cropped frame flush under the selection - `ViewModels/MainViewModel.cs` — `OnWebView2PreviewBitmapChanged` sets the element's Width/Height to
box → the box hugs the widget on all four sides. `QueryContentBoundsAsync` + the JS script are the cropped canvas size → the element grid AND the SelectionOverlay (both bound to element
DELETED. Width/Height) EQUAL the widget. `IsPreviewDragging` suppresses the auto-fit during a drag gesture.
- `MainWindow.xaml` — web `<Image>` Stretch set to Fill via the IsWebSource DataTrigger (unchanged). - `MainWindow.xaml.cs` — sets/clears `IsPreviewDragging` at drag/resize start and end.
- Invariant: **render at canvas size; crop to the alpha extent; Fill to the box.** - Invariant: **render at canvas size; crop to the alpha extent; SIZE THE BOX to the crop.**
- Unit tests: 3 new for `FindContentBounds` (tight box, no-alpha no-crop, full-alpha no-crop). - Unit tests: 10/10 (alpha bounds ×3, canvas-size mapping ×3, control size ×4).
**Property panel / sliders (2026-08-28):** web URI ✕ clears the textbox; web ✓/✕ icon buttons **Property panel / sliders (2026-08-28):** web URI ✕ clears the textbox; web ✓/✕ icon buttons
`IsTabStop="False"` so Tab walks X→Y→W→H→URI; Slider style `IsMoveToPointEnabled="True"` `IsTabStop="False"`; Slider style `IsMoveToPointEnabled="True"`. Confirmed fixed by the creator.
restores click-anywhere-on-bar. All three confirmed fixed by the creator.
Build: 0 warnings; tests 7/7 (WebView2Manager). Rejected for the record: box-sized viewport (upper-left Build: 0 warnings; tests 10/10. Rejected for the record: box-sized viewport, `Uniform`, CSS reset,
truncation), fixed 1920×1080 + `Uniform` (letterbox dead space), CSS margin/100% reset (enlarged gap), viewport==box, DOM-union crop (broke image), crop-without-container-size (left the gap).
viewport==box (reflow/truncation/scrollbars), DOM-union crop (broke the image).
## Key facts / landmines ## Key facts / landmines
@@ -52,8 +50,8 @@ viewport==box (reflow/truncation/scrollbars), DOM-union crop (broke the image).
- No `CODING.md`; conventions live in `AGENTS.md` + `ai.md`. - No `CODING.md`; conventions live in `AGENTS.md` + `ai.md`.
## Next steps ## Next steps
1. AWAIT creator's verdict on the ALPHA-BBOX crop — reload a widget URL at several box sizes; the 1. AWAIT creator's verdict on the container==content auto-fit — reload a widget; the selection frame
selection frame must snap flush to the widget with no dead space, image must stay identical for must equal the widget exactly (no dead space, including during resize). Follow-ups only if asked:
full-bleed widgets. If an animated widget causes visible jitter, add crop hysteresis as a follow-up. crop hysteresis for animated widgets, ticker/percent-display widgets.
2. Follow-up only if the creator asks: graceful handling of navigation errors / invalid URIs (TASK 17 ×9). 2. Graceful handling of navigation errors / invalid URIs (TASK 17 ×9) — only if the creator asks.
3. Continue queue: TASK 18 local recording, TASK 21 media source, TASK 22 audio sync offset. 3. Continue queue: TASK 18 local recording, TASK 21 media source, TASK 22 audio sync offset.
+3
View File
@@ -572,6 +572,7 @@ public partial class MainWindow : Window
{ {
_isResizing = true; _isResizing = true;
_resizeAspect = sel.ClipShape == ClipShape.Round ? 1 : sel.Width / Math.Max(1, sel.Height); _resizeAspect = sel.ClipShape == ClipShape.Round ? 1 : sel.Width / Math.Max(1, sel.Height);
_viewModel.IsPreviewDragging = true;
grid.CaptureMouse(); grid.CaptureMouse();
e.Handled = true; e.Handled = true;
return; return;
@@ -583,6 +584,7 @@ public partial class MainWindow : Window
_viewModel.SelectedElement = hit; _viewModel.SelectedElement = hit;
_isDraggingOverlay = true; _isDraggingOverlay = true;
_grabOffset = new Point(p.X - hit.X, p.Y - hit.Y); _grabOffset = new Point(p.X - hit.X, p.Y - hit.Y);
_viewModel.IsPreviewDragging = true;
grid.CaptureMouse(); grid.CaptureMouse();
e.Handled = true; e.Handled = true;
return; return;
@@ -643,6 +645,7 @@ public partial class MainWindow : Window
grid.ReleaseMouseCapture(); grid.ReleaseMouseCapture();
_isDraggingOverlay = false; _isDraggingOverlay = false;
_isResizing = false; _isResizing = false;
_viewModel.IsPreviewDragging = false;
} }
private bool HitHandle(Point mouseScreen, SceneElement element) private bool HitHandle(Point mouseScreen, SceneElement element)
+11 -13
View File
@@ -31,17 +31,15 @@ Stream widget bounding box: hug the widget on all four sides, pinned to canvas (
extents regardless. Lesson: verify declared mechanics actually execute; assume nothing ran. extents regardless. Lesson: verify declared mechanics actually execute; assume nothing ran.
## Current iteration (web bounding box — UNCOMMITTED, verification pending) ## Current iteration (web bounding box — UNCOMMITTED, verification pending)
ALPHA-BBOX CROP: `FindContentBounds` scans the captured Bgra32 frame and returns the bounding box CONTAINER == CONTENT. The element grid and the selection box now SIZE to the cropped frame:
of non-transparent pixels (the background is injected transparent, so alpha = the widget's extent). `WebView2Manager.ContentCanvasSize` maps the alpha-crop pixel rect to MASTER CANVAS units
Capture is cropped to that rect and `Stretch="Fill"` maps it flush under the box. No DOM query, no (1920/pixW per axis, DPI-correct) and the preview event carries it; `MainViewModel` sets the
layout timing — the crop rect IS the rendered widget by construction, so it can NEVER truncate Source.Width/Height (the grid + SelectionOverlay bindings) to that size. No Fill-stretching of a
visible content. Worst case (widget fills the canvas) = full frame = the confirmed-good render. cropped bitmap into a larger frame — the box IS the widget. Auto-fit is suppressed while the
Canvas-size viewport + Fill untouched. `QueryContentBoundsAsync` + the JS script are DELETED. drag/resize gesture is active (`IsPreviewDragging`, set by MainWindow code-behind).
## Iteration that just failed (`a5b9952` — wrapper-excluded DOM-union crop + image alignment) ## Iteration that just failed (`bccdb48` — alpha crop, but container stayed at Source.Width/Height)
Broke the image rendering — creator could not evaluate the box because the image was gone/ruined. Crop trimmed the bitmap but the element grid/box still rendered at the UNcropped user-set size, so
Why: the visible-element-union rect is measured ONCE on NavigationCompleted; widget layout isn't Fill stretched the tight widget across a larger frame and dead space floated between content bounds
settled then (async fonts/images, load animations, iframes), so the crop rect comes out tiny/wrong and the selection border. Lesson: fixing the BITMAP is not fixing the BOX — the layout container
and the Fill-stretched frame renders a broken/zoomed sliver. This was against the creator's explicit must match the cropped dimensions or the gap persists by construction.
"don't break the rendering" instruction. Lesson: measure actual RENDERED PIXELS (alpha), never
transient DOM state; and never ship a crop blind.
+11 -2
View File
@@ -20,7 +20,7 @@ public sealed class WebView2Manager : IDisposable
private readonly Dispatcher _dispatcher; private readonly Dispatcher _dispatcher;
private readonly Dictionary<string, WebSourceSession> _sessions = new(); private readonly Dictionary<string, WebSourceSession> _sessions = new();
public event Action<string, WriteableBitmap>? PreviewBitmapChanged; public event Action<string, WriteableBitmap, double, double>? PreviewBitmapChanged;
private sealed class WebSourceSession private sealed class WebSourceSession
{ {
@@ -182,6 +182,14 @@ public sealed class WebView2Manager : IDisposable
return (minX, minY, maxX - minX + 1, maxY - minY + 1); return (minX, minY, maxX - minX + 1, maxY - minY + 1);
} }
// The cropped pixel extent, expressed in MASTER CANVAS units (the WebView2 is
// 1920×1080 DIPs and captures at its physical size, so 1920/pixW maps capture
// pixels back to canvas units). The element grid + selection box are sized to
// this, so the container matches the content exactly instead of Fill-stretching
// a cropped bitmap into a larger frame.
internal static (double W, double H) ContentCanvasSize(int cropW, int cropH, int pixW, int pixH)
=> (cropW * (1920.0 / pixW), cropH * (1080.0 / pixH));
private async Task CaptureFrame(string sourceId) private async Task CaptureFrame(string sourceId)
{ {
if (!_sessions.TryGetValue(sourceId, out var session) || session.Disposed) return; if (!_sessions.TryGetValue(sourceId, out var session) || session.Disposed) return;
@@ -232,11 +240,12 @@ public sealed class WebView2Manager : IDisposable
wb.WritePixels(new Int32Rect(0, 0, cropW, cropH), outPixels, cropW * 4, 0); wb.WritePixels(new Int32Rect(0, 0, cropW, cropH), outPixels, cropW * 4, 0);
wb.Freeze(); wb.Freeze();
var (canvasW, canvasH) = ContentCanvasSize(cropW, cropH, pixW, pixH);
session.LatestFrame = new VideoFrame(cropW, cropH, outPixels); session.LatestFrame = new VideoFrame(cropW, cropH, outPixels);
_ = _dispatcher.BeginInvoke(() => _ = _dispatcher.BeginInvoke(() =>
{ {
PreviewBitmapChanged?.Invoke(sourceId, wb); PreviewBitmapChanged?.Invoke(sourceId, wb, canvasW, canvasH);
}); });
} }
catch (ObjectDisposedException) { } catch (ObjectDisposedException) { }
+10 -7
View File
@@ -917,16 +917,19 @@ capturing the primary display (which causes the infinity mirror effect).
8. ✅ Preview shows live web content (not just a blank rectangle) 8. ✅ Preview shows live web content (not just a blank rectangle)
9. ☐ Handle navigation errors, invalid URIs, timeout gracefully 9. ☐ Handle navigation errors, invalid URIs, timeout gracefully
**Rendering model (2026-08-28, canvas-size viewport + ALPHA-BBOX crop):** **Rendering model (2026-08-28, canvas-size viewport + alpha crop + CONTAINER==CONTENT):**
the page renders at the MASTER CANVAS size (1920×1080), stable, never tracked (no reflow/truncation; the page renders at the MASTER CANVAS size (1920×1080), stable, never tracked (no reflow/truncation;
scrollbars suppressed via `overflow:hidden`). `FindContentBounds` scans the Bgra32 capture and crops scrollbars suppressed via `overflow:hidden`). `FindContentBounds` scans the Bgra32 capture and crops
to the bounding box of non-transparent pixels — the widget's true rendered extent, measured from the to the bounding box of non-transparent pixels — the widget's true rendered extent, measured from the
frame itself (NO DOM query, immune to layout timing; can never truncate content; full-canvas widgets frame itself (NO DOM query, immune to layout timing, can never truncate content; full-canvas widgets
fall through to the full frame = prior verified-good image). `Stretch="Fill"` maps the cropped frame fall through to the full frame). `ContentCanvasSize` maps the cropped pixel rect to canvas units
flush under the selection box → box hugs the widget on all four sides. `QueryContentBoundsAsync` + (DPI-correct); `MainViewModel.OnWebView2PreviewBitmapChanged` sizes the element's Width/Height to it —
the JS content-bounds script are DELETED. ⚠️ Lesson bank: never return `JSON.stringify` from so the element grid AND the SelectionOverlay (both bound to the element's Width/Height) equal the
`ExecuteScriptAsync` (double-encodes); never measure DOM stuff on NavigationCompleted (unsettled widget exactly. Auto-fit suppressed while dragging (`IsPreviewDragging`). `Stretch="Fill"` is now
layout broke the image — `a5b9952`); measure rendered pixels instead. ~1:1; no cropped bitmap is stretched into a larger frame.
⚠️ Lesson bank: never return `JSON.stringify` from `ExecuteScriptAsync` (double-encodes); never
measure DOM on NavigationCompleted (unsettled layout broke the image — `a5b9952`); fixing the
bitmap crop without sizing the container leaves the gap (— `bccdb48`).
**Properties panel (2026-08-28):** web URI ✓/✕ icon buttons are `IsTabStop="False"` so Tab flows **Properties panel (2026-08-28):** web URI ✓/✕ icon buttons are `IsTabStop="False"` so Tab flows
X→Y→W→H→URI; the ✕ button now clears the URI textbox (was reverting to the pre-accept snapshot). X→Y→W→H→URI; the ✕ button now clears the URI textbox (was reverting to the pre-accept snapshot).
+21 -1
View File
@@ -2290,6 +2290,11 @@ public class MainViewModel : ViewModelBase
_ = RedesignateBackgroundAsync($"{MonitorKeyPrefix}{display.Index}"); _ = RedesignateBackgroundAsync($"{MonitorKeyPrefix}{display.Index}");
} }
// True while the user is dragging/resizing an element on the preview canvas
// (set by MainWindow code-behind). Suppresses the web source's auto-fit so it
// doesn't fight the gesture mid-drag.
public bool IsPreviewDragging { get; set; }
public void InitWebView2(Panel hostPanel) public void InitWebView2(Panel hostPanel)
{ {
_webView2Manager = new WebView2Manager(hostPanel, System.Windows.Application.Current.Dispatcher); _webView2Manager = new WebView2Manager(hostPanel, System.Windows.Application.Current.Dispatcher);
@@ -2297,11 +2302,26 @@ public class MainViewModel : ViewModelBase
RegisterLoadedWebSources(); RegisterLoadedWebSources();
} }
private void OnWebView2PreviewBitmapChanged(string sourceId, WriteableBitmap bitmap) private void OnWebView2PreviewBitmapChanged(string sourceId, WriteableBitmap bitmap, double canvasW, double canvasH)
{ {
foreach (var scene in Scenes) foreach (var scene in Scenes)
foreach (var el in scene.Elements.OfType<Source>().Where(s => s.Id == sourceId)) foreach (var el in scene.Elements.OfType<Source>().Where(s => s.Id == sourceId))
{
el.VideoImageSource = bitmap; el.VideoImageSource = bitmap;
// The container must equal the content: size the element grid
// (and therefore the selection box) to the cropped frame's canvas
// extent. Skipped while the user is dragging/resizing the element
// so the gesture isn't fought mid-drag.
if (IsPreviewDragging || canvasW < 1 || canvasH < 1) continue;
var w = Math.Clamp(Math.Round(canvasW), 1, 1920);
var h = Math.Clamp(Math.Round(canvasH), 1, 1080);
if (Math.Abs(el.Width - w) > 1 || Math.Abs(el.Height - h) > 1)
{
el.Width = w;
el.Height = h;
}
}
} }
public void Shutdown() public void Shutdown()
+21
View File
@@ -154,4 +154,25 @@ public sealed class WebView2ManagerTests
Assert.Equal((0, 0, w, h), (x, y, bw, bh)); Assert.Equal((0, 0, w, h), (x, y, bw, bh));
} }
[Fact]
public void ContentCanvasSize_Full_Frame_Maps_To_Canvas()
{
var (cw, ch) = WebView2Manager.ContentCanvasSize(1920, 1080, 1920, 1080);
Assert.Equal((1920.0, 1080.0), (cw, ch));
}
[Fact]
public void ContentCanvasSize_Half_Widget_Is_Half_Canvas()
{
var (cw, ch) = WebView2Manager.ContentCanvasSize(960, 540, 1920, 1080);
Assert.Equal((960.0, 540.0), (cw, ch));
}
[Fact]
public void ContentCanvasSize_Corrects_For_Dpi_Scale()
{
var (cw, ch) = WebView2Manager.ContentCanvasSize(1200, 675, 2400, 1350);
Assert.Equal((960.0, 540.0), (cw, ch));
}
} }