Web source: container == content — element grid + selection box sized to the cropped frame's canvas extent

- FindContentBounds already trimmed the bitmap; the grid/box still rendered at the uncropped user
  size, so Fill stretched the tight widget into a bigger frame = dead-between-content-and-border.
- ContentCanvasSize maps the crop rect to canvas units (1920/pixW per axis, DPI-correct) and the
  preview event now carries (bitmap, canvasW, canvasH).
- OnWebView2PreviewBitmapChanged sets the element Width/Height to that size; the grid and the
  SelectionOverlay (both bound to the element's Width/Height) now equal the widget exactly.
- IsPreviewDragging (set in MainWindow drag/resize) suppresses auto-fit mid-gesture.
- Tests 10/10 (alpha bounds, canvas-size mapping incl. DPI, control size); 0 warnings.
This commit is contained in:
2026-08-28 12:08:04 -07:00
parent bccdb4892c
commit ab29ec801d
7 changed files with 103 additions and 51 deletions
+26 -28
View File
@@ -6,36 +6,34 @@
working). **NOT pushed — user rule: never push without explicit instruction (2026-08-24).** No feature
branches pre-1.0: all work lands on `main` per work unit.
## The web-source rendering work (2026-08-28) — CURRENT MODEL: canvas-size viewport + ALPHA-BBOX crop
## The web-source rendering work (2026-08-28) — CURRENT MODEL: canvas-size viewport + alpha crop + container==content
**History of failures (see MyMistakes.md for the full log):** box-size rendering clipped full-bleed
widgets to their upper-left chunk (fixed — page renders at the master canvas 1920×1080, stable, the
one render the creator confirmed solid). Then four crop attempts failed in different ways: (1) the
`JSON.stringify` return double-encoded under `ExecuteScriptAsync`, silently disabling the crop;
(2) `scrollWidth/scrollHeight` at canvas size returns the whole viewport for full-canvas wrapper
bodies → no trim (ghost boundary); (3) union-of-DOM-elements measured an unsettled layout on
NavigationCompleted and BROKE the image (creator's "don't break the rendering" boundary crossed).
(4) Reverting to f39ebd9's scroll-crop was another dead end (#2 again).
**History of failures (see MyMistakes.md for the full log):**
1. box-sized viewport clipped full-bleed widgets to their upper-left chunk → render at the master
canvas 1920×1080 (stable), the only render the creator confirmed solid.
2. `JSON.stringify` return double-encoded under `ExecuteScriptAsync` → crop silently disabled.
3. `scrollWidth/scrollHeight` at canvas size = whole viewport for full-canvas wrapper bodies → no trim.
4. DOM-element-union crop measured an unsettled layout on NavigationCompleted → BROKE the image.
5. Alpha-bbox crop (from the frame itself) fixed the BITMAP but the grid/box stayed at the uncropped
user size → Fill stretched the tight widget into a big frame → dead space between content and border.
**The CURRENT MODEL (this commit):**
**The CURRENT MODEL (this change):**
- `Services/WebView2Manager.cs` — page renders at 1920×1080 (stable, never tracked); `overflow:hidden`
+ transparent-bg injection. `FindContentBounds` scans the Bgra32 capture per frame and crops to the
bounding box of NON-TRANSPARENT pixels — the widget's true rendered extent, measured from the frame
itself. No DOM query, no layout timing: it can never truncate visible content (worst case = full
frame = the confirmed-good image). `Stretch="Fill"` maps the cropped frame flush under the selection
box → the box hugs the widget on all four sides. `QueryContentBoundsAsync` + the JS script are
DELETED.
- `MainWindow.xaml` — web `<Image>` Stretch set to Fill via the IsWebSource DataTrigger (unchanged).
- Invariant: **render at canvas size; crop to the alpha extent; Fill to the box.**
- Unit tests: 3 new for `FindContentBounds` (tight box, no-alpha no-crop, full-alpha no-crop).
+ transparent-bg injection. `FindContentBounds` crops the Bgra32 capture to the bounding box of
non-transparent pixels each frame (immune to timing, can never truncate content). `ContentCanvasSize`
maps the crop rect to CANVAS UNITS (DPI-correct). Preview event now carries (bitmap, canvasW, canvasH).
- `ViewModels/MainViewModel.cs` — `OnWebView2PreviewBitmapChanged` sets the element's Width/Height to
the cropped canvas size → the element grid AND the SelectionOverlay (both bound to element
Width/Height) EQUAL the widget. `IsPreviewDragging` suppresses the auto-fit during a drag gesture.
- `MainWindow.xaml.cs` — sets/clears `IsPreviewDragging` at drag/resize start and end.
- Invariant: **render at canvas size; crop to the alpha extent; SIZE THE BOX to the crop.**
- Unit tests: 10/10 (alpha bounds ×3, canvas-size mapping ×3, control size ×4).
**Property panel / sliders (2026-08-28):** web URI ✕ clears the textbox; web ✓/✕ icon buttons
`IsTabStop="False"` so Tab walks X→Y→W→H→URI; Slider style `IsMoveToPointEnabled="True"`
restores click-anywhere-on-bar. All three confirmed fixed by the creator.
`IsTabStop="False"`; Slider style `IsMoveToPointEnabled="True"`. Confirmed fixed by the creator.
Build: 0 warnings; tests 7/7 (WebView2Manager). Rejected for the record: box-sized viewport (upper-left
truncation), fixed 1920×1080 + `Uniform` (letterbox dead space), CSS margin/100% reset (enlarged gap),
viewport==box (reflow/truncation/scrollbars), DOM-union crop (broke the image).
Build: 0 warnings; tests 10/10. Rejected for the record: box-sized viewport, `Uniform`, CSS reset,
viewport==box, DOM-union crop (broke image), crop-without-container-size (left the gap).
## Key facts / landmines
@@ -52,8 +50,8 @@ viewport==box (reflow/truncation/scrollbars), DOM-union crop (broke the image).
- No `CODING.md`; conventions live in `AGENTS.md` + `ai.md`.
## Next steps
1. AWAIT creator's verdict on the ALPHA-BBOX crop — reload a widget URL at several box sizes; the
selection frame must snap flush to the widget with no dead space, image must stay identical for
full-bleed widgets. If an animated widget causes visible jitter, add crop hysteresis as a follow-up.
2. Follow-up only if the creator asks: graceful handling of navigation errors / invalid URIs (TASK 17 ×9).
1. AWAIT creator's verdict on the container==content auto-fit — reload a widget; the selection frame
must equal the widget exactly (no dead space, including during resize). Follow-ups only if asked:
crop hysteresis for animated widgets, ticker/percent-display widgets.
2. Graceful handling of navigation errors / invalid URIs (TASK 17 ×9) — only if the creator asks.
3. Continue queue: TASK 18 local recording, TASK 21 media source, TASK 22 audio sync offset.
+3
View File
@@ -572,6 +572,7 @@ public partial class MainWindow : Window
{
_isResizing = true;
_resizeAspect = sel.ClipShape == ClipShape.Round ? 1 : sel.Width / Math.Max(1, sel.Height);
_viewModel.IsPreviewDragging = true;
grid.CaptureMouse();
e.Handled = true;
return;
@@ -583,6 +584,7 @@ public partial class MainWindow : Window
_viewModel.SelectedElement = hit;
_isDraggingOverlay = true;
_grabOffset = new Point(p.X - hit.X, p.Y - hit.Y);
_viewModel.IsPreviewDragging = true;
grid.CaptureMouse();
e.Handled = true;
return;
@@ -643,6 +645,7 @@ public partial class MainWindow : Window
grid.ReleaseMouseCapture();
_isDraggingOverlay = false;
_isResizing = false;
_viewModel.IsPreviewDragging = false;
}
private bool HitHandle(Point mouseScreen, SceneElement element)
+11 -13
View File
@@ -31,17 +31,15 @@ Stream widget bounding box: hug the widget on all four sides, pinned to canvas (
extents regardless. Lesson: verify declared mechanics actually execute; assume nothing ran.
## Current iteration (web bounding box — UNCOMMITTED, verification pending)
ALPHA-BBOX CROP: `FindContentBounds` scans the captured Bgra32 frame and returns the bounding box
of non-transparent pixels (the background is injected transparent, so alpha = the widget's extent).
Capture is cropped to that rect and `Stretch="Fill"` maps it flush under the box. No DOM query, no
layout timing — the crop rect IS the rendered widget by construction, so it can NEVER truncate
visible content. Worst case (widget fills the canvas) = full frame = the confirmed-good render.
Canvas-size viewport + Fill untouched. `QueryContentBoundsAsync` + the JS script are DELETED.
CONTAINER == CONTENT. The element grid and the selection box now SIZE to the cropped frame:
`WebView2Manager.ContentCanvasSize` maps the alpha-crop pixel rect to MASTER CANVAS units
(1920/pixW per axis, DPI-correct) and the preview event carries it; `MainViewModel` sets the
Source.Width/Height (the grid + SelectionOverlay bindings) to that size. No Fill-stretching of a
cropped bitmap into a larger frame — the box IS the widget. Auto-fit is suppressed while the
drag/resize gesture is active (`IsPreviewDragging`, set by MainWindow code-behind).
## Iteration that just failed (`a5b9952` — wrapper-excluded DOM-union crop + image alignment)
Broke the image rendering — creator could not evaluate the box because the image was gone/ruined.
Why: the visible-element-union rect is measured ONCE on NavigationCompleted; widget layout isn't
settled then (async fonts/images, load animations, iframes), so the crop rect comes out tiny/wrong
and the Fill-stretched frame renders a broken/zoomed sliver. This was against the creator's explicit
"don't break the rendering" instruction. Lesson: measure actual RENDERED PIXELS (alpha), never
transient DOM state; and never ship a crop blind.
## Iteration that just failed (`bccdb48` — alpha crop, but container stayed at Source.Width/Height)
Crop trimmed the bitmap but the element grid/box still rendered at the UNcropped user-set size, so
Fill stretched the tight widget across a larger frame and dead space floated between content bounds
and the selection border. Lesson: fixing the BITMAP is not fixing the BOX — the layout container
must match the cropped dimensions or the gap persists by construction.
+11 -2
View File
@@ -20,7 +20,7 @@ public sealed class WebView2Manager : IDisposable
private readonly Dispatcher _dispatcher;
private readonly Dictionary<string, WebSourceSession> _sessions = new();
public event Action<string, WriteableBitmap>? PreviewBitmapChanged;
public event Action<string, WriteableBitmap, double, double>? PreviewBitmapChanged;
private sealed class WebSourceSession
{
@@ -182,6 +182,14 @@ public sealed class WebView2Manager : IDisposable
return (minX, minY, maxX - minX + 1, maxY - minY + 1);
}
// The cropped pixel extent, expressed in MASTER CANVAS units (the WebView2 is
// 1920×1080 DIPs and captures at its physical size, so 1920/pixW maps capture
// pixels back to canvas units). The element grid + selection box are sized to
// this, so the container matches the content exactly instead of Fill-stretching
// a cropped bitmap into a larger frame.
internal static (double W, double H) ContentCanvasSize(int cropW, int cropH, int pixW, int pixH)
=> (cropW * (1920.0 / pixW), cropH * (1080.0 / pixH));
private async Task CaptureFrame(string sourceId)
{
if (!_sessions.TryGetValue(sourceId, out var session) || session.Disposed) return;
@@ -232,11 +240,12 @@ public sealed class WebView2Manager : IDisposable
wb.WritePixels(new Int32Rect(0, 0, cropW, cropH), outPixels, cropW * 4, 0);
wb.Freeze();
var (canvasW, canvasH) = ContentCanvasSize(cropW, cropH, pixW, pixH);
session.LatestFrame = new VideoFrame(cropW, cropH, outPixels);
_ = _dispatcher.BeginInvoke(() =>
{
PreviewBitmapChanged?.Invoke(sourceId, wb);
PreviewBitmapChanged?.Invoke(sourceId, wb, canvasW, canvasH);
});
}
catch (ObjectDisposedException) { }
+10 -7
View File
@@ -917,16 +917,19 @@ capturing the primary display (which causes the infinity mirror effect).
8. ✅ Preview shows live web content (not just a blank rectangle)
9. ☐ Handle navigation errors, invalid URIs, timeout gracefully
**Rendering model (2026-08-28, canvas-size viewport + ALPHA-BBOX crop):**
**Rendering model (2026-08-28, canvas-size viewport + alpha crop + CONTAINER==CONTENT):**
the page renders at the MASTER CANVAS size (1920×1080), stable, never tracked (no reflow/truncation;
scrollbars suppressed via `overflow:hidden`). `FindContentBounds` scans the Bgra32 capture and crops
to the bounding box of non-transparent pixels — the widget's true rendered extent, measured from the
frame itself (NO DOM query, immune to layout timing; can never truncate content; full-canvas widgets
fall through to the full frame = prior verified-good image). `Stretch="Fill"` maps the cropped frame
flush under the selection box → box hugs the widget on all four sides. `QueryContentBoundsAsync` +
the JS content-bounds script are DELETED. ⚠️ Lesson bank: never return `JSON.stringify` from
`ExecuteScriptAsync` (double-encodes); never measure DOM stuff on NavigationCompleted (unsettled
layout broke the image — `a5b9952`); measure rendered pixels instead.
frame itself (NO DOM query, immune to layout timing, can never truncate content; full-canvas widgets
fall through to the full frame). `ContentCanvasSize` maps the cropped pixel rect to canvas units
(DPI-correct); `MainViewModel.OnWebView2PreviewBitmapChanged` sizes the element's Width/Height to it —
so the element grid AND the SelectionOverlay (both bound to the element's Width/Height) equal the
widget exactly. Auto-fit suppressed while dragging (`IsPreviewDragging`). `Stretch="Fill"` is now
~1:1; no cropped bitmap is stretched into a larger frame.
⚠️ Lesson bank: never return `JSON.stringify` from `ExecuteScriptAsync` (double-encodes); never
measure DOM on NavigationCompleted (unsettled layout broke the image — `a5b9952`); fixing the
bitmap crop without sizing the container leaves the gap (— `bccdb48`).
**Properties panel (2026-08-28):** web URI ✓/✕ icon buttons are `IsTabStop="False"` so Tab flows
X→Y→W→H→URI; the ✕ button now clears the URI textbox (was reverting to the pre-accept snapshot).
+21 -1
View File
@@ -2290,6 +2290,11 @@ public class MainViewModel : ViewModelBase
_ = RedesignateBackgroundAsync($"{MonitorKeyPrefix}{display.Index}");
}
// True while the user is dragging/resizing an element on the preview canvas
// (set by MainWindow code-behind). Suppresses the web source's auto-fit so it
// doesn't fight the gesture mid-drag.
public bool IsPreviewDragging { get; set; }
public void InitWebView2(Panel hostPanel)
{
_webView2Manager = new WebView2Manager(hostPanel, System.Windows.Application.Current.Dispatcher);
@@ -2297,11 +2302,26 @@ public class MainViewModel : ViewModelBase
RegisterLoadedWebSources();
}
private void OnWebView2PreviewBitmapChanged(string sourceId, WriteableBitmap bitmap)
private void OnWebView2PreviewBitmapChanged(string sourceId, WriteableBitmap bitmap, double canvasW, double canvasH)
{
foreach (var scene in Scenes)
foreach (var el in scene.Elements.OfType<Source>().Where(s => s.Id == sourceId))
{
el.VideoImageSource = bitmap;
// The container must equal the content: size the element grid
// (and therefore the selection box) to the cropped frame's canvas
// extent. Skipped while the user is dragging/resizing the element
// so the gesture isn't fought mid-drag.
if (IsPreviewDragging || canvasW < 1 || canvasH < 1) continue;
var w = Math.Clamp(Math.Round(canvasW), 1, 1920);
var h = Math.Clamp(Math.Round(canvasH), 1, 1080);
if (Math.Abs(el.Width - w) > 1 || Math.Abs(el.Height - h) > 1)
{
el.Width = w;
el.Height = h;
}
}
}
public void Shutdown()
+21
View File
@@ -154,4 +154,25 @@ public sealed class WebView2ManagerTests
Assert.Equal((0, 0, w, h), (x, y, bw, bh));
}
[Fact]
public void ContentCanvasSize_Full_Frame_Maps_To_Canvas()
{
var (cw, ch) = WebView2Manager.ContentCanvasSize(1920, 1080, 1920, 1080);
Assert.Equal((1920.0, 1080.0), (cw, ch));
}
[Fact]
public void ContentCanvasSize_Half_Widget_Is_Half_Canvas()
{
var (cw, ch) = WebView2Manager.ContentCanvasSize(960, 540, 1920, 1080);
Assert.Equal((960.0, 540.0), (cw, ch));
}
[Fact]
public void ContentCanvasSize_Corrects_For_Dpi_Scale()
{
var (cw, ch) = WebView2Manager.ContentCanvasSize(1200, 675, 2400, 1350);
Assert.Equal((960.0, 540.0), (cw, ch));
}
}