Web source: off-screen WebView2 capture + layout persistence + transparent bg

Bundle exception (2026-08-27): this lands prior-session work plus today's
transparency fix as one commit per creator instruction.

WebView2Manager (new): off-screen WebView2 control parked at -5000/-5000 in
the window's visual tree, captures via CapturePreviewAsync every 100 ms into
a BGRA8 VideoFrame. Source Width/Height sync to the control via
PropertyChanged. JS transparent-background injection runs on every
NavigationStarting/NavigationCompleted so the page's own CSS does not paint
white over the capture.

Transparency fix: DefaultBackgroundColor = System.Drawing.Color.Transparent
set on the WPF control before EnsureCoreWebView2Async. The WPF SDK doc says
the value is forwarded to the controller on init; the IDL (ICoreWebView2-
Controller2) confirms alpha=0 makes the capture alpha-preserving. The
prior-session 'CoreWebView2.BackgroundColor' path was the wrong property
name and has been replaced.

LayoutStore: WebUri column added (idempotent ALTER TABLE) and round-tripped
on save/load (schema v11).

MainViewModel: WebView2Manager constructed on Loaded; loaded web sources
re-registered; Source.WebUri PropertyChanged forwards to manager.

MainWindow: off-screen host Canvas added to visual tree.

Tests: 3 scale-sync + WebView2_DefaultBackgroundColor_Is_Transparent +
LayoutStorePersistenceTests.WebSource_WebUri_Persists. 234/236 pass
(2 pre-existing failures unchanged).

Scale bug (DPI/physical-pixel mismatch) still open.
This commit is contained in:
2026-08-27 08:52:39 -07:00
parent f81048849a
commit fd6e02b687
10 changed files with 595 additions and 57 deletions
+104 -45
View File
@@ -1,57 +1,116 @@
# HANDOFF — Session State
## Branch
**`main`** @ `67baac6`, committed LOCALLY, working tree clean. **NOT pushed — user rule: never
push without explicit instruction (2026-08-24).** No feature branches pre-1.0: all work lands on
`main` per work unit.
**`main`** @ `67baac6`, working tree dirty (WebView2 web source feature + transparency fix).
**NOT pushed — user rule: never push without explicit instruction (2026-08-24).** No feature
branches pre-1.0: all work lands on `main` per work unit.
## Shipped (this session, all on main)
- TASK 26 — webcam (+) menu gates on the app-wide identity (test `WebcamMenuGateTests`).
- TASK 27 — YouTube Chat one-per-layout gate + legacy label heal (test `ChatLayerGateTests`).
- TASK 19/23 status flip — Control Surface UX verified shipped; map corrected.
- TASK 20 step 1 — global hotkeys F1-F9 (`Services/GlobalHotkeys.cs`, test `GlobalHotkeyTests`). Validated by creator.
- ON-AIR sign right of REC — gray offline / red public / green private (`OnAirBrush`), shared `StatusSignText` style. Test `OnAirSignTests`.
- **Broadcast metadata pull-out + launch geometry** (`b988e6d`) — white "Text" tab on preview's right edge slides out the broadcast form: Title, Description, Tags csv, Visibility, Made-for-Kids, read-only Scheduled Start. Edits persist to Settings `Broadcast.*` immediately; Update Broadcast PUTs `liveBroadcasts?part=snippet,status` via new `YouTubeStreamService.UpdateBroadcast`. Go Live prefills from the form + captures back what was inserted. Old Default Stream Title/Description removed from App Settings (properties deleted). Window: default 1920×1040, MinWidth 1366, MinHeight 768, Manual startup, size+position persisted on close and restored clamped. Test `BroadcastPullOutTests`.
- **Bug-fix pass 1** (`e6baa55` + `d9e352f`): Text tab **always visible** (creator revised the Live-only gating same day); tab **toggles** the drawer; **click-away collapses it** (`Window_PreviewMouseLeftButtonDown` checks `TextPullOutHost` ancestry); avatar initial-letter fallback added.
- **Bug-fix pass 2** (`1009800`): real crash fix + avatar root cause.
- **Update Broadcast button** (`89ab83b`): diff-before-push, offline=local only, live=API; Go Live empty-field prompt opens Text drawer.
- **TASK 20 step 2** (this session): hotkey config UI + persistence:
- `Services/HotkeyChord.cs` — modifier+key chord record with display/parse/FromWpf helpers
- `Services/GlobalHotkeys.cs` — modifier param on `IHotkeyRegistrar`, custom bindings dict, `Reattach()` for hot-swap
- `Services/LayoutStore.cs` — `LoadHotkeyBindings()` / `SaveHotkeyBindings()` (Settings table, no migration)
- `ViewModels/HotkeyConfigViewModel.cs` — working copy, capture mode, conflict detection, unbind, reset
- `HotkeyConfigDialog.xaml` + `.cs` — click-to-capture, Unbind per row, Reset/Save/Cancel
- `ViewModels/MainViewModel.cs` — loads bindings, `OpenHotkeyConfigCommand`, `HotkeyBindingsChanged` callback
- `MainWindow.xaml.cs` — passes bindings to manager, re-attaches on dialog save
- Tests: `HotkeyConfigTests` (round-trip, display string, storage serialization)
- **Thumbnail strip hotkey labels** (`d4aa5e1`): each scene shows its current binding (e.g. "Starting F1").
- **Pull-out tab renamed** to "Stream Settings"; **right-click Start Stream** → Change Account / Logout context menu (`67baac6`).
- **Scope Lock rules** (`AGENTS.md` + `scripts/scope-check.sh`): pre-task scope declaration, git history scan before editing, pre-commit audit script that validates `git diff` against declared file list. Born from the `d2114c7` incident.
- Suite: 231 total, 230 pass — only failure is the known pre-existing `AudioPipelineTests.Mix_HonorsProviderGains_AndGameMute_KillsTheLoopback`. Build 0 warnings.
## Uncommitted changes (WebView2 web source — to be committed this session)
**User exception (2026-08-27):** bundle the prior-session dirty work + today's transparency fix
into ONE commit instead of separate work units.
## ⚠️ Landmines
9 files:
- `Services/WebView2Manager.cs` (new) — off-screen WebView2 capture: PropertyChanged syncs
source W/H to control, JS transparent background injection on every nav, `CapturePreviewAsync`
→ BGRA `VideoFrame`; **`DefaultBackgroundColor = System.Drawing.Color.Transparent` set in the
constructor** (before `EnsureCoreWebView2Async`, per WPF SDK doc) — this is the real fix for
transparency. The old white-pixel alpha hack is removed.
- `ytLive.Tests/WebView2ManagerTests.cs` (new) — 4 tests: 3 scale-sync + 1
`WebView2_DefaultBackgroundColor_Is_Transparent`, all pass.
- `Models/Source.cs` — `DisplaySource` selector now routes `WebSource` to `VideoImageSource`
like `ChatBox`.
- `Services/LayoutStore.cs` — `WebUri` column added (idempotent `ALTER TABLE`), read/write in
the source load/save queries.
- `ViewModels/MainViewModel.cs` — `WebView2Manager` init + register/unregister lifecycle
(loaded web sources rehydrate, `WebUri` PropertyChanged forwards to manager).
- `MainWindow.xaml` + `.cs` — off-screen WebView2 host `Canvas` (offscreen-positioned at
-5000/-5000) added to the window's visual tree.
- `ytLive.csproj` — `Microsoft.Web.WebView2` 1.0.2903.40 package reference.
- `ytLive.Tests/LayoutStorePersistenceTests.cs` — `WebSource_WebUri_Persists` roundtrip test.
- **User verdict (2026-08-23):** "The AI wrote fucked up code after repeatedly and intentionally misunderstanding instructions." Trust nothing from those sessions without manual verification. Session 2026-08-24 reinforced it: shipped a "fix" that couldn't work (see Run-crash above) and a UX guess that missed — verify behavior, don't pattern-match.
Build: 0 warnings. Tests: **234/236 pass** (1 new pass + 2 pre-existing failures unchanged:
`AudioPipelineTests.Mix_HonorsProviderGains_AndGameMute_KillsTheLoopback` (HANDOFF-known,
unrelated) + `RoundClipInteractionTests.Round_Clip_Corner_Is_Grabbable_And_Shape_Is_Circle`
(pre-existing, verified by stash-and-rerun, unrelated).
## Two bugs — status
### Bug 1: Scale (web source doesn't render at configured size)
**PARTIALLY ADDRESSED.** PropertyChanged handler syncs source Width/Height to WebView2 control.
But `CapturePreviewAsync` captures at the HWND's physical pixel size (Width × DPI / 96), which
may not match WPF logical units on high-DPI displays. The capture pipeline scales to
`webView.Width/Height` but this fights the WPF Image element's own `UniformToFill`. Further
work needed.
### Bug 2: Transparency (web source loses transparency in capture)
**SOLVED (2026-08-27).** Root cause was looking at the wrong property: previous session tried
`CoreWebView2.BackgroundColor` and `AddScriptToExecuteOnDocumentCreated` — neither exists on
`CoreWebView2` in this SDK. The real property is **`WebView2.DefaultBackgroundColor`** on the
WPF control (a `System.Drawing.Color`, projected from `ICoreWebView2Controller2`). Confirmed
via `strings` on `Microsoft.Web.WebView2.Wpf.dll` and the IDL §7363: *"In the case of a
transparent `DefaultBackgroundColor` WebView will render hosting app content as the
background."* WPF doc: *"The most recent value set to this property before CoreWebView2Controller
has been initialized will be set on it after initialization."* Setting it in the constructor
(before `InitializeAsync → EnsureCoreWebView2Async`) makes the captured PNG have alpha=0 in
transparent regions. Combined with the existing CSS injection (`html/body { background:
transparent }` on every nav), the pixel-level white→transparent alpha hack is no longer needed
and has been removed.
**Verification path:** launch the app with a stream-elements widget URL, the animation should
over the scene's background with no white box. Visual check needed — no automated test for
capture alpha (would need a real rendered WebView2).
## ⚠️ Landmines (inherited from prior sessions + this session)
- **User verdict (2026-08-23 + 2026-08-26):** "The AI wrote fucked up code after repeatedly and
intentionally misunderstanding instructions." and "fuck you for wasting an entire fucking
day." Trust nothing without verification. **Verify behavior, don't pattern-match.**
- **USER RULE: never `git push` without explicit instruction.**
- **Pre-existing test failure is ONE** (`AudioPipelineTests.Mix_HonorsProviderGains_AndGameMute_KillsTheLoopback`) — unrelated to this session.
- **Real-MainWindow tests MUST set `LayoutPathOverride`** (ai.md) — DB pollution wiped real data once.
- **WPF handle forcing:** `WindowInteropHelper.Handle` pre-show returns Zero and does NOT raise `SourceInitialized`; use `EnsureHandle()` (does both).
- **Two MainWindows alive at once double-register hotkeys** — second window's RegisterHotKey fails silently-by-log. Tests close windows before opening the next.
- **Closing a MainWindow OVERWRITES saved window geometry** (Closing → SaveWindowGeometry). In tests that assert restore-from-DB, save state AFTER closing prior windows.
- **liveBroadcasts.update replaces the whole snippet part** — `UpdateBroadcast` echoes scheduledStartTime or YouTube clears the schedule.
- **contentDetails lock in created/ready** — latency/DVR/embed etc. cannot be edited while live; deliberately absent from the pull-out form.
- **Text Runs are not Visuals** — ANY new hit-testing code must type-guard before `VisualTreeHelper.GetParent` (it throws on non-Visuals).
- **Avatar images: never decode via BitmapImage.UriSource on this box** — use the HttpClient→StreamSource pattern in `MainWindow.LoadAvatarImage`.
- **App running locks ytLive.exe** — build fails MSB3027 until creator exits the app (or ask before killing PID).
- **`Source.Type` setter derives `IsBackground`** (Models/Source.cs:40) — any Type change must re-set IsBackground after.
- **vstest quirk:** only one `--tests:` param allowed; plain substrings work, alternation regexes silently match nothing.
- Notification.Wpf gotchas (ai.md/TASKS.md): area routing by XAML `Name`; unknown AreaName silently drops; `NeverExpires()` = `TimeSpan.MaxValue`; `ToHex()` = `#AARRGGBB`.
- **Dead code flagged, not touched:** `ShowWebcamCommand` / `CanShowWebcamInStagedScene` wired but unbound — audit item.
- **Pre-existing test failures are TWO** (not one — corrected 2026-08-27):
`AudioPipelineTests.Mix_HonorsProviderGains_AndGameMute_KillsTheLoopback` (HANDOFF-known,
sound channel related, can't test when not live) +
`RoundClipInteractionTests.Round_Clip_Corner_Is_Grabbable_And_Shape_Is_Circle` (verified
pre-existing by stash-and-rerun 2026-08-27). Both unrelated to web source.
- **Real-MainWindow tests MUST set `LayoutPathOverride`** (ai.md) — DB pollution wiped real
data once.
- **WPF handle forcing:** `WindowInteropHelper.Handle` pre-show returns Zero and does NOT
raise `SourceInitialized`; use `EnsureHandle()` (does both).
- **Two MainWindows alive at once double-register hotkeys** — second window's RegisterHotKey
fails silently-by-log. Tests close windows before opening the next.
- **Closing a MainWindow OVERWRITES saved window geometry** (Closing → SaveWindowGeometry).
In tests that assert restore-from-DB, save state AFTER closing prior windows.
- **liveBroadcasts.update replaces the whole snippet part** — `UpdateBroadcast` echoes
scheduledStartTime or YouTube clears the schedule.
- **contentDetails lock in created/ready** — latency/DVR/embed etc. cannot be edited while live.
- **Text Runs are not Visuals** — ANY new hit-testing code must type-guard before
`VisualTreeHelper.GetParent`.
- **Avatar images: never decode via BitmapImage.UriSource on this box** — use HttpClient→
StreamSource pattern.
- **App running locks ytLive.exe** — build fails MSB3027 until creator exits the app.
- **`Source.Type` setter derives `IsBackground`** (Models/Source.cs:40) — any Type change must
re-set IsBackground after.
- **vstest quirk:** only one `--tests:` param allowed; plain substrings work, alternation
regexes silently match nothing.
- **WebView2 SDK 1.0.2903.40 TFM `net8.0-windows10.0.19041.0`:** the property is
`Microsoft.Web.WebView2.Wpf.WebView2.DefaultBackgroundColor`, NOT `CoreWebView2.BackgroundColor`
(which throws CS1061 because it doesn't exist). Its TYPE is `System.Drawing.Color`, not
`System.Windows.Media.Color` — `using System.Drawing;` is required alongside
`System.Windows.Media` (otherwise `Color` is ambiguous). `AddScriptToExecuteOnDocumentCreated`
is also not on `CoreWebView2` — the equivalent is `WebView2`'s `CoreWebView2.AddScriptToExecuteOnDocumentCreated`
once initialized, or set per-page via `NavigationCompleted`/`NavigationStarting` events (what
the manager does).
- **WebView2 WPF control inherits from HwndHost, NOT Control** — no `Background` property.
- **CapturePreviewAsync preserves alpha** when the controller's `DefaultBackgroundColor` is
transparent — PNG decode → `BitmapImage` → `FormatConvertedBitmap(Bgra32)` keeps the alpha
channel. The previous session's diagnosis ("always opaque white") was based on the default
white `DefaultBackgroundColor`, not on CapturePreview itself.
## Next steps
1. Verify unpushed fixes on next launch: logo click opens About (no crash); avatar shows the real profile image (startup.log free of `Avatar load failed`); then PUSH pending creator go-ahead.
2. Continue queue: TASK 17 web source, TASK 18 local recording, TASK 21 media source, TASK 22 audio sync offset.
3. Loose ends: dead `ShowWebcamCommand`, AudioPipelineTests failure, Velopack update URL.
1. Launch the app with a stream-elements widget URL on a scene; visually confirm the animation
composites over the scene background (no white box). PENDING creator go-ahead.
2. Decide on the scale bug (Bug 1) — DPI/physical-pixel mismatch in the capture pipeline.
3. Continue queue: TASK 18 local recording, TASK 21 media source, TASK 22 audio sync offset.
4. Loose ends: dead `ShowWebcamCommand`, the two pre-existing test failures, Velopack update URL.
## Build
```bash