gramps/LlamaCasty
0
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Web source: fix rendering — fixed 1920×1080 viewport (OBS pattern), Stretch=Uniform, startup registration

Browse Source
This commit is contained in:
Llama Chile Shop
2026-08-28 10:23:59 -07:00
parent fd6e02b687
commit e917c41a48
6 changed files with 88 additions and 207 deletions
Download Patch File Download Diff File Expand all files Collapse all files
HANDOFF.md
+37 -121
View File
@@ -1,133 +1,49 @@
# HANDOFF — Session State
## Branch
**`main`** @ `67baac6`, working tree dirty (WebView2 web source feature + transparency fix).
**NOT pushed — user rule: never push without explicit instruction (2026-08-24).** No feature
## Branch / Commit State
**`main`**, dirty working tree being committed as one unit (web source rendering fix — user-confirmed
working). **NOT pushed — user rule: never push without explicit instruction (2026-08-24).** No feature
branches pre-1.0: all work lands on `main` per work unit.
## Uncommitted changes (WebView2 web source — to be committed this session)
**User exception (2026-08-27):** bundle the prior-session dirty work + today's transparency fix
into ONE commit instead of separate work units.
## The web-source rendering fix (2026-08-28) — USER-CONFIRMED WORKING
9 files:
- `Services/WebView2Manager.cs` (new) — off-screen WebView2 capture: PropertyChanged syncs
source W/H to control, JS transparent background injection on every nav, `CapturePreviewAsync`
→ BGRA `VideoFrame`; **`DefaultBackgroundColor = System.Drawing.Color.Transparent` set in the
constructor** (before `EnsureCoreWebView2Async`, per WPF SDK doc) — this is the real fix for
transparency. The old white-pixel alpha hack is removed.
- `ytLive.Tests/WebView2ManagerTests.cs` (new) — 4 tests: 3 scale-sync + 1
`WebView2_DefaultBackgroundColor_Is_Transparent`, all pass.
- `Models/Source.cs` — `DisplaySource` selector now routes `WebSource` to `VideoImageSource`
like `ChatBox`.
- `Services/LayoutStore.cs` — `WebUri` column added (idempotent `ALTER TABLE`), read/write in
the source load/save queries.
- `ViewModels/MainViewModel.cs` — `WebView2Manager` init + register/unregister lifecycle
(loaded web sources rehydrate, `WebUri` PropertyChanged forwards to manager).
- `MainWindow.xaml` + `.cs` — off-screen WebView2 host `Canvas` (offscreen-positioned at
-5000/-5000) added to the window's visual tree.
- `ytLive.csproj` — `Microsoft.Web.WebView2` 1.0.2903.40 package reference.
- `ytLive.Tests/LayoutStorePersistenceTests.cs` — `WebSource_WebUri_Persists` roundtrip test.
**The bug:** preloaded web resources rendered truncated (upper-left only) and misbehaved as the box
resized. The WebView2 control was being resized to the element box as it changed.
Build: 0 warnings. Tests: **234/236 pass** (1 new pass + 2 pre-existing failures unchanged:
`AudioPipelineTests.Mix_HonorsProviderGains_AndGameMute_KillsTheLoopback` (HANDOFF-known,
unrelated) + `RoundClipInteractionTests.Round_Clip_Corner_Is_Grabbable_And_Shape_Is_Circle`
(pre-existing, verified by stash-and-rerun, unrelated).
**The fix — one idea, three-line change, the OBS/CEF pattern:**
- `Services/WebView2Manager.cs` — viewport is FIXED at the master canvas size (1920×1080), never
tracks the box. Control lives off-screen at (-5000,-5000). Page renders full at its designed size;
the preview scales the captured frame independently via `<Image Stretch="Uniform">`. This is what
OBS obs-browser does (fixed-viewport texture + independent scene-item scale). Removed the
PropertyChanged resize hook. Added an immediate `CaptureFrame` on `NavigationCompleted` so content
appears instantly on load (was waiting for the next 100ms tick).
- `ViewModels/MainViewModel.cs` — `InitWebView2` now calls `RegisterLoadedWebSources()` itself;
before, registration only happened when `LoadLayout` ran after manager init, so preloaded sources
never registered at startup.
- `MainWindow.xaml` — scene-element `<Image>` had a LOCAL `Stretch="UniformToFill"` that outranked
the Style DataTrigger (WPF precedence). Removed the local attribute; Style default is
`UniformToFill`; `IsWebSource` trigger overrides to `Stretch="Uniform"` (no crop/truncation).
- `ytLive.Tests/WebView2ManagerTests.cs` — 3 scale-sync tests tested the removed behavior; replaced
with 2 tests asserting the fixed 1920×1080 viewport and that it does not track source dimensions.
## Two bugs — status
Build: 0 warnings, 0 errors. User verified it renders correctly.
### Bug 1: Scale (web source doesn't render at configured size)
**PARTIALLY ADDRESSED.** PropertyChanged handler syncs source Width/Height to WebView2 control.
But `CapturePreviewAsync` captures at the HWND's physical pixel size (Width × DPI / 96), which
may not match WPF logical units on high-DPI displays. The capture pipeline scales to
`webView.Width/Height` but this fights the WPF Image element's own `UniformToFill`. Further
work needed.
## Key facts / landmines
### Bug 2: Transparency (web source loses transparency in capture)
**SOLVED (2026-08-27).** Root cause was looking at the wrong property: previous session tried
`CoreWebView2.BackgroundColor` and `AddScriptToExecuteOnDocumentCreated` — neither exists on
`CoreWebView2` in this SDK. The real property is **`WebView2.DefaultBackgroundColor`** on the
WPF control (a `System.Drawing.Color`, projected from `ICoreWebView2Controller2`). Confirmed
via `strings` on `Microsoft.Web.WebView2.Wpf.dll` and the IDL §7363: *"In the case of a
transparent `DefaultBackgroundColor` WebView will render hosting app content as the
background."* WPF doc: *"The most recent value set to this property before CoreWebView2Controller
has been initialized will be set on it after initialization."* Setting it in the constructor
(before `InitializeAsync → EnsureCoreWebView2Async`) makes the captured PNG have alpha=0 in
transparent regions. Combined with the existing CSS injection (`html/body { background:
transparent }` on every nav), the pixel-level white→transparent alpha hack is no longer needed
and has been removed.
**Verification path:** launch the app with a stream-elements widget URL, the animation should
over the scene's background with no white box. Visual check needed — no automated test for
capture alpha (would need a real rendered WebView2).
## ⚠️ Landmines (inherited from prior sessions + this session)
- **User verdict (2026-08-23 + 2026-08-26):** "The AI wrote fucked up code after repeatedly and
intentionally misunderstanding instructions." and "fuck you for wasting an entire fucking
day." Trust nothing without verification. **Verify behavior, don't pattern-match.**
- **USER RULE: never `git push` without explicit instruction.**
- **Pre-existing test failures are TWO** (not one — corrected 2026-08-27):
`AudioPipelineTests.Mix_HonorsProviderGains_AndGameMute_KillsTheLoopback` (HANDOFF-known,
sound channel related, can't test when not live) +
`RoundClipInteractionTests.Round_Clip_Corner_Is_Grabbable_And_Shape_Is_Circle` (verified
pre-existing by stash-and-rerun 2026-08-27). Both unrelated to web source.
- **Real-MainWindow tests MUST set `LayoutPathOverride`** (ai.md) — DB pollution wiped real
data once.
- **WPF handle forcing:** `WindowInteropHelper.Handle` pre-show returns Zero and does NOT
raise `SourceInitialized`; use `EnsureHandle()` (does both).
- **Two MainWindows alive at once double-register hotkeys** — second window's RegisterHotKey
fails silently-by-log. Tests close windows before opening the next.
- **Closing a MainWindow OVERWRITES saved window geometry** (Closing → SaveWindowGeometry).
In tests that assert restore-from-DB, save state AFTER closing prior windows.
- **liveBroadcasts.update replaces the whole snippet part** — `UpdateBroadcast` echoes
scheduledStartTime or YouTube clears the schedule.
- **contentDetails lock in created/ready** — latency/DVR/embed etc. cannot be edited while live.
- **Text Runs are not Visuals** — ANY new hit-testing code must type-guard before
`VisualTreeHelper.GetParent`.
- **Avatar images: never decode via BitmapImage.UriSource on this box** — use HttpClient→
StreamSource pattern.
- **App running locks ytLive.exe** — build fails MSB3027 until creator exits the app.
- **`Source.Type` setter derives `IsBackground`** (Models/Source.cs:40) — any Type change must
re-set IsBackground after.
- **vstest quirk:** only one `--tests:` param allowed; plain substrings work, alternation
regexes silently match nothing.
- **WebView2 SDK 1.0.2903.40 TFM `net8.0-windows10.0.19041.0`:** the property is
`Microsoft.Web.WebView2.Wpf.WebView2.DefaultBackgroundColor`, NOT `CoreWebView2.BackgroundColor`
(which throws CS1061 because it doesn't exist). Its TYPE is `System.Drawing.Color`, not
`System.Windows.Media.Color` — `using System.Drawing;` is required alongside
`System.Windows.Media` (otherwise `Color` is ambiguous). `AddScriptToExecuteOnDocumentCreated`
is also not on `CoreWebView2` — the equivalent is `WebView2`'s `CoreWebView2.AddScriptToExecuteOnDocumentCreated`
once initialized, or set per-page via `NavigationCompleted`/`NavigationStarting` events (what
the manager does).
- **WebView2 WPF control inherits from HwndHost, NOT Control** — no `Background` property.
- **CapturePreviewAsync preserves alpha** when the controller's `DefaultBackgroundColor` is
transparent — PNG decode → `BitmapImage` → `FormatConvertedBitmap(Bgra32)` keeps the alpha
channel. The previous session's diagnosis ("always opaque white") was based on the default
white `DefaultBackgroundColor`, not on CapturePreview itself.
- **App running locks `ytLive.exe`** → build fails MSB3027 until creator exits the app.
- **Startup diagnostics:** unhandled exceptions + checkpoints go to `%APPDATA%\ytLlive\startup.log`
(`Helpers/AppLog.cs`). On a silent startup crash, read that.
- **Pre-existing test failures (unrelated to web source, do not chase):**
`AudioPipelineTests.Mix_HonorsProviderGains_AndGameMute_KillsTheLoopback` +
`RoundClipInteractionTests.Round_Clip_Corner_Is_Grabbable_And_Shape_Is_Circle`.
- **Real-MainWindow tests MUST set `LayoutPathOverride`** — DB pollution wiped real data once.
- **Good Dog Rule:** ONE integration test per change; no new feature branches pre-1.0.
- **Scope Lock:** declare the file list, run `scripts/scope-check.sh` before every commit.
- **Every feature change ships memory updates in the SAME commit** — `TASKS.md` status + this file.
- No `CODING.md`; conventions live in `AGENTS.md` + `ai.md`.
## Next steps
1. Launch the app with a stream-elements widget URL on a scene; visually confirm the animation
composites over the scene background (no white box). PENDING creator go-ahead.
2. Decide on the scale bug (Bug 1) — DPI/physical-pixel mismatch in the capture pipeline.
1. Await creator comments on the committed rendering fix; address what he raises.
2. Follow-up only if the creator asks: graceful handling of navigation errors / invalid URIs (TASK 17 ×9).
3. Continue queue: TASK 18 local recording, TASK 21 media source, TASK 22 audio sync offset.
4. Loose ends: dead `ShowWebcamCommand`, the two pre-existing test failures, Velopack update URL.
## Build
```bash
"/mnt/c/Program Files/dotnet/dotnet.exe" build "C:\Users\gramp\Documents\Code\projects\ytLive\ytLive.csproj"
"/mnt/c/Program Files/dotnet/dotnet.exe" vstest "C:\Users\gramp\Documents\Code\projects\ytLive\ytLive.Tests\bin\Debug\net8.0-windows10.0.19041.0\ytLive.Tests.dll"
```
## Polar.sh resources
| Resource | ID |
|---|---|
| Org ID | `c05fb364-b967-4f6c-adf2-8a144e46d085` |
| Product | `d105dfa1-497e-423b-8cd4-e0ee2e3abbc0` |
| Price | `7c1b6f70-cf8c-46df-9bc8-6703f9fa64d0` |
| Benefit | `80f66bc1-d5a3-44a1-b9ea-1709ae508535` |
| Checkout URL | `https://buy.polar.sh/polar_cl_ueMy8AyAClO0o0yo19IroZwKTbD587gzUsjEW3bFapw` |
| Validate endpoint | `POST https://api.polar.sh/v1/customer-portal/license-keys/validate` |
| Key prefix | `LCYT-` |
## v1 vision
> A streaming tool so intuitive it needs no manual — go live or record in minutes, OBS-free.
MainWindow.xaml
+3 -2
View File
@@ -841,12 +841,12 @@
<Grid.RenderTransform>
<ScaleTransform ScaleX="{Binding MirrorScale}"/>
</Grid.RenderTransform>
<Image Stretch="UniformToFill"
Source="{Binding DisplaySource}"
<Image Source="{Binding DisplaySource}"
RenderOptions.BitmapScalingMode="HighQuality">
<Image.Style>
<Style TargetType="Image">
<Setter Property="Visibility" Value="Collapsed"/>
<Setter Property="Stretch" Value="UniformToFill"/>
<Style.Triggers>
<DataTrigger Binding="{Binding IsImageSource}" Value="True">
<Setter Property="Visibility" Value="Visible"/>
@@ -856,6 +856,7 @@
</DataTrigger>
<DataTrigger Binding="{Binding IsWebSource}" Value="True">
<Setter Property="Visibility" Value="Visible"/>
<Setter Property="Stretch" Value="Uniform"/>
</DataTrigger>
<MultiDataTrigger>
<MultiDataTrigger.Conditions>
Services/WebView2Manager.cs
+6 -17
View File
@@ -40,10 +40,13 @@ public sealed class WebView2Manager : IDisposable
{
if (_sessions.ContainsKey(source.Id)) return;
// OBS pattern: the browser viewport is the canvas size (1920×1080),
// never the element box. The page renders full at its designed size;
// the preview scales the captured frame independently (Stretch=Uniform).
var webView = new WebView2
{
Width = source.Width > 0 ? source.Width : 640,
Height = source.Height > 0 ? source.Height : 480,
Width = 1920,
Height = 1080,
DefaultBackgroundColor = System.Drawing.Color.Transparent,
Visibility = System.Windows.Visibility.Visible,
};
@@ -65,21 +68,6 @@ public sealed class WebView2Manager : IDisposable
_sessions[source.Id] = session;
source.PropertyChanged += (_, e) =>
{
if (e.PropertyName == nameof(Source.Width) || e.PropertyName == nameof(Source.Height))
{
var w = Math.Max(1, (int)source.Width);
var h = Math.Max(1, (int)source.Height);
if (Math.Abs(session.Control.Width - w) > 0.5 ||
Math.Abs(session.Control.Height - h) > 0.5)
{
session.Control.Width = w;
session.Control.Height = h;
}
}
};
timer.Tick += (_, _) =>
{
if (!session.Disposed && session.Initialized)
@@ -132,6 +120,7 @@ public sealed class WebView2Manager : IDisposable
_ = cws.ExecuteScriptAsync(
"document.documentElement.style.background='transparent';" +
"document.body.style.background='transparent';");
_ = CaptureFrame(source.Id);
};
session.Initialized = true;
TASKS.md
+15 -9
View File
@@ -905,18 +905,24 @@ capturing the primary display (which causes the infinity mirror effect).
**Goal:** make the web source actually render URLs into the preview and stream output.
### Status: ☐ Not started — required for v1
### Status: ✅ Done — required for v1 (2026-08-28)
1. ☐ Add `Microsoft.Web.WebView2` NuGet package
2. ☐ Schema v10: `WebUri TEXT` column on `Source` table + migration in `LayoutStore.cs`
3. ☐ Persist `Source.WebUri` on save/load (currently in-memory only — lost on restart)
4. ☐ Hidden `WebView2` control per web source — navigates to `WebUri`, renders in-app
5. ☐ Frame capture from WebView2 (`CoreWebView2.CapturePreviewAsync` or `CompositionSurface`) → `VideoFrame` (BGRA8)
6. ☐ Wire into `FramePump` resolver — `Source { Type: WebSource }` → latest WebView2 frame
7. ☐ Wire into `SceneCompositor` — render web source as an image element at its position/size
8. ☐ Preview shows live web content (not just a blank rectangle)
1. ✅ Add `Microsoft.Web.WebView2` NuGet package
2. ✅ Schema v10: `WebUri TEXT` column on `Source` table + migration in `LayoutStore.cs`
3. ✅ Persist `Source.WebUri` on save/load (currently in-memory only — lost on restart)
4. ✅ Hidden off-screen `WebView2` control per web source — navigates to `WebUri`, renders in-app
5. ✅ Frame capture from WebView2 (`CoreWebView2.CapturePreviewAsync`) → `WriteableBitmap` (BGRA8)
6. ✅ Wire into `FramePump` resolver — `Source { Type: WebSource }` → latest WebView2 frame
7. ✅ Wire into `SceneCompositor` — render web source as an image element at its position/size
8. ✅ Preview shows live web content (not just a blank rectangle)
9. ☐ Handle navigation errors, invalid URIs, timeout gracefully
**Rendering model (2026-08-28, the fix that works):** WebView2 viewport is FIXED at the master
canvas size (1920×1080) in `Register`, never tracking the element box — the OBS/CEF pattern
(fixed render viewport, display box scaled independently via `Stretch="Uniform"`). Capturing the
box-sized viewport was the truncation bug. Display `Stretch`: web → `Uniform` (no crop),
everything else → `UniformToFill`. Graceful-error handling (item 9) is a follow-up.
### Design decisions
- WebView2 is the only option for Windows — it's pre-installed on Windows 10 20H2+ and Windows 11
ViewModels/MainViewModel.cs
+1
View File
@@ -2294,6 +2294,7 @@ public class MainViewModel : ViewModelBase
{
_webView2Manager = new WebView2Manager(hostPanel, System.Windows.Application.Current.Dispatcher);
_webView2Manager.PreviewBitmapChanged += OnWebView2PreviewBitmapChanged;
RegisterLoadedWebSources();
}
private void OnWebView2PreviewBitmapChanged(string sourceId, WriteableBitmap bitmap)
ytLive.Tests/WebView2ManagerTests.cs
+25 -57
View File
@@ -20,7 +20,29 @@ public sealed class WebView2ManagerTests
}
[Fact]
public void WebView2_Control_Size_Tracks_Source_Dimensions()
public void WebView2_Control_Size_Is_Fixed_At_Canvas_Resolution()
{
_app.Run(() =>
{
var panel = new Canvas();
var source = new Source
{
Type = SourceType.WebSource,
Width = 640,
Height = 480,
};
using var manager = new WebView2Manager(panel, Dispatcher.CurrentDispatcher);
manager.Register(source);
var webView = panel.Children.OfType<WebView2>().First();
Assert.Equal(1920.0, webView.Width);
Assert.Equal(1080.0, webView.Height);
});
}
[Fact]
public void WebView2_Control_Size_Does_Not_Track_Source_Dimensions()
{
_app.Run(() =>
{
@@ -36,66 +58,12 @@ public sealed class WebView2ManagerTests
manager.Register(source);
var webView = panel.Children.OfType<WebView2>().First();
Assert.Equal(640.0, webView.Width);
Assert.Equal(480.0, webView.Height);
source.Width = 1024;
source.Height = 768;
Assert.Equal(1024.0, webView.Width);
Assert.Equal(768.0, webView.Height);
});
}
[Fact]
public void WebView2_Control_Size_Does_Not_Drift_On_Minor_Rounding()
{
_app.Run(() =>
{
var panel = new Canvas();
var source = new Source
{
Type = SourceType.WebSource,
Width = 640,
Height = 480,
};
using var manager = new WebView2Manager(panel, Dispatcher.CurrentDispatcher);
manager.Register(source);
var webView = panel.Children.OfType<WebView2>().First();
source.Width = 640.4;
Assert.Equal(640.0, webView.Width);
source.Height = 480.4;
Assert.Equal(480.0, webView.Height);
});
}
[Fact]
public void WebView2_Control_Size_Minimum_Is_One()
{
_app.Run(() =>
{
var panel = new Canvas();
var source = new Source
{
Type = SourceType.WebSource,
Width = 640,
Height = 480,
};
using var manager = new WebView2Manager(panel, Dispatcher.CurrentDispatcher);
manager.Register(source);
var webView = panel.Children.OfType<WebView2>().First();
source.Width = 0;
source.Height = 0;
Assert.Equal(1.0, webView.Width);
Assert.Equal(1.0, webView.Height);
Assert.Equal(1920.0, webView.Width);
Assert.Equal(1080.0, webView.Height);
});
}