fix(web): inject transparency BEFORE page scripts — CapturePreviewAsync honors page CSS
The widget page paints html/body opaque, so CapturePreviewAsync output had
zero alpha-0 margins — every take since bccdb48 built the crop on 'the
capture is transparent' and never verified it. WebView2 spec is explicit:
'WebView will always honor a webpage's background content' and
DefaultBackgroundColor only shows through pages with no background style
(MicrosoftEdge/WebView2Feedback specs/BackgroundColor.md). The late
ExecuteScriptAsync injection (NavStarting/NavCompleted) ran after page CSS
and lost the fight.
Fix via CoreWebView2.AddScriptToExecuteOnDocumentCreatedAsync — the
documented pre-parse hook ('before the HTML document has been parsed and
before any other script included by the HTML document is run'), the same
mechanism OBS user.css uses. Nav handlers kept as post-load re-assertion.
Restores the take-23/24 stride-correct crop path (my take-25 removal was
wrong — it regressed the bounding box into a solid black box).
Adds one-shot diagnostics: raw capture PNG -> %TEMP%/ytLive-web-<id>.png
plus alpha min/max/mean/%zero and FindContentBounds result in startup.log,
so the next run proves the capture is transparent instead of guessing.
Build 0 warnings, 289/289 tests pass. MyMistakes.md records the full chain.
This commit is contained in:
+30
-22
@@ -2,37 +2,45 @@
|
||||
|
||||
## Branch / Commit State
|
||||
|
||||
**`main` HEAD = `081e4c1`** — take-24: PasteKey includes CropBounds — stale raster cache fixed.
|
||||
**`main` HEAD = `081e4c1`** — take-24 (PasteKey CropBounds). Working tree carries the
|
||||
real fix (uncommitted): pre-parse transparency injection + diagnostics.
|
||||
|
||||
```
|
||||
081e4c1 fix(web): add CropBounds to PasteKey — stale raster cache was corrupting transparency
|
||||
f6802c7 fix(web): stride-correct CropBounds Fill rendering in compositor (take-23)
|
||||
84aa70f docs: MyMistakes.md — take-22 full fix for widget scale/position + transparency
|
||||
ed9d7c1 fix(web): CropBounds metadata + Fill-style scaling in compositor — transparent margins preserved, widget fills element rect (BROKEN — reverted)
|
||||
5ca9778 docs: update HANDOFF + MyMistakes.md — take-21 web transparency fix landed
|
||||
e002847 fix(web): feed full canvas to compositor — transparent margins reveal layers beneath (take-21)
|
||||
70db344 perf(camera): MJPEG negotiation before frame reader — camera drives at its best rate
|
||||
87509bc (origin/main) docs: restructure TASKS.md into a catalog
|
||||
```
|
||||
## Web Overlay — the ROOT CAUSE (finally named, not guessed)
|
||||
|
||||
**Pushed to origin: `87509bc`.** 8 unpushed commits after it.
|
||||
WebView2's `CapturePreviewAsync` **always honors the page's own background** — the widget
|
||||
page paints html/body opaque, so the captured PNG has **no alpha-0 margins, ever**.
|
||||
`DefaultBackgroundColor=Transparent` only shows through pages without a background style.
|
||||
All prior takes (19-24) assumed the capture had transparent margins — it never did.
|
||||
`FindContentBounds` therefore had nothing to crop to, and every blend saw alpha=255 =
|
||||
black box + "transparency broken" + lost resizing. One bug, three symptoms.
|
||||
|
||||
## Web Overlay — Current State
|
||||
**Fix applied (verify by running):**
|
||||
1. `WebView2Manager.InitializeAsync`: `AddScriptToExecuteOnDocumentCreatedAsync(TransparentBackgroundScript)`
|
||||
— injects html/body `background:transparent` BEFORE the page parses/scripts run (the
|
||||
OBS user.css equivalent). NavStarting/NavCompleted keep the script as post-load re-assert.
|
||||
2. Restored the take-23/24 compositor crop path (stride-correct `BlitContentRaw` +
|
||||
`CropBounds` in PasteKey) — my previous take-25 gutting was wrong.
|
||||
3. Diagnostics: first capture per source dumps the RAW WebView2 PNG to
|
||||
`%TEMP%\ytLive-web-<id>.png` and logs alpha stats + FindContentBounds result to
|
||||
startup.log.
|
||||
|
||||
take-22 (`ed9d7c1`) BROKE: stride bug (crop width used for canvas buffer indexing) — REVERTED.
|
||||
take-23 (`f6802c7`): stride fixed, but PasteKey missing CropBounds → stale raster cached → transparent pixels corrupted to opaque.
|
||||
take-24 (`081e4c1`): FIX. `PasteKey` now includes `frame.CropBounds`. Different crop bounds = cache miss = fresh raster with correct alpha.
|
||||
## NEXT STEP (user run required once)
|
||||
|
||||
## Still open (needs your run + log)
|
||||
Run the app with the web overlay, then:
|
||||
- Read `%APPDATA%\ytLlive\startup.log` — the `alpha[min=..,max=..,mean=..,zero=..%]`
|
||||
line PROVES whether the capture is transparent now (mean near 0 + high zero% + a tight
|
||||
contentBounds = fixed) or still opaque (mean ~255 → injection didn't beat the page).
|
||||
- Agent can PIL-analyze `%TEMP%\ytLive-web-<id>.png` for the true alpha bbox.
|
||||
|
||||
- **Chat overlay missing from recording:** Chat shows in preview but not in recorded output.
|
||||
- **Audio/sync issues:** unspecified, no log yet.
|
||||
- **Webcam/capture speed:** unspecified, no log yet.
|
||||
## Still Open
|
||||
|
||||
- Chat overlay missing from recording (separate issue — not yet touched this session)
|
||||
- Audio/sync: non-event (headset volume), closed
|
||||
|
||||
## Landmines
|
||||
|
||||
- testhost shares startup.log with app — filter by time when triaging
|
||||
- Stale testhost/exe locks DLL: `taskkill /F /IM testhost.exe /IM ytLive.exe` first
|
||||
- testhost/exe lock DLLs: `taskkill /F /IM testhost.exe /IM ytLive.exe` before rebuild
|
||||
- Do NOT run full-suite vstest (WASAPI hang); flow = clean build + per-class + scope-check
|
||||
- Kill app before build: `/mnt/c/Windows/System32/taskkill.exe /F /IM ytLive.exe`
|
||||
- ffmpeg month-end pinned; `Startup.log` "Recording saved:" shows real final path
|
||||
- The PNG dump is written ONCE per session per source (DebugPngWritten flag)
|
||||
+43
-24
@@ -15,34 +15,53 @@
|
||||
|
||||
## 🔬 Recipes registry
|
||||
|
||||
### ⚠ SPIN GUARD TRIGGERED → RESOLVED (take-21, 22 broke, 23 fixed) — web overlay transparency + scale
|
||||
### ⚠ SPIN GUARD TRIGGERED → RESOLVED (? VERIFY) — web overlay transparency + bounding box
|
||||
|
||||
**Take-19 (`0f441d8`): FAILED.** Attempted to feed the compositor the FULL canvas instead of the
|
||||
alpha crop. User verdict: "didn't work at all — broke additional crap." Reverted (`1295e0e`).
|
||||
The crop→UniformToFill zoom insight was RIGHT, but the commit ALSO introduced `CanvasRing` (8-deep
|
||||
full-canvas buffer ring) alongside the fix — the ring reuse poisoned the paste cache.
|
||||
**THE ONE ROOT CAUSE THAT EXPLAINS EVERY FAILED TAKE:** WebView2's `CapturePreviewAsync`
|
||||
produces an **OPAQUE** PNG. From the WebView2 spec (sender: MicrosoftEdge/WebView2Feedback
|
||||
`specs/BackgroundColor.md`): "WebView will always honor a webpage's background content."
|
||||
`DefaultBackgroundColor = Transparent` only shows through pages with NO background style —
|
||||
the widget's own CSS paints html/body opaque. Every take below built on the false premise
|
||||
"the capture has transparent margins, alpha=0"; it never did. `FindContentBounds` then had
|
||||
no alpha-0 margins to find → wrong crop → black bounding box. The compositor blend saw
|
||||
alpha=255 → black over webcam = "transparency broken". Same bug, three symptoms.
|
||||
|
||||
**Take-21 (`e002847`): PARTIAL FIX.** Feed full canvas, transparent margins revealed — but the
|
||||
widget appeared shrunk and offset. `UniformToFill` used `src=1920×1080` for a 703×389 element
|
||||
→ scale=0.366. The preview used `Stretch="Fill"` on the CROPPED bitmap — different math.
|
||||
**THE FIX (the OBS way, applied 2026-09-08):** inject the transparency BEFORE the page
|
||||
parses using `CoreWebView2.AddScriptToExecuteOnDocumentCreatedAsync` — documented to run
|
||||
"before the HTML document has been parsed and before any other script included by the HTML
|
||||
document is run" (learn.microsoft.com/dotnet/api/microsoft.web.webview2.core.corewebview2.addscripttoexecuteondocumentcreatedasync).
|
||||
The old `ExecuteScriptAsync` on NavigationStarting/NavigationCompleted ran AFTER page
|
||||
scripts/CSS → widget page repainted background opaque → lost the fight. OBS browser sources
|
||||
do the same via a pre-parse user.css. Kept the nav handlers as a post-load re-assertion.
|
||||
|
||||
**Take-22 (`ed9d7c1`): BROKE IT.** Added `CropBounds` metadata but the pixel index in
|
||||
`BlitContentRaw` used `w = cbW` (crop width, e.g. 800) instead of `src.Width` (canvas stride
|
||||
= 1920) for buffer indexing. Since `s = src.BgraPixels` is the FULL canvas buffer, multiplying
|
||||
by crop width instead of canvas stride computed completely wrong memory offsets — garbage pixels,
|
||||
"original content destroyed." Reverted.
|
||||
**Take timeline (the honest record):**
|
||||
- `bccdb48` (Aug 28): added FindContentBounds crop — correct idea (tight bbox, no dead
|
||||
space), but the capture was OPAQUE so the bbox math was built on nothing.
|
||||
- `5348b5c` (Aug 28, 2 min later): reverted to full-frame no-crop — looked "good" for a
|
||||
full-bleed widget, but floating widgets regained dead space ("ghost boundary").
|
||||
- take-21 (`e002847`) full canvas → shrunken/offset widget (UnifomToFill of whole canvas
|
||||
into a small element = lost resizing).
|
||||
- take-22 (`ed9d7c1`) crop width used as canvas stride for buffer indexing → garbage.
|
||||
- take-23 (`f6802c7`) stride fixed with `src.Width`; PasteKey lacked CropBounds → stale
|
||||
raster cache → stale crop.
|
||||
- take-24 (`081e4c1`) CropBounds in PasteKey — STILL BROKEN because the SOURCE ALPHA WAS
|
||||
NEVER REAL.
|
||||
- take-25 (**ME, this session — the user's "RE-INTRODUCING THE BOUNDING-BOX PROBLEM"**):
|
||||
I removed FindContentBounds + the crop path entirely, betting full-canvas UniformToFill
|
||||
was the answer. It WASN'T — the source is opaque-black, so the element rendered as a
|
||||
SOLID BLACK BOX (the user's screenshot: "a black box in the lower right corner"). Killed
|
||||
the crop → dead space returned AND black box. The compositor math was ALREADY correct;
|
||||
gutting it was vandalism in response to a source-level bug.
|
||||
|
||||
**Take-23 (`f6802c7`): stride fix.** `BlitContentRaw` CropBounds path uses `src.Width`
|
||||
(canvas stride = 1920) for ALL pixel indexing. `srcStride` explicitly named to prevent recurrence.
|
||||
|
||||
**Take-24 (`081e4c1`): transparency still broken after scale fix — PasteKey missing CropBounds.**
|
||||
Scale was correct but transparent pixels showed black (backdrop) instead of revealing webcam.
|
||||
Root cause: `PasteKey` (compositor paste cache) was keyed on `(Pixels, Epoch, Width, Height,
|
||||
DstW, DstH, Round, Mirror)` — NOT `CropBounds`. Since the frame Width/Height are always
|
||||
1920×1080 (the canvas) regardless of crop bounds, the same key was used for all element
|
||||
sizes. A stale raster (wrong crop) was served from the cache after resize or first add,
|
||||
baking in opaque content over transparent regions. Fix: add `CropBounds` to `PasteKey`.
|
||||
`frame.CropBounds` is included in the cache key so different crop bounds = cache miss.
|
||||
**Rules, self-inflicted:**
|
||||
1. Instrument FIRST. This session added: first-capture PNG dump of the raw WebView2 PNG
|
||||
(to `%TEMP%\ytLive-web-<id>.png`) + alpha min/max/mean/%zero + FindContentBounds result
|
||||
logged to startup.log once per session. That's the diff between a five-take loop and a
|
||||
five-minute diagnosis.
|
||||
2. When the same symptom loops across takes, the PREMISE is wrong, not the code — the
|
||||
capture being transparent was the load-bearing premise and it was never verified.
|
||||
3. Do not delete code paths that fix one axis (crop=bbox) while debugging another
|
||||
(source alpha). Revert scope creep; keep layer contributions separable.
|
||||
|
||||
### Shrink / re-encode an image for the README (screenshots → small hero image)
|
||||
|
||||
|
||||
+57
-10
@@ -42,6 +42,7 @@ public sealed class WebView2Manager : IDisposable
|
||||
public int OutRingNext;
|
||||
public long Epoch;
|
||||
public WriteableBitmap? PreviewBitmap;
|
||||
public bool DebugPngWritten;
|
||||
|
||||
public byte[] RentOutBuffer(int size)
|
||||
{
|
||||
@@ -138,6 +139,19 @@ public sealed class WebView2Manager : IDisposable
|
||||
return _sessions.TryGetValue(sourceId, out var session) ? session.LatestFrame : null;
|
||||
}
|
||||
|
||||
// Injected BEFORE the document is parsed (AddScriptToExecuteOnDocumentCreatedAsync) so
|
||||
// the widget page's own CSS/scripts cannot repaint html/body opaque — the reason the
|
||||
// late NavigationStarting/NavigationCompleted ExecuteScriptAsync injection lost the
|
||||
// fight (page styles ran first). CapturePreviewAsync "will always honor a webpage's
|
||||
// background content" (MicrosoftEdge/WebView2Feedback specs/BackgroundColor.md), so a
|
||||
// transparent capture REQUIRES the transparency to be in place before the page paints.
|
||||
private const string TransparentBackgroundScript =
|
||||
"document.documentElement.style.background='transparent';" +
|
||||
"document.documentElement.style.overflow='hidden';" +
|
||||
"document.documentElement.style.margin='0';" +
|
||||
"if(document.body){document.body.style.background='transparent';" +
|
||||
"document.body.style.overflow='hidden';document.body.style.margin='0';}";
|
||||
|
||||
private async Task InitializeAsync(Source source, WebSourceSession session)
|
||||
{
|
||||
try
|
||||
@@ -146,22 +160,19 @@ public sealed class WebView2Manager : IDisposable
|
||||
|
||||
var cws = session.Control.CoreWebView2!;
|
||||
|
||||
// Pre-page hook: runs after the global object exists but BEFORE the document
|
||||
// parses / any page script runs — must complete before navigating away from
|
||||
// about:blank for it to apply to the widget document.
|
||||
await cws.AddScriptToExecuteOnDocumentCreatedAsync(TransparentBackgroundScript);
|
||||
|
||||
cws.NavigationStarting += (_, _) =>
|
||||
{
|
||||
_ = cws.ExecuteScriptAsync(
|
||||
"document.documentElement.style.background='transparent';" +
|
||||
"document.documentElement.style.overflow='hidden';" +
|
||||
"document.body.style.background='transparent';" +
|
||||
"document.body.style.overflow='hidden';");
|
||||
_ = cws.ExecuteScriptAsync(TransparentBackgroundScript);
|
||||
};
|
||||
|
||||
cws.NavigationCompleted += (_, _) =>
|
||||
{
|
||||
_ = cws.ExecuteScriptAsync(
|
||||
"document.documentElement.style.background='transparent';" +
|
||||
"document.documentElement.style.overflow='hidden';" +
|
||||
"document.body.style.background='transparent';" +
|
||||
"document.body.style.overflow='hidden';");
|
||||
_ = cws.ExecuteScriptAsync(TransparentBackgroundScript);
|
||||
_ = CaptureFrame(source.Id);
|
||||
};
|
||||
|
||||
@@ -231,6 +242,19 @@ public sealed class WebView2Manager : IDisposable
|
||||
await webView.CoreWebView2.CapturePreviewAsync(
|
||||
CoreWebView2CapturePreviewImageFormat.Png, ms);
|
||||
|
||||
if (!session.DebugPngWritten)
|
||||
{
|
||||
session.DebugPngWritten = true;
|
||||
try
|
||||
{
|
||||
ms.Position = 0;
|
||||
var dbg = Path.Combine(Path.GetTempPath(), $"ytLive-web-{sourceId}.png");
|
||||
File.WriteAllBytes(dbg, ms.ToArray());
|
||||
AppLog.Write($"WebView2Manager: debug capture dump -> {dbg}");
|
||||
}
|
||||
catch (Exception dex) { AppLog.Write($"WebView2Manager: debug dump failed: {dex.Message}"); }
|
||||
}
|
||||
|
||||
ms.Position = 0;
|
||||
|
||||
var bitmap = new BitmapImage();
|
||||
@@ -258,6 +282,29 @@ public sealed class WebView2Manager : IDisposable
|
||||
|
||||
var (cropX, cropY, cropW, cropH) = FindContentBounds(pixels, pixW, pixH, stride);
|
||||
|
||||
if (session.Epoch == 0)
|
||||
{
|
||||
int minA = 255, maxA = 0;
|
||||
long sumA = 0, nA = 0, zeroA = 0;
|
||||
for (var y = 0; y < pixH; y += 16)
|
||||
{
|
||||
var row = y * stride;
|
||||
for (var x = 0; x < pixW; x += 16)
|
||||
{
|
||||
var a = pixels[row + x * 4 + 3];
|
||||
if (a < minA) minA = a;
|
||||
if (a > maxA) maxA = a;
|
||||
sumA += a;
|
||||
if (a == 0) zeroA++;
|
||||
nA++;
|
||||
}
|
||||
}
|
||||
AppLog.Write(
|
||||
$"WebView2Manager: first capture for '{sourceId}' {pixW}x{pixH} " +
|
||||
$"alpha[min={minA},max={maxA},mean={sumA / (double)nA:F1},zero={100.0 * zeroA / nA:F1}%] " +
|
||||
$"contentBounds=({cropX},{cropY},{cropW},{cropH})");
|
||||
}
|
||||
|
||||
var outPixels = session.RentOutBuffer(cropW * cropH * 4);
|
||||
if (cropW == pixW && cropH == pixH && cropX == 0 && cropY == 0)
|
||||
Buffer.BlockCopy(pixels, 0, outPixels, 0, outPixels.Length);
|
||||
|
||||
Reference in New Issue
Block a user