diff --git a/HANDOFF.md b/HANDOFF.md index 1afdcb4..9ffc34e 100644 --- a/HANDOFF.md +++ b/HANDOFF.md @@ -2,37 +2,45 @@ ## Branch / Commit State -**`main` HEAD = `081e4c1`** — take-24: PasteKey includes CropBounds — stale raster cache fixed. +**`main` HEAD = `081e4c1`** — take-24 (PasteKey CropBounds). Working tree carries the +real fix (uncommitted): pre-parse transparency injection + diagnostics. -``` -081e4c1 fix(web): add CropBounds to PasteKey — stale raster cache was corrupting transparency -f6802c7 fix(web): stride-correct CropBounds Fill rendering in compositor (take-23) -84aa70f docs: MyMistakes.md — take-22 full fix for widget scale/position + transparency -ed9d7c1 fix(web): CropBounds metadata + Fill-style scaling in compositor — transparent margins preserved, widget fills element rect (BROKEN — reverted) -5ca9778 docs: update HANDOFF + MyMistakes.md — take-21 web transparency fix landed -e002847 fix(web): feed full canvas to compositor — transparent margins reveal layers beneath (take-21) -70db344 perf(camera): MJPEG negotiation before frame reader — camera drives at its best rate -87509bc (origin/main) docs: restructure TASKS.md into a catalog -``` +## Web Overlay — the ROOT CAUSE (finally named, not guessed) -**Pushed to origin: `87509bc`.** 8 unpushed commits after it. +WebView2's `CapturePreviewAsync` **always honors the page's own background** — the widget +page paints html/body opaque, so the captured PNG has **no alpha-0 margins, ever**. +`DefaultBackgroundColor=Transparent` only shows through pages without a background style. +All prior takes (19-24) assumed the capture had transparent margins — it never did. +`FindContentBounds` therefore had nothing to crop to, and every blend saw alpha=255 = +black box + "transparency broken" + lost resizing. One bug, three symptoms. -## Web Overlay — Current State +**Fix applied (verify by running):** +1. `WebView2Manager.InitializeAsync`: `AddScriptToExecuteOnDocumentCreatedAsync(TransparentBackgroundScript)` + — injects html/body `background:transparent` BEFORE the page parses/scripts run (the + OBS user.css equivalent). NavStarting/NavCompleted keep the script as post-load re-assert. +2. Restored the take-23/24 compositor crop path (stride-correct `BlitContentRaw` + + `CropBounds` in PasteKey) — my previous take-25 gutting was wrong. +3. Diagnostics: first capture per source dumps the RAW WebView2 PNG to + `%TEMP%\ytLive-web-.png` and logs alpha stats + FindContentBounds result to + startup.log. -take-22 (`ed9d7c1`) BROKE: stride bug (crop width used for canvas buffer indexing) — REVERTED. -take-23 (`f6802c7`): stride fixed, but PasteKey missing CropBounds → stale raster cached → transparent pixels corrupted to opaque. -take-24 (`081e4c1`): FIX. `PasteKey` now includes `frame.CropBounds`. Different crop bounds = cache miss = fresh raster with correct alpha. +## NEXT STEP (user run required once) -## Still open (needs your run + log) +Run the app with the web overlay, then: +- Read `%APPDATA%\ytLlive\startup.log` — the `alpha[min=..,max=..,mean=..,zero=..%]` + line PROVES whether the capture is transparent now (mean near 0 + high zero% + a tight + contentBounds = fixed) or still opaque (mean ~255 → injection didn't beat the page). +- Agent can PIL-analyze `%TEMP%\ytLive-web-.png` for the true alpha bbox. -- **Chat overlay missing from recording:** Chat shows in preview but not in recorded output. -- **Audio/sync issues:** unspecified, no log yet. -- **Webcam/capture speed:** unspecified, no log yet. +## Still Open + +- Chat overlay missing from recording (separate issue — not yet touched this session) +- Audio/sync: non-event (headset volume), closed ## Landmines - testhost shares startup.log with app — filter by time when triaging -- Stale testhost/exe locks DLL: `taskkill /F /IM testhost.exe /IM ytLive.exe` first +- testhost/exe lock DLLs: `taskkill /F /IM testhost.exe /IM ytLive.exe` before rebuild - Do NOT run full-suite vstest (WASAPI hang); flow = clean build + per-class + scope-check - Kill app before build: `/mnt/c/Windows/System32/taskkill.exe /F /IM ytLive.exe` -- ffmpeg month-end pinned; `Startup.log` "Recording saved:" shows real final path +- The PNG dump is written ONCE per session per source (DebugPngWritten flag) \ No newline at end of file diff --git a/MyMistakes.md b/MyMistakes.md index 7689a02..b87e252 100644 --- a/MyMistakes.md +++ b/MyMistakes.md @@ -15,34 +15,53 @@ ## 🔬 Recipes registry -### ⚠ SPIN GUARD TRIGGERED → RESOLVED (take-21, 22 broke, 23 fixed) — web overlay transparency + scale +### ⚠ SPIN GUARD TRIGGERED → RESOLVED (? VERIFY) — web overlay transparency + bounding box -**Take-19 (`0f441d8`): FAILED.** Attempted to feed the compositor the FULL canvas instead of the -alpha crop. User verdict: "didn't work at all — broke additional crap." Reverted (`1295e0e`). -The crop→UniformToFill zoom insight was RIGHT, but the commit ALSO introduced `CanvasRing` (8-deep -full-canvas buffer ring) alongside the fix — the ring reuse poisoned the paste cache. +**THE ONE ROOT CAUSE THAT EXPLAINS EVERY FAILED TAKE:** WebView2's `CapturePreviewAsync` +produces an **OPAQUE** PNG. From the WebView2 spec (sender: MicrosoftEdge/WebView2Feedback +`specs/BackgroundColor.md`): "WebView will always honor a webpage's background content." +`DefaultBackgroundColor = Transparent` only shows through pages with NO background style — +the widget's own CSS paints html/body opaque. Every take below built on the false premise +"the capture has transparent margins, alpha=0"; it never did. `FindContentBounds` then had +no alpha-0 margins to find → wrong crop → black bounding box. The compositor blend saw +alpha=255 → black over webcam = "transparency broken". Same bug, three symptoms. -**Take-21 (`e002847`): PARTIAL FIX.** Feed full canvas, transparent margins revealed — but the -widget appeared shrunk and offset. `UniformToFill` used `src=1920×1080` for a 703×389 element -→ scale=0.366. The preview used `Stretch="Fill"` on the CROPPED bitmap — different math. +**THE FIX (the OBS way, applied 2026-09-08):** inject the transparency BEFORE the page +parses using `CoreWebView2.AddScriptToExecuteOnDocumentCreatedAsync` — documented to run +"before the HTML document has been parsed and before any other script included by the HTML +document is run" (learn.microsoft.com/dotnet/api/microsoft.web.webview2.core.corewebview2.addscripttoexecuteondocumentcreatedasync). +The old `ExecuteScriptAsync` on NavigationStarting/NavigationCompleted ran AFTER page +scripts/CSS → widget page repainted background opaque → lost the fight. OBS browser sources +do the same via a pre-parse user.css. Kept the nav handlers as a post-load re-assertion. -**Take-22 (`ed9d7c1`): BROKE IT.** Added `CropBounds` metadata but the pixel index in -`BlitContentRaw` used `w = cbW` (crop width, e.g. 800) instead of `src.Width` (canvas stride -= 1920) for buffer indexing. Since `s = src.BgraPixels` is the FULL canvas buffer, multiplying -by crop width instead of canvas stride computed completely wrong memory offsets — garbage pixels, -"original content destroyed." Reverted. +**Take timeline (the honest record):** +- `bccdb48` (Aug 28): added FindContentBounds crop — correct idea (tight bbox, no dead + space), but the capture was OPAQUE so the bbox math was built on nothing. +- `5348b5c` (Aug 28, 2 min later): reverted to full-frame no-crop — looked "good" for a + full-bleed widget, but floating widgets regained dead space ("ghost boundary"). +- take-21 (`e002847`) full canvas → shrunken/offset widget (UnifomToFill of whole canvas + into a small element = lost resizing). +- take-22 (`ed9d7c1`) crop width used as canvas stride for buffer indexing → garbage. +- take-23 (`f6802c7`) stride fixed with `src.Width`; PasteKey lacked CropBounds → stale + raster cache → stale crop. +- take-24 (`081e4c1`) CropBounds in PasteKey — STILL BROKEN because the SOURCE ALPHA WAS + NEVER REAL. +- take-25 (**ME, this session — the user's "RE-INTRODUCING THE BOUNDING-BOX PROBLEM"**): + I removed FindContentBounds + the crop path entirely, betting full-canvas UniformToFill + was the answer. It WASN'T — the source is opaque-black, so the element rendered as a + SOLID BLACK BOX (the user's screenshot: "a black box in the lower right corner"). Killed + the crop → dead space returned AND black box. The compositor math was ALREADY correct; + gutting it was vandalism in response to a source-level bug. -**Take-23 (`f6802c7`): stride fix.** `BlitContentRaw` CropBounds path uses `src.Width` -(canvas stride = 1920) for ALL pixel indexing. `srcStride` explicitly named to prevent recurrence. - -**Take-24 (`081e4c1`): transparency still broken after scale fix — PasteKey missing CropBounds.** -Scale was correct but transparent pixels showed black (backdrop) instead of revealing webcam. -Root cause: `PasteKey` (compositor paste cache) was keyed on `(Pixels, Epoch, Width, Height, -DstW, DstH, Round, Mirror)` — NOT `CropBounds`. Since the frame Width/Height are always -1920×1080 (the canvas) regardless of crop bounds, the same key was used for all element -sizes. A stale raster (wrong crop) was served from the cache after resize or first add, -baking in opaque content over transparent regions. Fix: add `CropBounds` to `PasteKey`. -`frame.CropBounds` is included in the cache key so different crop bounds = cache miss. +**Rules, self-inflicted:** +1. Instrument FIRST. This session added: first-capture PNG dump of the raw WebView2 PNG + (to `%TEMP%\ytLive-web-.png`) + alpha min/max/mean/%zero + FindContentBounds result + logged to startup.log once per session. That's the diff between a five-take loop and a + five-minute diagnosis. +2. When the same symptom loops across takes, the PREMISE is wrong, not the code — the + capture being transparent was the load-bearing premise and it was never verified. +3. Do not delete code paths that fix one axis (crop=bbox) while debugging another + (source alpha). Revert scope creep; keep layer contributions separable. ### Shrink / re-encode an image for the README (screenshots → small hero image) diff --git a/Services/WebView2Manager.cs b/Services/WebView2Manager.cs index 9af04ad..bf4f9df 100644 --- a/Services/WebView2Manager.cs +++ b/Services/WebView2Manager.cs @@ -42,6 +42,7 @@ public sealed class WebView2Manager : IDisposable public int OutRingNext; public long Epoch; public WriteableBitmap? PreviewBitmap; + public bool DebugPngWritten; public byte[] RentOutBuffer(int size) { @@ -138,6 +139,19 @@ public sealed class WebView2Manager : IDisposable return _sessions.TryGetValue(sourceId, out var session) ? session.LatestFrame : null; } + // Injected BEFORE the document is parsed (AddScriptToExecuteOnDocumentCreatedAsync) so + // the widget page's own CSS/scripts cannot repaint html/body opaque — the reason the + // late NavigationStarting/NavigationCompleted ExecuteScriptAsync injection lost the + // fight (page styles ran first). CapturePreviewAsync "will always honor a webpage's + // background content" (MicrosoftEdge/WebView2Feedback specs/BackgroundColor.md), so a + // transparent capture REQUIRES the transparency to be in place before the page paints. + private const string TransparentBackgroundScript = + "document.documentElement.style.background='transparent';" + + "document.documentElement.style.overflow='hidden';" + + "document.documentElement.style.margin='0';" + + "if(document.body){document.body.style.background='transparent';" + + "document.body.style.overflow='hidden';document.body.style.margin='0';}"; + private async Task InitializeAsync(Source source, WebSourceSession session) { try @@ -146,22 +160,19 @@ public sealed class WebView2Manager : IDisposable var cws = session.Control.CoreWebView2!; + // Pre-page hook: runs after the global object exists but BEFORE the document + // parses / any page script runs — must complete before navigating away from + // about:blank for it to apply to the widget document. + await cws.AddScriptToExecuteOnDocumentCreatedAsync(TransparentBackgroundScript); + cws.NavigationStarting += (_, _) => { - _ = cws.ExecuteScriptAsync( - "document.documentElement.style.background='transparent';" + - "document.documentElement.style.overflow='hidden';" + - "document.body.style.background='transparent';" + - "document.body.style.overflow='hidden';"); + _ = cws.ExecuteScriptAsync(TransparentBackgroundScript); }; cws.NavigationCompleted += (_, _) => { - _ = cws.ExecuteScriptAsync( - "document.documentElement.style.background='transparent';" + - "document.documentElement.style.overflow='hidden';" + - "document.body.style.background='transparent';" + - "document.body.style.overflow='hidden';"); + _ = cws.ExecuteScriptAsync(TransparentBackgroundScript); _ = CaptureFrame(source.Id); }; @@ -231,6 +242,19 @@ public sealed class WebView2Manager : IDisposable await webView.CoreWebView2.CapturePreviewAsync( CoreWebView2CapturePreviewImageFormat.Png, ms); + if (!session.DebugPngWritten) + { + session.DebugPngWritten = true; + try + { + ms.Position = 0; + var dbg = Path.Combine(Path.GetTempPath(), $"ytLive-web-{sourceId}.png"); + File.WriteAllBytes(dbg, ms.ToArray()); + AppLog.Write($"WebView2Manager: debug capture dump -> {dbg}"); + } + catch (Exception dex) { AppLog.Write($"WebView2Manager: debug dump failed: {dex.Message}"); } + } + ms.Position = 0; var bitmap = new BitmapImage(); @@ -258,6 +282,29 @@ public sealed class WebView2Manager : IDisposable var (cropX, cropY, cropW, cropH) = FindContentBounds(pixels, pixW, pixH, stride); + if (session.Epoch == 0) + { + int minA = 255, maxA = 0; + long sumA = 0, nA = 0, zeroA = 0; + for (var y = 0; y < pixH; y += 16) + { + var row = y * stride; + for (var x = 0; x < pixW; x += 16) + { + var a = pixels[row + x * 4 + 3]; + if (a < minA) minA = a; + if (a > maxA) maxA = a; + sumA += a; + if (a == 0) zeroA++; + nA++; + } + } + AppLog.Write( + $"WebView2Manager: first capture for '{sourceId}' {pixW}x{pixH} " + + $"alpha[min={minA},max={maxA},mean={sumA / (double)nA:F1},zero={100.0 * zeroA / nA:F1}%] " + + $"contentBounds=({cropX},{cropY},{cropW},{cropH})"); + } + var outPixels = session.RentOutBuffer(cropW * cropH * 4); if (cropW == pixW && cropH == pixH && cropX == 0 && cropY == 0) Buffer.BlockCopy(pixels, 0, outPixels, 0, outPixels.Length);