diff --git a/HANDOFF.md b/HANDOFF.md index 156496e..303a69f 100644 --- a/HANDOFF.md +++ b/HANDOFF.md @@ -1,52 +1,76 @@ -# HANDOFF — 2026-09-10 (transparency hardening applied) +# HANDOFF — 2026-09-12 (transparency: ROOT CAUSE FOUND + FIXED, awaiting the verify take) ## Branch / Commit State -`main` HEAD will be = the new transparency-hardening commit (slice 14). Ahead of -origin by 19, NOT pushing (user ruling: no push until web-overlay transparency AND -audio-silence are addressed). +`main` HEAD lands THIS WORK UNIT: the compositor alpha fix (see ✅ below) + its ONE +integration test + memory updates (`MyMistakes.md`, `ai.md`, `HANDOFF.md`). Ahead of +origin by ~22 commits. **NOT pushing** — user ruling: no push until web-overlay +transparency AND audio-silence are addressed. Nothing uncommitted at end of session. -## ACTIVE THREAD (ONE problem at a time) +## ✅ HONEST STATUS — THE TRANSPARENCY BUG IS IDENTIFIED -**Web-uri transparency — second hardening applied (background-image wipe), awaiting take.** -The 15:34 take: the recording's box interior is the widget's OWN full-canvas opaque paint -(bright content strips at top/bottom + right bar over a black void) — NOT desktop, NOT a -compositor blend. The real-document dumps showed blank-transparent at +1s and the page's -backdrop appears later; a background-COLOR-only `!important` wipe (b4bba4b) leaves CSS -background-IMAGE (gradient/backdrop) intact — the OBS answer is `background: none -!important`, background-image too (obsproject/obs-studio#6659). +**Root cause found by reading the unread code path to the end (the handoff's live +suspect, now convicted):** -**Slice 15 (commit target, the fix):** -- `TransparentBackgroundScript` now also wipes `background-image:none!important` on - `html,body,html *` (gradients/backdrops die; ``/DOM art survives — OBS semantics). -- `NavigateCompleted` re-arms `WidgetDumpRemaining = 5` ~30s in, so this take dumps the - real document INSIDE the recording window (the missing evidence last time). -- Test extended (asserts `background-image:none!important` present, weak inline form absent). +`SceneCompositor.BlitContentRaw` (SceneCompositor.cs, partial-alpha `else if (sa > 0)` +branch) applied the OPAQUE-dst source-over blend onto the paste-cache raster's +TRANSPARENT base: `dst = (src*sa + dst*inv)/255` with dst black → color premultiplied by +sa, then `dst[+3] = 255`. A 50%-alpha widget pixel became darkened color + FULL alpha; at +paste time `BlendRowOpaque` saw alpha 255 → straight copy → the scene behind was +overwritten by darkened ink. Transparent margins (alpha 0) and opaque content (alpha 255) +survived, which is why every take showed a box while the dumps (raw capture) and the +preview (raw WriteableBitmap) stayed correct. The CSS-wipe fixes (`b4bba4b`/`0f72c53`) +were red herrings — they treated the PAGE as the villain, but the capture was transparent +from the start (the 15:51 dumps already proved it). -## THE ONE REMAINING STEP (verify, no further analysis) +**The fix (committed):** `BlitContentRaw` gained `transparentDst=false`; the raster call +passes `true` and writes straight color + straight alpha so the paste rows +(`BlendRowOpaque`/`BlendRowWeighted`) do the real source-over onto the opaque master. +The two blend rows were verified correct all along (HANDOFF accepted facts held); the +divergence was the sampler feeding them. Master paths are bit-identical (`transparentDst` +defaults false). -1. User records the Live scene, widget animating (pre-record ~35s so the 30s re-arm dump - fires mid-take). -2. Verdict: element rect shows the scene backdrop behind the widget art (no black box/void) - → transparency CLOSED, push gate #1 clears. If a black void persists BEYOND a visible - background-image, it's not background painting and only chroma-key remains — bring the - take and we do the compositor key, not more capture investigation. -3. Then the queued layer-order-save bug (reorder doesn't persist `SortOrder`) is next. +**Verification:** clean build 0 warnings (app + tests); `SceneCompositorTests` + +`SceneGraphTests` + `FramePumpTests` + `ChatOverlayLayerCacheTests` = 22 pass, the ONE +failure is the documented pre-existing `Composite_FullScene_MasterPixels` pixel (1380,700) +(reproduces with the fix stashed — see ai.md slice 9). The new +`PasteCache_SemiTransparentLayer_RevealsBackdrop_NotOpaqueInk` test FAILS on the old code +(exact signature: `pixel (16,16): expected rgb(127,0,128), got rgb(0,0,128)`) and PASSES +on the fix — non-vacuous, proven both ways. + +## NEXT STEP — ONE take (verify, no further analysis) + +The Good Dog Rule is satisfied (ONE integration test shipped with the fix). Record the +Live scene with the widget animating, pre-record ~35s so the 30s re-arm dump fires +mid-take, then read the verdict: +- Element rect shows the scene backdrop behind the widget art (no black box/void, no + darkened edge ring) → **transparency CLOSED, push gate #1 clears**. +- If anything persists, the fix's own test contract is the diagnostic: a translucent + pixel must read as scene-through-src, never inked — bring the take. ## Other threads (paused) -- **Audio silence** — `f3d578c` has per-5s `Audio live:` telemetry; next take with desktop - audio ACTIVE names the stage. Push gate #2. +- **Audio silence** — `f3d578c` has per-5s `Audio live:` telemetry; next take with + desktop audio ACTIVE names the stage. Push gate #2. - **Webcam missing** — "MJPG negotiation refused (being used by another process)". Queued. - **Web capture speed** — ~10-14Hz effective, user satisfied. Revisit only on request. +- **Layer order** — dragging an element over another does not persist `SortOrder`; user + explicitly asked it not be buried. Queued after transparency. +- **Known backfills when queued work resumes:** `Composite_FullScene_MasterPixels` pixel + (1380,700) cyan-vs-magenta (pre-existing, recorded in ai.md slice 9); vertical-tier + `BilinearScale` fresh allocation per frame. ## Landmines - testhost shares startup.log with the app — filter by time. -- App was running at commit time; `taskkill //F //IM ytLive.exe` (Windows `taskkill.exe`, - bash-quoted `//F //IM`) before rebuilds, and re-run if `MSB3021` copy-lock appears. -- Build/tests: `/mnt/c/Program Files/dotnet/dotnet.exe build …` / vstest. -- Probing: `/mnt/c/Program Files/Krita (x64)/bin/ffmpeg.exe` / `ffprobe.exe` — Windows exes - take Windows-style paths. -- Never re-derive the transparency story again — MyMistakes "SPIN GUARD → RESOLVED" is the - record (wash-rinse-repeat cost the user a whole session). \ No newline at end of file +- `taskkill //F //IM ytLive.exe` before rebuilds; re-run if `MSB3021` copy-lock. +- Build/tests: `/mnt/c/Program Files/dotnet/dotnet.exe build …` / vstest. 0 warnings rule. + FULL-suite vstest can hang (WASAPI teardown, pre-existing) — per-class filters are the + norm (`--TestCaseFilter:"FullyQualifiedName~…"`). +- Probing: `/mnt/c/Program Files/Krita (x64)/bin/ffmpeg.exe` / `ffprobe.exe` — Windows + exes take Windows-style paths (do NOT re-download Linux ffmpeg — user aborted that). +- Evidence artifacts (keep): `%TEMP%\ytLive-web-8d7234ec-…-w1..5.png` (15:51), + recordings `ty-20260910-12\51|1534|1551-*.mp4`, decoded frames at + `/mnt/c/Users/gramp/AppData/Local/Temp/w151.raw`. +- The full transparency story lives in `MyMistakes.md` → "SPIN GUARD → RESOLVED" (now + including the 2026-09-12 RESOLVED entry). GREP IT FIRST. Do not re-derive a fourth time. \ No newline at end of file diff --git a/MyMistakes.md b/MyMistakes.md index 62a6525..83bd230 100644 --- a/MyMistakes.md +++ b/MyMistakes.md @@ -93,6 +93,29 @@ closes. Self-inflicted again: the agent re-derived the whole transparency story this entry — the instrument said transparent because the capture had NOT been repainted yet. Do NOT re-derive this story a third time. +**2026-09-12 → RESOLVED — the box lived in the paste-cache RASTER, not the page.** +The 15:51 facts (alpha max=255, mean ~19, zero 57%, white rounded panel, real +transparent margins) + preview-correct + recording-box meant the capture transparency was +REAL all along; the fracture sat in `SceneCompositor.BlitContentRaw` +(SceneCompositor.cs:419-550) — the sampler that builds the element-space paste-cache +raster, the ONE path the take loop never read in full. Its partial-alpha branch applied +the OPAQUE-dst blend onto a TRANSPARENT raster base: `dst = (src*sa + dst*inv)/255` +with dst black → color PREMULTIPLIED by sa, then `dst[+3] = 255` — a 50%-alpha widget +pixel became darkened color + FULL alpha. At paste time `BlendRowOpaque` saw alpha 255 → +straight copy → the scene behind was overwritten by darkened ink. Fully-transparent +margins (alpha 0, `continue`) and fully-opaque content (alpha 255 branch) survived — +which is why every take showed a box while the dumps and the preview (raw WriteableBitmap, +unaffected by the compositor) stayed correct, and why the CSS-wipe fixes (`b4bba4b` / +`0f72c53`) were red herrings: they treated the page as the villain, but the capture was +transparent from the start. **FIX:** `BlitContentRaw` gained `transparentDst=false`; +the raster call site passes `true` and writes STRAIGHT color + straight alpha so the +paste rows (`BlendRowOpaque`/`BlendRowWeighted`) do the real source-over onto the opaque +master. The master paths are untouched (bit-identical). ONE test +`PasteCache_SemiTransparentLayer_RevealsBackdrop_NotOpaqueInk` fails on the old code with +exactly the bug encoded: 50%-blue over red reads `(0,0,128)` instead of `(127,0,128)` — +backdrop never shows through. Next: verify take (element rect shows the backdrop behind +the widget art), then push gate #1 clears. + ### Shrink / re-encode an image for the README (screenshots → small hero image) Worked out 2026-08-29 (the recipe was NEVER recorded the first time it was done, so diff --git a/Services/Compositor/SceneCompositor.cs b/Services/Compositor/SceneCompositor.cs index 06f6dab..85cf29a 100644 --- a/Services/Compositor/SceneCompositor.cs +++ b/Services/Compositor/SceneCompositor.cs @@ -308,7 +308,7 @@ public sealed class SceneCompositor } if (raster == null) raster = new VideoFrame(dw, dh, new byte[dw * dh * 4]); Array.Clear(raster.BgraPixels); - BlitContentRaw(raster.BgraPixels, dw, dh, 0, 0, ew, eh, frame, 1f, isRound, mirror); + BlitContentRaw(raster.BgraPixels, dw, dh, 0, 0, ew, eh, frame, 1f, isRound, mirror, transparentDst: true); lock (_pasteGate) { // Only the SOURCE frame's identity is a real key owner; drop any older @@ -419,7 +419,8 @@ public sealed class SceneCompositor private static void BlitContentRaw( byte[] dst, int dstW, int dstH, float ex, float ey, float ew, float eh, - VideoFrame src, float opacity, bool isRound, bool isMirror) + VideoFrame src, float opacity, bool isRound, bool isMirror, + bool transparentDst = false) { if (ew <= 0 || eh <= 0) return; @@ -529,11 +530,27 @@ public sealed class SceneCompositor } else if (sa > 0) { - var inv = 255 - sa; - dst[di] = (byte)((sb * sa + dst[di] * inv + 127) / 255); - dst[di + 1] = (byte)((sg * sa + dst[di + 1] * inv + 127) / 255); - dst[di + 2] = (byte)((sr * sa + dst[di + 2] * inv + 127) / 255); - dst[di + 3] = 255; + if (transparentDst) + { + // Element-raster pass on a TRANSPARENT base (the paste cache): + // keep straight color + straight alpha so the paste rows can + // source-over onto the opaque master. Premultiplying here (the + // opaque-dst blend) stamped alpha 255 onto already-alpha-scaled + // color — the paste over-wrote the scene with darkened ink, the + // web widget's "black box" in recordings. + dst[di] = (byte)sb; + dst[di + 1] = (byte)sg; + dst[di + 2] = (byte)sr; + dst[di + 3] = (byte)sa; + } + else + { + var inv = 255 - sa; + dst[di] = (byte)((sb * sa + dst[di] * inv + 127) / 255); + dst[di + 1] = (byte)((sg * sa + dst[di + 1] * inv + 127) / 255); + dst[di + 2] = (byte)((sr * sa + dst[di + 2] * inv + 127) / 255); + dst[di + 3] = 255; + } } } } diff --git a/ai.md b/ai.md index 321429a..97d53bd 100644 --- a/ai.md +++ b/ai.md @@ -758,7 +758,13 @@ seam:** `Func`, `Func` resolver, `FuncTHE page-transparency fix (the web widget's recording "black box"): a + /// semi-transparent layer flowing through the paste-cache raster must reveal the + /// backdrop through it (true source-over), never an opaque darkened block. The raster + /// is built on a TRANSPARENT base, but the old partial-alpha branch applied the + /// opaque-dst blend (premultiplied color + forced alpha 255), so every translucent + /// widget pixel pasted as opaque ink — the box in recordings while the raw-bitmap + /// preview stayed correct. A half-alpha blue widget over a red backdrop must read + /// (127,0,128) here; the buggy raster pastes (0,0,128). + [Fact] + public void PasteCache_SemiTransparentLayer_RevealsBackdrop_NotOpaqueInk() + { + var red = Solid(128, 64, 255, 0, 0); // live backdrop + var web = new byte[32 * 16 * 4]; // 2:1 widget canvas, half-alpha blue, uniform + for (var i = 0; i < web.Length; i += 4) + { + web[i] = 255; web[i + 1] = 0; web[i + 2] = 0; web[i + 3] = 128; + } + var semitransparent = new VideoFrame(32, 16, web); // NOT IsOpaque + + var background = new Source { Type = SourceType.DisplayCapture, IsBackground = true, CaptureKey = "monitor:0" }; + var widget = new Source { Type = SourceType.WebSource, X = 8, Y = 8, Width = 16, Height = 16 }; + var scene = new Scene { Name = "Live" }; + scene.Elements.Add(background); + scene.Elements.Add(widget); + + VideoFrame? FrameFor(SceneElement e) => e switch + { + Source { IsBackground: true } => red, + Source { Type: SourceType.WebSource } => semitransparent, + _ => null, + }; + + var options = new CompositorOptions + { + SourceRectX = 0, SourceRectY = 0, SourceRectWidth = 128, SourceRectHeight = 64, + OutputWidth = 128, OutputHeight = 64, + }; + + var compositor = new SceneCompositor(); + var build = compositor.Render(scene, FrameFor, null, options); // raster build + paste + var hit = compositor.Render(scene, FrameFor, null, options); // paste-cache hit + foreach (var output in new[] { build, hit }) + { + AssertColor(output, 16, 16, 127, 0, 128); // backdrop shows through the 50% blue + AssertColor(output, 4, 4, 255, 0, 0); // outside the element: untouched backdrop + } + Assert.Equal(build.BgraPixels, hit.BgraPixels); + } + /// take-16 regression gate — GC churn convicted mechanically: a ring-style /// producer hands out a small set of rotating arrays with a NEW EPOCH every frame /// (exactly what the capture/camera rings do). The compositor must re-rasterize INTO