fix(compositor): keep straight alpha in the paste-cache raster — the web widget's recording black box

The element-space raster builds on a TRANSPARENT base but BlitContentRaw's
partial-alpha branch applied the opaque-dst source-over blend: color premultiplied
by the sampled alpha, then alpha forced to 255. Pasting that raster saw a==255 and
straight-copied darkened ink over the scene — a recording box that the raw-bitmap
preview (correct alpha) never showed. Transparent margins and opaque content were
unaffected, which is why every take looped on the page/CSS while the capture was
transparent all along (15:51 dumps: alpha max 255, mean ~19, zero 57%).

BlitContentRaw now takes transparentDst; the raster call passes true and writes
straight color + straight alpha so the paste rows (BlendRowOpaque/Weighted) do the
real source-over onto the opaque master. Master paths byte-identical.

ONE integration test PasteCache_SemiTransparentLayer_RevealsBackdrop_NotOpaqueInk:
50%-blue over red reads (127,0,128) fixed vs (0,0,128) buggy — proven both ways
(verified by stashing the fix: fails before, passes after). Clean build, 0 warnings;
22/23 compositor-class tests pass, the sole failure the documented pre-existing
Composite_FullScene_MasterPixels pixel (1380,700).

Alpha-compositing model: standard source-over with producer-cached surfaces, the
OBS/libyuv paste model already cited in ai.md/MyMistakes (rawvideo recipe, row-blit
BLEND_NONE / straight-alpha branches); full story in MyMistakes (RESOLVED entry).

Per the good-dog rule: one integration test, memory updates (MyMistakes/ai.md/
HANDOFF) in the same commit.
This commit is contained in:
2026-09-12 08:57:58 -07:00
parent 0f72c53aa5
commit efe88b8a0b
5 changed files with 162 additions and 43 deletions
+59 -35
View File
@@ -1,52 +1,76 @@
# HANDOFF — 2026-09-10 (transparency hardening applied) # HANDOFF — 2026-09-12 (transparency: ROOT CAUSE FOUND + FIXED, awaiting the verify take)
## Branch / Commit State ## Branch / Commit State
`main` HEAD will be = the new transparency-hardening commit (slice 14). Ahead of `main` HEAD lands THIS WORK UNIT: the compositor alpha fix (see ✅ below) + its ONE
origin by 19, NOT pushing (user ruling: no push until web-overlay transparency AND integration test + memory updates (`MyMistakes.md`, `ai.md`, `HANDOFF.md`). Ahead of
audio-silence are addressed). origin by ~22 commits. **NOT pushing** — user ruling: no push until web-overlay
transparency AND audio-silence are addressed. Nothing uncommitted at end of session.
## ACTIVE THREAD (ONE problem at a time) ## ✅ HONEST STATUS — THE TRANSPARENCY BUG IS IDENTIFIED
**Web-uri transparency — second hardening applied (background-image wipe), awaiting take.** **Root cause found by reading the unread code path to the end (the handoff's live
The 15:34 take: the recording's box interior is the widget's OWN full-canvas opaque paint suspect, now convicted):**
(bright content strips at top/bottom + right bar over a black void) — NOT desktop, NOT a
compositor blend. The real-document dumps showed blank-transparent at +1s and the page's
backdrop appears later; a background-COLOR-only `!important` wipe (b4bba4b) leaves CSS
background-IMAGE (gradient/backdrop) intact — the OBS answer is `background: none
!important`, background-image too (obsproject/obs-studio#6659).
**Slice 15 (commit target, the fix):** `SceneCompositor.BlitContentRaw` (SceneCompositor.cs, partial-alpha `else if (sa > 0)`
- `TransparentBackgroundScript` now also wipes `background-image:none!important` on branch) applied the OPAQUE-dst source-over blend onto the paste-cache raster's
`html,body,html *` (gradients/backdrops die; `<img>`/DOM art survives — OBS semantics). TRANSPARENT base: `dst = (src*sa + dst*inv)/255` with dst black → color premultiplied by
- `NavigateCompleted` re-arms `WidgetDumpRemaining = 5` ~30s in, so this take dumps the sa, then `dst[+3] = 255`. A 50%-alpha widget pixel became darkened color + FULL alpha; at
real document INSIDE the recording window (the missing evidence last time). paste time `BlendRowOpaque` saw alpha 255 → straight copy → the scene behind was
- Test extended (asserts `background-image:none!important` present, weak inline form absent). overwritten by darkened ink. Transparent margins (alpha 0) and opaque content (alpha 255)
survived, which is why every take showed a box while the dumps (raw capture) and the
preview (raw WriteableBitmap) stayed correct. The CSS-wipe fixes (`b4bba4b`/`0f72c53`)
were red herrings — they treated the PAGE as the villain, but the capture was transparent
from the start (the 15:51 dumps already proved it).
## THE ONE REMAINING STEP (verify, no further analysis) **The fix (committed):** `BlitContentRaw` gained `transparentDst=false`; the raster call
passes `true` and writes straight color + straight alpha so the paste rows
(`BlendRowOpaque`/`BlendRowWeighted`) do the real source-over onto the opaque master.
The two blend rows were verified correct all along (HANDOFF accepted facts held); the
divergence was the sampler feeding them. Master paths are bit-identical (`transparentDst`
defaults false).
1. User records the Live scene, widget animating (pre-record ~35s so the 30s re-arm dump **Verification:** clean build 0 warnings (app + tests); `SceneCompositorTests` +
fires mid-take). `SceneGraphTests` + `FramePumpTests` + `ChatOverlayLayerCacheTests` = 22 pass, the ONE
2. Verdict: element rect shows the scene backdrop behind the widget art (no black box/void) failure is the documented pre-existing `Composite_FullScene_MasterPixels` pixel (1380,700)
→ transparency CLOSED, push gate #1 clears. If a black void persists BEYOND a visible (reproduces with the fix stashed — see ai.md slice 9). The new
background-image, it's not background painting and only chroma-key remains — bring the `PasteCache_SemiTransparentLayer_RevealsBackdrop_NotOpaqueInk` test FAILS on the old code
take and we do the compositor key, not more capture investigation. (exact signature: `pixel (16,16): expected rgb(127,0,128), got rgb(0,0,128)`) and PASSES
3. Then the queued layer-order-save bug (reorder doesn't persist `SortOrder`) is next. on the fix — non-vacuous, proven both ways.
## NEXT STEP — ONE take (verify, no further analysis)
The Good Dog Rule is satisfied (ONE integration test shipped with the fix). Record the
Live scene with the widget animating, pre-record ~35s so the 30s re-arm dump fires
mid-take, then read the verdict:
- Element rect shows the scene backdrop behind the widget art (no black box/void, no
darkened edge ring) → **transparency CLOSED, push gate #1 clears**.
- If anything persists, the fix's own test contract is the diagnostic: a translucent
pixel must read as scene-through-src, never inked — bring the take.
## Other threads (paused) ## Other threads (paused)
- **Audio silence** — `f3d578c` has per-5s `Audio live:` telemetry; next take with desktop - **Audio silence** — `f3d578c` has per-5s `Audio live:` telemetry; next take with
audio ACTIVE names the stage. Push gate #2. desktop audio ACTIVE names the stage. Push gate #2.
- **Webcam missing** — "MJPG negotiation refused (being used by another process)". Queued. - **Webcam missing** — "MJPG negotiation refused (being used by another process)". Queued.
- **Web capture speed** — ~10-14Hz effective, user satisfied. Revisit only on request. - **Web capture speed** — ~10-14Hz effective, user satisfied. Revisit only on request.
- **Layer order** — dragging an element over another does not persist `SortOrder`; user
explicitly asked it not be buried. Queued after transparency.
- **Known backfills when queued work resumes:** `Composite_FullScene_MasterPixels` pixel
(1380,700) cyan-vs-magenta (pre-existing, recorded in ai.md slice 9); vertical-tier
`BilinearScale` fresh allocation per frame.
## Landmines ## Landmines
- testhost shares startup.log with the app — filter by time. - testhost shares startup.log with the app — filter by time.
- App was running at commit time; `taskkill //F //IM ytLive.exe` (Windows `taskkill.exe`, - `taskkill //F //IM ytLive.exe` before rebuilds; re-run if `MSB3021` copy-lock.
bash-quoted `//F //IM`) before rebuilds, and re-run if `MSB3021` copy-lock appears. - Build/tests: `/mnt/c/Program Files/dotnet/dotnet.exe build …` / vstest. 0 warnings rule.
- Build/tests: `/mnt/c/Program Files/dotnet/dotnet.exe build …` / vstest. FULL-suite vstest can hang (WASAPI teardown, pre-existing) — per-class filters are the
- Probing: `/mnt/c/Program Files/Krita (x64)/bin/ffmpeg.exe` / `ffprobe.exe` — Windows exes norm (`--TestCaseFilter:"FullyQualifiedName~…"`).
take Windows-style paths. - Probing: `/mnt/c/Program Files/Krita (x64)/bin/ffmpeg.exe` / `ffprobe.exe` — Windows
- Never re-derive the transparency story again — MyMistakes "SPIN GUARD → RESOLVED" is the exes take Windows-style paths (do NOT re-download Linux ffmpeg — user aborted that).
record (wash-rinse-repeat cost the user a whole session). - Evidence artifacts (keep): `%TEMP%\ytLive-web-8d7234ec-…-w1..5.png` (15:51),
recordings `ty-20260910-12\51|1534|1551-*.mp4`, decoded frames at
`/mnt/c/Users/gramp/AppData/Local/Temp/w151.raw`.
- The full transparency story lives in `MyMistakes.md` → "SPIN GUARD → RESOLVED" (now
including the 2026-09-12 RESOLVED entry). GREP IT FIRST. Do not re-derive a fourth time.
+23
View File
@@ -93,6 +93,29 @@ closes. Self-inflicted again: the agent re-derived the whole transparency story
this entry — the instrument said transparent because the capture had NOT been this entry — the instrument said transparent because the capture had NOT been
repainted yet. Do NOT re-derive this story a third time. repainted yet. Do NOT re-derive this story a third time.
**2026-09-12 → RESOLVED — the box lived in the paste-cache RASTER, not the page.**
The 15:51 facts (alpha max=255, mean ~19, zero 57%, white rounded panel, real
transparent margins) + preview-correct + recording-box meant the capture transparency was
REAL all along; the fracture sat in `SceneCompositor.BlitContentRaw`
(SceneCompositor.cs:419-550) — the sampler that builds the element-space paste-cache
raster, the ONE path the take loop never read in full. Its partial-alpha branch applied
the OPAQUE-dst blend onto a TRANSPARENT raster base: `dst = (src*sa + dst*inv)/255`
with dst black → color PREMULTIPLIED by sa, then `dst[+3] = 255` — a 50%-alpha widget
pixel became darkened color + FULL alpha. At paste time `BlendRowOpaque` saw alpha 255 →
straight copy → the scene behind was overwritten by darkened ink. Fully-transparent
margins (alpha 0, `continue`) and fully-opaque content (alpha 255 branch) survived —
which is why every take showed a box while the dumps and the preview (raw WriteableBitmap,
unaffected by the compositor) stayed correct, and why the CSS-wipe fixes (`b4bba4b` /
`0f72c53`) were red herrings: they treated the page as the villain, but the capture was
transparent from the start. **FIX:** `BlitContentRaw` gained `transparentDst=false`;
the raster call site passes `true` and writes STRAIGHT color + straight alpha so the
paste rows (`BlendRowOpaque`/`BlendRowWeighted`) do the real source-over onto the opaque
master. The master paths are untouched (bit-identical). ONE test
`PasteCache_SemiTransparentLayer_RevealsBackdrop_NotOpaqueInk` fails on the old code with
exactly the bug encoded: 50%-blue over red reads `(0,0,128)` instead of `(127,0,128)` —
backdrop never shows through. Next: verify take (element rect shows the backdrop behind
the widget art), then push gate #1 clears.
### Shrink / re-encode an image for the README (screenshots → small hero image) ### Shrink / re-encode an image for the README (screenshots → small hero image)
Worked out 2026-08-29 (the recipe was NEVER recorded the first time it was done, so Worked out 2026-08-29 (the recipe was NEVER recorded the first time it was done, so
+19 -2
View File
@@ -308,7 +308,7 @@ public sealed class SceneCompositor
} }
if (raster == null) raster = new VideoFrame(dw, dh, new byte[dw * dh * 4]); if (raster == null) raster = new VideoFrame(dw, dh, new byte[dw * dh * 4]);
Array.Clear(raster.BgraPixels); Array.Clear(raster.BgraPixels);
BlitContentRaw(raster.BgraPixels, dw, dh, 0, 0, ew, eh, frame, 1f, isRound, mirror); BlitContentRaw(raster.BgraPixels, dw, dh, 0, 0, ew, eh, frame, 1f, isRound, mirror, transparentDst: true);
lock (_pasteGate) lock (_pasteGate)
{ {
// Only the SOURCE frame's identity is a real key owner; drop any older // Only the SOURCE frame's identity is a real key owner; drop any older
@@ -419,7 +419,8 @@ public sealed class SceneCompositor
private static void BlitContentRaw( private static void BlitContentRaw(
byte[] dst, int dstW, int dstH, byte[] dst, int dstW, int dstH,
float ex, float ey, float ew, float eh, float ex, float ey, float ew, float eh,
VideoFrame src, float opacity, bool isRound, bool isMirror) VideoFrame src, float opacity, bool isRound, bool isMirror,
bool transparentDst = false)
{ {
if (ew <= 0 || eh <= 0) return; if (ew <= 0 || eh <= 0) return;
@@ -528,6 +529,21 @@ public sealed class SceneCompositor
dst[di + 3] = 255; dst[di + 3] = 255;
} }
else if (sa > 0) else if (sa > 0)
{
if (transparentDst)
{
// Element-raster pass on a TRANSPARENT base (the paste cache):
// keep straight color + straight alpha so the paste rows can
// source-over onto the opaque master. Premultiplying here (the
// opaque-dst blend) stamped alpha 255 onto already-alpha-scaled
// color — the paste over-wrote the scene with darkened ink, the
// web widget's "black box" in recordings.
dst[di] = (byte)sb;
dst[di + 1] = (byte)sg;
dst[di + 2] = (byte)sr;
dst[di + 3] = (byte)sa;
}
else
{ {
var inv = 255 - sa; var inv = 255 - sa;
dst[di] = (byte)((sb * sa + dst[di] * inv + 127) / 255); dst[di] = (byte)((sb * sa + dst[di] * inv + 127) / 255);
@@ -538,6 +554,7 @@ public sealed class SceneCompositor
} }
} }
} }
}
/// <summary>Two-stage bilinear in 8.8 fixed point; result within ±1 of the /// <summary>Two-stage bilinear in 8.8 fixed point; result within ±1 of the
/// float StretchMath.SampleBgra (well inside every pixel test's ±2 tolerance). /// float StretchMath.SampleBgra (well inside every pixel test's ±2 tolerance).
+7 -1
View File
@@ -758,7 +758,13 @@ seam:** `Func<Scene?>`, `Func<SceneElement, VideoFrame?>` resolver, `Func<Compos
routes non-opaque layers through `BlitCachedLayer`: a layer rasterizes ONCE into an element-space, routes non-opaque layers through `BlitCachedLayer`: a layer rasterizes ONCE into an element-space,
TRANSPARENT-based frame keyed by (source-array identity, source W×H, ceil'd dst rect, round, TRANSPARENT-based frame keyed by (source-array identity, source W×H, ceil'd dst rect, round,
mirror), then every later tick PASTES it (integer position, row alpha-blend, opacity applied at mirror), then every later tick PASTES it (integer position, row alpha-blend, opacity applied at
paste). Producers hand out fresh immutable arrays, so array-identity keys can never serve stale paste). **Correction (2026-09-12, the web-widget "black box"):** the raster builds on a
TRANSPARENT base, but the sampler's PARTIAL-alpha branch used the opaque-dst blend — it
premultiplied the color into RGB and forced `alpha=255`. A translucent widget pixel then pasted
as opaque darkened ink (the box in recordings) while the raw-bitmap preview stayed correct.
`BlitContentRaw` now takes `transparentDst` (the raster call passes `true` and writes straight
color + straight alpha; the paste rows do the source-over). Master paths are bit-identical.
Guard: `PasteCache_SemiTransparentLayer_RevealsBackdrop_NotOpaqueInk`. Producers hand out fresh immutable arrays, so array-identity keys can never serve stale
content; dict bounded at 48, cleared wholesale on overflow. Only changing content (webcam device content; dict bounded at 48, cleared wholesale on overflow. Only changing content (webcam device
frames, web capture ticks, chat messages) resamples; the opaque backdrop keeps its memcpy path. frames, web capture ticks, chat messages) resamples; the opaque backdrop keeps its memcpy path.
Position/opacity drags are now near-free (no resample — paste params, not cache keys). Position/opacity drags are now near-free (no resample — paste params, not cache keys).
+49
View File
@@ -254,6 +254,55 @@ public class SceneCompositorTests
Assert.Equal(out2.BgraPixels, out3.BgraPixels); Assert.Equal(out2.BgraPixels, out3.BgraPixels);
} }
/// <summary>THE page-transparency fix (the web widget's recording "black box"): a
/// semi-transparent layer flowing through the paste-cache raster must reveal the
/// backdrop through it (true source-over), never an opaque darkened block. The raster
/// is built on a TRANSPARENT base, but the old partial-alpha branch applied the
/// opaque-dst blend (premultiplied color + forced alpha 255), so every translucent
/// widget pixel pasted as opaque ink — the box in recordings while the raw-bitmap
/// preview stayed correct. A half-alpha blue widget over a red backdrop must read
/// (127,0,128) here; the buggy raster pastes (0,0,128).</summary>
[Fact]
public void PasteCache_SemiTransparentLayer_RevealsBackdrop_NotOpaqueInk()
{
var red = Solid(128, 64, 255, 0, 0); // live backdrop
var web = new byte[32 * 16 * 4]; // 2:1 widget canvas, half-alpha blue, uniform
for (var i = 0; i < web.Length; i += 4)
{
web[i] = 255; web[i + 1] = 0; web[i + 2] = 0; web[i + 3] = 128;
}
var semitransparent = new VideoFrame(32, 16, web); // NOT IsOpaque
var background = new Source { Type = SourceType.DisplayCapture, IsBackground = true, CaptureKey = "monitor:0" };
var widget = new Source { Type = SourceType.WebSource, X = 8, Y = 8, Width = 16, Height = 16 };
var scene = new Scene { Name = "Live" };
scene.Elements.Add(background);
scene.Elements.Add(widget);
VideoFrame? FrameFor(SceneElement e) => e switch
{
Source { IsBackground: true } => red,
Source { Type: SourceType.WebSource } => semitransparent,
_ => null,
};
var options = new CompositorOptions
{
SourceRectX = 0, SourceRectY = 0, SourceRectWidth = 128, SourceRectHeight = 64,
OutputWidth = 128, OutputHeight = 64,
};
var compositor = new SceneCompositor();
var build = compositor.Render(scene, FrameFor, null, options); // raster build + paste
var hit = compositor.Render(scene, FrameFor, null, options); // paste-cache hit
foreach (var output in new[] { build, hit })
{
AssertColor(output, 16, 16, 127, 0, 128); // backdrop shows through the 50% blue
AssertColor(output, 4, 4, 255, 0, 0); // outside the element: untouched backdrop
}
Assert.Equal(build.BgraPixels, hit.BgraPixels);
}
/// <summary>take-16 regression gate — GC churn convicted mechanically: a ring-style /// <summary>take-16 regression gate — GC churn convicted mechanically: a ring-style
/// producer hands out a small set of rotating arrays with a NEW EPOCH every frame /// producer hands out a small set of rotating arrays with a NEW EPOCH every frame
/// (exactly what the capture/camera rings do). The compositor must re-rasterize INTO /// (exactly what the capture/camera rings do). The compositor must re-rasterize INTO