fix(web): re-point the widget diagnostic at the REAL document — the transparency premise gets verified

The one-shot dump + alpha log was bound to the FIRST capture ever = the initial
about:blank placeholder (alpha=0, rgb=0, 5/5 sessions) — a blind instrument. The
pre-parse transparency injection (1a39b09) is intact but was NEVER verified
(MyMistakes '? VERIFY'), and the 2026-09-10 take still shows a black, opaque
block. Stop guessing: NavigationCompleted for a non-about:blank document now
arms WidgetDumpRemaining=5; the next 5 post-paint captures dump
%TEMP%\ytLive-web-<id>-w1..5.png + shared AlphaStats(min/max/mean/zero%) +
FindContentBounds to startup.log.

The stats line alone names the branch: zero%≈opaque ⇒ the injection did not hold
for this widget's CSS (fix: !important stylesheet / chroma-key); large zero% +
tight bounds ⇒ capture IS transparent and the black lives in the compositor
blend. No new test: WebView2 runtime is not instantiable in the suite and the
re-point is log-only; 8/8 WebView2 tests still pass.
This commit is contained in:
2026-09-10 12:08:53 -07:00
parent f3d578c81b
commit ea347f211c
4 changed files with 125 additions and 91 deletions
+48 -74
View File
@@ -1,89 +1,63 @@
# HANDOFF — 2026-09-10 (afternoon)
# HANDOFF — 2026-09-10 (late afternoon)
## Branch / Commit State
`main` HEAD currently = `5e78065` (slice 11, committed). **Slice 12 (audio diagnostics) is
uncommitted** in the working tree. **Ahead of origin by 17, NOT pushing** — user ruling (2026-09-10):
do not push until the web-overlay **transparency AND audio-silence** issues are addressed.
Milestone tag `milestone-recording-timing` (annotated) on `c45cbc9` — rollback point:
`git reset --hard milestone-recording-timing`.
`main` HEAD currently = `f3d578c` (slice 12 audio telemetry). **Slice 13 (web transparency
diagnostic re-point) is uncommitted** in the working tree. Ahead of origin by 18, NOT pushing
(user ruling: no push until web-overlay transparency AND audio-silence are addressed).
## Timing saga — CLOSED (verified)
## ACTIVE THREAD (the user's directive: ONE problem at a time)
Slices 9 (`bd396e4`) + 10 (`c45cbc9`) committed. Takes `0949`/`0957` verified: counter
+1/frame, zero gaps/dups. User confirmed → **timing CLOSED.**
**Web-uri transparency: still a black, opaque block** (take `ty-20260910-1202-0000-2.mp4` —
user: "the web-uri resource transparency is still a black, opaque block. The desired animation
runs just fine."). The animation-speed work is done; transparency is now THE web problem.
## Slice 11 (web capture cadence) — committed `5e78065`
Institutional memory (MyMistakes, spin-guard entry): WebView2 `CapturePreviewAsync` honors the
page's CSS — the pre-parse injection (`1a39b09`, `AddScriptToExecuteOnDocumentCreatedAsync`)
is the recorded fix and is INTACT in `WebView2Manager.cs` (line ~157). The "? VERIFY" was never
closed because the diagnostic dump + alpha log bound to the FIRST capture ever = the about:blank
placeholder (blind; 5/5 sessions alpha=0 rgb=0).
10Hz capture → ~30Hz target with de-throttle flags + `CaptureScheduler` (latest-wins in-flight
drop). First verification take (`ty-20260910-1038-0000-2.mp4`, 1423 frames 23.7s) results:
**Slice 13 change (uncommitted):** re-point the instrument at reality —
- `WebSourceSession.WidgetDumpRemaining`; `NavigationCompleted` arms `= 5` when the newly-loaded
document is NOT about:blank.
- Next 5 captures dump `%TEMP%\ytLive-web-<id>-w1..5.png` + `AlphaStats(...)` + FindContentBounds
to startup.log.
- `AlphaStats` extracted from the old inline first-capture block.
- **Capture cost: 35-117ms per frame** (not the hoped 10-30ms). Effective cadence is ~10-14Hz,
NOT 30Hz — the scheduler's in-flight drop silently collapsed the target back to roughly the old
rate. The in-flight drop works (no stacking), but the decode path (BitmapImage →
FormatConvertedBitmap → FindContentBounds full-scan) is too expensive to reach 30Hz.
Web animation speed will not have improved meaningfully. Next: the capture cost decides — either
optimize the decode path (crop-only, cached bounds) or drop the target to ~15Hz.
- FramePump: healthy (300/300 frames, 0 dropped, 6 stalls worst 166ms).
- Camera: initialized (YUY2 640×480) but "MJPG negotiation refused (being used by another process)"
logged at startup — the webcam source was contested.
- Audio: **full-length silent AAC** (−91dB, 1124 frames). User reported: no transparency in web-uri,
no webcam video, no audio. The three symptoms are separate threads.
Build 0 warnings; 8/8 WebView2 tests pass (runtime not instantiable — re-point is log-only, no
new test). Suite otherwise 290/291 (pre-existing compositor pixel).
## Slice 12 (audio diagnostics) — uncommitted, in-tree
## THE DECISION LADDER (no more cargo-culting)
The silence is capture-side: the pipe ran for 23.95s, ffmpeg connected and read ~9.1MB of zeros.
Sources started (`Audio: using system default mic...` logged at 10:38:01.296), no failure
callbacks fired. The mixer's existing integration test (`Mix_WithFiltersDuckAndGain_Lands_On_AudioPipe`)
proves the loop→pipe path carries real audio when fed. Two remaining capture-side suspects:
(1) default render device mismatch — audio played on a non-default endpoint (common);
(2) both endpoints held exclusive by another process.
1. User launches the app with the widget sources active, waits ~10s, closes. (NO recording needed.)
2. Read the `widget capture [1..5/5]` lines in startup.log:
- **zero% ≈ 0 (opaque):** pre-parse injection did NOT hold for this widget's own CSS →
FIX = stronger transparent-background enforcement (document-level `!important` stylesheet
injected pre-parse + re-asserted, per the OBS user.css precedent) OR chroma-key the known
backdrop color in `CaptureFrame`. Pick after seeing the `-w1..5.png` PNGs.
- **zero% large + tight contentBounds:** the capture HAS transparent margins → the recording's
black block lives downstream → audit the compositor web-layer blend/underlay (PMA-vs-straight
alpha, black pre-fill). Do NOT touch the capture path.
3. Implement the branch's fix with ONE integration test (chroma-key math or compositor blend has
testable seams), docs same-commit, then ONE verification recording.
**Changes in tree:**
- `Services/Audio/AudioMixer.cs`: `FillAndMix` now returns `(MicRms, MicDrained, LoopDrained)`;
LiveLoopAsync accumulates per-5s telemetry → startup.log:
`Audio live: pipe connected=, dropped writes=, micLevel=, loopLevel=, drained, peakMix`;
StartLive/StopLive log start/stop lines.
- `Services/Audio/NamedPipeAudioWriter.cs`: new `DroppedWrites` counter — nonzero means audio
was dropped before ffmpeg connected (names "pipe never connected" stage).
- Docs: `ai.md` slice 12 entry.
## Other threads (paused)
Suite: 290/291 pass — same sole pre-existing compositor pixel failure.
## NEXT STEP
1. Commit slice 12 (scope-check passed on the three source/doc files).
2. User runs a recording with DESKTOP AUDIO ACTIVE (music/game playing → verify the "Desktop Audio"
footer bar moves during the take). Send startup.log lines containing `Audio live:`.
The log names the stage:
- `loopLevel > 0` + `peakMix > 0` + file silent → pipe-side (impossible per integration test; should not appear)
- `loopLevel ≈ 0` + `drained 0/0` → capture delivered nothing (default device mismatch or exclusive hold)
- `dropped writes > 0` → encoder pipe never connected (timing/ordering bug)
3. If capture-side: add explicit loopback-device selection (enumerate active render endpoints, log
the chosen one, allow user to pick) — the fix that makes the mismatch impossible.
4. Then return to web transparency (post-paint dump) and webcam.
## Still Open
- **Audio silence** — push gate reason #2. Diagnostic committed pending take (see above).
- **Web transparency UNVERIFIED** — instrument blind (about:blank). Needs post-paint re-point.
Push gate reason #1.
- **Web capture speed** — 30Hz target not achievable with current decode path; capture cost
35-117ms/frame → effective ~10-14Hz. Open question whether to optimize path or accept ~15Hz.
- **Webcam missing** in take-1038 — "MJPG negotiation refused (being used by another process)"
at startup; camera initialized but possibly contested. Separate thread, queued.
- Pre-existing compositor pixel test failure (never in scope).
- **Audio silence** — `f3d578c` added per-5s `Audio live:` telemetry; next take with desktop
audio ACTIVE + those lines names the stage. Push gate #2.
- **Web capture speed** — 30Hz NOT achieved (35-117ms/capture → effective ~10-14Hz). The
animation "runs fine" so the user is satisfied; revisit only if they want more.
- **Webcam missing** in the 10:38/12:02 takes — "MJPG negotiation refused (being used by
another process)" at startup. Queued behind transparency+audio.
## Landmines
- testhost shares startup.log with app — filter by time when triaging.
- Locked DLLs: `taskkill //F //IM testhost.exe //IM ytLive.exe` before rebuild.
- Build/tests via `/mnt/c/Program Files/dotnet/dotnet.exe build …` / `… vstest
"C:\Users\gramp\Documents\Code\projects\ytLive\ytLive.Tests\bin\Debug\net8.0-windows10.0.19041.0\ytLive.Tests.dll"`.
- Probing recordings: `/mnt/c/Program Files/Krita (x64)/bin/ffmpeg.exe` / `ffprobe.exe`.
- `tools/ticker`: constant ~+982ms offset on the FIRST line is cosmetic.
- WebView2 capture cost: 35-117ms per full-HD PNG encode+decode — the hard ceiling on
web animation capture rate. Slice 13 candidate if the user wants to pursue 30Hz.
- WASAPI loopback: captures the DEFAULT render endpoint only — if the user plays audio
through a non-default device, the recording is silently silent. This is likely the root cause
of the audio-silence issue. The fix requires device enumeration + selection.
- testhost shares startup.log with app — filter by time.
- `taskkill //F //IM testhost.exe //IM ytLive.exe` before rebuild.
- Build/tests: `/mnt/c/Program Files/dotnet/dotnet.exe build …` / vstest.
- Probing: `/mnt/c/Program Files/Krita (x64)/bin/ffmpeg.exe` / `ffprobe.exe`.
- Do NOT delete the crop/compositor paths while debugging source alpha (MyMistakes rule 3 —
take-25 vandalism).
- The user is frustrated with take-loops; the widget dump needs NO recording — an app launch
suffices. Ask for launch + 10s + close, not a take.