fix(web): re-point the widget diagnostic at the REAL document — the transparency premise gets verified
The one-shot dump + alpha log was bound to the FIRST capture ever = the initial
about:blank placeholder (alpha=0, rgb=0, 5/5 sessions) — a blind instrument. The
pre-parse transparency injection (1a39b09) is intact but was NEVER verified
(MyMistakes '? VERIFY'), and the 2026-09-10 take still shows a black, opaque
block. Stop guessing: NavigationCompleted for a non-about:blank document now
arms WidgetDumpRemaining=5; the next 5 post-paint captures dump
%TEMP%\ytLive-web-<id>-w1..5.png + shared AlphaStats(min/max/mean/zero%) +
FindContentBounds to startup.log.
The stats line alone names the branch: zero%≈opaque ⇒ the injection did not hold
for this widget's CSS (fix: !important stylesheet / chroma-key); large zero% +
tight bounds ⇒ capture IS transparent and the black lives in the compositor
blend. No new test: WebView2 runtime is not instantiable in the suite and the
re-point is log-only; 8/8 WebView2 tests still pass.
This commit is contained in:
+48
-74
@@ -1,89 +1,63 @@
|
||||
# HANDOFF — 2026-09-10 (afternoon)
|
||||
# HANDOFF — 2026-09-10 (late afternoon)
|
||||
|
||||
## Branch / Commit State
|
||||
|
||||
`main` HEAD currently = `5e78065` (slice 11, committed). **Slice 12 (audio diagnostics) is
|
||||
uncommitted** in the working tree. **Ahead of origin by 17, NOT pushing** — user ruling (2026-09-10):
|
||||
do not push until the web-overlay **transparency AND audio-silence** issues are addressed.
|
||||
Milestone tag `milestone-recording-timing` (annotated) on `c45cbc9` — rollback point:
|
||||
`git reset --hard milestone-recording-timing`.
|
||||
`main` HEAD currently = `f3d578c` (slice 12 audio telemetry). **Slice 13 (web transparency
|
||||
diagnostic re-point) is uncommitted** in the working tree. Ahead of origin by 18, NOT pushing
|
||||
(user ruling: no push until web-overlay transparency AND audio-silence are addressed).
|
||||
|
||||
## Timing saga — CLOSED (verified)
|
||||
## ACTIVE THREAD (the user's directive: ONE problem at a time)
|
||||
|
||||
Slices 9 (`bd396e4`) + 10 (`c45cbc9`) committed. Takes `0949`/`0957` verified: counter
|
||||
+1/frame, zero gaps/dups. User confirmed → **timing CLOSED.**
|
||||
**Web-uri transparency: still a black, opaque block** (take `ty-20260910-1202-0000-2.mp4` —
|
||||
user: "the web-uri resource transparency is still a black, opaque block. The desired animation
|
||||
runs just fine."). The animation-speed work is done; transparency is now THE web problem.
|
||||
|
||||
## Slice 11 (web capture cadence) — committed `5e78065`
|
||||
Institutional memory (MyMistakes, spin-guard entry): WebView2 `CapturePreviewAsync` honors the
|
||||
page's CSS — the pre-parse injection (`1a39b09`, `AddScriptToExecuteOnDocumentCreatedAsync`)
|
||||
is the recorded fix and is INTACT in `WebView2Manager.cs` (line ~157). The "? VERIFY" was never
|
||||
closed because the diagnostic dump + alpha log bound to the FIRST capture ever = the about:blank
|
||||
placeholder (blind; 5/5 sessions alpha=0 rgb=0).
|
||||
|
||||
10Hz capture → ~30Hz target with de-throttle flags + `CaptureScheduler` (latest-wins in-flight
|
||||
drop). First verification take (`ty-20260910-1038-0000-2.mp4`, 1423 frames 23.7s) results:
|
||||
**Slice 13 change (uncommitted):** re-point the instrument at reality —
|
||||
- `WebSourceSession.WidgetDumpRemaining`; `NavigationCompleted` arms `= 5` when the newly-loaded
|
||||
document is NOT about:blank.
|
||||
- Next 5 captures dump `%TEMP%\ytLive-web-<id>-w1..5.png` + `AlphaStats(...)` + FindContentBounds
|
||||
to startup.log.
|
||||
- `AlphaStats` extracted from the old inline first-capture block.
|
||||
|
||||
- **Capture cost: 35-117ms per frame** (not the hoped 10-30ms). Effective cadence is ~10-14Hz,
|
||||
NOT 30Hz — the scheduler's in-flight drop silently collapsed the target back to roughly the old
|
||||
rate. The in-flight drop works (no stacking), but the decode path (BitmapImage →
|
||||
FormatConvertedBitmap → FindContentBounds full-scan) is too expensive to reach 30Hz.
|
||||
Web animation speed will not have improved meaningfully. Next: the capture cost decides — either
|
||||
optimize the decode path (crop-only, cached bounds) or drop the target to ~15Hz.
|
||||
- FramePump: healthy (300/300 frames, 0 dropped, 6 stalls worst 166ms).
|
||||
- Camera: initialized (YUY2 640×480) but "MJPG negotiation refused (being used by another process)"
|
||||
logged at startup — the webcam source was contested.
|
||||
- Audio: **full-length silent AAC** (−91dB, 1124 frames). User reported: no transparency in web-uri,
|
||||
no webcam video, no audio. The three symptoms are separate threads.
|
||||
Build 0 warnings; 8/8 WebView2 tests pass (runtime not instantiable — re-point is log-only, no
|
||||
new test). Suite otherwise 290/291 (pre-existing compositor pixel).
|
||||
|
||||
## Slice 12 (audio diagnostics) — uncommitted, in-tree
|
||||
## THE DECISION LADDER (no more cargo-culting)
|
||||
|
||||
The silence is capture-side: the pipe ran for 23.95s, ffmpeg connected and read ~9.1MB of zeros.
|
||||
Sources started (`Audio: using system default mic...` logged at 10:38:01.296), no failure
|
||||
callbacks fired. The mixer's existing integration test (`Mix_WithFiltersDuckAndGain_Lands_On_AudioPipe`)
|
||||
proves the loop→pipe path carries real audio when fed. Two remaining capture-side suspects:
|
||||
(1) default render device mismatch — audio played on a non-default endpoint (common);
|
||||
(2) both endpoints held exclusive by another process.
|
||||
1. User launches the app with the widget sources active, waits ~10s, closes. (NO recording needed.)
|
||||
2. Read the `widget capture [1..5/5]` lines in startup.log:
|
||||
- **zero% ≈ 0 (opaque):** pre-parse injection did NOT hold for this widget's own CSS →
|
||||
FIX = stronger transparent-background enforcement (document-level `!important` stylesheet
|
||||
injected pre-parse + re-asserted, per the OBS user.css precedent) OR chroma-key the known
|
||||
backdrop color in `CaptureFrame`. Pick after seeing the `-w1..5.png` PNGs.
|
||||
- **zero% large + tight contentBounds:** the capture HAS transparent margins → the recording's
|
||||
black block lives downstream → audit the compositor web-layer blend/underlay (PMA-vs-straight
|
||||
alpha, black pre-fill). Do NOT touch the capture path.
|
||||
3. Implement the branch's fix with ONE integration test (chroma-key math or compositor blend has
|
||||
testable seams), docs same-commit, then ONE verification recording.
|
||||
|
||||
**Changes in tree:**
|
||||
- `Services/Audio/AudioMixer.cs`: `FillAndMix` now returns `(MicRms, MicDrained, LoopDrained)`;
|
||||
LiveLoopAsync accumulates per-5s telemetry → startup.log:
|
||||
`Audio live: pipe connected=, dropped writes=, micLevel=, loopLevel=, drained, peakMix`;
|
||||
StartLive/StopLive log start/stop lines.
|
||||
- `Services/Audio/NamedPipeAudioWriter.cs`: new `DroppedWrites` counter — nonzero means audio
|
||||
was dropped before ffmpeg connected (names "pipe never connected" stage).
|
||||
- Docs: `ai.md` slice 12 entry.
|
||||
## Other threads (paused)
|
||||
|
||||
Suite: 290/291 pass — same sole pre-existing compositor pixel failure.
|
||||
|
||||
## NEXT STEP
|
||||
|
||||
1. Commit slice 12 (scope-check passed on the three source/doc files).
|
||||
2. User runs a recording with DESKTOP AUDIO ACTIVE (music/game playing → verify the "Desktop Audio"
|
||||
footer bar moves during the take). Send startup.log lines containing `Audio live:`.
|
||||
The log names the stage:
|
||||
- `loopLevel > 0` + `peakMix > 0` + file silent → pipe-side (impossible per integration test; should not appear)
|
||||
- `loopLevel ≈ 0` + `drained 0/0` → capture delivered nothing (default device mismatch or exclusive hold)
|
||||
- `dropped writes > 0` → encoder pipe never connected (timing/ordering bug)
|
||||
3. If capture-side: add explicit loopback-device selection (enumerate active render endpoints, log
|
||||
the chosen one, allow user to pick) — the fix that makes the mismatch impossible.
|
||||
4. Then return to web transparency (post-paint dump) and webcam.
|
||||
|
||||
## Still Open
|
||||
|
||||
- **Audio silence** — push gate reason #2. Diagnostic committed pending take (see above).
|
||||
- **Web transparency UNVERIFIED** — instrument blind (about:blank). Needs post-paint re-point.
|
||||
Push gate reason #1.
|
||||
- **Web capture speed** — 30Hz target not achievable with current decode path; capture cost
|
||||
35-117ms/frame → effective ~10-14Hz. Open question whether to optimize path or accept ~15Hz.
|
||||
- **Webcam missing** in take-1038 — "MJPG negotiation refused (being used by another process)"
|
||||
at startup; camera initialized but possibly contested. Separate thread, queued.
|
||||
- Pre-existing compositor pixel test failure (never in scope).
|
||||
- **Audio silence** — `f3d578c` added per-5s `Audio live:` telemetry; next take with desktop
|
||||
audio ACTIVE + those lines names the stage. Push gate #2.
|
||||
- **Web capture speed** — 30Hz NOT achieved (35-117ms/capture → effective ~10-14Hz). The
|
||||
animation "runs fine" so the user is satisfied; revisit only if they want more.
|
||||
- **Webcam missing** in the 10:38/12:02 takes — "MJPG negotiation refused (being used by
|
||||
another process)" at startup. Queued behind transparency+audio.
|
||||
|
||||
## Landmines
|
||||
|
||||
- testhost shares startup.log with app — filter by time when triaging.
|
||||
- Locked DLLs: `taskkill //F //IM testhost.exe //IM ytLive.exe` before rebuild.
|
||||
- Build/tests via `/mnt/c/Program Files/dotnet/dotnet.exe build …` / `… vstest
|
||||
"C:\Users\gramp\Documents\Code\projects\ytLive\ytLive.Tests\bin\Debug\net8.0-windows10.0.19041.0\ytLive.Tests.dll"`.
|
||||
- Probing recordings: `/mnt/c/Program Files/Krita (x64)/bin/ffmpeg.exe` / `ffprobe.exe`.
|
||||
- `tools/ticker`: constant ~+982ms offset on the FIRST line is cosmetic.
|
||||
- WebView2 capture cost: 35-117ms per full-HD PNG encode+decode — the hard ceiling on
|
||||
web animation capture rate. Slice 13 candidate if the user wants to pursue 30Hz.
|
||||
- WASAPI loopback: captures the DEFAULT render endpoint only — if the user plays audio
|
||||
through a non-default device, the recording is silently silent. This is likely the root cause
|
||||
of the audio-silence issue. The fix requires device enumeration + selection.
|
||||
- testhost shares startup.log with app — filter by time.
|
||||
- `taskkill //F //IM testhost.exe //IM ytLive.exe` before rebuild.
|
||||
- Build/tests: `/mnt/c/Program Files/dotnet/dotnet.exe build …` / vstest.
|
||||
- Probing: `/mnt/c/Program Files/Krita (x64)/bin/ffmpeg.exe` / `ffprobe.exe`.
|
||||
- Do NOT delete the crop/compositor paths while debugging source alpha (MyMistakes rule 3 —
|
||||
take-25 vandalism).
|
||||
- The user is frustrated with take-loops; the widget dump needs NO recording — an app launch
|
||||
suffices. Ask for launch + 10s + close, not a take.
|
||||
Reference in New Issue
Block a user