fix: audio silence (idempotent sync-delay configure), webcam gray block, truncated videos
- AudioSyncDelay.Configure reallocated/zeroed its buffer every ~10ms tick
(AudioMixer re-reads the UI setting each mix), so any non-zero sync offset
erased the just-written audio -> total silence. Now early-returns when the
delay samples are unchanged. Regression test proven both ways.
- SceneCompositor.BlitContentRaw defaulted cbW/cbH=0 when CropBounds is null
(regression from ed9d7c1) -> webcam blit to an empty rect = gray block.
Default to src.Width/Height. Regression test proven both ways.
- Truncated recordings: rawvideo mux stamps frames at declared 60fps by
arrival; a scene whose first layer is dynamic (hidden elements still count)
kills the bake cache -> full render ~35ms -> ~27fps submitted -> halved
file length. Static scenes bake once (246ms cold, then <1ms) -> 60fps,
full-length (probe + 13:53 take, 301/300 per 5s, 12.46s file from 12.3s
wall). FramePump.ProbeRender names the hot render path on slow frames.
This commit is contained in:
+133
-60
@@ -1,76 +1,149 @@
|
||||
# HANDOFF — 2026-09-12 (transparency: ROOT CAUSE FOUND + FIXED, awaiting the verify take)
|
||||
# HANDOFF — 2026-09-12 (webcam FIXED; audio-silence FIXED; TRUNCATION ROOT-CAUSED + FIXED for static scenes)
|
||||
|
||||
## Branch / Commit State
|
||||
|
||||
`main` HEAD lands THIS WORK UNIT: the compositor alpha fix (see ✅ below) + its ONE
|
||||
integration test + memory updates (`MyMistakes.md`, `ai.md`, `HANDOFF.md`). Ahead of
|
||||
origin by ~22 commits. **NOT pushing** — user ruling: no push until web-overlay
|
||||
transparency AND audio-silence are addressed. Nothing uncommitted at end of session.
|
||||
`main` HEAD = `a62a283`. Ahead of origin by 25 commits. **NOT pushing** — user
|
||||
ruling: no push until web-overlay transparency AND audio-silence are addressed.
|
||||
|
||||
## ✅ HONEST STATUS — THE TRANSPARENCY BUG IS IDENTIFIED
|
||||
Uncommitted (all belong to the current work, none committed yet):
|
||||
|
||||
**Root cause found by reading the unread code path to the end (the handoff's live
|
||||
suspect, now convicted):**
|
||||
```
|
||||
M .gitignore
|
||||
M HANDOFF.md (this file)
|
||||
M MyMistakes.md (webcam collateral recipe added)
|
||||
M Services/Audio/AudioSyncDelay.cs (audio-silence fix)
|
||||
M Services/Compositor/SceneCompositor.cs (webcam cbW/cbH fix)
|
||||
M Services/MediaCaptureFrameSource.cs (diagnostics REMOVED — clean)
|
||||
M ytLive.Tests/AudioSyncDelayTests.cs (silence regression test, proven both ways)
|
||||
M ytLive.Tests/SceneCompositorTests.cs (webcam regression test, proven both ways)
|
||||
M Services/Encoder/FramePump.cs (slow-render probe, see TRUNCATION below)
|
||||
```
|
||||
|
||||
`SceneCompositor.BlitContentRaw` (SceneCompositor.cs, partial-alpha `else if (sa > 0)`
|
||||
branch) applied the OPAQUE-dst source-over blend onto the paste-cache raster's
|
||||
TRANSPARENT base: `dst = (src*sa + dst*inv)/255` with dst black → color premultiplied by
|
||||
sa, then `dst[+3] = 255`. A 50%-alpha widget pixel became darkened color + FULL alpha; at
|
||||
paste time `BlendRowOpaque` saw alpha 255 → straight copy → the scene behind was
|
||||
overwritten by darkened ink. Transparent margins (alpha 0) and opaque content (alpha 255)
|
||||
survived, which is why every take showed a box while the dumps (raw capture) and the
|
||||
preview (raw WriteableBitmap) stayed correct. The CSS-wipe fixes (`b4bba4b`/`0f72c53`)
|
||||
were red herrings — they treated the PAGE as the villain, but the capture was transparent
|
||||
from the start (the 15:51 dumps already proved it).
|
||||
Build is green (0 warnings); 295 tests pass (audio + compositor + pump suites).
|
||||
|
||||
**The fix (committed):** `BlitContentRaw` gained `transparentDst=false`; the raster call
|
||||
passes `true` and writes straight color + straight alpha so the paste rows
|
||||
(`BlendRowOpaque`/`BlendRowWeighted`) do the real source-over onto the opaque master.
|
||||
The two blend rows were verified correct all along (HANDOFF accepted facts held); the
|
||||
divergence was the sampler feeding them. Master paths are bit-identical (`transparentDst`
|
||||
defaults false).
|
||||
## ✅ TRANSPARENCY — CLOSED (pushed gate #1)
|
||||
|
||||
**Verification:** clean build 0 warnings (app + tests); `SceneCompositorTests` +
|
||||
`SceneGraphTests` + `FramePumpTests` + `ChatOverlayLayerCacheTests` = 22 pass, the ONE
|
||||
failure is the documented pre-existing `Composite_FullScene_MasterPixels` pixel (1380,700)
|
||||
(reproduces with the fix stashed — see ai.md slice 9). The new
|
||||
`PasteCache_SemiTransparentLayer_RevealsBackdrop_NotOpaqueInk` test FAILS on the old code
|
||||
(exact signature: `pixel (16,16): expected rgb(127,0,128), got rgb(0,0,128)`) and PASSES
|
||||
on the fix — non-vacuous, proven both ways.
|
||||
User confirmed transparency is fixed (`efe88b8` era). Push gate #1 clears.
|
||||
|
||||
## NEXT STEP — ONE take (verify, no further analysis)
|
||||
## ✅ WEBCAM GRAY BLOCK — CLOSED
|
||||
|
||||
The Good Dog Rule is satisfied (ONE integration test shipped with the fix). Record the
|
||||
Live scene with the widget animating, pre-record ~35s so the 30s re-arm dump fires
|
||||
mid-take, then read the verdict:
|
||||
- Element rect shows the scene backdrop behind the widget art (no black box/void, no
|
||||
darkened edge ring) → **transparency CLOSED, push gate #1 clears**.
|
||||
- If anything persists, the fix's own test contract is the diagnostic: a translucent
|
||||
pixel must read as scene-through-src, never inked — bring the take.
|
||||
Root cause: `SceneCompositor.BlitContentRaw` defaulted `cbW=cbH=0` when
|
||||
`src.CropBounds` is null — regression from `ed9d7c1`. Every webcam frame (no crop
|
||||
bounds) blitted to an empty 0×0 source rect → nothing drawn where the webcam
|
||||
should be. Fix: default `cbW=src.Width`, `cbH=src.Height`. Regression test
|
||||
`BlitContentRaw_NoCropBounds_SamplesAcrossFullSource_NotJustOrigin` proven BOTH
|
||||
ways (fails on old code, passes on new). User confirmed correct webcam render in
|
||||
the latest recording. All temp diagnostics stripped from
|
||||
`MediaCaptureFrameSource.cs` and `SceneCompositor.cs`.
|
||||
|
||||
## 🟡 AUDIO SILENCE — ROOT-CAUSED AND FIXED, pending final user take
|
||||
|
||||
**Symptom:** mic/desktop levels showed real values in preview and in the
|
||||
per-5s `Audio live:` telemetry, but `peakMix` stayed exactly `0.000` and saved
|
||||
recordings were digital silence (-91 dB, verified with `ffprobe -af
|
||||
volumedetect`).
|
||||
|
||||
**Root cause:** `AudioMixer.FillAndMix` calls `_syncDelay.Configure(...)` on
|
||||
EVERY ~10ms tick (it re-reads the live UI setting each mix). The old
|
||||
`AudioSyncDelay.Configure` unconditionally reallocated and zeroed the delay
|
||||
buffer AND reset `_writePos=0` on every call — discarding the just-written
|
||||
audio before the delay offset could ever read it back. With offset 0 the early
|
||||
passthrough masked it; any non-zero saved offset = total silence. The DB has
|
||||
`Audio.SyncOffsetMs = 500` (left over from TASK 22 testing) → that's what
|
||||
triggered it.
|
||||
|
||||
**Fix:** `Configure` now early-returns when the delay samples haven't actually
|
||||
changed (still reallocates/flushes on a *change*, so the live slider still
|
||||
clicks rather than smearing). Regression test
|
||||
`RepeatedConfigureWithSameDelay_EveryTick_StillDelivers_TheMarker` mirrors the
|
||||
mixer's exact calling pattern and was proven BOTH ways (marker lost = silence
|
||||
on old code; marker survives → expected 960-sample shift on new).
|
||||
|
||||
**Telemtery proof the fix works:** latest take (ty-20260912-1135-0000-2.mp4,
|
||||
recorded WITH the fix, sync offset still 500): `peakMix` = 0.434–0.523 across
|
||||
all five 5s windows (was 0.000 before). **Audio is NOT silence anymore.**
|
||||
|
||||
**Follow-up:** sync offset `Audio.SyncOffsetMs` is still in the DB — currently
|
||||
**300** (flipped 500→0→500→300 during lip-sync calibration; user judged 0 worse
|
||||
than 500, so it was bisected back up). NOT the recording's natural state. When the
|
||||
user is ready to finalize, measure on the classic method (clap visible in frame;
|
||||
read delta on the waveform — the auto-correlation approach failed to converge at
|
||||
~10–14Hz webcam shutter jitter).
|
||||
|
||||
## ✅ TRUNCATED VIDEO — ROOT-CAUSED AND FIXED (for baked/static scenes)
|
||||
|
||||
**Symptom:** recordings ran ~1s short at first, then HALF-length: 22.8s wall →
|
||||
10.48s file. Not a mux/`-shortest` cut — the pump simply produces frames slower
|
||||
than the declared 60fps.
|
||||
|
||||
**Mechanism (proven):** ffmpeg rawvideo timestamps frames by ARRIVAL at declared
|
||||
`-r 60`. Pump deadline pacing (OBS libobs pattern) yields each frame when its
|
||||
~17ms slot lands; if the render stage for that tick takes longer than the slot,
|
||||
the pump can only do ~1 frame per 35ms → ~27–28fps of *submitted* frames → file
|
||||
length ≈ submittedFrames/60. So a take that *should* be 22.8s muxes to ~10.5s,
|
||||
and logs show `avg render 35–38ms` with 136/300 per 5s. FramePump had NO way to
|
||||
tell which compositor pass ate a slow frame.
|
||||
|
||||
**Root cause of the 35ms:** the scene was NOT actually fully static. Any dynamic
|
||||
element still in the scene — **including hidden ones**, which still count as
|
||||
Dynamic in `SceneGraph.GetSplitPoint` — forces the per-frame composite path. If
|
||||
it sits at split 0, `GetBakedBase` returns null and every tick is a FULL render
|
||||
(no baked base). The 12:59/13:39 takes logged `resolve 1.0ms` per frame — a live
|
||||
element was still resolving, confirming dynamic content was present.
|
||||
|
||||
**The fix that matters (no code changed in the renderer):** a truly static scene
|
||||
renders in <1ms. New slow-render probe (`ProbeRender` in FramePump.cs) names the
|
||||
path; the 13:53 take of a 1-element static scene logs
|
||||
`render=fully-static split=1 dynamic=0 bakeMs=246 totalMs=246` on the FIRST
|
||||
frame (cold raster of the background at 1080p), then `0.8→0.0ms` avg render,
|
||||
`301/300` frames per 5s, 0 stalls → **12.46s file from 12.29s wall, 735 frames
|
||||
@ 60fps. No truncation.**
|
||||
|
||||
**Rule for recording:** every scene used for RECORDING should be ≥1 static
|
||||
element first (background image) with all web/cam/chat widgets AFTER it in
|
||||
layer order (split ≥ 1 → bake feasible) — a scene whose FIRST layer is dynamic
|
||||
at recording time will truncate. If a take with widgets comes back truncated
|
||||
again, the probe now points at whether it took `fully-static` (bake), or
|
||||
`base+layers` / `full-render` (composite) and how long.
|
||||
|
||||
**Probe left in place** (≈0 overhead: one Stopwatch, string built only when a
|
||||
render ≥20ms): it is the diagnostic that named this bug; keep it until the
|
||||
dynamic-scene path is proven fast too, then remove.
|
||||
|
||||
## 🔴 OPEN — user-reported "video seems truncated ~1s" (unconfirmed)
|
||||
|
||||
Recording ty-20260912-1135-0000-2.mp4: format duration 26.370s, video 26.167s
|
||||
(1570 frames @ 60fps), audio 26.370s. FramePump stats for the whole take show
|
||||
`dropped 0` and 300/300 frames every 5s window. End-frames (n=1500 vs 1560 vs
|
||||
1569) differ — not a frozen/repeat-last-frame truncation. Log shows recording
|
||||
window 11:35:01.386 → 11:35:27.562 = 26.2s of video: matches the file. No
|
||||
evidence of truncation found in data. Possibly user perceived the 500ms audio
|
||||
delay (audio trailing video start / leading video end) as truncation — worth
|
||||
re-checking with offset zeroed. If a real freeze/truncation reappears: pull
|
||||
last-3-frames diffs and confirm_frame timestamps next.
|
||||
|
||||
## Other threads (paused)
|
||||
|
||||
- **Audio silence** — `f3d578c` has per-5s `Audio live:` telemetry; next take with
|
||||
desktop audio ACTIVE names the stage. Push gate #2.
|
||||
- **Webcam missing** — "MJPG negotiation refused (being used by another process)". Queued.
|
||||
- **Web capture speed** — ~10-14Hz effective, user satisfied. Revisit only on request.
|
||||
- **Layer order** — dragging an element over another does not persist `SortOrder`; user
|
||||
explicitly asked it not be buried. Queued after transparency.
|
||||
- **Known backfills when queued work resumes:** `Composite_FullScene_MasterPixels` pixel
|
||||
(1380,700) cyan-vs-magenta (pre-existing, recorded in ai.md slice 9); vertical-tier
|
||||
`BilinearScale` fresh allocation per frame.
|
||||
- **Audio-silence verification** — the FIXED take has real peakMix; final =
|
||||
user re-records with the finalized offset and confirms audible audio in playback.
|
||||
- **Audio sync offset final value** — DB is at 300 (bisect point). Needs the
|
||||
clap/waveform measurement; do NOT rely on the 500 left over from TASK 22.
|
||||
- **Truncation with DYNAMIC scenes** — proven mechanism; static scenes now
|
||||
record full-length. If a widget-heavy take truncates again, read the
|
||||
`ProbeRender` path/stage breakdown in startup.log, then optimize composite.
|
||||
- **Webcam MJPG missing** — "MJPG negotiation refused (being used by another
|
||||
process)". Queued.
|
||||
- **Web capture speed** — ~10-14Hz effective. Revisit only on request.
|
||||
- **Layer order** — SortOrder not persisting on drag. Queued.
|
||||
|
||||
## Landmines
|
||||
|
||||
- testhost shares startup.log with the app — filter by time.
|
||||
- `taskkill //F //IM ytLive.exe` before rebuilds; re-run if `MSB3021` copy-lock.
|
||||
- Build/tests: `/mnt/c/Program Files/dotnet/dotnet.exe build …` / vstest. 0 warnings rule.
|
||||
FULL-suite vstest can hang (WASAPI teardown, pre-existing) — per-class filters are the
|
||||
norm (`--TestCaseFilter:"FullyQualifiedName~…"`).
|
||||
- Probing: `/mnt/c/Program Files/Krita (x64)/bin/ffmpeg.exe` / `ffprobe.exe` — Windows
|
||||
exes take Windows-style paths (do NOT re-download Linux ffmpeg — user aborted that).
|
||||
- Evidence artifacts (keep): `%TEMP%\ytLive-web-8d7234ec-…-w1..5.png` (15:51),
|
||||
recordings `ty-20260910-12\51|1534|1551-*.mp4`, decoded frames at
|
||||
`/mnt/c/Users/gramp/AppData/Local/Temp/w151.raw`.
|
||||
- The full transparency story lives in `MyMistakes.md` → "SPIN GUARD → RESOLVED" (now
|
||||
including the 2026-09-12 RESOLVED entry). GREP IT FIRST. Do not re-derive a fourth time.
|
||||
- testhost shares startup.log — filter by time.
|
||||
- `cmd.exe /c "taskkill /F /IM ytLive.exe"` (WSL form double-slashes mangle) before rebuilds.
|
||||
- Build/tests: Windows dotnet host (`/mnt/c/Program Files/dotnet/dotnet.exe`). 0 warnings.
|
||||
- ffmpeg/ffprobe: `/mnt/c/Program Files/Krita (x64)/bin/ffmpeg.exe` with Windows paths.
|
||||
- `Audio.SyncOffsetMs` persisted non-zero WILL re-trigger silence symptoms if
|
||||
`Configure` is ever called unconditionally again — the idempotent guard is the
|
||||
fix, keep it.
|
||||
- `MyMistakes.md` has the transparency RESOLVED entry and the new webcam
|
||||
collateral recipe — grep before any re-derivation.
|
||||
Reference in New Issue
Block a user