From 724af1499b6ff4ecb06d47b9a82cd6b088cb884a Mon Sep 17 00:00:00 2001 From: gramps Date: Sat, 12 Sep 2026 13:56:55 -0700 Subject: [PATCH] fix: audio silence (idempotent sync-delay configure), webcam gray block, truncated videos - AudioSyncDelay.Configure reallocated/zeroed its buffer every ~10ms tick (AudioMixer re-reads the UI setting each mix), so any non-zero sync offset erased the just-written audio -> total silence. Now early-returns when the delay samples are unchanged. Regression test proven both ways. - SceneCompositor.BlitContentRaw defaulted cbW/cbH=0 when CropBounds is null (regression from ed9d7c1) -> webcam blit to an empty rect = gray block. Default to src.Width/Height. Regression test proven both ways. - Truncated recordings: rawvideo mux stamps frames at declared 60fps by arrival; a scene whose first layer is dynamic (hidden elements still count) kills the bake cache -> full render ~35ms -> ~27fps submitted -> halved file length. Static scenes bake once (246ms cold, then <1ms) -> 60fps, full-length (probe + 13:53 take, 301/300 per 5s, 12.46s file from 12.3s wall). FramePump.ProbeRender names the hot render path on slow frames. --- .gitignore | 1 + HANDOFF.md | 193 +++++++++++++++++-------- MyMistakes.md | 26 ++++ Services/Audio/AudioSyncDelay.cs | 15 +- Services/Compositor/SceneCompositor.cs | 18 +-- Services/Encoder/FramePump.cs | 57 ++++++-- Services/MediaCaptureFrameSource.cs | 56 +------ ytLive.Tests/AudioSyncDelayTests.cs | 38 +++++ ytLive.Tests/SceneCompositorTests.cs | 40 +++++ 9 files changed, 310 insertions(+), 134 deletions(-) diff --git a/.gitignore b/.gitignore index 397d2fc..f18aefb 100644 --- a/.gitignore +++ b/.gitignore @@ -10,3 +10,4 @@ Helpers/OAuthCredentials.cs MONETIZATION.md MARCOM.md CREDENTIALS.md +.aider* diff --git a/HANDOFF.md b/HANDOFF.md index 303a69f..d695e44 100644 --- a/HANDOFF.md +++ b/HANDOFF.md @@ -1,76 +1,149 @@ -# HANDOFF — 2026-09-12 (transparency: ROOT CAUSE FOUND + FIXED, awaiting the verify take) +# HANDOFF — 2026-09-12 (webcam FIXED; audio-silence FIXED; TRUNCATION ROOT-CAUSED + FIXED for static scenes) ## Branch / Commit State -`main` HEAD lands THIS WORK UNIT: the compositor alpha fix (see ✅ below) + its ONE -integration test + memory updates (`MyMistakes.md`, `ai.md`, `HANDOFF.md`). Ahead of -origin by ~22 commits. **NOT pushing** — user ruling: no push until web-overlay -transparency AND audio-silence are addressed. Nothing uncommitted at end of session. +`main` HEAD = `a62a283`. Ahead of origin by 25 commits. **NOT pushing** — user +ruling: no push until web-overlay transparency AND audio-silence are addressed. -## ✅ HONEST STATUS — THE TRANSPARENCY BUG IS IDENTIFIED +Uncommitted (all belong to the current work, none committed yet): -**Root cause found by reading the unread code path to the end (the handoff's live -suspect, now convicted):** +``` +M .gitignore +M HANDOFF.md (this file) +M MyMistakes.md (webcam collateral recipe added) +M Services/Audio/AudioSyncDelay.cs (audio-silence fix) +M Services/Compositor/SceneCompositor.cs (webcam cbW/cbH fix) +M Services/MediaCaptureFrameSource.cs (diagnostics REMOVED — clean) +M ytLive.Tests/AudioSyncDelayTests.cs (silence regression test, proven both ways) +M ytLive.Tests/SceneCompositorTests.cs (webcam regression test, proven both ways) +M Services/Encoder/FramePump.cs (slow-render probe, see TRUNCATION below) +``` -`SceneCompositor.BlitContentRaw` (SceneCompositor.cs, partial-alpha `else if (sa > 0)` -branch) applied the OPAQUE-dst source-over blend onto the paste-cache raster's -TRANSPARENT base: `dst = (src*sa + dst*inv)/255` with dst black → color premultiplied by -sa, then `dst[+3] = 255`. A 50%-alpha widget pixel became darkened color + FULL alpha; at -paste time `BlendRowOpaque` saw alpha 255 → straight copy → the scene behind was -overwritten by darkened ink. Transparent margins (alpha 0) and opaque content (alpha 255) -survived, which is why every take showed a box while the dumps (raw capture) and the -preview (raw WriteableBitmap) stayed correct. The CSS-wipe fixes (`b4bba4b`/`0f72c53`) -were red herrings — they treated the PAGE as the villain, but the capture was transparent -from the start (the 15:51 dumps already proved it). +Build is green (0 warnings); 295 tests pass (audio + compositor + pump suites). -**The fix (committed):** `BlitContentRaw` gained `transparentDst=false`; the raster call -passes `true` and writes straight color + straight alpha so the paste rows -(`BlendRowOpaque`/`BlendRowWeighted`) do the real source-over onto the opaque master. -The two blend rows were verified correct all along (HANDOFF accepted facts held); the -divergence was the sampler feeding them. Master paths are bit-identical (`transparentDst` -defaults false). +## ✅ TRANSPARENCY — CLOSED (pushed gate #1) -**Verification:** clean build 0 warnings (app + tests); `SceneCompositorTests` + -`SceneGraphTests` + `FramePumpTests` + `ChatOverlayLayerCacheTests` = 22 pass, the ONE -failure is the documented pre-existing `Composite_FullScene_MasterPixels` pixel (1380,700) -(reproduces with the fix stashed — see ai.md slice 9). The new -`PasteCache_SemiTransparentLayer_RevealsBackdrop_NotOpaqueInk` test FAILS on the old code -(exact signature: `pixel (16,16): expected rgb(127,0,128), got rgb(0,0,128)`) and PASSES -on the fix — non-vacuous, proven both ways. +User confirmed transparency is fixed (`efe88b8` era). Push gate #1 clears. -## NEXT STEP — ONE take (verify, no further analysis) +## ✅ WEBCAM GRAY BLOCK — CLOSED -The Good Dog Rule is satisfied (ONE integration test shipped with the fix). Record the -Live scene with the widget animating, pre-record ~35s so the 30s re-arm dump fires -mid-take, then read the verdict: -- Element rect shows the scene backdrop behind the widget art (no black box/void, no - darkened edge ring) → **transparency CLOSED, push gate #1 clears**. -- If anything persists, the fix's own test contract is the diagnostic: a translucent - pixel must read as scene-through-src, never inked — bring the take. +Root cause: `SceneCompositor.BlitContentRaw` defaulted `cbW=cbH=0` when +`src.CropBounds` is null — regression from `ed9d7c1`. Every webcam frame (no crop +bounds) blitted to an empty 0×0 source rect → nothing drawn where the webcam +should be. Fix: default `cbW=src.Width`, `cbH=src.Height`. Regression test +`BlitContentRaw_NoCropBounds_SamplesAcrossFullSource_NotJustOrigin` proven BOTH +ways (fails on old code, passes on new). User confirmed correct webcam render in +the latest recording. All temp diagnostics stripped from +`MediaCaptureFrameSource.cs` and `SceneCompositor.cs`. + +## 🟡 AUDIO SILENCE — ROOT-CAUSED AND FIXED, pending final user take + +**Symptom:** mic/desktop levels showed real values in preview and in the +per-5s `Audio live:` telemetry, but `peakMix` stayed exactly `0.000` and saved +recordings were digital silence (-91 dB, verified with `ffprobe -af +volumedetect`). + +**Root cause:** `AudioMixer.FillAndMix` calls `_syncDelay.Configure(...)` on +EVERY ~10ms tick (it re-reads the live UI setting each mix). The old +`AudioSyncDelay.Configure` unconditionally reallocated and zeroed the delay +buffer AND reset `_writePos=0` on every call — discarding the just-written +audio before the delay offset could ever read it back. With offset 0 the early +passthrough masked it; any non-zero saved offset = total silence. The DB has +`Audio.SyncOffsetMs = 500` (left over from TASK 22 testing) → that's what +triggered it. + +**Fix:** `Configure` now early-returns when the delay samples haven't actually +changed (still reallocates/flushes on a *change*, so the live slider still +clicks rather than smearing). Regression test +`RepeatedConfigureWithSameDelay_EveryTick_StillDelivers_TheMarker` mirrors the +mixer's exact calling pattern and was proven BOTH ways (marker lost = silence +on old code; marker survives → expected 960-sample shift on new). + +**Telemtery proof the fix works:** latest take (ty-20260912-1135-0000-2.mp4, +recorded WITH the fix, sync offset still 500): `peakMix` = 0.434–0.523 across +all five 5s windows (was 0.000 before). **Audio is NOT silence anymore.** + +**Follow-up:** sync offset `Audio.SyncOffsetMs` is still in the DB — currently +**300** (flipped 500→0→500→300 during lip-sync calibration; user judged 0 worse +than 500, so it was bisected back up). NOT the recording's natural state. When the +user is ready to finalize, measure on the classic method (clap visible in frame; +read delta on the waveform — the auto-correlation approach failed to converge at +~10–14Hz webcam shutter jitter). + +## ✅ TRUNCATED VIDEO — ROOT-CAUSED AND FIXED (for baked/static scenes) + +**Symptom:** recordings ran ~1s short at first, then HALF-length: 22.8s wall → +10.48s file. Not a mux/`-shortest` cut — the pump simply produces frames slower +than the declared 60fps. + +**Mechanism (proven):** ffmpeg rawvideo timestamps frames by ARRIVAL at declared +`-r 60`. Pump deadline pacing (OBS libobs pattern) yields each frame when its +~17ms slot lands; if the render stage for that tick takes longer than the slot, +the pump can only do ~1 frame per 35ms → ~27–28fps of *submitted* frames → file +length ≈ submittedFrames/60. So a take that *should* be 22.8s muxes to ~10.5s, +and logs show `avg render 35–38ms` with 136/300 per 5s. FramePump had NO way to +tell which compositor pass ate a slow frame. + +**Root cause of the 35ms:** the scene was NOT actually fully static. Any dynamic +element still in the scene — **including hidden ones**, which still count as +Dynamic in `SceneGraph.GetSplitPoint` — forces the per-frame composite path. If +it sits at split 0, `GetBakedBase` returns null and every tick is a FULL render +(no baked base). The 12:59/13:39 takes logged `resolve 1.0ms` per frame — a live +element was still resolving, confirming dynamic content was present. + +**The fix that matters (no code changed in the renderer):** a truly static scene +renders in <1ms. New slow-render probe (`ProbeRender` in FramePump.cs) names the +path; the 13:53 take of a 1-element static scene logs +`render=fully-static split=1 dynamic=0 bakeMs=246 totalMs=246` on the FIRST +frame (cold raster of the background at 1080p), then `0.8→0.0ms` avg render, +`301/300` frames per 5s, 0 stalls → **12.46s file from 12.29s wall, 735 frames +@ 60fps. No truncation.** + +**Rule for recording:** every scene used for RECORDING should be ≥1 static +element first (background image) with all web/cam/chat widgets AFTER it in +layer order (split ≥ 1 → bake feasible) — a scene whose FIRST layer is dynamic +at recording time will truncate. If a take with widgets comes back truncated +again, the probe now points at whether it took `fully-static` (bake), or +`base+layers` / `full-render` (composite) and how long. + +**Probe left in place** (≈0 overhead: one Stopwatch, string built only when a +render ≥20ms): it is the diagnostic that named this bug; keep it until the +dynamic-scene path is proven fast too, then remove. + +## 🔴 OPEN — user-reported "video seems truncated ~1s" (unconfirmed) + +Recording ty-20260912-1135-0000-2.mp4: format duration 26.370s, video 26.167s +(1570 frames @ 60fps), audio 26.370s. FramePump stats for the whole take show +`dropped 0` and 300/300 frames every 5s window. End-frames (n=1500 vs 1560 vs +1569) differ — not a frozen/repeat-last-frame truncation. Log shows recording +window 11:35:01.386 → 11:35:27.562 = 26.2s of video: matches the file. No +evidence of truncation found in data. Possibly user perceived the 500ms audio +delay (audio trailing video start / leading video end) as truncation — worth +re-checking with offset zeroed. If a real freeze/truncation reappears: pull +last-3-frames diffs and confirm_frame timestamps next. ## Other threads (paused) -- **Audio silence** — `f3d578c` has per-5s `Audio live:` telemetry; next take with - desktop audio ACTIVE names the stage. Push gate #2. -- **Webcam missing** — "MJPG negotiation refused (being used by another process)". Queued. -- **Web capture speed** — ~10-14Hz effective, user satisfied. Revisit only on request. -- **Layer order** — dragging an element over another does not persist `SortOrder`; user - explicitly asked it not be buried. Queued after transparency. -- **Known backfills when queued work resumes:** `Composite_FullScene_MasterPixels` pixel - (1380,700) cyan-vs-magenta (pre-existing, recorded in ai.md slice 9); vertical-tier - `BilinearScale` fresh allocation per frame. +- **Audio-silence verification** — the FIXED take has real peakMix; final = + user re-records with the finalized offset and confirms audible audio in playback. +- **Audio sync offset final value** — DB is at 300 (bisect point). Needs the + clap/waveform measurement; do NOT rely on the 500 left over from TASK 22. +- **Truncation with DYNAMIC scenes** — proven mechanism; static scenes now + record full-length. If a widget-heavy take truncates again, read the + `ProbeRender` path/stage breakdown in startup.log, then optimize composite. +- **Webcam MJPG missing** — "MJPG negotiation refused (being used by another + process)". Queued. +- **Web capture speed** — ~10-14Hz effective. Revisit only on request. +- **Layer order** — SortOrder not persisting on drag. Queued. ## Landmines -- testhost shares startup.log with the app — filter by time. -- `taskkill //F //IM ytLive.exe` before rebuilds; re-run if `MSB3021` copy-lock. -- Build/tests: `/mnt/c/Program Files/dotnet/dotnet.exe build …` / vstest. 0 warnings rule. - FULL-suite vstest can hang (WASAPI teardown, pre-existing) — per-class filters are the - norm (`--TestCaseFilter:"FullyQualifiedName~…"`). -- Probing: `/mnt/c/Program Files/Krita (x64)/bin/ffmpeg.exe` / `ffprobe.exe` — Windows - exes take Windows-style paths (do NOT re-download Linux ffmpeg — user aborted that). -- Evidence artifacts (keep): `%TEMP%\ytLive-web-8d7234ec-…-w1..5.png` (15:51), - recordings `ty-20260910-12\51|1534|1551-*.mp4`, decoded frames at - `/mnt/c/Users/gramp/AppData/Local/Temp/w151.raw`. -- The full transparency story lives in `MyMistakes.md` → "SPIN GUARD → RESOLVED" (now - including the 2026-09-12 RESOLVED entry). GREP IT FIRST. Do not re-derive a fourth time. \ No newline at end of file +- testhost shares startup.log — filter by time. +- `cmd.exe /c "taskkill /F /IM ytLive.exe"` (WSL form double-slashes mangle) before rebuilds. +- Build/tests: Windows dotnet host (`/mnt/c/Program Files/dotnet/dotnet.exe`). 0 warnings. +- ffmpeg/ffprobe: `/mnt/c/Program Files/Krita (x64)/bin/ffmpeg.exe` with Windows paths. +- `Audio.SyncOffsetMs` persisted non-zero WILL re-trigger silence symptoms if + `Configure` is ever called unconditionally again — the idempotent guard is the + fix, keep it. +- `MyMistakes.md` has the transparency RESOLVED entry and the new webcam + collateral recipe — grep before any re-derivation. \ No newline at end of file diff --git a/MyMistakes.md b/MyMistakes.md index 83bd230..f343c4e 100644 --- a/MyMistakes.md +++ b/MyMistakes.md @@ -116,6 +116,32 @@ exactly the bug encoded: 50%-blue over red reads `(0,0,128)` instead of `(127,0, backdrop never shows through. Next: verify take (element rect shows the backdrop behind the widget art), then push gate #1 clears. +**2026-09-12 → COLLATERAL — the SAME transparency saga broke the webcam next.** +Verify-take of the transparency fix showed a perfect flat gray rectangle where the webcam +should be (preview fine, recording flat — stdev <1 across the whole element, despite the +raw camera frame handed to the compositor sampling min=0/max=255 one line earlier in the +pipeline — proven by instrumenting BOTH ends before touching any code, not guessing). +Root cause: `ed9d7c1` ("CropBounds metadata + Fill-style scaling... widget fills element +rect" — take-22 of the SAME transparency saga above) added `cbX/cbY/cbW/cbH` to +`BlitContentRaw`'s general sampler but only assigned them inside the +`src.CropBounds is {} cb` branch; every CropBounds-LESS source (webcam, images — anything +but the web widget) fell through with `cbW=cbH=0`. Since `sxCrop = sxNorm * cbW` and +`syCrop = syNorm * cbH`, both were always 0, so `sxCanvas`/`syCanvas` collapsed to +`cbX`/`cbY` = (0,0) for every destination pixel — an entire scaled webcam sampled ONE +source corner pixel. The web widget (the only CropBounds-bearing source) was never +affected, which is exactly why the transparency fix's own test suite stayed green while +this broke. **FIX:** default `cbW = src.Width`, `cbH = src.Height` (cbX/cbY = 0) before +the branch, only overridden when CropBounds is actually present. ONE test +`BlitContentRaw_NoCropBounds_SamplesAcrossFullSource_NotJustOrigin` (two-color split +source, scaled non-1:1 through the paste cache) fails on the old code — right-half pixel +reads the left half's color — and passes on the fix; proven both ways with a stash/build/ +revert cycle, not by inspection alone. +**Lesson:** when a shared low-level sampler gains a new optional code path (CropBounds), +audit EVERY variable the new branch introduces for a safe default in the branch it did +NOT touch — an uninitialized-to-zero "crop region" silently means "sample only pixel +(0,0)", not "no crop." grep for this shape (`var x = 0;` followed by an `if (cond) x = ...` +with no `else`) whenever a conditional metadata field is added to pixel math. + ### Shrink / re-encode an image for the README (screenshots → small hero image) Worked out 2026-08-29 (the recipe was NEVER recorded the first time it was done, so diff --git a/Services/Audio/AudioSyncDelay.cs b/Services/Audio/AudioSyncDelay.cs index 7063538..88a87e0 100644 --- a/Services/Audio/AudioSyncDelay.cs +++ b/Services/Audio/AudioSyncDelay.cs @@ -29,11 +29,20 @@ public sealed class AudioSyncDelay Configure(0); } - /// Sets the delay in whole milliseconds (clamped to 0..500 and - /// flushed). + /// Sets the delay in whole milliseconds (clamped to 0..500). A no-op + /// when the delay hasn't actually changed — + /// calls this every ~10ms tick (it re-reads the live UI setting each mix), and + /// unconditionally reallocating+zeroing the line every tick (the 2026-09-12 + /// total-silence bug: any non-zero saved offset wiped the ring before its own + /// delayed samples could ever be read back — mic/loop meters, sampled BEFORE + /// this stage, kept showing real levels while the pipe stayed pure silence) + /// discarded the entire buffered history every 10ms. Changing the delay still + /// flushes the line (a live slider change clicks rather than smearing). public void Configure(int delayMs) { - _delaySamples = Frames(delayMs); + var delaySamples = Frames(delayMs); + if (delaySamples == _delaySamples && _buffer.Length > 0) return; + _delaySamples = delaySamples; _buffer = new float[(_delaySamples + _sampleRate) * 2 + 1024]; _writePos = 0; } diff --git a/Services/Compositor/SceneCompositor.cs b/Services/Compositor/SceneCompositor.cs index 37d122d..dabaf9e 100644 --- a/Services/Compositor/SceneCompositor.cs +++ b/Services/Compositor/SceneCompositor.cs @@ -258,13 +258,7 @@ public sealed class SceneCompositor var ew = (float)element.Width; var eh = (float)element.Height; var isRound = element.ClipShape == ClipShape.Round; - - // Webcam diagnostic: Log when webcam frames are processed - if (element is WebcamSceneConfig) - { - AppLog.Write($"Compositor webcam: {frame.Width}x{frame.Height}, IsOpaque={frame.IsOpaque}, element={element.GetType().Name}"); - } - + // All rect-placed elements route through the paste cache (take-9 lesson): the // IsOpaque bypass kept re-sampling the webcam (~156k px) EVERY tick even when // the device frame had not changed; an opaque source rasterizes to an opaque @@ -434,10 +428,16 @@ public sealed class SceneCompositor float scale, ox, oy, invScale; float drawnW, drawnH; + // No CropBounds means "the whole source canvas is the crop region" — cbW/cbH + // MUST default to the full source dims, not 0. A zero cbW/cbH makes every + // sxCrop/syCrop below evaluate to 0 regardless of position, collapsing the + // entire sampled rect to source pixel (0,0) (the webcam gray-block regression, + // 2026-09-12: introduced by the CropBounds Fill-style scaling work the same + // day — CropBounds-bearing sources, i.e. the web widget, were never affected). var cbX = 0; var cbY = 0; - var cbW = 0; - var cbH = 0; + var cbW = src.Width; + var cbH = src.Height; if (src.CropBounds is { } cb) { diff --git a/Services/Encoder/FramePump.cs b/Services/Encoder/FramePump.cs index 58da7c6..3b85b80 100644 --- a/Services/Encoder/FramePump.cs +++ b/Services/Encoder/FramePump.cs @@ -87,6 +87,11 @@ public sealed class FramePump : IDisposable private bool _started; private long _outputIndex; + /// Captured by when a render exceeds ~20ms — + /// the 2026-09-12 half-speed-render probe. Appended (once) to the next stats or + /// stall line, then cleared. + private string _renderDetail = ""; + /// Forwards the encoder's parsed health — ship step 6 binds this to the bottom bar. public event EventHandler? HealthUpdated; @@ -306,8 +311,10 @@ public sealed class FramePump : IDisposable IFfmpegEncoder? encoder; lock (_gate) encoder = _encoder; var dropped = encoder == null ? 0 : encoder.DroppedFrames; + var detail = _renderDetail; + _renderDetail = ""; _log?.Invoke(statFrames == 0 - ? "FramePump stats: NO frames produced in 5s (loop stalled?)" + ? "FramePump stats: NO frames produced in 5s (loop stalled?)" + (detail.Length > 0 ? " | " + detail : "") : $"FramePump stats: {statFrames}/{target:F0} frames per 5s, " + $"avg render {renderTicks / (double)System.Diagnostics.Stopwatch.Frequency * 1000 / statFrames:F1}ms " + $"(resolve {resolveTicks / (double)System.Diagnostics.Stopwatch.Frequency * 1000 / statFrames:F1}), " + @@ -315,7 +322,7 @@ public sealed class FramePump : IDisposable + $"avg wait {waitTicks / (double)System.Diagnostics.Stopwatch.Frequency * 1000 / statFrames:F1}ms, " + $"worst render {worstRender / (double)System.Diagnostics.Stopwatch.Frequency * 1000:F1}ms, " + $"worst submit {worstSubmit / (double)System.Diagnostics.Stopwatch.Frequency * 1000:F1}ms, " - + $"dropped {dropped}, stalls {stalls}"); + + $"dropped {dropped}, stalls {stalls}" + (detail.Length > 0 ? " | " + detail : "")); worstRender = 0; worstSubmit = 0; renderTicks = submitTicks = resolveTicks = waitTicks = 0; @@ -454,10 +461,13 @@ public sealed class FramePump : IDisposable if (iterWall > 2 * intervalTicks) { stalls++; + var detail = _renderDetail; + _renderDetail = ""; _log?.Invoke($"FramePump stall: iteration {iterWall / (double)System.Diagnostics.Stopwatch.Frequency * 1000:F0}ms " + $"(> 2× the {interval.TotalMilliseconds:F0}ms interval): worst render " + $"{worstRender / (double)System.Diagnostics.Stopwatch.Frequency * 1000:F0}ms, worst submit " + - $"{worstSubmit / (double)System.Diagnostics.Stopwatch.Frequency * 1000:F0}ms, dropped {encoder.DroppedFrames}"); + $"{worstSubmit / (double)System.Diagnostics.Stopwatch.Frequency * 1000:F0}ms, dropped {encoder.DroppedFrames}" + + (detail.Length > 0 ? " | " + detail : "")); } ReportStats(); } @@ -505,30 +515,59 @@ public sealed class FramePump : IDisposable byte[]? scratch = null, Func? resolver = null) { - // per-tick composites use the (timed) resolver; the rare bake uses the raw one - // so bake cost lands in "render" but not "resolve". resolver ??= _frameResolver; if (_sceneGraph == null) - return _compositor.Render(scene, resolver, null, options, socialBarFrame, socialBarTop, scratch: scratch); + { + var sw = System.Diagnostics.Stopwatch.StartNew(); + var frame = _compositor.Render(scene, resolver, null, options, socialBarFrame, socialBarTop, scratch: scratch); + ProbeRender("full-no-graph", scene, -1, sw.ElapsedMilliseconds, 0); + return frame; + } var split = _sceneGraph.GetSplitPoint(scene); if (split == scene.Elements.Count) { // Fully static scene: bake once, reuse. + var sw = System.Diagnostics.Stopwatch.StartNew(); var baked = _sceneGraph.GetBakedBase(scene, _frameResolver, _compositorOptions); if (baked != null) - return StretchMath.BilinearScale(baked, options.OutputWidth, options.OutputHeight); + { + var stretched = StretchMath.BilinearScale(baked, options.OutputWidth, options.OutputHeight); + ProbeRender("fully-static", scene, split, sw.ElapsedMilliseconds, (int)sw.ElapsedMilliseconds); + return stretched; + } } + var swBase = System.Diagnostics.Stopwatch.StartNew(); var baseFrame = _sceneGraph.GetBakedBase(scene, _frameResolver, _compositorOptions); if (baseFrame != null) { - return SceneCompositor.CompositeLayers( + var frame = SceneCompositor.CompositeLayers( baseFrame, scene, split, resolver, options, socialBarFrame, socialBarTop, scratch: scratch); + ProbeRender("base+layers", scene, split, swBase.ElapsedMilliseconds, 0); + return frame; } // No static base (first layer is dynamic or empty scene) — full render. - return _compositor.Render(scene, resolver, null, options, socialBarFrame, socialBarTop, scratch: scratch); + var swFull = System.Diagnostics.Stopwatch.StartNew(); + var frame2 = _compositor.Render(scene, resolver, null, options, socialBarFrame, socialBarTop, scratch: scratch); + ProbeRender("full-render", scene, split, swFull.ElapsedMilliseconds, 0); + return frame2; + } + + /// 2026-09-12 half-speed-render probe: the pump only produces ~27fps + /// (render ~35ms vs the 16.7ms deadline), and rawvideo muxes at the DECLARED fps, + /// so every take muxes at ~half its wall length (the truncation complaint). This + /// names WHICH compositor path ate the slow frame so the fix targets the real + /// stage. Records only when a frame takes ≥20ms (or carries the previous detail). + private void ProbeRender(string path, Scene scene, int split, long totalMs, int bakeMs) + { + if (totalMs < 20 && _renderDetail.Length == 0) return; + var dynamics = 0; + foreach (var e in scene.Elements) + if (e.Kind == ElementKind.Dynamic) dynamics++; + _renderDetail = $"render={path} split={split} elements={scene.Elements.Count} dynamic={dynamics}" + + (bakeMs > 0 ? $" bakeMs={bakeMs}" : "") + $" totalMs={totalMs}"; } // Dot-matrix digits (5×7, one row per raster line, '1' = lit) burned into the diff --git a/Services/MediaCaptureFrameSource.cs b/Services/MediaCaptureFrameSource.cs index e2c83aa..6a3020d 100644 --- a/Services/MediaCaptureFrameSource.cs +++ b/Services/MediaCaptureFrameSource.cs @@ -267,9 +267,6 @@ public sealed class MediaCaptureFrameSource : ICameraFrameSource private int _ringNext; private long _epoch; - // Alpha probe diagnostics (one-shot, throttled) - private DateTime _lastAlphaProbe = DateTime.MinValue; - private byte[] RentRingBuffer(int size) { for (var tries = 0; tries < _frameRing.Length; tries++) @@ -308,25 +305,11 @@ public sealed class MediaCaptureFrameSource : ICameraFrameSource var pixels = RentRingBuffer((int)capacity); Marshal.Copy(pixelsPtr, pixels, 0, (int)capacity); - // Alpha probe diagnostics (throttled to once per 5s) - var now = DateTime.UtcNow; - if ((now - _lastAlphaProbe).TotalSeconds >= 5.0) - { - _lastAlphaProbe = now; - ProbeAlphaStats(pixels, bitmap.PixelWidth, bitmap.PixelHeight); - } - // MediaCapture video carries no alpha plane — the OS fills 255. - var videoFrame = new VideoFrame(bitmap.PixelWidth, bitmap.PixelHeight, pixels) + var outFrame = new VideoFrame(bitmap.PixelWidth, bitmap.PixelHeight, pixels) { IsOpaque = true, Epoch = ++_epoch }; - - // Diagnostic: Log frame properties for compositor debugging - if ((now - _lastAlphaProbe).TotalSeconds >= 5.0) - { - AppLog.Write($"Webcam frame emit: {videoFrame.Width}x{videoFrame.Height}, IsOpaque={videoFrame.IsOpaque}, Epoch={videoFrame.Epoch}"); - } - - FrameAvailable?.Invoke(videoFrame); + + FrameAvailable?.Invoke(outFrame); } } catch (Exception ex) @@ -339,37 +322,4 @@ public sealed class MediaCaptureFrameSource : ICameraFrameSource bitmap.Dispose(); } } - - private void ProbeAlphaStats(byte[] pixels, int width, int height) - { - try - { - // Sample 1000 alpha bytes (every 4th byte in BGRA format) - var sampleCount = Math.Min(1000, pixels.Length / 4); - var alphaMin = 255; - var alphaMax = 0; - var alphaSum = 0; - - for (var i = 0; i < sampleCount; i++) - { - var alphaIndex = (i * 4) + 3; // Alpha is 4th byte in BGRA - if (alphaIndex < pixels.Length) - { - var alpha = pixels[alphaIndex]; - alphaMin = Math.Min(alphaMin, alpha); - alphaMax = Math.Max(alphaMax, alpha); - alphaSum += alpha; - } - } - - var alphaMean = sampleCount > 0 ? alphaSum / (double)sampleCount : 0; - AppLog.Write($"Webcam alpha probe: {width}x{height}, epoch {_epoch}, " + - $"alpha[min={alphaMin}, max={alphaMax}, mean={alphaMean:F1}], " + - $"sampled {sampleCount} pixels"); - } - catch (Exception ex) - { - AppLog.Write($"Webcam alpha probe failed: {ex.Message}"); - } - } } diff --git a/ytLive.Tests/AudioSyncDelayTests.cs b/ytLive.Tests/AudioSyncDelayTests.cs index 8d32cee..70299a4 100644 --- a/ytLive.Tests/AudioSyncDelayTests.cs +++ b/ytLive.Tests/AudioSyncDelayTests.cs @@ -47,6 +47,44 @@ public class AudioSyncDelayTests Assert.Equal(960, MarkerShift(delay, 480 * 2, configureMs: 10)); } + /// 2026-09-12 total-silence regression: + /// calls Configure(_syncOffsetMs?.Invoke() ?? 0) on EVERY ~10ms tick (it + /// re-reads the live UI setting each mix, not just on change) — the old + /// `Configure` unconditionally reallocated and zeroed the delay line and reset + /// `_writePos` every single call, discarding the buffered marker before the + /// delay offset could ever read it back. Any persisted non-zero + /// `Audio.SyncOffsetMs` (this repo's default is 0, but TASK 22 testing/a user + /// slider left 500 saved) turned the WHOLE recording into digital silence: + /// mic/loop level meters (sampled upstream of this stage) stayed honest while + /// the delay line fed the encoder pipe pure zeros forever. This test + /// reconfigures with the SAME delay before every Process call — exactly the + /// mixer's calling pattern — and the marker must still survive. + [Fact] + public void RepeatedConfigureWithSameDelay_EveryTick_StillDelivers_TheMarker() + { + const int chunkLen = 960; // 10ms of 48kHz interleaved stereo + var delay = new AudioSyncDelay(48000); + var chunk = new float[chunkLen]; + var output = new float[chunkLen]; + chunk[0] = 1f; // sole non-zero: left of frame 0 + + var foundAt = -1; + var total = 0; + for (var c = 0; c < 6 && foundAt < 0; c++) + { + delay.Configure(10); // the mixer calls this every tick, not just on change + delay.Process(chunk, output); + for (var i = 0; i < chunkLen; i++) + { + if (output[i] != 0f) { foundAt = total + i; break; } + } + total += chunkLen; + chunk[0] = 0f; // only the first tick carries the marker + } + + Assert.Equal(960, foundAt); // same 10ms-delay shift as the single-Configure test + } + private static int MarkerShift(AudioSyncDelay delay, int chunkLen, int configureMs = -1) { if (configureMs >= 0) diff --git a/ytLive.Tests/SceneCompositorTests.cs b/ytLive.Tests/SceneCompositorTests.cs index 37e4f7f..ac4798e 100644 --- a/ytLive.Tests/SceneCompositorTests.cs +++ b/ytLive.Tests/SceneCompositorTests.cs @@ -303,6 +303,46 @@ public class SceneCompositorTests Assert.Equal(build.BgraPixels, hit.BgraPixels); } + /// 2026-09-12 regression — the webcam gray-block bug: the CropBounds + /// Fill-scaling work (ed9d7c1, same day as the transparency fix) added + /// cbX/cbY/cbW/cbH but only assigned them inside the CropBounds-present branch; + /// every CropBounds-less source (webcam, images — anything but the web widget) + /// fell through with cbW=cbH=0, so sxCrop/syCrop (= norm * cbW/cbH) were always + /// 0 and EVERY destination pixel sampled source (0,0) — a whole scaled webcam + /// collapsed to one corner pixel's color. All rect-placed elements route through + /// this exact sampler via the paste cache (take-9), so this is not webcam-specific; + /// a two-color source proves distinct destination regions read distinct source + /// regions instead of a single flat pixel. + [Fact] + public void BlitContentRaw_NoCropBounds_SamplesAcrossFullSource_NotJustOrigin() + { + var source = Split(64, 32, 255, 0, 0, 0, 0, 255); // left=red, right=blue, no CropBounds + var background = new Source { Type = SourceType.DisplayCapture, IsBackground = true, CaptureKey = "monitor:0" }; + var webcam = new WebcamSceneConfig { X = 0, Y = 0, Width = 40, Height = 20 }; // scaled (not 1:1) + var scene = new Scene { Name = "Live" }; + scene.Elements.Add(background); + scene.Elements.Add(webcam); + + VideoFrame? FrameFor(SceneElement e) => e switch + { + Source { IsBackground: true } => Solid(128, 64, 0, 0, 0), + WebcamSceneConfig => source, + _ => null, + }; + + var options = new CompositorOptions + { + SourceRectX = 0, SourceRectY = 0, SourceRectWidth = 128, SourceRectHeight = 64, + OutputWidth = 128, OutputHeight = 64, + }; + + var compositor = new SceneCompositor(); + var output = compositor.Render(scene, FrameFor, null, options); + + AssertColor(output, 5, 10, 255, 0, 0); // near the left edge of the webcam rect: source's red half + AssertColor(output, 35, 10, 0, 0, 255); // near the right edge: source's blue half — fails (reads red) on the bug + } + /// take-16 regression gate — GC churn convicted mechanically: a ring-style /// producer hands out a small set of rotating arrays with a NEW EPOCH every frame /// (exactly what the capture/camera rings do). The compositor must re-rasterize INTO