fix: audio silence (idempotent sync-delay configure), webcam gray block, truncated videos

- AudioSyncDelay.Configure reallocated/zeroed its buffer every ~10ms tick
  (AudioMixer re-reads the UI setting each mix), so any non-zero sync offset
  erased the just-written audio -> total silence. Now early-returns when the
  delay samples are unchanged. Regression test proven both ways.
- SceneCompositor.BlitContentRaw defaulted cbW/cbH=0 when CropBounds is null
  (regression from ed9d7c1) -> webcam blit to an empty rect = gray block.
  Default to src.Width/Height. Regression test proven both ways.
- Truncated recordings: rawvideo mux stamps frames at declared 60fps by
  arrival; a scene whose first layer is dynamic (hidden elements still count)
  kills the bake cache -> full render ~35ms -> ~27fps submitted -> halved
  file length. Static scenes bake once (246ms cold, then <1ms) -> 60fps,
  full-length (probe + 13:53 take, 301/300 per 5s, 12.46s file from 12.3s
  wall). FramePump.ProbeRender names the hot render path on slow frames.
This commit is contained in:
2026-09-12 13:56:55 -07:00
parent a62a283fd4
commit 724af1499b
9 changed files with 310 additions and 134 deletions
+1
View File
@@ -10,3 +10,4 @@ Helpers/OAuthCredentials.cs
MONETIZATION.md MONETIZATION.md
MARCOM.md MARCOM.md
CREDENTIALS.md CREDENTIALS.md
.aider*
+133 -60
View File
@@ -1,76 +1,149 @@
# HANDOFF — 2026-09-12 (transparency: ROOT CAUSE FOUND + FIXED, awaiting the verify take) # HANDOFF — 2026-09-12 (webcam FIXED; audio-silence FIXED; TRUNCATION ROOT-CAUSED + FIXED for static scenes)
## Branch / Commit State ## Branch / Commit State
`main` HEAD lands THIS WORK UNIT: the compositor alpha fix (see ✅ below) + its ONE `main` HEAD = `a62a283`. Ahead of origin by 25 commits. **NOT pushing** — user
integration test + memory updates (`MyMistakes.md`, `ai.md`, `HANDOFF.md`). Ahead of ruling: no push until web-overlay transparency AND audio-silence are addressed.
origin by ~22 commits. **NOT pushing** — user ruling: no push until web-overlay
transparency AND audio-silence are addressed. Nothing uncommitted at end of session.
## ✅ HONEST STATUS — THE TRANSPARENCY BUG IS IDENTIFIED Uncommitted (all belong to the current work, none committed yet):
**Root cause found by reading the unread code path to the end (the handoff's live ```
suspect, now convicted):** M .gitignore
M HANDOFF.md (this file)
M MyMistakes.md (webcam collateral recipe added)
M Services/Audio/AudioSyncDelay.cs (audio-silence fix)
M Services/Compositor/SceneCompositor.cs (webcam cbW/cbH fix)
M Services/MediaCaptureFrameSource.cs (diagnostics REMOVED — clean)
M ytLive.Tests/AudioSyncDelayTests.cs (silence regression test, proven both ways)
M ytLive.Tests/SceneCompositorTests.cs (webcam regression test, proven both ways)
M Services/Encoder/FramePump.cs (slow-render probe, see TRUNCATION below)
```
`SceneCompositor.BlitContentRaw` (SceneCompositor.cs, partial-alpha `else if (sa > 0)` Build is green (0 warnings); 295 tests pass (audio + compositor + pump suites).
branch) applied the OPAQUE-dst source-over blend onto the paste-cache raster's
TRANSPARENT base: `dst = (src*sa + dst*inv)/255` with dst black → color premultiplied by
sa, then `dst[+3] = 255`. A 50%-alpha widget pixel became darkened color + FULL alpha; at
paste time `BlendRowOpaque` saw alpha 255 → straight copy → the scene behind was
overwritten by darkened ink. Transparent margins (alpha 0) and opaque content (alpha 255)
survived, which is why every take showed a box while the dumps (raw capture) and the
preview (raw WriteableBitmap) stayed correct. The CSS-wipe fixes (`b4bba4b`/`0f72c53`)
were red herrings — they treated the PAGE as the villain, but the capture was transparent
from the start (the 15:51 dumps already proved it).
**The fix (committed):** `BlitContentRaw` gained `transparentDst=false`; the raster call ## ✅ TRANSPARENCY — CLOSED (pushed gate #1)
passes `true` and writes straight color + straight alpha so the paste rows
(`BlendRowOpaque`/`BlendRowWeighted`) do the real source-over onto the opaque master.
The two blend rows were verified correct all along (HANDOFF accepted facts held); the
divergence was the sampler feeding them. Master paths are bit-identical (`transparentDst`
defaults false).
**Verification:** clean build 0 warnings (app + tests); `SceneCompositorTests` + User confirmed transparency is fixed (`efe88b8` era). Push gate #1 clears.
`SceneGraphTests` + `FramePumpTests` + `ChatOverlayLayerCacheTests` = 22 pass, the ONE
failure is the documented pre-existing `Composite_FullScene_MasterPixels` pixel (1380,700)
(reproduces with the fix stashed — see ai.md slice 9). The new
`PasteCache_SemiTransparentLayer_RevealsBackdrop_NotOpaqueInk` test FAILS on the old code
(exact signature: `pixel (16,16): expected rgb(127,0,128), got rgb(0,0,128)`) and PASSES
on the fix — non-vacuous, proven both ways.
## NEXT STEP — ONE take (verify, no further analysis) ## ✅ WEBCAM GRAY BLOCK — CLOSED
The Good Dog Rule is satisfied (ONE integration test shipped with the fix). Record the Root cause: `SceneCompositor.BlitContentRaw` defaulted `cbW=cbH=0` when
Live scene with the widget animating, pre-record ~35s so the 30s re-arm dump fires `src.CropBounds` is null — regression from `ed9d7c1`. Every webcam frame (no crop
mid-take, then read the verdict: bounds) blitted to an empty 0×0 source rect → nothing drawn where the webcam
- Element rect shows the scene backdrop behind the widget art (no black box/void, no should be. Fix: default `cbW=src.Width`, `cbH=src.Height`. Regression test
darkened edge ring) → **transparency CLOSED, push gate #1 clears**. `BlitContentRaw_NoCropBounds_SamplesAcrossFullSource_NotJustOrigin` proven BOTH
- If anything persists, the fix's own test contract is the diagnostic: a translucent ways (fails on old code, passes on new). User confirmed correct webcam render in
pixel must read as scene-through-src, never inked — bring the take. the latest recording. All temp diagnostics stripped from
`MediaCaptureFrameSource.cs` and `SceneCompositor.cs`.
## 🟡 AUDIO SILENCE — ROOT-CAUSED AND FIXED, pending final user take
**Symptom:** mic/desktop levels showed real values in preview and in the
per-5s `Audio live:` telemetry, but `peakMix` stayed exactly `0.000` and saved
recordings were digital silence (-91 dB, verified with `ffprobe -af
volumedetect`).
**Root cause:** `AudioMixer.FillAndMix` calls `_syncDelay.Configure(...)` on
EVERY ~10ms tick (it re-reads the live UI setting each mix). The old
`AudioSyncDelay.Configure` unconditionally reallocated and zeroed the delay
buffer AND reset `_writePos=0` on every call — discarding the just-written
audio before the delay offset could ever read it back. With offset 0 the early
passthrough masked it; any non-zero saved offset = total silence. The DB has
`Audio.SyncOffsetMs = 500` (left over from TASK 22 testing) → that's what
triggered it.
**Fix:** `Configure` now early-returns when the delay samples haven't actually
changed (still reallocates/flushes on a *change*, so the live slider still
clicks rather than smearing). Regression test
`RepeatedConfigureWithSameDelay_EveryTick_StillDelivers_TheMarker` mirrors the
mixer's exact calling pattern and was proven BOTH ways (marker lost = silence
on old code; marker survives → expected 960-sample shift on new).
**Telemtery proof the fix works:** latest take (ty-20260912-1135-0000-2.mp4,
recorded WITH the fix, sync offset still 500): `peakMix` = 0.434–0.523 across
all five 5s windows (was 0.000 before). **Audio is NOT silence anymore.**
**Follow-up:** sync offset `Audio.SyncOffsetMs` is still in the DB — currently
**300** (flipped 500→0→500→300 during lip-sync calibration; user judged 0 worse
than 500, so it was bisected back up). NOT the recording's natural state. When the
user is ready to finalize, measure on the classic method (clap visible in frame;
read delta on the waveform — the auto-correlation approach failed to converge at
~10–14Hz webcam shutter jitter).
## ✅ TRUNCATED VIDEO — ROOT-CAUSED AND FIXED (for baked/static scenes)
**Symptom:** recordings ran ~1s short at first, then HALF-length: 22.8s wall →
10.48s file. Not a mux/`-shortest` cut — the pump simply produces frames slower
than the declared 60fps.
**Mechanism (proven):** ffmpeg rawvideo timestamps frames by ARRIVAL at declared
`-r 60`. Pump deadline pacing (OBS libobs pattern) yields each frame when its
~17ms slot lands; if the render stage for that tick takes longer than the slot,
the pump can only do ~1 frame per 35ms → ~27–28fps of *submitted* frames → file
length ≈ submittedFrames/60. So a take that *should* be 22.8s muxes to ~10.5s,
and logs show `avg render 35–38ms` with 136/300 per 5s. FramePump had NO way to
tell which compositor pass ate a slow frame.
**Root cause of the 35ms:** the scene was NOT actually fully static. Any dynamic
element still in the scene — **including hidden ones**, which still count as
Dynamic in `SceneGraph.GetSplitPoint` — forces the per-frame composite path. If
it sits at split 0, `GetBakedBase` returns null and every tick is a FULL render
(no baked base). The 12:59/13:39 takes logged `resolve 1.0ms` per frame — a live
element was still resolving, confirming dynamic content was present.
**The fix that matters (no code changed in the renderer):** a truly static scene
renders in <1ms. New slow-render probe (`ProbeRender` in FramePump.cs) names the
path; the 13:53 take of a 1-element static scene logs
`render=fully-static split=1 dynamic=0 bakeMs=246 totalMs=246` on the FIRST
frame (cold raster of the background at 1080p), then `0.8→0.0ms` avg render,
`301/300` frames per 5s, 0 stalls → **12.46s file from 12.29s wall, 735 frames
@ 60fps. No truncation.**
**Rule for recording:** every scene used for RECORDING should be ≥1 static
element first (background image) with all web/cam/chat widgets AFTER it in
layer order (split ≥ 1 → bake feasible) — a scene whose FIRST layer is dynamic
at recording time will truncate. If a take with widgets comes back truncated
again, the probe now points at whether it took `fully-static` (bake), or
`base+layers` / `full-render` (composite) and how long.
**Probe left in place** (≈0 overhead: one Stopwatch, string built only when a
render ≥20ms): it is the diagnostic that named this bug; keep it until the
dynamic-scene path is proven fast too, then remove.
## 🔴 OPEN — user-reported "video seems truncated ~1s" (unconfirmed)
Recording ty-20260912-1135-0000-2.mp4: format duration 26.370s, video 26.167s
(1570 frames @ 60fps), audio 26.370s. FramePump stats for the whole take show
`dropped 0` and 300/300 frames every 5s window. End-frames (n=1500 vs 1560 vs
1569) differ — not a frozen/repeat-last-frame truncation. Log shows recording
window 11:35:01.386 → 11:35:27.562 = 26.2s of video: matches the file. No
evidence of truncation found in data. Possibly user perceived the 500ms audio
delay (audio trailing video start / leading video end) as truncation — worth
re-checking with offset zeroed. If a real freeze/truncation reappears: pull
last-3-frames diffs and confirm_frame timestamps next.
## Other threads (paused) ## Other threads (paused)
- **Audio silence** — `f3d578c` has per-5s `Audio live:` telemetry; next take with - **Audio-silence verification** — the FIXED take has real peakMix; final =
desktop audio ACTIVE names the stage. Push gate #2. user re-records with the finalized offset and confirms audible audio in playback.
- **Webcam missing** — "MJPG negotiation refused (being used by another process)". Queued. - **Audio sync offset final value** — DB is at 300 (bisect point). Needs the
- **Web capture speed** — ~10-14Hz effective, user satisfied. Revisit only on request. clap/waveform measurement; do NOT rely on the 500 left over from TASK 22.
- **Layer order** — dragging an element over another does not persist `SortOrder`; user - **Truncation with DYNAMIC scenes** — proven mechanism; static scenes now
explicitly asked it not be buried. Queued after transparency. record full-length. If a widget-heavy take truncates again, read the
- **Known backfills when queued work resumes:** `Composite_FullScene_MasterPixels` pixel `ProbeRender` path/stage breakdown in startup.log, then optimize composite.
(1380,700) cyan-vs-magenta (pre-existing, recorded in ai.md slice 9); vertical-tier - **Webcam MJPG missing** — "MJPG negotiation refused (being used by another
`BilinearScale` fresh allocation per frame. process)". Queued.
- **Web capture speed** — ~10-14Hz effective. Revisit only on request.
- **Layer order** — SortOrder not persisting on drag. Queued.
## Landmines ## Landmines
- testhost shares startup.log with the app — filter by time. - testhost shares startup.log — filter by time.
- `taskkill //F //IM ytLive.exe` before rebuilds; re-run if `MSB3021` copy-lock. - `cmd.exe /c "taskkill /F /IM ytLive.exe"` (WSL form double-slashes mangle) before rebuilds.
- Build/tests: `/mnt/c/Program Files/dotnet/dotnet.exe build …` / vstest. 0 warnings rule. - Build/tests: Windows dotnet host (`/mnt/c/Program Files/dotnet/dotnet.exe`). 0 warnings.
FULL-suite vstest can hang (WASAPI teardown, pre-existing) — per-class filters are the - ffmpeg/ffprobe: `/mnt/c/Program Files/Krita (x64)/bin/ffmpeg.exe` with Windows paths.
norm (`--TestCaseFilter:"FullyQualifiedName~…"`). - `Audio.SyncOffsetMs` persisted non-zero WILL re-trigger silence symptoms if
- Probing: `/mnt/c/Program Files/Krita (x64)/bin/ffmpeg.exe` / `ffprobe.exe` — Windows `Configure` is ever called unconditionally again — the idempotent guard is the
exes take Windows-style paths (do NOT re-download Linux ffmpeg — user aborted that). fix, keep it.
- Evidence artifacts (keep): `%TEMP%\ytLive-web-8d7234ec-…-w1..5.png` (15:51), - `MyMistakes.md` has the transparency RESOLVED entry and the new webcam
recordings `ty-20260910-12\51|1534|1551-*.mp4`, decoded frames at collateral recipe — grep before any re-derivation.
`/mnt/c/Users/gramp/AppData/Local/Temp/w151.raw`.
- The full transparency story lives in `MyMistakes.md` → "SPIN GUARD → RESOLVED" (now
including the 2026-09-12 RESOLVED entry). GREP IT FIRST. Do not re-derive a fourth time.
+26
View File
@@ -116,6 +116,32 @@ exactly the bug encoded: 50%-blue over red reads `(0,0,128)` instead of `(127,0,
backdrop never shows through. Next: verify take (element rect shows the backdrop behind backdrop never shows through. Next: verify take (element rect shows the backdrop behind
the widget art), then push gate #1 clears. the widget art), then push gate #1 clears.
**2026-09-12 → COLLATERAL — the SAME transparency saga broke the webcam next.**
Verify-take of the transparency fix showed a perfect flat gray rectangle where the webcam
should be (preview fine, recording flat — stdev <1 across the whole element, despite the
raw camera frame handed to the compositor sampling min=0/max=255 one line earlier in the
pipeline — proven by instrumenting BOTH ends before touching any code, not guessing).
Root cause: `ed9d7c1` ("CropBounds metadata + Fill-style scaling... widget fills element
rect" — take-22 of the SAME transparency saga above) added `cbX/cbY/cbW/cbH` to
`BlitContentRaw`'s general sampler but only assigned them inside the
`src.CropBounds is {} cb` branch; every CropBounds-LESS source (webcam, images — anything
but the web widget) fell through with `cbW=cbH=0`. Since `sxCrop = sxNorm * cbW` and
`syCrop = syNorm * cbH`, both were always 0, so `sxCanvas`/`syCanvas` collapsed to
`cbX`/`cbY` = (0,0) for every destination pixel — an entire scaled webcam sampled ONE
source corner pixel. The web widget (the only CropBounds-bearing source) was never
affected, which is exactly why the transparency fix's own test suite stayed green while
this broke. **FIX:** default `cbW = src.Width`, `cbH = src.Height` (cbX/cbY = 0) before
the branch, only overridden when CropBounds is actually present. ONE test
`BlitContentRaw_NoCropBounds_SamplesAcrossFullSource_NotJustOrigin` (two-color split
source, scaled non-1:1 through the paste cache) fails on the old code — right-half pixel
reads the left half's color — and passes on the fix; proven both ways with a stash/build/
revert cycle, not by inspection alone.
**Lesson:** when a shared low-level sampler gains a new optional code path (CropBounds),
audit EVERY variable the new branch introduces for a safe default in the branch it did
NOT touch — an uninitialized-to-zero "crop region" silently means "sample only pixel
(0,0)", not "no crop." grep for this shape (`var x = 0;` followed by an `if (cond) x = ...`
with no `else`) whenever a conditional metadata field is added to pixel math.
### Shrink / re-encode an image for the README (screenshots → small hero image) ### Shrink / re-encode an image for the README (screenshots → small hero image)
Worked out 2026-08-29 (the recipe was NEVER recorded the first time it was done, so Worked out 2026-08-29 (the recipe was NEVER recorded the first time it was done, so
+12 -3
View File
@@ -29,11 +29,20 @@ public sealed class AudioSyncDelay
Configure(0); Configure(0);
} }
/// <summary>Sets the delay in whole milliseconds (clamped to 0..500 and /// <summary>Sets the delay in whole milliseconds (clamped to 0..500). A no-op
/// flushed).</summary> /// when the delay hasn't actually changed — <see cref="AudioMixer.FillAndMix"/>
/// calls this every ~10ms tick (it re-reads the live UI setting each mix), and
/// unconditionally reallocating+zeroing the line every tick (the 2026-09-12
/// total-silence bug: any non-zero saved offset wiped the ring before its own
/// delayed samples could ever be read back — mic/loop meters, sampled BEFORE
/// this stage, kept showing real levels while the pipe stayed pure silence)
/// discarded the entire buffered history every 10ms. Changing the delay still
/// flushes the line (a live slider change clicks rather than smearing).</summary>
public void Configure(int delayMs) public void Configure(int delayMs)
{ {
_delaySamples = Frames(delayMs); var delaySamples = Frames(delayMs);
if (delaySamples == _delaySamples && _buffer.Length > 0) return;
_delaySamples = delaySamples;
_buffer = new float[(_delaySamples + _sampleRate) * 2 + 1024]; _buffer = new float[(_delaySamples + _sampleRate) * 2 + 1024];
_writePos = 0; _writePos = 0;
} }
+9 -9
View File
@@ -258,13 +258,7 @@ public sealed class SceneCompositor
var ew = (float)element.Width; var ew = (float)element.Width;
var eh = (float)element.Height; var eh = (float)element.Height;
var isRound = element.ClipShape == ClipShape.Round; var isRound = element.ClipShape == ClipShape.Round;
// Webcam diagnostic: Log when webcam frames are processed
if (element is WebcamSceneConfig)
{
AppLog.Write($"Compositor webcam: {frame.Width}x{frame.Height}, IsOpaque={frame.IsOpaque}, element={element.GetType().Name}");
}
// All rect-placed elements route through the paste cache (take-9 lesson): the // All rect-placed elements route through the paste cache (take-9 lesson): the
// IsOpaque bypass kept re-sampling the webcam (~156k px) EVERY tick even when // IsOpaque bypass kept re-sampling the webcam (~156k px) EVERY tick even when
// the device frame had not changed; an opaque source rasterizes to an opaque // the device frame had not changed; an opaque source rasterizes to an opaque
@@ -434,10 +428,16 @@ public sealed class SceneCompositor
float scale, ox, oy, invScale; float scale, ox, oy, invScale;
float drawnW, drawnH; float drawnW, drawnH;
// No CropBounds means "the whole source canvas is the crop region" — cbW/cbH
// MUST default to the full source dims, not 0. A zero cbW/cbH makes every
// sxCrop/syCrop below evaluate to 0 regardless of position, collapsing the
// entire sampled rect to source pixel (0,0) (the webcam gray-block regression,
// 2026-09-12: introduced by the CropBounds Fill-style scaling work the same
// day — CropBounds-bearing sources, i.e. the web widget, were never affected).
var cbX = 0; var cbX = 0;
var cbY = 0; var cbY = 0;
var cbW = 0; var cbW = src.Width;
var cbH = 0; var cbH = src.Height;
if (src.CropBounds is { } cb) if (src.CropBounds is { } cb)
{ {
+48 -9
View File
@@ -87,6 +87,11 @@ public sealed class FramePump : IDisposable
private bool _started; private bool _started;
private long _outputIndex; private long _outputIndex;
/// <summary>Captured by <see cref="ProbeRender"/> when a render exceeds ~20ms —
/// the 2026-09-12 half-speed-render probe. Appended (once) to the next stats or
/// stall line, then cleared.</summary>
private string _renderDetail = "";
/// <summary>Forwards the encoder's parsed health — ship step 6 binds this to the bottom bar.</summary> /// <summary>Forwards the encoder's parsed health — ship step 6 binds this to the bottom bar.</summary>
public event EventHandler<StreamHealth>? HealthUpdated; public event EventHandler<StreamHealth>? HealthUpdated;
@@ -306,8 +311,10 @@ public sealed class FramePump : IDisposable
IFfmpegEncoder? encoder; IFfmpegEncoder? encoder;
lock (_gate) encoder = _encoder; lock (_gate) encoder = _encoder;
var dropped = encoder == null ? 0 : encoder.DroppedFrames; var dropped = encoder == null ? 0 : encoder.DroppedFrames;
var detail = _renderDetail;
_renderDetail = "";
_log?.Invoke(statFrames == 0 _log?.Invoke(statFrames == 0
? "FramePump stats: NO frames produced in 5s (loop stalled?)" ? "FramePump stats: NO frames produced in 5s (loop stalled?)" + (detail.Length > 0 ? " | " + detail : "")
: $"FramePump stats: {statFrames}/{target:F0} frames per 5s, " + : $"FramePump stats: {statFrames}/{target:F0} frames per 5s, " +
$"avg render {renderTicks / (double)System.Diagnostics.Stopwatch.Frequency * 1000 / statFrames:F1}ms " + $"avg render {renderTicks / (double)System.Diagnostics.Stopwatch.Frequency * 1000 / statFrames:F1}ms " +
$"(resolve {resolveTicks / (double)System.Diagnostics.Stopwatch.Frequency * 1000 / statFrames:F1}), " + $"(resolve {resolveTicks / (double)System.Diagnostics.Stopwatch.Frequency * 1000 / statFrames:F1}), " +
@@ -315,7 +322,7 @@ public sealed class FramePump : IDisposable
+ $"avg wait {waitTicks / (double)System.Diagnostics.Stopwatch.Frequency * 1000 / statFrames:F1}ms, " + $"avg wait {waitTicks / (double)System.Diagnostics.Stopwatch.Frequency * 1000 / statFrames:F1}ms, "
+ $"worst render {worstRender / (double)System.Diagnostics.Stopwatch.Frequency * 1000:F1}ms, " + $"worst render {worstRender / (double)System.Diagnostics.Stopwatch.Frequency * 1000:F1}ms, "
+ $"worst submit {worstSubmit / (double)System.Diagnostics.Stopwatch.Frequency * 1000:F1}ms, " + $"worst submit {worstSubmit / (double)System.Diagnostics.Stopwatch.Frequency * 1000:F1}ms, "
+ $"dropped {dropped}, stalls {stalls}"); + $"dropped {dropped}, stalls {stalls}" + (detail.Length > 0 ? " | " + detail : ""));
worstRender = 0; worstRender = 0;
worstSubmit = 0; worstSubmit = 0;
renderTicks = submitTicks = resolveTicks = waitTicks = 0; renderTicks = submitTicks = resolveTicks = waitTicks = 0;
@@ -454,10 +461,13 @@ public sealed class FramePump : IDisposable
if (iterWall > 2 * intervalTicks) if (iterWall > 2 * intervalTicks)
{ {
stalls++; stalls++;
var detail = _renderDetail;
_renderDetail = "";
_log?.Invoke($"FramePump stall: iteration {iterWall / (double)System.Diagnostics.Stopwatch.Frequency * 1000:F0}ms " + _log?.Invoke($"FramePump stall: iteration {iterWall / (double)System.Diagnostics.Stopwatch.Frequency * 1000:F0}ms " +
$"(> 2× the {interval.TotalMilliseconds:F0}ms interval): worst render " + $"(> 2× the {interval.TotalMilliseconds:F0}ms interval): worst render " +
$"{worstRender / (double)System.Diagnostics.Stopwatch.Frequency * 1000:F0}ms, worst submit " + $"{worstRender / (double)System.Diagnostics.Stopwatch.Frequency * 1000:F0}ms, worst submit " +
$"{worstSubmit / (double)System.Diagnostics.Stopwatch.Frequency * 1000:F0}ms, dropped {encoder.DroppedFrames}"); $"{worstSubmit / (double)System.Diagnostics.Stopwatch.Frequency * 1000:F0}ms, dropped {encoder.DroppedFrames}" +
(detail.Length > 0 ? " | " + detail : ""));
} }
ReportStats(); ReportStats();
} }
@@ -505,30 +515,59 @@ public sealed class FramePump : IDisposable
byte[]? scratch = null, byte[]? scratch = null,
Func<SceneElement, VideoFrame?>? resolver = null) Func<SceneElement, VideoFrame?>? resolver = null)
{ {
// per-tick composites use the (timed) resolver; the rare bake uses the raw one
// so bake cost lands in "render" but not "resolve".
resolver ??= _frameResolver; resolver ??= _frameResolver;
if (_sceneGraph == null) if (_sceneGraph == null)
return _compositor.Render(scene, resolver, null, options, socialBarFrame, socialBarTop, scratch: scratch); {
var sw = System.Diagnostics.Stopwatch.StartNew();
var frame = _compositor.Render(scene, resolver, null, options, socialBarFrame, socialBarTop, scratch: scratch);
ProbeRender("full-no-graph", scene, -1, sw.ElapsedMilliseconds, 0);
return frame;
}
var split = _sceneGraph.GetSplitPoint(scene); var split = _sceneGraph.GetSplitPoint(scene);
if (split == scene.Elements.Count) if (split == scene.Elements.Count)
{ {
// Fully static scene: bake once, reuse. // Fully static scene: bake once, reuse.
var sw = System.Diagnostics.Stopwatch.StartNew();
var baked = _sceneGraph.GetBakedBase(scene, _frameResolver, _compositorOptions); var baked = _sceneGraph.GetBakedBase(scene, _frameResolver, _compositorOptions);
if (baked != null) if (baked != null)
return StretchMath.BilinearScale(baked, options.OutputWidth, options.OutputHeight); {
var stretched = StretchMath.BilinearScale(baked, options.OutputWidth, options.OutputHeight);
ProbeRender("fully-static", scene, split, sw.ElapsedMilliseconds, (int)sw.ElapsedMilliseconds);
return stretched;
}
} }
var swBase = System.Diagnostics.Stopwatch.StartNew();
var baseFrame = _sceneGraph.GetBakedBase(scene, _frameResolver, _compositorOptions); var baseFrame = _sceneGraph.GetBakedBase(scene, _frameResolver, _compositorOptions);
if (baseFrame != null) if (baseFrame != null)
{ {
return SceneCompositor.CompositeLayers( var frame = SceneCompositor.CompositeLayers(
baseFrame, scene, split, resolver, options, socialBarFrame, socialBarTop, scratch: scratch); baseFrame, scene, split, resolver, options, socialBarFrame, socialBarTop, scratch: scratch);
ProbeRender("base+layers", scene, split, swBase.ElapsedMilliseconds, 0);
return frame;
} }
// No static base (first layer is dynamic or empty scene) — full render. // No static base (first layer is dynamic or empty scene) — full render.
return _compositor.Render(scene, resolver, null, options, socialBarFrame, socialBarTop, scratch: scratch); var swFull = System.Diagnostics.Stopwatch.StartNew();
var frame2 = _compositor.Render(scene, resolver, null, options, socialBarFrame, socialBarTop, scratch: scratch);
ProbeRender("full-render", scene, split, swFull.ElapsedMilliseconds, 0);
return frame2;
}
/// <summary>2026-09-12 half-speed-render probe: the pump only produces ~27fps
/// (render ~35ms vs the 16.7ms deadline), and rawvideo muxes at the DECLARED fps,
/// so every take muxes at ~half its wall length (the truncation complaint). This
/// names WHICH compositor path ate the slow frame so the fix targets the real
/// stage. Records only when a frame takes ≥20ms (or carries the previous detail).</summary>
private void ProbeRender(string path, Scene scene, int split, long totalMs, int bakeMs)
{
if (totalMs < 20 && _renderDetail.Length == 0) return;
var dynamics = 0;
foreach (var e in scene.Elements)
if (e.Kind == ElementKind.Dynamic) dynamics++;
_renderDetail = $"render={path} split={split} elements={scene.Elements.Count} dynamic={dynamics}" +
(bakeMs > 0 ? $" bakeMs={bakeMs}" : "") + $" totalMs={totalMs}";
} }
// Dot-matrix digits (5×7, one row per raster line, '1' = lit) burned into the // Dot-matrix digits (5×7, one row per raster line, '1' = lit) burned into the
+3 -53
View File
@@ -267,9 +267,6 @@ public sealed class MediaCaptureFrameSource : ICameraFrameSource
private int _ringNext; private int _ringNext;
private long _epoch; private long _epoch;
// Alpha probe diagnostics (one-shot, throttled)
private DateTime _lastAlphaProbe = DateTime.MinValue;
private byte[] RentRingBuffer(int size) private byte[] RentRingBuffer(int size)
{ {
for (var tries = 0; tries < _frameRing.Length; tries++) for (var tries = 0; tries < _frameRing.Length; tries++)
@@ -308,25 +305,11 @@ public sealed class MediaCaptureFrameSource : ICameraFrameSource
var pixels = RentRingBuffer((int)capacity); var pixels = RentRingBuffer((int)capacity);
Marshal.Copy(pixelsPtr, pixels, 0, (int)capacity); Marshal.Copy(pixelsPtr, pixels, 0, (int)capacity);
// Alpha probe diagnostics (throttled to once per 5s)
var now = DateTime.UtcNow;
if ((now - _lastAlphaProbe).TotalSeconds >= 5.0)
{
_lastAlphaProbe = now;
ProbeAlphaStats(pixels, bitmap.PixelWidth, bitmap.PixelHeight);
}
// MediaCapture video carries no alpha plane — the OS fills 255. // MediaCapture video carries no alpha plane — the OS fills 255.
var videoFrame = new VideoFrame(bitmap.PixelWidth, bitmap.PixelHeight, pixels) var outFrame = new VideoFrame(bitmap.PixelWidth, bitmap.PixelHeight, pixels)
{ IsOpaque = true, Epoch = ++_epoch }; { IsOpaque = true, Epoch = ++_epoch };
// Diagnostic: Log frame properties for compositor debugging FrameAvailable?.Invoke(outFrame);
if ((now - _lastAlphaProbe).TotalSeconds >= 5.0)
{
AppLog.Write($"Webcam frame emit: {videoFrame.Width}x{videoFrame.Height}, IsOpaque={videoFrame.IsOpaque}, Epoch={videoFrame.Epoch}");
}
FrameAvailable?.Invoke(videoFrame);
} }
} }
catch (Exception ex) catch (Exception ex)
@@ -339,37 +322,4 @@ public sealed class MediaCaptureFrameSource : ICameraFrameSource
bitmap.Dispose(); bitmap.Dispose();
} }
} }
private void ProbeAlphaStats(byte[] pixels, int width, int height)
{
try
{
// Sample 1000 alpha bytes (every 4th byte in BGRA format)
var sampleCount = Math.Min(1000, pixels.Length / 4);
var alphaMin = 255;
var alphaMax = 0;
var alphaSum = 0;
for (var i = 0; i < sampleCount; i++)
{
var alphaIndex = (i * 4) + 3; // Alpha is 4th byte in BGRA
if (alphaIndex < pixels.Length)
{
var alpha = pixels[alphaIndex];
alphaMin = Math.Min(alphaMin, alpha);
alphaMax = Math.Max(alphaMax, alpha);
alphaSum += alpha;
}
}
var alphaMean = sampleCount > 0 ? alphaSum / (double)sampleCount : 0;
AppLog.Write($"Webcam alpha probe: {width}x{height}, epoch {_epoch}, " +
$"alpha[min={alphaMin}, max={alphaMax}, mean={alphaMean:F1}], " +
$"sampled {sampleCount} pixels");
}
catch (Exception ex)
{
AppLog.Write($"Webcam alpha probe failed: {ex.Message}");
}
}
} }
+38
View File
@@ -47,6 +47,44 @@ public class AudioSyncDelayTests
Assert.Equal(960, MarkerShift(delay, 480 * 2, configureMs: 10)); Assert.Equal(960, MarkerShift(delay, 480 * 2, configureMs: 10));
} }
/// <summary>2026-09-12 total-silence regression: <see cref="AudioMixer.FillAndMix"/>
/// calls <c>Configure(_syncOffsetMs?.Invoke() ?? 0)</c> on EVERY ~10ms tick (it
/// re-reads the live UI setting each mix, not just on change) — the old
/// `Configure` unconditionally reallocated and zeroed the delay line and reset
/// `_writePos` every single call, discarding the buffered marker before the
/// delay offset could ever read it back. Any persisted non-zero
/// `Audio.SyncOffsetMs` (this repo's default is 0, but TASK 22 testing/a user
/// slider left 500 saved) turned the WHOLE recording into digital silence:
/// mic/loop level meters (sampled upstream of this stage) stayed honest while
/// the delay line fed the encoder pipe pure zeros forever. This test
/// reconfigures with the SAME delay before every Process call — exactly the
/// mixer's calling pattern — and the marker must still survive.</summary>
[Fact]
public void RepeatedConfigureWithSameDelay_EveryTick_StillDelivers_TheMarker()
{
const int chunkLen = 960; // 10ms of 48kHz interleaved stereo
var delay = new AudioSyncDelay(48000);
var chunk = new float[chunkLen];
var output = new float[chunkLen];
chunk[0] = 1f; // sole non-zero: left of frame 0
var foundAt = -1;
var total = 0;
for (var c = 0; c < 6 && foundAt < 0; c++)
{
delay.Configure(10); // the mixer calls this every tick, not just on change
delay.Process(chunk, output);
for (var i = 0; i < chunkLen; i++)
{
if (output[i] != 0f) { foundAt = total + i; break; }
}
total += chunkLen;
chunk[0] = 0f; // only the first tick carries the marker
}
Assert.Equal(960, foundAt); // same 10ms-delay shift as the single-Configure test
}
private static int MarkerShift(AudioSyncDelay delay, int chunkLen, int configureMs = -1) private static int MarkerShift(AudioSyncDelay delay, int chunkLen, int configureMs = -1)
{ {
if (configureMs >= 0) if (configureMs >= 0)
+40
View File
@@ -303,6 +303,46 @@ public class SceneCompositorTests
Assert.Equal(build.BgraPixels, hit.BgraPixels); Assert.Equal(build.BgraPixels, hit.BgraPixels);
} }
/// <summary>2026-09-12 regression — the webcam gray-block bug: the CropBounds
/// Fill-scaling work (ed9d7c1, same day as the transparency fix) added
/// cbX/cbY/cbW/cbH but only assigned them inside the CropBounds-present branch;
/// every CropBounds-less source (webcam, images — anything but the web widget)
/// fell through with cbW=cbH=0, so sxCrop/syCrop (= norm * cbW/cbH) were always
/// 0 and EVERY destination pixel sampled source (0,0) — a whole scaled webcam
/// collapsed to one corner pixel's color. All rect-placed elements route through
/// this exact sampler via the paste cache (take-9), so this is not webcam-specific;
/// a two-color source proves distinct destination regions read distinct source
/// regions instead of a single flat pixel.</summary>
[Fact]
public void BlitContentRaw_NoCropBounds_SamplesAcrossFullSource_NotJustOrigin()
{
var source = Split(64, 32, 255, 0, 0, 0, 0, 255); // left=red, right=blue, no CropBounds
var background = new Source { Type = SourceType.DisplayCapture, IsBackground = true, CaptureKey = "monitor:0" };
var webcam = new WebcamSceneConfig { X = 0, Y = 0, Width = 40, Height = 20 }; // scaled (not 1:1)
var scene = new Scene { Name = "Live" };
scene.Elements.Add(background);
scene.Elements.Add(webcam);
VideoFrame? FrameFor(SceneElement e) => e switch
{
Source { IsBackground: true } => Solid(128, 64, 0, 0, 0),
WebcamSceneConfig => source,
_ => null,
};
var options = new CompositorOptions
{
SourceRectX = 0, SourceRectY = 0, SourceRectWidth = 128, SourceRectHeight = 64,
OutputWidth = 128, OutputHeight = 64,
};
var compositor = new SceneCompositor();
var output = compositor.Render(scene, FrameFor, null, options);
AssertColor(output, 5, 10, 255, 0, 0); // near the left edge of the webcam rect: source's red half
AssertColor(output, 35, 10, 0, 0, 255); // near the right edge: source's blue half — fails (reads red) on the bug
}
/// <summary>take-16 regression gate — GC churn convicted mechanically: a ring-style /// <summary>take-16 regression gate — GC churn convicted mechanically: a ring-style
/// producer hands out a small set of rotating arrays with a NEW EPOCH every frame /// producer hands out a small set of rotating arrays with a NEW EPOCH every frame
/// (exactly what the capture/camera rings do). The compositor must re-rasterize INTO /// (exactly what the capture/camera rings do). The compositor must re-rasterize INTO