gramps
f3d578c81b
The recorded audio is full-length silent AAC (−91dB, 1124 frames/23.95s) —
the named pipe carried ~9.1MB of zeros the entire take. The mixer's loop
ran, the pipe connected, ffmpeg read it, and the file is full-length
silence. Sources started fine ('Audio: using system default mic...' logged),
no failure callbacks fired, and the existing integration test
(Mix_WithFiltersDuckAndGain_Lands_On_AudioPipe) proves the loop→pipe path
carries real audio when fed — the fault is capture-side.
Added permanent per-5s live-loop telemetry to startup.log so the next take
names the exact stage without new code:
Audio live: pipe connected= dropped writes= micLevel= loopLevel= drained
N/N samples peakMix N (per 5s)
- AudioMixer.FillAndMix now returns (MicRms, MicDrained, LoopDrained) for
the accumulation; StartLive/StopLive log start/stop lines.
- NamedPipeAudioWriter.DroppedWrites: nonzero = audio dropped before ffmpeg
connected (names 'pipe never connected' stage).
Likely root cause: WASAPI loopback captures the default render endpoint —
if audio plays on a non-default device the recording is silently silent.
The fix requires device enumeration + selection (slice 13 candidate).
Suite 290/291 — same sole pre-existing compositor pixel failure.
90 lines
5.4 KiB
Markdown
90 lines
5.4 KiB
Markdown
# HANDOFF — 2026-09-10 (afternoon)
|
||
|
||
## Branch / Commit State
|
||
|
||
`main` HEAD currently = `5e78065` (slice 11, committed). **Slice 12 (audio diagnostics) is
|
||
uncommitted** in the working tree. **Ahead of origin by 17, NOT pushing** — user ruling (2026-09-10):
|
||
do not push until the web-overlay **transparency AND audio-silence** issues are addressed.
|
||
Milestone tag `milestone-recording-timing` (annotated) on `c45cbc9` — rollback point:
|
||
`git reset --hard milestone-recording-timing`.
|
||
|
||
## Timing saga — CLOSED (verified)
|
||
|
||
Slices 9 (`bd396e4`) + 10 (`c45cbc9`) committed. Takes `0949`/`0957` verified: counter
|
||
+1/frame, zero gaps/dups. User confirmed → **timing CLOSED.**
|
||
|
||
## Slice 11 (web capture cadence) — committed `5e78065`
|
||
|
||
10Hz capture → ~30Hz target with de-throttle flags + `CaptureScheduler` (latest-wins in-flight
|
||
drop). First verification take (`ty-20260910-1038-0000-2.mp4`, 1423 frames 23.7s) results:
|
||
|
||
- **Capture cost: 35-117ms per frame** (not the hoped 10-30ms). Effective cadence is ~10-14Hz,
|
||
NOT 30Hz — the scheduler's in-flight drop silently collapsed the target back to roughly the old
|
||
rate. The in-flight drop works (no stacking), but the decode path (BitmapImage →
|
||
FormatConvertedBitmap → FindContentBounds full-scan) is too expensive to reach 30Hz.
|
||
Web animation speed will not have improved meaningfully. Next: the capture cost decides — either
|
||
optimize the decode path (crop-only, cached bounds) or drop the target to ~15Hz.
|
||
- FramePump: healthy (300/300 frames, 0 dropped, 6 stalls worst 166ms).
|
||
- Camera: initialized (YUY2 640×480) but "MJPG negotiation refused (being used by another process)"
|
||
logged at startup — the webcam source was contested.
|
||
- Audio: **full-length silent AAC** (−91dB, 1124 frames). User reported: no transparency in web-uri,
|
||
no webcam video, no audio. The three symptoms are separate threads.
|
||
|
||
## Slice 12 (audio diagnostics) — uncommitted, in-tree
|
||
|
||
The silence is capture-side: the pipe ran for 23.95s, ffmpeg connected and read ~9.1MB of zeros.
|
||
Sources started (`Audio: using system default mic...` logged at 10:38:01.296), no failure
|
||
callbacks fired. The mixer's existing integration test (`Mix_WithFiltersDuckAndGain_Lands_On_AudioPipe`)
|
||
proves the loop→pipe path carries real audio when fed. Two remaining capture-side suspects:
|
||
(1) default render device mismatch — audio played on a non-default endpoint (common);
|
||
(2) both endpoints held exclusive by another process.
|
||
|
||
**Changes in tree:**
|
||
- `Services/Audio/AudioMixer.cs`: `FillAndMix` now returns `(MicRms, MicDrained, LoopDrained)`;
|
||
LiveLoopAsync accumulates per-5s telemetry → startup.log:
|
||
`Audio live: pipe connected=, dropped writes=, micLevel=, loopLevel=, drained, peakMix`;
|
||
StartLive/StopLive log start/stop lines.
|
||
- `Services/Audio/NamedPipeAudioWriter.cs`: new `DroppedWrites` counter — nonzero means audio
|
||
was dropped before ffmpeg connected (names "pipe never connected" stage).
|
||
- Docs: `ai.md` slice 12 entry.
|
||
|
||
Suite: 290/291 pass — same sole pre-existing compositor pixel failure.
|
||
|
||
## NEXT STEP
|
||
|
||
1. Commit slice 12 (scope-check passed on the three source/doc files).
|
||
2. User runs a recording with DESKTOP AUDIO ACTIVE (music/game playing → verify the "Desktop Audio"
|
||
footer bar moves during the take). Send startup.log lines containing `Audio live:`.
|
||
The log names the stage:
|
||
- `loopLevel > 0` + `peakMix > 0` + file silent → pipe-side (impossible per integration test; should not appear)
|
||
- `loopLevel ≈ 0` + `drained 0/0` → capture delivered nothing (default device mismatch or exclusive hold)
|
||
- `dropped writes > 0` → encoder pipe never connected (timing/ordering bug)
|
||
3. If capture-side: add explicit loopback-device selection (enumerate active render endpoints, log
|
||
the chosen one, allow user to pick) — the fix that makes the mismatch impossible.
|
||
4. Then return to web transparency (post-paint dump) and webcam.
|
||
|
||
## Still Open
|
||
|
||
- **Audio silence** — push gate reason #2. Diagnostic committed pending take (see above).
|
||
- **Web transparency UNVERIFIED** — instrument blind (about:blank). Needs post-paint re-point.
|
||
Push gate reason #1.
|
||
- **Web capture speed** — 30Hz target not achievable with current decode path; capture cost
|
||
35-117ms/frame → effective ~10-14Hz. Open question whether to optimize path or accept ~15Hz.
|
||
- **Webcam missing** in take-1038 — "MJPG negotiation refused (being used by another process)"
|
||
at startup; camera initialized but possibly contested. Separate thread, queued.
|
||
- Pre-existing compositor pixel test failure (never in scope).
|
||
|
||
## Landmines
|
||
|
||
- testhost shares startup.log with app — filter by time when triaging.
|
||
- Locked DLLs: `taskkill //F //IM testhost.exe //IM ytLive.exe` before rebuild.
|
||
- Build/tests via `/mnt/c/Program Files/dotnet/dotnet.exe build …` / `… vstest
|
||
"C:\Users\gramp\Documents\Code\projects\ytLive\ytLive.Tests\bin\Debug\net8.0-windows10.0.19041.0\ytLive.Tests.dll"`.
|
||
- Probing recordings: `/mnt/c/Program Files/Krita (x64)/bin/ffmpeg.exe` / `ffprobe.exe`.
|
||
- `tools/ticker`: constant ~+982ms offset on the FIRST line is cosmetic.
|
||
- WebView2 capture cost: 35-117ms per full-HD PNG encode+decode — the hard ceiling on
|
||
web animation capture rate. Slice 13 candidate if the user wants to pursue 30Hz.
|
||
- WASAPI loopback: captures the DEFAULT render endpoint only — if the user plays audio
|
||
through a non-default device, the recording is silently silent. This is likely the root cause
|
||
of the audio-silence issue. The fix requires device enumeration + selection.
|