fix(rec): bounded encoder queue + drop policy + burned frame counter — the "1...23...4...56..." smeared-ticker take

slice 9 made the DURATION right but content still hiccuped; aggregates (301/300, uniform
file PTS) could not see it. Measured root cause: FfmpegEncoder.SubmitFrameAsync BLOCKED
on WriteAsync(8.3MB)+FlushAsync when ffmpeg lagged the pipe, and the burst while-loop
re-wrote that same stale composite per crossed slot — frozen runs.

OBS shape (derivative, wrapped pre-1.0): the encoder queue in libobs/obs-encoder.c —
encoder thread never couples back into the video thread; overflow = dropped data, never
a frozen producer. https://github.com/obsproject/obs-studio/blob/master/libobs/obs-encoder.c

- FfmpegEncoder: SubmitFrameAsync is now an enqueue (ArrayPool copy) into a bounded
  Channel (cap 120) drained by its own task; drop-newest + count when full;
  StopAsync flushes the queue then EOF (TryComplete). IFfmpegEncoder.DroppedFrames.
- FramePump: ONE fresh composite per iteration (burst loop deleted); worst-submit stat,
  stall logger (>2x interval names the stage), dropped/stalls in stats.
- Burned-in 6-digit dot-matrix frame counter (white box, bottom-right) on every composite
  — the clock-independent judge replacing the WSL ticker: +1/frame, jumps = counted drops.
- ONE new test Backpressure_QueueOverflow_DropsFrames_AndNeverBlocks (slow-sink fake:
  submit never blocks, drops counted, stop flushes exactly submitted-minus-dropped).
- Full suite 290 tests, 289 pass — sole failure the pre-existing compositor pixel test.
- Docs same-commit: ai.md slice 10 (+ encoder/stop-note corrections), MyMistakes point 8,
  HANDOFF.

Audio untouched (queued follow-up); web overlay still frozen pending timing closure.
This commit is contained in:
2026-09-10 09:32:55 -07:00
parent bd396e488c
commit c45cbc93b7
8 changed files with 414 additions and 79 deletions
+45 -6
View File
@@ -505,8 +505,9 @@ encoder step (not yet — this PR ships the seam + impl + tests only).
The encoder is a **thin orchestrator over `ffmpeg.exe`** — no H.264/AAC code in the app. It spawns the
subprocess (path from `IFfmpegLocator`), feeds raw BGRA master frames into stdin, and parses `-stats`
stderr lines into `StreamHealth` (bitrate/FPS/duration, dropped-from-frame-count). `FfmpegEncoder`
(`IFfmpegEncoder` seam) holds: `StartAsync` (locate → probe `-encoders` → spawn → stderr loop),
`SubmitFrameAsync` (serialized stdin writes under `SemaphoreSlim`), `StopAsync` (stdin EOF → ffmpeg
(`IFfmpegEncoder` seam) holds: `StartAsync` (locate → probe `-encoders` → spawn → stderr loop →
drain loop), `SubmitFrameAsync` (slice 10: bounded-queue ENQUEUE — never a pipe write; the drain task
owns stdin writes), `StopAsync` (flush queue → stdin EOF → ffmpeg
finalizes + exits by itself; a 10s watchdog kills it), `Dispose` (force-kill + wait), and the
`HealthUpdated`/`ProcessFailed` events. **Pattern:** the encoder never touches `Process` — it drives the
`IEncoderProcess` seam (`FfmpegEncoderProcess` wraps the real `Process`, redirected stdin/stdout/stderr
@@ -775,7 +776,9 @@ seam:** `Func<Scene?>`, `Func<SceneElement, VideoFrame?>` resolver, `Func<Compos
canon — stackoverflow.com/questions/5441464; timeBeginPeriod — learn.microsoft.com/en-us/windows/win32/api/timeapi/nf-timeapi-timebeginperiod):
`timeBeginPeriod(1)` for the pump's lifetime (paired `timeEndPeriod` in the finally), sleep only the
BULK of the remainder (request minus 2ms), `Thread.SpinWait` the last ~2ms across the deadline;
blown deadlines still rebase. Stats gained `avg wait Xms` so render+submit+wait must ≈ period —
blown deadlines still rebase. (SUPERSEDED in part: slice 9 deleted the rebase — deadlines only
advance; slice 10 replaced the burst re-write with one fresh composite per iteration.) Stats gained
`avg wait Xms` so render+submit+wait must ≈ period —
the accounting is closed, no stage can hide again. Same slice: the webcam's `IsOpaque` paste-cache
bypass was removed (it re-sampled ~156k px every tick even between identical device frames; the
cached paste beats the sampler on hits and costs the same on misses). Take 10 verdict: `≈300/300`
@@ -834,9 +837,45 @@ seam:** `Func<Scene?>`, `Func<SceneElement, VideoFrame?>` resolver, `Func<Compos
the next slot boundary, not immediately. (Unrelated pre-existing failure found the same day:
`Composite_FullScene_MasterPixels` pixel (1380,700) cyan-vs-magenta — reproduces with this fix stashed,
untouched by it, recorded as a follow-up.)
- **Stop ordering matters:** `StopAsync` stops the encoder (closes stdin → EOF → ffmpeg finalizes+exits)
**before** awaiting the loop, because closing stdin unblocks a write stuck on pipe backpressure — the
reverse order would deadlock. `ProcessFailed` self-stops the pump. `Failed` while live flips
- **Slice 10 — bounded encoder queue + drop policy + burned-in frame counter (2026-09-10, the
playback-pacing take):** slice 9 made the DURATION right but the CREATOR still read the recorded
ticker as "1...23...4...56..." — variable pacing. The aggregates (301/300, uniform PTS, 15.6s wall vs
15.74s file) could not see it, and the cause was finally measured in `FfmpegEncoder.SubmitFrameAsync`:
it BLOCKED on `WriteAsync(8.3MB) + FlushAsync` whenever ffmpeg lagged the pipe, and slice 9's
burst `while` loop then re-wrote that SAME composite for every slot that ticked past — frozen
content runs (the "smeared ticker"). Fixed the OBS way (the encoder queue in `libobs/obs-encoder.c`:
the encoder's thread NEVER couples back into the video thread; overload = dropped content, never a
frozen video thread):
(1) **`FfmpegEncoder.SubmitFrameAsync` is now an ENQUEUE** into a bounded `Channel<byte[]>` (cap 120 ≈
2s at 60fps) owned by a dedicated drain task with the stdin write + flush; the caller NEVER blocks on
the pipe. Pixels are copied into an `ArrayPool` buffer before enqueue (the write now happens later on
the drain thread, so the pump's scratch reusable the moment submit returns — same contract as before).
(2) **drop-on-overflow (creator's choice: freshness over coverage):** when the queue is full the NEWEST
frame is dropped and counted (`IFfmpegEncoder.DroppedFrames`, `Interlocked`); the session never freezes
or smears — it drops. Stop flushes the whole queue then EOF (`Channel.TryComplete` → drain writes
leftovers → closes stdin → ffmpeg finalizes+exits), so no accepted frame is ever lost at stop.
(3) **pump loops ONE submit per iteration** (replacing slice 9's burst): every due slot gets a FRESH
composite — no catch-up slot ever repeats frozen content; missed slots vanish as a count-based gap.
The deadline counter is still never reset to wall-now (slice 9 ruling).
(4) **burned-in frame counter (the new judge):** `_outputIndex` is burned into a 6-digit dot-matrix
strip (white box + black 5×7 glyphs, bottom-right corner) of EVERY composite before submit. The WSL
ticker is demoted because its own timers smear under Windows host load; decoding the recording and
reading the strip is clock-independent: the number advances +1 per frame and jumps by exactly the
counted drops (queue overflow OR skipped catch-up slots). Stats gained `worst submit Xms`, `dropped N`,
`stalls K`; a stall logger names any iteration > 2× interval with its render/submit split — with the
queue, submit is ~1ms, so a stall means render/resolve. The strip sits above the social bar (bar is
composited, then burned over) — a debug judge, tiny at 1080p. Test:
`Backpressure_QueueOverflow_DropsFrames_AndNeverBlocks` (the ONE: a 40ms-per-frame sink fake makes
every enqueue fill the queue; submit must return instantly, drops are counted, stop flushes exactly
submitted − dropped bytes). Full suite 289/290 (the pre-existing compositor pixel failure unchanged).
Audio untouched (follow-up). WSL ticker-under-load reliability check still to run (informational — the
burned counter is the judge).
- **Stop ordering matters:** `StopAsync` stops the encoder — since slice 10 it FLUSHES the pending
queue (`Channel.TryComplete` → drain writes the leftovers, closes stdin → EOF → ffmpeg finalizes+exits;
an accepted frame is never lost) — **before** awaiting the loop. The old reverse-order deadlock was
a submit stuck on pipe backpressure; the drain thread owns that wait now, and stopping the encoder
first keeps the pump's own (non-blocking) submits from racing the completed queue as a spurious drop.
`ProcessFailed` self-stops the pump. `Failed` while live flips
`StreamStatus.Error` (minimal).
- **Health stats (ship step 6, shipped 2026-08-13):** `HealthUpdated` is bound to the bottom bar —
`MainViewModel.OnFramePumpHealthUpdated` marshals to the UI thread (the encoder's stderr loop raises on