fix(rec): bounded encoder queue + drop policy + burned frame counter — the "1...23...4...56..." smeared-ticker take
slice 9 made the DURATION right but content still hiccuped; aggregates (301/300, uniform file PTS) could not see it. Measured root cause: FfmpegEncoder.SubmitFrameAsync BLOCKED on WriteAsync(8.3MB)+FlushAsync when ffmpeg lagged the pipe, and the burst while-loop re-wrote that same stale composite per crossed slot — frozen runs. OBS shape (derivative, wrapped pre-1.0): the encoder queue in libobs/obs-encoder.c — encoder thread never couples back into the video thread; overflow = dropped data, never a frozen producer. https://github.com/obsproject/obs-studio/blob/master/libobs/obs-encoder.c - FfmpegEncoder: SubmitFrameAsync is now an enqueue (ArrayPool copy) into a bounded Channel (cap 120) drained by its own task; drop-newest + count when full; StopAsync flushes the queue then EOF (TryComplete). IFfmpegEncoder.DroppedFrames. - FramePump: ONE fresh composite per iteration (burst loop deleted); worst-submit stat, stall logger (>2x interval names the stage), dropped/stalls in stats. - Burned-in 6-digit dot-matrix frame counter (white box, bottom-right) on every composite — the clock-independent judge replacing the WSL ticker: +1/frame, jumps = counted drops. - ONE new test Backpressure_QueueOverflow_DropsFrames_AndNeverBlocks (slow-sink fake: submit never blocks, drops counted, stop flushes exactly submitted-minus-dropped). - Full suite 290 tests, 289 pass — sole failure the pre-existing compositor pixel test. - Docs same-commit: ai.md slice 10 (+ encoder/stop-note corrections), MyMistakes point 8, HANDOFF. Audio untouched (queued follow-up); web overlay still frozen pending timing closure.
This commit is contained in:
@@ -505,8 +505,9 @@ encoder step (not yet — this PR ships the seam + impl + tests only).
|
||||
The encoder is a **thin orchestrator over `ffmpeg.exe`** — no H.264/AAC code in the app. It spawns the
|
||||
subprocess (path from `IFfmpegLocator`), feeds raw BGRA master frames into stdin, and parses `-stats`
|
||||
stderr lines into `StreamHealth` (bitrate/FPS/duration, dropped-from-frame-count). `FfmpegEncoder`
|
||||
(`IFfmpegEncoder` seam) holds: `StartAsync` (locate → probe `-encoders` → spawn → stderr loop),
|
||||
`SubmitFrameAsync` (serialized stdin writes under `SemaphoreSlim`), `StopAsync` (stdin EOF → ffmpeg
|
||||
(`IFfmpegEncoder` seam) holds: `StartAsync` (locate → probe `-encoders` → spawn → stderr loop →
|
||||
drain loop), `SubmitFrameAsync` (slice 10: bounded-queue ENQUEUE — never a pipe write; the drain task
|
||||
owns stdin writes), `StopAsync` (flush queue → stdin EOF → ffmpeg
|
||||
finalizes + exits by itself; a 10s watchdog kills it), `Dispose` (force-kill + wait), and the
|
||||
`HealthUpdated`/`ProcessFailed` events. **Pattern:** the encoder never touches `Process` — it drives the
|
||||
`IEncoderProcess` seam (`FfmpegEncoderProcess` wraps the real `Process`, redirected stdin/stdout/stderr
|
||||
@@ -775,7 +776,9 @@ seam:** `Func<Scene?>`, `Func<SceneElement, VideoFrame?>` resolver, `Func<Compos
|
||||
canon — stackoverflow.com/questions/5441464; timeBeginPeriod — learn.microsoft.com/en-us/windows/win32/api/timeapi/nf-timeapi-timebeginperiod):
|
||||
`timeBeginPeriod(1)` for the pump's lifetime (paired `timeEndPeriod` in the finally), sleep only the
|
||||
BULK of the remainder (request minus 2ms), `Thread.SpinWait` the last ~2ms across the deadline;
|
||||
blown deadlines still rebase. Stats gained `avg wait Xms` so render+submit+wait must ≈ period —
|
||||
blown deadlines still rebase. (SUPERSEDED in part: slice 9 deleted the rebase — deadlines only
|
||||
advance; slice 10 replaced the burst re-write with one fresh composite per iteration.) Stats gained
|
||||
`avg wait Xms` so render+submit+wait must ≈ period —
|
||||
the accounting is closed, no stage can hide again. Same slice: the webcam's `IsOpaque` paste-cache
|
||||
bypass was removed (it re-sampled ~156k px every tick even between identical device frames; the
|
||||
cached paste beats the sampler on hits and costs the same on misses). Take 10 verdict: `≈300/300`
|
||||
@@ -834,9 +837,45 @@ seam:** `Func<Scene?>`, `Func<SceneElement, VideoFrame?>` resolver, `Func<Compos
|
||||
the next slot boundary, not immediately. (Unrelated pre-existing failure found the same day:
|
||||
`Composite_FullScene_MasterPixels` pixel (1380,700) cyan-vs-magenta — reproduces with this fix stashed,
|
||||
untouched by it, recorded as a follow-up.)
|
||||
- **Stop ordering matters:** `StopAsync` stops the encoder (closes stdin → EOF → ffmpeg finalizes+exits)
|
||||
**before** awaiting the loop, because closing stdin unblocks a write stuck on pipe backpressure — the
|
||||
reverse order would deadlock. `ProcessFailed` self-stops the pump. `Failed` while live flips
|
||||
- **Slice 10 — bounded encoder queue + drop policy + burned-in frame counter (2026-09-10, the
|
||||
playback-pacing take):** slice 9 made the DURATION right but the CREATOR still read the recorded
|
||||
ticker as "1...23...4...56..." — variable pacing. The aggregates (301/300, uniform PTS, 15.6s wall vs
|
||||
15.74s file) could not see it, and the cause was finally measured in `FfmpegEncoder.SubmitFrameAsync`:
|
||||
it BLOCKED on `WriteAsync(8.3MB) + FlushAsync` whenever ffmpeg lagged the pipe, and slice 9's
|
||||
burst `while` loop then re-wrote that SAME composite for every slot that ticked past — frozen
|
||||
content runs (the "smeared ticker"). Fixed the OBS way (the encoder queue in `libobs/obs-encoder.c`:
|
||||
the encoder's thread NEVER couples back into the video thread; overload = dropped content, never a
|
||||
frozen video thread):
|
||||
(1) **`FfmpegEncoder.SubmitFrameAsync` is now an ENQUEUE** into a bounded `Channel<byte[]>` (cap 120 ≈
|
||||
2s at 60fps) owned by a dedicated drain task with the stdin write + flush; the caller NEVER blocks on
|
||||
the pipe. Pixels are copied into an `ArrayPool` buffer before enqueue (the write now happens later on
|
||||
the drain thread, so the pump's scratch reusable the moment submit returns — same contract as before).
|
||||
(2) **drop-on-overflow (creator's choice: freshness over coverage):** when the queue is full the NEWEST
|
||||
frame is dropped and counted (`IFfmpegEncoder.DroppedFrames`, `Interlocked`); the session never freezes
|
||||
or smears — it drops. Stop flushes the whole queue then EOF (`Channel.TryComplete` → drain writes
|
||||
leftovers → closes stdin → ffmpeg finalizes+exits), so no accepted frame is ever lost at stop.
|
||||
(3) **pump loops ONE submit per iteration** (replacing slice 9's burst): every due slot gets a FRESH
|
||||
composite — no catch-up slot ever repeats frozen content; missed slots vanish as a count-based gap.
|
||||
The deadline counter is still never reset to wall-now (slice 9 ruling).
|
||||
(4) **burned-in frame counter (the new judge):** `_outputIndex` is burned into a 6-digit dot-matrix
|
||||
strip (white box + black 5×7 glyphs, bottom-right corner) of EVERY composite before submit. The WSL
|
||||
ticker is demoted because its own timers smear under Windows host load; decoding the recording and
|
||||
reading the strip is clock-independent: the number advances +1 per frame and jumps by exactly the
|
||||
counted drops (queue overflow OR skipped catch-up slots). Stats gained `worst submit Xms`, `dropped N`,
|
||||
`stalls K`; a stall logger names any iteration > 2× interval with its render/submit split — with the
|
||||
queue, submit is ~1ms, so a stall means render/resolve. The strip sits above the social bar (bar is
|
||||
composited, then burned over) — a debug judge, tiny at 1080p. Test:
|
||||
`Backpressure_QueueOverflow_DropsFrames_AndNeverBlocks` (the ONE: a 40ms-per-frame sink fake makes
|
||||
every enqueue fill the queue; submit must return instantly, drops are counted, stop flushes exactly
|
||||
submitted − dropped bytes). Full suite 289/290 (the pre-existing compositor pixel failure unchanged).
|
||||
Audio untouched (follow-up). WSL ticker-under-load reliability check still to run (informational — the
|
||||
burned counter is the judge).
|
||||
- **Stop ordering matters:** `StopAsync` stops the encoder — since slice 10 it FLUSHES the pending
|
||||
queue (`Channel.TryComplete` → drain writes the leftovers, closes stdin → EOF → ffmpeg finalizes+exits;
|
||||
an accepted frame is never lost) — **before** awaiting the loop. The old reverse-order deadlock was
|
||||
a submit stuck on pipe backpressure; the drain thread owns that wait now, and stopping the encoder
|
||||
first keeps the pump's own (non-blocking) submits from racing the completed queue as a spurious drop.
|
||||
`ProcessFailed` self-stops the pump. `Failed` while live flips
|
||||
`StreamStatus.Error` (minimal).
|
||||
- **Health stats (ship step 6, shipped 2026-08-13):** `HealthUpdated` is bound to the bottom bar —
|
||||
`MainViewModel.OnFramePumpHealthUpdated` marshals to the UI thread (the encoder's stderr loop raises on
|
||||
|
||||
Reference in New Issue
Block a user