fix(pump+compositor): take-3 starvation — deadline pacing + row-blit fast paths
Two defects made the producer 17x slow (37s record -> 2.1s/127-frame file, rawvideo stamps by arrival): FramePump slept the FULL interval after each render (period = render+submit+interval) and SceneCompositor did per-pixel float sampling + Math.Round blends over all 2.07M master pixels, scanning the whole destination per overlay (258ms avg render vs 1.5ms submit). Both solutions are established, not invented — researched before coding per the derivative-work rule: - deadline pacing: OBS libobs/media-io/video-io.c video_thread (nextTick += intervalTicks, sleep only the remainder, rebase on overrun, never burst) - row blits: libyuv pattern (BSD-3, chromium.googlesource.com/libyuv/libyuv) — 1:1 aligned identity fast path, per-pixel alpha branch, integer fixed-point blend, overlay clipped to the intersection rect, skip the dead black pre-fill when the backdrop covers ONE integration test: Pump_Paces_To_The_Deadline_Compensating_Render_Cost (lands after a fake-seam lesson: pacing fakes must await, not complete synchronously, or the pump loop runs inline on StartAsync and hangs vstest). Clean build 0 warnings; FramePumpTests 10/10, SceneCompositor/SceneGraph/ SocialBar/StretchMath 20/20. Docs same commit: ai.md pipeline section, TASKS.md TASK 18 (webcam-in-output + rename modal verified from take 3), MyMistakes recipe, HANDOFF rewritten. Take 4 pending on the user's machine.
This commit is contained in:
+50
-55
@@ -2,72 +2,67 @@
|
||||
|
||||
## Branch / Commit State
|
||||
|
||||
**`main`**, clean tree, **11 local commits NOT pushed** until the user's "commit and push" lands
|
||||
with this file's commit (expect `main == origin/main` at handoff commit). Tonight's series
|
||||
(newest first):
|
||||
**`main`** — take-3 starvation fix commits land locally (see `git log`); push only when the user says so.
|
||||
The take-3 audit itself happened 2026-09-02/03: user recorded take 3 (`Downloads\recordings\`, manually
|
||||
renamed `llamacasty-recording-3.mp4`), ffprobe + `startup.log` + an extracted frame did the diagnosis.
|
||||
|
||||
- this commit — docs: HANDOFF rewrite
|
||||
- `4509bef` — docs: record-OR-stream ruling, scheduling scope, SYNC provenance, playout declined
|
||||
- `af0d372` — docs: take-two outcomes (arrival-stamping, resolver key, top-bar model, TASK 18 status)
|
||||
- `89fee6c` — fix: webcam identity→device key + always-present Start + [light]LABEL[pill] order
|
||||
- `97ffc42` — chore(pump): render/submit stage-timing stats, 5s windows
|
||||
- `8dcaee0` — fix(encoder): stderr logging + any-exit-is-failure + resolver null logging
|
||||
- `7f2bda8` — fix(audio): game meter × volume (`GameMeterHonestyTests`)
|
||||
- `5ead064` — fix(audio): `_delayedMix` NRE — killed the "known failure", the hang, AND the log flood; LoopbackGain honest
|
||||
- `5a1a3c5` — fix(18): stop ends everything — pills clear, failures roll back (`SessionTeardownTests`)
|
||||
- `688682d` — feat(9): transition(complete) close-out (never existed before)
|
||||
- `22b780e` — fix(ffmpeg): month-end re-pin (daily aged out → 404) + actionable wrap
|
||||
## What the take-3 audit found
|
||||
|
||||
## What tonight was
|
||||
- **37s recording → 2.1s file, 127 frames @60fps** = ~17x time-lapse, audio truncated to match
|
||||
(ffmpeg's rawvideo demuxer was blocked on the starved video input, so only ~2.3s of audio was consumed).
|
||||
The user's "webcam at 6x, garbled" = the same file-level time-compression, most visible on the webcam.
|
||||
- **The stage-timing seam paid for itself in one line**: `17/300 frames per 5s, avg render 258.1ms,
|
||||
avg submit 1.5ms` — the encoder/pipe was healthy; `SceneCompositor.Render` was the whole bottleneck.
|
||||
- **Two defects, both fixed (2026-09-03)** — research-first (OBS `libobs/media-io/video-io.c` deadline
|
||||
pacing + libyuv row-blit pattern; cited in the commit and `MyMistakes.md` recipe):
|
||||
1. `FramePump` slept the FULL interval after each render → period = render+interval. Now an absolute
|
||||
`nextTick += intervalTicks` deadline; on overrun rebase (no catch-up burst).
|
||||
2. Per-pixel float sampling + `Math.Round` blends over all 2.07M master px; `BlitOverlay` scanned the
|
||||
full destination per overlay. Now: 1:1 aligned row-walk fast path (alpha branch, integer fixed-point
|
||||
blend), `BlitOverlay` clipped to the intersection, backdrop-cover skips the black pre-fill.
|
||||
3. **Webcam-in-output VERIFIED** from the take-3 frame (bottom-right, border + social bar render too).
|
||||
- Test: `Pump_Paces_To_The_Deadline_Compensating_Render_Cost` (records the requested wait via the pacing
|
||||
seam). Per-class runs: FramePumpTests 10/10, SceneCompositor+SceneGraph+SocialBar+StretchMath 20/20.
|
||||
Clean build 0 warnings. **Landmine found the hard way:** a pacing fake returning
|
||||
`Task.CompletedTask` synchronously runs the whole pump loop on `StartAsync`'s continuation and HANGS
|
||||
the vstest run — fakes must genuinely await (`Task.Delay(d, ct)`) or yield.
|
||||
- Minor logged on take-3 stop: `FramePump: encoder stop failed: No process is associated` — cosmetic
|
||||
teardown race (process already exited before StopAsync's kill); follow-up only if it grows teeth.
|
||||
|
||||
Creator's first native runs since the refactor: startup crash → fixed (3 stacked faults); PM audit →
|
||||
v1-complete declaration + out-of-product list (TASKS.md file end); first-ever real recordings → the
|
||||
recording pipeline now demonstrably produces H.264+AAC MP4s (`Downloads\recordings\`, ffmpeg cached
|
||||
in `%APPDATA%\ytLlive\tools\`).
|
||||
## OPEN — do next, in order
|
||||
|
||||
**Test suite: ZERO known failures** (audio class was the last — root-caused to an uninitialized
|
||||
field; TASK 22's fault is recorded in its entry). Per-class Windows-host vstest runs everything
|
||||
including RealApp; only the FULL suite hangs (WASAPI teardown — don't run it).
|
||||
|
||||
## OPEN — do tomorrow, in order
|
||||
|
||||
1. **Take 3 (user records ~30s, record-only):** read `%APPDATA%\ytLlive\startup.log` — the new
|
||||
`FramePump stats: n/target frames per 5s, avg render Xms submit Yms` lines will name the stage
|
||||
behind the ~2fps producer starvation (symptoms: short file + 30x time-lapse + garbled audio,
|
||||
one cause — rawvideo stamps by arrival). Also check: webcam now IN the output (89fee6c),
|
||||
meter follows the volume knob, Stop slides pills off + Start present, top bar grouping feels right.
|
||||
Also: take-2's file stayed auto-named `ty-…-0000.mp4` — did the rename modal appear?
|
||||
2. **Then code, one integration test per change:** starvation fix (from #1's data) → radio pills /
|
||||
record-OR-stream enforcement (ruling captured in TASKS.md TASK 18; `BeginGoLive(alsoRecord)` dies)
|
||||
→ top bar Option A **pending user approval** (his "still not correct" message predates tonight's
|
||||
89fee6c — have him LOOK first) → empty-state ruling pending: disabled / rehearsal (my recommendation) /
|
||||
default-record → SYNC slider placement ruling pending (feature is his, permanently; position is open).
|
||||
3. **Un-asked questions** (don't nag, just have the answers ready): why-stupid confirmation on
|
||||
simultaneous rec+stream was given (VOD copy + hardware drag) ✓; "other issues" from take 1/2 —
|
||||
user mentioned them but never listed; take 2 preview questions.
|
||||
1. **Take 4 (user records ~30s, record-only)** → ffprobe the file (expect frames ≈ 60×seconds, duration
|
||||
≈ wall time) + read `startup.log` `FramePump stats` (expect `n/300`, `avg render` ≤ ~16ms). If n/300
|
||||
climbs only to ~45-55: next slice is the 8.3MB/frame LOH allocation → pool the master buffer.
|
||||
2. **Then code, one integration test per change** (queue from before, unchanged): radio pills /
|
||||
record-OR-stream enforcement (TASK 18 ruling; `BeginGoLive(alsoRecord)` dies) → top bar Option A
|
||||
**pending user approval** → empty-state ruling pending (disabled / rehearsal / default-record) →
|
||||
SYNC slider placement ruling pending.
|
||||
3. **Un-asked questions** (don't nag; answers ready): the recorder said the desktop capture "didn't last
|
||||
long enough to tell quality" — take 4 fixes that; "other issues" from takes 1/2 never listed.
|
||||
|
||||
## State of the app
|
||||
|
||||
Boots clean, records clean-ish (starvation pending), preview honest, top bar reordered per spec,
|
||||
webcam-key fixed but NEVER verified in a real recording until take 3. The user's layout DB is
|
||||
intact (real C920 row confirmed by direct sqlite read — the 'test-camera' log lines are testhost
|
||||
noise, both processes share startup.log).
|
||||
Boots clean. Recording pipeline: render starvation fixed pending take-4 confirmation; webcam + social bar
|
||||
confirmed IN the output. ffmpeg cached at `%APPDATA%\ytLlive\tools\` (month-end pin). Test suite per-class
|
||||
green; do NOT run full-suite vstest (WASAPI teardown hang, pre-existing).
|
||||
|
||||
## Landmines
|
||||
|
||||
- testhost shares startup.log with the app — filter by time when triaging.
|
||||
- Stale testhost/exe locks the DLL (MSB3027): `taskkill /F /IM ytLive.exe` / `testhost.exe` first.
|
||||
- Do NOT run full-suite vstest (hangs); do NOT claim suite totals — per-class only.
|
||||
- `AudioPipelineTests` is healthy now but was the "known failure" graveyard — any new failure
|
||||
there means a live-loop regression; read the logged stack (catches now log WITH stack + 5s throttle).
|
||||
- Real-`MainWindow` tests MUST use `LayoutPathOverride` + temp DB; `VolumePushOverride` seam exists
|
||||
so volume-slider tests never touch the machine's speakers.
|
||||
- verify.sh's full-suite step hangs from WSL — flow tonight: clean build (0 warnings) + per-class
|
||||
vstest + scope-check, all through the Windows dotnet.exe host.
|
||||
- ffmpeg pin: month-end rule recorded (TASKS.md); BtbN keeps dailies ~14 days.
|
||||
- Stale testhost/exe locks the DLL (MSB3027): `taskkill /F /IM testhost.exe` / `ytLive.exe` first.
|
||||
- Do NOT run full-suite vstest (hangs); do NOT claim suite totals — per-class only. verify.sh's step 2 IS
|
||||
the full-suite hang — flow: clean build + per-class vstest + scope-check.
|
||||
- Windows binaries (ffprobe/ffmpeg under `%APPDATA%\ytLlive\tools\`) need WINDOWS paths
|
||||
(`C:\Users\...`), never `/mnt/c/...` — the mount path reads as "No such file or directory".
|
||||
- Real-`MainWindow` tests MUST use `LayoutPathOverride` + temp DB; `VolumePushOverride` seam exists so
|
||||
volume-slider tests never touch the machine's speakers.
|
||||
- `AudioPipelineTests` was the "known failure" graveyard — a new failure there means a live-loop
|
||||
regression; read the logged stack (throttled, with stack).
|
||||
- Pacing-seam fakes must yield/await — synchronous completion runs the pump loop inline (see above).
|
||||
|
||||
## @ User note
|
||||
|
||||
No unsolicited roadmap/next-step lists — work the queue above, report what changed, keep responses
|
||||
SHORT (his words, twice tonight: walls of text are not getting read). Good dog: one integration test
|
||||
per change. Committing is expected; pushing on his word — tonight he said push with the handoff.
|
||||
He does not want tangents acknowledged twice (rename question = noise; the ask was ALWAYS the recording).
|
||||
Keep responses SHORT; work the queue; one integration test per change; committing is expected, pushing is
|
||||
his word.
|
||||
|
||||
@@ -47,6 +47,36 @@ packages, high-quality downscale via `TransformedBitmap`. Screenshots compress
|
||||
|
||||
---
|
||||
|
||||
### Feeding a rawvideo pipe at 60fps: deadline pacing + row-blit budget
|
||||
|
||||
Derived 2026-09-03 (take-3 diagnosis — the stats seam from `97ffc42` named the stage
|
||||
in one line: `17/300 frames per 5s, avg render 258.1ms, avg submit 1.5ms`).
|
||||
Both halves were solved by OBS/libyuv long ago; do not re-derive:
|
||||
|
||||
1. **Pacing is a DEADLINE, never a post-render sleep.** `sleep(interval)` after each
|
||||
frame makes the period `render + submit + interval` — the producer can hit ≤ half
|
||||
the declared rate even with a free render. OBS's `video_thread`
|
||||
(`libobs/media-io/video-io.c`) advances an absolute `nextTick += intervalTicks` and
|
||||
sleeps only the remainder; if the deadline blew, skip the wait AND the missed ticks
|
||||
(rebase, no catch-up burst — a burst queues stale frames). Critical with rawvideo:
|
||||
pts is stamped by ARRIVAL, so a starved producer silently time-lapses the file.
|
||||
2. **A 1080p frame is ~2.07M pixels — the hot path must be row-simple.** Per-pixel
|
||||
`Math.Round` + float source-over in managed code costs ~100ns/px = the whole 258ms.
|
||||
libyuv's pattern (https://chromium.googlesource.com/libyuv/libyuv/): branch per
|
||||
pixel on source alpha (opaque → 4-byte copy, transparent → skip), integer
|
||||
fixed-point blend `(s*a + d*(255-a) + 127)/255` otherwise; and ALWAYS clip the loop
|
||||
to the intersection rect (our social-bar overlay scanned all 2M dst px for a 64px
|
||||
strip). A full-cover 1:1 blit also obsoletes the opaque-black pre-fill — skip dead
|
||||
writes.
|
||||
3. **Hermetic pacing test:** inject the delay seam to RECORD the requested TimeSpan and
|
||||
genuinely await it (`Task.Delay(d, ct)`) — a fake that returns
|
||||
`Task.CompletedTask` synchronously makes the whole pump loop run on `StartAsync`'s
|
||||
sync continuation and hang the test run (hit this 2026-09-03; the existing fakes all
|
||||
yield for exactly this reason). Assert the REQUESTED wait (< interval with a
|
||||
≥cost-ms fake render) — never wall-clock rate, which flakes on loaded machines.
|
||||
|
||||
---
|
||||
|
||||
## Splitting a large file into partials — NEVER `awk … > SRC` while awking SRC
|
||||
|
||||
(2026-08-31, Commit D) Tried to split `SocialsDialogViewModel.cs` in one line:
|
||||
|
||||
@@ -54,12 +54,21 @@ public sealed class SceneCompositor
|
||||
else
|
||||
{
|
||||
if (split < 0) split = scene.Elements.Count;
|
||||
for (var i = 3; i < buffer.Length; i += 4)
|
||||
buffer[i] = 255; // opaque black base
|
||||
|
||||
var elements = scene.Elements;
|
||||
var liveCapture = elements.OfType<Source>().FirstOrDefault(s => s.IsBackground);
|
||||
var liveCaptureFrame = liveCapture != null && liveCapture.IsVisible ? frameFor(liveCapture) : null;
|
||||
|
||||
// A master-sized live backdrop paints every pixel of the base — the
|
||||
// opaque-black pre-fill (2M byte writes) would be dead work. Capture
|
||||
// frames are opaque BGRA by contract (the WinRT path ignores alpha),
|
||||
// so the 1:1 row walk covers the full rect.
|
||||
var covered = liveCaptureFrame != null
|
||||
&& liveCaptureFrame.Width == cropW && liveCaptureFrame.Height == cropH;
|
||||
if (!covered)
|
||||
for (var i = 3; i < buffer.Length; i += 4)
|
||||
buffer[i] = 255; // opaque black base
|
||||
|
||||
if (liveCaptureFrame != null)
|
||||
BlitContent(buffer, cropW, cropH, 0, 0, cropW, cropH, liveCaptureFrame, 1f, false, false);
|
||||
|
||||
@@ -281,19 +290,33 @@ public sealed class SceneCompositor
|
||||
{
|
||||
if (ew <= 0 || eh <= 0) return;
|
||||
|
||||
var x0 = Math.Max(0, (int)Math.Floor(ex));
|
||||
var y0 = Math.Max(0, (int)Math.Floor(ey));
|
||||
var x1 = Math.Min(dstW - 1, (int)Math.Ceiling(ex + ew));
|
||||
var y1 = Math.Min(dstH - 1, (int)Math.Ceiling(ey + eh));
|
||||
if (x0 > x1 || y0 > y1) return;
|
||||
|
||||
var (scale, ox, oy) = StretchMath.UniformToFill(ew, eh, src.Width, src.Height);
|
||||
|
||||
// 1:1 aligned fast path (the full-frame backdrop, master-sized overlays):
|
||||
// bilinear sampling at scale 1 with integer offsets is the identity, so the
|
||||
// blit is a row walk, not 2M float samples + rounds. Research-first (2026-09-03,
|
||||
// take-3 fix; libyuv's row-based BLEND_NONE pattern —
|
||||
// https://chromium.googlesource.com/libyuv/libyuv/): the 258ms/frame render
|
||||
// that time-lapsed take 3 lived in the general loop below.
|
||||
if (!isRound && !isMirror && scale == 1f && ox == 0f && oy == 0f
|
||||
&& ex == MathF.Floor(ex) && ey == MathF.Floor(ey))
|
||||
{
|
||||
Blit1To1Rows(dst, dstW, dstH, (int)ex, (int)ey, src, opacity);
|
||||
return;
|
||||
}
|
||||
|
||||
var drawnW = src.Width * scale;
|
||||
var drawnH = src.Height * scale;
|
||||
var radius = Math.Min(ew, eh) / 2f;
|
||||
var cx = ew / 2f;
|
||||
var cy = eh / 2f;
|
||||
|
||||
var x0 = Math.Max(0, (int)Math.Floor(ex));
|
||||
var y0 = Math.Max(0, (int)Math.Floor(ey));
|
||||
var x1 = Math.Min(dstW - 1, (int)Math.Ceiling(ex + ew));
|
||||
var y1 = Math.Min(dstH - 1, (int)Math.Ceiling(ey + eh));
|
||||
if (x0 > x1 || y0 > y1) return;
|
||||
|
||||
for (var y = y0; y <= y1; y++)
|
||||
{
|
||||
for (var x = x0; x <= x1; x++)
|
||||
@@ -317,6 +340,83 @@ public sealed class SceneCompositor
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>Row walk for a 1:1 aligned blit at integer offset (ex, ey): per-pixel
|
||||
/// branch on source alpha — opaque rows copy straight, transparent pixels are
|
||||
/// skipped, partial alpha blends in integer fixed-point (libyuv's pattern). The
|
||||
/// master buffer's destination alpha is always opaque (black base paints 255 and
|
||||
/// every blend writes 255), so dst alpha is never read — matching BlendPixel's
|
||||
/// source-over result for opaque dst within ±1.</summary>
|
||||
private static void Blit1To1Rows(byte[] dst, int dstW, int dstH, int ex, int ey, VideoFrame src, float opacity)
|
||||
{
|
||||
var op = (int)(Math.Clamp(opacity, 0f, 1f) * 256);
|
||||
if (op <= 0) return;
|
||||
|
||||
var x0 = Math.Max(0, ex);
|
||||
var y0 = Math.Max(0, ey);
|
||||
var x1 = Math.Min(dstW, ex + src.Width);
|
||||
var y1 = Math.Min(dstH, ey + src.Height);
|
||||
if (x0 >= x1 || y0 >= y1) return;
|
||||
|
||||
var rowLen = x1 - x0;
|
||||
for (var y = y0; y < y1; y++)
|
||||
{
|
||||
var si = ((y - ey) * src.Width + (x0 - ex)) * 4;
|
||||
var di = (y * dstW + x0) * 4;
|
||||
if (op == 256) BlendRowOpaque(dst, di, src.BgraPixels, si, rowLen);
|
||||
else BlendRowWeighted(dst, di, src.BgraPixels, si, rowLen, op);
|
||||
}
|
||||
}
|
||||
|
||||
private static void BlendRowOpaque(byte[] dst, int di, byte[] src, int si, int count)
|
||||
{
|
||||
for (var n = 0; n < count; n++)
|
||||
{
|
||||
var a = src[si + 3];
|
||||
if (a == 255)
|
||||
{
|
||||
dst[di] = src[si];
|
||||
dst[di + 1] = src[si + 1];
|
||||
dst[di + 2] = src[si + 2];
|
||||
dst[di + 3] = 255;
|
||||
}
|
||||
else if (a != 0)
|
||||
{
|
||||
var inv = 255 - a;
|
||||
dst[di] = (byte)((src[si] * a + dst[di] * inv + 127) / 255);
|
||||
dst[di + 1] = (byte)((src[si + 1] * a + dst[di + 1] * inv + 127) / 255);
|
||||
dst[di + 2] = (byte)((src[si + 2] * a + dst[di + 2] * inv + 127) / 255);
|
||||
dst[di + 3] = 255;
|
||||
}
|
||||
si += 4;
|
||||
di += 4;
|
||||
}
|
||||
}
|
||||
|
||||
private static void BlendRowWeighted(byte[] dst, int di, byte[] src, int si, int count, int op)
|
||||
{
|
||||
for (var n = 0; n < count; n++)
|
||||
{
|
||||
var a = src[si + 3] * op >> 8;
|
||||
if (a >= 255)
|
||||
{
|
||||
dst[di] = src[si];
|
||||
dst[di + 1] = src[si + 1];
|
||||
dst[di + 2] = src[si + 2];
|
||||
dst[di + 3] = 255;
|
||||
}
|
||||
else if (a > 0)
|
||||
{
|
||||
var inv = 255 - a;
|
||||
dst[di] = (byte)((src[si] * a + dst[di] * inv + 127) / 255);
|
||||
dst[di + 1] = (byte)((src[si + 1] * a + dst[di + 1] * inv + 127) / 255);
|
||||
dst[di + 2] = (byte)((src[si + 2] * a + dst[di + 2] * inv + 127) / 255);
|
||||
dst[di + 3] = 255;
|
||||
}
|
||||
si += 4;
|
||||
di += 4;
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>Centered OBS-style border stroke: rect ring (Traditional) or circle ring (Round).</summary>
|
||||
private static void DrawBorder(
|
||||
byte[] dst, int dstW, int dstH,
|
||||
@@ -351,21 +451,22 @@ public sealed class SceneCompositor
|
||||
}
|
||||
|
||||
/// <summary>1:1 copy of a master-sized overlay (flash, social bar) cropped to the
|
||||
/// active source rect. The overlay is positioned in master space by (sx0, sy0).</summary>
|
||||
/// active source rect. The overlay is positioned in master space by (sx0, sy0).
|
||||
/// Iterates only the intersection rect — the old loop walked every destination
|
||||
/// pixel to blit a ~64px strip (2M iterations of bounds checks per overlay).</summary>
|
||||
private static void BlitOverlay(
|
||||
byte[] dst, int dstW, int dstH, CompositorOptions options,
|
||||
VideoFrame overlay, int sx0, int sy0)
|
||||
{
|
||||
for (var y = 0; y < dstH; y++)
|
||||
var dx0 = Math.Max(0, sx0 - options.SourceRectX);
|
||||
var dy0 = Math.Max(0, sy0 - options.SourceRectY);
|
||||
var dx1 = Math.Min(dstW, sx0 - options.SourceRectX + overlay.Width);
|
||||
var dy1 = Math.Min(dstH, sy0 - options.SourceRectY + overlay.Height);
|
||||
for (var y = dy0; y < dy1; y++)
|
||||
{
|
||||
for (var x = 0; x < dstW; x++)
|
||||
{
|
||||
var sx = x + options.SourceRectX - sx0;
|
||||
var sy = y + options.SourceRectY - sy0;
|
||||
if (sx < 0 || sy < 0 || sx >= overlay.Width || sy >= overlay.Height) continue;
|
||||
var sample = StretchMath.SampleBgra(overlay.BgraPixels, overlay.Width, overlay.Height, sx, sy);
|
||||
BlendPixel(dst, (y * dstW + x) * 4, sample, 1f);
|
||||
}
|
||||
var si = ((y + options.SourceRectY - sy0) * overlay.Width
|
||||
+ dx0 + options.SourceRectX - sx0) * 4;
|
||||
BlendRowOpaque(dst, (y * dstW + dx0) * 4, overlay.BgraPixels, si, dx1 - dx0);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -218,6 +218,16 @@ public sealed class FramePump : IDisposable
|
||||
var interval = TimeSpan.FromSeconds(1d / Math.Max(1, options.Fps));
|
||||
var lastTick = System.Diagnostics.Stopwatch.StartNew();
|
||||
|
||||
// Deadline pacing (2026-09-03, take-3 fix): the frame interval is a DEADLINE,
|
||||
// not an afterthought sleep — the OBS libobs video-io.c pattern (researched
|
||||
// before coding; see https://github.com/obsproject/obs-studio/blob/master/
|
||||
// libobs/media-io/video-io.c). The old loop slept the FULL interval after
|
||||
// each render, so period = render + submit + interval: at take-3's 258ms
|
||||
// render that was 3.6fps stamped into a 60fps container — rawvideo stamps by
|
||||
// arrival, so 30 wall-seconds muxed as a 2.1s time-lapse, no error anywhere.
|
||||
var intervalTicks = Math.Max(1, (long)Math.Round(interval.TotalSeconds * System.Diagnostics.Stopwatch.Frequency));
|
||||
var nextTick = System.Diagnostics.Stopwatch.GetTimestamp();
|
||||
|
||||
// Stage timing (2026-09-01, take two): rawvideo carries no per-frame
|
||||
// timestamps — ffmpeg stamps frames by ARRIVAL at the declared fps. A producer
|
||||
// slower than the declared rate yields a time-lapsed, short file (observed:
|
||||
@@ -292,8 +302,22 @@ public sealed class FramePump : IDisposable
|
||||
submitTicks += submitSw.ElapsedTicks;
|
||||
statFrames++;
|
||||
ReportStats();
|
||||
|
||||
// Advance the deadline; cost already spent is not slept again.
|
||||
// Blew the frame budget: skip the wait AND the missed ticks —
|
||||
// rebase the clock rather than bursting a catch-up pile
|
||||
// (OBS rewinds its tick the same way; a burst would only
|
||||
// queue stale frames into the encoder).
|
||||
nextTick += intervalTicks;
|
||||
var lag = nextTick - System.Diagnostics.Stopwatch.GetTimestamp();
|
||||
if (lag <= 0)
|
||||
{
|
||||
nextTick = System.Diagnostics.Stopwatch.GetTimestamp() + intervalTicks;
|
||||
lag = 0;
|
||||
}
|
||||
await _pacingDelay(
|
||||
TimeSpan.FromSeconds(lag / (double)System.Diagnostics.Stopwatch.Frequency), ct);
|
||||
}
|
||||
await _pacingDelay(interval, ct);
|
||||
}
|
||||
}
|
||||
catch (OperationCanceledException)
|
||||
|
||||
@@ -973,7 +973,7 @@ click (volume sliders keep their manual `SetSliderValueFromClick`, harmless dupl
|
||||
|
||||
**Goal:** record the stream output to a local file, with or without simultaneously streaming.
|
||||
|
||||
### Status: ✅ Shipped `a9eb360` (2026-08-29) — code done (incl. manual-rename modal), build 0 warnings, 244/246 tests; **running-app verification IN PROGRESS (2026-09-01 takes 1–2)**: files now actually land (dead ffmpeg pin re-pinned month-end, locator wrap), webcam-in-output + top-bar reachability + audio-loop NREs fixed en route; take 3 pending with pump stage-timing to resolve the ~2fps producer starvation (time-lapse/short-file/garbled-audio = one root cause, `FramePump stats` will name the stage)
|
||||
### Status: ✅ Shipped `a9eb360` (2026-08-29) — code done (incl. manual-rename modal), build 0 warnings, 244/246 tests; **running-app verification (takes 1–3 done)**: files land (ffmpeg re-pinned month-end), **webcam-in-output + social bar visually CONFIRMED from take 3's extracted frame**, rename modal used for real (take 3 was named via it); take 3 exposed the ~2fps producer starvation and the pump stage-timing (`97ffc42`) named it in one line — `avg render 258.1ms` — **fixed 2026-09-03** (deadline pacing per OBS video-io.c + libyuv-style row blits in `SceneCompositor`, test `Pump_Paces_To_The_Deadline_Compensating_Render_Cost`); **take 4 pending** to verify 60fps lands (stats must show `n/300` and render ≤ ~16ms; next suspect if hiccuppy: 8.3MB/frame LOH alloc → buffer pool)
|
||||
|
||||
1. ✅ `EncoderOptions` extended with `StreamEnabled` / `RecordEnabled` / `RecordPath` (independent intent flags)
|
||||
2. ✅ `FfmpegArgs.Build` reworked into per-output blocks (stream `-f flv`, record `-f mp4`) via `AddVideoTags`
|
||||
|
||||
@@ -692,6 +692,24 @@ seam:** `Func<Scene?>`, `Func<SceneElement, VideoFrame?>` resolver, `Func<Compos
|
||||
loaded from the `LayoutStore` cache at startup and set fresh by `PrepareAndStartLiveAsync` before the pump
|
||||
starts. **The pump reads the URL once at startup**, which is why go-live ensures the stream BEFORE
|
||||
`StartAsync`.
|
||||
- **Deadline pacing + row-blit render (take-3 starvation fix, 2026-09-03):** take 3 recorded 30s into a
|
||||
2.1s file and the 5s stats line named the cause in one number — `17/300 frames per 5s, avg render
|
||||
258.1ms, avg submit 1.5ms`. Two defects, both fixed: (1) the pump slept the FULL interval after each
|
||||
render, so period = render + interval — OBS's `video_thread` (libobs/media-io/video-io.c) pattern
|
||||
replaces it: absolute `nextTick += intervalTicks` deadline, sleep only the remainder, and on overrun
|
||||
skip the wait AND the missed ticks (rebase, never burst stale frames). (2) the compositor did
|
||||
per-pixel float sampling + `Math.Round` blending over all 2.07M master pixels (backdrop) and scanned
|
||||
the whole destination per overlay (a 64px social-bar strip cost 2M iterations). `SceneCompositor` now
|
||||
follows the libyuv pattern (BSD-3, chromium.googlesource.com/libyuv/libyuv — cited per the
|
||||
derivative-work rule): 1:1 aligned blits take a row-walk fast path (bilinear at scale 1 + integer
|
||||
offset is the identity) with per-pixel alpha branch and integer fixed-point blend; `BlitOverlay`
|
||||
clips to the intersection rect; a full-cover live backdrop skips the opaque-black pre-fill. The
|
||||
general per-pixel path stays for scaled/round/mirrored elements (webcam) — pixel-identical, and it is
|
||||
only ~130k px. Test: `Pump_Paces_To_The_Deadline_Compensating_Render_Cost` records the requested wait
|
||||
via the pacing seam (a seam fake must genuinely await — a synchronous completed task runs the whole
|
||||
pump loop on `StartAsync`'s continuation and hangs the run; MyMistakes). **Next slice if take 4's
|
||||
stats show it:** 8.3MB fresh `byte[]` per frame → LOH churn/GC stalls — pool the master buffer if avg
|
||||
render lands ≤ budget but frames still hiccup.
|
||||
- **Stop ordering matters:** `StopAsync` stops the encoder (closes stdin → EOF → ffmpeg finalizes+exits)
|
||||
**before** awaiting the loop, because closing stdin unblocks a write stuck on pipe backpressure — the
|
||||
reverse order would deadlock. `ProcessFailed` self-stops the pump. `Failed` while live flips
|
||||
|
||||
@@ -324,4 +324,67 @@ public class FramePumpTests
|
||||
Assert.Equal(StreamStatus.Streaming, h.Status);
|
||||
Assert.Equal(59.9, h.FPS, 1);
|
||||
}
|
||||
|
||||
/// <summary>The ONE integration test for the take-3 starvation fix (2026-09-03):
|
||||
/// the frame interval is a DEADLINE — render cost eats into it, never piles on
|
||||
/// top (the OBS libobs video-io.c pacing pattern). The old loop slept the full
|
||||
/// interval AFTER each render, so 30 wall-seconds of 258ms renders muxed into a
|
||||
/// 2.1s 60fps time-lapse (rawvideo stamps by arrival). Here the fake render
|
||||
/// costs ≥60ms of a 200ms interval and the pacing seam records every requested
|
||||
/// wait: no request may reach the full interval (that was the bug), none may be
|
||||
/// negative, and a machine slow enough to blow every deadline is still honest
|
||||
/// (zero waits < interval passes too).</summary>
|
||||
[Fact]
|
||||
public async Task Pump_Paces_To_The_Deadline_Compensating_Render_Cost()
|
||||
{
|
||||
const int fps = 5; // 200ms interval — generous margin over the fake cost
|
||||
const int renderCostMs = 60; // Thread.Sleep is a guaranteed lower bound
|
||||
var interval = TimeSpan.FromSeconds(1d / fps);
|
||||
|
||||
var encoder = new FakeEncoder();
|
||||
var scene = BackgroundScene();
|
||||
var delays = new List<TimeSpan>();
|
||||
using var pump = new FramePump(
|
||||
sceneProvider: () => scene,
|
||||
frameResolver: _ =>
|
||||
{
|
||||
System.Threading.Thread.Sleep(renderCostMs);
|
||||
return null;
|
||||
},
|
||||
compositorOptions: () => new CompositorOptions
|
||||
{
|
||||
SourceRectX = 0, SourceRectY = 0, SourceRectWidth = 64, SourceRectHeight = 48,
|
||||
OutputWidth = 64, OutputHeight = 48,
|
||||
},
|
||||
encoderOptions: () => new EncoderOptions
|
||||
{
|
||||
RtmpUrl = "rtmp://a.rtmp.youtube.com/live2/abc",
|
||||
Width = 64, Height = 48, Fps = fps,
|
||||
},
|
||||
encoderFactory: () => encoder,
|
||||
pacingDelay: async (d, ct) =>
|
||||
{
|
||||
lock (delays) delays.Add(d);
|
||||
await Task.Delay(d, ct); // honors the request, like the real Task.Delay default
|
||||
});
|
||||
|
||||
await pump.StartAsync();
|
||||
var deadline = DateTime.UtcNow.AddSeconds(10);
|
||||
while (DateTime.UtcNow < deadline)
|
||||
{
|
||||
lock (delays) if (delays.Count >= 3) break;
|
||||
await Task.Delay(10);
|
||||
}
|
||||
await pump.StopAsync();
|
||||
|
||||
List<TimeSpan> got;
|
||||
lock (delays) got = delays.ToList();
|
||||
Assert.True(got.Count >= 3, $"pump paced only {got.Count} frames in 10s");
|
||||
foreach (var d in got)
|
||||
{
|
||||
Assert.True(d >= TimeSpan.Zero, $"requested wait {d} is negative");
|
||||
Assert.True(d < interval,
|
||||
$"requested wait {d} reaches the full {interval} interval — render cost must subtract from the deadline (the take-3 bug)");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user