perf(render): take-4 slice 2 — IsOpaque memcpy, integer bilinear, pump scratch pool
Take 4: pacing held (sync perfect) but avg render stayed 58.9ms — 2M managed row-walk iterations + a fresh 8.3MB buffer every tick (LOH churn into GC stalls inside the render measurement). - VideoFrame.IsOpaque: producer-contract flag (screen capture + webcam — DWM/MF fill alpha 255; media/chat/web/static NOT flagged). Full-cover aligned opaque backdrop = ONE Buffer.BlockCopy; black pre-fill skipped when it covers. - General BlitContent: integer 8.8 fixed-point bilinear + blend, row invariants hoisted, no per-pixel division/Math.Round. Within ±1 of the float reference (pixel tests allow ±2). Research per derivative-work rule: libyuv row/scale kernels (chromium.googlesource.com/libyuv/libyuv). - FramePump scratch pool (max 4, length-keyed, owned-by-reference): release strictly AFTER SubmitFrameAsync returns (stdin write copies); Contains-guard makes the transition Cut alias safe. - Removed the dead per-tick fromScene render + fromSceneProvider seam — BlendFrame consumes TransitionService.FromFrame captured at Start; the pump's render fed nothing. MainViewModel call site updated (signature). Bugs caught by the pixel probes pre-ship (recorded MyMistakes): first Bilinear double-shifted both stages (solid-255 sampled to ~1 -> general path drew nothing); sentinel 0xAB collided with an x+y pixel. FakeEncoder snapshots submitted frames (mirrors real copy semantics under recycling). ONE integration test: Pump_Pools_ScratchBuffers_Across_Frames_Without_ Stale_Pixels (alternating backdrops + repeated backing identity). Direct pin: Composite_OpaqueFullCover_Backdrop_CopiesEveryPixel_Into_Scratch. Clean build 0 warnings; 59/59 per-class + RealApp boot-smoke. take 5 verdict: expect avg render <= ~10ms, ~300/300 frames. Docs same commit: ai.md pipeline section, TASKS.md TASK 18, HANDOFF rewritten (Unit B spec + settled decisions queued).
This commit is contained in:
+67
-50
@@ -2,67 +2,84 @@
|
||||
|
||||
## Branch / Commit State
|
||||
|
||||
**`main`** — take-3 starvation fix commits land locally (see `git log`); push only when the user says so.
|
||||
The take-3 audit itself happened 2026-09-02/03: user recorded take 3 (`Downloads\recordings\`, manually
|
||||
renamed `llamacasty-recording-3.mp4`), ffprobe + `startup.log` + an extracted frame did the diagnosis.
|
||||
**`main`** — Unit A (render starvation, slice 2) committed locally this session; push on the user's word
|
||||
(his pattern: says "push" explicitly). Slice 1 (deadline pacing + row-blit, take-3 fix) is already on
|
||||
`origin/main` as `716a77f`.
|
||||
|
||||
## What the take-3 audit found
|
||||
## What just happened (2026-09-04, Unit A slice 2)
|
||||
|
||||
- **37s recording → 2.1s file, 127 frames @60fps** = ~17x time-lapse, audio truncated to match
|
||||
(ffmpeg's rawvideo demuxer was blocked on the starved video input, so only ~2.3s of audio was consumed).
|
||||
The user's "webcam at 6x, garbled" = the same file-level time-compression, most visible on the webcam.
|
||||
- **The stage-timing seam paid for itself in one line**: `17/300 frames per 5s, avg render 258.1ms,
|
||||
avg submit 1.5ms` — the encoder/pipe was healthy; `SceneCompositor.Render` was the whole bottleneck.
|
||||
- **Two defects, both fixed (2026-09-03)** — research-first (OBS `libobs/media-io/video-io.c` deadline
|
||||
pacing + libyuv row-blit pattern; cited in the commit and `MyMistakes.md` recipe):
|
||||
1. `FramePump` slept the FULL interval after each render → period = render+interval. Now an absolute
|
||||
`nextTick += intervalTicks` deadline; on overrun rebase (no catch-up burst).
|
||||
2. Per-pixel float sampling + `Math.Round` blends over all 2.07M master px; `BlitOverlay` scanned the
|
||||
full destination per overlay. Now: 1:1 aligned row-walk fast path (alpha branch, integer fixed-point
|
||||
blend), `BlitOverlay` clipped to the intersection, backdrop-cover skips the black pre-fill.
|
||||
3. **Webcam-in-output VERIFIED** from the take-3 frame (bottom-right, border + social bar render too).
|
||||
- Test: `Pump_Paces_To_The_Deadline_Compensating_Render_Cost` (records the requested wait via the pacing
|
||||
seam). Per-class runs: FramePumpTests 10/10, SceneCompositor+SceneGraph+SocialBar+StretchMath 20/20.
|
||||
Clean build 0 warnings. **Landmine found the hard way:** a pacing fake returning
|
||||
`Task.CompletedTask` synchronously runs the whole pump loop on `StartAsync`'s continuation and HANGS
|
||||
the vstest run — fakes must genuinely await (`Task.Delay(d, ct)`) or yield.
|
||||
- Minor logged on take-3 stop: `FramePump: encoder stop failed: No process is associated` — cosmetic
|
||||
teardown race (process already exited before StopAsync's kill); follow-up only if it grows teeth.
|
||||
Take 4 verdict: pacing HELD (no stall cliff, sync intact — user confirmed game+webcam in-sync) but render
|
||||
stayed **58.9ms** (budget 16.7) → file still ~3.4x time-lapse. Root: the 2M-iteration managed row walk +
|
||||
8.3MB fresh buffer every tick. Shipped:
|
||||
|
||||
## OPEN — do next, in order
|
||||
1. **`VideoFrame.IsOpaque`** producer-contract flag — set ONLY by `ScreenCaptureFrameSource` +
|
||||
`MediaCaptureFrameSource` (DWM/MF fill alpha 255). Full-canvas aligned blit of an opaque frame =
|
||||
ONE `Buffer.BlockCopy`; black pre-fill skipped when the backdrop covers.
|
||||
2. **Integer fixed-point bilinear** in `BlitContent` general path (webcam: round/mirror/scaled) — no
|
||||
divisions, no `Math.Round`; ±1 of the float reference (tests allow ±2).
|
||||
3. **Pump scratch pool** — `AcquireScratch`/`ReleaseScratch` (max 4, length-keyed, owned-by-reference so
|
||||
bake-cache/social-bar/static-art arrays can never be captured). Release strictly AFTER
|
||||
`SubmitFrameAsync` returns (stdin write copies). `FakeEncoder` now snapshots frames like the real
|
||||
encoder (holds would race legitimate recycling).
|
||||
4. **Dead code kill:** the per-tick `fromScene` render in the transition branch fed NOTHING
|
||||
(`BlendFrame` uses `TransitionService.FromFrame` captured at `Start`) — removed with the
|
||||
`fromSceneProvider` seam + `MainViewModel.cs` call site. Transition cost halves as a side effect.
|
||||
|
||||
1. **Take 4 (user records ~30s, record-only)** → ffprobe the file (expect frames ≈ 60×seconds, duration
|
||||
≈ wall time) + read `startup.log` `FramePump stats` (expect `n/300`, `avg render` ≤ ~16ms). If n/300
|
||||
climbs only to ~45-55: next slice is the 8.3MB/frame LOH allocation → pool the master buffer.
|
||||
2. **Then code, one integration test per change** (queue from before, unchanged): radio pills /
|
||||
record-OR-stream enforcement (TASK 18 ruling; `BeginGoLive(alsoRecord)` dies) → top bar Option A
|
||||
**pending user approval** → empty-state ruling pending (disabled / rehearsal / default-record) →
|
||||
SYNC slider placement ruling pending.
|
||||
3. **Un-asked questions** (don't nag; answers ready): the recorder said the desktop capture "didn't last
|
||||
long enough to tell quality" — take 4 fixes that; "other issues" from takes 1/2 never listed.
|
||||
Bugs caught by the pixel probes before shipping (see MyMistakes take-4 follow-ups): first `Bilinear`
|
||||
double-shifted (both stages scaled → solid-255 sampled to ~1 → general path drew nothing); sentinel
|
||||
0xAB collided with a legitimate `x+y` value; pacing-fake synchronous completion hangs vstest (known,
|
||||
re-trod).
|
||||
|
||||
## State of the app
|
||||
**Verification:** clean build 0 warnings (both projects); per-class vstest 59/59 (FramePump 11 incl. the
|
||||
new pooling test, SceneCompositor incl. NEW `Composite_OpaqueFullCover...`, SceneGraph, SocialBar, StretchMath,
|
||||
Camera/ScreenCapture/MediaVideoSource producers, WebcamOutputKey, SessionTeardown) + SourceNaming
|
||||
RealApp boot-smoke. Scope-check passed.
|
||||
|
||||
Boots clean. Recording pipeline: render starvation fixed pending take-4 confirmation; webcam + social bar
|
||||
confirmed IN the output. ffmpeg cached at `%APPDATA%\ytLlive\tools\` (month-end pin). Test suite per-class
|
||||
green; do NOT run full-suite vstest (WASAPI teardown hang, pre-existing).
|
||||
## OPEN — next, in order
|
||||
|
||||
1. **Take 5 (user, ~30s record-only):** read `%APPDATA%\ytLlive\startup.log` — expect
|
||||
`FramePump stats: ≈300/300 frames per 5s, avg render ≤ ~10ms, avg submit ~1ms` + ffprobe frames ≈
|
||||
60×duration. If render is under budget but `n/300` still short → suspect capture-side (the 1440p
|
||||
`DownscaleBgra` per-frame alloc loop on the capture thread) or residual GC — next slice.
|
||||
Also for the user to eyeball: webcam now at true 60fps, audio still in sync.
|
||||
2. **Unit B — the top bar + session logic (user spec 2026-09-04, decisions settled):**
|
||||
- Two-line top bar. Line 1: center = REC + **LIVE** pills (text renamed from ON-AIR; pills become
|
||||
mutually-exclusive RADIOS — record-OR-stream ruling), right = avatar + **Login/Logout** button
|
||||
(no account status light). Line 2: centered primary **Start** (grayed while NO pill armed —
|
||||
INVERTS the 2026-09-01 "unarmed Start records" rule; fix the map when landing) that becomes the
|
||||
Stop/End button while active.
|
||||
- Avatar right-click → **Change Account** (creator: "standard google thing ... on a portrait
|
||||
right-click"). Login = `SignInCommand` direct (context menu on Start dies; "Choose Record Folder"
|
||||
lives in gear → App Settings only).
|
||||
- LIVE pill stays login-gated (`CanToggleOnAir` exists ✓ 3a.viii).
|
||||
- REC+Start → `Microsoft.Win32.SaveFileDialog` (InitialDirectory = settings folder, default name
|
||||
`ty-…-0000.mp4`, NATIVE overwrite prompt covers exists/validate, Enter confirms). Cancel →
|
||||
abort + disarm pill (lit pill with no session is a lie). Up-front naming RETIRES the stop-time
|
||||
rename modal (assumption stated; user's dialog answer was about Go-Live confirmation).
|
||||
- LIVE+Start → **Go-Live dialog stays** as preflight: prefilled from Text-drawer `Broadcast.*`,
|
||||
unfilled fields visibly prompted, explicit confirm → `PrepareAndStartLiveAsync` (user: "going
|
||||
live is scary — confirmation allows back-out + testing up to go-live").
|
||||
- Bottom-bar metrics init/maintain: `ResetHealth` + `HealthUpdated` exist — verify on take 5/6.
|
||||
- F6 "start/end" hotkey routes through `HandleHotkey` — check it honors the new grayed-Start gate.
|
||||
- ONE integration test (hermetic): pills↔button state machine + record-path seam (an
|
||||
`internal static Func<SaveFileDialog-ish prompt>` override seam mirroring `RegistrarOverride`
|
||||
— never pop real dialogs in tests).
|
||||
3. **Follow-ups recorded (don't fix opportunistically):** vertical-tier `BilinearScale` per-frame
|
||||
alloc; cosmetic `FramePump: encoder stop failed: No process is associated` double-stop race;
|
||||
1440p capture downscale alloc.
|
||||
|
||||
## Landmines
|
||||
|
||||
- testhost shares startup.log with the app — filter by time when triaging.
|
||||
- Stale testhost/exe locks the DLL (MSB3027): `taskkill /F /IM testhost.exe` / `ytLive.exe` first.
|
||||
- Do NOT run full-suite vstest (hangs); do NOT claim suite totals — per-class only. verify.sh's step 2 IS
|
||||
the full-suite hang — flow: clean build + per-class vstest + scope-check.
|
||||
- Windows binaries (ffprobe/ffmpeg under `%APPDATA%\ytLlive\tools\`) need WINDOWS paths
|
||||
(`C:\Users\...`), never `/mnt/c/...` — the mount path reads as "No such file or directory".
|
||||
- Real-`MainWindow` tests MUST use `LayoutPathOverride` + temp DB; `VolumePushOverride` seam exists so
|
||||
volume-slider tests never touch the machine's speakers.
|
||||
- `AudioPipelineTests` was the "known failure" graveyard — a new failure there means a live-loop
|
||||
regression; read the logged stack (throttled, with stack).
|
||||
- Pacing-seam fakes must yield/await — synchronous completion runs the pump loop inline (see above).
|
||||
- Do NOT run full-suite vstest (WASAPI hang, pre-existing); flow = clean build + per-class + scope-check.
|
||||
- Pacing-seam fakes MUST await/yield (sync-completed task → pump runs inline on StartAsync → hang).
|
||||
- `FakeEncoder` snapshots submitted frames — keep any new fake encoder honest about buffer recycling.
|
||||
- Multi-stage fixed-point: shift only at the end (MyMistakes 2026-09-04).
|
||||
- Real-`MainWindow` tests: `LayoutPathOverride` + temp DB mandatory; `VolumePushOverride` for volume.
|
||||
- ffmpeg: month-end pinned build; `Startup.log` "Recording saved:" lines show the real final path.
|
||||
|
||||
## @ User note
|
||||
|
||||
He does not want tangents acknowledged twice (rename question = noise; the ask was ALWAYS the recording).
|
||||
Keep responses SHORT; work the queue; one integration test per change; committing is expected, pushing is
|
||||
his word.
|
||||
Recording fix FIRST (his order), UX queue right behind — spec + settled decisions above, don't re-ask.
|
||||
Keep responses SHORT; one integration test per change; commit every unit; push on his word only.
|
||||
|
||||
Reference in New Issue
Block a user