perf(pump): slice 18 — C4 blit-on-change composite cache
ty-1841: capture fixed (band ~20 updates/s, no tears) but FramePump stalls on EVERY iteration (totalMs 21-44, render=full-render split=0 elements=6 dynamic=4) — the render ceiling, ~22-28 composites/s, caps the desktop in a 60fps file. SceneGraph can't help: the live backdrop is element 0 and cannot be baked (a cached capture goes stale), so the cache lives at the pump. RenderFull wraps both full-render call sites: BuildFullRenderSignature hashes the full input identity (options rect, social bar, per-element layout/visual bits + the frame each element would resolve through the SAME resolver seam, using array identity + Epoch + CropBounds); unchanged identity reuses the last composite with one Buffer.BlockCopy (~3ms) instead of a ~30ms re-composite. Cache buffer is a separate long-lived array, written pre-burn/pre-recycle (the caller burns the frame counter and recycles scratch AFTER render). Engagement gated on the 1:1 config (the only deployed tier). Telemetry surfaces `cache NR/WH` on the 5s stats line. Same shape as OBS (sources cache their surface, the scene blits on update) — docs.obsproject.com/backend-design, the pattern this repo cites since the 2026-09-04 paste-cache slice. Good Dog: FullRenderCache_StaticInputs_RenderOnce_Then_Reuse_UntilInputChanges (static scene renders ONCE + byte-identical reuse; new frame+Epoch invalidates). 297/297 green, 0 warnings, verify.sh scope-locked (FramePump.cs, FramePumpTests.cs + ai.md/HANDOFF/MyMistakes). No push — device re-verify next.
This commit is contained in:
+49
-59
@@ -1,11 +1,11 @@
|
||||
# HANDOFF — 2026-09-14 (slice-17 concurrent capture conversions committed locally — device verify next)
|
||||
# HANDOFF — 2026-09-15 (slice-18 C4 composite cache committed locally — device verify next)
|
||||
|
||||
## Branch / Commit State
|
||||
|
||||
`main` HEAD = **slice-17 commit** (overlapping capture readbacks — committed LOCALLY, **NOT
|
||||
pushed**; web/A/V work stays commit-local until greenlight). Before it: slice-16 capture
|
||||
conversion fix, slice-15 pacing fix (FramePump), `c01206f` (composition capture), `b22d08e`
|
||||
(signed audio-sync, pushed). Working tree **clean**.
|
||||
`main` HEAD = **slice-18 commit** (C4 blit-on-change composite cache — committed LOCALLY, **NOT
|
||||
pushed**; web/A/V work stays commit-local until greenlight). Before it: slice-17 overlapping
|
||||
capture readbacks, slice-16 capture conversion fix, slice-15 pacing fix (FramePump), `c01206f`
|
||||
(composition capture), `b22d08e` (signed audio-sync, pushed). Working tree clean.
|
||||
|
||||
## ⚠️ Branding (2026-09-14, creator-corrected): product = **llamacasty**, internals = ytLive
|
||||
|
||||
@@ -13,61 +13,46 @@ The product is **llamacasty**; repo path, csproj `AssemblyName`/`RootNamespace`,
|
||||
(`%APPDATA%\ytLlive\...`), and most code names are the legacy **ytLive/ytLlive**. User-facing
|
||||
language says "llamacasty"; code/assembly/repo names stay ytLive. See `ai.md` → Brand.
|
||||
|
||||
## 🔬 Slice-16 build was still too frozen — take ty-20260914-1824 says the READBACK is the wall
|
||||
## ✅ Committed locally — slice 18: C4 = FramePump blit-on-change composite cache
|
||||
|
||||
Slice 15 fixed pacing (video 729 frames @60 = 12.15s ≈ audio 12.35s — pacing healthy). But the
|
||||
creator reports the desktop layer **still looks like missing frames / choppy vs live**. Decoded
|
||||
`ty-20260914-1824-0000-2.mp4` to `/mnt/c/tmpout/f1824.raw` and audited:
|
||||
- Desktop band: **6.8 content updates/s, 88% frozen**, one **4.85s freeze** at start. Worse,
|
||||
not better, than 1742.
|
||||
- BUT the new slice-16 telemetry proved the downscale fix WORKS: `conv avg 46-50ms, max ~61ms,
|
||||
~16-20 conversions/s, skip busy 37-83 per 2s, skip cadence 0, ring allocs 0`. The ring is
|
||||
steady-state (no allocs); the **GPU→CPU readback (`CreateCopyFromSurfaceAsync`) is ~45ms of
|
||||
the conversion** on a 240Hz-HDR box sharing the GPU with the encoder. **The wall was never
|
||||
the CPU downscale.**
|
||||
- FramePump worst render 165ms startup spike → 33-42ms sustained (whole-frame ~7.1/s ⇒ render
|
||||
is the SECOND cap, ~30 unique composites/s).
|
||||
- Delivery is healthy (60-100 arrivals/s) ⇒ focus-loss OS throttling is NOT the cause (that
|
||||
theory is now closed).
|
||||
The ty-1841 take (slice-17 build) proved capture fixed (band ~20 fresh updates/s, no tears, pacing
|
||||
clean) but the render is STILL the wall: **FramePump stall on EVERY iteration** (`totalMs 21-44`,
|
||||
`render=full-render split=0 elements=6 dynamic=4`, worst render 166ms startup spike) — only ~22-28
|
||||
composites/s. The SceneGraph split can't fix it: `GetSplitPoint` returns **0** because the
|
||||
live-capture backdrop is element 0 and CANNOT be baked (a cached capture goes stale).
|
||||
|
||||
## 🔬 Committed locally — slice 17: overlapping readbacks + monotonic publish + deeper pool
|
||||
**What** (`Services/Encoder/FramePump.cs`, + new Good Dog test in `ytLive.Tests/FramePumpTests.cs`):
|
||||
the full-render path now caches the last composite + its INPUT IDENTITY. `BuildFullRenderSignature`
|
||||
mirrors the compositor's own resolution (same resolver seam: element ref + layout/visual bits +
|
||||
resolved frame's array identity + Epoch + CropBounds + options + social bar) — unchanged identity →
|
||||
ONE `Buffer.BlockCopy` (~3ms) instead of the full re-composite (~30ms); changed identity → re-render.
|
||||
Cache buffer is a separate long-lived array, written pre-burn/pre-recycle (never the scratch pool).
|
||||
Gated on the 1:1 config (the only deployed tier). Telemetry: `cache {renders}R/{hits}H` on the 5s
|
||||
stats line + internal `CacheHits`/`CacheRenders`/`OutputIndex`.
|
||||
|
||||
**What** (`Services/ScreenCaptureFrameSource.cs` + new `Services/MonotonicGate.cs` + locked
|
||||
`Services/FrameRingBuffer.cs`):
|
||||
1. **MaxConcurrentConversions = 3** readbacks in flight (was one-in-flight `_framePending`),
|
||||
pool buffers 2 → **5** so in-flight frames fit.
|
||||
2. **MonotonicLatest publish gate** (`MonotonicGate`, new internal): a completed readback is
|
||||
published ONLY if its Epoch is strictly newer than the last published. Overlapping
|
||||
conversions can finish out of order; a slow OLDER completion must never overwrite a newer
|
||||
`LatestFrame` (backwards time hole = the mirror of the 1742 tear). Epoch via
|
||||
`Interlocked.Increment`.
|
||||
3. `FrameRingBuffer.Rent`/`ConsumeAllocations` now take `_lock` (rents are concurrent); the
|
||||
10ms floor and downscale stay; `DownscaleBgra` row-scratch is per-conversion locals (no
|
||||
shared `_row0/_row1`).
|
||||
4. **RESEARCH near-miss (docs'ed, MyMistakes):** shrinking the pool to 1920×1080 would have
|
||||
been wrong — Microsoft Docs (screen capture): *"the underlying Direct3D surface is always
|
||||
the size specified … **clipped**"* to the frame. Readback stays native; the lever is
|
||||
concurrency.
|
||||
|
||||
**Good Dog test:** `ScreenCaptureFrameSourceTests.PublishGate_TryPublish_OnlyStrictlyNewerWins`.
|
||||
**296/296 green, app build 0 warnings.** Scope-lock files (6): `Services/ScreenCaptureFrameSource.cs`,
|
||||
`Services/FrameRingBuffer.cs`, `Services/MonotonicGate.cs` (new), `ytLive.Tests/ScreenCaptureFrameSourceTests.cs`
|
||||
+ docs (ai.md Slice 17, MyMistakes slice-17 block, this HANDOFF). `SceneCompositor.cs`/`FramePump.cs`
|
||||
NOT touched (C4 is the next slice, pending this re-measure).
|
||||
**Good Dog test:** `FullRenderCache_StaticInputs_RenderOnce_Then_Reuse_UntilInputChanges` — static
|
||||
scene renders ONCE then hits (byte-identical above the burn strip), a new frame (new array + Epoch)
|
||||
invalidates + propagates. Existing `Pump_Pools_...` test now passes a STABLE scene (like production)
|
||||
and keys alternation on `OutputIndex` (the two resolver passes per tick double-advanced a call-count
|
||||
flip). **297/297 green, app + tests build 0 warnings.** Scope-locked (2 code files + ai.md +
|
||||
HANDOFF): `Services/Encoder/FramePump.cs`, `ytLive.Tests/FramePumpTests.cs`.
|
||||
|
||||
## ⚠️ Open items (before PUSHABLE)
|
||||
|
||||
- **Device re-verify (next step):** creator records the SAME tv-show scenario on the slice-17
|
||||
- **Device re-verify (next step):** creator records the SAME tv-show scenario on the slice-18
|
||||
build. Judge numerically:
|
||||
- startup.log telemetry: conversions/s should jump from ~17-20 to **≥ ~30-40**, `skip busy`
|
||||
falling, `ring allocs` ≈ 0, conv avg still ~40-50ms (readback isn't free, it just overlaps).
|
||||
- Decode + `/tmp/opencode/tear_audit.py`: desktop-band fresh updates/s up toward the render
|
||||
cap (~30+), frozen % well under 50%, no genuine mid-frame splits.
|
||||
- ffprobe: video ≈ audio ≈ wall.
|
||||
- If capture now feeds ≥ render's unique-composite rate and render still busts 16.6ms slots →
|
||||
**C4 slice** (Epoch-cached composite / blit-on-change), still 60fps. If capture still lags,
|
||||
the bind is GPU contention — re-measure before touching anything.
|
||||
- **No push yet** — commit-locally-until-greenlight for web/A/V work.
|
||||
- startup.log telemetry: `cache` line shows hits dominating on TV holds (`e.g. cache 1R/250H`),
|
||||
`avg render` drops toward the ~3ms BlockCopy, FramePump **stalls disappear** (the per-iteration
|
||||
stall was the C4 signature).
|
||||
- Decode + `/tmp/opencode/freeze_audit.py` / `band_timeline.py`: desktop-band fresh updates/s up
|
||||
toward ~60 (was ~20 first-11s; the render cap was the bind), no mid-frame splits.
|
||||
- ffprobe: video ≈ audio ≈ wall (pacing already healthy at slice 17 — unchanged expected).
|
||||
- If the desktop layer STILL reads choppy after cache hits dominate every static hold, the residual
|
||||
is the **24fps TV → 60fps container pulldown** (inherent 3:2-ish repeats; the 1841 gap histogram
|
||||
was 89×2-slot + 84×3-slot holds) — that's content, not the pipeline; decide with the creator
|
||||
whether it needs an adaptive cadence or is acceptable.
|
||||
- **No push yet** — commit-locally-until-greenlight for web/A/V work. After the take verdict, also
|
||||
re-measure the clap offset (`/tmp/opencode/avsync.py`), then decide push with the user.
|
||||
|
||||
## Open threads (carried)
|
||||
|
||||
@@ -85,16 +70,21 @@ NOT touched (C4 is the next slice, pending this re-measure).
|
||||
- Build/tests: **Windows dotnet host** (`/mnt/c/Program Files/dotnet/dotnet.exe`). 0 warnings —
|
||||
only `./scripts/verify.sh "<files>"`'s clean build counts. Building `ytLive.csproj` alone does
|
||||
NOT rebuild `ytLive.Tests.dll` — run the Tests csproj before `vstest`.
|
||||
- FramePump tests that assert per-frame CONTENT must pass a STABLE scene (`() => scene`) — the
|
||||
default NewPump scene is fresh-per-tick (ok for pacing tests, but it churns the C4 render
|
||||
signature and hides the cache). Cache-sensitive assertions also can't use a call-count resolver
|
||||
flip (the tick resolves twice: signature + render) — key alternation on `OutputIndex`.
|
||||
- ffmpeg/ffprobe: `/mnt/c/Program Files/Krita (x64)/bin/` with Windows paths.
|
||||
- `MyMistakes.md` has the **freeze-audit RECIPE**, the **A/V sync measurement recipe**, the
|
||||
**deadline-pacing** lessons, the **CoreMessaging DQ recipe**, and now the **WGC-CLIP** + two
|
||||
slice blocks — grep before re-deriving.
|
||||
**deadline-pacing** lessons, the **CoreMessaging DQ recipe**, and the **WGC-CLIP** + slice
|
||||
blocks — grep before re-deriving.
|
||||
- sqlite3 at `/home/gramps/android-sdk/platform-tools/sqlite3`.
|
||||
- `C:\tmpout` is for ffmpeg evidence artifacts (raw decodes / PNGs); keep them out of the repo.
|
||||
|
||||
## Next step
|
||||
|
||||
Creator records a tv-show take on the slice-17 build → read the startup.log telemetry line
|
||||
(conversions/s ≥ ~30-40, `skip busy` falling) + `tear_audit.py` cadence + ffprobe durations. If
|
||||
the desktop now tracks the render cap (~30+ updates/s, <50% frozen, no splits): C4 render slice
|
||||
next, then re-measure clap offset (`/tmp/opencode/avsync.py`), then decide push with the user.
|
||||
Creator records a tv-show take on the slice-18 build → read the startup.log telemetry: shift-stall
|
||||
frequency and the `cache NR/WH` line (hits must dominate on TV holds) + the band audit + ffprobe
|
||||
durations. If the desktop now tracks ~60 updates/s and stalls are gone: re-measure the clap offset,
|
||||
then decide push with the user. If the layer is still choppy on fully-static holds, the pulldown
|
||||
(readme) is the residual and it's a content decision, not a pipeline bug.
|
||||
@@ -668,3 +668,23 @@ tick's worth after muting, and if the pipe still carries pre-mute buffered bytes
|
||||
pipe first or assert on a multi-tick window. Do not blame a sync change for this — verify the grain
|
||||
against the clean tree in the same mode before touching the mixer.
|
||||
|
||||
---
|
||||
|
||||
### Slice-18 follow-up (2026-09-15) — C4 composite cache: test fakes must mirror production's STABLE scene and per-tick purity
|
||||
|
||||
The C4 blit-on-change cache keys on a hash of the resolved inputs, INCLUDING per-element reference
|
||||
identity (`RuntimeHelpers.GetHashCode(element)`). Two test-setup habits silently broke/starved it:
|
||||
|
||||
1. **`NewPump`'s default scene is fresh per tick** (`() => BackgroundScene()` — fine for pacing
|
||||
tests) — churned the element refs, so the signature NEVER matched and the cache looked broken
|
||||
(301 "renders" instead of 1). Production hands a STABLE `StagedScene`. **Rule: any FramePump test
|
||||
that asserts per-frame content or cache behavior must pass `scene: () => scene` with one Scene
|
||||
instance; the fresh-scene default is only for pacing/diagnostic tests.**
|
||||
2. **A call-count resolver flip (`flip++ % 2`) double-advances under a cache-aware pump** — the
|
||||
tick resolves TWICE (signature pass + compositor pass). Key per-tick content on the burned
|
||||
`OutputIndex` (stable until submit, which happens AFTER render) or on `encoder.Frames.Count`
|
||||
instead. A stable-per-tick token, not a call counter, is the deterministic alternation.
|
||||
3. New frames that must invalidate the cache need BOTH a fresh array AND a fresh Epoch (array
|
||||
identity alone is unchanged for a mutated-in-place array; `Epoch` is the monotonic generation
|
||||
marker the compositor's paste key and the C4 signature share).
|
||||
|
||||
|
||||
@@ -92,6 +92,28 @@ public sealed class FramePump : IDisposable
|
||||
/// stall line, then cleared.</summary>
|
||||
private string _renderDetail = "";
|
||||
|
||||
/// <summary>C4 blit-on-change cache (2026-09-15): the FULL-render path (split=0
|
||||
/// when the live-capture backdrop sits at element 0, or no SceneGraph) re-composites
|
||||
/// the ENTIRE frame every tick even when no input changed — the ty-1841 take logged
|
||||
/// a render stall on EVERY iteration (totalMs 21-44, one composite per ~30ms while
|
||||
/// the content moved ~20 updates/s). The SceneGraph split can never help here: the
|
||||
/// backdrop is element 0 and must stay dynamic (a baked capture goes stale), so the
|
||||
/// cache lives at the pump: the last full-render output plus the input identity that
|
||||
/// produced it. Unchanged identity → ONE BlockCopy (~3ms) reuses the composite;
|
||||
/// changed identity → re-composite. The cache buffer is a SEPARATE long-lived array,
|
||||
/// never the scratch pool — the caller burns the frame counter and recycles the
|
||||
/// scratch AFTER RenderScene returns, so the cache is written from the rendered
|
||||
/// scratch BEFORE returning (pre-burn, pre-recycle). The identity mirror is
|
||||
/// <see cref="BuildFullRenderSignature"/>: it resolves the same frames the compositor
|
||||
/// will (same resolver seam), so a changed frame (new capture, new webcam, new chat
|
||||
/// raster) forces a re-render and an unchanged one never serves stale bytes. Internal
|
||||
/// counters feed the Good Dog test + the 5s cache detail in the stats line.</summary>
|
||||
private byte[]? _fullCachePixels;
|
||||
private bool _fullCacheValid;
|
||||
private ulong _lastFullSignature;
|
||||
internal long CacheHits;
|
||||
internal long CacheRenders;
|
||||
|
||||
/// <summary>Forwards the encoder's parsed health — ship step 6 binds this to the bottom bar.</summary>
|
||||
public event EventHandler<StreamHealth>? HealthUpdated;
|
||||
|
||||
@@ -124,6 +146,11 @@ public sealed class FramePump : IDisposable
|
||||
|
||||
public bool IsRunning { get; private set; }
|
||||
|
||||
/// <summary>The burned counter of the LAST frame submitted to the encoder. Stable
|
||||
/// across a tick's multiple resolver passes (it advances only at burn/submit, which
|
||||
/// happens AFTER render) — tests key per-tick content on it.</summary>
|
||||
internal long OutputIndex => _outputIndex;
|
||||
|
||||
/// <summary>Never throws: failures are logged and surfaced via <see cref="Failed"/>,
|
||||
/// so the VM can fire-and-forget it from a sync command handler.</summary>
|
||||
public async Task StartAsync(CancellationToken cancellationToken = default)
|
||||
@@ -303,6 +330,7 @@ public sealed class FramePump : IDisposable
|
||||
long renderTicks = 0, submitTicks = 0, resolveTicks = 0, waitTicks = 0, worstRender = 0, worstSubmit = 0;
|
||||
int statFrames = 0;
|
||||
int stalls = 0;
|
||||
long lastCacheHits = 0, lastCacheRenders = 0;
|
||||
var statsNext = DateTime.UtcNow + TimeSpan.FromSeconds(5);
|
||||
void ReportStats()
|
||||
{
|
||||
@@ -313,6 +341,17 @@ public sealed class FramePump : IDisposable
|
||||
var dropped = encoder == null ? 0 : encoder.DroppedFrames;
|
||||
var detail = _renderDetail;
|
||||
_renderDetail = "";
|
||||
var cacheDetail = "";
|
||||
if (statFrames > 0)
|
||||
{
|
||||
// C4 window detail: rendered composites (R) vs cache reuses (H) since the
|
||||
// last report — hard numeric proof the blit-on-change cache engaged.
|
||||
var hits = CacheHits - lastCacheHits;
|
||||
var renders = CacheRenders - lastCacheRenders;
|
||||
lastCacheHits = CacheHits;
|
||||
lastCacheRenders = CacheRenders;
|
||||
if (hits > 0 || renders > 0) cacheDetail = $", cache {renders}R/{hits}H";
|
||||
}
|
||||
_log?.Invoke(statFrames == 0
|
||||
? "FramePump stats: NO frames produced in 5s (loop stalled?)" + (detail.Length > 0 ? " | " + detail : "")
|
||||
: $"FramePump stats: {statFrames}/{target:F0} frames per 5s, " +
|
||||
@@ -322,7 +361,7 @@ public sealed class FramePump : IDisposable
|
||||
+ $"avg wait {waitTicks / (double)System.Diagnostics.Stopwatch.Frequency * 1000 / statFrames:F1}ms, "
|
||||
+ $"worst render {worstRender / (double)System.Diagnostics.Stopwatch.Frequency * 1000:F1}ms, "
|
||||
+ $"worst submit {worstSubmit / (double)System.Diagnostics.Stopwatch.Frequency * 1000:F1}ms, "
|
||||
+ $"dropped {dropped}, stalls {stalls}" + (detail.Length > 0 ? " | " + detail : ""));
|
||||
+ $"dropped {dropped}, stalls {stalls}" + cacheDetail + (detail.Length > 0 ? " | " + detail : ""));
|
||||
worstRender = 0;
|
||||
worstSubmit = 0;
|
||||
renderTicks = submitTicks = resolveTicks = waitTicks = 0;
|
||||
@@ -523,7 +562,7 @@ public sealed class FramePump : IDisposable
|
||||
if (_sceneGraph == null)
|
||||
{
|
||||
var sw = System.Diagnostics.Stopwatch.StartNew();
|
||||
var frame = _compositor.Render(scene, resolver, null, options, socialBarFrame, socialBarTop, scratch: scratch);
|
||||
var frame = RenderFull(scene, options, socialBarFrame, socialBarTop, scratch, resolver);
|
||||
ProbeRender("full-no-graph", scene, -1, sw.ElapsedMilliseconds, 0);
|
||||
return frame;
|
||||
}
|
||||
@@ -554,11 +593,163 @@ public sealed class FramePump : IDisposable
|
||||
|
||||
// No static base (first layer is dynamic or empty scene) — full render.
|
||||
var swFull = System.Diagnostics.Stopwatch.StartNew();
|
||||
var frame2 = _compositor.Render(scene, resolver, null, options, socialBarFrame, socialBarTop, scratch: scratch);
|
||||
var frame2 = RenderFull(scene, options, socialBarFrame, socialBarTop, scratch, resolver);
|
||||
ProbeRender("full-render", scene, split, swFull.ElapsedMilliseconds, 0);
|
||||
return frame2;
|
||||
}
|
||||
|
||||
/// <summary>The full-render path (no SceneGraph, or no static base) wrapped in the
|
||||
/// C4 blit-on-change cache: when every resolved input plus every element's
|
||||
/// layout/visual bits match the last composite, reuse it with one BlockCopy instead
|
||||
/// of re-compositing. Mirror the compositor's resolution <i>before</i> deciding so a
|
||||
/// hit costs ~3ms of pure copy and a miss costs exactly the old full render.</summary>
|
||||
private VideoFrame RenderFull(
|
||||
Scene scene, CompositorOptions options, VideoFrame? socialBarFrame, int socialBarTop,
|
||||
byte[]? scratch, Func<SceneElement, VideoFrame?> resolver)
|
||||
{
|
||||
// 1:1 guard: the cache stores the OUTPUT bytes and scratch is source-sized —
|
||||
// at 1:1 they are the same buffer size. Off-size tiers (a vertical 1080×1920
|
||||
// tier, an H264-1080p tier) fall back to a per-tick full render; the deployed
|
||||
// config is master-sized and caching a per-tick fresh scale buffer is the known
|
||||
// vertical-tier follow-up (see ai.md), not silently baked here.
|
||||
var useCache = scratch != null && options.SourceRectWidth == options.OutputWidth
|
||||
&& options.SourceRectHeight == options.OutputHeight;
|
||||
var signature = useCache
|
||||
? BuildFullRenderSignature(scene, options, socialBarFrame, socialBarTop, resolver)
|
||||
: 0UL;
|
||||
|
||||
if (useCache && _fullCacheValid && signature == _lastFullSignature
|
||||
&& _fullCachePixels!.Length == scratch!.Length)
|
||||
{
|
||||
Buffer.BlockCopy(_fullCachePixels, 0, scratch, 0, scratch.Length);
|
||||
CacheHits++;
|
||||
return new VideoFrame(options.SourceRectWidth, options.SourceRectHeight, scratch);
|
||||
}
|
||||
|
||||
var rendered = _compositor.Render(scene, resolver, null, options, socialBarFrame, socialBarTop, scratch: scratch);
|
||||
CacheRenders++;
|
||||
if (useCache)
|
||||
{
|
||||
var pixels = rendered.BgraPixels;
|
||||
if (_fullCachePixels == null || _fullCachePixels.Length != pixels.Length)
|
||||
_fullCachePixels = new byte[pixels.Length];
|
||||
// PRE-burn / PRE-recycle: the caller burns the counter and recycles this
|
||||
// scratch after RenderScene returns — the cache keeps a clean copy, and the
|
||||
// burn on the next hit writes to a FRESH scratch copy, never the cache.
|
||||
Buffer.BlockCopy(pixels, 0, _fullCachePixels, 0, pixels.Length);
|
||||
_lastFullSignature = signature;
|
||||
_fullCacheValid = true;
|
||||
}
|
||||
else
|
||||
{
|
||||
_fullCacheValid = false; // size changed — never reuse a wrong-size buffer
|
||||
}
|
||||
return rendered;
|
||||
}
|
||||
|
||||
/// <summary>Hash the complete set of inputs the compositor's FULL render consumes:
|
||||
/// the crop/output dimensions, the social bar, and — in element order, mirroring
|
||||
/// <c>SceneCompositor.Render</c>'s iteration — every element's layout/visual fields
|
||||
/// plus the frame each one resolves (through the SAME resolver seam the render will
|
||||
/// use). A changed frame (new capture, new webcam Epoch, new chat raster) changes the
|
||||
/// hash → re-composite; an unchanged one reuses the cache. Buffer-array identity +
|
||||
/// Epoch + CropBounds is the compositor's own paste-key shape — producers hand out
|
||||
/// fresh immutable arrays, or recycled-ring arrays whose monotonic Epoch
|
||||
/// distinguishes generations (the capture take-11 fix), so address-identity alone is
|
||||
/// never trusted to key recycled content.</summary>
|
||||
private static ulong BuildFullRenderSignature(
|
||||
Scene scene, CompositorOptions options, VideoFrame? socialBarFrame, int socialBarTop,
|
||||
Func<SceneElement, VideoFrame?> resolver)
|
||||
{
|
||||
var h = 14695981039346656037UL; // FNV-1a offset basis
|
||||
h = Mix(h, (ulong)options.SourceRectX);
|
||||
h = Mix(h, (ulong)options.SourceRectY);
|
||||
h = Mix(h, (ulong)options.SourceRectWidth);
|
||||
h = Mix(h, (ulong)options.SourceRectHeight);
|
||||
h = Mix(h, (ulong)options.OutputWidth);
|
||||
h = Mix(h, (ulong)options.OutputHeight);
|
||||
h = Mix(h, (ulong)socialBarTop);
|
||||
if (socialBarFrame != null) h = MixFrame(h, socialBarFrame);
|
||||
else h = Mix(h, 0xFFFFFFFFFFFFFFFFUL);
|
||||
|
||||
var backdropResolved = false;
|
||||
foreach (var element in scene.Elements)
|
||||
{
|
||||
h = Mix(h, (ulong)System.Runtime.CompilerServices.RuntimeHelpers.GetHashCode(element));
|
||||
h = Mix(h, element.IsVisible ? 1UL : 0UL);
|
||||
h = MixDouble(h, element.X);
|
||||
h = MixDouble(h, element.Y);
|
||||
h = MixDouble(h, element.Width);
|
||||
h = MixDouble(h, element.Height);
|
||||
h = MixDouble(h, element.Opacity);
|
||||
h = Mix(h, (ulong)element.ClipShape);
|
||||
h = Mix(h, element.IsMirrored ? 1UL : 0UL);
|
||||
h = Mix(h, (ulong)element.BorderWidth);
|
||||
h = MixDouble(h, element.BorderOpacity);
|
||||
if (element.TryGetBorderColor(out var bcR, out var bcG, out var bcB))
|
||||
h = Mix(h, (ulong)((bcR << 16) | (bcG << 8) | bcB));
|
||||
else
|
||||
h = Mix(h, 0UL);
|
||||
|
||||
// Mirror the compositor's per-element frame resolution EXACTLY, so the hash
|
||||
// reacts to the same frames a render uses and never to a frame a render
|
||||
// ignores (a false-positive change only wastes one render; a false NEGATIVE
|
||||
// would serve stale bytes — that is what this mirror prevents).
|
||||
VideoFrame? frame = null;
|
||||
if (element is Source { IsBackground: true })
|
||||
{
|
||||
h = Mix(h, 1UL); // the live backdrop — first visible one is resolved, once
|
||||
if (element.IsVisible && !backdropResolved)
|
||||
{
|
||||
frame = resolver(element);
|
||||
backdropResolved = true;
|
||||
}
|
||||
}
|
||||
else if (element is Source { Type: SourceType.TextOverlay })
|
||||
{
|
||||
h = Mix(h, 2UL); // skipped kind — never resolved by the compositor
|
||||
}
|
||||
else if (element is Source { Type: SourceType.Background })
|
||||
{
|
||||
h = Mix(h, 3UL); // background image — resolved even when invisible (Render does)
|
||||
frame = resolver(element);
|
||||
}
|
||||
else
|
||||
{
|
||||
h = Mix(h, 4UL); // regular layer — resolved only when visible
|
||||
if (element.IsVisible) frame = resolver(element);
|
||||
}
|
||||
if (frame != null) h = MixFrame(h, frame);
|
||||
else h = Mix(h, 0UL);
|
||||
}
|
||||
return h;
|
||||
}
|
||||
|
||||
private static ulong Mix(ulong h, ulong v) => (h ^ v) * 0x9E3779B97F4A7C15UL;
|
||||
|
||||
private static ulong MixDouble(ulong h, double v)
|
||||
=> Mix(h, unchecked((ulong)BitConverter.DoubleToInt64Bits(v)));
|
||||
|
||||
private static ulong MixFrame(ulong h, VideoFrame frame)
|
||||
{
|
||||
h = Mix(h, (ulong)System.Runtime.CompilerServices.RuntimeHelpers.GetHashCode(frame.BgraPixels));
|
||||
h = Mix(h, (ulong)frame.Epoch);
|
||||
h = Mix(h, (ulong)frame.Width);
|
||||
h = Mix(h, (ulong)frame.Height);
|
||||
if (frame.CropBounds is { } cb)
|
||||
{
|
||||
h = Mix(h, (ulong)cb.X);
|
||||
h = Mix(h, (ulong)cb.Y);
|
||||
h = Mix(h, (ulong)cb.W);
|
||||
h = Mix(h, (ulong)cb.H);
|
||||
}
|
||||
else
|
||||
{
|
||||
h = Mix(h, 0xFFFFFFFFFFFFFFFFUL);
|
||||
}
|
||||
return h;
|
||||
}
|
||||
|
||||
/// <summary>2026-09-12 half-speed-render probe: the pump only produces ~27fps
|
||||
/// (render ~35ms vs the 16.7ms deadline), and rawvideo muxes at the DECLARED fps,
|
||||
/// so every take muxes at ~half its wall length (the truncation complaint). This
|
||||
|
||||
@@ -1090,6 +1090,43 @@ Full suite 290/291 passing, the sole failure the pre-existing compositor pixel t
|
||||
`PublishGate_TryPublish_OnlyStrictlyNewerWins`. Full suite **296/296 green, 0
|
||||
warnings**. NOT YET DEVICE-VERIFIED; target: telemetry frames/s jumps ≥ ~30-40 and the
|
||||
band audit drops below ~50% frozen. No push.
|
||||
- **Slice 18 — C4: the FramePump blit-on-change composite cache (2026-09-15, take ty-1841
|
||||
verdict):** the 1841 take proved capture fixed (band ~20 fresh updates/s, no tears, pacing
|
||||
clean) but logged a FramePump **stall on EVERY iteration** (`totalMs 21-44`,
|
||||
`render=full-render split=0 elements=6 dynamic=4`, worst render 166ms startup spike) →
|
||||
~22-28 composites/s hard-caps the desktop inside the 60fps file. The SceneGraph split
|
||||
CANNOT fix this wired: `GetSplitPoint` returns **0** because the live-capture backdrop is
|
||||
element 0 and must stay dynamic — a baked capture goes stale — so every tick runs the full
|
||||
re-composite even when no input changed (the compositor's per-layer paste cache already makes
|
||||
identical frames identical per-element; the waste is re-COMPOSITING the whole 8.3MB frame).
|
||||
Same shape as OBS (sources cache their surface, the compositor renders on update only —
|
||||
the pattern this file has cited since the 2026-09-04 paste-cache slice) at the FRAME level.
|
||||
`FramePump.RenderFull` now wraps both full-render call sites (no-SceneGraph + split=0
|
||||
fallback): it computes `BuildFullRenderSignature` (a deterministic hash of the crop/output
|
||||
dims, the social bar identity + top, and — per element, MIRRORING the compositor's own
|
||||
resolution — the element ref, layout/visual bits, and the frame each element would resolve
|
||||
through the SAME resolver seam, using buffer-array identity + Epoch + CropBounds; a changed
|
||||
frame forces a re-composite, an unchanged one never serves stale bytes) and, when it matches
|
||||
the last composite, reuses the cache with ONE `Buffer.BlockCopy` (~3ms) instead. The cache
|
||||
buffer is a **separate long-lived array** — never the scratch pool — because the caller burns
|
||||
the frame counter and recycles the scratch AFTER `RenderScene` returns; the cache is written
|
||||
from the rendered scratch BEFORE returning (pre-burn, pre-recycle). Engagement is gated on
|
||||
the 1:1 config (`SourceRect == Output`, the only deployed tier); the vertical tier's cached
|
||||
fresh-scale-buffer follow-up stays exactly where it was. Telemetry: internal `CacheHits`/
|
||||
`CacheRenders` counters surface as `cache {renders}R/{hits}H` on the 5s stats line, and the
|
||||
ticks' key is an internal `OutputIndex` (burned counter, stable across one tick's resolver
|
||||
passes). **Good Dog test**
|
||||
`FullRenderCache_StaticInputs_RenderOnce_Then_Reuse_UntilInputChanges`: static scene →
|
||||
exactly ONE composite (renders==1, hits>0, byte-identical content above the burn strip),
|
||||
then a new frame (new array + monotonic Epoch) invalidates and propagates. Existing pool test
|
||||
now passes a STABLE scene (production hands `StagedScene`; a fresh scene per tick churns the
|
||||
element refs inside the signature and hides the cache — the tests that care about cache
|
||||
behavior must mirror production) and keys its red/blue alternation on `OutputIndex`/frame
|
||||
count so the tick's two resolver passes (signature + render) don't double-advance a call-count
|
||||
flip. Full suite **297/297 green, 0 warnings**; verify.sh scope-locked to FramePump.cs +
|
||||
FramePumpTests.cs + docs. Target next take: `cache` hits dominate on TV holds, `avg render`
|
||||
drops toward the BlockCopy, stalls vanish, then the band audit + clap re-measure decide push.
|
||||
No push.
|
||||
- **Stop ordering matters:** `StopAsync` stops the encoder — since slice 10 it FLUSHES the pending
|
||||
queue (`Channel.TryComplete` → drain writes the leftovers, closes stdin → EOF → ffmpeg finalizes+exits;
|
||||
an accepted frame is never lost) — **before** awaiting the loop. The old reverse-order deadlock was
|
||||
|
||||
@@ -434,17 +434,23 @@ public class FramePumpTests
|
||||
/// its master buffers across frames (the 8.3MB-per-tick LOH churn that cost GC
|
||||
/// stalls inside "render") while EVERY frame's content stays correct — stale bytes
|
||||
/// from an earlier tick would show as a wrong color. The backdrop alternates
|
||||
/// red/blue per tick (submit snapshots pin the color), and a repeated backing-array
|
||||
/// identity proves the pool actually recycles (an inert pool keeps the pixel
|
||||
/// assertions green while leaving the churn in place).</summary>
|
||||
/// red/blue per OUTPUT frame (keyed on the burned index, which is stable across the
|
||||
/// tick's resolver passes) and a repeated backing-array identity proves the pool
|
||||
/// actually recycles (an inert pool keeps the pixel assertions green while leaving
|
||||
/// the churn in place). The scene instance is STABLE across ticks, like production
|
||||
/// (MainViewModel's StagedScene), so the compositor signature doesn't churn on
|
||||
/// unrelated element identity.</summary>
|
||||
[Fact]
|
||||
public async Task Pump_Pools_ScratchBuffers_Across_Frames_Without_Stale_Pixels()
|
||||
{
|
||||
var red = SceneCompositorTests.Solid(64, 48, 255, 0, 0);
|
||||
var blue = SceneCompositorTests.Solid(64, 48, 0, 0, 255);
|
||||
var flip = 0;
|
||||
var scene = BackgroundScene();
|
||||
var encoder = new FakeEncoder();
|
||||
using var pump = NewPump(encoder, resolve: _ => flip++ % 2 == 0 ? red : blue);
|
||||
FramePump? pumpHolder = null;
|
||||
using var pump = NewPump(encoder, scene: () => scene,
|
||||
resolve: _ => pumpHolder!.OutputIndex % 2 == 0 ? red : blue);
|
||||
pumpHolder = pump;
|
||||
|
||||
await pump.StartAsync();
|
||||
var deadline = DateTime.UtcNow.AddSeconds(10);
|
||||
@@ -462,6 +468,73 @@ public class FramePumpTests
|
||||
"every frame rode a fresh buffer — the scratch pool is inert");
|
||||
}
|
||||
|
||||
/// <summary>The ONE integration test for the C4 blit-on-change cache (2026-09-15):
|
||||
/// the ty-1841 take logged a render stall on EVERY iteration (totalMs 21-44, one
|
||||
/// full composite per ~30ms even though the content moved only ~20 updates/s) — the
|
||||
/// SceneGraph split can't help because the live backdrop sits at element 0 and cannot
|
||||
/// be baked (a cached capture goes stale). A static scene must render the composite
|
||||
/// ONCE then reuse it per tick (re-render == 1, cache hits > 0, byte-identical
|
||||
/// content above the burn strip); a new backdrop frame (new array + monotonic Epoch)
|
||||
/// must invalidate the cache and propagate the changed pixels.</summary>
|
||||
[Fact]
|
||||
public async Task FullRenderCache_StaticInputs_RenderOnce_Then_Reuse_UntilInputChanges()
|
||||
{
|
||||
var red = SceneCompositorTests.Solid(64, 48, 255, 0, 0);
|
||||
var current = red;
|
||||
// STABLE scene across ticks (production hands StagedScene) — a fresh scene per
|
||||
// tick would churn the element references inside the render signature and force
|
||||
// a re-composite every frame, hiding the cache entirely.
|
||||
var scene = BackgroundScene();
|
||||
var encoder = new FakeEncoder();
|
||||
using var pump = NewPump(encoder, scene: () => scene, resolve: _ => current);
|
||||
|
||||
await pump.StartAsync();
|
||||
var deadline = DateTime.UtcNow.AddSeconds(5);
|
||||
while (DateTime.UtcNow < deadline && (encoder.Frames.Count < 8 || pump.CacheHits < 6))
|
||||
await Task.Delay(10);
|
||||
|
||||
Assert.True(encoder.Frames.Count >= 8, $"only {encoder.Frames.Count} frames produced");
|
||||
Assert.Equal(1, pump.CacheRenders); // one composite for the whole static phase
|
||||
Assert.True(pump.CacheHits >= 6, $"cache reused only {pump.CacheHits} times");
|
||||
|
||||
// Every frame's content above the burn strip must be the same — the cache is
|
||||
// serving the identical pre-burn composite; the burn counter varies only the
|
||||
// bottom-right strip (rows 39..47, cols 27..64 in a 64×48 frame).
|
||||
var reference = encoder.Frames[^1].BgraPixels.AsSpan(0, 37 * 64 * 4).ToArray();
|
||||
Assert.All(encoder.Frames, f => Assert.True(
|
||||
f.BgraPixels.AsSpan(0, 37 * 64 * 4).SequenceEqual(reference),
|
||||
"a static input re-composited a different frame — the cache served stale bytes or never engaged"));
|
||||
|
||||
// Mutate: new backing array + Epoch guarantees a full signature miss.
|
||||
var blue = new VideoFrame(64, 48, new byte[64 * 48 * 4]) { Epoch = 42 };
|
||||
for (var i = 0; i < blue.BgraPixels.Length; i += 4)
|
||||
{
|
||||
blue.BgraPixels[i] = 255; // B
|
||||
blue.BgraPixels[i + 3] = 255; // A
|
||||
}
|
||||
current = blue;
|
||||
|
||||
VideoFrame? changed = null;
|
||||
deadline = DateTime.UtcNow.AddSeconds(5);
|
||||
while (DateTime.UtcNow < deadline && changed == null)
|
||||
{
|
||||
foreach (var f in encoder.Frames)
|
||||
{
|
||||
if (!f.BgraPixels.AsSpan(0, 37 * 64 * 4).SequenceEqual(reference))
|
||||
{
|
||||
changed = f;
|
||||
break;
|
||||
}
|
||||
}
|
||||
if (changed == null) await Task.Delay(10);
|
||||
}
|
||||
Assert.NotNull(changed);
|
||||
AssertColor(changed!, 0, 0, 0, 0, 255); // the new backdrop propagated
|
||||
Assert.True(pump.CacheRenders >= 2, "the changed input never re-composited");
|
||||
|
||||
await pump.StopAsync();
|
||||
}
|
||||
|
||||
/// <summary>One integration test for the take-8 off-UI fix: the pump loop must NOT
|
||||
/// run on the starting thread's SynchronizationContext (StartAsync fires from a UI
|
||||
/// command handler; the old loop inherited it, so every frame competed with the
|
||||
|
||||
Reference in New Issue
Block a user