gramps/LlamaCasty
0
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

perf(compositor): slice 5 — paste cache for non-opaque layers (take-7/8 data)

Browse Source 
The stamped build settled what slices 3-4 could not: chat cache works (resolve
~0.0ms) but render stayed 26-27ms -> 124-135/300. The cost was the compositor
re-rasterizing EVERY layer every tick: this Live scene re-samples chat (159k) +
web widget (271k) + image (95k) + cam (156k) ~ 680k px @ ~38ns — for layers
whose pixels do not change between chat/web/cam updates.

OBS shape: cache the surface, paste per tick. BlitCachedLayer rasterizes a
non-opaque layer ONCE into an element-space, transparent-based frame keyed by
(source-array identity, src W/H, ceil'd dst rect, round, mirror), then pastes:
integer position, row alpha-blend, opacity applied at paste. Producers hand out
fresh immutable arrays -> array-identity keys cannot serve stale content; dict
bounded (48, clears whole). Drag/opacity live in paste params, not keys, so
editing stops triggering resamples too. Opaque backdrop keeps the memcpy path;
the webcam keeps the direct path via its IsOpaque flag (revisit if take 9 is
borderline).

ONE integration test: PasteCache_RepeatRender_IsByteIdentical_And_ContentChange-
Propagates (byte-exact raster-vs-paste incl. round-clip margins, new-array
propagation); existing pixel suite guards sampler semantics. 85/85 across
compositor/pump/chat/capture/session classes, clean build 0 warnings. Also:
BuildStampTests.cs was written last commit but never staged — its own scope-check
slip, added here (the run had used the on-disk file; tracked now).

Docs same commit: ai.md slice 5 + stale 'general path only 130k' claim corrected,
TASKS.md take-9 gate, MyMistakes recipe (prove the stage; a fix that doesn't move
the stat wasn't the bottleneck), HANDOFF. take 9 expectation: 300/300, render
<= ~8ms -> saga closes, Unit B starts.
This commit is contained in:
Llama Chile Shop
2026-09-04 11:44:19 -07:00
parent 27bf74389d
commit 6af2026906
7 changed files with 258 additions and 16 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Services/Compositor/SceneCompositor.cs
+101 -2
View File
@@ -14,6 +14,19 @@ namespace ytLive.Services.Compositor;
/// </summary>
public sealed class SceneCompositor
{
// Paste cache (take-8 fix, 2026-09-04): the Live scene re-sampled ~680k bilinear
// pixels EVERY tick even though only the camera actually changes at that rate —
// chat/web/image layers are identical frame after frame. Same shape as OBS (the
// source surface is cached, the compositor blits): a scaled/round-clipped/mirrored
// layer rasterizes once and is pasted (row-copy/blend) on every later tick.
// Keyed by the SOURCE FRAME IDENTITY — every producer (capture, webcam, chat
// renderer, web capture, static cache) hands out immutable byte[], so "same pixels
// array + same target rect" ⇒ same raster. Bounds-checked eviction clears wholesale.
private sealed record PasteKey(byte[] Pixels, int SrcW, int SrcH, int DstW, int DstH, bool Round, bool Mirror);
private static readonly Dictionary<PasteKey, VideoFrame> _pasteCache = new();
private static readonly object _pasteGate = new();
private const int MaxPasteEntries = 48;
/// <summary>
/// Composite <paramref name="scene"/> into the tier's output frame. Layer order (back →
/// front): live background (the scene's <c>IsBackground</c> source) → background image →
@@ -240,12 +253,84 @@ public sealed class SceneCompositor
var ew = (float)element.Width;
var eh = (float)element.Height;
var isRound = element.ClipShape == ClipShape.Round;
BlitContent(buffer, cropW, cropH, ex, ey, ew, eh, frame,
(float)element.Opacity, isRound, element.IsMirrored);
if (frame.IsOpaque && element.Opacity >= 1f)
{
// Opaque full-strength layer (the live backdrop): raster it straight, no alpha path.
BlitContent(buffer, cropW, cropH, ex, ey, ew, eh, frame, 1f, isRound, element.IsMirrored);
}
else
{
BlitCachedLayer(buffer, cropW, cropH, ex, ey, ew, eh, frame, element, isRound);
}
if (element.HasBorder)
DrawBorder(buffer, cropW, cropH, ex, ey, ew, eh, element, isRound);
}
/// <summary>Non-opaque layer blit through the paste cache: the element-space raster
/// (scaled to the element rect, round-clipped, mirrored) depends only on the source
/// frame + rect + shape/mirror — NOT on position or opacity. Static content (chat,
/// web overlays, images, the media frame between updates) is sampled ONCE and pasted
/// at integer position with per-tixel blend; only layers whose source frame actually
/// changed (webcam at device fps) pay resampling.</summary>
private static void BlitCachedLayer(
byte[] buffer, int cropW, int cropH,
float ex, float ey, float ew, float eh,
VideoFrame frame, SceneElement element, bool isRound)
{
if (ew <= 0 || eh <= 0) return;
var dw = (int)System.Math.Ceiling(ew);
var dh = (int)System.Math.Ceiling(eh);
if (dw <= 0 || dh <= 0) return;
var mirror = element.IsMirrored;
PasteKey? key = null;
VideoFrame? raster = null;
lock (_pasteGate)
{
key = new PasteKey(frame.BgraPixels, frame.Width, frame.Height, dw, dh, isRound, mirror);
_pasteCache.TryGetValue(key, out raster);
}
if (raster == null)
{
// Rasterize in ELEMENT space on a TRANSPARENT base: the sampler writes 255
// alpha only where it actually draws, so untouched margins (outside the
// round clip, past the UniformToFill edges) stay alpha 0 and the paste
// reveals the master beneath instead of black.
var pixels = new byte[dw * dh * 4];
BlitContentRaw(pixels, dw, dh, 0, 0, ew, eh, frame, 1f, isRound, mirror);
raster = new VideoFrame(dw, dh, pixels);
lock (_pasteGate)
{
if (_pasteCache.Count >= MaxPasteEntries) _pasteCache.Clear();
_pasteCache[key!] = raster;
}
}
// Paste at integer position; opacity blends. The raster is never flagged
// IsOpaque — round clips/margins carry alpha 0 — so the row blend always runs.
var op = (int)(System.Math.Clamp(element.Opacity, 0f, 1f) * 256);
if (op <= 0) return;
var px = (int)System.Math.Floor(ex);
var py = (int)System.Math.Floor(ey);
var x0 = System.Math.Max(0, px);
var y0 = System.Math.Max(0, py);
var x1 = System.Math.Min(cropW, px + dw);
var y1 = System.Math.Min(cropH, py + dh);
if (x0 >= x1 || y0 >= y1) return;
var d = buffer.AsSpan();
var s = raster.BgraPixels;
for (var y = y0; y < y1; y++)
{
var si = ((y - py) * dw + (x0 - px)) * 4;
var len = x1 - x0;
if (op == 256)
BlendRowOpaque(d, (y * cropW + x0) * 4, s.AsSpan(si, len * 4), 0, len);
else
BlendRowWeighted(d, (y * cropW + x0) * 4, s.AsSpan(si, len * 4), 0, len, op);
}
}
private static VideoFrame Overlay(VideoFrame frame, CompositorOptions options, VideoFrame overlay, int sx0, int sy0)
{
var cropW = options.SourceRectWidth;
@@ -311,6 +396,20 @@ public sealed class SceneCompositor
return;
}
BlitContentRaw(dst, dstW, dstH, ex, ey, ew, eh, src, opacity, isRound, isMirror);
}
/// <summary>The general scaled/clipped/mirrored sampler (integer fixed-point
/// bilinear + source-over). Also rasterizes paste-cache entries in ELEMENT space
/// onto a TRANSPARENT base (alpha 0) so pasting over the master keeps the round
/// clip and transparency of the untouched margins.</summary>
private static void BlitContentRaw(
byte[] dst, int dstW, int dstH,
float ex, float ey, float ew, float eh,
VideoFrame src, float opacity, bool isRound, bool isMirror)
{
if (ew <= 0 || eh <= 0) return;
var (scale, ox, oy) = StretchMath.UniformToFill(ew, eh, src.Width, src.Height);
var drawnW = src.Width * scale;
var drawnH = src.Height * scale;
var radius = Math.Min(ew, eh) / 2f;