gramps/LlamaCasty
0
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Per-element composite timing + scene dump (measurement only)

Browse Source 
The cadence work ended with a question the log could not answer: the compositor
manages 6.8 composites/s at ~141ms while the 10:03 take did 29-34ms on the "same
scene shape" (full-render split=0 elements=6 dynamic=5). Nothing in the stats line
said WHICH of the six elements cost what, and nothing recorded the scene itself, so
the two takes could only be compared by eye. This closes that gap and nothing else:
no fix is attempted until a real take names the culprit.

- Services/Compositor/CompositeTimings.cs (new): per-window collector.
  Add(stage, ticks) accumulates; Describe(renders, minMs = 0.1) prints the averaged
  breakdown slowest-first, dropping sub-0.1ms noise.
- SceneCompositor.Render / CompositeLayers / BakeStaticBase / BlitBakedElement /
  BlitCachedLayer + SceneGraph.GetBakedBase: trailing optional CompositeTimings?
  Every stage is timed (backdrop, base, flash, social, ticker, zoom, scale, overlay,
  bake backdrop, bake overlays) and EVERY element gets its own entry keyed by
  z-index + name ("#2 Webcam"), with the nested "raster" sub-stage recording the
  paste-cache rasterize miss - the split between "re-samples every tick" and "slow
  paste" without a second session. Bake-pass elements are prefixed "bake #..." so a
  one-off rebake cannot masquerade as a per-composite cost. Callers that pass
  nothing (thumbnails, existing tests) are unaffected and pay nothing.
- FramePump: owns the collector, threads it through RenderScene/RenderFull/
  GetBakedBase/CompositeLayers, and logs "FramePump parts (N renders/5s): ..." once
  per 5s window beside the stats line (averaged per render, so it compares directly
  with "avg render"), then resets it. Also dumps the scene - "FramePump scene dump:"
  header plus one line per element (z-index, name, kind, source type, visibility,
  rect, opacity, clip) - at pump start and on every scene change.
- One Good Dog integration test (ytLive.Tests/CompositeTimingTests.cs): a real
  composite of a full-canvas element plus a 64x64 one through the real collector
  must rank the full-canvas element first, carry the raster sub-stage and the fixed
  stages, and the dump must name both elements with z-index/type/rect and echo the
  split point.

Derivative work (AGENTS.md): OBS's answer to "which source is slow" is the Source
Profiler - average execution time of each source's tick and render functions
(https://docs.obsproject.com/reference-libobs-util-source-profiler, util/profiler.h).
Same shape here: one timer per element per composite, reported as an average, plus
the stage split this case needs.

Docs in the same commit: ai.md (new subsection "Per-element composite timing + scene
dump", the closed gap corrected in "Frame cadence"), Services/index.md (new file row
+ FramePump/SceneCompositor rows), TASKS.md (unit marked shipped, next = creator
take), HANDOFF.md (rewritten: unpushed commit, next step, verify.sh whitelist hole).

Also recorded this session (from the creator): the launch command is
`dotnet run -c Release` - Release is the DEFAULT, Debug only when explicitly named,
and perf takes must read Release. ai.md -> Run updated.

Full suite 428 tests, 0 warnings; the only failures are the known audio env flakes
(both pass in isolation). Scope check passed.
This commit is contained in:
gramps
2026-10-16 18:23:20 -07:00
parent 3cb29ea45c
commit a3a4681aac
9 changed files with 408 additions and 60 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Services/Compositor/CompositeTimings.cs
+76
View File
@@ -0,0 +1,76 @@
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
namespace ytLive.Services.Compositor;
/// <summary>
/// Per-stage composite timing (webcam-cadence unit, 2026-10-16): wall time per render
/// pass, split into named stages — the backdrop, every element (keyed by z-index +
/// name), the global overlays, and the final output scale. The pump owns one instance
/// for the whole session, passes it into <see cref="SceneCompositor.Render"/> /
/// <c>CompositeLayers</c> / <c>BakeStaticBase</c>, and logs an averaged breakdown every
/// stats window — the "which of the 6 elements eats ~140ms" reading the old stats line
/// could not give (it reported render/acquire/resolve, never the culprit layer).
/// <para>Stage names are stable and greppable: <c>backdrop</c>, <c>base</c>,
/// <c>flash</c>, <c>social</c>, <c>ticker</c>, <c>zoom</c>, <c>scale</c>,
/// <c>bake backdrop</c>, <c>bake overlays</c>, and <c>#&lt;index&gt; &lt;name&gt;</c> per
/// element (bake-pass elements are prefixed <c>bake</c> so a one-off rebake cannot
/// masquerade as a per-composite cost). A <c>#&lt;i&gt; &lt;name&gt; raster</c> sub-stage NESTS inside its parent's total
/// (it is the paste-cache rasterize miss — the prime suspect when one element costs a
/// full resample every tick). Stage time INCLUDES that stage's frame resolve; resolve
/// is separately totalled in the stats line.</para>
/// <para>Single writer in production (the pump's render thread — every blit joins its
/// <see cref="StripPool"/> workers before returning); the lock only makes a test able
/// to share one across threads.</para>
/// </summary>
public sealed class CompositeTimings
{
private readonly object _gate = new();
private readonly Dictionary<string, long> _stages = new(StringComparer.Ordinal);
/// <summary>Accumulate one stage sample. Non-positive ticks are dropped — a
/// skipped layer must not create a zero-cost entry that dilutes the reading.</summary>
public void Add(string stage, long ticks)
{
if (ticks <= 0 || stage.Length == 0) return;
lock (_gate)
{
_stages.TryGetValue(stage, out var cur);
_stages[stage] = cur + ticks;
}
}
/// <summary>Clear every stage — called once per stats window after the parts line
/// is logged, so each window reports its own breakdown.</summary>
public void Reset()
{
lock (_gate) _stages.Clear();
}
/// <summary>Average milliseconds per render for every stage at or above
/// <paramref name="minMs"/>, sorted slowest-first:
/// <c>"#2 Webcam 88.1ms, backdrop 61.2ms, #2 Webcam raster 84.0ms"</c>.
/// <paramref name="renders"/> is the divisor (the window's render count, so the
/// numbers compare directly with the stats line's <c>avg render</c>). The floor
/// keeps a healthy pipeline from burying the contender in zeroes; the empty string
/// means nothing was composited (the fully-static shortcut and C4 cache hits do no
/// per-element work — an empty parts line there is the correct reading).</summary>
public string Describe(int renders, double minMs = 0.1)
{
List<KeyValuePair<string, long>> snap;
lock (_gate)
{
if (_stages.Count == 0) return string.Empty;
snap = _stages.ToList();
}
var freq = (double)Stopwatch.Frequency;
var parts = snap
.Select(kv => (Stage: kv.Key, Ms: kv.Value / freq * 1000.0 / System.Math.Max(1, renders)))
.Where(p => p.Ms >= minMs)
.OrderByDescending(p => p.Ms)
.Select(p => $"{p.Stage} {p.Ms:F1}ms");
return string.Join(", ", parts);
}
}
Services/Compositor/SceneCompositor.cs
+72 -17
View File
@@ -54,7 +54,8 @@ public sealed class SceneCompositor
VideoFrame? staticBase = null,
int split = -1,
byte[]? scratch = null,
VideoFrame? tickerFrame = null)
VideoFrame? tickerFrame = null,
CompositeTimings? timings = null)
{
if (scene == null) throw new ArgumentNullException(nameof(scene));
if (frameFor == null) throw new ArgumentNullException(nameof(frameFor));
@@ -76,12 +77,15 @@ public sealed class SceneCompositor
{
if (staticBase.Width != cropW || staticBase.Height != cropH)
throw new ArgumentException("The static base must match the source rect.", nameof(staticBase));
var tBase = System.Diagnostics.Stopwatch.GetTimestamp();
Buffer.BlockCopy(staticBase.BgraPixels, 0, buffer, 0, buffer.Length);
timings?.Add("base", System.Diagnostics.Stopwatch.GetTimestamp() - tBase);
}
else
{
if (split < 0) split = scene.Elements.Count;
var tBackdrop = System.Diagnostics.Stopwatch.GetTimestamp();
var elements = scene.Elements;
var liveCapture = elements.OfType<Source>().FirstOrDefault(s => s.IsBackground);
var liveCaptureFrame = liveCapture != null && liveCapture.IsVisible ? frameFor(liveCapture) : null;
@@ -149,8 +153,11 @@ public sealed class SceneCompositor
if (backgroundFrame != null)
BlitContent(buffer, cropW, cropH, 0, 0, cropW, cropH, backgroundFrame, 1f, false, false);
foreach (var element in elements)
timings?.Add("backdrop", System.Diagnostics.Stopwatch.GetTimestamp() - tBackdrop);
for (var ei = 0; ei < elements.Count; ei++)
{
var element = elements[ei];
if (!element.IsVisible) continue;
switch (element)
{
@@ -162,7 +169,7 @@ public sealed class SceneCompositor
}
var fullFrame = frameFor(element);
if (fullFrame == null) continue;
BlitBakedElement(buffer, cropW, cropH, options, element, fullFrame);
BlitBakedElement(buffer, cropW, cropH, options, element, fullFrame, timings, $"#{ei} {element.Name}");
}
}
@@ -181,17 +188,29 @@ public sealed class SceneCompositor
}
var frame = frameFor(element);
if (frame == null) continue;
BlitBakedElement(buffer, cropW, cropH, options, element, frame);
BlitBakedElement(buffer, cropW, cropH, options, element, frame, timings, $"#{i} {element.Name}");
}
if (flashFrame != null)
{
var tFlash = System.Diagnostics.Stopwatch.GetTimestamp();
BlitOverlay(buffer, cropW, cropH, options, flashFrame, 0, 0);
timings?.Add("flash", System.Diagnostics.Stopwatch.GetTimestamp() - tFlash);
}
if (socialBarFrame != null && socialBarFrame.Width > 0 && socialBarFrame.Height > 0)
{
var tSocial = System.Diagnostics.Stopwatch.GetTimestamp();
BlitOverlay(buffer, cropW, cropH, options, socialBarFrame, 0, socialBarTop);
timings?.Add("social", System.Diagnostics.Stopwatch.GetTimestamp() - tSocial);
}
if (tickerFrame != null && tickerFrame.Width > 0 && tickerFrame.Height > 0)
{
var tTicker = System.Diagnostics.Stopwatch.GetTimestamp();
BlitOverlay(buffer, cropW, cropH, options, tickerFrame, tickerFrame.OriginX, tickerFrame.OriginY);
timings?.Add("ticker", System.Diagnostics.Stopwatch.GetTimestamp() - tTicker);
}
// The zoom box is layer n+1: a live 2x view of a FIXED capture window sampled
// from the frame just composited (all layers). Drawn LAST — above everything.
@@ -199,10 +218,17 @@ public sealed class SceneCompositor
// the box; the content it shows never changes (creator ruling 2026-10-08).
var zoom = scene.Elements.OfType<ZoomSceneElement>().FirstOrDefault(z => z.IsZoomActive);
if (zoom != null)
{
var tZoom = System.Diagnostics.Stopwatch.GetTimestamp();
BlitZoom(buffer, cropW, cropH, options, zoom);
timings?.Add("zoom", System.Diagnostics.Stopwatch.GetTimestamp() - tZoom);
}
return StretchMath.BilinearScale(
var tScale = System.Diagnostics.Stopwatch.GetTimestamp();
var scaled = StretchMath.BilinearScale(
new VideoFrame(cropW, cropH, buffer), options.OutputWidth, options.OutputHeight);
timings?.Add("scale", System.Diagnostics.Stopwatch.GetTimestamp() - tScale);
return scaled;
}
/// <summary>
@@ -220,12 +246,14 @@ public sealed class SceneCompositor
Func<CompositorOptions> options,
VideoFrame? socialBarFrame = null,
int socialBarTop = 0,
VideoFrame? flashFrame = null)
VideoFrame? flashFrame = null,
CompositeTimings? timings = null)
{
if (split <= 0) return null;
var opts = options();
if (opts.SourceRectWidth <= 0 || opts.SourceRectHeight <= 0) return null;
var tBake = System.Diagnostics.Stopwatch.GetTimestamp();
var cropW = opts.SourceRectWidth;
var cropH = opts.SourceRectHeight;
var buffer = new byte[cropW * cropH * 4];
@@ -250,6 +278,8 @@ public sealed class SceneCompositor
BlitContent(buffer, cropW, cropH, 0, 0, cropW, cropH, backgroundFrame, 1f, false, false);
}
timings?.Add("bake backdrop", System.Diagnostics.Stopwatch.GetTimestamp() - tBake);
for (var i = 0; i < split; i++)
{
var element = scene.Elements[i];
@@ -264,14 +294,19 @@ public sealed class SceneCompositor
}
var frame = frameFor(element);
if (frame == null) continue;
BlitBakedElement(buffer, cropW, cropH, opts, element, frame);
BlitBakedElement(buffer, cropW, cropH, opts, element, frame, timings, $"bake #{i} {element.Name}");
}
var baked = new VideoFrame(cropW, cropH, buffer);
if (flashFrame != null)
baked = Overlay(baked, opts, flashFrame, 0, 0);
if (socialBarFrame != null && socialBarFrame.Width > 0 && socialBarFrame.Height > 0)
baked = Overlay(baked, opts, socialBarFrame, 0, socialBarTop);
if (flashFrame != null || (socialBarFrame != null && socialBarFrame.Width > 0 && socialBarFrame.Height > 0))
{
var tOverlays = System.Diagnostics.Stopwatch.GetTimestamp();
if (flashFrame != null)
baked = Overlay(baked, opts, flashFrame, 0, 0);
if (socialBarFrame != null && socialBarFrame.Width > 0 && socialBarFrame.Height > 0)
baked = Overlay(baked, opts, socialBarFrame, 0, socialBarTop);
timings?.Add("bake overlays", System.Diagnostics.Stopwatch.GetTimestamp() - tOverlays);
}
return baked;
}
@@ -291,7 +326,8 @@ public sealed class SceneCompositor
int socialBarTop = 0,
VideoFrame? flashFrame = null,
byte[]? scratch = null,
VideoFrame? tickerFrame = null)
VideoFrame? tickerFrame = null,
CompositeTimings? timings = null)
{
var cropW = options.SourceRectWidth;
var cropH = options.SourceRectHeight;
@@ -299,7 +335,9 @@ public sealed class SceneCompositor
var buffer = scratch != null && scratch.Length == baseFrame.BgraPixels.Length
? scratch
: new byte[baseFrame.BgraPixels.Length];
var tBase = System.Diagnostics.Stopwatch.GetTimestamp();
Buffer.BlockCopy(baseFrame.BgraPixels, 0, buffer, 0, baseFrame.BgraPixels.Length);
timings?.Add("base", System.Diagnostics.Stopwatch.GetTimestamp() - tBase);
for (var i = split; i < scene.Elements.Count; i++)
{
@@ -315,22 +353,35 @@ public sealed class SceneCompositor
}
var frame = frameFor(element);
if (frame == null) continue;
BlitBakedElement(buffer, cropW, cropH, options, element, frame);
BlitBakedElement(buffer, cropW, cropH, options, element, frame, timings, $"#{i} {element.Name}");
}
if (tickerFrame != null && tickerFrame.Width > 0 && tickerFrame.Height > 0)
{
var tTicker = System.Diagnostics.Stopwatch.GetTimestamp();
BlitOverlay(buffer, cropW, cropH, options, tickerFrame, tickerFrame.OriginX, tickerFrame.OriginY);
timings?.Add("ticker", System.Diagnostics.Stopwatch.GetTimestamp() - tTicker);
}
var zoom = scene.Elements.OfType<ZoomSceneElement>().FirstOrDefault(z => z.IsZoomActive);
if (zoom != null)
{
var tZoom = System.Diagnostics.Stopwatch.GetTimestamp();
BlitZoom(buffer, cropW, cropH, options, zoom);
timings?.Add("zoom", System.Diagnostics.Stopwatch.GetTimestamp() - tZoom);
}
return StretchMath.BilinearScale(new VideoFrame(cropW, cropH, buffer), options.OutputWidth, options.OutputHeight);
var tScale = System.Diagnostics.Stopwatch.GetTimestamp();
var scaled = StretchMath.BilinearScale(new VideoFrame(cropW, cropH, buffer), options.OutputWidth, options.OutputHeight);
timings?.Add("scale", System.Diagnostics.Stopwatch.GetTimestamp() - tScale);
return scaled;
}
private static void BlitBakedElement(
byte[] buffer, int cropW, int cropH, CompositorOptions options, SceneElement element, VideoFrame frame)
byte[] buffer, int cropW, int cropH, CompositorOptions options, SceneElement element, VideoFrame frame,
CompositeTimings? timings, string stage)
{
var tElement = System.Diagnostics.Stopwatch.GetTimestamp();
var ex = (float)(element.X - options.SourceRectX);
var ey = (float)(element.Y - options.SourceRectY);
var ew = (float)element.Width;
@@ -342,9 +393,10 @@ public sealed class SceneCompositor
// the device frame had not changed; an opaque source rasterizes to an opaque
// raster whose paste is a per-pixel copy branch, so hits beat the sampler and
// misses cost exactly what the bypass cost.
BlitCachedLayer(buffer, cropW, cropH, ex, ey, ew, eh, frame, element, isRound);
BlitCachedLayer(buffer, cropW, cropH, ex, ey, ew, eh, frame, element, isRound, timings, stage);
if (element.HasBorder)
DrawBorder(buffer, cropW, cropH, ex, ey, ew, eh, element, isRound);
timings?.Add(stage, System.Diagnostics.Stopwatch.GetTimestamp() - tElement);
}
/// <summary>Non-opaque layer blit through the paste cache: the element-space raster
@@ -356,7 +408,8 @@ public sealed class SceneCompositor
private static void BlitCachedLayer(
byte[] buffer, int cropW, int cropH,
float ex, float ey, float ew, float eh,
VideoFrame frame, SceneElement element, bool isRound)
VideoFrame frame, SceneElement element, bool isRound,
CompositeTimings? timings, string stage)
{
if (ew <= 0 || eh <= 0) return;
var dw = (int)System.Math.Ceiling(ew);
@@ -374,6 +427,7 @@ public sealed class SceneCompositor
if (raster == null)
{
var tRaster = System.Diagnostics.Stopwatch.GetTimestamp();
// Rasterize in ELEMENT space on a TRANSPARENT base: the sampler writes 255
// alpha only where it actually draws, so untouched margins (outside the
// round clip, past the UniformToFill edges) stay alpha 0 and the paste
@@ -398,6 +452,7 @@ public sealed class SceneCompositor
_pasteCache[key!] = raster;
_elementRasters[element] = raster;
}
timings?.Add(stage + " raster", System.Diagnostics.Stopwatch.GetTimestamp() - tRaster);
}
// Paste at integer position; opacity blends. The raster is never flagged