gramps/LlamaCasty
0
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

UNIT 0: make a dropped webcam frame measurable, not just inferable

Browse Source 
The creator is DHH and lipreads their own recordings, so a webcam that
judders is an accessibility defect, not a polish item. Take
ty-20261005-1001-821-0001.mp4 holds 25.6 distinct webcam images/s against
the camera's 30fps with 281 gaps of 50-100ms - roughly one hitch in five -
in a file that muxes flawlessly. Nothing in the app could see that: rawvideo
carries no per-frame timestamps (ffmpeg stamps by ARRIVAL at the declared
60fps), and the stats line divided every average by the number of frames
SUBMITTED, which deadline padding always fills to the full target. So the
pump reported "300/300 frames, avg render 12.4ms" while producing 60
distinct composites.

Helpers/FrameCadence.cs is one pure counter fed from three points:
MediaCaptureFrameSource logs the device's own SystemRelativeTime stamps
against our arrival times (TryAcquireLatestFrame keeps only the LATEST
frame, so falling behind is silent by design), and FramePump logs the
distinct webcam Epoch values that reached a composite. Read together they
name the culprit - device/USB, our reader, or our compositor - instead of
guessing. Render/resolve/wait now divide by the real render count, and
"frames per 5s from R renders" is the duplicate-on-lag ratio stated
honestly.

An out-of-order timestamp counts as a frame but produces no gap and does not
become the next interval's reference, so one bad stamp cannot invent a
multi-second hitch twice.

Also corrects a stale ai.md line (the "webcam renders at ~18fps" figure
came from a different session and counted every newly rendered layer) and
re-records the background-removal ruling as permanently out. Records the
agreed-but-unbuilt Units 1-2, the no-fallback-camera ruling, and the
SetFormatAsync dead end found by probing the 19041 projection.

Suite 402/402, 0 warnings. The one red, LayerReorderPersistenceTests'
RealMouseDrag, fails identically on a stashed clean tree - it needs an
interactive desktop session this shell lacks.
This commit is contained in:
gramps
2026-10-05 11:10:22 -07:00
parent b2f219739d
commit 55ca1029a3
9 changed files with 532 additions and 30 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Helpers/FrameCadence.cs
+123
View File
@@ -0,0 +1,123 @@
using System;
using System.Diagnostics;
namespace ytLive.Helpers;
/// <summary>
/// Arrival cadence for one window of a timestamped frame stream: how many frames
/// arrived, how far apart they were, and how many gaps were long enough to read
/// as a hitch. Fed any shared monotonic clock (<see cref="Stopwatch.GetTimestamp"/>
/// values, or a device's system-relative stamps on the same clock) and read through
/// <see cref="Summarize"/>.
///
/// Why it exists: a 60fps container makes a missing camera frame indistinguishable
/// from a padded repeat. rawvideo carries no per-frame timestamps — ffmpeg stamps
/// frames by ARRIVAL at the declared rate — so the OBS "duplicate on lag" loop
/// (FramePump) emits 300 authored frames per 5s whether the renderer managed 300
/// composites or 84. The recorded file therefore cannot name the stage that lost
/// the frames; only these counters can. The same counter run twice tells the two
/// apart: the camera's OWN timestamps vs our ARRIVAL timestamps. Steady camera
/// stamps with ragged arrivals means WE dropped it (compositor/reader); ragged in
/// both means the camera, USB link, or driver did.
/// </summary>
public sealed class FrameCadence
{
/// <summary>Default hitch threshold in ms. 50ms is ~1.5x a 30fps period — a
/// missed 30fps frame arrives as ~66ms — so this counts dropped frames without
/// flagging ordinary jitter (a 33ms frame can legitimately measure 35ms).</summary>
public const double DefaultHitchThresholdMs = 50;
private readonly double _hitchThresholdMs;
private long _firstTicks;
private long _previousTicks;
private double _gapTicksTotal;
private int _frames;
private int _gaps;
private int _hitches;
private double _longestGapMs;
public FrameCadence(double hitchThresholdMs = DefaultHitchThresholdMs)
{
if (hitchThresholdMs <= 0)
throw new ArgumentOutOfRangeException(nameof(hitchThresholdMs), hitchThresholdMs,
"Hitch threshold must be positive.");
_hitchThresholdMs = hitchThresholdMs;
}
public int Frames => _frames;
public int Gaps => _gaps;
public int Hitches => _hitches;
public double HitchThresholdMs => _hitchThresholdMs;
/// <summary>Records one frame at <paramref name="timestampTicks"/> on the shared
/// monotonic clock. Monotonicity is asserted, not trusted: an out-of-order stamp
/// (a device whose clock domain differs from ours, a wrapped counter, a dropped
/// timestamp) is counted as a frame but produces no gap AND does not become the
/// reference for the next one — otherwise a single bad stamp would poison both
/// its own interval and the one after it, inventing a multi-second "hitch".</summary>
public void Add(long timestampTicks)
{
if (_frames == 0)
{
_firstTicks = timestampTicks;
_previousTicks = timestampTicks;
_frames++;
return;
}
var delta = timestampTicks - _previousTicks;
_frames++;
if (delta <= 0) return;
var gapMs = delta * 1000d / Stopwatch.Frequency;
_gaps++;
_gapTicksTotal += gapMs;
if (gapMs > _longestGapMs) _longestGapMs = gapMs;
if (gapMs >= _hitchThresholdMs) _hitches++;
_previousTicks = timestampTicks;
}
public FrameCadenceSummary Summarize()
{
var spanMs = _frames >= 2
? (_previousTicks - _firstTicks) * 1000d / Stopwatch.Frequency
: 0d;
// Rate is gaps-per-span, not frames-per-span: N frames span N-1 periods, and
// counting the extra interval would report one extra frame per window.
var rate = _gaps > 0 && spanMs > 0 ? _gaps / (spanMs / 1000d) : 0d;
return new FrameCadenceSummary(
_frames, rate, spanMs, _gaps, _hitches, _longestGapMs,
_gaps > 0 ? _gapTicksTotal / _gaps : 0d, _hitchThresholdMs);
}
/// <summary>Starts a fresh window, keeping the configured threshold.</summary>
public void Reset()
{
_firstTicks = 0;
_previousTicks = 0;
_gapTicksTotal = 0;
_frames = 0;
_gaps = 0;
_hitches = 0;
_longestGapMs = 0;
}
}
/// <summary>One measured cadence window — see <see cref="FrameCadence"/>.</summary>
public readonly record struct FrameCadenceSummary(
int Frames,
double RatePerSecond,
double SpanMs,
int Gaps,
int Hitches,
double LongestGapMs,
double AverageGapMs,
double HitchThresholdMs)
{
/// <summary>One-line log form. Identical in shape from every producer (camera
/// source, frame pump) so two lines can be read against each other: same window
/// length, same fields, same order.</summary>
public string Describe(string label) => Frames == 0
? $"{label} no frames"
: $"{label} {Frames}f/{SpanMs / 1000d:F1}s {RatePerSecond:F1}/s, {Gaps} gaps, " +
$"{Hitches} >= {HitchThresholdMs:F0}ms, avg {AverageGapMs:F1}ms, longest {LongestGapMs:F1}ms";
}
Helpers/index.md
+1
View File
@@ -14,6 +14,7 @@ Cross-cutting utilities. See [`schema.md`](../schema.md) for the memory-map conv
| `InverseBoolToVisibilityConverter.cs` / `NotNullToVisibilityConverter.cs` | XAML value converters for visibility bindings |
| `AllTrueToVisibilityConverter.cs` | Multi-value AND-gate → Visible; gates the layer-row capture menu items on row-IsBackground + staged-scene-is-Live (TASK 25) |
| `CameraConflictProbe.cs` | Best-effort diagnostic: when a camera won't start, enumerates running processes known to hold cameras (OBS, Zoom, Teams, NVIDIA Broadcast, browsers, etc.) — Windows doesn't expose "which process has this device" via any public API, so this is a suspect list, not a verdict |
| `FrameCadence.cs` | Pure arrival-cadence counter (`FrameCadence` + `FrameCadenceSummary.Describe`): frames, gaps, hitches, longest/avg gap for a window of a timestamped stream. Fed by `MediaCaptureFrameSource` (camera stamps vs our arrivals) and `FramePump` (distinct webcam Epochs that reached the output) — UNIT 0, see ai.md "Frame cadence" |
Related: [`Themes/Controls.xaml`](../Themes/Controls.xaml) styles the lists this
class backs; [`Models/index.md`](../Models/index.md) and