UNIT 0: make a dropped webcam frame measurable, not just inferable
The creator is DHH and lipreads their own recordings, so a webcam that judders is an accessibility defect, not a polish item. Take ty-20261005-1001-821-0001.mp4 holds 25.6 distinct webcam images/s against the camera's 30fps with 281 gaps of 50-100ms - roughly one hitch in five - in a file that muxes flawlessly. Nothing in the app could see that: rawvideo carries no per-frame timestamps (ffmpeg stamps by ARRIVAL at the declared 60fps), and the stats line divided every average by the number of frames SUBMITTED, which deadline padding always fills to the full target. So the pump reported "300/300 frames, avg render 12.4ms" while producing 60 distinct composites. Helpers/FrameCadence.cs is one pure counter fed from three points: MediaCaptureFrameSource logs the device's own SystemRelativeTime stamps against our arrival times (TryAcquireLatestFrame keeps only the LATEST frame, so falling behind is silent by design), and FramePump logs the distinct webcam Epoch values that reached a composite. Read together they name the culprit - device/USB, our reader, or our compositor - instead of guessing. Render/resolve/wait now divide by the real render count, and "frames per 5s from R renders" is the duplicate-on-lag ratio stated honestly. An out-of-order timestamp counts as a frame but produces no gap and does not become the next interval's reference, so one bad stamp cannot invent a multi-second hitch twice. Also corrects a stale ai.md line (the "webcam renders at ~18fps" figure came from a different session and counted every newly rendered layer) and re-records the background-removal ruling as permanently out. Records the agreed-but-unbuilt Units 1-2, the no-fallback-camera ruling, and the SetFormatAsync dead end found by probing the 19041 projection. Suite 402/402, 0 warnings. The one red, LayerReorderPersistenceTests' RealMouseDrag, fails identically on a stashed clean tree - it needs an interactive desktop session this shell lacks.
This commit is contained in:
+94
-20
@@ -1,13 +1,79 @@
|
||||
# HANDOFF — current state
|
||||
|
||||
**Branch:** `main` (pre-1.0, no feature branches — creator ruling 2026-08-24).
|
||||
**Pushed 2026-10-04:** `origin/main` = `048cab7`. **NOT PUSHED since — local `main` is 5 commits
|
||||
ahead and the tree is clean.** The stale-push-state failure `MyMistakes.md` warns about has now bitten
|
||||
**Pushed 2026-10-04:** `origin/main` = `048cab7`. **NOT PUSHED since — local `main` is ahead and the
|
||||
tree is clean.** The stale-push-state failure `MyMistakes.md` warns about has now bitten
|
||||
twice, so the rule stands: **check `git rev-list --count origin/main..main` before trusting any line
|
||||
here about push state.**
|
||||
|
||||
Unpushed: `5ad0106` broadcast diagnostic → `74da99c` + `0fb2f9c` privacy/Test rulings →
|
||||
`8fbc370` diagnostic parse fix → `07a088a` **the explicit-bind fix** → `509ef36` window placement.
|
||||
`8fbc370` diagnostic parse fix → `07a088a` **the explicit-bind fix** → `509ef36` window placement →
|
||||
`b2f2197` handoff → `UNIT 0` frame-cadence measurement.
|
||||
|
||||
## 🚧 IN FLIGHT 2026-10-05 — UNIT 0 shipped: a dropped webcam frame is finally measurable
|
||||
|
||||
**The measurement the map never had.** Take `ty-20261005-1001-821-0001.mp4` (85.6s, 1920×1080,
|
||||
H.264 NVENC, 5124 frames @60) holds **25.6 distinct webcam images/s** in its lower-right region
|
||||
against the camera's 30fps: 1,153 normal 33ms intervals, **281 gaps of 50–100ms**, 96 bunched 17ms
|
||||
gaps. One hitch in five — the judder a DHH creator lipreads against, invisible in a file that muxes
|
||||
flawlessly. Measured with `signalstats`/YDIF over `crop=480:270:1400:780`; the bundled ffmpeg has no
|
||||
`mpdecimate`. ⛔ The old "webcam renders at ~18fps" claim was wrong (different session, counted all
|
||||
newly rendered layers) — corrected in `ai.md`.
|
||||
|
||||
**Shipped:** `Helpers/FrameCadence.cs` (pure counter: frames, gaps, hitches ≥50ms, longest/avg), fed
|
||||
from three points — `MediaCaptureFrameSource` logs `Camera cadence WxH: arrivals … | camera …`
|
||||
(our delivery vs the device's own `SystemRelativeTime`), and `FramePump` logs distinct webcam
|
||||
`Epoch`s that reached a composite. Full detail in `ai.md` → "Frame cadence".
|
||||
|
||||
**Also fixed, and it was a real measurement bug:** every pump average was divided by the number of
|
||||
frames **submitted**, which deadline padding always fills to the full 60fps target — so a pump fed
|
||||
one webcam frame per 200ms logged "300/300 frames, avg render 12.4ms". Render/resolve/wait now divide
|
||||
by `statRenders` and the line reads `300/300 frames per 5s from 60 renders`. That pair IS the
|
||||
duplicate-on-lag ratio. Test: `FramePumpTests.Stats_ReportRealRendersAndDistinctWebcamFrames_…`.
|
||||
|
||||
**⏭ NEXT STEP — needs the creator, one short recording (60s is plenty):** record with the webcam
|
||||
visible and paste nothing — I read `%APPDATA%\ytLlive\startup.log` for the `Camera cadence` and
|
||||
`FramePump stats` lines. Reading them together names the culprit: camera steady + arrivals ragged =
|
||||
we drop it (reader/compositor); ragged in both = device/USB/driver; pump `webcam` below `arrivals`
|
||||
= the compositor missed frames the reader already had. **Then Unit 1** (exclusive lock + 720p, see
|
||||
below) is judged against those numbers.
|
||||
|
||||
**Agreed but NOT yet built (decided 2026-10-05, so a later session does not re-derive it):**
|
||||
- **Unit 1 — exclusive camera lock + 720p on the camera the creator picked.** Open with
|
||||
`MediaCaptureSharingMode.ExclusiveControl`; if init is refused, reopen the **SAME camera** in
|
||||
`SharedReadOnly`. Target MJPG ≥30fps up to 1280×720, ranked by frame rate then size (this also fixes
|
||||
the existing candidate-ordering bug). **The creator REVERSED the ban on taking an exclusive camera
|
||||
lock** — that ban was never written into the map, which is why the reversal must be recorded this
|
||||
time. Accepted trade-off: an app needing camera *control* that opens it after us will fail.
|
||||
- ✅ **API probe DONE 2026-10-05 against the actual 19041.57 projection** (a throwaway
|
||||
compile-probe file, since deleted — do not re-derive, and note the build cost of finding out):
|
||||
`MediaCaptureSharingMode.ExclusiveControl` **is** projected, so the old comment in
|
||||
`MediaCaptureFrameSource.cs` ("Exclusive is not projected by the 19041 SDK") is wrong and must be
|
||||
corrected. `Exclusive` and `SharedReadOnly` are the **only two** `MediaCaptureSharingMode` values.
|
||||
`MediaFrameSource.SupportedFormats` and `CurrentFormat` exist.
|
||||
⛔ **`MediaFrameSource.SetFormatAsync(MediaFrameFormat)` is a DEAD END here** — it takes a
|
||||
`MediaFrameFormat`, and that type has **no public constructor and no static factory** in the
|
||||
projection (it is only obtainable from a live frame's `VideoMediaFrame.Format`). The two-argument
|
||||
`(subtype, properties)` overload in Microsoft's docs does not exist here. So the format change must
|
||||
go through the `VideoDeviceController.SetMediaStreamPropertiesAsync` path the code ALREADY has —
|
||||
it is only refused because `SharingMode` is `SharedReadOnly`. Under an exclusive lock the existing
|
||||
negotiation should simply start working, which is a much smaller change than planned.
|
||||
- **Remaining unknown (needs a real device, not research):** whether `SetMediaStreamPropertiesAsync`
|
||||
actually succeeds under `ExclusiveControl` on this C920. The `MediaFrameSource.SupportedFormats`
|
||||
route is the fallback probe if it refuses — enumerate, and log what the device offers before
|
||||
assuming anything.
|
||||
- **Unit 2 — blinking "SD" badge in the preview only**, when a better format was available and we did
|
||||
not get it. Top-right corner of the webcam, or the 1:30 point ON THE RIM for a round clip (a round
|
||||
webcam has no corner); outside the mirror transform. ~1Hz opacity blink (under the 3/sec
|
||||
photosensitivity limit), tooltip on hover only, **must never appear in the recording or stream**
|
||||
(the compositor draws from camera frames, not the WPF tree). Style lives in `Themes/Controls.xaml`
|
||||
— a DataTrigger storyboard cannot be stopped inside a DataTemplate (`PreviewPane.xaml:168-171`).
|
||||
- ⛔ **NO fallback-camera setting, ever** — creator ruling: *"no fallback - that's bypassing the
|
||||
user's explicit choice."* Picking a vcam is a user action in the camera picker. Corollary, also
|
||||
recorded in `ai.md`: **Windows Studio Effects is NOT a vcam** — it runs inside the physical camera's
|
||||
driver and never appears as a selectable device. Only Broadcast / XSplit VCam install one.
|
||||
- ⛔ **Background removal: permanently out.** Creator confirmed the existing note: *"no - not doing
|
||||
it. the note is correct and accurate."* `ai.md` said "milestone 2", which was stale — now corrected.
|
||||
|
||||
## 🚨 FIXED 2026-10-04 — broadcasts never went live because they were never BOUND
|
||||
|
||||
@@ -281,11 +347,12 @@ Detail and the open 11.12 re-check: [`TASKS/task-50-publish-recording.md`](TASKS
|
||||
|
||||
## Landmines
|
||||
|
||||
- **Frame pump jitter, unquantified (2026-10-04).** The same test stream logged **86 stalls
|
||||
per 5s** (`FramePump stall: iteration 40–50ms` against a 17ms budget; 142 such lines in an
|
||||
8-second window) while `FramePump stats` still read 301/300 frames, 0 dropped. Nominal 60fps
|
||||
with zero drops *and* repeated 3× overruns is a contradiction worth one measurement, not a
|
||||
guess. Do not "fix" it from the log alone.
|
||||
- **Frame pump jitter — counters now honest, cause NOT yet named (2026-10-05).** The same test
|
||||
stream logged **86 stalls per 5s** (`FramePump stall: iteration 40–50ms` against a 17ms budget)
|
||||
while `FramePump stats` read 301/300 frames, 0 dropped. UNIT 0 removed the reason those two
|
||||
numbers could disagree (the averages were diluted by padded submits) and added the counters that
|
||||
separate reader from compositor — **but no one has read a post-UNIT-0 log yet.** A creator
|
||||
recording is required first. Do not "fix" the pump before that reading.
|
||||
- **Stale fact inside `TASKS.md` (left alone, flagged here per the Scope Lock).** The "1.0 gates"
|
||||
section asserts "**TASK 36 is now shipped**", but the catalog row 36 reads **☐ Queued** and
|
||||
`task-36-gold-pass.md` still shows items 1–6 unchecked. The line most likely means *item 2*
|
||||
@@ -293,9 +360,10 @@ Detail and the open 11.12 re-check: [`TASKS/task-50-publish-recording.md`](TASKS
|
||||
flagging rather than fixing because it is not this unit's job.
|
||||
- **A stale `ytLive.exe` (PID 2544) locked `bin/.../ytLive.exe`** and broke `dotnet run` with
|
||||
MSB3027. Killed. **If the build fails to copy `ytLive.exe`, check for a running instance first.**
|
||||
- **`LayerReorderPersistenceTests.RealMouseDrag_OnTheLayerList_PersistsTheReorder` is flaky by
|
||||
environment** (needs an interactive desktop session; 2 fail / 1 pass in isolation). **Run the
|
||||
suite with ytLive CLOSED.** Not a regression.
|
||||
- **`LayerReorderPersistenceTests.RealMouseDrag_OnTheLayerList_PersistsTheReorder` needs an
|
||||
interactive desktop session.** As of 2026-10-05 it fails deterministically here (`ImgC` vs
|
||||
`ImgA`), verified identical on a clean tree. **Run the suite with ytLive CLOSED** and expect this
|
||||
one to fail in this shell. Not a regression — don't diagnose it from a WSL-shell failure.
|
||||
- **`RealAppHost.RunAsync` exists for a reason** — a frame-pump test MUST `await` inside it. A
|
||||
blocking wait occupies the one shared STA thread and **hangs the whole suite with no output.**
|
||||
Always background a `vstest` run and poll the log; a foreground piped `vstest` returns nothing
|
||||
@@ -308,11 +376,17 @@ Detail and the open 11.12 re-check: [`TASKS/task-50-publish-recording.md`](TASKS
|
||||
|
||||
## Test state
|
||||
|
||||
**395/395, 0 warnings, re-verified 2026-10-04 on a clean test-project build** (381 + 3 diagnostic
|
||||
`FirstItem` cases + 1 window fact + 3 placement theory cases + 1 bind test, less 1 removed).
|
||||
Build the **test** project, not just the app project, or VSTest runs a stale DLL.
|
||||
**402/402, 0 warnings, verified 2026-10-05** (395 + 6 `FrameCadenceTests` + 1 pump-stats integration
|
||||
test). Build the **test** project, not just the app project, or VSTest runs a stale DLL.
|
||||
**Launch exe for the creator's test run: `bin/Debug/net8.0-windows10.0.19041.0/ytLive.exe`.**
|
||||
|
||||
**Confirmed environmental flake, NOT a regression (re-checked 2026-10-05, and it now fails
|
||||
deterministically rather than 2-of-3):** `LayerReorderPersistenceTests.RealMouseDrag_OnTheLayerList_
|
||||
PersistsTheReorder` fails with `Expected "ImgC", Actual "ImgA"` at `RealAppHost.Run` line 68. **It
|
||||
fails identically on a stashed clean tree**, so it has nothing to do with UNIT 0 — it needs an
|
||||
interactive desktop session this shell does not have (the mouse drag never lands). Do not chase it;
|
||||
it is listed under Landmines as the known interactive-session test.
|
||||
|
||||
**⚠️ New flake, do not chase it as a regression:** on the *first* full-suite run,
|
||||
`AudioPipelineTests.MasterLimiter_CapsTheLiveMix_OnThePipe` failed with `Expected 0.891,
|
||||
Actual 0` — a **zero**, not a slightly-off value, which is the signature of the pipe not being
|
||||
@@ -335,12 +409,12 @@ whole-suite scheduling artifact of the same family as the `RealAppHost` STA land
|
||||
1. **~PUSH, then the creator's 60-second Test** — see the FIXED box above. Packaging is NOT next;
|
||||
the creator's ruling 2026-10-04: still in active development, so ship-time work waits and queued
|
||||
features come first.
|
||||
2. **Frame-pump jitter (queued, unquantified):** ~86 stalls per 5 s in one log window, 142 in an
|
||||
8-second window, while stats reported 301/300 frames and **zero** drops. Measure before touching
|
||||
anything — the counter and the missing-frame count currently disagree, and "fixing" the pump
|
||||
without knowing which number is wrong risks making it worse. **Creator to-do, not yet queued:
|
||||
long-run resource/consumption/recycling benchmarking** (how long the app stays healthy under full
|
||||
burden) — record only, do not start it.
|
||||
2. **Webcam cadence → Unit 1 (exclusive lock + 720p).** UNIT 0 is shipped and the counters are in
|
||||
`startup.log`; the reading needs one 60-second creator recording (details + the exact
|
||||
read-the-log-together table in the IN FLIGHT box above). Then Unit 1: exclusive camera lock +
|
||||
720p, and Unit 2: the blinking SD badge. **Creator to-do, not yet queued: long-run
|
||||
resource/consumption/recycling benchmarking** (how long the app stays healthy under full burden) —
|
||||
record only, do not start it.
|
||||
2. **The packaging project + `Package.appxmanifest`** (TASK 48 items 2–3) — the next code unit.
|
||||
Full trust, `webcam`/`microphone`, English only; Velopack deleted. Add the
|
||||
`DefaultRecordFolder()` comment in the same unit, and the WebView2 framework dependency +
|
||||
|
||||
@@ -0,0 +1,123 @@
|
||||
using System;
|
||||
using System.Diagnostics;
|
||||
|
||||
namespace ytLive.Helpers;
|
||||
|
||||
/// <summary>
|
||||
/// Arrival cadence for one window of a timestamped frame stream: how many frames
|
||||
/// arrived, how far apart they were, and how many gaps were long enough to read
|
||||
/// as a hitch. Fed any shared monotonic clock (<see cref="Stopwatch.GetTimestamp"/>
|
||||
/// values, or a device's system-relative stamps on the same clock) and read through
|
||||
/// <see cref="Summarize"/>.
|
||||
///
|
||||
/// Why it exists: a 60fps container makes a missing camera frame indistinguishable
|
||||
/// from a padded repeat. rawvideo carries no per-frame timestamps — ffmpeg stamps
|
||||
/// frames by ARRIVAL at the declared rate — so the OBS "duplicate on lag" loop
|
||||
/// (FramePump) emits 300 authored frames per 5s whether the renderer managed 300
|
||||
/// composites or 84. The recorded file therefore cannot name the stage that lost
|
||||
/// the frames; only these counters can. The same counter run twice tells the two
|
||||
/// apart: the camera's OWN timestamps vs our ARRIVAL timestamps. Steady camera
|
||||
/// stamps with ragged arrivals means WE dropped it (compositor/reader); ragged in
|
||||
/// both means the camera, USB link, or driver did.
|
||||
/// </summary>
|
||||
public sealed class FrameCadence
|
||||
{
|
||||
/// <summary>Default hitch threshold in ms. 50ms is ~1.5x a 30fps period — a
|
||||
/// missed 30fps frame arrives as ~66ms — so this counts dropped frames without
|
||||
/// flagging ordinary jitter (a 33ms frame can legitimately measure 35ms).</summary>
|
||||
public const double DefaultHitchThresholdMs = 50;
|
||||
|
||||
private readonly double _hitchThresholdMs;
|
||||
private long _firstTicks;
|
||||
private long _previousTicks;
|
||||
private double _gapTicksTotal;
|
||||
private int _frames;
|
||||
private int _gaps;
|
||||
private int _hitches;
|
||||
private double _longestGapMs;
|
||||
|
||||
public FrameCadence(double hitchThresholdMs = DefaultHitchThresholdMs)
|
||||
{
|
||||
if (hitchThresholdMs <= 0)
|
||||
throw new ArgumentOutOfRangeException(nameof(hitchThresholdMs), hitchThresholdMs,
|
||||
"Hitch threshold must be positive.");
|
||||
_hitchThresholdMs = hitchThresholdMs;
|
||||
}
|
||||
|
||||
public int Frames => _frames;
|
||||
public int Gaps => _gaps;
|
||||
public int Hitches => _hitches;
|
||||
public double HitchThresholdMs => _hitchThresholdMs;
|
||||
|
||||
/// <summary>Records one frame at <paramref name="timestampTicks"/> on the shared
|
||||
/// monotonic clock. Monotonicity is asserted, not trusted: an out-of-order stamp
|
||||
/// (a device whose clock domain differs from ours, a wrapped counter, a dropped
|
||||
/// timestamp) is counted as a frame but produces no gap AND does not become the
|
||||
/// reference for the next one — otherwise a single bad stamp would poison both
|
||||
/// its own interval and the one after it, inventing a multi-second "hitch".</summary>
|
||||
public void Add(long timestampTicks)
|
||||
{
|
||||
if (_frames == 0)
|
||||
{
|
||||
_firstTicks = timestampTicks;
|
||||
_previousTicks = timestampTicks;
|
||||
_frames++;
|
||||
return;
|
||||
}
|
||||
|
||||
var delta = timestampTicks - _previousTicks;
|
||||
_frames++;
|
||||
if (delta <= 0) return;
|
||||
var gapMs = delta * 1000d / Stopwatch.Frequency;
|
||||
_gaps++;
|
||||
_gapTicksTotal += gapMs;
|
||||
if (gapMs > _longestGapMs) _longestGapMs = gapMs;
|
||||
if (gapMs >= _hitchThresholdMs) _hitches++;
|
||||
_previousTicks = timestampTicks;
|
||||
}
|
||||
|
||||
public FrameCadenceSummary Summarize()
|
||||
{
|
||||
var spanMs = _frames >= 2
|
||||
? (_previousTicks - _firstTicks) * 1000d / Stopwatch.Frequency
|
||||
: 0d;
|
||||
// Rate is gaps-per-span, not frames-per-span: N frames span N-1 periods, and
|
||||
// counting the extra interval would report one extra frame per window.
|
||||
var rate = _gaps > 0 && spanMs > 0 ? _gaps / (spanMs / 1000d) : 0d;
|
||||
return new FrameCadenceSummary(
|
||||
_frames, rate, spanMs, _gaps, _hitches, _longestGapMs,
|
||||
_gaps > 0 ? _gapTicksTotal / _gaps : 0d, _hitchThresholdMs);
|
||||
}
|
||||
|
||||
/// <summary>Starts a fresh window, keeping the configured threshold.</summary>
|
||||
public void Reset()
|
||||
{
|
||||
_firstTicks = 0;
|
||||
_previousTicks = 0;
|
||||
_gapTicksTotal = 0;
|
||||
_frames = 0;
|
||||
_gaps = 0;
|
||||
_hitches = 0;
|
||||
_longestGapMs = 0;
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>One measured cadence window — see <see cref="FrameCadence"/>.</summary>
|
||||
public readonly record struct FrameCadenceSummary(
|
||||
int Frames,
|
||||
double RatePerSecond,
|
||||
double SpanMs,
|
||||
int Gaps,
|
||||
int Hitches,
|
||||
double LongestGapMs,
|
||||
double AverageGapMs,
|
||||
double HitchThresholdMs)
|
||||
{
|
||||
/// <summary>One-line log form. Identical in shape from every producer (camera
|
||||
/// source, frame pump) so two lines can be read against each other: same window
|
||||
/// length, same fields, same order.</summary>
|
||||
public string Describe(string label) => Frames == 0
|
||||
? $"{label} no frames"
|
||||
: $"{label} {Frames}f/{SpanMs / 1000d:F1}s {RatePerSecond:F1}/s, {Gaps} gaps, " +
|
||||
$"{Hitches} >= {HitchThresholdMs:F0}ms, avg {AverageGapMs:F1}ms, longest {LongestGapMs:F1}ms";
|
||||
}
|
||||
@@ -14,6 +14,7 @@ Cross-cutting utilities. See [`schema.md`](../schema.md) for the memory-map conv
|
||||
| `InverseBoolToVisibilityConverter.cs` / `NotNullToVisibilityConverter.cs` | XAML value converters for visibility bindings |
|
||||
| `AllTrueToVisibilityConverter.cs` | Multi-value AND-gate → Visible; gates the layer-row capture menu items on row-IsBackground + staged-scene-is-Live (TASK 25) |
|
||||
| `CameraConflictProbe.cs` | Best-effort diagnostic: when a camera won't start, enumerates running processes known to hold cameras (OBS, Zoom, Teams, NVIDIA Broadcast, browsers, etc.) — Windows doesn't expose "which process has this device" via any public API, so this is a suspect list, not a verdict |
|
||||
| `FrameCadence.cs` | Pure arrival-cadence counter (`FrameCadence` + `FrameCadenceSummary.Describe`): frames, gaps, hitches, longest/avg gap for a window of a timestamped stream. Fed by `MediaCaptureFrameSource` (camera stamps vs our arrivals) and `FramePump` (distinct webcam Epochs that reached the output) — UNIT 0, see ai.md "Frame cadence" |
|
||||
|
||||
Related: [`Themes/Controls.xaml`](../Themes/Controls.xaml) styles the lists this
|
||||
class backs; [`Models/index.md`](../Models/index.md) and
|
||||
|
||||
@@ -338,6 +338,27 @@ public sealed class FramePump : IDisposable
|
||||
int stalls = 0;
|
||||
long lastCacheHits = 0, lastCacheRenders = 0;
|
||||
var statsNext = DateTime.UtcNow + TimeSpan.FromSeconds(5);
|
||||
|
||||
// Real-render accounting (UNIT 0, 2026-10-05): every average above was
|
||||
// divided by statFrames — the number of frames SUBMITTED, which deadline
|
||||
// padding always fills to the full target even when the renderer only
|
||||
// produced a fifth of them. A 60fps container therefore reported "300/300
|
||||
// frames, avg render 12ms" while the source actually fed it 60 distinct
|
||||
// images. Render cost is now divided by the render count, and the render
|
||||
// count is reported beside the submit count — the two numbers side by side
|
||||
// ARE the duplicate-on-lag ratio, which is the only honest reading of how
|
||||
// much of the stream is a repeat.
|
||||
int statRenders = 0;
|
||||
|
||||
// Webcam cadence (UNIT 0, 2026-10-05): the pump sees each element's resolved
|
||||
// frame, and VideoFrame.Epoch is the webcam ring's monotonic generation — so a
|
||||
// NEW epoch means the webcam actually delivered something new between two of
|
||||
// our ticks. This is the counterpart to MediaCaptureFrameSource's arrivals
|
||||
// line: it measures what reached the OUTPUT, not what the camera produced. If
|
||||
// the camera reports 30/s and this reports 25/s, the loss is between them.
|
||||
var webcamCadence = new Helpers.FrameCadence();
|
||||
var seenWebcamEpochs = new HashSet<long>();
|
||||
|
||||
void ReportStats()
|
||||
{
|
||||
if (DateTime.UtcNow < statsNext) return;
|
||||
@@ -360,19 +381,23 @@ public sealed class FramePump : IDisposable
|
||||
}
|
||||
_log?.Invoke(statFrames == 0
|
||||
? "FramePump stats: NO frames produced in 5s (loop stalled?)" + (detail.Length > 0 ? " | " + detail : "")
|
||||
: $"FramePump stats: {statFrames}/{target:F0} frames per 5s, " +
|
||||
$"avg render {renderTicks / (double)System.Diagnostics.Stopwatch.Frequency * 1000 / statFrames:F1}ms " +
|
||||
$"(resolve {resolveTicks / (double)System.Diagnostics.Stopwatch.Frequency * 1000 / statFrames:F1}), " +
|
||||
: $"FramePump stats: {statFrames}/{target:F0} frames per 5s from {statRenders} renders, " +
|
||||
$"avg render {renderTicks / (double)System.Diagnostics.Stopwatch.Frequency * 1000 / Math.Max(1, statRenders):F1}ms " +
|
||||
$"(resolve {resolveTicks / (double)System.Diagnostics.Stopwatch.Frequency * 1000 / Math.Max(1, statRenders):F1}), " +
|
||||
$"avg submit {submitTicks / (double)System.Diagnostics.Stopwatch.Frequency * 1000 / statFrames:F1}ms, "
|
||||
+ $"avg wait {waitTicks / (double)System.Diagnostics.Stopwatch.Frequency * 1000 / statFrames:F1}ms, "
|
||||
+ $"avg wait {waitTicks / (double)System.Diagnostics.Stopwatch.Frequency * 1000 / Math.Max(1, statRenders):F1}ms, "
|
||||
+ $"worst render {worstRender / (double)System.Diagnostics.Stopwatch.Frequency * 1000:F1}ms, "
|
||||
+ $"worst submit {worstSubmit / (double)System.Diagnostics.Stopwatch.Frequency * 1000:F1}ms, "
|
||||
+ $"dropped {dropped}, stalls {stalls}" + cacheDetail + (detail.Length > 0 ? " | " + detail : ""));
|
||||
+ $"dropped {dropped}, stalls {stalls}{cacheDetail}, {webcamCadence.Summarize().Describe("webcam")}"
|
||||
+ (detail.Length > 0 ? " | " + detail : ""));
|
||||
worstRender = 0;
|
||||
worstSubmit = 0;
|
||||
renderTicks = submitTicks = resolveTicks = waitTicks = 0;
|
||||
statFrames = 0;
|
||||
statRenders = 0;
|
||||
stalls = 0;
|
||||
webcamCadence.Reset();
|
||||
seenWebcamEpochs.Clear();
|
||||
statsNext = DateTime.UtcNow + TimeSpan.FromSeconds(5);
|
||||
}
|
||||
|
||||
@@ -387,6 +412,8 @@ public sealed class FramePump : IDisposable
|
||||
var frame = _frameResolver(element);
|
||||
resolveSw.Stop();
|
||||
resolveTicks += resolveSw.ElapsedTicks;
|
||||
if (element.IsWebcam && frame != null && seenWebcamEpochs.Add(frame.Epoch))
|
||||
webcamCadence.Add(System.Diagnostics.Stopwatch.GetTimestamp());
|
||||
return frame;
|
||||
}
|
||||
|
||||
@@ -444,6 +471,7 @@ public sealed class FramePump : IDisposable
|
||||
renderSw.Stop();
|
||||
renderTicks += renderSw.ElapsedTicks;
|
||||
if (renderSw.ElapsedTicks > worstRender) worstRender = renderSw.ElapsedTicks;
|
||||
statRenders++;
|
||||
|
||||
IFfmpegEncoder? encoder;
|
||||
lock (_gate) encoder = _encoder;
|
||||
|
||||
@@ -50,7 +50,9 @@ public sealed class MediaCaptureFrameSource : ICameraFrameSource
|
||||
// Fallback ladder: prefer the camera's VideoPreview stream, but if the
|
||||
// reader refuses to start there (NoVideoFrameAvailable), retry against
|
||||
// its VideoRecord stream — some devices only deliver through it.
|
||||
// (MediaCaptureSharingMode.Exclusive is not projected by the 19041 SDK.)
|
||||
// (The exclusive-lock upgrade is a separate, later unit — creator
|
||||
// ruling 2026-10-05; see HANDOFF.md. It is NOT blocked by an SDK gap:
|
||||
// `MediaCaptureSharingMode.ExclusiveControl` IS projected in 19041.)
|
||||
if (!await TryStartAsync(preferVideoRecord: false))
|
||||
{
|
||||
if (!await TryStartAsync(preferVideoRecord: true))
|
||||
@@ -246,6 +248,7 @@ public sealed class MediaCaptureFrameSource : ICameraFrameSource
|
||||
_frameReader = reader;
|
||||
_lastError = null;
|
||||
_isFailed = false;
|
||||
StartCadenceWindow();
|
||||
return true;
|
||||
}
|
||||
|
||||
@@ -287,6 +290,30 @@ public sealed class MediaCaptureFrameSource : ICameraFrameSource
|
||||
}
|
||||
}
|
||||
|
||||
private const int CadenceWindowSeconds = 5;
|
||||
|
||||
private void StartCadenceWindow()
|
||||
{
|
||||
_arrivalCadence.Reset();
|
||||
_cameraCadence.Reset();
|
||||
_cadenceNextReportTicks = System.Diagnostics.Stopwatch.GetTimestamp()
|
||||
+ System.Diagnostics.Stopwatch.Frequency * CadenceWindowSeconds;
|
||||
}
|
||||
|
||||
/// <summary>Logs one cadence window and opens the next. The size rides along
|
||||
/// because the whole question started as "the webcam is soft": a future format
|
||||
/// change must be readable in the SAME line that proves the frame rate.</summary>
|
||||
private void ReportCadence(int width, int height)
|
||||
{
|
||||
_cadenceNextReportTicks = System.Diagnostics.Stopwatch.GetTimestamp()
|
||||
+ System.Diagnostics.Stopwatch.Frequency * CadenceWindowSeconds;
|
||||
var arrivals = _arrivalCadence.Summarize();
|
||||
var camera = _cameraCadence.Summarize();
|
||||
_arrivalCadence.Reset();
|
||||
_cameraCadence.Reset();
|
||||
AppLog.Write($"Camera cadence {width}x{height}: {arrivals.Describe("arrivals")} | {camera.Describe("camera")}");
|
||||
}
|
||||
|
||||
private void UnsubscribeCaptureEvents(MediaCapture capture)
|
||||
{
|
||||
capture.Failed -= OnCaptureFailed;
|
||||
@@ -349,12 +376,35 @@ public sealed class MediaCaptureFrameSource : ICameraFrameSource
|
||||
return fresh;
|
||||
}
|
||||
|
||||
// Arrival vs production cadence (UNIT 0, 2026-10-05): the recorded file cannot
|
||||
// tell a dropped camera frame from a padded repeat — rawvideo stamps frames by
|
||||
// ARRIVAL at the declared 60fps, so a webcam delivering 25 of its 30 frames still
|
||||
// muxes a full-length file. Two counters, one reader, settle it: the camera's OWN
|
||||
// SystemRelativeTime stamps (device/USB/driver cadence) against the moment the
|
||||
// reader handed each frame to us (our cadence, including coalescing drops — the
|
||||
// reader deliberately keeps only the LATEST frame, so falling behind is silent).
|
||||
// Steady camera + ragged arrivals = we are behind. Ragged in both = the device.
|
||||
private readonly FrameCadence _arrivalCadence = new();
|
||||
private readonly FrameCadence _cameraCadence = new();
|
||||
private long _cadenceNextReportTicks;
|
||||
|
||||
private void OnFrameArrived(MediaFrameReader sender, MediaFrameArrivedEventArgs args)
|
||||
{
|
||||
var arrivedTicks = System.Diagnostics.Stopwatch.GetTimestamp();
|
||||
using var frame = sender.TryAcquireLatestFrame();
|
||||
var videoFrame = frame?.VideoMediaFrame?.SoftwareBitmap;
|
||||
if (videoFrame == null) return;
|
||||
|
||||
// SystemRelativeTime is QPC-domain, the same clock Stopwatch reads here, so the
|
||||
// camera's own frame spacing is measurable without a second conversion. It is
|
||||
// NULLABLE in the projection (null = the device supplied no stamp), so the
|
||||
// production side of the comparison is then simply missing from the line —
|
||||
// arrivals alone still measure what reached us.
|
||||
_arrivalCadence.Add(arrivedTicks);
|
||||
if (frame!.SystemRelativeTime is { } cameraStamp)
|
||||
_cameraCadence.Add((long)(cameraStamp.TotalSeconds * System.Diagnostics.Stopwatch.Frequency));
|
||||
if (arrivedTicks >= _cadenceNextReportTicks) ReportCadence(videoFrame.PixelWidth, videoFrame.PixelHeight);
|
||||
|
||||
var bitmap = videoFrame.BitmapPixelFormat == BitmapPixelFormat.Bgra8
|
||||
? videoFrame
|
||||
: SoftwareBitmap.Convert(videoFrame, BitmapPixelFormat.Bgra8);
|
||||
|
||||
@@ -156,6 +156,19 @@ second encoder path needing a "redirect". Record+simulcast is one ffmpeg with tw
|
||||
adopts a solo camera as default. TASK 26's app-wide gate superseded by creator directive. The App
|
||||
Settings webcam selector is now **TASK 40 Unit A** (plan saved 2026-09-22). Remaining next slice
|
||||
(user-queued): static (+) catalog rows (Background, YouTubeEvent, quoted labels) with reason-greying.
|
||||
- **🔶 Webcam cadence → resolution — IN PROGRESS, creator-queued 2026-10-05** (DHH/lipreading: the
|
||||
webcam judders, ~1 hitch in 5, invisible in the output file). **UNIT 0 shipped** (counters +
|
||||
`Helpers/FrameCadence.cs`; see `ai.md` → "Frame cadence"). **Needs a creator recording before the
|
||||
next unit.** Remaining units, all agreed 2026-10-05, one Good Dog test each:
|
||||
- **Unit 1:** `MediaCaptureSharingMode.ExclusiveControl` + MJPG ≥30fps up to 1280×720 via the
|
||||
existing `SetMediaStreamPropertiesAsync` path (refused only because `SharedReadOnly`); same-camera
|
||||
`SharedReadOnly` retry. **The creator reversed the ban on taking an exclusive camera lock.**
|
||||
API probe already done — see HANDOFF.md, incl. the `SetFormatAsync` dead end.
|
||||
- **Unit 2:** blinking amber **"SD" badge**, preview only, never in the recording/stream, when a
|
||||
better format was available and not obtained. Top-right corner, or the 1:30 rim point when the
|
||||
clip is round.
|
||||
- ⛔ **No fallback-camera feature, ever** — *"that's bypassing the user's explicit choice."*
|
||||
- ⛔ **No background removal** — re-affirmed out 2026-10-05.
|
||||
- **TASK 40 — App Settings round** — plan saved 2026-09-22 (`TASKS/task-40-app-settings-round.md`),
|
||||
four units, one Good Dog test each: **A** active web camera section under Recordings (Change…
|
||||
sub-menu = live enumeration incl. vcams; Tune… config dialog via `ICameraControlProbe`, live-only
|
||||
@@ -357,7 +370,9 @@ See [`TASKS/research-youtube-api.md`](TASKS/research-youtube-api.md) — authori
|
||||
These were explicitly ruled out:
|
||||
|
||||
- **D3D11 swap compositor** (superseded by software compositor; TASK 3 item 16)
|
||||
- **Background removal** (ONNX/DirectML; TASK 3 item 19)
|
||||
- **Background removal** (ONNX/DirectML; TASK 3 item 19) — ⛔ **re-affirmed out 2026-10-05**: *"no - not
|
||||
doing it. the note is correct and accurate."* A vcam app does it free; an in-app model is a runtime,
|
||||
a GPU/driver matrix, and per-scene lighting tuning. **Never reopen without a new ruling.**
|
||||
- **OBS-style source expansion** (game capture, browser source, media playlist, VLC, color-key, MIDI — the minimal source set is deliberate; creators who need more have graduated to OBS)
|
||||
- **Multi-layout** (auto-save, single layout; multi-layout = OBS territory)
|
||||
- **Cross-process stream resume** (API makes it impossible; creator ruling 2026-09-01)
|
||||
|
||||
@@ -483,6 +483,11 @@ This replaces the old five-seeder cluster (`Seed{Starting,Brb,Ending,Chat}Backgr
|
||||
fails with "camera in use" (or, if init slips through, the device exposes no preview source). Fix:
|
||||
quit Broadcast while streaming, or pick its virtual "NVIDIA Broadcast" device from the picker and the
|
||||
app captures the processed feed. This is a real-device finding (Logitech `VID_046D&PID_082D`).
|
||||
Correction (2026-10-05): **Windows Studio Effects is NOT a second option of this kind** — it runs
|
||||
inside the physical camera's own driver, so it never appears as a selectable device. Only tools
|
||||
that install a virtual camera (NVIDIA Broadcast, XSplit VCam) do. "Pick the vcam" is a user action in
|
||||
the camera picker, never an automatic substitution: creator ruling 2026-10-05 — the app must not
|
||||
bypass the user's explicit camera choice, so there is deliberately **no fallback-camera setting**.
|
||||
- **TFM:** `net8.0-windows10.0.19041.0` (app + tests) pulls the WinRT projection from the SDK reference
|
||||
packs — no NuGet package, no capability manifest (unpackaged desktop app works; the Windows privacy
|
||||
camera toggle still applies). `EnableWindowsTargeting` keeps WSL builds working.
|
||||
@@ -597,8 +602,56 @@ This replaces the old five-seeder cluster (`Seed{Starting,Brb,Ending,Chat}Backgr
|
||||
- **GPU posture:** webcam frames are CPU (GPU-agnostic; WPF hardware-presents the preview anyway). Hardware
|
||||
encoders (NVENC/AMF/QSV) matter for the encoder task, not capture. D3DImage GPU compositing is deferred
|
||||
to the encoder task.
|
||||
- **Background removal = milestone 2** — ONNX Runtime + DirectML (CPU fallback), MediaPipe Selfie
|
||||
Segmentation (Apache-2.0), wired into the same `VideoFrame` seam. Not part of milestone 1.
|
||||
- **Background removal = PERMANENTLY OUT** (creator ruling 2026-10-05, confirmed: "no - not doing
|
||||
it. the note is correct and accurate"). The line here used to read "milestone 2", which was stale —
|
||||
it was never a plan, only a leftover from TASK 3's original scoping. The rationale that stands:
|
||||
a vcam app (NVIDIA Broadcast, XSplit VCam) does it free, and an in-app model means a runtime, a
|
||||
GPU/driver matrix, and per-scene lighting tuning — support hell for a solo dev
|
||||
(`TASKS/task-36-gold-pass.md`, `TASKS/task-03-capture-pipeline.md` item 19). Don't reopen it.
|
||||
|
||||
### Frame cadence — how a dropped frame became measurable (UNIT 0, 2026-10-05)
|
||||
|
||||
**The problem this exists to solve.** The recorded file cannot distinguish a dropped camera frame
|
||||
from a padded repeat. `-f rawvideo` carries no per-frame timestamps — ffmpeg assigns pts from frame
|
||||
order at the DECLARED fps — so a webcam delivering 25 of its 30 frames still muxes a full-length,
|
||||
correctly-timed 60fps file. Measured on take `ty-20261005-1001-821-0001.mp4` (85.6s, 1080p60): the
|
||||
lower-right webcam region held 25.6 distinct images/s against the camera's 30fps, with 1,153 normal
|
||||
33ms intervals, 281 gaps of 50–100ms, and 96 bunched 17ms gaps — roughly one hitch in five, which is
|
||||
what makes a face judder while the file looks flawless. (The "webcam renders at ~18fps" figure that
|
||||
used to sit in this map was WRONG: it came from a different session and counted every newly rendered
|
||||
layer, not webcam arrivals. Corrected 2026-10-05.)
|
||||
|
||||
**`Helpers/FrameCadence.cs`** — one pure counter, three feeds. `Add(timestamp)` on any shared
|
||||
monotonic clock; `Summarize()` → frames, rate (gaps ÷ span, NOT frames ÷ span — N frames span N-1
|
||||
periods), gaps, hitches (default threshold 50ms ≈ 1.5× a 30fps period), longest and average gap;
|
||||
`Describe(label)` gives every producer the same one-line shape so two lines can be read against each
|
||||
other. An out-of-order stamp counts as a frame but produces no gap AND does not become the reference
|
||||
for the next interval — otherwise one bad stamp invents a multi-second "hitch" twice.
|
||||
|
||||
**Three measurement points, and what each separates:**
|
||||
|
||||
| Line | Fed by | Answers |
|
||||
|---|---|---|
|
||||
| `Camera cadence WxH: arrivals … \| camera …` | `MediaCaptureFrameSource` | device/USB/driver cadence (`MediaFrame.SystemRelativeTime`, QPC domain, nullable) vs what the reader handed us. `TryAcquireLatestFrame` keeps only the LATEST frame, so falling behind is silent by design — this is where that shows |
|
||||
| `webcam …` on the pump's stats line | `FramePump.TimedResolver` | distinct webcam `VideoFrame.Epoch` values that reached a composite — what actually got into the output |
|
||||
| `frames per 5s from R renders` | `FramePump` | submits vs REAL composites — the duplicate-on-lag ratio, the only honest reading of how much of the stream is a repeat |
|
||||
|
||||
**Why the stats-line denominators changed (the actual bug).** Every average was divided by
|
||||
`statFrames`, the number of frames SUBMITTED — which deadline padding always fills to the full 60fps
|
||||
target. A pump fed one webcam frame per 200ms logged "300/300 frames per 5s, avg render 12.4ms"
|
||||
while producing 60 distinct composites: the render average was diluted ~5× and the loss was
|
||||
invisible. Render/resolve/wait are now divided by `statRenders`; submit stays per-frame (it really
|
||||
happens per submit). Regression test:
|
||||
`FramePumpTests.Stats_ReportRealRendersAndDistinctWebcamFrames_NotJustPaddedSubmits`.
|
||||
|
||||
**Reading the three together:** camera 30/s steady + arrivals 25/s = WE dropped it (reader or
|
||||
compositor). Ragged in both = the device, the USB link, or the driver. Pump `webcam` below
|
||||
`arrivals` = the compositor missed frames the reader had already delivered.
|
||||
|
||||
**Still unmeasured (next unit):** the camera is stuck at the C920 default 640×480 YUY2 @30 because
|
||||
all 185 `SetMediaStreamPropertiesAsync` attempts are refused under `SharedReadOnly`, so 720p MJPG is
|
||||
on the table and the compositor's headroom at ~3× the bytes is unknown. Measure first, then change
|
||||
the acquisition.
|
||||
|
||||
### Scene compositor (TASK 4 ship step 1 — shipped 2026-08-10, plan in TASKS.md)
|
||||
|
||||
@@ -943,7 +996,7 @@ scene each tick, resolves every element to its latest frame, composites it into
|
||||
and paces frames into the encoder at the tier's FPS. **rawvideo is stamped by ARRIVAL:** ffmpeg assigns
|
||||
pts from frame order at the declared fps — supply rate = output speed. A starved producer (pump < fps)
|
||||
ships a time-lapse, truncated file with NO error (take-2 lesson, 2026-09-01); the pump therefore logs
|
||||
`FramePump stats: n/target frames per 5s, avg render Xms (resolve R), avg submit Yms` so a slow stage names itself — and since 2026-09-04 so does the BUILD: every build gets a GUID stamped
|
||||
`FramePump stats: n/target frames per 5s from R renders, avg render Xms (resolve R2), avg submit Yms` so a slow stage names itself (the `frames … from R renders` pair and the render-count denominators arrived with UNIT 0 — see "Frame cadence" below; before that every average was divided by the SUBMITTED count, which deadline padding always fills to the full target, so a pump fed one webcam frame per 200ms still logged "300/300 frames" and its render average was diluted ~an order of magnitude) — and since 2026-09-04 so does the BUILD: every build gets a GUID stamped
|
||||
into `Helpers/BuildStamp` (csproj `GenerateBuildStamp` target, fresh per compile — no lying incremental build), logged at startup
|
||||
("Build xxxxxxxx (compiled ...)"). **Take 6 (stamp-less, ambiguous): render 35-41ms with slice 3 built or not unknown — attribution is why the stamp exists; takes 7+ are
|
||||
readable.** If the breakdown says resolve dominates, the suspects are pre-cached chat/config thrash or the web frame; if blit dominates, the general-path element sizes. **Pattern — everything is a constructor-injected
|
||||
|
||||
@@ -0,0 +1,111 @@
|
||||
using System;
|
||||
using System.Diagnostics;
|
||||
using Xunit;
|
||||
using ytLive.Helpers;
|
||||
|
||||
namespace ytLive.Tests;
|
||||
|
||||
/// <summary>
|
||||
/// UNIT 0 (2026-10-05): the camera and the pump could both drop frames invisibly —
|
||||
/// a 60fps rawvideo container stamps frames by ARRIVAL, so a missing camera frame
|
||||
/// and a padded repeat look identical in the recorded file, and the old stats line
|
||||
/// counted SUBMITTED frames (always the full padded target) over an unknown number
|
||||
/// of renders. These tests pin the one shared counter that makes the two measurable:
|
||||
/// gaps, hitches, and the longest stall in a window of a timestamped stream.
|
||||
/// </summary>
|
||||
public sealed class FrameCadenceTests
|
||||
{
|
||||
private static long Ms(double ms) => (long)(ms * Stopwatch.Frequency / 1000d);
|
||||
|
||||
private static long AtMs(double ms) => Ms(ms);
|
||||
|
||||
[Fact]
|
||||
public void SteadyStream_ReportsNominalRateAndNoHitches()
|
||||
{
|
||||
var cadence = new FrameCadence();
|
||||
|
||||
// 30 frames at a true 33.3ms period: 29 gaps, none over the 50ms threshold.
|
||||
for (var i = 0; i < 30; i++) cadence.Add(AtMs(i * (1000d / 30d)));
|
||||
|
||||
var s = cadence.Summarize();
|
||||
Assert.Equal(30, s.Frames);
|
||||
Assert.Equal(29, s.Gaps);
|
||||
Assert.Equal(0, s.Hitches);
|
||||
Assert.Equal(30d, Math.Round(s.RatePerSecond, 0));
|
||||
Assert.Equal(33.3, Math.Round(s.AverageGapMs, 1));
|
||||
Assert.InRange(s.LongestGapMs, 33.0, 33.5);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void DroppedFrames_CountAsHitchesAndLongestGap()
|
||||
{
|
||||
var cadence = new FrameCadence();
|
||||
|
||||
// 30fps with two dropped frames (66.6ms gaps) and one double-frame (0.1ms):
|
||||
// the pattern the 2026-10-05 recording showed in the webcam corner.
|
||||
foreach (var ms in new[]
|
||||
{
|
||||
0.0, 33.3, 33.4, 66.7, 100.0, 133.3, 199.9, 233.2, 266.5, 333.1, 366.4,
|
||||
})
|
||||
cadence.Add(AtMs(ms));
|
||||
|
||||
var s = cadence.Summarize();
|
||||
Assert.Equal(11, s.Frames);
|
||||
Assert.Equal(10, s.Gaps); // the bunched pair still counts as an interval
|
||||
Assert.Equal(2, s.Hitches); // the two dropped-frame gaps at 30fps
|
||||
Assert.Equal(66.6, Math.Round(s.LongestGapMs, 1));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void OutOfOrderStamp_CountsAsFrameWithoutPoisoningGaps()
|
||||
{
|
||||
var cadence = new FrameCadence();
|
||||
|
||||
cadence.Add(AtMs(0));
|
||||
cadence.Add(AtMs(33.3));
|
||||
cadence.Add(AtMs(10.0)); // out of order — a stamp from another clock domain
|
||||
cadence.Add(AtMs(66.6));
|
||||
|
||||
var s = cadence.Summarize();
|
||||
Assert.Equal(4, s.Frames);
|
||||
Assert.Equal(2, s.Gaps); // the two valid intervals only
|
||||
Assert.Equal(0, s.Hitches);
|
||||
Assert.Equal(33.3, Math.Round(s.LongestGapMs, 1));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Reset_StartsFreshWindowAndKeepsThreshold()
|
||||
{
|
||||
var cadence = new FrameCadence(hitchThresholdMs: 20);
|
||||
|
||||
cadence.Add(AtMs(0));
|
||||
cadence.Add(AtMs(100));
|
||||
Assert.Equal(1, cadence.Hitches);
|
||||
|
||||
cadence.Reset();
|
||||
|
||||
Assert.Equal(0, cadence.Frames);
|
||||
Assert.Equal(0, cadence.Gaps);
|
||||
Assert.Equal(20, cadence.Summarize().HitchThresholdMs);
|
||||
Assert.Contains("no frames", cadence.Summarize().Describe("cam"));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Describe_NamesTheWindowRateHitchesAndLongestGap()
|
||||
{
|
||||
var cadence = new FrameCadence();
|
||||
|
||||
foreach (var ms in new[] { 0.0, 16.7, 33.4, 50.1, 120.0 }) cadence.Add(AtMs(ms));
|
||||
|
||||
Assert.Equal(
|
||||
"webcam 5f/0.1s 33.3/s, 4 gaps, 1 >= 50ms, avg 30.0ms, longest 69.9ms",
|
||||
cadence.Summarize().Describe("webcam"));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void ThresholdsMustBePositive()
|
||||
{
|
||||
Assert.Throws<ArgumentOutOfRangeException>(() => new FrameCadence(0));
|
||||
Assert.Throws<ArgumentOutOfRangeException>(() => new FrameCadence(-1));
|
||||
}
|
||||
}
|
||||
@@ -1,5 +1,6 @@
|
||||
using System.Net;
|
||||
using System.Text;
|
||||
using System.Text.RegularExpressions;
|
||||
using Xunit;
|
||||
using ytLive.Models;
|
||||
using ytLive.Services;
|
||||
@@ -573,6 +574,52 @@ public class FramePumpTests
|
||||
$"resolver ran on the starting thread ({startThread}) — the loop is back on the caller context"));
|
||||
}
|
||||
|
||||
/// <summary>The ONE integration test for UNIT 0's stats correction (2026-10-05).
|
||||
/// The old line divided every average by the number of frames SUBMITTED — deadline
|
||||
/// padding always fills that to the full 60fps target, so a pump fed one webcam
|
||||
/// frame per 200ms still logged "300/300 frames per 5s" and the "avg render" was
|
||||
/// diluted by an order of magnitude. It also never reported the webcam at all, so
|
||||
/// a camera delivering 25 of its 30 frames was indistinguishable from one
|
||||
/// delivering 30. Here a webcam advancing a new Epoch every ~200ms must show up
|
||||
/// as FEWER distinct frames than submits, with the render count beside them.</summary>
|
||||
[Fact]
|
||||
public async Task Stats_ReportRealRendersAndDistinctWebcamFrames_NotJustPaddedSubmits()
|
||||
{
|
||||
var webcamPixels = SceneCompositorTests.Solid(64, 48, 0, 255, 0).BgraPixels;
|
||||
var scene = new Scene { Name = "Live" };
|
||||
scene.Elements.Add(new WebcamSceneConfig { Name = "Webcam", IsVisible = true });
|
||||
var encoder = new FakeEncoder();
|
||||
var log = new List<string>();
|
||||
var epoch = 0L;
|
||||
using var pump = NewPump(encoder, scene: () => scene, log: log, resolve: e =>
|
||||
{
|
||||
if (!e.IsWebcam) return null;
|
||||
// One NEW webcam frame every ~200ms — slower than the 60fps container,
|
||||
// exactly the case the old stats line hid completely.
|
||||
Thread.Sleep(200);
|
||||
epoch++;
|
||||
return new VideoFrame(64, 48, webcamPixels) { Epoch = epoch };
|
||||
});
|
||||
|
||||
await pump.StartAsync();
|
||||
var deadline = DateTime.UtcNow.AddSeconds(8);
|
||||
while (DateTime.UtcNow < deadline && !log.Any(l => l.Contains("webcam ")))
|
||||
await Task.Delay(50);
|
||||
await pump.StopAsync();
|
||||
|
||||
var stats = log.FirstOrDefault(l => l.StartsWith("FramePump stats:") && l.Contains("webcam "));
|
||||
Assert.True(stats != null, $"no cadence-bearing stats line in: {string.Join(" | ", log)}");
|
||||
|
||||
var submits = int.Parse(Regex.Match(stats!, @"stats: (\d+)/").Groups[1].Value);
|
||||
var renders = int.Parse(Regex.Match(stats!, @"from (\d+) renders").Groups[1].Value);
|
||||
var webcamFrames = int.Parse(Regex.Match(stats!, @"webcam (\d+)f/").Groups[1].Value);
|
||||
|
||||
Assert.True(submits > 0, stats);
|
||||
Assert.True(renders > 0 && renders <= submits, $"renders {renders} vs submits {submits}: {stats}");
|
||||
Assert.True(webcamFrames > 0 && webcamFrames < submits,
|
||||
$"webcam frames {webcamFrames} should be far fewer than the {submits} padded submits: {stats}");
|
||||
}
|
||||
|
||||
private sealed class InlineSyncContext : SynchronizationContext
|
||||
{
|
||||
public override void Post(SendOrPostCallback d, object? state) => d(state);
|
||||
|
||||
Reference in New Issue
Block a user