diff --git a/HANDOFF.md b/HANDOFF.md
index 72150ab..4c7dde0 100644
--- a/HANDOFF.md
+++ b/HANDOFF.md
@@ -1,13 +1,79 @@
# HANDOFF — current state
**Branch:** `main` (pre-1.0, no feature branches — creator ruling 2026-08-24).
-**Pushed 2026-10-04:** `origin/main` = `048cab7`. **NOT PUSHED since — local `main` is 5 commits
-ahead and the tree is clean.** The stale-push-state failure `MyMistakes.md` warns about has now bitten
+**Pushed 2026-10-04:** `origin/main` = `048cab7`. **NOT PUSHED since — local `main` is ahead and the
+tree is clean.** The stale-push-state failure `MyMistakes.md` warns about has now bitten
twice, so the rule stands: **check `git rev-list --count origin/main..main` before trusting any line
here about push state.**
Unpushed: `5ad0106` broadcast diagnostic → `74da99c` + `0fb2f9c` privacy/Test rulings →
-`8fbc370` diagnostic parse fix → `07a088a` **the explicit-bind fix** → `509ef36` window placement.
+`8fbc370` diagnostic parse fix → `07a088a` **the explicit-bind fix** → `509ef36` window placement →
+`b2f2197` handoff → `UNIT 0` frame-cadence measurement.
+
+## 🚧 IN FLIGHT 2026-10-05 — UNIT 0 shipped: a dropped webcam frame is finally measurable
+
+**The measurement the map never had.** Take `ty-20261005-1001-821-0001.mp4` (85.6s, 1920×1080,
+H.264 NVENC, 5124 frames @60) holds **25.6 distinct webcam images/s** in its lower-right region
+against the camera's 30fps: 1,153 normal 33ms intervals, **281 gaps of 50–100ms**, 96 bunched 17ms
+gaps. One hitch in five — the judder a DHH creator lipreads against, invisible in a file that muxes
+flawlessly. Measured with `signalstats`/YDIF over `crop=480:270:1400:780`; the bundled ffmpeg has no
+`mpdecimate`. ⛔ The old "webcam renders at ~18fps" claim was wrong (different session, counted all
+newly rendered layers) — corrected in `ai.md`.
+
+**Shipped:** `Helpers/FrameCadence.cs` (pure counter: frames, gaps, hitches ≥50ms, longest/avg), fed
+from three points — `MediaCaptureFrameSource` logs `Camera cadence WxH: arrivals … | camera …`
+(our delivery vs the device's own `SystemRelativeTime`), and `FramePump` logs distinct webcam
+`Epoch`s that reached a composite. Full detail in `ai.md` → "Frame cadence".
+
+**Also fixed, and it was a real measurement bug:** every pump average was divided by the number of
+frames **submitted**, which deadline padding always fills to the full 60fps target — so a pump fed
+one webcam frame per 200ms logged "300/300 frames, avg render 12.4ms". Render/resolve/wait now divide
+by `statRenders` and the line reads `300/300 frames per 5s from 60 renders`. That pair IS the
+duplicate-on-lag ratio. Test: `FramePumpTests.Stats_ReportRealRendersAndDistinctWebcamFrames_…`.
+
+**⏭ NEXT STEP — needs the creator, one short recording (60s is plenty):** record with the webcam
+visible and paste nothing — I read `%APPDATA%\ytLlive\startup.log` for the `Camera cadence` and
+`FramePump stats` lines. Reading them together names the culprit: camera steady + arrivals ragged =
+we drop it (reader/compositor); ragged in both = device/USB/driver; pump `webcam` below `arrivals`
+= the compositor missed frames the reader already had. **Then Unit 1** (exclusive lock + 720p, see
+below) is judged against those numbers.
+
+**Agreed but NOT yet built (decided 2026-10-05, so a later session does not re-derive it):**
+- **Unit 1 — exclusive camera lock + 720p on the camera the creator picked.** Open with
+ `MediaCaptureSharingMode.ExclusiveControl`; if init is refused, reopen the **SAME camera** in
+ `SharedReadOnly`. Target MJPG ≥30fps up to 1280×720, ranked by frame rate then size (this also fixes
+ the existing candidate-ordering bug). **The creator REVERSED the ban on taking an exclusive camera
+ lock** — that ban was never written into the map, which is why the reversal must be recorded this
+ time. Accepted trade-off: an app needing camera *control* that opens it after us will fail.
+ - ✅ **API probe DONE 2026-10-05 against the actual 19041.57 projection** (a throwaway
+ compile-probe file, since deleted — do not re-derive, and note the build cost of finding out):
+ `MediaCaptureSharingMode.ExclusiveControl` **is** projected, so the old comment in
+ `MediaCaptureFrameSource.cs` ("Exclusive is not projected by the 19041 SDK") is wrong and must be
+ corrected. `Exclusive` and `SharedReadOnly` are the **only two** `MediaCaptureSharingMode` values.
+ `MediaFrameSource.SupportedFormats` and `CurrentFormat` exist.
+ ⛔ **`MediaFrameSource.SetFormatAsync(MediaFrameFormat)` is a DEAD END here** — it takes a
+ `MediaFrameFormat`, and that type has **no public constructor and no static factory** in the
+ projection (it is only obtainable from a live frame's `VideoMediaFrame.Format`). The two-argument
+ `(subtype, properties)` overload in Microsoft's docs does not exist here. So the format change must
+ go through the `VideoDeviceController.SetMediaStreamPropertiesAsync` path the code ALREADY has —
+ it is only refused because `SharingMode` is `SharedReadOnly`. Under an exclusive lock the existing
+ negotiation should simply start working, which is a much smaller change than planned.
+ - **Remaining unknown (needs a real device, not research):** whether `SetMediaStreamPropertiesAsync`
+ actually succeeds under `ExclusiveControl` on this C920. The `MediaFrameSource.SupportedFormats`
+ route is the fallback probe if it refuses — enumerate, and log what the device offers before
+ assuming anything.
+- **Unit 2 — blinking "SD" badge in the preview only**, when a better format was available and we did
+ not get it. Top-right corner of the webcam, or the 1:30 point ON THE RIM for a round clip (a round
+ webcam has no corner); outside the mirror transform. ~1Hz opacity blink (under the 3/sec
+ photosensitivity limit), tooltip on hover only, **must never appear in the recording or stream**
+ (the compositor draws from camera frames, not the WPF tree). Style lives in `Themes/Controls.xaml`
+ — a DataTrigger storyboard cannot be stopped inside a DataTemplate (`PreviewPane.xaml:168-171`).
+- ⛔ **NO fallback-camera setting, ever** — creator ruling: *"no fallback - that's bypassing the
+ user's explicit choice."* Picking a vcam is a user action in the camera picker. Corollary, also
+ recorded in `ai.md`: **Windows Studio Effects is NOT a vcam** — it runs inside the physical camera's
+ driver and never appears as a selectable device. Only Broadcast / XSplit VCam install one.
+- ⛔ **Background removal: permanently out.** Creator confirmed the existing note: *"no - not doing
+ it. the note is correct and accurate."* `ai.md` said "milestone 2", which was stale — now corrected.
## 🚨 FIXED 2026-10-04 — broadcasts never went live because they were never BOUND
@@ -281,11 +347,12 @@ Detail and the open 11.12 re-check: [`TASKS/task-50-publish-recording.md`](TASKS
## Landmines
-- **Frame pump jitter, unquantified (2026-10-04).** The same test stream logged **86 stalls
- per 5s** (`FramePump stall: iteration 40–50ms` against a 17ms budget; 142 such lines in an
- 8-second window) while `FramePump stats` still read 301/300 frames, 0 dropped. Nominal 60fps
- with zero drops *and* repeated 3× overruns is a contradiction worth one measurement, not a
- guess. Do not "fix" it from the log alone.
+- **Frame pump jitter — counters now honest, cause NOT yet named (2026-10-05).** The same test
+ stream logged **86 stalls per 5s** (`FramePump stall: iteration 40–50ms` against a 17ms budget)
+ while `FramePump stats` read 301/300 frames, 0 dropped. UNIT 0 removed the reason those two
+ numbers could disagree (the averages were diluted by padded submits) and added the counters that
+ separate reader from compositor — **but no one has read a post-UNIT-0 log yet.** A creator
+ recording is required first. Do not "fix" the pump before that reading.
- **Stale fact inside `TASKS.md` (left alone, flagged here per the Scope Lock).** The "1.0 gates"
section asserts "**TASK 36 is now shipped**", but the catalog row 36 reads **☐ Queued** and
`task-36-gold-pass.md` still shows items 1–6 unchecked. The line most likely means *item 2*
@@ -293,9 +360,10 @@ Detail and the open 11.12 re-check: [`TASKS/task-50-publish-recording.md`](TASKS
flagging rather than fixing because it is not this unit's job.
- **A stale `ytLive.exe` (PID 2544) locked `bin/.../ytLive.exe`** and broke `dotnet run` with
MSB3027. Killed. **If the build fails to copy `ytLive.exe`, check for a running instance first.**
-- **`LayerReorderPersistenceTests.RealMouseDrag_OnTheLayerList_PersistsTheReorder` is flaky by
- environment** (needs an interactive desktop session; 2 fail / 1 pass in isolation). **Run the
- suite with ytLive CLOSED.** Not a regression.
+- **`LayerReorderPersistenceTests.RealMouseDrag_OnTheLayerList_PersistsTheReorder` needs an
+ interactive desktop session.** As of 2026-10-05 it fails deterministically here (`ImgC` vs
+ `ImgA`), verified identical on a clean tree. **Run the suite with ytLive CLOSED** and expect this
+ one to fail in this shell. Not a regression — don't diagnose it from a WSL-shell failure.
- **`RealAppHost.RunAsync` exists for a reason** — a frame-pump test MUST `await` inside it. A
blocking wait occupies the one shared STA thread and **hangs the whole suite with no output.**
Always background a `vstest` run and poll the log; a foreground piped `vstest` returns nothing
@@ -308,11 +376,17 @@ Detail and the open 11.12 re-check: [`TASKS/task-50-publish-recording.md`](TASKS
## Test state
-**395/395, 0 warnings, re-verified 2026-10-04 on a clean test-project build** (381 + 3 diagnostic
-`FirstItem` cases + 1 window fact + 3 placement theory cases + 1 bind test, less 1 removed).
-Build the **test** project, not just the app project, or VSTest runs a stale DLL.
+**402/402, 0 warnings, verified 2026-10-05** (395 + 6 `FrameCadenceTests` + 1 pump-stats integration
+test). Build the **test** project, not just the app project, or VSTest runs a stale DLL.
**Launch exe for the creator's test run: `bin/Debug/net8.0-windows10.0.19041.0/ytLive.exe`.**
+**Confirmed environmental flake, NOT a regression (re-checked 2026-10-05, and it now fails
+deterministically rather than 2-of-3):** `LayerReorderPersistenceTests.RealMouseDrag_OnTheLayerList_
+PersistsTheReorder` fails with `Expected "ImgC", Actual "ImgA"` at `RealAppHost.Run` line 68. **It
+fails identically on a stashed clean tree**, so it has nothing to do with UNIT 0 — it needs an
+interactive desktop session this shell does not have (the mouse drag never lands). Do not chase it;
+it is listed under Landmines as the known interactive-session test.
+
**⚠️ New flake, do not chase it as a regression:** on the *first* full-suite run,
`AudioPipelineTests.MasterLimiter_CapsTheLiveMix_OnThePipe` failed with `Expected 0.891,
Actual 0` — a **zero**, not a slightly-off value, which is the signature of the pipe not being
@@ -335,12 +409,12 @@ whole-suite scheduling artifact of the same family as the `RealAppHost` STA land
1. **~PUSH, then the creator's 60-second Test** — see the FIXED box above. Packaging is NOT next;
the creator's ruling 2026-10-04: still in active development, so ship-time work waits and queued
features come first.
-2. **Frame-pump jitter (queued, unquantified):** ~86 stalls per 5 s in one log window, 142 in an
- 8-second window, while stats reported 301/300 frames and **zero** drops. Measure before touching
- anything — the counter and the missing-frame count currently disagree, and "fixing" the pump
- without knowing which number is wrong risks making it worse. **Creator to-do, not yet queued:
- long-run resource/consumption/recycling benchmarking** (how long the app stays healthy under full
- burden) — record only, do not start it.
+2. **Webcam cadence → Unit 1 (exclusive lock + 720p).** UNIT 0 is shipped and the counters are in
+ `startup.log`; the reading needs one 60-second creator recording (details + the exact
+ read-the-log-together table in the IN FLIGHT box above). Then Unit 1: exclusive camera lock +
+ 720p, and Unit 2: the blinking SD badge. **Creator to-do, not yet queued: long-run
+ resource/consumption/recycling benchmarking** (how long the app stays healthy under full burden) —
+ record only, do not start it.
2. **The packaging project + `Package.appxmanifest`** (TASK 48 items 2–3) — the next code unit.
Full trust, `webcam`/`microphone`, English only; Velopack deleted. Add the
`DefaultRecordFolder()` comment in the same unit, and the WebView2 framework dependency +
diff --git a/Helpers/FrameCadence.cs b/Helpers/FrameCadence.cs
new file mode 100644
index 0000000..85ac51f
--- /dev/null
+++ b/Helpers/FrameCadence.cs
@@ -0,0 +1,123 @@
+using System;
+using System.Diagnostics;
+
+namespace ytLive.Helpers;
+
+///
+/// Arrival cadence for one window of a timestamped frame stream: how many frames
+/// arrived, how far apart they were, and how many gaps were long enough to read
+/// as a hitch. Fed any shared monotonic clock (
+/// values, or a device's system-relative stamps on the same clock) and read through
+/// .
+///
+/// Why it exists: a 60fps container makes a missing camera frame indistinguishable
+/// from a padded repeat. rawvideo carries no per-frame timestamps — ffmpeg stamps
+/// frames by ARRIVAL at the declared rate — so the OBS "duplicate on lag" loop
+/// (FramePump) emits 300 authored frames per 5s whether the renderer managed 300
+/// composites or 84. The recorded file therefore cannot name the stage that lost
+/// the frames; only these counters can. The same counter run twice tells the two
+/// apart: the camera's OWN timestamps vs our ARRIVAL timestamps. Steady camera
+/// stamps with ragged arrivals means WE dropped it (compositor/reader); ragged in
+/// both means the camera, USB link, or driver did.
+///
+public sealed class FrameCadence
+{
+ /// Default hitch threshold in ms. 50ms is ~1.5x a 30fps period — a
+ /// missed 30fps frame arrives as ~66ms — so this counts dropped frames without
+ /// flagging ordinary jitter (a 33ms frame can legitimately measure 35ms).
+ public const double DefaultHitchThresholdMs = 50;
+
+ private readonly double _hitchThresholdMs;
+ private long _firstTicks;
+ private long _previousTicks;
+ private double _gapTicksTotal;
+ private int _frames;
+ private int _gaps;
+ private int _hitches;
+ private double _longestGapMs;
+
+ public FrameCadence(double hitchThresholdMs = DefaultHitchThresholdMs)
+ {
+ if (hitchThresholdMs <= 0)
+ throw new ArgumentOutOfRangeException(nameof(hitchThresholdMs), hitchThresholdMs,
+ "Hitch threshold must be positive.");
+ _hitchThresholdMs = hitchThresholdMs;
+ }
+
+ public int Frames => _frames;
+ public int Gaps => _gaps;
+ public int Hitches => _hitches;
+ public double HitchThresholdMs => _hitchThresholdMs;
+
+ /// Records one frame at on the shared
+ /// monotonic clock. Monotonicity is asserted, not trusted: an out-of-order stamp
+ /// (a device whose clock domain differs from ours, a wrapped counter, a dropped
+ /// timestamp) is counted as a frame but produces no gap AND does not become the
+ /// reference for the next one — otherwise a single bad stamp would poison both
+ /// its own interval and the one after it, inventing a multi-second "hitch".
+ public void Add(long timestampTicks)
+ {
+ if (_frames == 0)
+ {
+ _firstTicks = timestampTicks;
+ _previousTicks = timestampTicks;
+ _frames++;
+ return;
+ }
+
+ var delta = timestampTicks - _previousTicks;
+ _frames++;
+ if (delta <= 0) return;
+ var gapMs = delta * 1000d / Stopwatch.Frequency;
+ _gaps++;
+ _gapTicksTotal += gapMs;
+ if (gapMs > _longestGapMs) _longestGapMs = gapMs;
+ if (gapMs >= _hitchThresholdMs) _hitches++;
+ _previousTicks = timestampTicks;
+ }
+
+ public FrameCadenceSummary Summarize()
+ {
+ var spanMs = _frames >= 2
+ ? (_previousTicks - _firstTicks) * 1000d / Stopwatch.Frequency
+ : 0d;
+ // Rate is gaps-per-span, not frames-per-span: N frames span N-1 periods, and
+ // counting the extra interval would report one extra frame per window.
+ var rate = _gaps > 0 && spanMs > 0 ? _gaps / (spanMs / 1000d) : 0d;
+ return new FrameCadenceSummary(
+ _frames, rate, spanMs, _gaps, _hitches, _longestGapMs,
+ _gaps > 0 ? _gapTicksTotal / _gaps : 0d, _hitchThresholdMs);
+ }
+
+ /// Starts a fresh window, keeping the configured threshold.
+ public void Reset()
+ {
+ _firstTicks = 0;
+ _previousTicks = 0;
+ _gapTicksTotal = 0;
+ _frames = 0;
+ _gaps = 0;
+ _hitches = 0;
+ _longestGapMs = 0;
+ }
+}
+
+/// One measured cadence window — see .
+public readonly record struct FrameCadenceSummary(
+ int Frames,
+ double RatePerSecond,
+ double SpanMs,
+ int Gaps,
+ int Hitches,
+ double LongestGapMs,
+ double AverageGapMs,
+ double HitchThresholdMs)
+{
+ /// One-line log form. Identical in shape from every producer (camera
+ /// source, frame pump) so two lines can be read against each other: same window
+ /// length, same fields, same order.
+ public string Describe(string label) => Frames == 0
+ ? $"{label} no frames"
+ : $"{label} {Frames}f/{SpanMs / 1000d:F1}s {RatePerSecond:F1}/s, {Gaps} gaps, " +
+ $"{Hitches} >= {HitchThresholdMs:F0}ms, avg {AverageGapMs:F1}ms, longest {LongestGapMs:F1}ms";
+}
\ No newline at end of file
diff --git a/Helpers/index.md b/Helpers/index.md
index 0de6cb2..3050aca 100644
--- a/Helpers/index.md
+++ b/Helpers/index.md
@@ -14,6 +14,7 @@ Cross-cutting utilities. See [`schema.md`](../schema.md) for the memory-map conv
| `InverseBoolToVisibilityConverter.cs` / `NotNullToVisibilityConverter.cs` | XAML value converters for visibility bindings |
| `AllTrueToVisibilityConverter.cs` | Multi-value AND-gate → Visible; gates the layer-row capture menu items on row-IsBackground + staged-scene-is-Live (TASK 25) |
| `CameraConflictProbe.cs` | Best-effort diagnostic: when a camera won't start, enumerates running processes known to hold cameras (OBS, Zoom, Teams, NVIDIA Broadcast, browsers, etc.) — Windows doesn't expose "which process has this device" via any public API, so this is a suspect list, not a verdict |
+| `FrameCadence.cs` | Pure arrival-cadence counter (`FrameCadence` + `FrameCadenceSummary.Describe`): frames, gaps, hitches, longest/avg gap for a window of a timestamped stream. Fed by `MediaCaptureFrameSource` (camera stamps vs our arrivals) and `FramePump` (distinct webcam Epochs that reached the output) — UNIT 0, see ai.md "Frame cadence" |
Related: [`Themes/Controls.xaml`](../Themes/Controls.xaml) styles the lists this
class backs; [`Models/index.md`](../Models/index.md) and
diff --git a/Services/Encoder/FramePump.cs b/Services/Encoder/FramePump.cs
index 6bc3573..0bc8e84 100644
--- a/Services/Encoder/FramePump.cs
+++ b/Services/Encoder/FramePump.cs
@@ -338,6 +338,27 @@ public sealed class FramePump : IDisposable
int stalls = 0;
long lastCacheHits = 0, lastCacheRenders = 0;
var statsNext = DateTime.UtcNow + TimeSpan.FromSeconds(5);
+
+ // Real-render accounting (UNIT 0, 2026-10-05): every average above was
+ // divided by statFrames — the number of frames SUBMITTED, which deadline
+ // padding always fills to the full target even when the renderer only
+ // produced a fifth of them. A 60fps container therefore reported "300/300
+ // frames, avg render 12ms" while the source actually fed it 60 distinct
+ // images. Render cost is now divided by the render count, and the render
+ // count is reported beside the submit count — the two numbers side by side
+ // ARE the duplicate-on-lag ratio, which is the only honest reading of how
+ // much of the stream is a repeat.
+ int statRenders = 0;
+
+ // Webcam cadence (UNIT 0, 2026-10-05): the pump sees each element's resolved
+ // frame, and VideoFrame.Epoch is the webcam ring's monotonic generation — so a
+ // NEW epoch means the webcam actually delivered something new between two of
+ // our ticks. This is the counterpart to MediaCaptureFrameSource's arrivals
+ // line: it measures what reached the OUTPUT, not what the camera produced. If
+ // the camera reports 30/s and this reports 25/s, the loss is between them.
+ var webcamCadence = new Helpers.FrameCadence();
+ var seenWebcamEpochs = new HashSet();
+
void ReportStats()
{
if (DateTime.UtcNow < statsNext) return;
@@ -360,19 +381,23 @@ public sealed class FramePump : IDisposable
}
_log?.Invoke(statFrames == 0
? "FramePump stats: NO frames produced in 5s (loop stalled?)" + (detail.Length > 0 ? " | " + detail : "")
- : $"FramePump stats: {statFrames}/{target:F0} frames per 5s, " +
- $"avg render {renderTicks / (double)System.Diagnostics.Stopwatch.Frequency * 1000 / statFrames:F1}ms " +
- $"(resolve {resolveTicks / (double)System.Diagnostics.Stopwatch.Frequency * 1000 / statFrames:F1}), " +
+ : $"FramePump stats: {statFrames}/{target:F0} frames per 5s from {statRenders} renders, " +
+ $"avg render {renderTicks / (double)System.Diagnostics.Stopwatch.Frequency * 1000 / Math.Max(1, statRenders):F1}ms " +
+ $"(resolve {resolveTicks / (double)System.Diagnostics.Stopwatch.Frequency * 1000 / Math.Max(1, statRenders):F1}), " +
$"avg submit {submitTicks / (double)System.Diagnostics.Stopwatch.Frequency * 1000 / statFrames:F1}ms, "
- + $"avg wait {waitTicks / (double)System.Diagnostics.Stopwatch.Frequency * 1000 / statFrames:F1}ms, "
+ + $"avg wait {waitTicks / (double)System.Diagnostics.Stopwatch.Frequency * 1000 / Math.Max(1, statRenders):F1}ms, "
+ $"worst render {worstRender / (double)System.Diagnostics.Stopwatch.Frequency * 1000:F1}ms, "
+ $"worst submit {worstSubmit / (double)System.Diagnostics.Stopwatch.Frequency * 1000:F1}ms, "
- + $"dropped {dropped}, stalls {stalls}" + cacheDetail + (detail.Length > 0 ? " | " + detail : ""));
+ + $"dropped {dropped}, stalls {stalls}{cacheDetail}, {webcamCadence.Summarize().Describe("webcam")}"
+ + (detail.Length > 0 ? " | " + detail : ""));
worstRender = 0;
worstSubmit = 0;
renderTicks = submitTicks = resolveTicks = waitTicks = 0;
statFrames = 0;
+ statRenders = 0;
stalls = 0;
+ webcamCadence.Reset();
+ seenWebcamEpochs.Clear();
statsNext = DateTime.UtcNow + TimeSpan.FromSeconds(5);
}
@@ -387,6 +412,8 @@ public sealed class FramePump : IDisposable
var frame = _frameResolver(element);
resolveSw.Stop();
resolveTicks += resolveSw.ElapsedTicks;
+ if (element.IsWebcam && frame != null && seenWebcamEpochs.Add(frame.Epoch))
+ webcamCadence.Add(System.Diagnostics.Stopwatch.GetTimestamp());
return frame;
}
@@ -444,6 +471,7 @@ public sealed class FramePump : IDisposable
renderSw.Stop();
renderTicks += renderSw.ElapsedTicks;
if (renderSw.ElapsedTicks > worstRender) worstRender = renderSw.ElapsedTicks;
+ statRenders++;
IFfmpegEncoder? encoder;
lock (_gate) encoder = _encoder;
diff --git a/Services/MediaCaptureFrameSource.cs b/Services/MediaCaptureFrameSource.cs
index 16c1d66..5aa3685 100644
--- a/Services/MediaCaptureFrameSource.cs
+++ b/Services/MediaCaptureFrameSource.cs
@@ -50,7 +50,9 @@ public sealed class MediaCaptureFrameSource : ICameraFrameSource
// Fallback ladder: prefer the camera's VideoPreview stream, but if the
// reader refuses to start there (NoVideoFrameAvailable), retry against
// its VideoRecord stream — some devices only deliver through it.
- // (MediaCaptureSharingMode.Exclusive is not projected by the 19041 SDK.)
+ // (The exclusive-lock upgrade is a separate, later unit — creator
+ // ruling 2026-10-05; see HANDOFF.md. It is NOT blocked by an SDK gap:
+ // `MediaCaptureSharingMode.ExclusiveControl` IS projected in 19041.)
if (!await TryStartAsync(preferVideoRecord: false))
{
if (!await TryStartAsync(preferVideoRecord: true))
@@ -246,6 +248,7 @@ public sealed class MediaCaptureFrameSource : ICameraFrameSource
_frameReader = reader;
_lastError = null;
_isFailed = false;
+ StartCadenceWindow();
return true;
}
@@ -287,6 +290,30 @@ public sealed class MediaCaptureFrameSource : ICameraFrameSource
}
}
+ private const int CadenceWindowSeconds = 5;
+
+ private void StartCadenceWindow()
+ {
+ _arrivalCadence.Reset();
+ _cameraCadence.Reset();
+ _cadenceNextReportTicks = System.Diagnostics.Stopwatch.GetTimestamp()
+ + System.Diagnostics.Stopwatch.Frequency * CadenceWindowSeconds;
+ }
+
+ /// Logs one cadence window and opens the next. The size rides along
+ /// because the whole question started as "the webcam is soft": a future format
+ /// change must be readable in the SAME line that proves the frame rate.
+ private void ReportCadence(int width, int height)
+ {
+ _cadenceNextReportTicks = System.Diagnostics.Stopwatch.GetTimestamp()
+ + System.Diagnostics.Stopwatch.Frequency * CadenceWindowSeconds;
+ var arrivals = _arrivalCadence.Summarize();
+ var camera = _cameraCadence.Summarize();
+ _arrivalCadence.Reset();
+ _cameraCadence.Reset();
+ AppLog.Write($"Camera cadence {width}x{height}: {arrivals.Describe("arrivals")} | {camera.Describe("camera")}");
+ }
+
private void UnsubscribeCaptureEvents(MediaCapture capture)
{
capture.Failed -= OnCaptureFailed;
@@ -349,12 +376,35 @@ public sealed class MediaCaptureFrameSource : ICameraFrameSource
return fresh;
}
+ // Arrival vs production cadence (UNIT 0, 2026-10-05): the recorded file cannot
+ // tell a dropped camera frame from a padded repeat — rawvideo stamps frames by
+ // ARRIVAL at the declared 60fps, so a webcam delivering 25 of its 30 frames still
+ // muxes a full-length file. Two counters, one reader, settle it: the camera's OWN
+ // SystemRelativeTime stamps (device/USB/driver cadence) against the moment the
+ // reader handed each frame to us (our cadence, including coalescing drops — the
+ // reader deliberately keeps only the LATEST frame, so falling behind is silent).
+ // Steady camera + ragged arrivals = we are behind. Ragged in both = the device.
+ private readonly FrameCadence _arrivalCadence = new();
+ private readonly FrameCadence _cameraCadence = new();
+ private long _cadenceNextReportTicks;
+
private void OnFrameArrived(MediaFrameReader sender, MediaFrameArrivedEventArgs args)
{
+ var arrivedTicks = System.Diagnostics.Stopwatch.GetTimestamp();
using var frame = sender.TryAcquireLatestFrame();
var videoFrame = frame?.VideoMediaFrame?.SoftwareBitmap;
if (videoFrame == null) return;
+ // SystemRelativeTime is QPC-domain, the same clock Stopwatch reads here, so the
+ // camera's own frame spacing is measurable without a second conversion. It is
+ // NULLABLE in the projection (null = the device supplied no stamp), so the
+ // production side of the comparison is then simply missing from the line —
+ // arrivals alone still measure what reached us.
+ _arrivalCadence.Add(arrivedTicks);
+ if (frame!.SystemRelativeTime is { } cameraStamp)
+ _cameraCadence.Add((long)(cameraStamp.TotalSeconds * System.Diagnostics.Stopwatch.Frequency));
+ if (arrivedTicks >= _cadenceNextReportTicks) ReportCadence(videoFrame.PixelWidth, videoFrame.PixelHeight);
+
var bitmap = videoFrame.BitmapPixelFormat == BitmapPixelFormat.Bgra8
? videoFrame
: SoftwareBitmap.Convert(videoFrame, BitmapPixelFormat.Bgra8);
diff --git a/TASKS.md b/TASKS.md
index 7a9a88f..4a5c4e2 100644
--- a/TASKS.md
+++ b/TASKS.md
@@ -156,6 +156,19 @@ second encoder path needing a "redirect". Record+simulcast is one ffmpeg with tw
adopts a solo camera as default. TASK 26's app-wide gate superseded by creator directive. The App
Settings webcam selector is now **TASK 40 Unit A** (plan saved 2026-09-22). Remaining next slice
(user-queued): static (+) catalog rows (Background, YouTubeEvent, quoted labels) with reason-greying.
+- **🔶 Webcam cadence → resolution — IN PROGRESS, creator-queued 2026-10-05** (DHH/lipreading: the
+ webcam judders, ~1 hitch in 5, invisible in the output file). **UNIT 0 shipped** (counters +
+ `Helpers/FrameCadence.cs`; see `ai.md` → "Frame cadence"). **Needs a creator recording before the
+ next unit.** Remaining units, all agreed 2026-10-05, one Good Dog test each:
+ - **Unit 1:** `MediaCaptureSharingMode.ExclusiveControl` + MJPG ≥30fps up to 1280×720 via the
+ existing `SetMediaStreamPropertiesAsync` path (refused only because `SharedReadOnly`); same-camera
+ `SharedReadOnly` retry. **The creator reversed the ban on taking an exclusive camera lock.**
+ API probe already done — see HANDOFF.md, incl. the `SetFormatAsync` dead end.
+ - **Unit 2:** blinking amber **"SD" badge**, preview only, never in the recording/stream, when a
+ better format was available and not obtained. Top-right corner, or the 1:30 rim point when the
+ clip is round.
+ - ⛔ **No fallback-camera feature, ever** — *"that's bypassing the user's explicit choice."*
+ - ⛔ **No background removal** — re-affirmed out 2026-10-05.
- **TASK 40 — App Settings round** — plan saved 2026-09-22 (`TASKS/task-40-app-settings-round.md`),
four units, one Good Dog test each: **A** active web camera section under Recordings (Change…
sub-menu = live enumeration incl. vcams; Tune… config dialog via `ICameraControlProbe`, live-only
@@ -357,7 +370,9 @@ See [`TASKS/research-youtube-api.md`](TASKS/research-youtube-api.md) — authori
These were explicitly ruled out:
- **D3D11 swap compositor** (superseded by software compositor; TASK 3 item 16)
-- **Background removal** (ONNX/DirectML; TASK 3 item 19)
+- **Background removal** (ONNX/DirectML; TASK 3 item 19) — ⛔ **re-affirmed out 2026-10-05**: *"no - not
+ doing it. the note is correct and accurate."* A vcam app does it free; an in-app model is a runtime,
+ a GPU/driver matrix, and per-scene lighting tuning. **Never reopen without a new ruling.**
- **OBS-style source expansion** (game capture, browser source, media playlist, VLC, color-key, MIDI — the minimal source set is deliberate; creators who need more have graduated to OBS)
- **Multi-layout** (auto-save, single layout; multi-layout = OBS territory)
- **Cross-process stream resume** (API makes it impossible; creator ruling 2026-09-01)
diff --git a/ai.md b/ai.md
index 29604f8..eeffc7e 100644
--- a/ai.md
+++ b/ai.md
@@ -483,6 +483,11 @@ This replaces the old five-seeder cluster (`Seed{Starting,Brb,Ending,Chat}Backgr
fails with "camera in use" (or, if init slips through, the device exposes no preview source). Fix:
quit Broadcast while streaming, or pick its virtual "NVIDIA Broadcast" device from the picker and the
app captures the processed feed. This is a real-device finding (Logitech `VID_046D&PID_082D`).
+ Correction (2026-10-05): **Windows Studio Effects is NOT a second option of this kind** — it runs
+ inside the physical camera's own driver, so it never appears as a selectable device. Only tools
+ that install a virtual camera (NVIDIA Broadcast, XSplit VCam) do. "Pick the vcam" is a user action in
+ the camera picker, never an automatic substitution: creator ruling 2026-10-05 — the app must not
+ bypass the user's explicit camera choice, so there is deliberately **no fallback-camera setting**.
- **TFM:** `net8.0-windows10.0.19041.0` (app + tests) pulls the WinRT projection from the SDK reference
packs — no NuGet package, no capability manifest (unpackaged desktop app works; the Windows privacy
camera toggle still applies). `EnableWindowsTargeting` keeps WSL builds working.
@@ -597,8 +602,56 @@ This replaces the old five-seeder cluster (`Seed{Starting,Brb,Ending,Chat}Backgr
- **GPU posture:** webcam frames are CPU (GPU-agnostic; WPF hardware-presents the preview anyway). Hardware
encoders (NVENC/AMF/QSV) matter for the encoder task, not capture. D3DImage GPU compositing is deferred
to the encoder task.
-- **Background removal = milestone 2** — ONNX Runtime + DirectML (CPU fallback), MediaPipe Selfie
- Segmentation (Apache-2.0), wired into the same `VideoFrame` seam. Not part of milestone 1.
+- **Background removal = PERMANENTLY OUT** (creator ruling 2026-10-05, confirmed: "no - not doing
+ it. the note is correct and accurate"). The line here used to read "milestone 2", which was stale —
+ it was never a plan, only a leftover from TASK 3's original scoping. The rationale that stands:
+ a vcam app (NVIDIA Broadcast, XSplit VCam) does it free, and an in-app model means a runtime, a
+ GPU/driver matrix, and per-scene lighting tuning — support hell for a solo dev
+ (`TASKS/task-36-gold-pass.md`, `TASKS/task-03-capture-pipeline.md` item 19). Don't reopen it.
+
+### Frame cadence — how a dropped frame became measurable (UNIT 0, 2026-10-05)
+
+**The problem this exists to solve.** The recorded file cannot distinguish a dropped camera frame
+from a padded repeat. `-f rawvideo` carries no per-frame timestamps — ffmpeg assigns pts from frame
+order at the DECLARED fps — so a webcam delivering 25 of its 30 frames still muxes a full-length,
+correctly-timed 60fps file. Measured on take `ty-20261005-1001-821-0001.mp4` (85.6s, 1080p60): the
+lower-right webcam region held 25.6 distinct images/s against the camera's 30fps, with 1,153 normal
+33ms intervals, 281 gaps of 50–100ms, and 96 bunched 17ms gaps — roughly one hitch in five, which is
+what makes a face judder while the file looks flawless. (The "webcam renders at ~18fps" figure that
+used to sit in this map was WRONG: it came from a different session and counted every newly rendered
+layer, not webcam arrivals. Corrected 2026-10-05.)
+
+**`Helpers/FrameCadence.cs`** — one pure counter, three feeds. `Add(timestamp)` on any shared
+monotonic clock; `Summarize()` → frames, rate (gaps ÷ span, NOT frames ÷ span — N frames span N-1
+periods), gaps, hitches (default threshold 50ms ≈ 1.5× a 30fps period), longest and average gap;
+`Describe(label)` gives every producer the same one-line shape so two lines can be read against each
+other. An out-of-order stamp counts as a frame but produces no gap AND does not become the reference
+for the next interval — otherwise one bad stamp invents a multi-second "hitch" twice.
+
+**Three measurement points, and what each separates:**
+
+| Line | Fed by | Answers |
+|---|---|---|
+| `Camera cadence WxH: arrivals … \| camera …` | `MediaCaptureFrameSource` | device/USB/driver cadence (`MediaFrame.SystemRelativeTime`, QPC domain, nullable) vs what the reader handed us. `TryAcquireLatestFrame` keeps only the LATEST frame, so falling behind is silent by design — this is where that shows |
+| `webcam …` on the pump's stats line | `FramePump.TimedResolver` | distinct webcam `VideoFrame.Epoch` values that reached a composite — what actually got into the output |
+| `frames per 5s from R renders` | `FramePump` | submits vs REAL composites — the duplicate-on-lag ratio, the only honest reading of how much of the stream is a repeat |
+
+**Why the stats-line denominators changed (the actual bug).** Every average was divided by
+`statFrames`, the number of frames SUBMITTED — which deadline padding always fills to the full 60fps
+target. A pump fed one webcam frame per 200ms logged "300/300 frames per 5s, avg render 12.4ms"
+while producing 60 distinct composites: the render average was diluted ~5× and the loss was
+invisible. Render/resolve/wait are now divided by `statRenders`; submit stays per-frame (it really
+happens per submit). Regression test:
+`FramePumpTests.Stats_ReportRealRendersAndDistinctWebcamFrames_NotJustPaddedSubmits`.
+
+**Reading the three together:** camera 30/s steady + arrivals 25/s = WE dropped it (reader or
+compositor). Ragged in both = the device, the USB link, or the driver. Pump `webcam` below
+`arrivals` = the compositor missed frames the reader had already delivered.
+
+**Still unmeasured (next unit):** the camera is stuck at the C920 default 640×480 YUY2 @30 because
+all 185 `SetMediaStreamPropertiesAsync` attempts are refused under `SharedReadOnly`, so 720p MJPG is
+on the table and the compositor's headroom at ~3× the bytes is unknown. Measure first, then change
+the acquisition.
### Scene compositor (TASK 4 ship step 1 — shipped 2026-08-10, plan in TASKS.md)
@@ -943,7 +996,7 @@ scene each tick, resolves every element to its latest frame, composites it into
and paces frames into the encoder at the tier's FPS. **rawvideo is stamped by ARRIVAL:** ffmpeg assigns
pts from frame order at the declared fps — supply rate = output speed. A starved producer (pump < fps)
ships a time-lapse, truncated file with NO error (take-2 lesson, 2026-09-01); the pump therefore logs
-`FramePump stats: n/target frames per 5s, avg render Xms (resolve R), avg submit Yms` so a slow stage names itself — and since 2026-09-04 so does the BUILD: every build gets a GUID stamped
+`FramePump stats: n/target frames per 5s from R renders, avg render Xms (resolve R2), avg submit Yms` so a slow stage names itself (the `frames … from R renders` pair and the render-count denominators arrived with UNIT 0 — see "Frame cadence" below; before that every average was divided by the SUBMITTED count, which deadline padding always fills to the full target, so a pump fed one webcam frame per 200ms still logged "300/300 frames" and its render average was diluted ~an order of magnitude) — and since 2026-09-04 so does the BUILD: every build gets a GUID stamped
into `Helpers/BuildStamp` (csproj `GenerateBuildStamp` target, fresh per compile — no lying incremental build), logged at startup
("Build xxxxxxxx (compiled ...)"). **Take 6 (stamp-less, ambiguous): render 35-41ms with slice 3 built or not unknown — attribution is why the stamp exists; takes 7+ are
readable.** If the breakdown says resolve dominates, the suspects are pre-cached chat/config thrash or the web frame; if blit dominates, the general-path element sizes. **Pattern — everything is a constructor-injected
diff --git a/ytLive.Tests/FrameCadenceTests.cs b/ytLive.Tests/FrameCadenceTests.cs
new file mode 100644
index 0000000..7569917
--- /dev/null
+++ b/ytLive.Tests/FrameCadenceTests.cs
@@ -0,0 +1,111 @@
+using System;
+using System.Diagnostics;
+using Xunit;
+using ytLive.Helpers;
+
+namespace ytLive.Tests;
+
+///
+/// UNIT 0 (2026-10-05): the camera and the pump could both drop frames invisibly —
+/// a 60fps rawvideo container stamps frames by ARRIVAL, so a missing camera frame
+/// and a padded repeat look identical in the recorded file, and the old stats line
+/// counted SUBMITTED frames (always the full padded target) over an unknown number
+/// of renders. These tests pin the one shared counter that makes the two measurable:
+/// gaps, hitches, and the longest stall in a window of a timestamped stream.
+///
+public sealed class FrameCadenceTests
+{
+ private static long Ms(double ms) => (long)(ms * Stopwatch.Frequency / 1000d);
+
+ private static long AtMs(double ms) => Ms(ms);
+
+ [Fact]
+ public void SteadyStream_ReportsNominalRateAndNoHitches()
+ {
+ var cadence = new FrameCadence();
+
+ // 30 frames at a true 33.3ms period: 29 gaps, none over the 50ms threshold.
+ for (var i = 0; i < 30; i++) cadence.Add(AtMs(i * (1000d / 30d)));
+
+ var s = cadence.Summarize();
+ Assert.Equal(30, s.Frames);
+ Assert.Equal(29, s.Gaps);
+ Assert.Equal(0, s.Hitches);
+ Assert.Equal(30d, Math.Round(s.RatePerSecond, 0));
+ Assert.Equal(33.3, Math.Round(s.AverageGapMs, 1));
+ Assert.InRange(s.LongestGapMs, 33.0, 33.5);
+ }
+
+ [Fact]
+ public void DroppedFrames_CountAsHitchesAndLongestGap()
+ {
+ var cadence = new FrameCadence();
+
+ // 30fps with two dropped frames (66.6ms gaps) and one double-frame (0.1ms):
+ // the pattern the 2026-10-05 recording showed in the webcam corner.
+ foreach (var ms in new[]
+ {
+ 0.0, 33.3, 33.4, 66.7, 100.0, 133.3, 199.9, 233.2, 266.5, 333.1, 366.4,
+ })
+ cadence.Add(AtMs(ms));
+
+ var s = cadence.Summarize();
+ Assert.Equal(11, s.Frames);
+ Assert.Equal(10, s.Gaps); // the bunched pair still counts as an interval
+ Assert.Equal(2, s.Hitches); // the two dropped-frame gaps at 30fps
+ Assert.Equal(66.6, Math.Round(s.LongestGapMs, 1));
+ }
+
+ [Fact]
+ public void OutOfOrderStamp_CountsAsFrameWithoutPoisoningGaps()
+ {
+ var cadence = new FrameCadence();
+
+ cadence.Add(AtMs(0));
+ cadence.Add(AtMs(33.3));
+ cadence.Add(AtMs(10.0)); // out of order — a stamp from another clock domain
+ cadence.Add(AtMs(66.6));
+
+ var s = cadence.Summarize();
+ Assert.Equal(4, s.Frames);
+ Assert.Equal(2, s.Gaps); // the two valid intervals only
+ Assert.Equal(0, s.Hitches);
+ Assert.Equal(33.3, Math.Round(s.LongestGapMs, 1));
+ }
+
+ [Fact]
+ public void Reset_StartsFreshWindowAndKeepsThreshold()
+ {
+ var cadence = new FrameCadence(hitchThresholdMs: 20);
+
+ cadence.Add(AtMs(0));
+ cadence.Add(AtMs(100));
+ Assert.Equal(1, cadence.Hitches);
+
+ cadence.Reset();
+
+ Assert.Equal(0, cadence.Frames);
+ Assert.Equal(0, cadence.Gaps);
+ Assert.Equal(20, cadence.Summarize().HitchThresholdMs);
+ Assert.Contains("no frames", cadence.Summarize().Describe("cam"));
+ }
+
+ [Fact]
+ public void Describe_NamesTheWindowRateHitchesAndLongestGap()
+ {
+ var cadence = new FrameCadence();
+
+ foreach (var ms in new[] { 0.0, 16.7, 33.4, 50.1, 120.0 }) cadence.Add(AtMs(ms));
+
+ Assert.Equal(
+ "webcam 5f/0.1s 33.3/s, 4 gaps, 1 >= 50ms, avg 30.0ms, longest 69.9ms",
+ cadence.Summarize().Describe("webcam"));
+ }
+
+ [Fact]
+ public void ThresholdsMustBePositive()
+ {
+ Assert.Throws(() => new FrameCadence(0));
+ Assert.Throws(() => new FrameCadence(-1));
+ }
+}
\ No newline at end of file
diff --git a/ytLive.Tests/FramePumpTests.cs b/ytLive.Tests/FramePumpTests.cs
index 8736cc0..96db11b 100644
--- a/ytLive.Tests/FramePumpTests.cs
+++ b/ytLive.Tests/FramePumpTests.cs
@@ -1,5 +1,6 @@
using System.Net;
using System.Text;
+using System.Text.RegularExpressions;
using Xunit;
using ytLive.Models;
using ytLive.Services;
@@ -573,6 +574,52 @@ public class FramePumpTests
$"resolver ran on the starting thread ({startThread}) — the loop is back on the caller context"));
}
+ /// The ONE integration test for UNIT 0's stats correction (2026-10-05).
+ /// The old line divided every average by the number of frames SUBMITTED — deadline
+ /// padding always fills that to the full 60fps target, so a pump fed one webcam
+ /// frame per 200ms still logged "300/300 frames per 5s" and the "avg render" was
+ /// diluted by an order of magnitude. It also never reported the webcam at all, so
+ /// a camera delivering 25 of its 30 frames was indistinguishable from one
+ /// delivering 30. Here a webcam advancing a new Epoch every ~200ms must show up
+ /// as FEWER distinct frames than submits, with the render count beside them.
+ [Fact]
+ public async Task Stats_ReportRealRendersAndDistinctWebcamFrames_NotJustPaddedSubmits()
+ {
+ var webcamPixels = SceneCompositorTests.Solid(64, 48, 0, 255, 0).BgraPixels;
+ var scene = new Scene { Name = "Live" };
+ scene.Elements.Add(new WebcamSceneConfig { Name = "Webcam", IsVisible = true });
+ var encoder = new FakeEncoder();
+ var log = new List();
+ var epoch = 0L;
+ using var pump = NewPump(encoder, scene: () => scene, log: log, resolve: e =>
+ {
+ if (!e.IsWebcam) return null;
+ // One NEW webcam frame every ~200ms — slower than the 60fps container,
+ // exactly the case the old stats line hid completely.
+ Thread.Sleep(200);
+ epoch++;
+ return new VideoFrame(64, 48, webcamPixels) { Epoch = epoch };
+ });
+
+ await pump.StartAsync();
+ var deadline = DateTime.UtcNow.AddSeconds(8);
+ while (DateTime.UtcNow < deadline && !log.Any(l => l.Contains("webcam ")))
+ await Task.Delay(50);
+ await pump.StopAsync();
+
+ var stats = log.FirstOrDefault(l => l.StartsWith("FramePump stats:") && l.Contains("webcam "));
+ Assert.True(stats != null, $"no cadence-bearing stats line in: {string.Join(" | ", log)}");
+
+ var submits = int.Parse(Regex.Match(stats!, @"stats: (\d+)/").Groups[1].Value);
+ var renders = int.Parse(Regex.Match(stats!, @"from (\d+) renders").Groups[1].Value);
+ var webcamFrames = int.Parse(Regex.Match(stats!, @"webcam (\d+)f/").Groups[1].Value);
+
+ Assert.True(submits > 0, stats);
+ Assert.True(renders > 0 && renders <= submits, $"renders {renders} vs submits {submits}: {stats}");
+ Assert.True(webcamFrames > 0 && webcamFrames < submits,
+ $"webcam frames {webcamFrames} should be far fewer than the {submits} padded submits: {stats}");
+ }
+
private sealed class InlineSyncContext : SynchronizationContext
{
public override void Post(SendOrPostCallback d, object? state) => d(state);