Read the first instrumented take: the judder did not reproduce, no element is slow
The measurement unit existed to name which of the six scene elements eats ~140ms per composite. The creator's Release recording (ty-20261016-1841-559-0002.mp4, 174.5s, 1920x1080@60) names none of them — the whole composite costs ~1ms: composites 301 per 5s = 60/s, every slot, 0 dropped, 0 stalls after warm-up avg render 1.0-1.1ms (worst 2.5ms) = ~7% of the 16.67ms budget parts "#5 HD Pro Webcam C920 0.4ms, #5 ... raster 0.2ms, backdrop 0.2ms" - the other four elements never reach the 0.1ms print floor C4 cache 151R/150H: half the ticks were a whole-frame BlockCopy webcam 30.0/s at the resolver, longest gap 50.0ms (one 60Hz frame of sampling phase, not a drop) the FILE 29.6 distinct webcam images/s, 82% of images held exactly 2 frames, longest hold 267ms (the healthy 30fps-in-60fps signature) The two stalls are not composite cost: 124ms on the very first render (warm-up) and 53ms entering the "Chat" scene (a rebake). So the historic slow numbers are not a slow scene but different configurations: - Oct 9 19:48 (only bad take still on disk) was the VERTICAL tier: "FramePump started (1080x1920 @ 60 fps) scene='Live' comp=608x1080" - the 607x1080 centre crop upscaled to 2M px, 92 renders/5s, avg render 45.0ms, worst 278ms. - Oct 5 12:01 (141ms) logged no scene=/comp= at all - the suffix did not exist yet, so the map's "21:9 -> 1920x803" explanation stays a hypothesis, not a reading. - Every slow window shows "cache xR/0H" (the whole-frame cache never reusing) while this take shows 151R/150H, and one Oct 5 take latches mid-take (1.5ms -> 3.0ms -> 45.2ms in 15s, then stays). That latch is the lead for the next take; it is a hypothesis and the parts line is what will decide it. Corrections to the map (same commit, stale facts replaced not appended): - "It was never Debug-vs-Release" is WITHDRAWN: same 1920x1080 tier, Oct 9 = 10.4ms vs this Release take's 1.0ms, and Build <hash> carries no configuration - every pre-2026-10-16 render number is config-unknown. - The "same scene shape" 10:01/12:01 comparison is superseded: it was an eye comparison of takes whose geometry was never logged. New: scripts/webcam-cadence.py - the ground-truth probe for what the FILE holds, beside what the pump says it did. It took three wrong readings to get right (H.264 re-encoding defeats exact-pixel comparison; a round-clip crop's corners leak 60fps background; a fixed crop across a scene switch reads background), all recorded in MyMistakes.md as a recipe, including an ffmpeg-native cross-check (tblend=difference + signalstats.YAVG - the same metric, the technique already in this repo's A/V-sync recipe) which agreed: 52.2% duplicates / 28.6 distinct/s vs 50.7% / 29.6/s on different windows of the same take. Next: no fix is allowed yet - the creator makes a Release take in the failing geometry (vertical 1080p60 tier first), and that take's parts line names the stage. Docs in the same commit: ai.md (new "What the first instrumented take said" section, the two corrections, the parts-line denominators, the tool in the inventory), TASKS.md (reading outcome), HANDOFF.md (rewritten), MyMistakes.md (recipe). Gates: clean build 0 warnings, full suite 428/428, scope check passed.
This commit is contained in:
+70
-54
@@ -1,70 +1,86 @@
|
||||
# HANDOFF — 2026-10-16 (work committed locally; NOT pushed — push at the next milestone)
|
||||
|
||||
## At a glance
|
||||
- **Branch** main, tracking `origin/main`. **origin/main = `931ab46`; HEAD = the tip commit "Per-element
|
||||
composite timing + scene dump (measurement only)"** — **2 ahead, 0 behind** (nothing pushed this
|
||||
session; per the commit/push policy, push only at sub-milestones/milestones, when the creator says
|
||||
so). This file cannot record its own commit's hash — amending a commit changes that hash — so the tip
|
||||
is named by subject.
|
||||
- The two local-only commits are `3cb29ea` (**the previous session's HANDOFF refresh — the old
|
||||
handoff claimed it was pushed; it never was**) and the tip. So "0 ahead" in the 2026-10-14 handoff
|
||||
was wrong — check `git log origin/main..HEAD`, never the handoff's own claim.
|
||||
- Working tree **clean** after the commit.
|
||||
- No app run, recording, or test session happened 2026-10-11 → 10-16 (verified against
|
||||
`%APPDATA%\ytLlive\startup.log`: last activity 2026-10-10 13:26). The creator was away for a family
|
||||
emergency and has only just returned — **there is still no take of the instrumented build.**
|
||||
- **Branch** main, tracking `origin/main`. **origin/main = `931ab46`; HEAD = the tip commit
|
||||
"Read the first instrumented take: the judder did not reproduce, no element is slow"** —
|
||||
**4 ahead, 0 behind** (nothing pushed this session; per the commit/push policy, push only when the
|
||||
creator says so). This file cannot record its own commit's hash — amending a commit changes that
|
||||
hash — so the tip is named by subject.
|
||||
- The four local-only commits: `3cb29ea` (**previous session's HANDOFF refresh — the old handoff
|
||||
claimed it was pushed; it never was**), `a3a4681` (per-element composite timing + scene dump),
|
||||
`84cbec2` (FramePumpTests thread-safe log sink), the tip. Always check
|
||||
`git log origin/main..HEAD` — never a handoff's own "0 ahead" claim.
|
||||
- Working tree clean after the tip commit.
|
||||
- **The creator's first take of the instrumented build happened today** (18:40–18:44, Release
|
||||
recording). **The judder did NOT reproduce and no element is slow** — full reading in `ai.md` →
|
||||
"✅ What the first instrumented take said". Nothing is queued to fix from it.
|
||||
|
||||
## This session's unit (committed, unpushed — the tip commit)
|
||||
- **Per-element composite timing + scene dump — MEASUREMENT ONLY (2026-10-16).** Closes the gap the
|
||||
cadence work left: the 141ms take and the 30ms take could not be compared because the log named no
|
||||
element and recorded no scene.
|
||||
- `Services/Compositor/CompositeTimings.cs` (new): collector; `Describe(renders, minMs = 0.1)`
|
||||
prints the averaged breakdown slowest-first.
|
||||
- `SceneCompositor.Render` / `CompositeLayers` / `BakeStaticBase` / `BlitBakedElement` /
|
||||
`BlitCachedLayer` + `SceneGraph.GetBakedBase`: trailing optional `CompositeTimings?` — every stage
|
||||
and every element timed (`#<i> <name>`, nested `raster` = paste-cache miss). Existing callers
|
||||
unaffected.
|
||||
- `FramePump`: `FramePump parts (N renders/5s): …` per stats window; `FramePump scene dump:` block
|
||||
(header + one line per element) at pump start and on scene change.
|
||||
- Gate test: `ytLive.Tests/CompositeTimingTests.cs` (one `[Fact]`, real composite through the real
|
||||
collector + dump formatter). Full suite 428 tests; 0 warnings; scope check passed.
|
||||
- Derivative citation: OBS Source Profiler (per-source tick/render average) — see `ai.md`
|
||||
"Per-element composite timing + scene dump".
|
||||
## This session (2 commits — both unpushed)
|
||||
1. **`84cbec2` — FramePumpTests race fix (own commit, creator-approved).** A full-suite failure
|
||||
(`Stats_ReportRealRendersAndDistinctWebcamFrames…`, "Collection was modified") was a **test-side
|
||||
data race**: a plain `List<string>` enumerated by the test while the pump logged from its own
|
||||
thread. Fixed with `ConcurrentBag<string>`. **Proof: full suite 428/428 passed, 0 warnings.**
|
||||
Lesson recorded in `MyMistakes.md` → "known failure label" (2nd instance).
|
||||
2. **The reading (this is the tip commit).** Take `ty-20261016-1841-559-0002.mp4` (174.5s, Release
|
||||
`e95154d9`, 1920×1080 1:1, recording): **60 composites/s, avg render 1.0–1.1ms, biggest element =
|
||||
webcam 0.4ms**, backdrop 0.2ms, other four under the 0.1ms floor, `cache 151R/150H`, webcam 30.0/s
|
||||
at the resolver (longest gap 50ms), and the FILE holds **29.6 distinct webcam images/s with 82% of
|
||||
images held exactly 2 frames**. No culprit element exists in this take.
|
||||
- **The historic slow numbers are different configurations:** Oct 9 19:48 (still on disk) was the
|
||||
**vertical** tier (`comp=608x1080`, 45ms, 92 renders/5s); Oct 5 12:01's geometry was never logged
|
||||
(the `scene=/comp=` suffix didn't exist) so the map's 21:9 story stays a hypothesis. Every slow
|
||||
window shows `cache xR/0H`; one Oct 5 take latches fast→slow (1.5ms → 45ms) within 15s.
|
||||
- **Corrected in the map (not appended):** "it was never Debug-vs-Release" is withdrawn (same
|
||||
1920×1080 tier: Oct 9 = 10.4ms vs today's Release 1.0ms, and `Build <hash>` records no
|
||||
configuration), and the "same scene shape" 10:01/12:01 comparison is superseded.
|
||||
- **Same commit ships the probe + its recipe.** `scripts/webcam-cadence.py` (new tool): the
|
||||
ground-truth check for what the FILE holds. Its three traps (H.264 re-encoding defeats
|
||||
exact-pixel comparison; a round-clip crop's corners leak 60fps background; a fixed crop across a
|
||||
scene switch reads background) cost three wrong readings — all in `MyMistakes.md`, with an
|
||||
ffmpeg-native cross-check (`tblend=difference` + `signalstats.YAVG`) that agreed (52.2%
|
||||
duplicates / 28.6 distinct/s).
|
||||
|
||||
## Next step
|
||||
- **Creator take on the instrumented build, Release** (`"/mnt/c/Program Files/dotnet/dotnet.exe" run
|
||||
-c Release`). Read the two new lines in `%APPDATA%\ytLlive\startup.log`:
|
||||
`FramePump parts (N renders/5s): …` (which element / which stage eats the budget) and the
|
||||
`FramePump scene dump:` block (the shape the take was made from). **No fix is to be attempted until
|
||||
the parts line names the element** — the 10:03 → 12:01 regression (29–34ms → 143–155ms) is still
|
||||
unexplained, and the competing explanations in the map (21:9 source rect vs. same-shape regression)
|
||||
are unresolved on purpose until a reading decides it.
|
||||
- **Creator take in the FAILING geometry — the vertical 1080p60 tier first** (creator's pick from the
|
||||
two options offered). Then, if the creator is willing, the 21:9 display config.
|
||||
- Read three lines in `%APPDATA%\ytLlive\startup.log`: `FramePump parts (N renders/5s): …` (which
|
||||
stage/element eats the budget), the `FramePump scene dump:` block (the shape), and `cache NR/NH`
|
||||
(0 hits = the slow state's signature). **No fix until that take names the stage** — the latch
|
||||
hypothesis (a composite pushed past the camera period can never reuse the whole-frame cache again)
|
||||
is unproven and the `parts` line is what decides it.
|
||||
|
||||
## Also still open (operational)
|
||||
- **verify.sh's known-failure whitelist is too loose (found 2026-10-16):** it counts matching names and
|
||||
passes if `KNOWN > 0`, so a NEW third failure rides along unnoticed. This session's full run showed
|
||||
`AudioPipelineTests.MasterLimiter_CapsTheLiveMix_OnThePipe` failing beside the two known ones — it
|
||||
passes in isolation (29/29), so it is an audio-device env flake like
|
||||
`Mix_HonorsProviderGains…` (system audio playing). **Follow-up: tighten the script** (require EVERY
|
||||
failure line to be in the whitelist) and add `MasterLimiter_CapsTheLiveMix_OnThePipe` to the list.
|
||||
Not fixed in this unit — out of scope (`scripts/verify.sh` was not in the declared file list).
|
||||
## Open (operational)
|
||||
- **verify.sh's known-failure whitelist is too loose (confirmed twice on 2026-10-16):** it passes when
|
||||
ANY known failure appears, so a new failure rides along unnoticed — and when the ONLY failure is a
|
||||
known flake (`LayerReorderPersistenceTests.RealMouseDrag_OnTheLayerList_PersistsTheReorder`, which
|
||||
passes in isolation) it reports "NOT the known pre-existing pair". **Follow-up: require EVERY
|
||||
failure line to be whitelisted**, and add `MasterLimiter_CapsTheLiveMix_OnThePipe` +
|
||||
`RealMouseDrag_OnTheLayerList_PersistsTheReorder` to the list. Not fixed here — `scripts/verify.sh`
|
||||
was never in a declared file list.
|
||||
- **A pre-existing stash from older work is still sitting there:** `stash@{0}` = "WIP on main:
|
||||
0464593 Compositor: fast-path opaque 1:1 when source fills canvas" (NOT this session's; do not drop
|
||||
it blindly).
|
||||
- **Vertical-format regression (STALE, not fixed):** 1080p60-vertical selection no longer shrinks the
|
||||
capture border in the preview. Mechanism mapped in `Controls/PreviewPane.xaml:319-322` (crop Rectangle
|
||||
bound to `OutputRectX/Y/W/H` + `IsOutputCropped`, driven by `MainViewModel.Streaming.cs`
|
||||
`ApplyStreamQuality`; `DimRects` in `MainViewModel.cs:91-94`). Root cause unknown — paused.
|
||||
- ⛔ Sudo password rotation pending (user; then update both password lines in `CREDENTIALS.md`).
|
||||
capture border in the preview (`Controls/PreviewPane.xaml:319-322` + `MainViewModel.Streaming.cs`
|
||||
`ApplyStreamQuality`). Root cause unknown — paused. Note the vertical tier is ALSO the only slow
|
||||
take still on disk, so if the next take reproduces the 45ms there, this area gets touched twice.
|
||||
- ⛔ Sudo password rotation pending (user; then update both `CREDENTIALS.md` password lines).
|
||||
- TASK 56 secret scan before `gramps/LlamaCasty` goes public.
|
||||
- TASK 55 disk hygiene queued (`startup.log` uncapped — now 22MB and growing with the new lines).
|
||||
- TASK 55 disk hygiene (`startup.log` uncapped — 23MB and the new lines add to it every 5s).
|
||||
|
||||
## Tests
|
||||
- Full suite **426 passed / 2 failed / 0 skipped / 0 warnings** (2026-10-16, 428 total incl. the new
|
||||
test). Both failures are the audio env flake (see above) and pass in isolation.
|
||||
- Pre-commit gate: `./scripts/verify.sh "<file1>" "<file2>" ...` (clean build + full suite + scope check).
|
||||
**Read its output — it can pass with an unlisted failure present (see the landmine above).**
|
||||
- Full suite **428/428 passed, 0 warnings** (2026-10-16, the gate run for `84cbec2`) — a completely
|
||||
clean run, including the audio tests that usually flake.
|
||||
- Known flaky-by-environment (all pass in isolation): `AudioPipelineTests.Mix_HonorsProviderGains…`,
|
||||
`MasterLimiter_CapsTheLiveMix_OnThePipe`, `LayerReorderPersistenceTests.RealMouseDrag…`,
|
||||
`Round_Clip_Corner…`. **Read verify.sh's output — it can pass with an unlisted failure present, or
|
||||
fail on a known flake (see above).**
|
||||
|
||||
## Secrets / paths (do not re-hunt)
|
||||
- Layout DB: `%APPDATA%\ytLlive\ytLlive.db` (SQLite, full-rewrite per save). Auth: `%APPDATA%\ytLlive\ytLlive.auth`
|
||||
(DPAPI). Startup log: `%APPDATA%\ytLlive\startup.log`.
|
||||
- Layout DB: `%APPDATA%\ytLlive\ytLlive.db` (SQLite, full-rewrite per save). Auth:
|
||||
`%APPDATA%\ytLlive\ytLlive.auth` (DPAPI). Startup log: `%APPDATA%\ytLlive\startup.log` (23MB).
|
||||
- Recordings land in `C:\Users\gramp\Downloads\recordings\` (`ty-<date>-<time>-<ms>-<seq>.mp4`; a
|
||||
duplicate name gets a `-2` suffix — the log's path and the file on disk can differ that way).
|
||||
- LAN credentials: `CREDENTIALS.md` (gitignored). Google Cloud OAuth project number `198374416003`,
|
||||
owned by `llamachile.shop@gmail.com` — verify at
|
||||
`https://console.cloud.google.com/apis/credentials?project=198374416003`.
|
||||
|
||||
@@ -382,6 +382,52 @@ delay + ~100ms pipeline = exactly the measured 2.11–2.22s. The lag ALSO scales
|
||||
"time since app launch" up to the ring depth — a take right after a restart reads as
|
||||
~370ms while a fully-warmed app reads ~2.2s. Same code, wildly different numbers.
|
||||
|
||||
### Measuring webcam CAMERA cadence in a recording — the MAD method (RECIPE, 2026-10-16)
|
||||
|
||||
Derived while reading the first instrumented take. Answers "did the camera's cadence reach the
|
||||
FILE?" — the ground truth beside the pump's own counters. Tool: **`scripts/webcam-cadence.py`**
|
||||
(`python3 scripts/webcam-cadence.py TAKE.mp4 --x X --y Y --size 180 --start SECONDS`).
|
||||
Three wrong readings were produced before it worked; each is a trap to skip:
|
||||
|
||||
1. **Exact pixel equality is useless.** H.264 re-encodes even a *fully static* region slightly
|
||||
differently every frame, so `cur == prev` reported **57.9 distinct images/s** on a take whose
|
||||
camera delivers 30/s — every frame "new". Use a THRESHOLD on the mean absolute difference
|
||||
(MAD) between consecutive frames: the distribution is bimodal (codec noise < 0.1, real camera
|
||||
motion > 1), so read the threshold off the histogram (0.5 for a moving subject) instead of
|
||||
guessing.
|
||||
2. **The crop must sit INSIDE a round clip.** The webcam is `clip=Round`, radius `r = min(w,h)/2`
|
||||
= 145 for a 526×290 box. A square crop's corners lie at `s/√2` from the centre, so `s = 240`
|
||||
→ 170 > 145 → the corners sampled the 60fps BACKGROUND behind the circle and the probe counted
|
||||
60/s. Keep `s ≤ 1.41·r` (180 is safe for r=145). **The crop rect comes from the pump's
|
||||
`FramePump scene dump:` line** (`rect=1425,748 526x290` → centre 1688,893).
|
||||
3. **One scene at a time.** Element rects differ per scene, so a crop carried across a scene
|
||||
switch sits on background and reports a multi-second "hold" that never happened (a 13.8s
|
||||
"freeze" in the first pass was just the `Chat` scene). Pick a window from the dump timestamps
|
||||
with `--start`.
|
||||
4. **A still subject also produces long holds.** Threshold counting cannot distinguish "the
|
||||
camera had nothing new to send" from "we repeated a frame" — read it BESIDE the pump's
|
||||
`FramePump stats` lines (`N renders/5s` = composites/s, `webcam N/s` = arrivals), which do not
|
||||
depend on subject motion.
|
||||
|
||||
**Healthy shape:** the hold histogram peaks at **2 frames** (30fps camera in a 60fps container:
|
||||
every image twice → ~50% duplicates), distinct/s ≈ the camera's own rate, longest holds in the
|
||||
low hundreds of ms. Oct 16 Release take: 29.6/s, **82% of images held exactly 2 frames**, longest
|
||||
267ms. A peak at 4+ frames, or distinct/s approaching the composites/s figure, is the judder.
|
||||
|
||||
**Cross-check with ffmpeg's own filters (no numpy, uses this file's A/V-sync technique):** the metric
|
||||
above is the mean absolute frame-to-frame difference, which is exactly what
|
||||
`tblend=all_mode=difference,signalstats,metadata=print` reports as `YAVG`. Same crop, 40s window:
|
||||
|
||||
```bash
|
||||
ffmpeg -v error -ss 45 -t 40 -i TAKE.mp4 \
|
||||
-vf "crop=180:180:1598:803,scale=64:64,format=gray,tblend=all_mode=difference,signalstats,metadata=print:file=-" \
|
||||
-f null - | grep YAVG | sed 's/.*YAVG=//' # count values <= threshold
|
||||
```
|
||||
|
||||
It agreed with the script on the same take (52.2% duplicates / 28.6 distinct/s vs 50.7% / 29.6/s —
|
||||
the windows differ slightly), so either path is trustworthy; use the script when the full take and
|
||||
the hold histogram matter, this pipeline for a fast spot check.
|
||||
|
||||
### Feeding a rawvideo pipe at 60fps: deadline pacing + row-blit budget
|
||||
|
||||
Derived 2026-09-03 (take-3 diagnosis — the stats seam from `97ffc42` named the stage
|
||||
|
||||
@@ -178,6 +178,18 @@ second encoder path needing a "redirect". Record+simulcast is one ffmpeg with tw
|
||||
(Release):** the parts line must name which of the 6 elements eats the ~140ms and the dump must
|
||||
show the shape of the slow take vs the fast one (10:03 → 12:01, 29–34ms → 143–155ms, same shape).
|
||||
**No fix until a real reading names the element** — the next unit is the fix that reading dictates.
|
||||
- **📖 FIRST INSTRUMENTED TAKE READ (2026-10-16, Release, recording, 1920×1080 1:1 master tier):
|
||||
THE JUDDER DID NOT REPRODUCE — and no element is expensive.** 60 composites/s, avg render
|
||||
1.0–1.1ms, biggest element in the scene = webcam **0.4ms** (raster 0.2ms), backdrop 0.2ms, the
|
||||
other four under the 0.1ms floor; webcam 30.0/s at the resolver (longest gap 50ms); the FILE holds
|
||||
29.6 distinct webcam images/s with 82% of images held exactly 2 frames. **Nothing to fix yet.**
|
||||
The historic slow numbers are different configurations, not a slow scene — Oct 9 19:48 was the
|
||||
**vertical** tier (`comp=608x1080`, 45ms) and Oct 5 12:01's geometry was never logged (the
|
||||
`scene=/comp=` suffix didn't exist); every slow window shows `cache xR/0H` (the whole-frame cache
|
||||
never reusing) while the healthy one shows `151R/150H`, and one Oct 5 take latches fast→slow
|
||||
mid-take. **NEXT: a Release take in the failing geometry** — vertical tier first, then the 21:9
|
||||
display config — and only that take's `parts` line names the stage. See `ai.md` → "✅ What the
|
||||
first instrumented take said".
|
||||
- **Unit 2 (later, unchanged):** blinking amber **"SD" badge**, preview only, never in the
|
||||
recording/stream, when a better format was available and not obtained. Top-right corner, or the
|
||||
1:30 rim point when the clip is round.
|
||||
|
||||
@@ -758,7 +758,10 @@ pure pixel work in the composite, not frame lookup. Two consequences, both load-
|
||||
143–155ms between sessions with nothing logged explaining it. That comparison was made by EYE on
|
||||
two takes because the log carried no per-element breakdown and no scene dump — both now exist
|
||||
(see "Per-element composite timing + scene dump" below); naming the culprit is a reading, not a
|
||||
guess, and guessing is explicitly out of bounds.
|
||||
guess, and guessing is explicitly out of bounds. **Superseded 2026-10-16:** "same scene shape" was
|
||||
an eye comparison of takes whose geometry was never logged, and the next bad take still on disk is
|
||||
the *vertical* tier (below). A Release 1920×1080 take now measures 1.0ms with no element over
|
||||
0.4ms — see "Per-element composite timing + scene dump" → what the first instrumented take said.
|
||||
|
||||
**Counter subtlety, so it is not re-derived:** the pump's `webcam` line read 13/s while the file held
|
||||
6.8/s. Not a bug — the counter stamps a frame when the resolver FIRST sees it, and one composite takes
|
||||
@@ -803,6 +806,10 @@ allowed until a real reading names the culprit.
|
||||
- Gate test: `ytLive.Tests/CompositeTimingTests.cs` (one `[Fact]`) — a REAL composite of a full-canvas
|
||||
element plus a 64×64 one must rank the full-canvas element first, carry the `raster` sub-stage and
|
||||
the fixed stages, and the dump must name both elements with z-index/type/rect and echo the split.
|
||||
- **`scripts/webcam-cadence.py`** — the other half of a reading: what the FILE holds (distinct webcam
|
||||
images/s + hold histogram), beside what the pump says it did. Recipe and its three traps (H.264
|
||||
re-encoding, the round-clip crop, scene switches): `MyMistakes.md` → "Measuring webcam CAMERA cadence
|
||||
in a recording".
|
||||
|
||||
**Read takes in Release.** The creator launches `dotnet run -c Release`; Debug pixel paths are
|
||||
several× slower, so a Debug number is not a reading of this pipeline (see Run).
|
||||
@@ -814,6 +821,57 @@ per-source timers via `util/profiler.h`). Same shape here: one timer per element
|
||||
as an average, plus the stage split. The `#<i> <name> raster` sub-stage is the extra this case needs —
|
||||
it separates "re-sampled every tick" from "slow paste" without another session.
|
||||
|
||||
### ✅ What the first instrumented take said (2026-10-16, Release) — the judder DID NOT reproduce
|
||||
|
||||
Take `ty-20261016-1841-559-0002.mp4` (174.5s, 1920×1080@60, Release build `e95154d9` — a **recording**,
|
||||
not preview; scene `Live`, switched to `Chat` at 28s, back to `Live` at 44s) and its log window:
|
||||
|
||||
| Reading | This take |
|
||||
|---|---|
|
||||
| composites | **301 per 5s = 60/s**, every slot; 0 dropped, 0 stalls after warm-up |
|
||||
| avg render | **1.0–1.1ms**, worst 2.5ms — ~7% of the 16.67ms budget |
|
||||
| `parts` | `#5 HD Pro Webcam C920 0.4ms, #5 … raster 0.2ms, backdrop 0.2ms` — the other four elements are under the 0.1ms print floor |
|
||||
| C4 full-composite cache | **151R/150H** per window — half the ticks were a whole-frame `BlockCopy` |
|
||||
| webcam at the resolver | **30.0/s**, longest gap **50.0ms** (one extra 60Hz frame = sampling phase, not a drop) |
|
||||
| the file itself | **29.6 distinct webcam images/s, 82% of images held exactly 2 frames**, longest hold 267ms (`scripts/webcam-cadence.py`) |
|
||||
|
||||
Nothing in this take is over budget and **no element is expensive — there is no culprit here to fix.**
|
||||
The two stalls in the take are not per-composite cost: 124ms on the very first render (warm-up) and
|
||||
53ms on the switch into `Chat` (a rebake).
|
||||
|
||||
**Scale of the disagreement:** the historic slow readings are 141ms (Oct 5 12:01) and 45ms (Oct 9
|
||||
19:48) — 40–140× the ~1ms this take needed per composite. Three facts about those takes, all read off
|
||||
the same log:
|
||||
|
||||
- **Oct 9 19:48 (`ty-20261009-1948-121-0000-2.mp4`) was the VERTICAL tier**, not a 16:9 one:
|
||||
`FramePump started (1080×1920 @ 60 fps) scene='Live' comp=608x1080` — the 607×1080 centre crop
|
||||
bilinear-upscaled to 1080×1920 (2M px), 92 renders/5s, avg render 45.0ms, worst 278ms, 92 stalls,
|
||||
`cache 92R/0H`. The file on disk is 11.6s and every frame is identical (a static desktop), so it
|
||||
cannot be read for cadence — only its cost.
|
||||
- **Oct 5 12:01 (`…1201-556-0001.mp4`, no longer on disk) logged `FramePump started (1920×1080 @ 60
|
||||
fps)` with NO `scene=/comp=` suffix** — that suffix did not exist yet, so its source rect was never
|
||||
recorded and the map's "21:9 → 1920×803" explanation stays a hypothesis.
|
||||
- **A take can latch mid-take.** Oct 5 12:29 goes 1.5ms (avg render) → 3.0ms → **45.2ms within 15
|
||||
seconds and stays there** (`cache 10R/0H`, then `98R/0H`). So the slow state is not a fixed property
|
||||
of a scene geometry. The consistent signature of every slow window is `cache xR/**0H**`: the C4
|
||||
whole-frame cache never reused once, so every tick composited. The healthy take shows the other
|
||||
stable state (composite 1ms → every second tick is byte-identical → whole-frame reuse). **The latch
|
||||
is the thing to test on a reproducing take** — it is a hypothesis, not a reading; the `parts` line
|
||||
is what will decide it.
|
||||
|
||||
**What this does NOT say.** This take is the 1:1 master tier and its scene logs
|
||||
`full-render split=0 elements=6 dynamic=4` (the bad-period takes logged `dynamic=5`), so it never
|
||||
exercises the failing geometry. "The healthy path is healthy" is not "the judder is gone". Reproducing
|
||||
it needs a Release take in the failing configuration (vertical tier first, then the 21:9 display
|
||||
config) — that take's `parts` line names the stage, and only then is a fix unit allowed.
|
||||
|
||||
**Reading the `parts` line's denominators:** like `avg render`, it is divided by `statRenders`, which
|
||||
counts C4 REUSES as well as real composites (the 301 above = 151 real + 150 whole-frame copies). So in
|
||||
a healthy 30fps-camera take both averages are ~2× optimistic; the `cache NR/NH` pair on the same line
|
||||
says how much of the window was real composite work. `CompositeTimings.Describe` drops stages under
|
||||
0.1ms by default, and **a missing or short parts line is itself a reading** — a fully-static window
|
||||
(every tick a C4 hit) composites no elements at all.
|
||||
|
||||
### Render budget — getting a composite inside 16.67ms (UNITs 4-5, 2026-10-05)
|
||||
|
||||
The pump emits one encoder frame per 16.67ms slot and **pads with a duplicate when a composite
|
||||
@@ -825,8 +883,13 @@ so a slow stage names itself, and logs the scene name + compositor source rect a
|
||||
21:9 display shrinks the source rect to 1920×803 ("fit", not "fill"), so the opaque-backdrop 1:1 row
|
||||
walk is skipped, the base is prefilled with opaque black (~2M byte writes), and the backdrop is
|
||||
resampled per-pixel (~1.54M pixels with an alpha branch). Every recorded A/B lines up:
|
||||
21:9 → 141ms, 16:9 → 30-33ms, 2560×1440 (fits to 1920×1080 exactly) → fast path. It was never
|
||||
Debug-vs-Release; today's Debug run on 16:9 with the game up measured 33ms.
|
||||
21:9 → 141ms, 16:9 → 30-33ms, 2560×1440 (fits to 1920×1080 exactly) → fast path.
|
||||
**Corrected 2026-10-16 — the "never Debug-vs-Release" half of that is withdrawn.** A Release
|
||||
1920×1080 take measures **1.0–1.1ms** where an Oct 9 take of the same tier measured 10.4ms, and
|
||||
`Build <hash>` carries no build configuration, so the config of every pre-2026-10-16 take is
|
||||
UNKNOWN: no historic render number may be compared against a Release take. The 21:9 explanation
|
||||
itself stays a hypothesis — the 141ms take's start line predates the `scene=/comp=` suffix, so its
|
||||
source rect was never recorded. A take can also latch mid-take (see the instrumentation section).
|
||||
|
||||
**Unit 4 — cheap wins, all pixel-identical:**
|
||||
- Opaque 1:1 fast paths expanded to the whole-canvas opaque case (a `BlockCopy`, not a per-pixel
|
||||
|
||||
@@ -0,0 +1,131 @@
|
||||
#!/usr/bin/env python3
|
||||
"""webcam-cadence.py — measure the CAMERA cadence that actually reached a recording.
|
||||
|
||||
This is the ground-truth probe for the webcam-judder work (`ai.md` → "Frame cadence"):
|
||||
the pump's own counters say what the compositor DID, this says what the FILE holds.
|
||||
|
||||
python3 scripts/webcam-cadence.py <video.mp4> <x> <y> <size> [--start SECONDS]
|
||||
|
||||
Get <x> <y> <size> from the pump's scene dump in startup.log — the element's `rect=` is
|
||||
in master pixels, e.g. `[5] HD Pro Webcam C920 … rect=1425,748 526x290` → centre
|
||||
(1425+526/2, 748+290/2) = (1688, 893); pass the crop top-left as x,y and `size` as the
|
||||
square side: --x 1598 --y 803 --size 180 (see below for why not 526x290).
|
||||
|
||||
Why it is done this way (each of these was a real wrong reading on 2026-10-16):
|
||||
|
||||
1. **Exact pixel equality is useless.** H.264 re-encodes even a completely static region
|
||||
slightly differently on every frame, so `cur == prev` reported "57.9 distinct images/s"
|
||||
— i.e. every frame new — for a scene whose camera delivers 30/s. Use a THRESHOLD on the
|
||||
mean absolute difference (MAD): a new camera frame moves the picture far more than codec
|
||||
noise. The MAD distribution is bimodal (codec noise well under 0.5, real motion over 1),
|
||||
so the threshold is read off the data, not guessed.
|
||||
|
||||
2. **The crop must sit INSIDE a round clip.** A round-clipped element (the webcam) has
|
||||
radius r = min(w,h)/2 = 145 for a 526x290 box. A square crop of side s has its corners
|
||||
at s/√2 from the centre: s = 240 → 170 > 145, so the crop corners showed the 60fps
|
||||
BACKGROUND behind the circle and the probe counted 60/s. Keep s ≤ 1.41·r (180 is safe
|
||||
for r=145) — or crop a circle-excluded region.
|
||||
|
||||
3. **One scene at a time.** Element rects differ per scene, so a fixed crop carried across
|
||||
a scene switch reads whatever is under it (background) and reports a multi-second "hold"
|
||||
that never happened. Use the `FramePump scene dump:` timestamps to pick a window with
|
||||
`--start`, or measure each scene's window separately.
|
||||
|
||||
4. **A still subject also produces long holds.** Threshold counting cannot tell "the camera
|
||||
had nothing new to send" from "we repeated a frame". Read this beside the pump's own
|
||||
`FramePump stats` (renders/5s = composites/s) and `webcam … N/s` lines, which do not
|
||||
depend on subject motion.
|
||||
|
||||
Expected healthy shape: the hold histogram peaks at **2 frames** (a 30fps camera in a 60fps
|
||||
container: every image twice → 50% duplicates) with distinct/s ≈ the camera's own rate and
|
||||
the longest holds in the low hundreds of ms. A peak at 4+ frames, or distinct/s near the
|
||||
composites/s figure, means the file is a slide show of held images — the judder.
|
||||
"""
|
||||
import argparse
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
import numpy as np
|
||||
|
||||
FFMPEG = shutil.which("ffmpeg")
|
||||
if FFMPEG is None:
|
||||
sys.exit("ffmpeg not on PATH (bundled copy: ffmpeg/ffmpeg.exe — use the Windows one from Windows)")
|
||||
BLOCK = 64
|
||||
|
||||
|
||||
def main() -> int:
|
||||
ap = argparse.ArgumentParser(description="Count distinct webcam images per second in a recording.")
|
||||
ap.add_argument("video")
|
||||
ap.add_argument("--x", type=int, required=True, help="crop left, master pixels")
|
||||
ap.add_argument("--y", type=int, required=True, help="crop top, master pixels")
|
||||
ap.add_argument("--size", type=int, default=180, help="square crop side, INSIDE the round clip (default 180)")
|
||||
ap.add_argument("--start", type=float, default=0.0, help="seconds to skip (single-scene window)")
|
||||
ap.add_argument("--fps", type=float, default=60.0, help="container fps (default 60)")
|
||||
ap.add_argument("--frames", type=int, default=0, help="stop after N frames (0 = whole file)")
|
||||
args = ap.parse_args()
|
||||
|
||||
vf = f"crop={args.size}:{args.size}:{args.x}:{args.y},scale={BLOCK}:{BLOCK},format=gray"
|
||||
cmd = [FFMPEG, "-v", "error"]
|
||||
if args.start:
|
||||
cmd += ["-ss", str(args.start)]
|
||||
cmd += ["-i", args.video, "-vf", vf, "-f", "rawvideo", "-"]
|
||||
print(f"# crop {args.size}x{args.size}+{args.x}+{args.y} @ {args.start}s of {args.video}", file=sys.stderr)
|
||||
|
||||
frame_bytes = BLOCK * BLOCK
|
||||
prev = None
|
||||
mads = []
|
||||
buf = b""
|
||||
proc = subprocess.Popen(cmd, stdout=subprocess.PIPE)
|
||||
assert proc.stdout is not None
|
||||
while True:
|
||||
chunk = proc.stdout.read(frame_bytes * 256 - len(buf))
|
||||
if not chunk:
|
||||
break
|
||||
buf += chunk
|
||||
while len(buf) >= frame_bytes:
|
||||
raw, buf = buf[:frame_bytes], buf[frame_bytes:]
|
||||
cur = np.frombuffer(raw, dtype=np.uint8).astype(np.int16)
|
||||
if prev is not None:
|
||||
mads.append(float(np.abs(cur - prev).mean()))
|
||||
prev = cur
|
||||
if args.frames and len(mads) + 1 >= args.frames:
|
||||
proc.terminate()
|
||||
break
|
||||
if args.frames and len(mads) + 1 >= args.frames:
|
||||
break
|
||||
proc.wait()
|
||||
if not mads:
|
||||
sys.exit("no frames decoded")
|
||||
|
||||
mads = np.array(mads)
|
||||
frames = len(mads) + 1
|
||||
dur = frames / args.fps
|
||||
print(f"frames={frames} ({dur:.1f}s @ {args.fps:.0f}fps)")
|
||||
print("MAD between consecutive frames (mean abs gray diff, 0-255):")
|
||||
for lo, hi in [(0, .1), (.1, .5), (.5, 1), (1, 2), (2, 4), (4, 8), (8, 16), (16, 1e9)]:
|
||||
n = int(((mads >= lo) & (mads < hi)).sum())
|
||||
label = f"{hi:g}" if hi < 1e9 else "999"
|
||||
print(f" [{lo:>4}-{label:>4}): {n:>7} ({n / len(mads) * 100:5.1f}%)")
|
||||
print(f" median={np.median(mads):.2f} p90={np.percentile(mads, 90):.2f} max={mads.max():.1f}")
|
||||
|
||||
# Thresholds read off the bimodal split; 0.5 is the operating point for a moving subject.
|
||||
for t in (0.5, 1.0):
|
||||
new = np.where(mads > t)[0]
|
||||
distinct = len(new) + 1
|
||||
holds = np.diff(np.concatenate([[0], new + 1, [frames]]))
|
||||
hist: dict[int, int] = {}
|
||||
for h in holds:
|
||||
hist[h] = hist.get(h, 0) + 1
|
||||
top = ", ".join(f"{k}x{hist[k]}" for k in sorted(hist)[:8])
|
||||
longest = sorted(hist.items(), reverse=True)[:4]
|
||||
print(f"\nthreshold MAD > {t}: distinct images={distinct} ({distinct / dur:.1f}/s), "
|
||||
f"duplicates={(frames - distinct) / frames * 100:.1f}%")
|
||||
print(f" hold histogram (frames-held x images) {top}")
|
||||
print(" longest holds: " + ", ".join(f"{k}f ({k / args.fps * 1000:.0f}ms) x{hist[k]}" for k, _ in longest))
|
||||
print(" want: peak at 2 frames, ~the camera's own rate, longest holds < ~300ms")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
Reference in New Issue
Block a user