Read the first instrumented take: the judder did not reproduce, no element is slow
The measurement unit existed to name which of the six scene elements eats ~140ms per composite. The creator's Release recording (ty-20261016-1841-559-0002.mp4, 174.5s, 1920x1080@60) names none of them — the whole composite costs ~1ms: composites 301 per 5s = 60/s, every slot, 0 dropped, 0 stalls after warm-up avg render 1.0-1.1ms (worst 2.5ms) = ~7% of the 16.67ms budget parts "#5 HD Pro Webcam C920 0.4ms, #5 ... raster 0.2ms, backdrop 0.2ms" - the other four elements never reach the 0.1ms print floor C4 cache 151R/150H: half the ticks were a whole-frame BlockCopy webcam 30.0/s at the resolver, longest gap 50.0ms (one 60Hz frame of sampling phase, not a drop) the FILE 29.6 distinct webcam images/s, 82% of images held exactly 2 frames, longest hold 267ms (the healthy 30fps-in-60fps signature) The two stalls are not composite cost: 124ms on the very first render (warm-up) and 53ms entering the "Chat" scene (a rebake). So the historic slow numbers are not a slow scene but different configurations: - Oct 9 19:48 (only bad take still on disk) was the VERTICAL tier: "FramePump started (1080x1920 @ 60 fps) scene='Live' comp=608x1080" - the 607x1080 centre crop upscaled to 2M px, 92 renders/5s, avg render 45.0ms, worst 278ms. - Oct 5 12:01 (141ms) logged no scene=/comp= at all - the suffix did not exist yet, so the map's "21:9 -> 1920x803" explanation stays a hypothesis, not a reading. - Every slow window shows "cache xR/0H" (the whole-frame cache never reusing) while this take shows 151R/150H, and one Oct 5 take latches mid-take (1.5ms -> 3.0ms -> 45.2ms in 15s, then stays). That latch is the lead for the next take; it is a hypothesis and the parts line is what will decide it. Corrections to the map (same commit, stale facts replaced not appended): - "It was never Debug-vs-Release" is WITHDRAWN: same 1920x1080 tier, Oct 9 = 10.4ms vs this Release take's 1.0ms, and Build <hash> carries no configuration - every pre-2026-10-16 render number is config-unknown. - The "same scene shape" 10:01/12:01 comparison is superseded: it was an eye comparison of takes whose geometry was never logged. New: scripts/webcam-cadence.py - the ground-truth probe for what the FILE holds, beside what the pump says it did. It took three wrong readings to get right (H.264 re-encoding defeats exact-pixel comparison; a round-clip crop's corners leak 60fps background; a fixed crop across a scene switch reads background), all recorded in MyMistakes.md as a recipe, including an ffmpeg-native cross-check (tblend=difference + signalstats.YAVG - the same metric, the technique already in this repo's A/V-sync recipe) which agreed: 52.2% duplicates / 28.6 distinct/s vs 50.7% / 29.6/s on different windows of the same take. Next: no fix is allowed yet - the creator makes a Release take in the failing geometry (vertical 1080p60 tier first), and that take's parts line names the stage. Docs in the same commit: ai.md (new "What the first instrumented take said" section, the two corrections, the parts-line denominators, the tool in the inventory), TASKS.md (reading outcome), HANDOFF.md (rewritten), MyMistakes.md (recipe). Gates: clean build 0 warnings, full suite 428/428, scope check passed.
This commit is contained in:
@@ -0,0 +1,131 @@
|
||||
#!/usr/bin/env python3
|
||||
"""webcam-cadence.py — measure the CAMERA cadence that actually reached a recording.
|
||||
|
||||
This is the ground-truth probe for the webcam-judder work (`ai.md` → "Frame cadence"):
|
||||
the pump's own counters say what the compositor DID, this says what the FILE holds.
|
||||
|
||||
python3 scripts/webcam-cadence.py <video.mp4> <x> <y> <size> [--start SECONDS]
|
||||
|
||||
Get <x> <y> <size> from the pump's scene dump in startup.log — the element's `rect=` is
|
||||
in master pixels, e.g. `[5] HD Pro Webcam C920 … rect=1425,748 526x290` → centre
|
||||
(1425+526/2, 748+290/2) = (1688, 893); pass the crop top-left as x,y and `size` as the
|
||||
square side: --x 1598 --y 803 --size 180 (see below for why not 526x290).
|
||||
|
||||
Why it is done this way (each of these was a real wrong reading on 2026-10-16):
|
||||
|
||||
1. **Exact pixel equality is useless.** H.264 re-encodes even a completely static region
|
||||
slightly differently on every frame, so `cur == prev` reported "57.9 distinct images/s"
|
||||
— i.e. every frame new — for a scene whose camera delivers 30/s. Use a THRESHOLD on the
|
||||
mean absolute difference (MAD): a new camera frame moves the picture far more than codec
|
||||
noise. The MAD distribution is bimodal (codec noise well under 0.5, real motion over 1),
|
||||
so the threshold is read off the data, not guessed.
|
||||
|
||||
2. **The crop must sit INSIDE a round clip.** A round-clipped element (the webcam) has
|
||||
radius r = min(w,h)/2 = 145 for a 526x290 box. A square crop of side s has its corners
|
||||
at s/√2 from the centre: s = 240 → 170 > 145, so the crop corners showed the 60fps
|
||||
BACKGROUND behind the circle and the probe counted 60/s. Keep s ≤ 1.41·r (180 is safe
|
||||
for r=145) — or crop a circle-excluded region.
|
||||
|
||||
3. **One scene at a time.** Element rects differ per scene, so a fixed crop carried across
|
||||
a scene switch reads whatever is under it (background) and reports a multi-second "hold"
|
||||
that never happened. Use the `FramePump scene dump:` timestamps to pick a window with
|
||||
`--start`, or measure each scene's window separately.
|
||||
|
||||
4. **A still subject also produces long holds.** Threshold counting cannot tell "the camera
|
||||
had nothing new to send" from "we repeated a frame". Read this beside the pump's own
|
||||
`FramePump stats` (renders/5s = composites/s) and `webcam … N/s` lines, which do not
|
||||
depend on subject motion.
|
||||
|
||||
Expected healthy shape: the hold histogram peaks at **2 frames** (a 30fps camera in a 60fps
|
||||
container: every image twice → 50% duplicates) with distinct/s ≈ the camera's own rate and
|
||||
the longest holds in the low hundreds of ms. A peak at 4+ frames, or distinct/s near the
|
||||
composites/s figure, means the file is a slide show of held images — the judder.
|
||||
"""
|
||||
import argparse
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
import numpy as np
|
||||
|
||||
FFMPEG = shutil.which("ffmpeg")
|
||||
if FFMPEG is None:
|
||||
sys.exit("ffmpeg not on PATH (bundled copy: ffmpeg/ffmpeg.exe — use the Windows one from Windows)")
|
||||
BLOCK = 64
|
||||
|
||||
|
||||
def main() -> int:
|
||||
ap = argparse.ArgumentParser(description="Count distinct webcam images per second in a recording.")
|
||||
ap.add_argument("video")
|
||||
ap.add_argument("--x", type=int, required=True, help="crop left, master pixels")
|
||||
ap.add_argument("--y", type=int, required=True, help="crop top, master pixels")
|
||||
ap.add_argument("--size", type=int, default=180, help="square crop side, INSIDE the round clip (default 180)")
|
||||
ap.add_argument("--start", type=float, default=0.0, help="seconds to skip (single-scene window)")
|
||||
ap.add_argument("--fps", type=float, default=60.0, help="container fps (default 60)")
|
||||
ap.add_argument("--frames", type=int, default=0, help="stop after N frames (0 = whole file)")
|
||||
args = ap.parse_args()
|
||||
|
||||
vf = f"crop={args.size}:{args.size}:{args.x}:{args.y},scale={BLOCK}:{BLOCK},format=gray"
|
||||
cmd = [FFMPEG, "-v", "error"]
|
||||
if args.start:
|
||||
cmd += ["-ss", str(args.start)]
|
||||
cmd += ["-i", args.video, "-vf", vf, "-f", "rawvideo", "-"]
|
||||
print(f"# crop {args.size}x{args.size}+{args.x}+{args.y} @ {args.start}s of {args.video}", file=sys.stderr)
|
||||
|
||||
frame_bytes = BLOCK * BLOCK
|
||||
prev = None
|
||||
mads = []
|
||||
buf = b""
|
||||
proc = subprocess.Popen(cmd, stdout=subprocess.PIPE)
|
||||
assert proc.stdout is not None
|
||||
while True:
|
||||
chunk = proc.stdout.read(frame_bytes * 256 - len(buf))
|
||||
if not chunk:
|
||||
break
|
||||
buf += chunk
|
||||
while len(buf) >= frame_bytes:
|
||||
raw, buf = buf[:frame_bytes], buf[frame_bytes:]
|
||||
cur = np.frombuffer(raw, dtype=np.uint8).astype(np.int16)
|
||||
if prev is not None:
|
||||
mads.append(float(np.abs(cur - prev).mean()))
|
||||
prev = cur
|
||||
if args.frames and len(mads) + 1 >= args.frames:
|
||||
proc.terminate()
|
||||
break
|
||||
if args.frames and len(mads) + 1 >= args.frames:
|
||||
break
|
||||
proc.wait()
|
||||
if not mads:
|
||||
sys.exit("no frames decoded")
|
||||
|
||||
mads = np.array(mads)
|
||||
frames = len(mads) + 1
|
||||
dur = frames / args.fps
|
||||
print(f"frames={frames} ({dur:.1f}s @ {args.fps:.0f}fps)")
|
||||
print("MAD between consecutive frames (mean abs gray diff, 0-255):")
|
||||
for lo, hi in [(0, .1), (.1, .5), (.5, 1), (1, 2), (2, 4), (4, 8), (8, 16), (16, 1e9)]:
|
||||
n = int(((mads >= lo) & (mads < hi)).sum())
|
||||
label = f"{hi:g}" if hi < 1e9 else "999"
|
||||
print(f" [{lo:>4}-{label:>4}): {n:>7} ({n / len(mads) * 100:5.1f}%)")
|
||||
print(f" median={np.median(mads):.2f} p90={np.percentile(mads, 90):.2f} max={mads.max():.1f}")
|
||||
|
||||
# Thresholds read off the bimodal split; 0.5 is the operating point for a moving subject.
|
||||
for t in (0.5, 1.0):
|
||||
new = np.where(mads > t)[0]
|
||||
distinct = len(new) + 1
|
||||
holds = np.diff(np.concatenate([[0], new + 1, [frames]]))
|
||||
hist: dict[int, int] = {}
|
||||
for h in holds:
|
||||
hist[h] = hist.get(h, 0) + 1
|
||||
top = ", ".join(f"{k}x{hist[k]}" for k in sorted(hist)[:8])
|
||||
longest = sorted(hist.items(), reverse=True)[:4]
|
||||
print(f"\nthreshold MAD > {t}: distinct images={distinct} ({distinct / dur:.1f}/s), "
|
||||
f"duplicates={(frames - distinct) / frames * 100:.1f}%")
|
||||
print(f" hold histogram (frames-held x images) {top}")
|
||||
print(" longest holds: " + ", ".join(f"{k}f ({k / args.fps * 1000:.0f}ms) x{hist[k]}" for k, _ in longest))
|
||||
print(" want: peak at 2 frames, ~the camera's own rate, longest holds < ~300ms")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
Reference in New Issue
Block a user