tools: scripts/avsync.py measures A/V sync from the file, not from an ear

Three numbers, from the recording itself: each stream's length, the drift implied by a length
mismatch (a clock problem), and the residual offset between mouth motion and speech (a constant
problem). Per-3s window table with correlations, so a weak reading announces itself instead of
reporting a confident wrong number, and a median over only the strongly-correlated windows — a
whole-clip search on periodic speech locks onto the wrong beat and returns nonsense (it said
-400 ms at the search limit on the clip where the windows said -185).

Validated on ty-20261112-1228-457-0000.mp4: lengths 13.271/13.305 s (match, so no drift) and
-185 ms (sound early), which agrees with the independent visual read of the same clip.

The creator is deaf, so "does it sound right" cannot be the loop. This is.
Needs numpy; uses the bundled ffmpeg.exe.
This commit is contained in:
2026-11-12 13:57:29 -08:00
parent e654a16b60
commit b216cbc56f
2 changed files with 194 additions and 0 deletions
+9
View File
@@ -139,6 +139,15 @@ save/load round trips are exact: an element removed in the UI (`RemoveElement`
close) does **not** come back after reload (`LayoutStorePersistenceTests` guards this).
## A/V sync measurement
`scripts/avsync.py <recording> [--crop WxH+X+Y]` reports what no ear can: each stream's length,
whether they drift (a length mismatch is a clock problem), and the residual offset between mouth
motion and speech (a constant problem), with a per-3s window table and a correlation so a weak
reading says so. It uses the bundled `ffmpeg.exe` and needs `numpy`. The creator is deaf, so this —
not a listening test — is how sync gets judged: feed it the clip instead of guessing, and quote the
numbers in the commit.
## Session performance evidence
`python3 scripts/session-stats.py` mines every `startup*.log` (primary, rotated, and per-instance)