gramps
b22d08eca6
Positive offsets still delay the whole mix via the delay line (lip-sync fix); negative offsets now ARM once at StartLive and drop |N| ms off the pipe's write head so audio events land earlier when audio runs BEHIND video. Slider relabeled AUDIO SYNC, Min −500, locked while live/recording (IsEditMode). LayoutStore and VM clamp to −500..500. OBS reference for eat-the-head negative sync: https://obsproject.com/kb/obs-studio/buffering-time (negative sync values pull audio earlier by discarding buffered player audio). Test: StartLive_NegativeOffset_AdvancesAudio_ByDroppingTheStreamHead (6x0.9 head must be eaten before 0.2 bed reaches the wire).
83 lines
5.1 KiB
Markdown
83 lines
5.1 KiB
Markdown
# HANDOFF — 2026-09-14 (SIGNED AUDIO SYNC ONLINE: −500..+500, negative advances by eating the stream head; slider locked while live/recording)
|
||
|
||
## Branch / Commit State
|
||
|
||
`main` HEAD = **`11a7af2`** (pre-live ring-backlog fix, committed, pushed NOT authorized).
|
||
Working tree **dirty** with the signed-sync work below — NOT yet committed. **NOT pushing**
|
||
(no-push ruling still in effect; the last two fixes — audio silence + ring backlog — are
|
||
committed locally and the user has not greenlit a push).
|
||
|
||
DB now: **`Audio.SyncOffsetMs = 0`** (confirmed via sqlite3 this session — the creator slid
|
||
the sync to 0 and the write finally stuck; sound is correct at 0). The 0.54s residual in the
|
||
1128 take ≈ 300ms injected offset (leftover DB value) + ~240ms natural (partly webcam
|
||
clap-quantization ±40–85ms, partly measurement).
|
||
|
||
## ✅ SHIPPED (this dirty tree) — signed audio sync, −500..+500
|
||
|
||
**What:** the audio-sync control is now a SIGNED offset. Positive = delay the mix (audio runs
|
||
AHEAD of video — existing `AudioSyncDelay` behavior, unchanged and live-reactive). Negative =
|
||
**advance** the audio (audio runs BEHIND video): OBS's "eat the head of the buffer" fix —
|
||
the mixer drops the first |N| ms of the written stream at the pipe, re-anchoring the audio
|
||
stream so every event lands |N| ms EARLIER relative to video.
|
||
|
||
**Mechanism (AudioMixer):** `StartLive` arms `_advanceSamplesRemaining = |N| ms → samples` at
|
||
go-live (a negative offset can only eat the HEAD of the stream; it is armed once, not
|
||
live-reactive). `LiveLoopAsync` skips `min(budget, mixBuffer.Length)` samples off each write
|
||
head while the budget lasts — the pipe writer accepts a partial chunk via `AsMemory(writeFrom)`.
|
||
Positive path untouched (delay line still re-reads the Func every tick).
|
||
|
||
**UI/plumbing:**
|
||
- `MainViewModel.Audio.cs` — clamp `Math.Clamp(value, -500, 500)`, doc updated.
|
||
- `LayoutStore.Settings.cs` — `LoadAudioSyncOffsetMs`/`SaveAudioSyncOffsetMs` clamp −500..500.
|
||
- `PreviewPane.xaml` — label **SYNC → "AUDIO SYNC"**, `Minimum="-500"`, tooltip explains both
|
||
directions (calibrate with a clap: clap late → negative; early → positive), and
|
||
**`IsEnabled="{Binding IsEditMode}"`** — the slider locks during live AND recording (gun
|
||
safety, same property `IsRecording`/`StreamStatus` already raise PropertyChanged for).
|
||
- `AudioSyncDelay` unchanged (still clamps negative→0 internally; header doc updated to point
|
||
at the mixer for the advance side).
|
||
|
||
**Regression test (the ONE integration test for this change):**
|
||
`StartLive_NegativeOffset_AdvancesAudio_ByDroppingTheStreamHead` in `AudioPipelineTests.cs` —
|
||
−40 ms advance (8-tick budget at 5ms interval), emits 6×0.9 fresh right after StartLive (≤
|
||
budget, so the head MUST be eaten) then a long 0.2 bed; asserts the wire max ≈ 0.2 (< 0.3).
|
||
Fails loudly if −N no longer drops the head (0.9 leaks).
|
||
|
||
**User directives (this session):**
|
||
- Signed −500..+500 with positive=delay / negative=advance, relabel "AUDIO SYNC", default 0.
|
||
- **Lock the sync control when live or recording** (done — `IsEditMode`).
|
||
- **NOTE ONLY, no fix:** the live recording is completely different from the "recording
|
||
results" shown in Chat view (recorded in `bugs.md` — do not rediscover as a surprise).
|
||
- **Before 1.0:** write a detailed USER-DOC tutorial on the audio-sync feature (see TASKS.md
|
||
note; add it to the gold-pass/1.0 checklist). `docs/` currently holds only the README image.
|
||
|
||
## Take verification so far (ring-backlog fix)
|
||
|
||
Take `ty-20260914-1128-0000-2.mp4` (1920×1080@60fps, 356 frames / 5.93s): audio clap RMS peak
|
||
3.260s; video diff-frame 163 → 2.717s; **offset ≈ +0.54s** — collapsed from +2.11/2.22s; the
|
||
remaining ~300ms was the baked-in `SyncOffsetMs=300` (now 0). Both streams `start_time=0` — not
|
||
an `avoid_negative_ts` artifact.
|
||
|
||
## Open threads
|
||
|
||
- **Verify signed sync on device:** re-run the clap take with a NEGATIVE offset to confirm the
|
||
advance direction end-to-end (the regression test proves the mixer; a take proves the file).
|
||
- **Audio-silence verification** — fixed code (`724af14`) confirmed; creator heard real audio.
|
||
- **Webcam MJPG missing / ~10–14Hz**, layer SortOrder, truncation-with-dynamic-scenes — queued.
|
||
- **Sync control tutorial in user docs — REQUIRED before 1.0** (creator directive).
|
||
|
||
## Landmines
|
||
|
||
- testhost shares startup.log — filter by time.
|
||
- `cmd.exe /c "taskkill /F /IM ytLive.exe"` (WSL double-slashes mangle) before rebuilds.
|
||
- Build/tests: **Windows dotnet host** (`/mnt/c/Program Files/dotnet/dotnet.exe`). 0 warnings.
|
||
- ffmpeg/ffprobe: `/mnt/c/Program Files/Krita (x64)/bin/` with Windows paths.
|
||
- `MyMistakes.md` has the **audio/video sync measurement recipe** (claps + cross-correlation)
|
||
— grep it before re-deriving.
|
||
- sqlite3 lives at `/home/gramps/android-sdk/platform-tools/sqlite3` (WSL) for the DB at
|
||
`/mnt/c/Users/gramp/AppData/Roaming/ytLlive/ytLlive.db`.
|
||
|
||
## Next step
|
||
|
||
Run the verify.sh gate (clean build, 0 warnings, full suite, scope check) on the dirty tree,
|
||
then commit the signed-sync work unit (no push). Optionally a device clap take with a negative
|
||
offset to validate the advance on the file. |