feat: signed A/V sync offset (−500..+500), negative advances by eating live stream head (OBS eat-head semantics)

Positive offsets still delay the whole mix via the delay line (lip-sync fix);
negative offsets now ARM once at StartLive and drop |N| ms off the pipe's write
head so audio events land earlier when audio runs BEHIND video. Slider relabeled
AUDIO SYNC, Min −500, locked while live/recording (IsEditMode). LayoutStore and
VM clamp to −500..500.

OBS reference for eat-the-head negative sync: https://obsproject.com/kb/obs-studio/buffering-time (negative sync values pull audio earlier by discarding buffered player audio).

Test: StartLive_NegativeOffset_AdvancesAudio_ByDroppingTheStreamHead (6x0.9 head
must be eaten before 0.2 bed reaches the wire).
This commit is contained in:
2026-09-14 12:30:44 -07:00
parent 11a7af2dc0
commit b22d08eca6
12 changed files with 219 additions and 144 deletions
+5 -4
View File
@@ -658,13 +658,14 @@
PreviewMouseLeftButtonUp="VolumeSlider_PreviewMouseLeftButtonUp" PreviewMouseLeftButtonUp="VolumeSlider_PreviewMouseLeftButtonUp"
LostMouseCapture="VolumeSlider_LostMouseCapture"/> LostMouseCapture="VolumeSlider_LostMouseCapture"/>
<StackPanel Orientation="Horizontal" Margin="10,0,0,0" VerticalAlignment="Center" <StackPanel Orientation="Horizontal" Margin="10,0,0,0" VerticalAlignment="Center"
ToolTip="Audio sync delay (ms): add a few ms so the mic/audio lands on the video when it runs ahead."> ToolTip="Audio sync (−500..+500 ms). Positive: delays the audio when it runs ahead of the video. Negative: advances it (drops the first |N| ms of the recording) when it runs behind. Calibrate with a clap: is the clap late in the video, move negative; early, move positive.">
<TextBlock Text="SYNC" FontSize="9" Foreground="#9898aa" <TextBlock Text="AUDIO SYNC" FontSize="9" Foreground="#9898aa"
VerticalAlignment="Center" Margin="0,0,5,0"/> VerticalAlignment="Center" Margin="0,0,5,0"/>
<Slider Width="70" Minimum="0" Maximum="500" <Slider Width="70" Minimum="-500" Maximum="500"
Value="{Binding AudioSyncOffsetMs, Mode=TwoWay}" Value="{Binding AudioSyncOffsetMs, Mode=TwoWay}"
IsEnabled="{Binding IsEditMode}"
VerticalAlignment="Center" VerticalAlignment="Center"
ToolTip="{Binding AudioSyncOffsetMs, StringFormat=Sync delay: {0} ms}"/> ToolTip="{Binding AudioSyncOffsetMs, StringFormat=Audio sync: {0} ms}"/>
<Border Width="8" Height="8" CornerRadius="4" Margin="6,0,0,0" <Border Width="8" Height="8" CornerRadius="4" Margin="6,0,0,0"
VerticalAlignment="Center" VerticalAlignment="Center"
Background="{Binding AudioSyncOffsetMs, Converter={StaticResource IntToSyncBrush}}"/> Background="{Binding AudioSyncOffsetMs, Converter={StaticResource IntToSyncBrush}}"/>
+60 -120
View File
@@ -1,143 +1,83 @@
# HANDOFF — 2026-09-14 (AUDIO−VIDEO SYNC FIX SHIPPED: StartLive now clears the pre-live ring backlog) # HANDOFF — 2026-09-14 (SIGNED AUDIO SYNC ONLINE: −500..+500, negative advances by eating the stream head; slider locked while live/recording)
## Branch / Commit State ## Branch / Commit State
`main` HEAD = `5ba4d70`. Ahead of origin by **27 commits**. Working tree **dirty**: `main` HEAD = **`11a7af2`** (pre-live ring-backlog fix, committed, pushed NOT authorized).
the sync fix is implemented + tests green below, HANDOFF/MyMistakes updated — NOT yet Working tree **dirty** with the signed-sync work below — NOT yet committed. **NOT pushing**
committed. **NOT pushing** — the prior no-push ruling (transparency + audio-silence) is (no-push ruling still in effect; the last two fixes — audio silence + ring backlog — are
clear on the first gate; the second was the sync issue, whose fix is now in the tree committed locally and the user has not greenlit a push).
but the user has not greenlit a push. On-board next: verify a take, then decide.
Heads-up for any reader: the previous HANDOFF (a62a283 era) described webcam/audio/ DB now: **`Audio.SyncOffsetMs = 0`** (confirmed via sqlite3 this session — the creator slid
truncation fixes as **uncommitted** — that file was stale on arrival. They were the sync to 0 and the write finally stuck; sound is correct at 0). The 0.54s residual in the
actually already committed: 1128 take ≈ 300ms injected offset (leftover DB value) + ~240ms natural (partly webcam
clap-quantization ±40–85ms, partly measurement).
- `724af14` fix: audio silence (idempotent `AudioSyncDelay.Configure`), webcam gray ## ✅ SHIPPED (this dirty tree) — signed audio sync, −500..+500
block (cbW/cbH default), truncated videos (SceneGraph split/static bake docs + FramePump probe)
- `5ba4d70` docs: audio-sync offset measured ~367ms w/ 300 offset → "set SyncOffsetMs=0"
Build was green (0 warnings) and 295 tests passing at the prior session end; no code **What:** the audio-sync control is now a SIGNED offset. Positive = delay the mix (audio runs
changed this session (analysis only). AHEAD of video — existing `AudioSyncDelay` behavior, unchanged and live-reactive). Negative =
**advance** the audio (audio runs BEHIND video): OBS's "eat the head of the buffer" fix —
the mixer drops the first |N| ms of the written stream at the pipe, re-anchoring the audio
stream so every event lands |N| ms EARLIER relative to video.
## ✅ SHIPPED — pre-live capture ring backlog fix (2026-09-14) **Mechanism (AudioMixer):** `StartLive` arms `_advanceSamplesRemaining = |N| ms → samples` at
go-live (a negative offset can only eat the HEAD of the stream; it is armed once, not
live-reactive). `LiveLoopAsync` skips `min(budget, mixBuffer.Length)` samples off each write
head while the budget lasts — the pipe writer accepts a partial chunk via `AsMemory(writeFrom)`.
Positive path untouched (delay line still re-reads the Func every tick).
**Root cause:** `AudioMixer.Start()` begins mic and loopback capture at **app startup** **UI/plumbing:**
for the level meters. The 2-second ring buffers (`_micBuffer` = 96k floats = 2.0s mono; - `MainViewModel.Audio.cs` — clamp `Math.Clamp(value, -500, 500)`, doc updated.
`_loopbackBuffer` = 192k floats = 2.0s stereo) fill continuously with pre-live audio. - `LayoutStore.Settings.cs` — `LoadAudioSyncOffsetMs`/`SaveAudioSyncOffsetMs` clamp −500..500.
`StartLive` → `LiveLoopAsync` drained from the ring's tail — the **oldest** sample — - `PreviewPane.xaml` — label **SYNC → "AUDIO SYNC"**, `Minimum="-500"`, tooltip explains both
so every recorded event landed ~2.0s late in the audio track (fixed lag, scaled with directions (calibrate with a clap: clap late → negative; early → positive), and
time-since-app-launch, capped at ring depth). **`IsEnabled="{Binding IsEditMode}"`** — the slider locks during live AND recording (gun
safety, same property `IsRecording`/`StreamStatus` already raise PropertyChanged for).
**Fix:** `AudioMixer.StartLive()` now runs `_micBuffer.Clear(); _loopbackBuffer.Clear();` - `AudioSyncDelay` unchanged (still clamps negative→0 internally; header doc updated to point
immediately after the pipe starts, before the drain task runs — the in-flight ≤10ms at the mixer for the advance side).
chunk loss is imperceptible and correct ("recording begins at go-live").
**Regression test (the ONE integration test for this change):** **Regression test (the ONE integration test for this change):**
`StartLive_DiscardsPreLiveBacklog_SoFirstAudioIsCurrent` in `StartLive_NegativeOffset_AdvancesAudio_ByDroppingTheStreamHead` in `AudioPipelineTests.cs` —
`ytLive.Tests/AudioPipelineTests.cs` — saturates the loopback ring with stale 0.8 −40 ms advance (8-tick budget at 5ms interval), emits 6×0.9 fresh right after StartLive (≤
(simulated pre-live meters), StartLive, then emits fresh 0.2; asserts the wire carries budget, so the head MUST be eaten) then a long 0.2 bed; asserts the wire max ≈ 0.2 (< 0.3).
~0.2 (max < 0.3), failing loudly if the stale 0.8 backlog survived the clear. Fails loudly if −N no longer drops the head (0.9 leaks).
**Test collateral:** the 3 existing pipe-emit tests pre-filled the rings BEFORE **User directives (this session):**
StartLive (relying on the bug as a reservoir). Their emits now land right after - Signed −500..+500 with positive=delay / negative=advance, relabel "AUDIO SYNC", default 0.
StartLive (post-clear, pre-connect — the pipe drops pre-connect writes, so this keeps - **Lock the sync control when live or recording** (done — `IsEditMode`).
the ring primed for the client) with the 2026-09-14 comments updated. All passed. - **NOTE ONLY, no fix:** the live recording is completely different from the "recording
results" shown in Chat view (recorded in `bugs.md` — do not rediscover as a surprise).
- **Before 1.0:** write a detailed USER-DOC tutorial on the audio-sync feature (see TASKS.md
note; add it to the gold-pass/1.0 checklist). `docs/` currently holds only the README image.
Verified: `AudioPipelineTests` 26/26 green. Room-native build 0 warnings. Full gate ## Take verification so far (ring-backlog fix)
pending (verify.sh clean build + full suite + scope check).
### Read/Write semantics confirmed (AudioRingBuffer.cs) Take `ty-20260914-1128-0000-2.mp4` (1920×1080@60fps, 356 frames / 5.93s): audio clap RMS peak
3.260s; video diff-frame 163 → 2.717s; **offset ≈ +0.54s** — collapsed from +2.11/2.22s; the
Thread-safe via `_sync`: Write evicts oldest when full; Read returns from remaining ~300ms was the baked-in `SyncOffsetMs=300` (now 0). Both streams `start_time=0` — not
`tail = (_head - _count + C) % C` — when full `tail == _head`, so every drained sample an `avoid_negative_ts` artifact.
is one lap behind the last write. `Clear()` is locked; it is the right seam.
**Root cause (confirmed two independent takes):** `AudioMixer.Start()` begins mic and
loopback capture at **app startup** for the level meters. The 2-second ring buffers
(`_micBuffer` = 96k floats = 2.0s mono; `_loopbackBuffer` = 192k floats = 2.0s stereo)
fill continuously with pre-live audio. `StartLive` → `LiveLoopAsync` drains from the
ring's tail — the **oldest** sample, which is the moment-of-go-live sample minus the
full ring depth — so every recorded event appears ~2.0s late in the audio track. The
lag is **fixed** for the entire recording and scales linearly with "how long the app was
open before you hit record", capped at the 2.0s ring depth (plus the configured sync
offset plus ~0.1s encoder pipeline).
**Read/Write semantics confirmed** in `AudioRingBuffer.cs` (thread-safe via `_sync`):
Write evicts oldest samples when full (`_count == _buffer.Length`); Read returns from
`tail = (_head - _count + C) % C` — when full, `tail == _head`, meaning every drained
sample is exactly one lap behind what was last written. `Clear()` exists and is locked;
it is the right seam.
**Only one Clear call exists** in the capture path: `RestartMic()` (AudioMixer.cs:132)
clears `_micBuffer` on device swap. **Neither buffer is cleared at `StartLive`** — this
is the bug.
**Taking:** the lag also explains the earlier 13:58 take that measured only ~367ms
with 300ms offset active: the app had just been restarted for that take, so the rings
were only partially pre-filled (lag = min(2.0s, time-since-app-launch)). Two takes
hours later (rings fully saturated) reproduce the full ~2.0s backlog consistently.
---
**Take 1** (ty-20260912-1923-0000-2.mp4, talking + 3 claps):
- Audio 17.25s, video 17.05s (1023 frames), no frame drops — NOT truncated.
- Per-clap offsets: +2.16 / +2.18 / +2.15s (audio later). Cross-correlation peak:
**+133 frames (+2.217s)**, sharp and unique. Fixed offset, no drift.
- DB: `Audio.SyncOffsetMs = 300` (not 0 as the 5ba4d70 commit message claimed —
the write never actually landed).
**Take 2** (ty-20260914-1104-0000-2.mp4, clean clap test):
- Audio 22.31s, video 22.10s (1326 frames), no frame drops.
- Audio clap peak at **14.04s**; video motion peak at **11.93s** (yavg 2.75 vs
background ~0.1–0.5). Offset: **+2.11s** — confirms the same mechanism.
- Tiny pre-clap blips at 0.2/1.5/4.5/13.5s are faint artifacts (< 100 RMS);
the 11617-RMS clap is unambiguous.
**Theory of the "cut off at the end":** audio events land ~2s late; when the video
ends the audio track is still playing the final 2s of real-time content, giving the
impression of a cut. The audio track's slightly longer duration (17.25 vs 17.05;
22.31 vs 22.10) is the tail of that delayed signal — real audio outlives the
pump-stopped video.
**Theories RULED OUT by data:**
- ❌ Frame truncation / slow full-render — video is real-time; frame counts match
wall-clock (FramePump log: 301/300 per 5s, dropped 0, avg render 14ms).
- ❌ Named-pipe-connect drop — would *shorten* audio (events earlier), opposite sign.
Pipe connected near-instantly in both takes.
- ❌ AudioSyncDelay alone — capped at 0..500ms; 300ms is a subset, not the whole.
- ❌ Audio drift/resampler rate — offset is constant, not growing.
## Open threads ## Open threads
- **Audio-silence verification** — FIXED code (idempotent `Configure`, committed - **Verify signed sync on device:** re-run the clap take with a NEGATIVE offset to confirm the
`724af14`); the 19:23 take had real peakMix. Final user confirmation still pending. advance direction end-to-end (the regression test proves the mixer; a take proves the file).
- **Audio sync offset DB value** — `Audio.SyncOffsetMs = 300` (the "set to 0" write - **Audio-silence verification** — fixed code (`724af14`) confirmed; creator heard real audio.
in 5ba4d70 never actually landed). Fix for the ~2s ring backlog is now SHIPPED (local, - **Webcam MJPG missing / ~10–14Hz**, layer SortOrder, truncation-with-dynamic-scenes — queued.
uncommitted). Next: re-record the clap pattern, run the cross-correlation recipe → expect - **Sync control tutorial in user docs — REQUIRED before 1.0** (creator directive).
residual ≈ 400ms (300ms configured sync + ~100ms pipeline); then optionally zero the
slider to hit ~100ms.
- **Truncation with DYNAMIC scenes** — proven mechanism + probe in place; static
scenes full-length. 19:23 take confirms dynamic scenes hold ~60fps (no truncation)
but with 30ms stall cadence.
- **Webcam MJPG missing**, **web capture ~10–14Hz**, **layer SortOrder not persisting**
— queued, unchanged.
## Landmines ## Landmines
- testhost shares startup.log — filter by time. - testhost shares startup.log — filter by time.
- `cmd.exe /c "taskkill /F /IM ytLive.exe"` (WSL form double-slashes mangle) before rebuilds. - `cmd.exe /c "taskkill /F /IM ytLive.exe"` (WSL double-slashes mangle) before rebuilds.
- Build/tests: Windows dotnet host (`/mnt/c/Program Files/dotnet/dotnet.exe`). 0 warnings. - Build/tests: **Windows dotnet host** (`/mnt/c/Program Files/dotnet/dotnet.exe`). 0 warnings.
- ffmpeg/ffprobe: `/mnt/c/Program Files/Krita (x64)/bin/ffmpeg.exe` with Windows paths. - ffmpeg/ffprobe: `/mnt/c/Program Files/Krita (x64)/bin/` with Windows paths.
- **`Audio.SyncOffsetMs` currently = 300 in the DB** (contradicts the "set to 0" - `MyMistakes.md` has the **audio/video sync measurement recipe** (claps + cross-correlation)
line in 5ba4d70; the write likely never happened). Non-zero offsets still depend on — grep it before re-deriving.
the idempotent `Configure` guard to avoid the silence bug. - sqlite3 lives at `/home/gramps/android-sdk/platform-tools/sqlite3` (WSL) for the DB at
- `MyMistakes.md` → Recipes registry now has the **audio/video sync measurement `/mnt/c/Users/gramp/AppData/Roaming/ytLlive/ytLlive.db`.
recipe** (claps + cross-correlation) — grep before re-deriving.
## Next step ## Next step
**Verify the shipped fix:** re-record with the same clap pattern and run the Run the verify.sh gate (clean build, 0 warnings, full suite, scope check) on the dirty tree,
cross-correlation recipe from `MyMistakes.md` → expect offset ≈ 400ms (300ms configured then commit the signed-sync work unit (no push). Optionally a device clap take with a negative
sync + ~100ms pipeline) with a `SyncOffsetMs=300` DB value, or ~100ms with the slider offset to validate the advance on the file.
zeroed. Then commit this session's work (StartLive clear + regression test + docs) and
decide with the user whether the no-push ruling is lifted.
+22
View File
@@ -495,3 +495,25 @@ stamp BEFORE theorizing. (Also this session: a sentinel-byte test where the sour
GENERATE the sentinel, and a fixed-point bilinear that shifted BOTH stages and silently drew nothing — GENERATE the sentinel, and a fixed-point bilinear that shifted BOTH stages and silently drew nothing —
see the rawvideo recipe.) see the rawvideo recipe.)
## Signed A/V sync: negative offset = eat the buffer head (OBS semantics), armed at go-live
(2026-09-14) The ring-backlog fix shrank A/V lag from ~2.2s to ~0.54s, and ~300ms of that was
baked into the DB (`Audio.SyncOffsetMs=300`) via a POSITIVE-only control. For audio running BEHIND
video you cannot push audio later — positive delay makes it worse. OBS's answer is a NEGATIVE offset
that eats the buffer head: drop the first |N| ms of the written stream so every audio event lands
|N| ms EARLIER relative to video. It only makes sense at the head of the stream, so it is armed once
at `StartLive` (positive stays live-reactive via the delay line). Rule: record the signed semantics
together — positive = delay (ahead), negative = eat-head advance (behind) — and lock the control
while live (`IsEditMode`) since a mid-stream advance flip is meaningless. Test recipe:
`StartLive_NegativeOffset_AdvancesAudio_ByDroppingTheStreamHead` (emit 6×0.9 into an 8-tick budget,
long 0.2 bed, wire must show only 0.2).
## A pipe-read test that samples one tick is a timing flake by construction
(2026-09-14, re-discovered) `Mix_HonorsProviderGains_AndGameMute_KillsTheLoopback` fails
sporadically in ISOLATION (3/3 on the clean tree) but passes in the full suite: it reads exactly one
tick's worth after muting, and if the pipe still carries pre-mute buffered bytes the read spikes at
0.4 instead of silence. Any caller of `ReadFullyAsync` on a live pipe must either close/drain the
pipe first or assert on a multi-tick window. Do not blame a sync change for this — verify the grain
against the clean tree in the same mode before touching the mixer.
+29 -2
View File
@@ -32,6 +32,7 @@ public sealed class AudioMixer : IDisposable
private readonly TimeSpan _mixInterval; private readonly TimeSpan _mixInterval;
private readonly Func<int>? _syncOffsetMs; private readonly Func<int>? _syncOffsetMs;
private readonly AudioSyncDelay _syncDelay; private readonly AudioSyncDelay _syncDelay;
private int _advanceSamplesRemaining;
private bool _started; private bool _started;
// Live path (TASK 9): per-source resampling → voice chain on the mic → // Live path (TASK 9): per-source resampling → voice chain on the mic →
@@ -168,6 +169,18 @@ public sealed class AudioMixer : IDisposable
_micBuffer.Clear(); _micBuffer.Clear();
_loopbackBuffer.Clear(); _loopbackBuffer.Clear();
// Negative sync offset (audio runs BEHIND video): OBS's fix is to "eat
// the head of the buffer" — discard the first |N| ms of the live stream
// so every subsequent audio event lands |N| ms EARLIER in the recording.
// Armed once at go-live (you can't eat the head mid-stream); positive
// offsets stay live-reactive through the delay line in FillAndMix.
var syncMs = _syncOffsetMs?.Invoke() ?? 0;
_advanceSamplesRemaining = syncMs < 0
? (int)Math.Round(OutputSampleRate / 1000.0 * -syncMs) * 2
: 0;
if (_advanceSamplesRemaining > 0)
_log?.Invoke($"Audio live: advance armed ({-syncMs} ms ahead)");
var cts = new CancellationTokenSource(); var cts = new CancellationTokenSource();
_liveCts = cts; _liveCts = cts;
_pipe = new NamedPipeAudioWriter(); _pipe = new NamedPipeAudioWriter();
@@ -293,11 +306,25 @@ public sealed class AudioMixer : IDisposable
try try
{ {
var (_, micDrained, loopDrained) = FillAndMix(micChunk, loopbackChunk, mixBuffer); var (_, micDrained, loopDrained) = FillAndMix(micChunk, loopbackChunk, mixBuffer);
await pipe.WriteAsync(mixBuffer, cancellationToken).ConfigureAwait(false);
// Negative offset: skip the armed |N| ms of stream head. The pipe
// writer tolerates a partial chunk (any float length is valid);
// dropping the head re-anchors the audio stream against the video
// stream, making each event land earlier — OBS's eat-the-head fix.
var writeFrom = 0;
if (_advanceSamplesRemaining > 0)
{
var drop = Math.Min(_advanceSamplesRemaining, mixBuffer.Length);
_advanceSamplesRemaining -= drop;
writeFrom = drop;
}
if (writeFrom < mixBuffer.Length)
await pipe.WriteAsync(mixBuffer.AsMemory(writeFrom), cancellationToken).ConfigureAwait(false);
_micDrainedTotal += micDrained; _micDrainedTotal += micDrained;
_loopDrainedTotal += loopDrained; _loopDrainedTotal += loopDrained;
for (var i = 0; i < mixBuffer.Length; i++) for (var i = writeFrom; i < mixBuffer.Length; i++)
{ {
var abs = Math.Abs(mixBuffer[i]); var abs = Math.Abs(mixBuffer[i]);
if (abs > _peakMix) _peakMix = abs; if (abs > _peakMix) _peakMix = abs;
+4 -2
View File
@@ -10,8 +10,10 @@ namespace ytLive.Services.Audio;
/// the same audio N ms later out — so it is trivially testable and carries no /// the same audio N ms later out — so it is trivially testable and carries no
/// state outside its own ring. /// state outside its own ring.
/// ///
/// Positive offsets only (0..500 ms). Advancing audio (<see cref="TimeSpan"/> /// Positive offsets only (0..500 ms). Advancing audio (the audio runs BEHIND
/// would need a video-side delay and is out of scope for the audio layer. /// the video) is handled upstream in <see cref="AudioMixer"/>, not here: the
/// mixer eats the first |N| ms of the live stream head (OBS's negative sync —
/// see <see cref="AudioMixer.StartLive"/>), keeping this line a pure delay.
/// Changing the delay flushes the line, so a live slider change clicks rather /// Changing the delay flushes the line, so a live slider change clicks rather
/// than smearing. Safe to call from any thread — the mixer owns it. /// than smearing. Safe to call from any thread — the mixer owns it.
/// </summary> /// </summary>
+6 -4
View File
@@ -29,17 +29,19 @@ public partial class LayoutStore : IDisposable
insert.ExecuteNonQuery(); insert.ExecuteNonQuery();
} }
/// <summary>The global audio-sync delay in milliseconds (TASK 22), 0..500, /// <summary>The global audio-sync offset in milliseconds (TASK 22), −500..500,
/// default 0. Restored at startup so the mixer starts on the user's offset.</summary> /// default 0: positive delays the mix (audio ahead of video), negative
/// advances it (audio behind video). Restored at startup so the mixer starts
/// on the user's offset.</summary>
public int LoadAudioSyncOffsetMs() public int LoadAudioSyncOffsetMs()
{ {
var value = GetSetting("Audio.SyncOffsetMs"); var value = GetSetting("Audio.SyncOffsetMs");
return int.TryParse(value, out var ms) ? Math.Clamp(ms, 0, 500) : 0; return int.TryParse(value, out var ms) ? Math.Clamp(ms, -500, 500) : 0;
} }
public void SaveAudioSyncOffsetMs(int ms) public void SaveAudioSyncOffsetMs(int ms)
{ {
UpsertSetting("Audio.SyncOffsetMs", Math.Clamp(ms, 0, 500).ToString()); UpsertSetting("Audio.SyncOffsetMs", Math.Clamp(ms, -500, 500).ToString());
} }
/// <summary>The user's chosen record output folder (TASK 18), or null to use the /// <summary>The user's chosen record output folder (TASK 18), or null to use the
+20
View File
@@ -17,7 +17,27 @@
### Design decisions ### Design decisions
- **Global offset first** — one setting for all audio sources. Per-source is v1.1+. - **Global offset first** — one setting for all audio sources. Per-source is v1.1+.
- **Signed offset (2026-09-14)** — WIDENED from positive-only 0..500 to **−500..+500**.
> The "positive-only / advance needs video-side delay / out of scope" line below is
> RETIRED. Negative offsets now ADVANCE the audio by eating the head of the live stream
> (OBS's negative-sync behavior): `AudioMixer.StartLive` arms `_advanceSamplesRemaining`
> = |N| ms → samples, and `LiveLoopAsync` drops that many samples off the write head.
> Positive keeps using the `AudioSyncDelay` line live-reactive. Slider relabelled
> "AUDIO SYNC", `Min="-500"`, and **locked (`IsEnabled = IsEditMode`) while live or
> recording** — a negative advance can only be armed at go-live, so it must not move
> mid-session. Clamps: `Pre-viewModel` + `LayoutStore` ±500; `AudioSyncDelay` still
> clamps negative→0 internally (pure delay line, unchanged).
> **Provenance (2026-09-14): creator-directive** — asked for both directions after the
> ring-backlog fix surfaced the residual (audio can run late too: capture cards, BT,
> webcams). Regression test: `StartLive_NegativeOffset_AdvancesAudio_ByDroppingTheStreamHead`.
- **Positive-only (delay audio)** — the physically-correct direction (audio runs ahead of the video). True "advance" needs a video-side delay and is PERMANENTLY OUT with per-source sync (TASKS.md → "Out of product" — v1.x phrasing retired 2026-09-01). - **Positive-only (delay audio)** — the physically-correct direction (audio runs ahead of the video). True "advance" needs a video-side delay and is PERMANENTLY OUT with per-source sync (TASKS.md → "Out of product" — v1.x phrasing retired 2026-09-01).
- **Simple slider** — 0 to +500 ms, default 0. No numeric input needed. - **Simple slider** — 0 to +500 ms, default 0. No numeric input needed.
- **Visual feedback** — "sync OK" status dot shows when an offset is dialled in. - **Visual feedback** — "sync OK" status dot shows when an offset is dialled in.
### ❗ REQUIRED before 1.0 (creator directive 2026-09-14)
A **detailed user-doc tutorial** on the audio-sync feature (what -500..+500 means, the
clap-calibration recipe both directions, and that it locks while live). `docs/` currently
holds only the README image — the tutorial is unstarted. Add it to the 1.0/gold-pass checklist.
+6 -5
View File
@@ -155,16 +155,17 @@ public partial class MainViewModel
private set => SetProperty(ref _micSourceName, value); private set => SetProperty(ref _micSourceName, value);
} }
/// <summary>Global audio-sync delay in milliseconds (0..500, default 0), /// <summary>Global audio-sync offset in milliseconds (−500..500, default 0):
/// applied to the whole mix so audio lands on the video when it runs ahead /// POSITIVE delays the whole mix so audio lands on the video when it runs
/// — the OBS fix for lip-sync. Consumed live by the mixer's delay line via /// AHEAD (OBS's lip-sync fix); NEGATIVE advances it by dropping the first
/// a Func seam; persisted with the layout.</summary> /// |N| ms of the live stream head when audio runs BEHIND the video.
/// Consumed live by the mixer via a Func seam; persisted with the layout.</summary>
public int AudioSyncOffsetMs public int AudioSyncOffsetMs
{ {
get => _audioSyncOffsetMs; get => _audioSyncOffsetMs;
set set
{ {
var clamped = Math.Clamp(value, 0, 500); var clamped = Math.Clamp(value, -500, 500);
if (SetProperty(ref _audioSyncOffsetMs, clamped)) if (SetProperty(ref _audioSyncOffsetMs, clamped))
ScheduleSave(); ScheduleSave();
} }
+9 -5
View File
@@ -626,11 +626,15 @@ devices, no timers).
pipe lifecycle. pipe lifecycle.
- **Audio sync offset (TASK 22):** after the limiter, `FillAndMix` routes the whole interleaved stereo - **Audio sync offset (TASK 22):** after the limiter, `FillAndMix` routes the whole interleaved stereo
mix through a **`AudioSyncDelay`** (`Services/Audio/AudioSyncDelay.cs`) — a pure delay line whose mix through a **`AudioSyncDelay`** (`Services/Audio/AudioSyncDelay.cs`) — a pure delay line whose
offset comes from a `Func<int> syncOffsetMs` seam (the VM's global `AudioSyncOffsetMs`, 0..500 ms, offset comes from a `Func<int> syncOffsetMs` seam (the VM's global `AudioSyncOffsetMs`, **−500..+500
persisted as `Audio.SyncOffsetMs`). This is OBS's documented fix for audio running ahead of the video ms**, persisted as `Audio.SyncOffsetMs`). This is OBS's documented fix for lip-sync: **positive**
(delay it a few ms); positive-only, since advancing audio would need a video-side delay. A slider on delays audio when it runs ahead of video; **negative** advances audio when it runs behind by eating
the mic bar + a status dot (`IntToSyncBrushConverter`) surface it. Changing the delay flushes the line the first `|N|` ms of the live stream head (see `AudioMixer.StartLive`; the delay line stays
(a live change clicks rather than smears). clamped 0..500 internally — negative bypasses it entirely). The advance is armed once at `StartLive`
(eating the head mid-stream is impossible); positive is live-reactive (re-read per tick). A slider
on the mic bar (`"AUDIO SYNC"`, locked while live/recording via `IsEditMode`) + a status dot
(`IntToSyncBrushConverter`) surface it. Changing the delay flushes the line (a live change clicks
rather than smears).
- **TRAX — free background music (TASK 8):** `MusicPlayer` = NAudio `MediaFoundationReader` - **TRAX — free background music (TASK 8):** `MusicPlayer` = NAudio `MediaFoundationReader`
(mp3/wav/m4a) → `VolumeWaveProvider16` at the hardcoded **0.20** bed (no slider) → `WaveOutEvent` on (mp3/wav/m4a) → `VolumeWaveProvider16` at the hardcoded **0.20** bed (no slider) → `WaveOutEvent` on
the default device, **looping on any clean natural end** (`PlaybackStopped` with `e.Exception == null` the default device, **looping on any clean natural end** (`PlaybackStopped` with `e.Exception == null`
+7
View File
@@ -7,3 +7,10 @@ separate devices. If iTunes audio appears on the mic meter, it is likely acousti
coupling — the mic picking up speaker/headphone output. Verify by muting the mic; coupling — the mic picking up speaker/headphone output. Verify by muting the mic;
if the meter still shows levels, check Windows Sound settings for stereo mix or if the meter still shows levels, check Windows Sound settings for stereo mix or
"listen to this device" routing. "listen to this device" routing.
## Live recording ≠ "recording results" shown in Chat view (NOTE ONLY — no fix)
User note (2026-09-14): the recording produced while live is completely different
from the "recording results" surfaced in the Chat view. Do NOT attempt to fix or
reconcile them pre-1.0 — this entry exists so the discrepancy is never rediscovered
as a surprise. When the reconciliation work is eventually queued, start by mapping
both recording paths side by side.
+48
View File
@@ -478,6 +478,54 @@ public class AudioPipelineTests
Assert.True(floats.Max() < 0.3f, "stale pre-live backlog leaked onto the wire"); Assert.True(floats.Max() < 0.3f, "stale pre-live backlog leaked onto the wire");
} }
[Fact]
public async Task StartLive_NegativeOffset_AdvancesAudio_ByDroppingTheStreamHead()
{
// THE integration test for the signed-sync branch (2026-09-14): a
// NEGATIVE offset must ADVANCE audio — OBS eats the buffer head so the
// audio lands earlier on the video. StartLive arms a |N| ms sample
// budget; the live loop skips that much off the write head. With a -40ms
// offset at a 5ms mix interval (480 samples/tick at 48k), the budget is
// 3840 samples = a full 8 ticks. The first 6 ticks carry fresh 0.9 audio,
// which MUST be eaten; the long 0.2 bed behind it is what the wire shows
// (0.2 on the wire, never 0.9). WITHOUT the advance, 0.9 leaks straight
// through and the test fails.
var pipeName = "ytllive_test_" + Guid.NewGuid().ToString("N");
var mic = new FakeSource();
var loopback = new FakeSource();
using var mixer = new AudioMixer(
mic, loopback,
log: null,
syncOffsetMs: () => -40,
mixInterval: TimeSpan.FromMilliseconds(5));
using var client = new NamedPipeClientStream(".", pipeName, PipeDirection.In, PipeOptions.Asynchronous);
var connected = client.ConnectAsync();
mixer.StartLive(pipeName);
// 6 ticks of head 0.9 (all within the 8-tick advance budget) immediately
// after StartLive, then a long 0.2 bed. Zero mic, unity gains, silent
// ducker, quiet limiter — the loopback bed rides the wire ~1:1.
var head = new float[480];
Array.Fill(head, 0.9f);
for (var i = 0; i < 6; i++)
loopback.Emit(new AudioSample(head, 48000, 2));
var bed = new float[480];
Array.Fill(bed, 0.2f);
for (var i = 0; i < 60; i++)
loopback.Emit(new AudioSample(bed, 48000, 2));
await connected.WaitAsync(TimeSpan.FromSeconds(5));
var bytes = await ReadFullyAsync(client, 240 * 2 * 4, TimeSpan.FromSeconds(5));
mixer.StopLive();
var floats = new float[bytes.Length / 4];
Buffer.BlockCopy(bytes, 0, floats, 0, bytes.Length);
Assert.True(floats.Length > 0, "expected wired audio");
Assert.True(floats.Max() < 0.3f, "the negative-offset head was NOT eaten (0.9 leaked)");
Assert.True(floats.Any(f => MathF.Abs(f) > 0.05f), "the pipe carried only silence");
}
private static async Task<byte[]> ReadFullyAsync(NamedPipeClientStream client, int count, TimeSpan timeout) private static async Task<byte[]> ReadFullyAsync(NamedPipeClientStream client, int count, TimeSpan timeout)
{ {
var ms = new MemoryStream(); var ms = new MemoryStream();
+3 -2
View File
@@ -6,8 +6,9 @@ namespace ytLive.Tests;
/// <summary> /// <summary>
/// TASK 22: the global audio-sync delay line. It is a pure fixed positive /// TASK 22: the global audio-sync delay line. It is a pure fixed positive
/// delay on the interleaved stereo mix — the OBS fix for audio running ahead /// delay on the interleaved stereo mix — the OBS fix for audio running ahead
/// of video. Positive-only: advancing audio needs a video-side delay and is /// of video. Positive-only: the line clamps negatives to 0; the NEGATIVE
/// out of the audio layer's scope. /// (advance) side — audio running behind video, the OBS "eat the buffer head"
/// case — is covered by <see cref="AudioPipelineTests.StartLive_NegativeOffset_AdvancesAudio_ByDroppingTheStreamHead"/>.
/// </summary> /// </summary>
public class AudioSyncDelayTests public class AudioSyncDelayTests
{ {