feat: signed A/V sync offset (−500..+500), negative advances by eating live stream head (OBS eat-head semantics)
Positive offsets still delay the whole mix via the delay line (lip-sync fix); negative offsets now ARM once at StartLive and drop |N| ms off the pipe's write head so audio events land earlier when audio runs BEHIND video. Slider relabeled AUDIO SYNC, Min −500, locked while live/recording (IsEditMode). LayoutStore and VM clamp to −500..500. OBS reference for eat-the-head negative sync: https://obsproject.com/kb/obs-studio/buffering-time (negative sync values pull audio earlier by discarding buffered player audio). Test: StartLive_NegativeOffset_AdvancesAudio_ByDroppingTheStreamHead (6x0.9 head must be eaten before 0.2 bed reaches the wire).
This commit is contained in:
@@ -658,13 +658,14 @@
|
||||
PreviewMouseLeftButtonUp="VolumeSlider_PreviewMouseLeftButtonUp"
|
||||
LostMouseCapture="VolumeSlider_LostMouseCapture"/>
|
||||
<StackPanel Orientation="Horizontal" Margin="10,0,0,0" VerticalAlignment="Center"
|
||||
ToolTip="Audio sync delay (ms): add a few ms so the mic/audio lands on the video when it runs ahead.">
|
||||
<TextBlock Text="SYNC" FontSize="9" Foreground="#9898aa"
|
||||
ToolTip="Audio sync (−500..+500 ms). Positive: delays the audio when it runs ahead of the video. Negative: advances it (drops the first |N| ms of the recording) when it runs behind. Calibrate with a clap: is the clap late in the video, move negative; early, move positive.">
|
||||
<TextBlock Text="AUDIO SYNC" FontSize="9" Foreground="#9898aa"
|
||||
VerticalAlignment="Center" Margin="0,0,5,0"/>
|
||||
<Slider Width="70" Minimum="0" Maximum="500"
|
||||
<Slider Width="70" Minimum="-500" Maximum="500"
|
||||
Value="{Binding AudioSyncOffsetMs, Mode=TwoWay}"
|
||||
IsEnabled="{Binding IsEditMode}"
|
||||
VerticalAlignment="Center"
|
||||
ToolTip="{Binding AudioSyncOffsetMs, StringFormat=Sync delay: {0} ms}"/>
|
||||
ToolTip="{Binding AudioSyncOffsetMs, StringFormat=Audio sync: {0} ms}"/>
|
||||
<Border Width="8" Height="8" CornerRadius="4" Margin="6,0,0,0"
|
||||
VerticalAlignment="Center"
|
||||
Background="{Binding AudioSyncOffsetMs, Converter={StaticResource IntToSyncBrush}}"/>
|
||||
|
||||
+60
-120
@@ -1,143 +1,83 @@
|
||||
# HANDOFF — 2026-09-14 (AUDIO−VIDEO SYNC FIX SHIPPED: StartLive now clears the pre-live ring backlog)
|
||||
# HANDOFF — 2026-09-14 (SIGNED AUDIO SYNC ONLINE: −500..+500, negative advances by eating the stream head; slider locked while live/recording)
|
||||
|
||||
## Branch / Commit State
|
||||
|
||||
`main` HEAD = `5ba4d70`. Ahead of origin by **27 commits**. Working tree **dirty**:
|
||||
the sync fix is implemented + tests green below, HANDOFF/MyMistakes updated — NOT yet
|
||||
committed. **NOT pushing** — the prior no-push ruling (transparency + audio-silence) is
|
||||
clear on the first gate; the second was the sync issue, whose fix is now in the tree
|
||||
but the user has not greenlit a push. On-board next: verify a take, then decide.
|
||||
`main` HEAD = **`11a7af2`** (pre-live ring-backlog fix, committed, pushed NOT authorized).
|
||||
Working tree **dirty** with the signed-sync work below — NOT yet committed. **NOT pushing**
|
||||
(no-push ruling still in effect; the last two fixes — audio silence + ring backlog — are
|
||||
committed locally and the user has not greenlit a push).
|
||||
|
||||
Heads-up for any reader: the previous HANDOFF (a62a283 era) described webcam/audio/
|
||||
truncation fixes as **uncommitted** — that file was stale on arrival. They were
|
||||
actually already committed:
|
||||
DB now: **`Audio.SyncOffsetMs = 0`** (confirmed via sqlite3 this session — the creator slid
|
||||
the sync to 0 and the write finally stuck; sound is correct at 0). The 0.54s residual in the
|
||||
1128 take ≈ 300ms injected offset (leftover DB value) + ~240ms natural (partly webcam
|
||||
clap-quantization ±40–85ms, partly measurement).
|
||||
|
||||
- `724af14` fix: audio silence (idempotent `AudioSyncDelay.Configure`), webcam gray
|
||||
block (cbW/cbH default), truncated videos (SceneGraph split/static bake docs + FramePump probe)
|
||||
- `5ba4d70` docs: audio-sync offset measured ~367ms w/ 300 offset → "set SyncOffsetMs=0"
|
||||
## ✅ SHIPPED (this dirty tree) — signed audio sync, −500..+500
|
||||
|
||||
Build was green (0 warnings) and 295 tests passing at the prior session end; no code
|
||||
changed this session (analysis only).
|
||||
**What:** the audio-sync control is now a SIGNED offset. Positive = delay the mix (audio runs
|
||||
AHEAD of video — existing `AudioSyncDelay` behavior, unchanged and live-reactive). Negative =
|
||||
**advance** the audio (audio runs BEHIND video): OBS's "eat the head of the buffer" fix —
|
||||
the mixer drops the first |N| ms of the written stream at the pipe, re-anchoring the audio
|
||||
stream so every event lands |N| ms EARLIER relative to video.
|
||||
|
||||
## ✅ SHIPPED — pre-live capture ring backlog fix (2026-09-14)
|
||||
**Mechanism (AudioMixer):** `StartLive` arms `_advanceSamplesRemaining = |N| ms → samples` at
|
||||
go-live (a negative offset can only eat the HEAD of the stream; it is armed once, not
|
||||
live-reactive). `LiveLoopAsync` skips `min(budget, mixBuffer.Length)` samples off each write
|
||||
head while the budget lasts — the pipe writer accepts a partial chunk via `AsMemory(writeFrom)`.
|
||||
Positive path untouched (delay line still re-reads the Func every tick).
|
||||
|
||||
**Root cause:** `AudioMixer.Start()` begins mic and loopback capture at **app startup**
|
||||
for the level meters. The 2-second ring buffers (`_micBuffer` = 96k floats = 2.0s mono;
|
||||
`_loopbackBuffer` = 192k floats = 2.0s stereo) fill continuously with pre-live audio.
|
||||
`StartLive` → `LiveLoopAsync` drained from the ring's tail — the **oldest** sample —
|
||||
so every recorded event landed ~2.0s late in the audio track (fixed lag, scaled with
|
||||
time-since-app-launch, capped at ring depth).
|
||||
|
||||
**Fix:** `AudioMixer.StartLive()` now runs `_micBuffer.Clear(); _loopbackBuffer.Clear();`
|
||||
immediately after the pipe starts, before the drain task runs — the in-flight ≤10ms
|
||||
chunk loss is imperceptible and correct ("recording begins at go-live").
|
||||
**UI/plumbing:**
|
||||
- `MainViewModel.Audio.cs` — clamp `Math.Clamp(value, -500, 500)`, doc updated.
|
||||
- `LayoutStore.Settings.cs` — `LoadAudioSyncOffsetMs`/`SaveAudioSyncOffsetMs` clamp −500..500.
|
||||
- `PreviewPane.xaml` — label **SYNC → "AUDIO SYNC"**, `Minimum="-500"`, tooltip explains both
|
||||
directions (calibrate with a clap: clap late → negative; early → positive), and
|
||||
**`IsEnabled="{Binding IsEditMode}"`** — the slider locks during live AND recording (gun
|
||||
safety, same property `IsRecording`/`StreamStatus` already raise PropertyChanged for).
|
||||
- `AudioSyncDelay` unchanged (still clamps negative→0 internally; header doc updated to point
|
||||
at the mixer for the advance side).
|
||||
|
||||
**Regression test (the ONE integration test for this change):**
|
||||
`StartLive_DiscardsPreLiveBacklog_SoFirstAudioIsCurrent` in
|
||||
`ytLive.Tests/AudioPipelineTests.cs` — saturates the loopback ring with stale 0.8
|
||||
(simulated pre-live meters), StartLive, then emits fresh 0.2; asserts the wire carries
|
||||
~0.2 (max < 0.3), failing loudly if the stale 0.8 backlog survived the clear.
|
||||
`StartLive_NegativeOffset_AdvancesAudio_ByDroppingTheStreamHead` in `AudioPipelineTests.cs` —
|
||||
−40 ms advance (8-tick budget at 5ms interval), emits 6×0.9 fresh right after StartLive (≤
|
||||
budget, so the head MUST be eaten) then a long 0.2 bed; asserts the wire max ≈ 0.2 (< 0.3).
|
||||
Fails loudly if −N no longer drops the head (0.9 leaks).
|
||||
|
||||
**Test collateral:** the 3 existing pipe-emit tests pre-filled the rings BEFORE
|
||||
StartLive (relying on the bug as a reservoir). Their emits now land right after
|
||||
StartLive (post-clear, pre-connect — the pipe drops pre-connect writes, so this keeps
|
||||
the ring primed for the client) with the 2026-09-14 comments updated. All passed.
|
||||
**User directives (this session):**
|
||||
- Signed −500..+500 with positive=delay / negative=advance, relabel "AUDIO SYNC", default 0.
|
||||
- **Lock the sync control when live or recording** (done — `IsEditMode`).
|
||||
- **NOTE ONLY, no fix:** the live recording is completely different from the "recording
|
||||
results" shown in Chat view (recorded in `bugs.md` — do not rediscover as a surprise).
|
||||
- **Before 1.0:** write a detailed USER-DOC tutorial on the audio-sync feature (see TASKS.md
|
||||
note; add it to the gold-pass/1.0 checklist). `docs/` currently holds only the README image.
|
||||
|
||||
Verified: `AudioPipelineTests` 26/26 green. Room-native build 0 warnings. Full gate
|
||||
pending (verify.sh clean build + full suite + scope check).
|
||||
## Take verification so far (ring-backlog fix)
|
||||
|
||||
### Read/Write semantics confirmed (AudioRingBuffer.cs)
|
||||
|
||||
Thread-safe via `_sync`: Write evicts oldest when full; Read returns from
|
||||
`tail = (_head - _count + C) % C` — when full `tail == _head`, so every drained sample
|
||||
is one lap behind the last write. `Clear()` is locked; it is the right seam.
|
||||
|
||||
**Root cause (confirmed two independent takes):** `AudioMixer.Start()` begins mic and
|
||||
loopback capture at **app startup** for the level meters. The 2-second ring buffers
|
||||
(`_micBuffer` = 96k floats = 2.0s mono; `_loopbackBuffer` = 192k floats = 2.0s stereo)
|
||||
fill continuously with pre-live audio. `StartLive` → `LiveLoopAsync` drains from the
|
||||
ring's tail — the **oldest** sample, which is the moment-of-go-live sample minus the
|
||||
full ring depth — so every recorded event appears ~2.0s late in the audio track. The
|
||||
lag is **fixed** for the entire recording and scales linearly with "how long the app was
|
||||
open before you hit record", capped at the 2.0s ring depth (plus the configured sync
|
||||
offset plus ~0.1s encoder pipeline).
|
||||
|
||||
**Read/Write semantics confirmed** in `AudioRingBuffer.cs` (thread-safe via `_sync`):
|
||||
Write evicts oldest samples when full (`_count == _buffer.Length`); Read returns from
|
||||
`tail = (_head - _count + C) % C` — when full, `tail == _head`, meaning every drained
|
||||
sample is exactly one lap behind what was last written. `Clear()` exists and is locked;
|
||||
it is the right seam.
|
||||
|
||||
**Only one Clear call exists** in the capture path: `RestartMic()` (AudioMixer.cs:132)
|
||||
clears `_micBuffer` on device swap. **Neither buffer is cleared at `StartLive`** — this
|
||||
is the bug.
|
||||
|
||||
**Taking:** the lag also explains the earlier 13:58 take that measured only ~367ms
|
||||
with 300ms offset active: the app had just been restarted for that take, so the rings
|
||||
were only partially pre-filled (lag = min(2.0s, time-since-app-launch)). Two takes
|
||||
hours later (rings fully saturated) reproduce the full ~2.0s backlog consistently.
|
||||
|
||||
---
|
||||
|
||||
**Take 1** (ty-20260912-1923-0000-2.mp4, talking + 3 claps):
|
||||
- Audio 17.25s, video 17.05s (1023 frames), no frame drops — NOT truncated.
|
||||
- Per-clap offsets: +2.16 / +2.18 / +2.15s (audio later). Cross-correlation peak:
|
||||
**+133 frames (+2.217s)**, sharp and unique. Fixed offset, no drift.
|
||||
- DB: `Audio.SyncOffsetMs = 300` (not 0 as the 5ba4d70 commit message claimed —
|
||||
the write never actually landed).
|
||||
|
||||
**Take 2** (ty-20260914-1104-0000-2.mp4, clean clap test):
|
||||
- Audio 22.31s, video 22.10s (1326 frames), no frame drops.
|
||||
- Audio clap peak at **14.04s**; video motion peak at **11.93s** (yavg 2.75 vs
|
||||
background ~0.1–0.5). Offset: **+2.11s** — confirms the same mechanism.
|
||||
- Tiny pre-clap blips at 0.2/1.5/4.5/13.5s are faint artifacts (< 100 RMS);
|
||||
the 11617-RMS clap is unambiguous.
|
||||
|
||||
**Theory of the "cut off at the end":** audio events land ~2s late; when the video
|
||||
ends the audio track is still playing the final 2s of real-time content, giving the
|
||||
impression of a cut. The audio track's slightly longer duration (17.25 vs 17.05;
|
||||
22.31 vs 22.10) is the tail of that delayed signal — real audio outlives the
|
||||
pump-stopped video.
|
||||
|
||||
**Theories RULED OUT by data:**
|
||||
- ❌ Frame truncation / slow full-render — video is real-time; frame counts match
|
||||
wall-clock (FramePump log: 301/300 per 5s, dropped 0, avg render 14ms).
|
||||
- ❌ Named-pipe-connect drop — would *shorten* audio (events earlier), opposite sign.
|
||||
Pipe connected near-instantly in both takes.
|
||||
- ❌ AudioSyncDelay alone — capped at 0..500ms; 300ms is a subset, not the whole.
|
||||
- ❌ Audio drift/resampler rate — offset is constant, not growing.
|
||||
Take `ty-20260914-1128-0000-2.mp4` (1920×1080@60fps, 356 frames / 5.93s): audio clap RMS peak
|
||||
3.260s; video diff-frame 163 → 2.717s; **offset ≈ +0.54s** — collapsed from +2.11/2.22s; the
|
||||
remaining ~300ms was the baked-in `SyncOffsetMs=300` (now 0). Both streams `start_time=0` — not
|
||||
an `avoid_negative_ts` artifact.
|
||||
|
||||
## Open threads
|
||||
|
||||
- **Audio-silence verification** — FIXED code (idempotent `Configure`, committed
|
||||
`724af14`); the 19:23 take had real peakMix. Final user confirmation still pending.
|
||||
- **Audio sync offset DB value** — `Audio.SyncOffsetMs = 300` (the "set to 0" write
|
||||
in 5ba4d70 never actually landed). Fix for the ~2s ring backlog is now SHIPPED (local,
|
||||
uncommitted). Next: re-record the clap pattern, run the cross-correlation recipe → expect
|
||||
residual ≈ 400ms (300ms configured sync + ~100ms pipeline); then optionally zero the
|
||||
slider to hit ~100ms.
|
||||
- **Truncation with DYNAMIC scenes** — proven mechanism + probe in place; static
|
||||
scenes full-length. 19:23 take confirms dynamic scenes hold ~60fps (no truncation)
|
||||
but with 30ms stall cadence.
|
||||
- **Webcam MJPG missing**, **web capture ~10–14Hz**, **layer SortOrder not persisting**
|
||||
— queued, unchanged.
|
||||
- **Verify signed sync on device:** re-run the clap take with a NEGATIVE offset to confirm the
|
||||
advance direction end-to-end (the regression test proves the mixer; a take proves the file).
|
||||
- **Audio-silence verification** — fixed code (`724af14`) confirmed; creator heard real audio.
|
||||
- **Webcam MJPG missing / ~10–14Hz**, layer SortOrder, truncation-with-dynamic-scenes — queued.
|
||||
- **Sync control tutorial in user docs — REQUIRED before 1.0** (creator directive).
|
||||
|
||||
## Landmines
|
||||
|
||||
- testhost shares startup.log — filter by time.
|
||||
- `cmd.exe /c "taskkill /F /IM ytLive.exe"` (WSL form double-slashes mangle) before rebuilds.
|
||||
- Build/tests: Windows dotnet host (`/mnt/c/Program Files/dotnet/dotnet.exe`). 0 warnings.
|
||||
- ffmpeg/ffprobe: `/mnt/c/Program Files/Krita (x64)/bin/ffmpeg.exe` with Windows paths.
|
||||
- **`Audio.SyncOffsetMs` currently = 300 in the DB** (contradicts the "set to 0"
|
||||
line in 5ba4d70; the write likely never happened). Non-zero offsets still depend on
|
||||
the idempotent `Configure` guard to avoid the silence bug.
|
||||
- `MyMistakes.md` → Recipes registry now has the **audio/video sync measurement
|
||||
recipe** (claps + cross-correlation) — grep before re-deriving.
|
||||
- `cmd.exe /c "taskkill /F /IM ytLive.exe"` (WSL double-slashes mangle) before rebuilds.
|
||||
- Build/tests: **Windows dotnet host** (`/mnt/c/Program Files/dotnet/dotnet.exe`). 0 warnings.
|
||||
- ffmpeg/ffprobe: `/mnt/c/Program Files/Krita (x64)/bin/` with Windows paths.
|
||||
- `MyMistakes.md` has the **audio/video sync measurement recipe** (claps + cross-correlation)
|
||||
— grep it before re-deriving.
|
||||
- sqlite3 lives at `/home/gramps/android-sdk/platform-tools/sqlite3` (WSL) for the DB at
|
||||
`/mnt/c/Users/gramp/AppData/Roaming/ytLlive/ytLlive.db`.
|
||||
|
||||
## Next step
|
||||
|
||||
**Verify the shipped fix:** re-record with the same clap pattern and run the
|
||||
cross-correlation recipe from `MyMistakes.md` → expect offset ≈ 400ms (300ms configured
|
||||
sync + ~100ms pipeline) with a `SyncOffsetMs=300` DB value, or ~100ms with the slider
|
||||
zeroed. Then commit this session's work (StartLive clear + regression test + docs) and
|
||||
decide with the user whether the no-push ruling is lifted.
|
||||
Run the verify.sh gate (clean build, 0 warnings, full suite, scope check) on the dirty tree,
|
||||
then commit the signed-sync work unit (no push). Optionally a device clap take with a negative
|
||||
offset to validate the advance on the file.
|
||||
@@ -495,3 +495,25 @@ stamp BEFORE theorizing. (Also this session: a sentinel-byte test where the sour
|
||||
GENERATE the sentinel, and a fixed-point bilinear that shifted BOTH stages and silently drew nothing —
|
||||
see the rawvideo recipe.)
|
||||
|
||||
## Signed A/V sync: negative offset = eat the buffer head (OBS semantics), armed at go-live
|
||||
|
||||
(2026-09-14) The ring-backlog fix shrank A/V lag from ~2.2s to ~0.54s, and ~300ms of that was
|
||||
baked into the DB (`Audio.SyncOffsetMs=300`) via a POSITIVE-only control. For audio running BEHIND
|
||||
video you cannot push audio later — positive delay makes it worse. OBS's answer is a NEGATIVE offset
|
||||
that eats the buffer head: drop the first |N| ms of the written stream so every audio event lands
|
||||
|N| ms EARLIER relative to video. It only makes sense at the head of the stream, so it is armed once
|
||||
at `StartLive` (positive stays live-reactive via the delay line). Rule: record the signed semantics
|
||||
together — positive = delay (ahead), negative = eat-head advance (behind) — and lock the control
|
||||
while live (`IsEditMode`) since a mid-stream advance flip is meaningless. Test recipe:
|
||||
`StartLive_NegativeOffset_AdvancesAudio_ByDroppingTheStreamHead` (emit 6×0.9 into an 8-tick budget,
|
||||
long 0.2 bed, wire must show only 0.2).
|
||||
|
||||
## A pipe-read test that samples one tick is a timing flake by construction
|
||||
|
||||
(2026-09-14, re-discovered) `Mix_HonorsProviderGains_AndGameMute_KillsTheLoopback` fails
|
||||
sporadically in ISOLATION (3/3 on the clean tree) but passes in the full suite: it reads exactly one
|
||||
tick's worth after muting, and if the pipe still carries pre-mute buffered bytes the read spikes at
|
||||
0.4 instead of silence. Any caller of `ReadFullyAsync` on a live pipe must either close/drain the
|
||||
pipe first or assert on a multi-tick window. Do not blame a sync change for this — verify the grain
|
||||
against the clean tree in the same mode before touching the mixer.
|
||||
|
||||
|
||||
@@ -32,6 +32,7 @@ public sealed class AudioMixer : IDisposable
|
||||
private readonly TimeSpan _mixInterval;
|
||||
private readonly Func<int>? _syncOffsetMs;
|
||||
private readonly AudioSyncDelay _syncDelay;
|
||||
private int _advanceSamplesRemaining;
|
||||
private bool _started;
|
||||
|
||||
// Live path (TASK 9): per-source resampling → voice chain on the mic →
|
||||
@@ -168,6 +169,18 @@ public sealed class AudioMixer : IDisposable
|
||||
_micBuffer.Clear();
|
||||
_loopbackBuffer.Clear();
|
||||
|
||||
// Negative sync offset (audio runs BEHIND video): OBS's fix is to "eat
|
||||
// the head of the buffer" — discard the first |N| ms of the live stream
|
||||
// so every subsequent audio event lands |N| ms EARLIER in the recording.
|
||||
// Armed once at go-live (you can't eat the head mid-stream); positive
|
||||
// offsets stay live-reactive through the delay line in FillAndMix.
|
||||
var syncMs = _syncOffsetMs?.Invoke() ?? 0;
|
||||
_advanceSamplesRemaining = syncMs < 0
|
||||
? (int)Math.Round(OutputSampleRate / 1000.0 * -syncMs) * 2
|
||||
: 0;
|
||||
if (_advanceSamplesRemaining > 0)
|
||||
_log?.Invoke($"Audio live: advance armed ({-syncMs} ms ahead)");
|
||||
|
||||
var cts = new CancellationTokenSource();
|
||||
_liveCts = cts;
|
||||
_pipe = new NamedPipeAudioWriter();
|
||||
@@ -293,11 +306,25 @@ public sealed class AudioMixer : IDisposable
|
||||
try
|
||||
{
|
||||
var (_, micDrained, loopDrained) = FillAndMix(micChunk, loopbackChunk, mixBuffer);
|
||||
await pipe.WriteAsync(mixBuffer, cancellationToken).ConfigureAwait(false);
|
||||
|
||||
// Negative offset: skip the armed |N| ms of stream head. The pipe
|
||||
// writer tolerates a partial chunk (any float length is valid);
|
||||
// dropping the head re-anchors the audio stream against the video
|
||||
// stream, making each event land earlier — OBS's eat-the-head fix.
|
||||
var writeFrom = 0;
|
||||
if (_advanceSamplesRemaining > 0)
|
||||
{
|
||||
var drop = Math.Min(_advanceSamplesRemaining, mixBuffer.Length);
|
||||
_advanceSamplesRemaining -= drop;
|
||||
writeFrom = drop;
|
||||
}
|
||||
|
||||
if (writeFrom < mixBuffer.Length)
|
||||
await pipe.WriteAsync(mixBuffer.AsMemory(writeFrom), cancellationToken).ConfigureAwait(false);
|
||||
|
||||
_micDrainedTotal += micDrained;
|
||||
_loopDrainedTotal += loopDrained;
|
||||
for (var i = 0; i < mixBuffer.Length; i++)
|
||||
for (var i = writeFrom; i < mixBuffer.Length; i++)
|
||||
{
|
||||
var abs = Math.Abs(mixBuffer[i]);
|
||||
if (abs > _peakMix) _peakMix = abs;
|
||||
|
||||
@@ -10,8 +10,10 @@ namespace ytLive.Services.Audio;
|
||||
/// the same audio N ms later out — so it is trivially testable and carries no
|
||||
/// state outside its own ring.
|
||||
///
|
||||
/// Positive offsets only (0..500 ms). Advancing audio (<see cref="TimeSpan"/>
|
||||
/// would need a video-side delay and is out of scope for the audio layer.
|
||||
/// Positive offsets only (0..500 ms). Advancing audio (the audio runs BEHIND
|
||||
/// the video) is handled upstream in <see cref="AudioMixer"/>, not here: the
|
||||
/// mixer eats the first |N| ms of the live stream head (OBS's negative sync —
|
||||
/// see <see cref="AudioMixer.StartLive"/>), keeping this line a pure delay.
|
||||
/// Changing the delay flushes the line, so a live slider change clicks rather
|
||||
/// than smearing. Safe to call from any thread — the mixer owns it.
|
||||
/// </summary>
|
||||
|
||||
@@ -29,17 +29,19 @@ public partial class LayoutStore : IDisposable
|
||||
insert.ExecuteNonQuery();
|
||||
}
|
||||
|
||||
/// <summary>The global audio-sync delay in milliseconds (TASK 22), 0..500,
|
||||
/// default 0. Restored at startup so the mixer starts on the user's offset.</summary>
|
||||
/// <summary>The global audio-sync offset in milliseconds (TASK 22), −500..500,
|
||||
/// default 0: positive delays the mix (audio ahead of video), negative
|
||||
/// advances it (audio behind video). Restored at startup so the mixer starts
|
||||
/// on the user's offset.</summary>
|
||||
public int LoadAudioSyncOffsetMs()
|
||||
{
|
||||
var value = GetSetting("Audio.SyncOffsetMs");
|
||||
return int.TryParse(value, out var ms) ? Math.Clamp(ms, 0, 500) : 0;
|
||||
return int.TryParse(value, out var ms) ? Math.Clamp(ms, -500, 500) : 0;
|
||||
}
|
||||
|
||||
public void SaveAudioSyncOffsetMs(int ms)
|
||||
{
|
||||
UpsertSetting("Audio.SyncOffsetMs", Math.Clamp(ms, 0, 500).ToString());
|
||||
UpsertSetting("Audio.SyncOffsetMs", Math.Clamp(ms, -500, 500).ToString());
|
||||
}
|
||||
|
||||
/// <summary>The user's chosen record output folder (TASK 18), or null to use the
|
||||
|
||||
@@ -17,7 +17,27 @@
|
||||
### Design decisions
|
||||
|
||||
- **Global offset first** — one setting for all audio sources. Per-source is v1.1+.
|
||||
- **Signed offset (2026-09-14)** — WIDENED from positive-only 0..500 to **−500..+500**.
|
||||
|
||||
> The "positive-only / advance needs video-side delay / out of scope" line below is
|
||||
> RETIRED. Negative offsets now ADVANCE the audio by eating the head of the live stream
|
||||
> (OBS's negative-sync behavior): `AudioMixer.StartLive` arms `_advanceSamplesRemaining`
|
||||
> = |N| ms → samples, and `LiveLoopAsync` drops that many samples off the write head.
|
||||
> Positive keeps using the `AudioSyncDelay` line live-reactive. Slider relabelled
|
||||
> "AUDIO SYNC", `Min="-500"`, and **locked (`IsEnabled = IsEditMode`) while live or
|
||||
> recording** — a negative advance can only be armed at go-live, so it must not move
|
||||
> mid-session. Clamps: `Pre-viewModel` + `LayoutStore` ±500; `AudioSyncDelay` still
|
||||
> clamps negative→0 internally (pure delay line, unchanged).
|
||||
> **Provenance (2026-09-14): creator-directive** — asked for both directions after the
|
||||
> ring-backlog fix surfaced the residual (audio can run late too: capture cards, BT,
|
||||
> webcams). Regression test: `StartLive_NegativeOffset_AdvancesAudio_ByDroppingTheStreamHead`.
|
||||
- **Positive-only (delay audio)** — the physically-correct direction (audio runs ahead of the video). True "advance" needs a video-side delay and is PERMANENTLY OUT with per-source sync (TASKS.md → "Out of product" — v1.x phrasing retired 2026-09-01).
|
||||
- **Simple slider** — 0 to +500 ms, default 0. No numeric input needed.
|
||||
- **Visual feedback** — "sync OK" status dot shows when an offset is dialled in.
|
||||
|
||||
### ❗ REQUIRED before 1.0 (creator directive 2026-09-14)
|
||||
|
||||
A **detailed user-doc tutorial** on the audio-sync feature (what -500..+500 means, the
|
||||
clap-calibration recipe both directions, and that it locks while live). `docs/` currently
|
||||
holds only the README image — the tutorial is unstarted. Add it to the 1.0/gold-pass checklist.
|
||||
|
||||
|
||||
@@ -155,16 +155,17 @@ public partial class MainViewModel
|
||||
private set => SetProperty(ref _micSourceName, value);
|
||||
}
|
||||
|
||||
/// <summary>Global audio-sync delay in milliseconds (0..500, default 0),
|
||||
/// applied to the whole mix so audio lands on the video when it runs ahead
|
||||
/// — the OBS fix for lip-sync. Consumed live by the mixer's delay line via
|
||||
/// a Func seam; persisted with the layout.</summary>
|
||||
/// <summary>Global audio-sync offset in milliseconds (−500..500, default 0):
|
||||
/// POSITIVE delays the whole mix so audio lands on the video when it runs
|
||||
/// AHEAD (OBS's lip-sync fix); NEGATIVE advances it by dropping the first
|
||||
/// |N| ms of the live stream head when audio runs BEHIND the video.
|
||||
/// Consumed live by the mixer via a Func seam; persisted with the layout.</summary>
|
||||
public int AudioSyncOffsetMs
|
||||
{
|
||||
get => _audioSyncOffsetMs;
|
||||
set
|
||||
{
|
||||
var clamped = Math.Clamp(value, 0, 500);
|
||||
var clamped = Math.Clamp(value, -500, 500);
|
||||
if (SetProperty(ref _audioSyncOffsetMs, clamped))
|
||||
ScheduleSave();
|
||||
}
|
||||
|
||||
@@ -626,11 +626,15 @@ devices, no timers).
|
||||
pipe lifecycle.
|
||||
- **Audio sync offset (TASK 22):** after the limiter, `FillAndMix` routes the whole interleaved stereo
|
||||
mix through a **`AudioSyncDelay`** (`Services/Audio/AudioSyncDelay.cs`) — a pure delay line whose
|
||||
offset comes from a `Func<int> syncOffsetMs` seam (the VM's global `AudioSyncOffsetMs`, 0..500 ms,
|
||||
persisted as `Audio.SyncOffsetMs`). This is OBS's documented fix for audio running ahead of the video
|
||||
(delay it a few ms); positive-only, since advancing audio would need a video-side delay. A slider on
|
||||
the mic bar + a status dot (`IntToSyncBrushConverter`) surface it. Changing the delay flushes the line
|
||||
(a live change clicks rather than smears).
|
||||
offset comes from a `Func<int> syncOffsetMs` seam (the VM's global `AudioSyncOffsetMs`, **−500..+500
|
||||
ms**, persisted as `Audio.SyncOffsetMs`). This is OBS's documented fix for lip-sync: **positive**
|
||||
delays audio when it runs ahead of video; **negative** advances audio when it runs behind by eating
|
||||
the first `|N|` ms of the live stream head (see `AudioMixer.StartLive`; the delay line stays
|
||||
clamped 0..500 internally — negative bypasses it entirely). The advance is armed once at `StartLive`
|
||||
(eating the head mid-stream is impossible); positive is live-reactive (re-read per tick). A slider
|
||||
on the mic bar (`"AUDIO SYNC"`, locked while live/recording via `IsEditMode`) + a status dot
|
||||
(`IntToSyncBrushConverter`) surface it. Changing the delay flushes the line (a live change clicks
|
||||
rather than smears).
|
||||
- **TRAX — free background music (TASK 8):** `MusicPlayer` = NAudio `MediaFoundationReader`
|
||||
(mp3/wav/m4a) → `VolumeWaveProvider16` at the hardcoded **0.20** bed (no slider) → `WaveOutEvent` on
|
||||
the default device, **looping on any clean natural end** (`PlaybackStopped` with `e.Exception == null`
|
||||
|
||||
@@ -7,3 +7,10 @@ separate devices. If iTunes audio appears on the mic meter, it is likely acousti
|
||||
coupling — the mic picking up speaker/headphone output. Verify by muting the mic;
|
||||
if the meter still shows levels, check Windows Sound settings for stereo mix or
|
||||
"listen to this device" routing.
|
||||
|
||||
## Live recording ≠ "recording results" shown in Chat view (NOTE ONLY — no fix)
|
||||
User note (2026-09-14): the recording produced while live is completely different
|
||||
from the "recording results" surfaced in the Chat view. Do NOT attempt to fix or
|
||||
reconcile them pre-1.0 — this entry exists so the discrepancy is never rediscovered
|
||||
as a surprise. When the reconciliation work is eventually queued, start by mapping
|
||||
both recording paths side by side.
|
||||
|
||||
@@ -478,6 +478,54 @@ public class AudioPipelineTests
|
||||
Assert.True(floats.Max() < 0.3f, "stale pre-live backlog leaked onto the wire");
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task StartLive_NegativeOffset_AdvancesAudio_ByDroppingTheStreamHead()
|
||||
{
|
||||
// THE integration test for the signed-sync branch (2026-09-14): a
|
||||
// NEGATIVE offset must ADVANCE audio — OBS eats the buffer head so the
|
||||
// audio lands earlier on the video. StartLive arms a |N| ms sample
|
||||
// budget; the live loop skips that much off the write head. With a -40ms
|
||||
// offset at a 5ms mix interval (480 samples/tick at 48k), the budget is
|
||||
// 3840 samples = a full 8 ticks. The first 6 ticks carry fresh 0.9 audio,
|
||||
// which MUST be eaten; the long 0.2 bed behind it is what the wire shows
|
||||
// (0.2 on the wire, never 0.9). WITHOUT the advance, 0.9 leaks straight
|
||||
// through and the test fails.
|
||||
var pipeName = "ytllive_test_" + Guid.NewGuid().ToString("N");
|
||||
var mic = new FakeSource();
|
||||
var loopback = new FakeSource();
|
||||
using var mixer = new AudioMixer(
|
||||
mic, loopback,
|
||||
log: null,
|
||||
syncOffsetMs: () => -40,
|
||||
mixInterval: TimeSpan.FromMilliseconds(5));
|
||||
|
||||
using var client = new NamedPipeClientStream(".", pipeName, PipeDirection.In, PipeOptions.Asynchronous);
|
||||
var connected = client.ConnectAsync();
|
||||
mixer.StartLive(pipeName);
|
||||
|
||||
// 6 ticks of head 0.9 (all within the 8-tick advance budget) immediately
|
||||
// after StartLive, then a long 0.2 bed. Zero mic, unity gains, silent
|
||||
// ducker, quiet limiter — the loopback bed rides the wire ~1:1.
|
||||
var head = new float[480];
|
||||
Array.Fill(head, 0.9f);
|
||||
for (var i = 0; i < 6; i++)
|
||||
loopback.Emit(new AudioSample(head, 48000, 2));
|
||||
var bed = new float[480];
|
||||
Array.Fill(bed, 0.2f);
|
||||
for (var i = 0; i < 60; i++)
|
||||
loopback.Emit(new AudioSample(bed, 48000, 2));
|
||||
|
||||
await connected.WaitAsync(TimeSpan.FromSeconds(5));
|
||||
var bytes = await ReadFullyAsync(client, 240 * 2 * 4, TimeSpan.FromSeconds(5));
|
||||
mixer.StopLive();
|
||||
|
||||
var floats = new float[bytes.Length / 4];
|
||||
Buffer.BlockCopy(bytes, 0, floats, 0, bytes.Length);
|
||||
Assert.True(floats.Length > 0, "expected wired audio");
|
||||
Assert.True(floats.Max() < 0.3f, "the negative-offset head was NOT eaten (0.9 leaked)");
|
||||
Assert.True(floats.Any(f => MathF.Abs(f) > 0.05f), "the pipe carried only silence");
|
||||
}
|
||||
|
||||
private static async Task<byte[]> ReadFullyAsync(NamedPipeClientStream client, int count, TimeSpan timeout)
|
||||
{
|
||||
var ms = new MemoryStream();
|
||||
|
||||
@@ -6,8 +6,9 @@ namespace ytLive.Tests;
|
||||
/// <summary>
|
||||
/// TASK 22: the global audio-sync delay line. It is a pure fixed positive
|
||||
/// delay on the interleaved stereo mix — the OBS fix for audio running ahead
|
||||
/// of video. Positive-only: advancing audio needs a video-side delay and is
|
||||
/// out of the audio layer's scope.
|
||||
/// of video. Positive-only: the line clamps negatives to 0; the NEGATIVE
|
||||
/// (advance) side — audio running behind video, the OBS "eat the buffer head"
|
||||
/// case — is covered by <see cref="AudioPipelineTests.StartLive_NegativeOffset_AdvancesAudio_ByDroppingTheStreamHead"/>.
|
||||
/// </summary>
|
||||
public class AudioSyncDelayTests
|
||||
{
|
||||
|
||||
Reference in New Issue
Block a user