Third measurement, and it names the start-up defect the creator heard as "started off
wrong, then corrected" (ty-20261112-1222-748-0000-2.mp4):
wall clock (audio start -> audio stop) 12.614 s
video: 754 frames 12.588 s correct
audio: 585 packets 12.477 s 137 ms short
pipe connected 160 ms after the session opened
first 150 ms of the audio track digital silence (the mic delay line's head)
The rate is fixed — 12.5 s of show, no accumulating drift — but starting the clock at
connect made the audio stream begin 160 ms after the video stream, so every word sat 160 ms
late and the track came out that much shorter. The silence at the head is the 175 ms mic
delay doing its job, not a fault.
Now the clock runs from session open. Time that passes before ffmpeg connects stays owed,
and it is paid the moment the pipe connects — out of the capture ring, so it is the audio
that was missed rather than padding. Debt larger than one 200 ms chunk continues to be paid
over the following ticks.
674 pass, 0 warnings.
Second measurement of the same recording family (ty-20261112-1133-701-0000-2.mp4):
wall clock (audio start -> audio stop) 6.477 s
video: 386 frames 6.455 s correct
audio: 294 packets 6.252 s 226 ms SHORT
Same magnitude as the original bug, opposite sign — the clock fixed the tick-rate error and
exposed the next one underneath. Audio started 49 ms before the pump, ffmpeg connected ~80 ms
after, and every chunk written in that window was dropped (dropped writes 27). The loop still
counted those frames as produced, so the track lost exactly that much time.
Now: the clock does not start until the pipe is connected, and AudioClock.Advance counts a
frame only when its write reached the pipe. A dropped write stays owed and is paid off on the
next tick, so a start-up gap costs content (silence, which there was anyway) instead of
length.
Two measurements, two different signs, one lesson: the audio track's LENGTH has to be the
clock's, and anything that removes samples without removing time — a fast tick, a dropped
write — shows up as drift no static offset can fix.
674 tests (one notification-timing flake under full-suite load; passes alone), 0 warnings.
Measured on the creator's recording (ty-20261112-1118-423-0000-2.mp4), which drifts
out of sync over ten seconds:
wall clock (pump start -> audio stop) 9.875 s
video: 594 frames, uniform 1/60 9.888 s correct
audio: 485,376 samples @ 48 kHz 10.112 s +2.1%, 224 ms long
So it was not VFR and not wall-clock PTS: the video is CFR and matches the clock (the
only PTS anomaly in the file is one 38 ms gap from two lost frames). The audio track is
what was too long, and a static offset cannot fix a rate error — which is why the opening
looked aligned (the 175 ms mic delay cancels the camera's constant pipeline latency) and
the end did not.
Cause: LiveLoopAsync wrote a fixed 480 frames per iteration and trusted that each
iteration was 10 ms. Under a raised timer resolution they land near 9.8 ms — 102 ticks
per second where 100 are expected, i.e. 102% of an audio second per wall second. FFmpeg
stamps audio by sample count, so every second of show pushed the sound ~2% later against
a correct picture. Video never had the bug because the pump paces off a stopwatch.
Fix: Services/Audio/AudioClock asks the clock how many frames are owed
(expected - produced, clamped to one 200 ms chunk) and the loop produces exactly that,
padding on underrun. A stalled loop pays its debt over the following ticks instead of
dumping one huge write. FillAndMix takes a frame count, and the master limiter/meter take
spans, so a partial chunk is processed without touching stale buffer tail.
674 tests (one log-rotation flake under full-suite load while the app itself is running
and writing startup.log; passes alone), 0 warnings. Ear check pending.
The webcam's frames reach the compositor later than the mic's samples reach the audio
pipe, so the voice runs ahead of the lips. The existing lip-sync offset could not fix
that: it delays the WHOLE mix, which drags desktop and alert audio — neither of which
has a camera to follow — out of sync in the process.
So the delay line learned channels (mono for a source, stereo for the mix, offset always
in milliseconds so the frame count is what stays constant), and FillAndMix runs a second
line over micChunk before the mix. Slider-side: presets off / 150 / 200 in the LIP-SYNC
bay under the whole-mix control, persisted as Audio.MicSyncOffsetMs.
Default 175 ms is the creator's own number for a C920 pipeline (they asked for
+150..+200). It is a setting, not a constant, because the right figure belongs to the
camera and the machine — a faster camera wants less, no camera wants none.
Both delay lines are reconfigured every tick, so the 2026-09-12 trap still applies and is
documented at both sites: a Configure that reallocates discards the buffered voice every
10 ms.
673 pass, 0 warnings. Ear check pending: this is an audio change.
A video file was picture-only: the decode pipe passed `-an`, so a starting-soon
or BRB clip with music played silent everywhere. Adds sound to every clip and
collapses the two decoders into one.
- `Services/VideoClipPipe.cs`: the shared pipe behind `MediaVideoSource` (TASK 21)
and `AlertClipDecoder` (TASK 47) — video (`rawvideo bgra`, scaled) + audio
(`f32le -ac 2 -ar 48000`, 50 ms chunks, byte-carry across reads), both paced to
real time, loop restarts both so picture and sound stay together. Alert
behaviour is unchanged.
- `MediaVideoSourceManager` applies the layer's Level/Mute per session and raises
`AudioAvailable`; `MainViewModel` feeds it to the new
`AudioMixer.EnqueueDesktopMediaAudio`, which joins the DESKTOP/GAME ring so the
desktop fader and the mic auto-ducker apply exactly as they do for game audio
and TRAX (creator ruling 2026-11-05: audio joins the group).
- Per-layer `MediaMuted` + existing `MediaVolume` surfaced as Mute/Level in the
properties panel; `MediaMuted` persisted (ALTER TABLE guard + SELECT/INSERT).
- Level/mute are per SESSION because one decoder is shared per file path — the
last layer touched wins; one decoder per layer would play the file twice.
Test: `MediaSourceAudioTests` — the pipe emits 50 ms chunks at 48k stereo, the
manager scales by level and drops when muted, and a media layer's sound moves the
desktop meter. Alert/decode suites unchanged and green.
Docs: task-21 closed on item 10, the "silent soundtrack" facts corrected in
frame-pipeline/output/overlays, HANDOFF rewritten. verify.sh green (529 passed;
the two documented pre-existing failures only).
Add independent AppLog message types and explicit Class 1/2 severity. Wrap and color log rows, pin Class 1 above Class 2, and let acknowledgement return entries to the current-launch timeline without changing file format or rotation.
Ref: https://logtape.org/manual/levels
Unit 9, the one that matters most for this creator.
MASTER STRIP (top, nearest the preview):
- one ladder fed by the MEASURED mix — `AudioMixer` now measures the buffer the encoder receives
(`MasterLevel`, `MasterImbalanceDb`), so the master cannot show a source instead of the output
- `-12.4 dB` readout, and ONE L/R flag that appears above a 6 dB gap instead of a second ladder
- bitrate / FPS / dropped / duration from the existing health poll
MONO OUTPUT:
- `AudioFold.ToMono` averages each stereo pair into both channels, applied after the limiter and
the sync delay, so the stream AND the recording get what was folded
- `MonoOutput` is persisted (`Audio.MonoOutput`) and restored at startup: an accessibility switch
that resets each session is an accessibility failure
- one click in the strip, remembered forever
Guard: AudioFoldTests (hard-left reaches the right at half amplitude, hard-right — the inaudible
case — reaches the left, centred material keeps its level, mono buffers untouched) and
MasterStripTests (the strip binds the measured mix + the switch, the store round-trips it, and the
fold is the tested helper rather than new arithmetic). 33 audio tests green.
Unit 6. Mirrors the mic bay for the desktop/game bed, plus the badge that matters to a creator who
cannot hear the duck: it shows while the ducker is actually attenuating, and it prints the gain the
mixer applied (−12.0 DUCK by design).
- `AudioMixer.DuckGain` exposes `AutoDucker.CurrentGain` instead of leaving it internal.
- VM: `GameVolumeDb` / `GameVolumeDbLabel` (fader face), `GameMeterFraction` (measured level),
`IsGameDucking` + `GameDuckLabel` refreshed on the level tick — the same moment the ducker runs,
so the badge cannot lag behind the mix.
- The bay: header chip + source, ladder, mute dot (uses the meter band, so mute reads at a glance),
dB readout, fader.
Guard: GameBayTests — the dock binds all four, `AutoDucker.DuckGain` is −12.04 dB and the label is
produced through the same dB scale as the faders, and the mixer property exists so the badge cannot
become a hardcoded number.
TASK 43's alert box grows a real video celebration. Per-alert IAlertClipDecoder
(ffmpeg bgra + f32le pipes, real-time paced, disposed at drain) plays the shipped
Assets/alert-default.mp4 (stamped into the Asset table at startup) unless the
creator picks their own file — path reference only, never stored in the DB; the
six AlertRenderer animations stay the fallback. ~0.3s fade rides the alpha
envelope on straight-source copies (EOF freeze-frames then fades out); audio
forwards to a new AudioMixer alert ring (8s, 48k stereo) drained at unity — no
duck, creator ruling — scaled by volume × fade. An auto-composed marquee ticker
('Funder — Super Chat · $10.00', 140px/s) scrolls top-of-frame via a
FramePump._alertTicker seam through Render/CompositeLayers, mixed into the cache
signature (dynamic overlay, never baked). New Stream Alerts section in LeftPanel.
Derivative-work references (how OBS/Streamlabs alert boxes do per-alert video):
- https://support.streamlabs.com/hc/en-us/articles/217741147-Setting-Up-Your-Streamlabs-Alerts (custom image/video per alert type + variations)
- https://obsproject.com/kb/stream-tutorial-2-alerts (alert overlay as an on-screen zone)
- https://streamlabs.com/content-hub/widgets/alert-box (per-event alert playback)
Good Dog: AlertLayerVideoTests drives a fake IAlertClipDecoder through the whole
lifecycle in one pass (custom path wins, decoder spawns/disposes, fade envelope
0→127→255, audio volume×fade, ticker scrolls, EOF fade-drain to idle). It caught
the clip branch of Advance not clearing _current before AdvanceToNext — the layer
stayed IsPlaying after drain (MyMistakes post-mortem).
Full vstest 319/319; clean build 0 warnings; scope check green.
Positive offsets still delay the whole mix via the delay line (lip-sync fix);
negative offsets now ARM once at StartLive and drop |N| ms off the pipe's write
head so audio events land earlier when audio runs BEHIND video. Slider relabeled
AUDIO SYNC, Min −500, locked while live/recording (IsEditMode). LayoutStore and
VM clamp to −500..500.
OBS reference for eat-the-head negative sync: https://obsproject.com/kb/obs-studio/buffering-time (negative sync values pull audio earlier by discarding buffered player audio).
Test: StartLive_NegativeOffset_AdvancesAudio_ByDroppingTheStreamHead (6x0.9 head
must be eaten before 0.2 bed reaches the wire).
AudioMixer.Start() begins mic/loopback capture at app startup to feed the
level meters, so both 2s ring buffers fill with pre-live audio. StartLive()
drained from the oldest tail sample, putting every recorded event ~2.2s late
in the audio track (confirmed by clap analysis + cross-correlation on two
takes: +2.11 to +2.22s).
Fix: AudioMixer.StartLive() now runs _micBuffer.Clear() + _loopbackBuffer.Clear()
immediately after the pipe starts, before the drain task runs. Recording now
begins at go-live; the <=10ms in-flight chunk evicted by Clear() is imperceptible.
Regression test: StartLive_DiscardsPreLiveBacklog_SoFirstAudioIsCurrent
saturates the loopback ring with stale 0.8 pre-live audio, then asserts the
wire carries fresh post-live 0.2 (max < 0.3).
Reference (external, per derivative-work rule): OBS 'Audio mixer' keeps its
buffers fed continuously and syncs the stream start timestamp at record time
rather than replaying pre-live capture; a go-live flush of the capture buffer
is the accepted pattern for live tools restarting a stream.
Docs: HANDOFF.md (fix shipped), MyMistakes.md (A/V sync measurement recipe).
The recorded audio is full-length silent AAC (−91dB, 1124 frames/23.95s) —
the named pipe carried ~9.1MB of zeros the entire take. The mixer's loop
ran, the pipe connected, ffmpeg read it, and the file is full-length
silence. Sources started fine ('Audio: using system default mic...' logged),
no failure callbacks fired, and the existing integration test
(Mix_WithFiltersDuckAndGain_Lands_On_AudioPipe) proves the loop→pipe path
carries real audio when fed — the fault is capture-side.
Added permanent per-5s live-loop telemetry to startup.log so the next take
names the exact stage without new code:
Audio live: pipe connected= dropped writes= micLevel= loopLevel= drained
N/N samples peakMix N (per 5s)
- AudioMixer.FillAndMix now returns (MicRms, MicDrained, LoopDrained) for
the accumulation; StartLive/StopLive log start/stop lines.
- NamedPipeAudioWriter.DroppedWrites: nonzero = audio dropped before ffmpeg
connected (names 'pipe never connected' stage).
Likely root cause: WASAPI loopback captures the default render endpoint —
if audio plays on a non-default device the recording is silently silent.
The fix requires device enumeration + selection (slice 13 candidate).
Suite 290/291 — same sole pre-existing compositor pixel failure.
The creator-declared known failure (Mix_HonorsProviderGains) and AudioPipelineTests'
standalone hang shared ONE root cause, born in TASK 22: AudioMixer.FillAndMix
dereferenced _delayedMix (declared float[]? , never assigned) as delayed.Length —
every live-mix tick NRE'd before the pipe write, so NO audio ever reached the wire
(tests starved -> hung/fail; app -> swallowed catch logged only ex.Message, 10ms
flood). Now: ctor-allocated + null-check, catch logs WITH stack via AppLog and is
throttled 5s, and a cancelled token breaks out before logging.
Bonus contract fix: AudioGainProvider.LoopbackGain returned unity on a disproven
premise ('loopback scales with endpoint volume'). The creator's tonight observation —
volume at 20%, meter pegged — proves the WASAPI tap is pre-endpoint-volume, so the
mixer must multiply by GameAudioVolume for stream honesty (this is what ai.md always
specified; the 'failing' test encoded the same and was RIGHT).
AudioPipelineTests: 25/25 in 26ms, standalone, no hang. Known-failure count: ZERO.
ai.md tests paragraph rewritten (no more suite-total claims, both ex-'knowns'
explained); TASK 22 regression recorded; MyMistakes: the known-failure-label rules.
Creator ruling 2026-09-01 (stuck REC pill after the failed first recording):
- StopStream clears RecordPillOn + OnAirPillOn — intent resets with reality.
- OnFramePumpFailed: dispatcher-marshalled FULL rollback via StopStream (was:
toast + Error-status only when live — record-only sessions zombied with
IsRecording=true over a dead encoder; the 20:34 attempt proved it). Toast copy
says 'Recording stopped' vs 'stream pipeline stopped'.
- Same zombie class in the three go-live prep failure branches: StreamStatus.Error
limbo replaced by StopStream() (audio loop + recording + pills unwind; a created
broadcast still gets the close-out).
- AudioMixer.StopLive made explicitly idempotent (Stop() twice, rollback paths that
never reached StartLive).
- Seams: OnFramePumpFailed + IsRecording setter internal (InternalsVisibleTo; test
pattern mirrors LayoutPathOverride).
ONE integration test: SessionTeardownTests (real window + temp DB: pump death with
lit pill -> no zombie, no End button, Offline). AudioPipelineTests confirmed to hang
STANDALONE (pre-existing, the declared-known audio class) — ai.md test-count
paragraph corrected to stop claiming a suite total that cannot currently be measured.
- Elements scrollbar no longer deselects element on click (check ScrollViewer, not StackPanel)
- Web source: URI + check/revert buttons merged into single DockPanel row, consistent order (✓ rightmost)
- Countdown timer moved from Elements panel to inline in source list row, range clamped to 1-30 min
- Volume slider click-to-position restored: e.Handled prevents IncreaseLarge from overriding position, Thumb drag still works
- Desktop audio meter: asymmetric smoothing (attack 0.2, decay 0.08) for faster response to silence
- 207 tests passing, 0 warnings
gramps