Third measurement, and it names the start-up defect the creator heard as "started off
wrong, then corrected" (ty-20261112-1222-748-0000-2.mp4):
wall clock (audio start -> audio stop) 12.614 s
video: 754 frames 12.588 s correct
audio: 585 packets 12.477 s 137 ms short
pipe connected 160 ms after the session opened
first 150 ms of the audio track digital silence (the mic delay line's head)
The rate is fixed — 12.5 s of show, no accumulating drift — but starting the clock at
connect made the audio stream begin 160 ms after the video stream, so every word sat 160 ms
late and the track came out that much shorter. The silence at the head is the 175 ms mic
delay doing its job, not a fault.
Now the clock runs from session open. Time that passes before ffmpeg connects stays owed,
and it is paid the moment the pipe connects — out of the capture ring, so it is the audio
that was missed rather than padding. Debt larger than one 200 ms chunk continues to be paid
over the following ticks.
674 pass, 0 warnings.
Second measurement of the same recording family (ty-20261112-1133-701-0000-2.mp4):
wall clock (audio start -> audio stop) 6.477 s
video: 386 frames 6.455 s correct
audio: 294 packets 6.252 s 226 ms SHORT
Same magnitude as the original bug, opposite sign — the clock fixed the tick-rate error and
exposed the next one underneath. Audio started 49 ms before the pump, ffmpeg connected ~80 ms
after, and every chunk written in that window was dropped (dropped writes 27). The loop still
counted those frames as produced, so the track lost exactly that much time.
Now: the clock does not start until the pipe is connected, and AudioClock.Advance counts a
frame only when its write reached the pipe. A dropped write stays owed and is paid off on the
next tick, so a start-up gap costs content (silence, which there was anyway) instead of
length.
Two measurements, two different signs, one lesson: the audio track's LENGTH has to be the
clock's, and anything that removes samples without removing time — a fast tick, a dropped
write — shows up as drift no static offset can fix.
674 tests (one notification-timing flake under full-suite load; passes alone), 0 warnings.
Measured on the creator's recording (ty-20261112-1118-423-0000-2.mp4), which drifts
out of sync over ten seconds:
wall clock (pump start -> audio stop) 9.875 s
video: 594 frames, uniform 1/60 9.888 s correct
audio: 485,376 samples @ 48 kHz 10.112 s +2.1%, 224 ms long
So it was not VFR and not wall-clock PTS: the video is CFR and matches the clock (the
only PTS anomaly in the file is one 38 ms gap from two lost frames). The audio track is
what was too long, and a static offset cannot fix a rate error — which is why the opening
looked aligned (the 175 ms mic delay cancels the camera's constant pipeline latency) and
the end did not.
Cause: LiveLoopAsync wrote a fixed 480 frames per iteration and trusted that each
iteration was 10 ms. Under a raised timer resolution they land near 9.8 ms — 102 ticks
per second where 100 are expected, i.e. 102% of an audio second per wall second. FFmpeg
stamps audio by sample count, so every second of show pushed the sound ~2% later against
a correct picture. Video never had the bug because the pump paces off a stopwatch.
Fix: Services/Audio/AudioClock asks the clock how many frames are owed
(expected - produced, clamped to one 200 ms chunk) and the loop produces exactly that,
padding on underrun. A stalled loop pays its debt over the following ticks instead of
dumping one huge write. FillAndMix takes a frame count, and the master limiter/meter take
spans, so a partial chunk is processed without touching stale buffer tail.
674 tests (one log-rotation flake under full-suite load while the app itself is running
and writing startup.log; passes alone), 0 warnings. Ear check pending.
The webcam's frames reach the compositor later than the mic's samples reach the audio
pipe, so the voice runs ahead of the lips. The existing lip-sync offset could not fix
that: it delays the WHOLE mix, which drags desktop and alert audio — neither of which
has a camera to follow — out of sync in the process.
So the delay line learned channels (mono for a source, stereo for the mix, offset always
in milliseconds so the frame count is what stays constant), and FillAndMix runs a second
line over micChunk before the mix. Slider-side: presets off / 150 / 200 in the LIP-SYNC
bay under the whole-mix control, persisted as Audio.MicSyncOffsetMs.
Default 175 ms is the creator's own number for a C920 pipeline (they asked for
+150..+200). It is a setting, not a constant, because the right figure belongs to the
camera and the machine — a faster camera wants less, no camera wants none.
Both delay lines are reconfigured every tick, so the 2026-09-12 trap still applies and is
documented at both sites: a Configure that reallocates discards the buffered voice every
10 ms.
673 pass, 0 warnings. Ear check pending: this is an audio change.
A video file was picture-only: the decode pipe passed `-an`, so a starting-soon
or BRB clip with music played silent everywhere. Adds sound to every clip and
collapses the two decoders into one.
- `Services/VideoClipPipe.cs`: the shared pipe behind `MediaVideoSource` (TASK 21)
and `AlertClipDecoder` (TASK 47) — video (`rawvideo bgra`, scaled) + audio
(`f32le -ac 2 -ar 48000`, 50 ms chunks, byte-carry across reads), both paced to
real time, loop restarts both so picture and sound stay together. Alert
behaviour is unchanged.
- `MediaVideoSourceManager` applies the layer's Level/Mute per session and raises
`AudioAvailable`; `MainViewModel` feeds it to the new
`AudioMixer.EnqueueDesktopMediaAudio`, which joins the DESKTOP/GAME ring so the
desktop fader and the mic auto-ducker apply exactly as they do for game audio
and TRAX (creator ruling 2026-11-05: audio joins the group).
- Per-layer `MediaMuted` + existing `MediaVolume` surfaced as Mute/Level in the
properties panel; `MediaMuted` persisted (ALTER TABLE guard + SELECT/INSERT).
- Level/mute are per SESSION because one decoder is shared per file path — the
last layer touched wins; one decoder per layer would play the file twice.
Test: `MediaSourceAudioTests` — the pipe emits 50 ms chunks at 48k stereo, the
manager scales by level and drops when muted, and a media layer's sound moves the
desktop meter. Alert/decode suites unchanged and green.
Docs: task-21 closed on item 10, the "silent soundtrack" facts corrected in
frame-pipeline/output/overlays, HANDOFF rewritten. verify.sh green (529 passed;
the two documented pre-existing failures only).
Add independent AppLog message types and explicit Class 1/2 severity. Wrap and color log rows, pin Class 1 above Class 2, and let acknowledgement return entries to the current-launch timeline without changing file format or rotation.
Ref: https://logtape.org/manual/levels
Creator request: a sound-byte browse + label under Built-in video, with an option to loop the
sound while the clip plays. The video itself is always one pass to completion.
- Source gains AlertAudioPath and AlertAudioLoop, persisted (column + migration + INSERT +
SELECT + reader, the whole chain — that chain is where the blur crashed once).
- The sound byte plays through MediaPlayer on the default output device; MediaEnded re-seeks to
zero while looping, and StopClip kills it so a tail cannot bleed into the next alert.
- The clip's OWN soundtrack moved off the mixer and onto the device as well
(Services/Audio/AlertAudioOutput: one WaveOutEvent + buffered float provider).
Why the move: the creator asked that alert audio — the clip's track or the added sound byte —
appear in the desktop sound bar. Only audio that reaches the default device is picked up by the
WASAPI loopback that feeds both the meter and the stream. The old EnqueueAlertAudio route put the
sound in the stream but left the meter flat and the room silent.
Tests: the sound byte and its loop flag round-trip through SQLite, and a fresh alert layer starts
with no sound byte and no loop. Release 0 warnings.
Reported: pegging the TRAX fader changes nothing audible, the ladder never reaches red, and the
ladder does not scale with the fader.
Cause (two defects in the same setters):
- UserGain wrote the OUTPUT DEVICE's volume. NAudio's WaveOut ignores that, so the fader was inert.
- LocalGain (the desktop-mute follow) recomputed the sample-chain volume WITHOUT the user gain, so
every duck/mute silently discarded whatever the creator had set.
Fix: one `EffectiveVolume(user, local)` = MusicVolume × user × local, applied to the sample chain by
both setters. The meter sits AFTER that stage, so the ladder now scales with the fader and can reach
the peak band on a loud track — exactly what pegging should look like. Max user gain is +6 dB, so
"pegged" means the file at unity, not a magic boost.
Guard: MusicLevelTests pins the arithmetic (default half, pegged double, mute multiplies to zero),
that the user gain survives the mute follow, and that nothing may drive the output device's volume
again — the no-op that made this invisible.
Reported: the TRAX bar moves, but its sampling looks far slower than the mic meter.
Cause: the TRAX number was a raw instantaneous peak per metering block — a stutter sampled by the UI
timer, with no smoothing. The mic looks fluid because it is smoothed at audio-callback rate.
Fix: the player now smooths its measured level with the SAME curve the mic uses
(MeterScale.NextLevel: fast attack, gentle decay), and the dock's meter clock runs at ~30 Hz instead
of 20. Two ladders that move alike, because they are the same ballistics.
Guard: MeterScaleTests pins the attack/decay behaviour alongside peak-hold.
Creator: "why is there no meter for when trax is playing?"
Routing answer: TRAX rides the desktop loopback, so it already moved the GAME ladder — there was no
separate TRAX signal in the mix. But that ladder cannot answer the question a deaf creator actually
has: "is the music producing sound at all?" The play glyph cannot either — an open file with a dead
decoder still reports Playing.
So the player now measures its own output: \`MusicPlayer\` wraps the volume stage in an NAudio
\`MeteringSampleProvider\` and exposes \`Level\`; \`ITraxPlayer.Level\` keeps the fake honest. The VM maps
it through the same display scale (\`TraxMeterFraction\`), zeroed when not playing, and the TRAX bay
carries a 24-segment ladder above its fader.
This reverses my earlier "no ladder, nothing honest to show" — the honest thing arrived with a real
measurement. TraxBayTests flipped accordingly (it now asserts the ladder AND its source).
Unit 9, the one that matters most for this creator.
MASTER STRIP (top, nearest the preview):
- one ladder fed by the MEASURED mix — `AudioMixer` now measures the buffer the encoder receives
(`MasterLevel`, `MasterImbalanceDb`), so the master cannot show a source instead of the output
- `-12.4 dB` readout, and ONE L/R flag that appears above a 6 dB gap instead of a second ladder
- bitrate / FPS / dropped / duration from the existing health poll
MONO OUTPUT:
- `AudioFold.ToMono` averages each stereo pair into both channels, applied after the limiter and
the sync delay, so the stream AND the recording get what was folded
- `MonoOutput` is persisted (`Audio.MonoOutput`) and restored at startup: an accessibility switch
that resets each session is an accessibility failure
- one click in the strip, remembered forever
Guard: AudioFoldTests (hard-left reaches the right at half amplitude, hard-right — the inaudible
case — reaches the left, centred material keeps its level, mono buffers untouched) and
MasterStripTests (the strip binds the measured mix + the switch, the store round-trips it, and the
fold is the tested helper rather than new arithmetic). 33 audio tests green.
Unit 7. The bay shows the track name (left-click toggles playback, right-click opens the picker —
the label stays the control), a play/pause button, the duck badge, and a dB level.
No ladder here, deliberately: nothing measures the music bed, and a decorative ladder would break
the measured-not-painted rule. A test asserts the bay has no SegmentMeter.
The level is real: `ITraxPlayer.UserGain` is a new creator gain multiplied onto the fixed design
volume, separate from `LocalGain` (which follows the desktop mute). The VM's `TraxVolumeDb` is the
same dB scale as the other faders, so unity means the bed's design level.
Unit 6. Mirrors the mic bay for the desktop/game bed, plus the badge that matters to a creator who
cannot hear the duck: it shows while the ducker is actually attenuating, and it prints the gain the
mixer applied (−12.0 DUCK by design).
- `AudioMixer.DuckGain` exposes `AutoDucker.CurrentGain` instead of leaving it internal.
- VM: `GameVolumeDb` / `GameVolumeDbLabel` (fader face), `GameMeterFraction` (measured level),
`IsGameDucking` + `GameDuckLabel` refreshed on the level tick — the same moment the ducker runs,
so the badge cannot lag behind the mix.
- The bay: header chip + source, ladder, mute dot (uses the meter band, so mute reads at a glance),
dB readout, fader.
Guard: GameBayTests — the dock binds all four, `AutoDucker.DuckGain` is −12.04 dB and the label is
produced through the same dB scale as the faders, and the mixer property exists so the badge cannot
become a hardcoded number.
`AudioLevelMeter` measured one number for everything. It now also keeps smoothed per-channel
levels with the same attack/decay, plus `ImbalanceDb`. Mono samples report both sides equal (a mono
source is centred), so the two never disagree about whether audio exists.
The dock shows ONE ladder per source — two ladders per bay was visual noise for a rare failure —
but this creator is profoundly deaf on the right, so the one case that matters is "the sound is
panned away from me". That is now measurable and will drive a flag, not a second ladder.
Semantics that mattered: a silent side is compared at the −60 dBFS floor, so a fully one-sided
signal reads ≈54 dB rather than "nothing to compare". My first version returned 0 for exactly the
case the feature exists for; the test caught it.
Guard: StereoMeterTests — hard left, hard right (the inaudible case), balanced stereo, mono
centred, and silence-is-not-an-imbalance. 31 audio tests green.
TASK 43's alert box grows a real video celebration. Per-alert IAlertClipDecoder
(ffmpeg bgra + f32le pipes, real-time paced, disposed at drain) plays the shipped
Assets/alert-default.mp4 (stamped into the Asset table at startup) unless the
creator picks their own file — path reference only, never stored in the DB; the
six AlertRenderer animations stay the fallback. ~0.3s fade rides the alpha
envelope on straight-source copies (EOF freeze-frames then fades out); audio
forwards to a new AudioMixer alert ring (8s, 48k stereo) drained at unity — no
duck, creator ruling — scaled by volume × fade. An auto-composed marquee ticker
('Funder — Super Chat · $10.00', 140px/s) scrolls top-of-frame via a
FramePump._alertTicker seam through Render/CompositeLayers, mixed into the cache
signature (dynamic overlay, never baked). New Stream Alerts section in LeftPanel.
Derivative-work references (how OBS/Streamlabs alert boxes do per-alert video):
- https://support.streamlabs.com/hc/en-us/articles/217741147-Setting-Up-Your-Streamlabs-Alerts (custom image/video per alert type + variations)
- https://obsproject.com/kb/stream-tutorial-2-alerts (alert overlay as an on-screen zone)
- https://streamlabs.com/content-hub/widgets/alert-box (per-event alert playback)
Good Dog: AlertLayerVideoTests drives a fake IAlertClipDecoder through the whole
lifecycle in one pass (custom path wins, decoder spawns/disposes, fade envelope
0→127→255, audio volume×fade, ticker scrolls, EOF fade-drain to idle). It caught
the clip branch of Advance not clearing _current before AdvanceToNext — the layer
stayed IsPlaying after drain (MyMistakes post-mortem).
Full vstest 319/319; clean build 0 warnings; scope check green.
Positive offsets still delay the whole mix via the delay line (lip-sync fix);
negative offsets now ARM once at StartLive and drop |N| ms off the pipe's write
head so audio events land earlier when audio runs BEHIND video. Slider relabeled
AUDIO SYNC, Min −500, locked while live/recording (IsEditMode). LayoutStore and
VM clamp to −500..500.
OBS reference for eat-the-head negative sync: https://obsproject.com/kb/obs-studio/buffering-time (negative sync values pull audio earlier by discarding buffered player audio).
Test: StartLive_NegativeOffset_AdvancesAudio_ByDroppingTheStreamHead (6x0.9 head
must be eaten before 0.2 bed reaches the wire).
AudioMixer.Start() begins mic/loopback capture at app startup to feed the
level meters, so both 2s ring buffers fill with pre-live audio. StartLive()
drained from the oldest tail sample, putting every recorded event ~2.2s late
in the audio track (confirmed by clap analysis + cross-correlation on two
takes: +2.11 to +2.22s).
Fix: AudioMixer.StartLive() now runs _micBuffer.Clear() + _loopbackBuffer.Clear()
immediately after the pipe starts, before the drain task runs. Recording now
begins at go-live; the <=10ms in-flight chunk evicted by Clear() is imperceptible.
Regression test: StartLive_DiscardsPreLiveBacklog_SoFirstAudioIsCurrent
saturates the loopback ring with stale 0.8 pre-live audio, then asserts the
wire carries fresh post-live 0.2 (max < 0.3).
Reference (external, per derivative-work rule): OBS 'Audio mixer' keeps its
buffers fed continuously and syncs the stream start timestamp at record time
rather than replaying pre-live capture; a go-live flush of the capture buffer
is the accepted pattern for live tools restarting a stream.
Docs: HANDOFF.md (fix shipped), MyMistakes.md (A/V sync measurement recipe).
- AudioSyncDelay.Configure reallocated/zeroed its buffer every ~10ms tick
(AudioMixer re-reads the UI setting each mix), so any non-zero sync offset
erased the just-written audio -> total silence. Now early-returns when the
delay samples are unchanged. Regression test proven both ways.
- SceneCompositor.BlitContentRaw defaulted cbW/cbH=0 when CropBounds is null
(regression from ed9d7c1) -> webcam blit to an empty rect = gray block.
Default to src.Width/Height. Regression test proven both ways.
- Truncated recordings: rawvideo mux stamps frames at declared 60fps by
arrival; a scene whose first layer is dynamic (hidden elements still count)
kills the bake cache -> full render ~35ms -> ~27fps submitted -> halved
file length. Static scenes bake once (246ms cold, then <1ms) -> 60fps,
full-length (probe + 13:53 take, 301/300 per 5s, 12.46s file from 12.3s
wall). FramePump.ProbeRender names the hot render path on slow frames.
The recorded audio is full-length silent AAC (−91dB, 1124 frames/23.95s) —
the named pipe carried ~9.1MB of zeros the entire take. The mixer's loop
ran, the pipe connected, ffmpeg read it, and the file is full-length
silence. Sources started fine ('Audio: using system default mic...' logged),
no failure callbacks fired, and the existing integration test
(Mix_WithFiltersDuckAndGain_Lands_On_AudioPipe) proves the loop→pipe path
carries real audio when fed — the fault is capture-side.
Added permanent per-5s live-loop telemetry to startup.log so the next take
names the exact stage without new code:
Audio live: pipe connected= dropped writes= micLevel= loopLevel= drained
N/N samples peakMix N (per 5s)
- AudioMixer.FillAndMix now returns (MicRms, MicDrained, LoopDrained) for
the accumulation; StartLive/StopLive log start/stop lines.
- NamedPipeAudioWriter.DroppedWrites: nonzero = audio dropped before ffmpeg
connected (names 'pipe never connected' stage).
Likely root cause: WASAPI loopback captures the default render endpoint —
if audio plays on a non-default device the recording is silently silent.
The fix requires device enumeration + selection (slice 13 candidate).
Suite 290/291 — same sole pre-existing compositor pixel failure.
The creator-declared known failure (Mix_HonorsProviderGains) and AudioPipelineTests'
standalone hang shared ONE root cause, born in TASK 22: AudioMixer.FillAndMix
dereferenced _delayedMix (declared float[]? , never assigned) as delayed.Length —
every live-mix tick NRE'd before the pipe write, so NO audio ever reached the wire
(tests starved -> hung/fail; app -> swallowed catch logged only ex.Message, 10ms
flood). Now: ctor-allocated + null-check, catch logs WITH stack via AppLog and is
throttled 5s, and a cancelled token breaks out before logging.
Bonus contract fix: AudioGainProvider.LoopbackGain returned unity on a disproven
premise ('loopback scales with endpoint volume'). The creator's tonight observation —
volume at 20%, meter pegged — proves the WASAPI tap is pre-endpoint-volume, so the
mixer must multiply by GameAudioVolume for stream honesty (this is what ai.md always
specified; the 'failing' test encoded the same and was RIGHT).
AudioPipelineTests: 25/25 in 26ms, standalone, no hang. Known-failure count: ZERO.
ai.md tests paragraph rewritten (no more suite-total claims, both ex-'knowns'
explained); TASK 22 regression recorded; MyMistakes: the known-failure-label rules.
Creator ruling 2026-09-01 (stuck REC pill after the failed first recording):
- StopStream clears RecordPillOn + OnAirPillOn — intent resets with reality.
- OnFramePumpFailed: dispatcher-marshalled FULL rollback via StopStream (was:
toast + Error-status only when live — record-only sessions zombied with
IsRecording=true over a dead encoder; the 20:34 attempt proved it). Toast copy
says 'Recording stopped' vs 'stream pipeline stopped'.
- Same zombie class in the three go-live prep failure branches: StreamStatus.Error
limbo replaced by StopStream() (audio loop + recording + pills unwind; a created
broadcast still gets the close-out).
- AudioMixer.StopLive made explicitly idempotent (Stop() twice, rollback paths that
never reached StartLive).
- Seams: OnFramePumpFailed + IsRecording setter internal (InternalsVisibleTo; test
pattern mirrors LayoutPathOverride).
ONE integration test: SessionTeardownTests (real window + temp DB: pump death with
lit pill -> no zombie, no End button, Offline). AudioPipelineTests confirmed to hang
STANDALONE (pre-existing, the declared-known audio class) — ai.md test-count
paragraph corrected to stop claiming a suite total that cannot currently be measured.
Issue 3: Desktop meter now decays 2.5x faster than it rises
(DecaySmoothing=0.5 vs Smoothing=0.2). When rms drops below current
level the higher alpha pulls the bar down quickly; when audio returns
it rises smoothly. 207 tests pass, 0 warnings.
1. Error text shows in red inside the desktop audio meter bar (scrollable)
when TRAX load or playback fails — user gets feedback, not just a
silent red dot.
2. LoadTrax no longer nulls _music on error — the track path is retained
so the user sees what failed and can retry.
3. OnTraxPlaybackEnded now receives the NAudio exception and surfaces it.
4. TRAX dot is green (playing) or gray (idle/no track) — no more yellow
or red states.
207 tests, 0 warnings.
- Elements scrollbar no longer deselects element on click (check ScrollViewer, not StackPanel)
- Web source: URI + check/revert buttons merged into single DockPanel row, consistent order (✓ rightmost)
- Countdown timer moved from Elements panel to inline in source list row, range clamped to 1-30 min
- Volume slider click-to-position restored: e.Handled prevents IncreaseLarge from overriding position, Thumb drag still works
- Desktop audio meter: asymmetric smoothing (attack 0.2, decay 0.08) for faster response to silence
- 207 tests passing, 0 warnings
gramps