A video file was picture-only: the decode pipe passed `-an`, so a starting-soon
or BRB clip with music played silent everywhere. Adds sound to every clip and
collapses the two decoders into one.
- `Services/VideoClipPipe.cs`: the shared pipe behind `MediaVideoSource` (TASK 21)
and `AlertClipDecoder` (TASK 47) — video (`rawvideo bgra`, scaled) + audio
(`f32le -ac 2 -ar 48000`, 50 ms chunks, byte-carry across reads), both paced to
real time, loop restarts both so picture and sound stay together. Alert
behaviour is unchanged.
- `MediaVideoSourceManager` applies the layer's Level/Mute per session and raises
`AudioAvailable`; `MainViewModel` feeds it to the new
`AudioMixer.EnqueueDesktopMediaAudio`, which joins the DESKTOP/GAME ring so the
desktop fader and the mic auto-ducker apply exactly as they do for game audio
and TRAX (creator ruling 2026-11-05: audio joins the group).
- Per-layer `MediaMuted` + existing `MediaVolume` surfaced as Mute/Level in the
properties panel; `MediaMuted` persisted (ALTER TABLE guard + SELECT/INSERT).
- Level/mute are per SESSION because one decoder is shared per file path — the
last layer touched wins; one decoder per layer would play the file twice.
Test: `MediaSourceAudioTests` — the pipe emits 50 ms chunks at 48k stereo, the
manager scales by level and drops when muted, and a media layer's sound moves the
desktop meter. Alert/decode suites unchanged and green.
Docs: task-21 closed on item 10, the "silent soundtrack" facts corrected in
frame-pipeline/output/overlays, HANDOFF rewritten. verify.sh green (529 passed;
the two documented pre-existing failures only).
Add independent AppLog message types and explicit Class 1/2 severity. Wrap and color log rows, pin Class 1 above Class 2, and let acknowledgement return entries to the current-launch timeline without changing file format or rotation.
Ref: https://logtape.org/manual/levels
Creator request: a sound-byte browse + label under Built-in video, with an option to loop the
sound while the clip plays. The video itself is always one pass to completion.
- Source gains AlertAudioPath and AlertAudioLoop, persisted (column + migration + INSERT +
SELECT + reader, the whole chain — that chain is where the blur crashed once).
- The sound byte plays through MediaPlayer on the default output device; MediaEnded re-seeks to
zero while looping, and StopClip kills it so a tail cannot bleed into the next alert.
- The clip's OWN soundtrack moved off the mixer and onto the device as well
(Services/Audio/AlertAudioOutput: one WaveOutEvent + buffered float provider).
Why the move: the creator asked that alert audio — the clip's track or the added sound byte —
appear in the desktop sound bar. Only audio that reaches the default device is picked up by the
WASAPI loopback that feeds both the meter and the stream. The old EnqueueAlertAudio route put the
sound in the stream but left the meter flat and the room silent.
Tests: the sound byte and its loop flag round-trip through SQLite, and a fresh alert layer starts
with no sound byte and no loop. Release 0 warnings.
Reported: pegging the TRAX fader changes nothing audible, the ladder never reaches red, and the
ladder does not scale with the fader.
Cause (two defects in the same setters):
- UserGain wrote the OUTPUT DEVICE's volume. NAudio's WaveOut ignores that, so the fader was inert.
- LocalGain (the desktop-mute follow) recomputed the sample-chain volume WITHOUT the user gain, so
every duck/mute silently discarded whatever the creator had set.
Fix: one `EffectiveVolume(user, local)` = MusicVolume × user × local, applied to the sample chain by
both setters. The meter sits AFTER that stage, so the ladder now scales with the fader and can reach
the peak band on a loud track — exactly what pegging should look like. Max user gain is +6 dB, so
"pegged" means the file at unity, not a magic boost.
Guard: MusicLevelTests pins the arithmetic (default half, pegged double, mute multiplies to zero),
that the user gain survives the mute follow, and that nothing may drive the output device's volume
again — the no-op that made this invisible.
Reported: the TRAX bar moves, but its sampling looks far slower than the mic meter.
Cause: the TRAX number was a raw instantaneous peak per metering block — a stutter sampled by the UI
timer, with no smoothing. The mic looks fluid because it is smoothed at audio-callback rate.
Fix: the player now smooths its measured level with the SAME curve the mic uses
(MeterScale.NextLevel: fast attack, gentle decay), and the dock's meter clock runs at ~30 Hz instead
of 20. Two ladders that move alike, because they are the same ballistics.
Guard: MeterScaleTests pins the attack/decay behaviour alongside peak-hold.
Creator: "why is there no meter for when trax is playing?"
Routing answer: TRAX rides the desktop loopback, so it already moved the GAME ladder — there was no
separate TRAX signal in the mix. But that ladder cannot answer the question a deaf creator actually
has: "is the music producing sound at all?" The play glyph cannot either — an open file with a dead
decoder still reports Playing.
So the player now measures its own output: \`MusicPlayer\` wraps the volume stage in an NAudio
\`MeteringSampleProvider\` and exposes \`Level\`; \`ITraxPlayer.Level\` keeps the fake honest. The VM maps
it through the same display scale (\`TraxMeterFraction\`), zeroed when not playing, and the TRAX bay
carries a 24-segment ladder above its fader.
This reverses my earlier "no ladder, nothing honest to show" — the honest thing arrived with a real
measurement. TraxBayTests flipped accordingly (it now asserts the ladder AND its source).
Unit 9, the one that matters most for this creator.
MASTER STRIP (top, nearest the preview):
- one ladder fed by the MEASURED mix — `AudioMixer` now measures the buffer the encoder receives
(`MasterLevel`, `MasterImbalanceDb`), so the master cannot show a source instead of the output
- `-12.4 dB` readout, and ONE L/R flag that appears above a 6 dB gap instead of a second ladder
- bitrate / FPS / dropped / duration from the existing health poll
MONO OUTPUT:
- `AudioFold.ToMono` averages each stereo pair into both channels, applied after the limiter and
the sync delay, so the stream AND the recording get what was folded
- `MonoOutput` is persisted (`Audio.MonoOutput`) and restored at startup: an accessibility switch
that resets each session is an accessibility failure
- one click in the strip, remembered forever
Guard: AudioFoldTests (hard-left reaches the right at half amplitude, hard-right — the inaudible
case — reaches the left, centred material keeps its level, mono buffers untouched) and
MasterStripTests (the strip binds the measured mix + the switch, the store round-trips it, and the
fold is the tested helper rather than new arithmetic). 33 audio tests green.
Unit 7. The bay shows the track name (left-click toggles playback, right-click opens the picker —
the label stays the control), a play/pause button, the duck badge, and a dB level.
No ladder here, deliberately: nothing measures the music bed, and a decorative ladder would break
the measured-not-painted rule. A test asserts the bay has no SegmentMeter.
The level is real: `ITraxPlayer.UserGain` is a new creator gain multiplied onto the fixed design
volume, separate from `LocalGain` (which follows the desktop mute). The VM's `TraxVolumeDb` is the
same dB scale as the other faders, so unity means the bed's design level.
Unit 6. Mirrors the mic bay for the desktop/game bed, plus the badge that matters to a creator who
cannot hear the duck: it shows while the ducker is actually attenuating, and it prints the gain the
mixer applied (−12.0 DUCK by design).
- `AudioMixer.DuckGain` exposes `AutoDucker.CurrentGain` instead of leaving it internal.
- VM: `GameVolumeDb` / `GameVolumeDbLabel` (fader face), `GameMeterFraction` (measured level),
`IsGameDucking` + `GameDuckLabel` refreshed on the level tick — the same moment the ducker runs,
so the badge cannot lag behind the mix.
- The bay: header chip + source, ladder, mute dot (uses the meter band, so mute reads at a glance),
dB readout, fader.
Guard: GameBayTests — the dock binds all four, `AutoDucker.DuckGain` is −12.04 dB and the label is
produced through the same dB scale as the faders, and the mixer property exists so the badge cannot
become a hardcoded number.
`AudioLevelMeter` measured one number for everything. It now also keeps smoothed per-channel
levels with the same attack/decay, plus `ImbalanceDb`. Mono samples report both sides equal (a mono
source is centred), so the two never disagree about whether audio exists.
The dock shows ONE ladder per source — two ladders per bay was visual noise for a rare failure —
but this creator is profoundly deaf on the right, so the one case that matters is "the sound is
panned away from me". That is now measurable and will drive a flag, not a second ladder.
Semantics that mattered: a silent side is compared at the −60 dBFS floor, so a fully one-sided
signal reads ≈54 dB rather than "nothing to compare". My first version returned 0 for exactly the
case the feature exists for; the test caught it.
Guard: StereoMeterTests — hard left, hard right (the inaudible case), balanced stereo, mono
centred, and silence-is-not-an-imbalance. 31 audio tests green.
TASK 43's alert box grows a real video celebration. Per-alert IAlertClipDecoder
(ffmpeg bgra + f32le pipes, real-time paced, disposed at drain) plays the shipped
Assets/alert-default.mp4 (stamped into the Asset table at startup) unless the
creator picks their own file — path reference only, never stored in the DB; the
six AlertRenderer animations stay the fallback. ~0.3s fade rides the alpha
envelope on straight-source copies (EOF freeze-frames then fades out); audio
forwards to a new AudioMixer alert ring (8s, 48k stereo) drained at unity — no
duck, creator ruling — scaled by volume × fade. An auto-composed marquee ticker
('Funder — Super Chat · $10.00', 140px/s) scrolls top-of-frame via a
FramePump._alertTicker seam through Render/CompositeLayers, mixed into the cache
signature (dynamic overlay, never baked). New Stream Alerts section in LeftPanel.
Derivative-work references (how OBS/Streamlabs alert boxes do per-alert video):
- https://support.streamlabs.com/hc/en-us/articles/217741147-Setting-Up-Your-Streamlabs-Alerts (custom image/video per alert type + variations)
- https://obsproject.com/kb/stream-tutorial-2-alerts (alert overlay as an on-screen zone)
- https://streamlabs.com/content-hub/widgets/alert-box (per-event alert playback)
Good Dog: AlertLayerVideoTests drives a fake IAlertClipDecoder through the whole
lifecycle in one pass (custom path wins, decoder spawns/disposes, fade envelope
0→127→255, audio volume×fade, ticker scrolls, EOF fade-drain to idle). It caught
the clip branch of Advance not clearing _current before AdvanceToNext — the layer
stayed IsPlaying after drain (MyMistakes post-mortem).
Full vstest 319/319; clean build 0 warnings; scope check green.
Positive offsets still delay the whole mix via the delay line (lip-sync fix);
negative offsets now ARM once at StartLive and drop |N| ms off the pipe's write
head so audio events land earlier when audio runs BEHIND video. Slider relabeled
AUDIO SYNC, Min −500, locked while live/recording (IsEditMode). LayoutStore and
VM clamp to −500..500.
OBS reference for eat-the-head negative sync: https://obsproject.com/kb/obs-studio/buffering-time (negative sync values pull audio earlier by discarding buffered player audio).
Test: StartLive_NegativeOffset_AdvancesAudio_ByDroppingTheStreamHead (6x0.9 head
must be eaten before 0.2 bed reaches the wire).
AudioMixer.Start() begins mic/loopback capture at app startup to feed the
level meters, so both 2s ring buffers fill with pre-live audio. StartLive()
drained from the oldest tail sample, putting every recorded event ~2.2s late
in the audio track (confirmed by clap analysis + cross-correlation on two
takes: +2.11 to +2.22s).
Fix: AudioMixer.StartLive() now runs _micBuffer.Clear() + _loopbackBuffer.Clear()
immediately after the pipe starts, before the drain task runs. Recording now
begins at go-live; the <=10ms in-flight chunk evicted by Clear() is imperceptible.
Regression test: StartLive_DiscardsPreLiveBacklog_SoFirstAudioIsCurrent
saturates the loopback ring with stale 0.8 pre-live audio, then asserts the
wire carries fresh post-live 0.2 (max < 0.3).
Reference (external, per derivative-work rule): OBS 'Audio mixer' keeps its
buffers fed continuously and syncs the stream start timestamp at record time
rather than replaying pre-live capture; a go-live flush of the capture buffer
is the accepted pattern for live tools restarting a stream.
Docs: HANDOFF.md (fix shipped), MyMistakes.md (A/V sync measurement recipe).
- AudioSyncDelay.Configure reallocated/zeroed its buffer every ~10ms tick
(AudioMixer re-reads the UI setting each mix), so any non-zero sync offset
erased the just-written audio -> total silence. Now early-returns when the
delay samples are unchanged. Regression test proven both ways.
- SceneCompositor.BlitContentRaw defaulted cbW/cbH=0 when CropBounds is null
(regression from ed9d7c1) -> webcam blit to an empty rect = gray block.
Default to src.Width/Height. Regression test proven both ways.
- Truncated recordings: rawvideo mux stamps frames at declared 60fps by
arrival; a scene whose first layer is dynamic (hidden elements still count)
kills the bake cache -> full render ~35ms -> ~27fps submitted -> halved
file length. Static scenes bake once (246ms cold, then <1ms) -> 60fps,
full-length (probe + 13:53 take, 301/300 per 5s, 12.46s file from 12.3s
wall). FramePump.ProbeRender names the hot render path on slow frames.
The recorded audio is full-length silent AAC (−91dB, 1124 frames/23.95s) —
the named pipe carried ~9.1MB of zeros the entire take. The mixer's loop
ran, the pipe connected, ffmpeg read it, and the file is full-length
silence. Sources started fine ('Audio: using system default mic...' logged),
no failure callbacks fired, and the existing integration test
(Mix_WithFiltersDuckAndGain_Lands_On_AudioPipe) proves the loop→pipe path
carries real audio when fed — the fault is capture-side.
Added permanent per-5s live-loop telemetry to startup.log so the next take
names the exact stage without new code:
Audio live: pipe connected= dropped writes= micLevel= loopLevel= drained
N/N samples peakMix N (per 5s)
- AudioMixer.FillAndMix now returns (MicRms, MicDrained, LoopDrained) for
the accumulation; StartLive/StopLive log start/stop lines.
- NamedPipeAudioWriter.DroppedWrites: nonzero = audio dropped before ffmpeg
connected (names 'pipe never connected' stage).
Likely root cause: WASAPI loopback captures the default render endpoint —
if audio plays on a non-default device the recording is silently silent.
The fix requires device enumeration + selection (slice 13 candidate).
Suite 290/291 — same sole pre-existing compositor pixel failure.
The creator-declared known failure (Mix_HonorsProviderGains) and AudioPipelineTests'
standalone hang shared ONE root cause, born in TASK 22: AudioMixer.FillAndMix
dereferenced _delayedMix (declared float[]? , never assigned) as delayed.Length —
every live-mix tick NRE'd before the pipe write, so NO audio ever reached the wire
(tests starved -> hung/fail; app -> swallowed catch logged only ex.Message, 10ms
flood). Now: ctor-allocated + null-check, catch logs WITH stack via AppLog and is
throttled 5s, and a cancelled token breaks out before logging.
Bonus contract fix: AudioGainProvider.LoopbackGain returned unity on a disproven
premise ('loopback scales with endpoint volume'). The creator's tonight observation —
volume at 20%, meter pegged — proves the WASAPI tap is pre-endpoint-volume, so the
mixer must multiply by GameAudioVolume for stream honesty (this is what ai.md always
specified; the 'failing' test encoded the same and was RIGHT).
AudioPipelineTests: 25/25 in 26ms, standalone, no hang. Known-failure count: ZERO.
ai.md tests paragraph rewritten (no more suite-total claims, both ex-'knowns'
explained); TASK 22 regression recorded; MyMistakes: the known-failure-label rules.
Creator ruling 2026-09-01 (stuck REC pill after the failed first recording):
- StopStream clears RecordPillOn + OnAirPillOn — intent resets with reality.
- OnFramePumpFailed: dispatcher-marshalled FULL rollback via StopStream (was:
toast + Error-status only when live — record-only sessions zombied with
IsRecording=true over a dead encoder; the 20:34 attempt proved it). Toast copy
says 'Recording stopped' vs 'stream pipeline stopped'.
- Same zombie class in the three go-live prep failure branches: StreamStatus.Error
limbo replaced by StopStream() (audio loop + recording + pills unwind; a created
broadcast still gets the close-out).
- AudioMixer.StopLive made explicitly idempotent (Stop() twice, rollback paths that
never reached StartLive).
- Seams: OnFramePumpFailed + IsRecording setter internal (InternalsVisibleTo; test
pattern mirrors LayoutPathOverride).
ONE integration test: SessionTeardownTests (real window + temp DB: pump death with
lit pill -> no zombie, no End button, Offline). AudioPipelineTests confirmed to hang
STANDALONE (pre-existing, the declared-known audio class) — ai.md test-count
paragraph corrected to stop claiming a suite total that cannot currently be measured.
Issue 3: Desktop meter now decays 2.5x faster than it rises
(DecaySmoothing=0.5 vs Smoothing=0.2). When rms drops below current
level the higher alpha pulls the bar down quickly; when audio returns
it rises smoothly. 207 tests pass, 0 warnings.
1. Error text shows in red inside the desktop audio meter bar (scrollable)
when TRAX load or playback fails — user gets feedback, not just a
silent red dot.
2. LoadTrax no longer nulls _music on error — the track path is retained
so the user sees what failed and can retry.
3. OnTraxPlaybackEnded now receives the NAudio exception and surfaces it.
4. TRAX dot is green (playing) or gray (idle/no track) — no more yellow
or red states.
207 tests, 0 warnings.
- Elements scrollbar no longer deselects element on click (check ScrollViewer, not StackPanel)
- Web source: URI + check/revert buttons merged into single DockPanel row, consistent order (✓ rightmost)
- Countdown timer moved from Elements panel to inline in source list row, range clamped to 1-30 min
- Volume slider click-to-position restored: e.Handled prevents IncreaseLarge from overriding position, Thumb drag still works
- Desktop audio meter: asymmetric smoothing (attack 0.2, decay 0.08) for faster response to silence
- 207 tests passing, 0 warnings
gramps