feat(media): play the clip's soundtrack and share one decode pipe

A video file was picture-only: the decode pipe passed `-an`, so a starting-soon
or BRB clip with music played silent everywhere. Adds sound to every clip and
collapses the two decoders into one.

- `Services/VideoClipPipe.cs`: the shared pipe behind `MediaVideoSource` (TASK 21)
  and `AlertClipDecoder` (TASK 47) — video (`rawvideo bgra`, scaled) + audio
  (`f32le -ac 2 -ar 48000`, 50 ms chunks, byte-carry across reads), both paced to
  real time, loop restarts both so picture and sound stay together. Alert
  behaviour is unchanged.
- `MediaVideoSourceManager` applies the layer's Level/Mute per session and raises
  `AudioAvailable`; `MainViewModel` feeds it to the new
  `AudioMixer.EnqueueDesktopMediaAudio`, which joins the DESKTOP/GAME ring so the
  desktop fader and the mic auto-ducker apply exactly as they do for game audio
  and TRAX (creator ruling 2026-11-05: audio joins the group).
- Per-layer `MediaMuted` + existing `MediaVolume` surfaced as Mute/Level in the
  properties panel; `MediaMuted` persisted (ALTER TABLE guard + SELECT/INSERT).
- Level/mute are per SESSION because one decoder is shared per file path — the
  last layer touched wins; one decoder per layer would play the file twice.

Test: `MediaSourceAudioTests` — the pipe emits 50 ms chunks at 48k stereo, the
manager scales by level and drops when muted, and a media layer's sound moves the
desktop meter. Alert/decode suites unchanged and green.

Docs: task-21 closed on item 10, the "silent soundtrack" facts corrected in
frame-pipeline/output/overlays, HANDOFF rewritten. verify.sh green (529 passed;
the two documented pre-existing failures only).
This commit is contained in:
2026-11-05 07:54:06 -08:00
parent 776f84fb88
commit 2670ac022b
21 changed files with 669 additions and 328 deletions
+28 -1
View File
@@ -6,6 +6,7 @@ using System.Windows.Media.Imaging;
using Microsoft.Win32;
using ytLive.Models;
using ytLive.Services;
using ytLive.Services.Audio;
namespace ytLive.ViewModels;
@@ -105,6 +106,11 @@ public partial class MainViewModel
{
_mediaManager.SetLooping(path, AnyMediaReferenceLoops(path));
}
else if (e.PropertyName is nameof(Source.MediaVolume) or nameof(Source.MediaMuted)
&& source.MediaPath is { } audioPath)
{
ApplyMediaAudioSettings(audioPath);
}
}
_mediaHandlers[source.Id] = OnSourceChanged;
@@ -121,7 +127,11 @@ public partial class MainViewModel
private void EnsureMediaSession(string? path)
{
if (string.IsNullOrWhiteSpace(path)) return;
if (_mediaManager.HasSession(path)) return;
if (_mediaManager.HasSession(path))
{
ApplyMediaAudioSettings(path);
return;
}
_ = AcquireMediaAsync(path);
}
@@ -133,8 +143,25 @@ public partial class MainViewModel
// case the bitmap is already published and will not be raised again for us.
if (_mediaManager.GetPreviewBitmap(path) is { } bitmap)
OnMediaPreviewBitmapChanged(path, bitmap);
ApplyMediaAudioSettings(path);
}
/// <summary>The file's sound controls: one decoder is shared per path, so the level and
/// mute are per session — the layer that was last touched is the one that wins.</summary>
private void ApplyMediaAudioSettings(string path)
{
var source = Scenes.SelectMany(s => s.Elements).OfType<Source>()
.FirstOrDefault(s => s.IsMediaSource && s.MediaPath == path);
if (source == null) return;
_mediaManager.SetAudioSettings(path, source.MediaVolume, source.MediaMuted);
}
/// <summary>A clip's own soundtrack joins the desktop/game group, so the desktop fader and
/// the voice ducker treat it exactly like game audio (TASK 21 audio slice).</summary>
private void OnMediaAudioAvailable(string path, AudioSample sample)
=> _audioMixer.EnqueueDesktopMediaAudio(sample);
/// <summary>Shared sessions or not, the loop state is the path's: any live media
/// reference asking for a loop makes the one decoder loop (recorded rule, TASK 21).</summary>
private bool AnyMediaReferenceLoops(string path)