feat(media): play the clip's soundtrack and share one decode pipe
A video file was picture-only: the decode pipe passed `-an`, so a starting-soon or BRB clip with music played silent everywhere. Adds sound to every clip and collapses the two decoders into one. - `Services/VideoClipPipe.cs`: the shared pipe behind `MediaVideoSource` (TASK 21) and `AlertClipDecoder` (TASK 47) — video (`rawvideo bgra`, scaled) + audio (`f32le -ac 2 -ar 48000`, 50 ms chunks, byte-carry across reads), both paced to real time, loop restarts both so picture and sound stay together. Alert behaviour is unchanged. - `MediaVideoSourceManager` applies the layer's Level/Mute per session and raises `AudioAvailable`; `MainViewModel` feeds it to the new `AudioMixer.EnqueueDesktopMediaAudio`, which joins the DESKTOP/GAME ring so the desktop fader and the mic auto-ducker apply exactly as they do for game audio and TRAX (creator ruling 2026-11-05: audio joins the group). - Per-layer `MediaMuted` + existing `MediaVolume` surfaced as Mute/Level in the properties panel; `MediaMuted` persisted (ALTER TABLE guard + SELECT/INSERT). - Level/mute are per SESSION because one decoder is shared per file path — the last layer touched wins; one decoder per layer would play the file twice. Test: `MediaSourceAudioTests` — the pipe emits 50 ms chunks at 48k stereo, the manager scales by level and drops when muted, and a media layer's sound moves the desktop meter. Alert/decode suites unchanged and green. Docs: task-21 closed on item 10, the "silent soundtrack" facts corrected in frame-pipeline/output/overlays, HANDOFF rewritten. verify.sh green (529 passed; the two documented pre-existing failures only).
This commit is contained in:
+12
-10
@@ -1,20 +1,21 @@
|
||||
# HANDOFF — 2026-11-05 (TASK 21 — video file layer)
|
||||
# HANDOFF — 2026-11-05 (TASK 21 — video file layer + its audio)
|
||||
|
||||
## State
|
||||
|
||||
- `main`; TASK 21 is complete: a video file added from the (+) menu decodes, composites into preview
|
||||
and stream, loops on toggle, and stops with its last scene reference. 3·17 (text layers) completed
|
||||
2026-11-04.
|
||||
- New test `MediaSourceAcquisitionTests` (add → one decoder starts + frames reach the layer + loop flip
|
||||
reaches the decoder + remove → stops). App build is 0 warnings.
|
||||
- **Media is picture-only:** `MediaVideoSource` decodes with `-an`, so a video file's own soundtrack
|
||||
is silent on stream — the loopback does not carry it because nothing plays it. Two routes recorded
|
||||
in `task-21` item 10 (alert-style mixer feed vs TRAX-style local play); no volume slider until one exists.
|
||||
- `main`; TASK 21 is complete. A video file added from the (+) menu decodes, composites into preview and
|
||||
stream, loops on toggle, and **plays its own soundtrack** (Mute + Level per layer, mixed into the
|
||||
desktop/game group so the desktop fader and the ducker apply). Removing it stops the decoder.
|
||||
- Decoding is now ONE pipe: `Services/VideoClipPipe.cs` behind both `MediaVideoSource` (TASK 21) and
|
||||
`AlertClipDecoder` (TASK 47) — "a clip is a clip". Alert behaviour is unchanged and its tests stay green.
|
||||
- Tests: `MediaSourceAcquisitionTests` (add → decoder starts, frames, loop, remove) and
|
||||
`MediaSourceAudioTests` (chunks, level scaling, mute, sound reaching the desktop meter). App build 0 warnings.
|
||||
|
||||
## Next action
|
||||
|
||||
- Next queued unit: TASK 49 (chat profanity filter). Day 3's TASK 32/33 resilience slices are still
|
||||
outstanding — 32·4 (fragmented MP4) is the smallest.
|
||||
- Agreed next feature work (designed, not started): green/blue screen detection + keying through the clip
|
||||
pipe — suggest-only by default, an opt-in "Auto Green/Blue Screen" setting, alert clips included.
|
||||
- Do not push absent a checkpoint request.
|
||||
|
||||
## Landmines
|
||||
@@ -23,5 +24,6 @@
|
||||
mouse input. Do not run it while the desktop is in use.
|
||||
- Integration tests constructing `MainWindow` must set a temporary `LayoutPathOverride` and clear it in `finally`.
|
||||
- Media sessions are keyed per **path**, not per layer: acquire through `EnsureMediaSession`, release
|
||||
through `ReleaseMediaIfUnused`, never `AcquireAsync` twice for one path.
|
||||
through `ReleaseMediaIfUnused`, never `AcquireAsync` twice for one path. Level/mute are therefore
|
||||
per session too.
|
||||
- Bundled FFmpeg stays outside the single-file exe (`ExcludeFromSingleFile="true"`).
|
||||
|
||||
Reference in New Issue
Block a user