Files

T

gramps 2670ac022b feat(media): play the clip's soundtrack and share one decode pipe

A video file was picture-only: the decode pipe passed `-an`, so a starting-soon
or BRB clip with music played silent everywhere. Adds sound to every clip and
collapses the two decoders into one.

- `Services/VideoClipPipe.cs`: the shared pipe behind `MediaVideoSource` (TASK 21)
  and `AlertClipDecoder` (TASK 47) — video (`rawvideo bgra`, scaled) + audio
  (`f32le -ac 2 -ar 48000`, 50 ms chunks, byte-carry across reads), both paced to
  real time, loop restarts both so picture and sound stay together. Alert
  behaviour is unchanged.
- `MediaVideoSourceManager` applies the layer's Level/Mute per session and raises
  `AudioAvailable`; `MainViewModel` feeds it to the new
  `AudioMixer.EnqueueDesktopMediaAudio`, which joins the DESKTOP/GAME ring so the
  desktop fader and the mic auto-ducker apply exactly as they do for game audio
  and TRAX (creator ruling 2026-11-05: audio joins the group).
- Per-layer `MediaMuted` + existing `MediaVolume` surfaced as Mute/Level in the
  properties panel; `MediaMuted` persisted (ALTER TABLE guard + SELECT/INSERT).
- Level/mute are per SESSION because one decoder is shared per file path — the
  last layer touched wins; one decoder per layer would play the file twice.

Test: `MediaSourceAudioTests` — the pipe emits 50 ms chunks at 48k stereo, the
manager scales by level and drops when muted, and a media layer's sound moves the
desktop meter. Alert/decode suites unchanged and green.

Docs: task-21 closed on item 10, the "silent soundtrack" facts corrected in
frame-pipeline/output/overlays, HANDOFF rewritten. verify.sh green (529 passed;
the two documented pre-existing failures only).

2026-11-05 07:54:06 -08:00

1.8 KiB

Raw Blame History

HANDOFF — 2026-11-05 (TASK 21 — video file layer + its audio)

State

main; TASK 21 is complete. A video file added from the (+) menu decodes, composites into preview and stream, loops on toggle, and plays its own soundtrack (Mute + Level per layer, mixed into the desktop/game group so the desktop fader and the ducker apply). Removing it stops the decoder.
Decoding is now ONE pipe: Services/VideoClipPipe.cs behind both MediaVideoSource (TASK 21) and AlertClipDecoder (TASK 47) — "a clip is a clip". Alert behaviour is unchanged and its tests stay green.
Tests: MediaSourceAcquisitionTests (add → decoder starts, frames, loop, remove) and MediaSourceAudioTests (chunks, level scaling, mute, sound reaching the desktop meter). App build 0 warnings.

Next action

Next queued unit: TASK 49 (chat profanity filter). Day 3's TASK 32/33 resilience slices are still outstanding — 32·4 (fragmented MP4) is the smallest.
Agreed next feature work (designed, not started): green/blue screen detection + keying through the clip pipe — suggest-only by default, an opt-in "Auto Green/Blue Screen" setting, alert clips included.
Do not push absent a checkpoint request.

Landmines

LayerReorderPersistenceTests.RealMouseDrag_OnTheLayerList_PersistsTheReorder injects physical mouse input. Do not run it while the desktop is in use.
Integration tests constructing MainWindow must set a temporary LayoutPathOverride and clear it in finally.
Media sessions are keyed per path, not per layer: acquire through EnsureMediaSession, release through ReleaseMediaIfUnused, never AcquireAsync twice for one path. Level/mute are therefore per session too.
Bundled FFmpeg stays outside the single-file exe (ExcludeFromSingleFile="true").

1.8 KiB Raw Blame History

HANDOFF — 2026-11-05 (TASK 21 — video file layer + its audio)

State

Next action

Landmines

1.8 KiB

Raw Blame History