Commit Graph
3 Commits
Author SHA1 Message Date
gramps 2670ac022b feat(media): play the clip's soundtrack and share one decode pipe
A video file was picture-only: the decode pipe passed `-an`, so a starting-soon
or BRB clip with music played silent everywhere. Adds sound to every clip and
collapses the two decoders into one.

- `Services/VideoClipPipe.cs`: the shared pipe behind `MediaVideoSource` (TASK 21)
  and `AlertClipDecoder` (TASK 47) — video (`rawvideo bgra`, scaled) + audio
  (`f32le -ac 2 -ar 48000`, 50 ms chunks, byte-carry across reads), both paced to
  real time, loop restarts both so picture and sound stay together. Alert
  behaviour is unchanged.
- `MediaVideoSourceManager` applies the layer's Level/Mute per session and raises
  `AudioAvailable`; `MainViewModel` feeds it to the new
  `AudioMixer.EnqueueDesktopMediaAudio`, which joins the DESKTOP/GAME ring so the
  desktop fader and the mic auto-ducker apply exactly as they do for game audio
  and TRAX (creator ruling 2026-11-05: audio joins the group).
- Per-layer `MediaMuted` + existing `MediaVolume` surfaced as Mute/Level in the
  properties panel; `MediaMuted` persisted (ALTER TABLE guard + SELECT/INSERT).
- Level/mute are per SESSION because one decoder is shared per file path — the
  last layer touched wins; one decoder per layer would play the file twice.

Test: `MediaSourceAudioTests` — the pipe emits 50 ms chunks at 48k stereo, the
manager scales by level and drops when muted, and a media layer's sound moves the
desktop meter. Alert/decode suites unchanged and green.

Docs: task-21 closed on item 10, the "silent soundtrack" facts corrected in
frame-pipeline/output/overlays, HANDOFF rewritten. verify.sh green (529 passed;
the two documented pre-existing failures only).
2026-11-05 07:54:06 -08:00
gramps 77d7208bfa feat(media): add a video file layer that decodes into the scene
TASK 21's picker slice: the decoder, manager, pacing and loop mechanism were
already shipped but nothing ever acquired a session, so a media Source rendered
nothing. Adds "Video File" to the (+) menu (video-filtered OpenFileDialog),
one decode session per distinct path with a HasSession guard, release when the
last scene reference goes, and a post-load reconcile. Loop is a manager-level
per-path answer ("any live reference loops") applied before StartAsync and
updated live via SetLooping. Properties panel shows the file with Change… and
Loop. Volume stays unwired: media audio rides the desktop loopback, TRAX-style.

Reference: OBS's media source (file → decode → composite into the scene) is the
established answer for this feature; the lifecycle shape mirrors this repo's
WebView2/ScreenCapture registration rather than inventing a new one.

Test: ytLive.Tests/MediaSourceAcquisitionTests.cs — add starts one decoder and
frames reach the layer, loop flip reaches the decoder, remove stops it.

Docs: task-21 closed, TASKS row + Day-4 row + stale 3·17 row corrected,
frame-pipeline acquisition note, HANDOFF rewritten. verify.sh green.
2026-11-05 07:17:43 -08:00
gramps 480ead0fc7 TASK 21 slice 1 step 3: IMediaFrameSource + MediaVideoSourceManager
Refactor MediaVideoSource onto IMediaFrameSource (Key/FrameAvailable/Completed/
StartAsync/StopAsync, renaming Start->StartAsync, FrameReady->FrameAvailable),
behavior preserved. Add MediaVideoSourceManager: app-wide decode-session owner
refcounted by MediaPath with Func<string,IMediaFrameSource?> factory seam,
Acquire/Release/ReleaseAll/GetLatestFrame, coalescing each file's frames onto
the UI dispatcher onto one shared WriteableBitmap; MediaFailed + PreviewBitmapChanged.

Mirrors ScreenCaptureManager (screen-capture session ownership + dispatcher
coalescing), per the codebase precedent and derivative-work rule.

Tests: MediaVideoSourceManagerTests (5 unit + 1 integration: single shared
bitmap, coalesce-to-latest); MediaVideoSourceTests updated for renames.
10/10 media tests pass, build 0 warnings. Docs (TASKS/HANDOFF/ai.md) updated.
2026-08-31 19:22:02 -07:00