# TASK 21 — Media source (video file playback)

> Catalog: [`TASKS.md`](../TASKS.md) — status and requirements live here.


**Goal:** play video files (MP4, MOV, AVI) into scenes — starting soon videos, BRB loops, intro/outro clips.

### Status: ✅ Complete 2026-11-05 — a video file added in the (+) menu decodes, composites into the
preview and the stream, loops on the creator's toggle, and stops when the layer or its last scene
reference goes away, and **the file's own soundtrack plays into the stream** (item 10).

1. ✅ `MediaSourceType` enum: `Video`, `Audio` (audio-only files via media source)
2. ✅ `SourceType.MediaSource` addition to the enum
3. ✅ `MediaSourceModel`: `MediaPath`, `MediaIsLooping`, `MediaVolume` (0-1), `MediaPlaybackState` — persisted in LayoutStore (schema migration + SELECT/INSERT + round-trip test)
4. ✅ `MediaVideoSource` implements `IMediaFrameSource`: FFmpeg raw video decoder → `VideoFrame` pipeline — spawns ffmpeg `-f rawvideo -pix_fmt bgra`, drains the pipe via pure `RawVideoFrameReader` (`Services/RawVideoFrameReader.cs`), raises `FrameAvailable`/`Completed`; lifecycle is `StartAsync`/`StopAsync`; decode process behind `IDecodeProcess`/`FfmpegDecodeProcess` seam (binary-stdout mirror of `IEncoderProcess`). Shipped 2026-08-31; refactored onto `IMediaFrameSource` with this slice.
4b. ✅ Slice 2a — native-FPS probe seam: `FfmpegFrameRateParser` (pure, prefers `avg_frame_rate=` then `r_frame_rate=`, rational N/N/M, unknown→null) + `IFrameRateProbe`/`FfmpegFrameRateProbe` (derives sibling `ffprobe.exe` from the located ffmpeg dir, reuses the `IDecodeProcess` seam; null if ffprobe absent) + `FfmpegLocator.ProbeFileName` now also extracts `ffprobe.exe` from the pinned archive (conditional). Tests: `FfmpegFrameRateParserTests` (6 pure units + 1 probe integration via fake locator/process) + `FfmpegLocatorTests` still green — 15/15.
5. ✅ `IMediaFrameSource` + `MediaVideoSourceManager` (`Services/IMediaFrameSource.cs`, `Services/MediaVideoSourceManager.cs`): app-wide decode-session ownership refcounted by `MediaPath` with a `Func<string, IMediaFrameSource?>` factory seam; `AcquireAsync`/`ReleaseAsync`/`ReleaseAllAsync`/`GetLatestFrame`/`HasSession`/`GetActiveKeys`/`SetLooping`; coalesces frames onto the UI dispatcher onto a single shared `WriteableBitmap` per file (mirror of `ScreenCaptureManager`); `MediaFailed` + `PreviewBitmapChanged`. Unit tests + one integration test (bitmap share/coalesce) — `MediaVideoSourceManagerTests`.
6. ✅ Native-FPS pacing (slice 2b): `MediaVideoSource` takes optional `IFrameRateProbe?` + `Func<TimeSpan,CancellationToken,Task>? delay` seams, probes FPS once in `RunAsync`, and delays by 1/fps after each emitted frame; unknown/absent probe → no pacing. Test: `MediaVideoSource_PacesFramesByProbedFps`.
7. ✅ Wire into `FramePump` resolver — `Source { Type: MediaSource }` → latest video frame, and the same
   resolver serves preview and thumbnails (`FramePump` defaults `resolver` to the frame resolver).
8. ✅ Wire into `SceneCompositor` — media renders through the generic `frameFor(element)` path at the
   element's rect, so no compositor change was needed.
9. ✅ Loop control — `IMediaFrameSource.Looping`; `MediaVideoSource` restarts the decode on natural EOF
   when `Looping` (fresh process per pass). `MediaIsLooping` reaches the session through the manager's
   per-path `loopProvider` at start and `SetLooping` on a live toggle. Rule (recorded): the path's
   session loops if **any** live media reference has it on, because one decoder is shared per path.
10. ✅ File audio + volume — the shared clip pipe decodes the soundtrack (`f32le -ac 2 -ar 48000 -vn`,
    50 ms chunks paced to real time, byte-carry across reads) and the layer carries **Mute** + **Level**.
    The manager applies them per session (one decoder per path, so the last layer touched wins — the
    alternative would play the same file twice) and the audio joins the DESKTOP/GAME group in
    `AudioMixer.EnqueueDesktopMediaAudio`, so the desktop fader and the voice ducker behave exactly as
    they do for game audio and TRAX. Test: `MediaSourceAudioTests` (chunks, level scaling, mute, and the
    layer's sound reaching the desktop meter).
11. ✅ UI: `AddMediaCommand` (+ menu → "Video File", video-filtered `OpenFileDialog`), `ChangeMediaFileCommand`
    ("Change…" in the properties panel), loop toggle, video name display.
12. ✅ Schema migration for media source settings (file path, loop, volume) — shipped with item 3.
13. ✅ Tests: decode/loop/pacing at the unit level, and `MediaSourceAcquisitionTests` (the Good Dog for the
    wiring): add a file → one decoder starts, frames reach the layer, loop flip reaches the decoder, remove → stops.

### Design decisions

- **FFmpeg handles all formats** — no codec-specific code. FFmpeg already in the project.
- **Audio joins the desktop/game group** (creator ruling 2026-11-05) — the clip's sound is decoded by
  the same pipe as its picture, mixed directly into the stream (not played locally and captured), and
  grouped with game audio so one fader and the ducker cover it. A per-layer Mute/Level sits on top.
  Chosen over TRAX-style local playback because it works regardless of the creator's desktop-capture
  state, and because an alert clip needs the same direct route anyway.
- **This pairs with TRAX** — TRAX is background music, media source is background video. Together they make non-Live scenes (Starting/BRB/Ending) feel polished.
- **Not a full NLE** — no trimming, no multi-track, no effects. Just "play this video in the scene."
- **Layers are defined in Live first** (creator 2026-11-05): this unit adds the layer and the picker in
  the ordinary (+) menu, with no per-scene availability rules. Which scenes may host a video layer is a
  later pass, and scene-only layers (an ending credit roll, for instance) are defined after that pass.