diff --git a/Controls/LeftPanel.xaml b/Controls/LeftPanel.xaml index 9ecfed0..e6f0ce8 100644 --- a/Controls/LeftPanel.xaml +++ b/Controls/LeftPanel.xaml @@ -746,7 +746,7 @@ - diff --git a/HANDOFF.md b/HANDOFF.md index c7f8b84..3e103ce 100644 --- a/HANDOFF.md +++ b/HANDOFF.md @@ -7,7 +7,9 @@ 2026-11-04. - New test `MediaSourceAcquisitionTests` (add → one decoder starts + frames reach the layer + loop flip reaches the decoder + remove → stops). App build is 0 warnings. -- Volume is deliberately unwired: media audio rides the desktop loopback, so a slider would do nothing. +- **Media is picture-only:** `MediaVideoSource` decodes with `-an`, so a video file's own soundtrack + is silent on stream — the loopback does not carry it because nothing plays it. Two routes recorded + in `task-21` item 10 (alert-style mixer feed vs TRAX-style local play); no volume slider until one exists. ## Next action diff --git a/TASKS/task-21-media-source.md b/TASKS/task-21-media-source.md index a35d66f..bc43088 100644 --- a/TASKS/task-21-media-source.md +++ b/TASKS/task-21-media-source.md @@ -7,7 +7,8 @@ ### Status: ✅ Complete 2026-11-05 — a video file added in the (+) menu decodes, composites into the preview and the stream, loops on the creator's toggle, and stops when the layer or its last scene -reference goes away. Audio is the desktop loopback (TRAX-style); a per-file volume route is not built. +reference goes away. **Picture only: the file's own audio is not played anywhere** (the decode pipe +passes `-an`), so a video with a soundtrack is silent on stream — the audio route is open, item 10. 1. ✅ `MediaSourceType` enum: `Video`, `Audio` (audio-only files via media source) 2. ✅ `SourceType.MediaSource` addition to the enum @@ -24,9 +25,13 @@ reference goes away. Audio is the desktop loopback (TRAX-style); a per-file volu when `Looping` (fresh process per pass). `MediaIsLooping` reaches the session through the manager's per-path `loopProvider` at start and `SetLooping` on a live toggle. Rule (recorded): the path's session loops if **any** live media reference has it on, because one decoder is shared per path. -10. ⏸ Volume control — deliberately not wired: media audio is captured by the desktop loopback like - TRAX, so a per-file slider would be a control with no audible effect. Revisit only with a local - audio route for the file's own stream. +10. ⏸ File audio + volume — **the file's soundtrack is silent everywhere today**: the decode pipe + passes `-an`, so nothing plays it and the desktop loopback has nothing to carry. Two established + routes when this is taken: **alert-style** (a second `f32le` pipe into the mixer — + `AlertClipDecoder.RunAudioAsync` is the working model, independent of desktop capture) or + **TRAX-style** (`MusicPlayer` plays it to the default device, so the loopback, the desktop fader + and ducking all apply — at the cost of a second clock to hold in sync with the video). Product + call; not started, and no volume slider until one of them exists. 11. ✅ UI: `AddMediaCommand` (+ menu → "Video File", video-filtered `OpenFileDialog`), `ChangeMediaFileCommand` ("Change…" in the properties panel), loop toggle, video name display. 12. ✅ Schema migration for media source settings (file path, loop, volume) — shipped with item 3. @@ -36,7 +41,10 @@ reference goes away. Audio is the desktop loopback (TRAX-style); a per-file volu ### Design decisions - **FFmpeg handles all formats** — no codec-specific code. FFmpeg already in the project. -- **Audio plays through the desktop channel** — media source audio is captured by the WASAPI loopback (like TRAX/game audio). No separate audio routing needed. +- **Audio is NOT wired, and it is not hiding behind the loopback** — `MediaVideoSource` decodes with + `-an`, so a video file's soundtrack is silent on stream today (the loopback only carries what the + machine actually plays). Item 10 records the two established routes; do not describe media as + audible until one is built. - **This pairs with TRAX** — TRAX is background music, media source is background video. Together they make non-Live scenes (Starting/BRB/Ending) feel polished. - **Not a full NLE** — no trimming, no multi-track, no effects. Just "play this video in the scene." - **Layers are defined in Live first** (creator 2026-11-05): this unit adds the layer and the picker in diff --git a/docs/ai/frame-pipeline.md b/docs/ai/frame-pipeline.md index 45b80c6..1960ae2 100644 --- a/docs/ai/frame-pipeline.md +++ b/docs/ai/frame-pipeline.md @@ -799,8 +799,9 @@ Full suite 290/291 passing, the sole failure the pre-existing compositor pixel t released (`GetActiveKeys`). Loop is a manager-level per-path answer: the ctor takes `Func? loopProvider` (production: "any live media reference loops"), applied before `StartAsync`, and `SetLooping(key, bool)` applies a live toggle. A file swap releases the old path after - the layer no longer references it. Volume stays unwired on purpose — sound rides the desktop loopback - (TRAX-style), so a per-file slider would be a control with no effect. + the layer no longer references it. **A video file's own soundtrack is silent today** — the decode pipe + is `-an`, so nothing plays it and the loopback has nothing to carry; wiring it (alert-style mixer feed + or TRAX-style local playback) is open, and no volume slider exists until then. - **Social bar on the output (bar bug-fix branch):** the `FramePump` takes an optional `socialBar: Func<(VideoFrame? Frame, SocialBarPosition Position)>?` seam, re-read **every frame** (so a mid-stream position flip applies immediately). The strip is pre-rasterized by `Compositor/SocialBarRenderer.cs` diff --git a/docs/ai/overlays.md b/docs/ai/overlays.md index 7446358..c37c1d2 100644 --- a/docs/ai/overlays.md +++ b/docs/ai/overlays.md @@ -99,8 +99,9 @@ published decisions (recorded in `TASKS/task-43-native-alerts.md`): `RawVideoFrameReader` feed loop, same `IFrameRateProbe` + delay pacing, both behind `IDecodeProcess`. What differs is ownership and role, not decoding: alerts are per-play and disposed at drain, media is a refcounted per-path session; alerts also run an - `f32le` audio pipe into the mixer, media deliberately sends no audio (the desktop - loopback carries it, TRAX-style); alerts play once, media loops. A shared raw-video pump + `f32le` audio pipe into the mixer, media runs none at all (`-an`) — so a media file's + soundtrack is silent today, and task-21 item 10 holds the two routes for changing that; + alerts play once, media loops. A shared raw-video pump is the obvious consolidation if a third consumer appears — do not do it for two. Fades in/out via **`FadeDurationSeconds = 0.30`** (`AlertOverlayLayer` const): fade-in by `elapsed` on straight-alpha frame copies;