docs(media): correct the audio claim — a video file's sound is silent, not loopback-carried

This commit is contained in:
2026-11-05 07:22:15 -08:00
parent db563c8644
commit 776f84fb88
5 changed files with 23 additions and 11 deletions
+1 -1
View File
@@ -746,7 +746,7 @@
<CheckBox Style="{StaticResource PropertyPillCheck}" Margin="10,4,0,4" <CheckBox Style="{StaticResource PropertyPillCheck}" Margin="10,4,0,4"
Content="Loop" IsChecked="{Binding SelectedElement.MediaIsLooping, Mode=TwoWay}" Content="Loop" IsChecked="{Binding SelectedElement.MediaIsLooping, Mode=TwoWay}"
ToolTip="Restart from the beginning at the end of the file — for starting-soon and BRB loops"/> ToolTip="Restart from the beginning at the end of the file — for starting-soon and BRB loops"/>
<TextBlock Text="Audio plays through your desktop audio, like TRAX." <TextBlock Text="Picture only — a video's own sound is not played yet."
Style="{StaticResource PropertyHint}"/> Style="{StaticResource PropertyHint}"/>
</StackPanel> </StackPanel>
+3 -1
View File
@@ -7,7 +7,9 @@
2026-11-04. 2026-11-04.
- New test `MediaSourceAcquisitionTests` (add → one decoder starts + frames reach the layer + loop flip - New test `MediaSourceAcquisitionTests` (add → one decoder starts + frames reach the layer + loop flip
reaches the decoder + remove → stops). App build is 0 warnings. reaches the decoder + remove → stops). App build is 0 warnings.
- Volume is deliberately unwired: media audio rides the desktop loopback, so a slider would do nothing. - **Media is picture-only:** `MediaVideoSource` decodes with `-an`, so a video file's own soundtrack
is silent on stream — the loopback does not carry it because nothing plays it. Two routes recorded
in `task-21` item 10 (alert-style mixer feed vs TRAX-style local play); no volume slider until one exists.
## Next action ## Next action
+13 -5
View File
@@ -7,7 +7,8 @@
### Status: ✅ Complete 2026-11-05 — a video file added in the (+) menu decodes, composites into the ### Status: ✅ Complete 2026-11-05 — a video file added in the (+) menu decodes, composites into the
preview and the stream, loops on the creator's toggle, and stops when the layer or its last scene preview and the stream, loops on the creator's toggle, and stops when the layer or its last scene
reference goes away. Audio is the desktop loopback (TRAX-style); a per-file volume route is not built. reference goes away. **Picture only: the file's own audio is not played anywhere** (the decode pipe
passes `-an`), so a video with a soundtrack is silent on stream — the audio route is open, item 10.
1. ✅ `MediaSourceType` enum: `Video`, `Audio` (audio-only files via media source) 1. ✅ `MediaSourceType` enum: `Video`, `Audio` (audio-only files via media source)
2. ✅ `SourceType.MediaSource` addition to the enum 2. ✅ `SourceType.MediaSource` addition to the enum
@@ -24,9 +25,13 @@ reference goes away. Audio is the desktop loopback (TRAX-style); a per-file volu
when `Looping` (fresh process per pass). `MediaIsLooping` reaches the session through the manager's when `Looping` (fresh process per pass). `MediaIsLooping` reaches the session through the manager's
per-path `loopProvider` at start and `SetLooping` on a live toggle. Rule (recorded): the path's per-path `loopProvider` at start and `SetLooping` on a live toggle. Rule (recorded): the path's
session loops if **any** live media reference has it on, because one decoder is shared per path. session loops if **any** live media reference has it on, because one decoder is shared per path.
10. ⏸ Volume control — deliberately not wired: media audio is captured by the desktop loopback like 10. ⏸ File audio + volume — **the file's soundtrack is silent everywhere today**: the decode pipe
TRAX, so a per-file slider would be a control with no audible effect. Revisit only with a local passes `-an`, so nothing plays it and the desktop loopback has nothing to carry. Two established
audio route for the file's own stream. routes when this is taken: **alert-style** (a second `f32le` pipe into the mixer —
`AlertClipDecoder.RunAudioAsync` is the working model, independent of desktop capture) or
**TRAX-style** (`MusicPlayer` plays it to the default device, so the loopback, the desktop fader
and ducking all apply — at the cost of a second clock to hold in sync with the video). Product
call; not started, and no volume slider until one of them exists.
11. ✅ UI: `AddMediaCommand` (+ menu → "Video File", video-filtered `OpenFileDialog`), `ChangeMediaFileCommand` 11. ✅ UI: `AddMediaCommand` (+ menu → "Video File", video-filtered `OpenFileDialog`), `ChangeMediaFileCommand`
("Change…" in the properties panel), loop toggle, video name display. ("Change…" in the properties panel), loop toggle, video name display.
12. ✅ Schema migration for media source settings (file path, loop, volume) — shipped with item 3. 12. ✅ Schema migration for media source settings (file path, loop, volume) — shipped with item 3.
@@ -36,7 +41,10 @@ reference goes away. Audio is the desktop loopback (TRAX-style); a per-file volu
### Design decisions ### Design decisions
- **FFmpeg handles all formats** — no codec-specific code. FFmpeg already in the project. - **FFmpeg handles all formats** — no codec-specific code. FFmpeg already in the project.
- **Audio plays through the desktop channel** — media source audio is captured by the WASAPI loopback (like TRAX/game audio). No separate audio routing needed. - **Audio is NOT wired, and it is not hiding behind the loopback** — `MediaVideoSource` decodes with
`-an`, so a video file's soundtrack is silent on stream today (the loopback only carries what the
machine actually plays). Item 10 records the two established routes; do not describe media as
audible until one is built.
- **This pairs with TRAX** — TRAX is background music, media source is background video. Together they make non-Live scenes (Starting/BRB/Ending) feel polished. - **This pairs with TRAX** — TRAX is background music, media source is background video. Together they make non-Live scenes (Starting/BRB/Ending) feel polished.
- **Not a full NLE** — no trimming, no multi-track, no effects. Just "play this video in the scene." - **Not a full NLE** — no trimming, no multi-track, no effects. Just "play this video in the scene."
- **Layers are defined in Live first** (creator 2026-11-05): this unit adds the layer and the picker in - **Layers are defined in Live first** (creator 2026-11-05): this unit adds the layer and the picker in
+3 -2
View File
@@ -799,8 +799,9 @@ Full suite 290/291 passing, the sole failure the pre-existing compositor pixel t
released (`GetActiveKeys`). Loop is a manager-level per-path answer: the ctor takes released (`GetActiveKeys`). Loop is a manager-level per-path answer: the ctor takes
`Func<string,bool>? loopProvider` (production: "any live media reference loops"), applied before `Func<string,bool>? loopProvider` (production: "any live media reference loops"), applied before
`StartAsync`, and `SetLooping(key, bool)` applies a live toggle. A file swap releases the old path after `StartAsync`, and `SetLooping(key, bool)` applies a live toggle. A file swap releases the old path after
the layer no longer references it. Volume stays unwired on purpose — sound rides the desktop loopback the layer no longer references it. **A video file's own soundtrack is silent today** — the decode pipe
(TRAX-style), so a per-file slider would be a control with no effect. is `-an`, so nothing plays it and the loopback has nothing to carry; wiring it (alert-style mixer feed
or TRAX-style local playback) is open, and no volume slider exists until then.
- **Social bar on the output (bar bug-fix branch):** the `FramePump` takes an optional - **Social bar on the output (bar bug-fix branch):** the `FramePump` takes an optional
`socialBar: Func<(VideoFrame? Frame, SocialBarPosition Position)>?` seam, re-read **every frame** (so a `socialBar: Func<(VideoFrame? Frame, SocialBarPosition Position)>?` seam, re-read **every frame** (so a
mid-stream position flip applies immediately). The strip is pre-rasterized by `Compositor/SocialBarRenderer.cs` mid-stream position flip applies immediately). The strip is pre-rasterized by `Compositor/SocialBarRenderer.cs`
+3 -2
View File
@@ -99,8 +99,9 @@ published decisions (recorded in `TASKS/task-43-native-alerts.md`):
`RawVideoFrameReader` feed loop, same `IFrameRateProbe` + delay pacing, both behind `RawVideoFrameReader` feed loop, same `IFrameRateProbe` + delay pacing, both behind
`IDecodeProcess`. What differs is ownership and role, not decoding: alerts are per-play `IDecodeProcess`. What differs is ownership and role, not decoding: alerts are per-play
and disposed at drain, media is a refcounted per-path session; alerts also run an and disposed at drain, media is a refcounted per-path session; alerts also run an
`f32le` audio pipe into the mixer, media deliberately sends no audio (the desktop `f32le` audio pipe into the mixer, media runs none at all (`-an`) — so a media file's
loopback carries it, TRAX-style); alerts play once, media loops. A shared raw-video pump soundtrack is silent today, and task-21 item 10 holds the two routes for changing that;
alerts play once, media loops. A shared raw-video pump
is the obvious consolidation if a third consumer appears — do not do it for two. is the obvious consolidation if a third consumer appears — do not do it for two.
Fades in/out via **`FadeDurationSeconds = 0.30`** Fades in/out via **`FadeDurationSeconds = 0.30`**
(`AlertOverlayLayer` const): fade-in by `elapsed` on straight-alpha frame copies; (`AlertOverlayLayer` const): fade-in by `elapsed` on straight-alpha frame copies;