TASK 21 slice 1 step 4: wire media into resolver + preview routing
Wire the MediaVideoSourceManager into the live pipeline (new partial
ViewModels/MainViewModel.Media.cs mirrors Webcam/Background partials):
readonly _mediaManager assigned in the core ctor (decode at 1920x1080 master,
dispatcher-coalesced preview), OnMediaPreviewBitmapChanged adopts the shared
bitmap onto every IsMediaSource with a matching MediaPath, a
Source { IsMediaSource, MediaPath } case in ResolveOutputFrame feeds
GetLatestFrame, and the manager is disposed on shutdown.
Source.DisplaySource now routes VideoImageSource for MediaSource (Source.cs),
so media previews/canvas show decoded video.
Compositor needs no change: BlitContent UniformToFill-scales any frame to the
element rect. Decoder/manager/seam derived-work mirrors ScreenCaptureManager
(citation in commit for slice-step-3).
Test (one unit for this change, mirroring the live-capture case):
Source_DisplaySource_IsVideoImageSource_WhenMediaSource. 14/14 media +
DisplaySource tests pass, build 0 warnings. Docs (TASKS/HANDOFF/ai.md) updated.
This commit is contained in:
+20
-12
@@ -34,13 +34,21 @@ This session:
|
||||
single shared bitmap, coalesce-to-latest). **Verified: all 10 media tests pass
|
||||
(5 manager + 3 reader + 1 decoder integration + 1), build 0 warnings.**
|
||||
|
||||
Next (slice 1 step 4 — VM wire-in): create `_mediaManager` in the ctor with
|
||||
`path => new MediaVideoSource(path, <W>, <H>, new FfmpegLocator(), new FfmpegDecodeProcess())`,
|
||||
subscribe `OnMediaPreviewBitmapChanged` (set `source.VideoImageSource` for every
|
||||
`IsMediaSource` with that `MediaPath`), add the `Source { IsMediaSource: true, MediaPath: not null }`
|
||||
case to `ResolveOutputFrame` (`MainViewModel.cs:447`), add `or SourceType.MediaSource`
|
||||
to `Source.DisplaySource` (Source.cs:119-121) so media previews route `VideoImageSource`,
|
||||
and dispose the manager on shutdown. Then slices 2 (native-FPS pacing) and 3 (loop).
|
||||
Slice 1 step 4 (wire-in) shipped: `_mediaManager` wired in the core ctor
|
||||
(`path => new MediaVideoSource(path, 1920, 1080, new FfmpegLocator(), new FfmpegDecodeProcess())`,
|
||||
dispatched preview), `OnMediaPreviewBitmapChanged` sets `VideoImageSource` on every
|
||||
`IsMediaSource` with a matching `MediaPath`, a `Source { IsMediaSource, MediaPath }`
|
||||
case in `ResolveOutputFrame` feeds `_mediaManager.GetLatestFrame`, `Source.DisplaySource`
|
||||
routes `VideoImageSource` for `MediaSource`, and the manager is disposed on shutdown.
|
||||
Lives in the new `ViewModels/MainViewModel.Media.cs` partial. **Verified: 14/14
|
||||
media + DisplaySource tests pass, build 0 warnings.**
|
||||
|
||||
Next (slice 1 step 4's remaining open end): no session acquisition yet — nothing
|
||||
`AcquireAsync`s a media path, so no frames flow until the UI picker (a later slice)
|
||||
starts a session on add and releases on remove.
|
||||
|
||||
Next (slice 2): native-FPS pacing (ffprobe FPS probe + per-frame pacing seam).
|
||||
Slice 3: loop control (restart decode on `Completed` when `MediaIsLooping`).
|
||||
|
||||
**Landmine:** a stale `testhost` can lock `ytLive.Tests`'s `ytLive.dll` and break
|
||||
`dotnet build` of the test project — kill it first (`cmd.exe /c "taskkill /PID N /F"`)
|
||||
@@ -50,11 +58,11 @@ pure tests run in WSL; user verifies the GUI suites on native Windows PowerShell
|
||||
|
||||
## Next
|
||||
|
||||
1. Commit slice 1 step 3 (this unit): manager + `IMediaFrameSource` + tests. Commit
|
||||
is already covered by `TASKS.md`/`HANDOFF.md`; this HANDOFF is the shipping doc.
|
||||
2. Slice 1 step 4: wire `_mediaManager` into the resolver + preview routing (see above).
|
||||
3. Slice 2: native-FPS pacing (ffprobe FPS probe + per-frame pacing seam).
|
||||
4. Await user push checkpoint for the pending TASK 22/21 commits.
|
||||
1. Commit slice 1 step 4 (this unit): resolver/preview wire-in + `Source.DisplaySource` media branch + tests.
|
||||
2. Slice 2: native-FPS pacing (ffprobe FPS probe + per-frame pacing seam).
|
||||
3. Slice 3: loop control (restart decode on `Completed` when `MediaIsLooping`).
|
||||
4. UI picker slice: acquire/release media sessions on add/remove (the remaining slice-1-step-4 open end).
|
||||
5. Await user push checkpoint for the pending TASK 22/21 commits.
|
||||
|
||||
## The directive (2026-08-31, user)
|
||||
|
||||
|
||||
+1
-1
@@ -118,7 +118,7 @@ public class Source : SceneElement
|
||||
|
||||
public override ImageSource? DisplaySource => IsLiveCapture
|
||||
? (VideoImageSource ?? _imageSource)
|
||||
: (Type is SourceType.ChatBox or SourceType.WebSource ? VideoImageSource : _imageSource);
|
||||
: (Type is SourceType.ChatBox or SourceType.WebSource or SourceType.MediaSource ? VideoImageSource : _imageSource);
|
||||
|
||||
public override bool IsImageSource => Type == SourceType.Image;
|
||||
|
||||
|
||||
@@ -1044,7 +1044,7 @@ The tasks below are ordered by dependency and risk. Each task builds on the prev
|
||||
4. ✅ `MediaVideoSource` implements `IMediaFrameSource`: FFmpeg raw video decoder → `VideoFrame` pipeline — spawns ffmpeg `-f rawvideo -pix_fmt bgra`, drains the pipe via pure `RawVideoFrameReader` (`Services/RawVideoFrameReader.cs`), raises `FrameAvailable`/`Completed`; lifecycle is `StartAsync`/`StopAsync`; decode process behind `IDecodeProcess`/`FfmpegDecodeProcess` seam (binary-stdout mirror of `IEncoderProcess`). Shipped 2026-08-31; refactored onto `IMediaFrameSource` with this slice.
|
||||
5. ✅ `IMediaFrameSource` + `MediaVideoSourceManager` (`Services/IMediaFrameSource.cs`, `Services/MediaVideoSourceManager.cs`): app-wide decode-session ownership refcounted by `MediaPath` with a `Func<string, IMediaFrameSource?>` factory seam; `AcquireAsync`/`ReleaseAsync`/`ReleaseAllAsync`/`GetLatestFrame`; coalesces frames onto the UI dispatcher onto a single shared `WriteableBitmap` per file (mirror of `ScreenCaptureManager`); `MediaFailed` + `PreviewBitmapChanged`. Unit tests + one integration test (bitmap share/coalesce) — `MediaVideoSourceManagerTests`, 5/5 pass.
|
||||
6. 🔶 Native-FPS pacing — frames currently emit as fast as the pipe produces them; ffprobe FPS probe + per-frame pacing seam still open (slice 2).
|
||||
7. ☐ Wire into `FramePump` resolver — `Source { Type: MediaSource }` → latest video frame
|
||||
7. ☐ Wire into `FramePump` resolver — `Source { Type: MediaSource }` → latest video frame — **resolver + preview routing + manager wired; session acquisition (start/stop on add/remove) still open (comes with the UI picker)**
|
||||
8. ☐ Wire into `SceneCompositor` — render media source as an image element at its position/size
|
||||
9. ☐ Loop control — `IsLooping` property, restart on end
|
||||
10. ☐ Volume control — per-source volume slider for audio playback
|
||||
|
||||
@@ -0,0 +1,35 @@
|
||||
using System.Linq;
|
||||
using System.Windows.Media.Imaging;
|
||||
using ytLive.Models;
|
||||
using ytLive.Services;
|
||||
|
||||
namespace ytLive.ViewModels;
|
||||
|
||||
// Media source (TASK 21): decode sessions are owned app-wide by
|
||||
// MediaVideoSourceManager, refcounted by MediaPath. Modeled on the Webcam/Background
|
||||
// partials: the readonly manager field is assigned in the core constructor, the
|
||||
// shared WriteableBitmap published per file is adopted by every media Source whose
|
||||
// MediaPath matches, and ResolveOutputFrame feeds the live compositor.
|
||||
public partial class MainViewModel
|
||||
{
|
||||
// Decode resolution: the 1920x1080 master. Media frames are scaled to the
|
||||
// element rect by the compositor (UniformToFill), so this is the capture size.
|
||||
private const int MediaDecodeWidth = 1920;
|
||||
private const int MediaDecodeHeight = 1080;
|
||||
|
||||
private readonly MediaVideoSourceManager _mediaManager;
|
||||
|
||||
// MediaVideoSourceManager publishes one shared WriteableBitmap per file; every
|
||||
// media Source pointing at that path picks it up for the preview/canvas.
|
||||
private void OnMediaPreviewBitmapChanged(string path, WriteableBitmap bitmap)
|
||||
{
|
||||
foreach (var source in Scenes.SelectMany(s => s.Elements.OfType<Source>())
|
||||
.Where(s => s.IsMediaSource && s.MediaPath == path))
|
||||
source.VideoImageSource = bitmap;
|
||||
}
|
||||
|
||||
// Called from the encoder error path is out of scope (TASK 21 sees the decoder
|
||||
// fail at startup); log to the debug console for now, surfaced via the UI later.
|
||||
private static void OnMediaFailed(string path, string message)
|
||||
=> System.Diagnostics.Debug.WriteLine($"MediaSource: '{path}' failed: {message}");
|
||||
}
|
||||
@@ -282,6 +282,14 @@ public partial class MainViewModel : ViewModelBase
|
||||
_screenCaptureManager.CaptureFailed += (key, message) =>
|
||||
AppLog.Write($"ScreenCaptureManager: capture '{key}' failed: {message}");
|
||||
|
||||
_mediaManager = new MediaVideoSourceManager(
|
||||
path => new MediaVideoSource(
|
||||
path, MediaDecodeWidth, MediaDecodeHeight,
|
||||
new FfmpegLocator(), new FfmpegDecodeProcess()),
|
||||
System.Windows.Application.Current?.Dispatcher);
|
||||
_mediaManager.PreviewBitmapChanged += OnMediaPreviewBitmapChanged;
|
||||
_mediaManager.MediaFailed += OnMediaFailed;
|
||||
|
||||
_chatLayer = new ChatOverlayLayer(new ChatBoxRenderer());
|
||||
|
||||
_sceneGraph = new SceneGraph();
|
||||
@@ -397,6 +405,7 @@ public partial class MainViewModel : ViewModelBase
|
||||
_framePump.Dispose();
|
||||
_cameraManager.Dispose();
|
||||
_screenCaptureManager.Dispose();
|
||||
_mediaManager.Dispose();
|
||||
_webView2Manager?.Dispose();
|
||||
_layoutStore.Dispose();
|
||||
_musicPlayer.Dispose();
|
||||
@@ -452,6 +461,7 @@ public partial class MainViewModel : ViewModelBase
|
||||
Source { IsLiveCapture: true, CaptureKey: not null } live => _screenCaptureManager.GetLatestFrame(live.CaptureKey),
|
||||
Source { Type: SourceType.ChatBox } chatBox => RenderChatBox(chatBox),
|
||||
Source { Type: SourceType.WebSource } web => _webView2Manager?.GetLatestFrame(web.Id),
|
||||
Source { IsMediaSource: true, MediaPath: not null } media => _mediaManager.GetLatestFrame(media.MediaPath),
|
||||
Source { AssetId: not null } image => StaticPixelCache.Get(image.AssetId),
|
||||
_ => null,
|
||||
};
|
||||
|
||||
@@ -680,8 +680,10 @@ seam:** `Func<Scene?>`, `Func<SceneElement, VideoFrame?>` resolver, `Func<Compos
|
||||
(`Services/MediaVideoSourceManager.cs`): refcounted by `MediaPath`, `Func<string, IMediaFrameSource?>` factory
|
||||
seam (mirror of `ScreenCaptureManager`), `AcquireAsync`/`ReleaseAsync`/`ReleaseAllAsync`/`GetLatestFrame`,
|
||||
coalescing each file's frames onto the UI dispatcher onto one shared `WriteableBitmap`. Frames emit as fast as
|
||||
the pipe produces them; native-FPS pacing (ffprobe probe) and the resolver/preview wire-in (`ResolveOutputFrame`
|
||||
case + `Source.DisplaySource`) are the next slices.
|
||||
the pipe produces them. Wired into the live pipeline (slice 1 step 4): `ResolveOutputFrame` reads
|
||||
`_mediaManager.GetLatestFrame(MediaPath)` for a `MediaSource`, and `Source.DisplaySource`
|
||||
(`Models/Source.cs`) routes `VideoImageSource` for `MediaSource` so the preview/canvas show decoded video.
|
||||
Still open: session acquisition on add/remove (UI picker), native-FPS pacing (ffprobe probe), loop control.
|
||||
- **Social bar on the output (bar bug-fix branch):** the `FramePump` takes an optional
|
||||
`socialBar: Func<(VideoFrame? Frame, SocialBarPosition Position)>?` seam, re-read **every frame** (so a
|
||||
mid-stream position flip applies immediately). The strip is pre-rasterized by `Compositor/SocialBarRenderer.cs`
|
||||
|
||||
@@ -93,6 +93,20 @@ public class BackgroundTests
|
||||
Assert.Same(bitmap, source.DisplaySource);
|
||||
}
|
||||
|
||||
// TASK 21: a media source's DisplaySource routes its live video frames (like
|
||||
// Display/Window capture), so media previews show decoded video, not the stale
|
||||
// default art.
|
||||
[Fact]
|
||||
public void Source_DisplaySource_IsVideoImageSource_WhenMediaSource()
|
||||
{
|
||||
var source = new Source { Type = SourceType.MediaSource };
|
||||
var bitmap = new System.Windows.Media.Imaging.WriteableBitmap(
|
||||
2, 2, 96, 96, System.Windows.Media.PixelFormats.Bgra32, null);
|
||||
source.VideoImageSource = bitmap;
|
||||
|
||||
Assert.Same(bitmap, source.DisplaySource);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Source_DisplaySource_FallsBackToImage_WhenLiveCaptureWithNoVideoImage()
|
||||
{
|
||||
|
||||
Reference in New Issue
Block a user