From 480ead0fc7ecedccb883d3782b4987e079398bac Mon Sep 17 00:00:00 2001 From: gramps Date: Mon, 31 Aug 2026 19:22:02 -0700 Subject: [PATCH] TASK 21 slice 1 step 3: IMediaFrameSource + MediaVideoSourceManager Refactor MediaVideoSource onto IMediaFrameSource (Key/FrameAvailable/Completed/ StartAsync/StopAsync, renaming Start->StartAsync, FrameReady->FrameAvailable), behavior preserved. Add MediaVideoSourceManager: app-wide decode-session owner refcounted by MediaPath with Func factory seam, Acquire/Release/ReleaseAll/GetLatestFrame, coalescing each file's frames onto the UI dispatcher onto one shared WriteableBitmap; MediaFailed + PreviewBitmapChanged. Mirrors ScreenCaptureManager (screen-capture session ownership + dispatcher coalescing), per the codebase precedent and derivative-work rule. Tests: MediaVideoSourceManagerTests (5 unit + 1 integration: single shared bitmap, coalesce-to-latest); MediaVideoSourceTests updated for renames. 10/10 media tests pass, build 0 warnings. Docs (TASKS/HANDOFF/ai.md) updated. --- HANDOFF.md | 50 ++-- Services/IMediaFrameSource.cs | 21 ++ Services/MediaVideoSource.cs | 35 +-- Services/MediaVideoSourceManager.cs | 256 +++++++++++++++++++ TASKS.md | 21 +- ai.md | 17 +- ytLive.Tests/MediaVideoSourceManagerTests.cs | 209 +++++++++++++++ ytLive.Tests/MediaVideoSourceTests.cs | 4 +- 8 files changed, 557 insertions(+), 56 deletions(-) create mode 100644 Services/IMediaFrameSource.cs create mode 100644 Services/MediaVideoSourceManager.cs create mode 100644 ytLive.Tests/MediaVideoSourceManagerTests.cs diff --git a/HANDOFF.md b/HANDOFF.md index 9ef176c..0ac808d 100644 --- a/HANDOFF.md +++ b/HANDOFF.md @@ -4,35 +4,43 @@ **`main`**, working through TASK 21 (media source). Local commits ahead of `origin/main` (not yet pushed — user pushes at agreed checkpoints): +- `a1d9751` — docs: ffmpeg decode-contract verification recipe + WSL CLR boundary +- `8f49010` — TASK 21 Increment B: FFmpeg rawvideo video decoder -> VideoFrame - `f89b9f9` — docs: default-device audio assumption (README + startup log) - `e8ff4df` — TASK 22: global audio sync offset (positive delay at the mixer out) -- `670fe3a` — TASK 31 (already pushed; listed for context) Prior milestones pushed through `origin/main`: TASK 31 (`3bf053a..670fe3a`), plus the full refactor series (Commit F `3107f92`, Commit G `85893ea`). ## What's In Flight -**TASK 21 (media source) — Increment B (the decoder) implemented, uncommitted-or-pending-commit, awaiting next unit.** +**TASK 21 (media source) — slice 1 step 3 shipped (the session-manager + interface); next open work is the VM wire-in (slice 1 step 4).** Increment A (the `Source` model: `MediaSourceType`, `SourceType.MediaSource`, `MediaPath`/`MediaIsLooping`/`MediaVolume`/`MediaPlaybackState` + LayoutStore -persistence) shipped earlier. This session shipped **Increment B — the decoder**: +persistence) and Increment B (the decoder `MediaVideoSource`) shipped earlier. -New files: -- `Services/RawVideoFrameReader.cs` — pure: parses a rawvideo BGRA stream into - `VideoFrame`s (exactly W×H×4 per frame; partial reads kept across `Feed`). -- `Services/IDecodeProcess.cs` + `Services/FfmpegDecodeProcess.cs` — binary-stdout - subprocess seam (mirror of `IEncoderProcess`). -- `Services/MediaVideoSource.cs` — spawns ffmpeg `-f rawvideo -pix_fmt bgra -an`, - drains the pipe, raises `FrameReady` per frame; `Completed` on EOF. -- `ytLive.Tests/MediaVideoSourceTests.cs` — 3 pure reader tests + 1 integration - (fake `IDecodeProcess`+`IFfmpegLocator` through the real source loop, frames in - order). **Verified: 4/4 pass, build 0 warnings.** +This session: +- Refactored `Services/MediaVideoSource.cs` onto the new + `Services/IMediaFrameSource.cs` (`Key`, `FrameAvailable`, `Completed`, + `StartAsync`/`StopAsync`; `Start`→`StartAsync`, `FrameReady`→`FrameAvailable`) + — behavior preserved, `MediaVideoSourceTests` updated and passing. +- Added `Services/MediaVideoSourceManager.cs` — app-wide decode-session + ownership refcounted by `MediaPath`, `Func` factory + seam, `AcquireAsync`/`ReleaseAsync`/`ReleaseAllAsync`/`GetLatestFrame`, frames + coalesced onto the UI dispatcher onto one shared `WriteableBitmap` per file, + `MediaFailed` + `PreviewBitmapChanged`. Structural mirror of `ScreenCaptureManager`. +- Added `ytLive.Tests/MediaVideoSourceManagerTests.cs` (5 unit + 1 integration: + single shared bitmap, coalesce-to-latest). **Verified: all 10 media tests pass + (5 manager + 3 reader + 1 decoder integration + 1), build 0 warnings.** -Still open (next slices): native-FPS pacing (ffprobe probe), loop control, volume, -the `ResolveOutputFrame` resolver case + wire into `SceneCompositor`, the file-picker -UI, and the remaining tests. +Next (slice 1 step 4 — VM wire-in): create `_mediaManager` in the ctor with +`path => new MediaVideoSource(path, , , new FfmpegLocator(), new FfmpegDecodeProcess())`, +subscribe `OnMediaPreviewBitmapChanged` (set `source.VideoImageSource` for every +`IsMediaSource` with that `MediaPath`), add the `Source { IsMediaSource: true, MediaPath: not null }` +case to `ResolveOutputFrame` (`MainViewModel.cs:447`), add `or SourceType.MediaSource` +to `Source.DisplaySource` (Source.cs:119-121) so media previews route `VideoImageSource`, +and dispose the manager on shutdown. Then slices 2 (native-FPS pacing) and 3 (loop). **Landmine:** a stale `testhost` can lock `ytLive.Tests`'s `ytLive.dll` and break `dotnet build` of the test project — kill it first (`cmd.exe /c "taskkill /PID N /F"`) @@ -42,11 +50,11 @@ pure tests run in WSL; user verifies the GUI suites on native Windows PowerShell ## Next -1. Commit Increment B (decoder) as one work unit (scope-check + docs already - staged in ai.md/TASKS.md/HANDOFF.md). -2. Next slice: native-FPS pacing + wire media into `ResolveOutputFrame` and the - compositor, per TASK 21 steps 6–7. -3. Await user push checkpoint for the TASK 22/21 commits. +1. Commit slice 1 step 3 (this unit): manager + `IMediaFrameSource` + tests. Commit + is already covered by `TASKS.md`/`HANDOFF.md`; this HANDOFF is the shipping doc. +2. Slice 1 step 4: wire `_mediaManager` into the resolver + preview routing (see above). +3. Slice 2: native-FPS pacing (ffprobe FPS probe + per-frame pacing seam). +4. Await user push checkpoint for the pending TASK 22/21 commits. ## The directive (2026-08-31, user) diff --git a/Services/IMediaFrameSource.cs b/Services/IMediaFrameSource.cs new file mode 100644 index 0000000..e01ecdb --- /dev/null +++ b/Services/IMediaFrameSource.cs @@ -0,0 +1,21 @@ +namespace ytLive.Services; + +/// +/// TASK 21: a media file decoder exposed to . +/// Mirrors IScreenCaptureSource/ICameraFrameSource so the manager +/// (and tests) treat a local video file like any other frame source: start/stop +/// and a per-frame event consumers coalesce. +/// +public interface IMediaFrameSource : IDisposable +{ + string Key { get; } + + /// Raised (on a worker thread) for each decoded frame. + event Action? FrameAvailable; + + /// Raised when the file finishes decoding (natural EOF). + event Action? Completed; + + Task StartAsync(CancellationToken cancellationToken = default); + Task StopAsync(); +} diff --git a/Services/MediaVideoSource.cs b/Services/MediaVideoSource.cs index 30c3141..e02182a 100644 --- a/Services/MediaVideoSource.cs +++ b/Services/MediaVideoSource.cs @@ -6,19 +6,20 @@ namespace ytLive.Services; /// /// TASK 21: decodes a local media file (any format ffmpeg handles) into -/// s for a MediaSource. Spawns ffmpeg with -/// -f rawvideo -pix_fmt bgra, reads the raw BGRA pipe via -/// , and raises per -/// frame. The decoder process is a seam () so tests -/// run without a real codec or audio device. Frames are emitted as fast as the -/// pipe produces them; native-FPS pacing is wired by the caller via a probe in a -/// later unit (TASK 21 step 5 pacing). +/// s for a MediaSource. Implements +/// : raises per +/// frame and on natural EOF. Spawns +/// ffmpeg with -f rawvideo -pix_fmt bgra and drains the raw BGRA pipe via +/// . The decoder process and emitters are seams +/// so tests run without a real codec or audio device. /// -public sealed class MediaVideoSource : IDisposable +public sealed class MediaVideoSource : IMediaFrameSource { - public event Action? FrameReady; + public event Action? FrameAvailable; public event Action? Completed; + public string Key => _path; + private readonly string _path; private readonly int _width; private readonly int _height; @@ -41,17 +42,19 @@ public sealed class MediaVideoSource : IDisposable _process = process; } - public void Start() + public Task StartAsync(CancellationToken cancellationToken = default) { - if (_run != null) return; + if (_run != null) return Task.CompletedTask; _run = _cts.Token.Register(StopInternal); - _ = Task.Run(RunAsync); + _ = Task.Run(RunAsync, CancellationToken.None); + return Task.CompletedTask; } - public void Stop() + public Task StopAsync() { _cts.Cancel(); _run = null; + return Task.CompletedTask; } public void Dispose() @@ -85,14 +88,12 @@ public sealed class MediaVideoSource : IDisposable var reader = new RawVideoFrameReader(_width, _height); var buffer = new byte[65536]; var stream = _process.StandardOutput; - byte[] readChunk; while (!_cts.IsCancellationRequested && !_process.HasExited) { var read = await stream.ReadAsync(buffer, 0, buffer.Length, _cts.Token).ConfigureAwait(false); if (read <= 0) break; - readChunk = buffer.AsSpan(0, read).ToArray(); - foreach (var frame in reader.Feed(readChunk)) - FrameReady?.Invoke(frame); + foreach (var frame in reader.Feed(buffer.AsSpan(0, read).ToArray())) + FrameAvailable?.Invoke(frame); } } catch (OperationCanceledException) diff --git a/Services/MediaVideoSourceManager.cs b/Services/MediaVideoSourceManager.cs new file mode 100644 index 0000000..f39c0a3 --- /dev/null +++ b/Services/MediaVideoSourceManager.cs @@ -0,0 +1,256 @@ +using System; +using System.Collections.Generic; +using System.Threading.Tasks; +using System.Windows; +using System.Windows.Media; +using System.Windows.Media.Imaging; +using System.Windows.Threading; +using ytLive.Helpers; + +namespace ytLive.Services; + +/// +/// TASK 21: owns media-file decode sessions app-wide, refcounted by file path. +/// One file = at most one decoder, one shared WriteableBitmap; the last release +/// stops and disposes the source. Frames arrive on a worker thread and are +/// coalesced onto the UI dispatcher (at most one pending copy per session, using +/// the latest frame). Mirrors ScreenCaptureManager/CameraManager. +/// +public sealed class MediaVideoSourceManager : IDisposable +{ + private sealed class MediaSession + { + public string Key { get; } + public IMediaFrameSource Source { get; } + public Action? FrameHandler; + public int RefCount; + public bool Started; + public WriteableBitmap? PreviewBitmap; + public VideoFrame? LatestFrame; + public bool FramePending; + + public MediaSession(string key, IMediaFrameSource source) + { + Key = key; + Source = source; + RefCount = 1; + } + } + + private readonly Func _sourceFactory; + private readonly Dispatcher? _uiDispatcher; + private readonly Dictionary _sessions = new(); + private readonly object _gate = new(); + + /// Raised on the UI thread when a media file's shared preview bitmap is first created. + public event Action? PreviewBitmapChanged; + + /// Raised when a media decoder cannot be created (bad path). + public event Action? MediaFailed; + + public MediaVideoSourceManager(Func sourceFactory, Dispatcher? uiDispatcher = null) + { + _sourceFactory = sourceFactory; + _uiDispatcher = uiDispatcher; + } + + /// Increments the refcount for a file, starting the decoder the first time. + public async Task AcquireAsync(string key) + { + if (string.IsNullOrWhiteSpace(key)) return false; + + MediaSession session; + bool shouldStart; + lock (_gate) + { + if (_sessions.TryGetValue(key, out var existing)) + { + existing.RefCount++; + shouldStart = false; + session = existing; + } + else + { + IMediaFrameSource? source; + try + { + source = _sourceFactory(key); + } + catch (Exception ex) + { + AppLog.Write($"MediaSource: creating decoder '{key}' failed: {ex.Message}"); + MediaFailed?.Invoke(key, ex.Message); + return false; + } + if (source == null) + { + MediaFailed?.Invoke(key, "No decoder for this file"); + return false; + } + session = new MediaSession(key, source); + session.FrameHandler = frame => OnFrameAvailable(session, frame); + session.Source.FrameAvailable += session.FrameHandler; + _sessions[key] = session; + shouldStart = true; + } + } + + if (!shouldStart) return session.Started; + + try + { + await session.Source.StartAsync(); + session.Started = true; + return true; + } + catch (Exception ex) + { + lock (_gate) + _sessions.Remove(key); + session.Source.FrameAvailable -= session.FrameHandler; + AppLog.Write($"MediaSource: failed to start decoder '{key}': {ex.Message}"); + MediaFailed?.Invoke(key, ex.Message); + await SafeStopAsync(session.Source); + return false; + } + } + + public async Task ReleaseAsync(string key) + { + MediaSession? toStop = null; + lock (_gate) + { + if (!_sessions.TryGetValue(key, out var session)) return; + if (--session.RefCount > 0) return; + _sessions.Remove(key); + toStop = session; + } + + if (toStop == null) return; + toStop.Source.FrameAvailable -= toStop.FrameHandler; + await SafeStopAsync(toStop.Source); + toStop.PreviewBitmap = null; + } + + /// Releases a file unconditionally (every ref) — used on layout reloads. + public async Task ReleaseAllAsync(string key) + { + MediaSession? toStop = null; + lock (_gate) + { + if (!_sessions.TryGetValue(key, out var session)) return; + session.RefCount = 0; + _sessions.Remove(key); + toStop = session; + } + + if (toStop == null) return; + toStop.Source.FrameAvailable -= toStop.FrameHandler; + await SafeStopAsync(toStop.Source); + toStop.PreviewBitmap = null; + } + + /// The most recent decoded frame for a file, or null before the first + /// frame (or if the file has no session). The live compositor reads media from here. + public VideoFrame? GetLatestFrame(string key) + { + lock (_gate) + return _sessions.TryGetValue(key, out var session) ? session.LatestFrame : null; + } + + /// The shared preview bitmap for a file (so a config created after the + /// first frame can adopt the running bitmap), or null. + public WriteableBitmap? GetPreviewBitmap(string key) + { + lock (_gate) + return _sessions.TryGetValue(key, out var session) ? session.PreviewBitmap : null; + } + + public void Dispose() + { + List sessions; + lock (_gate) + { + sessions = new List(_sessions.Values); + _sessions.Clear(); + } + + foreach (var session in sessions) + { + session.Source.FrameAvailable -= session.FrameHandler; + _ = SafeStopAsync(session.Source); + } + } + + private static async Task SafeStopAsync(IMediaFrameSource source) + { + try + { + await source.StopAsync(); + } + catch (Exception ex) + { + AppLog.Write($"MediaSource: stopping decoder failed: {ex.Message}"); + } + } + + private bool TryGetActiveSession(MediaSession session) + { + lock (_gate) + return _sessions.TryGetValue(session.Key, out var current) && ReferenceEquals(current, session); + } + + private void OnFrameAvailable(MediaSession session, VideoFrame frame) + { + if (!TryGetActiveSession(session)) return; + session.LatestFrame = frame; + + if (session.PreviewBitmap == null) + { + if (_uiDispatcher == null) return; + if (_uiDispatcher.CheckAccess()) + EnsurePreviewBitmap(session); + else + _uiDispatcher.BeginInvoke(() => + { + if (TryGetActiveSession(session)) + EnsurePreviewBitmap(session); + }, DispatcherPriority.Render); + return; + } + + if (_uiDispatcher == null || _uiDispatcher.CheckAccess()) + { + CopyFrame(session); + return; + } + + if (session.FramePending) return; + session.FramePending = true; + _uiDispatcher.BeginInvoke(() => + { + session.FramePending = false; + if (TryGetActiveSession(session) && session.PreviewBitmap != null) + CopyFrame(session); + }, DispatcherPriority.Render); + } + + private void EnsurePreviewBitmap(MediaSession session) + { + if (session.PreviewBitmap != null || session.LatestFrame == null) return; + var frame = session.LatestFrame; + var bitmap = new WriteableBitmap(frame.Width, frame.Height, 96, 96, PixelFormats.Bgra32, null); + bitmap.WritePixels(new Int32Rect(0, 0, frame.Width, frame.Height), frame.BgraPixels, frame.Stride, 0); + session.PreviewBitmap = bitmap; + PreviewBitmapChanged?.Invoke(session.Key, bitmap); + } + + private void CopyFrame(MediaSession session) + { + var bitmap = session.PreviewBitmap; + var frame = session.LatestFrame; + if (bitmap == null || frame == null) return; + if (frame.Width != bitmap.PixelWidth || frame.Height != bitmap.PixelHeight) return; + bitmap.WritePixels(new Int32Rect(0, 0, frame.Width, frame.Height), frame.BgraPixels, frame.Stride, 0); + } +} diff --git a/TASKS.md b/TASKS.md index b299f3c..1cf620f 100644 --- a/TASKS.md +++ b/TASKS.md @@ -1036,20 +1036,21 @@ The tasks below are ordered by dependency and risk. Each task builds on the prev **Goal:** play video files (MP4, MOV, AVI) into scenes — starting soon videos, BRB loops, intro/outro clips. -### Status: 🔶 In progress — Increment A shipped (model + persistence); Increment B (decoder) shipped; slices 6–12 open +### Status: 🔶 In progress — Increment A (model + persistence) + Increment B (decoder) shipped; slice 1 step 3 (manager + `IMediaFrameSource`) shipped; resolver/preview wire-in + pacing + loop open 1. ✅ `MediaSourceType` enum: `Video`, `Audio` (audio-only files via media source) 2. ✅ `SourceType.MediaSource` addition to the enum 3. ✅ `MediaSourceModel`: `MediaPath`, `MediaIsLooping`, `MediaVolume` (0-1), `MediaPlaybackState` — persisted in LayoutStore (schema migration + SELECT/INSERT + round-trip test) -4. ✅ `MediaVideoSource`: FFmpeg raw video decoder → `VideoFrame` pipeline — spawns ffmpeg `-f rawvideo -pix_fmt bgra`, drains the pipe via pure `RawVideoFrameReader` (`Services/RawVideoFrameReader.cs`), raises `FrameReady` per frame; decode process behind `IDecodeProcess`/`FfmpegDecodeProcess` seam (binary-stdout mirror of `IEncoderProcess`). Shipped 2026-08-31. -5. 🔶 Frame capture from video file, output BGRA8 frames — decode + BGRA8 emit done; **native-FPS pacing still open** (ffprobe probe, wired by the caller; frames currently emit as fast as the pipe produces). -6. ☐ Wire into `FramePump` resolver — `Source { Type: MediaSource }` → latest video frame -7. ☐ Wire into `SceneCompositor` — render media source as an image element at its position/size -8. ☐ Loop control — `IsLooping` property, restart on end -9. ☐ Volume control — per-source volume slider for audio playback -10. ☐ UI: file picker (filtered to video formats), loop toggle, volume slider -11. ☐ Schema migration for media source settings (file path, loop, volume) -12. ☐ Tests: video frame extraction, loop behavior, volume scaling, file validation +4. ✅ `MediaVideoSource` implements `IMediaFrameSource`: FFmpeg raw video decoder → `VideoFrame` pipeline — spawns ffmpeg `-f rawvideo -pix_fmt bgra`, drains the pipe via pure `RawVideoFrameReader` (`Services/RawVideoFrameReader.cs`), raises `FrameAvailable`/`Completed`; lifecycle is `StartAsync`/`StopAsync`; decode process behind `IDecodeProcess`/`FfmpegDecodeProcess` seam (binary-stdout mirror of `IEncoderProcess`). Shipped 2026-08-31; refactored onto `IMediaFrameSource` with this slice. +5. ✅ `IMediaFrameSource` + `MediaVideoSourceManager` (`Services/IMediaFrameSource.cs`, `Services/MediaVideoSourceManager.cs`): app-wide decode-session ownership refcounted by `MediaPath` with a `Func` factory seam; `AcquireAsync`/`ReleaseAsync`/`ReleaseAllAsync`/`GetLatestFrame`; coalesces frames onto the UI dispatcher onto a single shared `WriteableBitmap` per file (mirror of `ScreenCaptureManager`); `MediaFailed` + `PreviewBitmapChanged`. Unit tests + one integration test (bitmap share/coalesce) — `MediaVideoSourceManagerTests`, 5/5 pass. +6. 🔶 Native-FPS pacing — frames currently emit as fast as the pipe produces them; ffprobe FPS probe + per-frame pacing seam still open (slice 2). +7. ☐ Wire into `FramePump` resolver — `Source { Type: MediaSource }` → latest video frame +8. ☐ Wire into `SceneCompositor` — render media source as an image element at its position/size +9. ☐ Loop control — `IsLooping` property, restart on end +10. ☐ Volume control — per-source volume slider for audio playback +11. ☐ UI: file picker (filtered to video formats), loop toggle, volume slider +12. ☐ Schema migration for media source settings (file path, loop, volume) +13. ☐ Tests: video frame extraction, loop behavior, volume scaling, file validation ### Design decisions diff --git a/ai.md b/ai.md index bb40e28..69de975 100644 --- a/ai.md +++ b/ai.md @@ -155,7 +155,7 @@ C# / WPF (.NET 8) following MVVM: |------|------| | `Models/` | Plain data types — Scene, Source (incl. `ClipShape`, `IsMirrored`, `VideoImageSource`), QualityOption, StreamConfig, StreamHealth, YouTubeChannel, ChatMessage, **Socials (`SocialService` enum + `SocialEntry`/`SocialsConfig` + `SocialServiceIcons`) — the social bar** | | `ViewModels/` | MainViewModel — `public partial class`, one file per functional area (Scenes, Background, Webcam, Audio, Trax, Socials, Streaming, Chat, Overlays, Account, License, Recording — split complete, see `ViewModels/index.md`); **Chat.cs is a thin delegating facade over `Services/ChatOverlayLayer.cs` (Commit G, first true decomposition)**; GoLiveViewModel, ReuseImageViewModel, CameraPickerViewModel, **SocialsDialogViewModel** | -| `Services/` | YouTube OAuth2, stream/broadcast management, live chat polling, LayoutStore (SQLite), **SocialValidator (`ISocialValidator` seam + `HttpSocialValidator` default)**, **webcam: `VideoFrame` seam + `CameraDeviceInfo`/`ICameraEnumerator`/`ICameraFrameSource` interfaces + `MediaCaptureCameraEnumerator`/`MediaCaptureFrameSource` (WinRT) + `CameraManager`**, **screen capture: `IFullScreenDetector`/`Win32FullScreenDetector` + `IScreenCaptureSource`/`ScreenCaptureFrameSource` (WinRT GraphicsCapture) + `ScreenCaptureManager` + `ScreenCaptureSourceFactory` + `Direct3D11Helper`/`CaptureInterop` (COM bridges)**, **media source: `MediaVideoSource` (spawns ffmpeg `rawvideo` BGRA decode) + pure `RawVideoFrameReader` + `IDecodeProcess`/`FfmpegDecodeProcess` process seam (binary-stdout mirror of `IEncoderProcess`; see "Media source")**, **compositor: `SceneCompositor` + `CompositorOptions` + pure `StretchMath` + `StaticPixelCache` (see "Scene compositor")**, **audio: `IAudioSource` seam + `WasapiLoopbackAudioSource`/`WasapiMicAudioSource` (NAudio WASAPI) + `AudioMixer` + pure `AudioLevelMeter`/`WaveToFloat`/`VoiceFilterChain`/`LowShelfFilter`/`HighShelfFilter`/`NoiseGate`/`Compressor`/`AutoDucker`/`AudioRingBuffer`/`TinyResampler`/`AudioSyncDelay` + `MusicPlayer` + `IAudioPipeWriter`/`NamedPipeAudioWriter` (see "Live audio capture")**, **encoder: `IFfmpegEncoder`/`FfmpegEncoder` + `IEncoderProcess`/`FfmpegEncoderProcess` + `IFfmpegLocator`/`FfmpegLocator` + pure `FfmpegArgs`/`FfmpegProgressParser`/`FfmpegEncoderPicker` + the `FramePump` frame producer (see "Live encoder" + "Live frame pipeline")**, **notifications: `INotificationService` seam (`AppNotificationSeverity` Info/Success/Warning/Error) + `NotificationService` (Notification.Wpf toasts, see "Toast notifications")** | +| `Services/` | YouTube OAuth2, stream/broadcast management, live chat polling, LayoutStore (SQLite), **SocialValidator (`ISocialValidator` seam + `HttpSocialValidator` default)**, **webcam: `VideoFrame` seam + `CameraDeviceInfo`/`ICameraEnumerator`/`ICameraFrameSource` interfaces + `MediaCaptureCameraEnumerator`/`MediaCaptureFrameSource` (WinRT) + `CameraManager`**, **screen capture: `IFullScreenDetector`/`Win32FullScreenDetector` + `IScreenCaptureSource`/`ScreenCaptureFrameSource` (WinRT GraphicsCapture) + `ScreenCaptureManager` + `ScreenCaptureSourceFactory` + `Direct3D11Helper`/`CaptureInterop` (COM bridges)**, **media source: `IMediaFrameSource` + `MediaVideoSource` (spawns ffmpeg `rawvideo` BGRA decode) + `MediaVideoSourceManager` (refcount-by-path session owner) + pure `RawVideoFrameReader` + `IDecodeProcess`/`FfmpegDecodeProcess` process seam (binary-stdout mirror of `IEncoderProcess`; see "Media source")**, **compositor: `SceneCompositor` + `CompositorOptions` + pure `StretchMath` + `StaticPixelCache` (see "Scene compositor")**, **audio: `IAudioSource` seam + `WasapiLoopbackAudioSource`/`WasapiMicAudioSource` (NAudio WASAPI) + `AudioMixer` + pure `AudioLevelMeter`/`WaveToFloat`/`VoiceFilterChain`/`LowShelfFilter`/`HighShelfFilter`/`NoiseGate`/`Compressor`/`AutoDucker`/`AudioRingBuffer`/`TinyResampler`/`AudioSyncDelay` + `MusicPlayer` + `IAudioPipeWriter`/`NamedPipeAudioWriter` (see "Live audio capture")**, **encoder: `IFfmpegEncoder`/`FfmpegEncoder` + `IEncoderProcess`/`FfmpegEncoderProcess` + `IFfmpegLocator`/`FfmpegLocator` + pure `FfmpegArgs`/`FfmpegProgressParser`/`FfmpegEncoderPicker` + the `FramePump` frame producer (see "Live encoder" + "Live frame pipeline")**, **notifications: `INotificationService` seam (`AppNotificationSeverity` Info/Success/Warning/Error) + `NotificationService` (Notification.Wpf toasts, see "Toast notifications")** | | `Helpers/` | ViewModelBase (INotifyPropertyChanged), RelayCommand, ImageCache, AppLog (file logger), FocusPreservingListBox, OAuthCredentials, **TokenStore (DPAPI session persistence)**, visibility converters | | `Themes/` | `Controls.xaml` — the single dark-theme source, merged once in `App.xaml` (see `Themes/index.md`) | | `MainWindow.xaml` | Dark theme; layout: top bar (controls), center (preview + live controls below), left (scenes/sources), right (chat), bottom (gear + stream stats + resolution) | @@ -672,11 +672,16 @@ seam:** `Func`, `Func` resolver, `Func` factory + seam (mirror of `ScreenCaptureManager`), `AcquireAsync`/`ReleaseAsync`/`ReleaseAllAsync`/`GetLatestFrame`, + coalescing each file's frames onto the UI dispatcher onto one shared `WriteableBitmap`. Frames emit as fast as + the pipe produces them; native-FPS pacing (ffprobe probe) and the resolver/preview wire-in (`ResolveOutputFrame` + case + `Source.DisplaySource`) are the next slices. - **Social bar on the output (bar bug-fix branch):** the `FramePump` takes an optional `socialBar: Func<(VideoFrame? Frame, SocialBarPosition Position)>?` seam, re-read **every frame** (so a mid-stream position flip applies immediately). The strip is pre-rasterized by `Compositor/SocialBarRenderer.cs` diff --git a/ytLive.Tests/MediaVideoSourceManagerTests.cs b/ytLive.Tests/MediaVideoSourceManagerTests.cs new file mode 100644 index 0000000..cca3b83 --- /dev/null +++ b/ytLive.Tests/MediaVideoSourceManagerTests.cs @@ -0,0 +1,209 @@ +using System.Threading; +using System.Windows.Media.Imaging; +using System.Windows.Threading; +using Xunit; +using ytLive.Services; + +namespace ytLive.Tests; + +/// +/// MediaVideoSourceManager owns media-file decode sessions, refcounted by file +/// path and coalesced onto the UI dispatcher — a structural mirror of +/// ScreenCaptureManager. These tests pin that contract with a fake +/// IMediaFrameSource (the real ffmpeg decode is exercised on Windows at runtime). +/// +public class MediaVideoSourceManagerTests +{ + private sealed class FakeMediaSource : IMediaFrameSource + { + private readonly List? _started; + private readonly List? _stopped; + + public string Key { get; } + public event Action? FrameAvailable; + public event Action? Completed; + + public FakeMediaSource(string key, List? started = null, List? stopped = null) + { + Key = key; + _started = started; + _stopped = stopped; + } + + public Task StartAsync(CancellationToken cancellationToken = default) + { + _started?.Add(Key); + return Task.CompletedTask; + } + + public Task StopAsync() + { + _stopped?.Add(Key); + return Task.CompletedTask; + } + + public void Pump(VideoFrame frame) => FrameAvailable?.Invoke(frame); + public void Complete() => Completed?.Invoke(); + public void Dispose() { } + } + + private sealed class FailingMediaSource : IMediaFrameSource + { + public string Key { get; } + public event Action? FrameAvailable; + public event Action? Completed; + public FailingMediaSource(string key) => Key = key; + public Task StartAsync(CancellationToken cancellationToken = default) + => throw new InvalidOperationException("access denied"); + public Task StopAsync() => Task.CompletedTask; + public void Pump(VideoFrame frame) => FrameAvailable?.Invoke(frame); + public void Complete() => Completed?.Invoke(); + public void Dispose() { } + } + + private sealed class DispatcherPump : IDisposable + { + private readonly Thread _thread; + private readonly ManualResetEventSlim _ready = new(false); + private Dispatcher? _dispatcher; + + public Dispatcher Dispatcher => _dispatcher!; + + public DispatcherPump() + { + _thread = new Thread(() => + { + _dispatcher = Dispatcher.CurrentDispatcher; + _ready.Set(); + Dispatcher.Run(); + }); + _thread.IsBackground = true; + _thread.SetApartmentState(ApartmentState.STA); + _thread.Start(); + _ready.Wait(); + } + + public void Drain() => Dispatcher.Invoke(() => { }, DispatcherPriority.ContextIdle); + + public void Dispose() + { + Dispatcher.InvokeShutdown(); + _thread.Join(3000); + } + } + + private static byte[] Pixels(params byte[] raw) => raw; + + [Fact] + public async Task Refcount_TwoAcquires_OneDecoderUntilLastRelease() + { + var started = new List(); + var stopped = new List(); + var manager = new MediaVideoSourceManager(key => new FakeMediaSource(key, started, stopped)); + + Assert.True(await manager.AcquireAsync(@"C:\videos\clip.mp4")); + Assert.True(await manager.AcquireAsync(@"C:\videos\clip.mp4")); + Assert.Single(started); + + await manager.ReleaseAsync(@"C:\videos\clip.mp4"); + Assert.Empty(stopped); + + await manager.ReleaseAsync(@"C:\videos\clip.mp4"); + Assert.Single(stopped); + } + + [Fact] + public async Task ReleaseAll_StopsEvenWithMultipleRefs() + { + var started = new List(); + var stopped = new List(); + var manager = new MediaVideoSourceManager(key => new FakeMediaSource(key, started, stopped)); + + await manager.AcquireAsync(@"C:\videos\clip.mp4"); + await manager.AcquireAsync(@"C:\videos\clip.mp4"); + await manager.ReleaseAllAsync(@"C:\videos\clip.mp4"); + + Assert.Single(stopped); + } + + [Fact] + public async Task Acquire_FailingSource_ReturnsFalseAndRaisesMediaFailed() + { + var manager = new MediaVideoSourceManager(key => new FailingMediaSource(key)); + string? failedKey = null; + manager.MediaFailed += (key, _) => failedKey = key; + + Assert.False(await manager.AcquireAsync(@"C:\videos\clip.mp4")); + Assert.Equal(@"C:\videos\clip.mp4", failedKey); + } + + [Fact] + public async Task Acquire_EmptyKey_ReturnsFalse() + { + var manager = new MediaVideoSourceManager(key => new FakeMediaSource(key)); + Assert.False(await manager.AcquireAsync(" ")); + } + + [Fact] + public async Task GetLatestFrame_ReturnsLatestPump_UntilReleased() + { + FakeMediaSource? captured = null; + var manager = new MediaVideoSourceManager(key => captured = new FakeMediaSource(key)); + + Assert.Null(manager.GetLatestFrame(@"C:\videos\clip.mp4")); + + Assert.True(await manager.AcquireAsync(@"C:\videos\clip.mp4")); + Assert.Null(manager.GetLatestFrame(@"C:\videos\clip.mp4")); + + var first = new VideoFrame(2, 2, Pixels(1, 0, 0, 255, 2, 0, 0, 255, 3, 0, 0, 255, 4, 0, 0, 255)); + var second = new VideoFrame(2, 2, Pixels(5, 0, 0, 255, 6, 0, 0, 255, 7, 0, 0, 255, 8, 0, 0, 255)); + captured!.Pump(first); + captured.Pump(second); + + Assert.Same(second, manager.GetLatestFrame(@"C:\videos\clip.mp4")); + + await manager.ReleaseAsync(@"C:\videos\clip.mp4"); + Assert.Null(manager.GetLatestFrame(@"C:\videos\clip.mp4")); + } + + // The one integration test for this branch: one file creates one shared + // WriteableBitmap, published once, and back-to-back frames coalesce to the + // latest (a single pending UI copy per session) — mirror of ScreenCapture. + [Fact] + public async Task Frames_ShareOneBitmap_AndCoalesceToLatest() + { + using var pump = new DispatcherPump(); + FakeMediaSource? captured = null; + var manager = new MediaVideoSourceManager(key => captured = new FakeMediaSource(key), pump.Dispatcher); + + WriteableBitmap? published = null; + var publishedCount = 0; + manager.PreviewBitmapChanged += (_, bitmap) => { published = bitmap; publishedCount++; }; + + Assert.True(await manager.AcquireAsync(@"C:\videos\clip.mp4")); + + var first = new VideoFrame(2, 2, Pixels(1, 0, 0, 255, 2, 0, 0, 255, 3, 0, 0, 255, 4, 0, 0, 255)); + captured!.Pump(first); + pump.Drain(); + + Assert.NotNull(published); + Assert.Equal(1, publishedCount); + + var dims = pump.Dispatcher.Invoke(() => new[] { published!.PixelWidth, published.PixelHeight }); + Assert.Equal(new[] { 2, 2 }, dims); + + var second = new VideoFrame(2, 2, Pixels(5, 0, 0, 255, 6, 0, 0, 255, 7, 0, 0, 255, 8, 0, 0, 255)); + var third = new VideoFrame(2, 2, Pixels(9, 0, 0, 255, 10, 0, 0, 255, 11, 0, 0, 255, 12, 0, 0, 255)); + captured.Pump(second); + captured.Pump(third); + pump.Drain(); + + var result = pump.Dispatcher.Invoke(() => + { + var bytes = new byte[16]; + published!.CopyPixels(new System.Windows.Int32Rect(0, 0, 2, 2), bytes, 8, 0); + return bytes; + }); + Assert.Equal(third.BgraPixels, result); + } +} diff --git a/ytLive.Tests/MediaVideoSourceTests.cs b/ytLive.Tests/MediaVideoSourceTests.cs index 7a365a3..8738ccc 100644 --- a/ytLive.Tests/MediaVideoSourceTests.cs +++ b/ytLive.Tests/MediaVideoSourceTests.cs @@ -66,10 +66,10 @@ public class MediaVideoSourceTests var seen = new List(); var completed = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously); - source.FrameReady += f => seen.Add(((byte[])f.BgraPixels.Clone())); + source.FrameAvailable += f => seen.Add(((byte[])f.BgraPixels.Clone())); source.Completed += () => completed.TrySetResult(true); - source.Start(); + await source.StartAsync(); await completed.Task.WaitAsync(TimeSpan.FromSeconds(5)); Assert.Equal(2, seen.Count);