TASK 21 Increment B: FFmpeg rawvideo video decoder -> VideoFrame
Adds the codec-agnostic decoder half of the media source. Spawns ffmpeg with -f rawvideo -pix_fmt bgra (reusing the already-shipped ffmpeg via IFfmpegLocator) and drains the raw BGRA stdout pipe into VideoFrames. - Services/RawVideoFrameReader.cs: pure rawvideo BGRA stream -> frames (partial reads kept across Feed; no ffmpeg needed to test) - Services/IDecodeProcess.cs + FfmpegDecodeProcess.cs: binary-stdout subprocess seam, mirror of the encoder's IEncoderProcess - Services/MediaVideoSource.cs: owns the decode, raises FrameReady/Completed - MediaVideoSourceTests: 3 pure reader + 1 integration (fake decode process through the real source loop, frames in order) Reference (external scan): ffmpeg rawvideo pipe decode is the canonical codec-agnostic frame feeds pattern (ffmpeg docs -f rawvideo; how OBS/media pipelines push frames to a compositor). Verified: 4/4 tests, build 0 warnings. Native-FPS pacing + resolver/compositor wiring are the next slice.
This commit is contained in:
+38
-36
@@ -2,54 +2,56 @@
|
||||
|
||||
## Branch / Commit State
|
||||
|
||||
**`main`**, working toward TASK 22 (audio sync offset). Prior milestone **TASK 31**
|
||||
(SceneGraph + baked-crust compositor) is fully done, committed (`670fe3a`), pushed
|
||||
(`origin/main` `3bf053a..670fe3a`), and verified by the user on native Windows
|
||||
PowerShell (only the 2 known pre-existing failures remained).
|
||||
**`main`**, working through TASK 21 (media source). Local commits ahead of
|
||||
`origin/main` (not yet pushed — user pushes at agreed checkpoints):
|
||||
- `f89b9f9` — docs: default-device audio assumption (README + startup log)
|
||||
- `e8ff4df` — TASK 22: global audio sync offset (positive delay at the mixer out)
|
||||
- `670fe3a` — TASK 31 (already pushed; listed for context)
|
||||
|
||||
Prior milestones pushed through `origin/main`: TASK 31 (`3bf053a..670fe3a`), plus
|
||||
the full refactor series (Commit F `3107f92`, Commit G `85893ea`).
|
||||
|
||||
## What's In Flight
|
||||
|
||||
**TASK 22 (audio sync offset) — implemented, uncommitted, awaiting commit + user
|
||||
verification on Windows.**
|
||||
**TASK 21 (media source) — Increment B (the decoder) implemented, uncommitted-or-pending-commit, awaiting next unit.**
|
||||
|
||||
Scope (Good-Dog single use-case; user chose positive-only slider at the mixer out):
|
||||
a global `AudioSyncOffsetMs` (0..500 ms, default 0) that delays the whole
|
||||
interleaved-stereo mix so audio lands on the video when it runs ahead — OBS's
|
||||
documented lip-sync fix. Positive-only: advancing audio needs a video-side delay
|
||||
(out of the audio layer's scope; tracked for v1.1).
|
||||
Increment A (the `Source` model: `MediaSourceType`, `SourceType.MediaSource`,
|
||||
`MediaPath`/`MediaIsLooping`/`MediaVolume`/`MediaPlaybackState` + LayoutStore
|
||||
persistence) shipped earlier. This session shipped **Increment B — the decoder**:
|
||||
|
||||
Files touched (this work unit):
|
||||
- `Services/Audio/AudioSyncDelay.cs` (new) — pure delay line, flushed on `Configure`.
|
||||
- `Services/Audio/AudioMixer.cs` — `Func<int> syncOffsetMs` seam + `_syncDelay`
|
||||
applied post-limiter in `FillAndMix`.
|
||||
- `Services/LayoutStore.Settings.cs` — `LoadAudioSyncOffsetMs`/`SaveAudioSyncOffsetMs`
|
||||
(`Audio.SyncOffsetMs`).
|
||||
- `ViewModels/MainViewModel.cs` — load offset at startup + save in `SaveLayoutNow`.
|
||||
- `ViewModels/MainViewModel.Audio.cs` — `AudioSyncOffsetMs` property (clamped, `ScheduleSave`).
|
||||
- `Controls/PreviewPane.xaml` — SYNC slider (0..500) + status dot on the mic bar.
|
||||
- `Helpers/IntToSyncBrushConverter.cs` (new) + registered in `Themes/Controls.xaml`.
|
||||
- `ytLive.Tests/AudioSyncDelayTests.cs` (new) — 3 tests.
|
||||
- `ai.md`, `TASKS.md`, `HANDOFF.md`.
|
||||
New files:
|
||||
- `Services/RawVideoFrameReader.cs` — pure: parses a rawvideo BGRA stream into
|
||||
`VideoFrame`s (exactly W×H×4 per frame; partial reads kept across `Feed`).
|
||||
- `Services/IDecodeProcess.cs` + `Services/FfmpegDecodeProcess.cs` — binary-stdout
|
||||
subprocess seam (mirror of `IEncoderProcess`).
|
||||
- `Services/MediaVideoSource.cs` — spawns ffmpeg `-f rawvideo -pix_fmt bgra -an`,
|
||||
drains the pipe, raises `FrameReady` per frame; `Completed` on EOF.
|
||||
- `ytLive.Tests/MediaVideoSourceTests.cs` — 3 pure reader tests + 1 integration
|
||||
(fake `IDecodeProcess`+`IFfmpegLocator` through the real source loop, frames in
|
||||
order). **Verified: 4/4 pass, build 0 warnings.**
|
||||
|
||||
**Verified:** `ytLive.csproj` and `ytLive.Tests.csproj` build with 0 warnings;
|
||||
`AudioSyncDelayTests` (3/3) pass via vstest filter. The full audio pipeline /
|
||||
RealAppHost suites hang headless (pre-existing) — must be verified on native
|
||||
Windows PowerShell by the user.
|
||||
Still open (next slices): native-FPS pacing (ffprobe probe), loop control, volume,
|
||||
the `ResolveOutputFrame` resolver case + wire into `SceneCompositor`, the file-picker
|
||||
UI, and the remaining tests.
|
||||
|
||||
**Landmine:** a stale `testhost` can lock `ytLive.Tests`'s `ytLive.dll` and break
|
||||
`dotnet build` of the test project — kill it first (`cmd.exe /c "taskkill /PID N /F"`)
|
||||
if the build hits MSB3027.
|
||||
on MSB3027. Full-suite vstest hangs headless (RealAppHost/WASAPI) — only filtered
|
||||
pure tests run in WSL; user verifies the GUI suites on native Windows PowerShell
|
||||
(which may also hang if WASAPI startup blocks — pre-existing, not this change).
|
||||
|
||||
## Next
|
||||
|
||||
1. User runs `scripts/verify.sh` (or native Windows vstest) to confirm the full
|
||||
suite — expect only the 2 known pre-existing failures.
|
||||
2. On confirmation, run `scripts/scope-check.sh` with the declared file list,
|
||||
commit as one work unit, await the user's push go-ahead (sub-milestone).
|
||||
1. Commit Increment B (decoder) as one work unit (scope-check + docs already
|
||||
staged in ai.md/TASKS.md/HANDOFF.md).
|
||||
2. Next slice: native-FPS pacing + wire media into `ResolveOutputFrame` and the
|
||||
compositor, per TASK 21 steps 6–7.
|
||||
3. Await user push checkpoint for the TASK 22/21 commits.
|
||||
|
||||
## The directive (2026-08-31, user)
|
||||
|
||||
Rewrite the project into functional components to aid AI retrieval — the
|
||||
`Services/ChatOverlayLayer.cs` style (owner-state extraction), not line-count
|
||||
chasing. TASK 31's `SceneGraph.cs` continued that pattern (an owner object owning
|
||||
the scene collection + mutation surface).
|
||||
Rewrite the project into functional components to aid AI retrieval —
|
||||
`Services/ChatOverlayLayer.cs` / `SceneGraph.cs` style (owner-state extraction),
|
||||
not line-count chasing. Audio assumption is now an explicit contract (README
|
||||
"Audio Assumption"): the app uses system defaults, never fights Windows device
|
||||
locking, and does not debug user audio issues — OS's problem, not ours.
|
||||
|
||||
@@ -0,0 +1,36 @@
|
||||
using System.Diagnostics;
|
||||
using System.IO;
|
||||
|
||||
namespace ytLive.Services;
|
||||
|
||||
/// <summary>
|
||||
/// The real <see cref="IDecodeProcess"/>: a <see cref="Process"/> with stdout
|
||||
/// redirected as raw bytes for reading <c>rawvideo</c> frames. Constructed by
|
||||
/// <see cref="MediaVideoSource"/> for the media-file decoder subprocess.
|
||||
/// </summary>
|
||||
public sealed class FfmpegDecodeProcess : IDecodeProcess
|
||||
{
|
||||
private readonly Process _process;
|
||||
|
||||
public FfmpegDecodeProcess() => _process = new Process { EnableRaisingEvents = true };
|
||||
|
||||
public void Start(ProcessStartInfo startInfo)
|
||||
{
|
||||
_process.StartInfo = startInfo;
|
||||
_process.Start();
|
||||
}
|
||||
|
||||
public Stream StandardOutput => _process.StandardOutput.BaseStream;
|
||||
|
||||
public bool HasExited => _process.HasExited;
|
||||
public int ExitCode => _process.ExitCode;
|
||||
public void Kill()
|
||||
{
|
||||
if (!_process.HasExited) _process.Kill();
|
||||
}
|
||||
|
||||
public Task WaitForExitAsync(CancellationToken cancellationToken = default)
|
||||
=> _process.WaitForExitAsync(cancellationToken);
|
||||
|
||||
public void Dispose() => _process.Dispose();
|
||||
}
|
||||
@@ -0,0 +1,24 @@
|
||||
using System.Diagnostics;
|
||||
using System.IO;
|
||||
|
||||
namespace ytLive.Services;
|
||||
|
||||
/// <summary>
|
||||
/// TASK 21: seam around the media decoder's subprocess (ffmpeg <c>rawvideo</c>).
|
||||
/// Exposes the raw binary stdout (frames) and exit control, so
|
||||
/// <see cref="MediaVideoSource"/> never touches <c>System.Diagnostics.Process</c>
|
||||
/// directly and tests can fake the whole decode. Mirrors the encoder's
|
||||
/// <c>IEncoderProcess</c> but with a binary <see cref="StandardOutput"/>.
|
||||
/// </summary>
|
||||
public interface IDecodeProcess : IDisposable
|
||||
{
|
||||
void Start(ProcessStartInfo startInfo);
|
||||
|
||||
/// <summary>Raw binary stdout — the <c>rawvideo</c> BGRA frame bytes.</summary>
|
||||
Stream StandardOutput { get; }
|
||||
|
||||
bool HasExited { get; }
|
||||
int ExitCode { get; }
|
||||
void Kill();
|
||||
Task WaitForExitAsync(CancellationToken cancellationToken = default);
|
||||
}
|
||||
@@ -0,0 +1,117 @@
|
||||
using System.Diagnostics;
|
||||
using System.IO;
|
||||
using ytLive.Services.Encoder;
|
||||
|
||||
namespace ytLive.Services;
|
||||
|
||||
/// <summary>
|
||||
/// TASK 21: decodes a local media file (any format ffmpeg handles) into
|
||||
/// <see cref="VideoFrame"/>s for a MediaSource. Spawns ffmpeg with
|
||||
/// <c>-f rawvideo -pix_fmt bgra</c>, reads the raw BGRA pipe via
|
||||
/// <see cref="RawVideoFrameReader"/>, and raises <see cref="FrameReady"/> per
|
||||
/// frame. The decoder process is a seam (<see cref="IDecodeProcess"/>) so tests
|
||||
/// run without a real codec or audio device. Frames are emitted as fast as the
|
||||
/// pipe produces them; native-FPS pacing is wired by the caller via a probe in a
|
||||
/// later unit (TASK 21 step 5 pacing).
|
||||
/// </summary>
|
||||
public sealed class MediaVideoSource : IDisposable
|
||||
{
|
||||
public event Action<VideoFrame>? FrameReady;
|
||||
public event Action? Completed;
|
||||
|
||||
private readonly string _path;
|
||||
private readonly int _width;
|
||||
private readonly int _height;
|
||||
private readonly IFfmpegLocator _locator;
|
||||
private readonly IDecodeProcess _process;
|
||||
private readonly CancellationTokenSource _cts = new();
|
||||
private IDisposable? _run;
|
||||
|
||||
public MediaVideoSource(
|
||||
string path,
|
||||
int width,
|
||||
int height,
|
||||
IFfmpegLocator locator,
|
||||
IDecodeProcess process)
|
||||
{
|
||||
_path = path;
|
||||
_width = width;
|
||||
_height = height;
|
||||
_locator = locator;
|
||||
_process = process;
|
||||
}
|
||||
|
||||
public void Start()
|
||||
{
|
||||
if (_run != null) return;
|
||||
_run = _cts.Token.Register(StopInternal);
|
||||
_ = Task.Run(RunAsync);
|
||||
}
|
||||
|
||||
public void Stop()
|
||||
{
|
||||
_cts.Cancel();
|
||||
_run = null;
|
||||
}
|
||||
|
||||
public void Dispose()
|
||||
{
|
||||
_cts.Cancel();
|
||||
_cts.Dispose();
|
||||
_process.Dispose();
|
||||
}
|
||||
|
||||
private void StopInternal() => KillProcess();
|
||||
|
||||
private async Task RunAsync()
|
||||
{
|
||||
try
|
||||
{
|
||||
var ffmpegPath = await _locator.LocateAsync(_cts.Token).ConfigureAwait(false);
|
||||
var args =
|
||||
$"-hide_banner -loglevel error -i \"{_path}\" " +
|
||||
$"-f rawvideo -pix_fmt bgra -vf scale={_width}:{_height} -an -";
|
||||
var startInfo = new ProcessStartInfo
|
||||
{
|
||||
FileName = ffmpegPath,
|
||||
Arguments = args,
|
||||
UseShellExecute = false,
|
||||
RedirectStandardOutput = true,
|
||||
RedirectStandardError = true,
|
||||
CreateNoWindow = true,
|
||||
};
|
||||
_process.Start(startInfo);
|
||||
|
||||
var reader = new RawVideoFrameReader(_width, _height);
|
||||
var buffer = new byte[65536];
|
||||
var stream = _process.StandardOutput;
|
||||
byte[] readChunk;
|
||||
while (!_cts.IsCancellationRequested && !_process.HasExited)
|
||||
{
|
||||
var read = await stream.ReadAsync(buffer, 0, buffer.Length, _cts.Token).ConfigureAwait(false);
|
||||
if (read <= 0) break;
|
||||
readChunk = buffer.AsSpan(0, read).ToArray();
|
||||
foreach (var frame in reader.Feed(readChunk))
|
||||
FrameReady?.Invoke(frame);
|
||||
}
|
||||
}
|
||||
catch (OperationCanceledException)
|
||||
{
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
Fail($"Media decode failed: {ex.Message}");
|
||||
return;
|
||||
}
|
||||
|
||||
if (!_cts.IsCancellationRequested)
|
||||
Completed?.Invoke();
|
||||
}
|
||||
|
||||
private void Fail(string message) => Console.Error.WriteLine(message);
|
||||
|
||||
private void KillProcess()
|
||||
{
|
||||
try { _process.Kill(); } catch { /* already exited */ }
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,59 @@
|
||||
namespace ytLive.Services;
|
||||
|
||||
/// <summary>
|
||||
/// TASK 21: parses a stream of tightly-packed raw BGRA frames into
|
||||
/// <see cref="VideoFrame"/>s. The consumer of an ffmpeg <c>-f rawvideo -pix_fmt
|
||||
/// bgra</c> stdout pipe: every frame is exactly <c>width*height*4</c> bytes, so a
|
||||
/// frame is emitted whenever that many bytes have accumulated. Pure and fully
|
||||
/// deterministic, so it is tested with a synthetic byte stream — no ffmpeg, no
|
||||
/// process, no audio device needed.
|
||||
/// </summary>
|
||||
public sealed class RawVideoFrameReader
|
||||
{
|
||||
private readonly int _frameBytes;
|
||||
private readonly int _width;
|
||||
private readonly int _height;
|
||||
private byte[]? _pending;
|
||||
|
||||
public RawVideoFrameReader(int width, int height)
|
||||
{
|
||||
_width = width;
|
||||
_height = height;
|
||||
_frameBytes = width * height * 4;
|
||||
}
|
||||
|
||||
/// <summary>Feeds <paramref name="bytes"/> read from the pipe and returns any
|
||||
/// whole frames reconstructed so far. Partial/pending bytes are kept for the
|
||||
/// next call — frames never split or merge across reads.</summary>
|
||||
public List<VideoFrame> Feed(byte[] bytes)
|
||||
{
|
||||
var pending = _pending;
|
||||
byte[] combined;
|
||||
if (pending != null && pending.Length > 0)
|
||||
{
|
||||
combined = new byte[pending.Length + bytes.Length];
|
||||
Buffer.BlockCopy(pending, 0, combined, 0, pending.Length);
|
||||
Buffer.BlockCopy(bytes, 0, combined, pending.Length, bytes.Length);
|
||||
_pending = null;
|
||||
}
|
||||
else
|
||||
{
|
||||
combined = bytes;
|
||||
}
|
||||
|
||||
var frames = new List<VideoFrame>();
|
||||
var offset = 0;
|
||||
while (combined.Length - offset >= _frameBytes)
|
||||
{
|
||||
var frame = new byte[_frameBytes];
|
||||
Buffer.BlockCopy(combined, offset, frame, 0, _frameBytes);
|
||||
frames.Add(new VideoFrame(_width, _height, frame));
|
||||
offset += _frameBytes;
|
||||
}
|
||||
|
||||
if (offset < combined.Length)
|
||||
_pending = combined[offset..];
|
||||
|
||||
return frames;
|
||||
}
|
||||
}
|
||||
@@ -1036,13 +1036,13 @@ The tasks below are ordered by dependency and risk. Each task builds on the prev
|
||||
|
||||
**Goal:** play video files (MP4, MOV, AVI) into scenes — starting soon videos, BRB loops, intro/outro clips.
|
||||
|
||||
### Status: 🔶 In progress — Increment A shipped (model + persistence), slices 4–12 open
|
||||
### Status: 🔶 In progress — Increment A shipped (model + persistence); Increment B (decoder) shipped; slices 6–12 open
|
||||
|
||||
1. ✅ `MediaSourceType` enum: `Video`, `Audio` (audio-only files via media source)
|
||||
2. ✅ `SourceType.MediaSource` addition to the enum
|
||||
3. ✅ `MediaSourceModel`: `MediaPath`, `MediaIsLooping`, `MediaVolume` (0-1), `MediaPlaybackState` — persisted in LayoutStore (schema migration + SELECT/INSERT + round-trip test)
|
||||
4. ☐ `VideoFrameSource`: FFmpeg-based video decoder → `VideoFrame` pipeline
|
||||
5. ☐ Frame capture from video file (decode at native FPS, output BGRA8 frames)
|
||||
4. ✅ `MediaVideoSource`: FFmpeg raw video decoder → `VideoFrame` pipeline — spawns ffmpeg `-f rawvideo -pix_fmt bgra`, drains the pipe via pure `RawVideoFrameReader` (`Services/RawVideoFrameReader.cs`), raises `FrameReady` per frame; decode process behind `IDecodeProcess`/`FfmpegDecodeProcess` seam (binary-stdout mirror of `IEncoderProcess`). Shipped 2026-08-31.
|
||||
5. 🔶 Frame capture from video file, output BGRA8 frames — decode + BGRA8 emit done; **native-FPS pacing still open** (ffprobe probe, wired by the caller; frames currently emit as fast as the pipe produces).
|
||||
6. ☐ Wire into `FramePump` resolver — `Source { Type: MediaSource }` → latest video frame
|
||||
7. ☐ Wire into `SceneCompositor` — render media source as an image element at its position/size
|
||||
8. ☐ Loop control — `IsLooping` property, restart on end
|
||||
|
||||
@@ -155,7 +155,7 @@ C# / WPF (.NET 8) following MVVM:
|
||||
|------|------|
|
||||
| `Models/` | Plain data types — Scene, Source (incl. `ClipShape`, `IsMirrored`, `VideoImageSource`), QualityOption, StreamConfig, StreamHealth, YouTubeChannel, ChatMessage, **Socials (`SocialService` enum + `SocialEntry`/`SocialsConfig` + `SocialServiceIcons`) — the social bar** |
|
||||
| `ViewModels/` | MainViewModel — `public partial class`, one file per functional area (Scenes, Background, Webcam, Audio, Trax, Socials, Streaming, Chat, Overlays, Account, License, Recording — split complete, see `ViewModels/index.md`); **Chat.cs is a thin delegating facade over `Services/ChatOverlayLayer.cs` (Commit G, first true decomposition)**; GoLiveViewModel, ReuseImageViewModel, CameraPickerViewModel, **SocialsDialogViewModel** |
|
||||
| `Services/` | YouTube OAuth2, stream/broadcast management, live chat polling, LayoutStore (SQLite), **SocialValidator (`ISocialValidator` seam + `HttpSocialValidator` default)**, **webcam: `VideoFrame` seam + `CameraDeviceInfo`/`ICameraEnumerator`/`ICameraFrameSource` interfaces + `MediaCaptureCameraEnumerator`/`MediaCaptureFrameSource` (WinRT) + `CameraManager`**, **screen capture: `IFullScreenDetector`/`Win32FullScreenDetector` + `IScreenCaptureSource`/`ScreenCaptureFrameSource` (WinRT GraphicsCapture) + `ScreenCaptureManager` + `ScreenCaptureSourceFactory` + `Direct3D11Helper`/`CaptureInterop` (COM bridges)**, **compositor: `SceneCompositor` + `CompositorOptions` + pure `StretchMath` + `StaticPixelCache` (see "Scene compositor")**, **audio: `IAudioSource` seam + `WasapiLoopbackAudioSource`/`WasapiMicAudioSource` (NAudio WASAPI) + `AudioMixer` + pure `AudioLevelMeter`/`WaveToFloat`/`VoiceFilterChain`/`LowShelfFilter`/`HighShelfFilter`/`NoiseGate`/`Compressor`/`AutoDucker`/`AudioRingBuffer`/`TinyResampler`/`AudioSyncDelay` + `MusicPlayer` + `IAudioPipeWriter`/`NamedPipeAudioWriter` (see "Live audio capture")**, **encoder: `IFfmpegEncoder`/`FfmpegEncoder` + `IEncoderProcess`/`FfmpegEncoderProcess` + `IFfmpegLocator`/`FfmpegLocator` + pure `FfmpegArgs`/`FfmpegProgressParser`/`FfmpegEncoderPicker` + the `FramePump` frame producer (see "Live encoder" + "Live frame pipeline")**, **notifications: `INotificationService` seam (`AppNotificationSeverity` Info/Success/Warning/Error) + `NotificationService` (Notification.Wpf toasts, see "Toast notifications")** |
|
||||
| `Services/` | YouTube OAuth2, stream/broadcast management, live chat polling, LayoutStore (SQLite), **SocialValidator (`ISocialValidator` seam + `HttpSocialValidator` default)**, **webcam: `VideoFrame` seam + `CameraDeviceInfo`/`ICameraEnumerator`/`ICameraFrameSource` interfaces + `MediaCaptureCameraEnumerator`/`MediaCaptureFrameSource` (WinRT) + `CameraManager`**, **screen capture: `IFullScreenDetector`/`Win32FullScreenDetector` + `IScreenCaptureSource`/`ScreenCaptureFrameSource` (WinRT GraphicsCapture) + `ScreenCaptureManager` + `ScreenCaptureSourceFactory` + `Direct3D11Helper`/`CaptureInterop` (COM bridges)**, **media source: `MediaVideoSource` (spawns ffmpeg `rawvideo` BGRA decode) + pure `RawVideoFrameReader` + `IDecodeProcess`/`FfmpegDecodeProcess` process seam (binary-stdout mirror of `IEncoderProcess`; see "Media source")**, **compositor: `SceneCompositor` + `CompositorOptions` + pure `StretchMath` + `StaticPixelCache` (see "Scene compositor")**, **audio: `IAudioSource` seam + `WasapiLoopbackAudioSource`/`WasapiMicAudioSource` (NAudio WASAPI) + `AudioMixer` + pure `AudioLevelMeter`/`WaveToFloat`/`VoiceFilterChain`/`LowShelfFilter`/`HighShelfFilter`/`NoiseGate`/`Compressor`/`AutoDucker`/`AudioRingBuffer`/`TinyResampler`/`AudioSyncDelay` + `MusicPlayer` + `IAudioPipeWriter`/`NamedPipeAudioWriter` (see "Live audio capture")**, **encoder: `IFfmpegEncoder`/`FfmpegEncoder` + `IEncoderProcess`/`FfmpegEncoderProcess` + `IFfmpegLocator`/`FfmpegLocator` + pure `FfmpegArgs`/`FfmpegProgressParser`/`FfmpegEncoderPicker` + the `FramePump` frame producer (see "Live encoder" + "Live frame pipeline")**, **notifications: `INotificationService` seam (`AppNotificationSeverity` Info/Success/Warning/Error) + `NotificationService` (Notification.Wpf toasts, see "Toast notifications")** |
|
||||
| `Helpers/` | ViewModelBase (INotifyPropertyChanged), RelayCommand, ImageCache, AppLog (file logger), FocusPreservingListBox, OAuthCredentials, **TokenStore (DPAPI session persistence)**, visibility converters |
|
||||
| `Themes/` | `Controls.xaml` — the single dark-theme source, merged once in `App.xaml` (see `Themes/index.md`) |
|
||||
| `MainWindow.xaml` | Dark theme; layout: top bar (controls), center (preview + live controls below), left (scenes/sources), right (chat), bottom (gear + stream stats + resolution) |
|
||||
@@ -669,6 +669,14 @@ seam:** `Func<Scene?>`, `Func<SceneElement, VideoFrame?>` resolver, `Func<Compos
|
||||
background → `StaticPixelCache.Get(AssetId)`. `BuildCompositorOptions` rounds the VM's `OutputRect*`
|
||||
doubles to ints — the vertical 607.5 half-pixel crop rounds to a perfectly-centered **608** (`Math.Round`,
|
||||
ToEven); `BuildEncoderOptions` fills W×H/FPS/bitrate from the tier once the URL provider yields one.
|
||||
- **Media source decoder (TASK 21, Increment B — 2026-08-31):** `MediaVideoSource` (`Services/MediaVideoSource.cs`)
|
||||
decodes a local file to `VideoFrame`s by spawning ffmpeg with
|
||||
`-f rawvideo -pix_fmt bgra -vf scale=WxH -an` and draining the raw BGRA stdout pipe through the pure
|
||||
`RawVideoFrameReader` (`Services/RawVideoFrameReader.cs`), raising `FrameReady` per frame. The subprocess sits
|
||||
behind `IDecodeProcess`/`FfmpegDecodeProcess` (binary-stdout mirror of the encoder's `IEncoderProcess`), and
|
||||
the path comes from `IFfmpegLocator` — so it's codec-agnostic, uses the ffmpeg already shipped, and is
|
||||
testable with a fake process. Frames emit as fast as the pipe produces them; native-FPS pacing (ffprobe probe,
|
||||
wired by the caller) and the resolver/compositor wiring (`ResolveOutputFrame` case) are the next slice.
|
||||
- **Social bar on the output (bar bug-fix branch):** the `FramePump` takes an optional
|
||||
`socialBar: Func<(VideoFrame? Frame, SocialBarPosition Position)>?` seam, re-read **every frame** (so a
|
||||
mid-stream position flip applies immediately). The strip is pre-rasterized by `Compositor/SocialBarRenderer.cs`
|
||||
|
||||
@@ -0,0 +1,97 @@
|
||||
using System.Diagnostics;
|
||||
using System.IO;
|
||||
using Xunit;
|
||||
using ytLive.Services;
|
||||
using ytLive.Services.Encoder;
|
||||
|
||||
namespace ytLive.Tests;
|
||||
|
||||
/// <summary>
|
||||
/// TASK 21: the media decoder path. <see cref="RawVideoFrameReader"/> is pure
|
||||
/// (synthetic bytes, no ffmpeg); the one integration test drives the real
|
||||
/// <see cref="MediaVideoSource"/> source loop against a fake decoder process.
|
||||
/// </summary>
|
||||
public class MediaVideoSourceTests
|
||||
{
|
||||
// ─── Pure reader (unit) ───
|
||||
|
||||
[Fact]
|
||||
public void Reader_EmitsOneFramePerFrameBytes()
|
||||
{
|
||||
var w = 2; var h = 2;
|
||||
var reader = new RawVideoFrameReader(w, h);
|
||||
var frameBytes = new byte[w * h * 4];
|
||||
|
||||
var frames = reader.Feed(frameBytes);
|
||||
Assert.Single(frames);
|
||||
Assert.Equal(w, frames[0].Width);
|
||||
Assert.Equal(h, frames[0].Height);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Reader_SplitsMultipleFramesFromOneFeed()
|
||||
{
|
||||
var reader = new RawVideoFrameReader(1, 1); // 4 bytes each
|
||||
var frames = reader.Feed(new byte[12]); // 3 frames
|
||||
Assert.Equal(3, frames.Count);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Reader_KeepsPartialBytesAcrossFeeds()
|
||||
{
|
||||
var reader = new RawVideoFrameReader(1, 1);
|
||||
Assert.Empty(reader.Feed(new byte[2])); // half a frame
|
||||
Assert.Empty(reader.Feed(new byte[1])); // 3/4
|
||||
var frames = reader.Feed(new byte[1]); // completes the frame
|
||||
Assert.Single(frames);
|
||||
Assert.Empty(reader.Feed(new byte[0])); // no leftover
|
||||
}
|
||||
|
||||
// ─── The one integration test: source loop against a fake decode process ───
|
||||
|
||||
[Fact]
|
||||
public async Task MediaVideoSource_EmitsFramesInOrderFromFakeDecoder()
|
||||
{
|
||||
const int w = 2, h = 1; // frame bytes = 8
|
||||
var frameA = new byte[] { 1, 2, 3, 4, 5, 6, 7, 8 };
|
||||
var frameB = new byte[] { 9, 10, 11, 12, 13, 14, 15, 16 };
|
||||
var payload = new byte[frameA.Length + frameB.Length];
|
||||
Buffer.BlockCopy(frameA, 0, payload, 0, frameA.Length);
|
||||
Buffer.BlockCopy(frameB, 0, payload, frameA.Length, frameB.Length);
|
||||
|
||||
using var source = new MediaVideoSource(
|
||||
"clip.mp4", w, h,
|
||||
new FakeLocator(),
|
||||
new FakeDecodeProcess(payload));
|
||||
|
||||
var seen = new List<byte[]>();
|
||||
var completed = new TaskCompletionSource<bool>(TaskCreationOptions.RunContinuationsAsynchronously);
|
||||
source.FrameReady += f => seen.Add(((byte[])f.BgraPixels.Clone()));
|
||||
source.Completed += () => completed.TrySetResult(true);
|
||||
|
||||
source.Start();
|
||||
await completed.Task.WaitAsync(TimeSpan.FromSeconds(5));
|
||||
|
||||
Assert.Equal(2, seen.Count);
|
||||
Assert.Equal(frameA, seen[0]);
|
||||
Assert.Equal(frameB, seen[1]);
|
||||
}
|
||||
|
||||
private sealed class FakeDecodeProcess : IDecodeProcess
|
||||
{
|
||||
private readonly MemoryStream _stream;
|
||||
public FakeDecodeProcess(byte[] bytes) => _stream = new MemoryStream(bytes);
|
||||
public void Start(ProcessStartInfo startInfo) { }
|
||||
public Stream StandardOutput => _stream;
|
||||
public bool HasExited => _stream.Position >= _stream.Length;
|
||||
public int ExitCode => 0;
|
||||
public void Kill() { }
|
||||
public Task WaitForExitAsync(CancellationToken ct = default) => Task.CompletedTask;
|
||||
public void Dispose() => _stream.Dispose();
|
||||
}
|
||||
|
||||
private sealed class FakeLocator : IFfmpegLocator
|
||||
{
|
||||
public Task<string> LocateAsync(CancellationToken ct = default) => Task.FromResult("ffmpeg.exe");
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user