Compare commits

..

2 Commits

24 changed files with 1673 additions and 64 deletions
+44 -60
View File
@@ -7,69 +7,53 @@
## Session state (last updated: 2026-08-12) ## Session state (last updated: 2026-08-12)
- **Branch:** `main` — the social bar v2 push is **committed + pushed** (`9873eed`, - **Branch:** `main`. TASK 4 ship step 4 (WASAPI audio capture) is **built and
18 files, +1878/321, 112 tests passing, 0 warnings). Working tree is **dirty tested** but **NOT committed** — the working tree is dirty with the audio
with the post-push docs backfill only** (ai.md, README.md, TASKS.md schema layer (6 new files in `Services/Audio/` + `ytLive.Tests/AudioMixerTests.cs`,
section, THIRD-PARTY-NOTICES.txt) — ready to commit when the user says so. the `NAudio.Wasapi` 2.2.1 package reference, the THIRD-PARTY-NOTICES entry,
- **Finished this session (round 3):** user's live-test report said the mastodon the `MainViewModel` wiring, the `FfmpegEncoder.cs:139` CS8602 fix) and its
icon still didn't show for `@gramps@llamachile.tube`. Root cause: the identity memory updates (TASKS.md, ai.md, Services/index.md, HANDOFF.md). Ready to
domain `llamachile.tube` is YunoHost-SSO-gated — `/.well-known/nodeinfo`, commit when the user says so.
`/@gramps`, and webfinger all answer with the SSO login page, so nodeinfo - **Finished this session:** TASK 4 ship step 4 — the live audio capture layer:
returned nothing and the icon fell back to the honeycomb glyph. The real `IAudioSource`/`AudioSample` seam, `WasapiLoopbackAudioSource`
instance lives at `mastodon.llamachile.tube`, and only the bare root (`WasapiLoopbackCapture` on the default render device), `WasapiMicAudioSource`
`https://llamachile.tube/` 302s to it ("default app" redirect). Fix: (`WasapiCapture`, NAudio device resolved by `FriendlyName` matching
`HttpSocialValidator.TryFetchFediverseSoftwareAsync` now, when nodeinfo on the `MicSourceName` via a re-read `Func<string?>` provider, default-endpoint
identity domain fails, follows the root redirect (`ResolveInstanceHostAsync`, fallback), `AudioMixer` (owns both sources, starts/stops with go-live, feeds
reads `resp.RequestMessage.RequestUri.Host`) and re-runs the nodeinfo lookup the pure `AudioLevelMeter``MicLevelChanged`), `WaveToFloat` (IEEE-float /
on the resolved host. New test `FediverseHandle_RootRedirectToSubdomain_ResolvesSoftware`. PCM16 / extensible). `MainViewModel` constructs the mixer, `BeginGoLive`
Build: **0 warnings**. Tests: **112 passing**. success → `Start()`, `StopStream``Stop()`, `MicLevelChanged` marshalled to
- **What landed (all rounds this session):** the UI thread → `AudioLevel`. Loopback samples are currently dropped (the
1. **Fediverse icon resolution**`DetectService` maps `@user@domain` to the encoder's AAC mix, ship step 5, consumes them). Build: **0 warnings** (a
new `SocialService.Fediverse` (was `Link`/chain icon). On validate, pre-existing CS8602 in `FfmpegEncoder.cs:139` surfaced during the rebuild and
`HttpSocialValidator` best-effort GETs `https://{domain}/.well-known/nodeinfo`, was fixed with `process!`). Tests: **139 passing** (was 122; +17 new audio).
follows the first nodeinfo `links[].href`, reads `software.name`, and returns
it in `SocialLookupResult.FediverseSoftware`. `SocialEntry`/`SocialSlotViewModel`
carry `FediverseSoftware`; `SocialServiceIcons.LogoDataForFediverse(software)`
maps it to a bundled logo (mastodon/peertube/pixelfed/misskey/lemmy/pleroma/
firefish — Simple Icons CC0), falling back to the `FediverseIconData`
honeycomb glyph for unknown software (GoToSocial/Sharkey/Akkoma aren't in
Simple Icons). Nodeinfo failure still validates — the glyph falls back.
2. **Redirect resolution (round 3)** — identity domains that 302 their root
to the real instance (YunoHost default-app subdomains) resolve software via
the root redirect when identity-domain nodeinfo is SSO-blocked.
3. **Persistence** — new `SocialEntry.Software TEXT` column via
`MigrateSocialEntryTable()` (column-presence pattern, same as the others);
saved/loaded alongside Service/Handle/ProfileUrl.
4. **Icon colors**`IconButton` style gains `Foreground="#d0d0d0"`
(`Themes/Controls.xaml`); the dialog's trash button overrides
`Foreground="#e94560"` (`SocialsDialog.xaml`). All other `IconButton`
usages are `Path` content with explicit `Fill`, so unaffected.
- **Landmines:** - **Landmines:**
- `LayoutStore.Socials` is only populated by `Load()` — tests must call - `WaveFormatExtensible.SubFormat` is a **`Guid`** in NAudio 2.x, not a
`store.Load()` before asserting it. `WaveFormatEncoding` — compare it to
- `DetectService("justaname")`**Website** with empty handle; only `NAudio.Dmo.AudioMediaSubtypes.MEDIASUBTYPE_IEEE_FLOAT` (the IEEE-float
unparseable input or `@user@domain` yields Link/Fediverse. subtype constant; it lives in the `NAudio.Dmo` namespace, not CoreAudioApi).
- Dialog sign-in provider is `Func<Task<YouTubeChannel?>>`; test fakes must - Use the **`NAudio.Wasapi` 2.2.1 feature package**, not the `NAudio`
return `Task.FromResult` (sync-completed) so the fire-and-forget command meta-package — the wasapi types come from that package (`WasapiCapture` in
settles before the next assert. `NAudio.CoreAudioApi`, `WasapiLoopbackCapture` in `NAudio.Wave`); `NAudio.Core`
- `SocialBarBottomTop = 1040` literal lives in `MainWindow.xaml.cs` (the VM's comes in transitively.
`MasterFrameHeight` is private). - The audio meter is an exponential smoother (0.2 factor) — a single pushed
- `Cancel_AbortsInFlightValidation_WithoutMutatingSlot` relies on the sample only moves 20% toward its RMS. Tests must push repeatedly before
BlockingValidator's gate completing synchronously (no asserting a converged level.
`RunContinuationsAsynchronously`) — the assertions run after - Tests never instantiate `MainViewModel` directly except the round-clip
`Gate.TrySetResult` returns because the awaited continuation executes inline. integration test (a real `MainWindow`), which never goes live — so the mixer
- Nodeinfo cancellation test cancels mid-lookup via a counting stub; the is constructed but never started there; NAudio types are only constructed,
post-fetch `ct.IsCancellationRequested` check is what reports `Canceled`. never touching devices. Keep it that way.
- Round-3 redirect fallback fires only when identity-domain nodeinfo fails; - Mic device resolution must re-read `MicSourceName` at each `Start` (the app
the root follow happens automatically (HttpClient default auto-redirect), persists only the DisplayName, not a device ID).
and `RequestMessage.RequestUri.Host` is read from the final response. - Windows-only: all capture runs only while live (privacy indicator otherwise).
- **Next step:** TASK 4 ship step 3 — the encoder + RTMP push (FFmpeg subprocess:
frames via stdin, stderr health parsing, FLV mux + push to the cached reusable - **Next step:** TASK 4 ship step 5 — the frame-pipeline wiring
stream's ingestion URL). Nothing else queued — do not expand the task queue on (`CameraManager`/`ScreenCaptureManager` → compositor resolver → encoder
your own. construction, plus the loopback → encoder AAC mix). Nothing else queued — do
not expand the task queue on your own.
- **Secret/DB/port facts live:** OAuth client id/secret in `Helpers/OAuthCredentials.cs`; - **Secret/DB/port facts live:** OAuth client id/secret in `Helpers/OAuthCredentials.cs`;
OAuth session token in `Helpers/TokenStore.cs` (DPAPI → `%APPDATA%\ytLlive\ytLlive.auth`); OAuth session token in `Helpers/TokenStore.cs` (DPAPI → `%APPDATA%\ytLlive\ytLlive.auth`);
layout DB `%APPDATA%\ytLlive\ytLlive.db` (schema v8; the new `SocialEntry.Software` layout DB `%APPDATA%\ytLlive\ytLlive.db` (schema v8; `SocialEntry.Software`
column is a column-presence migration like the others, no version bump); OAuth callback column is a column-presence migration like the others, no version bump); OAuth callback
`http://localhost:8765/oauth2/callback`; crash log `%APPDATA%\ytLlive\startup.log`. `http://localhost:8765/oauth2/callback`; crash log `%APPDATA%\ytLlive\startup.log`.
+35
View File
@@ -0,0 +1,35 @@
namespace ytLive.Services.Audio;
/// <summary>
/// Computes the smoothed mic level (0..1) from captured samples. Pure —
/// unit-tested; the <see cref="AudioMixer"/> only feeds it and forwards the
/// result. RMS-based so it tracks perceived loudness, smoothed so the meter
/// does not flicker.
/// </summary>
public sealed class AudioLevelMeter
{
private const float Smoothing = 0.2f;
private float _level;
public float Level => _level;
public float Push(AudioSample sample)
{
if (sample.Samples.Length == 0)
return _level;
double sumSquares = 0;
var count = 0;
foreach (var value in sample.Samples)
{
sumSquares += value * value;
count++;
}
var rms = (float)Math.Sqrt(sumSquares / count);
_level = _level * (1 - Smoothing) + rms * Smoothing;
return _level;
}
public void Reset() => _level = 0;
}
+92
View File
@@ -0,0 +1,92 @@
using ytLive.Services;
namespace ytLive.Services.Audio;
/// <summary>
/// Owns the mic + desktop/game capture sources (TASK 4 ship step 4) and drives
/// the live mic meter. Runs only while live: started when go-live succeeds,
/// stopped on end-stream. Mic samples are level-metered and forwarded; loopback
/// samples are currently dropped (consumed by the encoder mix in a later step).
/// </summary>
public sealed class AudioMixer : IDisposable
{
private readonly IAudioSource _mic;
private readonly IAudioSource _loopback;
private readonly AudioLevelMeter _meter;
private readonly Action<string>? _log;
private bool _started;
public AudioMixer(IAudioSource mic, IAudioSource loopback, Action<string>? log = null)
{
_mic = mic;
_loopback = loopback;
_meter = new AudioLevelMeter();
_log = log;
_mic.SampleReady += OnMicSample;
_loopback.SampleReady += OnLoopbackSample;
_mic.Failed += OnMicFailed;
_loopback.Failed += OnLoopbackFailed;
}
/// <summary>Current smoothed mic level (0..1).</summary>
public float MicLevel => _meter.Level;
/// <summary>Raised whenever the smoothed mic level changes.</summary>
public event Action<float>? MicLevelChanged;
public void Start()
{
if (_started)
return;
_started = true;
_meter.Reset();
_loopback.Start();
_mic.Start();
}
public void Stop()
{
if (!_started)
return;
_started = false;
_mic.Stop();
_loopback.Stop();
_meter.Reset();
MicLevelChanged?.Invoke(0);
}
public void Dispose()
{
Stop();
_mic.SampleReady -= OnMicSample;
_loopback.SampleReady -= OnLoopbackSample;
_mic.Failed -= OnMicFailed;
_loopback.Failed -= OnLoopbackFailed;
_mic.Dispose();
_loopback.Dispose();
}
private void OnMicSample(AudioSample sample)
{
MicLevelChanged?.Invoke(_meter.Push(sample));
}
private void OnLoopbackSample(AudioSample sample)
{
// Desktop/game audio: captured for the future encoder mix; no UI yet.
}
private void OnMicFailed(Exception ex)
{
_log?.Invoke($"Mic capture failed: {ex.Message}");
MicLevelChanged?.Invoke(0);
}
private void OnLoopbackFailed(Exception ex)
{
_log?.Invoke($"Desktop audio capture failed: {ex.Message}");
}
}
+20
View File
@@ -0,0 +1,20 @@
namespace ytLive.Services.Audio;
/// <summary>
/// A chunk of interleaved PCM float samples (-1..1) with its format. This is
/// the unit every <see cref="IAudioSource"/> produces and the
/// <see cref="AudioMixer"/> consumes (TASK 4 ship step 4).
/// </summary>
public sealed class AudioSample
{
public float[] Samples { get; }
public int SampleRate { get; }
public int Channels { get; }
public AudioSample(float[] samples, int sampleRate, int channels)
{
Samples = samples;
SampleRate = sampleRate;
Channels = channels;
}
}
+22
View File
@@ -0,0 +1,22 @@
namespace ytLive.Services.Audio;
/// <summary>
/// A live audio capture source (TASK 4 ship step 4 seam): produces PCM float
/// chunks, runs only while live. The default implementations wrap NAudio's
/// WASAPI capture (mic) / loopback (desktop/game); the mixer and the tests
/// consume this interface, never NAudio directly.
/// </summary>
public interface IAudioSource : IDisposable
{
/// <summary>Starts capturing. Safe to call only once per Stop.</summary>
void Start();
/// <summary>Stops capturing; a later Start begins a fresh session.</summary>
void Stop();
/// <summary>Raises each captured chunk (interleaved PCM float, -1..1).</summary>
event Action<AudioSample>? SampleReady;
/// <summary>Raises when capture dies or fails to start (e.g. no device).</summary>
event Action<Exception>? Failed;
}
@@ -0,0 +1,69 @@
using NAudio.Wave;
namespace ytLive.Services.Audio;
/// <summary>
/// Captures the user's desktop/game audio (TASK 4 ship step 4) via WASAPI
/// loopback on the default render device. Starts/stops with go-live only.
/// </summary>
public sealed class WasapiLoopbackAudioSource : IAudioSource
{
private WasapiLoopbackCapture? _capture;
public event Action<AudioSample>? SampleReady;
public event Action<Exception>? Failed;
public void Start()
{
if (_capture != null)
throw new InvalidOperationException("Already started.");
try
{
_capture = new WasapiLoopbackCapture();
_capture.DataAvailable += OnDataAvailable;
_capture.RecordingStopped += OnRecordingStopped;
_capture.StartRecording();
}
catch (Exception ex)
{
Stop();
Failed?.Invoke(ex);
}
}
public void Stop()
{
if (_capture == null)
return;
try
{
_capture.StopRecording();
}
catch
{
}
_capture.Dispose();
_capture = null;
}
public void Dispose() => Stop();
private void OnDataAvailable(object? sender, WaveInEventArgs e)
{
if (e.BytesRecorded <= 0)
return;
var format = _capture?.WaveFormat ?? WaveFormat.CreateIeeeFloatWaveFormat(48000, 2);
var samples = WaveToFloat.Convert(e.Buffer, e.BytesRecorded, format);
if (samples.Length > 0)
SampleReady?.Invoke(new AudioSample(samples, format.SampleRate, format.Channels));
}
private void OnRecordingStopped(object? sender, StoppedEventArgs e)
{
if (e.Exception != null)
Failed?.Invoke(e.Exception);
}
}
+102
View File
@@ -0,0 +1,102 @@
using NAudio.CoreAudioApi;
using NAudio.Wave;
namespace ytLive.Services.Audio;
/// <summary>
/// Captures the selected microphone (TASK 4 ship step 4) via WASAPI. Resolves
/// the NAudio device by FriendlyName matching <c>MicSourceName</c> (the app only
/// persists DisplayName), falling back to the default capture endpoint.
/// </summary>
public sealed class WasapiMicAudioSource : IAudioSource
{
private readonly Func<string?> _micNameProvider;
private WasapiCapture? _capture;
/// <param name="micNameProvider">Returns the current mic DisplayName; read
/// at each Start so a device picked mid-session takes effect next go-live.</param>
public WasapiMicAudioSource(Func<string?> micNameProvider)
{
_micNameProvider = micNameProvider;
}
public event Action<AudioSample>? SampleReady;
public event Action<Exception>? Failed;
public void Start()
{
if (_capture != null)
throw new InvalidOperationException("Already started.");
try
{
var device = ResolveDevice();
_capture = device != null ? new WasapiCapture(device) : new WasapiCapture();
_capture.DataAvailable += OnDataAvailable;
_capture.RecordingStopped += OnRecordingStopped;
_capture.StartRecording();
}
catch (Exception ex)
{
Stop();
Failed?.Invoke(ex);
}
}
public void Stop()
{
if (_capture == null)
return;
try
{
_capture.StopRecording();
}
catch
{
}
_capture.Dispose();
_capture = null;
}
public void Dispose() => Stop();
private MMDevice? ResolveDevice()
{
var micName = _micNameProvider();
if (string.IsNullOrWhiteSpace(micName))
return null;
try
{
using var enumerator = new MMDeviceEnumerator();
foreach (var endpoint in enumerator.EnumerateAudioEndPoints(DataFlow.Capture, DeviceState.Active))
{
if (string.Equals(endpoint.FriendlyName, micName, StringComparison.OrdinalIgnoreCase))
return endpoint;
}
}
catch
{
}
return null;
}
private void OnDataAvailable(object? sender, WaveInEventArgs e)
{
if (e.BytesRecorded <= 0)
return;
var format = _capture?.WaveFormat ?? WaveFormat.CreateIeeeFloatWaveFormat(48000, 1);
var samples = WaveToFloat.Convert(e.Buffer, e.BytesRecorded, format);
if (samples.Length > 0)
SampleReady?.Invoke(new AudioSample(samples, format.SampleRate, format.Channels));
}
private void OnRecordingStopped(object? sender, StoppedEventArgs e)
{
if (e.Exception != null)
Failed?.Invoke(e.Exception);
}
}
+41
View File
@@ -0,0 +1,41 @@
using NAudio.Dmo;
using NAudio.Wave;
namespace ytLive.Services.Audio;
/// <summary>
/// Converts NAudio's raw WASAPI capture buffers (byte[], whatever bit depth the
/// device's mix format reports) into interleaved PCM float samples. Pure —
/// unit-tested; the two WASAPI sources share it.
/// </summary>
public static class WaveToFloat
{
public static float[] Convert(byte[] buffer, int bytesRecorded, WaveFormat format)
{
if (format.Encoding == WaveFormatEncoding.IeeeFloat && format.BitsPerSample == 32)
return ConvertIeeeFloat(buffer, bytesRecorded);
if (format.Encoding == WaveFormatEncoding.Pcm && format.BitsPerSample == 16)
return ConvertPcm16(buffer, bytesRecorded);
if (format is WaveFormatExtensible extensible
&& extensible.SubFormat == AudioMediaSubtypes.MEDIASUBTYPE_IEEE_FLOAT)
return ConvertIeeeFloat(buffer, bytesRecorded);
return ConvertPcm16(buffer, bytesRecorded);
}
private static float[] ConvertIeeeFloat(byte[] buffer, int bytesRecorded)
{
var count = bytesRecorded / 4;
var result = new float[count];
Buffer.BlockCopy(buffer, 0, result, 0, count * 4);
return result;
}
private static float[] ConvertPcm16(byte[] buffer, int bytesRecorded)
{
var count = bytesRecorded / 2;
var result = new float[count];
for (var i = 0; i < count; i++)
result[i] = BitConverter.ToInt16(buffer, i * 2) / 32768f;
return result;
}
}
+27
View File
@@ -0,0 +1,27 @@
namespace ytLive.Services.Encoder;
/// <summary>
/// Everything the FFmpeg subprocess encoder needs for one go-live (TASK 4 ship
/// step 3). <see cref="RtmpUrl"/> is the FULL ingestion URL — the reusable
/// stream's ingest address plus its stream key (<c>rtmp://a.rtmp.youtube.com/live2/&lt;key&gt;</c>).
/// Resolution/FPS/bitrate come from the active quality tier; audio is a silent
/// placeholder track until the WASAPI capture step replaces the input.
/// </summary>
public sealed class EncoderOptions
{
public string RtmpUrl { get; init; } = string.Empty;
public int Width { get; init; } = 1920;
public int Height { get; init; } = 1080;
public int Fps { get; init; } = 60;
public int BitrateKbps { get; init; } = 8000;
/// <summary>H.264 encoder name for <c>-c:v</c>; the encoder probes and prefers
/// nvenc → qsv → amf → libopenh264 when not forced.</summary>
public string? VideoEncoder { get; init; }
public int AudioSampleRate { get; init; } = 48000;
public int AudioChannels { get; init; } = 2;
/// <summary>GOP in frames = Fps × 4s — the YouTube keyframe ≤ 4s compliance bound.</summary>
public int GopSize => Fps * 4;
}
+46
View File
@@ -0,0 +1,46 @@
namespace ytLive.Services.Encoder;
/// <summary>
/// Builds the FFmpeg command line for a live RTMP push (TASK 4 ship step 3):
/// raw BGRA frames via stdin (paced <c>-re</c>), silent placeholder audio via lavfi
/// <c>anullsrc</c> (the WASAPI step replaces this input), H.264 + AAC encoding, FLV
/// muxing to the ingestion URL. Pure — the encoder just starts
/// <c>ffmpeg.exe [Build(...)]</c>.
/// </summary>
public static class FfmpegArgs
{
public static IReadOnlyList<string> Build(EncoderOptions options, string videoEncoder)
{
var gop = options.GopSize;
return
[
"-hide_banner",
"-loglevel", "info",
"-stats",
"-stats_period", "0.5",
"-re",
"-f", "rawvideo",
"-pix_fmt", "bgra",
"-video_size", $"{options.Width}x{options.Height}",
"-framerate", options.Fps.ToString(),
"-i", "pipe:0",
"-f", "lavfi",
"-i", $"anullsrc=channel_layout=stereo:sample_rate={options.AudioSampleRate}",
"-c:v", videoEncoder,
"-b:v", $"{options.BitrateKbps}k",
"-maxrate", $"{options.BitrateKbps}k",
"-bufsize", $"{options.BitrateKbps * 2}k",
"-g", gop.ToString(),
"-keyint_min", gop.ToString(),
"-sc_threshold", "0",
"-bf", "0",
"-pix_fmt", "yuv420p",
"-c:a", "aac",
"-b:a", "128k",
"-ar", options.AudioSampleRate.ToString(),
"-ac", options.AudioChannels.ToString(),
"-f", "flv",
options.RtmpUrl,
];
}
}
+263
View File
@@ -0,0 +1,263 @@
using System.Diagnostics;
using ytLive.Helpers;
using ytLive.Models;
namespace ytLive.Services.Encoder;
/// <summary>
/// The default <see cref="IFfmpegEncoder"/>: spawns <c>ffmpeg.exe</c> (resolved via
/// <see cref="IFfmpegLocator"/>), feeds raw BGRA frames into stdin, and parses the
/// <c>-stats</c> progress lines into <see cref="StreamHealth"/>. Encoder choice is
/// probed from the binary's <c>-encoders</c> listing (hardware NVENC/QSV/AMF first,
/// OpenH264 software fallback — never libx264, see the license posture) unless
/// <see cref="EncoderOptions.VideoEncoder"/> forces one.
///
/// Graceful stop = close stdin (EOF) → ffmpeg finalizes the FLV and exits by itself;
/// a watchdogs kill fires only if it hasn't exited shortly after EOF.
/// </summary>
public sealed class FfmpegEncoder : IFfmpegEncoder
{
public event EventHandler<StreamHealth>? HealthUpdated;
public event EventHandler<string>? ProcessFailed;
private readonly IFfmpegLocator _locator;
private readonly Func<IEncoderProcess> _processFactory;
private readonly object _gate = new();
private IEncoderProcess? _process;
private EncoderOptions? _options;
private StreamHealth _health = new() { Status = StreamStatus.Offline };
private Task? _stderrLoop;
private bool _stopRequested;
public FfmpegEncoder(
IFfmpegLocator locator,
Func<IEncoderProcess>? processFactory = null)
{
_locator = locator;
_processFactory = processFactory ?? (() => new FfmpegEncoderProcess());
}
public bool IsRunning { get; private set; }
public async Task StartAsync(EncoderOptions options, CancellationToken cancellationToken = default)
{
if (options == null) throw new ArgumentNullException(nameof(options));
if (string.IsNullOrWhiteSpace(options.RtmpUrl))
throw new ArgumentException("An RTMP ingestion URL is required.", nameof(options));
lock (_gate)
{
if (IsRunning) throw new InvalidOperationException("The encoder is already running.");
_options = options;
}
var ffmpegPath = await _locator.LocateAsync(cancellationToken).ConfigureAwait(false);
var encoder = options.VideoEncoder ?? await ProbeEncoderAsync(ffmpegPath, cancellationToken).ConfigureAwait(false);
var args = FfmpegArgs.Build(options, encoder);
var startInfo = new ProcessStartInfo
{
FileName = ffmpegPath,
UseShellExecute = false,
RedirectStandardInput = true,
RedirectStandardOutput = true,
RedirectStandardError = true,
CreateNoWindow = true,
};
foreach (var arg in args) startInfo.ArgumentList.Add(arg);
IEncoderProcess process;
try
{
process = _processFactory();
process.Start(startInfo);
}
catch (Exception ex)
{
AppLog.Write(ex, "FFmpeg encoder: failed to start subprocess");
lock (_gate) _options = null;
throw;
}
lock (_gate)
{
_process = process;
IsRunning = true;
_health = new StreamHealth { Status = StreamStatus.Streaming };
}
_stopRequested = false;
_stderrLoop = RunStderrLoopAsync(process);
}
/// <summary>
/// Write one raw BGRA frame to ffmpeg's stdin. Serialized internally; callers
/// (the compositor pump) may race freely. Frames are written as-is — the caller
/// paces to capture rate (the compositor's job, ship step 5).
/// </summary>
public async Task SubmitFrameAsync(VideoFrame frame, CancellationToken cancellationToken = default)
{
if (frame == null) throw new ArgumentNullException(nameof(frame));
IEncoderProcess? process;
lock (_gate)
{
if (!IsRunning) throw new InvalidOperationException("The encoder is not running.");
process = _process;
}
var bytes = frame.BgraPixels;
await process!.StandardInput.WriteAsync(bytes, cancellationToken).ConfigureAwait(false);
await process.StandardInput.FlushAsync(cancellationToken).ConfigureAwait(false);
}
public async Task StopAsync(CancellationToken cancellationToken = default)
{
IEncoderProcess? process;
Task? loop;
lock (_gate)
{
if (!IsRunning) return;
_stopRequested = true;
process = _process;
loop = _stderrLoop;
}
try
{
process!.StandardInput.Dispose(); // EOF → ffmpeg finalizes + exits
}
catch (Exception ex)
{
AppLog.Write(ex, "FFmpeg encoder: closing stdin failed");
}
try
{
using var timeout = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
timeout.CancelAfter(TimeSpan.FromSeconds(10));
await process!.WaitForExitAsync(timeout.Token).ConfigureAwait(false);
}
catch (OperationCanceledException)
{
AppLog.Write("FFmpeg encoder: did not exit after stdin EOF — killing");
process!.Kill();
}
catch (Exception ex)
{
AppLog.Write(ex, "FFmpeg encoder: waiting for exit failed");
process!.Kill();
}
try
{
if (loop != null) await loop.ConfigureAwait(false);
}
catch (Exception ex)
{
AppLog.Write(ex, "FFmpeg encoder: stderr loop faulted during stop");
}
process.Dispose();
lock (_gate)
{
IsRunning = false;
_health.Status = StreamStatus.Offline;
_health.LastError = null;
_process = null;
_options = null;
}
AppLog.Write($"FFmpeg encoder stopped (exit {process.ExitCode})");
}
public void Dispose()
{
lock (_gate)
{
if (!IsRunning) return;
_process?.Kill();
_process?.Dispose();
_process = null;
IsRunning = false;
}
}
private async Task<string> ProbeEncoderAsync(string ffmpegPath, CancellationToken cancellationToken)
{
try
{
var startInfo = new ProcessStartInfo
{
FileName = ffmpegPath,
UseShellExecute = false,
RedirectStandardOutput = true,
RedirectStandardError = true,
CreateNoWindow = true,
};
startInfo.ArgumentList.Add("-hide_banner");
startInfo.ArgumentList.Add("-encoders");
using var probe = _processFactory();
probe.Start(startInfo);
using var cts = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
cts.CancelAfter(TimeSpan.FromSeconds(15));
var output = await probe.StandardOutput.ReadToEndAsync(cts.Token).ConfigureAwait(false);
return FfmpegEncoderPicker.Pick(output);
}
catch (Exception ex)
{
AppLog.Write(ex, "FFmpeg encoder: encoder probe failed — falling back to software");
return FfmpegEncoderPicker.Preference[^1];
}
}
private Task RunStderrLoopAsync(IEncoderProcess process)
{
return Task.Run(async () =>
{
try
{
while (true)
{
var line = await process.StandardError.ReadLineAsync().ConfigureAwait(false);
if (line == null) break;
OnStderrLine(process, line);
}
var code = process.ExitCode;
var stillRunning = false;
lock (_gate) stillRunning = IsRunning;
if (stillRunning && !_stopRequested && code != 0)
{
AppLog.Write($"FFmpeg encoder: subprocess exited unexpectedly ({code})");
_health.LastError = $"FFmpeg exited with code {code}";
ProcessFailed?.Invoke(this, $"FFmpeg exited with code {code}");
}
}
catch (Exception ex)
{
AppLog.Write(ex, "FFmpeg encoder: stderr loop faulted");
}
});
}
private void OnStderrLine(IEncoderProcess process, string line)
{
var progress = FfmpegProgressParser.TryParse(line);
if (progress == null) return;
var dropped = Math.Max(0, (long)Math.Round(progress.Value.Fps * progress.Value.Duration.TotalSeconds) - progress.Value.Frame);
lock (_gate)
{
_health.CurrentBitrate = progress.Value.BitrateKbps;
_health.FPS = progress.Value.Fps;
_health.DroppedFrames = (int)dropped;
_health.StreamDuration = progress.Value.Duration;
}
HealthUpdated?.Invoke(this, _health);
}
}
+35
View File
@@ -0,0 +1,35 @@
namespace ytLive.Services.Encoder;
/// <summary>
/// Picks the best available H.264 encoder from ffmpeg's <c>-encoders</c> listing,
/// honoring the license posture (no GPL libx264): hardware NVENC → QSV → AMF, then
/// the OpenH264 software fallback. Pure parser — the <c>-encoders</c> probe output is
/// fetched by the encoder via an <see cref="IEncoderProcess"/> and fed here.
/// </summary>
public static class FfmpegEncoderPicker
{
/// <summary>Preference order, best first. All ship in the pinned BtbN lgpl-shared build.</summary>
public static readonly string[] Preference =
[
"h264_nvenc",
"h264_qsv",
"h264_amf",
"libopenh264",
];
/// <summary>
/// First <see cref="Preference"/> entry present in the probe output, or the
/// software fallback (which the pinned build always contains) if none matched.
/// Never returns libx264 — it is GPL and would contaminate the paid product.
/// </summary>
public static string Pick(string probeOutput)
{
var available = probeOutput.Split('\n');
foreach (var name in Preference)
{
if (available.Any(line => line.Contains(name, StringComparison.Ordinal)))
return name;
}
return Preference[^1];
}
}
+38
View File
@@ -0,0 +1,38 @@
using System.Diagnostics;
using System.IO;
namespace ytLive.Services.Encoder;
/// <summary>
/// The real <see cref="IEncoderProcess"/>: a <see cref="Process"/> with all three
/// std streams redirected. Constructed by <see cref="FfmpegEncoder"/> for both the
/// encoder subprocess and the <c>-encoders</c> probe.
/// </summary>
public sealed class FfmpegEncoderProcess : IEncoderProcess
{
private readonly Process _process;
public FfmpegEncoderProcess() => _process = new Process { EnableRaisingEvents = true };
public void Start(ProcessStartInfo startInfo)
{
_process.StartInfo = startInfo;
_process.Start();
}
public Stream StandardInput => _process.StandardInput.BaseStream;
public TextReader StandardOutput => _process.StandardOutput;
public TextReader StandardError => _process.StandardError;
public bool HasExited => _process.HasExited;
public int ExitCode => _process.ExitCode;
public void Kill()
{
if (!_process.HasExited) _process.Kill();
}
public Task WaitForExitAsync(CancellationToken cancellationToken = default)
=> _process.WaitForExitAsync(cancellationToken);
public void Dispose() => _process.Dispose();
}
+47
View File
@@ -0,0 +1,47 @@
using System.Text.RegularExpressions;
namespace ytLive.Services.Encoder;
/// <summary>A decoded ffmpeg <c>-stats</c> progress line (pure data).</summary>
public readonly record struct FfmpegProgress(
long Frame,
double Fps,
double BitrateKbps,
TimeSpan Duration,
long SizeBytes);
/// <summary>
/// Pure parser for ffmpeg's periodic <c>frame= fps= size= time= bitrate=</c> stderr
/// lines (the <c>-stats</c>/<c>-stats_period</c> output). Unit-tested in isolation
/// so the encoder loop stays a thin wire.
/// </summary>
public static class FfmpegProgressParser
{
// frame= 123 fps= 59.9 q=28.0 size= 1024KiB time=00:00:02.04 bitrate= 4000.1kbits/s speed=1.00x
private static readonly Regex Line = new(
@"frame=\s*(?<frame>\d+)\s+fps=\s*(?<fps>[\d.]+).*?"
+ @"size=\s*(?<size>\d+)KiB.*?"
+ @"time=(?<time>\d{2}):(?<min>\d{2}):(?<sec>[\d.]+).*?"
+ @"bitrate=\s*(?<bitrate>[\d.]+)kbits/s",
RegexOptions.Compiled | RegexOptions.CultureInvariant);
/// <summary>Returns null for non-progress lines (errors, warnings, banner).</summary>
public static FfmpegProgress? TryParse(string line)
{
if (string.IsNullOrWhiteSpace(line)) return null;
var m = Line.Match(line);
if (!m.Success) return null;
var frame = long.Parse(m.Groups["frame"].Value);
var fps = double.Parse(m.Groups["fps"].Value, System.Globalization.CultureInfo.InvariantCulture);
var size = long.Parse(m.Groups["size"].Value);
var bitrate = double.Parse(m.Groups["bitrate"].Value, System.Globalization.CultureInfo.InvariantCulture);
var hours = int.Parse(m.Groups["time"].Value);
var minutes = int.Parse(m.Groups["min"].Value);
var seconds = double.Parse(m.Groups["sec"].Value, System.Globalization.CultureInfo.InvariantCulture);
var duration = TimeSpan.FromSeconds(hours * 3600 + minutes * 60 + seconds);
return new FfmpegProgress(frame, fps, bitrate, duration, size * 1024);
}
}
+26
View File
@@ -0,0 +1,26 @@
using System.Diagnostics;
using System.IO;
namespace ytLive.Services.Encoder;
/// <summary>
/// Seam around a spawned subprocess (the encoder's ffmpeg and the encoder probe).
/// Exposes the redirected stdin (binary frames), stdout (probe listing) and stderr
/// (progress lines) plus exit control, so the encoder logic never touches
/// <c>System.Diagnostics.Process</c> directly and tests can fake the whole thing.
/// </summary>
public interface IEncoderProcess : IDisposable
{
void Start(ProcessStartInfo startInfo);
/// <summary>Raw binary stdin — the encoder writes BGRA frames here.</summary>
Stream StandardInput { get; }
TextReader StandardOutput { get; }
TextReader StandardError { get; }
bool HasExited { get; }
int ExitCode { get; }
void Kill();
Task WaitForExitAsync(CancellationToken cancellationToken = default);
}
+25
View File
@@ -0,0 +1,25 @@
using ytLive.Models;
namespace ytLive.Services.Encoder;
/// <summary>
/// The live encoder seam (TASK 4 ship step 3): start an FFmpeg subprocess that
/// encodes raw BGRA frames from stdin and pushes FLV to an RTMP ingestion URL,
/// raising parsed health stats from stderr. The frame producer (compositor →
/// capture managers, TASK 4 ship step 5) feeds <see cref="SubmitFrameAsync"/> at
/// capture rate; this service serializes writes, parses progress, and tears the
/// process down gracefully. Constructor-injected <see cref="IFfmpegLocator"/> +
/// process factory keep it hermetic (tests fake both).
/// </summary>
public interface IFfmpegEncoder : IDisposable
{
/// <summary>Fires on each parsed ffmpeg <c>-stats</c> progress line (~2 Hz).</summary>
event EventHandler<StreamHealth>? HealthUpdated;
/// <summary>Fires when the subprocess dies unexpectedly (non-zero exit while live).</summary>
event EventHandler<string>? ProcessFailed;
Task StartAsync(EncoderOptions options, CancellationToken cancellationToken = default);
Task SubmitFrameAsync(VideoFrame frame, CancellationToken cancellationToken = default);
Task StopAsync(CancellationToken cancellationToken = default);
}
+15
View File
@@ -33,7 +33,22 @@ External-facing logic: YouTube API, persistence. See
| `Compositor/StaticPixelCache.cs` | Asset bytes → cached BGRA8 `VideoFrame`, decoded once per content hash — the output path's raw-pixel source (the preview uses ImageCache's `BitmapImage`) | | `Compositor/StaticPixelCache.cs` | Asset bytes → cached BGRA8 `VideoFrame`, decoded once per content hash — the output path's raw-pixel source (the preview uses ImageCache's `BitmapImage`) |
| `Encoder/IFfmpegLocator.cs` | **TASK 4 ship step 2 seam**: resolves an absolute path to `ffmpeg.exe` (PATH first → cached `%APPDATA%\ytLlive\tools\ffmpeg.exe` → pinned BtbN LGPL-shared zip download). The encoder step and tests fake it | | `Encoder/IFfmpegLocator.cs` | **TASK 4 ship step 2 seam**: resolves an absolute path to `ffmpeg.exe` (PATH first → cached `%APPDATA%\ytLlive\tools\ffmpeg.exe` → pinned BtbN LGPL-shared zip download). The encoder step and tests fake it |
| `SocialValidator.cs` | `ISocialValidator` seam + `HttpSocialValidator` default: validates a social handle/URL by constructing the canonical profile URL and issuing an HTTP GET — the "act of validation" before adding to the bar. 200/redirect = exists; 404/connection failure = rejected. Fediverse `@user@domain` additionally does a best-effort nodeinfo lookup (`/.well-known/nodeinfo``software.name`) so the entry can show the instance's real logo; nodeinfo failure still validates (generic fediverse glyph). If the identity domain's nodeinfo is blocked (SSO) but the bare root 302s to the real instance (YunoHost default-app subdomains), the lookup follows the redirect and asks that host instead. Constructor takes optional `HttpClient` for tests | | `SocialValidator.cs` | `ISocialValidator` seam + `HttpSocialValidator` default: validates a social handle/URL by constructing the canonical profile URL and issuing an HTTP GET — the "act of validation" before adding to the bar. 200/redirect = exists; 404/connection failure = rejected. Fediverse `@user@domain` additionally does a best-effort nodeinfo lookup (`/.well-known/nodeinfo``software.name`) so the entry can show the instance's real logo; nodeinfo failure still validates (generic fediverse glyph). If the identity domain's nodeinfo is blocked (SSO) but the bare root 302s to the real instance (YunoHost default-app subdomains), the lookup follows the redirect and asks that host instead. Constructor takes optional `HttpClient` for tests |
| `Encoder/EncoderOptions.cs` | One go-live's encoder config: full `RtmpUrl` (ingest + stream key), W×H/FPS/bitrate from the quality tier, `VideoEncoder` (null = probe), audio sample rate/channels, `GopSize` = FPS×4 (the ≤4s keyframe bound) |
| `Encoder/IFfmpegEncoder.cs` | **TASK 4 ship step 3 seam**: start/feed/stop the live encoder; `HealthUpdated` (parsed `StreamHealth`), `ProcessFailed` on unexpected non-zero exit. Constructor-injected locator + process factory (tests fake both) |
| `Encoder/FfmpegEncoder.cs` | The default encoder: spawn `ffmpeg.exe` (probe `-encoders` first → hardware NVENC/QSV/AMF, OpenH264 fallback — never libx264), feed raw BGRA frames into stdin, parse `-stats` lines into `StreamHealth`, graceful stop via stdin EOF + 10s kill watchdog. Not yet constructed by the app (ship step 5 wires it) |
| `Encoder/IEncoderProcess.cs` | Seam around the spawned subprocess (probe + encoder): redirected stdin/stdout/stderr + exit control — the encoder never touches `Process` directly |
| `Encoder/FfmpegEncoderProcess.cs` | The real process wrapper (`Process` with all three std streams redirected) |
| `Encoder/FfmpegArgs.cs` | Pure FFmpeg command-line builder: rawvideo `pipe:0` input, `anullsrc` silent audio (WASAPI replaces it), H.264+AAC, closed GOP (`-g fps×4 -keyint_min -sc_threshold 0 -bf 0`, `yuv420p`), FLV → RTMP |
| `Encoder/FfmpegProgressParser.cs` | Pure parser for `frame=/fps=/size=/time=/bitrate=` stats lines → `FfmpegProgress` |
| `Encoder/FfmpegEncoderPicker.cs` | Pure H.264 encoder picker from `-encoders` output: NVENC → QSV → AMF → OpenH264; **never returns libx264** (GPL) |
| `Encoder/FfmpegLocator.cs` | The default locator: PATH probe wins, then the cache, then pull+extract (`ffmpeg.exe` **+ the `libav*.dll` family** via a staging dir, failures via `AppLog`). Pinned to BtbN `autobuild-2026-08-09-13-03` **lgpl-shared** (dynamic linking = LGPL compliance without static-relink §6 material; no GPL libx264; NVENC/QSV/AMF + libopenh264 + native AAC — see `ai.md` → Licensing). Search dirs / tools dir / downloader constructor-injected for hermetic tests. Not yet constructed by the app (the encoder step wires it) | | `Encoder/FfmpegLocator.cs` | The default locator: PATH probe wins, then the cache, then pull+extract (`ffmpeg.exe` **+ the `libav*.dll` family** via a staging dir, failures via `AppLog`). Pinned to BtbN `autobuild-2026-08-09-13-03` **lgpl-shared** (dynamic linking = LGPL compliance without static-relink §6 material; no GPL libx264; NVENC/QSV/AMF + libopenh264 + native AAC — see `ai.md` → Licensing). Search dirs / tools dir / downloader constructor-injected for hermetic tests. Not yet constructed by the app (the encoder step wires it) |
| `Audio/IAudioSource.cs` | **TASK 4 ship step 4 seam**: live capture source — `Start`/`Stop`/`SampleReady(AudioSample)`/`Failed(Exception)`, `IDisposable`; runs only while live. The app consumes this seam; tests inject hermetic fakes |
| `Audio/AudioSample.cs` | One captured chunk: interleaved PCM float (-1..1) + sample rate + channels |
| `Audio/WasapiLoopbackAudioSource.cs` | Desktop/game capture: NAudio `WasapiLoopbackCapture` on the default render device — automatic at unity, zero UI |
| `Audio/WasapiMicAudioSource.cs` | Mic capture: NAudio `WasapiCapture` with the device resolved by `FriendlyName` matching `MicSourceName` (DisplayName only), default capture endpoint fallback; name provider `Func<string?>` re-read at each `Start` so a mic picked mid-session takes effect next go-live |
| `Audio/AudioMixer.cs` | Owns both sources; `Start`/`Stop` follow go-live (`MainViewModel.BeginGoLive`/`StopStream`). Mic samples → `AudioLevelMeter``MicLevelChanged`; loopback samples currently dropped (the future encoder AAC mix consumes them). Failures log via `AppLog`; mic failure zeroes the meter, loopback failure doesn't kill the mic |
| `Audio/AudioLevelMeter.cs` | Pure smoothed RMS level (0..1): `Push(AudioSample)` + `Reset` — the unit-tested math behind `AudioLevel` |
| `Audio/WaveToFloat.cs` | Pure WASAPI buffer → float conversion: IEEE float 32-bit direct, PCM 16-bit normalized, `WaveFormatExtensible` IEEE-float subformat GUID, trailing partial samples ignored |
Related: constructed in [`ViewModels/MainViewModel.cs`](../ViewModels/MainViewModel.cs) Related: constructed in [`ViewModels/MainViewModel.cs`](../ViewModels/MainViewModel.cs)
(no DI container yet). Models in [`Models/index.md`](../Models/index.md). (no DI container yet). Models in [`Models/index.md`](../Models/index.md).
+67 -2
View File
@@ -199,8 +199,8 @@ Preview shows the transition too (WYSIWYG). No wipes/slides/LUTs beyond the four
1.**Ship step 1 — the output compositor SHIPPED** (2026-08-10) 1.**Ship step 1 — the output compositor SHIPPED** (2026-08-10)
2.**Ship step 2 — the FFmpeg locator SHIPPED** (2026-08-10) 2.**Ship step 2 — the FFmpeg locator SHIPPED** (2026-08-10)
3. **Encoder + RTMP push** — the FFmpeg subprocess: frames via stdin, stderr health parsing, FLV mux + push to the cached reusable stream's ingestion URL 3. **Encoder + RTMP push SHIPPED** (2026-08-12) — the FFmpeg subprocess: raw BGRA frames via stdin, stderr health parsing, FLV mux + push to the ingestion URL (see the ship step 3 plan below)
4. **WASAPI audio capture** loopback (desktop/game) + the picked mic feeding `AudioLevel` so the realtime meter comes alive (req 7) 4. **WASAPI audio capture SHIPPED** (2026-08-12) — NAudio loopback (desktop/game) + the picked mic feeding `AudioLevel`, so the realtime meter comes alive (see the ship step 4 plan below)
5.**Frame-pipeline wiring**`CameraManager`/`ScreenCaptureManager` → compositor resolver → encoder 5.**Frame-pipeline wiring**`CameraManager`/`ScreenCaptureManager` → compositor resolver → encoder
6.**Health stats** — bitrate, FPS, dropped frames reported live in the bottom bar (req 5) 6.**Health stats** — bitrate, FPS, dropped frames reported live in the bottom bar (req 5)
7.**One-click go live + private-only enforcement** — Go Live always creates/updates the broadcast with `privacyStatus = "private"` + PRIVATE badge (req 8, test-verifiable) 7.**One-click go live + private-only enforcement** — Go Live always creates/updates the broadcast with `privacyStatus = "private"` + PRIVATE badge (req 8, test-verifiable)
@@ -361,6 +361,71 @@ Tests: the hermetic `FfmpegLocatorTests` integration test (PATH → cache → do
fake downloader serving a real in-memory zip) + edge/unit cases (shared-build DLL extraction, zero-byte fake downloader serving a real in-memory zip) + edge/unit cases (shared-build DLL extraction, zero-byte
cache refresh, empty payload, missing zip entry, downloader failure) — **78 passing**. cache refresh, empty payload, missing zip entry, downloader failure) — **78 passing**.
#### Ship step 3 — Encoder + RTMP push (the FFmpeg subprocess)
**Goal:** encode raw BGRA master frames into H.264+AAC FLV and push them to the reusable stream's RTMP
ingestion URL — one battle-tested subprocess doing encode + mux + push + reconnect, the app feeding
frames via stdin and parsing stderr for health (req 2).
**Decisions (locked):** the encoder is a thin orchestrator over `ffmpeg.exe` — no H.264/AAC code in the
app. Arguments (pure `FfmpegArgs.Build`): `-re -f rawvideo -pix_fmt bgra -video_size WxH -framerate FPS
-i pipe:0` (frames in), a **silent placeholder audio track** via `-f lavfi -i anullsrc` (the WASAPI
capture step replaces this input), `-c:v <encoder> -b:v K -maxrate K -bufsize 2K` + **`-g fps×4`
`-keyint_min fps×4` `-sc_threshold 0` `-bf 0` `-pix_fmt yuv420p`** (the keyframe ≤4s / closed-GOP /
H.264 compliance), `-c:a aac -ar 48000 -ac 2`, `-f flv <rtmpUrl>`. Encoder choice is **probed from the
binary's `-encoders` listing** (`FfmpegEncoderPicker`, pure): hardware NVENC → QSV → AMF, then OpenH264
software fallback — **never libx264** (GPL; see `ai.md` → Licensing). The seam (`IFfmpegEncoder` +
`IEncoderProcess`, constructor-injected locator + process factory) keeps it hermetic — tests fake the
whole subprocess (probe + encoder), no real binary.
**Behavior:** `StartAsync` (locate → probe → spawn → stderr loop), `SubmitFrameAsync` (serialized BGRA
stdin writes, ~2 Hz health via `HealthUpdated`/`StreamHealth` — bitrate/FPS/duration/dropped-from-frame-
count), `StopAsync` (stdin EOF → ffmpeg finalizes + exits by itself; 10s watchdog kill), `ProcessFailed`
on a non-zero unexpected exit.
**Built (2026-08-12):** `EncoderOptions` + `IFfmpegEncoder`/`FfmpegEncoder` + `IEncoderProcess`/
`FfmpegEncoderProcess` + pure `FfmpegArgs`/`FfmpegProgressParser`/`FfmpegEncoderPicker` in
`Services/Encoder/`. Not yet constructed by the app (the frame-pipeline wiring, ship step 5, owns it).
Tests: `FfmpegEncoderTests` integration (probe → spawn with NVENC preferred → frames into stdin →
progress parsed → graceful stop, no kill) + units (args compliance/GOP, progress parser, picker
preference + GPL guard, no-URL/not-running/noop stops, process-death `ProcessFailed`) — **122 passing**.
#### Ship step 4 — WASAPI audio capture (the meter comes alive)
**Goal:** capture desktop/game audio (loopback) and the picked mic, feed the mic level into `AudioLevel`
so the realtime meter reads something other than 0, run capture **only while live** (req 7).
**Decisions (locked):**
1. **NAudio `NAudio.Wasapi` 2.2.1** — the wasapi feature package (not the `NAudio` meta-package): it
carries the capture types (`WasapiCapture`/`WasapiLoopbackCapture` + the MMDevice enumeration) with
`NAudio.Core`/`NAudio.Asio` pulled in transitively. MIT — recorded in `THIRD-PARTY-NOTICES.txt` (item 9).
2. **`IAudioSource` seam** (`Start`/`Stop`/`SampleReady`/`Failed`, IDisposable) — the app consumes the
seam; the two WASAPI implementations wrap NAudio; tests inject hermetic fakes (no real audio devices,
no timers). Loopback = `WasapiLoopbackCapture` on the default render device; mic = `WasapiCapture`
with the NAudio device resolved by `FriendlyName` matching `MicSourceName` (the app only persists the
DisplayName), falling back to the default capture endpoint. Mic device resolution is re-read at each
`Start` via a name provider so a mic picked mid-session takes effect next go-live.
3. **`AudioMixer` owns both sources** — starts/stops both with go-live (`BeginGoLive` success → `Start`,
`StopStream` → `Stop`). Mic samples feed a pure `AudioLevelMeter` (RMS, exponential smoothing) and
raise `MicLevelChanged`, marshalled to the UI thread into `AudioLevel`; desktop samples are currently
dropped (consumed by the encoder's AAC mix in a later step). Capture failures are logged via `AppLog`
(mic failure also zeroes the meter); loopback failure doesn't kill the mic.
4. **Byte→float** — pure `WaveToFloat.Convert` handles the WASAPI mix formats: IEEE float 32-bit (direct)
and PCM 16-bit (normalized to -1..1), including `WaveFormatExtensible` with the IEEE-float subformat
GUID. Trailing partial samples are ignored.
**Built (2026-08-12):** `Services/Audio/` ships `IAudioSource` + `AudioSample`, `WasapiLoopbackAudioSource`,
`WasapiMicAudioSource`, `AudioMixer`, `AudioLevelMeter`, `WaveToFloat`; `MainViewModel` constructs the
mixer (mic source fed `() => MicSourceName`), starts it on go-live and stops it on end-stream, and maps
`MicLevelChanged` → `AudioLevel`. A pre-existing CS8602 in `FfmpegEncoder.cs:139` surfaced during this
step's rebuild and was fixed (`process!`) — build **0 warnings**. Tests: `AudioMixerTests` (mixer
lifecycle/forwarding/failure against fakes, meter RMS/smoothing/reset, `WaveToFloat` float/PCM16/
extensible/truncation) — **139 passing**.
**Deferred (later ship steps):** wiring the desktop-capture samples into the encoder's AAC mix (replaces
the `-f lavfi -i anullsrc` placeholder; ship step 5 owns the encoder construction), WASAPI capture while
not live, and any audio UI beyond the existing mic controls.
--- ---
## TASK 5 — YouTube Live Stream Management ## TASK 5 — YouTube Live Stream Management
+8
View File
@@ -79,6 +79,14 @@ the guardrails — the "never do" list is there on purpose.
obligations; it is listed here per this file's "list everything" obligations; it is listed here per this file's "list everything"
policy. policy.
9. NAudio (WASAPI audio capture — loopback + mic)
Copyright (c) Mark Heath and contributors
License: MIT
Home: https://github.com/naudio/NAudio
Used as: the WASAPI loopback (desktop/game audio) and mic capture sources
behind the `IAudioSource` seam (TASK 4 ship step 4). MIT imposes
no source offer; this notice is kept per this file's policy.
-------------------------------------------------------------------------------- --------------------------------------------------------------------------------
MISCELLANEOUS MISCELLANEOUS
+26
View File
@@ -14,6 +14,7 @@ using Microsoft.Win32;
using ytLive.Helpers; using ytLive.Helpers;
using ytLive.Models; using ytLive.Models;
using ytLive.Services; using ytLive.Services;
using ytLive.Services.Audio;
namespace ytLive.ViewModels; namespace ytLive.ViewModels;
@@ -79,6 +80,12 @@ public class MainViewModel : ViewModelBase
private SocialsConfig? _socials; private SocialsConfig? _socials;
private readonly IMicrophoneEnumerator _microphoneEnumerator; private readonly IMicrophoneEnumerator _microphoneEnumerator;
// Live audio capture (TASK 4 ship step 4): desktop/game via WASAPI loopback,
// mic via WASAPI capture, both owned by the mixer and running only while
// live. Mic level feeds AudioLevel (the meter); loopback is for the future
// encoder mix. Private by design — no UI beyond the existing mic controls.
private readonly AudioMixer _audioMixer;
// Screen backdrop: a permanent live capture (desktop/game) that every scene // Screen backdrop: a permanent live capture (desktop/game) that every scene
// shows at the bottom layer. One shared capture session per key — the // shows at the bottom layer. One shared capture session per key — the
// ScreenCaptureManager refcounts by key, mirroring CameraManager. // ScreenCaptureManager refcounts by key, mirroring CameraManager.
@@ -832,6 +839,12 @@ public class MainViewModel : ViewModelBase
_microphoneEnumerator = new WinRtMicrophoneEnumerator(); _microphoneEnumerator = new WinRtMicrophoneEnumerator();
_audioMixer = new AudioMixer(
new WasapiMicAudioSource(() => MicSourceName),
new WasapiLoopbackAudioSource(),
message => AppLog.Write(message));
_audioMixer.MicLevelChanged += OnMicLevelChanged;
_fullScreenDetector = new Win32FullScreenDetector(); _fullScreenDetector = new Win32FullScreenDetector();
foreach (var display in _fullScreenDetector.GetDisplays()) foreach (var display in _fullScreenDetector.GetDisplays())
Displays.Add(display); Displays.Add(display);
@@ -1725,6 +1738,7 @@ public class MainViewModel : ViewModelBase
? "ytLlive" ? "ytLlive"
: $"{dialog.StreamTitle} — ytLlive"; : $"{dialog.StreamTitle} — ytLlive";
StreamStatus = StreamStatus.Streaming; StreamStatus = StreamStatus.Streaming;
_audioMixer.Start();
} }
} }
@@ -1732,6 +1746,7 @@ public class MainViewModel : ViewModelBase
{ {
StreamStatus = StreamStatus.Offline; StreamStatus = StreamStatus.Offline;
WindowTitle = "ytLlive"; WindowTitle = "ytLlive";
_audioMixer.Stop();
// Graceful end completes the session = signs out (the DPAPI token is // Graceful end completes the session = signs out (the DPAPI token is
// cleared so the next Start Stream requires a fresh sign-in). A crash // cleared so the next Start Stream requires a fresh sign-in). A crash
// never runs this, so the token survives and the creator stays signed in. // never runs this, so the token survives and the creator stays signed in.
@@ -1742,6 +1757,17 @@ public class MainViewModel : ViewModelBase
AppLog.Write("Stream ended; session signed out"); AppLog.Write("Stream ended; session signed out");
} }
private void OnMicLevelChanged(float level)
{
// NAudio raises on its capture thread; marshal to the UI thread so the
// meter binding updates safely.
var dispatcher = System.Windows.Application.Current?.Dispatcher;
if (dispatcher != null && !dispatcher.CheckAccess())
dispatcher.BeginInvoke(() => AudioLevel = level);
else
AudioLevel = level;
}
private void UpdateLiveVisuals() private void UpdateLiveVisuals()
{ {
var live = IsLive; var live = IsLive;
+54 -2
View File
@@ -91,7 +91,7 @@ C# / WPF (.NET 8) following MVVM:
|------|------| |------|------|
| `Models/` | Plain data types — Scene, Source (incl. `ClipShape`, `IsMirrored`, `VideoImageSource`), QualityOption, StreamConfig, StreamHealth, YouTubeChannel, ChatMessage, **Socials (`SocialService` enum + `SocialEntry`/`SocialsConfig` + `SocialServiceIcons`) — the social bar** | | `Models/` | Plain data types — Scene, Source (incl. `ClipShape`, `IsMirrored`, `VideoImageSource`), QualityOption, StreamConfig, StreamHealth, YouTubeChannel, ChatMessage, **Socials (`SocialService` enum + `SocialEntry`/`SocialsConfig` + `SocialServiceIcons`) — the social bar** |
| `ViewModels/` | MainViewModel — exposes collections + commands for the UI; GoLiveViewModel, ReuseImageViewModel, CameraPickerViewModel, **SocialsDialogViewModel** | | `ViewModels/` | MainViewModel — exposes collections + commands for the UI; GoLiveViewModel, ReuseImageViewModel, CameraPickerViewModel, **SocialsDialogViewModel** |
| `Services/` | YouTube OAuth2, stream/broadcast management, live chat polling, LayoutStore (SQLite), **SocialValidator (`ISocialValidator` seam + `HttpSocialValidator` default)**, **webcam: `VideoFrame` seam + `CameraDeviceInfo`/`ICameraEnumerator`/`ICameraFrameSource` interfaces + `MediaCaptureCameraEnumerator`/`MediaCaptureFrameSource` (WinRT) + `CameraManager`**, **screen capture: `IFullScreenDetector`/`Win32FullScreenDetector` + `IScreenCaptureSource`/`ScreenCaptureFrameSource` (WinRT GraphicsCapture) + `ScreenCaptureManager` + `ScreenCaptureSourceFactory` + `Direct3D11Helper`/`CaptureInterop` (COM bridges)** | | `Services/` | YouTube OAuth2, stream/broadcast management, live chat polling, LayoutStore (SQLite), **SocialValidator (`ISocialValidator` seam + `HttpSocialValidator` default)**, **webcam: `VideoFrame` seam + `CameraDeviceInfo`/`ICameraEnumerator`/`ICameraFrameSource` interfaces + `MediaCaptureCameraEnumerator`/`MediaCaptureFrameSource` (WinRT) + `CameraManager`**, **screen capture: `IFullScreenDetector`/`Win32FullScreenDetector` + `IScreenCaptureSource`/`ScreenCaptureFrameSource` (WinRT GraphicsCapture) + `ScreenCaptureManager` + `ScreenCaptureSourceFactory` + `Direct3D11Helper`/`CaptureInterop` (COM bridges)**, **audio: `IAudioSource` seam + `WasapiLoopbackAudioSource`/`WasapiMicAudioSource` (NAudio WASAPI) + `AudioMixer` + pure `AudioLevelMeter`/`WaveToFloat` (see "Live audio capture")** |
| `Helpers/` | ViewModelBase (INotifyPropertyChanged), RelayCommand, ImageCache, AppLog (file logger), FocusPreservingListBox, OAuthCredentials, **TokenStore (DPAPI session persistence)**, visibility converters | | `Helpers/` | ViewModelBase (INotifyPropertyChanged), RelayCommand, ImageCache, AppLog (file logger), FocusPreservingListBox, OAuthCredentials, **TokenStore (DPAPI session persistence)**, visibility converters |
| `Themes/` | `Controls.xaml` — the single dark-theme source, merged once in `App.xaml` (see `Themes/index.md`) | | `Themes/` | `Controls.xaml` — the single dark-theme source, merged once in `App.xaml` (see `Themes/index.md`) |
| `MainWindow.xaml` | Dark theme; layout: top bar (controls), center (preview), left (scenes/sources), right (chat), bottom (health) | | `MainWindow.xaml` | Dark theme; layout: top bar (controls), center (preview), left (scenes/sources), right (chat), bottom (health) |
@@ -114,7 +114,7 @@ C# / WPF (.NET 8) following MVVM:
- `Helpers/OAuthCredentials.cs` contains the real ClientId/ClientSecret. Auth is complete and the session **persists via Windows DPAPI** (`Helpers/TokenStore.cs``%APPDATA%\ytLlive\ytLlive.auth`, CurrentUser scope), reloaded best-effort at startup with a proactive refresh of a near-expiry access token. Sign-in/Change Account lives **inside the Start Stream dialog** (two-state flow — no separate Connect button). A **graceful End Livestream signs out**: `StopStream()` clears the session + token, so the next go-live needs a fresh sign-in; a crash never runs End, so the token survives and the creator stays signed in. `YouTubeAuthService` takes an optional `HttpClient` + `sessionChanged` callback (test seam + save hook; services are still constructed in `MainViewModel`) - `Helpers/OAuthCredentials.cs` contains the real ClientId/ClientSecret. Auth is complete and the session **persists via Windows DPAPI** (`Helpers/TokenStore.cs``%APPDATA%\ytLlive\ytLlive.auth`, CurrentUser scope), reloaded best-effort at startup with a proactive refresh of a near-expiry access token. Sign-in/Change Account lives **inside the Start Stream dialog** (two-state flow — no separate Connect button). A **graceful End Livestream signs out**: `StopStream()` clears the session + token, so the next go-live needs a fresh sign-in; a crash never runs End, so the token survives and the creator stays signed in. `YouTubeAuthService` takes an optional `HttpClient` + `sessionChanged` callback (test seam + save hook; services are still constructed in `MainViewModel`)
- Scene/source/asset layout + the social bar persist (SQLite, schema v8); the OAuth session persists (DPAPI); the paid-unlock state does not (yet — itch.io key verification pending) - Scene/source/asset layout + the social bar persist (SQLite, schema v8); the OAuth session persists (DPAPI); the paid-unlock state does not (yet — itch.io key verification pending)
- `YouTubeStreamService` uses hardcoded `1080p`/`60fps` and per-broadcast streams — must switch to the v3 `variable` reusable stream - `YouTubeStreamService` uses hardcoded `1080p`/`60fps` and per-broadcast streams — must switch to the v3 `variable` reusable stream
- Webcam capture is shipped (milestone 1); the live desktop/game backdrop is shipped (ship task #1); **the output compositor (TASK 4 ship step 1) is SHIPPED**, **the FFmpeg locator (TASK 4 ship step 2) is SHIPPED** — full plan in `TASKS.md`; the encoder subprocess + RTMP push, audio capture, and the frame-pipeline wiring follow (each its own PR) - Webcam capture is shipped (milestone 1); the live desktop/game backdrop is shipped (ship task #1); **the output compositor (TASK 4 ship step 1) is SHIPPED**, **the FFmpeg locator (TASK 4 ship step 2) is SHIPPED**, **the encoder + RTMP push (TASK 4 ship step 3) is SHIPPED**, **WASAPI audio capture (TASK 4 ship step 4) is SHIPPED** — full plan in `TASKS.md`; the frame-pipeline wiring follows (its own PR)
- `StreamConfig` defaults (`TargetBitrate=6000`, `Resolution="1920x1080"`) are stale — the live dropdown drives `StreamHealth.CurrentBitrate`/`FPS` instead - `StreamConfig` defaults (`TargetBitrate=6000`, `Resolution="1920x1080"`) are stale — the live dropdown drives `StreamHealth.CurrentBitrate`/`FPS` instead
### Screen backdrop capture (TASK 3 ship task #1) ### Screen backdrop capture (TASK 3 ship task #1)
@@ -330,6 +330,58 @@ const). Constructor-injected search dirs / tools dir / downloader (`Func<string,
Task<byte[]>>`) keep it hermetic: tests fake the network with a real in-memory zip. Constructed in the Task<byte[]>>`) keep it hermetic: tests fake the network with a real in-memory zip. Constructed in the
encoder step (not yet — this PR ships the seam + impl + tests only). encoder step (not yet — this PR ships the seam + impl + tests only).
### Live encoder + RTMP push (TASK 4 ship step 3 — shipped 2026-08-12, plan in TASKS.md)
The encoder is a **thin orchestrator over `ffmpeg.exe`** — no H.264/AAC code in the app. It spawns the
subprocess (path from `IFfmpegLocator`), feeds raw BGRA master frames into stdin, and parses `-stats`
stderr lines into `StreamHealth` (bitrate/FPS/duration, dropped-from-frame-count). `FfmpegEncoder`
(`IFfmpegEncoder` seam) holds: `StartAsync` (locate → probe `-encoders` → spawn → stderr loop),
`SubmitFrameAsync` (serialized stdin writes under `SemaphoreSlim`), `StopAsync` (stdin EOF → ffmpeg
finalizes + exits by itself; a 10s watchdog kills it), `Dispose` (force-kill + wait), and the
`HealthUpdated`/`ProcessFailed` events. **Pattern:** the encoder never touches `Process` — it drives the
`IEncoderProcess` seam (`FfmpegEncoderProcess` wraps the real `Process`, redirected stdin/stdout/stderr
+ exit control); a `Func<IEncoderProcess>` factory + the locator are constructor-injected, so the
integration test fakes the whole subprocess (probe + encoder) with a Channel-backed `TextReader` whose
`Complete()` is EOF (`null`), never a `ChannelClosedException`.
**Decisions (locked):** args are pure (`FfmpegArgs.Build`, no string building in the encoder):
`-re -f rawvideo -pix_fmt bgra -video_size WxH -framerate FPS -i pipe:0` + a **silent placeholder
`-f lavfi -i anullsrc`** track (WASAPI capture, ship step 4, replaces it) + `-c:v <enc> -b:v K
-maxrate K -bufsize 2K` + **`-g fps×4 -keyint_min fps×4 -sc_threshold 0 -bf 0 -pix_fmt yuv420p`**
(≤4s keyframes, closed GOP, H.264 compliance) + `-c:a aac -ar 48000 -ac 2 -f flv <rtmpUrl>`.
**Encoder choice is probed from the binary's `-encoders` listing** (`FfmpegEncoderPicker`, pure):
NVENC → QSV → AMF → OpenH264 fallback, **never libx264** (GPL; see Licensing). `EncoderOptions.VideoEncoder`
forces one and skips the probe. `EncoderOptions` also carries W×H/FPS/bitrate from the quality tier and
`GopSize` = FPS×4. Not yet constructed by the app — the frame-pipeline wiring (ship step 5) owns it.
### Live audio capture (TASK 4 ship step 4 — shipped 2026-08-12, plan in TASKS.md)
**Capture runs only while live and is KISS by rule**: desktop/game audio is automatic (WASAPI loopback,
zero UI), the mic is the creator's only audio control (meter/mute/volume already shipped). The whole
layer sits behind an **`IAudioSource` seam** (`Services/Audio/`: `Start`/`Stop`/`SampleReady`/`Failed`,
IDisposable) so the app never touches NAudio directly and tests inject hermetic fakes (no devices, no
timers).
- **Sources (NAudio `NAudio.Wasapi` 2.2.1, MIT — item 9 in `THIRD-PARTY-NOTICES.txt`):**
`WasapiLoopbackAudioSource` = `WasapiLoopbackCapture` on the default render device;
`WasapiMicAudioSource` = `WasapiCapture` with the device resolved by `FriendlyName` matching
`MicSourceName` (the app only persists the **DisplayName**), falling back to the default capture
endpoint. Mic device resolution re-reads the name provider `Func<string?>` at each `Start`, so a mic
picked mid-session takes effect next go-live.
- **`AudioMixer`** owns both sources; `Start`/`Stop` follow go-live (`MainViewModel.BeginGoLive` success
`_audioMixer.Start()`, `StopStream``Stop()`). Mic samples feed a pure **`AudioLevelMeter`** (RMS
with 0.2 exponential smoothing) → `MicLevelChanged` → marshalled to the UI thread → `AudioLevel`.
Desktop samples are currently **dropped** — the future encoder's AAC mix (ship step 5) consumes them,
replacing the `-f lavfi -i anullsrc` placeholder. Failures log via `AppLog`; a mic failure zeroes the
meter, a loopback failure never kills the mic.
- **`WaveToFloat`** (pure, shared): WASAPI mix formats → interleaved float — IEEE float 32-bit direct,
PCM 16-bit normalized to -1..1, `WaveFormatExtensible` with the IEEE-float subformat GUID
(`NAudio.Dmo.AudioMediaSubtypes.MEDIASUBTYPE_IEEE_FLOAT`), trailing partial samples ignored.
- `FfmpegEncoder.cs:139` pre-existing CS8602 fixed (`process!`) — build **0 warnings**; **139 passing**.
**Deferred:** loopback→encoder mix wiring and the encoder construction (ship step 5); capture while not
live is deliberately not shipped (privacy indicator otherwise).
### Social bar (TASK 14 — shipped 2026-08-12, plan in TASKS.md) ### Social bar (TASK 14 — shipped 2026-08-12, plan in TASKS.md)
A **global bar layer** (never a Source, no sources-list row) that sits over the bottom or top of the A **global bar layer** (never a Source, no sources-list row) that sits over the bottom or top of the
+280
View File
@@ -0,0 +1,280 @@
using NAudio.Wave;
using Xunit;
using ytLive.Services.Audio;
namespace ytLive.Tests;
/// <summary>
/// TASK 4 ship step 4: WASAPI audio capture behind the <see cref="IAudioSource"/>
/// seam. The units pin down the pure pieces — byte→float conversion, the level
/// meter math, and the mixer lifecycle/forwarding against fakes. No real audio
/// devices (NAudio device resolution is a thin wrapper left to a manual smoke
/// test), no timers.
/// </summary>
public class AudioMixerTests
{
private sealed class FakeSource : IAudioSource
{
public int StartCount { get; private set; }
public int StopCount { get; private set; }
public bool Disposed { get; private set; }
public event Action<AudioSample>? SampleReady;
public event Action<Exception>? Failed;
public void Start() => StartCount++;
public void Stop() => StopCount++;
public void Dispose() => Disposed = true;
public void Emit(AudioSample sample) => SampleReady?.Invoke(sample);
public void Fail(Exception ex) => Failed?.Invoke(ex);
}
[Fact]
public void Start_StartsBothSources()
{
var mic = new FakeSource();
var loopback = new FakeSource();
var mixer = new AudioMixer(mic, loopback);
mixer.Start();
Assert.Equal(1, mic.StartCount);
Assert.Equal(1, loopback.StartCount);
}
[Fact]
public void Start_IsIdempotent()
{
var mic = new FakeSource();
var mixer = new AudioMixer(mic, new FakeSource());
mixer.Start();
mixer.Start();
Assert.Equal(1, mic.StartCount);
}
[Fact]
public void Stop_StopsBothAndResetsLevel()
{
var mic = new FakeSource();
var loopback = new FakeSource();
var mixer = new AudioMixer(mic, loopback);
mixer.Start();
mic.Emit(new AudioSample(new[] { 0.8f }, 48000, 1));
var last = -1f;
mixer.MicLevelChanged += l => last = l;
mixer.Stop();
Assert.Equal(1, mic.StopCount);
Assert.Equal(1, loopback.StopCount);
Assert.Equal(0f, last);
Assert.Equal(0f, mixer.MicLevel);
}
[Fact]
public void Stop_WithNoStart_DoesNothing()
{
var mic = new FakeSource();
var mixer = new AudioMixer(mic, new FakeSource());
mixer.Stop();
Assert.Equal(0, mic.StopCount);
}
[Fact]
public void MicSamples_DriveMicLevelChanged()
{
var mic = new FakeSource();
var mixer = new AudioMixer(mic, new FakeSource());
var levels = new List<float>();
mixer.MicLevelChanged += l => levels.Add(l);
mixer.Start();
mic.Emit(new AudioSample(new[] { 1f, 1f, 1f, 1f }, 48000, 1));
mic.Emit(new AudioSample(new[] { -1f, -1f, -1f, -1f }, 48000, 1));
Assert.NotEmpty(levels);
Assert.All(levels, l => Assert.InRange(l, 0f, 1f));
}
[Fact]
public void LoopbackSamples_DoNotChangeMicLevel()
{
var loopback = new FakeSource();
var mixer = new AudioMixer(new FakeSource(), loopback);
var levels = new List<float>();
mixer.MicLevelChanged += l => levels.Add(l);
mixer.Start();
loopback.Emit(new AudioSample(new[] { 1f, 1f, 1f, 1f }, 48000, 2));
Assert.Empty(levels);
Assert.Equal(0f, mixer.MicLevel);
}
[Fact]
public void MicFailure_LogsAndResetsLevel()
{
var mic = new FakeSource();
var logs = new List<string>();
var mixer = new AudioMixer(mic, new FakeSource(), m => logs.Add(m));
mixer.Start();
mic.Emit(new AudioSample(new[] { 0.5f }, 48000, 1));
var last = -1f;
mixer.MicLevelChanged += l => last = l;
mic.Fail(new InvalidOperationException("boom"));
Assert.Contains(logs, l => l.Contains("boom"));
Assert.Equal(0f, last);
}
[Fact]
public void LoopbackFailure_LogsButKeepsMic()
{
var loopback = new FakeSource();
var logs = new List<string>();
var mixer = new AudioMixer(new FakeSource(), loopback, m => logs.Add(m));
mixer.Start();
var last = -1f;
mixer.MicLevelChanged += l => last = l;
loopback.Fail(new InvalidOperationException("boom"));
Assert.Contains(logs, l => l.Contains("boom"));
Assert.Equal(-1f, last);
}
[Fact]
public void Dispose_StopsAndDisposesSources()
{
var mic = new FakeSource();
var loopback = new FakeSource();
var mixer = new AudioMixer(mic, loopback);
mixer.Start();
mixer.Dispose();
Assert.True(mic.Disposed);
Assert.True(loopback.Disposed);
Assert.Equal(1, mic.StopCount);
}
[Fact]
public void Dispose_UnsubscribesEvents()
{
var mic = new FakeSource();
var loopback = new FakeSource();
var mixer = new AudioMixer(mic, loopback);
mixer.Dispose();
var levels = new List<float>();
mixer.MicLevelChanged += l => levels.Add(l);
mic.Emit(new AudioSample(new[] { 1f }, 48000, 1));
Assert.Empty(levels);
}
}
public class AudioLevelMeterTests
{
[Fact]
public void ConstantSine_ConvergesToRms()
{
var meter = new AudioLevelMeter();
var sample = new AudioSample(new[] { 0.5f, -0.5f, 0.5f, -0.5f }, 48000, 1);
float level = 0;
for (var i = 0; i < 30; i++)
level = meter.Push(sample);
Assert.InRange(level, 0.45f, 0.55f);
}
[Fact]
public void Silence_DrivesTowardZero()
{
var meter = new AudioLevelMeter();
meter.Push(new AudioSample(new[] { 1f, 1f, 1f, 1f }, 48000, 1));
var samples = new float[1024];
var sample = new AudioSample(samples, 48000, 1);
for (var i = 0; i < 50; i++)
meter.Push(sample);
Assert.Equal(0f, meter.Level, 3);
}
[Fact]
public void EmptySample_KeepsLevel()
{
var meter = new AudioLevelMeter();
meter.Push(new AudioSample(new[] { 1f, 1f, 1f, 1f }, 48000, 1));
var before = meter.Level;
var level = meter.Push(new AudioSample(Array.Empty<float>(), 48000, 1));
Assert.Equal(before, level);
}
[Fact]
public void Reset_ZerosLevel()
{
var meter = new AudioLevelMeter();
meter.Push(new AudioSample(new[] { 1f, 1f, 1f, 1f }, 48000, 1));
meter.Reset();
Assert.Equal(0f, meter.Level);
}
}
public class WaveToFloatTests
{
private static byte[] FloatSamples(params float[] values)
{
var bytes = new byte[values.Length * 4];
Buffer.BlockCopy(values, 0, bytes, 0, bytes.Length);
return bytes;
}
[Fact]
public void IeeeFloat32_PreservesValues()
{
var format = WaveFormat.CreateIeeeFloatWaveFormat(48000, 2);
var samples = WaveToFloat.Convert(FloatSamples(0.25f, -0.5f, 1f), 12, format);
Assert.Equal(3, samples.Length);
Assert.Equal(0.25f, samples[0], 4);
Assert.Equal(-0.5f, samples[1], 4);
Assert.Equal(1f, samples[2], 4);
}
[Fact]
public void Pcm16_NormalizesToUnitRange()
{
var format = WaveFormat.CreateCustomFormat(WaveFormatEncoding.Pcm, 48000, 1, 48000 * 2, 2, 16);
var bytes = new byte[] { 0x00, 0x00, 0xFF, 0x7F, 0x00, 0x80 };
var samples = WaveToFloat.Convert(bytes, 6, format);
Assert.Equal(3, samples.Length);
Assert.Equal(0f, samples[0], 4);
Assert.Equal(1f, samples[1], 4);
Assert.Equal(-1f, samples[2], 4);
}
[Fact]
public void TruncatedTrailingBytes_AreIgnored()
{
var format = WaveFormat.CreateIeeeFloatWaveFormat(48000, 2);
var bytes = new byte[] { 0, 0, 0x80, 0x3F, 1, 2, 3 }; // 1 float + 3 stray bytes
var samples = WaveToFloat.Convert(bytes, bytes.Length, format);
Assert.Single(samples);
Assert.Equal(1f, samples[0], 4);
}
}
+290
View File
@@ -0,0 +1,290 @@
using System.Diagnostics;
using System.Threading.Channels;
using Xunit;
using ytLive.Models;
using ytLive.Services;
using ytLive.Services.Encoder;
namespace ytLive.Tests;
/// <summary>
/// The FFmpeg subprocess encoder (TASK 4 ship step 3). The integration test drives
/// the full lifecycle against fakes: encoder probe → subprocess start with the right
/// args → raw frames into stdin → stderr progress parsed into health → graceful stop.
/// The units pin down the pure pieces (args, progress parser, encoder picker) and
/// the failure edges. No real ffmpeg binary, no network.
/// </summary>
public class FfmpegEncoderTests
{
private sealed class QueuedReader : TextReader
{
private readonly Channel<string?> _channel = Channel.CreateUnbounded<string?>();
public void Enqueue(string s) => _channel.Writer.TryWrite(s);
public void Complete() => _channel.Writer.TryComplete();
public override string? ReadLine() => ReadLineAsync().GetAwaiter().GetResult();
public override async Task<string?> ReadLineAsync()
{
try
{
return await _channel.Reader.ReadAsync().AsTask().ConfigureAwait(false);
}
catch (ChannelClosedException)
{
return null; // stream EOF — the process exited and the pipe closed
}
}
}
private sealed class FakeEncoderProcess : IEncoderProcess
{
public ProcessStartInfo? StartInfo { get; private set; }
public MemoryStream Stdin { get; } = new();
public Stream StandardInput => Stdin;
public TextReader StandardOutput { get; }
public bool HasExited { get; private set; }
public int ExitCode { get; private set; }
public bool Killed { get; private set; }
public bool Started { get; private set; }
private readonly TaskCompletionSource _exit =
new(TaskCreationOptions.RunContinuationsAsynchronously);
private readonly QueuedReader _error = new();
public FakeEncoderProcess(string probeOutput = "")
=> StandardOutput = new StringReader(probeOutput);
public TextReader StandardError => _error;
public void EnqueueStderr(string line) => _error.Enqueue(line);
public void Start(ProcessStartInfo startInfo)
{
StartInfo = startInfo;
Started = true;
}
public void SignalExit(int code = 0)
{
ExitCode = code;
HasExited = true;
_error.Complete();
_exit.TrySetResult();
}
public void Kill()
{
Killed = true;
_error.Complete();
_exit.TrySetResult();
}
public Task WaitForExitAsync(CancellationToken cancellationToken = default) => _exit.Task;
public void Dispose()
{
_error.Complete();
_exit.TrySetResult();
}
}
private sealed class FakeEncoderProcessFactory
{
private readonly Queue<FakeEncoderProcess> _processes = new();
public void Return(FakeEncoderProcess p) => _processes.Enqueue(p);
public FakeEncoderProcess Create() => _processes.Dequeue();
}
private sealed class StubLocator : IFfmpegLocator
{
public Task<string> LocateAsync(CancellationToken cancellationToken = default)
=> Task.FromResult(@"C:\tools\ffmpeg.exe");
}
private static byte[] BgraFrame(int w, int h, byte b, byte g, byte r)
{
var bytes = new byte[w * h * 4];
for (var i = 0; i < bytes.Length; i += 4)
{
bytes[i] = b;
bytes[i + 1] = g;
bytes[i + 2] = r;
bytes[i + 3] = 255;
}
return bytes;
}
[Fact]
public async Task Start_ProbesEncoder_FeedsFrames_ParsesHealth_StopsGracefully()
{
const string encoders =
" V..... h264_nvenc NVIDIA NVENC H.264 encoder (codec h264)\n" +
" V..... libopenh264 OpenH264 H.264 (codec h264)\n";
var probe = new FakeEncoderProcess(encoders);
var encoderProc = new FakeEncoderProcess();
var factory = new FakeEncoderProcessFactory();
factory.Return(probe);
factory.Return(encoderProc);
using var encoder = new FfmpegEncoder(new StubLocator(), factory.Create);
var health = new TaskCompletionSource<StreamHealth>(TaskCreationOptions.RunContinuationsAsynchronously);
encoder.HealthUpdated += (_, h) => health.TrySetResult(h);
var options = new EncoderOptions
{
RtmpUrl = "rtmp://a.rtmp.youtube.com/live2/abc-xyz",
Width = 1920,
Height = 1080,
Fps = 60,
BitrateKbps = 8000,
};
await encoder.StartAsync(options);
Assert.True(probe.Started, "the -encoders probe must run");
Assert.True(encoderProc.Started, "the encoder subprocess must start");
Assert.Equal(@"C:\tools\ffmpeg.exe", encoderProc.StartInfo!.FileName);
var args = encoderProc.StartInfo.ArgumentList.ToArray();
var c = Array.IndexOf(args, "-c:v");
Assert.True(
args[c + 1] == "h264_nvenc",
"hardware NVENC must be preferred over the listed openh264");
await encoder.SubmitFrameAsync(new VideoFrame(1920, 1080, BgraFrame(1920, 1080, 255, 0, 0)));
await encoder.SubmitFrameAsync(new VideoFrame(1920, 1080, BgraFrame(1920, 1080, 0, 255, 0)));
Assert.Equal(2 * 1920 * 1080 * 4, encoderProc.Stdin.Length);
encoderProc.EnqueueStderr(
"frame= 120 fps= 59.9 q=28.0 size= 1024KiB time=00:00:02.00 bitrate= 4000.1kbits/s speed=1.00x");
var h = await health.Task.WaitAsync(TimeSpan.FromSeconds(5));
Assert.Equal(4000.1, h.CurrentBitrate, 1);
Assert.Equal(59.9, h.FPS, 1);
Assert.Equal(TimeSpan.FromSeconds(2), h.StreamDuration);
encoderProc.SignalExit();
await encoder.StopAsync();
Assert.False(encoder.IsRunning);
Assert.False(encoderProc.Killed, "graceful stop must not kill the process");
Assert.Equal(StreamStatus.Offline, h.Status);
}
[Fact]
public async Task Start_WithoutRtmpUrl_Throws()
{
using var encoder = new FfmpegEncoder(new StubLocator());
await Assert.ThrowsAsync<ArgumentException>(() => encoder.StartAsync(new EncoderOptions()));
}
[Fact]
public async Task SubmitFrame_WhenNotRunning_Throws()
{
using var encoder = new FfmpegEncoder(new StubLocator());
await Assert.ThrowsAsync<InvalidOperationException>(
() => encoder.SubmitFrameAsync(new VideoFrame(2, 2, new byte[16])));
}
[Fact]
public async Task Stop_WithoutStart_IsNoop()
{
using var encoder = new FfmpegEncoder(new StubLocator());
await encoder.StopAsync();
Assert.False(encoder.IsRunning);
}
[Fact]
public async Task ProcessDeath_RaisesFailed()
{
var encoderProc = new FakeEncoderProcess();
var factory = new FakeEncoderProcessFactory();
factory.Return(new FakeEncoderProcess());
factory.Return(encoderProc);
using var encoder = new FfmpegEncoder(new StubLocator(), factory.Create);
var failed = new TaskCompletionSource<string>(TaskCreationOptions.RunContinuationsAsynchronously);
encoder.ProcessFailed += (_, msg) => failed.TrySetResult(msg);
await encoder.StartAsync(new EncoderOptions { RtmpUrl = "rtmp://x/y" });
encoderProc.SignalExit(1);
var msg = await failed.Task.WaitAsync(TimeSpan.FromSeconds(5));
Assert.Contains("1", msg);
}
[Fact]
public void Args_Gop_IsFpsTimesFour()
{
var options = new EncoderOptions { Fps = 60 };
Assert.Equal(240, options.GopSize);
var args = FfmpegArgs.Build(options, "libopenh264").ToArray();
var g = Array.IndexOf(args, "-g");
Assert.Equal("240", args[g + 1]);
var keyint = Array.IndexOf(args, "-keyint_min");
Assert.Equal("240", args[keyint + 1]);
}
[Fact]
public void Args_IncludeInputOutputAndCompliance()
{
var args = FfmpegArgs.Build(
new EncoderOptions
{
RtmpUrl = "rtmp://a.rtmp.youtube.com/live2/key",
Width = 1920,
Height = 1080,
Fps = 60,
BitrateKbps = 8000,
},
"libopenh264").ToArray();
Assert.Contains("-f", args);
Assert.Contains("rawvideo", args);
Assert.Contains("pipe:0", args);
Assert.Contains("1920x1080", args);
Assert.Contains("anullsrc=channel_layout=stereo:sample_rate=48000", args);
Assert.Contains("-sc_threshold", args);
Assert.Contains("-bf", args);
Assert.Contains("yuv420p", args);
Assert.Contains("aac", args);
Assert.Contains("flv", args);
Assert.Contains("rtmp://a.rtmp.youtube.com/live2/key", args);
}
[Fact]
public void ProgressParser_ParsesRealStatsLine()
{
var p = FfmpegProgressParser.TryParse(
"frame= 123 fps= 59.9 q=28.0 size= 1024KiB time=00:00:02.04 bitrate= 4000.1kbits/s speed=1.00x");
Assert.NotNull(p);
Assert.Equal(123, p.Value.Frame);
Assert.Equal(59.9, p.Value.Fps, 1);
Assert.Equal(4000.1, p.Value.BitrateKbps, 1);
Assert.Equal(2.04, p.Value.Duration.TotalSeconds, 2);
Assert.Equal(1024 * 1024, p.Value.SizeBytes);
}
[Fact]
public void ProgressParser_IgnoresBannerAndErrors()
{
Assert.Null(FfmpegProgressParser.TryParse("ffmpeg version 6.1 Copyright (c) 2000-2026 the FFmpeg developers"));
Assert.Null(FfmpegProgressParser.TryParse("Error while opening encoder for output stream #0:0"));
Assert.Null(FfmpegProgressParser.TryParse(""));
}
[Fact]
public void EncoderPicker_PrefersHardware_NeverLibx264()
{
var listing =
" V..... libx264 libx264 H.264 / AVC (codec h264)\n" +
" V..... libopenh264 OpenH264 H.264 (codec h264)\n" +
" V..... h264_nvenc NVIDIA NVENC H.264 encoder (codec h264)\n";
Assert.Equal("h264_nvenc", FfmpegEncoderPicker.Pick(listing));
var onlyGpl = " V..... libx264 libx264 H.264 / AVC (codec h264)\n";
Assert.Equal("libopenh264", FfmpegEncoderPicker.Pick(onlyGpl));
var software = " V..... libopenh264 OpenH264 H.264 (codec h264)\n";
Assert.Equal("libopenh264", FfmpegEncoderPicker.Pick(software));
Assert.Equal("libopenh264", FfmpegEncoderPicker.Pick("no encoders at all"));
}
}
+1
View File
@@ -36,6 +36,7 @@
<ItemGroup> <ItemGroup>
<PackageReference Include="Microsoft.Data.Sqlite" Version="8.0.10"/> <PackageReference Include="Microsoft.Data.Sqlite" Version="8.0.10"/>
<PackageReference Include="NAudio.Wasapi" Version="2.2.1"/>
<PackageReference Include="SQLitePCLRaw.bundle_e_sqlite3" Version="2.1.12"/> <PackageReference Include="SQLitePCLRaw.bundle_e_sqlite3" Version="2.1.12"/>
<PackageReference Include="System.Security.Cryptography.ProtectedData" Version="8.0.0"/> <PackageReference Include="System.Security.Cryptography.ProtectedData" Version="8.0.0"/>
</ItemGroup> </ItemGroup>