TASK 4 audio follow-up: game audio bar + mic status dot + always-on capture — the footer's second audio control (desktop/game, a mirror of the mic bar: meter + mute + volume) appears only while a full-screen game is producing sound (IGameAudioDetector seam + GameAudioHysteresis: show ~500ms of fullscreen+sound, hide ~1s after leaving fullscreen, silence never hides an active bar; VM polls on a 250ms timer); capture now runs for the app's lifetime so both meters preview live (started at startup via StartMicCaptureAsync, disposed in Shutdown — no longer go-live driven); MIC label is a button with a status dot (Models/MicStatus: green via the source Started event, yellow = mic problem, red = no device); PickMicrophone swaps the live device immediately via AudioMixer.RestartMic; fixed a latent ?.Invoke(meter.Push(...)) short-circuit that skipped the meter update when nothing subscribed — 167 tests passing, 0 warnings

This commit is contained in:
2026-08-13 11:29:44 -07:00
parent ea250c02a6
commit 9f9ed34627
20 changed files with 897 additions and 82 deletions
+46 -5
View File
@@ -4,15 +4,18 @@ namespace ytLive.Services.Audio;
/// <summary>
/// Owns the mic + desktop/game capture sources (TASK 4 ship step 4) and drives
/// the live mic meter. Runs only while live: started when go-live succeeds,
/// stopped on end-stream. Mic samples are level-metered and forwarded; loopback
/// samples are currently dropped (consumed by the encoder mix in a later step).
/// the footer meters. Capture runs for the app's lifetime (started once at
/// startup, stopped on shutdown) so both bars stay live in preview: mic samples
/// are level-metered and forwarded, loopback samples feed the game bar's meter.
/// The mixer surfaces mic connection state (Connected/Failed) for the status
/// dot and can restart the mic source mid-session when a device is re-picked.
/// </summary>
public sealed class AudioMixer : IDisposable
{
private readonly IAudioSource _mic;
private readonly IAudioSource _loopback;
private readonly AudioLevelMeter _meter;
private readonly AudioLevelMeter _loopbackMeter;
private readonly Action<string>? _log;
private bool _started;
@@ -21,8 +24,10 @@ public sealed class AudioMixer : IDisposable
_mic = mic;
_loopback = loopback;
_meter = new AudioLevelMeter();
_loopbackMeter = new AudioLevelMeter();
_log = log;
_mic.Started += OnMicStarted;
_mic.SampleReady += OnMicSample;
_loopback.SampleReady += OnLoopbackSample;
_mic.Failed += OnMicFailed;
@@ -32,9 +37,21 @@ public sealed class AudioMixer : IDisposable
/// <summary>Current smoothed mic level (0..1).</summary>
public float MicLevel => _meter.Level;
/// <summary>Current smoothed desktop/game level (0..1).</summary>
public float LoopbackLevel => _loopbackMeter.Level;
/// <summary>Raised whenever the smoothed mic level changes.</summary>
public event Action<float>? MicLevelChanged;
/// <summary>Raised whenever the smoothed desktop/game level changes.</summary>
public event Action<float>? LoopbackLevelChanged;
/// <summary>Raised when the mic capture comes up (the status dot goes green).</summary>
public event Action? MicConnected;
/// <summary>Raised when the mic capture fails or dies (the status dot goes yellow).</summary>
public event Action<Exception>? MicFailed;
public void Start()
{
if (_started)
@@ -46,6 +63,17 @@ public sealed class AudioMixer : IDisposable
_mic.Start();
}
/// <summary>Swaps the mic source without touching loopback — used when the
/// creator picks a different device mid-session. The level resets and the
/// new source raises <see cref="MicConnected"/> or <see cref="MicFailed"/>.</summary>
public void RestartMic()
{
_mic.Stop();
_meter.Reset();
MicLevelChanged?.Invoke(0);
_mic.Start();
}
public void Stop()
{
if (!_started)
@@ -55,12 +83,15 @@ public sealed class AudioMixer : IDisposable
_mic.Stop();
_loopback.Stop();
_meter.Reset();
_loopbackMeter.Reset();
MicLevelChanged?.Invoke(0);
LoopbackLevelChanged?.Invoke(0);
}
public void Dispose()
{
Stop();
_mic.Started -= OnMicStarted;
_mic.SampleReady -= OnMicSample;
_loopback.SampleReady -= OnLoopbackSample;
_mic.Failed -= OnMicFailed;
@@ -69,20 +100,30 @@ public sealed class AudioMixer : IDisposable
_loopback.Dispose();
}
private void OnMicStarted()
{
MicConnected?.Invoke();
}
private void OnMicSample(AudioSample sample)
{
MicLevelChanged?.Invoke(_meter.Push(sample));
// Push unconditionally: the ?. on the event would otherwise skip the
// argument (and the meter update) when nothing is subscribed yet.
var level = _meter.Push(sample);
MicLevelChanged?.Invoke(level);
}
private void OnLoopbackSample(AudioSample sample)
{
// Desktop/game audio: captured for the future encoder mix; no UI yet.
var level = _loopbackMeter.Push(sample);
LoopbackLevelChanged?.Invoke(level);
}
private void OnMicFailed(Exception ex)
{
_log?.Invoke($"Mic capture failed: {ex.Message}");
MicLevelChanged?.Invoke(0);
MicFailed?.Invoke(ex);
}
private void OnLoopbackFailed(Exception ex)
+8 -3
View File
@@ -2,9 +2,10 @@ namespace ytLive.Services.Audio;
/// <summary>
/// A live audio capture source (TASK 4 ship step 4 seam): produces PCM float
/// chunks, runs only while live. The default implementations wrap NAudio's
/// WASAPI capture (mic) / loopback (desktop/game); the mixer and the tests
/// consume this interface, never NAudio directly.
/// chunks. The default implementations wrap NAudio's WASAPI capture (mic) /
/// loopback (desktop/game); the mixer and the tests consume this interface,
/// never NAudio directly. Capture now runs for the app's lifetime so the
/// footer meters stay live in preview — the mixer owns start/stop.
/// </summary>
public interface IAudioSource : IDisposable
{
@@ -14,6 +15,10 @@ public interface IAudioSource : IDisposable
/// <summary>Stops capturing; a later Start begins a fresh session.</summary>
void Stop();
/// <summary>Raises once capture is live (right after recording starts).
/// Never raised when Start fails — <see cref="Failed"/> fires instead.</summary>
event Action? Started;
/// <summary>Raises each captured chunk (interleaved PCM float, -1..1).</summary>
event Action<AudioSample>? SampleReady;
+4 -1
View File
@@ -4,12 +4,14 @@ namespace ytLive.Services.Audio;
/// <summary>
/// Captures the user's desktop/game audio (TASK 4 ship step 4) via WASAPI
/// loopback on the default render device. Starts/stops with go-live only.
/// loopback on the default render device. Runs for the app's lifetime so the
/// game audio bar stays live in preview.
/// </summary>
public sealed class WasapiLoopbackAudioSource : IAudioSource
{
private WasapiLoopbackCapture? _capture;
public event Action? Started;
public event Action<AudioSample>? SampleReady;
public event Action<Exception>? Failed;
@@ -24,6 +26,7 @@ public sealed class WasapiLoopbackAudioSource : IAudioSource
_capture.DataAvailable += OnDataAvailable;
_capture.RecordingStopped += OnRecordingStopped;
_capture.StartRecording();
Started?.Invoke();
}
catch (Exception ex)
{
+4 -1
View File
@@ -14,12 +14,14 @@ public sealed class WasapiMicAudioSource : IAudioSource
private WasapiCapture? _capture;
/// <param name="micNameProvider">Returns the current mic DisplayName; read
/// at each Start so a device picked mid-session takes effect next go-live.</param>
/// at each Start so a device picked mid-session takes effect immediately
/// (the mixer restarts the mic on pick).</param>
public WasapiMicAudioSource(Func<string?> micNameProvider)
{
_micNameProvider = micNameProvider;
}
public event Action? Started;
public event Action<AudioSample>? SampleReady;
public event Action<Exception>? Failed;
@@ -35,6 +37,7 @@ public sealed class WasapiMicAudioSource : IAudioSource
_capture.DataAvailable += OnDataAvailable;
_capture.RecordingStopped += OnRecordingStopped;
_capture.StartRecording();
Started?.Invoke();
}
catch (Exception ex)
{
+46
View File
@@ -0,0 +1,46 @@
namespace ytLive.Services;
/// <summary>
/// Default <see cref="IGameAudioDetector"/>: samples a full-screen monitor
/// provider (the existing <c>IFullScreenDetector</c>) and the live loopback
/// level and advances a <see cref="GameAudioHysteresis"/>. The pure transitions
/// live in GameAudioHysteresis (unit-tested); this wrapper owns the providers
/// and the raised-changed event. WPF-free — the VM owns the poll timer.
/// </summary>
public sealed class GameAudioDetector : IGameAudioDetector
{
private const float SoundFloor = 0.005f;
private readonly Func<int?> _foregroundFullScreenMonitorProvider;
private readonly Func<float> _loopbackLevelProvider;
private readonly Func<DateTime> _now;
private readonly GameAudioHysteresis _hysteresis = new();
/// <param name="foregroundFullScreenMonitorProvider">Returns the monitor a
/// full-screen foreground window covers, or null when windowed/none.</param>
/// <param name="loopbackLevelProvider">Current smoothed desktop/game level (0..1).</param>
/// <param name="now">Clock for the hysteresis windows; injectable for tests.</param>
public GameAudioDetector(
Func<int?> foregroundFullScreenMonitorProvider,
Func<float> loopbackLevelProvider,
Func<DateTime>? now = null)
{
_foregroundFullScreenMonitorProvider = foregroundFullScreenMonitorProvider;
_loopbackLevelProvider = loopbackLevelProvider;
_now = now ?? (() => DateTime.Now);
}
public bool IsGameAudioActive => _hysteresis.IsActive;
public event Action<bool>? IsGameAudioActiveChanged;
public void Poll()
{
var before = _hysteresis.IsActive;
var isFullScreen = _foregroundFullScreenMonitorProvider() != null;
var hasSound = _loopbackLevelProvider() > SoundFloor;
_hysteresis.Update(isFullScreen, hasSound, _now());
if (before != _hysteresis.IsActive)
IsGameAudioActiveChanged?.Invoke(_hysteresis.IsActive);
}
}
+57
View File
@@ -0,0 +1,57 @@
namespace ytLive.Services;
/// <summary>
/// Pure show/hide state machine for the game audio bar (TASK 4 game audio).
/// SHOW: a full-screen foreground app has been producing desktop audio for at
/// least <see cref="ShowAfterSound"/> — "a working game with sound". HIDE: the
/// app leaves fullscreen/foreground for <see cref="HideAfterWindowed"/> — the
/// game is no longer up in the preview. Silence NEVER hides an active bar; it
/// only matters for the initial show. No timers; <see cref="Update"/> is fed by
/// the caller with a clock.
/// </summary>
public sealed class GameAudioHysteresis
{
private readonly TimeSpan _showAfterSound = TimeSpan.FromMilliseconds(500);
private readonly TimeSpan _hideAfterWindowed = TimeSpan.FromSeconds(1);
private DateTime? _soundSince;
private DateTime? _windowedSince;
public bool IsActive { get; private set; }
public void Update(bool isFullScreen, bool hasSound, DateTime now)
{
if (isFullScreen)
{
_windowedSince = null;
if (IsActive)
return;
if (!hasSound)
{
_soundSince = null;
return;
}
_soundSince ??= now;
if (now - _soundSince >= _showAfterSound)
IsActive = true;
}
else
{
_soundSince = null;
if (!IsActive)
{
_windowedSince = null;
return;
}
_windowedSince ??= now;
if (now - _windowedSince >= _hideAfterWindowed)
{
IsActive = false;
_windowedSince = null;
}
}
}
}
+20
View File
@@ -0,0 +1,20 @@
namespace ytLive.Services;
/// <summary>
/// Detects "a working game with sound is up in the preview" (TASK 4 game audio
/// bar): becomes active once a full-screen foreground app is producing desktop
/// audio, and stays active as long as that full-screen app remains — silence
/// never hides the bar, only the game leaving fullscreen/foreground does. Seam
/// so the VM and tests never touch Win32 or audio interop directly.
/// </summary>
public interface IGameAudioDetector
{
/// <summary>True while the game audio bar should be visible.</summary>
bool IsGameAudioActive { get; }
/// <summary>Raised when the bar should appear or disappear.</summary>
event Action<bool>? IsGameAudioActiveChanged;
/// <summary>Samples the injected providers and advances the state machine.</summary>
void Poll();
}
+8 -5
View File
@@ -44,13 +44,16 @@ External-facing logic: YouTube API, persistence. See
| `Encoder/FfmpegEncoderPicker.cs` | Pure H.264 encoder picker from `-encoders` output: NVENC → QSV → AMF → OpenH264; **never returns libx264** (GPL) |
| `Encoder/FramePump.cs` | **TASK 4 ship step 5**: the live frame producer — while live it snapshots the active scene each tick, resolves every element to its latest frame (`Func<SceneElement, VideoFrame?>` resolver), composites it into the tier's output frame, and paces frames into the encoder at the tier's FPS. All collaborators constructor-injected seams; free of WPF and the capture managers. `StartAsync` never throws (failures log + `Failed`); no RTMP URL = encoder skipped; `StopAsync` stops the encoder before awaiting the loop (backpressure deadlock); `ProcessFailed` self-stops. Since the bar bug-fix branch it takes an optional `socialBar: Func<(VideoFrame? Frame, SocialBarPosition Position)>?` seam — a pre-rendered bar strip composited last (above the flash) at the top edge or `SourceRectHeight bar height`. See `ai.md` "Live frame pipeline" |
| `Encoder/FfmpegLocator.cs` | The default locator: PATH probe wins, then the cache, then pull+extract (`ffmpeg.exe` **+ the `libav*.dll` family** via a staging dir, failures via `AppLog`). Pinned to BtbN `autobuild-2026-08-09-13-03` **lgpl-shared** (dynamic linking = LGPL compliance without static-relink §6 material; no GPL libx264; NVENC/QSV/AMF + libopenh264 + native AAC — see `ai.md` → Licensing). Search dirs / tools dir / downloader constructor-injected for hermetic tests. Wired into the encoder since ship step 5 |
| `Audio/IAudioSource.cs` | **TASK 4 ship step 4 seam**: live capture source — `Start`/`Stop`/`SampleReady(AudioSample)`/`Failed(Exception)`, `IDisposable`; runs only while live. The app consumes this seam; tests inject hermetic fakes |
| `Audio/IAudioSource.cs` | **TASK 4 ship step 4 seam**: live capture source — `Start`/`Stop`/`Started`/`SampleReady(AudioSample)`/`Failed(Exception)`, `IDisposable`. Capture now runs for the app's lifetime (started at startup) so the footer meters preview live. The app consumes this seam; tests inject hermetic fakes |
| `Audio/AudioSample.cs` | One captured chunk: interleaved PCM float (-1..1) + sample rate + channels |
| `Audio/WasapiLoopbackAudioSource.cs` | Desktop/game capture: NAudio `WasapiLoopbackCapture` on the default render device — automatic at unity, zero UI |
| `Audio/WasapiMicAudioSource.cs` | Mic capture: NAudio `WasapiCapture` with the device resolved by `FriendlyName` matching `MicSourceName` (DisplayName only), default capture endpoint fallback; name provider `Func<string?>` re-read at each `Start` so a mic picked mid-session takes effect next go-live |
| `Audio/AudioMixer.cs` | Owns both sources; `Start`/`Stop` follow go-live (`MainViewModel.BeginGoLive`/`StopStream`). Mic samples → `AudioLevelMeter``MicLevelChanged`; loopback samples currently dropped (the future encoder AAC mix consumes them). Failures log via `AppLog`; mic failure zeroes the meter, loopback failure doesn't kill the mic |
| `Audio/AudioLevelMeter.cs` | Pure smoothed RMS level (0..1): `Push(AudioSample)` + `Reset` — the unit-tested math behind `AudioLevel` |
| `Audio/WasapiLoopbackAudioSource.cs` | Desktop/game capture: NAudio `WasapiLoopbackCapture` on the default render device — automatic at unity; the game bar's meter consumes it |
| `Audio/WasapiMicAudioSource.cs` | Mic capture: NAudio `WasapiCapture` with the device resolved by `FriendlyName` matching `MicSourceName` (DisplayName only), default capture endpoint fallback; name provider `Func<string?>` re-read at each `Start` so a device picked mid-session takes effect immediately (the mixer restarts the mic on pick) |
| `Audio/AudioMixer.cs` | Owns both sources; capture starts once at startup (`MainViewModel.StartMicCaptureAsync`) and stops on `Shutdown` — NOT go-live (preview monitoring). Mic samples → `AudioLevelMeter``MicLevelChanged`; loopback samples → the game bar's meter via `LoopbackLevelChanged`. Surfaces mic connection state: `MicConnected`/`MicFailed` (drives the status dot) + `RestartMic()` (device swap mid-session, keeps loopback). Failures log via `AppLog`; mic failure zeroes the meter, loopback failure doesn't kill the mic. Note: the meter `Push` is unconditional (the `?.` on the event would otherwise skip the argument when nothing is subscribed) |
| `Audio/AudioLevelMeter.cs` | Pure smoothed RMS level (0..1): `Push(AudioSample)` + `Reset` — the unit-tested math behind `AudioLevel` and `GameAudioLevel` |
| `Audio/WaveToFloat.cs` | Pure WASAPI buffer → float conversion: IEEE float 32-bit direct, PCM 16-bit normalized, `WaveFormatExtensible` IEEE-float subformat GUID, trailing partial samples ignored |
| `IGameAudioDetector.cs` | **TASK 4 game audio bar seam**: `IsGameAudioActive` + `IsGameAudioActiveChanged` + `Poll()` — detects "a working game with sound is up in the preview" |
| `GameAudioHysteresis.cs` | Pure show/hide state machine for the game bar: SHOW = full-screen app holds sound ~500ms; HIDE = the app leaves fullscreen ~1s. **Silence never hides an active bar** — only the game leaving the preview does (creator's rule). No timers; `Update(isFullScreen, hasSound, now)` |
| `GameAudioDetector.cs` | Default `IGameAudioDetector`: composes a full-screen monitor provider (`IFullScreenDetector.GetForegroundFullScreenMonitorIndex`) + the live loopback level (floor 0.5%) into a `GameAudioHysteresis`. WPF-free; the VM owns the 250ms poll timer |
Related: constructed in [`ViewModels/MainViewModel.cs`](../ViewModels/MainViewModel.cs)
(no DI container yet). Models in [`Models/index.md`](../Models/index.md).