TASK 4 audio follow-up: game audio bar + mic status dot + always-on capture — the footer's second audio control (desktop/game, a mirror of the mic bar: meter + mute + volume) appears only while a full-screen game is producing sound (IGameAudioDetector seam + GameAudioHysteresis: show ~500ms of fullscreen+sound, hide ~1s after leaving fullscreen, silence never hides an active bar; VM polls on a 250ms timer); capture now runs for the app's lifetime so both meters preview live (started at startup via StartMicCaptureAsync, disposed in Shutdown — no longer go-live driven); MIC label is a button with a status dot (Models/MicStatus: green via the source Started event, yellow = mic problem, red = no device); PickMicrophone swaps the live device immediately via AudioMixer.RestartMic; fixed a latent ?.Invoke(meter.Push(...)) short-circuit that skipped the meter update when nothing subscribed — 167 tests passing, 0 warnings

This commit is contained in:
2026-08-13 11:29:44 -07:00
parent ea250c02a6
commit 9f9ed34627
20 changed files with 897 additions and 82 deletions
+30 -17
View File
@@ -91,7 +91,7 @@ C# / WPF (.NET 8) following MVVM:
|------|------|
| `Models/` | Plain data types — Scene, Source (incl. `ClipShape`, `IsMirrored`, `VideoImageSource`), QualityOption, StreamConfig, StreamHealth, YouTubeChannel, ChatMessage, **Socials (`SocialService` enum + `SocialEntry`/`SocialsConfig` + `SocialServiceIcons`) — the social bar** |
| `ViewModels/` | MainViewModel — exposes collections + commands for the UI; GoLiveViewModel, ReuseImageViewModel, CameraPickerViewModel, **SocialsDialogViewModel** |
| `Services/` | YouTube OAuth2, stream/broadcast management, live chat polling, LayoutStore (SQLite), **SocialValidator (`ISocialValidator` seam + `HttpSocialValidator` default)**, **webcam: `VideoFrame` seam + `CameraDeviceInfo`/`ICameraEnumerator`/`ICameraFrameSource` interfaces + `MediaCaptureCameraEnumerator`/`MediaCaptureFrameSource` (WinRT) + `CameraManager`**, **screen capture: `IFullScreenDetector`/`Win32FullScreenDetector` + `IScreenCaptureSource`/`ScreenCaptureFrameSource` (WinRT GraphicsCapture) + `ScreenCaptureManager` + `ScreenCaptureSourceFactory` + `Direct3D11Helper`/`CaptureInterop` (COM bridges)**, **compositor: `SceneCompositor` + `CompositorOptions` + pure `StretchMath` + `StaticPixelCache` (see "Scene compositor")**, **audio: `IAudioSource` seam + `WasapiLoopbackAudioSource`/`WasapiMicAudioSource` (NAudio WASAPI) + `AudioMixer` + pure `AudioLevelMeter`/`WaveToFloat` (see "Live audio capture")**, **encoder: `IFfmpegEncoder`/`FfmpegEncoder` + `IEncoderProcess`/`FfmpegEncoderProcess` + `IFfmpegLocator`/`FfmpegLocator` + pure `FfmpegArgs`/`FfmpegProgressParser`/`FfmpegEncoderPicker` + the `FramePump` frame producer (see "Live encoder" + "Live frame pipeline")** |
| `Services/` | YouTube OAuth2, stream/broadcast management, live chat polling, LayoutStore (SQLite), **SocialValidator (`ISocialValidator` seam + `HttpSocialValidator` default)**, **webcam: `VideoFrame` seam + `CameraDeviceInfo`/`ICameraEnumerator`/`ICameraFrameSource` interfaces + `MediaCaptureCameraEnumerator`/`MediaCaptureFrameSource` (WinRT) + `CameraManager`**, **screen capture: `IFullScreenDetector`/`Win32FullScreenDetector` + `IScreenCaptureSource`/`ScreenCaptureFrameSource` (WinRT GraphicsCapture) + `ScreenCaptureManager` + `ScreenCaptureSourceFactory` + `Direct3D11Helper`/`CaptureInterop` (COM bridges)**, **compositor: `SceneCompositor` + `CompositorOptions` + pure `StretchMath` + `StaticPixelCache` (see "Scene compositor")**, **audio: `IAudioSource` seam + `WasapiLoopbackAudioSource`/`WasapiMicAudioSource` (NAudio WASAPI) + `AudioMixer` + pure `AudioLevelMeter`/`WaveToFloat` + game bar: `IGameAudioDetector`/`GameAudioHysteresis`/`GameAudioDetector` (see "Live audio capture")**, **encoder: `IFfmpegEncoder`/`FfmpegEncoder` + `IEncoderProcess`/`FfmpegEncoderProcess` + `IFfmpegLocator`/`FfmpegLocator` + pure `FfmpegArgs`/`FfmpegProgressParser`/`FfmpegEncoderPicker` + the `FramePump` frame producer (see "Live encoder" + "Live frame pipeline")** |
| `Helpers/` | ViewModelBase (INotifyPropertyChanged), RelayCommand, ImageCache, AppLog (file logger), FocusPreservingListBox, OAuthCredentials, **TokenStore (DPAPI session persistence)**, visibility converters |
| `Themes/` | `Controls.xaml` — the single dark-theme source, merged once in `App.xaml` (see `Themes/index.md`) |
| `MainWindow.xaml` | Dark theme; layout: top bar (controls), center (preview), left (scenes/sources), right (chat), bottom (health) |
@@ -355,34 +355,47 @@ forces one and skips the probe. `EncoderOptions` also carries W×H/FPS/bitrate f
`GopSize` = FPS×4. Constructed by `MainViewModel` since ship step 5 (`new FfmpegEncoder(new FfmpegLocator())`),
driven by the `FramePump` below.
### Live audio capture (TASK 4 ship step 4 — shipped 2026-08-12, plan in TASKS.md)
### Live audio capture (TASK 4 ship step 4 — shipped 2026-08-12, game audio bar follow-up 2026-08-13, plans in TASKS.md)
**Capture runs only while live and is KISS by rule**: desktop/game audio is automatic (WASAPI loopback,
zero UI), the mic is the creator's only audio control (meter/mute/volume already shipped). The whole
layer sits behind an **`IAudioSource` seam** (`Services/Audio/`: `Start`/`Stop`/`SampleReady`/`Failed`,
IDisposable) so the app never touches NAudio directly and tests inject hermetic fakes (no devices, no
timers).
**Capture runs for the app's lifetime and is KISS by rule**: desktop/game audio is automatic (WASAPI
loopback), the mic is the creator's only audio control (meter/mute/volume already shipped). The whole
layer sits behind an **`IAudioSource` seam** (`Services/Audio/`: `Start`/`Stop`/`Started`/`SampleReady`/
`Failed`, IDisposable) so the app never touches NAudio directly and tests inject hermetic fakes (no
devices, no timers).
- **Sources (NAudio `NAudio.Wasapi` 2.2.1, MIT — item 9 in `THIRD-PARTY-NOTICES.txt`):**
`WasapiLoopbackAudioSource` = `WasapiLoopbackCapture` on the default render device;
`WasapiMicAudioSource` = `WasapiCapture` with the device resolved by `FriendlyName` matching
`MicSourceName` (the app only persists the **DisplayName**), falling back to the default capture
endpoint. Mic device resolution re-reads the name provider `Func<string?>` at each `Start`, so a mic
picked mid-session takes effect next go-live.
- **`AudioMixer`** owns both sources; `Start`/`Stop` follow go-live (`MainViewModel.BeginGoLive` success
→ `_audioMixer.Start()`, `StopStream` → `Stop()`). Mic samples feed a pure **`AudioLevelMeter`** (RMS
with 0.2 exponential smoothing) → `MicLevelChanged` → marshalled to the UI thread → `AudioLevel`.
Desktop samples are currently **dropped** — a later step's AAC mix consumes them, replacing the
`-f lavfi -i anullsrc` placeholder (the encoder construction itself shipped in ship step 5). Failures log via `AppLog`; a mic failure zeroes the
meter, a loopback failure never kills the mic.
picked mid-session takes effect **immediately** (the mixer restarts the mic on pick). Both sources
raise `Started` once their capture loop actually begins — the mixer turns that into `MicConnected`.
- **`AudioMixer`** owns both sources; **`StartMicCaptureAsync` starts the mixer once at startup** and
`Shutdown` disposes it — NOT go-live — so both footer meters preview live (`BeginGoLive`/`StopStream`
no longer touch the mixer). Mic samples feed a pure **`AudioLevelMeter`** (RMS with 0.2 exponential
smoothing) → `MicLevelChanged` → marshalled to the UI thread → `AudioLevel`; loopback samples feed a
second meter → `LoopbackLevelChanged` → the game bar's `GameAudioLevel`. **Mic connection state
surfaces as events:** `MicConnected` (source `Started`), `MicFailed` (source `Failed`), and
`RestartMic()` re-resolves + restarts just the mic (loopback keeps running). Failures log via
`AppLog`; a mic failure zeroes the meter, a loopback failure never kills the mic. Meter `Push` is
unconditional (a `?.` on the event would skip the argument — and the meter update — when nothing is
subscribed yet).
- **Mic status dot (`Models/MicStatus.cs`)** on the footer's MIC button: green = `MicConnected`,
yellow = `MicFailed` (in use/unplugged), red = no mic device at startup (the mixer is never started,
so loopback and the game bar can't run either — no capture devices at all).
- **Game audio bar** (desktop/game, only while a full-screen game is up in the preview):
`IGameAudioDetector` seam (`Services/IGameAudioDetector.cs`), pure `GameAudioHysteresis` (SHOW after
~500ms of fullscreen + sound, HIDE after ~1s away from fullscreen, **silence never hides an active
bar**), and the default `GameAudioDetector` composing `IFullScreenDetector` + the live loopback level
(floor 0.5%). WPF-free — the VM owns a 250ms `DispatcherTimer` that polls it and flips
`IsGameAudioBarVisible`.
- **`WaveToFloat`** (pure, shared): WASAPI mix formats → interleaved float — IEEE float 32-bit direct,
PCM 16-bit normalized to -1..1, `WaveFormatExtensible` with the IEEE-float subformat GUID
(`NAudio.Dmo.AudioMediaSubtypes.MEDIASUBTYPE_IEEE_FLOAT`), trailing partial samples ignored.
- `FfmpegEncoder.cs:139` pre-existing CS8602 fixed (`process!`) — build **0 warnings**; **139 passing**.
- Build **0 warnings**; **167 passing** (mixer/hysteresis/game-detector unit tests).
**Deferred:** the loopback→encoder AAC mix wiring (a later step — the encoder construction + frame
pipeline shipped in ship step 5); capture while not live is deliberately not shipped (privacy indicator
otherwise).
pipeline shipped in ship step 5).
### Live frame pipeline (TASK 4 ship step 5 — shipped 2026-08-12, plan in TASKS.md)