TASK 4 ship step 4: WASAPI audio capture — NAudio loopback + mic behind an IAudioSource seam, AudioMixer driving AudioLevel while live, pure level meter + WaveToFloat, NAudio.Wasapi 2.2.1 (MIT, notices item 9) — 139 tests passing, 0 warnings
This commit is contained in:
@@ -91,7 +91,7 @@ C# / WPF (.NET 8) following MVVM:
|
||||
|------|------|
|
||||
| `Models/` | Plain data types — Scene, Source (incl. `ClipShape`, `IsMirrored`, `VideoImageSource`), QualityOption, StreamConfig, StreamHealth, YouTubeChannel, ChatMessage, **Socials (`SocialService` enum + `SocialEntry`/`SocialsConfig` + `SocialServiceIcons`) — the social bar** |
|
||||
| `ViewModels/` | MainViewModel — exposes collections + commands for the UI; GoLiveViewModel, ReuseImageViewModel, CameraPickerViewModel, **SocialsDialogViewModel** |
|
||||
| `Services/` | YouTube OAuth2, stream/broadcast management, live chat polling, LayoutStore (SQLite), **SocialValidator (`ISocialValidator` seam + `HttpSocialValidator` default)**, **webcam: `VideoFrame` seam + `CameraDeviceInfo`/`ICameraEnumerator`/`ICameraFrameSource` interfaces + `MediaCaptureCameraEnumerator`/`MediaCaptureFrameSource` (WinRT) + `CameraManager`**, **screen capture: `IFullScreenDetector`/`Win32FullScreenDetector` + `IScreenCaptureSource`/`ScreenCaptureFrameSource` (WinRT GraphicsCapture) + `ScreenCaptureManager` + `ScreenCaptureSourceFactory` + `Direct3D11Helper`/`CaptureInterop` (COM bridges)** |
|
||||
| `Services/` | YouTube OAuth2, stream/broadcast management, live chat polling, LayoutStore (SQLite), **SocialValidator (`ISocialValidator` seam + `HttpSocialValidator` default)**, **webcam: `VideoFrame` seam + `CameraDeviceInfo`/`ICameraEnumerator`/`ICameraFrameSource` interfaces + `MediaCaptureCameraEnumerator`/`MediaCaptureFrameSource` (WinRT) + `CameraManager`**, **screen capture: `IFullScreenDetector`/`Win32FullScreenDetector` + `IScreenCaptureSource`/`ScreenCaptureFrameSource` (WinRT GraphicsCapture) + `ScreenCaptureManager` + `ScreenCaptureSourceFactory` + `Direct3D11Helper`/`CaptureInterop` (COM bridges)**, **audio: `IAudioSource` seam + `WasapiLoopbackAudioSource`/`WasapiMicAudioSource` (NAudio WASAPI) + `AudioMixer` + pure `AudioLevelMeter`/`WaveToFloat` (see "Live audio capture")** |
|
||||
| `Helpers/` | ViewModelBase (INotifyPropertyChanged), RelayCommand, ImageCache, AppLog (file logger), FocusPreservingListBox, OAuthCredentials, **TokenStore (DPAPI session persistence)**, visibility converters |
|
||||
| `Themes/` | `Controls.xaml` — the single dark-theme source, merged once in `App.xaml` (see `Themes/index.md`) |
|
||||
| `MainWindow.xaml` | Dark theme; layout: top bar (controls), center (preview), left (scenes/sources), right (chat), bottom (health) |
|
||||
@@ -114,7 +114,7 @@ C# / WPF (.NET 8) following MVVM:
|
||||
- `Helpers/OAuthCredentials.cs` contains the real ClientId/ClientSecret. Auth is complete and the session **persists via Windows DPAPI** (`Helpers/TokenStore.cs` → `%APPDATA%\ytLlive\ytLlive.auth`, CurrentUser scope), reloaded best-effort at startup with a proactive refresh of a near-expiry access token. Sign-in/Change Account lives **inside the Start Stream dialog** (two-state flow — no separate Connect button). A **graceful End Livestream signs out**: `StopStream()` clears the session + token, so the next go-live needs a fresh sign-in; a crash never runs End, so the token survives and the creator stays signed in. `YouTubeAuthService` takes an optional `HttpClient` + `sessionChanged` callback (test seam + save hook; services are still constructed in `MainViewModel`)
|
||||
- Scene/source/asset layout + the social bar persist (SQLite, schema v8); the OAuth session persists (DPAPI); the paid-unlock state does not (yet — itch.io key verification pending)
|
||||
- `YouTubeStreamService` uses hardcoded `1080p`/`60fps` and per-broadcast streams — must switch to the v3 `variable` reusable stream
|
||||
- Webcam capture is shipped (milestone 1); the live desktop/game backdrop is shipped (ship task #1); **the output compositor (TASK 4 ship step 1) is SHIPPED**, **the FFmpeg locator (TASK 4 ship step 2) is SHIPPED**, **the encoder + RTMP push (TASK 4 ship step 3) is SHIPPED** — full plan in `TASKS.md`; WASAPI audio capture and the frame-pipeline wiring follow (each its own PR)
|
||||
- Webcam capture is shipped (milestone 1); the live desktop/game backdrop is shipped (ship task #1); **the output compositor (TASK 4 ship step 1) is SHIPPED**, **the FFmpeg locator (TASK 4 ship step 2) is SHIPPED**, **the encoder + RTMP push (TASK 4 ship step 3) is SHIPPED**, **WASAPI audio capture (TASK 4 ship step 4) is SHIPPED** — full plan in `TASKS.md`; the frame-pipeline wiring follows (its own PR)
|
||||
- `StreamConfig` defaults (`TargetBitrate=6000`, `Resolution="1920x1080"`) are stale — the live dropdown drives `StreamHealth.CurrentBitrate`/`FPS` instead
|
||||
|
||||
### Screen backdrop capture (TASK 3 ship task #1)
|
||||
@@ -354,6 +354,34 @@ NVENC → QSV → AMF → OpenH264 fallback, **never libx264** (GPL; see Licensi
|
||||
forces one and skips the probe. `EncoderOptions` also carries W×H/FPS/bitrate from the quality tier and
|
||||
`GopSize` = FPS×4. Not yet constructed by the app — the frame-pipeline wiring (ship step 5) owns it.
|
||||
|
||||
### Live audio capture (TASK 4 ship step 4 — shipped 2026-08-12, plan in TASKS.md)
|
||||
|
||||
**Capture runs only while live and is KISS by rule**: desktop/game audio is automatic (WASAPI loopback,
|
||||
zero UI), the mic is the creator's only audio control (meter/mute/volume already shipped). The whole
|
||||
layer sits behind an **`IAudioSource` seam** (`Services/Audio/`: `Start`/`Stop`/`SampleReady`/`Failed`,
|
||||
IDisposable) so the app never touches NAudio directly and tests inject hermetic fakes (no devices, no
|
||||
timers).
|
||||
|
||||
- **Sources (NAudio `NAudio.Wasapi` 2.2.1, MIT — item 9 in `THIRD-PARTY-NOTICES.txt`):**
|
||||
`WasapiLoopbackAudioSource` = `WasapiLoopbackCapture` on the default render device;
|
||||
`WasapiMicAudioSource` = `WasapiCapture` with the device resolved by `FriendlyName` matching
|
||||
`MicSourceName` (the app only persists the **DisplayName**), falling back to the default capture
|
||||
endpoint. Mic device resolution re-reads the name provider `Func<string?>` at each `Start`, so a mic
|
||||
picked mid-session takes effect next go-live.
|
||||
- **`AudioMixer`** owns both sources; `Start`/`Stop` follow go-live (`MainViewModel.BeginGoLive` success
|
||||
→ `_audioMixer.Start()`, `StopStream` → `Stop()`). Mic samples feed a pure **`AudioLevelMeter`** (RMS
|
||||
with 0.2 exponential smoothing) → `MicLevelChanged` → marshalled to the UI thread → `AudioLevel`.
|
||||
Desktop samples are currently **dropped** — the future encoder's AAC mix (ship step 5) consumes them,
|
||||
replacing the `-f lavfi -i anullsrc` placeholder. Failures log via `AppLog`; a mic failure zeroes the
|
||||
meter, a loopback failure never kills the mic.
|
||||
- **`WaveToFloat`** (pure, shared): WASAPI mix formats → interleaved float — IEEE float 32-bit direct,
|
||||
PCM 16-bit normalized to -1..1, `WaveFormatExtensible` with the IEEE-float subformat GUID
|
||||
(`NAudio.Dmo.AudioMediaSubtypes.MEDIASUBTYPE_IEEE_FLOAT`), trailing partial samples ignored.
|
||||
- `FfmpegEncoder.cs:139` pre-existing CS8602 fixed (`process!`) — build **0 warnings**; **139 passing**.
|
||||
|
||||
**Deferred:** loopback→encoder mix wiring and the encoder construction (ship step 5); capture while not
|
||||
live is deliberately not shipped (privacy indicator otherwise).
|
||||
|
||||
### Social bar (TASK 14 — shipped 2026-08-12, plan in TASKS.md)
|
||||
|
||||
A **global bar layer** (never a Source, no sources-list row) that sits over the bottom or top of the
|
||||
|
||||
Reference in New Issue
Block a user