TASK 4 ship step 4: WASAPI audio capture — NAudio loopback + mic behind an IAudioSource seam, AudioMixer driving AudioLevel while live, pure level meter + WaveToFloat, NAudio.Wasapi 2.2.1 (MIT, notices item 9) — 139 tests passing, 0 warnings
This commit is contained in:
+43
-30
@@ -7,37 +7,50 @@
|
||||
|
||||
## Session state (last updated: 2026-08-12)
|
||||
|
||||
- **Branch:** `main`. TASK 4 ship step 3 (encoder + RTMP push) is **built and
|
||||
tested** but **NOT committed** — the working tree is dirty with the encoder
|
||||
(10 new files in `Services/Encoder/` + `ytLive.Tests/FfmpegEncoderTests.cs`) and
|
||||
its memory updates (TASKS.md, ai.md, Services/index.md). Ready to commit when
|
||||
the user says so.
|
||||
- **Finished this session:** TASK 4 ship step 3 — the FFmpeg subprocess encoder:
|
||||
`EncoderOptions` + `IFfmpegEncoder`/`FfmpegEncoder` + `IEncoderProcess`/
|
||||
`FfmpegEncoderProcess` + pure `FfmpegArgs`/`FfmpegProgressParser`/
|
||||
`FfmpegEncoderPicker` in `Services/Encoder/`. StartAsync (locate → probe
|
||||
`-encoders` → spawn → stderr loop), SubmitFrameAsync (serialized BGRA stdin),
|
||||
StopAsync (stdin EOF → ffmpeg finalizes; 10s kill watchdog), ProcessFailed on
|
||||
unexpected non-zero exit. Not yet constructed by the app (ship step 5 wiring).
|
||||
Build: **0 warnings**. Tests: **122 passing** (was 112; +10 new).
|
||||
- **Branch:** `main`. TASK 4 ship step 4 (WASAPI audio capture) is **built and
|
||||
tested** but **NOT committed** — the working tree is dirty with the audio
|
||||
layer (6 new files in `Services/Audio/` + `ytLive.Tests/AudioMixerTests.cs`,
|
||||
the `NAudio.Wasapi` 2.2.1 package reference, the THIRD-PARTY-NOTICES entry,
|
||||
the `MainViewModel` wiring, the `FfmpegEncoder.cs:139` CS8602 fix) and its
|
||||
memory updates (TASKS.md, ai.md, Services/index.md, HANDOFF.md). Ready to
|
||||
commit when the user says so.
|
||||
- **Finished this session:** TASK 4 ship step 4 — the live audio capture layer:
|
||||
`IAudioSource`/`AudioSample` seam, `WasapiLoopbackAudioSource`
|
||||
(`WasapiLoopbackCapture` on the default render device), `WasapiMicAudioSource`
|
||||
(`WasapiCapture`, NAudio device resolved by `FriendlyName` matching
|
||||
`MicSourceName` via a re-read `Func<string?>` provider, default-endpoint
|
||||
fallback), `AudioMixer` (owns both sources, starts/stops with go-live, feeds
|
||||
the pure `AudioLevelMeter` → `MicLevelChanged`), `WaveToFloat` (IEEE-float /
|
||||
PCM16 / extensible). `MainViewModel` constructs the mixer, `BeginGoLive`
|
||||
success → `Start()`, `StopStream` → `Stop()`, `MicLevelChanged` marshalled to
|
||||
the UI thread → `AudioLevel`. Loopback samples are currently dropped (the
|
||||
encoder's AAC mix, ship step 5, consumes them). Build: **0 warnings** (a
|
||||
pre-existing CS8602 in `FfmpegEncoder.cs:139` surfaced during the rebuild and
|
||||
was fixed with `process!`). Tests: **139 passing** (was 122; +17 new audio).
|
||||
- **Landmines:**
|
||||
- `ChannelReader.ReadAsync` on a completed channel **throws**
|
||||
`ChannelClosedException` — it does NOT return `null` like a StreamReader EOF.
|
||||
The test fake (`QueuedReader` in `FfmpegEncoderTests.cs`) catches it and
|
||||
returns `null`, or the encoder's stderr loop treats it as a fault and
|
||||
`ProcessFailed` never fires (that's exactly what happened on the first run —
|
||||
see commit history).
|
||||
- The probe process (`FakeEncoderProcess`) must be a *separate* `IEncoderProcess`
|
||||
instance from the encoder process in `Start_*` tests — `StartAsync` calls the
|
||||
factory twice (probe → encoder), and the fake can't simulate both roles at
|
||||
once.
|
||||
- `StopAsync` waits the full `ExitTimeout` if the fake's process doesn't signal
|
||||
exit — fakes must call `SignalExit()` inside `StopAsync`'s stdin-EOF path.
|
||||
- Windows-only: `FfmpegEncoderProcess` sets `UseShellExecute=false` +
|
||||
`RedirectStandardXxx=true` — never spawn with a shell.
|
||||
- **Next step:** TASK 4 ship step 4 — WASAPI audio capture (loopback + mic)
|
||||
feeding `AudioLevel` (req 7). Nothing else queued — do not expand the task
|
||||
queue on your own.
|
||||
- `WaveFormatExtensible.SubFormat` is a **`Guid`** in NAudio 2.x, not a
|
||||
`WaveFormatEncoding` — compare it to
|
||||
`NAudio.Dmo.AudioMediaSubtypes.MEDIASUBTYPE_IEEE_FLOAT` (the IEEE-float
|
||||
subtype constant; it lives in the `NAudio.Dmo` namespace, not CoreAudioApi).
|
||||
- Use the **`NAudio.Wasapi` 2.2.1 feature package**, not the `NAudio`
|
||||
meta-package — the wasapi types come from that package (`WasapiCapture` in
|
||||
`NAudio.CoreAudioApi`, `WasapiLoopbackCapture` in `NAudio.Wave`); `NAudio.Core`
|
||||
comes in transitively.
|
||||
- The audio meter is an exponential smoother (0.2 factor) — a single pushed
|
||||
sample only moves 20% toward its RMS. Tests must push repeatedly before
|
||||
asserting a converged level.
|
||||
- Tests never instantiate `MainViewModel` directly except the round-clip
|
||||
integration test (a real `MainWindow`), which never goes live — so the mixer
|
||||
is constructed but never started there; NAudio types are only constructed,
|
||||
never touching devices. Keep it that way.
|
||||
- Mic device resolution must re-read `MicSourceName` at each `Start` (the app
|
||||
persists only the DisplayName, not a device ID).
|
||||
- Windows-only: all capture runs only while live (privacy indicator otherwise).
|
||||
|
||||
- **Next step:** TASK 4 ship step 5 — the frame-pipeline wiring
|
||||
(`CameraManager`/`ScreenCaptureManager` → compositor resolver → encoder
|
||||
construction, plus the loopback → encoder AAC mix). Nothing else queued — do
|
||||
not expand the task queue on your own.
|
||||
|
||||
- **Secret/DB/port facts live:** OAuth client id/secret in `Helpers/OAuthCredentials.cs`;
|
||||
OAuth session token in `Helpers/TokenStore.cs` (DPAPI → `%APPDATA%\ytLlive\ytLlive.auth`);
|
||||
|
||||
Reference in New Issue
Block a user