TASK 4 ship step 5: live frame pipeline — FramePump paces the active scene's composite into the encoder, ScreenCaptureManager.GetLatestFrame, MainViewModel resolver/option-builder/pump wiring, FramePumpTests (7) + GetLatestFrame test — 147 tests passing, 0 warnings

This commit is contained in:
2026-08-13 08:57:56 -07:00
parent 58c0f8e8c4
commit e72ba71165
9 changed files with 700 additions and 54 deletions
+28 -43
View File
@@ -5,52 +5,37 @@
> a problem. Conventions: [`schema.md`](schema.md). Rewrite this file at session
> end, compaction, or any interruption.
## Session state (last updated: 2026-08-12)
## Session state (last updated: 2026-08-13)
- **Branch:** `main`. TASK 4 ship step 4 (WASAPI audio capture) is **built and
tested** but **NOT committed** — the working tree is dirty with the audio
layer (6 new files in `Services/Audio/` + `ytLive.Tests/AudioMixerTests.cs`,
the `NAudio.Wasapi` 2.2.1 package reference, the THIRD-PARTY-NOTICES entry,
the `MainViewModel` wiring, the `FfmpegEncoder.cs:139` CS8602 fix) and its
memory updates (TASKS.md, ai.md, Services/index.md, HANDOFF.md). Ready to
commit when the user says so.
- **Finished this session:** TASK 4 ship step 4 — the live audio capture layer:
`IAudioSource`/`AudioSample` seam, `WasapiLoopbackAudioSource`
(`WasapiLoopbackCapture` on the default render device), `WasapiMicAudioSource`
(`WasapiCapture`, NAudio device resolved by `FriendlyName` matching
`MicSourceName` via a re-read `Func<string?>` provider, default-endpoint
fallback), `AudioMixer` (owns both sources, starts/stops with go-live, feeds
the pure `AudioLevelMeter``MicLevelChanged`), `WaveToFloat` (IEEE-float /
PCM16 / extensible). `MainViewModel` constructs the mixer, `BeginGoLive`
success → `Start()`, `StopStream``Stop()`, `MicLevelChanged` marshalled to
the UI thread → `AudioLevel`. Loopback samples are currently dropped (the
encoder's AAC mix, ship step 5, consumes them). Build: **0 warnings** (a
pre-existing CS8602 in `FfmpegEncoder.cs:139` surfaced during the rebuild and
was fixed with `process!`). Tests: **139 passing** (was 122; +17 new audio).
- **Branch:** `main`. TASK 4 ship step 5 (the live frame pipeline) is **shipped
and committed** — the `FramePump` frame producer
(`Services/Encoder/FramePump.cs`), `ScreenCaptureManager.GetLatestFrame(key)`,
the `MainViewModel` resolver/option-builders/pump lifecycle wiring, `FramePumpTests`
(7) + the `GetLatestFrame` test (1), and the memory updates (TASKS.md, ai.md,
Services/index.md — the previous session left `Services/index.md` stale and
HANDOFF unrewritten; both were fixed before the commit). Build **0 warnings**,
**147 tests passing**. A Windows patch reboot killed the prior session right
after the work was verified but before the commit.
- **Finished this session:** ship step 5 (re-verified: 0 warnings, 147/147 pass),
the `Services/index.md` FramePump row + de-stale'd encoder rows, and this
HANDOFF rewrite.
- **Landmines:**
- `WaveFormatExtensible.SubFormat` is a **`Guid`** in NAudio 2.x, not a
`WaveFormatEncoding` — compare it to
`NAudio.Dmo.AudioMediaSubtypes.MEDIASUBTYPE_IEEE_FLOAT` (the IEEE-float
subtype constant; it lives in the `NAudio.Dmo` namespace, not CoreAudioApi).
- Use the **`NAudio.Wasapi` 2.2.1 feature package**, not the `NAudio`
meta-package — the wasapi types come from that package (`WasapiCapture` in
`NAudio.CoreAudioApi`, `WasapiLoopbackCapture` in `NAudio.Wave`); `NAudio.Core`
comes in transitively.
- The audio meter is an exponential smoother (0.2 factor) — a single pushed
sample only moves 20% toward its RMS. Tests must push repeatedly before
asserting a converged level.
- The pump reads the active scene on a background thread while the UI can still
edit it — a concurrent-mutation exception is contained (logged + `Failed` +
the pump stops), not a crash. The background thread + video pipeline is the
new reality since ship step 5.
- `StopAsync` must stop the encoder (closes stdin) **before** awaiting the pump
loop — closing stdin unblocks a write stuck on pipe backpressure; the reverse
order deadlocks.
- `FramePump.IsRunning` must be set true before the loop starts (a completed-task
delay can run the first iteration synchronously on the caller's thread).
- `StartAsync` never throws; the VM fires-and-forgets it. `Failed` while live
flips `StreamStatus.Error` (minimal — real health surfacing is ship step 6).
- Tests never instantiate `MainViewModel` directly except the round-clip
integration test (a real `MainWindow`), which never goes live — so the mixer
is constructed but never started there; NAudio types are only constructed,
never touching devices. Keep it that way.
- Mic device resolution must re-read `MicSourceName` at each `Start` (the app
persists only the DisplayName, not a device ID).
- Windows-only: all capture runs only while live (privacy indicator otherwise).
- **Next step:** TASK 4 ship step 5 — the frame-pipeline wiring
(`CameraManager`/`ScreenCaptureManager` → compositor resolver → encoder
construction, plus the loopback → encoder AAC mix). Nothing else queued — do
not expand the task queue on your own.
integration test (a real `MainWindow`), which never goes live — keep it that way.
- **Next step:** TASK 4 ship step 6 — health stats: bind `FramePump.HealthUpdated`
(bitrate/FPS/duration) into the bottom bar. Nothing else queued — do not expand
the task queue on your own.
- **Secret/DB/port facts live:** OAuth client id/secret in `Helpers/OAuthCredentials.cs`;
OAuth session token in `Helpers/TokenStore.cs` (DPAPI → `%APPDATA%\ytLlive\ytLlive.auth`);