TASK 4 ship step 5: live frame pipeline — FramePump paces the active scene's composite into the encoder, ScreenCaptureManager.GetLatestFrame, MainViewModel resolver/option-builder/pump wiring, FramePumpTests (7) + GetLatestFrame test — 147 tests passing, 0 warnings
This commit is contained in:
+28
-43
@@ -5,52 +5,37 @@
|
||||
> a problem. Conventions: [`schema.md`](schema.md). Rewrite this file at session
|
||||
> end, compaction, or any interruption.
|
||||
|
||||
## Session state (last updated: 2026-08-12)
|
||||
## Session state (last updated: 2026-08-13)
|
||||
|
||||
- **Branch:** `main`. TASK 4 ship step 4 (WASAPI audio capture) is **built and
|
||||
tested** but **NOT committed** — the working tree is dirty with the audio
|
||||
layer (6 new files in `Services/Audio/` + `ytLive.Tests/AudioMixerTests.cs`,
|
||||
the `NAudio.Wasapi` 2.2.1 package reference, the THIRD-PARTY-NOTICES entry,
|
||||
the `MainViewModel` wiring, the `FfmpegEncoder.cs:139` CS8602 fix) and its
|
||||
memory updates (TASKS.md, ai.md, Services/index.md, HANDOFF.md). Ready to
|
||||
commit when the user says so.
|
||||
- **Finished this session:** TASK 4 ship step 4 — the live audio capture layer:
|
||||
`IAudioSource`/`AudioSample` seam, `WasapiLoopbackAudioSource`
|
||||
(`WasapiLoopbackCapture` on the default render device), `WasapiMicAudioSource`
|
||||
(`WasapiCapture`, NAudio device resolved by `FriendlyName` matching
|
||||
`MicSourceName` via a re-read `Func<string?>` provider, default-endpoint
|
||||
fallback), `AudioMixer` (owns both sources, starts/stops with go-live, feeds
|
||||
the pure `AudioLevelMeter` → `MicLevelChanged`), `WaveToFloat` (IEEE-float /
|
||||
PCM16 / extensible). `MainViewModel` constructs the mixer, `BeginGoLive`
|
||||
success → `Start()`, `StopStream` → `Stop()`, `MicLevelChanged` marshalled to
|
||||
the UI thread → `AudioLevel`. Loopback samples are currently dropped (the
|
||||
encoder's AAC mix, ship step 5, consumes them). Build: **0 warnings** (a
|
||||
pre-existing CS8602 in `FfmpegEncoder.cs:139` surfaced during the rebuild and
|
||||
was fixed with `process!`). Tests: **139 passing** (was 122; +17 new audio).
|
||||
- **Branch:** `main`. TASK 4 ship step 5 (the live frame pipeline) is **shipped
|
||||
and committed** — the `FramePump` frame producer
|
||||
(`Services/Encoder/FramePump.cs`), `ScreenCaptureManager.GetLatestFrame(key)`,
|
||||
the `MainViewModel` resolver/option-builders/pump lifecycle wiring, `FramePumpTests`
|
||||
(7) + the `GetLatestFrame` test (1), and the memory updates (TASKS.md, ai.md,
|
||||
Services/index.md — the previous session left `Services/index.md` stale and
|
||||
HANDOFF unrewritten; both were fixed before the commit). Build **0 warnings**,
|
||||
**147 tests passing**. A Windows patch reboot killed the prior session right
|
||||
after the work was verified but before the commit.
|
||||
- **Finished this session:** ship step 5 (re-verified: 0 warnings, 147/147 pass),
|
||||
the `Services/index.md` FramePump row + de-stale'd encoder rows, and this
|
||||
HANDOFF rewrite.
|
||||
- **Landmines:**
|
||||
- `WaveFormatExtensible.SubFormat` is a **`Guid`** in NAudio 2.x, not a
|
||||
`WaveFormatEncoding` — compare it to
|
||||
`NAudio.Dmo.AudioMediaSubtypes.MEDIASUBTYPE_IEEE_FLOAT` (the IEEE-float
|
||||
subtype constant; it lives in the `NAudio.Dmo` namespace, not CoreAudioApi).
|
||||
- Use the **`NAudio.Wasapi` 2.2.1 feature package**, not the `NAudio`
|
||||
meta-package — the wasapi types come from that package (`WasapiCapture` in
|
||||
`NAudio.CoreAudioApi`, `WasapiLoopbackCapture` in `NAudio.Wave`); `NAudio.Core`
|
||||
comes in transitively.
|
||||
- The audio meter is an exponential smoother (0.2 factor) — a single pushed
|
||||
sample only moves 20% toward its RMS. Tests must push repeatedly before
|
||||
asserting a converged level.
|
||||
- The pump reads the active scene on a background thread while the UI can still
|
||||
edit it — a concurrent-mutation exception is contained (logged + `Failed` +
|
||||
the pump stops), not a crash. The background thread + video pipeline is the
|
||||
new reality since ship step 5.
|
||||
- `StopAsync` must stop the encoder (closes stdin) **before** awaiting the pump
|
||||
loop — closing stdin unblocks a write stuck on pipe backpressure; the reverse
|
||||
order deadlocks.
|
||||
- `FramePump.IsRunning` must be set true before the loop starts (a completed-task
|
||||
delay can run the first iteration synchronously on the caller's thread).
|
||||
- `StartAsync` never throws; the VM fires-and-forgets it. `Failed` while live
|
||||
flips `StreamStatus.Error` (minimal — real health surfacing is ship step 6).
|
||||
- Tests never instantiate `MainViewModel` directly except the round-clip
|
||||
integration test (a real `MainWindow`), which never goes live — so the mixer
|
||||
is constructed but never started there; NAudio types are only constructed,
|
||||
never touching devices. Keep it that way.
|
||||
- Mic device resolution must re-read `MicSourceName` at each `Start` (the app
|
||||
persists only the DisplayName, not a device ID).
|
||||
- Windows-only: all capture runs only while live (privacy indicator otherwise).
|
||||
|
||||
- **Next step:** TASK 4 ship step 5 — the frame-pipeline wiring
|
||||
(`CameraManager`/`ScreenCaptureManager` → compositor resolver → encoder
|
||||
construction, plus the loopback → encoder AAC mix). Nothing else queued — do
|
||||
not expand the task queue on your own.
|
||||
integration test (a real `MainWindow`), which never goes live — keep it that way.
|
||||
- **Next step:** TASK 4 ship step 6 — health stats: bind `FramePump.HealthUpdated`
|
||||
(bitrate/FPS/duration) into the bottom bar. Nothing else queued — do not expand
|
||||
the task queue on your own.
|
||||
|
||||
- **Secret/DB/port facts live:** OAuth client id/secret in `Helpers/OAuthCredentials.cs`;
|
||||
OAuth session token in `Helpers/TokenStore.cs` (DPAPI → `%APPDATA%\ytLlive\ytLlive.auth`);
|
||||
|
||||
Reference in New Issue
Block a user