From 3904b626b923862982b3a20ebcfcdfa2955cf9b5 Mon Sep 17 00:00:00 2001 From: gramps Date: Sun, 16 Aug 2026 18:16:10 -0700 Subject: [PATCH] dokunfuk --- .gitignore | 1 + HANDOFF.md | 50 ++++++++++++++++++++++----------------------- Services/index.md | 6 +++--- TASKS.md | 38 +++++++++++++++++----------------- ViewModels/index.md | 4 ++-- ai.md | 34 +++++++++++++++--------------- 6 files changed, 67 insertions(+), 66 deletions(-) diff --git a/.gitignore b/.gitignore index 088129f..397d2fc 100644 --- a/.gitignore +++ b/.gitignore @@ -9,3 +9,4 @@ Helpers/OAuthCredentials.cs # Confidential business files (gitignored, not committed) MONETIZATION.md MARCOM.md +CREDENTIALS.md diff --git a/HANDOFF.md b/HANDOFF.md index d1db8f9..5daa4eb 100644 --- a/HANDOFF.md +++ b/HANDOFF.md @@ -5,11 +5,11 @@ > a problem. Conventions: [`schema.md`](schema.md). Rewrite this file at session > end, compaction, or any interruption. -## Session state (last updated: 2026-08-16) +## Session state (last updated: 2026-08-16, renumbering pass) -- **Branch:** `main`, tracking `origin/main`. Working tree: the **TASK 5 reusable-stream slice** is +- **Branch:** `main`, tracking `origin/main`. Working tree: the **TASK 9 reusable-stream slice** is committed + pushed (details below). Local branches `social-bar`/`webcam-validation` untouched (no secrets). -- **TASK 5 — REUSABLE STREAM + HEALTH BANNER — SHIPPED 2026-08-16 (items 1–3).** +- **TASK 9 — REUSABLE STREAM + HEALTH BANNER — SHIPPED 2026-08-16 (items 1–3).** - **Items 1–2 (reusable stream):** `_rtmpUrlProvider` now yields a real RTMP URL. `YouTubeStreamService` gains `GetOrCreateReusableStreamAsync` (lists `liveStreams?mine=true`, reuses the existing `cdn.isReusable` stream, inserts once on first use with @@ -32,18 +32,18 @@ - **CONFIDENTIAL files created (gitignored):** `MONETIZATION.md` (pricing, billing research, unlock mechanics) and `MARCOM.md` (launch marketing strategy, positioning, platform strategy). These are NOT committed to the public repo. `Helpers/OAuthCredentials.cs` was already gitignored. -- **TASK 5 items 6-7 scoped (not built):** visibility unlock (remove temporary "always Private" enforcement) +- **TASK 9 items 6-7 scoped (not built):** visibility unlock (remove temporary "always Private" enforcement) and full broadcast form (Core + Advanced tabs with all YouTube API-supported fields). These are the next technical tasks after live chat (item 4) and error handling (item 5). -- **TASK 23 trimmed** to technical scope only (billing decision, unlock mechanism, bug report, alerts gating). +- **TASK 12 trimmed** to technical scope only (billing decision, unlock mechanism, bug report, alerts gating). Business details moved to `MONETIZATION.md`. -- **TASK 26 added** — marcom/launch kit. Business details moved to `MARCOM.md`. Queued after all v1 features. +- **TASK 15 added** — marcom/launch kit. Business details moved to `MARCOM.md`. Queued after all v1 features. - **Backlog updated** — removed v0.3 (stream scheduling, not needed for casual streamers). Multi-destination restreaming noted as "congrats, you're ready for OBS" moment. -- **Resume point (next branch):** **TASK 5 item 4 — live chat** (`liveChat/messages` poll, right-panel +- **Resume point (next branch):** **TASK 9 item 4 — live chat** (`liveChat/messages` poll, right-panel render, Super Chat + membership badges). Then item 5 (error handling), item 6 (visibility unlock), - item 7 (full broadcast form), then TASK 23 (billing + unlock + support). -- **TASK 25 — MASTER LIMITER — COMMITTED + PUSHED 2026-08-15.** Queued from the TRAX discussion: the + item 7 (full broadcast form), then TASK 12 (billing + unlock + support). +- **TASK 14 — MASTER LIMITER — COMMITTED + PUSHED 2026-08-15.** Queued from the TRAX discussion: the live mix summed mic + loopback with no ceiling, so hot gains could pass 0 dBFS and clip the AAC encode. New pure `Services/Audio/MasterLimiter.cs` (−1 dBFS ceiling, instant attack per frame, smoothed release) applied at the end of `AudioMixer.FillAndMix`. ONE integration test @@ -56,27 +56,27 @@ TRAX music played). The whole `IGameAudioDetector`/`GameAudioDetector`/`GameAudioHysteresis` stack + the VM's 250ms poll timer + its two test files were **deleted**; the bar is now permanently overlaid at the bottom of the preview. Build **0 warnings**, full suite green. -- **TASK 24 — POST-PAUSE POLISH BATCH — SHIPPED + COMMITTED + PUSHED 2026-08-15.** All 8 creator +- **TASK 13 — POST-PAUSE POLISH BATCH — SHIPPED + COMMITTED + PUSHED 2026-08-15.** All 8 creator review issues fixed in one branch (details below). Build **0 warnings**, full suite **197 passing** (ONE integration test for that branch: `AudioPipelineTests.Mix_HonorsProviderGains_AndGameMute_KillsTheLoopback`). -- **AUDIO MILESTONE (TASK 9) — SHIPPED + COMMITTED + PUSHED.** Real stream audio + voice filters + +- **AUDIO MILESTONE (TASK 8) — SHIPPED + COMMITTED + PUSHED.** Real stream audio + voice filters + auto-duck + free TRAX background music. Commits `6d71ace` (milestone) + `f2f6401` (secrets cleanup), force-pushed (`725f5a7...f2f6401`). - **MONETIZATION LOCKED (2026-08-14, creator) — committed `86fcd86`.** One paid line = **annual subscription**: early access **$49.99/yr** → **$99/yr list at GA**, **grandfather-while-subscribed**, lapse → list on renewal. Free tier unchanged (branding flash = the billboard + no Alerts; Alerts is the paid feature). **Billing NOT itch-locked**; candidates Gumroad (native affiliates = - tiebreaker) / Lemon Squeezy. Support = in-app bug-report → git issues. Full policy in `ai.md` → - Monetization; scoped plan in `TASKS.md` **TASK 23**. + tiebreaker) / Lemon Squeezy. Support = in-app bug-report → git issues. Full policy in `ai.md` → + Monetization; scoped plan in `TASKS.md` **TASK 12**. - **Secrets scrubbed from git history.** The DO token + passwords were purged via `git filter-branch` + `git gc --prune=now --aggressive` (all-refs scan = 0 hits); values still exist in chat — keep treating as **compromised**; rotate the DO API token. -- **Shipped, all pushed:** TASK 5 items 1–3 (reusable stream + health banner, 2026-08-16), TASK 25 - master limiter, TASK 24 polish batch, TASK 21 (creator-hub About, `3d92bd0`), TASK 8 (meter +10 dB), - TASK 4 ship step 7 (one-click go-live + private-only), TASK 9 audio milestone (`6d71ace`), secrets +- **Shipped, all pushed:** TASK 9 items 1–3 (reusable stream + health banner, 2026-08-16), TASK 14 + master limiter, TASK 13 polish batch, TASK 11 (creator-hub About, `3d92bd0`), TASK 7 (meter +10 dB), + TASK 4 ship step 7 (one-click go-live + private-only), TASK 8 audio milestone (`6d71ace`), secrets cleanup (`f2f6401`), monetization docs (`86fcd86`). -## TASK 5 — reusable stream + health banner 2026-08-16 (what changed, items 1–3) +## TASK 9 — reusable stream + health banner 2026-08-16 (what changed, items 1–3) The recorded resume point: the "last blocker" was that go-live ran the visual flow but never pushed — `_rtmpUrlProvider` returned null, so `FramePump.StartAsync` skipped the encoder entirely. @@ -103,7 +103,7 @@ The recorded resume point: the "last blocker" was that go-live ran the visual fl `FramePumpTests.ReusableStream_Url_From_Service_Feeds_Encoder_Startup` (real service + real pump + hermetic HTTP: the reusable stream's URL lands in `EncoderOptions.RtmpUrl` and the encoder starts). -**Out of scope this branch (next branches):** TASK 5 item 4 live chat, item 5 the YouTube error-code +**Out of scope this branch (next branches):** TASK 9 item 4 live chat, item 5 the YouTube error-code mappings, and the design's bottom-strip YouTube logo/green-red dot (clickable → dialog). ### Item 3 — report-by-exception health banner (same session, second branch-worth of scope) @@ -129,9 +129,9 @@ mappings, and the design's bottom-strip YouTube logo/green-red dot (clickable + the ONE integration test `GetStreamHealthAsync_Report_By_Exception_Banner_Only_On_Warning_Or_Error` (real service JSON parse → real reporter: good → no banner, error issue → banner text + error color). -## TASK 24 — the 8-issue polish batch — SHIPPED 2026-08-15 (what changed) +## TASK 13 — the 8-issue polish batch — SHIPPED 2026-08-15 (what changed) -The creator's review of the TASK 9 build, all fixed in one branch (full record in `TASKS.md` TASK 24). +The creator's review of the TASK 8 build, all fixed in one branch (full record in `TASKS.md` TASK 13). 1. ✅ **Desktop/game audio volume slider had no effect** — the `AudioMixer` gain seams defaulted to unity (the VM never passed them): the slider/mute were decorative, stream AND local. Fixed with a @@ -175,7 +175,7 @@ sound or TRAX played, so it sat hidden during normal use. Fix = the bar is now * - **MainWindow.xaml:** dropped the `Visibility` binding on the preview-overlay game bar; it renders unconditionally (TRAX button + "Desktop Audio" label + meter + mute + volume unchanged). -## TASK 25 — master limiter 2026-08-15 (what changed) +## TASK 14 — master limiter 2026-08-15 (what changed) Came out of the TRAX discussion (file-size guard? 20% cap? sound-event balance?). Verdict: two of the three instincts were already satisfied, one real gap existed: @@ -195,10 +195,10 @@ three instincts were already satisfied, one real gap existed: 0.95 loopback bed capped to exactly 0.891 on the wire while staying audible. Plus 3 unit tests for the pure math. -## TASK 9 audio milestone — SHIPPED 2026-08-14 (what changed) +## TASK 8 audio milestone — SHIPPED 2026-08-14 (what changed) The creator's feature review settled this as the single next branch ("all the audio issues done and -tested — a huge milestone"). Final spec and full shipped-state records live in `TASKS.md` (TASK 9) +tested — a huge milestone"). Final spec and full shipped-state records live in `TASKS.md` (TASK 8) and `ai.md` ("Live audio capture"). Highlights: - **Real audio into the encoder.** `FfmpegArgs` now builds `-f f32le -ar 48000 -ac 2 -i \\.\pipe\ytllive_audio` @@ -207,7 +207,7 @@ and `ai.md` ("Live audio capture"). Highlights: before `_framePump.StopAsync()`. - **2-input mix (mic + loopback)**, honest gains: `micGain = MicVolume` (mute = 0), `loopbackGain = GameMuted ? 0 : GameAudioVolume` × duck. No third music channel. -- **Voice chain on the mic** (TASK 22), before meter AND mix: bass 120 Hz +4 dB → treble 8 kHz +3 dB → +- **Voice chain on the mic** (TASK 10), before meter AND mix: bass 120 Hz +4 dB → treble 8 kHz +3 dB → gate (0.005 / hysteresis 0.5) → compressor (0.5, 4:1). Pure TDF2 DSP, per-sample, always on. - **Auto-duck:** mic RMS > 0.02 → loopback ×0.25 (−12 dB), attack 0.05 / release 0.005. - **TRAX (free BGM):** `MusicPlayer` = MediaFoundationReader → `VolumeWaveProvider16` at fixed **0.20** → @@ -224,7 +224,7 @@ and `ai.md` ("Live audio capture"). Highlights: start-of-stream silence race was seen and eliminated). **Out of scope this branch:** IP webcam, chat box, alt-key crop, credits, bg removal, music-off-VOD -track (YouTube mutes VODs with copyrighted music — future feature), `PremiumUrl` (TASK 23 seam). +track (YouTube mutes VODs with copyrighted music — future feature), `PremiumUrl` (TASK 12 seam). - **Landmines:** - Never add another test that constructs `new App()` — use `RealAppHost.Run(...)` (shared STA host diff --git a/Services/index.md b/Services/index.md index 1b2e63b..de14d72 100644 --- a/Services/index.md +++ b/Services/index.md @@ -6,10 +6,10 @@ External-facing logic: YouTube API, persistence. See | File | Purpose | |------|---------| | `YouTubeAuthService.cs` | OAuth2 via Google: loopback callback (`http://localhost:8765/oauth2/callback`), token exchange, refresh, channel fetch. Constructor takes optional `HttpClient` + `sessionChanged` callback (test seam + save hook); session persists via `Helpers/TokenStore` (DPAPI); `ClearSession()` signs out (called by `MainViewModel.StopStream` on End Livestream) | -| `YouTubeStreamService.cs` | Broadcast/stream management via the v3 API. **`CreateBroadcast` always sends `privacyStatus = "private"`** (ship step 7 — private-only until v1) and the one-click v3 flags (`enableAutoStart/Stop`, `enableMonitorStream=false`, `latencyPreference=low`); injectable `HttpClient? http = null` ctor seam so tests can fake the API. **`GetOrCreateReusableStreamAsync` (TASK 5, shipped 2026-08-16)** returns the channel's `ReusableStream` — lists `liveStreams?mine=true` and reuses the existing `cdn.isReusable` stream, inserting once only on first use (`resolution=variable`, `frameRate=variable`, `isReusable=true`); a stream id given to `CreateBroadcast` binds at insert via `contentDetails.boundStreamId`. RTMP URL = `ingestionAddress + "/" + streamName`. **`GetStreamHealthAsync(streamId)` (TASK 5 item 3, shipped 2026-08-16)** polls `liveStreams?part=status` → `StreamHealth` with parsed `configurationIssues[]` | -| `StreamHealthReporter.cs` | Pure report-by-exception decision (TASK 5 item 3): `BannerFor(issues)` → `HealthIssueReport(Text?, IsError)` — null text on good/ok/noData/info-only, first warning/error issue produces its type text, error beats warning for color | +| `YouTubeStreamService.cs` | Broadcast/stream management via the v3 API. **`CreateBroadcast` always sends `privacyStatus = "private"`** (ship step 7 — private-only until v1) and the one-click v3 flags (`enableAutoStart/Stop`, `enableMonitorStream=false`, `latencyPreference=low`); injectable `HttpClient? http = null` ctor seam so tests can fake the API. **`GetOrCreateReusableStreamAsync` (TASK 9, shipped 2026-08-16)** returns the channel's `ReusableStream` — lists `liveStreams?mine=true` and reuses the existing `cdn.isReusable` stream, inserting once only on first use (`resolution=variable`, `frameRate=variable`, `isReusable=true`); a stream id given to `CreateBroadcast` binds at insert via `contentDetails.boundStreamId`. RTMP URL = `ingestionAddress + "/" + streamName`. **`GetStreamHealthAsync(streamId)` (TASK 9 item 3, shipped 2026-08-16)** polls `liveStreams?part=status` → `StreamHealth` with parsed `configurationIssues[]` | +| `StreamHealthReporter.cs` | Pure report-by-exception decision (TASK 9 item 3): `BannerFor(issues)` → `HealthIssueReport(Text?, IsError)` — null text on good/ok/noData/info-only, first warning/error issue produces its type text, error beats warning for color | | `YouTubeChatService.cs` | Polls `liveChat/messages`, raises `MessageReceived`; `IDisposable` | -| `LayoutStore.cs` | SQLite persistence (`Microsoft.Data.Sqlite`) at `%APPDATA%\ytLlive\ytLlive.db`; assets stored as BLOBs keyed by SHA-256 content hash; save/open layout files; schema `user_version` 6 (`Source.ClipShape`/`IsMirrored` via `ALTER TABLE` for pre-v2 DBs; v3 = singleton `Webcam` + per-scene `WebcamSceneConfig`, migrated idempotently **without backfill** — the stale `Source.DeviceId` column remains but is no longer read/written; v4 = `WebcamSceneConfig.RectWidth`/`RectHeight`, the pre-Round rect for the round-to-rect restore; v5 = `Source.IsBackdrop` + `Source.CaptureKey`, the live-capture backdrop; v6 = `Scene.HasBackdrop` — Live-only policy, one-time backfill turns Starting/BRB/Chat/Ending off + drops their backdrop sources; `MainViewModel.EnforceBackdropPolicy` re-normalizes on every load). Settings key/value table holds app state: `MicSourceName`, `ReusableStream` (TASK 5 — the cached reusable stream's id/address/name so the pump has its RTMP URL at startup) | +| `LayoutStore.cs` | SQLite persistence (`Microsoft.Data.Sqlite`) at `%APPDATA%\ytLlive\ytLlive.db`; assets stored as BLOBs keyed by SHA-256 content hash; save/open layout files; schema `user_version` 6 (`Source.ClipShape`/`IsMirrored` via `ALTER TABLE` for pre-v2 DBs; v3 = singleton `Webcam` + per-scene `WebcamSceneConfig`, migrated idempotently **without backfill** — the stale `Source.DeviceId` column remains but is no longer read/written; v4 = `WebcamSceneConfig.RectWidth`/`RectHeight`, the pre-Round rect for the round-to-rect restore; v5 = `Source.IsBackdrop` + `Source.CaptureKey`, the live-capture backdrop; v6 = `Scene.HasBackdrop` — Live-only policy, one-time backfill turns Starting/BRB/Chat/Ending off + drops their backdrop sources; `MainViewModel.EnforceBackdropPolicy` re-normalizes on every load). Settings key/value table holds app state: `MicSourceName`, `ReusableStream` (TASK 9 — the cached reusable stream's id/address/name so the pump has its RTMP URL at startup) | | `VideoFrame.cs` | Normalized CPU frame seam (`Width`/`Height`/tightly-packed BGRA `byte[]`) — the only pixel type the rest of the app knows about; future capture sources (screen, background-removed webcam) feed the same seam | | `CameraDeviceInfo.cs` | `(Id, DisplayName)` for a physical capture device | | `ICameraEnumerator.cs` | `GetCamerasAsync()` — seam so the picker/`CameraManager` never touch WinRT (tests inject fakes) | diff --git a/TASKS.md b/TASKS.md index 1491bb7..2367d01 100644 --- a/TASKS.md +++ b/TASKS.md @@ -115,7 +115,7 @@ Lifecycle: `created → ready → [testing] → live → complete` (transitional 18. ☐ **Chat box** — YouTube live chat rendered *on* the stream so viewers read along in-video 19. ❌ **Background removal (milestone 2)** — ONNX Runtime + DirectML, MediaPipe Selfie Segmentation — deliberately NOT in this build 20. ☐ **Alerts** — Super Chat / membership / subscribe pop-ins; build after the six; **the one paid feature** (see Monetization in `ai.md`) -21. ✅ **Logo + richer in-app About (2026-08-13, queued → 2026-08-14 SHIPPED)** — the ytLlive wordmark in the top bar opens the About overlay (already wired); the About overlay is now the **creator hub**: the real logo (the "llama fortnite superman logo" from the creator's vault, copied to `Assets/llama-logo.png` — the creator's own art, no third-party license), plus **creator-hub links** — llama chile shop on YouTube (`MainViewModel.ChannelUrl`), Mastodon (`https://mastodon.llamachile.tube/@gramps`), **Buy me a coffee** (`https://buymeacoffee.com/llamachiley` — live), and **Unlock Premium** (greyed "coming soon" — the **billing** product URL is a tabled seam, `PremiumUrl`, until TASK 23 picks the subscription provider). A **Licenses & legal** button flips the About overlay to an in-app scrolling panel that loads the full `THIRD-PARTY-NOTICES.txt` text (`MainViewModel.ShowLicenses` reads the shipped file from the executable directory on first open; graceful "not found" fallback — **never the OS viewer, everything stays in-app**); "← Back to About" returns. The About overlay is also the in-app home of the notices — the top-bar About button that opened the file in the OS viewer was removed on 2026-08-13 for exactly this reason. Integration test (Good Dog Rule — ONE): `AboutHubTests.About_Opens_InApp_Licensing_Loads_Shipped_Notices` drives the real window + VM, asserting the hub opens, the link URLs are real, the licensing panel loads the shipped notices text (contains "Third-Party Notices" + "LGPL"), and back returns to the hub. 174 tests passing, 0 warnings +21. ✅ **Logo + richer in-app About (2026-08-13, queued → 2026-08-14 SHIPPED)** — the ytLlive wordmark in the top bar opens the About overlay (already wired); the About overlay is now the **creator hub**: the real logo (the "llama fortnite superman logo" from the creator's vault, copied to `Assets/llama-logo.png` — the creator's own art, no third-party license), plus **creator-hub links** — llama chile shop on YouTube (`MainViewModel.ChannelUrl`), Mastodon (`https://mastodon.llamachile.tube/@gramps`), **Buy me a coffee** (`https://buymeacoffee.com/llamachiley` — live), and **Unlock Premium** (greyed "coming soon" — the **billing** product URL is a tabled seam, `PremiumUrl`, until TASK 12 picks the subscription provider). A **Licenses & legal** button flips the About overlay to an in-app scrolling panel that loads the full `THIRD-PARTY-NOTICES.txt` text (`MainViewModel.ShowLicenses` reads the shipped file from the executable directory on first open; graceful "not found" fallback — **never the OS viewer, everything stays in-app**); "← Back to About" returns. The About overlay is also the in-app home of the notices — the top-bar About button that opened the file in the OS viewer was removed on 2026-08-13 for exactly this reason. Integration test (Good Dog Rule — ONE): `AboutHubTests.About_Opens_InApp_Licensing_Loads_Shipped_Notices` drives the real window + VM, asserting the hub opens, the link URLs are real, the licensing panel loads the shipped notices text (contains "Third-Party Notices" + "LGPL"), and back returns to the hub. 174 tests passing, 0 warnings 22. ✅ **Voice filters on the mic channel (2026-08-13 queued → SHIPPED 2026-08-14 inside TASK 9, the audio milestone)** — the standard four applied to the sound input path (before the meter/encoder mix): **bass boost, treble, noise suppression, compressor** (set decided with the creator 2026-08-13). Noise suppression = a **pure-C# noise gate** (creator chose over RNNoise / a second ffmpeg `afftdn` pipe, 2026-08-14 — KISS). Always-on — no UI knobs; the mic stays the creator's single audio control ### The Minimal Source Set (design decision — do not expand casually) @@ -204,8 +204,8 @@ Preview shows the transition too (WYSIWYG). No wipes/slides/LUTs beyond the four 3. ✅ **Encoder + RTMP push SHIPPED** (2026-08-12) — the FFmpeg subprocess: raw BGRA frames via stdin, stderr health parsing, FLV mux + push to the ingestion URL (see the ship step 3 plan below) 4. ✅ **WASAPI audio capture SHIPPED** (2026-08-12) — NAudio loopback (desktop/game) + the picked mic feeding `AudioLevel`, so the realtime meter comes alive (see the ship step 4 plan below) 5. ✅ **Frame-pipeline wiring SHIPPED** (2026-08-12) — `CameraManager`/`ScreenCaptureManager` → compositor resolver → encoder, driven by a paced `FramePump` (see the ship step 5 plan below) -6. ✅ **Health stats SHIPPED** (2026-08-13) — `FramePump.HealthUpdated` (encoder's parsed bitrate/FPS/dropped/duration, already forwarded from `FfmpegEncoder.OnStderrLine`) now lands in the bottom bar: `MainViewModel.OnFramePumpHealthUpdated` marshals to the UI thread (the stderr loop raises on a background thread) and copies into `CurrentHealth` (the bottom bar's existing binding); `ResetHealth` zeroes dropped/duration on go-live and on End so stats never linger from a previous session (bitrate/FPS stay on the tier's targets). The bar lights up with real values once TASK 5 supplies the RTMP URL (until then the pump skips the encoder and the bar shows the tier's targets) -7. ✅ **One-click go live + private-only enforcement SHIPPED** (2026-08-14) — the Go Live dialog is **locked to Private** (no dropdown, `GoLiveViewModel.Visibility` is a get-only "Private"); `YouTubeStreamService.CreateBroadcast` **always sends `privacyStatus = "private"`** (dialog + service enforcement, requirement 8 — nothing can go out non-private) and gained an injectable `HttpClient? http = null` seam for tests; `BeginGoLive` now calls `CreateBroadcastAsync` and remembers `_currentBroadcastId` for TASK 5's bind/transition (failure → `StreamStatus.Error`, never a crash; `StopStream` clears the ID); the REC sign shows a **PRIVATE badge** (dark-red border, next to REC, `IsLivePrivate`) when the live stream is private; settings' dead "Default Visibility" dropdown + `MainViewModel.Visibilities`/`DefaultStreamVisibility` removed. The broadcast-insert integration test asserts the request body carries `"privacyStatus":"private"` (2 new tests → 173 passing, 0 warnings) +6. ✅ **Health stats SHIPPED** (2026-08-13) — `FramePump.HealthUpdated` (encoder's parsed bitrate/FPS/dropped/duration, already forwarded from `FfmpegEncoder.OnStderrLine`) now lands in the bottom bar: `MainViewModel.OnFramePumpHealthUpdated` marshals to the UI thread (the stderr loop raises on a background thread) and copies into `CurrentHealth` (the bottom bar's existing binding); `ResetHealth` zeroes dropped/duration on go-live and on End so stats never linger from a previous session (bitrate/FPS stay on the tier's targets). The bar lights up with real values once TASK 9 supplies the RTMP URL (until then the pump skips the encoder and the bar shows the tier's targets) +7. ✅ **One-click go live + private-only enforcement SHIPPED** (2026-08-14) — the Go Live dialog is **locked to Private** (no dropdown, `GoLiveViewModel.Visibility` is a get-only "Private"); `YouTubeStreamService.CreateBroadcast` **always sends `privacyStatus = "private"`** (dialog + service enforcement, requirement 8 — nothing can go out non-private) and gained an injectable `HttpClient? http = null` seam for tests; `BeginGoLive` now calls `CreateBroadcastAsync` and remembers `_currentBroadcastId` for TASK 9's bind/transition (failure → `StreamStatus.Error`, never a crash; `StopStream` clears the ID); the REC sign shows a **PRIVATE badge** (dark-red border, next to REC, `IsLivePrivate`) when the live stream is private; settings' dead "Default Visibility" dropdown + `MainViewModel.Visibilities`/`DefaultStreamVisibility` removed. The broadcast-insert integration test asserts the request body carries `"privacyStatus":"private"` (2 new tests → 173 passing, 0 warnings) The pipeline chain the encoder needs doesn't exist yet: **scene compositing** (the master 1920×1080 frame without the preview's editing chrome) → **audio capture** (WASAPI, feeds the meter) → **H.264+AAC encode** @@ -440,7 +440,7 @@ composite into the tier's output frame → pace into the encoder's stdin at the 1. **Video pipeline first** — the `-f lavfi -i anullsrc` silent track stays; mixing the loopback/mic WASAPI samples into the encoder's AAC track is its own later step. 2. **RTMP URL via a provider seam** — `MainViewModel._rtmpUrlProvider` is a `Func` returning - null today (the reusable stream's ingest URL lands with TASK 5); when it yields null the pump logs + null today (the reusable stream's ingest URL lands with TASK 9); when it yields null the pump logs and skips the encoder entirely, so go-live runs the existing visual flow without pushing. **Design:** @@ -526,15 +526,15 @@ the validator → persisted), compositor bar overlay (top/bottom + above-flash), --- -## TASK 9 — Audio milestone: real stream audio + voice filters + TRAX music (2026-08-14) +## TASK 8 — Audio milestone: real stream audio + voice filters + TRAX music (2026-08-14) -**Goal:** every audio issue done and tested in one branch — real mic/game audio reaches the encoder (replacing the `anullsrc` silence), `MicVolume`/`GameAudioVolume`/mute become real pre-AAC gains, TASK 22's voice filters land, auto-duck keeps the creator's voice over game + music, and a free background-music source ("TRAX") plays into the sound bar and the stream. +**Goal:** every audio issue done and tested in one branch — real mic/game audio reaches the encoder (replacing the `anullsrc` silence), `MicVolume`/`GameAudioVolume`/mute become real pre-AAC gains, TASK 10's voice filters land, auto-duck keeps the creator's voice over game + music, and a free background-music source ("TRAX") plays into the sound bar and the stream. ### Status: ✅ SHIPPED 2026-08-14 — 196 tests passing, 0 warnings (ONE integration test: `AudioPipelineTests.Mix_WithFiltersDuckAndGain_Lands_On_AudioPipe`) 1. ✅ **Real audio into the encoder** — the mixer's loopback samples were **dropped today** (`AudioMixer.OnLoopbackSample` metered only) and `MicVolume` was meter-only; `FfmpegArgs.cs:28` ran `-f lavfi -i anullsrc` (silence). Transport: **ffmpeg reads a Windows named pipe** — `-f f32le -ar 48000 -ac 2 -i \\.\pipe\ytllive_audio` replaces the anullsrc block, plus explicit `-map 0:v -map 1:a`; pipe name via `EncoderOptions.AudioPipeName` (`DefaultAudioPipeName = "ytllive_audio"`). Mixer stays **two inputs (mic + loopback)** — no N-source abstraction, no `StereoMixer` class. The VM owns the pipe lifecycle (`BeginGoLive` → `_audioMixer.StartLive(pipeName)`, `StopStream` → `_audioMixer.StopLive()` before the pump stops); `FfmpegEncoder.cs` is untouched. 2. ✅ **Honest gains reach the stream** — `MicVolume` scales the mic channel; `GameAudioVolume` + mute scale the loopback channel (the "mute in preview but stream still plays" footgun dies with it). One knob per input = KISS. Implemented as `Func` gain seams on `AudioMixer` (`micGain`/`loopbackGain`), wired by `MainViewModel`. -3. ✅ **Voice filters (TASK 22, before the meter AND the mix)** — `VoiceFilterChain`: bass (`LowShelfFilter` 120 Hz +4 dB) → treble (`HighShelfFilter` 8 kHz +3 dB) → **noise gate** (pure C#, creator's choice) → compressor (threshold 0.5, 4:1). Pure stateful DSP (TDF2), sine-in unit tests. Meter stays post-filter. +3. ✅ **Voice filters (TASK 10, before the meter AND the mix)** — `VoiceFilterChain`: bass (`LowShelfFilter` 120 Hz +4 dB) → treble (`HighShelfFilter` 8 kHz +3 dB) → **noise gate** (pure C#, creator's choice) → compressor (threshold 0.5, 4:1). Pure stateful DSP (TDF2), sine-in unit tests. Meter stays post-filter. 4. ✅ **Auto-duck** — pure envelope (`AutoDucker`): mic RMS > 0.02 → game+music dip ×0.25 (~12 dB), attack 0.05 / release 0.005, recover on release. Always-on, no knobs. Strong form of the creator's "voice always over the game volume" idea. 5. ✅ **TRAX — background music, FREE** (keeps the one paid line = Alerts + flash removal). `MusicPlayer`: NAudio `MediaFoundationReader` (mp3/wav/m4a) → `VolumeWaveProvider16` at the hardcoded **0.20** (fixed, not changeable) → `WaveOutEvent` on the default device (added the sibling `NAudio.WinMM` 2.2.1 package — `WaveOutEvent` isn't in `NAudio.Wasapi`), loop on end. **No 3rd mixer input and no `MusicVolume`** — music plays on the desktop, the existing loopback picks it up: heard in headphones, the sound-bar meter bounces, and the stream carries it through the loopback channel (ducked with game when the voice is hot). Known wrinkle (out of scope): YouTube mutes VODs carrying copyrighted music — future "music on live, off VOD". 6. ✅ **TRAX footer control (final spec, agreed 2026-08-14)** — `YtButtonSecondary`, **status dot + "TRAX"** text, in the center footer stack beside MIC: @@ -546,7 +546,7 @@ the validator → persisted), compositor bar overlay (top/bottom + above-flash), 7. ✅ **Sound bar shows music** — `IsGameAudioBarVisible = gameDetectorProducingSound || isMusicPlaying`; relabelled "Game Audio Capture" → **"Desktop Audio"** (TRAX rides the same channel). **Follow-up (2026-08-15): the gating is gone — the game audio bar is ALWAYS visible** (the `IGameAudioDetector`/`GameAudioDetector`/`GameAudioHysteresis` stack + the VM's 250ms poll timer and their tests were deleted; the bar kept hiding the desktop meter when no full-screen game with sound was up, and the creator wants it pinned). 8. ✅ **Capture hardening (design changed at build time)** — the plan's "force IEEE-float 48 kHz on both WASAPI sources" was **dropped**: NAudio 2.2.1's `WasapiCapture` exposes no overridable `GetDefaultMixFormat`, so both sources capture the device's own mix format and the mixer's pure `TinyResampler` normalizes any rate/channel count to 48 kHz stereo (the resampler IS the design, not a fallback). Stop path disposes the audio pipe (EOF) in `MainViewModel.StopStream` before the pump stops, so both ffmpeg inputs end in order. 9. ✅ **Schema v9** — single-row `Music` (`TrackPath`, `IsEnabled`; volume is the 0.20 constant) + migration in `LayoutStore.cs` (`user_version` 9; `Save` gained an optional `Music? music = null` param so existing 3-arg callers still compile; load query + null reset). -10. ✅ **Docs in the same commit** — `ai.md` audio section rewritten ("one knob per input", 2-in mix, auto-duck, TRAX free, VOD-mute wrinkle), TASK 22's status, this task's status, HANDOFF. +10. ✅ **Docs in the same commit** — `ai.md` audio section rewritten ("one knob per input", 2-in mix, auto-duck, TRAX free, VOD-mute wrinkle), TASK 10's status, this task's status, HANDOFF. **Files (new):** `Services/Audio/VoiceFilterChain.cs` (+ `LowShelfFilter`/`HighShelfFilter`/`NoiseGate`/`Compressor`), `Services/Audio/AudioRingBuffer.cs` (async-arrival source buffers), `Services/Audio/AutoDucker.cs`, `Services/Audio/TinyResampler.cs`, `Services/Audio/MusicPlayer.cs`, `Services/Audio/NamedPipeAudioWriter.cs` (+ `IAudioPipeWriter` seam), `Models/Music.cs`. @@ -554,11 +554,11 @@ the validator → persisted), compositor bar overlay (top/bottom + above-flash), **Testing (Good Dog Rule — ONE integration test):** `AudioPipelineTests.Mix_WithFiltersDuckAndGain_Lands_On_AudioPipe` — real filter chain + mixer + ducker + `NamedPipeAudioWriter`, fake `IAudioSource`s, test-side `NamedPipeClientStream` reads the bytes and asserts post-filter/post-gain/post-duck mixed stereo (ring buffers pre-filled so the first pipe tick already carries real audio — avoids a start-of-stream silence race). Units: each DSP stage (known sine-in → expected gain), ring buffer wrap/underflow/overwrite, ducker envelope, resampler (down/up/stateful/identity), `FfmpegArgs` pipe+map, schema v9 roundtrip + no-music-row. Existing tests stay green. -**Out of scope (this branch):** IP webcam (video-only when it lands — never an audio input), chat box source (TASK 3 item 18), alt-key crop, credits, background removal, music-off-VOD, `PremiumUrl` (TASK 23 seam). +**Out of scope (this branch):** IP webcam (video-only when it lands — never an audio input), chat box source (TASK 3 item 18), alt-key crop, credits, background removal, music-off-VOD, `PremiumUrl` (TASK 12 seam). --- -## TASK 5 — YouTube Live Stream Management +## TASK 9 — YouTube Live Stream Management **Goal:** Create/bind broadcasts, monitor YouTube-side stream health — the v3 way. @@ -594,7 +594,7 @@ the validator → persisted), compositor bar overlay (top/bottom + above-flash), --- -## TASK 23 — Monetization: subscription billing + in-app unlock + support +## TASK 12 — Monetization: subscription billing + in-app unlock + support **Goal:** annual subscription billing, in-app unlock seam, and in-app bug-report → git-issues support loop. @@ -610,7 +610,7 @@ the validator → persisted), compositor bar overlay (top/bottom + above-flash), --- -## TASK 6 — Layout Persistence (SQLite) +## TASK 5 — Layout Persistence (SQLite) **Goal:** Scenes, sources, and asset bytes survive restarts; assets are always available. @@ -621,7 +621,7 @@ the validator → persisted), compositor bar overlay (top/bottom + above-flash), 3. ✅ File-model save/open — the active layout file is tracked (default is the AppData DB); **Save Layout As… / Open Layout…** switch the active file; auto-save writes to whatever is active 4. ✅ Auto-save (invisible) — ~1.5s debounce on scene/source add/remove/reorder/rename/hide + any source transform change; flush on window close 5. ✅ Startup — load the active file; seed the five canonical scenes only when the DB is empty; (+) re-adds a missing canonical scene and is hidden once all five are present; adding beyond the five is rejected -6. ✅ Schema v1 → v8 — webcam columns (v2), singleton `Webcam` + per-scene `WebcamSceneConfig` (v3), `RectWidth`/`RectHeight` round-to-rect restore (v4), `Source.IsBackdrop` + `Source.CaptureKey` (v5), `Scene.HasBackdrop` — backdrop **Live-only by policy** (v6, one-time backfill + `EnforceBackdropPolicy` on every load), `Scene.HasSocialBar` (v7, dropped per-scene toggle — column back-compat, unread), `Socials.BarEnabled` (v8); the `SocialEntry.Software` fediverse-software column is a **column-presence migration** (commented v8→v9, no version bump — `user_version` stays 8); `WindowHandle` stays in-memory (per-session); save = transactional rewrite; orphaned assets pruned. **v9 lands in the TASK 9 audio milestone** (single-row `Music` — `TrackPath`/`IsEnabled`) +6. ✅ Schema v1 → v8 — webcam columns (v2), singleton `Webcam` + per-scene `WebcamSceneConfig` (v3), `RectWidth`/`RectHeight` round-to-rect restore (v4), `Source.IsBackdrop` + `Source.CaptureKey` (v5), `Scene.HasBackdrop` — backdrop **Live-only by policy** (v6, one-time backfill + `EnforceBackdropPolicy` on every load), `Scene.HasSocialBar` (v7, dropped per-scene toggle — column back-compat, unread), `Socials.BarEnabled` (v8); the `SocialEntry.Software` fediverse-software column is a **column-presence migration** (commented v8→v9, no version bump — `user_version` stays 8); `WindowHandle` stays in-memory (per-session); save = transactional rewrite; orphaned assets pruned. **v9 lands in the TASK 8 audio milestone** (single-row `Music` — `TrackPath`/`IsEnabled`) ### Design decisions @@ -655,7 +655,7 @@ the validator → persisted), compositor bar overlay (top/bottom + above-flash), --- -## TASK 7 — UI polish batch (scenes/sources rows, dedup naming, social bar) +## TASK 6 — UI polish batch (scenes/sources rows, dedup naming, social bar) **Goal:** clean up the two side lists and the social bar per gramps's review. @@ -675,7 +675,7 @@ the validator → persisted), compositor bar overlay (top/bottom + above-flash), --- -## TASK 8 — Meter scaling amplification (voice meter uses the full bar) +## TASK 7 — Meter scaling amplification (voice meter uses the full bar) **Goal:** the mic (and game) meters feel light — speech peaks should peg into the red at maxed volume. @@ -691,12 +691,12 @@ the validator → persisted), compositor bar overlay (top/bottom + above-flash), --- -## TASK 24 — Post-pause polish batch: the creator's 8 review issues (2026-08-15, shipped) +## TASK 13 — Post-pause polish batch: the creator's 8 review issues (2026-08-15, shipped) **Status: ✅ SHIPPED 2026-08-15 — 197 tests passing, 0 warnings (ONE integration test: `AudioPipelineTests.Mix_HonorsProviderGains_AndGameMute_KillsTheLoopback`).** -The creator reviewed the TASK 9 build and filed 8 issues. All fixed in one branch on `main`: +The creator reviewed the TASK 8 build and filed 8 issues. All fixed in one branch on `main`: 1. ✅ **Desktop/game audio volume slider had no effect** (headset OR stream) — the `AudioMixer`'s `micGain`/`loopbackGain` seams defaulted to unity because the VM never passed them; the sliders @@ -736,7 +736,7 @@ The creator reviewed the TASK 9 build and filed 8 issues. All fixed in one branc --- -## TASK 25 — Master limiter on the live mix (2026-08-15) +## TASK 14 — Master limiter on the live mix (2026-08-15) **Queued by the creator during the TRAX discussion:** "should the soundtrack be limited to avoid squandering resources / should it cap at 20% / how do the three sound events balance?" Review @@ -766,7 +766,7 @@ No changes to `MusicPlayer`, the 0.20 cap, the ducker, or the meter zones. Build --- -## TASK 26 — Social media launch kit +## TASK 15 — Social media launch kit **Goal:** marcom/social media assets and strategy for v1 launch. diff --git a/ViewModels/index.md b/ViewModels/index.md index ed1f5b6..62edeb5 100644 --- a/ViewModels/index.md +++ b/ViewModels/index.md @@ -4,9 +4,9 @@ MVVM layer. See [`schema.md`](../schema.md) for the memory-map conventions. | File | Purpose | |------|---------| -| `MainViewModel.cs` | The app brain: scenes/elements collections + commands, stream state (`IsLive`/`IsOffline`/`IsConnected`), chat feed, overlays, layout save/open, **resolution dropdown** (`QualityOptions`, `SelectedQuality`, `ResolutionHelp`, `ApplyStreamQuality`) that computes the output rect over the 1920×1080 master (`OutputRectX/Y/W/H`, `DimRects`, `IsOutputCropped`, `ResolutionBadgeText`). Auth: loads the saved DPAPI session at startup (`LoadSavedSessionAsync`), `SignInAsync` feeds the GoLive dialog; **go-live (ship step 7): `BeginGoLive` accepts the dialog → sets `StreamVisibility` (always "Private") → fires `CreateBroadcastAsync` (stores `_currentBroadcastId` for TASK 5's bind/transition; failure → `StreamStatus.Error`) + `_framePump.StartAsync()`**; **End Livestream signs out** (`StopStream` clears session + token + `_currentBroadcastId` — crash-safe). **Connected account:** top-bar avatar/name (`AccountAvatarUrl`/`AccountDisplayName` via `SyncConnectedAccount`, cleared on End). **Element rows:** scenes are pure selection rows (no per-row icons); source rows carry edit/visibility/trash (`EditElementCommand` sets `element.IsEditing`, `ToggleElementVisibilityCommand` flips `IsVisible`, `RemoveSourceCommand`; the Backdrop's trash slot stays **reserved** via `HiddenBoolToVisibilityConverter` so the edit/eye icons keep their fixed columns — TASK 24); duplicate resource names get a no-space incrementing suffix via shared `NextSourceName` (Image, Image2, Image3… — next free number derived from actual names, so deletions never collide; used by `AddSource` + `AddReusedImage`). **Audio (KISS — the mic is the creator's only audio control; capture SHIPPED, runs for the app's lifetime so the meters preview live):** the sound meter is a **READ-ONLY realtime level display**: fill = `Math.Min(1, AudioLevelMeter.ToDisplay((float)AudioLevel) * MicVolume)` (`MeterFillWidth`/`MeterBrush`, green→yellow→red, zone markers at 60%/80% — `ToDisplay` maps the raw linear RMS onto a −60..0 dBFS scale with +10 dB amplification so real speech/game levels occupy the bar and peaks hit red at maxed volume), where `AudioLevel` (live input, 0 with no input) is fed by the audio mixer once capture lands and `MicVolume` acts as a gain on ambient noise; while the slider is dragged the bar previews the slider position (`SetVolumeAdjusting`), returning to the live level on release (0 with no input — clicking the meter does nothing); `MicVolume` (default 0.8) + read-only `MicMuted`/`MicMuteText`/`ToggleMicMuteCommand` — **MicVolume drives MicMuted** (muted ⇔ volume 0): sliding to 0 flips the speaker to muted, sliding up from 0 clears it; muting stores the prior volume, unmuting restores it (default 0.8 if unknown) and flashes the meter to the restored position ~300ms (`BeginVolumeFlash`/`EndVolumeFlash`); `MicSourceName` = picked voice source, shown left-justified inside the meter bar (the fill runs at 75% opacity so the text + ruler markings show through); `OpenMicPickerCommand`/`PickMicrophone()` open the `MicPickerDialog` (a picked device takes effect immediately — `PickMicrophone` swaps the live source via `_audioMixer.RestartMic()`, loopback keeps running); the **MIC label is a button** (`OpenMicPickerCommand`) with a **status dot** (`MicStatus`, `Models/MicStatus`: green = `MicConnected` via the source's `Started` event, yellow = `MicFailed` — in use/unplugged, red = no mic device at startup; `MicStatusBrush`/`MicStatusToolTip`); capture starts once at startup (`StartMicCaptureAsync`) — NOT go-live (`BeginGoLive`/`StopStream` no longer touch the mixer) — zero devices = red dot + the mixer never starts. **Game audio bar** (desktop/game, **overlaid at the bottom of the preview window** — bottom-center chip, a mirror of the mic bar; **always visible** — the TASK 4 show/hide gating (`IGameAudioDetector`/`GameAudioDetector`/`GameAudioHysteresis` + the 250ms poll timer) was **removed 2026-08-15** because the bar kept hiding the desktop meter whenever no full-screen game with sound was up): `GameAudioLevel` (loopback meter via `LoopbackLevelChanged`, scaled by volume), `GameMuted`/`GameMuteText`/`ToggleGameMuteCommand`, `GameAudioVolume` (0..1 volume slider), `Begin/End/CancelGameVolumeFlash` (mirrors the mic bar's volume flash); the game speaker + slider share the mic bar's `PreviewMouseLeftButtonDown/Up` + `LostMouseCapture` code-behind pattern. **Webcam:** owns `CameraManager` (MediaCapture), `AddWebcamToActiveSceneAsync` (picker → default 480×270 bottom-right placement → acquire; the Windows camera picker ALWAYS opens so the creator chooses — never silently reuses the previous camera; picking a different camera swaps the app-wide identity via `SwapWebcamIdentityAsync`, same path as `ChangeWebcamAsync`; propagates the running shared bitmap via `CameraManager.GetPreviewBitmap` first, so a webcam added mid-session never renders a transparent container; greys out when the active scene already has a config via `CanAddWebcamToActiveScene`), `ChangeWebcamAsync` (device swap — `ReleaseAllAsync` old + re-acquire), `ShowWebcamInActiveScene` (reveal hidden config / empty-canvas right-click), `ReacquireWebcam` after layout load (re-propagates the shared bitmap to every config), `OnCameraPreviewBitmapChanged` forwards the shared bitmap into every `WebcamSceneConfig.VideoImageSource`; `SelectedElement` drives the preview overlay + `internal ClampWebcamToBounds(config, sceneName)` (per-scene size cap seam: 50%-per-dimension everywhere, half-screen-AREA in Chat via `MaxWebcamWidthFor`/`MaxWebcamHeightFor`). **Scenes (five-scene catalog):** empty DB seeds Starting/Live/BRB/Chat/Ending (`SceneCatalog.All`); `AddScene(name)` accepts only canonical, missing names; `MissingScenes`/`ShowAddScene` drive the "+" button (hidden once all five exist; its menu lists only the missing scenes via `AddSceneCommand`). **Screen backdrop (Live-only by policy):** owns `ScreenCaptureManager` + `ScreenCaptureSourceFactory` (WinRT GraphicsCapture), `internal static EnsureBackdrop` heals one per backdrop-enabled scene (gated on `Scene.HasBackdrop`), `ReacquireScreenCaptures`/`RefreshBackdropAutoCapture`/`NoteBackgroundWindow` key by full-screen game monitor or the **primary display** via `Win32FullScreenDetector` (`GetDisplays()`/`PrimaryMonitorIndex()`), `ChangeBackdropCaptureAsync` (OS picker) + `SetBackdropCapture(DisplayInfo)` (in-app "Capture Display"), `RedesignateBackdropAsync` re-targets all backdrops and releases orphaned sessions, `internal static EnforceBackdropPolicy` normalizes `Scene.HasBackdrop` by name on every load + strips lingering non-Live backdrops, `CanChangeBackdrop` gates the capture menu to Live (the only scene with a backdrop), `BackdropImage` hides the preview watermark (`ShowPreviewPlaceholder`). **REC sign (2026-08-13):** top-center indicator always visible — `RecDotBrush` (offline `#555555`, live `#e94560`, live-private `#8f1f1f`), `RecTextBrush` (dim offline, white live), `RecDotOpacity` (0.55 offline, pulsing 1.0/0.35 live via `_recDotPulse` flipped on the live tick), `IsLivePrivate` (`IsLive && StreamVisibility == "Private"`) — notified from the `StreamStatus` + `StreamVisibility` setters; `LiveIndicatorVisible`/`LivePulseOpacity` removed. **Mic mute icon:** a second 16px clickable glyph (mic, red + slash when muted) between the meter and the speaker on the mic bar — same `ToggleMicMuteCommand`. **TASK 24 (2026-08-15):** the gain seams (`AudioGainProvider` → `micGain`/`loopbackGain`) reach the mixer at construction so volume sliders + mutes are stream-honest; the game bar also scales TRAX locally (`MusicPlayer.LocalGain`); `MicSourceName` persists via `LayoutStore`'s `Settings` table + restores before first `Start`; TRAX moved into the game audio bar (left of "Desktop Audio") in the preview overlay. **Health stats (TASK 4 ship step 6, 2026-08-13):** `OnFramePumpHealthUpdated` marshals `FramePump.HealthUpdated` (encoder's parsed bitrate/FPS/dropped/duration — raised on the stderr thread) onto the UI thread into `CurrentHealth` (bottom bar bindings); `ResetHealth(status)` zeroes dropped/duration on go-live/End so stats never linger | +| `MainViewModel.cs` | The app brain: scenes/elements collections + commands, stream state (`IsLive`/`IsOffline`/`IsConnected`), chat feed, overlays, layout save/open, **resolution dropdown** (`QualityOptions`, `SelectedQuality`, `ResolutionHelp`, `ApplyStreamQuality`) that computes the output rect over the 1920×1080 master (`OutputRectX/Y/W/H`, `DimRects`, `IsOutputCropped`, `ResolutionBadgeText`). Auth: loads the saved DPAPI session at startup (`LoadSavedSessionAsync`), `SignInAsync` feeds the GoLive dialog; **go-live (ship step 7): `BeginGoLive` accepts the dialog → sets `StreamVisibility` (always "Private") → fires `CreateBroadcastAsync` (stores `_currentBroadcastId` for TASK 9's bind/transition; failure → `StreamStatus.Error`) + `_framePump.StartAsync()`**; **End Livestream signs out** (`StopStream` clears session + token + `_currentBroadcastId` — crash-safe). **Connected account:** top-bar avatar/name (`AccountAvatarUrl`/`AccountDisplayName` via `SyncConnectedAccount`, cleared on End). **Element rows:** scenes are pure selection rows (no per-row icons); source rows carry edit/visibility/trash (`EditElementCommand` sets `element.IsEditing`, `ToggleElementVisibilityCommand` flips `IsVisible`, `RemoveSourceCommand`; the Backdrop's trash slot stays **reserved** via `HiddenBoolToVisibilityConverter` so the edit/eye icons keep their fixed columns — TASK 13); duplicate resource names get a no-space incrementing suffix via shared `NextSourceName` (Image, Image2, Image3… — next free number derived from actual names, so deletions never collide; used by `AddSource` + `AddReusedImage`). **Audio (KISS — the mic is the creator's only audio control; capture SHIPPED, runs for the app's lifetime so the meters preview live):** the sound meter is a **READ-ONLY realtime level display**: fill = `Math.Min(1, AudioLevelMeter.ToDisplay((float)AudioLevel) * MicVolume)` (`MeterFillWidth`/`MeterBrush`, green→yellow→red, zone markers at 60%/80% — `ToDisplay` maps the raw linear RMS onto a −60..0 dBFS scale with +10 dB amplification so real speech/game levels occupy the bar and peaks hit red at maxed volume), where `AudioLevel` (live input, 0 with no input) is fed by the audio mixer once capture lands and `MicVolume` acts as a gain on ambient noise; while the slider is dragged the bar previews the slider position (`SetVolumeAdjusting`), returning to the live level on release (0 with no input — clicking the meter does nothing); `MicVolume` (default 0.8) + read-only `MicMuted`/`MicMuteText`/`ToggleMicMuteCommand` — **MicVolume drives MicMuted** (muted ⇔ volume 0): sliding to 0 flips the speaker to muted, sliding up from 0 clears it; muting stores the prior volume, unmuting restores it (default 0.8 if unknown) and flashes the meter to the restored position ~300ms (`BeginVolumeFlash`/`EndVolumeFlash`); `MicSourceName` = picked voice source, shown left-justified inside the meter bar (the fill runs at 75% opacity so the text + ruler markings show through); `OpenMicPickerCommand`/`PickMicrophone()` open the `MicPickerDialog` (a picked device takes effect immediately — `PickMicrophone` swaps the live source via `_audioMixer.RestartMic()`, loopback keeps running); the **MIC label is a button** (`OpenMicPickerCommand`) with a **status dot** (`MicStatus`, `Models/MicStatus`: green = `MicConnected` via the source's `Started` event, yellow = `MicFailed` — in use/unplugged, red = no mic device at startup; `MicStatusBrush`/`MicStatusToolTip`); capture starts once at startup (`StartMicCaptureAsync`) — NOT go-live (`BeginGoLive`/`StopStream` no longer touch the mixer) — zero devices = red dot + the mixer never starts. **Game audio bar** (desktop/game, **overlaid at the bottom of the preview window** — bottom-center chip, a mirror of the mic bar; **always visible** — the TASK 4 show/hide gating (`IGameAudioDetector`/`GameAudioDetector`/`GameAudioHysteresis` + the 250ms poll timer) was **removed 2026-08-15** because the bar kept hiding the desktop meter whenever no full-screen game with sound was up): `GameAudioLevel` (loopback meter via `LoopbackLevelChanged`, scaled by volume), `GameMuted`/`GameMuteText`/`ToggleGameMuteCommand`, `GameAudioVolume` (0..1 volume slider), `Begin/End/CancelGameVolumeFlash` (mirrors the mic bar's volume flash); the game speaker + slider share the mic bar's `PreviewMouseLeftButtonDown/Up` + `LostMouseCapture` code-behind pattern. **Webcam:** owns `CameraManager` (MediaCapture), `AddWebcamToActiveSceneAsync` (picker → default 480×270 bottom-right placement → acquire; the Windows camera picker ALWAYS opens so the creator chooses — never silently reuses the previous camera; picking a different camera swaps the app-wide identity via `SwapWebcamIdentityAsync`, same path as `ChangeWebcamAsync`; propagates the running shared bitmap via `CameraManager.GetPreviewBitmap` first, so a webcam added mid-session never renders a transparent container; greys out when the active scene already has a config via `CanAddWebcamToActiveScene`), `ChangeWebcamAsync` (device swap — `ReleaseAllAsync` old + re-acquire), `ShowWebcamInActiveScene` (reveal hidden config / empty-canvas right-click), `ReacquireWebcam` after layout load (re-propagates the shared bitmap to every config), `OnCameraPreviewBitmapChanged` forwards the shared bitmap into every `WebcamSceneConfig.VideoImageSource`; `SelectedElement` drives the preview overlay + `internal ClampWebcamToBounds(config, sceneName)` (per-scene size cap seam: 50%-per-dimension everywhere, half-screen-AREA in Chat via `MaxWebcamWidthFor`/`MaxWebcamHeightFor`). **Scenes (five-scene catalog):** empty DB seeds Starting/Live/BRB/Chat/Ending (`SceneCatalog.All`); `AddScene(name)` accepts only canonical, missing names; `MissingScenes`/`ShowAddScene` drive the "+" button (hidden once all five exist; its menu lists only the missing scenes via `AddSceneCommand`). **Screen backdrop (Live-only by policy):** owns `ScreenCaptureManager` + `ScreenCaptureSourceFactory` (WinRT GraphicsCapture), `internal static EnsureBackdrop` heals one per backdrop-enabled scene (gated on `Scene.HasBackdrop`), `ReacquireScreenCaptures`/`RefreshBackdropAutoCapture`/`NoteBackgroundWindow` key by full-screen game monitor or the **primary display** via `Win32FullScreenDetector` (`GetDisplays()`/`PrimaryMonitorIndex()`), `ChangeBackdropCaptureAsync` (OS picker) + `SetBackdropCapture(DisplayInfo)` (in-app "Capture Display"), `RedesignateBackdropAsync` re-targets all backdrops and releases orphaned sessions, `internal static EnforceBackdropPolicy` normalizes `Scene.HasBackdrop` by name on every load + strips lingering non-Live backdrops, `CanChangeBackdrop` gates the capture menu to Live (the only scene with a backdrop), `BackdropImage` hides the preview watermark (`ShowPreviewPlaceholder`). **REC sign (2026-08-13):** top-center indicator always visible — `RecDotBrush` (offline `#555555`, live `#e94560`, live-private `#8f1f1f`), `RecTextBrush` (dim offline, white live), `RecDotOpacity` (0.55 offline, pulsing 1.0/0.35 live via `_recDotPulse` flipped on the live tick), `IsLivePrivate` (`IsLive && StreamVisibility == "Private"`) — notified from the `StreamStatus` + `StreamVisibility` setters; `LiveIndicatorVisible`/`LivePulseOpacity` removed. **Mic mute icon:** a second 16px clickable glyph (mic, red + slash when muted) between the meter and the speaker on the mic bar — same `ToggleMicMuteCommand`. **TASK 24 (2026-08-15):** the gain seams (`AudioGainProvider` → `micGain`/`loopbackGain`) reach the mixer at construction so volume sliders + mutes are stream-honest; the game bar also scales TRAX locally (`MusicPlayer.LocalGain`); `MicSourceName` persists via `LayoutStore`'s `Settings` table + restores before first `Start`; TRAX moved into the game audio bar (left of "Desktop Audio") in the preview overlay. **Health stats (TASK 4 ship step 6, 2026-08-13):** `OnFramePumpHealthUpdated` marshals `FramePump.HealthUpdated` (encoder's parsed bitrate/FPS/dropped/duration — raised on the stderr thread) onto the UI thread into `CurrentHealth` (bottom bar bindings); `ResetHealth(status)` zeroes dropped/duration on go-live/End so stats never linger | | `GoLiveViewModel.cs` | Start Stream dialog: **account row** (saved channel shown with Change Account, or Sign in to YouTube; Start gated on `IsSignedIn`/`IsBusy`) + title/description + **locked Visibility (always "Private" — ship step 7, no dropdown)**, start/cancel requests | -| **About hub (TASK 21, in `MainViewModel`)** | the About overlay is the in-app creator hub: `ChannelUrl`/`MastodonUrl`/`CoffeeUrl` (public consts, live) + `PremiumUrl` (tabled billing seam — TASK 23 → the greyed "Unlock Premium" button lights up when filled); `OpenLicensesCommand` → `ShowLicenses()` loads the shipped `THIRD-PARTY-NOTICES.txt` into `LicensesText` and sets `IsLicensesOpen` (in-app scrolling panel, never the OS viewer); `BackToAbout_Click` in `MainWindow.xaml.cs` flips it off. | +| **About hub (TASK 11, in `MainViewModel`)** | the About overlay is the in-app creator hub: `ChannelUrl`/`MastodonUrl`/`CoffeeUrl` (public consts, live) + `PremiumUrl` (tabled billing seam — TASK 12 → the greyed "Unlock Premium" button lights up when filled); `OpenLicensesCommand` → `ShowLicenses()` loads the shipped `THIRD-PARTY-NOTICES.txt` into `LicensesText` and sets `IsLicensesOpen` (in-app scrolling panel, never the OS viewer); `BackToAbout_Click` in `MainWindow.xaml.cs` flips it off. | | `ReuseImageViewModel.cs` | Add Image dialog: candidate list (`ReuseImageCandidate`), reuse/new/cancel | | `CameraPickerViewModel.cs` | Add Webcam dialog: async camera list (loading / has / none states), `UseRequested(CameraDeviceInfo)`/`CancelRequested` | | `MicPickerViewModel.cs` | Select Microphone dialog (voice source): async mic list (loading / has / none states), `UseRequested(MicrophoneDeviceInfo)`/`CancelRequested` | diff --git a/ai.md b/ai.md index 19f2f00..987f746 100644 --- a/ai.md +++ b/ai.md @@ -112,7 +112,7 @@ C# / WPF (.NET 8) following MVVM: ### Current limitations / TODOs - `Helpers/OAuthCredentials.cs` contains the real ClientId/ClientSecret. Auth is complete and the session **persists via Windows DPAPI** (`Helpers/TokenStore.cs` → `%APPDATA%\ytLlive\ytLlive.auth`, CurrentUser scope), reloaded best-effort at startup with a proactive refresh of a near-expiry access token. Sign-in/Change Account lives **inside the Start Stream dialog** (two-state flow — no separate Connect button). A **graceful End Livestream signs out**: `StopStream()` clears the session + token, so the next go-live needs a fresh sign-in; a crash never runs End, so the token survives and the creator stays signed in. `YouTubeAuthService` takes an optional `HttpClient` + `sessionChanged` callback (test seam + save hook; services are still constructed in `MainViewModel`) -- Scene/source/asset layout + the social bar persist (SQLite, schema v8); the OAuth session persists (DPAPI); the paid-unlock state does not (yet — subscription entitlement verification pending, TASK 23) +- Scene/source/asset layout + the social bar persist (SQLite, schema v8); the OAuth session persists (DPAPI); the paid-unlock state does not (yet — subscription entitlement verification pending, TASK 12) - `YouTubeStreamService` manages the **variable reusable stream** (shipped 2026-08-16): `GetOrCreateReusableStreamAsync` lists `liveStreams?mine=true` and reuses the existing `cdn.isReusable` stream, creating it only on first use (`resolution=variable`, `frameRate=variable`); the stream is cached via `LayoutStore` (`SaveReusableStream`/`LoadReusableStream`, Settings table) and bound at broadcast insert (`contentDetails.boundStreamId`). Health (shipped 2026-08-16): `GetStreamHealthAsync(streamId)` polls `liveStreams?part=status` for `healthStatus` + `configurationIssues[]` → `StreamHealth`; banner decision in pure `StreamHealthReporter` - Webcam capture is shipped (milestone 1); the live desktop/game backdrop is shipped (ship task #1); **the output compositor (TASK 4 ship step 1) is SHIPPED**, **the FFmpeg locator (TASK 4 ship step 2) is SHIPPED**, **the encoder + RTMP push (TASK 4 ship step 3) is SHIPPED**, **WASAPI audio capture (TASK 4 ship step 4) is SHIPPED** — full plan in `TASKS.md`; the frame-pipeline wiring follows (its own PR) - `StreamConfig` defaults (`TargetBitrate=6000`, `Resolution="1920x1080"`) are stale — the live dropdown drives `StreamHealth.CurrentBitrate`/`FPS` instead @@ -356,7 +356,7 @@ integration test fakes the whole subprocess (probe + encoder) with a Channel-bac `-re -f rawvideo -pix_fmt bgra -video_size WxH -framerate FPS -i pipe:0` + a **real audio input — the mixer writes IEEE-float stereo to a Windows named pipe** (`-f f32le -ar 48000 -ac 2 -i \\.\pipe\ytllive_audio`, name via `EncoderOptions.AudioPipeName`; replaced the old `-f lavfi -i -anullsrc` silence in the TASK 9 audio milestone) + explicit `-map 0:v -map 1:a` + `-c:v -b:v K +anullsrc` silence in the TASK 8 audio milestone) + explicit `-map 0:v -map 1:a` + `-c:v -b:v K -maxrate K -bufsize 2K` + **`-g fps×4 -keyint_min fps×4 -sc_threshold 0 -bf 0 -pix_fmt yuv420p`** (≤4s keyframes, closed GOP, H.264 compliance) + `-c:a aac -ar 48000 -ac 2 -f flv `. **Encoder choice is probed from the binary's `-encoders` listing** (`FfmpegEncoderPicker`, pure): @@ -365,7 +365,7 @@ forces one and skips the probe. `EncoderOptions` also carries W×H/FPS/bitrate f `GopSize` = FPS×4. Constructed by `MainViewModel` since ship step 5 (`new FfmpegEncoder(new FfmpegLocator())`), driven by the `FramePump` below. -### Live audio capture (TASK 4 ship step 4 — shipped 2026-08-12; game audio bar 2026-08-13; **TASK 9 audio milestone: real stream audio + filters + duck + TRAX, shipped 2026-08-14**, plan in TASKS.md) +### Live audio capture (TASK 4 ship step 4 — shipped 2026-08-12; game audio bar 2026-08-13; **TASK 8 audio milestone: real stream audio + filters + duck + TRAX, shipped 2026-08-14**, plan in TASKS.md) **Capture runs for the app's lifetime and is KISS by rule**: desktop/game audio is automatic (WASAPI loopback), the mic is the creator's only audio control (meter/mute/volume already shipped). The whole @@ -397,7 +397,7 @@ devices, no timers). `Failed`), and `RestartMic()` re-resolves + restarts just the mic (loopback keeps running). Failures log via `AppLog`; a mic failure zeroes the meter, a loopback failure never kills the mic. Meter `Push` is unconditional (a `?.` on the event would skip the meter update when nothing is subscribed yet). -- **Go-live audio (TASK 9):** `BeginGoLive` → `StartLive(pipeName)` (idempotent) creates the named pipe +- **Go-live audio (TASK 8):** `BeginGoLive` → `StartLive(pipeName)` (idempotent) creates the named pipe server (`NamedPipeAudioWriter`, behind the `IAudioPipeWriter` seam) and a live loop task; every `mixInterval` (default 10 ms) `FillAndMix` reads a chunk from each ring buffer (silence-fills underruns), computes the post-filter mic RMS → `AutoDucker` (threshold 0.02, duck ×0.25 / −12 dB, @@ -412,7 +412,7 @@ devices, no timers). `StopLive()` (closes the pipe → ffmpeg audio EOF) **BEFORE** stopping the frame pump (video EOF) — the reverse order stalls on pipe backpressure. `FfmpegEncoder` itself is untouched; the VM owns the pipe lifecycle. -- **TRAX — free background music (TASK 9):** `MusicPlayer` = NAudio `MediaFoundationReader` +- **TRAX — free background music (TASK 8):** `MusicPlayer` = NAudio `MediaFoundationReader` (mp3/wav/m4a) → `VolumeWaveProvider16` at the hardcoded **0.20** bed (no slider) → `WaveOutEvent` on the default device, **looping on any clean natural end** (`PlaybackStopped` with `e.Exception == null` → rewind + replay; the earlier `Position >= Length` check was unreliable for `MediaFoundationReader` @@ -440,12 +440,12 @@ devices, no timers). chip, a mirror of the mic bar: meter + mute + volume slider). It's monitoring UI **in the preview grid** — the meter is display-only, but its **volume slider + mute are stream-honest** (via `AudioGainProvider.LoopbackGain`) **and drive TRAX locally** (`MusicPlayer.LocalGain`). Relabelled - **"Desktop Audio"** (TASK 9) since TRAX rides the same channel. + **"Desktop Audio"** (TASK 8) since TRAX rides the same channel. - **`WaveToFloat`** (pure, shared): WASAPI mix formats → interleaved float — IEEE float 32-bit direct, PCM 16-bit normalized to -1..1, `WaveFormatExtensible` with the IEEE-float subformat GUID (`NAudio.Dmo.AudioMediaSubtypes.MEDIASUBTYPE_IEEE_FLOAT`), trailing partial samples ignored. - Build **0 warnings**; **197 passing** (DSP/ring-buffer/ducker/resampler/pipe + mixer/hysteresis/ - game-detector/meter-scale unit tests + the TASK 9 integration test + game-detector/meter-scale unit tests + the TASK 8 integration test `AudioPipelineTests.Mix_WithFiltersDuckAndGain_Lands_On_AudioPipe` + the polish-batch integration test `Mix_HonorsProviderGains_AndGameMute_KillsTheLoopback` proving the provider gains reach the pipe and mute silences the loopback). @@ -462,7 +462,7 @@ seam:** `Func`, `Func` resolver, `Func` that now yields - the reusable stream's ingest URL (TASK 5, shipped 2026-08-16): loaded from the `LayoutStore` cache at + the reusable stream's ingest URL (TASK 9, shipped 2026-08-16): loaded from the `LayoutStore` cache at startup and set fresh by `PrepareAndStartLiveAsync` before the pump starts. **The pump reads the URL once at startup**, which is why go-live ensures the stream BEFORE `StartAsync`. - **Stop ordering matters:** `StopAsync` stops the encoder (closes stdin → EOF → ffmpeg finalizes+exits) @@ -474,7 +474,7 @@ seam:** `Func`, `Func` resolver, `Func`, `Func` resolver, `Func