diff --git a/HANDOFF.md b/HANDOFF.md index 5daa4eb..6697972 100644 --- a/HANDOFF.md +++ b/HANDOFF.md @@ -5,7 +5,7 @@ > a problem. Conventions: [`schema.md`](schema.md). Rewrite this file at session > end, compaction, or any interruption. -## Session state (last updated: 2026-08-16, renumbering pass) +## Session state (last updated: 2026-08-16, renumbering finalised) - **Branch:** `main`, tracking `origin/main`. Working tree: the **TASK 9 reusable-stream slice** is committed + pushed (details below). Local branches `social-bar`/`webcam-validation` untouched (no secrets). @@ -35,15 +35,15 @@ - **TASK 9 items 6-7 scoped (not built):** visibility unlock (remove temporary "always Private" enforcement) and full broadcast form (Core + Advanced tabs with all YouTube API-supported fields). These are the next technical tasks after live chat (item 4) and error handling (item 5). -- **TASK 12 trimmed** to technical scope only (billing decision, unlock mechanism, bug report, alerts gating). +- **TASK 10 trimmed** to technical scope only (billing decision, unlock mechanism, bug report, alerts gating). Business details moved to `MONETIZATION.md`. -- **TASK 15 added** — marcom/launch kit. Business details moved to `MARCOM.md`. Queued after all v1 features. +- **TASK 13 added** — marcom/launch kit. Business details moved to `MARCOM.md`. Queued after all v1 features. - **Backlog updated** — removed v0.3 (stream scheduling, not needed for casual streamers). Multi-destination restreaming noted as "congrats, you're ready for OBS" moment. - **Resume point (next branch):** **TASK 9 item 4 — live chat** (`liveChat/messages` poll, right-panel render, Super Chat + membership badges). Then item 5 (error handling), item 6 (visibility unlock), - item 7 (full broadcast form), then TASK 12 (billing + unlock + support). -- **TASK 14 — MASTER LIMITER — COMMITTED + PUSHED 2026-08-15.** Queued from the TRAX discussion: the + item 7 (full broadcast form), then TASK 10 (billing + unlock + support). +- **TASK 12 — MASTER LIMITER — COMMITTED + PUSHED 2026-08-15.** Queued from the TRAX discussion: the live mix summed mic + loopback with no ceiling, so hot gains could pass 0 dBFS and clip the AAC encode. New pure `Services/Audio/MasterLimiter.cs` (−1 dBFS ceiling, instant attack per frame, smoothed release) applied at the end of `AudioMixer.FillAndMix`. ONE integration test @@ -56,7 +56,7 @@ TRAX music played). The whole `IGameAudioDetector`/`GameAudioDetector`/`GameAudioHysteresis` stack + the VM's 250ms poll timer + its two test files were **deleted**; the bar is now permanently overlaid at the bottom of the preview. Build **0 warnings**, full suite green. -- **TASK 13 — POST-PAUSE POLISH BATCH — SHIPPED + COMMITTED + PUSHED 2026-08-15.** All 8 creator +- **TASK 11 — POST-PAUSE POLISH BATCH — SHIPPED + COMMITTED + PUSHED 2026-08-15.** All 8 creator review issues fixed in one branch (details below). Build **0 warnings**, full suite **197 passing** (ONE integration test for that branch: `AudioPipelineTests.Mix_HonorsProviderGains_AndGameMute_KillsTheLoopback`). - **AUDIO MILESTONE (TASK 8) — SHIPPED + COMMITTED + PUSHED.** Real stream audio + voice filters + @@ -67,12 +67,12 @@ lapse → list on renewal. Free tier unchanged (branding flash = the billboard + no Alerts; Alerts is the paid feature). **Billing NOT itch-locked**; candidates Gumroad (native affiliates = tiebreaker) / Lemon Squeezy. Support = in-app bug-report → git issues. Full policy in `ai.md` → - Monetization; scoped plan in `TASKS.md` **TASK 12**. + Monetization; scoped plan in `TASKS.md` **TASK 10**. - **Secrets scrubbed from git history.** The DO token + passwords were purged via `git filter-branch` + `git gc --prune=now --aggressive` (all-refs scan = 0 hits); values still exist in chat — keep treating as **compromised**; rotate the DO API token. -- **Shipped, all pushed:** TASK 9 items 1–3 (reusable stream + health banner, 2026-08-16), TASK 14 - master limiter, TASK 13 polish batch, TASK 11 (creator-hub About, `3d92bd0`), TASK 7 (meter +10 dB), +- **Shipped, all pushed:** TASK 9 items 1–3 (reusable stream + health banner, 2026-08-16), TASK 12 + master limiter, TASK 11 polish batch, creator-hub About (`3d92bd0`), TASK 7 (meter +10 dB), TASK 4 ship step 7 (one-click go-live + private-only), TASK 8 audio milestone (`6d71ace`), secrets cleanup (`f2f6401`), monetization docs (`86fcd86`). @@ -129,9 +129,9 @@ mappings, and the design's bottom-strip YouTube logo/green-red dot (clickable + the ONE integration test `GetStreamHealthAsync_Report_By_Exception_Banner_Only_On_Warning_Or_Error` (real service JSON parse → real reporter: good → no banner, error issue → banner text + error color). -## TASK 13 — the 8-issue polish batch — SHIPPED 2026-08-15 (what changed) +## TASK 11 — the 8-issue polish batch — SHIPPED 2026-08-15 (what changed) -The creator's review of the TASK 8 build, all fixed in one branch (full record in `TASKS.md` TASK 13). +The creator's review of the TASK 8 build, all fixed in one branch (full record in `TASKS.md` TASK 11). 1. ✅ **Desktop/game audio volume slider had no effect** — the `AudioMixer` gain seams defaulted to unity (the VM never passed them): the slider/mute were decorative, stream AND local. Fixed with a @@ -175,7 +175,7 @@ sound or TRAX played, so it sat hidden during normal use. Fix = the bar is now * - **MainWindow.xaml:** dropped the `Visibility` binding on the preview-overlay game bar; it renders unconditionally (TRAX button + "Desktop Audio" label + meter + mute + volume unchanged). -## TASK 14 — master limiter 2026-08-15 (what changed) +## TASK 12 — master limiter 2026-08-15 (what changed) Came out of the TRAX discussion (file-size guard? 20% cap? sound-event balance?). Verdict: two of the three instincts were already satisfied, one real gap existed: @@ -207,7 +207,7 @@ and `ai.md` ("Live audio capture"). Highlights: before `_framePump.StopAsync()`. - **2-input mix (mic + loopback)**, honest gains: `micGain = MicVolume` (mute = 0), `loopbackGain = GameMuted ? 0 : GameAudioVolume` × duck. No third music channel. -- **Voice chain on the mic** (TASK 10), before meter AND mix: bass 120 Hz +4 dB → treble 8 kHz +3 dB → +- **Voice chain on the mic** (TASK 8), before meter AND mix: bass 120 Hz +4 dB → treble 8 kHz +3 dB → gate (0.005 / hysteresis 0.5) → compressor (0.5, 4:1). Pure TDF2 DSP, per-sample, always on. - **Auto-duck:** mic RMS > 0.02 → loopback ×0.25 (−12 dB), attack 0.05 / release 0.005. - **TRAX (free BGM):** `MusicPlayer` = MediaFoundationReader → `VolumeWaveProvider16` at fixed **0.20** → @@ -224,7 +224,7 @@ and `ai.md` ("Live audio capture"). Highlights: start-of-stream silence race was seen and eliminated). **Out of scope this branch:** IP webcam, chat box, alt-key crop, credits, bg removal, music-off-VOD -track (YouTube mutes VODs with copyrighted music — future feature), `PremiumUrl` (TASK 12 seam). +track (YouTube mutes VODs with copyrighted music — future feature), `PremiumUrl` (TASK 10 seam). - **Landmines:** - Never add another test that constructs `new App()` — use `RealAppHost.Run(...)` (shared STA host diff --git a/TASKS.md b/TASKS.md index 2367d01..9ecb35a 100644 --- a/TASKS.md +++ b/TASKS.md @@ -5,8 +5,9 @@ > **Checklist markers** — every task's Status list uses the same states: > 1. ✅ — completed (green check) -> 2. ☐ — not completed / pending (empty box) -> 3. ❌ — exception (blocked, known-issue, or deliberately excluded from this build) +> 2. 🔶 — in progress (amber diamond) +> 3. ☐ — not completed / pending (empty box) +> 4. ❌ — exception (blocked, known-issue, or deliberately excluded from this build) ## YouTube Live API — research facts (authoritative, v3 build) @@ -53,6 +54,8 @@ Lifecycle: `created → ready → [testing] → live → complete` (transitional --- +--- + ## TASK 2 — YouTube OAuth2 Authentication **Goal:** Fully working Google OAuth2 flow — user clicks "YouTube", browser opens, authorization callback lands, channel info is stored. @@ -89,6 +92,8 @@ Lifecycle: `created → ready → [testing] → live → complete` (transitional --- +--- + ## TASK 3 — Capture Pipeline (Scenes/Sources) **Goal:** Real video preview in the center panel — the minimal source set below, composited per scene. @@ -115,7 +120,7 @@ Lifecycle: `created → ready → [testing] → live → complete` (transitional 18. ☐ **Chat box** — YouTube live chat rendered *on* the stream so viewers read along in-video 19. ❌ **Background removal (milestone 2)** — ONNX Runtime + DirectML, MediaPipe Selfie Segmentation — deliberately NOT in this build 20. ☐ **Alerts** — Super Chat / membership / subscribe pop-ins; build after the six; **the one paid feature** (see Monetization in `ai.md`) -21. ✅ **Logo + richer in-app About (2026-08-13, queued → 2026-08-14 SHIPPED)** — the ytLlive wordmark in the top bar opens the About overlay (already wired); the About overlay is now the **creator hub**: the real logo (the "llama fortnite superman logo" from the creator's vault, copied to `Assets/llama-logo.png` — the creator's own art, no third-party license), plus **creator-hub links** — llama chile shop on YouTube (`MainViewModel.ChannelUrl`), Mastodon (`https://mastodon.llamachile.tube/@gramps`), **Buy me a coffee** (`https://buymeacoffee.com/llamachiley` — live), and **Unlock Premium** (greyed "coming soon" — the **billing** product URL is a tabled seam, `PremiumUrl`, until TASK 12 picks the subscription provider). A **Licenses & legal** button flips the About overlay to an in-app scrolling panel that loads the full `THIRD-PARTY-NOTICES.txt` text (`MainViewModel.ShowLicenses` reads the shipped file from the executable directory on first open; graceful "not found" fallback — **never the OS viewer, everything stays in-app**); "← Back to About" returns. The About overlay is also the in-app home of the notices — the top-bar About button that opened the file in the OS viewer was removed on 2026-08-13 for exactly this reason. Integration test (Good Dog Rule — ONE): `AboutHubTests.About_Opens_InApp_Licensing_Loads_Shipped_Notices` drives the real window + VM, asserting the hub opens, the link URLs are real, the licensing panel loads the shipped notices text (contains "Third-Party Notices" + "LGPL"), and back returns to the hub. 174 tests passing, 0 warnings +21. ✅ **Logo + richer in-app About (2026-08-13, queued → 2026-08-14 SHIPPED)** — the ytLlive wordmark in the top bar opens the About overlay (already wired); the About overlay is now the **creator hub**: the real logo (the "llama fortnite superman logo" from the creator's vault, copied to `Assets/llama-logo.png` — the creator's own art, no third-party license), plus **creator-hub links** — llama chile shop on YouTube (`MainViewModel.ChannelUrl`), Mastodon (`https://mastodon.llamachile.tube/@gramps`), **Buy me a coffee** (`https://buymeacoffee.com/llamachiley` — live), and **Unlock Premium** (greyed "coming soon" — the **billing** product URL is a tabled seam, `PremiumUrl`, until TASK 10 picks the subscription provider). A **Licenses & legal** button flips the About overlay to an in-app scrolling panel that loads the full `THIRD-PARTY-NOTICES.txt` text (`MainViewModel.ShowLicenses` reads the shipped file from the executable directory on first open; graceful "not found" fallback — **never the OS viewer, everything stays in-app**); "← Back to About" returns. The About overlay is also the in-app home of the notices — the top-bar About button that opened the file in the OS viewer was removed on 2026-08-13 for exactly this reason. Integration test (Good Dog Rule — ONE): `AboutHubTests.About_Opens_InApp_Licensing_Loads_Shipped_Notices` drives the real window + VM, asserting the hub opens, the link URLs are real, the licensing panel loads the shipped notices text (contains "Third-Party Notices" + "LGPL"), and back returns to the hub. 174 tests passing, 0 warnings 22. ✅ **Voice filters on the mic channel (2026-08-13 queued → SHIPPED 2026-08-14 inside TASK 9, the audio milestone)** — the standard four applied to the sound input path (before the meter/encoder mix): **bass boost, treble, noise suppression, compressor** (set decided with the creator 2026-08-13). Noise suppression = a **pure-C# noise gate** (creator chose over RNNoise / a second ffmpeg `afftdn` pipe, 2026-08-14 — KISS). Always-on — no UI knobs; the mic stays the creator's single audio control ### The Minimal Source Set (design decision — do not expand casually) @@ -193,6 +198,8 @@ Preview shows the transition too (WYSIWYG). No wipes/slides/LUTs beyond the four --- +--- + ## TASK 4 — RTMP Ingest to YouTube **Goal:** Push encoded video to YouTube's RTMP ingest. @@ -526,88 +533,6 @@ the validator → persisted), compositor bar overlay (top/bottom + above-flash), --- -## TASK 8 — Audio milestone: real stream audio + voice filters + TRAX music (2026-08-14) - -**Goal:** every audio issue done and tested in one branch — real mic/game audio reaches the encoder (replacing the `anullsrc` silence), `MicVolume`/`GameAudioVolume`/mute become real pre-AAC gains, TASK 10's voice filters land, auto-duck keeps the creator's voice over game + music, and a free background-music source ("TRAX") plays into the sound bar and the stream. - -### Status: ✅ SHIPPED 2026-08-14 — 196 tests passing, 0 warnings (ONE integration test: `AudioPipelineTests.Mix_WithFiltersDuckAndGain_Lands_On_AudioPipe`) - -1. ✅ **Real audio into the encoder** — the mixer's loopback samples were **dropped today** (`AudioMixer.OnLoopbackSample` metered only) and `MicVolume` was meter-only; `FfmpegArgs.cs:28` ran `-f lavfi -i anullsrc` (silence). Transport: **ffmpeg reads a Windows named pipe** — `-f f32le -ar 48000 -ac 2 -i \\.\pipe\ytllive_audio` replaces the anullsrc block, plus explicit `-map 0:v -map 1:a`; pipe name via `EncoderOptions.AudioPipeName` (`DefaultAudioPipeName = "ytllive_audio"`). Mixer stays **two inputs (mic + loopback)** — no N-source abstraction, no `StereoMixer` class. The VM owns the pipe lifecycle (`BeginGoLive` → `_audioMixer.StartLive(pipeName)`, `StopStream` → `_audioMixer.StopLive()` before the pump stops); `FfmpegEncoder.cs` is untouched. -2. ✅ **Honest gains reach the stream** — `MicVolume` scales the mic channel; `GameAudioVolume` + mute scale the loopback channel (the "mute in preview but stream still plays" footgun dies with it). One knob per input = KISS. Implemented as `Func` gain seams on `AudioMixer` (`micGain`/`loopbackGain`), wired by `MainViewModel`. -3. ✅ **Voice filters (TASK 10, before the meter AND the mix)** — `VoiceFilterChain`: bass (`LowShelfFilter` 120 Hz +4 dB) → treble (`HighShelfFilter` 8 kHz +3 dB) → **noise gate** (pure C#, creator's choice) → compressor (threshold 0.5, 4:1). Pure stateful DSP (TDF2), sine-in unit tests. Meter stays post-filter. -4. ✅ **Auto-duck** — pure envelope (`AutoDucker`): mic RMS > 0.02 → game+music dip ×0.25 (~12 dB), attack 0.05 / release 0.005, recover on release. Always-on, no knobs. Strong form of the creator's "voice always over the game volume" idea. -5. ✅ **TRAX — background music, FREE** (keeps the one paid line = Alerts + flash removal). `MusicPlayer`: NAudio `MediaFoundationReader` (mp3/wav/m4a) → `VolumeWaveProvider16` at the hardcoded **0.20** (fixed, not changeable) → `WaveOutEvent` on the default device (added the sibling `NAudio.WinMM` 2.2.1 package — `WaveOutEvent` isn't in `NAudio.Wasapi`), loop on end. **No 3rd mixer input and no `MusicVolume`** — music plays on the desktop, the existing loopback picks it up: heard in headphones, the sound-bar meter bounces, and the stream carries it through the loopback channel (ducked with game when the voice is hot). Known wrinkle (out of scope): YouTube mutes VODs carrying copyrighted music — future "music on live, off VOD". -6. ✅ **TRAX footer control (final spec, agreed 2026-08-14)** — `YtButtonSecondary`, **status dot + "TRAX"** text, in the center footer stack beside MIC: - - **Status dot** (same 8px Ellipse pattern as MIC/Socials): **red** no track loaded · **yellow** loaded not playing · **green** playing. - - **Left-click** → toggle play/pause; no track loaded → opens the picker instead. - - **Right-click** → always opens the in-app track picker (`TraxButton_PreviewMouseRightButtonUp`, code-behind pattern like `GameSpeaker_MouseLeftButtonUp`, `e.Handled = true` so no context menu). - - **Tooltip** → playing/loaded track name; "No track — right-click to choose background music" when empty. - - **No slider** — volume is the 0.20 constant. Picker = plain OS `OpenFileDialog` filtered to mp3/wav/m4a/aac/flac/ogg. -7. ✅ **Sound bar shows music** — `IsGameAudioBarVisible = gameDetectorProducingSound || isMusicPlaying`; relabelled "Game Audio Capture" → **"Desktop Audio"** (TRAX rides the same channel). **Follow-up (2026-08-15): the gating is gone — the game audio bar is ALWAYS visible** (the `IGameAudioDetector`/`GameAudioDetector`/`GameAudioHysteresis` stack + the VM's 250ms poll timer and their tests were deleted; the bar kept hiding the desktop meter when no full-screen game with sound was up, and the creator wants it pinned). -8. ✅ **Capture hardening (design changed at build time)** — the plan's "force IEEE-float 48 kHz on both WASAPI sources" was **dropped**: NAudio 2.2.1's `WasapiCapture` exposes no overridable `GetDefaultMixFormat`, so both sources capture the device's own mix format and the mixer's pure `TinyResampler` normalizes any rate/channel count to 48 kHz stereo (the resampler IS the design, not a fallback). Stop path disposes the audio pipe (EOF) in `MainViewModel.StopStream` before the pump stops, so both ffmpeg inputs end in order. -9. ✅ **Schema v9** — single-row `Music` (`TrackPath`, `IsEnabled`; volume is the 0.20 constant) + migration in `LayoutStore.cs` (`user_version` 9; `Save` gained an optional `Music? music = null` param so existing 3-arg callers still compile; load query + null reset). -10. ✅ **Docs in the same commit** — `ai.md` audio section rewritten ("one knob per input", 2-in mix, auto-duck, TRAX free, VOD-mute wrinkle), TASK 10's status, this task's status, HANDOFF. - -**Files (new):** `Services/Audio/VoiceFilterChain.cs` (+ `LowShelfFilter`/`HighShelfFilter`/`NoiseGate`/`Compressor`), `Services/Audio/AudioRingBuffer.cs` (async-arrival source buffers), `Services/Audio/AutoDucker.cs`, `Services/Audio/TinyResampler.cs`, `Services/Audio/MusicPlayer.cs`, `Services/Audio/NamedPipeAudioWriter.cs` (+ `IAudioPipeWriter` seam), `Models/Music.cs`. - -**Files (changed):** `AudioMixer.cs` (filter chain before meter+mix, loopback into the mix, gains via `Func` seams, ducker, silence-filler so the pipe never stalls, `StartLive`/`StopLive`), `WasapiMicAudioSource.cs` (device mix format — resampler normalizes; no forced format), `FfmpegArgs.cs` (named-pipe input + `-map`, name via `EncoderOptions.AudioPipeName`), `EncoderOptions.cs` (`AudioPipeName`), `ytLive.csproj` (`NAudio.WinMM` 2.2.1 for `WaveOutEvent`), `LayoutStore.cs` (schema v9), `MainViewModel.cs`/`MainWindow.xaml`/`MainWindow.xaml.cs` (footer TRAX group, sound-bar visibility OR music, go-live starts the pipe, end closes it first). - -**Testing (Good Dog Rule — ONE integration test):** `AudioPipelineTests.Mix_WithFiltersDuckAndGain_Lands_On_AudioPipe` — real filter chain + mixer + ducker + `NamedPipeAudioWriter`, fake `IAudioSource`s, test-side `NamedPipeClientStream` reads the bytes and asserts post-filter/post-gain/post-duck mixed stereo (ring buffers pre-filled so the first pipe tick already carries real audio — avoids a start-of-stream silence race). Units: each DSP stage (known sine-in → expected gain), ring buffer wrap/underflow/overwrite, ducker envelope, resampler (down/up/stateful/identity), `FfmpegArgs` pipe+map, schema v9 roundtrip + no-music-row. Existing tests stay green. - -**Out of scope (this branch):** IP webcam (video-only when it lands — never an audio input), chat box source (TASK 3 item 18), alt-key crop, credits, background removal, music-off-VOD, `PremiumUrl` (TASK 12 seam). - ---- - -## TASK 9 — YouTube Live Stream Management - -**Goal:** Create/bind broadcasts, monitor YouTube-side stream health — the v3 way. - -### Status: ⏳ In progress — items 1–3 SHIPPED (reusable stream 2026-08-16; report-by-exception health 2026-08-16); items 4–7 still open (each its own branch/PR) - -1. ☑ **Broadcast creation** — title/description/privacy/scheduledStartTime via API, with the v3 flags above (SHIPPED: `CreateBroadcast` sends `enableAutoStart/Stop`, `enableMonitorStream=false`, `latencyPreference=low`, `selfDeclaredMadeForKids=false`) -2. ☑ **Reusable stream** — create once, cache + reuse; bind to broadcast (SHIPPED: `GetOrCreateReusableStreamAsync` lists-then-inserts the `variable`/`isReusable` stream, cached via `LayoutStore` Settings, bound at broadcast insert via `boundStreamId`; `_rtmpUrlProvider` yields the ingest URL so go-live actually encodes + pushes) -3. ☑ **Health monitoring** — poll `liveStreams.list` `healthStatus` + `configurationIssues[]`, surface banner only on warning/error (SHIPPED: `GetStreamHealthAsync(streamId)` 30s while live; pure `StreamHealthReporter.BannerFor` = report-by-exception; banner strip under the top bar, amber warning / dark-red error, via `HealthIssueBanner`/`HealthIssueBackground`; poll failures log-only; ONE integration test `GetStreamHealthAsync_Report_By_Exception_Banner_Only_On_Warning_Or_Error`) -4. ☐ Live chat — poll `liveChat/messages`, render in right panel, support Super Chat + membership badges -5. ☐ Error handling — the YouTube error codes: `errorStreamInactive`, `invalidTransition`, `redundantTransition`, `liveStreamDeletionNotAllowed`, `liveStreamModificationNotAllowed`, `liveBroadcastBindingNotAllowed` -6. ☐ **Visibility picker** — remove temporary "always Private" enforcement (shipped as test-only; now unlocked for v1). User picks Private/Unlisted/Public from the go-live dialog. Trivial: remove the hardcoded override in `YouTubeStreamService.CreateBroadcast` (currently `privacyStatus = "private"` regardless of dialog selection) -7. ☐ **Full broadcast form** — expose all YouTube API-supported fields in the go-live dialog. Core tab: title, description, visibility, made-for-kids, schedule (start + optional end). Advanced tab (expandable, sane defaults): latency (Normal/Low/Ultra-Low), DVR, embed, record-from-start, projection (rectangular/360°), closed captions, auto-start, auto-stop, monitor stream, region restrictions. Monetization via `liveBroadcasts.update` (insert-only on that resource) — separate step after broadcast creation. Remove unsupported `categoryId` (not a `liveBroadcast` field, silently ignored) - -### Design decisions (v3) - -1. **One-click go-live** — `liveBroadcasts.insert` with `enableAutoStart=true`, `enableAutoStop=true`, `enableMonitorStream=false`, `selfDeclaredMadeForKids=false`, `latencyPreference=low`. No `transition(live)` call, no testing stage, no liveStarting polling. Encoder starts → YouTube brings it live by itself. -2. **Variable reusable stream** — `cdn.resolution=variable`, `cdn.frameRate=variable`, `isReusable=true`. Create once per channel, cache ingestion URL + stream name, reuse for every broadcast. Any quality tier works without recreation; auto step-down needs no API calls. -3. **Report-by-exception** — poll `liveStreams.list`; banner only on `healthStatus` warning/error issues (`configurationIssues[]`). Bottom strip = YouTube logo + green/red connection dot (clickable → opens the dialog). -4. **One dialog, three states** — `not connected` (sign-in) / `connected-offline` (all editable) / `live` (title + description + visibility editable; quality + account greyed out). Both entry points (Start Stream button + bottom strip) open it; prefilled from saved session profile. -5. **Live edits** — `liveBroadcasts.update` with part=`snippet,status` for title/description/privacy. -6. **End stream** — stop encoder → `transition(complete)`, with `enableAutoStop` as the safety net. -7. **Broadcast ID == Video ID** — one ID to track status, health, and the auto-created VOD (`recordFromStart` + `enableDvr`). - -### Requirements: - -1. **Broadcast creation** — title/description/privacy/scheduledStartTime via API, with the v3 flags above -2. **Reusable stream** — create once, cache + reuse; bind to broadcast -3. **Health monitoring** — poll `liveStreams.list` `healthStatus` + `configurationIssues[]`, surface banner only on warning/error -4. **Live chat** — poll `liveChat/messages`, render in right panel, support Super Chat + membership badges -5. **Error handling** — the YouTube error codes: `errorStreamInactive`, `invalidTransition`, `redundantTransition`, `liveStreamDeletionNotAllowed`, `liveStreamModificationNotAllowed`, `liveBroadcastBindingNotAllowed` -6. **Visibility picker** — Private/Unlisted/Public from the go-live dialog -7. **Full broadcast form** — Core + Advanced tabs with all API-supported fields - ---- - -## TASK 12 — Monetization: subscription billing + in-app unlock + support - -**Goal:** annual subscription billing, in-app unlock seam, and in-app bug-report → git-issues support loop. - -**Business details (pricing, billing research, revenue projections) in `MONETIZATION.md` (gitignored).** - -### Status: 🔶 Scoped — nothing built - -1. ☐ **Billing provider decision** — Gumroad vs Lemon Squeezy (see MONETIZATION.md for comparison) -2. ☐ **Unlock mechanism** — license-key entry + online/offline verification + persisted entitlement (extends `LayoutStore`, schema bump) + renewal/lapse handling -3. ☐ **In-app bug-report → git issues** — built-in report control; good error codes first, optional log attachment with transparency about what's being sent -4. ☐ **"Unlock Premium" seam** — the About hub's greyed button (`PremiumUrl`) lights up and routes to billing/unlock -5. ☐ **Alerts gating** — Super Chat pop-ins gated behind subscription state - --- ## TASK 5 — Layout Persistence (SQLite) @@ -655,6 +580,8 @@ the validator → persisted), compositor bar overlay (top/bottom + above-flash), --- +--- + ## TASK 6 — UI polish batch (scenes/sources rows, dedup naming, social bar) **Goal:** clean up the two side lists and the social bar per gramps's review. @@ -675,6 +602,8 @@ the validator → persisted), compositor bar overlay (top/bottom + above-flash), --- +--- + ## TASK 7 — Meter scaling amplification (voice meter uses the full bar) **Goal:** the mic (and game) meters feel light — speech peaks should peg into the red at maxed volume. @@ -691,7 +620,99 @@ the validator → persisted), compositor bar overlay (top/bottom + above-flash), --- -## TASK 13 — Post-pause polish batch: the creator's 8 review issues (2026-08-15, shipped) +--- + +## TASK 8 — Audio milestone: real stream audio + voice filters + TRAX music (2026-08-14) + +**Goal:** every audio issue done and tested in one branch — real mic/game audio reaches the encoder (replacing the `anullsrc` silence), `MicVolume`/`GameAudioVolume`/mute become real pre-AAC gains, TASK 10's voice filters land, auto-duck keeps the creator's voice over game + music, and a free background-music source ("TRAX") plays into the sound bar and the stream. + +### Status: ✅ SHIPPED 2026-08-14 — 196 tests passing, 0 warnings (ONE integration test: `AudioPipelineTests.Mix_WithFiltersDuckAndGain_Lands_On_AudioPipe`) + +1. ✅ **Real audio into the encoder** — the mixer's loopback samples were **dropped today** (`AudioMixer.OnLoopbackSample` metered only) and `MicVolume` was meter-only; `FfmpegArgs.cs:28` ran `-f lavfi -i anullsrc` (silence). Transport: **ffmpeg reads a Windows named pipe** — `-f f32le -ar 48000 -ac 2 -i \\.\pipe\ytllive_audio` replaces the anullsrc block, plus explicit `-map 0:v -map 1:a`; pipe name via `EncoderOptions.AudioPipeName` (`DefaultAudioPipeName = "ytllive_audio"`). Mixer stays **two inputs (mic + loopback)** — no N-source abstraction, no `StereoMixer` class. The VM owns the pipe lifecycle (`BeginGoLive` → `_audioMixer.StartLive(pipeName)`, `StopStream` → `_audioMixer.StopLive()` before the pump stops); `FfmpegEncoder.cs` is untouched. +2. ✅ **Honest gains reach the stream** — `MicVolume` scales the mic channel; `GameAudioVolume` + mute scale the loopback channel (the "mute in preview but stream still plays" footgun dies with it). One knob per input = KISS. Implemented as `Func` gain seams on `AudioMixer` (`micGain`/`loopbackGain`), wired by `MainViewModel`. +3. ✅ **Voice filters (TASK 10, before the meter AND the mix)** — `VoiceFilterChain`: bass (`LowShelfFilter` 120 Hz +4 dB) → treble (`HighShelfFilter` 8 kHz +3 dB) → **noise gate** (pure C#, creator's choice) → compressor (threshold 0.5, 4:1). Pure stateful DSP (TDF2), sine-in unit tests. Meter stays post-filter. +4. ✅ **Auto-duck** — pure envelope (`AutoDucker`): mic RMS > 0.02 → game+music dip ×0.25 (~12 dB), attack 0.05 / release 0.005, recover on release. Always-on, no knobs. Strong form of the creator's "voice always over the game volume" idea. +5. ✅ **TRAX — background music, FREE** (keeps the one paid line = Alerts + flash removal). `MusicPlayer`: NAudio `MediaFoundationReader` (mp3/wav/m4a) → `VolumeWaveProvider16` at the hardcoded **0.20** (fixed, not changeable) → `WaveOutEvent` on the default device (added the sibling `NAudio.WinMM` 2.2.1 package — `WaveOutEvent` isn't in `NAudio.Wasapi`), loop on end. **No 3rd mixer input and no `MusicVolume`** — music plays on the desktop, the existing loopback picks it up: heard in headphones, the sound-bar meter bounces, and the stream carries it through the loopback channel (ducked with game when the voice is hot). Known wrinkle (out of scope): YouTube mutes VODs carrying copyrighted music — future "music on live, off VOD". +6. ✅ **TRAX footer control (final spec, agreed 2026-08-14)** — `YtButtonSecondary`, **status dot + "TRAX"** text, in the center footer stack beside MIC: + - **Status dot** (same 8px Ellipse pattern as MIC/Socials): **red** no track loaded · **yellow** loaded not playing · **green** playing. + - **Left-click** → toggle play/pause; no track loaded → opens the picker instead. + - **Right-click** → always opens the in-app track picker (`TraxButton_PreviewMouseRightButtonUp`, code-behind pattern like `GameSpeaker_MouseLeftButtonUp`, `e.Handled = true` so no context menu). + - **Tooltip** → playing/loaded track name; "No track — right-click to choose background music" when empty. + - **No slider** — volume is the 0.20 constant. Picker = plain OS `OpenFileDialog` filtered to mp3/wav/m4a/aac/flac/ogg. +7. ✅ **Sound bar shows music** — `IsGameAudioBarVisible = gameDetectorProducingSound || isMusicPlaying`; relabelled "Game Audio Capture" → **"Desktop Audio"** (TRAX rides the same channel). **Follow-up (2026-08-15): the gating is gone — the game audio bar is ALWAYS visible** (the `IGameAudioDetector`/`GameAudioDetector`/`GameAudioHysteresis` stack + the VM's 250ms poll timer and their tests were deleted; the bar kept hiding the desktop meter when no full-screen game with sound was up, and the creator wants it pinned). +8. ✅ **Capture hardening (design changed at build time)** — the plan's "force IEEE-float 48 kHz on both WASAPI sources" was **dropped**: NAudio 2.2.1's `WasapiCapture` exposes no overridable `GetDefaultMixFormat`, so both sources capture the device's own mix format and the mixer's pure `TinyResampler` normalizes any rate/channel count to 48 kHz stereo (the resampler IS the design, not a fallback). Stop path disposes the audio pipe (EOF) in `MainViewModel.StopStream` before the pump stops, so both ffmpeg inputs end in order. +9. ✅ **Schema v9** — single-row `Music` (`TrackPath`, `IsEnabled`; volume is the 0.20 constant) + migration in `LayoutStore.cs` (`user_version` 9; `Save` gained an optional `Music? music = null` param so existing 3-arg callers still compile; load query + null reset). +10. ✅ **Docs in the same commit** — `ai.md` audio section rewritten ("one knob per input", 2-in mix, auto-duck, TRAX free, VOD-mute wrinkle), TASK 10's status, this task's status, HANDOFF. + +**Files (new):** `Services/Audio/VoiceFilterChain.cs` (+ `LowShelfFilter`/`HighShelfFilter`/`NoiseGate`/`Compressor`), `Services/Audio/AudioRingBuffer.cs` (async-arrival source buffers), `Services/Audio/AutoDucker.cs`, `Services/Audio/TinyResampler.cs`, `Services/Audio/MusicPlayer.cs`, `Services/Audio/NamedPipeAudioWriter.cs` (+ `IAudioPipeWriter` seam), `Models/Music.cs`. + +**Files (changed):** `AudioMixer.cs` (filter chain before meter+mix, loopback into the mix, gains via `Func` seams, ducker, silence-filler so the pipe never stalls, `StartLive`/`StopLive`), `WasapiMicAudioSource.cs` (device mix format — resampler normalizes; no forced format), `FfmpegArgs.cs` (named-pipe input + `-map`, name via `EncoderOptions.AudioPipeName`), `EncoderOptions.cs` (`AudioPipeName`), `ytLive.csproj` (`NAudio.WinMM` 2.2.1 for `WaveOutEvent`), `LayoutStore.cs` (schema v9), `MainViewModel.cs`/`MainWindow.xaml`/`MainWindow.xaml.cs` (footer TRAX group, sound-bar visibility OR music, go-live starts the pipe, end closes it first). + +**Testing (Good Dog Rule — ONE integration test):** `AudioPipelineTests.Mix_WithFiltersDuckAndGain_Lands_On_AudioPipe` — real filter chain + mixer + ducker + `NamedPipeAudioWriter`, fake `IAudioSource`s, test-side `NamedPipeClientStream` reads the bytes and asserts post-filter/post-gain/post-duck mixed stereo (ring buffers pre-filled so the first pipe tick already carries real audio — avoids a start-of-stream silence race). Units: each DSP stage (known sine-in → expected gain), ring buffer wrap/underflow/overwrite, ducker envelope, resampler (down/up/stateful/identity), `FfmpegArgs` pipe+map, schema v9 roundtrip + no-music-row. Existing tests stay green. + +**Out of scope (this branch):** IP webcam (video-only when it lands — never an audio input), chat box source (TASK 3 item 18), alt-key crop, credits, background removal, music-off-VOD, `PremiumUrl` (TASK 10 seam). + +--- + +--- + +## TASK 9 — YouTube Live Stream Management + +**Goal:** Create/bind broadcasts, monitor YouTube-side stream health — the v3 way. + +### Status: ⏳ In progress — items 1–3 SHIPPED (reusable stream 2026-08-16; report-by-exception health 2026-08-16); items 4–7 still open (each its own branch/PR) + +1. ☑ **Broadcast creation** — title/description/privacy/scheduledStartTime via API, with the v3 flags above (SHIPPED: `CreateBroadcast` sends `enableAutoStart/Stop`, `enableMonitorStream=false`, `latencyPreference=low`, `selfDeclaredMadeForKids=false`) +2. ☑ **Reusable stream** — create once, cache + reuse; bind to broadcast (SHIPPED: `GetOrCreateReusableStreamAsync` lists-then-inserts the `variable`/`isReusable` stream, cached via `LayoutStore` Settings, bound at broadcast insert via `boundStreamId`; `_rtmpUrlProvider` yields the ingest URL so go-live actually encodes + pushes) +3. ☑ **Health monitoring** — poll `liveStreams.list` `healthStatus` + `configurationIssues[]`, surface banner only on warning/error (SHIPPED: `GetStreamHealthAsync(streamId)` 30s while live; pure `StreamHealthReporter.BannerFor` = report-by-exception; banner strip under the top bar, amber warning / dark-red error, via `HealthIssueBanner`/`HealthIssueBackground`; poll failures log-only; ONE integration test `GetStreamHealthAsync_Report_By_Exception_Banner_Only_On_Warning_Or_Error`) +4. ☐ Live chat — poll `liveChat/messages`, render in right panel, support Super Chat + membership badges +5. ☐ Error handling — the YouTube error codes: `errorStreamInactive`, `invalidTransition`, `redundantTransition`, `liveStreamDeletionNotAllowed`, `liveStreamModificationNotAllowed`, `liveBroadcastBindingNotAllowed` +6. ☐ **Visibility picker** — remove temporary "always Private" enforcement (shipped as test-only; now unlocked for v1). User picks Private/Unlisted/Public from the go-live dialog. Trivial: remove the hardcoded override in `YouTubeStreamService.CreateBroadcast` (currently `privacyStatus = "private"` regardless of dialog selection) +7. ☐ **Full broadcast form** — expose all YouTube API-supported fields in the go-live dialog. Core tab: title, description, visibility, made-for-kids, schedule (start + optional end). Advanced tab (expandable, sane defaults): latency (Normal/Low/Ultra-Low), DVR, embed, record-from-start, projection (rectangular/360°), closed captions, auto-start, auto-stop, monitor stream, region restrictions. Monetization via `liveBroadcasts.update` (insert-only on that resource) — separate step after broadcast creation. Remove unsupported `categoryId` (not a `liveBroadcast` field, silently ignored) + +### Design decisions (v3) + +1. **One-click go-live** — `liveBroadcasts.insert` with `enableAutoStart=true`, `enableAutoStop=true`, `enableMonitorStream=false`, `selfDeclaredMadeForKids=false`, `latencyPreference=low`. No `transition(live)` call, no testing stage, no liveStarting polling. Encoder starts → YouTube brings it live by itself. +2. **Variable reusable stream** — `cdn.resolution=variable`, `cdn.frameRate=variable`, `isReusable=true`. Create once per channel, cache ingestion URL + stream name, reuse for every broadcast. Any quality tier works without recreation; auto step-down needs no API calls. +3. **Report-by-exception** — poll `liveStreams.list`; banner only on `healthStatus` warning/error issues (`configurationIssues[]`). Bottom strip = YouTube logo + green/red connection dot (clickable → opens the dialog). +4. **One dialog, three states** — `not connected` (sign-in) / `connected-offline` (all editable) / `live` (title + description + visibility editable; quality + account greyed out). Both entry points (Start Stream button + bottom strip) open it; prefilled from saved session profile. +5. **Live edits** — `liveBroadcasts.update` with part=`snippet,status` for title/description/privacy. +6. **End stream** — stop encoder → `transition(complete)`, with `enableAutoStop` as the safety net. +7. **Broadcast ID == Video ID** — one ID to track status, health, and the auto-created VOD (`recordFromStart` + `enableDvr`). + +### Requirements: + +1. **Broadcast creation** — title/description/privacy/scheduledStartTime via API, with the v3 flags above +2. **Reusable stream** — create once, cache + reuse; bind to broadcast +3. **Health monitoring** — poll `liveStreams.list` `healthStatus` + `configurationIssues[]`, surface banner only on warning/error +4. **Live chat** — poll `liveChat/messages`, render in right panel, support Super Chat + membership badges +5. **Error handling** — the YouTube error codes: `errorStreamInactive`, `invalidTransition`, `redundantTransition`, `liveStreamDeletionNotAllowed`, `liveStreamModificationNotAllowed`, `liveBroadcastBindingNotAllowed` +6. **Visibility picker** — Private/Unlisted/Public from the go-live dialog +7. **Full broadcast form** — Core + Advanced tabs with all API-supported fields + +--- + +--- + +## TASK 10 — Monetization: subscription billing + in-app unlock + support + +**Goal:** annual subscription billing, in-app unlock seam, and in-app bug-report → git-issues support loop. + +**Business details (pricing, billing research, revenue projections) in `MONETIZATION.md` (gitignored).** + +### Status: 🔶 Scoped — nothing built + +1. ☐ **Billing provider decision** — Gumroad vs Lemon Squeezy (see MONETIZATION.md for comparison) +2. ☐ **Unlock mechanism** — license-key entry + online/offline verification + persisted entitlement (extends `LayoutStore`, schema bump) + renewal/lapse handling +3. ☐ **In-app bug-report → git issues** — built-in report control; good error codes first, optional log attachment with transparency about what's being sent +4. ☐ **"Unlock Premium" seam** — the About hub's greyed button (`PremiumUrl`) lights up and routes to billing/unlock +5. ☐ **Alerts gating** — Super Chat pop-ins gated behind subscription state + +--- + +--- + +## TASK 11 — Post-pause polish batch: the creator's 8 review issues (2026-08-15, shipped) **Status: ✅ SHIPPED 2026-08-15 — 197 tests passing, 0 warnings (ONE integration test: `AudioPipelineTests.Mix_HonorsProviderGains_AndGameMute_KillsTheLoopback`).** @@ -736,7 +757,9 @@ The creator reviewed the TASK 8 build and filed 8 issues. All fixed in one branc --- -## TASK 14 — Master limiter on the live mix (2026-08-15) +--- + +## TASK 12 — Master limiter on the live mix (2026-08-15) **Queued by the creator during the TRAX discussion:** "should the soundtrack be limited to avoid squandering resources / should it cap at 20% / how do the three sound events balance?" Review @@ -766,7 +789,9 @@ No changes to `MusicPlayer`, the 0.20 cap, the ducker, or the meter zones. Build --- -## TASK 15 — Social media launch kit +--- + +## TASK 13 — Social media launch kit **Goal:** marcom/social media assets and strategy for v1 launch. @@ -789,3 +814,5 @@ No changes to `MusicPlayer`, the 0.20 cap, the ducker, or the meter zones. Build 1. v0.2 — Recording to local file (recordings carry the branding flash — see TASK 3 / `ai.md` Monetization) 2. v0.4 — Multi-destination restreaming (if needed; casual streamers may outgrow ytLlive first — "congrats, you're ready for OBS") 3. v0.5 — Stream clipping + +--- diff --git a/ViewModels/index.md b/ViewModels/index.md index 62edeb5..4d864b7 100644 --- a/ViewModels/index.md +++ b/ViewModels/index.md @@ -4,7 +4,7 @@ MVVM layer. See [`schema.md`](../schema.md) for the memory-map conventions. | File | Purpose | |------|---------| -| `MainViewModel.cs` | The app brain: scenes/elements collections + commands, stream state (`IsLive`/`IsOffline`/`IsConnected`), chat feed, overlays, layout save/open, **resolution dropdown** (`QualityOptions`, `SelectedQuality`, `ResolutionHelp`, `ApplyStreamQuality`) that computes the output rect over the 1920×1080 master (`OutputRectX/Y/W/H`, `DimRects`, `IsOutputCropped`, `ResolutionBadgeText`). Auth: loads the saved DPAPI session at startup (`LoadSavedSessionAsync`), `SignInAsync` feeds the GoLive dialog; **go-live (ship step 7): `BeginGoLive` accepts the dialog → sets `StreamVisibility` (always "Private") → fires `CreateBroadcastAsync` (stores `_currentBroadcastId` for TASK 9's bind/transition; failure → `StreamStatus.Error`) + `_framePump.StartAsync()`**; **End Livestream signs out** (`StopStream` clears session + token + `_currentBroadcastId` — crash-safe). **Connected account:** top-bar avatar/name (`AccountAvatarUrl`/`AccountDisplayName` via `SyncConnectedAccount`, cleared on End). **Element rows:** scenes are pure selection rows (no per-row icons); source rows carry edit/visibility/trash (`EditElementCommand` sets `element.IsEditing`, `ToggleElementVisibilityCommand` flips `IsVisible`, `RemoveSourceCommand`; the Backdrop's trash slot stays **reserved** via `HiddenBoolToVisibilityConverter` so the edit/eye icons keep their fixed columns — TASK 13); duplicate resource names get a no-space incrementing suffix via shared `NextSourceName` (Image, Image2, Image3… — next free number derived from actual names, so deletions never collide; used by `AddSource` + `AddReusedImage`). **Audio (KISS — the mic is the creator's only audio control; capture SHIPPED, runs for the app's lifetime so the meters preview live):** the sound meter is a **READ-ONLY realtime level display**: fill = `Math.Min(1, AudioLevelMeter.ToDisplay((float)AudioLevel) * MicVolume)` (`MeterFillWidth`/`MeterBrush`, green→yellow→red, zone markers at 60%/80% — `ToDisplay` maps the raw linear RMS onto a −60..0 dBFS scale with +10 dB amplification so real speech/game levels occupy the bar and peaks hit red at maxed volume), where `AudioLevel` (live input, 0 with no input) is fed by the audio mixer once capture lands and `MicVolume` acts as a gain on ambient noise; while the slider is dragged the bar previews the slider position (`SetVolumeAdjusting`), returning to the live level on release (0 with no input — clicking the meter does nothing); `MicVolume` (default 0.8) + read-only `MicMuted`/`MicMuteText`/`ToggleMicMuteCommand` — **MicVolume drives MicMuted** (muted ⇔ volume 0): sliding to 0 flips the speaker to muted, sliding up from 0 clears it; muting stores the prior volume, unmuting restores it (default 0.8 if unknown) and flashes the meter to the restored position ~300ms (`BeginVolumeFlash`/`EndVolumeFlash`); `MicSourceName` = picked voice source, shown left-justified inside the meter bar (the fill runs at 75% opacity so the text + ruler markings show through); `OpenMicPickerCommand`/`PickMicrophone()` open the `MicPickerDialog` (a picked device takes effect immediately — `PickMicrophone` swaps the live source via `_audioMixer.RestartMic()`, loopback keeps running); the **MIC label is a button** (`OpenMicPickerCommand`) with a **status dot** (`MicStatus`, `Models/MicStatus`: green = `MicConnected` via the source's `Started` event, yellow = `MicFailed` — in use/unplugged, red = no mic device at startup; `MicStatusBrush`/`MicStatusToolTip`); capture starts once at startup (`StartMicCaptureAsync`) — NOT go-live (`BeginGoLive`/`StopStream` no longer touch the mixer) — zero devices = red dot + the mixer never starts. **Game audio bar** (desktop/game, **overlaid at the bottom of the preview window** — bottom-center chip, a mirror of the mic bar; **always visible** — the TASK 4 show/hide gating (`IGameAudioDetector`/`GameAudioDetector`/`GameAudioHysteresis` + the 250ms poll timer) was **removed 2026-08-15** because the bar kept hiding the desktop meter whenever no full-screen game with sound was up): `GameAudioLevel` (loopback meter via `LoopbackLevelChanged`, scaled by volume), `GameMuted`/`GameMuteText`/`ToggleGameMuteCommand`, `GameAudioVolume` (0..1 volume slider), `Begin/End/CancelGameVolumeFlash` (mirrors the mic bar's volume flash); the game speaker + slider share the mic bar's `PreviewMouseLeftButtonDown/Up` + `LostMouseCapture` code-behind pattern. **Webcam:** owns `CameraManager` (MediaCapture), `AddWebcamToActiveSceneAsync` (picker → default 480×270 bottom-right placement → acquire; the Windows camera picker ALWAYS opens so the creator chooses — never silently reuses the previous camera; picking a different camera swaps the app-wide identity via `SwapWebcamIdentityAsync`, same path as `ChangeWebcamAsync`; propagates the running shared bitmap via `CameraManager.GetPreviewBitmap` first, so a webcam added mid-session never renders a transparent container; greys out when the active scene already has a config via `CanAddWebcamToActiveScene`), `ChangeWebcamAsync` (device swap — `ReleaseAllAsync` old + re-acquire), `ShowWebcamInActiveScene` (reveal hidden config / empty-canvas right-click), `ReacquireWebcam` after layout load (re-propagates the shared bitmap to every config), `OnCameraPreviewBitmapChanged` forwards the shared bitmap into every `WebcamSceneConfig.VideoImageSource`; `SelectedElement` drives the preview overlay + `internal ClampWebcamToBounds(config, sceneName)` (per-scene size cap seam: 50%-per-dimension everywhere, half-screen-AREA in Chat via `MaxWebcamWidthFor`/`MaxWebcamHeightFor`). **Scenes (five-scene catalog):** empty DB seeds Starting/Live/BRB/Chat/Ending (`SceneCatalog.All`); `AddScene(name)` accepts only canonical, missing names; `MissingScenes`/`ShowAddScene` drive the "+" button (hidden once all five exist; its menu lists only the missing scenes via `AddSceneCommand`). **Screen backdrop (Live-only by policy):** owns `ScreenCaptureManager` + `ScreenCaptureSourceFactory` (WinRT GraphicsCapture), `internal static EnsureBackdrop` heals one per backdrop-enabled scene (gated on `Scene.HasBackdrop`), `ReacquireScreenCaptures`/`RefreshBackdropAutoCapture`/`NoteBackgroundWindow` key by full-screen game monitor or the **primary display** via `Win32FullScreenDetector` (`GetDisplays()`/`PrimaryMonitorIndex()`), `ChangeBackdropCaptureAsync` (OS picker) + `SetBackdropCapture(DisplayInfo)` (in-app "Capture Display"), `RedesignateBackdropAsync` re-targets all backdrops and releases orphaned sessions, `internal static EnforceBackdropPolicy` normalizes `Scene.HasBackdrop` by name on every load + strips lingering non-Live backdrops, `CanChangeBackdrop` gates the capture menu to Live (the only scene with a backdrop), `BackdropImage` hides the preview watermark (`ShowPreviewPlaceholder`). **REC sign (2026-08-13):** top-center indicator always visible — `RecDotBrush` (offline `#555555`, live `#e94560`, live-private `#8f1f1f`), `RecTextBrush` (dim offline, white live), `RecDotOpacity` (0.55 offline, pulsing 1.0/0.35 live via `_recDotPulse` flipped on the live tick), `IsLivePrivate` (`IsLive && StreamVisibility == "Private"`) — notified from the `StreamStatus` + `StreamVisibility` setters; `LiveIndicatorVisible`/`LivePulseOpacity` removed. **Mic mute icon:** a second 16px clickable glyph (mic, red + slash when muted) between the meter and the speaker on the mic bar — same `ToggleMicMuteCommand`. **TASK 24 (2026-08-15):** the gain seams (`AudioGainProvider` → `micGain`/`loopbackGain`) reach the mixer at construction so volume sliders + mutes are stream-honest; the game bar also scales TRAX locally (`MusicPlayer.LocalGain`); `MicSourceName` persists via `LayoutStore`'s `Settings` table + restores before first `Start`; TRAX moved into the game audio bar (left of "Desktop Audio") in the preview overlay. **Health stats (TASK 4 ship step 6, 2026-08-13):** `OnFramePumpHealthUpdated` marshals `FramePump.HealthUpdated` (encoder's parsed bitrate/FPS/dropped/duration — raised on the stderr thread) onto the UI thread into `CurrentHealth` (bottom bar bindings); `ResetHealth(status)` zeroes dropped/duration on go-live/End so stats never linger | +| `MainViewModel.cs` | The app brain: scenes/elements collections + commands, stream state (`IsLive`/`IsOffline`/`IsConnected`), chat feed, overlays, layout save/open, **resolution dropdown** (`QualityOptions`, `SelectedQuality`, `ResolutionHelp`, `ApplyStreamQuality`) that computes the output rect over the 1920×1080 master (`OutputRectX/Y/W/H`, `DimRects`, `IsOutputCropped`, `ResolutionBadgeText`). Auth: loads the saved DPAPI session at startup (`LoadSavedSessionAsync`), `SignInAsync` feeds the GoLive dialog; **go-live (ship step 7): `BeginGoLive` accepts the dialog → sets `StreamVisibility` (always "Private") → fires `CreateBroadcastAsync` (stores `_currentBroadcastId` for TASK 9's bind/transition; failure → `StreamStatus.Error`) + `_framePump.StartAsync()`**; **End Livestream signs out** (`StopStream` clears session + token + `_currentBroadcastId` — crash-safe). **Connected account:** top-bar avatar/name (`AccountAvatarUrl`/`AccountDisplayName` via `SyncConnectedAccount`, cleared on End). **Element rows:** scenes are pure selection rows (no per-row icons); source rows carry edit/visibility/trash (`EditElementCommand` sets `element.IsEditing`, `ToggleElementVisibilityCommand` flips `IsVisible`, `RemoveSourceCommand`; the Backdrop's trash slot stays **reserved** via `HiddenBoolToVisibilityConverter` so the edit/eye icons keep their fixed columns — TASK 13); duplicate resource names get a no-space incrementing suffix via shared `NextSourceName` (Image, Image2, Image3… — next free number derived from actual names, so deletions never collide; used by `AddSource` + `AddReusedImage`). **Audio (KISS — the mic is the creator's only audio control; capture SHIPPED, runs for the app's lifetime so the meters preview live):** the sound meter is a **READ-ONLY realtime level display**: fill = `Math.Min(1, AudioLevelMeter.ToDisplay((float)AudioLevel) * MicVolume)` (`MeterFillWidth`/`MeterBrush`, green→yellow→red, zone markers at 60%/80% — `ToDisplay` maps the raw linear RMS onto a −60..0 dBFS scale with +10 dB amplification so real speech/game levels occupy the bar and peaks hit red at maxed volume), where `AudioLevel` (live input, 0 with no input) is fed by the audio mixer once capture lands and `MicVolume` acts as a gain on ambient noise; while the slider is dragged the bar previews the slider position (`SetVolumeAdjusting`), returning to the live level on release (0 with no input — clicking the meter does nothing); `MicVolume` (default 0.8) + read-only `MicMuted`/`MicMuteText`/`ToggleMicMuteCommand` — **MicVolume drives MicMuted** (muted ⇔ volume 0): sliding to 0 flips the speaker to muted, sliding up from 0 clears it; muting stores the prior volume, unmuting restores it (default 0.8 if unknown) and flashes the meter to the restored position ~300ms (`BeginVolumeFlash`/`EndVolumeFlash`); `MicSourceName` = picked voice source, shown left-justified inside the meter bar (the fill runs at 75% opacity so the text + ruler markings show through); `OpenMicPickerCommand`/`PickMicrophone()` open the `MicPickerDialog` (a picked device takes effect immediately — `PickMicrophone` swaps the live source via `_audioMixer.RestartMic()`, loopback keeps running); the **MIC label is a button** (`OpenMicPickerCommand`) with a **status dot** (`MicStatus`, `Models/MicStatus`: green = `MicConnected` via the source's `Started` event, yellow = `MicFailed` — in use/unplugged, red = no mic device at startup; `MicStatusBrush`/`MicStatusToolTip`); capture starts once at startup (`StartMicCaptureAsync`) — NOT go-live (`BeginGoLive`/`StopStream` no longer touch the mixer) — zero devices = red dot + the mixer never starts. **Game audio bar** (desktop/game, **overlaid at the bottom of the preview window** — bottom-center chip, a mirror of the mic bar; **always visible** — the TASK 4 show/hide gating (`IGameAudioDetector`/`GameAudioDetector`/`GameAudioHysteresis` + the 250ms poll timer) was **removed 2026-08-15** because the bar kept hiding the desktop meter whenever no full-screen game with sound was up): `GameAudioLevel` (loopback meter via `LoopbackLevelChanged`, scaled by volume), `GameMuted`/`GameMuteText`/`ToggleGameMuteCommand`, `GameAudioVolume` (0..1 volume slider), `Begin/End/CancelGameVolumeFlash` (mirrors the mic bar's volume flash); the game speaker + slider share the mic bar's `PreviewMouseLeftButtonDown/Up` + `LostMouseCapture` code-behind pattern. **Webcam:** owns `CameraManager` (MediaCapture), `AddWebcamToActiveSceneAsync` (picker → default 480×270 bottom-right placement → acquire; the Windows camera picker ALWAYS opens so the creator chooses — never silently reuses the previous camera; picking a different camera swaps the app-wide identity via `SwapWebcamIdentityAsync`, same path as `ChangeWebcamAsync`; propagates the running shared bitmap via `CameraManager.GetPreviewBitmap` first, so a webcam added mid-session never renders a transparent container; greys out when the active scene already has a config via `CanAddWebcamToActiveScene`), `ChangeWebcamAsync` (device swap — `ReleaseAllAsync` old + re-acquire), `ShowWebcamInActiveScene` (reveal hidden config / empty-canvas right-click), `ReacquireWebcam` after layout load (re-propagates the shared bitmap to every config), `OnCameraPreviewBitmapChanged` forwards the shared bitmap into every `WebcamSceneConfig.VideoImageSource`; `SelectedElement` drives the preview overlay + `internal ClampWebcamToBounds(config, sceneName)` (per-scene size cap seam: 50%-per-dimension everywhere, half-screen-AREA in Chat via `MaxWebcamWidthFor`/`MaxWebcamHeightFor`). **Scenes (five-scene catalog):** empty DB seeds Starting/Live/BRB/Chat/Ending (`SceneCatalog.All`); `AddScene(name)` accepts only canonical, missing names; `MissingScenes`/`ShowAddScene` drive the "+" button (hidden once all five exist; its menu lists only the missing scenes via `AddSceneCommand`). **Screen backdrop (Live-only by policy):** owns `ScreenCaptureManager` + `ScreenCaptureSourceFactory` (WinRT GraphicsCapture), `internal static EnsureBackdrop` heals one per backdrop-enabled scene (gated on `Scene.HasBackdrop`), `ReacquireScreenCaptures`/`RefreshBackdropAutoCapture`/`NoteBackgroundWindow` key by full-screen game monitor or the **primary display** via `Win32FullScreenDetector` (`GetDisplays()`/`PrimaryMonitorIndex()`), `ChangeBackdropCaptureAsync` (OS picker) + `SetBackdropCapture(DisplayInfo)` (in-app "Capture Display"), `RedesignateBackdropAsync` re-targets all backdrops and releases orphaned sessions, `internal static EnforceBackdropPolicy` normalizes `Scene.HasBackdrop` by name on every load + strips lingering non-Live backdrops, `CanChangeBackdrop` gates the capture menu to Live (the only scene with a backdrop), `BackdropImage` hides the preview watermark (`ShowPreviewPlaceholder`). **REC sign (2026-08-13):** top-center indicator always visible — `RecDotBrush` (offline `#555555`, live `#e94560`, live-private `#8f1f1f`), `RecTextBrush` (dim offline, white live), `RecDotOpacity` (0.55 offline, pulsing 1.0/0.35 live via `_recDotPulse` flipped on the live tick), `IsLivePrivate` (`IsLive && StreamVisibility == "Private"`) — notified from the `StreamStatus` + `StreamVisibility` setters; `LiveIndicatorVisible`/`LivePulseOpacity` removed. **Mic mute icon:** a second 16px clickable glyph (mic, red + slash when muted) between the meter and the speaker on the mic bar — same `ToggleMicMuteCommand`. **TASK 11 (2026-08-15):** the gain seams (`AudioGainProvider` → `micGain`/`loopbackGain`) reach the mixer at construction so volume sliders + mutes are stream-honest; the game bar also scales TRAX locally (`MusicPlayer.LocalGain`); `MicSourceName` persists via `LayoutStore`'s `Settings` table + restores before first `Start`; TRAX moved into the game audio bar (left of "Desktop Audio") in the preview overlay. **Health stats (TASK 4 ship step 6, 2026-08-13):** `OnFramePumpHealthUpdated` marshals `FramePump.HealthUpdated` (encoder's parsed bitrate/FPS/dropped/duration — raised on the stderr thread) onto the UI thread into `CurrentHealth` (bottom bar bindings); `ResetHealth(status)` zeroes dropped/duration on go-live/End so stats never linger | | `GoLiveViewModel.cs` | Start Stream dialog: **account row** (saved channel shown with Change Account, or Sign in to YouTube; Start gated on `IsSignedIn`/`IsBusy`) + title/description + **locked Visibility (always "Private" — ship step 7, no dropdown)**, start/cancel requests | | **About hub (TASK 11, in `MainViewModel`)** | the About overlay is the in-app creator hub: `ChannelUrl`/`MastodonUrl`/`CoffeeUrl` (public consts, live) + `PremiumUrl` (tabled billing seam — TASK 12 → the greyed "Unlock Premium" button lights up when filled); `OpenLicensesCommand` → `ShowLicenses()` loads the shipped `THIRD-PARTY-NOTICES.txt` into `LicensesText` and sets `IsLicensesOpen` (in-app scrolling panel, never the OS viewer); `BackToAbout_Click` in `MainWindow.xaml.cs` flips it off. | | `ReuseImageViewModel.cs` | Add Image dialog: candidate list (`ReuseImageCandidate`), reuse/new/cancel | diff --git a/ai.md b/ai.md index 987f746..b76f900 100644 --- a/ai.md +++ b/ai.md @@ -112,7 +112,7 @@ C# / WPF (.NET 8) following MVVM: ### Current limitations / TODOs - `Helpers/OAuthCredentials.cs` contains the real ClientId/ClientSecret. Auth is complete and the session **persists via Windows DPAPI** (`Helpers/TokenStore.cs` → `%APPDATA%\ytLlive\ytLlive.auth`, CurrentUser scope), reloaded best-effort at startup with a proactive refresh of a near-expiry access token. Sign-in/Change Account lives **inside the Start Stream dialog** (two-state flow — no separate Connect button). A **graceful End Livestream signs out**: `StopStream()` clears the session + token, so the next go-live needs a fresh sign-in; a crash never runs End, so the token survives and the creator stays signed in. `YouTubeAuthService` takes an optional `HttpClient` + `sessionChanged` callback (test seam + save hook; services are still constructed in `MainViewModel`) -- Scene/source/asset layout + the social bar persist (SQLite, schema v8); the OAuth session persists (DPAPI); the paid-unlock state does not (yet — subscription entitlement verification pending, TASK 12) +- Scene/source/asset layout + the social bar persist (SQLite, schema v8); the OAuth session persists (DPAPI); the paid-unlock state does not (yet — subscription entitlement verification pending, TASK 10) - `YouTubeStreamService` manages the **variable reusable stream** (shipped 2026-08-16): `GetOrCreateReusableStreamAsync` lists `liveStreams?mine=true` and reuses the existing `cdn.isReusable` stream, creating it only on first use (`resolution=variable`, `frameRate=variable`); the stream is cached via `LayoutStore` (`SaveReusableStream`/`LoadReusableStream`, Settings table) and bound at broadcast insert (`contentDetails.boundStreamId`). Health (shipped 2026-08-16): `GetStreamHealthAsync(streamId)` polls `liveStreams?part=status` for `healthStatus` + `configurationIssues[]` → `StreamHealth`; banner decision in pure `StreamHealthReporter` - Webcam capture is shipped (milestone 1); the live desktop/game backdrop is shipped (ship task #1); **the output compositor (TASK 4 ship step 1) is SHIPPED**, **the FFmpeg locator (TASK 4 ship step 2) is SHIPPED**, **the encoder + RTMP push (TASK 4 ship step 3) is SHIPPED**, **WASAPI audio capture (TASK 4 ship step 4) is SHIPPED** — full plan in `TASKS.md`; the frame-pipeline wiring follows (its own PR) - `StreamConfig` defaults (`TargetBitrate=6000`, `Resolution="1920x1080"`) are stale — the live dropdown drives `StreamHealth.CurrentBitrate`/`FPS` instead @@ -610,8 +610,8 @@ annual subscription** (settled 2026-08-14 — replaces the earlier "one-time unl - **Billing:** NOT locked to itch.io (creator's call 2026-08-14). Hunted fact — **itch.io has no native subscription billing** (no annual/recurring product billing; only pay-what-you-want, pre-orders, early-access, keys, and a Patreon integration). Candidates for annual subs + license-key verification: - **Gumroad** (its native affiliate program is a tiebreaker if the affiliate angle in TASK 12 matters) - and **Lemon Squeezy**. Decision deferred to TASK 12. + **Gumroad** (its native affiliate program is a tiebreaker if the affiliate angle in TASK 10 matters) + and **Lemon Squeezy**. Decision deferred to TASK 10. - **Support (creator's model):** in-app bug-reporting mechanism → issues into git; most queries are how-tos / feature requests / manual-skimmers. Maintenance cadence = "when I get around to it" with emergency patches; not a 24/7 service promise.