Files
LlamaCasty/TASKS.md
T
gramps fbc8562cf4 perf(capture): slice 8 — buffer ring + paste-cache Epoch; gen2 visibility (take-11 spikes)
Take 11 (c10ce06c) validated the off-UI architecture: typical frames land
work ~10ms + wait ~6.8ms = 16.7 exactly on the deadline; 212/300 best yet.
The ENTIRE remaining gap is periodic 35-65ms render spikes that WORSENED
across the take (189 -> 147) — the signature of gen2 GC pauses. Biggest
churner is structural: the screen capture minted a fresh ~8.3MB byte[] per
DWM frame (~500MB/s of LOH), a producer OBS never does (it owns fixed
surface pools).

- ScreenCaptureFrameSource: 4-deep buffer ring with size-matched slots (a
  <=17ms consumer cannot be lapped at 60Hz) + reused downscale row scratch.
- VideoFrame.Epoch: monotonic per producer frame. The paste cache keys on
  array IDENTITY, so recycled arrays MUST be distinguished — epoch joins the
  PasteKey. Producers handing fresh arrays leave it 0 (key unchanged effect).
- Stats print 'gen2 +N' per 5s window: next take acquits or convicts GC
  without another guess (rule: prove the stage).
- Test (the ONE): PasteCache_RecycledArrayWithNewEpoch_ReRasterizes_NotStaleHits
  — same array, new content, bumped epoch; fails on the old key by
  construction. 37/37 compositor/pump, clean build.
- Next suspect if gen2 stays hot: the 10Hz WebView2 capture (full-canvas PNG
  decode + fresh arrays on the UI thread) — recorded, untouched.

Creator audio ask queued in the same working session (+40% post-mix master
gain before the -1dBFS limiter) lands as its own commit next.
2026-09-04 12:36:14 -07:00

1633 lines
158 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# ytLlive — Task List
> Task queue and authoritative research. Memory-map conventions: [`schema.md`](schema.md);
> architecture/decisions: [`ai.md`](ai.md). Update statuses here whenever a task moves.
> **Checklist markers** — every task's Status list uses the same states:
> 1. ✅ — completed (green check)
> 2. 🔶 — in progress (amber diamond)
> 3. ☐ — not completed / pending (empty box)
> 4. ❌ — exception (blocked, known-issue, or deliberately excluded from this build)
> **2026-09-01 — v1 = feature-complete ruling:** there is no v1.x. Everything queues to v1 or to
> **"Out of product — permanently"** (file end). Read those two sections before adding or reviving
> anything here.
## YouTube Live API — research facts (authoritative, v3 build)
Lifecycle: `created → ready → [testing] → live → complete` (transitional `liveStarting` / `testStarting`).
1. **liveBroadcasts.insert** requires: `snippet.title`, `snippet.scheduledStartTime`, `status.privacyStatus`, `status.selfDeclaredMadeForKids` (COPPA).
2. **liveStreams.insert** requires: `snippet.title`, `cdn.frameRate`, `cdn.ingestionType`, `cdn.resolution`. **None of the four (except title) can ever change after creation** — changing them means delete + recreate the stream. This is the hard constraint behind the quality grey-out.
3. **Title / description / privacy**: editable at any time, including while live (`liveBroadcasts.update`, part=`snippet,status`).
4. **contentDetails** (DVR, recordFromStart, monitorStream, embed, latency): editable only in `created` / `ready`.
5. **Transition to live** only allowed when the bound stream's `status.streamStatus == active`.
### Two features that reshape the design
1. **enableAutoStart / enableAutoStop** — instant one-click go-live, no transition call. With `enableAutoStart=true` we never call `transition(live)`: the broadcast auto-goes-live the moment the encoder starts. Combined with `enableMonitorStream=false` (our preview pane replaces YouTube's monitor stream — the thing that forces a testing stage), the flow is **create → bind → Start Stream → encoder starts → YouTube brings it live**. No testing, no transition polling, no liveStarting stuck-state handling.
2. **`cdn.resolution=variable` / `cdn.frameRate=variable`** — free auto step-down. YouTube auto-detects what we send; since we ARE the encoder we can drop bitrate/resolution on the fly with zero API calls. Declaring an explicit resolution instead (e.g. 1080p) requires a new stream, which can't happen mid-broadcast. Variable is the enabler for the whole auto step-down feature. **(Claim-check 2026-09-01: the enabler is real, the governor is not built — the drop-side policy ships as TASK 33, v1 per the complete-v1 ruling.)**
### Compliance gotchas (maps perfectly to report-by-exception)
1. `liveStreams.status.healthStatus`: `good | ok | bad | noData` plus `configurationIssues[]` with `type` + `severity` (`info|warning|error`). Literally built for report-by-exception — poll it, render nothing on good/ok, surface a banner only on warning/error. No need to invent our own health logic.
2. Encoder must comply or YouTube flags it: keyframes ≤ 4s (`gopSizeLong`), closed GOP, H.264, audio AAC/MP3 @ 44.1/48kHz, mono/stereo only.
3. Error codes to handle: `errorStreamInactive`, `invalidTransition`, `redundantTransition`, `liveStreamDeletionNotAllowed`, `liveStreamModificationNotAllowed`, `liveBroadcastBindingNotAllowed`.
### Tips we should take advantage of
1. **Reusable streams** (`isReusable=true`): one stream per channel, cache its ingestion URL + stream name, reuse for every broadcast. No rebinding dance each go-live. This is exactly the manual-stream-key baseline.
2. **Backup ingestion address**: YouTube provides a simultaneous-push backup — future hardening, not v1.
3. **`recordFromStart` + `enableDvr` default true** → every live is auto-recorded and immediately replayable. Free VOD archive, matches the v0.2 recording goal.
4. **`latencyPreference`: `normal | low | ultraLow`** — for homelab streamers talking to chat, `low` (or `ultraLow`, capped at 1080p) is a real feature.
5. **Broadcast ID == Video ID** — one ID to track everything.
---
## TASK 1 — Initial Scaffold
**Goal:** Working C# / WPF project with MVVM architecture, dark-theme main window, and YouTube service stubs.
### Status: ✅ Done
1. ✅ Models: Scene, Source, StreamConfig, StreamHealth, YouTubeChannel, ChatMessage
2. ✅ Services: YouTubeAuthService (OAuth2), YouTubeStreamService (broadcast/health), YouTubeChatService (chat polling)
3. ✅ MainViewModel: scene management, stream controls, chat
4. ✅ MainWindow: scene/source panel, preview area, chat panel, status bar
5. ✅ Clean build, 0 warnings (WSL + Windows)
---
---
## TASK 2 — YouTube OAuth2 Authentication
**Goal:** Fully working Google OAuth2 flow — user clicks "YouTube", browser opens, authorization callback lands, channel info is stored.
### Status: ✅ Done
1. ✅ Two-state Start/End Stream button, go-live dialog (account + title/description/visibility), red top bar, pulsing LIVE badge + elapsed timer, preview glow, taskbar red dot
2. ✅ Real OAuth2 wiring — baked-in Google credentials (desktop client; loopback callback) + `YouTubeAuthService` complete: browser launch, `HttpListener` callback, token exchange, refresh, channel fetch
3. ✅ Token persistence via Windows DPAPI (`Helpers/TokenStore.cs` → `%APPDATA%\ytLlive\ytLlive.auth`), best-effort reload + proactive refresh at startup, saved after every exchange/refresh
4. ✅ Account sign-in/change surfaced in the GoLive dialog (saved account shown with "Change Account"; "Sign in to YouTube" when none; Start disabled until signed in)
5. ✅ ~~**End Livestream signs out**~~ **SUPERSEDED by TASK 18 (2026-08-29): explicit sign-out only** — `StopStream()` no longer clears the session (sign out via Logout / Change Account, `SignOutYouTubeAsync`); stopping a stream or recording leaves the creator signed in. Originally shipped as: a graceful end completes the session: `StopStream()` calls `YouTubeAuthService.ClearSession()` + `TokenStore.Clear()` + `IsConnected = false`, so the next Start Stream dialog requires a fresh sign-in. A crash never runs End, so the DPAPI token survives and the creator stays signed in. Resume/reconnect after a midstream crash is deliberately deferred to TASK 3: the socket can't be resumed (it dies with the process), so "resume" = fast reconnect with a saved broadcast ID/stream key within YouTube's disconnect-grace window; too slow and `enableAutoStop` ends the broadcast. **Orphan closed 2026-09-01:** this mechanism is owned by **TASK 32 (Stream resilience)** — in-session blip retry inside the grace window; cross-process "resume" of a dead broadcast is officially out-of-product (the API makes it impossible — creator ruling).
6. ✅ Tests in `ytLive.Tests` (xUnit, net8.0-windows): TokenStore DPAPI roundtrip/corrupt/missing/clear + mocked exchange channel-parse + refresh expiry bump + `ClearSession` — 7 passing
**Design constraint:** Sign-in must NEVER block core exploration. Users can build scenes, add sources, and audition the software without authenticating. But **going live requires authentication** — the "Start Stream" dialog is where the account sign-in lives, alongside all stream metadata.
**Two-state flow:** There is no separate "Connect" button. The top bar shows a single button — **Start Stream** when idle, **End Stream** when live. Clicking Start Stream opens one dialog that supplies everything: account (previously-saved account shown as default, with a Change Account action) + title/description/visibility.
**Live indicators (unmissable):** Top bar + window title bar flip red, a pulsing **● LIVE** badge with elapsed timer appears in the top bar, the preview area gets a red glow, and the taskbar icon shows a red overlay dot. Window title bar shows the stream title once validated.
### Requirements:
1. **Google Cloud OAuth credentials** — client ID + secret, **baked into `Helpers/OAuthCredentials.cs`** (desktop "Desktop app" OAuth client; loopback callback — no console redirect URI registration needed; creators never configure)
2. **Local HTTP listener** — `HttpListener` on `http://localhost:PORT/oauth2/callback` to catch the redirect
3. **Browser launch** — open the authorization URL in the default browser
4. **Token persistence** — store access/refresh tokens securely (Windows DPAPI), reload on startup
5. **UI state** — account shown in the Start Stream dialog; "Change Account" action triggers re-auth
6. **Go Live gated on auth** — Start Stream dialog requires sign-in to enable the Start button; scene building works without it
### Tests:
1. Mock token exchange response, verify channel info parsed
2. Verify token refresh triggers when near expiry
3. Verify credential load/save roundtrip
---
---
## TASK 3 — Capture Pipeline (Scenes/Sources)
**Goal:** Real video preview in the center panel — the minimal source set below, composited per scene.
### Status: 🔶 In progress — 27 of 30 items done (1 excluded: background removal); items 16, 17, 20 open
1. ✅ **Milestone 1 — webcam** — MediaCapture (WinRT SDK projection) with device enumeration, CPU-first frame source, refcounted `CameraManager`, picker dialog, clip shapes (Traditional + Round) + mirror, 480×270 default placement — schema v2
2. ✅ **Schema v3 (Ship Branch A)** — multi-scene webcam (singleton `Webcam` + per-scene `WebcamSceneConfig`), right-click border/context menu, static OBS-style borders, 50%-per-dimension webcam size cap, device-swap (`ReleaseAllAsync`) — 25 tests passing
3. ✅ **Schema v4** — round→rect restore persisted (`WebcamSceneConfig.RectWidth`/`RectHeight`) + one-time legacy-square 16:9 heal on load
4. ✅ **Screen backdrop (ship task #1, schema v5)** — live desktop/game capture as a permanent, non-deletable bottom layer (`Source.IsBackdrop`), auto-detecting the full-screen game at launch/focus (else the **primary display** — never assumed monitor 0) via `Win32FullScreenDetector` (now with `GetDisplays()`/`PrimaryMonitorIndex()` for the in-app display picker), content re-designated via the OS `GraphicsCapturePicker` ("Change Capture…") or the in-app "Capture Display" submenu, refcounted/shared capture sessions in `ScreenCaptureManager` mirroring `CameraManager` — 45 tests passing
5. ✅ **Schema v6 — backdrop Live-only by policy** — `Scene.HasBackdrop`, enforced by scene name on every load (`EnforceBackdropPolicy`: Starting/BRB/Chat/Ending never carry one; the one-time v5→v6 backfill covers all four), the scene context-menu "Backdrop" checkbox is gone (policy owns the flag), preview watermark hides when the backdrop renders, capture changed to `WindowsRuntimeMarshal.TryGetDataUnsafe` (the CsWinRT-safe frame-read) + downscale to the 1920×1080 master + 5s-throttled error logging (was flooding `startup.log` with 5 MB of cast errors and burning CPU), round webcam no longer re-rasterizes an `ImageBrush` every frame (Image + EllipseGeometry clip) — the live-mode stutter fix
6. ✅ **The five-scene catalog (`SceneCatalog`)** — Starting/Live/BRB/Chat/Ending is the product — work with less, never more; the (+) button only shows when a canonical scene is missing and re-adds it (its menu lists only the missing ones) — 62 tests passing
7. ✅ **Webcam-after-session-start fix** — a webcam added to a scene after the camera was already running (e.g. Chat) previously rendered a transparent container — `CameraManager.GetPreviewBitmap` + propagation in `AddWebcamToActiveSceneAsync`/`ReacquireWebcam` now hands the running shared frames to any newly added `WebcamSceneConfig` — 65 tests passing
8. ✅ **Chat scene webcam size cap** — raised from 50%-per-dimension (960×540) to half the screen AREA (~1358×764 @16:9, `MaxWebcamWidthFor`/`MaxWebcamHeightFor` keyed by canonical name) so the viewer sees the creator better
9. ✅ **"Add Webcam" always opens the camera picker** — deleting one scene's webcam then re-adding used to resurrect the old camera when another scene still used it — `SwapWebcamIdentityAsync` now swaps the app-wide identity if a different camera is chosen, same path as "Change Webcam…"
10. ✅ **Webcam resource validation + first-frame proof** — `MediaCaptureFrameSource` validates post-init (VideoDeviceId match, stream properties ≥1, `reader.StartAsync()` status read + throws on non-Success); subscribes `capture.Failed` + `CameraStreamStateChanged` → `SourceFailed` event on the seam; fallback ladder (VideoPreview → VideoRecord). `CameraManager.AcquireAsync` requires first-frame proof (4s timeout): returns true only after a real frame arrives — silent empty box impossible. `MainViewModel` subscribes `CameraFailed` → red `WebcamError` chip in preview + MessageBox names suspect apps (`CameraConflictProbe`). 19041 SDK projection gaps: `Exclusive`/`DeviceLost` not projected; `CameraStreamState.Failed` compared by `(int)2`. 81 tests passing
11. ✅ **Scenes/sources UI** — add/reorder/rename, image + background overlays with move/resize/opacity/reuse
12. ✅ **Audio UX shipped (UI)** — the bottom-bar footer is now two lines (dropped/duration moved under bitrate/fps), with the mic's sound meter + mute button + volume slider grouped CENTERED on the footer's top line, beneath the preview panel (meter: 288px, muted slate track with ruler graduations + muted yellow/red zone tints, green→yellow→red fill; mute = speaker icon → red do-not-symbol when muted, and the slider and speaker stay in sync (volume 0 ⇔ muted — sliding off flips the speaker to muted, sliding up from 0 clears it); mic volume defaults to 80%, muting zeroes the meter and restores the prior volume on unmute (which flashes the meter to the restored position ~300ms before it returns to the live level); the meter is a READ-ONLY realtime level display (fill = live level × volume — volume is a gain on ambient noise; while the slider is dragged the bar previews the slider position and bounces back to the live level on release, which is 0 with no input — clicking the meter does nothing), clicking the MIC label opens a microphone picker whose chosen source shows left-justified inside the meter bar (fill at 75% opacity so the name + ruler markings show through); slim dimensional slider — gradient track/fill, gloss-sphere thumb; the old flat pink 18px-filled one is gone), everything else on line 2 (bitrate/fps/dropped/duration/health left, quality + gear right) — the creator's only audio control, desktop/game audio is automatic (KISS rule). **Post-test follow-up (2026-08-13, game audio bar branch):** the **game audio bar** (desktop/game, a mirror of the mic bar: meter + mute + volume) is **overlaid at the bottom of the preview window** (bottom-center dark chip — monitoring UI only, never on the live output) and appears only while a **full-screen game is producing sound** in the preview (`IGameAudioDetector` seam + default `GameAudioDetector` polling `IFullScreenDetector` + the live loopback level, floor 0.5%, into a pure `GameAudioHysteresis`: SHOW after ~500ms of fullscreen+sound, HIDE ~1s after leaving fullscreen, **silence never hides an active bar**; the VM polls it on a 250ms `DispatcherTimer`); **capture now runs for the app's lifetime** (mixer started once at startup via `StartMicCaptureAsync`, disposed in `Shutdown` — not go-live) so the meters preview live; the **MIC label became a button with a status dot** (`Models/MicStatus`: green = connected via the source `Started` event, yellow = mic problem, red = no device); picking a mic takes effect immediately (`AudioMixer.RestartMic`, loopback keeps running); fixed a latent `?.Invoke(meter.Push(...))` short-circuit that skipped the meter update when nothing was subscribed. **Round 2 (2026-08-13):** the meters were dead on a flat scale (real speech/game RMS is ~0.01..0.1 linear) — the raw level is now mapped via `AudioLevelMeter.ToDisplay` (−60..0 dBFS spread across 0..1) so typical input reads ~1/3..2/3 of the bar at default volume. 169 tests passing, 0 warnings. **Round 3 (2026-08-13):** the mic bar gained a **mic mute icon** (a microphone glyph in the speaker's 16px style, red + slash when muted) between the meter and the speaker — both mutes adjacent with spacing between them, same `ToggleMicMuteCommand`; and the top-center LIVE badge became an **always-visible REC sign** — dark gray dot + dim "REC" offline, bright red (#e94560) + "REC" + elapsed while live, **darker red (#8f1f1f)** when live with a **private** stream (driven by the dialog's chosen visibility) — `RecDotBrush`/`RecTextBrush`/`RecDotOpacity`/`IsLivePrivate`, pulsing while live. **Queued:** task 22 (voice filters). Voice-filter note: the meter's `ToDisplay` input is the pre-filter mic level; when filters land they must apply BEFORE the meter/encoder mix
13. ✅ The connected YouTube account's avatar/name shows in the top bar next to Start Stream (`SyncConnectedAccount`); the scenes list is content-height now (no dead space before SOURCES)
14. ✅ **Social bar v2 (six-slot dialog, sign-in gate, real logos)** — global bar layer (never a Source, no sources-list row), content-sized, centered, GREEN glow when ON, top/bottom snap-drag (default BOTTOM, persisted `SocialBarPosition`; drag clamps to 0/1040, tie→bottom). Footer Social button gets a state dot (green=ON). Dialog "Social Media Site Promotion" (`SocialsDialog` + `ViewModels/SocialsDialogViewModel`, WPF-free + injected `ISocialValidator`/sign-in/sign-out fakes): ON/OFF bar switch (`SocialsConfig.BarEnabled`, schema v8), 6 fixed slots — row 1 always YouTube (signed-in → channel handle; signed-out → sign-in gate → OAuth; delete → confirm sign-out, mirrors `StopStream`), row 2 free, rows 3-6 lock icons on freemium (Premium seam: all six). Validation: `DetectService` (URL domain / fediverse `@user@domain` / bare→Website) → async `ISocialValidator` on confirm/Save; valid snaps to text + real service logo (bundled SVG path data via `LogoDataFor`, Simple Icons CC0 — initials badges gone); invalid → red do-not, stays editable, Save blocked. LCR justify dropped (`BarJustify` unread), per-scene toggle dropped (`Scene.HasSocialBar` back-compat). **Post-test fixes (2026-08-12):** footer label "Socials" (not "Social"); fediverse `@user@domain` validates — the full handle is the identity end-to-end (`DetectService`/`CanonicalUrlFor`/`HttpSocialValidator` build `https://domain/@user`, no domain loss); **Cancel is a hard stop** — `ISocialValidator.LookupAsync` takes a `CancellationToken`, dialog VM owns a CTS, Cancel/X/Save abort in-flight lookups (HTTP request killed, canceled continuations never touch slot state), and `ConfirmEdit` skips re-submitting identical text (LostFocus on dismiss never re-fires a lookup). `HttpSocialValidator` now has real tests (fake `HttpMessageHandler`). 105 tests passing. **Post-test fixes (2026-08-12, round 2):** fediverse `@user@domain` no longer shows a generic chain — it resolves to the instance's actual software via nodeinfo (`/.well-known/nodeinfo` → `software.name`; `SocialService.Fediverse` enum member + `SocialEntry.FediverseSoftware` persisted in a new `SocialEntry.Software` column, schema migration by column-presence) and renders that software's bundled logo (`LogoDataForFediverse`: mastodon/peertube/pixelfed/misskey/lemmy/pleroma/firefish, generic fediverse honeycomb fallback). Dialog row-2 edit/trash icons were too dark — `IconButton` style gains `Foreground=#d0d0d0`; trash overrides `#e94560` (app red). 112 tests passing. **Post-test fixes (2026-08-12, round 3):** a fediverse handle whose identity domain is itself a redirect (e.g. YunoHost default-app subdomains — `@user@llamachile.tube` where the mastodon instance lives at `mastodon.llamachile.tube`) now still resolves its software: nodeinfo on the identity domain is SSO-blocked, so `HttpSocialValidator` follows the bare root `https://domain/` 302 to the real instance host and re-runs the nodeinfo lookup there.
15. ✅ **Window capture** — absorbed into the Screen picker (no separate source type); dedicated window-as-source work is pending
16. ❌ **Scene compositing (D3DImage/MediaElement)** — SUPERSEDED, permanently out (2026-09-01): the software output compositor (TASK 4 step 1) + the XAML preview ARE the design; a D3D11 swap stays a possibility behind the `VideoFrame` seam, not a feature
17. ☐ **Text source** — live text ("Starting soon", "Back in 5", handle, callout)
18. ✅ **Chat box** — YouTube live chat rendered *on* the stream so viewers read along in-video — `ChatBoxRenderer` (WPF FormattedText → VideoFrame), configurable font size/color/badges/timestamps/max-messages via Elements panel, schema v10, persisted across save/load
19. ❌ **Background removal (milestone 2)** — ONNX Runtime + DirectML, MediaPipe Selfie Segmentation — deliberately NOT in this build
20. ☐ **Alerts** — Super Chat / membership / subscribe pop-ins; build after the six; **free and ungated** (the stale "one paid feature" label predates the 2026-08-31 monetization reversal and is corrected 2026-09-01 — the paid swap is the branding flash ONLY; see Monetization in `ai.md`). **v1 IN** per the complete-v1 ruling. **Precursor — reward-event capture (2026-09-01):** Alerts render from the app's canonical `RewardEvent` feed (all 7 `liveChatMessage` reward types, persisted to SQLite + the session report — see TASK 10 → "Related work — monetization awareness"), so it consumes that already-built data rather than re-integrating `liveChatMessages`. That capture is the required prior milestone.
21. ✅ **Show Desktop toggle** — `Source.ShowDesktop` bool persisted in DB (schema migration + LayoutStore read/write). When checked, primary monitor captures regardless of fullscreen game state. Toggling off falls back to running game or releases to static placeholder. `ClearBackdropCaptureAsync` clears `CaptureKey` + `VideoImageSource` so static fallback renders. Toggle on Live backdrop context menu (IsCheckable MenuItem).
22. ✅ **Top bar — status light + avatar + Log In button** — red/green/pulsing ellipse (disconnected/connected/live). Avatar border visible only when connected, loads via `BitmapImage` code-behind. "Log In" button visible when disconnected, calls `StartStreamCommand` → GoLiveWindow. `ShowStartStream` now requires `IsConnected`.
23. ✅ **Chat preview — one-at-a-time mock messages** — `RunMockChatPreviewAsync`: simple async loop, displays `MockChatMessages[n]` via `Take(n+1)`, 1000ms between messages, wraps at end. `ChatPreviewEnabled` property on Source (default true). Stops on real messages, restarts on fade timer clear.
24. ✅ **All backdrops seeded + EnsureDefaultBackdrop** — Live, Chat, Ending backdrops added (ending-backdrop.jpg replaced). `EnsureDefaultBackdrop()` called on every scene switch. `SeedEndingBackdrop` replaces existing (not skip-if-exists). Chat scene no longer skipped.
25. ✅ **SceneCatalog fix** — Chat scene display name corrected to "Chat" (was "YouTube Chat" — that name belongs to the layer/SourceType.ChatBox, not the scene).
26. ✅ **Left panel spacing + context menu reorder** — HR margins matched, Row 4 changed from `*` to `Auto`. Context menu reordered: Border Thickness first, then Opacity, Color, Effect.
21. ✅ **Logo + richer in-app About (2026-08-13, queued → 2026-08-14 SHIPPED)** — the ytLlive wordmark in the top bar opens the About overlay (already wired); the About overlay is now the **creator hub**: the real logo (the "llama fortnite superman logo" from the creator's vault, copied to `Assets/llama-logo.png` — the creator's own art, no third-party license), plus **creator-hub links** — llama chile shop on YouTube (`MainViewModel.ChannelUrl`), Mastodon (`https://mastodon.llamachile.tube/@gramps`), **Buy me a coffee** (`https://buymeacoffee.com/llamachiley` — live), and **Unlock Premium** (greyed "coming soon" — the **billing** product URL is a tabled seam, `PremiumUrl`, until TASK 10 picks the subscription provider). A **Licenses & legal** button flips the About overlay to an in-app scrolling panel that loads the full `THIRD-PARTY-NOTICES.txt` text (`MainViewModel.ShowLicenses` reads the shipped file from the executable directory on first open; graceful "not found" fallback — **never the OS viewer, everything stays in-app**); "← Back to About" returns. The About overlay is also the in-app home of the notices — the top-bar About button that opened the file in the OS viewer was removed on 2026-08-13 for exactly this reason. Integration test (Good Dog Rule — ONE): `AboutHubTests.About_Opens_InApp_Licensing_Loads_Shipped_Notices` drives the real window + VM, asserting the hub opens, the link URLs are real, the licensing panel loads the shipped notices text (contains "Third-Party Notices" + "LGPL"), and back returns to the hub. 174 tests passing, 0 warnings
22. ✅ **Voice filters on the mic channel (2026-08-13 queued → SHIPPED 2026-08-14 inside TASK 9, the audio milestone)** — the standard four applied to the sound input path (before the meter/encoder mix): **bass boost, treble, noise suppression, compressor** (set decided with the creator 2026-08-13). Noise suppression = a **pure-C# noise gate** (creator chose over RNNoise / a second ffmpeg `afftdn` pipe, 2026-08-14 — KISS). Always-on — no UI knobs; the mic stays the creator's single audio control
27. ✅ **Webcam row gates on the app-wide identity (2026-08-24, TASK 26)** — the (+) menu's Webcam item greys out whenever a webcam exists **anywhere** (`CanAddWebcam` = `StagedScene != null && _webcam == null`; renamed from `CanAddWebcamToStagedScene` whose per-scene rule let a second picker run from a scene lacking the config), raised at both `_webcam` mutation sites (create / last-config removal) + scene staging + elements change. Creator's visual pass found it: minis/no-capture-rows/capture-controls all good. ONE integration test `WebcamMenuGateTests.CanAddWebcam_Gates_On_The_AppWide_Webcam_Identity` (real window + temp DB seeded with a webcam in Starting; asserts greyed while Live staged, re-enabled after `RemoveSourceCommand` clears the last config + the `Webcam` DB row). Stale map fixed in the same commit: ai.md's "empty-canvas right-click Show Webcam" claim dropped — that XAML never shipped (`ShowWebcamCommand`/`CanShowWebcamInStagedScene` are wired but unbound dead code, audit item). 224 tests (223 pass; the pre-existing AudioPipelineTests failure is unrelated)
28. ✅ **YouTube Chat layer: one-per-layout gate + legacy label heal (2026-08-24, TASK 27)** — same rule as TASK 26 applied to the chat layer: the (+) menu's YouTube Chat item greys out while any scene carries a ChatBox source (`CanAddYouTubeChat`, raised on staging + elements change; `AddSource` refuses a second), tooltip "One chat layer at a time — it's already in your stream". Plus the creator's label fix: layers added by commit `65641d8` were named "Chat"; LoadLayout now heals exactly that un-renamed default to "YouTube Chat" so the Layers row matches the (+) picklist (creator renames untouched, idempotent). ONE integration test `ChatLayerGateTests.LegacyChatName_Heals_And_CanAddYouTubeChat_Gates_On_The_Existing_Layer` (real window + temp DB with a legacy "Chat" row: healed on load + persisted on save, greyed cross-scene and in-scene, re-enabled after remove). 225 tests (224 pass; pre-existing AudioPipelineTests failure unrelated)
29. ✅ **Broadcast metadata pull-out + launch geometry (2026-08-24)** — creator ask: "a tab-pullout on the right side of the preview pane — white tab, YouTube-red label reading 'Text' — opening a form with all settable liveBroadcast fields; fields that can't be filled before launch greyed out; saving/updating remote content; field data saved and pre-loaded on app run". Shipped as specified with one correction: the API is the inverse of "more fields once live" for contentDetails (those lock in created/ready) — the pull-out carries the always-editable snippet/status set: Title, Description, Tags (csv), Visibility (private/unlisted/public), Made-for-Kids, plus read-only Scheduled Start. **Always visible** (creator revised same day from Live-only gating). White 30px tab, rotated red "Text", right edge vertically centered; click slides a dark 320px drawer left over the preview (200ms CubicEase). Every edit persists to Settings keys (`Broadcast.*`) immediately; **Update Broadcast** button calls the new `YouTubeStreamService.UpdateBroadcast` (PUT `liveBroadcasts?part=snippet,status`, echoes scheduledStartTime because update replaces the whole snippet part). Go Live prefills from the form and captures what was inserted (`CaptureGoLive`). The old un-persisted Default Stream Title/Description fields came OFF the App Settings overlay (replaced by the form); `DefaultStreamTitle/Description` VM properties deleted. **Launch geometry** (same unit): default 1920×1040, MinWidth 1366, MinHeight 768; `WindowStartupLocation=Manual` + window size/position persisted on close via `RestoreBounds` (maximized-safe), restored in ctor clamped to minimums and the primary work area (disconnected-secondary fallback). ONE integration test `BroadcastPullOutTests.Metadata_Persists_WindowRestores_Clamped_And_UpdatePatchesRemote`. 228 tests (227 pass; pre-existing AudioPipelineTests failure unrelated)
30. ✅ **Gear menu cleanup + Default Location for Recordings + single Start button (2026-08-29)** — **KISS: removed** Gear-menu `Save Layout`/`Save Layout As…`/`Open Layout…` (redundant with auto-save; multi-layout = OBS) + dead `SaveLayoutCommand`/`SaveLayoutAsCommand`/`OpenLayoutCommand` + `SaveLayoutAs()`/`OpenLayoutFile()` methods. **App Settings panel:** first real setting — `Default Location for Recordings` (configured folder via `ChooseRecordFolderCommand` + "Use Downloads" reset via `ResetRecordFolderCommand`, resolved path shown). Fallback chain: configured → `%USERPROFILE%\Downloads` → `MyVideos`. `StartRecordFile()` + `ChooseRecordFolder()` `InitialDirectory` updated to use the new fallback. **Single Start button:** `PrimaryButtonText`/`IsStreamingStart` + their `OnPropertyChanged` calls removed; button `Content="Start"` in XAML — pills are the intent surface, button is always just Start. `Choose Record Folder` fast-path kept in the button context menu. Monetization (About links/premium) deferred to a later task. ONE fallback test `DefaultRecordFolder_Falls_Back_To_Downloads`. build 0 warnings, 247 tests (2 known failures, pre-existing).
### The Minimal Source Set (design decision — do not expand casually)
ytLlive is YouTube-only and 90% of users are casual. OBS's long source list is off-putting; we ship
the hot few and nothing esoteric. If a user needs more, they've graduated to OBS.
1. **Webcam** — the face cam. Non-negotiable.
2. **Screen** — the main event (game, slides, browser). One source; a picker chooses a monitor *or* a
window. (Window capture is absorbed here — no separate source type.)
3. **Background** — a full-canvas backdrop image. Fills the whole scene automatically, zero fiddling.
Kept separate from Image on purpose: same pixels, but this one needs no positioning.
4. **Image** — a floating graphic/logo overlay (watermark, badge, corner branding). Free-positioned.
5. **Text** — live text ("Starting soon", "Back in 5", handle, callout). Casual streamers live on this.
6. **Chat box** — YouTube live chat rendered *on* the stream so viewers read along in-video. YT-native.
7. **Alerts** — Super Chat / membership / subscribe pop-ins. The dopamine source. **The one big lift**
(Super Chat event streaming + on-stream rendering/animation); build after the six. **Free and
ungated** — paid = flash removal only (stale "one paid feature" corrected 2026-09-01). Consumes
the canonical `RewardEvent` feed (reward-event capture, TASK 10 → related work).
Deliberately NOT supported: game capture, browser source, media playlist, VLC, color-key voodoo, MIDI.
### Source memory model (design decision)
1. A scene has resources. Resources can be shared across scenes.
2. A resource exists exactly once in memory, no matter how many scenes use it (a logo in five
scenes = one loaded bitmap).
3. Every resource carries a **catalog of scenes**: one usage entry per scene it appears in, each
entry dictating that scene's use — placement (X/Y/Width/Height), opacity, z-order, enabled,
scale mode, crop.
4. Usages are named `{resourceName}.{sceneName}` — whatever the user named the resource, dot, the
scene name: `logo.starting`, `logo.live`, `myPic.brb`. Not a hardcoded "logo".
5. A webcam in two scenes = one capture session, two catalog entries.
6. Refcount by catalog size: the last usage removed → the resource is disposed and evicted.
7. The resource (not a per-scene node) owns everything `IDisposable`.
### Scene transitions (design decision)
Scene switching while live must never stutter. Supported types, most → least economical:
1. **Cut** — instant switch. The default. Zero cost.
2. **Fade** — short crossfade (~300ms).
3. **Move** — a simple, economical move transition, done to perfection and memory-efficient. The
smart streamer's bread and butter.
4. **Custom (media) transitions** — require media elements (video/stinger playback during the
transition). Heavier, but creators pay for these, so we support them. Their media follows the
same resource memory model: loaded once, catalogued by scene.
Preview shows the transition too (WYSIWYG). No wipes/slides/LUTs beyond the four above.
### Requirements:
1. **Screen** — Windows.Graphics.Capture (WinRT), enumerate displays/windows, picker
2. **Webcam** — MediaCapture (WinRT SDK projection) with device enumeration — ✅ **milestone 1 done**:
1. TFM bumped to `net8.0-windows10.0.19041.0` (app **and** tests) so the WinRT projection resolves from the SDK reference packs — no NuGet package, no capability manifest (unpackaged desktop app)
2. `MediaCaptureFrameSource` (CPU-first: `MemoryPreference = Cpu`, BGRA8 via `CreateFrameReaderAsync`), `MediaCaptureCameraEnumerator` (`DeviceInformation.FindAllAsync(DeviceClass.VideoCapture)`)
3. `CameraManager`: refcounted by `DeviceId`, one shared `WriteableBitmap` app-wide, dispatcher-coalesced UI updates (~render rate, latest-frame drop), placeholder/`AppLog` + warning on failure
4. `CameraPickerDialog` (mirror of `ReuseImageDialog`) — "Searching for cameras…" / list / "No cameras found" states
5. One webcam app-wide: Add → Webcam greyed out once one exists ("it's already in your stream" tooltip); persisted `DeviceId` re-acquires after layout load
6. Default placement 16:9 **480×270**, bottom-right, 32px margin; drag/resize/selection shared with Image sources
7. **Clip shapes: Traditional + Round** (phone view dropped — the 9:16 phone output is the vertical output-crop tier); **mirror**; both persisted in the layout DB (schema v2) and toggled from the source chip
8. **Background removal = milestone 2** (ONNX Runtime + DirectML, MediaPipe Selfie Segmentation) — not in this build
3. **Background / Image / Text** — static sources positioned/scaled/opacity
4. **Chat box** — rendered from the live chat poll (right panel is the same feed, raw)
5. **Scene compositing** — per-scene source layering (z-order = sources list order, top-to-bottom
back-to-front), preview rendered via D3DImage or MediaElement
6. **Branding flash** — the topmost full-frame "made with ytLlive!" layer at ~25% opacity, ~1s on /
300s off (see Monetization in `ai.md`), gated on `BrandFlashEnabled` + live/recording. Lives in the
preview compositor now (`BrandFlashLayer` in `MainWindow.xaml` CanvasGrid, driven by
`BrandFlashActive`/`BrandFlashTimer` in `MainViewModel`); the encoder output renders the same layer,
and v0.2 local recordings carry it too
7. **Drag/drop placement & reorder** — intuitive, visual (per design principle):
1. **Preview:** click-drag a source in the center panel to reposition it; resize via handles
2. **Scenes list:** drag rows to reorder scenes
3. **Sources list:** drag rows to reorder sources (this *is* the z-order) — implemented
---
---
## TASK 4 — RTMP Ingest to YouTube
**Goal:** Push encoded video to YouTube's RTMP ingest.
### Status: ✅ Done — all 7 items shipped
1. ✅ **Ship step 1 — the output compositor SHIPPED** (2026-08-10)
2. ✅ **Ship step 2 — the FFmpeg locator SHIPPED** (2026-08-10)
3. ✅ **Encoder + RTMP push SHIPPED** (2026-08-12) — the FFmpeg subprocess: raw BGRA frames via stdin, stderr health parsing, FLV mux + push to the ingestion URL (see the ship step 3 plan below)
4. ✅ **WASAPI audio capture SHIPPED** (2026-08-12) — NAudio loopback (desktop/game) + the picked mic feeding `AudioLevel`, so the realtime meter comes alive (see the ship step 4 plan below)
5. ✅ **Frame-pipeline wiring SHIPPED** (2026-08-12) — `CameraManager`/`ScreenCaptureManager` → compositor resolver → encoder, driven by a paced `FramePump` (see the ship step 5 plan below)
6. ✅ **Health stats SHIPPED** (2026-08-13) — `FramePump.HealthUpdated` (encoder's parsed bitrate/FPS/dropped/duration, already forwarded from `FfmpegEncoder.OnStderrLine`) now lands in the bottom bar: `MainViewModel.OnFramePumpHealthUpdated` marshals to the UI thread (the stderr loop raises on a background thread) and copies into `CurrentHealth` (the bottom bar's existing binding); `ResetHealth` zeroes dropped/duration on go-live and on End so stats never linger from a previous session (bitrate/FPS stay on the tier's targets). The bar lights up with real values once TASK 9 supplies the RTMP URL (until then the pump skips the encoder and the bar shows the tier's targets)
7. ✅ **One-click go live + private-only enforcement SHIPPED** (2026-08-14) — the Go Live dialog is **locked to Private** (no dropdown, `GoLiveViewModel.Visibility` is a get-only "Private"); `YouTubeStreamService.CreateBroadcast` **always sends `privacyStatus = "private"`** (dialog + service enforcement, requirement 8 — nothing can go out non-private) and gained an injectable `HttpClient? http = null` seam for tests; `BeginGoLive` now calls `CreateBroadcastAsync` and remembers `_currentBroadcastId` for TASK 9's bind/transition (failure → `StreamStatus.Error`, never a crash; `StopStream` clears the ID); the REC sign shows a **PRIVATE badge** (dark-red border, next to REC, `IsLivePrivate`) when the live stream is private; settings' dead "Default Visibility" dropdown + `MainViewModel.Visibilities`/`DefaultStreamVisibility` removed. The broadcast-insert integration test asserts the request body carries `"privacyStatus":"private"` (2 new tests → 173 passing, 0 warnings)
The pipeline chain the encoder needs doesn't exist yet: **scene compositing** (the master 1920×1080 frame
without the preview's editing chrome) → **audio capture** (WASAPI, feeds the meter) → **H.264+AAC encode**
→ **vertical-tier crop/scale** → **RTMP push** → **health stats** into the bottom bar. Nothing can encode
until a frame source exists, so the compositor is ship step 1. The pipeline is
`CameraManager + ScreenCaptureManager → compositor resolver → compositor → encoder → RTMP`.
### Requirements:
1. **Encoding** — H.264 (hardware via NVENC/AMD, fallback x264) + AAC audio; **must comply**: keyframes ≤ 4s (gopSizeLong), closed GOP, AAC/MP3 @ 44.1/48kHz, mono/stereo only. **License posture (decided): GPL-free build** — NVENC (NVIDIA) / QSV (Intel) / AMF (AMD) + OpenH264 software fallback + built-in AAC; no libx264 (GPL contaminates a paid product). Output containers are identical either way (H.264+AAC in `.flv` for RTMP, `.mp4`/`.ts` for VOD) — the format is NOT the differentiator, the license and per-GPU quality are. **License guardrails (never violate — see `ai.md` → "Licensing — do not violate"):** only BtbN `lgpl`/`lgpl-shared` builds; never GPL (gyan.dev) or `nonfree` (fdk-aac); never static for distribution (LGPL §6 relink material); never link FFmpeg into the app; never drop `THIRD-PARTY-NOTICES.txt` from the app/About screen.
2. **RTMP push** — **FFmpeg subprocess (decided)**: app feeds raw frames via stdin, parses stderr for health; one battle-tested binary does encode + FLV mux + push. **No self-heal (claim-check 2026-09-01): ffmpeg's reconnect flags are input-side; an RTMP *output* push does not reconnect itself — retry is app-side, owned by TASK 32.** **Binary distribution (decided): check-then-pull** — probe `where ffmpeg`/PATH at first go-live; if absent, download a **pinned** build (**BtbN LGPL-shared win64** zip, ~75 MB — gyan.dev's builds are GPLv3 and ship libx264, which violates the license posture; BtbN's LGPL variant drops x264/x265 while keeping NVENC/QSV/AMF + libopenh264 + native AAC) to `%APPDATA%\ytLlive\tools\ffmpeg.exe` (extract `ffmpeg.exe` **plus the `libav*.dll` family**) and cache it, offline-friendly. Behind an `IFfmpegLocator` seam so tests fake it (ship step 2, below). Push goes to the cached reusable stream's ingestion URL
3. **Quality ladder** — the offered tiers, with **1080p60 @ 8 Mbps as the standard/default**:
1. 720p30 @ 6 Mbps
2. 720p60 @ 6 Mbps
3. 1080p30 @ 8 Mbps
4. **1080p60 @ 8 Mbps** (default — mainstream ceiling, GPU hardware-encoded so the gaming
machine never notices; upload headroom stays comfortable)
5. Vertical 1080×1920 @ 60fps @ 8 Mbps (9:16 phone tier)
The composition master is always 1920×1080; a tier is an output rect + target resolution
(see `ai.md` "Resolution tiers"). Vertical output = the centered 607×1080 crop of the master
scaled to 1080×1920 (semi-crop preview is already implemented; the encoder applies the same rect).
1080p60 is the ceiling by design — "if you want 1440 or 4K or 8K → OBS is your solution"; the app
targets the most mainstream creator, not power users.
Ladder is sculpted by a **cached probe** (IP-only TCP vs public ingest host; no auth required).
Quality is greyed out while live because the declared resolution can't change mid-stream — but
with `variable`, we can **auto step-down** bitrate/resolution on the fly with zero API calls
(no stream recreation); 60fps presumes a hardware encoder — no hardware encoder → auto
fallback to 720p60/1080p30
4. **Stream key management** — reuse the cached reusable stream (one per channel) instead of creating a new one per go-live; prefill default YouTube ingest URL `rtmp://a.rtmp.youtube.com/live2`
5. **Health stats** — bitrate, FPS, dropped frames reported live in the bottom bar (encoder-side)
6. **One-click go live** — defaults that work out of the box
7. **Audio capture (feeds the meter — this task ships the wiring)** — WASAPI loopback (desktop/game at unity, zero UI — "it just is") + the picked mic (`MicSourceName` from the `MicPickerDialog`). The mic capture feeds `AudioLevel` so the realtime meter comes alive (today it reads 0 — the mixer feed is pending, see `ai.md` audio notes). AAC mono/stereo @ 48 kHz per the compliance rules.
8. **Private-only go live until v1 (reputation guard, decided 2026-08-10)** — until the v1 release, go-live is **locked to private streams only** so a software error can never publish something public/unlisted that damages the creator's reputation. RTMP push itself has no privacy — privacy lives on the YouTube **live broadcast object**, which this app already controls via its OAuth API calls. So the lock is purely API-side: the Go Live flow always creates/updates the broadcast with `privacyStatus = "private"` and a guard **refuses** to set anything else (same spirit as the Live-only backdrop policy). The UI shows a clear "PRIVATE" badge next to the stream state so the creator always knows who can see them. Enforcement must be verifiable in the auth-service tests (fake the broadcast-insert/update call, assert `privacyStatus` is forced to private).
9. **v1 release gate: bundle the full license texts (decided 2026-08-10)** — `THIRD-PARTY-NOTICES.txt` currently links the canonical license texts rather than embedding them. At the **v1 (GA) release**, the full texts of every license it names (LGPL v2.1+, BSD-2-Clause, MIT, Apache-2.0) MUST be bundled alongside it (shipped in the app output, e.g. a `licenses/` folder next to the notices file, still reachable from the About screen). This is a **release blocker for v1, not a task to queue early** — do it in the release pass. The repo should treat this like the private-only go-live gate: a checkbox that cannot silently lapse.
#### Ship step 1 — Scene compositor (the frame source)
**Goal:** a pure-CPU software compositor producing the encoder's master frame (BGRA8, the `VideoFrame`
seam) from the scene model. The preview stays XAML (the editing view); the compositor is the **output
view** — WPF's `RenderTargetBitmap` can't be used (software-rendered + captures chrome). Two renderers
must agree, so the XAML (`MainWindow.xaml` CanvasGrid + element DataTemplate) is the contract.
**Decisions (locked 2026-08-10):** **Path A CPU blitter** — GPU effort belongs to NVENC (the encoder),
not composition; with an FFmpeg subprocess the master crosses a CPU readback to the pipe every frame
anyway, so GPU compositing buys ~nothing at this layer count (2-3 live layers; static layers
pre-composite once). A D3D11 compositor can replace this one later **behind the same seam** (the CPU
master buffer stays the contract). **Render the output rect directly**: compositor is constructed with
`CompositorOptions {SourceRectX/Y/W/H, OutputWidth, OutputHeight}`; 16:9 tiers = full 1920×1080 1:1;
vertical (9:16) = composite the centered 607×1080 crop then bilinear-upscale to 1080×1920. Reuses
`MainViewModel.OutputRectX/Y/W/H` (note `(1920−607)/2 = 656.5` → align to integer pixels for output).
**Render spec (back → front, mirror the XAML exactly):**
1. Backdrop — the Live scene's `IsBackdrop` Source (`CaptureKey` → live frame), `UniformToFill`
full-frame (XAML's separate `BackdropImage` layer; the backdrop *element* renders nothing — its
DataTemplate Image is Collapsed for DisplayCapture).
2. Background — the scene's `Background` Source, `UniformToFill` full-frame (the `ActiveBackgroundImage`
layer, not per-element).
3. Elements in `Scene.Elements` order (back→front), skip `IsVisible=false`. What actually renders:
1. `Source` Type `Image` → static asset, `UniformToFill` cover-crop into (X, Y, W, H)
2. `WebcamSceneConfig` → latest frame by `DeviceId`: Traditional = `UniformToFill` rect; Round = circle
diameter `min(W,H)` (alpha 0 outside — true circle, not oval); mirror = horizontal flip around
element center (`MirrorScale`); opacity = per-pixel multiply (content + border); border = stroked
rect / centered circle at `RoundBorderSize`, width `BorderWidth`, alpha `BorderOpacity`
3. `Background` / `IsBackdrop` / `TextOverlay` are NOT per-element (layers above; Text not shipped)
4. Branding flash — pre-rendered full-frame "made with ytLlive!" at 25% alpha when live +
`BrandFlashEnabled` + timer active. Passed in as a `VideoFrame?` (compositor core stays pure byte-math,
no WPF; likely a bundled asset rather than runtime text rendering).
5. NOT in output (preview chrome only): SelectionOverlay, DimRects, output-rect outline, badge, placeholder.
**New files (all in `Services/Compositor/`):**
1. `SceneCompositor.cs` — `Render(Scene, frameFor: Func<SceneElement, VideoFrame?>, flashFrame:
VideoFrame?, CompositorOptions) → VideoFrame` (output-sized). The caller's `frameFor` resolver maps
each element to its frame (webcam → DeviceId, image → AssetId via `StaticPixelCache`, backdrop →
CaptureKey) — the compositor stays pure/hermetic/no WPF.
2. `CompositorOptions.cs` — source-rect + output W×H.
3. `StretchMath.cs` — `UniformToFill` cover-crop, ellipse mask, bilinear scale (pure, unit-tested).
4. `StaticPixelCache.cs` — asset `byte[]` → cached BGRA `VideoFrame` (WPF `BitmapDecoder` + `CopyPixels`,
decode once per content hash).
**Test plan (Good Dog Rule — ONE integration test):** `SceneCompositorTests` — a scene with backdrop
(solid red fake frame) + round webcam (solid green) + image (solid blue) → render 16:9 master → assert
per-layer probe pixels (corner = backdrop color, element center = webcam color, outside the round clip =
backdrop color, mirrored element swaps left/right); a vertical-tier variant asserts 1080×1920 output +
crop fidelity. Focused unit tests on `StretchMath`. Tests push frames directly — no capture managers
involved (they wire in a later step).
**Same-PR housekeeping:** fix the stale comment `MainViewModel.cs:324` ("shown under the meter on line 2"
→ "shown left-justified INSIDE the meter bar" — `ai.md` is the authority); this task's requirements now
include the explicit audio-capture/meter wiring (#7 above).
**Out of scope (later ship steps):** FFmpeg locator + license posture (covered in requirements 1-2),
encoder + RTMP push, WASAPI audio capture (loopback + mic) feeding `AudioLevel`, wiring
`CameraManager`/`ScreenCaptureManager` into the frame pipeline, brand-flash timer wiring, health stats
(bitrate/FPS/dropped).
**Built (2026-08-10):** all four files shipped in `Services/Compositor/`, `SceneElement.TryGetBorderColor`
made public (shared hex parse with the compositor — no duplicated color parsing), the stale
`MainViewModel.cs:324` comment corrected, and the pre-existing CS1998 in `YouTubeAuthServiceTests`
cleaned up — build **0 warnings**. Tests: the `SceneCompositorTests` integration test (full-scene master
pixels, vertical tier, flash) + 4 `StretchMath` units — **72 passing**.
#### Ship step 2 — FFmpeg locator (the encoder's binary)
**Goal:** resolve a usable `ffmpeg.exe` on demand (the encoder's one external dependency), never shipping
a binary in the repo. Returns an absolute path; downloads only when neither PATH nor the local cache
provides one.
**Decisions (locked 2026-08-10):**
1. **BtbN LGPL-shared win64 build** — not gyan.dev (gyan's "essentials" is GPLv3 and ships libx264, which
violates requirement 1's license posture) and **not the static lgpl build**: LGPLv2.1 §6 wants
relinkable object files for static linking, but the **shared** (dynamic-DLL) variant sidesteps that —
compliance is "license text + source offer + unmodified binaries" (see `THIRD-PARTY-NOTICES.txt` and
`ai.md` → Licensing). Drops libx264/libx265 while keeping NVENC/QSV/AMF, libopenh264 (the LGPL-legal
H.264 software fallback) and native AAC — exactly the requirement-1 encoder profile.
2. **Pinned URL** — **RE-PINNED 2026-09-01**: `https://github.com/BtbN/FFmpeg-Builds/releases/download/autobuild-2026-08-31-13-27/ffmpeg-N-126342-gf88b741dbf-win64-lgpl-shared.zip`
(BtbN **lgpl-shared** variant, ffmpeg N-126342; ~75 MB zip — earlier "~30 MB" estimate corrected). A dated autobuild tag is immutable; BtbN retention
keeps the last 14 daily builds + each month-end build for 2 years, so a cold cache after retention
expiry 404s — a logged, recoverable failure (the seam throws; the encoder step surfaces it). The
first pin (`autobuild-2026-08-09-13-03`, a daily) aged out on 2026-09-01 — the first real recording
attempt — proving the rule; the new pin is deliberately the **month-end** build (2-year retention).
Dated tags carry versioned asset names (`ffmpeg-N-…-win64-lgpl-shared.zip`), extraction is
name-agnostic; download failures now wrap in `IOException` with an actionable message
("refresh `FfmpegLocator.PinnedUrl` or install ffmpeg on PATH") instead of leaking a raw 404. Once
cached, the URL is never touched again. The pin is a single `const`, bumpable in one place — and must
always stay on the **shared** variant (never `gpl`, `nonfree`, or static; see ai.md Licensing).
3. **Check-then-pull order** — (1) PATH probe (the user's own install wins), (2) cached
`%APPDATA%\ytLlive\tools\ffmpeg.exe`, (3) download + extract. Extract `ffmpeg.exe` **plus the
`libav*.dll` family** (the shared build's bin/ folder; Windows resolves the DLLs from the exe's own
directory) into a staging dir then move into place — a crash never leaves a corrupt or partial cache.
4. **Seam** — `IFfmpegLocator.LocateAsync(CancellationToken)`: search dirs, tools dir, and the downloader
(`Func<string, CancellationToken, Task<byte[]>>`) are constructor-injected with production defaults, so
tests fake the network (feeding a real in-memory zip) and never touch disk outside a temp dir.
**New files (all in `Services/Encoder/`):**
1. `IFfmpegLocator.cs` — the seam.
2. `FfmpegLocator.cs` — the impl (PATH probe → cache → pull+extract exe + DLLs), failures logged via `AppLog`.
3. `THIRD-PARTY-NOTICES.txt` (repo root) — the LGPL/BSD/MIT notices + source offer, copied to the build
output. *(Surfacing changed on 2026-08-13: the top-bar About button that opened the file in the OS
viewer is GONE — the notices are reachable in-app via the logo → About overlay instead.)*
**Test plan:** the hermetic integration test drives the full decision ladder against a temp tools dir and
a fake downloader returning a real in-memory zip (`.../bin/ffmpeg.exe` entry): PATH hit wins without
downloading, cache hit skips the network, cold cache downloads → extracts → `ffmpeg.exe` lands in the
tools dir, and a second call serves the cache (downloader invoked exactly once). Focused unit tests:
**shared-build DLLs extract alongside the exe**, empty zip throws, missing entry throws, empty download
throws, downloader failure propagates, zero-byte cache is refreshed.
**Same-PR housekeeping:** requirement 2's stale binary facts corrected in this plan (~30 MB → ~75 MB zip;
"gyan.dev/BtB N" → BtbN LGPL-shared only, with the why); the "never do" licensing guardrails recorded in
`ai.md` so the reasoning survives.
**Out of scope (later ship steps):** the FFmpeg subprocess encoder (frames in via stdin, stderr health
parsing), RTMP push, WASAPI audio capture, the frame-pipeline wiring, health stats.
**Built (2026-08-10):** `IFfmpegLocator` + `FfmpegLocator` shipped in `Services/Encoder/`, pinned to the
**lgpl-shared** build `autobuild-2026-08-09-13-03` (extracts `ffmpeg.exe` + the `libav*.dll` family via a
staging dir). `THIRD-PARTY-NOTICES.txt` (repo root) ships to the build output; the "never do" licensing
guardrails are recorded in `ai.md` — build **0 warnings**.
Tests: the hermetic `FfmpegLocatorTests` integration test (PATH → cache → download decision ladder with a
fake downloader serving a real in-memory zip) + edge/unit cases (shared-build DLL extraction, zero-byte
cache refresh, empty payload, missing zip entry, downloader failure) — **78 passing**.
#### Ship step 3 — Encoder + RTMP push (the FFmpeg subprocess)
**Goal:** encode raw BGRA master frames into H.264+AAC FLV and push them to the reusable stream's RTMP
ingestion URL — one battle-tested subprocess doing encode + mux + push (retry is app-side, TASK 32 — ffmpeg does not self-reconnect an RTMP output), the app feeding
frames via stdin and parsing stderr for health (req 2).
**Decisions (locked):** the encoder is a thin orchestrator over `ffmpeg.exe` — no H.264/AAC code in the
app. Arguments (pure `FfmpegArgs.Build`): `-re -f rawvideo -pix_fmt bgra -video_size WxH -framerate FPS
-i pipe:0` (frames in), a **silent placeholder audio track** via `-f lavfi -i anullsrc` (the WASAPI
capture step replaces this input), `-c:v <encoder> -b:v K -maxrate K -bufsize 2K` + **`-g fps×4`
`-keyint_min fps×4` `-sc_threshold 0` `-bf 0` `-pix_fmt yuv420p`** (the keyframe ≤4s / closed-GOP /
H.264 compliance), `-c:a aac -ar 48000 -ac 2`, `-f flv <rtmpUrl>`. Encoder choice is **probed from the
binary's `-encoders` listing** (`FfmpegEncoderPicker`, pure): hardware NVENC → QSV → AMF, then OpenH264
software fallback — **never libx264** (GPL; see `ai.md` → Licensing). The seam (`IFfmpegEncoder` +
`IEncoderProcess`, constructor-injected locator + process factory) keeps it hermetic — tests fake the
whole subprocess (probe + encoder), no real binary.
**Behavior:** `StartAsync` (locate → probe → spawn → stderr loop), `SubmitFrameAsync` (serialized BGRA
stdin writes, ~2 Hz health via `HealthUpdated`/`StreamHealth` — bitrate/FPS/duration/dropped-from-frame-
count), `StopAsync` (stdin EOF → ffmpeg finalizes + exits by itself; 10s watchdog kill), `ProcessFailed`
on a non-zero unexpected exit.
**Built (2026-08-12):** `EncoderOptions` + `IFfmpegEncoder`/`FfmpegEncoder` + `IEncoderProcess`/
`FfmpegEncoderProcess` + pure `FfmpegArgs`/`FfmpegProgressParser`/`FfmpegEncoderPicker` in
`Services/Encoder/`. Not yet constructed by the app (the frame-pipeline wiring, ship step 5, owns it).
Tests: `FfmpegEncoderTests` integration (probe → spawn with NVENC preferred → frames into stdin →
progress parsed → graceful stop, no kill) + units (args compliance/GOP, progress parser, picker
preference + GPL guard, no-URL/not-running/noop stops, process-death `ProcessFailed`) — **122 passing**.
#### Ship step 4 — WASAPI audio capture (the meter comes alive)
**Goal:** capture desktop/game audio (loopback) and the picked mic, feed the mic level into `AudioLevel`
so the realtime meter reads something other than 0, run capture **only while live** (req 7).
**Decisions (locked):**
1. **NAudio `NAudio.Wasapi` 2.2.1** — the wasapi feature package (not the `NAudio` meta-package): it
carries the capture types (`WasapiCapture`/`WasapiLoopbackCapture` + the MMDevice enumeration) with
`NAudio.Core`/`NAudio.Asio` pulled in transitively. MIT — recorded in `THIRD-PARTY-NOTICES.txt` (item 9).
2. **`IAudioSource` seam** (`Start`/`Stop`/`SampleReady`/`Failed`, IDisposable) — the app consumes the
seam; the two WASAPI implementations wrap NAudio; tests inject hermetic fakes (no real audio devices,
no timers). Loopback = `WasapiLoopbackCapture` on the default render device; mic = `WasapiCapture`
with the NAudio device resolved by `FriendlyName` matching `MicSourceName` (the app only persists the
DisplayName), falling back to the default capture endpoint. Mic device resolution is re-read at each
`Start` via a name provider so a mic picked mid-session takes effect next go-live.
3. **`AudioMixer` owns both sources** — starts/stops both with go-live (`BeginGoLive` success → `Start`,
`StopStream` → `Stop`). Mic samples feed a pure `AudioLevelMeter` (RMS, exponential smoothing) and
raise `MicLevelChanged`, marshalled to the UI thread into `AudioLevel`; desktop samples are currently
dropped (consumed by the encoder's AAC mix in a later step). Capture failures are logged via `AppLog`
(mic failure also zeroes the meter); loopback failure doesn't kill the mic.
4. **Byte→float** — pure `WaveToFloat.Convert` handles the WASAPI mix formats: IEEE float 32-bit (direct)
and PCM 16-bit (normalized to -1..1), including `WaveFormatExtensible` with the IEEE-float subformat
GUID. Trailing partial samples are ignored.
**Built (2026-08-12):** `Services/Audio/` ships `IAudioSource` + `AudioSample`, `WasapiLoopbackAudioSource`,
`WasapiMicAudioSource`, `AudioMixer`, `AudioLevelMeter`, `WaveToFloat`; `MainViewModel` constructs the
mixer (mic source fed `() => MicSourceName`), starts it on go-live and stops it on end-stream, and maps
`MicLevelChanged` → `AudioLevel`. A pre-existing CS8602 in `FfmpegEncoder.cs:139` surfaced during this
step's rebuild and was fixed (`process!`) — build **0 warnings**. Tests: `AudioMixerTests` (mixer
lifecycle/forwarding/failure against fakes, meter RMS/smoothing/reset, `WaveToFloat` float/PCM16/
extensible/truncation) — **139 passing**.
**Deferred (later ship steps):** wiring the desktop-capture samples into the encoder's AAC mix (replaces
the `-f lavfi -i anullsrc` placeholder; the encoder construction itself shipped in ship step 5).
*(The "capture while not live" + "audio UI beyond the mic controls" deferrals were SHIPPED on the
2026-08-13 game audio bar branch — capture is now always-on for preview and the game bar is the second
audio UI. The mixer's short-circuit meter fix + `Started`/`RestartMic` seams live in the same branch.)*
#### Ship step 5 — Frame-pipeline wiring (the encoder gets a frame source)
**Goal:** the chain `CameraManager`/`ScreenCaptureManager` → compositor resolver → encoder, driven while
live by a paced frame pump: snapshot the active scene → resolve each element to its latest frame →
composite into the tier's output frame → pace into the encoder's stdin at the tier's FPS.
**Decisions (locked via user Q&A, 2026-08-12):**
1. **Video pipeline first** — the `-f lavfi -i anullsrc` silent track stays; mixing the loopback/mic
WASAPI samples into the encoder's AAC track is its own later step.
2. **RTMP URL via a provider seam** — `MainViewModel._rtmpUrlProvider` is a `Func<string?>` returning
null today (the reusable stream's ingest URL lands with TASK 9); when it yields null the pump logs
and skips the encoder entirely, so go-live runs the existing visual flow without pushing.
**Design:**
1. `Services/Encoder/FramePump.cs` — the frame producer. All collaborators constructor-injected seams
(`Func<Scene?>`, `Func<SceneElement, VideoFrame?>` resolver, `Func<CompositorOptions>`,
`Func<EncoderOptions?>`, `Func<IFfmpegEncoder>`, `Action<string>` log, injectable pacing delay) so it
stays free of WPF and of the capture managers and is hermetic in tests. `StartAsync` never throws
(failures log + surface via `Failed` — the VM fires-and-forgets from the sync command handler);
loop = snapshot → render → `SubmitFrameAsync`, paced at `1/options.Fps` (default `Task.Delay`; tests
inject `Task.Yield`). `StopAsync` stops the encoder (closes stdin) BEFORE awaiting the loop — closing
stdin unblocks a write stuck on pipe backpressure, so stop can't deadlock on the pump. `ProcessFailed`
self-stops the pump. `HealthUpdated` forwards the encoder's stats (ship step 6 binds the bottom bar).
2. `ScreenCaptureManager.GetLatestFrame(key)` — mirrors `CameraManager.GetLatestFrame(deviceId)`; the
backdrop's live frame for the compositor.
3. `MainViewModel` — owns the resolver (`WebcamSceneConfig` → `GetLatestFrame(WebcamId)`;
`Source.IsLiveCapture` → `GetLatestFrame(CaptureKey)`; image/background → `StaticPixelCache.Get(AssetId)`),
builds `CompositorOptions` from the tier + `OutputRect*` (doubles rounded to ints — the vertical
607.5 half-pixel crop rounds to a perfectly-centered 608), builds `EncoderOptions` from the tier when
the URL provider returns one, constructs the real `FfmpegEncoder(new FfmpegLocator())`, starts the
pump on go-live, stops it on end-stream, disposes in `Shutdown`, and flips `StreamStatus.Error` when
the pump fails while live (minimal — detailed health surfacing is ship step 6).
**Test plan (Good Dog Rule — ONE integration test):** `FramePumpTests.Start_CompositesScene_FeedsEncoder_StopsCleanly`
drives the full lifecycle against fakes — real `SceneCompositor` + real `FramePump`, fake `IFfmpegEncoder`
— asserting the composited red backdrop frame actually reaches the encoder at the tier size and that stop
tears everything down. Units: no-URL start skips the encoder, re-entrant start/stop no-ops, encoder
start-failure raises `Failed` + disposes, `ProcessFailed` self-stops the pump, `HealthUpdated` forwards.
`ScreenCaptureManagerTests.GetLatestFrame_ReturnsLatestPump_UntilReleased` pins the new accessor.
**Out of scope (later ship steps):** the loopback/mic → AAC mix (replaces `anullsrc`), health stats in the
bottom bar (ship step 6), scene-switching transitions, and any flash-frame wiring.
**Built (2026-08-12):** `FramePump` shipped in `Services/Encoder/`, `ScreenCaptureManager.GetLatestFrame`
added, `MainViewModel` wired end-to-end (resolver + both option builders + pump lifecycle), `FramePumpTests`
(7) + `GetLatestFrame` test (1) added — build **0 warnings**, **147 tests passing**. Known consideration:
the pump reads the active scene on a background thread while the UI can still edit it; a concurrent-mutation
exception is contained (logged + `Failed` + pump stops) rather than crashing.
#### Ship step 5.5 — Social bar bug fixes + the bar on the live output (2026-08-13)
**Bug 1 — bar wouldn't reliably change position (root cause found, then simplified):** the original
drag set `Canvas.SetTop(bar, …)` with a local value, which permanently overrides the
`Canvas.Top="{Binding SocialBarTop}"` binding — the release-time `SetSocialBarPosition` →
`PropertyChanged(SocialBarTop)` could never beat it. First fix added direction-snapping during the drag
(`SocialBarSnap.Decide`, ±6px deadzone) + `bar.ClearValue(Canvas.TopProperty)` on release — but that
still misbehaved for shaky hands (jitter around the deadzone: it snapped up reliably, then refused to
come back down and snapped back to top). **Superseded by a click-toggle (KISS, user decision):** clicking
the bar in the preview flips it top ⇄ bottom (`MainViewModel.ToggleSocialBarPosition` → the existing
`SetSocialBarPosition`), the bar rides `{Binding SocialBarTop}` alone (no local values, no deadzone, no
jitter sensitivity), and `SocialBarSnap` was removed. The `ClearValue` lesson stands: never set a local
value on a property the binding owns.
**Bug 2 — Mastodon showed the generic 7-star honeycomb (root cause found):** the DB row
`@gramps@llamachile.tube` had `Software = NULL` — nodeinfo was only ever resolved against the identity domain
(`llamachile.tube`, a landing page), never probed for the real instance at `mastodon.llamachile.tube`.
Fixed on three fronts: `HttpSocialValidator` now **probes well-known subdomains** (mastodon. → social. →
… `FediverseSubdomainCandidates`) when the identity domain and its redirect both come up empty, under a
~15s linked-CTS budget, with an optional `Action<string>` log; `MainViewModel` **heals** any fediverse entry
missing a software name on layout load (`HealFediverseSoftwareAsync` — static, testable; instance wrapper
runs it off the UI thread, applies via the dispatcher, saves); `SocialEntry.FediverseSoftware` is now
**settable** and raises `PropertyChanged` for `LogoData`, so the heal updates the icon in place. If the user
later re-saves the entry with the icon fixed, the name persists with it.
**Compositor rendering (user-approved scope):** the social bar now appears **on the live output**, not just
the preview — `Compositor/SocialBarRenderer.cs` rasterizes the entries into a transparent straight-alpha
BGRA strip (1920-wide, 40px content + 24px glow pad, green `#2ecc71` glow baked in) via `RenderTargetBitmap`
(WPF glue like `StaticPixelCache`; the compositor core stays pure). `SceneCompositor.Render` takes optional
`socialBarFrame` + `socialBarTop` (master space) and blits it **last — above the branding flash** (the old
`BlitFlash` generalized to offset `BlitOverlay`). `FramePump` gains a `socialBar:` seam
(`Func<(VideoFrame?, SocialBarPosition)>`, re-read every frame) and places the bar at `0` or
`SourceRectHeight − bar height`. `MainViewModel` owns the frame (`RenderSocialBarFrame`, re-rendered on
load/save/notify) and feeds the seam.
**Tests (+6 → 153 passing, 0 warnings):** settable `FediverseSoftware` updates `LogoData`,
subdomain-probe unit + null-when-silent unit, the branch's **one integration test**
`Socials_HealMissingFediverseSoftware_RoundTripsThroughDb` (temp-DB roundtrip: NULL software → healed via
the validator → persisted), compositor bar overlay (top/bottom + above-flash), `FramePump` bar pass-through
(Top then flipped to Bottom mid-run — the seam is re-read each frame), and both `ISocialValidator` fakes
(`FakeValidator`/`BlockingValidator`) gained `ResolveFediverseSoftwareAsync`. The drag-snap units
(`SocialBarSnap`) were removed with the click-toggle supersession.
**Not included (say the word):** rewriting the healed entry's `ProfileUrl` to `https://mastodon.llamachile.tube/@gramps`.
---
---
## TASK 5 — Layout Persistence (SQLite)
**Goal:** Scenes, sources, and asset bytes survive restarts; assets are always available.
### Status: ✅ Done
1. ✅ SQLite database (`Microsoft.Data.Sqlite`) at `%APPDATA%\ytLlive\ytLlive.db`; schema versioned via `PRAGMA user_version` (currently **v8**)
2. ✅ Assets live in the DB (BLOB keyed by SHA-256 content hash), never file paths — deleting the original file never breaks a scene
3. ✅ File-model save/open — the active layout file is tracked (default is the AppData DB); **Save Layout As… / Open Layout…** switch the active file; auto-save writes to whatever is active
4. ✅ Auto-save (invisible) — ~1.5s debounce on scene/source add/remove/reorder/rename/hide + any source transform change; flush on window close
5. ✅ Startup — load the active file; seed the five canonical scenes only when the DB is empty; (+) re-adds a missing canonical scene and is hidden once all five are present; adding beyond the five is rejected
6. ✅ Schema v1 → v9 — webcam columns (v2), singleton `Webcam` + per-scene `WebcamSceneConfig` (v3), `RectWidth`/`RectHeight` round-to-rect restore (v4), `Source.IsBackdrop` + `Source.CaptureKey` (v5), `Scene.HasBackdrop` — backdrop **Live-only by policy** (v6, one-time backfill + `EnforceBackdropPolicy` on every load), `Scene.HasSocialBar` (v7, dropped per-scene toggle — column back-compat, unread), `Socials.BarEnabled` (v8); the `SocialEntry.Software` fediverse-software column is a **column-presence migration** (commented v8→v9, no version bump — `user_version` stays 8); v9 = single-row `Music`. **Load-time rule:** sources load with `IsBackground` derived from `Type` (OR'd with the persisted column so legacy DisplayCapture backdrops keep their flag) — rows saved before the Type setter derived the flag heal on load
7. ✅ WindowHandle stays in-memory (per-session); save = transactional rewrite; orphaned assets pruned. **v9 lands in the TASK 8 audio milestone** (single-row `Music` — `TrackPath`/`IsEnabled`)
### Design decisions
1. **SQLite database** (`Microsoft.Data.Sqlite`) at `%APPDATA%\ytLlive\ytLlive.db`; schema versioned
via `PRAGMA user_version`.
2. **Assets live in the DB, not on disk** — `Asset` table stores image bytes (BLOB) keyed by a
SHA-256 content hash (unique). Identical image content collapses to one row regardless of file
name — the 1:M resource memory model, enforced by the database. No file paths; deleting the
original file never breaks a scene.
3. **File-model save/open** — the active layout file is tracked (default is the AppData DB).
**Save Layout As… / Open Layout…** switch the active file; auto-save writes to whatever is active.
4. **Auto-save (invisible)** — ~1.5s debounce on scene add/remove/reorder/rename/hide, source
add/remove/reorder, and any source transform change; flush on window close.
5. **Schema** — `Scene` (Id, Name, IsHidden, IsChatScene, HasBackdrop, HasSocialBar, SortOrder), `Asset` (Id, Hash, Data,
PixelWidth, PixelHeight), `Source` (Id, SceneId FK cascade, AssetId FK, Type, Name, IsEnabled,
X/Y/Width/Height/Opacity, MonitorIndex, DeviceId, ClipShape, IsMirrored, SortOrder), `Socials` (Id,
BarPosition, BarJustify — back-compat, unread, BarEnabled added via `ALTER`), `SocialEntry` (Id,
SocialsId FK cascade, Service, Handle, ProfileUrl, SortOrder) — `user_version` **8** (v1 → v2 =
`ALTER TABLE` adds the two webcam columns; v3 = singleton `Webcam` + per-scene `WebcamSceneConfig`;
v4 = `WebcamSceneConfig.RectWidth`/`RectHeight` for the round-to-rect restore; v5 = `Source.IsBackdrop`
+ `Source.CaptureKey` for the live-capture backdrop; v6 = `Scene.HasBackdrop` — the backdrop is
**Live-only by policy** (one-time backfill turns Starting/BRB/Chat/Ending off and drops their backdrop
sources; `EnforceBackdropPolicy` re-normalizes every load); v7 = `Scene.HasSocialBar` (per-scene toggle
dropped — column back-compat, unread); v8 = `Socials.BarEnabled`). The `SocialEntry.Software`
fediverse-software column is a column-presence migration (commented v8→v9, no version bump).
`WindowHandle` stays in-memory (per-session). Save = transactional rewrite; orphaned assets pruned.
6. **Startup** — load the active file; seed the five canonical scenes
(Starting/Live/BRB/Chat/Ending, `SceneCatalog`) only when the DB is empty. The (+)
button re-adds a missing canonical scene and is hidden once all five are present;
adding beyond the five is rejected — work with less, never more.
---
---
## TASK 6 — UI polish batch (scenes/sources rows, dedup naming, social bar)
**Goal:** clean up the two side lists and the social bar per gramps's review.
### Status: ✅ Done
1. ✅ Scene rows are pure selection rows — the per-row edit/trash/visibility icons and the inline rename TextBox are gone (`EditSceneCommand`/`RemoveSceneCommand`/`ToggleSceneVisibilityCommand` + handlers + `Scene.IsEditing` removed; `IsHidden` stays persisted + dims hidden rows)
2. ✅ Source rows gained the trio — edit (inline rename via new `EditElementCommand` + `SceneElement.IsEditing`), visibility eye (new `ToggleElementVisibilityCommand` flips `SceneElement.IsVisible`; the eye style now binds `IsVisible`, open/slashed + row dims to 0.45 when hidden), and the existing trash
3. ✅ Duplicate resource names get a no-space incrementing suffix via shared `NextSourceName` (Image, Image2, Image3…) — next free number derived from the names actually in the scene, so deleting a middle resource never collides (`AddSource` + `AddReusedImage` both use it)
4. ✅ Social bar renders the full validated handle — `MaxWidth=200` + `TextTrimming` removed from BOTH `SocialBarRenderer` and the preview template (mastodon `@gramps@…` no longer cuts off)
5. ✅ Side panels stay fixed-width (left 220 / right 300) — deliberate: they never re-layout on resize, the preview absorbs it
6. ✅ Focus-loss capture lag documented as a known OS limit in `ai.md` — deferred by user decision (no code change)
### Design decisions
1. **Next free number from names, not type counts** — the old scheme counted elements by `SourceType` (`count == 0 ? baseName : base+count+1`), which collided after deletions; the new helper scans actual names.
2. **One WPF App per AppDomain** — the real-App tests (round-clip + naming) share `RealAppHost` (a dedicated STA thread owning the single `App`) via the `RealApp` serial collection, instead of each calling `new App()`.
---
---
## TASK 7 — Meter scaling amplification (voice meter uses the full bar)
**Goal:** the mic (and game) meters feel light — speech peaks should peg into the red at maxed volume.
### Status: ✅ Done
1. ✅ `AudioLevelMeter.ToDisplay` now adds **+10 dB of input amplification** before the −60..0 dBFS → 0..1 log mapping (was raw dB): speech peaks (~0.2 RMS, −14 dBFS) read ~0.93 → red zone; normal speech (~0.05, −26 dBFS) ~0.73 → yellow; background noise ≤0.001 linear (−60 dBFS) still reads 0 (the meter never idles on it)
2. ✅ Unit tests updated to the new mapping + new `ToDisplay_Pushes_Speech_Peaks_Into_Red_At_Maxed_Volume` (0.2 → 0.92..0.95, 0.05 → 0.72..0.75)
### Design decisions
1. **Amplify at the mapping, not in the VM** — one knob (`ToDisplay`), shared by the mic bar and the game bar; the `× MicVolume` gain-on-noise behavior is untouched (raising the slider still moves ambient noise up the bar).
2. **+10 dB, not more** — louder boosts push quiet speech into the lower half and make the noise floor visible; +10 puts speech peaks solidly in red while idle stays at zero.
---
---
## TASK 8 — Audio milestone: real stream audio + voice filters + TRAX music (2026-08-14)
**Goal:** every audio issue done and tested in one branch — real mic/game audio reaches the encoder (replacing the `anullsrc` silence), `MicVolume`/`GameAudioVolume`/mute become real pre-AAC gains, TASK 10's voice filters land, auto-duck keeps the creator's voice over game + music, and a free background-music source ("TRAX") plays into the sound bar and the stream.
### Status: ✅ SHIPPED 2026-08-14 — 196 tests passing, 0 warnings (ONE integration test: `AudioPipelineTests.Mix_WithFiltersDuckAndGain_Lands_On_AudioPipe`)
1. ✅ **Real audio into the encoder** — the mixer's loopback samples were **dropped today** (`AudioMixer.OnLoopbackSample` metered only) and `MicVolume` was meter-only; `FfmpegArgs.cs:28` ran `-f lavfi -i anullsrc` (silence). Transport: **ffmpeg reads a Windows named pipe** — `-f f32le -ar 48000 -ac 2 -i \\.\pipe\ytllive_audio` replaces the anullsrc block, plus explicit `-map 0:v -map 1:a`; pipe name via `EncoderOptions.AudioPipeName` (`DefaultAudioPipeName = "ytllive_audio"`). Mixer stays **two inputs (mic + loopback)** — no N-source abstraction, no `StereoMixer` class. The VM owns the pipe lifecycle (`BeginGoLive` → `_audioMixer.StartLive(pipeName)`, `StopStream` → `_audioMixer.StopLive()` before the pump stops); `FfmpegEncoder.cs` is untouched.
2. ✅ **Honest gains reach the stream** — `MicVolume` scales the mic channel; `GameAudioVolume` + mute scale the loopback channel (the "mute in preview but stream still plays" footgun dies with it). One knob per input = KISS. Implemented as `Func<double>` gain seams on `AudioMixer` (`micGain`/`loopbackGain`), wired by `MainViewModel`.
3. ✅ **Voice filters (TASK 10, before the meter AND the mix)** — `VoiceFilterChain`: bass (`LowShelfFilter` 120 Hz +4 dB) → treble (`HighShelfFilter` 8 kHz +3 dB) → **noise gate** (pure C#, creator's choice) → compressor (threshold 0.5, 4:1). Pure stateful DSP (TDF2), sine-in unit tests. Meter stays post-filter.
4. ✅ **Auto-duck** — pure envelope (`AutoDucker`): mic RMS > 0.02 → game+music dip ×0.25 (~12 dB), attack 0.05 / release 0.005, recover on release. Always-on, no knobs. Strong form of the creator's "voice always over the game volume" idea.
5. ✅ **TRAX — background music, FREE** (keeps the one paid line = Alerts + flash removal). `MusicPlayer`: NAudio `MediaFoundationReader` (mp3/wav/m4a) → `VolumeWaveProvider16` at the hardcoded **0.20** (fixed, not changeable) → `WaveOutEvent` on the default device (added the sibling `NAudio.WinMM` 2.2.1 package — `WaveOutEvent` isn't in `NAudio.Wasapi`), loop on end. **No 3rd mixer input and no `MusicVolume`** — music plays on the desktop, the existing loopback picks it up: heard in headphones, the sound-bar meter bounces, and the stream carries it through the loopback channel (ducked with game when the voice is hot). Known wrinkle (out of scope): YouTube mutes VODs carrying copyrighted music — future "music on live, off VOD".
6. ✅ **TRAX footer control (final spec, agreed 2026-08-14)** — `YtButtonSecondary`, **status dot + "TRAX"** text, in the center footer stack beside MIC:
- **Status dot** (same 8px Ellipse pattern as MIC/Socials): **red** no track loaded · **yellow** loaded not playing · **green** playing.
- **Left-click** → toggle play/pause; no track loaded → opens the picker instead.
- **Right-click** → always opens the in-app track picker (`TraxButton_PreviewMouseRightButtonUp`, code-behind pattern like `GameSpeaker_MouseLeftButtonUp`, `e.Handled = true` so no context menu).
- **Tooltip** → playing/loaded track name; "No track — right-click to choose background music" when empty.
- **No slider** — volume is the 0.20 constant. Picker = plain OS `OpenFileDialog` filtered to mp3/wav/m4a/aac/flac/ogg.
7. ✅ **Sound bar shows music** — `IsGameAudioBarVisible = gameDetectorProducingSound || isMusicPlaying`; relabelled "Game Audio Capture" → **"Desktop Audio"** (TRAX rides the same channel). **Follow-up (2026-08-15): the gating is gone — the game audio bar is ALWAYS visible** (the `IGameAudioDetector`/`GameAudioDetector`/`GameAudioHysteresis` stack + the VM's 250ms poll timer and their tests were deleted; the bar kept hiding the desktop meter when no full-screen game with sound was up, and the creator wants it pinned).
8. ✅ **Capture hardening (design changed at build time)** — the plan's "force IEEE-float 48 kHz on both WASAPI sources" was **dropped**: NAudio 2.2.1's `WasapiCapture` exposes no overridable `GetDefaultMixFormat`, so both sources capture the device's own mix format and the mixer's pure `TinyResampler` normalizes any rate/channel count to 48 kHz stereo (the resampler IS the design, not a fallback). Stop path disposes the audio pipe (EOF) in `MainViewModel.StopStream` before the pump stops, so both ffmpeg inputs end in order.
9. ✅ **Schema v9** — single-row `Music` (`TrackPath`, `IsEnabled`; volume is the 0.20 constant) + migration in `LayoutStore.cs` (`user_version` 9; `Save` gained an optional `Music? music = null` param so existing 3-arg callers still compile; load query + null reset).
10. ✅ **Docs in the same commit** — `ai.md` audio section rewritten ("one knob per input", 2-in mix, auto-duck, TRAX free, VOD-mute wrinkle), TASK 10's status, this task's status, HANDOFF.
**Files (new):** `Services/Audio/VoiceFilterChain.cs` (+ `LowShelfFilter`/`HighShelfFilter`/`NoiseGate`/`Compressor`), `Services/Audio/AudioRingBuffer.cs` (async-arrival source buffers), `Services/Audio/AutoDucker.cs`, `Services/Audio/TinyResampler.cs`, `Services/Audio/MusicPlayer.cs`, `Services/Audio/NamedPipeAudioWriter.cs` (+ `IAudioPipeWriter` seam), `Models/Music.cs`.
**Files (changed):** `AudioMixer.cs` (filter chain before meter+mix, loopback into the mix, gains via `Func<double>` seams, ducker, silence-filler so the pipe never stalls, `StartLive`/`StopLive`), `WasapiMicAudioSource.cs` (device mix format — resampler normalizes; no forced format), `FfmpegArgs.cs` (named-pipe input + `-map`, name via `EncoderOptions.AudioPipeName`), `EncoderOptions.cs` (`AudioPipeName`), `ytLive.csproj` (`NAudio.WinMM` 2.2.1 for `WaveOutEvent`), `LayoutStore.cs` (schema v9), `MainViewModel.cs`/`MainWindow.xaml`/`MainWindow.xaml.cs` (footer TRAX group, sound-bar visibility OR music, go-live starts the pipe, end closes it first).
**Testing (Good Dog Rule — ONE integration test):** `AudioPipelineTests.Mix_WithFiltersDuckAndGain_Lands_On_AudioPipe` — real filter chain + mixer + ducker + `NamedPipeAudioWriter`, fake `IAudioSource`s, test-side `NamedPipeClientStream` reads the bytes and asserts post-filter/post-gain/post-duck mixed stereo (ring buffers pre-filled so the first pipe tick already carries real audio — avoids a start-of-stream silence race). Units: each DSP stage (known sine-in → expected gain), ring buffer wrap/underflow/overwrite, ducker envelope, resampler (down/up/stateful/identity), `FfmpegArgs` pipe+map, schema v9 roundtrip + no-music-row. Existing tests stay green.
**Out of scope (this branch):** IP webcam (video-only when it lands — never an audio input), chat box source (TASK 3 item 18), alt-key crop, credits, background removal, music-off-VOD, `PremiumUrl` (TASK 10 seam).
---
---
## TASK 9 — YouTube Live Stream Management
**Goal:** Create/bind broadcasts, monitor YouTube-side stream health — the v3 way.
### Status: ⏳ In progress — items 1–3 SHIPPED (reusable stream 2026-08-16; report-by-exception health 2026-08-16); item 4 SHIPPED (status reconciled 2026-09-01); item 5 open; item 6 = deliberate lock (TASK 36); item 7 scoped to core fields — advanced tab is out-of-product
1. ☑ **Broadcast creation** — title/description/privacy/scheduledStartTime via API, with the v3 flags above (SHIPPED: `CreateBroadcast` sends `enableAutoStart/Stop`, `enableMonitorStream=false`, `latencyPreference=low`, `selfDeclaredMadeForKids=false`)
2. ☑ **Reusable stream** — create once, cache + reuse; bind to broadcast (SHIPPED: `GetOrCreateReusableStreamAsync` lists-then-inserts the `variable`/`isReusable` stream, cached via `LayoutStore` Settings, bound at broadcast insert via `boundStreamId`; `_rtmpUrlProvider` yields the ingest URL so go-live actually encodes + pushes)
3. ☑ **Health monitoring** — poll `liveStreams.list` `healthStatus` + `configurationIssues[]`, surface banner only on warning/error (SHIPPED: `GetStreamHealthAsync(streamId)` 30s while live; pure `StreamHealthReporter.BannerFor` = report-by-exception; banner strip under the top bar, amber warning / dark-red error, via `HealthIssueBanner`/`HealthIssueBackground`; poll failures log-only; ONE integration test `GetStreamHealthAsync_Report_By_Exception_Banner_Only_On_Warning_Or_Error`)
4. ☑ Live chat — poll `liveChat/messages`, render in right panel, support Super Chat + membership badges — **SHIPPED** (status reconciled 2026-09-01: `YouTubeChatService` polls + parses `superChatEvent`/`newSponsorEvent` with badge/level on `ChatMessage`; the panel MOVED to the left-panel live state by Control Surface UX, "right panel" superseded; the remaining reward types land with the monetization chain, TASK 10 → related work)
5. ☐ Error handling — the YouTube error codes: `errorStreamInactive`, `invalidTransition`, `redundantTransition`, `liveStreamDeletionNotAllowed`, `liveStreamModificationNotAllowed`, `liveBroadcastBindingNotAllowed`
6. ❌ **Visibility picker** — DELIBERATE TEST-PHASE LOCK (creator decision 2026-09-01), not open work: "always Private" stays during multi-month real-world testing so breakage VODs never clutter the channel. `recordFromStart`/DVR stay on — Private VODs are invisible review tapes; bulk-delete pre-GA. Unlock = TASK 36 gold-pass item (remove the override in `YouTubeStreamService.CreateBroadcast`, wire the dialog selection). See `ai.md` → YouTube Live API constraints.
7. 🔶 **Full broadcast form** — RESCOPED 2026-09-01, not the old two-tab plan. **Core fields (SHIPPED as the Text drawer, 2026-08-24):** title, description, tags, visibility, made-for-kids — live-editable. **Scheduling:** ships as **TASK 34** (Scheduled checkbox + datetime in the same drawer). **Advanced tab is PERMANENTLY OUT** (the 10% margin): latency (locked `low`), DVR/record-from-start (locked on), embed, projection, CC, region restrictions stay fixed at sane defaults, invisible — every exposed field is a support ticket. `categoryId` removal: done (not a `liveBroadcast` field). Monetization enablement (if ever needed) rides the reward-events chain via `liveBroadcasts.update`, not a form field
### Design decisions (v3)
1. **One-click go-live** — `liveBroadcasts.insert` with `enableAutoStart=true`, `enableAutoStop=true`, `enableMonitorStream=false`, `selfDeclaredMadeForKids=false`, `latencyPreference=low`. No `transition(live)` call, no testing stage, no liveStarting polling. Encoder starts → YouTube brings it live by itself.
2. **Variable reusable stream** — `cdn.resolution=variable`, `cdn.frameRate=variable`, `isReusable=true`. Create once per channel, cache ingestion URL + stream name, reuse for every broadcast. Any quality tier works without recreation; auto step-down needs no API calls.
3. **Report-by-exception** — poll `liveStreams.list`; banner only on `healthStatus` warning/error issues (`configurationIssues[]`). Bottom strip = YouTube logo + green/red connection dot (clickable → opens the dialog).
4. **One dialog, three states** — `not connected` (sign-in) / `connected-offline` (all editable) / `live` (title + description + visibility editable; quality + account greyed out). Both entry points (Start Stream button + bottom strip) open it; prefilled from saved session profile.
5. **Live edits** — `liveBroadcasts.update` with part=`snippet,status` for title/description/privacy.
6. **End stream** — stop encoder → `transition(complete)`, with `enableAutoStop` as the safety net. **SHIPPED 2026-09-01** (`EndBroadcastAsync`, wired into `StopStream` after RTMP EOF; the call had been specced since TASK 9 but never existed — found during recording verification)
7. **Broadcast ID == Video ID** — one ID to track status, health, and the auto-created VOD (`recordFromStart` + `enableDvr`).
### Requirements:
1. **Broadcast creation** — title/description/privacy/scheduledStartTime via API, with the v3 flags above
2. **Reusable stream** — create once, cache + reuse; bind to broadcast
3. **Health monitoring** — poll `liveStreams.list` `healthStatus` + `configurationIssues[]`, surface banner only on warning/error
4. **Live chat** — poll `liveChat/messages`, render in right panel, support Super Chat + membership badges
5. **Error handling** — the YouTube error codes: `errorStreamInactive`, `invalidTransition`, `redundantTransition`, `liveStreamDeletionNotAllowed`, `liveStreamModificationNotAllowed`, `liveBroadcastBindingNotAllowed`
6. **Visibility picker** — Private/Unlisted/Public from the go-live dialog
7. **Full broadcast form** — Core + Advanced tabs with all API-supported fields
---
---
## TASK 10 — Monetization: watermark-only subscription + Polar billing
**Goal:** annual subscription via Polar.sh that removes the branding watermark. All features are
free — the only difference between free and paid is the watermark.
**Related work — monetization awareness (2026-09-01, ungated, free):** this task's billing is the
*payer* side; the *product* also carries built-in, ungated monetization awareness for every creator —
reward-event capture (all 7 `liveChatMessage` reward event types → canonical `RewardEvent` → SQLite
persistence), per-session/broadcast reports (members, Super Chat $, stickers, gifts, milestones — the
in-app replacement for the emailed "stream activity report"), and a journey tracker (position % +
ETA vs. versioned, date-aware YPP thresholds; the Tier-2 bar doubles for new applicants 2027-02-01).
Details and design notes: `ai.md` → Monetization. This is tracked as related work under this task's
narrative, not a separate task number — it ships in its own code slices with one integration test each.
**Scope: v1** (2026-09-01 complete-v1 ruling — there is no v1.x): slice order = reward-event capture →
session report → journey tracker → Alerts (TASK 3 item 20).
**Business details (pricing, Polar product/checkout/discounts) in `MONETIZATION.md` (gitignored).**
**Polar product:** `d105dfa1-497e-423b-8cd4-e0ee2e3abbc0` (LlamaCasty, $99/yr, currently `private`)
**Checkout:** `llamacasty.com` → Polar hosted page (~80% configured)
**Discounts:** `LLAMAFOUNDER` (100% off, 50 uses, `once`), `LLAMA50` (50% off, 12 months, unlimited)
**Org ID:** `c05fb364-b967-4f6c-adf2-8a144e46d085` (org-scoped OAT — `organization_id` can be omitted from API calls)
**Key prefix:** `LCYT-`
**Validate endpoint:** `POST https://api.polar.sh/v1/customer-portal/license-keys/validate` (no auth required for client-side validation)
### Status: 🔶 In progress — steps 1–7 shipped, Velopack update URL pending
1. ✅ **Billing provider** — Polar (polar.sh) chosen and configured. Product, price, license key benefit, discounts, checkout link all created.
2. ✅ **License key entry + verification UI** — About overlay swap panel (same pattern as Licenses panel). Text input + "Activate" button + status message. "Enter License Key" button in the hub.
3. ✅ **Offline entitlement** — `LayoutStore` Settings table: `LicenseKey`, `LicenseValidatedAt`, `IsPremium`. 14-day offline grace period. Startup re-validates online, falls back to cache.
4. ✅ **BrandFlash → watermark toggle** — `IsPremium` property setter flips `BrandFlashEnabled = !value`. Flash stops immediately on premium activation. Flash asset VideoFrame rendering deferred to TASK 4 compositor.
5. ✅ **Remove social slot locks** — all 6 social bar slots open to everyone (no `IsPremium` gating on slots 3-6) — shipped as TASK 10c
6. ✅ **"Unlock Premium" button** — `PremiumUrl` set to Polar checkout; `IsPremiumAvailable` is true; button lights up in the About hub.
7. ✅ **Renewal/lapse handling** — startup re-validates against Polar API. Lapse → `IsPremium = false` → flash returns. Grace period: 14 days offline.
8. 🔶 **Velopack auto-updates** — NuGet packages + bootstrap in `App.xaml.cs`. Update URL self-hosted on DO droplet (pending configuration).
---
---
## TASK 11 — Post-pause polish batch: the creator's 8 review issues (2026-08-15, shipped)
**Status: ✅ SHIPPED 2026-08-15 — 197 tests passing, 0 warnings (ONE integration test:
`AudioPipelineTests.Mix_HonorsProviderGains_AndGameMute_KillsTheLoopback`).**
The creator reviewed the TASK 8 build and filed 8 issues. All fixed in one branch on `main`:
1. ✅ **Desktop/game audio volume slider had no effect** (headset OR stream) — the `AudioMixer`'s
`micGain`/`loopbackGain` seams defaulted to unity because the VM never passed them; the sliders
were decorative. Fixed: new `AudioGainProvider` (`Services/Audio/`) hands `MicMuted`/`MicVolume`/
`GameMuted`/`GameAudioVolume` to the mixer at construction, read live each mix tick.
2. ✅ **Desktop/game mute button had no effect** — same root cause; `GameMuted` now zeroes the
loopback on the stream. Locally, `MusicPlayer.LocalGain` (new) is scaled by the game bar on every
volume/mute change + on TRAX load, so the creator **hears** the control work on the music (the
track rides the loopback channel, so it ducks locally exactly as it does on stream).
3. ✅ **TRAX played once then stopped** — `MusicPlayer.OnPlaybackStopped` only looped when
`Position >= Length`, unreliable for `MediaFoundationReader`; now any clean stop
(`e.Exception == null`) rewinds + replays; only errors/explicit stops surface via `PlaybackEnded`.
4. ✅ **TRAX/MIC buttons swapped** — TRAX no longer sits between MIC and the (mic-specific) meter;
the footer's mic cluster is MIC + meter + mute + volume.
5. ✅ **Mic source persists across restarts** — `LayoutStore` gained a `Settings` key/value table
(`SaveMicSourceName`/`LoadMicSourceName`); the VM saves on pick and restores before the mixer's
first `Start`, so a restart reconnects the same already-vetted device (green dot) or reports it
missing (yellow) instead of falling back to the default endpoint.
6. ✅ **TRAX moved into the game audio bar** — left of the "Desktop Audio" label in the preview
overlay (the creator's follow-up pick), no longer in the footer; its tooltip teaches the clicks
("left-click pauses/plays, right-click chooses a track").
7. ✅ **Backdrop's icons shifted right** — the trash button's `Collapsed` released its column; a new
`HiddenBoolToVisibilityConverter` keeps the slot reserved (`Hidden`), so edit/eye stay in their
fixed columns for every source row.
8. ✅ **No separation between scenes and sources** — a 1px hairline with top/bottom padding now sits
between the two listboxes in the left panel.
### Design decisions
1. **The gain seams are the one source of truth** — `AudioGainProvider` is a thin seam (four Funcs)
so the mixer contract and the mute⇔zero-volume rule are testable without constructing the VM.
2. **Stream-honest AND locally audible** — the game bar's controls now do two things: scale the
loopback on the stream and scale the music in the headphones. Native game audio is untouched
(system output, OBS-style non-monitored); an OBS-style monitor loop is out of scope.
3. **Persist the DisplayName, not the device ID** — the FriendlyName match was already the mic's
identity end-to-end; `Settings` just makes it survive restarts.
---
---
## TASK 12 — Master limiter on the live mix (2026-08-15)
**Queued by the creator during the TRAX discussion:** "should the soundtrack be limited to avoid
squandering resources / should it cap at 20% / how do the three sound events balance?" Review
conclusions (all three instincts checked out, only ONE real gap):
- **File size — no guard needed.** `MusicPlayer` uses `MediaFoundationReader`, which **streams from
disk** (progressive source): memory is flat (~a few MB) regardless of file size; CPU negligible.
- **20% cap — already enforced by construction.** `MusicPlayer.MusicVolume = 0.20f` + music rides the
SAME WASAPI loopback (and therefore the same gain) as game audio, so music:game is always exactly
**0.20:1 at any slider position** — it literally cannot rise above 20% of the current desktop volume.
- **The gap:** `AudioMixer.FillAndMix` summed mic + loopback with **no output ceiling** — mic 100% +
loud game/music could pass 0 dBFS and clip the AAC encode.
**Shipped:**
1. ✅ **`Services/Audio/MasterLimiter.cs`** (pure, unit-tested) — a **−1 dBFS ceiling** (`Ceiling =
0.891`), **instant attack per frame** (a hot frame is scaled exactly to the ceiling — no overshoot),
**smoothed release** toward unity so loud passages don't pump; gain never exceeds 1 (no boosting).
Applied at the end of `AudioMixer.FillAndMix`, right before the pipe write.
2. ✅ **Unit tests** — over-ceiling frames trimmed to ≤ ceiling; sub-ceiling frames never boosted; gain
recovers to unity after the loud frame ends.
3. ✅ **ONE integration test** (`MasterLimiter_CapsTheLiveMix_OnThePipe`) — real mixer + pipe harness:
a 0.95 loopback bed (hotter than the ceiling) is capped to exactly 0.891 on the wire while staying
audible.
4. ✅ **Docs in the same commit** — `ai.md` (go-live audio section), `Services/index.md` (new row).
No changes to `MusicPlayer`, the 0.20 cap, the ducker, or the meter zones. Build 0 warnings.
---
---
## TASK 13 — Social media launch kit
**Goal:** marcom/social media assets and strategy for v1 launch.
**Business details (positioning, messaging, platform strategy, launch assets) in `MARCOM.md` (gitignored).**
### Status: 🔶 Scoped — nothing built; queued after all v1 features ship
1. ☐ Product positioning & messaging (one-liner, elevator pitch, competitive positioning)
2. ☐ Social media swipe files (pre-written posts for supporters)
3. ☐ Launch day assets (demo video, screenshots, GIFs, social graphics, press kit)
4. ☐ Platform strategy (YouTube, Reddit, indie dev communities, Product Hunt)
5. ☐ Founder story (67-year-old dev building his own streaming app)
6. ☐ Email announcement templates
7. ☐ "Build in public" livestream angle (stream the coding of ytLlive with ytLlive)
---
## TASK 14 — Creator feedback batch: quick fixes (TRAX, meter, backdrop, sliders)
**Goal:** address creator's immediate UX feedback from testing session.
### Status: 🔶 In progress — Branch 2 shipped
1. ✅ TRAX volume — `LocalGain = GameAudioVolume * 0.2` so TRAX is 20% of the desktop audio slider (was 100%, now quieter)
2. ✅ TRAX tooltip — replaced `ToolTip` binding with `ToolTipService.ToolTip` + `Placement="Top"` to fix z-order inside Viewbox overlay
3. ✅ Mic meter boost — `MeterLevel` multiplied by 1.2× before clamping so the bar reads ~20% higher at same input level
4. ✅ Click-to-position sliders — clicking anywhere on the mic/game volume slider track now jumps the thumb to that position before starting the drag
5. ✅ Backdrop visibility — the dedicated `<Image>` now binds `Visibility` to `BackdropVisible` (ViewModel property); compositor checks `IsVisible` before blit
6. ✅ Rename "Backdrop" → "Game Capture" in `EnsureBackdrop()` + test assertions
7. ✅ Elements panel — new "ELEMENTS" section beneath Sources list; per-element config with live preview, ✕ revert, ✓ applied indicator
8. ✅ Webcam border config — color hex input + color swatch picker, border thickness 0–10 slider
9. ✅ Countdown source — `SourceType.Countdown`, pick list (Starting/BRB scenes only), timer minutes 1–60 default 15
10. ✅ Web source — `SourceType.WebSource`, pick list (all scenes), URI text input
11. ✅ Elements scrollbar deselect fix — clicking the Elements panel scrollbar no longer deselects the selected element (scrollbar lives inside the ScrollViewer, not the StackPanel; guard now checks `ElementsScrollViewer` instead of `ElementsPanel`)
12. ✅ Web source URI + buttons inline — merged URI TextBox and check/revert buttons into a single DockPanel row (✓ rightmost, ✕ left); buttons no longer wrap to a separate line
13. ✅ Web source URI hidden for webcam — URI section was showing for webcam due to WPF binding path `SelectedElement.Type` not resolving on `WebcamSceneConfig`; replaced with `IsWebSource` virtual property on `SceneElement` (mirrors existing `IsWebcam` pattern), overridden in `Source` to return `Type == SourceType.WebSource`, bound via `BoolToVis`
14. ✅ Check/revert consistent order + accept — all three element sections (webcam border, countdown timer, web URI) now use consistent [✕][✓] button order; green check buttons enabled with click handlers that save current values as snapshot baseline (so revert undoes to last accepted state, not initial selection)
---
## TASK 15 — Stock scene background images
**Goal:** branded background images for each canonical scene (Starting, Live, BRB, Chat, Ending)
that both provide working hints to creators and self-promote LlamaCasty.
### Status: ✅ Shipped — all 5 scenes seeded
1. ✅ **Starting backdrop** — `Assets/starting-backdrop.jpg` (embedded resource), `SeedStartingBackdrop()` in MainViewModel, idempotent
2. ✅ **BRB backdrop** — `Assets/brb-backdrop.jpg` (user-provided Gemini image), `SeedBrbBackdrop()` in MainViewModel, idempotent
3. ✅ **Live backdrop** — `Assets/live-backdrop.jpg` (user-provided image), seeded via `SeedLiveBackdropAsset()`, used as the static fallback (TASK 16)
4. ✅ **Ending backdrop** — `Assets/ending-backdrop.jpg` (user-provided image), `SeedEndingBackdrop()` replaces existing (not skip-if-exists)
5. ✅ **Chat backdrop** — `Assets/chat-backdrop.jpg` (user-provided image), `SeedChatBackdrop()` at startup
6. ✅ `EnsureDefaultBackdrop()` called on every scene switch — seeds the default backdrop if scene has none
### Design decisions
- Images are user-provided, not generated — the creator owns the brand look
- Each scene gets its own image (Starting = "Starting Soon", Live = live branding, BRB = "Be Right Back", etc.)
- The images double as the no-game backdrop fallback (TASK 16), so the Live scene image should work as a desktop replacement
---
## TASK 16 — Kill infinity display (no-game backdrop fallback)
**Goal:** when no full-screen game is detected, show a static branded placeholder instead of
capturing the primary display (which causes the infinity mirror effect).
### Status: ✅ Shipped
**Changes:**
1. `ResolveAutoCaptureKey()` returns `null` when no full-screen game is detected (was: primary monitor)
2. `ReacquireScreenCaptures()` — clears stale `CaptureKey` when `auto == null` (handles DB upgrades);
skips `AcquireAsync` when no keys are set
3. `SeedLiveBackdropAsset()` — sets `AssetId` on the backdrop element itself (NOT a Background source);
called from `ReacquireScreenCaptures()` after `EnsureBackdrop()` guarantees the element exists
4. `Source.DisplaySource` falls back to `_imageSource` when `VideoImageSource` is null for live captures
5. `Assets/live-backdrop.jpg` — user-provided image, added as `<Resource>` in csproj
6. `ResolveOutputFrame()` in compositor — `Source { IsLiveCapture: true, CaptureKey: not null }` already skips null keys; fallback pattern `Source { AssetId: not null }` picks up the static image
**Rendering flow:**
| Scenario | Preview | Compositor |
|---|---|---|
| Game detected | Live capture (opaque, covers static asset on backdrop) | `ScreenCaptureManager.GetLatestFrame` |
| No game | Static image via `DisplaySource` → `_imageSource` on backdrop | `StaticPixelCache.Get(assetId)` |
| Manual capture pick | Live capture | Live frame |
---
## TASK 17 — Web source rendering (WebView2)
**Goal:** make the web source actually render URLs into the preview and stream output.
### Status: ✅ Done — required for v1 (2026-08-28)
1. ✅ Add `Microsoft.Web.WebView2` NuGet package
2. ✅ Schema v10: `WebUri TEXT` column on `Source` table + migration in `LayoutStore.cs`
3. ✅ Persist `Source.WebUri` on save/load (currently in-memory only — lost on restart)
4. ✅ Hidden off-screen `WebView2` control per web source — navigates to `WebUri`, renders in-app
5. ✅ Frame capture from WebView2 (`CoreWebView2.CapturePreviewAsync`) → `WriteableBitmap` (BGRA8)
6. ✅ Wire into `FramePump` resolver — `Source { Type: WebSource }` → latest WebView2 frame
7. ✅ Wire into `SceneCompositor` — render web source as an image element at its position/size
8. ✅ Preview shows live web content (not just a blank rectangle)
9. ☐ Handle navigation errors, invalid URIs, timeout gracefully
**Rendering model (2026-08-28, canvas-size viewport + ALPHA-BBOX crop — DONE, creator-verified):**
the page renders at the MASTER CANVAS size (1920×1080), stable, never tracked (no reflow/truncation;
scrollbars suppressed via `overflow:hidden`). `FindContentBounds` scans the Bgra32 capture and crops
to the bounding box of non-transparent pixels — the widget's true rendered extent, measured from the
frame itself (NO DOM query, immune to layout timing; can never truncate content; full-canvas widgets
fall through to the full frame = prior verified-good image). `Stretch="Fill"` maps the cropped frame
flush under the selection box → box hugs the widget on all four sides. `QueryContentBoundsAsync` +
the JS content-bounds script are DELETED. **RESULT: bounding verified PERFECT by the creator with
two widgets.** Root-cause note on the earlier "remaining defect/gap": that was a WIDGET GLOW EFFECT
(the widget's own CSS glow pushes out its perceived borders), not a code bug — the alpha-bbox was
already hugging the glow halo. Lesson: test with a second, plain widget before changing code.
⚠️ Lesson bank: never return `JSON.stringify` from `ExecuteScriptAsync` (double-encodes); never
measure DOM stuff on NavigationCompleted (unsettled layout broke the image — `a5b9952`); avoid
sizing the container to the crop (`ab29ec8` reverted — made it worse); a widget glow effect can
fake a gap — verify with a plain widget first; measure rendered pixels instead.
**Properties panel (2026-08-28):** web URI ✓/✕ icon buttons are `IsTabStop="False"` so Tab flows
X→Y→W→H→URI; the ✕ button now clears the URI textbox (was reverting to the pre-accept snapshot).
Slider style gained `IsMoveToPointEnabled="True"` — click-anywhere-on-bar jumps the thumb to the
click (volume sliders keep their manual `SetSliderValueFromClick`, harmless duplication).
### Design decisions
- WebView2 is the only option for Windows — it's pre-installed on Windows 10 20H2+ and Windows 11
- The web source is a standard element — positioned/sized/opacitied like any image source
- Frame capture rate can be lower than video FPS (5-10 fps for web content is fine)
- This enables Streamlabs/StreamElements overlays via web URLs
---
## TASK 18 — Local recording
**Goal:** record the stream output to a local file, with or without simultaneously streaming.
### Status: ✅ Shipped `a9eb360` (2026-08-29) — code done (incl. manual-rename modal), build 0 warnings, 244/246 tests; **running-app verification (takes 1–5 done)**: files land (ffmpeg re-pinned month-end), **webcam-in-output + social bar visually CONFIRMED from take 3's extracted frame**, rename modal used for real (take 3 was named via it); take 3 exposed the ~2fps producer starvation and the pump stage-timing (`97ffc42`) named it in one line — `avg render 258.1ms` — **slice 1 fixed 2026-09-03** (deadline pacing per OBS video-io.c + libyuv-style row blits in `SceneCompositor`, test `Pump_Paces_To_The_Deadline_Compensating_Render_Cost`); **take 4: pacing held but render stayed 58.9ms** (2M-iteration row walk + 8.3MB/tick LOH) — **slice 2 shipped 2026-09-04**: `VideoFrame.IsOpaque` producer-contract flag → full-cover backdrop is one `Buffer.BlockCopy`; integer fixed-point bilinear general path; pump scratch pool (release strictly post-submit, owned-by-reference so cache frames are untouchable); dead per-tick `fromScene` render + the `fromSceneProvider` seam removed (BlendFrame uses `TransitionService.FromFrame` — the old render fed nothing). Tests `Pump_Pools_ScratchBuffers_Across_Frames_Without_Stale_Pixels` (the ONE) + `Composite_OpaqueFullCover_Backdrop_CopiesEveryPixel_Into_Scratch`; 59/59 per-class green, clean build 0 warnings. **take 5 ran: render 58.9→25.5ms (`138/300` ≈ 2.2x still) — cause: the per-tick chat raster (`RenderFrame` hit `RenderTargetBitmap` every tick whenever the message buffer was non-empty — the buffer survives sessions); slice 3 shipped 2026-09-04: `ChatOverlayLayer` rasters on message/config change and blits a cached frame every tick (OBS text-source pattern; test `ChatOverlayLayerCacheTests`).** **take 6 ran WITHOUT attribution (35-41ms — build provenance unknown; slice 3 effectiveness UNCONFIRMED) → build-stamp shipped instead of guessing again: `Helpers/BuildStamp` GUID per build (csproj GenerateBuildStamp; incremental builds can no longer lie), wordmark superscript + startup.log line; stats split `render (resolve)` so the next take names the stage. **takes 7–8 (stamped f190587b): chat fix CONFIRMED (`resolve ≈0`) but render stayed 26-27ms — compositor re-rasterizing STATIC layers every tick; slice 5 shipped: `BlitCachedLayer` pastes once-rasterized element-space layers (OBS surface-cache pattern; test `PasteCache_RepeatRender_IsByteIdentical_And_ContentChangePropagates`). **take 9 ran (c65a3cde, paste cache): render 26.5→22.4ms yet period stayed ~37ms — the gap was Task.Delay's 15.6ms sleep quantum padding every sub-tick wait: the true ceiling, hidden until then (why takes 7→9 looked like zero change). Slice 6 shipped: timeBeginPeriod(1) for the pump life + bulk-sleep + 2ms spin tail + `avg wait` stat (accounting closes) + webcam now routes through the paste cache. **take 10 ran (59a02a5b): the new `wait` stat exposed the LAST structural bug — `render 22 + wait 10` against a 16.7ms deadline is impossible for a rebasing pacer: the wait was the producer QUEUED BEHIND THE LIVE PREVIEW — the pump's await-continuations inherit the UI SynchronizationContext (StartAsync fires from a command handler), so the loop had been rendering on the dispatcher all along. Slice 7: Task.Run the loop (OBS keeps media threads off-UI for this exact reason), StaticPixelCache locked + chat raster marshalled to the dispatcher on cache-miss (RTB/DrawingVisual are UI-thread objects), SustainedLowLatency GC, `worst render` stat, webcam routed through the paste cache; test `Pump_Produces_OffTheStartingContext`, 70/70 green. **take 11 ran (c10ce06c): off-UI loop WORKED — typical frames land exactly on the 16.7ms deadline (work ~10 + wait ~6.8; 212/300 best yet); the remaining gap is periodic 35-65ms spikes worsening across a take = gen2 GC pauses, fed by the capture path's fresh ~8.3MB array per DWM frame (~500MB/s). Slice 8: 4-deep capture buffer ring + `VideoFrame.Epoch` in the identity-keyed paste cache + `gen2 +N` printed per stats window; test `PasteCache_RecycledArrayWithNewEpoch_ReRasterizes_NotStaleHits` (fails on the old key), 37/37. Also this session (creator ask): post-mix master gain +40% (before the −1dBFS limiter, so no new clipping). **take 12: `gen2` ~0-1/window, `worst render` → ~16-20ms, n/300 → 300, audio ~40% louder in the file.**** Known remaining churn (follow-ups, not silently done): vertical tier's final `BilinearScale` still allocates per frame; one slow tick (~15-25ms) per arriving chat message — debounced off-tick re-render if take 6 shows burst loss. **User UX spec (2026-09-04) queued behind this**: two-line top bar (LIVE rename, radio pills, Login/Logout button + avatar right-click Change Account, no account light; line 2 centered Start↔Stop grayed-until-armed) + up-front SaveFileDialog for REC (native overwrite prompt; retires the stop-time rename modal) + Go-Live dialog KEPT as preflight confirmation prefilled from the Text drawer — decisions captured in HANDOFF
1. ✅ `EncoderOptions` extended with `StreamEnabled` / `RecordEnabled` / `RecordPath` (independent intent flags)
2. ✅ `FfmpegArgs.Build` reworked into per-output blocks (stream `-f flv`, record `-f mp4`) via `AddVideoTags`
3. ✅ Two modes via pills: record-only / stream-only (single ffmpeg, one output). ~~stream+record~~ — **REMOVED by creator ruling 2026-09-01: the VOD is already the copy; dual encode drags mid-range hardware and degrades both outputs ("we're not them"). Pending implementation: pills become mutually exclusive radios + `BeginGoLive(alsoRecord)` param dies.**
4. ✅ Top-bar REC + ON-AIR pill toggles, status lights (REC green when recording, ON-AIR green when live), dynamic primary-button text, account status-light tooltip
5. ✅ Record-folder persistence (`LayoutStore` `RecordFolder` key) + picker (`ChooseRecordFolderCommand`)
6. 🟡 Branding flash carries into local recordings — same frame path as streaming (verify in the running app)
7. ✅ Output folder `%APPDATA%\ytLlive\recordings\` default; auto-name `ty-<yyyymmdd>-<hhmm start>-0000.mp4` at start, **rename-on-stop** to `ty-…-<hh2mm2 length>.mp4` (numeric suffix on collision)
Remaining: running-app verification of the rename + dual output.
### Design decisions
- **Record OR stream, never both (creator ruling 2026-09-01):** YouTube's VOD already keeps the copy
(downloadable from Studio), and a second concurrent encode degrades BOTH outputs on anything short of
a performant chassis. Recording's real jobs: the offline mux-check bench and the no-account session.
Pills become radios (arming one disarms the other); the one-stop ruling ("Stop ends everything")
already assumes a single session kind.
- FFmpeg supports multiple outputs natively (`-f flv rtmp://... -f mp4 file.mp4`) — kept as generic machinery, but only one block is ever enabled now
- MP4 is the default container; crash-safety arrives as **fragmented MP4** (TASK 32 slice 4 — in record mode the local file IS the archive), retiring the old "MKV as an option" note
- Record-only mode is useful for pre-recorded content or testing without going live
- The branding flash carries into local recordings (free tier billboard extends to recordings)
---
## v1 execution order
The tasks below are ordered by dependency and risk. Each task builds on the previous.
1. ✅ **TASK 9.4** — Live chat (right panel) — wire `YouTubeChatService.Start()`, parse messages, render in panel. Foundation for chat box source.
2. ✅ **TASK 3.18** — Chat box source — renders chat ON the stream. Depends on TASK 9.4 (same message parsing).
3. ✅ **TASK 10** — Polar billing — license key entry + watermark toggle + Velopack auto-updates (steps 1-7 shipped; Velopack update URL pending).
4. ✅ **TASK 19/23** — Control Surface UX — director's control room: thumbnails above central monitor, transitions (Cut/Fade/Move), edit mode offline only, left panel two-state (layers/props ↔ chat), right panel eliminated. Verified shipped 2026-08-24.
5. ✅ **TASK 20** — Hotkeys — global keyboard shortcuts. Shipped 2026-08-26: F1-F9 defaults + config UI with modifier chords, persistence, conflict detection, unbinding.
5b. ✅ **Broadcast metadata pull-out + launch geometry** — shipped 2026-08-24 (row 29 above). Live-screen "Text" tab → broadcast form, persistence + remote update; window default/minimums bumped (1920×1040 / 1366×768) with size+position restore.
6. ✅ **TASK 17** — Web source (WebView2) — shipped 2026-08-28; enables alert ecosystem.
7. ✅ **TASK 18** — Local recording — shipped 2026-08-29 (running-app verification pending in handoff).
8. 🔶 **TASK 21** — Media source — video file playback for non-Live scenes. **Remaining: the UI picker slice** (handoff spec in HANDOFF.md).
9. ✅ **TASK 22** — Audio sync offset — shipped 2026-08-31 (status reconciled 2026-09-01).
10. ✅ **TASK 15** — Stock bg images — all 5 scenes seeded (Starting, BRB, Live, Chat, Ending). `EnsureDefaultBackdrop()` runs on every scene switch.
11. **TASK 32** — Stream resilience (blip retry → grace countdown on the ON-AIR sign → one-click Back on air → crash-safe recording → pre-flight).
12. **TASK 33** — Bandwidth auto step-down (depends on TASK 32's restart machinery).
13. **TASK 34** — Scheduled streams (Text-drawer schedule + adoption at Start).
14. **TASK 35** — Scene-linked audio ("scenes remember the room").
15. **Text source (TASK 3 item 17)** + **the monetization-awareness chain + Alerts (TASK 10 related work → TASK 3 item 20)** — capture → report → journey → alerts, one slice per integration test.
16. **TASK 13** — Social media launch kit (MARCOM) — after features, before gold.
17. **TASK 36** — Gold pass, always last: visibility unlock + flash-live enable + screens/layers audit + native verification suite + expiry reminders + signing/installer/EULA/Velopack.
---
## TASK 19 — Scene transitions → MERGED into TASK 19/23 (Control Surface UX)
**Superseded by the Control Surface UX task.** Cut/Fade/Move transitions, thumbnails, edit mode, left panel two-state — all shipped as one cohesive feature. See the Control Surface UX section in `ai.md`.
### Design decisions
- **Cut is free** — instant switch, no blending, zero CPU cost. This is the default.
- **Fade is the minimum expectation** — 300ms crossfade is universal. Every streaming tool ships this.
- **Move is economical** — simple slide animation, no video decoding needed. Good middle ground.
- **Custom is premium** — media/stinger transitions require video playback. Heavy but expected by mid-tier streamers.
- Transitions happen in the compositor, not post-encode. The preview sees the same blend as the live output.
---
## TASK 20 — Hotkeys (keyboard shortcuts)
**Goal:** keyboard shortcuts for scene switching and common actions — the single biggest UX gap.
### Status: ✅ Done — shipped 2026-08-26 (steps 1-2; all 9 checklist items complete; handoff flags it shipped)
1. ✅ `GlobalHotkeyManager` service — `Services/GlobalHotkeys.cs`: registers OS-level hotkeys on the window HWND via `RegisterHotKey`/`WM_HOTKEY` (`IHotkeyRegistrar` seam for tests; `RegistrarOverride` mirrors `LayoutPathOverride`). Wired in `MainWindow.OnSourceInitialized`.
2. ✅ Scene switching hotkeys — F1-F5 stage/transition the five canonical scenes.
3. ✅ Common action hotkeys — F6 start/end stream, F7 mute mic, F8 mute desktop audio, F9 TRAX play/pause. Dispatch lives in `MainViewModel.HandleHotkey` and honors command gating.
4. ✅ Hotkey configuration UI — `HotkeyConfigDialog` (XAML Window + `HotkeyConfigViewModel`). Click-to-capture, "Unbind" per row, Reset to Defaults. Conflict detection shows a warning; user must unbind the conflicting action first. Modifier chords supported from day 1 (Ctrl+F10, Alt+Shift+F12, etc.).
5. ✅ Global hotkeys — work even when app is not focused (that is what `RegisterHotKey` gives us)
6. ✅ Conflict detection — warns when a chord is already bound to another action; user must unbind first.
7. ✅ Persistence — `LoadHotkeyBindings()` / `SaveHotkeyBindings()` in `LayoutStore`, stored as `"modifiers+vk"` in Settings table. No schema migration needed.
8. ✅ Test: `GlobalHotkeyTests.WmHotkey_StagesScene_And_TogglesMic_And_RevokesOnClose` — real window + fake registrar, a genuine `WM_HOTKEY` posted through the window's own `HwndSource` hook stages the mapped scene and flips mic mute; close revokes every registration.
9. ✅ Test: `HotkeyConfigTests` — round-trip persistence, display string formatting, storage serialization.
### Design decisions
- **Global hotkeys are mandatory** — streamers are in-game and cannot alt-tab. F1-F5 must work from anywhere.
- **Modifier chords from day 1** — Ctrl+F10, Alt+F1, etc. for Stream Deck support and power users.
- **Conflict = warning, not swap** — user must unbind the conflicting action before reassigning. No surprise reassignments.
- **Unbinding** — every action can be set to "None" (no hotkey). Explicitly unbound actions are absent from the DB.
- **No Stream Deck yet** — bare keyboard first. Stream Deck support (physical devices) is v1.1+.
---
## TASK 21 — Media source (video file playback)
**Goal:** play video files (MP4, MOV, AVI) into scenes — starting soon videos, BRB loops, intro/outro clips.
### Status: 🔶 In progress — Increment A (model + persistence) + Increment B (decoder) + slice 1 (manager + resolver/preview wire-in) + slice 2a/2b (ffprobe probe + native-FPS pacing) + slice 3 (loop mechanism) shipped. **Remaining: the UI picker slice (AddMedia command + file dialog + Acquire/Release wiring + loop-flag wiring) — GUI, handed off to build/verify natively on Windows (see HANDOFF → "UI picker slice — handoff spec").**
1. ✅ `MediaSourceType` enum: `Video`, `Audio` (audio-only files via media source)
2. ✅ `SourceType.MediaSource` addition to the enum
3. ✅ `MediaSourceModel`: `MediaPath`, `MediaIsLooping`, `MediaVolume` (0-1), `MediaPlaybackState` — persisted in LayoutStore (schema migration + SELECT/INSERT + round-trip test)
4. ✅ `MediaVideoSource` implements `IMediaFrameSource`: FFmpeg raw video decoder → `VideoFrame` pipeline — spawns ffmpeg `-f rawvideo -pix_fmt bgra`, drains the pipe via pure `RawVideoFrameReader` (`Services/RawVideoFrameReader.cs`), raises `FrameAvailable`/`Completed`; lifecycle is `StartAsync`/`StopAsync`; decode process behind `IDecodeProcess`/`FfmpegDecodeProcess` seam (binary-stdout mirror of `IEncoderProcess`). Shipped 2026-08-31; refactored onto `IMediaFrameSource` with this slice.
4b. ✅ Slice 2a — native-FPS probe seam: `FfmpegFrameRateParser` (pure, prefers `avg_frame_rate=` then `r_frame_rate=`, rational N/N/M, unknown→null) + `IFrameRateProbe`/`FfmpegFrameRateProbe` (derives sibling `ffprobe.exe` from the located ffmpeg dir, reuses the `IDecodeProcess` seam; null if ffprobe absent) + `FfmpegLocator.ProbeFileName` now also extracts `ffprobe.exe` from the pinned archive (conditional). Tests: `FfmpegFrameRateParserTests` (6 pure units + 1 probe integration via fake locator/process) + `FfmpegLocatorTests` still green — 15/15.
5. ✅ `IMediaFrameSource` + `MediaVideoSourceManager` (`Services/IMediaFrameSource.cs`, `Services/MediaVideoSourceManager.cs`): app-wide decode-session ownership refcounted by `MediaPath` with a `Func<string, IMediaFrameSource?>` factory seam; `AcquireAsync`/`ReleaseAsync`/`ReleaseAllAsync`/`GetLatestFrame`; coalesces frames onto the UI dispatcher onto a single shared `WriteableBitmap` per file (mirror of `ScreenCaptureManager`); `MediaFailed` + `PreviewBitmapChanged`. Unit tests + one integration test (bitmap share/coalesce) — `MediaVideoSourceManagerTests`, 5/5 pass.
6. ✅ Native-FPS pacing (slice 2b): `MediaVideoSource` takes optional `IFrameRateProbe?` + `Func<TimeSpan,CancellationToken,Task>? delay` seams, probes FPS once in `RunAsync`, and delays by 1/fps after each emitted frame; unknown/absent probe → no pacing. Test: `MediaVideoSource_PacesFramesByProbedFps` (fake probe + recording delay, one delay per frame ≈ 1ms).
7. ☐ Wire into `FramePump` resolver — `Source { Type: MediaSource }` → latest video frame — **resolver + preview routing + manager wired; session acquisition (start/stop on add/remove) still open (comes with the UI picker)**
8. ☐ Wire into `SceneCompositor` — render media source as an image element at its position/size
9. ✅ Loop control (mechanism) — `IMediaFrameSource.Looping`; `MediaVideoSource` takes a `Func<IDecodeProcess>` process factory and restarts the decode on natural EOF when `Looping` (fresh process per pass, since a `Process` can't be re-`Start()`ed). Test: `MediaVideoSource_LoopsUntilLoopDisabled` (single frame re-emits across passes, `Completed` only after loop cleared). Wiring `Source.MediaIsLooping` into the flag lands with the UI-picker (acquisition) slice.
10. ☐ Volume control — per-source volume slider for audio playback
11. ☐ UI: file picker (filtered to video formats), loop toggle, volume slider
12. ☐ Schema migration for media source settings (file path, loop, volume)
13. ☐ Tests: video frame extraction, loop behavior, volume scaling, file validation
### Design decisions
- **FFmpeg handles all formats** — no codec-specific code. FFmpeg already in the project.
- **Audio plays through the desktop channel** — media source audio is captured by the WASAPI loopback (like TRAX/game audio). No separate audio routing needed.
- **This pairs with TRAX** — TRAX is background music, media source is background video. Together they make non-Live scenes (Starting/BRB/Ending) feel polished.
- **Not a full NLE** — no trimming, no multi-track, no effects. Just "play this video in the scene."
### UI picker slice — spec (imported from HANDOFF 2026-09-01 so it can't be lost by a rewrite)
Nothing `AcquireAsync`s a media path yet, so **no frames flow in a running app** — decoder, manager,
pacing, and loop mechanism are shipped and unit-tested, but a media Source has no way to start a
session. Deliverables, in order:
1. **File picker + add/remove commands** (mirror `MainViewModel.Trax.cs:68` `OpenFileDialog` usage).
A "Media" source action opens an `OpenFileDialog` filtered to video (mp4/mov/avi); on OK set
`Source.MediaPath`, add to scene, and **`AcquireAsync(MediaPath)`**; on remove
**`ReleaseAsync(MediaPath)`** (refcounted — not `ReleaseAllAsync`, unless the path leaves the
layout entirely). On `StagedScene` layout loads/teardown, release every media path no longer
present and acquire new ones so sessions track the live layout.
2. **Wire `Source.MediaIsLooping` → `IMediaFrameSource.Looping`** — needs a manager-level per-**path**
loop provider. Ambiguity to rule + record: sessions are refcounted per `MediaPath` and shared
across scenes, but `MediaIsLooping` is per-`Source`. Chosen rule: the path's session `Looping` =
**any** live reference has it on (shared behaviour). Record it in the slice commit.
3. **Loop toggle + volume slider** in the source context menu / inspector (items 10/11).
4. `MediaVolume`/`MediaPlaybackState` wiring — audio path for media is future work (desktop loopback
already carries it, TRAX-style; `MediaVolume` slider can drive the local file's audio separately
later — out of this slice).
**Design notes to preserve:** `ResolveOutputFrame` reads `_mediaManager.GetLatestFrame(MediaPath)`;
`OnMediaPreviewBitmapChanged` adopts the shared `WriteableBitmap` per path; `MediaFailed` →
`OnMediaFailed` (currently `Debug.WriteLine` — surface in UI via toast, TASK 24 stack). The compositor
scales any frame size — media renders through the generic `frameFor(element)` path, no compositor
change. First GUI smoke test: add a short real mp4 on native Windows, confirm it plays and previews.
---
## TASK 22 — Audio sync offset
**Goal:** per-source audio delay compensation to prevent lip-sync drift from USB mics and capture cards.
### Status: ✅ Done (shipped TASK 22, 2026-08-31)
1. ☑ `AudioSyncOffsetMs` on `MainViewModel` (global, default 0, range 0..500 ms) — positive-only: OBS's documented fix delays the audio so it lands on video when it runs ahead; advancing audio would need a video-side delay (out of the audio layer's scope, v1.1+).
2. ☑ Applied in `AudioMixer` — post-mix interleaved-stereo delay via the pure `AudioSyncDelay` line (`Services/Audio/AudioSyncDelay.cs`), fed through a `Func<int> syncOffsetMs` seam each mix tick.
3. ☑ UI: compact "SYNC" slider (0..500) on the mic bar with a status dot (green = no-op, amber = offset set) via `IntToSyncBrushConverter`. **Provenance (recorded 2026-09-01 after a creator "I never ordered this" scare): the creator explicitly asked for OBS's delay-filter lip-sync fix BUILT NATIVELY — the feature is his, not AI drift. Its placement (preview-rail vs mic-settings/gear) remains an open design question; do not remove the capability.**
4. ☑ Persisted in `LayoutStore.Settings` (`Audio.SyncOffsetMs`) via `LoadAudioSyncOffsetMs`/`SaveAudioSyncOffsetMs`; saved from `SaveLayoutNow`.
5. ☑ Tests: `AudioSyncDelayTests` — zero-delay identity, negative→0 clamp, >500 ms clamp to 500 ms, and 10 ms → 960 interleaved-sample shift.
6. ⚠ **Post-ship regression (found 2026-09-01, first real launch):** this task's sync-delay line `_delayedMix` was declared nullable and never initialized — the first `delayed.Length` deref NRE'd EVERY live-mix tick, silently killing all live/record audio, hanging `AudioPipelineTests`, and being mislabeled a "known failure". Fixed in the recording-verification pass (init + null-check + throttled loop errors logged with stack); `AudioPipelineTests` 25/25 green afterward. Lesson recorded in MyMistakes.
### Design decisions
- **Global offset first** — one setting for all audio sources. Per-source is v1.1+.
- **Positive-only (delay audio)** — the physically-correct direction (audio runs ahead of the video). True "advance" needs a video-side delay and is PERMANENTLY OUT with per-source sync (TASKS.md → "Out of product" — v1.x phrasing retired 2026-09-01).
- **Simple slider** — 0 to +500 ms, default 0. No numeric input needed.
- **Visual feedback** — "sync OK" status dot shows when an offset is dialled in.
---
## TASK 23 — Studio mode → MERGED into TASK 19/23 (Control Surface UX)
**Superseded by the Control Surface UX task.** Studio mode is now the entire app's paradigm — the director's control surface with 5 thumbnails + central monitor. See the Control Surface UX section in `ai.md`.
---
## TASK 24 — Toast notifications (in-app, non-blocking)
### Status: ✅ Done (shipped 2026-08-23, branch `task24-notifications`)
Replace blocking MessageBoxes with in-window toasts; promote actionable log-only failures.
### Requirements:
1. Library: **Notification.Wpf 11.0.0** (Platonenkov fork of Federerer/Notifications.Wpf) — MIT, active, targets net8.0-windows. Notice #10 added to `THIRD-PARTY-NOTICES.txt`.
2. Placement: bottom-right above the footer (`NotificationArea x:Name="ToastArea"`, last child of MainWindow's root grid → topmost z-order, `Grid.RowSpan=4`, `Margin="0,0,12,96"`, outside the preview Viewbox), max 4 stacked.
3. Behavior: Info ~4s / Success ~4s / Warning ~8s auto-dismiss; **Error sticky until dismissed**.
4. Styling: dark card tints matching the health-banner language (slate `#3a3b52` info, green `#1d5c38` success, amber `#b8860b` warning, dark-red `#8f1f1f` error); corner radius 8; keep-visible-on-hover.
5. Threading: all Show calls marshal via a Dispatcher captured at construction (encoder/pump events fire off-thread).
### Migrations (MessageBox → toast):
- Webcam acquire failure ×2 (`AddWebcamToStagedSceneAsync`, `SwapWebcamIdentityAsync`) → Warning
- Image read failure (`PickImageBytes`) → Warning
- Sign-in unsuccessful → Warning; sign-in failed → Error
### Promotions (log-only → toast, each keeps its AppLog line):
- Go-live prep ×3: reusable stream unavailable / broadcast insert failed / prep exception → Error
- Frame-pump death while live (`OnFramePumpFailed`) → Error
- Mic missing at startup (`StartMicCaptureAsync`) → Warning (once)
- Premium lapse at startup re-validation → Warning
- Saved-session refresh failure (signed out notice) → Info
Deliberately left alone: offline license re-validation skip (log-only — would spam every launch); SocialsDialog slot-delete YesNo confirm (stays modal).
### Tests:
- Unit: pure severity→request mapping (`NotificationServiceTests`) — area routing, lifetimes, tints.
- Integration (the ONE): real window-hosted `ToastArea` + real service — Info auto-dismisses, Error sticks (`NotificationAreaIntegrationTests`, RealApp collection).
### Library facts (cost a hunt — see also ai.md):
- Area routing matches the area's XAML `Name` against the request's `AreaName`; unknown name silently drops.
- `NeverExpires()` = `ExpirationTime = TimeSpan.MaxValue` (not null); `NotificationColor.ToHex()` returns `#AARRGGBB`.
- Overlay-window shutdown caveat avoided entirely by using the in-window area mode.
---
## TASK 25 — Background consolidation: one locked Background per screen + mini-view rule
### Status: ✅ Done (shipped 2026-08-23, branch `task25-backgrounds`)
The creator rejected the multi-concept background model ("why are there multiple background
things?"). Locked model, verbatim intent:
1. **Exactly ONE background per screen**, named "Background", at position 0.
2. It cannot be re-ordered or deleted; **no Background/Screen item in the (+) menu**.
3. Context-sensitivity exists **only on the Live screen** (Show Desktop toggle + monitor
switching — behavior kept intact).
4. **Exactly five default background images** live in the DB; stale rows are cleaned up.
### What shipped:
- **Seeder consolidation:** the five near-clone seeders (`Seed{Starting,Brb,Ending,Chat}Background`,
`SeedLiveBackgroundAsset`, `EnsureDefaultBackground`) are gone. One path now:
`EnsureBackground(scene)` → `CreateBackground(name)` — Live = `DisplayCapture` row (capture
machinery untouched), everything else = static `Background` art row; both named "Background".
- **Heal on every load:** `NormalizeBackgrounds(scenes)` keeps the correctly-flavored row,
converts a wrong-flavor survivor in place (`Source.Type`'s setter derives `IsBackground` —
conversions must re-assert the flag), drops duplicates, seeds missing ones, renames, pins to
index 0; non-canonical scenes lose backgrounds + flag. `HealBackgrounds()` stamps default art
(`Assets/{scene}-background.jpg` via `AddAsset`; custom Browse art wins). This fixed the real DB's
rot: four scenes carried a stray "Game Capture" duplicate beside their static row, and Settings
held 16 orphaned `BackgroundUseDefault_{guid}` keys.
- **Settings purge:** `LayoutStore.Save` deletes `BackgroundUseDefault_{id}` / `BackgroundPath_{id}`
keys whose id is no longer a Source row.
- **(+) menu:** Screen + Background items removed → Webcam/Image/Text/Countdown/Web/YouTube Chat;
`AddSource` also hard-refuses DisplayCapture/WindowCapture/Background parameters.
- **Capture controls Live-only:** `CanChangeBackground` = staged scene is Live. The preview
CanvasGrid menu (Show Desktop/Capture Desktop/Refresh Desktop) binds it directly; the layer-row
context menu MultiBindings it with the row's `IsBackground` through a new
`Helpers/AllTrueToVisibilityConverter`. Non-Live screens keep the Use-default pill + Browse.
- **Mini rule:** minis never render live captures. While Live is staged its mini shows the green
placeholder; unstaged it shows `live-background.jpg`. Fixed by making `LoadBackgroundImage`
flavor-blind (`IsBackground`) and gating staged-Live to the placeholder in `RefreshSnapshotsAsync`.
(Real-time rendering stays center-monitor-only — preview lag during live gameplay is a known
unsolved OS-level problem and must not be compounded.)
### Tests (suite went 221/218 → 223/220; the two stale background-policy landmine tests healed here):
- Integration (the ONE): `BackgroundHealIntegrationTests` — seeds a dirty temp DB (duplicate rows,
misnamed Live row, orphaned keys), drives the real window, asserts one Background per scene at
index 0 with correct flavor/name, save purges the orphaned keys.
- Unit: SceneCatalogTests rewritten for NormalizeBackgrounds/EnsureBackground flavors;
BackgroundTests empty-scene flavor updated; SourceNamingTests excludes the always-present
Background from numbered-name expectations.
---
## TASK 30 — Gear menu cleanup + Default Location for Recordings + preview layout restructure
### Status: ✅ Done (shipped 2026-08-29, commits `6031a56` → `61b850f` → `455a409`)
The creator's KISS review: gear menu cleanup, single Start button, record-location setting,
and a full layout restructure moving all live controls into the preview pane.
### What shipped:
**Gear menu cleanup:**
- Removed `Save Layout` / `Save Layout As…` / `Open Layout…` from Gear menu + dead VM
commands/methods (`SaveLayoutCommand`, `SaveLayoutAsCommand`, `OpenLayoutCommand`,
`SaveLayoutAs()`, `OpenLayoutFile()`). Multi-layout = use OBS.
**Default Location for Recordings (App Settings):**
- First real setting panel in the Gear menu: "Default Location for Recordings".
- `DefaultRecordFolder()` (internal static): configured → `%USERPROFILE%\Downloads` → `MyVideos`.
- `RecordFolderDisplay` property, `ResetRecordFolderCommand`, Browse + "Use Downloads" reset.
- `TextBox` binding fixed to `Mode=OneWay` (get-only computed property).
- ONE fallback test: `DefaultRecordFolder_Falls_Back_To_Downloads`.
**Single Start button:**
- Button `Content="Start"` hardcoded in XAML. Removed `PrimaryButtonText` / `IsStreamingStart`
properties + all `OnPropertyChanged` references. Pills = intent, lights = state, button = trigger.
**Preview layout restructure — all live controls under preview:**
- Socials button moved from footer (scenes/sources column) to preview bottom row, left of
TRAX button. `MouseEnter`/`MouseLeave` handlers pulse the `SocialBarElement`
`DropShadowEffect` (sine wave BlurRadius 24→48, Opacity 0.7→1.0 at 30ms tick).
- **Mic controls moved from footer into preview pane** (vertical StackPanel below Socials+TRAX):
MIC button + sound meter + mute grid + volume slider — the same footer controls, relocated.
- Preview bottom row: vertical `StackPanel Grid.Row="2"` → Line 1 (Socials + TRAX), Line 2
(MIC + meter + mute + slider). All centered.
**Bottom bar restructured:**
- Reduced to single row (mic row removed), tighter padding (`Padding="16,4"`).
- Gear icon moved to far left. Stream stats (Bitrate/FPS/Dropped/Duration) centered under
preview pane. Resolution dropdown stays far right.
**Avatar fix:**
- Avatar `BitmapImage` no longer loads in the constructor (stale image from last session).
- Only loads when `IsConnected` fires `true` via `OnViewModelPropertyChanged`.
### Tests:
- Build 0 warnings, 247 tests (2 pre-existing known failures excluded).
---
## TASK 31 — SceneGraph component + static/dynamic compositor optimization
**Goal:** extract the scene collection, element inventory, and mutation surface into a standalone
`SceneGraph` component. Classify elements as static or dynamic. Enable the compositor to bake
static layers once and only recompose dynamic layers per frame.
### Status: ✅ Done (2026-08-31)
What landed — core optimization + SceneGraph component, all in one commit:
- `ElementKind` (Static/Dynamic) on `SceneElement` base, overridden in `Source` and `WebcamSceneConfig`
- `Services/SceneGraph.cs`: owns `Scenes` collection (ViewModel's `Scenes` property delegates to it);
mutation surface `AddElement`/`InsertElement`/`RemoveElement`/`MoveElement` (each invalidates the bake);
queries `GetBackground`/`GetWebcam`/`GetChatBoxes`/`GetSplitPoint`/`IsStatic`; and the
`GetBakedBase` bake-cache (keyed by scene id + static element identities below the split)
- `SceneCompositor.BakeStaticBase`/`CompositeLayers` + `Render(.., staticBase, split)` overload —
split-aware, builds the cached base in source-rect space, composites dynamic/above-split per frame
- `FramePump` takes an optional `SceneGraph` and uses the optimized path when wired (falls back to full render)
- `MainViewModel`: routes element mutations through the graph; invalidates the bake on static
layout/opacity/visibility/useDefaultBackground changes and after background heal / EnsureBackground
- Integration test `SceneGraphTests.BakedStaticBase_WithDynamicLayer_CompositesCorrectly` (audio-free,
verifies bake-once + cache hit + dynamic-on-top pixels + static-mutation invalidation +
dynamic-only non-invalidation)
**Defensive deviations from this spec (decision 2026-08-31):**
1. `ChatOverlayLayer` keeps taking `IEnumerable<Scene>` rather than `SceneGraph.GetChatBoxes()` — it is
deliberately decoupled from the graph (`// without owning the scene graph`). Forcing the graph in would
couple a WPF-bound layer to it and violate that documented seam.
2. Background static helpers (`EnsureBackground`/`CreateBackground`/`NormalizeBackgrounds`) stay on the
ViewModel because `BackgroundTests.cs` unit-tests `MainViewModel.EnsureBackground` directly. Queries
moved to SceneGraph; helpers + their invalidation wiring stayed on the VM.
3. The full ViewModel-as-facade extraction (moving the ~all-binding-surface off Scenes/StagedScene/
LiveScene) was deliberately NOT done in this unsupervised pass — it is the "touch 11 files across 4
layers" regression risk the handoff flagged. SceneGraph owns the collection + mutation surface now;
the remaining VM call sites binding to those still work because `Scenes` delegates. Revisit after 1.0.
**Verification:** SceneCompositorTests 4, StretchMathTests 4, BackgroundTests 16, SceneCatalogTests 18,
LayoutStorePersistenceTests 12, FramePumpTests 9, SceneGraphTests 1 all green. RealAppHost GUI/collection
tests (SourceNaming, RoundClip, BackgroundHeal, ...) construct a real `MainWindow` — **CORRECTED
2026-09-01:** they DO run from WSL when invoked per-class through the Windows `dotnet.exe` vstest host
(the old "cannot run headless" claim conflated them with the full-suite WASAPI hang). Two first-launch
crashes this refactor shipped with were caught on the first real native launch and fixed same day:
ctor-order `SceneGraph` NRE (`null!` field assigned after first ctor use — now field-initialized) and
window-scope `EyeButton`/`EyeIconStyle` consumed via `StaticResource` from the extracted `LeftPanel`
(UserControl namescopes can't see window resources — styles moved to `Themes/Controls.xaml`, the
app-scope rule honored). The RoundClip "known failure" was then root-caused to stale test code
(`window.FindName` across namescopes + `VisualTreeHelper.HitTest` where `UIElement.InputHitTest`
models input) — test green 2026-09-01, the sole remaining known failure is the audio one.
### Design
**Element classification:**
Every `SceneElement` exposes `ElementKind Kind` — `Static` or `Dynamic`:
| Type | Kind | Why |
|------|------|-----|
| `Source` (Background art) | Static | Pixels don't change at runtime |
| `Source` (Image) | Static | Pixels don't change at runtime |
| `Source` (DisplayCapture) | Dynamic | Live game/desktop feed |
| `Source` (ChatBox) | Dynamic | Live chat messages arrive continuously |
| `Source` (WebSource) | Dynamic | WebView content can change |
| `WebcamSceneConfig` | Dynamic | Camera feed, changes every frame |
**Split point:**
The **split point** is the index of the first dynamic element in a scene's z-ordered `Elements`.
Everything below it = baked base. Everything from it upward (including static layers above dynamic
elements) = composited per frame. If zero dynamic elements exist: entire scene is baked,
compositor skipped entirely.
**BakedSceneCache:**
```
BakedSceneCache {
VideoFrame BaseFrame // composited static layers below split point
int Version // incremented on mutation
SceneElement[] BakedElements // for invalidation tracking
}
```
Invalidation triggers (re-bake): element added/removed/reordered, static element moved/resized/
opacity changed, `IsVisible` toggled on a static element, background asset changed, scene switch.
No invalidation when: only a dynamic element's pixels changed (webcam frame, chat message),
or a dynamic element moved/resized (per-frame compositing concern, not a base rebake).
**Compositor integration:**
```
Render(scene, frameFor, options):
1. Find split point (first Dynamic element index)
2. If split == scene.Elements.Count:
→ return cached base (or bake if stale)
3. Otherwise:
→ start from cached base (or bake if stale)
→ composite elements[split..] on top using frameFor
```
**SceneGraph interface:**
```
SceneGraph {
// Collection
ObservableCollection<Scene> Scenes
Scene? StagedScene
Scene? LiveScene
// Mutation surface (single owner of element operations)
AddElement(scene, element)
RemoveElement(scene, element)
MoveElement(scene, element, newIndex)
// Queries (replace scattered LINQ)
GetBackground(scene): Source?
GetWebcam(scene): WebcamSceneConfig?
GetChatBoxes(): IEnumerable<Source>
GetSplitPoint(scene): int
IsStatic(scene): bool
// Events
SceneChanged
ElementAdded/Removed/Moved
SplitPointChanged
}
```
**What moves into SceneGraph:**
| Current location | Moves to |
|-----------------|----------|
| `MainViewModel.Scenes.cs` — Scenes collection, StagedScene, scene switching | `SceneGraph` |
| `MainViewModel.Background.cs` — `NormalizeBackgrounds`, `EnsureBackground`, background queries | `SceneGraph` (background helpers) |
| `MainViewModel.Sources.cs` — `AddSource`, `RemoveElement`, element inventory | `SceneGraph` (mutation surface) |
| `MainViewModel.Webcam.cs` — webcam queries across scenes | `SceneGraph.GetWebcam(scene)` |
| `ChatOverlayLayer` — `scenes.SelectMany(...).OfType<Source>().Where(ChatBox)` | `SceneGraph.GetChatBoxes()` |
**What stays on the ViewModel:**
The binding surface — `StagedScene` setter still raises `OnPropertyChanged` for
`ShowEmptySceneHint`, `CanAddWebcam`, etc. But now it delegates to `SceneGraph` for actual
state queries. ViewModel becomes a thin binding facade over the SceneGraph (same pattern as
`MainViewModel.Chat.cs` over `ChatOverlayLayer`).
The compositor resolver (`frameFor` callback) stays in the ViewModel/encoder layer — it bridges
WPF concepts (CameraManager, ScreenCaptureManager, ImageCache) into the pure `VideoFrame` seam.
SceneGraph doesn't know about capture sessions; it just knows element kinds.
**Migration path:**
1. Extract `SceneGraph` as a standalone class in `Services/`
2. Move `Scenes` collection + `StagedScene`/`LiveScene` + wiring
3. Move element mutation surface (add/remove/move)
4. Move background normalization/queries
5. Add `ElementKind` to `SceneElement` base
6. Add `GetSplitPoint` + `BakedSceneCache`
7. Update compositor to use split point
8. Wire ViewModel as thin facade
9. One integration test: baked static base + dynamic layer composite
### Tests
ONE integration test: `SceneGraphTests.BakedStaticBase_WithDynamicLayer_CompositesCorrectly`
— a scene with static background + image + dynamic webcam; assert the baked base is cached
(renders once), dynamic layer composited on top, static element mutation invalidates the cache,
dynamic-only pixel change does not.
---
## TASK 32 — Stream resilience (queued 2026-09-01 — merges the orphaned TASK 2 resume clause into the pills/signs architecture)
**Goal:** a dropped stream heals itself when physically possible, tells the truth when it isn't, and
gets the creator back on air in one click. The ON-AIR sign + primary button in the top bar are the
surface — they already mean "reality" (pills = intent, lights = reality, TASK 18/30); reality just
isn't two-state. Resumption of a completed broadcast is explicitly **out of scope** (impossible via
the API — creator ruling 2026-09-01): no lying in the copy.
**Slices (one integration test each, in order):**
1. ☐ **Blip retry** — app-side encoder restart: push dies while live (ProcessFailed / stdin
backpressure death) → bounded-backoff relaunch (2s/5s/10s, N attempts) against the cached
constant reusable-stream ingest URL. Success = push running again while the broadcast is still
live; viewers never know. Fixes the map's stale "ffmpeg does reconnect" claim (corrected
2026-09-01: reconnect flags are input-side only). Test: fake process fails once then starts →
pump recovers without surfacing to the VM.
2. ☐ **Measured grace + the sign** — while the push is dead, escalate the existing health poll
(30s → ~5s). ON-AIR dot goes **amber "RECONNECTING"** with a mm:ss countdown to the observed
YouTube cutoff; the grace length is a **measured, pinned constant** — probe it on a real private
test stream, cite the observation (spin-guard rule; do NOT invent a number). Poll reports the
broadcast `complete` → dot **red "OFF AIR"**, primary button relabels **"● Back on air"** and
pulses. Test: fake status provider drives the dot through live → reconnecting → cut-off.
3. ☐ **One-click Back on air** — the relabeled button fires the existing go-live path with saved
`Broadcast.*` + cached stream, dialog skipped: new broadcast, new VOD (accepted cost, stated
in-product). Test: seeded saved form + fake stream service → click creates + pushes, no dialog.
4. ☐ **Crash-safe recording** — fragmented MP4 on the record block (`-movflags
frag_keyframe+empty_moov+default_base_moof`): plain MP4's trailing moov atom means a power loss
kills the WHOLE file. Rename-on-stop unchanged. Closes the TASK 18 "MKV as an option" design-note
remnant (fragmented MP4 chosen over MKV: single container, YouTube-upload-native — verify player/
editor tolerance, cite the research). Test: `FfmpegArgs` emits the movflags only on the record block.
5. ☐ **Pre-flight** — before broadcast insert, one TCP probe to the ingest host:1935 (~2s timeout).
Unreachable → Error toast, no insert, no ghost broadcast ("check VPN/network"). The "forgot the
VPN" story, solved as prevention instead of recovery. Test: fake probe failure aborts go-live before any API call.
### Design decisions
- **The sign is the status, the button is the action** — no new chrome, no log lines, no spinners.
- **Honest windows:** countdown = observed grace; at zero we say OFF AIR, we don't pretend to reconnect.
- **Why us > incumbents:** constant reusable stream + persisted form make retry/restart cheap; OBS
shows a reconnecting log line and Streamlabs silently dies.
---
## TASK 33 — Bandwidth auto step-down (queued 2026-09-01; v1 per the complete-v1 ruling)
**Goal:** when the upload can't hold the tier, drop it automatically instead of drowning viewers in
freeze frames. The `variable` reusable stream (shipped) makes this zero-API; the governor that decides
WHEN is what TASKS.md:25 always claimed existed and was never built.
1. ☐ Pure `StepDownPolicy` — sliding window over the encoder's reported dropped-frame rate
(`StreamHealth`, already parsed): sustained overrun → downgrade exactly one 16:9 tier
(1080p60 → 1080p30 → 720p60 → 720p30). Hysteresis cooldown; **one-way within a broadcast**
(no oscillation — a recovered connection doesn't climb back mid-show; the creator may manually
step up). Vertical tier stays a manual choice.
2. ☐ Restart machinery = TASK 32 slice 1 (tier change is an encoder relaunch at new W×H/FPS over the
same pipe/URL; the master canvas is 1920×1080 regardless, so geometry never rewrites).
3. ☐ Surface: the resolution badge + dropdown update to the landed tier; ONE Info toast
("Quality lowered to hold your stream") — no nagging after that.
4. ☐ Tests: pure-policy units (drop-rate windows, one-way, cooldown) + the ONE integration: fake
health stream with sustained drops → exactly one lower-tier encoder restart.
---
## TASK 34 — Scheduled streams, Text-drawer version (queued 2026-09-01; option (b) with the creator's placement)
**Scope ruling (2026-09-01):** this is the whole scheduling story — announce a LIVE show (countdown
watch-page + subscriber notifications), then the creator shows up and pushes. "Going live is our
schedule." Scheduled *playout* of pre-recorded files: discussed and DECLINED by the creator.
Premiere/upload scheduling: out of product (see closed list).
**Goal:** announce "Friday 8pm" from inside the app — YouTube notifies subscribers and runs the
channel countdown; come Friday, one click goes live into the scheduled broadcast. Scheduling is a
"my channel" concern, so it lives in the always-visible **Text drawer**, not the go-live modal.
1. ☐ Drawer gains ☑ *Scheduled livestream* + date/time (replacing the read-only "Scheduled Start"
row). Checking + Save **creates** the broadcast: insert `liveBroadcasts` with a future
`scheduledStartTime`, bound to the cached reusable stream — the drawer's Save path grows its first
create operation (today it can only update an existing broadcast).
2. ☐ Invariant: **at most one open scheduled broadcast**; unchecking before airtime →
`liveBroadcasts.delete` (legal in created/ready).
3. ☐ **Adoption:** go-live checks for the open scheduled broadcast and pushes into it instead of
inserting a new one. Mid-stream title/desc edits ride the existing Update path (unchanged).
4. ☐ `LiveBroadcastFormViewModel` owns `IsScheduled` + `ScheduledStart` (nullable), persisted with the
other `Broadcast.*` keys; local time in, RFC3339 offset on the wire — the timezone edge is where the
bugs will live: test it (DST boundary, non-UTC creator).
5. ☐ **Research flags (cite before building — spin guard):** (a) pushing *before* the scheduled start
with `enableAutoStart` — live immediately or held? Decides a "starting early — it goes live NOW"
confirmation. (b) delete semantics for a ready broadcast bound to a reusable stream.
6. ☐ Test (the ONE): schedule → persists + inserts (fake service captures a future scheduledStartTime
+ boundStreamId); start → adopts instead of inserting; uncheck → delete called.
- **Test-phase note:** dark by design while the visibility lock holds (Private notifies nobody); the
creator can still exercise insert/adopt/cancel. Full shine arrives with TASK 36's unlock — pleasing
coupling, deliberate.
---
## TASK 35 — Scene-linked audio: "scenes remember the room" (queued 2026-09-01; spec approved by the creator same day)
**Goal:** the BRB scene mutes the mic; the Chat scene brings it back. The creator stops fiddling with
audio at transitions — the scene carries the room state. OBS needs scripts/plugins; native = the
creator-proof pitch.
1. ☐ `Scene` gains nullable `SceneMutesMic` / `SceneMutesDesktop` — `null` = don't touch (the default;
scenes are passive until told). Schema column bump + roundtrip.
2. ☐ Applied on **live transitions only** (thumbnail click / hotkey), ONCE at entry — never on
offline staging (no surprise stream-state edits while auditioning).
3. ☐ The creator is boss: a manual mute/unmute mid-scene is never overridden until the next entry.
4. ☐ BRB ships `SceneMutesMic = true`; other scenes default `null` — creator's choice.
5. ☐ Edit UI: two checkboxes in the scene's Properties section (left panel, edit mode) — no new chrome.
6. ☐ Test (the ONE): transition applies scene state; `null` leaves state untouched; manual override
survives until the next entry.
---
## TASK 36 — Gold pass (queued 2026-09-01; the going-gate, always the last work unit)
**Goal:** the single deliberate pass that turns the dev product into the shipped product. Nothing here
may be done opportunistically mid-development — each item has a lock/comment pointing at this task.
1. ☐ **Visibility unlock** — remove the `CreateBroadcast` Private override (TASK 9 item 6); dialog
selection rules. The channel-protection stance ends at GA, by hand, here.
2. ☐ **Branding flash goes live** — end the preview-only pre-GA posture: composite the escalating
obnoxious-promo flash onto output + recordings for free users. Paid removes it; that stays the
ONLY paid delta. (Escalation curve finalized here, first.)
3. ☐ **Screens/layers settings audit** — the retired 2026-08-22 landmine (ai.md): fine-tooth-comb
every screen's Background layer properties, pill persistence, context-menu visibility, (+) items.
4. ☐ **Native-Windows verification suite** — the checklist TASKS/HANDOFF never had: real webcam,
real mp4 decode/loop/pacing (TASK 21 picker must have landed), real recording file opens +
plays (fragmented MP4), a real private go-live end-to-end, GUI/RealApp test classes green on
the desktop (they've only ever run against fakes).
5. ☐ **License-expiry reminders** — T-14 / T-7 / T-1 toasts via the existing notification stack
("renews <date> — manage here" → `PremiumUrl`); lapsed: flash returns (the product itself is the
notice) + one Info toast; active paid users see ZERO license UI. Research: does Polar's
validate response carry `expires_at` reliably — cite before building.
6. ☐ **Release engineering** (the `Distribution.md` plan, 644 lines, finally tasked): code signing
(HARD blocker — an unsigned installer is a SmartScreen scarewall), installer + Velopack update URL
(TASK 10 item 8), EULA draft/review, THIRD-PARTY-NOTICES license-texts gate (TASK 4 req 9).
Build posture (agreed ceiling): compile flags max out at `HARDENED` (release-only anti-debug/
tamper checks) + `MOCK_REWARDS` (fake reward payloads for Alerts/dev) — **additive-only**, never
branching normal code paths, both compiled every CI build so they can't rot. Obfuscation /
assembly-splitting stay GA-time decisions, not compile-time ones.
---
## v1 = the finished product (creator ruling, 2026-09-01)
**There is no v1.x.** v1 ships feature-complete: everything in the queue above lands, or is
explicitly buried below. The parking lot is closed — nothing is "deferred to a later version"
anymore; the only two states are **v1 task** and **out of product, permanently**. The 10% margin of
error is this list, bounded and written down — not vibes.
**Monetization posture restated (what "final" means):** free gives everything; the only difference
is the branding flash. License activation flips exactly one bit (`IsPremium` → flash off). Nagware =
the escalating obnoxious self-promo flash itself — never modals, never feature-locks. Pre-GA the
flash is preview-only (see ai.md → Monetization; TASK 36 flips it live).
## Out of product — permanently (the 10% margin, closed list — do not re-litigate)
| Idea | Verdict & reason |
|------|------------------|
| Stream Deck / Loupedeck integration | The device sends keystrokes — **global hotkeys already ARE the integration** (TASK 20). Redundant, not deferred. |
| Per-source audio sync offset | Global offset shipped (TASK 22); per-source is the OBS mixer rabbit hole. Condemned by the Audio Assumption (one knob). |
| Profiles / presets (multi-config) | Same hydra as multi-layout, already ruled at TASK 30: **that's what OBS is for.** |
| Chroma key / background removal | Already ❌ (TASK 3 item 19). NVIDIA Broadcast does it free; model+GPU+lighting caveats = support hell for a solo dev. Stays buried. |
| Virtual camera output (Zoom/Discord) | Different product, device-driver swamp. Our output is one YouTube stream. |
| Replay buffer (instant replay) | Game-clipping culture, not our persona. **The most valuable corpse here** — if the list is ever reopened at all, this is candidate #1, and it reopens by creator ruling, not by AI suggestion. |
| Multi-destination restreaming | "Casual streamers outgrow LlamaCasty first" — the map already said it; the per-destination audio/monitoring expectations are a different product. |
| Stream clipping | YouTube killed clips; trimming local MP4 is an editor. |
| Advanced broadcast-form tab (latency/DVR/embed/projection/CC/region) | TASK 9 item 7 rescoped: fixed sane defaults, invisible. Every exposed field is a support ticket. |
| In-app bug-report mechanism | Never built, never will be: support = email + GitHub issues. (ai.md's support line corrected to match, 2026-09-01. Second-place corpse if the list ever reopens: with replay buffer.) |
| D3DImage/GPU preview compositor | TASK 3 item 16 superseded: XAML preview + software output compositor are the design; a D3D11 swap remains a seam-respecting possibility, not a feature. |
| Simultaneous record + stream | Creator ruling 2026-09-01: the VOD is the copy; dual encode drags mediocre hardware and degrades both outputs. "We're not them." Two modes, radio pills, one stop. |
| Premiere / video-upload pipeline | Uploading finished videos is Studio's job (quota-brutal, processing-state polling, API premiere support unproven). Watch-parties are fandom behavior, not our persona. "Going live is our schedule." |
| Scheduled playout (pre-recorded file airs as live) | Discussed fully 2026-09-01 and declined by the creator — "let's not." If it ever returns, it returns by ruling, with machine-on caveats and the media-audio slice as its price of admission. |
*If a future session is tempted by anything on this list, the answer is already written. New ideas
must survive this page before they get a task number.*