Files

T

gramps 4b9a8e515f Control Surface UX: pill toggle, background rename, per-scene properties

- Sliding pill toggle (green ON / red OFF) replaces checkbox
- Geometry mismatch error for custom backgrounds
- Backdrop -> Background: C# rename, SQL column rename with migration
- Source.IsBackground: new -> override (fixes WPF binding)
- Legacy 'Backdrop' display name migrated to 'Background'
- Per-scene UseDefaultBackground + CustomBackgroundPath
- Asset files renamed backdrop -> background
- StagedScene/LiveScene split, thumbnail strip, edit gating
- TransitionService (Cut/Fade/Move)
- Context menu: Capture Desktop, Refresh Desktop
- Resolution badge removed, ON-AIR status light added

2026-08-22 18:10:59 -07:00

101 KiB

Raw Blame History

ytLlive — Task List

Task queue and authoritative research. Memory-map conventions: schema.md; architecture/decisions: ai.md. Update statuses here whenever a task moves.

Checklist markers — every task's Status list uses the same states:

✅ — completed (green check)

🔶 — in progress (amber diamond)

☐ — not completed / pending (empty box)

❌ — exception (blocked, known-issue, or deliberately excluded from this build)

YouTube Live API — research facts (authoritative, v3 build)

Lifecycle: created → ready → [testing] → live → complete (transitional liveStarting / testStarting).

liveBroadcasts.insert requires: snippet.title, snippet.scheduledStartTime, status.privacyStatus, status.selfDeclaredMadeForKids (COPPA).
liveStreams.insert requires: snippet.title, cdn.frameRate, cdn.ingestionType, cdn.resolution. None of the four (except title) can ever change after creation — changing them means delete + recreate the stream. This is the hard constraint behind the quality grey-out.
Title / description / privacy: editable at any time, including while live (liveBroadcasts.update, part=snippet,status).
contentDetails (DVR, recordFromStart, monitorStream, embed, latency): editable only in created / ready.
Transition to live only allowed when the bound stream's status.streamStatus == active.

Two features that reshape the design

enableAutoStart / enableAutoStop — instant one-click go-live, no transition call. With enableAutoStart=true we never call transition(live): the broadcast auto-goes-live the moment the encoder starts. Combined with enableMonitorStream=false (our preview pane replaces YouTube's monitor stream — the thing that forces a testing stage), the flow is create → bind → Start Stream → encoder starts → YouTube brings it live. No testing, no transition polling, no liveStarting stuck-state handling.
cdn.resolution=variable / cdn.frameRate=variable — free auto step-down. YouTube auto-detects what we send; since we ARE the encoder we can drop bitrate/resolution on the fly with zero API calls. Declaring an explicit resolution instead (e.g. 1080p) requires a new stream, which can't happen mid-broadcast. Variable is the enabler for the whole auto step-down feature.

Compliance gotchas (maps perfectly to report-by-exception)

liveStreams.status.healthStatus: good | ok | bad | noData plus configurationIssues[] with type + severity (info|warning|error). Literally built for report-by-exception — poll it, render nothing on good/ok, surface a banner only on warning/error. No need to invent our own health logic.
Encoder must comply or YouTube flags it: keyframes ≤ 4s (gopSizeLong), closed GOP, H.264, audio AAC/MP3 @ 44.1/48kHz, mono/stereo only.
Error codes to handle: errorStreamInactive, invalidTransition, redundantTransition, liveStreamDeletionNotAllowed, liveStreamModificationNotAllowed, liveBroadcastBindingNotAllowed.

Tips we should take advantage of

Reusable streams (isReusable=true): one stream per channel, cache its ingestion URL + stream name, reuse for every broadcast. No rebinding dance each go-live. This is exactly the manual-stream-key baseline.
Backup ingestion address: YouTube provides a simultaneous-push backup — future hardening, not v1.
recordFromStart + enableDvr default true → every live is auto-recorded and immediately replayable. Free VOD archive, matches the v0.2 recording goal.
latencyPreference: normal | low | ultraLow — for homelab streamers talking to chat, low (or ultraLow, capped at 1080p) is a real feature.
Broadcast ID == Video ID — one ID to track everything.

TASK 1 — Initial Scaffold

Goal: Working C# / WPF project with MVVM architecture, dark-theme main window, and YouTube service stubs.

Status: ✅ Done

✅ Models: Scene, Source, StreamConfig, StreamHealth, YouTubeChannel, ChatMessage
✅ Services: YouTubeAuthService (OAuth2), YouTubeStreamService (broadcast/health), YouTubeChatService (chat polling)
✅ MainViewModel: scene management, stream controls, chat
✅ MainWindow: scene/source panel, preview area, chat panel, status bar
✅ Clean build, 0 warnings (WSL + Windows)

TASK 2 — YouTube OAuth2 Authentication

Goal: Fully working Google OAuth2 flow — user clicks "YouTube", browser opens, authorization callback lands, channel info is stored.

Status: ✅ Done

✅ Two-state Start/End Stream button, go-live dialog (account + title/description/visibility), red top bar, pulsing LIVE badge + elapsed timer, preview glow, taskbar red dot
✅ Real OAuth2 wiring — baked-in Google credentials (desktop client; loopback callback) + YouTubeAuthService complete: browser launch, HttpListener callback, token exchange, refresh, channel fetch
✅ Token persistence via Windows DPAPI (Helpers/TokenStore.cs → %APPDATA%\ytLlive\ytLlive.auth), best-effort reload + proactive refresh at startup, saved after every exchange/refresh
✅ Account sign-in/change surfaced in the GoLive dialog (saved account shown with "Change Account"; "Sign in to YouTube" when none; Start disabled until signed in)
✅ End Livestream signs out — a graceful end completes the session: StopStream() calls YouTubeAuthService.ClearSession() + TokenStore.Clear() + IsConnected = false, so the next Start Stream dialog requires a fresh sign-in. A crash never runs End, so the DPAPI token survives and the creator stays signed in. Resume/reconnect after a midstream crash is deliberately deferred to TASK 3: the socket can't be resumed (it dies with the process), so "resume" = fast reconnect with a saved broadcast ID/stream key within YouTube's disconnect-grace window; too slow and enableAutoStop ends the broadcast
✅ Tests in ytLive.Tests (xUnit, net8.0-windows): TokenStore DPAPI roundtrip/corrupt/missing/clear + mocked exchange channel-parse + refresh expiry bump + ClearSession — 7 passing

Design constraint: Sign-in must NEVER block core exploration. Users can build scenes, add sources, and audition the software without authenticating. But going live requires authentication — the "Start Stream" dialog is where the account sign-in lives, alongside all stream metadata.

Two-state flow: There is no separate "Connect" button. The top bar shows a single button — Start Stream when idle, End Stream when live. Clicking Start Stream opens one dialog that supplies everything: account (previously-saved account shown as default, with a Change Account action) + title/description/visibility.

Live indicators (unmissable): Top bar + window title bar flip red, a pulsing ● LIVE badge with elapsed timer appears in the top bar, the preview area gets a red glow, and the taskbar icon shows a red overlay dot. Window title bar shows the stream title once validated.

Requirements:

Google Cloud OAuth credentials — client ID + secret, baked into Helpers/OAuthCredentials.cs (desktop "Desktop app" OAuth client; loopback callback — no console redirect URI registration needed; creators never configure)
Local HTTP listener — HttpListener on http://localhost:PORT/oauth2/callback to catch the redirect
Browser launch — open the authorization URL in the default browser
Token persistence — store access/refresh tokens securely (Windows DPAPI), reload on startup
UI state — account shown in the Start Stream dialog; "Change Account" action triggers re-auth
Go Live gated on auth — Start Stream dialog requires sign-in to enable the Start button; scene building works without it

Tests:

Mock token exchange response, verify channel info parsed
Verify token refresh triggers when near expiry
Verify credential load/save roundtrip

TASK 3 — Capture Pipeline (Scenes/Sources)

Goal: Real video preview in the center panel — the minimal source set below, composited per scene.

Status: 🔶 In progress

✅ Milestone 1 — webcam — MediaCapture (WinRT SDK projection) with device enumeration, CPU-first frame source, refcounted CameraManager, picker dialog, clip shapes (Traditional + Round) + mirror, 480×270 default placement — schema v2
✅ Schema v3 (Ship Branch A) — multi-scene webcam (singleton Webcam + per-scene WebcamSceneConfig), right-click border/context menu, static OBS-style borders, 50%-per-dimension webcam size cap, device-swap (ReleaseAllAsync) — 25 tests passing
✅ Schema v4 — round→rect restore persisted (WebcamSceneConfig.RectWidth/RectHeight) + one-time legacy-square 16:9 heal on load
✅ Screen backdrop (ship task #1, schema v5) — live desktop/game capture as a permanent, non-deletable bottom layer (Source.IsBackdrop), auto-detecting the full-screen game at launch/focus (else the primary display — never assumed monitor 0) via Win32FullScreenDetector (now with GetDisplays()/PrimaryMonitorIndex() for the in-app display picker), content re-designated via the OS GraphicsCapturePicker ("Change Capture…") or the in-app "Capture Display" submenu, refcounted/shared capture sessions in ScreenCaptureManager mirroring CameraManager — 45 tests passing
✅ Schema v6 — backdrop Live-only by policy — Scene.HasBackdrop, enforced by scene name on every load (EnforceBackdropPolicy: Starting/BRB/Chat/Ending never carry one; the one-time v5→v6 backfill covers all four), the scene context-menu "Backdrop" checkbox is gone (policy owns the flag), preview watermark hides when the backdrop renders, capture changed to WindowsRuntimeMarshal.TryGetDataUnsafe (the CsWinRT-safe frame-read) + downscale to the 1920×1080 master + 5s-throttled error logging (was flooding startup.log with 5 MB of cast errors and burning CPU), round webcam no longer re-rasterizes an ImageBrush every frame (Image + EllipseGeometry clip) — the live-mode stutter fix
✅ The five-scene catalog (SceneCatalog) — Starting/Live/BRB/Chat/Ending is the product — work with less, never more; the (+) button only shows when a canonical scene is missing and re-adds it (its menu lists only the missing ones) — 62 tests passing
✅ Webcam-after-session-start fix — a webcam added to a scene after the camera was already running (e.g. Chat) previously rendered a transparent container — CameraManager.GetPreviewBitmap + propagation in AddWebcamToActiveSceneAsync/ReacquireWebcam now hands the running shared frames to any newly added WebcamSceneConfig — 65 tests passing
✅ Chat scene webcam size cap — raised from 50%-per-dimension (960×540) to half the screen AREA (~1358×764 @16:9, MaxWebcamWidthFor/MaxWebcamHeightFor keyed by canonical name) so the viewer sees the creator better
✅ "Add Webcam" always opens the camera picker — deleting one scene's webcam then re-adding used to resurrect the old camera when another scene still used it — SwapWebcamIdentityAsync now swaps the app-wide identity if a different camera is chosen, same path as "Change Webcam…"
✅ Webcam resource validation + first-frame proof — MediaCaptureFrameSource validates post-init (VideoDeviceId match, stream properties ≥1, reader.StartAsync() status read + throws on non-Success); subscribes capture.Failed + CameraStreamStateChanged → SourceFailed event on the seam; fallback ladder (VideoPreview → VideoRecord). CameraManager.AcquireAsync requires first-frame proof (4s timeout): returns true only after a real frame arrives — silent empty box impossible. MainViewModel subscribes CameraFailed → red WebcamError chip in preview + MessageBox names suspect apps (CameraConflictProbe). 19041 SDK projection gaps: Exclusive/DeviceLost not projected; CameraStreamState.Failed compared by (int)2. 81 tests passing
✅ Scenes/sources UI — add/reorder/rename, image + background overlays with move/resize/opacity/reuse
✅ Audio UX shipped (UI) — the bottom-bar footer is now two lines (dropped/duration moved under bitrate/fps), with the mic's sound meter + mute button + volume slider grouped CENTERED on the footer's top line, beneath the preview panel (meter: 288px, muted slate track with ruler graduations + muted yellow/red zone tints, green→yellow→red fill; mute = speaker icon → red do-not-symbol when muted, and the slider and speaker stay in sync (volume 0 ⇔ muted — sliding off flips the speaker to muted, sliding up from 0 clears it); mic volume defaults to 80%, muting zeroes the meter and restores the prior volume on unmute (which flashes the meter to the restored position ~300ms before it returns to the live level); the meter is a READ-ONLY realtime level display (fill = live level × volume — volume is a gain on ambient noise; while the slider is dragged the bar previews the slider position and bounces back to the live level on release, which is 0 with no input — clicking the meter does nothing), clicking the MIC label opens a microphone picker whose chosen source shows left-justified inside the meter bar (fill at 75% opacity so the name + ruler markings show through); slim dimensional slider — gradient track/fill, gloss-sphere thumb; the old flat pink 18px-filled one is gone), everything else on line 2 (bitrate/fps/dropped/duration/health left, quality + gear right) — the creator's only audio control, desktop/game audio is automatic (KISS rule). Post-test follow-up (2026-08-13, game audio bar branch): the game audio bar (desktop/game, a mirror of the mic bar: meter + mute + volume) is overlaid at the bottom of the preview window (bottom-center dark chip — monitoring UI only, never on the live output) and appears only while a full-screen game is producing sound in the preview (IGameAudioDetector seam + default GameAudioDetector polling IFullScreenDetector + the live loopback level, floor 0.5%, into a pure GameAudioHysteresis: SHOW after ~500ms of fullscreen+sound, HIDE ~1s after leaving fullscreen, silence never hides an active bar; the VM polls it on a 250ms DispatcherTimer); capture now runs for the app's lifetime (mixer started once at startup via StartMicCaptureAsync, disposed in Shutdown — not go-live) so the meters preview live; the MIC label became a button with a status dot (Models/MicStatus: green = connected via the source Started event, yellow = mic problem, red = no device); picking a mic takes effect immediately (AudioMixer.RestartMic, loopback keeps running); fixed a latent ?.Invoke(meter.Push(...)) short-circuit that skipped the meter update when nothing was subscribed. Round 2 (2026-08-13): the meters were dead on a flat scale (real speech/game RMS is ~0.01..0.1 linear) — the raw level is now mapped via AudioLevelMeter.ToDisplay (−60..0 dBFS spread across 0..1) so typical input reads ~1/3..2/3 of the bar at default volume. 169 tests passing, 0 warnings. Round 3 (2026-08-13): the mic bar gained a mic mute icon (a microphone glyph in the speaker's 16px style, red + slash when muted) between the meter and the speaker — both mutes adjacent with spacing between them, same ToggleMicMuteCommand; and the top-center LIVE badge became an always-visible REC sign — dark gray dot + dim "REC" offline, bright red (#e94560) + "REC" + elapsed while live, darker red (#8f1f1f) when live with a private stream (driven by the dialog's chosen visibility) — RecDotBrush/RecTextBrush/RecDotOpacity/IsLivePrivate, pulsing while live. Queued: task 22 (voice filters). Voice-filter note: the meter's ToDisplay input is the pre-filter mic level; when filters land they must apply BEFORE the meter/encoder mix
✅ The connected YouTube account's avatar/name shows in the top bar next to Start Stream (SyncConnectedAccount); the scenes list is content-height now (no dead space before SOURCES)
✅ Social bar v2 (six-slot dialog, sign-in gate, real logos) — global bar layer (never a Source, no sources-list row), content-sized, centered, GREEN glow when ON, top/bottom snap-drag (default BOTTOM, persisted SocialBarPosition; drag clamps to 0/1040, tie→bottom). Footer Social button gets a state dot (green=ON). Dialog "Social Media Site Promotion" (SocialsDialog + ViewModels/SocialsDialogViewModel, WPF-free + injected ISocialValidator/sign-in/sign-out fakes): ON/OFF bar switch (SocialsConfig.BarEnabled, schema v8), 6 fixed slots — row 1 always YouTube (signed-in → channel handle; signed-out → sign-in gate → OAuth; delete → confirm sign-out, mirrors StopStream), row 2 free, rows 3-6 lock icons on freemium (Premium seam: all six). Validation: DetectService (URL domain / fediverse @user@domain / bare→Website) → async ISocialValidator on confirm/Save; valid snaps to text + real service logo (bundled SVG path data via LogoDataFor, Simple Icons CC0 — initials badges gone); invalid → red do-not, stays editable, Save blocked. LCR justify dropped (BarJustify unread), per-scene toggle dropped (Scene.HasSocialBar back-compat). Post-test fixes (2026-08-12): footer label "Socials" (not "Social"); fediverse @user@domain validates — the full handle is the identity end-to-end (DetectService/CanonicalUrlFor/HttpSocialValidator build https://domain/@user, no domain loss); Cancel is a hard stop — ISocialValidator.LookupAsync takes a CancellationToken, dialog VM owns a CTS, Cancel/X/Save abort in-flight lookups (HTTP request killed, canceled continuations never touch slot state), and ConfirmEdit skips re-submitting identical text (LostFocus on dismiss never re-fires a lookup). HttpSocialValidator now has real tests (fake HttpMessageHandler). 105 tests passing. Post-test fixes (2026-08-12, round 2): fediverse @user@domain no longer shows a generic chain — it resolves to the instance's actual software via nodeinfo (/.well-known/nodeinfo → software.name; SocialService.Fediverse enum member + SocialEntry.FediverseSoftware persisted in a new SocialEntry.Software column, schema migration by column-presence) and renders that software's bundled logo (LogoDataForFediverse: mastodon/peertube/pixelfed/misskey/lemmy/pleroma/firefish, generic fediverse honeycomb fallback). Dialog row-2 edit/trash icons were too dark — IconButton style gains Foreground=#d0d0d0; trash overrides #e94560 (app red). 112 tests passing. Post-test fixes (2026-08-12, round 3): a fediverse handle whose identity domain is itself a redirect (e.g. YunoHost default-app subdomains — @user@llamachile.tube where the mastodon instance lives at mastodon.llamachile.tube) now still resolves its software: nodeinfo on the identity domain is SSO-blocked, so HttpSocialValidator follows the bare root https://domain/ 302 to the real instance host and re-runs the nodeinfo lookup there.
☐ Window capture — absorbed into the Screen picker (no separate source type); dedicated window-as-source work is pending
☐ Scene compositing — the D3DImage/MediaElement preview compositor (this task's requirement 5; the output compositor ships as TASK 4 ship step 1)
☐ Text source — live text ("Starting soon", "Back in 5", handle, callout)
✅ Chat box — YouTube live chat rendered on the stream so viewers read along in-video — ChatBoxRenderer (WPF FormattedText → VideoFrame), configurable font size/color/badges/timestamps/max-messages via Elements panel, schema v10, persisted across save/load
❌ Background removal (milestone 2) — ONNX Runtime + DirectML, MediaPipe Selfie Segmentation — deliberately NOT in this build
☐ Alerts — Super Chat / membership / subscribe pop-ins; build after the six; the one paid feature (see Monetization in ai.md)
✅ Show Desktop toggle — Source.ShowDesktop bool persisted in DB (schema migration + LayoutStore read/write). When checked, primary monitor captures regardless of fullscreen game state. Toggling off falls back to running game or releases to static placeholder. ClearBackdropCaptureAsync clears CaptureKey + VideoImageSource so static fallback renders. Toggle on Live backdrop context menu (IsCheckable MenuItem).
✅ Top bar — status light + avatar + Log In button — red/green/pulsing ellipse (disconnected/connected/live). Avatar border visible only when connected, loads via BitmapImage code-behind. "Log In" button visible when disconnected, calls StartStreamCommand → GoLiveWindow. ShowStartStream now requires IsConnected.
✅ Chat preview — one-at-a-time mock messages — RunMockChatPreviewAsync: simple async loop, displays MockChatMessages[n] via Take(n+1), 1000ms between messages, wraps at end. ChatPreviewEnabled property on Source (default true). Stops on real messages, restarts on fade timer clear.
✅ All backdrops seeded + EnsureDefaultBackdrop — Live, Chat, Ending backdrops added (ending-backdrop.jpg replaced). EnsureDefaultBackdrop() called on every scene switch. SeedEndingBackdrop replaces existing (not skip-if-exists). Chat scene no longer skipped.
✅ SceneCatalog fix — Chat scene display name corrected to "Chat" (was "YouTube Chat" — that name belongs to the layer/SourceType.ChatBox, not the scene).
✅ Left panel spacing + context menu reorder — HR margins matched, Row 4 changed from * to Auto. Context menu reordered: Border Thickness first, then Opacity, Color, Effect.
✅ Logo + richer in-app About (2026-08-13, queued → 2026-08-14 SHIPPED) — the ytLlive wordmark in the top bar opens the About overlay (already wired); the About overlay is now the creator hub: the real logo (the "llama fortnite superman logo" from the creator's vault, copied to Assets/llama-logo.png — the creator's own art, no third-party license), plus creator-hub links — llama chile shop on YouTube (MainViewModel.ChannelUrl), Mastodon (https://mastodon.llamachile.tube/@gramps), Buy me a coffee (https://buymeacoffee.com/llamachiley — live), and Unlock Premium (greyed "coming soon" — the billing product URL is a tabled seam, PremiumUrl, until TASK 10 picks the subscription provider). A Licenses & legal button flips the About overlay to an in-app scrolling panel that loads the full THIRD-PARTY-NOTICES.txt text (MainViewModel.ShowLicenses reads the shipped file from the executable directory on first open; graceful "not found" fallback — never the OS viewer, everything stays in-app); "← Back to About" returns. The About overlay is also the in-app home of the notices — the top-bar About button that opened the file in the OS viewer was removed on 2026-08-13 for exactly this reason. Integration test (Good Dog Rule — ONE): AboutHubTests.About_Opens_InApp_Licensing_Loads_Shipped_Notices drives the real window + VM, asserting the hub opens, the link URLs are real, the licensing panel loads the shipped notices text (contains "Third-Party Notices" + "LGPL"), and back returns to the hub. 174 tests passing, 0 warnings
✅ Voice filters on the mic channel (2026-08-13 queued → SHIPPED 2026-08-14 inside TASK 9, the audio milestone) — the standard four applied to the sound input path (before the meter/encoder mix): bass boost, treble, noise suppression, compressor (set decided with the creator 2026-08-13). Noise suppression = a pure-C# noise gate (creator chose over RNNoise / a second ffmpeg afftdn pipe, 2026-08-14 — KISS). Always-on — no UI knobs; the mic stays the creator's single audio control

The Minimal Source Set (design decision — do not expand casually)

ytLlive is YouTube-only and 90% of users are casual. OBS's long source list is off-putting; we ship the hot few and nothing esoteric. If a user needs more, they've graduated to OBS.

Webcam — the face cam. Non-negotiable.
Screen — the main event (game, slides, browser). One source; a picker chooses a monitor or a window. (Window capture is absorbed here — no separate source type.)
Background — a full-canvas backdrop image. Fills the whole scene automatically, zero fiddling. Kept separate from Image on purpose: same pixels, but this one needs no positioning.
Image — a floating graphic/logo overlay (watermark, badge, corner branding). Free-positioned.
Text — live text ("Starting soon", "Back in 5", handle, callout). Casual streamers live on this.
Chat box — YouTube live chat rendered on the stream so viewers read along in-video. YT-native.
Alerts — Super Chat / membership / subscribe pop-ins. The dopamine source. The one big lift (Super Chat event streaming + on-stream rendering/animation); build after the six. Also the one paid feature — see Monetization in ai.md.

Deliberately NOT supported: game capture, browser source, media playlist, VLC, color-key voodoo, MIDI.

Source memory model (design decision)

A scene has resources. Resources can be shared across scenes.
A resource exists exactly once in memory, no matter how many scenes use it (a logo in five scenes = one loaded bitmap).
Every resource carries a catalog of scenes: one usage entry per scene it appears in, each entry dictating that scene's use — placement (X/Y/Width/Height), opacity, z-order, enabled, scale mode, crop.
Usages are named {resourceName}.{sceneName} — whatever the user named the resource, dot, the scene name: logo.starting, logo.live, myPic.brb. Not a hardcoded "logo".
A webcam in two scenes = one capture session, two catalog entries.
Refcount by catalog size: the last usage removed → the resource is disposed and evicted.
The resource (not a per-scene node) owns everything IDisposable.

Scene transitions (design decision)

Scene switching while live must never stutter. Supported types, most → least economical:

Cut — instant switch. The default. Zero cost.
Fade — short crossfade (~300ms).
Move — a simple, economical move transition, done to perfection and memory-efficient. The smart streamer's bread and butter.
Custom (media) transitions — require media elements (video/stinger playback during the transition). Heavier, but creators pay for these, so we support them. Their media follows the same resource memory model: loaded once, catalogued by scene.

Preview shows the transition too (WYSIWYG). No wipes/slides/LUTs beyond the four above.

Requirements:

Screen — Windows.Graphics.Capture (WinRT), enumerate displays/windows, picker
Webcam — MediaCapture (WinRT SDK projection) with device enumeration — ✅ milestone 1 done:
1. TFM bumped to net8.0-windows10.0.19041.0 (app and tests) so the WinRT projection resolves from the SDK reference packs — no NuGet package, no capability manifest (unpackaged desktop app)
2. MediaCaptureFrameSource (CPU-first: MemoryPreference = Cpu, BGRA8 via CreateFrameReaderAsync), MediaCaptureCameraEnumerator (DeviceInformation.FindAllAsync(DeviceClass.VideoCapture))
3. CameraManager: refcounted by DeviceId, one shared WriteableBitmap app-wide, dispatcher-coalesced UI updates (~render rate, latest-frame drop), placeholder/AppLog + warning on failure
4. CameraPickerDialog (mirror of ReuseImageDialog) — "Searching for cameras…" / list / "No cameras found" states
5. One webcam app-wide: Add → Webcam greyed out once one exists ("it's already in your stream" tooltip); persisted DeviceId re-acquires after layout load
6. Default placement 16:9 480×270, bottom-right, 32px margin; drag/resize/selection shared with Image sources
7. Clip shapes: Traditional + Round (phone view dropped — the 9:16 phone output is the vertical output-crop tier); mirror; both persisted in the layout DB (schema v2) and toggled from the source chip
8. Background removal = milestone 2 (ONNX Runtime + DirectML, MediaPipe Selfie Segmentation) — not in this build
Background / Image / Text — static sources positioned/scaled/opacity
Chat box — rendered from the live chat poll (right panel is the same feed, raw)
Scene compositing — per-scene source layering (z-order = sources list order, top-to-bottom back-to-front), preview rendered via D3DImage or MediaElement
Branding flash — the topmost full-frame "made with ytLlive!" layer at ~25% opacity, ~1s on / 300s off (see Monetization in ai.md), gated on BrandFlashEnabled + live/recording. Lives in the preview compositor now (BrandFlashLayer in MainWindow.xaml CanvasGrid, driven by BrandFlashActive/BrandFlashTimer in MainViewModel); the encoder output renders the same layer, and v0.2 local recordings carry it too
Drag/drop placement & reorder — intuitive, visual (per design principle):
1. Preview: click-drag a source in the center panel to reposition it; resize via handles
2. Scenes list: drag rows to reorder scenes
3. Sources list: drag rows to reorder sources (this is the z-order) — implemented

TASK 4 — RTMP Ingest to YouTube

Goal: Push encoded video to YouTube's RTMP ingest.

Status: 🔶 In progress

✅ Ship step 1 — the output compositor SHIPPED (2026-08-10)
✅ Ship step 2 — the FFmpeg locator SHIPPED (2026-08-10)
✅ Encoder + RTMP push SHIPPED (2026-08-12) — the FFmpeg subprocess: raw BGRA frames via stdin, stderr health parsing, FLV mux + push to the ingestion URL (see the ship step 3 plan below)
✅ WASAPI audio capture SHIPPED (2026-08-12) — NAudio loopback (desktop/game) + the picked mic feeding AudioLevel, so the realtime meter comes alive (see the ship step 4 plan below)
✅ Frame-pipeline wiring SHIPPED (2026-08-12) — CameraManager/ScreenCaptureManager → compositor resolver → encoder, driven by a paced FramePump (see the ship step 5 plan below)
✅ Health stats SHIPPED (2026-08-13) — FramePump.HealthUpdated (encoder's parsed bitrate/FPS/dropped/duration, already forwarded from FfmpegEncoder.OnStderrLine) now lands in the bottom bar: MainViewModel.OnFramePumpHealthUpdated marshals to the UI thread (the stderr loop raises on a background thread) and copies into CurrentHealth (the bottom bar's existing binding); ResetHealth zeroes dropped/duration on go-live and on End so stats never linger from a previous session (bitrate/FPS stay on the tier's targets). The bar lights up with real values once TASK 9 supplies the RTMP URL (until then the pump skips the encoder and the bar shows the tier's targets)
✅ One-click go live + private-only enforcement SHIPPED (2026-08-14) — the Go Live dialog is locked to Private (no dropdown, GoLiveViewModel.Visibility is a get-only "Private"); YouTubeStreamService.CreateBroadcast always sends privacyStatus = "private" (dialog + service enforcement, requirement 8 — nothing can go out non-private) and gained an injectable HttpClient? http = null seam for tests; BeginGoLive now calls CreateBroadcastAsync and remembers _currentBroadcastId for TASK 9's bind/transition (failure → StreamStatus.Error, never a crash; StopStream clears the ID); the REC sign shows a PRIVATE badge (dark-red border, next to REC, IsLivePrivate) when the live stream is private; settings' dead "Default Visibility" dropdown + MainViewModel.Visibilities/DefaultStreamVisibility removed. The broadcast-insert integration test asserts the request body carries "privacyStatus":"private" (2 new tests → 173 passing, 0 warnings)

The pipeline chain the encoder needs doesn't exist yet: scene compositing (the master 1920×1080 frame without the preview's editing chrome) → audio capture (WASAPI, feeds the meter) → H.264+AAC encode → vertical-tier crop/scale → RTMP push → health stats into the bottom bar. Nothing can encode until a frame source exists, so the compositor is ship step 1. The pipeline is CameraManager + ScreenCaptureManager → compositor resolver → compositor → encoder → RTMP.

Requirements:

Encoding — H.264 (hardware via NVENC/AMD, fallback x264) + AAC audio; must comply: keyframes ≤ 4s (gopSizeLong), closed GOP, AAC/MP3 @ 44.1/48kHz, mono/stereo only. License posture (decided): GPL-free build — NVENC (NVIDIA) / QSV (Intel) / AMF (AMD) + OpenH264 software fallback + built-in AAC; no libx264 (GPL contaminates a paid product). Output containers are identical either way (H.264+AAC in .flv for RTMP, .mp4/.ts for VOD) — the format is NOT the differentiator, the license and per-GPU quality are. License guardrails (never violate — see ai.md → "Licensing — do not violate"): only BtbN lgpl/lgpl-shared builds; never GPL (gyan.dev) or nonfree (fdk-aac); never static for distribution (LGPL §6 relink material); never link FFmpeg into the app; never drop THIRD-PARTY-NOTICES.txt from the app/About screen.
RTMP push — FFmpeg subprocess (decided): app feeds raw frames via stdin, parses stderr for health; one battle-tested binary does encode + FLV mux + push + reconnect. Binary distribution (decided): check-then-pull — probe where ffmpeg/PATH at first go-live; if absent, download a pinned build (BtbN LGPL-shared win64 zip, ~75 MB — gyan.dev's builds are GPLv3 and ship libx264, which violates the license posture; BtbN's LGPL variant drops x264/x265 while keeping NVENC/QSV/AMF + libopenh264 + native AAC) to %APPDATA%\ytLlive\tools\ffmpeg.exe (extract ffmpeg.exe plus the libav*.dll family) and cache it, offline-friendly. Behind an IFfmpegLocator seam so tests fake it (ship step 2, below). Push goes to the cached reusable stream's ingestion URL
Quality ladder — the offered tiers, with 1080p60 @ 8 Mbps as the standard/default:
1. 720p30 @ 6 Mbps
2. 720p60 @ 6 Mbps
3. 1080p30 @ 8 Mbps
4. 1080p60 @ 8 Mbps (default — mainstream ceiling, GPU hardware-encoded so the gaming machine never notices; upload headroom stays comfortable)
5. Vertical 1080×1920 @ 60fps @ 8 Mbps (9:16 phone tier) The composition master is always 1920×1080; a tier is an output rect + target resolution (see ai.md "Resolution tiers"). Vertical output = the centered 607×1080 crop of the master scaled to 1080×1920 (semi-crop preview is already implemented; the encoder applies the same rect). 1080p60 is the ceiling by design — "if you want 1440 or 4K or 8K → OBS is your solution"; the app targets the most mainstream creator, not power users. Ladder is sculpted by a cached probe (IP-only TCP vs public ingest host; no auth required). Quality is greyed out while live because the declared resolution can't change mid-stream — but with variable, we can auto step-down bitrate/resolution on the fly with zero API calls (no stream recreation); 60fps presumes a hardware encoder — no hardware encoder → auto fallback to 720p60/1080p30
Stream key management — reuse the cached reusable stream (one per channel) instead of creating a new one per go-live; prefill default YouTube ingest URL rtmp://a.rtmp.youtube.com/live2
Health stats — bitrate, FPS, dropped frames reported live in the bottom bar (encoder-side)
One-click go live — defaults that work out of the box
Audio capture (feeds the meter — this task ships the wiring) — WASAPI loopback (desktop/game at unity, zero UI — "it just is") + the picked mic (MicSourceName from the MicPickerDialog). The mic capture feeds AudioLevel so the realtime meter comes alive (today it reads 0 — the mixer feed is pending, see ai.md audio notes). AAC mono/stereo @ 48 kHz per the compliance rules.
Private-only go live until v1 (reputation guard, decided 2026-08-10) — until the v1 release, go-live is locked to private streams only so a software error can never publish something public/unlisted that damages the creator's reputation. RTMP push itself has no privacy — privacy lives on the YouTube live broadcast object, which this app already controls via its OAuth API calls. So the lock is purely API-side: the Go Live flow always creates/updates the broadcast with privacyStatus = "private" and a guard refuses to set anything else (same spirit as the Live-only backdrop policy). The UI shows a clear "PRIVATE" badge next to the stream state so the creator always knows who can see them. Enforcement must be verifiable in the auth-service tests (fake the broadcast-insert/update call, assert privacyStatus is forced to private).
v1 release gate: bundle the full license texts (decided 2026-08-10) — THIRD-PARTY-NOTICES.txt currently links the canonical license texts rather than embedding them. At the v1 (GA) release, the full texts of every license it names (LGPL v2.1+, BSD-2-Clause, MIT, Apache-2.0) MUST be bundled alongside it (shipped in the app output, e.g. a licenses/ folder next to the notices file, still reachable from the About screen). This is a release blocker for v1, not a task to queue early — do it in the release pass. The repo should treat this like the private-only go-live gate: a checkbox that cannot silently lapse.

Ship step 1 — Scene compositor (the frame source)

Goal: a pure-CPU software compositor producing the encoder's master frame (BGRA8, the VideoFrame seam) from the scene model. The preview stays XAML (the editing view); the compositor is the output view — WPF's RenderTargetBitmap can't be used (software-rendered + captures chrome). Two renderers must agree, so the XAML (MainWindow.xaml CanvasGrid + element DataTemplate) is the contract.

Decisions (locked 2026-08-10): Path A CPU blitter — GPU effort belongs to NVENC (the encoder), not composition; with an FFmpeg subprocess the master crosses a CPU readback to the pipe every frame anyway, so GPU compositing buys ~nothing at this layer count (2-3 live layers; static layers pre-composite once). A D3D11 compositor can replace this one later behind the same seam (the CPU master buffer stays the contract). Render the output rect directly: compositor is constructed with CompositorOptions {SourceRectX/Y/W/H, OutputWidth, OutputHeight}; 16:9 tiers = full 1920×1080 1:1; vertical (9:16) = composite the centered 607×1080 crop then bilinear-upscale to 1080×1920. Reuses MainViewModel.OutputRectX/Y/W/H (note (1920−607)/2 = 656.5 → align to integer pixels for output).

Render spec (back → front, mirror the XAML exactly):

Backdrop — the Live scene's IsBackdrop Source (CaptureKey → live frame), UniformToFill full-frame (XAML's separate BackdropImage layer; the backdrop element renders nothing — its DataTemplate Image is Collapsed for DisplayCapture).
Background — the scene's Background Source, UniformToFill full-frame (the ActiveBackgroundImage layer, not per-element).
Elements in Scene.Elements order (back→front), skip IsVisible=false. What actually renders:
1. Source Type Image → static asset, UniformToFill cover-crop into (X, Y, W, H)
2. WebcamSceneConfig → latest frame by DeviceId: Traditional = UniformToFill rect; Round = circle diameter min(W,H) (alpha 0 outside — true circle, not oval); mirror = horizontal flip around element center (MirrorScale); opacity = per-pixel multiply (content + border); border = stroked rect / centered circle at RoundBorderSize, width BorderWidth, alpha BorderOpacity
3. Background / IsBackdrop / TextOverlay are NOT per-element (layers above; Text not shipped)
Branding flash — pre-rendered full-frame "made with ytLlive!" at 25% alpha when live + BrandFlashEnabled + timer active. Passed in as a VideoFrame? (compositor core stays pure byte-math, no WPF; likely a bundled asset rather than runtime text rendering).
NOT in output (preview chrome only): SelectionOverlay, DimRects, output-rect outline, badge, placeholder.

New files (all in Services/Compositor/):

SceneCompositor.cs — Render(Scene, frameFor: Func<SceneElement, VideoFrame?>, flashFrame: VideoFrame?, CompositorOptions) → VideoFrame (output-sized). The caller's frameFor resolver maps each element to its frame (webcam → DeviceId, image → AssetId via StaticPixelCache, backdrop → CaptureKey) — the compositor stays pure/hermetic/no WPF.
CompositorOptions.cs — source-rect + output W×H.
StretchMath.cs — UniformToFill cover-crop, ellipse mask, bilinear scale (pure, unit-tested).
StaticPixelCache.cs — asset byte[] → cached BGRA VideoFrame (WPF BitmapDecoder + CopyPixels, decode once per content hash).

Test plan (Good Dog Rule — ONE integration test): SceneCompositorTests — a scene with backdrop (solid red fake frame) + round webcam (solid green) + image (solid blue) → render 16:9 master → assert per-layer probe pixels (corner = backdrop color, element center = webcam color, outside the round clip = backdrop color, mirrored element swaps left/right); a vertical-tier variant asserts 1080×1920 output + crop fidelity. Focused unit tests on StretchMath. Tests push frames directly — no capture managers involved (they wire in a later step).

Same-PR housekeeping: fix the stale comment MainViewModel.cs:324 ("shown under the meter on line 2" → "shown left-justified INSIDE the meter bar" — ai.md is the authority); this task's requirements now include the explicit audio-capture/meter wiring (#7 above).

Out of scope (later ship steps): FFmpeg locator + license posture (covered in requirements 1-2), encoder + RTMP push, WASAPI audio capture (loopback + mic) feeding AudioLevel, wiring CameraManager/ScreenCaptureManager into the frame pipeline, brand-flash timer wiring, health stats (bitrate/FPS/dropped).

Built (2026-08-10): all four files shipped in Services/Compositor/, SceneElement.TryGetBorderColor made public (shared hex parse with the compositor — no duplicated color parsing), the stale MainViewModel.cs:324 comment corrected, and the pre-existing CS1998 in YouTubeAuthServiceTests cleaned up — build 0 warnings. Tests: the SceneCompositorTests integration test (full-scene master pixels, vertical tier, flash) + 4 StretchMath units — 72 passing.

Ship step 2 — FFmpeg locator (the encoder's binary)

Goal: resolve a usable ffmpeg.exe on demand (the encoder's one external dependency), never shipping a binary in the repo. Returns an absolute path; downloads only when neither PATH nor the local cache provides one.

Decisions (locked 2026-08-10):

BtbN LGPL-shared win64 build — not gyan.dev (gyan's "essentials" is GPLv3 and ships libx264, which violates requirement 1's license posture) and not the static lgpl build: LGPLv2.1 §6 wants relinkable object files for static linking, but the shared (dynamic-DLL) variant sidesteps that — compliance is "license text + source offer + unmodified binaries" (see THIRD-PARTY-NOTICES.txt and ai.md → Licensing). Drops libx264/libx265 while keeping NVENC/QSV/AMF, libopenh264 (the LGPL-legal H.264 software fallback) and native AAC — exactly the requirement-1 encoder profile.
Pinned URL — https://github.com/BtbN/FFmpeg-Builds/releases/download/autobuild-2026-08-09-13-03/ffmpeg-master-latest-win64-lgpl-shared.zip (~75 MB zip — earlier "~30 MB" estimate corrected). A dated autobuild tag is immutable; BtbN retention keeps the last 14 daily builds + each month-end build for 2 years, so a cold cache after retention expiry 404s — a logged, recoverable failure (the seam throws; the encoder step surfaces it). Once cached, the URL is never touched again. The pin is a single const, bumpable in one place — and must always stay on the shared variant (never gpl, nonfree, or static; see ai.md Licensing).
Check-then-pull order — (1) PATH probe (the user's own install wins), (2) cached %APPDATA%\ytLlive\tools\ffmpeg.exe, (3) download + extract. Extract ffmpeg.exe plus the libav*.dll family (the shared build's bin/ folder; Windows resolves the DLLs from the exe's own directory) into a staging dir then move into place — a crash never leaves a corrupt or partial cache.
Seam — IFfmpegLocator.LocateAsync(CancellationToken): search dirs, tools dir, and the downloader (Func<string, CancellationToken, Task<byte[]>>) are constructor-injected with production defaults, so tests fake the network (feeding a real in-memory zip) and never touch disk outside a temp dir.

New files (all in Services/Encoder/):

IFfmpegLocator.cs — the seam.
FfmpegLocator.cs — the impl (PATH probe → cache → pull+extract exe + DLLs), failures logged via AppLog.
THIRD-PARTY-NOTICES.txt (repo root) — the LGPL/BSD/MIT notices + source offer, copied to the build output. (Surfacing changed on 2026-08-13: the top-bar About button that opened the file in the OS viewer is GONE — the notices are reachable in-app via the logo → About overlay instead.)

Test plan: the hermetic integration test drives the full decision ladder against a temp tools dir and a fake downloader returning a real in-memory zip (.../bin/ffmpeg.exe entry): PATH hit wins without downloading, cache hit skips the network, cold cache downloads → extracts → ffmpeg.exe lands in the tools dir, and a second call serves the cache (downloader invoked exactly once). Focused unit tests: shared-build DLLs extract alongside the exe, empty zip throws, missing entry throws, empty download throws, downloader failure propagates, zero-byte cache is refreshed.

Same-PR housekeeping: requirement 2's stale binary facts corrected in this plan (~30 MB → ~75 MB zip; "gyan.dev/BtB N" → BtbN LGPL-shared only, with the why); the "never do" licensing guardrails recorded in ai.md so the reasoning survives.

Out of scope (later ship steps): the FFmpeg subprocess encoder (frames in via stdin, stderr health parsing), RTMP push, WASAPI audio capture, the frame-pipeline wiring, health stats.

Built (2026-08-10): IFfmpegLocator + FfmpegLocator shipped in Services/Encoder/, pinned to the lgpl-shared build autobuild-2026-08-09-13-03 (extracts ffmpeg.exe + the libav*.dll family via a staging dir). THIRD-PARTY-NOTICES.txt (repo root) ships to the build output; the "never do" licensing guardrails are recorded in ai.md — build 0 warnings. Tests: the hermetic FfmpegLocatorTests integration test (PATH → cache → download decision ladder with a fake downloader serving a real in-memory zip) + edge/unit cases (shared-build DLL extraction, zero-byte cache refresh, empty payload, missing zip entry, downloader failure) — 78 passing.

Ship step 3 — Encoder + RTMP push (the FFmpeg subprocess)

Goal: encode raw BGRA master frames into H.264+AAC FLV and push them to the reusable stream's RTMP ingestion URL — one battle-tested subprocess doing encode + mux + push + reconnect, the app feeding frames via stdin and parsing stderr for health (req 2).

Decisions (locked): the encoder is a thin orchestrator over ffmpeg.exe — no H.264/AAC code in the app. Arguments (pure FfmpegArgs.Build): -re -f rawvideo -pix_fmt bgra -video_size WxH -framerate FPS -i pipe:0 (frames in), a silent placeholder audio track via -f lavfi -i anullsrc (the WASAPI capture step replaces this input), -c:v <encoder> -b:v K -maxrate K -bufsize 2K + -g fps×4 -keyint_min fps×4 -sc_threshold 0 -bf 0 -pix_fmt yuv420p (the keyframe ≤4s / closed-GOP / H.264 compliance), -c:a aac -ar 48000 -ac 2, -f flv <rtmpUrl>. Encoder choice is probed from the binary's -encoders listing (FfmpegEncoderPicker, pure): hardware NVENC → QSV → AMF, then OpenH264 software fallback — never libx264 (GPL; see ai.md → Licensing). The seam (IFfmpegEncoder + IEncoderProcess, constructor-injected locator + process factory) keeps it hermetic — tests fake the whole subprocess (probe + encoder), no real binary.

Behavior: StartAsync (locate → probe → spawn → stderr loop), SubmitFrameAsync (serialized BGRA stdin writes, ~2 Hz health via HealthUpdated/StreamHealth — bitrate/FPS/duration/dropped-from-frame- count), StopAsync (stdin EOF → ffmpeg finalizes + exits by itself; 10s watchdog kill), ProcessFailed on a non-zero unexpected exit.

Built (2026-08-12): EncoderOptions + IFfmpegEncoder/FfmpegEncoder + IEncoderProcess/ FfmpegEncoderProcess + pure FfmpegArgs/FfmpegProgressParser/FfmpegEncoderPicker in Services/Encoder/. Not yet constructed by the app (the frame-pipeline wiring, ship step 5, owns it). Tests: FfmpegEncoderTests integration (probe → spawn with NVENC preferred → frames into stdin → progress parsed → graceful stop, no kill) + units (args compliance/GOP, progress parser, picker preference + GPL guard, no-URL/not-running/noop stops, process-death ProcessFailed) — 122 passing.

Ship step 4 — WASAPI audio capture (the meter comes alive)

Goal: capture desktop/game audio (loopback) and the picked mic, feed the mic level into AudioLevel so the realtime meter reads something other than 0, run capture only while live (req 7).

Decisions (locked):

NAudio NAudio.Wasapi 2.2.1 — the wasapi feature package (not the NAudio meta-package): it carries the capture types (WasapiCapture/WasapiLoopbackCapture + the MMDevice enumeration) with NAudio.Core/NAudio.Asio pulled in transitively. MIT — recorded in THIRD-PARTY-NOTICES.txt (item 9).
IAudioSource seam (Start/Stop/SampleReady/Failed, IDisposable) — the app consumes the seam; the two WASAPI implementations wrap NAudio; tests inject hermetic fakes (no real audio devices, no timers). Loopback = WasapiLoopbackCapture on the default render device; mic = WasapiCapture with the NAudio device resolved by FriendlyName matching MicSourceName (the app only persists the DisplayName), falling back to the default capture endpoint. Mic device resolution is re-read at each Start via a name provider so a mic picked mid-session takes effect next go-live.
AudioMixer owns both sources — starts/stops both with go-live (BeginGoLive success → Start, StopStream → Stop). Mic samples feed a pure AudioLevelMeter (RMS, exponential smoothing) and raise MicLevelChanged, marshalled to the UI thread into AudioLevel; desktop samples are currently dropped (consumed by the encoder's AAC mix in a later step). Capture failures are logged via AppLog (mic failure also zeroes the meter); loopback failure doesn't kill the mic.
Byte→float — pure WaveToFloat.Convert handles the WASAPI mix formats: IEEE float 32-bit (direct) and PCM 16-bit (normalized to -1..1), including WaveFormatExtensible with the IEEE-float subformat GUID. Trailing partial samples are ignored.

Built (2026-08-12): Services/Audio/ ships IAudioSource + AudioSample, WasapiLoopbackAudioSource, WasapiMicAudioSource, AudioMixer, AudioLevelMeter, WaveToFloat; MainViewModel constructs the mixer (mic source fed () => MicSourceName), starts it on go-live and stops it on end-stream, and maps MicLevelChanged → AudioLevel. A pre-existing CS8602 in FfmpegEncoder.cs:139 surfaced during this step's rebuild and was fixed (process!) — build 0 warnings. Tests: AudioMixerTests (mixer lifecycle/forwarding/failure against fakes, meter RMS/smoothing/reset, WaveToFloat float/PCM16/ extensible/truncation) — 139 passing.

Deferred (later ship steps): wiring the desktop-capture samples into the encoder's AAC mix (replaces the -f lavfi -i anullsrc placeholder; the encoder construction itself shipped in ship step 5). (The "capture while not live" + "audio UI beyond the mic controls" deferrals were SHIPPED on the 2026-08-13 game audio bar branch — capture is now always-on for preview and the game bar is the second audio UI. The mixer's short-circuit meter fix + Started/RestartMic seams live in the same branch.)

Ship step 5 — Frame-pipeline wiring (the encoder gets a frame source)

Goal: the chain CameraManager/ScreenCaptureManager → compositor resolver → encoder, driven while live by a paced frame pump: snapshot the active scene → resolve each element to its latest frame → composite into the tier's output frame → pace into the encoder's stdin at the tier's FPS.

Decisions (locked via user Q&A, 2026-08-12):

Video pipeline first — the -f lavfi -i anullsrc silent track stays; mixing the loopback/mic WASAPI samples into the encoder's AAC track is its own later step.
RTMP URL via a provider seam — MainViewModel._rtmpUrlProvider is a Func<string?> returning null today (the reusable stream's ingest URL lands with TASK 9); when it yields null the pump logs and skips the encoder entirely, so go-live runs the existing visual flow without pushing.

Design:

Services/Encoder/FramePump.cs — the frame producer. All collaborators constructor-injected seams (Func<Scene?>, Func<SceneElement, VideoFrame?> resolver, Func<CompositorOptions>, Func<EncoderOptions?>, Func<IFfmpegEncoder>, Action<string> log, injectable pacing delay) so it stays free of WPF and of the capture managers and is hermetic in tests. StartAsync never throws (failures log + surface via Failed — the VM fires-and-forgets from the sync command handler); loop = snapshot → render → SubmitFrameAsync, paced at 1/options.Fps (default Task.Delay; tests inject Task.Yield). StopAsync stops the encoder (closes stdin) BEFORE awaiting the loop — closing stdin unblocks a write stuck on pipe backpressure, so stop can't deadlock on the pump. ProcessFailed self-stops the pump. HealthUpdated forwards the encoder's stats (ship step 6 binds the bottom bar).
ScreenCaptureManager.GetLatestFrame(key) — mirrors CameraManager.GetLatestFrame(deviceId); the backdrop's live frame for the compositor.
MainViewModel — owns the resolver (WebcamSceneConfig → GetLatestFrame(WebcamId); Source.IsLiveCapture → GetLatestFrame(CaptureKey); image/background → StaticPixelCache.Get(AssetId)), builds CompositorOptions from the tier + OutputRect* (doubles rounded to ints — the vertical 607.5 half-pixel crop rounds to a perfectly-centered 608), builds EncoderOptions from the tier when the URL provider returns one, constructs the real FfmpegEncoder(new FfmpegLocator()), starts the pump on go-live, stops it on end-stream, disposes in Shutdown, and flips StreamStatus.Error when the pump fails while live (minimal — detailed health surfacing is ship step 6).

Test plan (Good Dog Rule — ONE integration test): FramePumpTests.Start_CompositesScene_FeedsEncoder_StopsCleanly drives the full lifecycle against fakes — real SceneCompositor + real FramePump, fake IFfmpegEncoder — asserting the composited red backdrop frame actually reaches the encoder at the tier size and that stop tears everything down. Units: no-URL start skips the encoder, re-entrant start/stop no-ops, encoder start-failure raises Failed + disposes, ProcessFailed self-stops the pump, HealthUpdated forwards. ScreenCaptureManagerTests.GetLatestFrame_ReturnsLatestPump_UntilReleased pins the new accessor.

Out of scope (later ship steps): the loopback/mic → AAC mix (replaces anullsrc), health stats in the bottom bar (ship step 6), scene-switching transitions, and any flash-frame wiring.

Built (2026-08-12): FramePump shipped in Services/Encoder/, ScreenCaptureManager.GetLatestFrame added, MainViewModel wired end-to-end (resolver + both option builders + pump lifecycle), FramePumpTests (7) + GetLatestFrame test (1) added — build 0 warnings, 147 tests passing. Known consideration: the pump reads the active scene on a background thread while the UI can still edit it; a concurrent-mutation exception is contained (logged + Failed + pump stops) rather than crashing.

Bug 1 — bar wouldn't reliably change position (root cause found, then simplified): the original drag set Canvas.SetTop(bar, …) with a local value, which permanently overrides the Canvas.Top="{Binding SocialBarTop}" binding — the release-time SetSocialBarPosition → PropertyChanged(SocialBarTop) could never beat it. First fix added direction-snapping during the drag (SocialBarSnap.Decide, ±6px deadzone) + bar.ClearValue(Canvas.TopProperty) on release — but that still misbehaved for shaky hands (jitter around the deadzone: it snapped up reliably, then refused to come back down and snapped back to top). Superseded by a click-toggle (KISS, user decision): clicking the bar in the preview flips it top ⇄ bottom (MainViewModel.ToggleSocialBarPosition → the existing SetSocialBarPosition), the bar rides {Binding SocialBarTop} alone (no local values, no deadzone, no jitter sensitivity), and SocialBarSnap was removed. The ClearValue lesson stands: never set a local value on a property the binding owns.

Bug 2 — Mastodon showed the generic 7-star honeycomb (root cause found): the DB row @gramps@llamachile.tube had Software = NULL — nodeinfo was only ever resolved against the identity domain (llamachile.tube, a landing page), never probed for the real instance at mastodon.llamachile.tube. Fixed on three fronts: HttpSocialValidator now probes well-known subdomains (mastodon. → social. → … FediverseSubdomainCandidates) when the identity domain and its redirect both come up empty, under a ~15s linked-CTS budget, with an optional Action<string> log; MainViewModel heals any fediverse entry missing a software name on layout load (HealFediverseSoftwareAsync — static, testable; instance wrapper runs it off the UI thread, applies via the dispatcher, saves); SocialEntry.FediverseSoftware is now settable and raises PropertyChanged for LogoData, so the heal updates the icon in place. If the user later re-saves the entry with the icon fixed, the name persists with it.

Compositor rendering (user-approved scope): the social bar now appears on the live output, not just the preview — Compositor/SocialBarRenderer.cs rasterizes the entries into a transparent straight-alpha BGRA strip (1920-wide, 40px content + 24px glow pad, green #2ecc71 glow baked in) via RenderTargetBitmap (WPF glue like StaticPixelCache; the compositor core stays pure). SceneCompositor.Render takes optional socialBarFrame + socialBarTop (master space) and blits it last — above the branding flash (the old BlitFlash generalized to offset BlitOverlay). FramePump gains a socialBar: seam (Func<(VideoFrame?, SocialBarPosition)>, re-read every frame) and places the bar at 0 or SourceRectHeight − bar height. MainViewModel owns the frame (RenderSocialBarFrame, re-rendered on load/save/notify) and feeds the seam.

Tests (+6 → 153 passing, 0 warnings): settable FediverseSoftware updates LogoData, subdomain-probe unit + null-when-silent unit, the branch's one integration test Socials_HealMissingFediverseSoftware_RoundTripsThroughDb (temp-DB roundtrip: NULL software → healed via the validator → persisted), compositor bar overlay (top/bottom + above-flash), FramePump bar pass-through (Top then flipped to Bottom mid-run — the seam is re-read each frame), and both ISocialValidator fakes (FakeValidator/BlockingValidator) gained ResolveFediverseSoftwareAsync. The drag-snap units (SocialBarSnap) were removed with the click-toggle supersession.

Not included (say the word): rewriting the healed entry's ProfileUrl to https://mastodon.llamachile.tube/@gramps.

TASK 5 — Layout Persistence (SQLite)

Goal: Scenes, sources, and asset bytes survive restarts; assets are always available.

Status: ✅ Done

✅ SQLite database (Microsoft.Data.Sqlite) at %APPDATA%\ytLlive\ytLlive.db; schema versioned via PRAGMA user_version (currently v8)
✅ Assets live in the DB (BLOB keyed by SHA-256 content hash), never file paths — deleting the original file never breaks a scene
✅ File-model save/open — the active layout file is tracked (default is the AppData DB); Save Layout As… / Open Layout… switch the active file; auto-save writes to whatever is active
✅ Auto-save (invisible) — ~1.5s debounce on scene/source add/remove/reorder/rename/hide + any source transform change; flush on window close
✅ Startup — load the active file; seed the five canonical scenes only when the DB is empty; (+) re-adds a missing canonical scene and is hidden once all five are present; adding beyond the five is rejected
✅ Schema v1 → v8 — webcam columns (v2), singleton Webcam + per-scene WebcamSceneConfig (v3), RectWidth/RectHeight round-to-rect restore (v4), Source.IsBackdrop + Source.CaptureKey (v5), Scene.HasBackdrop — backdrop Live-only by policy (v6, one-time backfill + EnforceBackdropPolicy on every load), Scene.HasSocialBar (v7, dropped per-scene toggle — column back-compat, unread), Socials.BarEnabled (v8); the SocialEntry.Software fediverse-software column is a column-presence migration (commented v8→v9, no version bump — user_version stays 8); WindowHandle stays in-memory (per-session); save = transactional rewrite; orphaned assets pruned. v9 lands in the TASK 8 audio milestone (single-row Music — TrackPath/IsEnabled)

Design decisions

SQLite database (Microsoft.Data.Sqlite) at %APPDATA%\ytLlive\ytLlive.db; schema versioned via PRAGMA user_version.
Assets live in the DB, not on disk — Asset table stores image bytes (BLOB) keyed by a SHA-256 content hash (unique). Identical image content collapses to one row regardless of file name — the 1:M resource memory model, enforced by the database. No file paths; deleting the original file never breaks a scene.
File-model save/open — the active layout file is tracked (default is the AppData DB). Save Layout As… / Open Layout… switch the active file; auto-save writes to whatever is active.
Auto-save (invisible) — ~1.5s debounce on scene add/remove/reorder/rename/hide, source add/remove/reorder, and any source transform change; flush on window close.
Schema — Scene (Id, Name, IsHidden, IsChatScene, HasBackdrop, HasSocialBar, SortOrder), Asset (Id, Hash, Data, PixelWidth, PixelHeight), Source (Id, SceneId FK cascade, AssetId FK, Type, Name, IsEnabled, X/Y/Width/Height/Opacity, MonitorIndex, DeviceId, ClipShape, IsMirrored, SortOrder), Socials (Id, BarPosition, BarJustify — back-compat, unread, BarEnabled added via ALTER), SocialEntry (Id, SocialsId FK cascade, Service, Handle, ProfileUrl, SortOrder) — user_version 8 (v1 → v2 = ALTER TABLE adds the two webcam columns; v3 = singleton Webcam + per-scene WebcamSceneConfig; v4 = WebcamSceneConfig.RectWidth/RectHeight for the round-to-rect restore; v5 = Source.IsBackdrop
- Source.CaptureKey for the live-capture backdrop; v6 = Scene.HasBackdrop — the backdrop is Live-only by policy (one-time backfill turns Starting/BRB/Chat/Ending off and drops their backdrop sources; EnforceBackdropPolicy re-normalizes every load); v7 = Scene.HasSocialBar (per-scene toggle dropped — column back-compat, unread); v8 = Socials.BarEnabled). The SocialEntry.Software fediverse-software column is a column-presence migration (commented v8→v9, no version bump). WindowHandle stays in-memory (per-session). Save = transactional rewrite; orphaned assets pruned.
Startup — load the active file; seed the five canonical scenes (Starting/Live/BRB/Chat/Ending, SceneCatalog) only when the DB is empty. The (+) button re-adds a missing canonical scene and is hidden once all five are present; adding beyond the five is rejected — work with less, never more.

Goal: clean up the two side lists and the social bar per gramps's review.

Status: ✅ Done

✅ Scene rows are pure selection rows — the per-row edit/trash/visibility icons and the inline rename TextBox are gone (EditSceneCommand/RemoveSceneCommand/ToggleSceneVisibilityCommand + handlers + Scene.IsEditing removed; IsHidden stays persisted + dims hidden rows)
✅ Source rows gained the trio — edit (inline rename via new EditElementCommand + SceneElement.IsEditing), visibility eye (new ToggleElementVisibilityCommand flips SceneElement.IsVisible; the eye style now binds IsVisible, open/slashed + row dims to 0.45 when hidden), and the existing trash
✅ Duplicate resource names get a no-space incrementing suffix via shared NextSourceName (Image, Image2, Image3…) — next free number derived from the names actually in the scene, so deleting a middle resource never collides (AddSource + AddReusedImage both use it)
✅ Social bar renders the full validated handle — MaxWidth=200 + TextTrimming removed from BOTH SocialBarRenderer and the preview template (mastodon @gramps@… no longer cuts off)
✅ Side panels stay fixed-width (left 220 / right 300) — deliberate: they never re-layout on resize, the preview absorbs it
✅ Focus-loss capture lag documented as a known OS limit in ai.md — deferred by user decision (no code change)

Design decisions

Next free number from names, not type counts — the old scheme counted elements by SourceType (count == 0 ? baseName : base+count+1), which collided after deletions; the new helper scans actual names.
One WPF App per AppDomain — the real-App tests (round-clip + naming) share RealAppHost (a dedicated STA thread owning the single App) via the RealApp serial collection, instead of each calling new App().

TASK 7 — Meter scaling amplification (voice meter uses the full bar)

Goal: the mic (and game) meters feel light — speech peaks should peg into the red at maxed volume.

Status: ✅ Done

✅ AudioLevelMeter.ToDisplay now adds +10 dB of input amplification before the −60..0 dBFS → 0..1 log mapping (was raw dB): speech peaks (~0.2 RMS, −14 dBFS) read ~0.93 → red zone; normal speech (~0.05, −26 dBFS) ~0.73 → yellow; background noise ≤0.001 linear (−60 dBFS) still reads 0 (the meter never idles on it)
✅ Unit tests updated to the new mapping + new ToDisplay_Pushes_Speech_Peaks_Into_Red_At_Maxed_Volume (0.2 → 0.92..0.95, 0.05 → 0.72..0.75)

Design decisions

Amplify at the mapping, not in the VM — one knob (ToDisplay), shared by the mic bar and the game bar; the × MicVolume gain-on-noise behavior is untouched (raising the slider still moves ambient noise up the bar).
+10 dB, not more — louder boosts push quiet speech into the lower half and make the noise floor visible; +10 puts speech peaks solidly in red while idle stays at zero.

TASK 8 — Audio milestone: real stream audio + voice filters + TRAX music (2026-08-14)

Goal: every audio issue done and tested in one branch — real mic/game audio reaches the encoder (replacing the anullsrc silence), MicVolume/GameAudioVolume/mute become real pre-AAC gains, TASK 10's voice filters land, auto-duck keeps the creator's voice over game + music, and a free background-music source ("TRAX") plays into the sound bar and the stream.

Status: ✅ SHIPPED 2026-08-14 — 196 tests passing, 0 warnings (ONE integration test: `AudioPipelineTests.Mix_WithFiltersDuckAndGain_Lands_On_AudioPipe`)

✅ Real audio into the encoder — the mixer's loopback samples were dropped today (AudioMixer.OnLoopbackSample metered only) and MicVolume was meter-only; FfmpegArgs.cs:28 ran -f lavfi -i anullsrc (silence). Transport: ffmpeg reads a Windows named pipe — -f f32le -ar 48000 -ac 2 -i \\.\pipe\ytllive_audio replaces the anullsrc block, plus explicit -map 0:v -map 1:a; pipe name via EncoderOptions.AudioPipeName (DefaultAudioPipeName = "ytllive_audio"). Mixer stays two inputs (mic + loopback) — no N-source abstraction, no StereoMixer class. The VM owns the pipe lifecycle (BeginGoLive → _audioMixer.StartLive(pipeName), StopStream → _audioMixer.StopLive() before the pump stops); FfmpegEncoder.cs is untouched.
✅ Honest gains reach the stream — MicVolume scales the mic channel; GameAudioVolume + mute scale the loopback channel (the "mute in preview but stream still plays" footgun dies with it). One knob per input = KISS. Implemented as Func<double> gain seams on AudioMixer (micGain/loopbackGain), wired by MainViewModel.
✅ Voice filters (TASK 10, before the meter AND the mix) — VoiceFilterChain: bass (LowShelfFilter 120 Hz +4 dB) → treble (HighShelfFilter 8 kHz +3 dB) → noise gate (pure C#, creator's choice) → compressor (threshold 0.5, 4:1). Pure stateful DSP (TDF2), sine-in unit tests. Meter stays post-filter.
✅ Auto-duck — pure envelope (AutoDucker): mic RMS > 0.02 → game+music dip ×0.25 (~12 dB), attack 0.05 / release 0.005, recover on release. Always-on, no knobs. Strong form of the creator's "voice always over the game volume" idea.
✅ TRAX — background music, FREE (keeps the one paid line = Alerts + flash removal). MusicPlayer: NAudio MediaFoundationReader (mp3/wav/m4a) → VolumeWaveProvider16 at the hardcoded 0.20 (fixed, not changeable) → WaveOutEvent on the default device (added the sibling NAudio.WinMM 2.2.1 package — WaveOutEvent isn't in NAudio.Wasapi), loop on end. No 3rd mixer input and no MusicVolume — music plays on the desktop, the existing loopback picks it up: heard in headphones, the sound-bar meter bounces, and the stream carries it through the loopback channel (ducked with game when the voice is hot). Known wrinkle (out of scope): YouTube mutes VODs carrying copyrighted music — future "music on live, off VOD".
✅ TRAX footer control (final spec, agreed 2026-08-14) — YtButtonSecondary, status dot + "TRAX" text, in the center footer stack beside MIC:
- Status dot (same 8px Ellipse pattern as MIC/Socials): red no track loaded · yellow loaded not playing · green playing.
- Left-click → toggle play/pause; no track loaded → opens the picker instead.
- Right-click → always opens the in-app track picker (TraxButton_PreviewMouseRightButtonUp, code-behind pattern like GameSpeaker_MouseLeftButtonUp, e.Handled = true so no context menu).
- Tooltip → playing/loaded track name; "No track — right-click to choose background music" when empty.
- No slider — volume is the 0.20 constant. Picker = plain OS OpenFileDialog filtered to mp3/wav/m4a/aac/flac/ogg.
✅ Sound bar shows music — IsGameAudioBarVisible = gameDetectorProducingSound || isMusicPlaying; relabelled "Game Audio Capture" → "Desktop Audio" (TRAX rides the same channel). Follow-up (2026-08-15): the gating is gone — the game audio bar is ALWAYS visible (the IGameAudioDetector/GameAudioDetector/GameAudioHysteresis stack + the VM's 250ms poll timer and their tests were deleted; the bar kept hiding the desktop meter when no full-screen game with sound was up, and the creator wants it pinned).
✅ Capture hardening (design changed at build time) — the plan's "force IEEE-float 48 kHz on both WASAPI sources" was dropped: NAudio 2.2.1's WasapiCapture exposes no overridable GetDefaultMixFormat, so both sources capture the device's own mix format and the mixer's pure TinyResampler normalizes any rate/channel count to 48 kHz stereo (the resampler IS the design, not a fallback). Stop path disposes the audio pipe (EOF) in MainViewModel.StopStream before the pump stops, so both ffmpeg inputs end in order.
✅ Schema v9 — single-row Music (TrackPath, IsEnabled; volume is the 0.20 constant) + migration in LayoutStore.cs (user_version 9; Save gained an optional Music? music = null param so existing 3-arg callers still compile; load query + null reset).
✅ Docs in the same commit — ai.md audio section rewritten ("one knob per input", 2-in mix, auto-duck, TRAX free, VOD-mute wrinkle), TASK 10's status, this task's status, HANDOFF.

Files (new): Services/Audio/VoiceFilterChain.cs (+ LowShelfFilter/HighShelfFilter/NoiseGate/Compressor), Services/Audio/AudioRingBuffer.cs (async-arrival source buffers), Services/Audio/AutoDucker.cs, Services/Audio/TinyResampler.cs, Services/Audio/MusicPlayer.cs, Services/Audio/NamedPipeAudioWriter.cs (+ IAudioPipeWriter seam), Models/Music.cs.

Files (changed): AudioMixer.cs (filter chain before meter+mix, loopback into the mix, gains via Func<double> seams, ducker, silence-filler so the pipe never stalls, StartLive/StopLive), WasapiMicAudioSource.cs (device mix format — resampler normalizes; no forced format), FfmpegArgs.cs (named-pipe input + -map, name via EncoderOptions.AudioPipeName), EncoderOptions.cs (AudioPipeName), ytLive.csproj (NAudio.WinMM 2.2.1 for WaveOutEvent), LayoutStore.cs (schema v9), MainViewModel.cs/MainWindow.xaml/MainWindow.xaml.cs (footer TRAX group, sound-bar visibility OR music, go-live starts the pipe, end closes it first).

Testing (Good Dog Rule — ONE integration test): AudioPipelineTests.Mix_WithFiltersDuckAndGain_Lands_On_AudioPipe — real filter chain + mixer + ducker + NamedPipeAudioWriter, fake IAudioSources, test-side NamedPipeClientStream reads the bytes and asserts post-filter/post-gain/post-duck mixed stereo (ring buffers pre-filled so the first pipe tick already carries real audio — avoids a start-of-stream silence race). Units: each DSP stage (known sine-in → expected gain), ring buffer wrap/underflow/overwrite, ducker envelope, resampler (down/up/stateful/identity), FfmpegArgs pipe+map, schema v9 roundtrip + no-music-row. Existing tests stay green.

Out of scope (this branch): IP webcam (video-only when it lands — never an audio input), chat box source (TASK 3 item 18), alt-key crop, credits, background removal, music-off-VOD, PremiumUrl (TASK 10 seam).

TASK 9 — YouTube Live Stream Management

Goal: Create/bind broadcasts, monitor YouTube-side stream health — the v3 way.

Status: ⏳ In progress — items 1–3 SHIPPED (reusable stream 2026-08-16; report-by-exception health 2026-08-16); items 4–7 still open (each its own branch/PR)

☑ Broadcast creation — title/description/privacy/scheduledStartTime via API, with the v3 flags above (SHIPPED: CreateBroadcast sends enableAutoStart/Stop, enableMonitorStream=false, latencyPreference=low, selfDeclaredMadeForKids=false)
☑ Reusable stream — create once, cache + reuse; bind to broadcast (SHIPPED: GetOrCreateReusableStreamAsync lists-then-inserts the variable/isReusable stream, cached via LayoutStore Settings, bound at broadcast insert via boundStreamId; _rtmpUrlProvider yields the ingest URL so go-live actually encodes + pushes)
☑ Health monitoring — poll liveStreams.list healthStatus + configurationIssues[], surface banner only on warning/error (SHIPPED: GetStreamHealthAsync(streamId) 30s while live; pure StreamHealthReporter.BannerFor = report-by-exception; banner strip under the top bar, amber warning / dark-red error, via HealthIssueBanner/HealthIssueBackground; poll failures log-only; ONE integration test GetStreamHealthAsync_Report_By_Exception_Banner_Only_On_Warning_Or_Error)
☐ Live chat — poll liveChat/messages, render in right panel, support Super Chat + membership badges
☐ Error handling — the YouTube error codes: errorStreamInactive, invalidTransition, redundantTransition, liveStreamDeletionNotAllowed, liveStreamModificationNotAllowed, liveBroadcastBindingNotAllowed
☐ Visibility picker — remove temporary "always Private" enforcement (shipped as test-only; now unlocked for v1). User picks Private/Unlisted/Public from the go-live dialog. Trivial: remove the hardcoded override in YouTubeStreamService.CreateBroadcast (currently privacyStatus = "private" regardless of dialog selection)
☐ Full broadcast form — expose all YouTube API-supported fields in the go-live dialog. Core tab: title, description, visibility, made-for-kids, schedule (start + optional end). Advanced tab (expandable, sane defaults): latency (Normal/Low/Ultra-Low), DVR, embed, record-from-start, projection (rectangular/360°), closed captions, auto-start, auto-stop, monitor stream, region restrictions. Monetization via liveBroadcasts.update (insert-only on that resource) — separate step after broadcast creation. Remove unsupported categoryId (not a liveBroadcast field, silently ignored)

Design decisions (v3)

One-click go-live — liveBroadcasts.insert with enableAutoStart=true, enableAutoStop=true, enableMonitorStream=false, selfDeclaredMadeForKids=false, latencyPreference=low. No transition(live) call, no testing stage, no liveStarting polling. Encoder starts → YouTube brings it live by itself.
Variable reusable stream — cdn.resolution=variable, cdn.frameRate=variable, isReusable=true. Create once per channel, cache ingestion URL + stream name, reuse for every broadcast. Any quality tier works without recreation; auto step-down needs no API calls.
Report-by-exception — poll liveStreams.list; banner only on healthStatus warning/error issues (configurationIssues[]). Bottom strip = YouTube logo + green/red connection dot (clickable → opens the dialog).
One dialog, three states — not connected (sign-in) / connected-offline (all editable) / live (title + description + visibility editable; quality + account greyed out). Both entry points (Start Stream button + bottom strip) open it; prefilled from saved session profile.
Live edits — liveBroadcasts.update with part=snippet,status for title/description/privacy.
End stream — stop encoder → transition(complete), with enableAutoStop as the safety net.
Broadcast ID == Video ID — one ID to track status, health, and the auto-created VOD (recordFromStart + enableDvr).

Requirements:

Broadcast creation — title/description/privacy/scheduledStartTime via API, with the v3 flags above
Reusable stream — create once, cache + reuse; bind to broadcast
Health monitoring — poll liveStreams.list healthStatus + configurationIssues[], surface banner only on warning/error
Live chat — poll liveChat/messages, render in right panel, support Super Chat + membership badges
Error handling — the YouTube error codes: errorStreamInactive, invalidTransition, redundantTransition, liveStreamDeletionNotAllowed, liveStreamModificationNotAllowed, liveBroadcastBindingNotAllowed
Visibility picker — Private/Unlisted/Public from the go-live dialog
Full broadcast form — Core + Advanced tabs with all API-supported fields

TASK 10 — Monetization: watermark-only subscription + Polar billing

Goal: annual subscription via Polar.sh that removes the branding watermark. All features are free — the only difference between free and paid is the watermark.

Business details (pricing, Polar product/checkout/discounts) in MONETIZATION.md (gitignored).

Polar product: d105dfa1-497e-423b-8cd4-e0ee2e3abbc0 (LlamaCasty, $99/yr, currently private) Checkout: llamacasty.com → Polar hosted page (~80% configured) Discounts: LLAMAFOUNDER (100% off, 50 uses, once), LLAMA50 (50% off, 12 months, unlimited) Org ID: c05fb364-b967-4f6c-adf2-8a144e46d085 (org-scoped OAT — organization_id can be omitted from API calls) Key prefix: LCYT- Validate endpoint: POST https://api.polar.sh/v1/customer-portal/license-keys/validate (no auth required for client-side validation)

Status: 🔶 In progress — steps 1–7 shipped, Velopack update URL pending

✅ Billing provider — Polar (polar.sh) chosen and configured. Product, price, license key benefit, discounts, checkout link all created.
✅ License key entry + verification UI — About overlay swap panel (same pattern as Licenses panel). Text input + "Activate" button + status message. "Enter License Key" button in the hub.
✅ Offline entitlement — LayoutStore Settings table: LicenseKey, LicenseValidatedAt, IsPremium. 14-day offline grace period. Startup re-validates online, falls back to cache.
✅ BrandFlash → watermark toggle — IsPremium property setter flips BrandFlashEnabled = !value. Flash stops immediately on premium activation. Flash asset VideoFrame rendering deferred to TASK 4 compositor.
✅ Remove social slot locks — all 6 social bar slots open to everyone (no IsPremium gating on slots 3-6) — shipped as TASK 10c
✅ "Unlock Premium" button — PremiumUrl set to Polar checkout; IsPremiumAvailable is true; button lights up in the About hub.
✅ Renewal/lapse handling — startup re-validates against Polar API. Lapse → IsPremium = false → flash returns. Grace period: 14 days offline.
🔶 Velopack auto-updates — NuGet packages + bootstrap in App.xaml.cs. Update URL self-hosted on DO droplet (pending configuration).

TASK 11 — Post-pause polish batch: the creator's 8 review issues (2026-08-15, shipped)

Status: ✅ SHIPPED 2026-08-15 — 197 tests passing, 0 warnings (ONE integration test: AudioPipelineTests.Mix_HonorsProviderGains_AndGameMute_KillsTheLoopback).

The creator reviewed the TASK 8 build and filed 8 issues. All fixed in one branch on main:

✅ Desktop/game audio volume slider had no effect (headset OR stream) — the AudioMixer's micGain/loopbackGain seams defaulted to unity because the VM never passed them; the sliders were decorative. Fixed: new AudioGainProvider (Services/Audio/) hands MicMuted/MicVolume/ GameMuted/GameAudioVolume to the mixer at construction, read live each mix tick.
✅ Desktop/game mute button had no effect — same root cause; GameMuted now zeroes the loopback on the stream. Locally, MusicPlayer.LocalGain (new) is scaled by the game bar on every volume/mute change + on TRAX load, so the creator hears the control work on the music (the track rides the loopback channel, so it ducks locally exactly as it does on stream).
✅ TRAX played once then stopped — MusicPlayer.OnPlaybackStopped only looped when Position >= Length, unreliable for MediaFoundationReader; now any clean stop (e.Exception == null) rewinds + replays; only errors/explicit stops surface via PlaybackEnded.
✅ TRAX/MIC buttons swapped — TRAX no longer sits between MIC and the (mic-specific) meter; the footer's mic cluster is MIC + meter + mute + volume.
✅ Mic source persists across restarts — LayoutStore gained a Settings key/value table (SaveMicSourceName/LoadMicSourceName); the VM saves on pick and restores before the mixer's first Start, so a restart reconnects the same already-vetted device (green dot) or reports it missing (yellow) instead of falling back to the default endpoint.
✅ TRAX moved into the game audio bar — left of the "Desktop Audio" label in the preview overlay (the creator's follow-up pick), no longer in the footer; its tooltip teaches the clicks ("left-click pauses/plays, right-click chooses a track").
✅ Backdrop's icons shifted right — the trash button's Collapsed released its column; a new HiddenBoolToVisibilityConverter keeps the slot reserved (Hidden), so edit/eye stay in their fixed columns for every source row.
✅ No separation between scenes and sources — a 1px hairline with top/bottom padding now sits between the two listboxes in the left panel.

Design decisions

The gain seams are the one source of truth — AudioGainProvider is a thin seam (four Funcs) so the mixer contract and the mute⇔zero-volume rule are testable without constructing the VM.
Stream-honest AND locally audible — the game bar's controls now do two things: scale the loopback on the stream and scale the music in the headphones. Native game audio is untouched (system output, OBS-style non-monitored); an OBS-style monitor loop is out of scope.
Persist the DisplayName, not the device ID — the FriendlyName match was already the mic's identity end-to-end; Settings just makes it survive restarts.

TASK 12 — Master limiter on the live mix (2026-08-15)

Queued by the creator during the TRAX discussion: "should the soundtrack be limited to avoid squandering resources / should it cap at 20% / how do the three sound events balance?" Review conclusions (all three instincts checked out, only ONE real gap):

File size — no guard needed. MusicPlayer uses MediaFoundationReader, which streams from disk (progressive source): memory is flat (~a few MB) regardless of file size; CPU negligible.
20% cap — already enforced by construction. MusicPlayer.MusicVolume = 0.20f + music rides the SAME WASAPI loopback (and therefore the same gain) as game audio, so music:game is always exactly 0.20:1 at any slider position — it literally cannot rise above 20% of the current desktop volume.
The gap: AudioMixer.FillAndMix summed mic + loopback with no output ceiling — mic 100% + loud game/music could pass 0 dBFS and clip the AAC encode.

Shipped:

✅ Services/Audio/MasterLimiter.cs (pure, unit-tested) — a −1 dBFS ceiling (Ceiling = 0.891), instant attack per frame (a hot frame is scaled exactly to the ceiling — no overshoot), smoothed release toward unity so loud passages don't pump; gain never exceeds 1 (no boosting). Applied at the end of AudioMixer.FillAndMix, right before the pipe write.
✅ Unit tests — over-ceiling frames trimmed to ≤ ceiling; sub-ceiling frames never boosted; gain recovers to unity after the loud frame ends.
✅ ONE integration test (MasterLimiter_CapsTheLiveMix_OnThePipe) — real mixer + pipe harness: a 0.95 loopback bed (hotter than the ceiling) is capped to exactly 0.891 on the wire while staying audible.
✅ Docs in the same commit — ai.md (go-live audio section), Services/index.md (new row).

No changes to MusicPlayer, the 0.20 cap, the ducker, or the meter zones. Build 0 warnings.

Goal: marcom/social media assets and strategy for v1 launch.

Business details (positioning, messaging, platform strategy, launch assets) in MARCOM.md (gitignored).

Status: 🔶 Scoped — nothing built; queued after all v1 features ship

☐ Product positioning & messaging (one-liner, elevator pitch, competitive positioning)
☐ Social media swipe files (pre-written posts for supporters)
☐ Launch day assets (demo video, screenshots, GIFs, social graphics, press kit)
☐ Platform strategy (YouTube, Reddit, indie dev communities, Product Hunt)
☐ Founder story (67-year-old dev building his own streaming app)
☐ Email announcement templates
☐ "Build in public" livestream angle (stream the coding of ytLlive with ytLlive)

TASK 14 — Creator feedback batch: quick fixes (TRAX, meter, backdrop, sliders)

Goal: address creator's immediate UX feedback from testing session.

Status: 🔶 In progress — Branch 2 shipped

✅ TRAX volume — LocalGain = GameAudioVolume * 0.2 so TRAX is 20% of the desktop audio slider (was 100%, now quieter)
✅ TRAX tooltip — replaced ToolTip binding with ToolTipService.ToolTip + Placement="Top" to fix z-order inside Viewbox overlay
✅ Mic meter boost — MeterLevel multiplied by 1.2× before clamping so the bar reads ~20% higher at same input level
✅ Click-to-position sliders — clicking anywhere on the mic/game volume slider track now jumps the thumb to that position before starting the drag
✅ Backdrop visibility — the dedicated <Image> now binds Visibility to BackdropVisible (ViewModel property); compositor checks IsVisible before blit
✅ Rename "Backdrop" → "Game Capture" in EnsureBackdrop() + test assertions
✅ Elements panel — new "ELEMENTS" section beneath Sources list; per-element config with live preview, ✕ revert, ✓ applied indicator
✅ Webcam border config — color hex input + color swatch picker, border thickness 0–10 slider
✅ Countdown source — SourceType.Countdown, pick list (Starting/BRB scenes only), timer minutes 1–60 default 15
✅ Web source — SourceType.WebSource, pick list (all scenes), URI text input
✅ Elements scrollbar deselect fix — clicking the Elements panel scrollbar no longer deselects the selected element (scrollbar lives inside the ScrollViewer, not the StackPanel; guard now checks ElementsScrollViewer instead of ElementsPanel)
✅ Web source URI + buttons inline — merged URI TextBox and check/revert buttons into a single DockPanel row (✓ rightmost, ✕ left); buttons no longer wrap to a separate line
✅ Web source URI hidden for webcam — URI section was showing for webcam due to WPF binding path SelectedElement.Type not resolving on WebcamSceneConfig; replaced with IsWebSource virtual property on SceneElement (mirrors existing IsWebcam pattern), overridden in Source to return Type == SourceType.WebSource, bound via BoolToVis
✅ Check/revert consistent order + accept — all three element sections (webcam border, countdown timer, web URI) now use consistent [✕][✓] button order; green check buttons enabled with click handlers that save current values as snapshot baseline (so revert undoes to last accepted state, not initial selection)

TASK 15 — Stock scene background images

Goal: branded background images for each canonical scene (Starting, Live, BRB, Chat, Ending) that both provide working hints to creators and self-promote LlamaCasty.

Status: ✅ Shipped — all 5 scenes seeded

✅ Starting backdrop — Assets/starting-backdrop.jpg (embedded resource), SeedStartingBackdrop() in MainViewModel, idempotent
✅ BRB backdrop — Assets/brb-backdrop.jpg (user-provided Gemini image), SeedBrbBackdrop() in MainViewModel, idempotent
✅ Live backdrop — Assets/live-backdrop.jpg (user-provided image), seeded via SeedLiveBackdropAsset(), used as the static fallback (TASK 16)
✅ Ending backdrop — Assets/ending-backdrop.jpg (user-provided image), SeedEndingBackdrop() replaces existing (not skip-if-exists)
✅ Chat backdrop — Assets/chat-backdrop.jpg (user-provided image), SeedChatBackdrop() at startup
✅ EnsureDefaultBackdrop() called on every scene switch — seeds the default backdrop if scene has none

Design decisions

Images are user-provided, not generated — the creator owns the brand look
Each scene gets its own image (Starting = "Starting Soon", Live = live branding, BRB = "Be Right Back", etc.)
The images double as the no-game backdrop fallback (TASK 16), so the Live scene image should work as a desktop replacement

TASK 16 — Kill infinity display (no-game backdrop fallback)

Goal: when no full-screen game is detected, show a static branded placeholder instead of capturing the primary display (which causes the infinity mirror effect).

Status: ✅ Shipped

Changes:

ResolveAutoCaptureKey() returns null when no full-screen game is detected (was: primary monitor)
ReacquireScreenCaptures() — clears stale CaptureKey when auto == null (handles DB upgrades); skips AcquireAsync when no keys are set
SeedLiveBackdropAsset() — sets AssetId on the backdrop element itself (NOT a Background source); called from ReacquireScreenCaptures() after EnsureBackdrop() guarantees the element exists
Source.DisplaySource falls back to _imageSource when VideoImageSource is null for live captures
Assets/live-backdrop.jpg — user-provided image, added as <Resource> in csproj
ResolveOutputFrame() in compositor — Source { IsLiveCapture: true, CaptureKey: not null } already skips null keys; fallback pattern Source { AssetId: not null } picks up the static image

Rendering flow:

Scenario	Preview	Compositor
Game detected	Live capture (opaque, covers static asset on backdrop)	`ScreenCaptureManager.GetLatestFrame`
No game	Static image via `DisplaySource` → `_imageSource` on backdrop	`StaticPixelCache.Get(assetId)`
Manual capture pick	Live capture	Live frame

TASK 17 — Web source rendering (WebView2)

Goal: make the web source actually render URLs into the preview and stream output.

Status: ☐ Not started — required for v1

☐ Add Microsoft.Web.WebView2 NuGet package
☐ Schema v10: WebUri TEXT column on Source table + migration in LayoutStore.cs
☐ Persist Source.WebUri on save/load (currently in-memory only — lost on restart)
☐ Hidden WebView2 control per web source — navigates to WebUri, renders in-app
☐ Frame capture from WebView2 (CoreWebView2.CapturePreviewAsync or CompositionSurface) → VideoFrame (BGRA8)
☐ Wire into FramePump resolver — Source { Type: WebSource } → latest WebView2 frame
☐ Wire into SceneCompositor — render web source as an image element at its position/size
☐ Preview shows live web content (not just a blank rectangle)
☐ Handle navigation errors, invalid URIs, timeout gracefully

Design decisions

WebView2 is the only option for Windows — it's pre-installed on Windows 10 20H2+ and Windows 11
The web source is a standard element — positioned/sized/opacitied like any image source
Frame capture rate can be lower than video FPS (5-10 fps for web content is fine)
This enables Streamlabs/StreamElements overlays via web URLs

TASK 18 — Local recording

Goal: record the stream output to a local file, with or without simultaneously streaming.

Status: ☐ Not started — required for v1

☐ Extend EncoderOptions with OutputPath? and recording mode flags
☐ FfmpegArgs.Build gains a local-file branch: MP4/MKV container for file output
☐ Three modes:
- Record only — no RTMP push, just local file (for pre-recorded content)
- Stream only — RTMP push, no local file (current behavior)
- Stream + record — dual output via -f flv rtmp://... -f mp4 file.mp4
☐ Recording controls in UI (REC button, file path picker, duration display)
☐ Schema migration for recording preferences (default output path, format)
☐ Branding flash appears in local recordings too (free tier)
☐ Output folder: %APPDATA%\ytLlive\recordings\ with timestamped filenames

Design decisions

FFmpeg supports multiple outputs natively (-f flv rtmp://... -f mp4 file.mp4) — no second subprocess needed
MP4 is the default container (widely compatible); MKV as an option (crash-safe, can be remuxed)
Record-only mode is useful for pre-recorded content or testing without going live
The branding flash carries into local recordings (free tier billboard extends to recordings)

v1 execution order

The tasks below are ordered by dependency and risk. Each task builds on the previous.

✅ TASK 9.4 — Live chat (right panel) — wire YouTubeChatService.Start(), parse messages, render in panel. Foundation for chat box source.
✅ TASK 3.18 — Chat box source — renders chat ON the stream. Depends on TASK 9.4 (same message parsing).
✅ TASK 10 — Polar billing — license key entry + watermark toggle + Velopack auto-updates (steps 1-7 shipped; Velopack update URL pending).
TASK 19/23 — Control Surface UX — director's control room: thumbnails above central monitor, transitions (Cut/Fade/Move), edit mode offline only, left panel two-state (layers/props ↔ chat), right panel eliminated.
TASK 20 — Hotkeys — global keyboard shortcuts. Small scope, huge UX impact.
TASK 17 — Web source (WebView2) — enables alert ecosystem.
TASK 18 — Local recording — independent, but pairs with stream.
TASK 21 — Media source — video file playback for non-Live scenes.
TASK 22 — Audio sync offset — small, quality-of-life.
✅ TASK 15 — Stock bg images — all 5 scenes seeded (Starting, BRB, Live, Chat, Ending). EnsureDefaultBackdrop() runs on every scene switch.
TASK 13 — Social media launch kit — after all v1 features ship.

TASK 19 — Scene transitions → MERGED into TASK 19/23 (Control Surface UX)

Superseded by the Control Surface UX task. Cut/Fade/Move transitions, thumbnails, edit mode, left panel two-state — all shipped as one cohesive feature. See the Control Surface UX section in ai.md.

Design decisions

Cut is free — instant switch, no blending, zero CPU cost. This is the default.
Fade is the minimum expectation — 300ms crossfade is universal. Every streaming tool ships this.
Move is economical — simple slide animation, no video decoding needed. Good middle ground.
Custom is premium — media/stinger transitions require video playback. Heavy but expected by mid-tier streamers.
Transitions happen in the compositor, not post-encode. The preview sees the same blend as the live output.

TASK 20 — Hotkeys (keyboard shortcuts)

Goal: keyboard shortcuts for scene switching and common actions — the single biggest UX gap.

Status: ☐ Not started — required for v1

☐ HotkeyManager service — register/unregister hotkeys per scene + common actions
☐ Scene switching hotkeys — F1-F5 for the 5 canonical scenes (configurable)
☐ Common action hotkeys — Start/Stop stream, Mute/Unmute mic, Mute/Unmute game, Toggle TRAX
☐ Hotkey configuration UI — Settings dialog or per-scene hotkey picker
☐ Global hotkeys — work even when app is not focused (for gaming)
☐ Conflict detection — warn when two actions share the same hotkey
☐ Schema migration for hotkey bindings (persist across restarts)
☐ Tests: hotkey registration, conflict detection, scene switch via hotkey

Design decisions

Global hotkeys are mandatory — streamers are in-game and cannot alt-tab. F1-F5 must work from anywhere.
Simple config — one row per action: [Action] [Current Key] [Change]. No complex macro system.
No Stream Deck yet — bare keyboard first. Stream Deck support (physical devices) is v1.1+.

TASK 21 — Media source (video file playback)

Goal: play video files (MP4, MOV, AVI) into scenes — starting soon videos, BRB loops, intro/outro clips.

Status: ☐ Not started — required for v1

☐ MediaSourceType enum: Video, Audio (audio-only files via media source)
☐ SourceType.MediaSource addition to the enum
☐ MediaSourceModel: FilePath, IsLooping, Volume (0-1), PlaybackState
☐ VideoFrameSource: FFmpeg-based video decoder → VideoFrame pipeline
☐ Frame capture from video file (decode at native FPS, output BGRA8 frames)
☐ Wire into FramePump resolver — Source { Type: MediaSource } → latest video frame
☐ Wire into SceneCompositor — render media source as an image element at its position/size
☐ Loop control — IsLooping property, restart on end
☐ Volume control — per-source volume slider for audio playback
☐ UI: file picker (filtered to video formats), loop toggle, volume slider
☐ Schema migration for media source settings (file path, loop, volume)
☐ Tests: video frame extraction, loop behavior, volume scaling, file validation

Design decisions

FFmpeg handles all formats — no codec-specific code. FFmpeg already in the project.
Audio plays through the desktop channel — media source audio is captured by the WASAPI loopback (like TRAX/game audio). No separate audio routing needed.
This pairs with TRAX — TRAX is background music, media source is background video. Together they make non-Live scenes (Starting/BRB/Ending) feel polished.
Not a full NLE — no trimming, no multi-track, no effects. Just "play this video in the scene."

TASK 22 — Audio sync offset

Goal: per-source audio delay compensation to prevent lip-sync drift from USB mics and capture cards.

Status: ☐ Not started — required for v1

☐ AudioSyncOffset property on AudioSource models (default 0ms, range -500ms to +500ms)
☐ Apply offset in AudioMixer — delay or advance audio samples relative to video
☐ UI: offset slider per audio source (or global offset for simplicity)
☐ Persist offset in LayoutStore (schema migration)
☐ Tests: offset application, positive/negative delay, boundary values

Design decisions

Global offset first — one setting for all audio sources. Per-source is v1.1+.
Simple slider — -500ms to +500ms, default 0. No numeric input needed.
Visual feedback — show a "sync OK" indicator when offset is applied.

TASK 23 — Studio mode → MERGED into TASK 19/23 (Control Surface UX)

Superseded by the Control Surface UX task. Studio mode is now the entire app's paradigm — the director's control surface with 5 thumbnails + central monitor. See the Control Surface UX section in ai.md.

Backlog (future versions)

v1.1 — Stream Deck / Loupedeck integration (requires hotkey foundation from TASK 20)
v1.1 — Per-source audio sync offset (global offset ships in TASK 22)
v1.1 — Multiple profiles/presets (save different configs for different stream types)
v1.1 — Chroma key filter (green screen removal, or ONNX background removal)
v1.1 — Virtual camera output (Zoom/Discord/Teams)
v1.1 — Replay buffer (instant replay with hotkey)
v2 — Multi-destination restreaming (if needed; casual streamers may outgrow LlamaCasty first)
v2 — Stream clipping
v2 — Export/import settings

101 KiB Raw Blame History Unescape Escape

ytLlive — Task List

YouTube Live API — research facts (authoritative, v3 build)

Two features that reshape the design

Compliance gotchas (maps perfectly to report-by-exception)

Tips we should take advantage of

TASK 1 — Initial Scaffold

Status: ✅ Done

TASK 2 — YouTube OAuth2 Authentication

Status: ✅ Done

Requirements:

Tests:

TASK 3 — Capture Pipeline (Scenes/Sources)

Status: 🔶 In progress

The Minimal Source Set (design decision — do not expand casually)

Source memory model (design decision)

Scene transitions (design decision)

Requirements:

TASK 4 — RTMP Ingest to YouTube

Status: 🔶 In progress

Requirements:

Ship step 1 — Scene compositor (the frame source)

Ship step 2 — FFmpeg locator (the encoder's binary)

Ship step 3 — Encoder + RTMP push (the FFmpeg subprocess)

Ship step 4 — WASAPI audio capture (the meter comes alive)

Ship step 5 — Frame-pipeline wiring (the encoder gets a frame source)

Ship step 5.5 — Social bar bug fixes + the bar on the live output (2026-08-13)

TASK 5 — Layout Persistence (SQLite)

Status: ✅ Done

Design decisions

TASK 6 — UI polish batch (scenes/sources rows, dedup naming, social bar)

Status: ✅ Done

Design decisions

TASK 7 — Meter scaling amplification (voice meter uses the full bar)

Status: ✅ Done

Design decisions

TASK 8 — Audio milestone: real stream audio + voice filters + TRAX music (2026-08-14)

Status: ✅ SHIPPED 2026-08-14 — 196 tests passing, 0 warnings (ONE integration test: AudioPipelineTests.Mix_WithFiltersDuckAndGain_Lands_On_AudioPipe)

TASK 9 — YouTube Live Stream Management

Status: ⏳ In progress — items 1–3 SHIPPED (reusable stream 2026-08-16; report-by-exception health 2026-08-16); items 4–7 still open (each its own branch/PR)

Design decisions (v3)

Requirements:

TASK 10 — Monetization: watermark-only subscription + Polar billing

Status: 🔶 In progress — steps 1–7 shipped, Velopack update URL pending

TASK 11 — Post-pause polish batch: the creator's 8 review issues (2026-08-15, shipped)

Design decisions

TASK 12 — Master limiter on the live mix (2026-08-15)

TASK 13 — Social media launch kit

Status: 🔶 Scoped — nothing built; queued after all v1 features ship

TASK 14 — Creator feedback batch: quick fixes (TRAX, meter, backdrop, sliders)

Status: 🔶 In progress — Branch 2 shipped

TASK 15 — Stock scene background images

Status: ✅ Shipped — all 5 scenes seeded

Design decisions

TASK 16 — Kill infinity display (no-game backdrop fallback)

Status: ✅ Shipped

TASK 17 — Web source rendering (WebView2)

Status: ☐ Not started — required for v1

Design decisions

TASK 18 — Local recording

Status: ☐ Not started — required for v1

Design decisions

v1 execution order

TASK 19 — Scene transitions → MERGED into TASK 19/23 (Control Surface UX)

Design decisions

TASK 20 — Hotkeys (keyboard shortcuts)

Status: ☐ Not started — required for v1

Design decisions

TASK 21 — Media source (video file playback)

Status: ☐ Not started — required for v1

Design decisions

TASK 22 — Audio sync offset

Status: ☐ Not started — required for v1

Design decisions

TASK 23 — Studio mode → MERGED into TASK 19/23 (Control Surface UX)

Backlog (future versions)

101 KiB

Raw Blame History

Status: ✅ SHIPPED 2026-08-14 — 196 tests passing, 0 warnings (ONE integration test: `AudioPipelineTests.Mix_WithFiltersDuckAndGain_Lands_On_AudioPipe`)