From cc3b1c92349b40f5d1c0f044197a74e070ef3652 Mon Sep 17 00:00:00 2001 From: gramps Date: Mon, 10 Aug 2026 11:18:37 -0700 Subject: [PATCH] TASKS.md reformat: task-list statuses under every task + all lists numbered Fix the 4659-char '### Status:' mega-heading that rendered TASK 3's status in title font through TASK 4. Every task now opens with a TASK-1-style '### Status:' block using task-list markers (green check = done, empty box = pending, red check = exception). Converted every remaining bullet list to numbered lists; added a marker legend at the top; corrected TASK 4 requirement 2's stale binary facts (static -> lgpl-shared, extract exe + the libav*.dll family). --- TASKS.md | 280 ++++++++++++++++++++++++++++++++----------------------- 1 file changed, 164 insertions(+), 116 deletions(-) diff --git a/TASKS.md b/TASKS.md index b99aa98..e716ca0 100644 --- a/TASKS.md +++ b/TASKS.md @@ -3,15 +3,20 @@ > Task queue and authoritative research. Memory-map conventions: [`schema.md`](schema.md); > architecture/decisions: [`ai.md`](ai.md). Update statuses here whenever a task moves. +> **Checklist markers** — every task's Status list uses the same states: +> 1. ✅ — completed (green check) +> 2. ☐ — not completed / pending (empty box) +> 3. ❌ — exception (blocked, known-issue, or deliberately excluded from this build) + ## YouTube Live API — research facts (authoritative, v3 build) Lifecycle: `created → ready → [testing] → live → complete` (transitional `liveStarting` / `testStarting`). -- **liveBroadcasts.insert** requires: `snippet.title`, `snippet.scheduledStartTime`, `status.privacyStatus`, `status.selfDeclaredMadeForKids` (COPPA). -- **liveStreams.insert** requires: `snippet.title`, `cdn.frameRate`, `cdn.ingestionType`, `cdn.resolution`. **None of the four (except title) can ever change after creation** — changing them means delete + recreate the stream. This is the hard constraint behind the quality grey-out. -- **Title / description / privacy**: editable at any time, including while live (`liveBroadcasts.update`, part=`snippet,status`). -- **contentDetails** (DVR, recordFromStart, monitorStream, embed, latency): editable only in `created` / `ready`. -- **Transition to live** only allowed when the bound stream's `status.streamStatus == active`. +1. **liveBroadcasts.insert** requires: `snippet.title`, `snippet.scheduledStartTime`, `status.privacyStatus`, `status.selfDeclaredMadeForKids` (COPPA). +2. **liveStreams.insert** requires: `snippet.title`, `cdn.frameRate`, `cdn.ingestionType`, `cdn.resolution`. **None of the four (except title) can ever change after creation** — changing them means delete + recreate the stream. This is the hard constraint behind the quality grey-out. +3. **Title / description / privacy**: editable at any time, including while live (`liveBroadcasts.update`, part=`snippet,status`). +4. **contentDetails** (DVR, recordFromStart, monitorStream, embed, latency): editable only in `created` / `ready`. +5. **Transition to live** only allowed when the bound stream's `status.streamStatus == active`. ### Two features that reshape the design @@ -20,17 +25,17 @@ Lifecycle: `created → ready → [testing] → live → complete` (transitional ### Compliance gotchas (maps perfectly to report-by-exception) -- `liveStreams.status.healthStatus`: `good | ok | bad | noData` plus `configurationIssues[]` with `type` + `severity` (`info|warning|error`). Literally built for report-by-exception — poll it, render nothing on good/ok, surface a banner only on warning/error. No need to invent our own health logic. -- Encoder must comply or YouTube flags it: keyframes ≤ 4s (`gopSizeLong`), closed GOP, H.264, audio AAC/MP3 @ 44.1/48kHz, mono/stereo only. -- Error codes to handle: `errorStreamInactive`, `invalidTransition`, `redundantTransition`, `liveStreamDeletionNotAllowed`, `liveStreamModificationNotAllowed`, `liveBroadcastBindingNotAllowed`. +1. `liveStreams.status.healthStatus`: `good | ok | bad | noData` plus `configurationIssues[]` with `type` + `severity` (`info|warning|error`). Literally built for report-by-exception — poll it, render nothing on good/ok, surface a banner only on warning/error. No need to invent our own health logic. +2. Encoder must comply or YouTube flags it: keyframes ≤ 4s (`gopSizeLong`), closed GOP, H.264, audio AAC/MP3 @ 44.1/48kHz, mono/stereo only. +3. Error codes to handle: `errorStreamInactive`, `invalidTransition`, `redundantTransition`, `liveStreamDeletionNotAllowed`, `liveStreamModificationNotAllowed`, `liveBroadcastBindingNotAllowed`. ### Tips we should take advantage of -- **Reusable streams** (`isReusable=true`): one stream per channel, cache its ingestion URL + stream name, reuse for every broadcast. No rebinding dance each go-live. This is exactly the manual-stream-key baseline. -- **Backup ingestion address**: YouTube provides a simultaneous-push backup — future hardening, not v1. -- **`recordFromStart` + `enableDvr` default true** → every live is auto-recorded and immediately replayable. Free VOD archive, matches the v0.2 recording goal. -- **`latencyPreference`: `normal | low | ultraLow`** — for homelab streamers talking to chat, `low` (or `ultraLow`, capped at 1080p) is a real feature. -- **Broadcast ID == Video ID** — one ID to track everything. +1. **Reusable streams** (`isReusable=true`): one stream per channel, cache its ingestion URL + stream name, reuse for every broadcast. No rebinding dance each go-live. This is exactly the manual-stream-key baseline. +2. **Backup ingestion address**: YouTube provides a simultaneous-push backup — future hardening, not v1. +3. **`recordFromStart` + `enableDvr` default true** → every live is auto-recorded and immediately replayable. Free VOD archive, matches the v0.2 recording goal. +4. **`latencyPreference`: `normal | low | ultraLow`** — for homelab streamers talking to chat, `low` (or `ultraLow`, capped at 1080p) is a real feature. +5. **Broadcast ID == Video ID** — one ID to track everything. --- @@ -39,11 +44,12 @@ Lifecycle: `created → ready → [testing] → live → complete` (transitional **Goal:** Working C# / WPF project with MVVM architecture, dark-theme main window, and YouTube service stubs. ### Status: ✅ Done -- Models: Scene, Source, StreamConfig, StreamHealth, YouTubeChannel, ChatMessage -- Services: YouTubeAuthService (OAuth2), YouTubeStreamService (broadcast/health), YouTubeChatService (chat polling) -- MainViewModel: scene management, stream controls, chat -- MainWindow: scene/source panel, preview area, chat panel, status bar -- Clean build, 0 warnings (WSL + Windows) + +1. ✅ Models: Scene, Source, StreamConfig, StreamHealth, YouTubeChannel, ChatMessage +2. ✅ Services: YouTubeAuthService (OAuth2), YouTubeStreamService (broadcast/health), YouTubeChatService (chat polling) +3. ✅ MainViewModel: scene management, stream controls, chat +4. ✅ MainWindow: scene/source panel, preview area, chat panel, status bar +5. ✅ Clean build, 0 warnings (WSL + Windows) --- @@ -51,6 +57,15 @@ Lifecycle: `created → ready → [testing] → live → complete` (transitional **Goal:** Fully working Google OAuth2 flow — user clicks "YouTube", browser opens, authorization callback lands, channel info is stored. +### Status: ✅ Done + +1. ✅ Two-state Start/End Stream button, go-live dialog (account + title/description/visibility), red top bar, pulsing LIVE badge + elapsed timer, preview glow, taskbar red dot +2. ✅ Real OAuth2 wiring — baked-in Google credentials (desktop client; loopback callback) + `YouTubeAuthService` complete: browser launch, `HttpListener` callback, token exchange, refresh, channel fetch +3. ✅ Token persistence via Windows DPAPI (`Helpers/TokenStore.cs` → `%APPDATA%\ytLlive\ytLlive.auth`), best-effort reload + proactive refresh at startup, saved after every exchange/refresh +4. ✅ Account sign-in/change surfaced in the GoLive dialog (saved account shown with "Change Account"; "Sign in to YouTube" when none; Start disabled until signed in) +5. ✅ **End Livestream signs out** — a graceful end completes the session: `StopStream()` calls `YouTubeAuthService.ClearSession()` + `TokenStore.Clear()` + `IsConnected = false`, so the next Start Stream dialog requires a fresh sign-in. A crash never runs End, so the DPAPI token survives and the creator stays signed in. Resume/reconnect after a midstream crash is deliberately deferred to TASK 3: the socket can't be resumed (it dies with the process), so "resume" = fast reconnect with a saved broadcast ID/stream key within YouTube's disconnect-grace window; too slow and `enableAutoStop` ends the broadcast +6. ✅ Tests in `ytLive.Tests` (xUnit, net8.0-windows): TokenStore DPAPI roundtrip/corrupt/missing/clear + mocked exchange channel-parse + refresh expiry bump + `ClearSession` — 7 passing + **Design constraint:** Sign-in must NEVER block core exploration. Users can build scenes, add sources, and audition the software without authenticating. But **going live requires authentication** — the "Start Stream" dialog is where the account sign-in lives, alongside all stream metadata. **Two-state flow:** There is no separate "Connect" button. The top bar shows a single button — **Start Stream** when idle, **End Stream** when live. Clicking Start Stream opens one dialog that supplies everything: account (previously-saved account shown as default, with a Change Account action) + title/description/visibility. @@ -68,17 +83,9 @@ Lifecycle: `created → ready → [testing] → live → complete` (transitional ### Tests: -- Mock token exchange response, verify channel info parsed -- Verify token refresh triggers when near expiry -- Verify credential load/save roundtrip - -### Status: ✅ Complete -- ✅ Two-state Start/End Stream button, go-live dialog (account + title/description/visibility), red top bar, pulsing LIVE badge + elapsed timer, preview glow, taskbar red dot -- ✅ Real OAuth2 wiring — baked-in Google credentials (desktop client; loopback callback) + `YouTubeAuthService` complete: browser launch, `HttpListener` callback, token exchange, refresh, channel fetch -- ✅ Token persistence via Windows DPAPI (`Helpers/TokenStore.cs` → `%APPDATA%\ytLlive\ytLlive.auth`), best-effort reload + proactive refresh at startup, saved after every exchange/refresh -- ✅ Account sign-in/change surfaced in the GoLive dialog (saved account shown with "Change Account"; "Sign in to YouTube" when none; Start disabled until signed in) -- ✅ **End Livestream signs out** — a graceful end completes the session: `StopStream()` calls `YouTubeAuthService.ClearSession()` + `TokenStore.Clear()` + `IsConnected = false`, so the next Start Stream dialog requires a fresh sign-in. A crash never runs End, so the DPAPI token survives and the creator stays signed in. Resume/reconnect after a midstream crash is deliberately deferred to TASK 3: the socket can't be resumed (it dies with the process), so "resume" = fast reconnect with a saved broadcast ID/stream key within YouTube's disconnect-grace window; too slow and `enableAutoStop` ends the broadcast -- ✅ Tests in `ytLive.Tests` (xUnit, net8.0-windows): TokenStore DPAPI roundtrip/corrupt/missing/clear + mocked exchange channel-parse + refresh expiry bump + `ClearSession` — 7 passing +1. Mock token exchange response, verify channel info parsed +2. Verify token refresh triggers when near expiry +3. Verify credential load/save roundtrip --- @@ -86,6 +93,27 @@ Lifecycle: `created → ready → [testing] → live → complete` (transitional **Goal:** Real video preview in the center panel — the minimal source set below, composited per scene. +### Status: 🔶 In progress + +1. ✅ **Milestone 1 — webcam** — MediaCapture (WinRT SDK projection) with device enumeration, CPU-first frame source, refcounted `CameraManager`, picker dialog, clip shapes (Traditional + Round) + mirror, 480×270 default placement — schema v2 +2. ✅ **Schema v3 (Ship Branch A)** — multi-scene webcam (singleton `Webcam` + per-scene `WebcamSceneConfig`), right-click border/context menu, static OBS-style borders, 50%-per-dimension webcam size cap, device-swap (`ReleaseAllAsync`) — 25 tests passing +3. ✅ **Schema v4** — round→rect restore persisted (`WebcamSceneConfig.RectWidth`/`RectHeight`) + one-time legacy-square 16:9 heal on load +4. ✅ **Screen backdrop (ship task #1, schema v5)** — live desktop/game capture as a permanent, non-deletable bottom layer (`Source.IsBackdrop`), auto-detecting the full-screen game at launch/focus (else the **primary display** — never assumed monitor 0) via `Win32FullScreenDetector` (now with `GetDisplays()`/`PrimaryMonitorIndex()` for the in-app display picker), content re-designated via the OS `GraphicsCapturePicker` ("Change Capture…") or the in-app "Capture Display" submenu, refcounted/shared capture sessions in `ScreenCaptureManager` mirroring `CameraManager` — 45 tests passing +5. ✅ **Schema v6 — backdrop Live-only by policy** — `Scene.HasBackdrop`, enforced by scene name on every load (`EnforceBackdropPolicy`: Starting/BRB/Chat/Ending never carry one; the one-time v5→v6 backfill covers all four), the scene context-menu "Backdrop" checkbox is gone (policy owns the flag), preview watermark hides when the backdrop renders, capture changed to `WindowsRuntimeMarshal.TryGetDataUnsafe` (the CsWinRT-safe frame-read) + downscale to the 1920×1080 master + 5s-throttled error logging (was flooding `startup.log` with 5 MB of cast errors and burning CPU), round webcam no longer re-rasterizes an `ImageBrush` every frame (Image + EllipseGeometry clip) — the live-mode stutter fix +6. ✅ **The five-scene catalog (`SceneCatalog`)** — Starting/Live/BRB/Chat/Ending is the product — work with less, never more; the (+) button only shows when a canonical scene is missing and re-adds it (its menu lists only the missing ones) — 62 tests passing +7. ✅ **Webcam-after-session-start fix** — a webcam added to a scene after the camera was already running (e.g. Chat) previously rendered a transparent container — `CameraManager.GetPreviewBitmap` + propagation in `AddWebcamToActiveSceneAsync`/`ReacquireWebcam` now hands the running shared frames to any newly added `WebcamSceneConfig` — 65 tests passing +8. ✅ **Chat scene webcam size cap** — raised from 50%-per-dimension (960×540) to half the screen AREA (~1358×764 @16:9, `MaxWebcamWidthFor`/`MaxWebcamHeightFor` keyed by canonical name) so the viewer sees the creator better +9. ✅ **"Add Webcam" always opens the camera picker** — deleting one scene's webcam then re-adding used to resurrect the old camera when another scene still used it — `SwapWebcamIdentityAsync` now swaps the app-wide identity if a different camera is chosen, same path as "Change Webcam…" +10. ✅ **Scenes/sources UI** — add/reorder/rename, image + background overlays with move/resize/opacity/reuse +11. ✅ **Audio UX shipped (UI)** — the bottom-bar footer is now two lines (dropped/duration moved under bitrate/fps), with the mic's sound meter + mute button + volume slider grouped CENTERED on the footer's top line, beneath the preview panel (meter: 288px, muted slate track with ruler graduations + muted yellow/red zone tints, green→yellow→red fill; mute = speaker icon → red do-not-symbol when muted, and the slider and speaker stay in sync (volume 0 ⇔ muted — sliding off flips the speaker to muted, sliding up from 0 clears it); mic volume defaults to 80%, muting zeroes the meter and restores the prior volume on unmute (which flashes the meter to the restored position ~300ms before it returns to the live level); the meter is a READ-ONLY realtime level display (fill = live level × volume — volume is a gain on ambient noise; while the slider is dragged the bar previews the slider position and bounces back to the live level on release, which is 0 with no input — clicking the meter does nothing), clicking the MIC label opens a microphone picker whose chosen source shows left-justified inside the meter bar (fill at 75% opacity so the name + ruler markings show through); slim dimensional slider — gradient track/fill, gloss-sphere thumb; the old flat pink 18px-filled one is gone), everything else on line 2 (bitrate/fps/dropped/duration/health left, quality + gear right) — the creator's only audio control, desktop/game audio is automatic (KISS rule) +12. ✅ The connected YouTube account's avatar/name shows in the top bar next to Start Stream (`SyncConnectedAccount`); the scenes list is content-height now (no dead space before SOURCES) +13. ☐ **Window capture** — absorbed into the Screen picker (no separate source type); dedicated window-as-source work is pending +14. ☐ **Scene compositing** — the D3DImage/MediaElement preview compositor (this task's requirement 5; the output compositor ships as TASK 4 ship step 1) +15. ☐ **Text source** — live text ("Starting soon", "Back in 5", handle, callout) +16. ☐ **Chat box** — YouTube live chat rendered *on* the stream so viewers read along in-video +17. ❌ **Background removal (milestone 2)** — ONNX Runtime + DirectML, MediaPipe Selfie Segmentation — deliberately NOT in this build +18. ☐ **Alerts** — Super Chat / membership / subscribe pop-ins; build after the six; **the one paid feature** (see Monetization in `ai.md`) + ### The Minimal Source Set (design decision — do not expand casually) ytLlive is YouTube-only and 90% of users are casual. OBS's long source list is off-putting; we ship @@ -107,17 +135,17 @@ Deliberately NOT supported: game capture, browser source, media playlist, VLC, c ### Source memory model (design decision) -- A scene has resources. Resources can be shared across scenes. -- A resource exists exactly once in memory, no matter how many scenes use it (a logo in five - scenes = one loaded bitmap). -- Every resource carries a **catalog of scenes**: one usage entry per scene it appears in, each - entry dictating that scene's use — placement (X/Y/Width/Height), opacity, z-order, enabled, - scale mode, crop. -- Usages are named `{resourceName}.{sceneName}` — whatever the user named the resource, dot, the - scene name: `logo.starting`, `logo.live`, `myPic.brb`. Not a hardcoded "logo". -- A webcam in two scenes = one capture session, two catalog entries. -- Refcount by catalog size: the last usage removed → the resource is disposed and evicted. -- The resource (not a per-scene node) owns everything `IDisposable`. +1. A scene has resources. Resources can be shared across scenes. +2. A resource exists exactly once in memory, no matter how many scenes use it (a logo in five + scenes = one loaded bitmap). +3. Every resource carries a **catalog of scenes**: one usage entry per scene it appears in, each + entry dictating that scene's use — placement (X/Y/Width/Height), opacity, z-order, enabled, + scale mode, crop. +4. Usages are named `{resourceName}.{sceneName}` — whatever the user named the resource, dot, the + scene name: `logo.starting`, `logo.live`, `myPic.brb`. Not a hardcoded "logo". +5. A webcam in two scenes = one capture session, two catalog entries. +6. Refcount by catalog size: the last usage removed → the resource is disposed and evicted. +7. The resource (not a per-scene node) owns everything `IDisposable`. ### Scene transitions (design decision) @@ -137,14 +165,14 @@ Preview shows the transition too (WYSIWYG). No wipes/slides/LUTs beyond the four 1. **Screen** — Windows.Graphics.Capture (WinRT), enumerate displays/windows, picker 2. **Webcam** — MediaCapture (WinRT SDK projection) with device enumeration — ✅ **milestone 1 done**: - - TFM bumped to `net8.0-windows10.0.19041.0` (app **and** tests) so the WinRT projection resolves from the SDK reference packs — no NuGet package, no capability manifest (unpackaged desktop app) - - `MediaCaptureFrameSource` (CPU-first: `MemoryPreference = Cpu`, BGRA8 via `CreateFrameReaderAsync`), `MediaCaptureCameraEnumerator` (`DeviceInformation.FindAllAsync(DeviceClass.VideoCapture)`) - - `CameraManager`: refcounted by `DeviceId`, one shared `WriteableBitmap` app-wide, dispatcher-coalesced UI updates (~render rate, latest-frame drop), placeholder/`AppLog` + warning on failure - - `CameraPickerDialog` (mirror of `ReuseImageDialog`) — "Searching for cameras…" / list / "No cameras found" states - - One webcam app-wide: Add → Webcam greyed out once one exists ("it's already in your stream" tooltip); persisted `DeviceId` re-acquires after layout load - - Default placement 16:9 **480×270**, bottom-right, 32px margin; drag/resize/selection shared with Image sources - - **Clip shapes: Traditional + Round** (phone view dropped — the 9:16 phone output is the vertical output-crop tier); **mirror**; both persisted in the layout DB (schema v2) and toggled from the source chip - - **Background removal = milestone 2** (ONNX Runtime + DirectML, MediaPipe Selfie Segmentation) — not in this build + 1. TFM bumped to `net8.0-windows10.0.19041.0` (app **and** tests) so the WinRT projection resolves from the SDK reference packs — no NuGet package, no capability manifest (unpackaged desktop app) + 2. `MediaCaptureFrameSource` (CPU-first: `MemoryPreference = Cpu`, BGRA8 via `CreateFrameReaderAsync`), `MediaCaptureCameraEnumerator` (`DeviceInformation.FindAllAsync(DeviceClass.VideoCapture)`) + 3. `CameraManager`: refcounted by `DeviceId`, one shared `WriteableBitmap` app-wide, dispatcher-coalesced UI updates (~render rate, latest-frame drop), placeholder/`AppLog` + warning on failure + 4. `CameraPickerDialog` (mirror of `ReuseImageDialog`) — "Searching for cameras…" / list / "No cameras found" states + 5. One webcam app-wide: Add → Webcam greyed out once one exists ("it's already in your stream" tooltip); persisted `DeviceId` re-acquires after layout load + 6. Default placement 16:9 **480×270**, bottom-right, 32px margin; drag/resize/selection shared with Image sources + 7. **Clip shapes: Traditional + Round** (phone view dropped — the 9:16 phone output is the vertical output-crop tier); **mirror**; both persisted in the layout DB (schema v2) and toggled from the source chip + 8. **Background removal = milestone 2** (ONNX Runtime + DirectML, MediaPipe Selfie Segmentation) — not in this build 3. **Background / Image / Text** — static sources positioned/scaled/opacity 4. **Chat box** — rendered from the live chat poll (right panel is the same feed, raw) 5. **Scene compositing** — per-scene source layering (z-order = sources list order, top-to-bottom @@ -155,11 +183,9 @@ Preview shows the transition too (WYSIWYG). No wipes/slides/LUTs beyond the four `BrandFlashActive`/`BrandFlashTimer` in `MainViewModel`); the encoder output renders the same layer, and v0.2 local recordings carry it too 7. **Drag/drop placement & reorder** — intuitive, visual (per design principle): - - **Preview:** click-drag a source in the center panel to reposition it; resize via handles - - **Scenes list:** drag rows to reorder scenes - - **Sources list:** drag rows to reorder sources (this *is* the z-order) — implemented - -### Status: 🔶 In progress — milestone 1 (webcam) shipped; **schema v3 (Ship Branch A) shipped**: multi-scene webcam (singleton `Webcam` + per-scene `WebcamSceneConfig`), right-click border/context menu, static OSB-style borders, 50%-per-dimension webcam size cap, device-swap (`ReleaseAllAsync`); **schema v4**: round→rect restore persisted (`WebcamSceneConfig.RectWidth`/`RectHeight`) + one-time legacy-square 16:9 heal on load — 25 tests passing; **screen backdrop (ship task #1) shipped**: live desktop/game capture as a permanent, non-deletable bottom layer (`Source.IsBackdrop`, schema v5), auto-detecting the full-screen game at launch/focus (else the **primary display** — never assumed monitor 0) via `Win32FullScreenDetector` (now with `GetDisplays()`/`PrimaryMonitorIndex()` for the in-app display picker), content re-designated via the OS `GraphicsCapturePicker` ("Change Capture…") or the in-app "Capture Display" submenu, refcounted/shared capture sessions in `ScreenCaptureManager` mirroring `CameraManager` — 45 tests passing; **schema v6**: `Scene.HasBackdrop`, now **Live-only by policy** — the backdrop is enforced by scene name on every load (`EnforceBackdropPolicy`: Starting/BRB/Chat/Ending never carry one; the one-time v5→v6 backfill covers all four), the scene context-menu "Backdrop" checkbox is gone (policy owns the flag), preview watermark hides when the backdrop renders, capture changed to `WindowsRuntimeMarshal.TryGetDataUnsafe` (the CsWinRT-safe frame-read) + downscale to the 1920×1080 master + 5s-throttled error logging (was flooding `startup.log` with 5 MB of cast errors and burning CPU), round webcam no longer re-rasterizes an `ImageBrush` every frame (Image + EllipseGeometry clip) — the live-mode stutter fix; **the five-scene catalog (`SceneCatalog`)**: Starting/Live/BRB/Chat/Ending is the product — work with less, never more; the (+) button only shows when a canonical scene is missing and re-adds it (its menu lists only the missing ones); **webcam-after-session-start fix**: a webcam added to a scene after the camera was already running (e.g. Chat) previously rendered a transparent container — `CameraManager.GetPreviewBitmap` + propagation in `AddWebcamToActiveSceneAsync`/`ReacquireWebcam` now hands the running shared frames to any newly added `WebcamSceneConfig` — 62 tests passing; the **Chat scene's webcam size cap** is raised from 50%-per-dimension (960×540) to half the screen AREA (~1358×764 @16:9, `MaxWebcamWidthFor`/`MaxWebcamHeightFor` keyed by canonical name) so the viewer sees the creator better — 65 tests passing; **"Add Webcam" always opens the camera picker** (deleting one scene's webcam then re-adding used to resurrect the old camera when another scene still used it — `SwapWebcamIdentityAsync` now swaps the app-wide identity if a different camera is chosen, same path as "Change Webcam…"); scenes/sources UI (add/reorder/rename, image + background overlays with move/resize/opacity/reuse) built; **audio UX shipped (UI)**: the bottom-bar footer is now two lines (dropped/duration moved under bitrate/fps), with the mic's sound meter + mute button + volume slider grouped CENTERED on the footer's top line, beneath the preview panel (meter: 288px, muted slate track with ruler graduations + muted yellow/red zone tints, green→yellow→red fill; mute = speaker icon → red do-not-symbol when muted, and the slider and speaker stay in sync (volume 0 ⇔ muted — sliding off flips the speaker to muted, sliding up from 0 clears it); mic volume defaults to 80%, muting zeroes the meter and restores the prior volume on unmute (which flashes the meter to the restored position ~300ms before it returns to the live level); the meter is a READ-ONLY realtime level display (fill = live level × volume — volume is a gain on ambient noise; while the slider is dragged the bar previews the slider position and bounces back to the live level on release, which is 0 with no input — clicking the meter does nothing), clicking the MIC label opens a microphone picker whose chosen source shows left-justified inside the meter bar (fill at 75% opacity so the name + ruler markings show through); slim dimensional slider — gradient track/fill, gloss-sphere thumb; the old flat pink 18px-filled one is gone), everything else on line 2 (bitrate/fps/dropped/duration/health left, quality + gear right) — the creator's only audio control, desktop/game audio is automatic (KISS rule); the connected YouTube account's avatar/name shows in the top bar next to Start Stream (`SyncConnectedAccount`); the scenes list is content-height now (no dead space before SOURCES); window capture, compositing, encoding pending + 1. **Preview:** click-drag a source in the center panel to reposition it; resize via handles + 2. **Scenes list:** drag rows to reorder scenes + 3. **Sources list:** drag rows to reorder sources (this *is* the z-order) — implemented --- @@ -167,17 +193,33 @@ Preview shows the transition too (WYSIWYG). No wipes/slides/LUTs beyond the four **Goal:** Push encoded video to YouTube's RTMP ingest. +### Status: 🔶 In progress + +1. ✅ **Ship step 1 — the output compositor SHIPPED** (2026-08-10) +2. ✅ **Ship step 2 — the FFmpeg locator SHIPPED** (2026-08-10) +3. ☐ **Encoder + RTMP push** — the FFmpeg subprocess: frames via stdin, stderr health parsing, FLV mux + push to the cached reusable stream's ingestion URL +4. ☐ **WASAPI audio capture** — loopback (desktop/game) + the picked mic feeding `AudioLevel` so the realtime meter comes alive (req 7) +5. ☐ **Frame-pipeline wiring** — `CameraManager`/`ScreenCaptureManager` → compositor resolver → encoder +6. ☐ **Health stats** — bitrate, FPS, dropped frames reported live in the bottom bar (req 5) +7. ☐ **One-click go live + private-only enforcement** — Go Live always creates/updates the broadcast with `privacyStatus = "private"` + PRIVATE badge (req 8, test-verifiable) + +The pipeline chain the encoder needs doesn't exist yet: **scene compositing** (the master 1920×1080 frame +without the preview's editing chrome) → **audio capture** (WASAPI, feeds the meter) → **H.264+AAC encode** +→ **vertical-tier crop/scale** → **RTMP push** → **health stats** into the bottom bar. Nothing can encode +until a frame source exists, so the compositor is ship step 1. The pipeline is +`CameraManager + ScreenCaptureManager → compositor resolver → compositor → encoder → RTMP`. + ### Requirements: 1. **Encoding** — H.264 (hardware via NVENC/AMD, fallback x264) + AAC audio; **must comply**: keyframes ≤ 4s (gopSizeLong), closed GOP, AAC/MP3 @ 44.1/48kHz, mono/stereo only. **License posture (decided): GPL-free build** — NVENC (NVIDIA) / QSV (Intel) / AMF (AMD) + OpenH264 software fallback + built-in AAC; no libx264 (GPL contaminates a paid product). Output containers are identical either way (H.264+AAC in `.flv` for RTMP, `.mp4`/`.ts` for VOD) — the format is NOT the differentiator, the license and per-GPU quality are. **License guardrails (never violate — see `ai.md` → "Licensing — do not violate"):** only BtbN `lgpl`/`lgpl-shared` builds; never GPL (gyan.dev) or `nonfree` (fdk-aac); never static for distribution (LGPL §6 relink material); never link FFmpeg into the app; never drop `THIRD-PARTY-NOTICES.txt` from the app/About screen. -2. **RTMP push** — **FFmpeg subprocess (decided)**: app feeds raw frames via stdin, parses stderr for health; one battle-tested binary does encode + FLV mux + push + reconnect. **Binary distribution (decided): check-then-pull** — probe `where ffmpeg`/PATH at first go-live; if absent, download a **pinned** build (**BtbN LGPL win64 static** zip, ~75 MB — gyan.dev's builds are GPLv3 and ship libx264, which violates the license posture; BtbN's LGPL variant drops x264/x265 while keeping NVENC/QSV/AMF + libopenh264 + native AAC) to `%APPDATA%\ytLlive\tools\ffmpeg.exe` (extract just `ffmpeg.exe` from the zip) and cache it, offline-friendly. Behind an `IFfmpegLocator` seam so tests fake it (ship step 2, below). Push goes to the cached reusable stream's ingestion URL +2. **RTMP push** — **FFmpeg subprocess (decided)**: app feeds raw frames via stdin, parses stderr for health; one battle-tested binary does encode + FLV mux + push + reconnect. **Binary distribution (decided): check-then-pull** — probe `where ffmpeg`/PATH at first go-live; if absent, download a **pinned** build (**BtbN LGPL-shared win64** zip, ~75 MB — gyan.dev's builds are GPLv3 and ship libx264, which violates the license posture; BtbN's LGPL variant drops x264/x265 while keeping NVENC/QSV/AMF + libopenh264 + native AAC) to `%APPDATA%\ytLlive\tools\ffmpeg.exe` (extract `ffmpeg.exe` **plus the `libav*.dll` family**) and cache it, offline-friendly. Behind an `IFfmpegLocator` seam so tests fake it (ship step 2, below). Push goes to the cached reusable stream's ingestion URL 3. **Quality ladder** — the offered tiers, with **1080p60 @ 8 Mbps as the standard/default**: - - 720p30 @ 6 Mbps - - 720p60 @ 6 Mbps - - 1080p30 @ 8 Mbps - - **1080p60 @ 8 Mbps** (default — mainstream ceiling, GPU hardware-encoded so the gaming - machine never notices; upload headroom stays comfortable) - - Vertical 1080×1920 @ 60fps @ 8 Mbps (9:16 phone tier) + 1. 720p30 @ 6 Mbps + 2. 720p60 @ 6 Mbps + 3. 1080p30 @ 8 Mbps + 4. **1080p60 @ 8 Mbps** (default — mainstream ceiling, GPU hardware-encoded so the gaming + machine never notices; upload headroom stays comfortable) + 5. Vertical 1080×1920 @ 60fps @ 8 Mbps (9:16 phone tier) The composition master is always 1920×1080; a tier is an output rect + target resolution (see `ai.md` "Resolution tiers"). Vertical output = the centered 607×1080 crop of the master scaled to 1080×1920 (semi-crop preview is already implemented; the encoder applies the same rect). @@ -195,13 +237,6 @@ Preview shows the transition too (WYSIWYG). No wipes/slides/LUTs beyond the four 8. **Private-only go live until v1 (reputation guard, decided 2026-08-10)** — until the v1 release, go-live is **locked to private streams only** so a software error can never publish something public/unlisted that damages the creator's reputation. RTMP push itself has no privacy — privacy lives on the YouTube **live broadcast object**, which this app already controls via its OAuth API calls. So the lock is purely API-side: the Go Live flow always creates/updates the broadcast with `privacyStatus = "private"` and a guard **refuses** to set anything else (same spirit as the Live-only backdrop policy). The UI shows a clear "PRIVATE" badge next to the stream state so the creator always knows who can see them. Enforcement must be verifiable in the auth-service tests (fake the broadcast-insert/update call, assert `privacyStatus` is forced to private). 9. **v1 release gate: bundle the full license texts (decided 2026-08-10)** — `THIRD-PARTY-NOTICES.txt` currently links the canonical license texts rather than embedding them. At the **v1 (GA) release**, the full texts of every license it names (LGPL v2.1+, BSD-2-Clause, MIT, Apache-2.0) MUST be bundled alongside it (shipped in the app output, e.g. a `licenses/` folder next to the notices file, still reachable from the About screen). This is a **release blocker for v1, not a task to queue early** — do it in the release pass. The repo should treat this like the private-only go-live gate: a checkbox that cannot silently lapse. -### Status: 🔶 In progress — **ship step 1 (the output compositor) SHIPPED** (2026-08-10); **ship step 2 (the FFmpeg locator) SHIPPED** (2026-08-10); encoder/RTMP/audio follow it - -The pipeline chain the encoder needs doesn't exist yet: **scene compositing** (the master 1920×1080 frame -without the preview's editing chrome) → **audio capture** (WASAPI, feeds the meter) → **H.264+AAC encode** -→ **vertical-tier crop/scale** → **RTMP push** → **health stats** into the bottom bar. Nothing can encode -until a frame source exists, so the compositor is ship step 1. - #### Ship step 1 — Scene compositor (the frame source) **Goal:** a pure-CPU software compositor producing the encoder's master frame (BGRA8, the `VideoFrame` @@ -225,26 +260,26 @@ vertical (9:16) = composite the centered 607×1080 crop then bilinear-upscale to 2. Background — the scene's `Background` Source, `UniformToFill` full-frame (the `ActiveBackgroundImage` layer, not per-element). 3. Elements in `Scene.Elements` order (back→front), skip `IsVisible=false`. What actually renders: - - `Source` Type `Image` → static asset, `UniformToFill` cover-crop into (X, Y, W, H) - - `WebcamSceneConfig` → latest frame by `DeviceId`: Traditional = `UniformToFill` rect; Round = circle - diameter `min(W,H)` (alpha 0 outside — true circle, not oval); mirror = horizontal flip around - element center (`MirrorScale`); opacity = per-pixel multiply (content + border); border = stroked - rect / centered circle at `RoundBorderSize`, width `BorderWidth`, alpha `BorderOpacity` - - `Background` / `IsBackdrop` / `TextOverlay` are NOT per-element (layers above; Text not shipped) + 1. `Source` Type `Image` → static asset, `UniformToFill` cover-crop into (X, Y, W, H) + 2. `WebcamSceneConfig` → latest frame by `DeviceId`: Traditional = `UniformToFill` rect; Round = circle + diameter `min(W,H)` (alpha 0 outside — true circle, not oval); mirror = horizontal flip around + element center (`MirrorScale`); opacity = per-pixel multiply (content + border); border = stroked + rect / centered circle at `RoundBorderSize`, width `BorderWidth`, alpha `BorderOpacity` + 3. `Background` / `IsBackdrop` / `TextOverlay` are NOT per-element (layers above; Text not shipped) 4. Branding flash — pre-rendered full-frame "made with ytLlive!" at 25% alpha when live + `BrandFlashEnabled` + timer active. Passed in as a `VideoFrame?` (compositor core stays pure byte-math, no WPF; likely a bundled asset rather than runtime text rendering). 5. NOT in output (preview chrome only): SelectionOverlay, DimRects, output-rect outline, badge, placeholder. **New files (all in `Services/Compositor/`):** -- `SceneCompositor.cs` — `Render(Scene, frameFor: Func, flashFrame: - VideoFrame?, CompositorOptions) → VideoFrame` (output-sized). The caller's `frameFor` resolver maps - each element to its frame (webcam → DeviceId, image → AssetId via `StaticPixelCache`, backdrop → - CaptureKey) — the compositor stays pure/hermetic/no WPF. -- `CompositorOptions.cs` — source-rect + output W×H. -- `StretchMath.cs` — `UniformToFill` cover-crop, ellipse mask, bilinear scale (pure, unit-tested). -- `StaticPixelCache.cs` — asset `byte[]` → cached BGRA `VideoFrame` (WPF `BitmapDecoder` + `CopyPixels`, - decode once per content hash). +1. `SceneCompositor.cs` — `Render(Scene, frameFor: Func, flashFrame: + VideoFrame?, CompositorOptions) → VideoFrame` (output-sized). The caller's `frameFor` resolver maps + each element to its frame (webcam → DeviceId, image → AssetId via `StaticPixelCache`, backdrop → + CaptureKey) — the compositor stays pure/hermetic/no WPF. +2. `CompositorOptions.cs` — source-rect + output W×H. +3. `StretchMath.cs` — `UniformToFill` cover-crop, ellipse mask, bilinear scale (pure, unit-tested). +4. `StaticPixelCache.cs` — asset `byte[]` → cached BGRA `VideoFrame` (WPF `BitmapDecoder` + `CopyPixels`, + decode once per content hash). **Test plan (Good Dog Rule — ONE integration test):** `SceneCompositorTests` — a scene with backdrop (solid red fake frame) + round webcam (solid green) + image (solid blue) → render 16:9 master → assert @@ -275,32 +310,32 @@ a binary in the repo. Returns an absolute path; downloads only when neither PATH provides one. **Decisions (locked 2026-08-10):** -- **BtbN LGPL-shared win64 build** — not gyan.dev (gyan's "essentials" is GPLv3 and ships libx264, which - violates requirement 1's license posture) and **not the static lgpl build**: LGPLv2.1 §6 wants - relinkable object files for static linking, but the **shared** (dynamic-DLL) variant sidesteps that — - compliance is "license text + source offer + unmodified binaries" (see `THIRD-PARTY-NOTICES.txt` and - `ai.md` → Licensing). Drops libx264/libx265 while keeping NVENC/QSV/AMF, libopenh264 (the LGPL-legal - H.264 software fallback) and native AAC — exactly the requirement-1 encoder profile. -- **Pinned URL** — `https://github.com/BtbN/FFmpeg-Builds/releases/download/autobuild-2026-08-09-13-03/ffmpeg-master-latest-win64-lgpl-shared.zip` - (~75 MB zip — earlier "~30 MB" estimate corrected). A dated autobuild tag is immutable; BtbN retention - keeps the last 14 daily builds + each month-end build for 2 years, so a cold cache after retention - expiry 404s — a logged, recoverable failure (the seam throws; the encoder step surfaces it). Once - cached, the URL is never touched again. The pin is a single `const`, bumpable in one place — and must - always stay on the **shared** variant (never `gpl`, `nonfree`, or static; see ai.md Licensing). -- **Check-then-pull order** — (1) PATH probe (the user's own install wins), (2) cached - `%APPDATA%\ytLlive\tools\ffmpeg.exe`, (3) download + extract. Extract `ffmpeg.exe` **plus the - `libav*.dll` family** (the shared build's bin/ folder; Windows resolves the DLLs from the exe's own - directory) into a staging dir then move into place — a crash never leaves a corrupt or partial cache. -- **Seam** — `IFfmpegLocator.LocateAsync(CancellationToken)`: search dirs, tools dir, and the downloader - (`Func>`) are constructor-injected with production defaults, so - tests fake the network (feeding a real in-memory zip) and never touch disk outside a temp dir. +1. **BtbN LGPL-shared win64 build** — not gyan.dev (gyan's "essentials" is GPLv3 and ships libx264, which + violates requirement 1's license posture) and **not the static lgpl build**: LGPLv2.1 §6 wants + relinkable object files for static linking, but the **shared** (dynamic-DLL) variant sidesteps that — + compliance is "license text + source offer + unmodified binaries" (see `THIRD-PARTY-NOTICES.txt` and + `ai.md` → Licensing). Drops libx264/libx265 while keeping NVENC/QSV/AMF, libopenh264 (the LGPL-legal + H.264 software fallback) and native AAC — exactly the requirement-1 encoder profile. +2. **Pinned URL** — `https://github.com/BtbN/FFmpeg-Builds/releases/download/autobuild-2026-08-09-13-03/ffmpeg-master-latest-win64-lgpl-shared.zip` + (~75 MB zip — earlier "~30 MB" estimate corrected). A dated autobuild tag is immutable; BtbN retention + keeps the last 14 daily builds + each month-end build for 2 years, so a cold cache after retention + expiry 404s — a logged, recoverable failure (the seam throws; the encoder step surfaces it). Once + cached, the URL is never touched again. The pin is a single `const`, bumpable in one place — and must + always stay on the **shared** variant (never `gpl`, `nonfree`, or static; see ai.md Licensing). +3. **Check-then-pull order** — (1) PATH probe (the user's own install wins), (2) cached + `%APPDATA%\ytLlive\tools\ffmpeg.exe`, (3) download + extract. Extract `ffmpeg.exe` **plus the + `libav*.dll` family** (the shared build's bin/ folder; Windows resolves the DLLs from the exe's own + directory) into a staging dir then move into place — a crash never leaves a corrupt or partial cache. +4. **Seam** — `IFfmpegLocator.LocateAsync(CancellationToken)`: search dirs, tools dir, and the downloader + (`Func>`) are constructor-injected with production defaults, so + tests fake the network (feeding a real in-memory zip) and never touch disk outside a temp dir. **New files (all in `Services/Encoder/`):** -- `IFfmpegLocator.cs` — the seam. -- `FfmpegLocator.cs` — the impl (PATH probe → cache → pull+extract exe + DLLs), failures logged via `AppLog`. -- `THIRD-PARTY-NOTICES.txt` (repo root) — the LGPL/BSD/MIT notices + source offer, copied to the build - output and surfaced via the top-bar **About** button (`MainWindow` code-behind, opens the file in the - OS viewer). +1. `IFfmpegLocator.cs` — the seam. +2. `FfmpegLocator.cs` — the impl (PATH probe → cache → pull+extract exe + DLLs), failures logged via `AppLog`. +3. `THIRD-PARTY-NOTICES.txt` (repo root) — the LGPL/BSD/MIT notices + source offer, copied to the build + output and surfaced via the top-bar **About** button (`MainWindow` code-behind, opens the file in the + OS viewer). **Test plan:** the hermetic integration test drives the full decision ladder against a temp tools dir and a fake downloader returning a real in-memory zip (`.../bin/ffmpeg.exe` entry): PATH hit wins without @@ -330,6 +365,14 @@ cache refresh, empty payload, missing zip entry, downloader failure) — **78 pa **Goal:** Create/bind broadcasts, monitor YouTube-side stream health — the v3 way. +### Status: ⏳ Not started + +1. ☐ Broadcast creation — title/description/privacy/scheduledStartTime via API, with the v3 flags above +2. ☐ Reusable stream — create once, cache + reuse; bind to broadcast +3. ☐ Health monitoring — poll `liveStreams.list` `healthStatus` + `configurationIssues[]`, surface banner only on warning/error +4. ☐ Live chat — poll `liveChat/messages`, render in right panel, support Super Chat + membership badges +5. ☐ Error handling — the YouTube error codes: `errorStreamInactive`, `invalidTransition`, `redundantTransition`, `liveStreamDeletionNotAllowed`, `liveStreamModificationNotAllowed`, `liveBroadcastBindingNotAllowed` + ### Design decisions (v3) 1. **One-click go-live** — `liveBroadcasts.insert` with `enableAutoStart=true`, `enableAutoStop=true`, `enableMonitorStream=false`, `selfDeclaredMadeForKids=false`, `latencyPreference=low`. No `transition(live)` call, no testing stage, no liveStarting polling. Encoder starts → YouTube brings it live by itself. @@ -348,14 +391,21 @@ cache refresh, empty payload, missing zip entry, downloader failure) — **78 pa 4. **Live chat** — poll `liveChat/messages`, render in right panel, support Super Chat + membership badges 5. **Error handling** — the YouTube error codes: `errorStreamInactive`, `invalidTransition`, `redundantTransition`, `liveStreamDeletionNotAllowed`, `liveStreamModificationNotAllowed`, `liveBroadcastBindingNotAllowed` -### Status: Not started - --- ## TASK 6 — Layout Persistence (SQLite) **Goal:** Scenes, sources, and asset bytes survive restarts; assets are always available. +### Status: ✅ Done + +1. ✅ SQLite database (`Microsoft.Data.Sqlite`) at `%APPDATA%\ytLlive\ytLlive.db`; schema versioned via `PRAGMA user_version` (currently **v6**) +2. ✅ Assets live in the DB (BLOB keyed by SHA-256 content hash), never file paths — deleting the original file never breaks a scene +3. ✅ File-model save/open — the active layout file is tracked (default is the AppData DB); **Save Layout As… / Open Layout…** switch the active file; auto-save writes to whatever is active +4. ✅ Auto-save (invisible) — ~1.5s debounce on scene/source add/remove/reorder/rename/hide + any source transform change; flush on window close +5. ✅ Startup — load the active file; seed the five canonical scenes only when the DB is empty; (+) re-adds a missing canonical scene and is hidden once all five are present; adding beyond the five is rejected +6. ✅ Schema v1 → v6 — webcam columns (v2), singleton `Webcam` + per-scene `WebcamSceneConfig` (v3), `RectWidth`/`RectHeight` round-to-rect restore (v4), `Source.IsBackdrop` + `Source.CaptureKey` (v5), `Scene.HasBackdrop` — backdrop **Live-only by policy** (v6, one-time backfill + `EnforceBackdropPolicy` on every load); `WindowHandle` stays in-memory (per-session); save = transactional rewrite; orphaned assets pruned + ### Design decisions 1. **SQLite database** (`Microsoft.Data.Sqlite`) at `%APPDATA%\ytLlive\ytLlive.db`; schema versioned @@ -375,21 +425,19 @@ cache refresh, empty payload, missing zip entry, downloader failure) — **78 pa `Webcam` + per-scene `WebcamSceneConfig`; v4 = `WebcamSceneConfig.RectWidth`/`RectHeight` for the round-to-rect restore; v5 = `Source.IsBackdrop` + `Source.CaptureKey` for the live-capture backdrop; v6 = `Scene.HasBackdrop` — the backdrop is **Live-only by policy** - (one-time backfill turns Starting/BRB/Chat/Ending off and drops their backdrop - sources; `EnforceBackdropPolicy` re-normalizes every load). `WindowHandle` stays in-memory - (per-session). Save = transactional rewrite; orphaned assets pruned. + (one-time backfill turns Starting/BRB/Chat/Ending off and drops their backdrop + sources; `EnforceBackdropPolicy` re-normalizes every load). `WindowHandle` stays in-memory + (per-session). Save = transactional rewrite; orphaned assets pruned. 6. **Startup** — load the active file; seed the five canonical scenes (Starting/Live/BRB/Chat/Ending, `SceneCatalog`) only when the DB is empty. The (+) button re-adds a missing canonical scene and is hidden once all five are present; adding beyond the five is rejected — work with less, never more. -### Status: ✅ Implemented - --- ## Backlog (future versions) -- v0.2 — Recording to local file (recordings carry the branding flash — see TASK 3 / `ai.md` Monetization) -- v0.3 — Stream scheduling -- v0.4 — Multi-destination restreaming -- v0.5 — Stream clipping +1. v0.2 — Recording to local file (recordings carry the branding flash — see TASK 3 / `ai.md` Monetization) +2. v0.3 — Stream scheduling +3. v0.4 — Multi-destination restreaming +4. v0.5 — Stream clipping