Files
LlamaCasty/TASKS/task-03-capture-pipeline.md
gramps d4aa588da0 feat(alerts): TASK 43 — native events & alerts (six unique animations) + chat parity
Replace the external StreamElements feed with NATIVE YouTube events per the creator's
2026-09-23 question.

Chat parity half:
- YouTubeChatService decodes ALL SIX liveChat/messages event types into
  ChatMessage.Kind (superChat/superSticker/newSponsor/membershipGifting/
  giftMembershipReceived/memberMilestoneChat) — the four formerly-empty overlay
  rows are real now — and re-arms its one-shot poll on the server's
  pollingIntervalMillis (streamList connection semantics; clamp 1000-6000ms,
  maxResults=2000). ParsePage internal static seam + ChatPage record for
  deterministic tests; optional HttpClient ctor seam kept; InjectSimulatedMessage
  (TASK 41) preserved.

Alerts half (creator rulings: one celebration zone, six UNIQUE animations, no
menus/polls, sub mention = chat row only, NO viewer count):
- New SourceType.AlertBox 'Stream Alerts' (one per layout, CanAddAlerts gate
  mirroring chat; idle = transparent).
- AlertRenderer: six distinct branded animations (SuperChat slide-up/shine/
  count-up, SuperSticker scale-pop, NewMember drop-in/flash, MemberGift slide-left/
  chip-fan, GiftReceived confetti, MemberMilestone rise/growth-bar), every card
  drawing the 'made with LlamaCasty!' brand line.
- AlertOverlayLayer: true component (Commit-G pattern) — queue (cap 10, drop tail
  never stall), 33ms UI ticker, cache-first RenderFrame + UpdatePreview,
  Advance(double) as the deterministic test clock; ChatEventKind.None rows never
  enqueue.
- Wired: resolver RenderAlertBox, _alertLayer ctor + dispose, LoadLayout previews,
  Add menu item (Controls/LeftPanel.xaml), TestSessionViewModel sims tagged
  (member->NewMember, superchat->SuperChat).

Good Dog ONE integration test: AlertLayerTests (RealApp STA, real WPF raster) —
ParsePage classifies all six kinds + cadence fields; None rows enqueue nothing;
six events play pairwise-distinct moving frames then drain to null.

Gate: clean build 0 warnings; full suite 316/317, the one failure
(LayerReorderPersistenceTests.RealMouseDrag) repros on the clean tree — the
known environmental class (real-mouse-drag no-ops with a game/fullscreen window
focused).

References (OBS/overlay ecosystem):
- streamList semantics: https://developers.google.com/youtube/v3/live/docs/liveChatMessages/streamList
- OBS alert-box pattern (designated celebration zone, idle transparent): creator-chosen model
2026-09-24 08:27:00 -07:00

28 KiB
Raw Permalink Blame History

TASK 3 — Capture Pipeline (Scenes/Sources)

Catalog: TASKS.md — status and requirements live here.

Goal: Real video preview in the center panel — the minimal source set below, composited per scene.

Status: 🔶 In progress — Alerts (item 20) SHIPPED 2026-09-24 via TASK 43; still open: item 17 (Text source), item 20's RewardEvents SQLite persistence half (16 compositing + 19 background removal excluded)

  1. ✅ Milestone 1 — webcam — MediaCapture (WinRT SDK projection) with device enumeration, CPU-first frame source, refcounted CameraManager, picker dialog, clip shapes (Traditional + Round) + mirror, 480×270 default placement — schema v2
  2. ✅ Schema v3 (Ship Branch A) — multi-scene webcam (singleton Webcam + per-scene WebcamSceneConfig), right-click border/context menu, static OBS-style borders, 50%-per-dimension webcam size cap, device-swap (ReleaseAllAsync) — 25 tests passing
  3. ✅ Schema v4 — round→rect restore persisted (WebcamSceneConfig.RectWidth/RectHeight) + one-time legacy-square 16:9 heal on load
  4. ✅ Screen backdrop (ship task #1, schema v5) — live desktop/game capture as a permanent, non-deletable bottom layer (Source.IsBackdrop), auto-detecting the full-screen game at launch/focus (else the primary display — never assumed monitor 0) via Win32FullScreenDetector (now with GetDisplays()/PrimaryMonitorIndex() for the in-app display picker), content re-designated via the OS GraphicsCapturePicker ("Change Capture…") or the in-app "Capture Display" submenu, refcounted/shared capture sessions in ScreenCaptureManager mirroring CameraManager — 45 tests passing
  5. ✅ Schema v6 — backdrop Live-only by policy — Scene.HasBackdrop, enforced by scene name on every load (EnforceBackdropPolicy: Starting/BRB/Chat/Ending never carry one; the one-time v5→v6 backfill covers all four), the scene context-menu "Backdrop" checkbox is gone (policy owns the flag), preview watermark hides when the backdrop renders, capture changed to WindowsRuntimeMarshal.TryGetDataUnsafe (the CsWinRT-safe frame-read) + downscale to the 1920×1080 master + 5s-throttled error logging (was flooding startup.log with 5 MB of cast errors and burning CPU), round webcam no longer re-rasterizes an ImageBrush every frame (Image + EllipseGeometry clip) — the live-mode stutter fix
  6. ✅ The five-scene catalog (SceneCatalog) — Starting/Live/BRB/Chat/Ending is the product — work with less, never more; the (+) button only shows when a canonical scene is missing and re-adds it (its menu lists only the missing ones) — 62 tests passing
  7. ✅ Webcam-after-session-start fix — a webcam added to a scene after the camera was already running (e.g. Chat) previously rendered a transparent container — CameraManager.GetPreviewBitmap + propagation in AddWebcamToActiveSceneAsync/ReacquireWebcam now hands the running shared frames to any newly added WebcamSceneConfig — 65 tests passing
  8. ✅ Chat scene webcam size cap — raised from 50%-per-dimension (960×540) to half the screen AREA (~1358×764 @16:9, MaxWebcamWidthFor/MaxWebcamHeightFor keyed by canonical name) so the viewer sees the creator better
  9. ✅ "Add Webcam" always opens the camera picker — deleting one scene's webcam then re-adding used to resurrect the old camera when another scene still used it — SwapWebcamIdentityAsync now swaps the app-wide identity if a different camera is chosen, same path as "Change Webcam…"
  10. ✅ Webcam resource validation + first-frame proof — MediaCaptureFrameSource validates post-init (VideoDeviceId match, stream properties ≥1, reader.StartAsync() status read + throws on non-Success); subscribes capture.Failed + CameraStreamStateChanged → SourceFailed event on the seam; fallback ladder (VideoPreview → VideoRecord). CameraManager.AcquireAsync requires first-frame proof (4s timeout): returns true only after a real frame arrives — silent empty box impossible. MainViewModel subscribes CameraFailed → red WebcamError chip in preview + MessageBox names suspect apps (CameraConflictProbe). 19041 SDK projection gaps: Exclusive/DeviceLost not projected; CameraStreamState.Failed compared by (int)2. 81 tests passing
  11. ✅ Scenes/sources UI — add/reorder/rename, image + background overlays with move/resize/opacity/reuse
  12. ✅ Audio UX shipped (UI) — the bottom-bar footer is now two lines (dropped/duration moved under bitrate/fps), with the mic's sound meter + mute button + volume slider grouped CENTERED on the footer's top line, beneath the preview panel (meter: 288px, muted slate track with ruler graduations + muted yellow/red zone tints, green→yellow→red fill; mute = speaker icon → red do-not-symbol when muted, and the slider and speaker stay in sync (volume 0 ⇔ muted — sliding off flips the speaker to muted, sliding up from 0 clears it); mic volume defaults to 80%, muting zeroes the meter and restores the prior volume on unmute (which flashes the meter to the restored position ~300ms before it returns to the live level); the meter is a READ-ONLY realtime level display (fill = live level × volume — volume is a gain on ambient noise; while the slider is dragged the bar previews the slider position and bounces back to the live level on release, which is 0 with no input — clicking the meter does nothing), clicking the MIC label opens a microphone picker whose chosen source shows left-justified inside the meter bar (fill at 75% opacity so the name + ruler markings show through); slim dimensional slider — gradient track/fill, gloss-sphere thumb; the old flat pink 18px-filled one is gone), everything else on line 2 (bitrate/fps/dropped/duration/health left, quality + gear right) — the creator's only audio control, desktop/game audio is automatic (KISS rule). Post-test follow-up (2026-08-13, game audio bar branch): the game audio bar (desktop/game, a mirror of the mic bar: meter + mute + volume) is overlaid at the bottom of the preview window (bottom-center dark chip — monitoring UI only, never on the live output) and appears only while a full-screen game is producing sound in the preview (IGameAudioDetector seam + default GameAudioDetector polling IFullScreenDetector + the live loopback level, floor 0.5%, into a pure GameAudioHysteresis: SHOW after ~500ms of fullscreen+sound, HIDE ~1s after leaving fullscreen, silence never hides an active bar; the VM polls it on a 250ms DispatcherTimer); capture now runs for the app's lifetime (mixer started once at startup via StartMicCaptureAsync, disposed in Shutdown — not go-live) so the meters preview live; the MIC label became a button with a status dot (Models/MicStatus: green = connected via the source Started event, yellow = mic problem, red = no device); picking a mic takes effect immediately (AudioMixer.RestartMic, loopback keeps running); fixed a latent ?.Invoke(meter.Push(...)) short-circuit that skipped the meter update when nothing was subscribed. Round 2 (2026-08-13): the meters were dead on a flat scale (real speech/game RMS is ~0.01..0.1 linear) — the raw level is now mapped via AudioLevelMeter.ToDisplay (−60..0 dBFS spread across 0..1) so typical input reads ~1/3..2/3 of the bar at default volume. 169 tests passing, 0 warnings. Round 3 (2026-08-13): the mic bar gained a mic mute icon (a microphone glyph in the speaker's 16px style, red + slash when muted) between the meter and the speaker — both mutes adjacent with spacing between them, same ToggleMicMuteCommand; and the top-center LIVE badge became an always-visible REC sign — dark gray dot + dim "REC" offline, bright red (#e94560) + "REC" + elapsed while live, darker red (#8f1f1f) when live with a private stream (driven by the dialog's chosen visibility) — RecDotBrush/RecTextBrush/RecDotOpacity/IsLivePrivate, pulsing while live. Queued: task 22 (voice filters). Voice-filter note: the meter's ToDisplay input is the pre-filter mic level; when filters land they must apply BEFORE the meter/encoder mix
  13. ✅ The connected YouTube account's avatar/name shows in the top bar next to Start Stream (SyncConnectedAccount); the scenes list is content-height now (no dead space before SOURCES)
  14. ✅ Social bar v2 (six-slot dialog, sign-in gate, real logos) — global bar layer (never a Source, no sources-list row), content-sized, centered, GREEN glow when ON, top/bottom snap-drag (default BOTTOM, persisted SocialBarPosition; drag clamps to 0/1040, tie→bottom). Footer Social button gets a state dot (green=ON). Dialog "Social Media Site Promotion" (SocialsDialog + ViewModels/SocialsDialogViewModel, WPF-free + injected ISocialValidator/sign-in/sign-out fakes): ON/OFF bar switch (SocialsConfig.BarEnabled, schema v8), 6 fixed slots — row 1 always YouTube (signed-in → channel handle; signed-out → sign-in gate → OAuth; delete → confirm sign-out, mirrors StopStream), row 2 free, rows 3-6 lock icons on freemium (Premium seam: all six). Validation: DetectService (URL domain / fediverse @user@domain / bare→Website) → async ISocialValidator on confirm/Save; valid snaps to text + real service logo (bundled SVG path data via LogoDataFor, Simple Icons CC0 — initials badges gone); invalid → red do-not, stays editable, Save blocked. LCR justify dropped (BarJustify unread), per-scene toggle dropped (Scene.HasSocialBar back-compat). Post-test fixes (2026-08-12): footer label "Socials" (not "Social"); fediverse @user@domain validates — the full handle is the identity end-to-end (DetectService/CanonicalUrlFor/HttpSocialValidator build https://domain/@user, no domain loss); Cancel is a hard stop — ISocialValidator.LookupAsync takes a CancellationToken, dialog VM owns a CTS, Cancel/X/Save abort in-flight lookups (HTTP request killed, canceled continuations never touch slot state), and ConfirmEdit skips re-submitting identical text (LostFocus on dismiss never re-fires a lookup). HttpSocialValidator now has real tests (fake HttpMessageHandler). 105 tests passing. Post-test fixes (2026-08-12, round 2): fediverse @user@domain no longer shows a generic chain — it resolves to the instance's actual software via nodeinfo (/.well-known/nodeinfo → software.name; SocialService.Fediverse enum member + SocialEntry.FediverseSoftware persisted in a new SocialEntry.Software column, schema migration by column-presence) and renders that software's bundled logo (LogoDataForFediverse: mastodon/peertube/pixelfed/misskey/lemmy/pleroma/firefish, generic fediverse honeycomb fallback). Dialog row-2 edit/trash icons were too dark — IconButton style gains Foreground=#d0d0d0; trash overrides #e94560 (app red). 112 tests passing. Post-test fixes (2026-08-12, round 3): a fediverse handle whose identity domain is itself a redirect (e.g. YunoHost default-app subdomains — @user@llamachile.tube where the mastodon instance lives at mastodon.llamachile.tube) now still resolves its software: nodeinfo on the identity domain is SSO-blocked, so HttpSocialValidator follows the bare root https://domain/ 302 to the real instance host and re-runs the nodeinfo lookup there.
  15. ✅ Window capture — absorbed into the Screen picker (no separate source type); dedicated window-as-source work is pending
  16. ❌ Scene compositing (D3DImage/MediaElement) — SUPERSEDED, permanently out (2026-09-01): the software output compositor (TASK 4 step 1) + the XAML preview ARE the design; a D3D11 swap stays a possibility behind the VideoFrame seam, not a feature
  17. ☐ Text source — live text ("Starting soon", "Back in 5", handle, callout)
  18. ✅ Chat box — YouTube live chat rendered on the stream so viewers read along in-video — ChatBoxRenderer (WPF FormattedText → VideoFrame), configurable font size/color/badges/timestamps/max-messages via Elements panel, schema v10, persisted across save/load
  19. ❌ Background removal (milestone 2) — ONNX Runtime + DirectML, MediaPipe Selfie Segmentation — deliberately NOT in this build
  20. ✅ Alerts — Super Chat / membership pop-ins (SHIPPED 2026-09-24 via TASK 43, NATIVE) — free and ungated (see Monetization in ai.md). v1 IN per the complete-v1 ruling. The deliverable is now native events + native animations, no third-party widget: the chat feed (liveChat/messages, six types decoded — TASK 43 half 1) drives a SourceType.AlertBox ("Stream Alerts", one celebration zone per layout) where each event plays its own unique branded animation (AlertRenderer/AlertOverlayLayer, 33ms ticker, Advance test clock). Creator rulings: a free-subscriber mention is a plain chat row — NO sub alert (YouTube emits no public event for free subs), NO viewer count (vetoed), and every card draws the "made with LlamaCasty!" brand line (each broadcast = free product placement). Precursor (still open): the reward-event persistence half — the canonical RewardEvents SQLite table (all six types persisted per broadcast) + superChatEvents.list backfill + the session-report rollup (TASK 10 → "Related work — monetization awareness"). The shipped alert path parses events into ChatMessage directly in memory; the report reads that table once it exists.
  21. ✅ Show Desktop toggle — Source.ShowDesktop bool persisted in DB (schema migration + LayoutStore read/write). When checked, primary monitor captures regardless of fullscreen game state. Toggling off falls back to running game or releases to static placeholder. ClearBackdropCaptureAsync clears CaptureKey + VideoImageSource so static fallback renders. Toggle on Live backdrop context menu (IsCheckable MenuItem).
  22. ✅ Top bar — status light + avatar + Log In button — red/green/pulsing ellipse (disconnected/connected/live). Avatar border visible only when connected, loads via BitmapImage code-behind. "Log In" button visible when disconnected, calls StartStreamCommand → GoLiveWindow. ShowStartStream now requires IsConnected.
  23. ✅ Chat preview — one-at-a-time mock messages — RunMockChatPreviewAsync: simple async loop, displays MockChatMessages[n] via Take(n+1), 1000ms between messages, wraps at end. ChatPreviewEnabled property on Source (default true). Stops on real messages, restarts on fade timer clear.
  24. ✅ All backdrops seeded + EnsureDefaultBackdrop — Live, Chat, Ending backdrops added (ending-backdrop.jpg replaced). EnsureDefaultBackdrop() called on every scene switch. SeedEndingBackdrop replaces existing (not skip-if-exists). Chat scene no longer skipped.
  25. ✅ SceneCatalog fix — Chat scene display name corrected to "Chat" (was "YouTube Chat" — that name belongs to the layer/SourceType.ChatBox, not the scene).
  26. ✅ Left panel spacing + context menu reorder — HR margins matched, Row 4 changed from * to Auto. Context menu reordered: Border Thickness first, then Opacity, Color, Effect.
  27. ✅ Logo + richer in-app About (2026-08-13, queued → 2026-08-14 SHIPPED) — the ytLlive wordmark in the top bar opens the About overlay (already wired); the About overlay is now the creator hub: the real logo (the "llama fortnite superman logo" from the creator's vault, copied to Assets/llama-logo.png — the creator's own art, no third-party license), plus creator-hub links — llama chile shop on YouTube (MainViewModel.ChannelUrl), Mastodon (https://mastodon.llamachile.tube/@gramps), Buy me a coffee (https://buymeacoffee.com/llamachiley — live), and Unlock Premium (greyed "coming soon" — the billing product URL is a tabled seam, PremiumUrl, until TASK 10 picks the subscription provider). A Licenses & legal button flips the About overlay to an in-app scrolling panel that loads the full THIRD-PARTY-NOTICES.txt text (MainViewModel.ShowLicenses reads the shipped file from the executable directory on first open; graceful "not found" fallback — never the OS viewer, everything stays in-app); "← Back to About" returns. The About overlay is also the in-app home of the notices — the top-bar About button that opened the file in the OS viewer was removed on 2026-08-13 for exactly this reason. Integration test (Good Dog Rule — ONE): AboutHubTests.About_Opens_InApp_Licensing_Loads_Shipped_Notices drives the real window + VM, asserting the hub opens, the link URLs are real, the licensing panel loads the shipped notices text (contains "Third-Party Notices" + "LGPL"), and back returns to the hub. 174 tests passing, 0 warnings
  28. ✅ Voice filters on the mic channel (2026-08-13 queued → SHIPPED 2026-08-14 inside TASK 9, the audio milestone) — the standard four applied to the sound input path (before the meter/encoder mix): bass boost, treble, noise suppression, compressor (set decided with the creator 2026-08-13). Noise suppression = a pure-C# noise gate (creator chose over RNNoise / a second ffmpeg afftdn pipe, 2026-08-14 — KISS). Always-on — no UI knobs; the mic stays the creator's single audio control
  29. ✅ Webcam row gates on the app-wide identity (2026-08-24, TASK 26) — the (+) menu's Webcam item greys out whenever a webcam exists anywhere (CanAddWebcam = StagedScene != null && _webcam == null; renamed from CanAddWebcamToStagedScene whose per-scene rule let a second picker run from a scene lacking the config), raised at both _webcam mutation sites (create / last-config removal) + scene staging + elements change. Creator's visual pass found it: minis/no-capture-rows/capture-controls all good. ONE integration test WebcamMenuGateTests.CanAddWebcam_Gates_On_The_AppWide_Webcam_Identity (real window + temp DB seeded with a webcam in Starting; asserts greyed while Live staged, re-enabled after RemoveSourceCommand clears the last config + the Webcam DB row). Stale map fixed in the same commit: ai.md's "empty-canvas right-click Show Webcam" claim dropped — that XAML never shipped (ShowWebcamCommand/CanShowWebcamInStagedScene are wired but unbound dead code, audit item). 224 tests (223 pass; the pre-existing AudioPipelineTests failure is unrelated)
  30. ✅ YouTube Chat layer: one-per-layout gate + legacy label heal (2026-08-24, TASK 27) — same rule as TASK 26 applied to the chat layer: the (+) menu's YouTube Chat item greys out while any scene carries a ChatBox source (CanAddYouTubeChat, raised on staging + elements change; AddSource refuses a second), tooltip "One chat layer at a time — it's already in your stream". Plus the creator's label fix: layers added by commit 65641d8 were named "Chat"; LoadLayout now heals exactly that un-renamed default to "YouTube Chat" so the Layers row matches the (+) picklist (creator renames untouched, idempotent). ONE integration test ChatLayerGateTests.LegacyChatName_Heals_And_CanAddYouTubeChat_Gates_On_The_Existing_Layer (real window + temp DB with a legacy "Chat" row: healed on load + persisted on save, greyed cross-scene and in-scene, re-enabled after remove). 225 tests (224 pass; pre-existing AudioPipelineTests failure unrelated)
  31. ✅ Broadcast metadata pull-out + launch geometry (2026-08-24) — creator ask: "a tab-pullout on the right side of the preview pane — white tab, YouTube-red label reading 'Text' — opening a form with all settable liveBroadcast fields; fields that can't be filled before launch greyed out; saving/updating remote content; field data saved and pre-loaded on app run". Shipped as specified with one correction: the API is the inverse of "more fields once live" for contentDetails (those lock in created/ready) — the pull-out carries the always-editable snippet/status set: Title, Description, Tags (csv), Visibility (private/unlisted/public), Made-for-Kids, plus read-only Scheduled Start. Always visible (creator revised same day from Live-only gating). White 30px tab, rotated red "Text", right edge vertically centered; click slides a dark 320px drawer left over the preview (200ms CubicEase). Every edit persists to Settings keys (Broadcast.*) immediately; Update Broadcast button calls the new YouTubeStreamService.UpdateBroadcast (PUT liveBroadcasts?part=snippet,status, echoes scheduledStartTime because update replaces the whole snippet part). Go Live prefills from the form and captures what was inserted (CaptureGoLive). The old un-persisted Default Stream Title/Description fields came OFF the App Settings overlay (replaced by the form); DefaultStreamTitle/Description VM properties deleted. Launch geometry (same unit): default 1920×1040, MinWidth 1366, MinHeight 768; WindowStartupLocation=Manual + window size/position persisted on close via RestoreBounds (maximized-safe), restored in ctor clamped to minimums and the primary work area (disconnected-secondary fallback). ONE integration test BroadcastPullOutTests.Metadata_Persists_WindowRestores_Clamped_And_UpdatePatchesRemote. 228 tests (227 pass; pre-existing AudioPipelineTests failure unrelated)
  32. ✅ Gear menu cleanup + Default Location for Recordings + single Start button (2026-08-29) — KISS: removed Gear-menu Save Layout/Save Layout As…/Open Layout… (redundant with auto-save; multi-layout = OBS) + dead SaveLayoutCommand/SaveLayoutAsCommand/OpenLayoutCommand + SaveLayoutAs()/OpenLayoutFile() methods. App Settings panel: first real setting — Default Location for Recordings (configured folder via ChooseRecordFolderCommand + "Use Downloads" reset via ResetRecordFolderCommand, resolved path shown). Fallback chain: configured → %USERPROFILE%\Downloads → MyVideos. StartRecordFile() + ChooseRecordFolder() InitialDirectory updated to use the new fallback. Single Start button: PrimaryButtonText/IsStreamingStart + their OnPropertyChanged calls removed; button Content="Start" in XAML — pills are the intent surface, button is always just Start. Choose Record Folder fast-path kept in the button context menu. Monetization (About links/premium) deferred to a later task. ONE fallback test DefaultRecordFolder_Falls_Back_To_Downloads. build 0 warnings, 247 tests (2 known failures, pre-existing).

The Minimal Source Set (design decision — do not expand casually)

ytLlive is YouTube-only and 90% of users are casual. OBS's long source list is off-putting; we ship the hot few and nothing esoteric. If a user needs more, they've graduated to OBS.

  1. Webcam — the face cam. Non-negotiable.
  2. Screen — the main event (game, slides, browser). One source; a picker chooses a monitor or a window. (Window capture is absorbed here — no separate source type.)
  3. Background — a full-canvas backdrop image. Fills the whole scene automatically, zero fiddling. Kept separate from Image on purpose: same pixels, but this one needs no positioning.
  4. Image — a floating graphic/logo overlay (watermark, badge, corner branding). Free-positioned.
  5. Text — live text ("Starting soon", "Back in 5", handle, callout). Casual streamers live on this.
  6. Chat box — YouTube live chat rendered on the stream so viewers read along in-video. YT-native.
  7. Alerts — Super Chat / membership pop-ins. The dopamine source. The one big lift — DONE (2026-09-24, TASK 43): native events + native animations. One SourceType.AlertBox celebration zone; six event types = six unique branded animations; free-sub mention = chat row only; no viewer count. Free and ungated — paid = flash removal only (corrected 2026-09-01). Pending: the RewardEvents SQLite persistence half (item 20; the report + journey read it, TASK 10 → related work).

Deliberately NOT supported: game capture, browser source, media playlist, VLC, color-key voodoo, MIDI.

Source memory model (design decision)

  1. A scene has resources. Resources can be shared across scenes.
  2. A resource exists exactly once in memory, no matter how many scenes use it (a logo in five scenes = one loaded bitmap).
  3. Every resource carries a catalog of scenes: one usage entry per scene it appears in, each entry dictating that scene's use — placement (X/Y/Width/Height), opacity, z-order, enabled, scale mode, crop.
  4. Usages are named {resourceName}.{sceneName} — whatever the user named the resource, dot, the scene name: logo.starting, logo.live, myPic.brb. Not a hardcoded "logo".
  5. A webcam in two scenes = one capture session, two catalog entries.
  6. Refcount by catalog size: the last usage removed → the resource is disposed and evicted.
  7. The resource (not a per-scene node) owns everything IDisposable.

Scene transitions (design decision)

Scene switching while live must never stutter. Supported types, most → least economical:

  1. Cut — instant switch. The default. Zero cost.
  2. Fade — short crossfade (~300ms).
  3. Move — a simple, economical move transition, done to perfection and memory-efficient. The smart streamer's bread and butter.
  4. Custom (media) transitions — require media elements (video/stinger playback during the transition). Heavier, but creators pay for these, so we support them. Their media follows the same resource memory model: loaded once, catalogued by scene.

Preview shows the transition too (WYSIWYG). No wipes/slides/LUTs beyond the four above.

Requirements:

  1. Screen — Windows.Graphics.Capture (WinRT), enumerate displays/windows, picker
  2. Webcam — MediaCapture (WinRT SDK projection) with device enumeration — ✅ milestone 1 done:
    1. TFM bumped to net8.0-windows10.0.19041.0 (app and tests) so the WinRT projection resolves from the SDK reference packs — no NuGet package, no capability manifest (unpackaged desktop app)
    2. MediaCaptureFrameSource (CPU-first: MemoryPreference = Cpu, BGRA8 via CreateFrameReaderAsync), MediaCaptureCameraEnumerator (DeviceInformation.FindAllAsync(DeviceClass.VideoCapture))
    3. CameraManager: refcounted by DeviceId, one shared WriteableBitmap app-wide, dispatcher-coalesced UI updates (~render rate, latest-frame drop), placeholder/AppLog + warning on failure
    4. CameraPickerDialog (mirror of ReuseImageDialog) — "Searching for cameras…" / list / "No cameras found" states
    5. One webcam app-wide: Add → Webcam greyed out once one exists ("it's already in your stream" tooltip); persisted DeviceId re-acquires after layout load
    6. Default placement 16:9 480×270, bottom-right, 32px margin; drag/resize/selection shared with Image sources
    7. Clip shapes: Traditional + Round (phone view dropped — the 9:16 phone output is the vertical output-crop tier); mirror; both persisted in the layout DB (schema v2) and toggled from the source chip
    8. Background removal = milestone 2 (ONNX Runtime + DirectML, MediaPipe Selfie Segmentation) — not in this build
  3. Background / Image / Text — static sources positioned/scaled/opacity
  4. Chat box — rendered from the live chat poll (right panel is the same feed, raw)
  5. Scene compositing — per-scene source layering (z-order = sources list order, top-to-bottom back-to-front), preview rendered via D3DImage or MediaElement
  6. Branding flash — the topmost full-frame "made with ytLlive!" layer at ~25% opacity, ~1s on / 300s off (see Monetization in ai.md), gated on BrandFlashEnabled + live/recording. Lives in the preview compositor now (BrandFlashLayer in MainWindow.xaml CanvasGrid, driven by BrandFlashActive/BrandFlashTimer in MainViewModel); the encoder output renders the same layer, and v0.2 local recordings carry it too
  7. Drag/drop placement & reorder — intuitive, visual (per design principle):
    1. Preview: click-drag a source in the center panel to reposition it; resize via handles
    2. Scenes list: drag rows to reorder scenes
    3. Sources list: drag rows to reorder sources (this is the z-order) — implemented