d4aa588da0
Replace the external StreamElements feed with NATIVE YouTube events per the creator's 2026-09-23 question. Chat parity half: - YouTubeChatService decodes ALL SIX liveChat/messages event types into ChatMessage.Kind (superChat/superSticker/newSponsor/membershipGifting/ giftMembershipReceived/memberMilestoneChat) — the four formerly-empty overlay rows are real now — and re-arms its one-shot poll on the server's pollingIntervalMillis (streamList connection semantics; clamp 1000-6000ms, maxResults=2000). ParsePage internal static seam + ChatPage record for deterministic tests; optional HttpClient ctor seam kept; InjectSimulatedMessage (TASK 41) preserved. Alerts half (creator rulings: one celebration zone, six UNIQUE animations, no menus/polls, sub mention = chat row only, NO viewer count): - New SourceType.AlertBox 'Stream Alerts' (one per layout, CanAddAlerts gate mirroring chat; idle = transparent). - AlertRenderer: six distinct branded animations (SuperChat slide-up/shine/ count-up, SuperSticker scale-pop, NewMember drop-in/flash, MemberGift slide-left/ chip-fan, GiftReceived confetti, MemberMilestone rise/growth-bar), every card drawing the 'made with LlamaCasty!' brand line. - AlertOverlayLayer: true component (Commit-G pattern) — queue (cap 10, drop tail never stall), 33ms UI ticker, cache-first RenderFrame + UpdatePreview, Advance(double) as the deterministic test clock; ChatEventKind.None rows never enqueue. - Wired: resolver RenderAlertBox, _alertLayer ctor + dispose, LoadLayout previews, Add menu item (Controls/LeftPanel.xaml), TestSessionViewModel sims tagged (member->NewMember, superchat->SuperChat). Good Dog ONE integration test: AlertLayerTests (RealApp STA, real WPF raster) — ParsePage classifies all six kinds + cadence fields; None rows enqueue nothing; six events play pairwise-distinct moving frames then drain to null. Gate: clean build 0 warnings; full suite 316/317, the one failure (LayerReorderPersistenceTests.RealMouseDrag) repros on the clean tree — the known environmental class (real-mouse-drag no-ops with a game/fullscreen window focused). References (OBS/overlay ecosystem): - streamList semantics: https://developers.google.com/youtube/v3/live/docs/liveChatMessages/streamList - OBS alert-box pattern (designated celebration zone, idle transparent): creator-chosen model
28 KiB
28 KiB
TASK 3 — Capture Pipeline (Scenes/Sources)
Catalog:
TASKS.md— status and requirements live here.
Goal: Real video preview in the center panel — the minimal source set below, composited per scene.
Status: 🔶 In progress — Alerts (item 20) SHIPPED 2026-09-24 via TASK 43; still open: item 17 (Text source), item 20's RewardEvents SQLite persistence half (16 compositing + 19 background removal excluded)
- ✅ Milestone 1 — webcam — MediaCapture (WinRT SDK projection) with device enumeration, CPU-first frame source, refcounted
CameraManager, picker dialog, clip shapes (Traditional + Round) + mirror, 480×270 default placement — schema v2 - ✅ Schema v3 (Ship Branch A) — multi-scene webcam (singleton
Webcam+ per-sceneWebcamSceneConfig), right-click border/context menu, static OBS-style borders, 50%-per-dimension webcam size cap, device-swap (ReleaseAllAsync) — 25 tests passing - ✅ Schema v4 — round→rect restore persisted (
WebcamSceneConfig.RectWidth/RectHeight) + one-time legacy-square 16:9 heal on load - ✅ Screen backdrop (ship task #1, schema v5) — live desktop/game capture as a permanent, non-deletable bottom layer (
Source.IsBackdrop), auto-detecting the full-screen game at launch/focus (else the primary display — never assumed monitor 0) viaWin32FullScreenDetector(now withGetDisplays()/PrimaryMonitorIndex()for the in-app display picker), content re-designated via the OSGraphicsCapturePicker("Change Capture…") or the in-app "Capture Display" submenu, refcounted/shared capture sessions inScreenCaptureManagermirroringCameraManager— 45 tests passing - ✅ Schema v6 — backdrop Live-only by policy —
Scene.HasBackdrop, enforced by scene name on every load (EnforceBackdropPolicy: Starting/BRB/Chat/Ending never carry one; the one-time v5→v6 backfill covers all four), the scene context-menu "Backdrop" checkbox is gone (policy owns the flag), preview watermark hides when the backdrop renders, capture changed toWindowsRuntimeMarshal.TryGetDataUnsafe(the CsWinRT-safe frame-read) + downscale to the 1920×1080 master + 5s-throttled error logging (was floodingstartup.logwith 5 MB of cast errors and burning CPU), round webcam no longer re-rasterizes anImageBrushevery frame (Image + EllipseGeometry clip) — the live-mode stutter fix - ✅ The five-scene catalog (
SceneCatalog) — Starting/Live/BRB/Chat/Ending is the product — work with less, never more; the (+) button only shows when a canonical scene is missing and re-adds it (its menu lists only the missing ones) — 62 tests passing - ✅ Webcam-after-session-start fix — a webcam added to a scene after the camera was already running (e.g. Chat) previously rendered a transparent container —
CameraManager.GetPreviewBitmap+ propagation inAddWebcamToActiveSceneAsync/ReacquireWebcamnow hands the running shared frames to any newly addedWebcamSceneConfig— 65 tests passing - ✅ Chat scene webcam size cap — raised from 50%-per-dimension (960×540) to half the screen AREA (~1358×764 @16:9,
MaxWebcamWidthFor/MaxWebcamHeightForkeyed by canonical name) so the viewer sees the creator better - ✅ "Add Webcam" always opens the camera picker — deleting one scene's webcam then re-adding used to resurrect the old camera when another scene still used it —
SwapWebcamIdentityAsyncnow swaps the app-wide identity if a different camera is chosen, same path as "Change Webcam…" - ✅ Webcam resource validation + first-frame proof —
MediaCaptureFrameSourcevalidates post-init (VideoDeviceId match, stream properties ≥1,reader.StartAsync()status read + throws on non-Success); subscribescapture.Failed+CameraStreamStateChanged→SourceFailedevent on the seam; fallback ladder (VideoPreview → VideoRecord).CameraManager.AcquireAsyncrequires first-frame proof (4s timeout): returns true only after a real frame arrives — silent empty box impossible.MainViewModelsubscribesCameraFailed→ redWebcamErrorchip in preview + MessageBox names suspect apps (CameraConflictProbe). 19041 SDK projection gaps:Exclusive/DeviceLostnot projected;CameraStreamState.Failedcompared by(int)2. 81 tests passing - ✅ Scenes/sources UI — add/reorder/rename, image + background overlays with move/resize/opacity/reuse
- ✅ Audio UX shipped (UI) — the bottom-bar footer is now two lines (dropped/duration moved under bitrate/fps), with the mic's sound meter + mute button + volume slider grouped CENTERED on the footer's top line, beneath the preview panel (meter: 288px, muted slate track with ruler graduations + muted yellow/red zone tints, green→yellow→red fill; mute = speaker icon → red do-not-symbol when muted, and the slider and speaker stay in sync (volume 0 ⇔ muted — sliding off flips the speaker to muted, sliding up from 0 clears it); mic volume defaults to 80%, muting zeroes the meter and restores the prior volume on unmute (which flashes the meter to the restored position ~300ms before it returns to the live level); the meter is a READ-ONLY realtime level display (fill = live level × volume — volume is a gain on ambient noise; while the slider is dragged the bar previews the slider position and bounces back to the live level on release, which is 0 with no input — clicking the meter does nothing), clicking the MIC label opens a microphone picker whose chosen source shows left-justified inside the meter bar (fill at 75% opacity so the name + ruler markings show through); slim dimensional slider — gradient track/fill, gloss-sphere thumb; the old flat pink 18px-filled one is gone), everything else on line 2 (bitrate/fps/dropped/duration/health left, quality + gear right) — the creator's only audio control, desktop/game audio is automatic (KISS rule). Post-test follow-up (2026-08-13, game audio bar branch): the game audio bar (desktop/game, a mirror of the mic bar: meter + mute + volume) is overlaid at the bottom of the preview window (bottom-center dark chip — monitoring UI only, never on the live output) and appears only while a full-screen game is producing sound in the preview (
IGameAudioDetectorseam + defaultGameAudioDetectorpollingIFullScreenDetector+ the live loopback level, floor 0.5%, into a pureGameAudioHysteresis: SHOW after ~500ms of fullscreen+sound, HIDE ~1s after leaving fullscreen, silence never hides an active bar; the VM polls it on a 250msDispatcherTimer); capture now runs for the app's lifetime (mixer started once at startup viaStartMicCaptureAsync, disposed inShutdown— not go-live) so the meters preview live; the MIC label became a button with a status dot (Models/MicStatus: green = connected via the sourceStartedevent, yellow = mic problem, red = no device); picking a mic takes effect immediately (AudioMixer.RestartMic, loopback keeps running); fixed a latent?.Invoke(meter.Push(...))short-circuit that skipped the meter update when nothing was subscribed. Round 2 (2026-08-13): the meters were dead on a flat scale (real speech/game RMS is ~0.01..0.1 linear) — the raw level is now mapped viaAudioLevelMeter.ToDisplay(−60..0 dBFS spread across 0..1) so typical input reads ~1/3..2/3 of the bar at default volume. 169 tests passing, 0 warnings. Round 3 (2026-08-13): the mic bar gained a mic mute icon (a microphone glyph in the speaker's 16px style, red + slash when muted) between the meter and the speaker — both mutes adjacent with spacing between them, sameToggleMicMuteCommand; and the top-center LIVE badge became an always-visible REC sign — dark gray dot + dim "REC" offline, bright red (#e94560) + "REC" + elapsed while live, darker red (#8f1f1f) when live with a private stream (driven by the dialog's chosen visibility) —RecDotBrush/RecTextBrush/RecDotOpacity/IsLivePrivate, pulsing while live. Queued: task 22 (voice filters). Voice-filter note: the meter'sToDisplayinput is the pre-filter mic level; when filters land they must apply BEFORE the meter/encoder mix - ✅ The connected YouTube account's avatar/name shows in the top bar next to Start Stream (
SyncConnectedAccount); the scenes list is content-height now (no dead space before SOURCES) - ✅ Social bar v2 (six-slot dialog, sign-in gate, real logos) — global bar layer (never a Source, no sources-list row), content-sized, centered, GREEN glow when ON, top/bottom snap-drag (default BOTTOM, persisted
SocialBarPosition; drag clamps to 0/1040, tie→bottom). Footer Social button gets a state dot (green=ON). Dialog "Social Media Site Promotion" (SocialsDialog+ViewModels/SocialsDialogViewModel, WPF-free + injectedISocialValidator/sign-in/sign-out fakes): ON/OFF bar switch (SocialsConfig.BarEnabled, schema v8), 6 fixed slots — row 1 always YouTube (signed-in → channel handle; signed-out → sign-in gate → OAuth; delete → confirm sign-out, mirrorsStopStream), row 2 free, rows 3-6 lock icons on freemium (Premium seam: all six). Validation:DetectService(URL domain / fediverse@user@domain/ bare→Website) → asyncISocialValidatoron confirm/Save; valid snaps to text + real service logo (bundled SVG path data viaLogoDataFor, Simple Icons CC0 — initials badges gone); invalid → red do-not, stays editable, Save blocked. LCR justify dropped (BarJustifyunread), per-scene toggle dropped (Scene.HasSocialBarback-compat). Post-test fixes (2026-08-12): footer label "Socials" (not "Social"); fediverse@user@domainvalidates — the full handle is the identity end-to-end (DetectService/CanonicalUrlFor/HttpSocialValidatorbuildhttps://domain/@user, no domain loss); Cancel is a hard stop —ISocialValidator.LookupAsynctakes aCancellationToken, dialog VM owns a CTS, Cancel/X/Save abort in-flight lookups (HTTP request killed, canceled continuations never touch slot state), andConfirmEditskips re-submitting identical text (LostFocus on dismiss never re-fires a lookup).HttpSocialValidatornow has real tests (fakeHttpMessageHandler). 105 tests passing. Post-test fixes (2026-08-12, round 2): fediverse@user@domainno longer shows a generic chain — it resolves to the instance's actual software via nodeinfo (/.well-known/nodeinfo→software.name;SocialService.Fediverseenum member +SocialEntry.FediverseSoftwarepersisted in a newSocialEntry.Softwarecolumn, schema migration by column-presence) and renders that software's bundled logo (LogoDataForFediverse: mastodon/peertube/pixelfed/misskey/lemmy/pleroma/firefish, generic fediverse honeycomb fallback). Dialog row-2 edit/trash icons were too dark —IconButtonstyle gainsForeground=#d0d0d0; trash overrides#e94560(app red). 112 tests passing. Post-test fixes (2026-08-12, round 3): a fediverse handle whose identity domain is itself a redirect (e.g. YunoHost default-app subdomains —@user@llamachile.tubewhere the mastodon instance lives atmastodon.llamachile.tube) now still resolves its software: nodeinfo on the identity domain is SSO-blocked, soHttpSocialValidatorfollows the bare roothttps://domain/302 to the real instance host and re-runs the nodeinfo lookup there. - ✅ Window capture — absorbed into the Screen picker (no separate source type); dedicated window-as-source work is pending
- ❌ Scene compositing (D3DImage/MediaElement) — SUPERSEDED, permanently out (2026-09-01): the software output compositor (TASK 4 step 1) + the XAML preview ARE the design; a D3D11 swap stays a possibility behind the
VideoFrameseam, not a feature - ☐ Text source — live text ("Starting soon", "Back in 5", handle, callout)
- ✅ Chat box — YouTube live chat rendered on the stream so viewers read along in-video —
ChatBoxRenderer(WPF FormattedText → VideoFrame), configurable font size/color/badges/timestamps/max-messages via Elements panel, schema v10, persisted across save/load - ❌ Background removal (milestone 2) — ONNX Runtime + DirectML, MediaPipe Selfie Segmentation — deliberately NOT in this build
- ✅ Alerts — Super Chat / membership pop-ins (SHIPPED 2026-09-24 via TASK 43, NATIVE) —
free and ungated (see Monetization in
ai.md). v1 IN per the complete-v1 ruling. The deliverable is now native events + native animations, no third-party widget: the chat feed (liveChat/messages, six types decoded — TASK 43 half 1) drives aSourceType.AlertBox("Stream Alerts", one celebration zone per layout) where each event plays its own unique branded animation (AlertRenderer/AlertOverlayLayer, 33ms ticker,Advancetest clock). Creator rulings: a free-subscriber mention is a plain chat row — NO sub alert (YouTube emits no public event for free subs), NO viewer count (vetoed), and every card draws the "made with LlamaCasty!" brand line (each broadcast = free product placement). Precursor (still open): the reward-event persistence half — the canonicalRewardEventsSQLite table (all six types persisted per broadcast) +superChatEvents.listbackfill + the session-report rollup (TASK 10 → "Related work — monetization awareness"). The shipped alert path parses events intoChatMessagedirectly in memory; the report reads that table once it exists. - ✅ Show Desktop toggle —
Source.ShowDesktopbool persisted in DB (schema migration + LayoutStore read/write). When checked, primary monitor captures regardless of fullscreen game state. Toggling off falls back to running game or releases to static placeholder.ClearBackdropCaptureAsyncclearsCaptureKey+VideoImageSourceso static fallback renders. Toggle on Live backdrop context menu (IsCheckable MenuItem). - ✅ Top bar — status light + avatar + Log In button — red/green/pulsing ellipse (disconnected/connected/live). Avatar border visible only when connected, loads via
BitmapImagecode-behind. "Log In" button visible when disconnected, callsStartStreamCommand→ GoLiveWindow.ShowStartStreamnow requiresIsConnected. - ✅ Chat preview — one-at-a-time mock messages —
RunMockChatPreviewAsync: simple async loop, displaysMockChatMessages[n]viaTake(n+1), 1000ms between messages, wraps at end.ChatPreviewEnabledproperty on Source (default true). Stops on real messages, restarts on fade timer clear. - ✅ All backdrops seeded + EnsureDefaultBackdrop — Live, Chat, Ending backdrops added (ending-backdrop.jpg replaced).
EnsureDefaultBackdrop()called on every scene switch.SeedEndingBackdropreplaces existing (not skip-if-exists). Chat scene no longer skipped. - ✅ SceneCatalog fix — Chat scene display name corrected to "Chat" (was "YouTube Chat" — that name belongs to the layer/SourceType.ChatBox, not the scene).
- ✅ Left panel spacing + context menu reorder — HR margins matched, Row 4 changed from
*toAuto. Context menu reordered: Border Thickness first, then Opacity, Color, Effect. - ✅ Logo + richer in-app About (2026-08-13, queued → 2026-08-14 SHIPPED) — the ytLlive wordmark in the top bar opens the About overlay (already wired); the About overlay is now the creator hub: the real logo (the "llama fortnite superman logo" from the creator's vault, copied to
Assets/llama-logo.png— the creator's own art, no third-party license), plus creator-hub links — llama chile shop on YouTube (MainViewModel.ChannelUrl), Mastodon (https://mastodon.llamachile.tube/@gramps), Buy me a coffee (https://buymeacoffee.com/llamachiley— live), and Unlock Premium (greyed "coming soon" — the billing product URL is a tabled seam,PremiumUrl, until TASK 10 picks the subscription provider). A Licenses & legal button flips the About overlay to an in-app scrolling panel that loads the fullTHIRD-PARTY-NOTICES.txttext (MainViewModel.ShowLicensesreads the shipped file from the executable directory on first open; graceful "not found" fallback — never the OS viewer, everything stays in-app); "← Back to About" returns. The About overlay is also the in-app home of the notices — the top-bar About button that opened the file in the OS viewer was removed on 2026-08-13 for exactly this reason. Integration test (Good Dog Rule — ONE):AboutHubTests.About_Opens_InApp_Licensing_Loads_Shipped_Noticesdrives the real window + VM, asserting the hub opens, the link URLs are real, the licensing panel loads the shipped notices text (contains "Third-Party Notices" + "LGPL"), and back returns to the hub. 174 tests passing, 0 warnings - ✅ Voice filters on the mic channel (2026-08-13 queued → SHIPPED 2026-08-14 inside TASK 9, the audio milestone) — the standard four applied to the sound input path (before the meter/encoder mix): bass boost, treble, noise suppression, compressor (set decided with the creator 2026-08-13). Noise suppression = a pure-C# noise gate (creator chose over RNNoise / a second ffmpeg
afftdnpipe, 2026-08-14 — KISS). Always-on — no UI knobs; the mic stays the creator's single audio control - ✅ Webcam row gates on the app-wide identity (2026-08-24, TASK 26) — the (+) menu's Webcam item greys out whenever a webcam exists anywhere (
CanAddWebcam=StagedScene != null && _webcam == null; renamed fromCanAddWebcamToStagedScenewhose per-scene rule let a second picker run from a scene lacking the config), raised at both_webcammutation sites (create / last-config removal) + scene staging + elements change. Creator's visual pass found it: minis/no-capture-rows/capture-controls all good. ONE integration testWebcamMenuGateTests.CanAddWebcam_Gates_On_The_AppWide_Webcam_Identity(real window + temp DB seeded with a webcam in Starting; asserts greyed while Live staged, re-enabled afterRemoveSourceCommandclears the last config + theWebcamDB row). Stale map fixed in the same commit: ai.md's "empty-canvas right-click Show Webcam" claim dropped — that XAML never shipped (ShowWebcamCommand/CanShowWebcamInStagedSceneare wired but unbound dead code, audit item). 224 tests (223 pass; the pre-existing AudioPipelineTests failure is unrelated) - ✅ YouTube Chat layer: one-per-layout gate + legacy label heal (2026-08-24, TASK 27) — same rule as TASK 26 applied to the chat layer: the (+) menu's YouTube Chat item greys out while any scene carries a ChatBox source (
CanAddYouTubeChat, raised on staging + elements change;AddSourcerefuses a second), tooltip "One chat layer at a time — it's already in your stream". Plus the creator's label fix: layers added by commit65641d8were named "Chat"; LoadLayout now heals exactly that un-renamed default to "YouTube Chat" so the Layers row matches the (+) picklist (creator renames untouched, idempotent). ONE integration testChatLayerGateTests.LegacyChatName_Heals_And_CanAddYouTubeChat_Gates_On_The_Existing_Layer(real window + temp DB with a legacy "Chat" row: healed on load + persisted on save, greyed cross-scene and in-scene, re-enabled after remove). 225 tests (224 pass; pre-existing AudioPipelineTests failure unrelated) - ✅ Broadcast metadata pull-out + launch geometry (2026-08-24) — creator ask: "a tab-pullout on the right side of the preview pane — white tab, YouTube-red label reading 'Text' — opening a form with all settable liveBroadcast fields; fields that can't be filled before launch greyed out; saving/updating remote content; field data saved and pre-loaded on app run". Shipped as specified with one correction: the API is the inverse of "more fields once live" for contentDetails (those lock in created/ready) — the pull-out carries the always-editable snippet/status set: Title, Description, Tags (csv), Visibility (private/unlisted/public), Made-for-Kids, plus read-only Scheduled Start. Always visible (creator revised same day from Live-only gating). White 30px tab, rotated red "Text", right edge vertically centered; click slides a dark 320px drawer left over the preview (200ms CubicEase). Every edit persists to Settings keys (
Broadcast.*) immediately; Update Broadcast button calls the newYouTubeStreamService.UpdateBroadcast(PUTliveBroadcasts?part=snippet,status, echoes scheduledStartTime because update replaces the whole snippet part). Go Live prefills from the form and captures what was inserted (CaptureGoLive). The old un-persisted Default Stream Title/Description fields came OFF the App Settings overlay (replaced by the form);DefaultStreamTitle/DescriptionVM properties deleted. Launch geometry (same unit): default 1920×1040, MinWidth 1366, MinHeight 768;WindowStartupLocation=Manual+ window size/position persisted on close viaRestoreBounds(maximized-safe), restored in ctor clamped to minimums and the primary work area (disconnected-secondary fallback). ONE integration testBroadcastPullOutTests.Metadata_Persists_WindowRestores_Clamped_And_UpdatePatchesRemote. 228 tests (227 pass; pre-existing AudioPipelineTests failure unrelated) - ✅ Gear menu cleanup + Default Location for Recordings + single Start button (2026-08-29) — KISS: removed Gear-menu
Save Layout/Save Layout As…/Open Layout…(redundant with auto-save; multi-layout = OBS) + deadSaveLayoutCommand/SaveLayoutAsCommand/OpenLayoutCommand+SaveLayoutAs()/OpenLayoutFile()methods. App Settings panel: first real setting —Default Location for Recordings(configured folder viaChooseRecordFolderCommand+ "Use Downloads" reset viaResetRecordFolderCommand, resolved path shown). Fallback chain: configured →%USERPROFILE%\Downloads→MyVideos.StartRecordFile()+ChooseRecordFolder()InitialDirectoryupdated to use the new fallback. Single Start button:PrimaryButtonText/IsStreamingStart+ theirOnPropertyChangedcalls removed; buttonContent="Start"in XAML — pills are the intent surface, button is always just Start.Choose Record Folderfast-path kept in the button context menu. Monetization (About links/premium) deferred to a later task. ONE fallback testDefaultRecordFolder_Falls_Back_To_Downloads. build 0 warnings, 247 tests (2 known failures, pre-existing).
The Minimal Source Set (design decision — do not expand casually)
ytLlive is YouTube-only and 90% of users are casual. OBS's long source list is off-putting; we ship the hot few and nothing esoteric. If a user needs more, they've graduated to OBS.
- Webcam — the face cam. Non-negotiable.
- Screen — the main event (game, slides, browser). One source; a picker chooses a monitor or a window. (Window capture is absorbed here — no separate source type.)
- Background — a full-canvas backdrop image. Fills the whole scene automatically, zero fiddling. Kept separate from Image on purpose: same pixels, but this one needs no positioning.
- Image — a floating graphic/logo overlay (watermark, badge, corner branding). Free-positioned.
- Text — live text ("Starting soon", "Back in 5", handle, callout). Casual streamers live on this.
- Chat box — YouTube live chat rendered on the stream so viewers read along in-video. YT-native.
- Alerts — Super Chat / membership pop-ins. The dopamine source. The one big lift — DONE
(2026-09-24, TASK 43): native events + native animations. One
SourceType.AlertBoxcelebration zone; six event types = six unique branded animations; free-sub mention = chat row only; no viewer count. Free and ungated — paid = flash removal only (corrected 2026-09-01). Pending: theRewardEventsSQLite persistence half (item 20; the report + journey read it, TASK 10 → related work).
Deliberately NOT supported: game capture, browser source, media playlist, VLC, color-key voodoo, MIDI.
Source memory model (design decision)
- A scene has resources. Resources can be shared across scenes.
- A resource exists exactly once in memory, no matter how many scenes use it (a logo in five scenes = one loaded bitmap).
- Every resource carries a catalog of scenes: one usage entry per scene it appears in, each entry dictating that scene's use — placement (X/Y/Width/Height), opacity, z-order, enabled, scale mode, crop.
- Usages are named
{resourceName}.{sceneName}— whatever the user named the resource, dot, the scene name:logo.starting,logo.live,myPic.brb. Not a hardcoded "logo". - A webcam in two scenes = one capture session, two catalog entries.
- Refcount by catalog size: the last usage removed → the resource is disposed and evicted.
- The resource (not a per-scene node) owns everything
IDisposable.
Scene transitions (design decision)
Scene switching while live must never stutter. Supported types, most → least economical:
- Cut — instant switch. The default. Zero cost.
- Fade — short crossfade (~300ms).
- Move — a simple, economical move transition, done to perfection and memory-efficient. The smart streamer's bread and butter.
- Custom (media) transitions — require media elements (video/stinger playback during the transition). Heavier, but creators pay for these, so we support them. Their media follows the same resource memory model: loaded once, catalogued by scene.
Preview shows the transition too (WYSIWYG). No wipes/slides/LUTs beyond the four above.
Requirements:
- Screen — Windows.Graphics.Capture (WinRT), enumerate displays/windows, picker
- Webcam — MediaCapture (WinRT SDK projection) with device enumeration — ✅ milestone 1 done:
- TFM bumped to
net8.0-windows10.0.19041.0(app and tests) so the WinRT projection resolves from the SDK reference packs — no NuGet package, no capability manifest (unpackaged desktop app) MediaCaptureFrameSource(CPU-first:MemoryPreference = Cpu, BGRA8 viaCreateFrameReaderAsync),MediaCaptureCameraEnumerator(DeviceInformation.FindAllAsync(DeviceClass.VideoCapture))CameraManager: refcounted byDeviceId, one sharedWriteableBitmapapp-wide, dispatcher-coalesced UI updates (~render rate, latest-frame drop), placeholder/AppLog+ warning on failureCameraPickerDialog(mirror ofReuseImageDialog) — "Searching for cameras…" / list / "No cameras found" states- One webcam app-wide: Add → Webcam greyed out once one exists ("it's already in your stream" tooltip); persisted
DeviceIdre-acquires after layout load - Default placement 16:9 480×270, bottom-right, 32px margin; drag/resize/selection shared with Image sources
- Clip shapes: Traditional + Round (phone view dropped — the 9:16 phone output is the vertical output-crop tier); mirror; both persisted in the layout DB (schema v2) and toggled from the source chip
- Background removal = milestone 2 (ONNX Runtime + DirectML, MediaPipe Selfie Segmentation) — not in this build
- TFM bumped to
- Background / Image / Text — static sources positioned/scaled/opacity
- Chat box — rendered from the live chat poll (right panel is the same feed, raw)
- Scene compositing — per-scene source layering (z-order = sources list order, top-to-bottom back-to-front), preview rendered via D3DImage or MediaElement
- Branding flash — the topmost full-frame "made with ytLlive!" layer at ~25% opacity, ~1s on /
300s off (see Monetization in
ai.md), gated onBrandFlashEnabled+ live/recording. Lives in the preview compositor now (BrandFlashLayerinMainWindow.xamlCanvasGrid, driven byBrandFlashActive/BrandFlashTimerinMainViewModel); the encoder output renders the same layer, and v0.2 local recordings carry it too - Drag/drop placement & reorder — intuitive, visual (per design principle):
- Preview: click-drag a source in the center panel to reposition it; resize via handles
- Scenes list: drag rows to reorder scenes
- Sources list: drag rows to reorder sources (this is the z-order) — implemented