Files
ytLlive/ai.md
T
gramps 18a21010bb Scene compositor (TASK 4 ship step 1): encoder master-frame scaffolding
Software compositor that renders a scene into the encoder's master VideoFrame,
mirroring the XAML preview minus editing chrome: backdrop -> background ->
elements (UniformToFill cover-crop, round clip, mirror, opacity, border) ->
branding flash.

NOTE FOR USERS: this change shows NO difference in the app's UI — it is pure
backend scaffolding laying the groundwork for live video capture/streaming.
The preview you see is unchanged.

- Services/Compositor/: SceneCompositor (Render(scene, frameFor resolver,
  flashFrame, CompositorOptions)), CompositorOptions (source rect + output
  size; 16:9 full master, vertical 607x1080 -> 1080x1920), StretchMath (pure
  UniformToFill + bilinear), StaticPixelCache (asset bytes -> BGRA8 frame)
- Frame sources injected via Func<SceneElement, VideoFrame?> resolver, so the
  compositor is pure, WPF-free, and hermetic to test (D3D11 upgrade behind the
  same seam later)
- SceneElement.TryGetBorderColor public (shared hex parse), stale
  MainViewModel comment fixed, pre-existing CS1998 in YouTubeAuthServiceTests
  cleaned up
- Tests: SceneCompositorTests integration (full scene + vertical tier + flash)
  + StretchMath units, docs updated (72 tests passing, 0 warnings)
2026-08-10 10:15:58 -07:00

36 KiB
Raw Blame History

ytLlive — AI Guide

Memory map entry point. Conventions live in schema.md; task status and YouTube API research in TASKS.md; directory maps in each folder's index.md. Reading order: this file → TASKS.md<dir>/index.md → source.

Response style

No default "Plans & Pitfalls" / planning boilerplate. Respond directly and concisely: do the queued work, then report what changed and what's next. Skip feature pitch, step-by-step implementation plans, pros/cons tables, and "potential pitfalls" sections unless the user explicitly asks for a plan first. A short diff-style summary beats a proposal document every time.

No-Fluff Mode (on demand)

Invoke with "no-fluff mode" (or similar) when you want ruthless review instead of reassurance. In that mode:

  • Strip all polite pleasantries, emojis, transitions, and conversational padding.
  • Treat the user's input as a draft to be methodically deconstructed or strengthened — argue, correct, and sharpen rather than agree.
  • Give unvarnished truth, not reassurance.

This is an occasional, explicitly-invoked mode — never the default. The default response style above stays in effect unless invoked.

Run

# From WSL, ALWAYS use the Windows dotnet host — never Linux `dotnet` for this project:
"/mnt/c/Program Files/dotnet/dotnet.exe" build "C:\Users\gramp\Documents\Code\projects\ytLive\ytLive.csproj"
"/mnt/c/Program Files/dotnet/dotnet.exe" run

EnableWindowsTargeting=true in ytLive.csproj lets a cold restore work from WSL, but a Linux dotnet run/build re-downloads 100M+ of windowsdesktop.app.* packs into the Linux NuGet cache (which lacks them) over the slow 9p /mnt/c bridge — twice, because the WPF _wpftmp generated project triggers a second restore (203s observed). The Windows cache has the SQLite packages and the packs resolve from C:\Program Files\dotnet\packs, so the Windows host never re-downloads. Never use --no-restore right after an interrupted restore — the stale project.assets.json produces misleading NETSDK1064 "package not found" errors. Running requires Windows anyway.

Tests

xUnit in ytLive.Tests (net8.0-windows10.0.19041.0, matches the app TFM). Run on Windows — WSL can't run net8.0-windows tests:

dotnet.exe vstest "C:\...\ytLive.Tests\bin\Debug\net8.0-windows10.0.19041.0\ytLive.Tests.dll"

Good Dog Rule: ONE integration test per branch, ONE test per PR. Current: TokenStore DPAPI roundtrip/corrupt/missing/clear, OAuth exchange/refresh/ClearSession, CameraManager refcount + frame pump + failure handling (fakes for the WinRT seams), real-MainWindow round-clip interaction test, LayoutStore delete roundtrip, LayoutStore pre-round-rect-dims roundtrip, LayoutStore backdrop roundtrip, LayoutStore HasBackdrop roundtrip + v5→v6 non-Live backfill, ScreenCaptureManager refcount + shared-bitmap + coalescing (fake IScreenCaptureSource + a real background-STA Dispatcher), BackdropTests (EnsureBackdrop insert/idempotent/heal + HasBackdrop gate, IsLiveCapture, DisplaySource, INPC), SceneCatalogTests (the five canonical scenes, Live-only backdrop policy, EnforceBackdropPolicy), WebcamSafeguardTests (the per-scene size clamp incl. the Chat half-screen-area cap), SceneCompositorTests (the full-scene composite integration test: backdrop + round webcam + mirrored/bordered images + flash; the vertical tier 1080×1920 crop/scale), StretchMathTests (UniformToFill cover-crop + bilinear) — 72 passing.

Real-MainWindow tests MUST be hermetic (DB pollution bug)

The integration test boots a real MainWindowMainViewModel → real LayoutStore (%APPDATA%\ytLlive\ytLlive.db). Shutdown() on close saves the layout (full rewrite: DELETE all scenes/sources, re-insert), so any source a test adds would be persisted over the user's real ones — this happened and wiped the real webcam source (DeviceId replaced by the test's fake test-camera). Rule: a test that constructs MainWindow MUST first set MainViewModel.LayoutPathOverride to a temp DB path and reset it (plus SqliteConnection.ClearAllPools() + delete) in finally. The seam is internal static string? LayoutPathOverride (line ~529 in MainViewModel.cs), ytLive.csproj has InternalsVisibleTo("ytLive.Tests").

The layout DB is a full rewrite per save (delete all, re-insert from memory), so save/load round trips are exact: an element removed in the UI (RemoveElementscene.Elements.RemoveOnElementsChanged → debounced ScheduleSave, plus Shutdown on close) does not come back after reload (LayoutStorePersistenceTests guards this).

Architecture

C# / WPF (.NET 8) following MVVM:

Path Role
Models/ Plain data types — Scene, Source (incl. ClipShape, IsMirrored, VideoImageSource), QualityOption, StreamConfig, StreamHealth, YouTubeChannel, ChatMessage
ViewModels/ MainViewModel — exposes collections + commands for the UI; GoLiveViewModel, ReuseImageViewModel, CameraPickerViewModel
Services/ YouTube OAuth2, stream/broadcast management, live chat polling, LayoutStore (SQLite), webcam: VideoFrame seam + CameraDeviceInfo/ICameraEnumerator/ICameraFrameSource interfaces + MediaCaptureCameraEnumerator/MediaCaptureFrameSource (WinRT) + CameraManager, screen capture: IFullScreenDetector/Win32FullScreenDetector + IScreenCaptureSource/ScreenCaptureFrameSource (WinRT GraphicsCapture) + ScreenCaptureManager + ScreenCaptureSourceFactory + Direct3D11Helper/CaptureInterop (COM bridges)
Helpers/ ViewModelBase (INotifyPropertyChanged), RelayCommand, ImageCache, AppLog (file logger), FocusPreservingListBox, OAuthCredentials, TokenStore (DPAPI session persistence), visibility converters
Themes/ Controls.xaml — the single dark-theme source, merged once in App.xaml (see Themes/index.md)
MainWindow.xaml Dark theme; layout: top bar (controls), center (preview), left (scenes/sources), right (chat), bottom (health)

Key patterns

  • ViewModelBase.SetProperty<T>() for property change notifications
  • RelayCommand for all button actions; commands gate on state (e.g. Start only when Offline). Typed CommandParameters — no stringly-typed command tokens: menu items that pick a source type pass the enum value itself (CommandParameter="{x:Static models:SourceType.DisplayCapture}"), so a typo breaks the build instead of silently adding an Image; AddSource still falls back to Enum.TryParse<SourceType>(..., true) for safety. The webcam item is its own AddWebcamCommand (it greys out via CanAddWebcamToActiveScene and isn't a SourceType — webcams are WebcamSceneConfig, not Source rows)
  • Audio is KISS by rule — the whole of audio is one knob: desktop/game audio is automatic (WASAPI loopback from the default output at unity, zero UI — "it just is"); the mic is the creator's only audio control — sound meter + mute button + volume slider (MicVolume, defaults to 0.8) all sit together on the footer's top line, CENTERED beneath the preview panel. Meter: 288px, muted slate track (#3a3b52) with ruler graduations and muted yellow/red zone tints at 60%/80%; fill = green → yellow → red via MeterFillWidth/MeterBrush; the meter is a READ-ONLY realtime level display — it shows the live input level scaled by the volume (raising the volume moves ambient noise up the bar), NOT the volume setting: the fill is Math.Min(1, AudioLevel * MicVolume) (AudioLevel is fed by the audio mixer once capture lands, 0 with no input) and 0 while muted. While the volume slider is being dragged the bar previews the slider position (SetVolumeAdjusting, from PreviewMouseLeftButtonDown/Up + LostMouseCapture handlers) so the creator sees where they're setting it; on release it returns to the live level — with no input it bounces back to 0, exactly as it does today. Clicking the meter does nothing; clicking the MIC label opens the mic picker (OpenMicPickerCommand), and the picked voice source name (MicSourceName) is shown left-justified INSIDE the meter bar (FontSize 10, ellipsized to the bar) — the fill runs at 75% opacity so the text and the ruler markings stay visible through it. Mute (ToggleMicMuteCommand/MicMuted) is a plain clickable speaker icon (MicSpeaker_MouseLeftButtonUp code-behind handler — not a Button, Stretch="Uniform" so the glyph is never clipped) that swaps to a red do-not-symbol (slashed speaker) when muted. The slider and the speaker can never disagree: MicMuted is read-only, derived from MicVolume == 0 — sliding the volume off flips the speaker to muted (storing the prior level in _volumeBeforeMute), sliding it up from 0 clears the mute indicator (and the stored level); the speaker button just runs the volume to 0 or restores it (default 0.8 if unknown). Muting zeroes the meter; unmuting flashes the meter to the restored position for ~300ms (BeginVolumeFlash/EndVolumeFlash on a DispatcherTimer, cancelled if the slider is grabbed) before it returns to the live level. Line 2 of the footer holds everything else: stream stats (bitrate/fps/dropped/duration/health) on the left, quality dropdown + gear on the right. The slider is a slim dimensional style in Themes/Controls.xaml (gradient track, beveled green fill on a 5px pill, gloss-sphere thumb with drop shadow — deliberately NOT flat). No device pickers (never show device names — no "install a device you didn't know existed"), no filter stacks, no monitoring, no routing — OBS's confusion (dynamic mixer, unintuitive names, four required filters) is deliberately absent. A production-ready mic chain (high-pass → noise gate → compressor) will be applied invisibly in the mixer, unconfigurable. Capture runs only while live (privacy indicator stays off otherwise). Capture pipeline = IAudioSource seam + NAudio WasapiCapture/WasapiLoopbackCapture + AudioMixer (pending — the UI is in place now). The connected account's avatar/name shows in the top bar next to Start Stream (AccountAvatarUrl/AccountDisplayName via SyncConnectedAccount), so the creator always sees WHICH account will go live.
  • ViewModels are constructed in XAML (<vm:MainViewModel/> as DataContext)
  • Services are currently instantiated in MainViewModel's constructor — no DI container yet
  • Layout persists to SQLite (Microsoft.Data.Sqlite); scenes/sources/asset bytes stored in the DB, asset identity is a SHA-256 content hash (1:M reuse, no file paths — assets are always available)
  • Five-scene catalog (Models/SceneCatalog.cs): the product is exactly Starting/Live/BRB/Chat/Ending — work with less, never more (the escape hatch for "more" is OBS). Scenes are matched by name (SceneCatalog.Is, case-insensitive trim). Empty DBs seed all five; the scenes-header "+" (ShowAddScene/MissingScenes on MainViewModel) only appears while ≥1 canonical scene is missing and its menu lists only the missing ones, re-adding them by name (AddSceneCommand). Renaming a canonical scene makes it missing again; AddScene rejects non-canonical names.
  • Theming: all custom styles live in Themes/Controls.xaml, merged in App.xaml — never duplicate styles per-window (dialog duplicates were consolidated into this dictionary)
  • Resolution tiers (bottom bar): 1080p60@8 (default) → 1080p30@8 → 720p60@6 → 720p30@6 → Vertical 1080p60@8 (9:16, 1080×1920). The composition master frame is always 1920×1080 — a tier is an output rect + target resolution over that master, so source geometry is never rewritten (no rounding drift). 16:9 tiers use the full frame; the vertical tier uses a centered 607×1080 window and the preview dims the cropped side strips at 55% black with an accent outline (semi-crop — the cut area stays visible). A resolution badge in the preview corner shows the active tier; the bottom bar shows bitrate/FPS. A tooltip explains finding upload bandwidth — an in-app speed test was deliberately dropped (unreliable). The future encoder crops the master to the rect and scales to the tier's Width×Height
  • Crash diagnosis: AppLog writes startup checkpoints to %APPDATA%\ytLlive\startup.log; App.xaml.cs logs DispatcherUnhandledException/AppDomain.UnhandledException. When WPF won't run from WSL, this log is how you find the failure (it caught the MenuItemRole.Separator XAML crash and the ComboBox SelectionBoxItem bug)

Current limitations / TODOs

  • Helpers/OAuthCredentials.cs contains the real ClientId/ClientSecret. Auth is complete and the session persists via Windows DPAPI (Helpers/TokenStore.cs%APPDATA%\ytLlive\ytLlive.auth, CurrentUser scope), reloaded best-effort at startup with a proactive refresh of a near-expiry access token. Sign-in/Change Account lives inside the Start Stream dialog (two-state flow — no separate Connect button). A graceful End Livestream signs out: StopStream() clears the session + token, so the next go-live needs a fresh sign-in; a crash never runs End, so the token survives and the creator stays signed in. YouTubeAuthService takes an optional HttpClient + sessionChanged callback (test seam + save hook; services are still constructed in MainViewModel)
  • Scene/source/asset layout persists (SQLite, schema v6); the OAuth session persists (DPAPI); the paid-unlock state does not (yet — itch.io key verification pending)
  • YouTubeStreamService uses hardcoded 1080p/60fps and per-broadcast streams — must switch to the v3 variable reusable stream
  • Webcam capture is shipped (milestone 1); the live desktop/game backdrop is shipped (ship task #1); the output compositor (TASK 4 ship step 1) is the next build — full plan in TASKS.md; window capture (non-backdrop), the encoder + RTMP push, and audio capture follow it
  • StreamConfig defaults (TargetBitrate=6000, Resolution="1920x1080") are stale — the live dropdown drives StreamHealth.CurrentBitrate/FPS instead

Screen backdrop capture (TASK 3 ship task #1)

The backdrop is the live desktop/game capture as a permanent, non-deletable bottom layer rendered in every scene — the "Screen" source from the minimal set, done as content-swap instead of a normal draggable source.

  • Model (schema v6): Source.IsBackdrop (persisted) marks the one backdrop per scene; Source.CaptureKey (persisted) names the target — monitor:<n>, window:<hwnd>, or picker:<displayname>. The backdrop is a real Source of Type DisplayCapture, inserted first (MainViewModel.EnsureBackdrop(scene), internal static — runs on layout load + every AddScene, healing any scene missing one), fixed at X=0/Y=0/1920×1080, and excluded from drag/hit-test/remove/reorder (remove is guarded in RemoveElement; IsDraggableElement never matches live types; the element template sets IsHitTestVisible=false for backdrops; the row's remove button and "Remove Source" menu item are hidden). Scene.HasBackdrop (persisted, default off) is the Live-only policy flag — the backdrop belongs to the canonical Live scene alone (see SceneCatalog). EnsureBackdrop returns null for a flag-less scene, so Starting/BRB/Chat/Ending compose their own layers. The one-time v5→v6 backfill turns those four scenes off and drops their backdrop sources, and EnforceBackdropPolicy (internal static, runs after every load) re-normalizes the flag by scene name and strips any backdrop that lingers in a non-Live scene — the flag is owned by policy, never the user. There is no scene-list "Backdrop" checkbox anymore (the old ToggleSceneBackdropCommand is gone); "Change Capture…"/"Refresh Capture"/"Capture Display" only show in the Live scene's preview menu (CanChangeBackdrop). The static Background, if any, renders above the backdrop.
  • Detection (launch + focus only, no live session listener): Win32FullScreenDetector = GetForegroundWindow + DwmGetWindowAttribute(DWMWA_EXTENDED_FRAME_BOUNDS) + MonitorFromWindow + GetMonitorInfo; a window is full-screen when its frame covers all four monitor edges; own process is excluded; monitor index = EnumDisplayMonitors enumeration order (the same order ScreenCaptureSourceFactory uses to map index→HMONITOR via Win32FullScreenDetector.GetMonitorHandle). IFullScreenDetector also exposes GetDisplays() (DisplayInfo: index/name/resolution/bounds/IsPrimary, friendly name via EnumDisplayDevices) + PrimaryMonitorIndex() for the in-app "Capture Display" submenu. At launch ReacquireScreenCaptures keys the backdrop to the full-screen game's monitor (or the primary display — never assumed monitor 0). On Deactivated, NoteBackgroundWindow samples the foreground ~250ms later (so an alt-tab to a game lands before the app re-activates); on the next Activated, RefreshBackdropAutoCapture re-runs detection against that sample and re-designates. Null detection (our app, the desktop, a normal window) leaves the current capture alone.
  • Capture (WinRT GraphicsCapture): ScreenCaptureFrameSource creates a free-threaded Direct3D11CaptureFramePool (2 buffers, B8G8R8A8UIntNormalized) + GraphicsCaptureSession; frames → SoftwareBitmap.CreateCopyFromSurfaceAsync (alpha ignored) → VideoFrame (BGRA8), bytes read via WindowsRuntimeMarshal.TryGetDataUnsafe (the same CsWinRT-safe read the webcam path uses) — the IMemoryBufferByteAccess ComImport cast threw Invalid cast on every frame under CsWinRT, which flooded startup.log (~5 MB in a session) and burned CPU, so it is gone. Surfaces larger than the 1920×1080 master are downscaled bilinearly to the master (DownscaleBgra) before the copy, and per-frame conversion failures are logged at most once per 5 s (ErrorLogThrottle). DRM-protected content delivers black frames (OS limitation, documented). Frame pool pauses while the app is minimized — capture keeps running, the pool just stops delivering.
  • Ownership: ScreenCaptureManager mirrors CameraManager — refcounted by target key, one shared WriteableBitmap per key, dispatcher-coalesced latest-frame copies, PreviewBitmapChanged/CaptureFailed events, ReleaseAllAsync on re-designation. ScreenCaptureSourceFactory.Resolve(key) parses the key into a source; PickAsync() shows the OS GraphicsCapturePicker ("Change Capture…", owner window set via the IInitializeWithWindow ComImport) and returns a transient picker: key — a reload falls back to auto-detection.
  • CsWinRT projection gaps hand-rolled: Windows.Graphics.Direct3D11.Direct3D11Helper is not projected, so Direct3D11Helper P/Invokes d3d11.dll!D3D11CreateDevice (hardware, BGRA_SUPPORT, explicit 11.1-first feature array) → QI IDXGIDevice → the WinRT interop export CreateDirect3D11DeviceFromDXGIDeviceMarshalInterface<IDirect3DDevice>.FromAbi (one shared device per process). Do not switch back to the QI-for-IDirect3DDxgiInterfaceAccess trick: the raw D3D11 device no longer exposes that interface on newer Windows (verified E_NOINTERFACE on build 26200, hardware and WARP alike) while CreateDirect3D11DeviceFromDXGIDevice keeps working. IInitializeWithWindow are ComImports in CaptureInterop.cs. All WinRT projections were verified by reflection against the built Microsoft.Windows.SDK.NET.dll before writing the interop.
  • Known v1 limits: full-desktop captures are CPU-copied at native resolution (GPU downscale = encoder task); window capture (window:<hwnd>) and multi-monitor live re-targeting beyond the auto-detected game are behind the picker; picker-based captures don't survive reload.
  • GPU posture: same as webcam — CPU frames, WPF hardware-presents; D3DImage GPU compositing deferred to the encoder task.
  • Preview watermark: the "Preview" placeholder hides while a backdrop renders — ShowPreviewPlaceholder now also checks BackdropImage (raised on backdrop change), so a scene with live capture shows the feed instead of the "nothing here" label.

Webcam capture (TASK 3 milestone 1)

  • Seam-first: everything above the WinRT layer speaks only VideoFrame (normalized tightly-packed BGRA8) + CameraDeviceInfo/ICameraEnumerator/ICameraFrameSource interfaces. Tests inject fakes; screen capture and background removal later feed the same seam.
  • CPU-first: MediaCaptureInitializationSettings { MemoryPreference = Cpu, StreamingCaptureMode = Video, SharingMode = SharedReadOnly }, frames pulled via CreateFrameReaderAsync(colorSource, MediaEncodingSubtypes.Bgra8) — the pipeline does any format conversion, so every FrameArrived yields a ready BGRA8 SoftwareBitmap (bytes read via WindowsRuntimeMarshal.TryGetDataUnsafe, not marshalled copies).
  • Source pick, not first hit: the frame reader is bound to the first source that is VideoPreview (preferred) or VideoRecord, not blindly the first preview source. If a camera exposes neither, the failure names the device and the stream types it does expose. SharedReadOnly lets the capture coexist with other apps that share the camera.
  • Known failure: NVIDIA Broadcast — it opens the physical webcam exclusively, so InitializeAsync fails with "camera in use" (or, if init slips through, the device exposes no preview source). Fix: quit Broadcast while streaming, or pick its virtual "NVIDIA Broadcast" device from the picker and the app captures the processed feed. This is a real-device finding (Logitech VID_046D&PID_082D).
  • TFM: net8.0-windows10.0.19041.0 (app + tests) pulls the WinRT projection from the SDK reference packs — no NuGet package, no capability manifest (unpackaged desktop app works; the Windows privacy camera toggle still applies). EnableWindowsTargeting keeps WSL builds working.
  • One webcam, many scenes (schema v3): a singleton Webcam row holds the identity (Id/DeviceId/Name); each scene gets its own WebcamSceneConfig (position/size/clip/mirror/ border/IsVisible). Scene.Elements holds images (Source) and, at most once, the webcam (WebcamSceneConfig); Scene.WebcamConfig is the accessor. CameraManager refcounts capture sessions by DeviceId (a session starts at RefCount = 1; repeat acquire bumps it; the last release stops + disposes). "Add Webcam" ALWAYS opens the Windows camera picker — the creator is never silently handed the previous camera (which used to happen after deleting one scene's webcam while another scene still used it; that identity survived, so re-adding bypassed the choice). Picking a different camera than the current app-wide one swaps it everywhere via SwapWebcamIdentityAsync (the same path "Change Webcam…" uses), keeping the singleton honest; picking the same one just places the config. The Add Webcam menu greys out when the active scene already has a config (CanAddWebcamToActiveScene); showing a hidden webcam reuses the existing config (CanShowWebcamInActiveScene / empty-canvas right-click "Show Webcam" — which, on a config-less canvas, delegates to Add Webcam and so also picks). Removing the last webcam config anywhere clears the identity (_webcam = null), which also drops "Change Webcam…".
  • Round→rect restores the aspect (persisted, schema v4): SceneElement.ToggleClipShape() snapshots the rectangular Width/Height into public RectWidth/RectHeight before going Round and restores them when switching back — otherwise the Round resize lock (square) would leave a square behind. The rect dims are persisted (WebcamSceneConfig.RectWidth/RectHeight, nullable), so a reloaded Round webcam still restores its pre-Round aspect instead of staying square. A one-time HealLegacySquareRect (load only) widens a pre-v4 Traditional config that ended up square to 16:9 (keeps height; Round and explicit rect dims are untouched).
  • Device swap / layout reload: ChangeWebcamAsync (picker) and ReacquireWebcam (after load) release the old device with ReleaseAllAsync — a forced full drop that zeroes the refcount and stops the source regardless of how many scenes held it (the per-config count isn't known once the scenes are replaced) — then AcquireAsync the new device once per config.
  • Shared bitmap, coalesced updates: one WriteableBitmap per active camera, created on the UI thread at the device's frame size (first frame), forwarded to every WebcamSceneConfig.VideoImageSource via PreviewBitmapChanged. Frames arrive on a worker thread; CameraManager coalesces onto the dispatcher (at most one pending copy per session, at Render priority, always copying the latest frame) so a 60fps device never drowns the render thread.
  • Webcam added mid-session must get the live frames: PreviewBitmapChanged fires once (the first frame creates the shared bitmap); later frames only mutate that bitmap in place, so a config that didn't exist at first-frame time would never receive it — the empty/transparent container you'd see adding a webcam to Chat while Live already had the camera. CameraManager.GetPreviewBitmap(deviceId) exposes the current shared bitmap; AddWebcamToActiveSceneAsync assigns it to the new config right before AcquireAsync, and ReacquireWebcam re-propagates it to every config after a reload.
  • Clip/mirror/border: per-element ClipShape (Traditional rectangle / Round ellipse) + IsMirrored (ScaleX = -1) + the OSB-standard static border (BorderColor #RRGGBB or ""=none, BorderOpacity 01, BorderWidth 020, BorderAnimation None|Pulse|Chase|Rainbow|Shimmer|MarchingAnts|Glow| Electricity|Sparkles). Rendered in the preview DataTemplate; toggled from the element's right-click context menu (webcam menu: Change Webcam…, Border Effect submenu — all 9 items enabled, values persist, rendering stays static until the animation tier ships — Border Color, Opacity/Thickness sliders, Hide in this scene, Remove); persisted in the layout DB. The Add menu shows when no webcam exists; the empty preview canvas has its own Show Webcam entry.
    • The Round webcam is an Image Stretch="UniformToFill" with an EllipseGeometry clip (Center=0.5,0.5 RadiusX/Y=0.5), inside a Viewbox Stretch="Uniform" holding a 1x1 Grid, so it renders as a true circle (diameter = the shorter element dimension) instead of an oval stretched to the element rect — the traditional Image keeps UniformToFill over the full rect. The clip is geometry, not an ImageBrush: a brush re-rasterizes the frequently-updated WriteableBitmap per frame on the render thread, which is what made the live webcam crawl while round. The Round border is a centered Ellipse at Width/Height = RoundBorderSize.
    • Resizing locks to a square (_resizeAspect = 1) while ClipShape == Round.
    • Webcam size clamp: ClampWebcamToBounds(config, sceneName) (internal — test seam) enforces the max per dimension at resize + load and no less than 10% of the master (192×108). The cap is picked by canonical scene name: 50% per dimension (960×540) everywhere except the Chat scene, which may reach half the screen area (~1358×764 @16:9) so the viewer sees the creator better (MaxWebcamWidthFor/MaxWebcamHeightFor; a renamed Chat loses the bigger cap). RoundBorderSize follows the clamped height. WebcamSafeguardTests guards the clamp.
    • Hit-testing: a Grid without Background only hit-tests where its children draw, so clicks in the empty corners of a round clip fell through to Window_PreviewMouseLeftButtonDown and deselected the element — making the corner handle ungrabbable. The element Grid carries Background="Transparent" (whole rect draggable; the empty canvas Grid uses the same trick for right-click Show Webcam) and the SelectionOverlay (dashed border + corner dot) is IsHitTestVisible="False" so it never intercepts the click. Two things make the webcam menu work: (1) the ContextMenu pins its own DataContext to PlacementTarget.DataContext — a ContextMenu isn't in the visual tree, so without it the Click-handler DataContext: patterns (and the IsChecked/slider bindings) silently fail; (2) Themes/Controls.xaml ships a full dark MenuItem template — PART_Popup (submenu popups), a popup ItemsPresenter (the Border Opacity/Thickness sliders live in Items, so they render in a hover flyout), a checkmark column, and a arrow driven by HasItems. An earlier bare Border + Header template dropped all three: submenus never opened, sliders never rendered, checkmarks never showed — the menu looked dead even though the Click handlers were fine.
  • GPU posture: webcam frames are CPU (GPU-agnostic; WPF hardware-presents the preview anyway). Hardware encoders (NVENC/AMF/QSV) matter for the encoder task, not capture. D3DImage GPU compositing is deferred to the encoder task.
  • Background removal = milestone 2 — ONNX Runtime + DirectML (CPU fallback), MediaPipe Selfie Segmentation (Apache-2.0), wired into the same VideoFrame seam. Not part of milestone 1.

Scene compositor (TASK 4 ship step 1 — shipped 2026-08-10, plan in TASKS.md)

The encoder needs the master 1920×1080 frame without the preview's editing chrome (SelectionOverlay, DimRects, output-rect outline, badge, placeholder). WPF's RenderTargetBitmap is software-rendered and captures the visual tree including chrome, so the preview can't be captured — the output is a second, parallel software compositor over the VideoFrame (BGRA8) seam, and the XAML preview (MainWindow.xaml CanvasGrid + element DataTemplate) is the rendering contract it replicates. Two renderers must agree: geometry, UniformToFill cover-crop, round clip, mirror, border, z-order. Preview stays XAML (editing view); the compositor is the output view.

  • Render the active tier's output rect directly (CompositorOptions {SourceRectX/Y/W/H, OutputWidth, OutputHeight}, fed from MainViewModel.OutputRect*): 16:9 = full 1920×1080 1:1; the vertical 9:16 tier = composite the centered 607×1080 crop then bilinear-upscale to 1080×1920.
  • CPU posture: with FFmpeg as a subprocess the master crosses a CPU readback to the pipe every frame anyway, so GPU compositing buys little at this layer count (2-3 live layers; static layers pre-composite once into a cached base). GPU effort belongs to NVENC (the encoder), not composition; if composition grows (wipes, filters, many layers), a D3D11 compositor can replace this one behind the same seam — the CPU master buffer stays the contract.
  • Branding flash is composited by the output path too (it's on the live output, per Monetization), passed in as a pre-rendered VideoFrame? — the compositor core stays pure byte-math, no WPF. Likely a bundled asset rather than runtime text rendering (deterministic, no font/layout risk).
  • Frame sources are injected via a Func<SceneElement, VideoFrame?> resolver (SceneCompositor.Render(scene, frameFor, flashFrame, options)) — the caller maps each element to its frame (webcam → DeviceId, image → AssetId via StaticPixelCache, backdrop → CaptureKey), so the compositor is pure, WPF-free, and hermetic to test. The capture managers wire into that resolver in the encoder step, not the compositor step. The master buffer (the compositor's return value) is the seam a future D3D11 compositor would honor identically.

Design Principle

This software is so intuitive that even the most right-brained person can easily intuit and use it.

Apply this to every UI decision:

  • One-click go-live with working defaults
  • Prefilled YouTube defaults (RTMP URL, bitrate, resolution, latency)
  • Visual/drag-and-drop scene building over property panels
  • Every action produces a visible outcome — no dead ends

Monetization (design decision — the branding flash is the sword)

Free forever: all streams unlimited, no time caps, no subscription, no per-feature paywalls. The one paid line is a one-time unlock (delivered via itch.io — they handle hosting, payment, and key delivery; we never own a server or a key shop):

  • Free: a periodic full-frame branding flash — "made with ytLlive!" rendered big and centered at ~25% opacity for about one second (soft 250ms fade in/out), repeated every 300s, on the live output (and on v0.2 local recordings). Implemented as BrandFlashLayer in the preview compositor (MainWindow.xaml CanvasGrid) + BrandFlashTimer in MainViewModel — cadence 300s, first flash ~5s after go-live, only while live or recording. An always-on watermark can be cropped or covered; an intermittent full-frame flash can't be cropped and is impractical to edit around on a live feed.
  • Paid (one-time): branding flash removed (flips BrandFlashEnabled off) + Alerts (Super Chat / membership / subscribe pop-ins).

Deliberately rejected: always-on watermark (obscurable — replaced by the flash), hard stream-time cutoffs (the worst dead end — a stream dying mid-broadcast reads as broken, and YouTube streams routinely run 2-4 hours), soft-limit nagging, freemium tiers, and donation-only (relies on the kindness of strangers). Resolution/quality ceilings are deferred — that decision belongs to the resolution & streaming-constraints conversation, not monetization.

Auth gates Go Live, but not exploration

The app is fully usable without authentication: users can build scenes, add sources, compose previews, and audition the software with zero commitment. But going live requires authentication — it's the one capability gated behind YouTube sign-in. The sign-in should never pressure the user ("sign in (optional)", not a modal wall): the two-state top bar shows Start Stream (offline) / End Stream (live), and the Start Stream dialog hosts the account — a saved session appears as the default with "Change Account"; with none saved, a "Sign in to YouTube" button starts OAuth and the Start button stays disabled until signed in.

Account assumption (do not build an account setup flow)

Connecting uses Google OAuth ("Sign in with Google") to link an existing YouTube creator account. ytLlive never creates or sets up accounts — that is YouTube's job. If the creator has no YouTube channel, they go to YouTube first. This assumption is explicit and must never be silently replaced by an in-app account-creation step. Zero state = the Start Stream dialog's "Sign in to YouTube" button; going live is unreachable until an account is connected.

YouTube Live API — design constraints (do not violate)

These are the hard facts behind every decision. Full list in TASKS.md.

  • One-click go-live — never call transition(live). Insert the broadcast with enableAutoStart=true, enableAutoStop=true, enableMonitorStream=false, selfDeclaredMadeForKids=false, latencyPreference=low. The encoder starting brings YouTube live. enableMonitorStream=false is what lets us skip the testing stage.
  • Variable reusable streamliveStreams.insert once per channel with cdn.resolution=variable, cdn.frameRate=variable, isReusable=true; cache the ingestion URL + stream name and reuse for every broadcast. Any quality tier works without recreating the stream, and auto step-down is done by us dropping bitrate on the fly (zero API calls).
  • Quality is greyed out while live — resolution/frameRate/ingestionType are immutable after stream creation; editing title/description/privacy is fine at any time.
  • Report-by-exception health — poll liveStreams.list; render nothing on good/ok, surface a banner only on configurationIssues[] with warning/error severity. Bottom strip = YouTube logo
    • green/red connection dot (clickable → opens the dialog).
  • One dialog, three states — not connected / connected-offline (all editable) / live (title + description + visibility editable; quality + account greyed out). Both entry points (Start Stream button + bottom strip) open it; prefilled from saved session profile.
  • End stream — stop encoder → transition(complete), enableAutoStop as the safety net.
  • Encoder compliance — keyframes ≤ 4s (gopSizeLong), closed GOP, H.264, AAC/MP3 @ 44.1/48kHz, mono/stereo only. YouTube flags violations via health status.
  • Broadcast ID == Video ID — one ID tracks status, health, and the auto-created VOD (recordFromStart + enableDvr default true).