The bar now renders ONE surface from the FIRST decision the creator makes — Record or Stream — with each world being the whole bar. Creator directive: "I wrote the fucking thing and I still can't figure-out how to do stuff", "forget this one dog plan bullshit". OBS/streaming-tool reference for the one-surface paradigm: https://obsproject.com (single mode toggle + context actions) — cited per spin-guard habit; the go-live/test pattern follows CEV (https://cev-desktop.aSean.xyz) precedent. - Mode: single segmented REC|ON-AIR switch (SegmentToggle/SegmentLabel in Themes/Controls.xaml); radio-exclusive world selection, one tap to flip. - Record world: switch + Start Recording, zero YouTube identity. - Stream world: switch + Go Live + Test + account zone right (Sign In until connected, then avatar with Change Account/Logout context menu). - Test is a child of Stream: procs only stream-armed AND signed-in. - Running: one reality line "dot word elapsed" (REC/LIVE/TEST; green/red/ gold) + End; worlds + switch retire. - Gear moved up from bottom bar, ~3 wordmark letters past the brand, single click opens Settings/Bug/Feature/About menu (TASK 40 Unit B shipped early). - Fix folded in: BeginTestStream now arms OnAirPillOn explicitly (unarmed pump booted with zero encoder outputs -> "At least one output is required" forced-stop cascade from TASK 41's pipeline). - VM: world/reality props + RaiseTopBarModes() wired into StreamStatus, pill, IsRecording, IsTestStream, IsConnected setters. Doc: ai.md top-bar model, ViewModels/index.md, Controls/index.md, TASKS.md, HANDOFF.md, new TASKS/task-42-top-bar-redesign.md; task-40 note. Tests: TopBarModeTests.cs (new Good Dog), PillRadioTests + TestStreamTests gates updated. Build 0 warnings; full suite 314/315 — single abort is the pre-existing env-dependent RealMouseDrag test (Path of Exile 2 running)
152 KiB
ytLlive — AI Guide
Memory map entry point. Conventions live in
schema.md; task status and YouTube API research inTASKS.md; brand/marketing palette and launch strategy inMARCOM.md(gitignored — read it before any branding or marketing work); directory maps in each folder'sindex.md. Reading order: this file →TASKS.md→MARCOM.md→<dir>/index.md→ source.
Response style
No default "Plans & Pitfalls" / planning boilerplate. Respond directly and concisely: do the queued work, then report what changed and what's next. Skip feature pitch, step-by-step implementation plans, pros/cons tables, and "potential pitfalls" sections unless the user explicitly asks for a plan first. A short diff-style summary beats a proposal document every time.
Derivative work (spin guard mandated in AGENTS.md)
Nothing we build is novel — streaming/overlay/WebView2 problems were solved by OBS, CEV, and WPF ecosystems long before us. Two consequences:
- Proactive: new features start with a quick external scan, not head-first design (citation goes in the commit message).
- Objectively triggered spin guard: a second failed fix for the same symptom means STOP theorizing and research the established answer externally; there is no unique bug in this repo that the wider ecosystem hasn't hit. This project's hard lesson: the web-source bounding saga (2026-08) burned 9 commits rediscovering that a browser source is a fixed canvas — OBS keeps it at a stable page size, crops/hugs content via the box, and clips at the edge. The apparent "gap" that ended the saga was a widget's own CSS glow effect, not a bug.
Derived-solution rule (2026-08-29, driven by the image-shrink incident): when
you work out any one-off, reusable solution (recipe / workaround / how-to), write
it into MyMistakes.md → Recipes registry that same session, and grep it
before ever re-deriving. A solution recorded once ends the loop; an un-recorded
one guarantees the user hears it derived again (see AGENTS.md → Working rules → 🔬).
LAN infrastructure (192.168.50.x)
All LAN credentials live in CREDENTIALS.md (gitignored). Server IPs and roles:
| Host | IP | Role |
|---|---|---|
| corsair | .132 | Gaming/streaming primary — RTX 5070 Ti (this machine) |
| llamavault | .86 | ODROID-HC4, 12TB RAID5, media storage (NFS). SSH: ssh mshallop@192.168.50.86 |
| pivault | .158 | RPi5 8GB, 10.83TB RAID5 NAS |
| jarvis | .210 | AMD Ryzen 5 5600X, RX 6600 XT — AI inference, SearXNG, Docker registry, Immich |
| ultron | .108 | AMD Ryzen 7 7840HS — AI orchestrator, llama-server, Qdrant |
| gordito | .144 | Home Assistant, Alarmo, Zigbee |
| officerfriendly | .26 | Pi-hole DNS |
| octopi | .197 | OctoPrint (Prusa MK4) |
| BigBlinkyRouter | .1 | ASUS ROG GT-BE98 Pro, WiFi 7 |
Web infra (DO droplet): llamacasty.com + llamachile.tube at 143.244.176.131.
No-Fluff Mode (on demand)
Invoke with "no-fluff mode" (or similar) when you want ruthless review instead of reassurance. In that mode:
- Strip all polite pleasantries, emojis, transitions, and conversational padding.
- Treat the user's input as a draft to be methodically deconstructed or strengthened — argue, correct, and sharpen rather than agree.
- Give unvarnished truth, not reassurance.
This is an occasional, explicitly-invoked mode — never the default. The default response style above stays in effect unless invoked.
Brand
Brand palette and official assets live in MARCOM.md (gitignored — read it for hex values,
asset file names, and marketing copy). The brand red #e94560 is the primary accent used
throughout the UI.
Product name vs repo/assembly branding (2026-09-14)
The product is llamacasty (llamacasty.com, llamachile.tube, llgit.llamachile.tube). The
repo path, csproj AssemblyName/RootNamespace, DB/log paths (%APPDATA%\ytLlive\...),
and most internal namings are the legacy ytLive/ytLlive — a re-brand that never renamed
the internals. Treat the two as separate: external/user-facing language says "llamacasty",
code/assembly/repo names stay ytLive. If a full internal re-brand is ever done, this note
and the UI locator (Premium/wordmark) are the checklists; it is NOT a goal pre-1.0.
AI transparency
This project is built through AI-assisted pair-programming. All code is generated under the supervision of an experienced developer with decades of coding experience. Architecture decisions, product direction, and quality gates are human-owned; AI accelerates implementation. This is a testimonial to successful human-AI collaboration, not autonomous code generation.
Run
# From WSL, ALWAYS use the Windows dotnet host — never Linux `dotnet` for this project:
"/mnt/c/Program Files/dotnet/dotnet.exe" build "C:\Users\gramp\Documents\Code\projects\ytLive\ytLive.csproj"
"/mnt/c/Program Files/dotnet/dotnet.exe" run
Pre-commit gate (write it down — I forgot once): ALWAYS run
./scripts/verify.sh "<file1>" "<file2>" ... (declared scope) before committing. It forces a
clean build (0 warnings), the full test suite (passes only the 2 known pre-existing [FAIL]s), and
the scope check in one shot. An incremental build can report 0 Warning(s) while skipping
recompiles — that's how CS8601 slipped past on 2026-08-29. Never claim 0 warnings from anything but
verify.sh's clean build.
EnableWindowsTargeting=true in ytLive.csproj lets a cold restore work from WSL, but a Linux
dotnet run/build re-downloads 100M+ of windowsdesktop.app.* packs into the Linux NuGet cache
(which lacks them) over the slow 9p /mnt/c bridge — twice, because the WPF _wpftmp generated
project triggers a second restore (203s observed). The Windows cache has the SQLite packages and the
packs resolve from C:\Program Files\dotnet\packs, so the Windows host never re-downloads.
Never use --no-restore right after an interrupted restore — the stale project.assets.json
produces misleading NETSDK1064 "package not found" errors. Running requires Windows anyway.
Tests
xUnit in ytLive.Tests (net8.0-windows10.0.19041.0, matches the app TFM). Run on Windows —
WSL can't run net8.0-windows tests:
dotnet.exe vstest "C:\...\ytLive.Tests\bin\Debug\net8.0-windows10.0.19041.0\ytLive.Tests.dll"
Good Dog Rule: ONE integration test per change (no feature branches pre-1.0 — work lands on
main, see AGENTS.md). Current: TokenStore DPAPI
roundtrip/corrupt/missing/clear, OAuth exchange/refresh/ClearSession, CameraManager refcount +
frame pump + failure handling (fakes for the WinRT seams), real-MainWindow round-clip
interaction test, LayoutStore delete roundtrip, LayoutStore pre-round-rect-dims roundtrip,
LayoutStore background roundtrip, LayoutStore HasBackground roundtrip + v5→v6 non-Live backfill,
ScreenCaptureManager refcount + shared-bitmap + coalescing
(fake IScreenCaptureSource + a real background-STA Dispatcher), BackgroundTests, SceneCatalogTests,
the TASK 25 dirty-layout heal (BackgroundHealIntegrationTests), WebcamSafeguardTests,
SceneCompositorTests (full-scene composite + vertical tier), StretchMathTests, FfmpegLocatorTests,
FfmpegEncoderTests, FramePumpTests, the TASK 8/11/12 audio chain (AudioPipelineTests + MasterLimiter),
the Socials fediverse-heal roundtrip, AboutHubTests, NotificationAreaIntegrationTests (TASK 24),
GlobalHotkeyTests + HotkeyConfigTests (TASK 20), WebcamMenuGateTests (TASK 26), ChatLayerGateTests
(TASK 27), BroadcastPullOutTests (TASK 29), DefaultRecordFolder fallback (TASK 30), WebView2ManagerTests
(TASK 17), RecordingFileTests + OnAirSignTests (TASK 18), SessionTeardownTests +
WebcamOutputKeyTests + GameMeterHonestyTests (2026-09-01 recording-verification fixes) — ZERO
known failures as of 2026-09-01. AudioPipelineTests 25/25 green in 26ms —
the "known failing" Mix_HonorsProviderGains… and the class's notorious STANDALONE HANG shared
one root cause: TASK 22's _delayedMix (nullable, never initialized) was dereferenced
(delayed.Length on null) every live-mix tick — a swallowed NRE starved the pipe the tests
read and flooded startup.log. Test was right, code drifted from the map's own contract
(loopbackGain = GameAudioVolume — now honored; the "unity because loopback scales with the
endpoint" assumption was disproven by the creator's 20%-volume meter observation). The former
RoundClip "known failure" (stale-test layers: namescoped FindName + VisualTreeHelper.HitTest
where UIElement.InputHitTest models input — see MyMistakes) was fixed the same day.
Per-class runs through the Windows dotnet.exe host execute the RealApp/MainWindow suites fine;
only the FULL-suite run still hangs (WASAPI teardown, pre-existing) — and the whole-suite
"247 total" era count is stale; trust per-class results.
Reward-event capture (monetization awareness, see the Monetization section) will add its integration
tests here when it ships: one real chat-poll payload containing all seven reward event types →
assert the persisted canonical RewardEvent rows (type + amount/currency/tier/memberLevel/participant
ids) round-trip into SQLite.
Real-MainWindow tests MUST be hermetic (DB pollution bug)
The integration test boots a real MainWindow → MainViewModel → real LayoutStore
(%APPDATA%\ytLlive\ytLlive.db). Shutdown() on close saves the layout (full rewrite:
DELETE all scenes/sources, re-insert), so any source a test adds would be persisted over the
user's real ones — this happened and wiped the real webcam source (DeviceId replaced by the
test's fake test-camera). Rule: a test that constructs MainWindow MUST first set
MainViewModel.LayoutPathOverride to a temp DB path and reset it (plus
SqliteConnection.ClearAllPools() + delete) in finally. The seam is
internal static string? LayoutPathOverride (line ~529 in MainViewModel.cs),
ytLive.csproj has InternalsVisibleTo("ytLive.Tests").
The layout DB is a full rewrite per save (delete all, re-insert from memory), so
save/load round trips are exact: an element removed in the UI (RemoveElement →
scene.Elements.Remove → OnElementsChanged → debounced ScheduleSave, plus Shutdown on
close) does not come back after reload (LayoutStorePersistenceTests guards this).
Architecture
C# / WPF (.NET 8) following MVVM:
| Path | Role |
|---|---|
Models/ |
Plain data types — Scene, Source (incl. ClipShape, IsMirrored, VideoImageSource), QualityOption, StreamConfig, StreamHealth, YouTubeChannel, ChatMessage, Socials (SocialService enum + SocialEntry/SocialsConfig + SocialServiceIcons) — the social bar |
ViewModels/ |
MainViewModel — public partial class, one file per functional area (Scenes, Background, Webcam, Audio, Trax, Socials, Streaming, Chat, Overlays, Account, License, Recording — split complete, see ViewModels/index.md); Chat.cs is a thin delegating facade over Services/ChatOverlayLayer.cs (Commit G, first true decomposition); GoLiveViewModel, ReuseImageViewModel, CameraPickerViewModel, SocialsDialogViewModel |
Services/ |
YouTube OAuth2, stream/broadcast management, live chat polling, LayoutStore (SQLite), SocialValidator (ISocialValidator seam + HttpSocialValidator default), webcam: VideoFrame seam + CameraDeviceInfo/ICameraEnumerator/ICameraFrameSource interfaces + MediaCaptureCameraEnumerator/MediaCaptureFrameSource (WinRT) + CameraManager, screen capture: IFullScreenDetector/Win32FullScreenDetector + IScreenCaptureSource/ScreenCaptureFrameSource (WinRT GraphicsCapture) + ScreenCaptureManager + ScreenCaptureSourceFactory + Direct3D11Helper/CaptureInterop (COM bridges), media source: IMediaFrameSource + MediaVideoSource (spawns ffmpeg rawvideo BGRA decode) + MediaVideoSourceManager (refcount-by-path session owner) + pure RawVideoFrameReader + IDecodeProcess/FfmpegDecodeProcess process seam (binary-stdout mirror of IEncoderProcess; see "Media source"), compositor: SceneCompositor + CompositorOptions + pure StretchMath + StaticPixelCache (see "Scene compositor"), audio: IAudioSource seam + WasapiLoopbackAudioSource/WasapiMicAudioSource (NAudio WASAPI) + AudioMixer + pure AudioLevelMeter/WaveToFloat/VoiceFilterChain/LowShelfFilter/HighShelfFilter/NoiseGate/Compressor/AutoDucker/AudioRingBuffer/TinyResampler/AudioSyncDelay + MusicPlayer + IAudioPipeWriter/NamedPipeAudioWriter (see "Live audio capture"), encoder: IFfmpegEncoder/FfmpegEncoder + IEncoderProcess/FfmpegEncoderProcess + IFfmpegLocator/FfmpegLocator + pure FfmpegArgs/FfmpegProgressParser/FfmpegEncoderPicker + the FramePump frame producer (see "Live encoder" + "Live frame pipeline"), notifications: INotificationService seam (AppNotificationSeverity Info/Success/Warning/Error) + NotificationService (Notification.Wpf toasts, see "Toast notifications") |
Helpers/ |
ViewModelBase (INotifyPropertyChanged), RelayCommand, ImageCache, AppLog (file logger), FocusPreservingListBox, OAuthCredentials, TokenStore (DPAPI session persistence), BuildStamp (wordmark release counter #N, +1 per commit, generated by GenerateBuildStamp in ytLive.csproj via git rev-list; the per-build GUID now logs to startup.log only — 2026-09-04), visibility converters |
Themes/ |
Controls.xaml — the single dark-theme source, merged once in App.xaml (see Themes/index.md) |
MainWindow.xaml |
Dark theme; layout: top bar (controls), center (preview + live controls below), left (scenes/sources), right (chat), bottom (gear + stream stats + resolution) |
Key patterns
ViewModelBase.SetProperty<T>()for property change notificationsRelayCommandfor all button actions; commands gate on state (e.g. Start only when Offline). TypedCommandParameters — no stringly-typed command tokens: menu items that pick a source type pass the enum value itself (CommandParameter="{x:Static models:SourceType.DisplayCapture}"), so a typo breaks the build instead of silently adding an Image;AddSourcestill falls back toEnum.TryParse<SourceType>(..., true)for safety. The webcam item is its ownAddWebcamCommand(it greys out viaCanAddWebcamToActiveSceneand isn't aSourceType— webcams areWebcamSceneConfig, notSourcerows)- Audio is KISS by rule — the whole of audio is one knob: desktop/game audio is automatic (WASAPI loopback from the default output at unity, zero UI — "it just is"); the mic is the creator's only audio control — sound meter + mute button + volume slider (
MicVolume, defaults to 0.8) all sit together on the second line of the preview bottom row, BELOW the Socials+TRAX row, CENTERED beneath the preview panel. The TRAX button + Desktop Audio meter sit on the first line of that same row. Meter: 288px, muted slate track (#3a3b52) with ruler graduations and muted yellow/red zone tints at 60%/80%; fill = green → yellow → red viaMeterFillWidth/MeterBrush; the meter is a READ-ONLY realtime level display — it shows the live input level scaled by the volume (raising the volume moves ambient noise up the bar), NOT the volume setting: the fill isMath.Min(1, AudioLevelMeter.ToDisplay(AudioLevel) * MicVolume)—ToDisplaymaps the raw linear RMS onto a −60..0 dBFS display scale, because real speech sits around −40..−20 dBFS (0.01..0.1 linear) which would leave a flat scale dead (AudioLevelis fed by the audio mixer once capture lands, 0 with no input) and 0 while muted. While the volume slider is being dragged the bar previews the slider position (SetVolumeAdjusting, fromPreviewMouseLeftButtonDown/Up+LostMouseCapturehandlers) so the creator sees where they're setting it; on release it returns to the live level — with no input it bounces back to 0, exactly as it does today. Clicking the meter does nothing; clicking the MIC label opens the mic picker (OpenMicPickerCommand), and the picked voice source name (MicSourceName) is shown left-justified INSIDE the meter bar (FontSize 10, ellipsized to the bar) — the fill runs at 75% opacity so the text and the ruler markings stay visible through it. Mute (ToggleMicMuteCommand/MicMuted) is a plain clickable speaker icon (MicSpeaker_MouseLeftButtonUpcode-behind handler — not a Button,Stretch="Uniform"so the glyph is never clipped) that swaps to a red do-not-symbol (slashed speaker) when muted. The slider and the speaker can never disagree:MicMutedis read-only, derived fromMicVolume == 0— sliding the volume off flips the speaker to muted (storing the prior level in_volumeBeforeMute), sliding it up from 0 clears the mute indicator (and the stored level); the speaker button just runs the volume to 0 or restores it (default 0.8 if unknown). Muting zeroes the meter; unmuting flashes the meter to the restored position for ~300ms (BeginVolumeFlash/EndVolumeFlashon a DispatcherTimer, cancelled if the slider is grabbed) before it returns to the live level. Line 2 of the footer holds everything else: stream stats (bitrate/fps/dropped/duration/health) on the left, quality dropdown + gear on the right. The slider is a slim dimensional style inThemes/Controls.xaml(gradient track, beveled green fill on a 5px pill, gloss-sphere thumb with drop shadow — deliberately NOT flat). No device pickers (never show device names — no "install a device you didn't know existed"), no filter stacks, no monitoring, no routing — OBS's confusion (dynamic mixer, unintuitive names, four required filters) is deliberately absent. A production-ready mic chain (high-pass → noise gate → compressor) will be applied invisibly in the mixer, unconfigurable. Capture runs only while live (privacy indicator stays off otherwise). Capture pipeline =IAudioSourceseam + NAudioWasapiCapture/WasapiLoopbackCapture+AudioMixer(pending — the UI is in place now). Mic mute icon (2026-08-13): a second 16px clickable glyph — a microphone, red + slash when muted — sits between the meter and the speaker (both mutes adjacent, spacing between the icons) and reuses the sameMicSpeaker_MouseLeftButtonUp→ToggleMicMuteCommandhandler. REC / ON-AIR pills + signs (TASK 18, 2026-08-29): the top-center area is now two sliding pill toggles (intent) next to two status signs (reality). REC pill (RecordPillOn) is the local-recording intent — works signed-out; ON-AIR pill (OnAirPillOn) is the streaming intent — greyed/disabled untilIsConnected. The REC sign (RecDotBrush/RecTextBrush/RecDotOpacity) is dark-gray + dim offline and green-tinged, pulsing only while actually recording (IsRecording), not merely pill-on. The ON-AIR sign (OnAirBrush=#555offline,#22c55elive) is green while actually streaming (IsLive); thePRIVATEbadge (IsLivePrivate) still shows when the broadcast is private. Both sign labels share theStatusSignTextstyle (Themes/Controls.xaml) so REC and ON-AIR can't drift apart — status color lives on the dot only, never the label. The elapsed timer shows whileIsLive || IsRecording. The connected account's avatar/name shows in the top bar next to the primary button (AccountAvatarUrl/AccountDisplayNameviaSyncConnectedAccount), so the creator always sees WHICH account will go live. - ViewModels are constructed in XAML (
<vm:MainViewModel/>as DataContext) - True decomposition beats partial-shuffling (2026-08-31, Commit G):
MainViewModelis a ~4100-line god-object split into 17 partials — but partials are a myth of decomposition: every partial shares the same class, sameSetPropertystate, same collaborators viaScenes/StagedScene/IsLive/Source.VideoImageSource, so an AI fetching one partial still reconstructs the whole class. The only thing that actually improves retrieval is a boundary where a feature owns its own state + collaborators.Services/ChatOverlayLayer.csis the model (extracted fromMainViewModel.Chat.cs194→44): the layer owns the message buffer,ChatBoxRenderer, fade/mock timers, preview renders, and the liveRenderFrame; the VM keeps only the binding surface (ChatMessagesdelegates to_chatLayer.Messagesso XAML + LeftPanelCollectionChangedhold; the two computed gate props stay on the VM because theirOnPropertyChangedis raised from VM setters). Diagnostic — glue vs. component: before extracting, countOnPropertyChanged/SetPropertytouches (glue) and bound-property reads per partial. Zero/low-glue + a cohesive state blob (renderer/timers/buffer) = extract (chat). High glue over an already-extracted service (Audio, Webcam, Background, Scenes —AudioMixer/CameraManager/ScreenCaptureManager/ChatBoxRendereralready exist) = leave as VM glue; forcing it adds coupling, doesn't remove it. Don't manufacture seams to hit a line count — the line count is a guideline for context, not a design goal. - SceneGraph component + baked-crust compositor (TASK 31):
Services/SceneGraph.csowns the scene collection (Scenes— the ViewModel'sScenesproperty delegates to it) and the element mutation surface (AddElement/InsertElement/RemoveElement/MoveElement, each invalidating the bake cache) plus queries that were scattered LINQ (GetBackground/GetWebcam/GetChatBoxes/GetSplitPoint/IsStatic). Elements exposeElementKind Kind(Static= images/background art,Dynamic= webcam/live capture/chat/web). The split point is the index of the first dynamic element;SceneGraph.GetBakedBasebakes/caches all static layers below it (keyed by scene id + static element identities), andSceneCompositor.BakeStaticBase/CompositeLayers/theRender(.., staticBase, split)overload composite the dynamic/above-split layers per frame.FramePump.RenderSceneuses the optimized path when aSceneGraphis wired in (falls back to full render without one). Invariant: dynamic-only pixel changes never invalidate; static layout/opacity/visibility/asset changes do (viaInvalidateBakefrom the VM's element-property and background-heal paths). Defensive deviation from the TASK 31 spec:ChatOverlayLayerkeeps takingIEnumerable<Scene>instead of depending onGetChatBoxes()— it is deliberately decoupled from the graph (its doc comment says "without owning the scene graph"); and the static background helpers (EnsureBackground/NormalizeBackgrounds) stay on the ViewModel becauseBackgroundTests.csunit-testsMainViewModel.EnsureBackgrounddirectly. Queries + invalidation moved; helpers stayed. - Services are currently instantiated in MainViewModel's constructor — no DI container yet
- Layout persists to SQLite (
Microsoft.Data.Sqlite); scenes/sources/asset bytes stored in the DB, asset identity is a SHA-256 content hash (1:M reuse, no file paths — assets are always available). Loaded sources always deriveIsBackgroundfromType(OR'd with the persisted column, so legacy DisplayCapture backdrops keep their flag) — pre-derivation rows withIsBackground=0heal on load - Element z-order is a unified space, not per-table (2026-09-20):
SourceandWebcamSceneConfiglive in different tables, and a webcam can be interleaved between sources in a scene. Both tables'SortOrderis stamped from the SAME counter — the element's index withinscene.Elements— andLoadmerges the two tables' rows per scene by that shared z (tie-break sources-first, so legacy per-table rows read back stacked the way they were saved). Never re-introduce per-type sort counters + "append all Sources then all configs", or a webcam slotted mid-stack reverts on reload (creator-reproduced bug). - Five-scene catalog (
Models/SceneCatalog.cs): the product is exactly Starting/Live/BRB/Chat/Ending — work with less, never more (the escape hatch for "more" is OBS). Scenes are matched by name (SceneCatalog.Is, case-insensitive trim). Empty DBs seed all five; the scenes-header "+" (ShowAddScene/MissingScenesonMainViewModel) only appears while ≥1 canonical scene is missing and its menu lists only the missing ones, re-adding them by name (AddSceneCommand). Renaming a canonical scene makes it missing again;AddScenerejects non-canonical names. - Theming: all custom styles live in
Themes/Controls.xaml, merged inApp.xaml— never duplicate styles per-window (dialog duplicates were consolidated into this dictionary) - Resolution tiers: 1080p60@8 (default) → 1080p30@8 → 720p60@6 → 720p30@6 → Vertical 1080p60@8 (9:16, 1080×1920). The composition master frame is always 1920×1080 — a tier is an output rect + target resolution over that master, so source geometry is never rewritten (no rounding drift). 16:9 tiers use the full frame; the vertical tier uses a centered 607×1080 window and the preview dims the cropped side strips at 55% black with an accent outline (semi-crop — the cut area stays visible). A resolution badge in the preview corner shows the active tier. Bottom bar: gear icon far left, stream stats (bitrate/FPS/dropped/duration) centered under preview, resolution dropdown far right. A tooltip explains finding upload bandwidth — an in-app speed test was deliberately dropped (unreliable). The future encoder crops the master to the rect and scales to the tier's Width×Height
- Crash diagnosis:
AppLogwrites startup checkpoints to%APPDATA%\ytLlive\startup.log;App.xaml.cslogsDispatcherUnhandledException/AppDomain.UnhandledException. When WPF won't run from WSL, this log is how you find the failure (it caught theMenuItemRole.SeparatorXAML crash and the ComboBox SelectionBoxItem bug)
⚠️ Known issue: screens/layers settings audit — RETIRED INTO THE GOLD PASS (2026-08-22, dispositioned 2026-09-01)
The AI hallucinated through multiple commits that night on background/scene property placement, repeatedly misreading the spec and introducing bugs. Specifically: moved properties to the wrong superclass level, used new instead of override breaking WPF bindings, removed working code, and added UI elements that weren't spec'd. The audit was never run since. The creator ruled (2026-09-01): it is a going-gold requirement, not session debt — the fine-tooth-comb pass (every screen's Background layer properties, pill toggle persistence, context menu visibility, "Add Layer"/"(+)" menu items) is a named checklist item under TASK 36 (gold pass) and this landmine is closed here.
Current limitations / TODOs
Helpers/OAuthCredentials.cscontains the real ClientId/ClientSecret. Auth is complete and the session persists via Windows DPAPI (Helpers/TokenStore.cs→%APPDATA%\ytLlive\ytLlive.auth, CurrentUser scope), reloaded best-effort at startup with a proactive refresh of a near-expiry access token. Sign-in/Change Account lives inside the Start Stream dialog (two-state flow — no separate Connect button). Sign-out is explicit only (Logout / Change Account →SignOutYouTubeAsync):StopStream()does NOT clear the session — TASK 18's 2026-08-29 reversal ("stopping a recording leaves the creator signed in") — the older "graceful End signs out" line here was stale and is corrected against the code (2026-09-01).YouTubeAuthServicetakes an optionalHttpClient+sessionChangedcallback (test seam + save hook; services are still constructed inMainViewModel)- Scene/source/asset layout + the social bar persist (SQLite, schema v8); the OAuth session persists (DPAPI); the paid-unlock state persists (LayoutStore Settings table —
LicenseKey/LicenseValidatedAt/IsPremium, 14-day offline grace) YouTubeStreamServicemanages the variable reusable stream (shipped 2026-08-16):GetOrCreateReusableStreamAsynclistsliveStreams?mine=trueand reuses the existingcdn.isReusablestream, creating it only on first use (resolution=variable,frameRate=variable); the stream is cached viaLayoutStore(SaveReusableStream/LoadReusableStream, Settings table) and bound at broadcast insert (boundStreamId). Health (shipped 2026-08-16):GetStreamHealthAsync(streamId)pollsliveStreams?part=statusforhealthStatus+configurationIssues[]→StreamHealth; banner decision in pureStreamHealthReporter- Webcam capture is shipped (milestone 1); the live desktop/game backdrop is shipped (ship task #1); the output compositor (TASK 4 ship step 1) is SHIPPED, the FFmpeg locator (TASK 4 ship step 2) is SHIPPED, the encoder + RTMP push (TASK 4 ship step 3) is SHIPPED, WASAPI audio capture (TASK 4 ship step 4) is SHIPPED, frame-pipeline wiring (TASK 4 ship step 5) is SHIPPED, health stats (TASK 4 ship step 6) is SHIPPED, one-click go-live + private-only enforcement (TASK 4 ship step 7) is SHIPPED — TASK 4 (RTMP Ingest) is fully done; full plan in
TASKS.md StreamConfigdefaults (TargetBitrate=6000,Resolution="1920x1080") are stale — the live dropdown drivesStreamHealth.CurrentBitrate/FPSinstead- v1 task queue (2026-08-19): tasks 19-23 added for v1 feature completeness; TASK 3.18 (chat box) shipped; TASK 16 (infinity display) shipped; TASK 10 (Polar billing) shipped — steps 1-7 (Velopack bootstrap, PolarLicenseService, license entry UI, LayoutStore entitlement, BrandFlash toggle, startup re-validation, PremiumUrl wired); Velopack update URL pending:
- TASK 10: Polar billing — license key entry, offline entitlement, brand flash gating, Velopack auto-updates
- TASK 19/23 merged: Control Surface UX — director's control room, transitions (Cut/Fade/Move), thumbnails above central monitor, edit mode offline only, right panel eliminated
- TASK 20: Hotkeys (global keyboard shortcuts) — the single biggest UX gap
- Broadcast metadata pull-out + launch geometry — shipped 2026-08-24 ("Text" tab on the Live screen, see its section below)
- TASK 21: Media source (video file playback) — starting soon videos, BRB loops
- TASK 22: Audio sync offset — lip-sync correction for USB mics/capture cards
- These were identified through competitive analysis and are table-stakes for any streaming software in 2025-2026. Full details in
TASKS.md.
Control Surface UX (TASK 19/23 — shipped 2026-08-24, verified by code survey)
The app is a director's control surface, not an editor. This replaces the OBS-style scene list with a TV-control-room metaphor.
Layout: Two-column. Left panel = one panel, two states. Center = thumbnails above central monitor.
- Left panel: When offline (edit mode) → LAYERS + PROPERTIES (same as current, minus SCREENS). When live/recording → YouTube Live Chat (entire panel replaced, never coexists with layers/properties). Mode is implicit (derived from
StreamStatus), no toggle button. - Thumbnails: Five scene previews above the central monitor, 16:9, drag-reorderable. Click = stage (offline) or transition (live). Active scene gets brand-red glow border.
- Central monitor: Shows staged scene (offline, editable in edit mode) or live output (live, read-only). During transitions: compositor-blended animation.
- Right panel: Eliminated. Chat moved to left panel (live mode).
- Edit gating:
IsEditMode=StreamStatus == Offline && !IsRecording. When live/recording: all drag/resize, context menus, right-click property adjustments disabled. Gun safety — no accidental edits mid-stream.
Scene reference split: ActiveScene → StagedScene (central monitor target, editable offline) + LiveScene (encoder output). Go Live sets LiveScene = StagedScene. Thumbnail click sets StagedScene (offline) or fires transition (live).
Transitions: TransitionService (new file) — Cut/Fade/Move with configurable duration (default 300ms). SceneCompositor.RenderBlend() blends two scenes' frames. FramePump feeds LiveScene to encoder; during transitions, feeds blended frames. Go Live always = Cut (clean start).
Thumbnails: Scene.Snapshot property. SceneCompositor.Render() at 320×180 → WriteableBitmap. Only staged scene has live backdrop render; other 4 = static snapshots.
Thumbnails: Scene.Snapshot property. RefreshSnapshotsAsync fills minis from each scene's background art (LoadBackgroundImage); minis never render live captures — while the Live screen is staged its mini shows the green placeholder, unstaged it shows live-background.jpg (real-time surface = center monitor only; user decision 2026-08-23).
Global hotkeys (TASK 20 — step 1 shipped 2026-08-24)
Services/GlobalHotkeys.cs:HotkeyIdenum (9 actions),IHotkeyRegistrarseam,Win32HotkeyRegistrar(RegisterHotKey, no modifiers),GlobalHotkeyManager. Fixed defaults: F1-F5 canonical scenes, F6 start/end stream, F7 mic mute, F8 desktop-audio mute, F9 TRAX play/pause.MainWindow.OnSourceInitializedcreates the manager and forwardsActivated→MainViewModel.HandleHotkey; windowCloseddetaches (unregister all).MainViewModel.HandleHotkeymaps ids toTransitionToScene/start-end commands (honoringCanExecute)/mute toggles/TraxLeftClick.- Tests:
RegistrarOverridestatic seam mirrorsLayoutPathOverride. Landmine: forcing a WPF window's handle withWindowInteropHelper.Handledoes NOT raiseSourceInitializedand returns Zero pre-show — useEnsureHandle()in tests; it both creates the HWND and fires the event.
Broadcast metadata pull-out + launch geometry (2026-08-24)
The "Text" tab (always visible, 2026-08-24 revision): a white 30px tab with rotated YouTube-red "Text" sits on the right edge of the preview, vertically centered — shown regardless of login/stream state (creator decision; original IsLive gating removed same day). Click toggles the drawer (ToggleDrawerCommand); any left-click outside the form also collapses it (Window_PreviewMouseLeftButtonDown checks TextPullOutHost ancestry), as does the ✕. The drawer is a 320px dark panel over the preview's right edge (200ms CubicEase on Width, BroadcastForm.IsDrawerOpen drives it). The Update button still greys until CurrentBroadcastId exists.
Fields — the always-editable snippet/status set (⚠️ creator premise inverted: contentDetails fields like latency/DVR/embed are editable ONLY in created/ready, NOT while live; they are deliberately absent):
- Title / Description / Tags (comma-separated string →
snippet.tags[]) / Visibility ComboBox (private/unlisted/public) / Made-for-Kids CheckBox — all live-editable pre- and post-launch - Scheduled Start is read-only ("set at Go Live")
Plumbing: Models/BroadcastMetadata POCO; Services/LiveBroadcastFormViewModel (owns field state, persists every edit to Settings keys Broadcast.* immediately via Func<LayoutStore> seam — survives layout re-pointing; owns drawer state; UpdateBroadcastCommand gated by CurrentBroadcastId != null && !IsUpdating). YouTubeStreamService.UpdateBroadcast PUTs liveBroadcasts?part=snippet,status and must echo scheduledStartTime — update replaces the whole snippet part, omitting it clears the schedule. Go Live prefills the dialog from the form (BroadcastForm.Title/Description) and calls CaptureGoLive(title, description, now) on confirm. MainViewModel.CurrentBroadcastId exposes the private _currentBroadcastId; notified on create/end.
The old Default Stream Title/Description (un-persisted, App Settings overlay) are DELETED — the form owns those fields end-to-end; Go Live prefill routes through it.
Launch geometry: default 1920×1040, MinWidth 1366, MinHeight 768, WindowStartupLocation=Manual. Window size/position persist on close via RestoreBounds (+ WindowState Normal/Maximized; Minimized treated as Normal) into Settings keys Window.*; restored in the MainWindow ctor BEFORE show, clamped to the minimums and the primary SystemParameters.WorkArea (a position saved on a since-disconnected secondary monitor falls back on-screen). Multi-monitor restore is not supported yet.
Test seam note: YouTubeAuthService.SetSession() is public — tests inject a channel with future TokenExpiry so EnsureToken() passes offline; pair with an HttpClient(new StubHandler()).
Screen backdrop capture (TASK 3 ship task #1) — model reworked by TASK 25
The Background is a locked per-screen layer (TASK 25): exactly one per canonical
scene, always named "Background", pinned at index 0, never addable/removable/reorderable,
absent from the (+) menu. Flavor is decided by scene: Live = DisplayCapture (the real-time
desktop/game surface with Show Desktop / monitor switch / fullscreen-game detect, falling back
to live-background.jpg); Starting/BRB/Chat/Ending = static Background art layers.
MainViewModel.NormalizeBackgrounds(scenes) (internal static) heals any loaded layout to this
model on every load: keeps the correctly-flavored row (or converts a survivor in place — note
Source.Type's setter derives IsBackground, so conversions must re-assert the flag), drops
duplicates/stale rows, seeds missing ones via CreateBackground(name), renames, moves to index 0;
non-canonical scenes get HasBackground=false and lose any backgrounds. HealBackgrounds()
(instance) runs Normalize + stamps default art (StampDefaultBackgroundAsset reads
Assets/{scene}-background.jpg pack resources through AddAsset; custom Browse art wins).
This replaces the old five-seeder cluster (Seed{Starting,Brb,Ending,Chat}Background,
SeedLiveBackgroundAsset, EnsureDefaultBackground) and EnforceBackgroundPolicy.
- Model (schema v6):
Source.IsBackground(persisted) marks the one Background per scene;Source.CaptureKey(persisted) names the target —monitor:<n>,window:<hwnd>, orpicker:<displayname>. The Live row is a realSourceofType DisplayCapture, inserted first (MainViewModel.EnsureBackground(scene), internal static — also used by theStagedScenesetter so a scene switch can never expose a missing Background), fixed at X=0/Y=0/1920×1080, and excluded from drag/hit-test/remove/reorder (remove is guarded inRemoveElement; the element template setsIsHitTestVisible=falsefor backgrounds; the row's remove button and "Remove Source" menu item are hidden).Scene.HasBackground(persisted) is true for all five canonical scenes (seeSceneCatalog.HasBackground). Capture controls — "Change Capture…"/"Refresh Capture"/"Capture Display"/"Show Desktop" — exist only while the Live scene is staged (CanChangeBackground= staged scene is Live; the preview CanvasGrid menu binds it directly, the layer-row menu MultiBindings it with the row's ownIsBackgroundviaHelpers/AllTrueToVisibilityConverter). Non-Live screens keep the properties panel pill ("Use default background") + Browse for custom art. - Settings hygiene:
LayoutStore.Savepurges orphanedBackgroundUseDefault_{id}/BackgroundPath_{id}Settings keys whose id is no longer a live Source row (TASK 25 heal drops duplicates; their keys must not accumulate). - Layering invariant (regression lesson, 2026-08-23):
ActiveBackgroundImagerenders aboveBackgroundImage, so it is fed by static-art rows only (Type == Background). The capture-flavored Live row must never populate it or it permanently covers the desktop/game capture; Live's canvas showsDisplaySource(capture, falling back to its AssetId art). Flavor-blindIsBackgroundlookups are for persistence/heal/minis — not this overlay. - Detection (event-driven):
Win32FullScreenDetectorhooksSetWinEventHook(EVENT_SYSTEM_FOREGROUND)— fires on the UI thread whenever the foreground window changes. The callback runs the sameGetForegroundWindow+DwmGetWindowAttribute(DWMWA_EXTENDED_FRAME_BOUNDS)+MonitorFromWindow+GetMonitorInfocheck; a window is full-screen when its frame covers all four monitor edges; own process is excluded; monitor index =EnumDisplayMonitorsenumeration order (the same orderScreenCaptureSourceFactoryuses to map index→HMONITOR viaWin32FullScreenDetector.GetMonitorHandle).IFullScreenDetectoralso exposesGetDisplays()(DisplayInfo: index/name/resolution/bounds/IsPrimary, friendly name viaEnumDisplayDevices) +PrimaryMonitorIndex()for the in-app "Capture Display" submenu +FullscreenMonitorChangedevent +StartWatching()/StopWatching(). At launchReacquireScreenCapturesheals backgrounds (HealBackgrounds()), then keys the Live background to the full-screen game's monitor; when no game is detected,ResolveAutoCaptureKey()returnsnull— staleCaptureKeyvalues are cleared, no capture session is started, andSource.DisplaySourcefalls back to_imageSource(the static asset on the backdrop element).StartWatching()is called at the end ofReacquireScreenCaptures.OnFullscreenMonitorChangedauto-redesignates when a fullscreen game appears on a different monitor; null (no fullscreen / own window) is ignored — the existing capture keeps running so loading screens / transitions don't break the stream.RefreshBackdropAutoCapture(bound to "Refresh Capture" context menu +Activatedhandler) reads the current foreground directly as a manual fallback. Deduplication:_lastReportedMonitorprevents redundant redesignates from rapid foreground changes. - Capture (WinRT GraphicsCapture):
ScreenCaptureFrameSourcecreates a free-threadedDirect3D11CaptureFramePool(2 buffers,B8G8R8A8UIntNormalized) +GraphicsCaptureSession; frames →SoftwareBitmap.CreateCopyFromSurfaceAsync(alpha ignored) →VideoFrame(BGRA8), bytes read viaWindowsRuntimeMarshal.TryGetDataUnsafe(the same CsWinRT-safe read the webcam path uses) — theIMemoryBufferByteAccessComImport cast threwInvalid caston every frame under CsWinRT, which floodedstartup.log(~5 MB in a session) and burned CPU, so it is gone. Surfaces larger than the 1920×1080 master are downscaled to the master (DownscaleBgra, integer 8.8 fixed-point bilinear — slice 16: the same two-stage math asSceneCompositor.Bilinear; the previous double-per-pixel version was ~30-45ms quiet / ~150ms under 240Hz-HDR load and froze the desktop layer ~90% of a take). Conversions run as up to MaxConcurrentConversions (3) overlapping readbacks (slice 17: the OS readback, not the downscale, is the ~47ms wall — see Slice 17) with a monotonic LatestFrame publish gate (MonotonicGate: a slow OLDER completion can never overwrite a newer frame), and are spaced by a 10ms floor (MinConvertInterval): the monitor delivers at the 240Hz DWM cadence (~4.2ms), far too fast for the ~60/s the pump can use, so the extra arrivals are dropped (skip busy/skip cadencetelemetry). Hand-out buffers come from a reuse-distance ring (FrameRingBuffer, depth 8, redLine 4): a buffer is only rewritten ≥4 rents after its last hand-out, else a fresh one is allocated — a frame a consumer still holds (session.LatestFramesurvives conversions; the dispatcher preview copy lags) can never be read-while-overwritten (the 1742 tear). Rolling telemetry (frames/s, conv avg/max ms, drops, ring allocs) is logged tostartup.logevery 2s while converting. Per-frame conversion failures are logged at most once per 5 s (ErrorLogThrottle). DRM-protected content delivers black frames (OS limitation, documented). Frame pool pauses while the app is minimized — capture keeps running, the pool just stops delivering. - Ownership:
ScreenCaptureManagermirrorsCameraManager— refcounted by target key, one sharedWriteableBitmapper key, dispatcher-coalesced latest-frame copies,PreviewBitmapChanged/CaptureFailedevents,ReleaseAllAsyncon re-designation.ScreenCaptureSourceFactory.Resolve(key)parses the key into a source;PickAsync()shows the OSGraphicsCapturePicker("Change Capture…", owner window set via theIInitializeWithWindowComImport) and returns a transientpicker:key — a reload falls back to auto-detection. - "Capture Window…" — in-app window pin (TASK 38, 2026-09-21): the Live backdrop context menus
(PreviewPane canvas + layer-row) offer a submenu listing visible top-level windows of other processes
(
Win32WindowEnumerator, anEnumWindowspass filtered likeWin32FullScreenDetector: visible, non-cloaked viaDWMWA_CLOAKED, non-empty rect, other-process, titled). Picking one runsRedesignateBackgroundAsync("window:0x{hwnd:X}")— the same full-bleed layer-0 capture path as the game/desktop, keyed straight intoScreenCaptureSourceFactory(window:already resolved there). Session-scoped, never persisted:window:(andpicker:) keys areIsTransientCaptureKey—ReacquireScreenCapturesstrips them before auto-detection on every load, because an HWND can be recycled to an unrelated window after restart (resurrecting a dead pin could capture the wrong window). Pin wins while alive:OnFullscreenMonitorChangedskips re-designation whileIsAliveWindowPin(current.CaptureKey)is true — the auto game/desktop detector never steals an explicit pin. No dead ends: if the pinned window's capture session dies mid-session,OnScreenCaptureFailed(transient key only) re-runsReacquireScreenCaptures→ auto-fallback (fullscreen game → desktop → static art). Seam:IWindowEnumerator+WindowInfo(Label = "Title — Process"),WindowEnumeratorOverridestatic test seam mirrorsCameraEnumeratorOverride. - CsWinRT projection gaps hand-rolled:
Windows.Graphics.Direct3D11.Direct3D11Helperis not projected, soDirect3D11HelperP/Invokesd3d11.dll!D3D11CreateDevice(hardware, BGRA_SUPPORT, explicit 11.1-first feature array) → QIIDXGIDevice→ the WinRT interop exportCreateDirect3D11DeviceFromDXGIDevice→MarshalInterface<IDirect3DDevice>.FromAbi(one shared device per process). Do not switch back to the QI-for-IDirect3DDxgiInterfaceAccesstrick: the raw D3D11 device no longer exposes that interface on newer Windows (verified E_NOINTERFACE on build 26200, hardware and WARP alike) whileCreateDirect3D11DeviceFromDXGIDevicekeeps working.IInitializeWithWindoware ComImports inCaptureInterop.cs. All WinRT projections were verified by reflection against the builtMicrosoft.Windows.SDK.NET.dllbefore writing the interop. - Known v1 limits: full-desktop captures are CPU-copied at native resolution (GPU downscale = encoder task); multi-monitor live re-targeting beyond the auto-detected game is behind the picker; picker- and window-pin captures don't survive reload (session-scoped by design — see the "Capture Window…" bullet).
- Focus-loss capture lag (known OS limit, NOT an in-app throttle — deferred): when the app window loses
focus the preview visibly slows (mouse-movement lag). Nothing in the repo checks focus to slow capture —
the only focus hooks (
FullscreenMonitorChangedevent +RefreshBackdropAutoCapture) re-target the backdrop, never pace it. The lag is inherited:Windows.Graphics.Captureis content-driven and DWM/OS-throttled while the app is in the background (worse under a full-screen game on 24H2/26100), GPU readback (CreateCopyFromSurfaceAsync) contends with the foreground game, the source's one-in-flight_framePendinggate drops frames during stalls, and the manager'sDispatcherPriority.Rendercopies only run as fast as WPF presents the window. Slice 16 shrank the per-conversion stall (fast downscale + 10ms cadence floor) but the OS-level delivery throttle under focus loss remains. Recorded 2026-08-13; no mitigation attempted yet (deferred by user decision). - GPU posture: same as webcam — CPU frames, WPF hardware-presents; D3DImage GPU compositing deferred to the encoder task.
- Preview watermark: the "Preview" placeholder hides while a background capture renders —
ShowPreviewPlaceholderalso checksBackgroundImage(raised on change), so a scene with live capture shows the feed instead of the "nothing here" label.
Webcam capture (TASK 3 milestone 1)
- Seam-first: everything above the WinRT layer speaks only
VideoFrame(normalized tightly-packed BGRA8) +CameraDeviceInfo/ICameraEnumerator/ICameraFrameSourceinterfaces. Tests inject fakes; screen capture and background removal later feed the same seam. - CPU-first:
MediaCaptureInitializationSettings { MemoryPreference = Cpu, StreamingCaptureMode = Video, SharingMode = SharedReadOnly }, frames pulled viaCreateFrameReaderAsync(colorSource, MediaEncodingSubtypes.Bgra8)— the pipeline does any format conversion, so everyFrameArrivedyields a ready BGRA8SoftwareBitmap(bytes read viaWindowsRuntimeMarshal.TryGetDataUnsafe, not marshalled copies). - Reader-output subtype ladder (2026-09-15 webcam-take fix): the reader is created per-candidate
by
ReaderSubtypeCandidates(activeSubtype)— Bgra8 for uncompressed cameras (fast path), NV12 then source-default for cameras sitting in MJPG. A camera another app left in MJPG mode cannot be read as Bgra8 by aMediaFrameReader(StartAsync→OutputFormatNotSupported; the MJPG pipeline exposes NV12, not BGRA) — exactly what a browser grabbing the webcam produces. When the app can't re-negotiate (ourSetMediaStreamPropertiesAsyncrefuses with "file in use / CaptureMode is SharedReadOnly" under msedge + NVIDIA Broadcast contention), an MJPG-active reader otherwise aborts the whole acquisition. Each candidate is allocated AND started under its OWN catch — WinRT answers an unsupported subtype with a THROW (E_INVALIDARG), not a status; one rejected format must never abort the ladder. Failures degrade to the next candidate; rejections are logged + collected into the final error; only the outer catch nets device-level errors (seeMyMistakes.md→ WINRT RESOURCE-ALLOCATION RECIPE). Anything not Bgra8 is converted inOnFrameArrived, which already handled non-BGRA software bitmaps. - Source pick, not first hit: the frame reader is bound to the first source that is
VideoPreview(preferred) orVideoRecord, not blindly the first preview source. If a camera exposes neither, the failure names the device and the stream types it does expose.SharedReadOnlylets the capture coexist with other apps that share the camera. - Known failure: NVIDIA Broadcast — it opens the physical webcam exclusively, so
InitializeAsyncfails with "camera in use" (or, if init slips through, the device exposes no preview source). Fix: quit Broadcast while streaming, or pick its virtual "NVIDIA Broadcast" device from the picker and the app captures the processed feed. This is a real-device finding (LogitechVID_046D&PID_082D). - TFM:
net8.0-windows10.0.19041.0(app + tests) pulls the WinRT projection from the SDK reference packs — no NuGet package, no capability manifest (unpackaged desktop app works; the Windows privacy camera toggle still applies).EnableWindowsTargetingkeeps WSL builds working. - One webcam, many scenes (schema v3) — app-level default (2026-09-17): a singleton
Webcamrow holds the identity (Id/DeviceId/Name); each scene gets its ownWebcamSceneConfig(position/size/clip/mirror/border/IsVisible).Scene.Elementsholds images (Source) and, at most once, the webcam (WebcamSceneConfig);Scene.WebcamConfigis the accessor.CameraManagerrefcounts capture sessions byDeviceId(a session starts atRefCount = 1; repeat acquire bumps it; the last release stops + disposes). The webcam is an app-level resource — ONE selection (the app default), usable in every scene that can host it. "Add Webcam" places the existing default DIRECTLY (no picker); the Windows camera picker runs only for the initial selection (_webcam == null). Choosing a different camera in the picker swaps it app-wide viaSwapWebcamIdentityAsync(same path "Change Webcam…" uses). Per-scene gate, not app-wide (TASK 26 superseded by creator directive 2026-09-17):CanAddWebcam=StagedScene is { WebcamConfig: null } && IsWebcamAttainable— a scene holding its own placement greys the row (one webcam per stream), but ANOTHER scene holding one does not, so Chat's camera never greys Live. Attainable means a live lock, not just a saved identity:IsWebcamAttainable=_webcam != null && CameraManager.IsRunning(_webcam.DeviceId)— an identity whose device was unplugged, or whose session can't start, leaves the row greyed withWebcamAddToolTip= "No webcam is currently available…" (identity alone no longer suffices). The max-1 reason gets the gentle "a second camera means you've graduated to OBS" line. Removing a scene's placement keeps the identity (it's the app default;_webcamis never nulled by removal) — Add stays offered for the same camera, because the app holds its own base lock (below). - Startup resource lifecycle (2026-09-17): the webcam is a resource the app allocates and
locks, validated at startup — this answers "why is the WebCam row greyed" from inside the app
instead of a DB spelunk.
MainViewModel.ValidateWebcamResourceStartupAsync(fired fire-and-forget right afterLoadLayout, stored asWebcamStartupValidationTaskso tests await it) enumerates the OS once and tri-states on the count: 0 → app runs on, layer inactive, no alarm; 1 → attemptCameraManager.AcquireAsync(device)as an app-wide lock — on failure a persistent red alert (WebcamLockAlert, bottom of the Layers panel with a Retry button) re-polls every 5 s (_webcamLockPollTimer) until a lock succeeds, and any first real frame also clears it; ≥2 → deliberately no auto-lock — selection belongs to the App Settings dialog (gear) queued — TASK 40 Unit A (App Settings round, plan saved 2026-09-22). The single-camera branch always acquires (it no longer skips when configs already hold the session): that acquire is the app's own BASE ref, so removing every scene's placement leavesRefCount = 1and the session alive — the gate stays satisfied and the Web Cam row stays offered, i.e. the app default outlives the scenes that render it. Acquire on a running session just bumps the refcount.CameraManager.IsRunning(deviceId)= a session exists (started OR still starting — a mid-start session is not "locked yet", it's in flight), and a rolled-back (failed) session leaves the dictionary so the pass can retry. After a successful single-cam lock, the pass adopts the camera as the app default if no identity exists yet (fresh layout) — the Web Cam layer is then offered in Live/Chat immediately (CanAddWebcamtrue); an already-loaded identity is never overwritten. Test seams mirrorLayoutPathOverride:CameraEnumeratorOverride/CameraFrameSourceFactoryOverrideletWebcamStartupResourceTestsdrive the probe without real hardware (0-cam no-alarm, 1-cam locked → identity adopted, lock-fail → alert → clears on retry). Attribution correction: the "NVIDIA Broadcast opens the webcam exclusively" failure below is a suspect-list claim —CameraConflictProbereads only running process names, no device handles; contention symptoms fit shared-mode/bandwidth just as well (seeMyMistakes.md). The YouTube Chat layer follows the same one-per-layout rule (TASK 27):CanAddYouTubeChatgreys out the (+) item while any scene carries a ChatBox source (raised on staging + elements change;AddSourcerefuses a second as defense in depth). Chat layers created before the "YouTube Chat" default still carried the legacy name "Chat" (commit65641d8era) — LoadLayout heals exactly that un-renamed default to "YouTube Chat"; creator renames are untouched. - Round→rect restores the aspect (persisted, schema v4):
SceneElement.ToggleClipShape()snapshots the rectangular Width/Height into publicRectWidth/RectHeightbefore going Round and restores them when switching back — otherwise the Round resize lock (square) would leave a square behind. The rect dims are persisted (WebcamSceneConfig.RectWidth/RectHeight, nullable), so a reloaded Round webcam still restores its pre-Round aspect instead of staying square. A one-timeHealLegacySquareRect(load only) widens a pre-v4Traditionalconfig that ended up square to 16:9 (keeps height; Round and explicit rect dims are untouched). - Device swap / layout reload:
ChangeWebcamAsync(picker) andReacquireWebcam(after load) release the old device withReleaseAllAsync— a forced full drop that zeroes the refcount and stops the source regardless of how many scenes held it (the per-config count isn't known once the scenes are replaced) — thenAcquireAsyncthe new device once per config. - Shared bitmap, coalesced updates: one
WriteableBitmapper active camera, created on the UI thread at the device's frame size (first frame), forwarded to everyWebcamSceneConfig.VideoImageSourceviaPreviewBitmapChanged. Frames arrive on a worker thread;CameraManagercoalesces onto the dispatcher (at most one pending copy per session, atRenderpriority, always copying the latest frame) so a 60fps device never drowns the render thread. - Webcam added mid-session must get the live frames:
PreviewBitmapChangedfires once (the first frame creates the shared bitmap); later frames only mutate that bitmap in place, so a config that didn't exist at first-frame time would never receive it — the empty/transparent container you'd see adding a webcam to Chat while Live already had the camera.CameraManager.GetPreviewBitmap(deviceId)exposes the current shared bitmap;AddWebcamToActiveSceneAsyncassigns it to the new config right beforeAcquireAsync, andReacquireWebcamre-propagates it to every config after a reload. - Clip/mirror/border: per-element
ClipShape(Traditional rectangle / Round ellipse) +IsMirrored(ScaleX = -1) + the OSB-standard static border (BorderColor#RRGGBBor""=none,BorderOpacity0–1,BorderWidth0–20,BorderAnimationNone|Pulse|Chase|Rainbow|Shimmer|MarchingAnts|Glow| Electricity|Sparkles). Rendered in the preview DataTemplate; toggled from the element's right-click context menu (webcam menu: Change Webcam…, Border Effect submenu — all 9 items enabled, values persist, rendering stays static until the animation tier ships — Border Color, Opacity/Thickness sliders, Hide in this scene, Remove); persisted in the layout DB. The Add menu shows when no webcam exists; the empty preview canvas has its own Show Webcam entry.- The Round webcam is an
Image Stretch="UniformToFill"with anEllipseGeometryclip (Center=0.5,0.5RadiusX/Y=0.5), inside aViewbox Stretch="Uniform"holding a1x1Grid, so it renders as a true circle (diameter = the shorter element dimension) instead of an oval stretched to the element rect — the traditionalImagekeepsUniformToFillover the full rect. The clip is geometry, not anImageBrush: a brush re-rasterizes the frequently-updatedWriteableBitmapper frame on the render thread, which is what made the live webcam crawl while round. The Round border is a centeredEllipseatWidth/Height = RoundBorderSize. - Resizing locks to a square (
_resizeAspect = 1) whileClipShape == Round. - Webcam size clamp:
ClampWebcamToBounds(config, sceneName)(internal — test seam) enforces the max per dimension at resize + load and no less than 10% of the master (192×108). The cap is picked by canonical scene name: 50% per dimension (960×540) everywhere except the Chat scene, which may reach half the screen area (~1358×764 @16:9) so the viewer sees the creator better (MaxWebcamWidthFor/MaxWebcamHeightFor; a renamed Chat loses the bigger cap).RoundBorderSizefollows the clamped height.WebcamSafeguardTestsguards the clamp. - Hit-testing: a
GridwithoutBackgroundonly hit-tests where its children draw, so clicks in the empty corners of a round clip fell through toWindow_PreviewMouseLeftButtonDownand deselected the element — making the corner handle ungrabbable. The element Grid carriesBackground="Transparent"(whole rect draggable; the empty canvas Grid uses the same trick for right-click Show Webcam) and theSelectionOverlay(dashed border + corner dot) isIsHitTestVisible="False"so it never intercepts the click. Two things make the webcam menu work: (1) theContextMenupins its ownDataContexttoPlacementTarget.DataContext— aContextMenuisn't in the visual tree, so without it the Click-handlerDataContext:patterns (and the IsChecked/slider bindings) silently fail; (2)Themes/Controls.xamlships a full darkMenuItemtemplate —PART_Popup(submenu popups), a popupItemsPresenter(the Border Opacity/Thickness sliders live in Items, so they render in a hover flyout), a✓checkmark column, and a›arrow driven byHasItems. An earlier bareBorder + Headertemplate dropped all three: submenus never opened, sliders never rendered, checkmarks never showed — the menu looked dead even though the Click handlers were fine.
- The Round webcam is an
- GPU posture: webcam frames are CPU (GPU-agnostic; WPF hardware-presents the preview anyway). Hardware encoders (NVENC/AMF/QSV) matter for the encoder task, not capture. D3DImage GPU compositing is deferred to the encoder task.
- Background removal = milestone 2 — ONNX Runtime + DirectML (CPU fallback), MediaPipe Selfie
Segmentation (Apache-2.0), wired into the same
VideoFrameseam. Not part of milestone 1.
Scene compositor (TASK 4 ship step 1 — shipped 2026-08-10, plan in TASKS.md)
The encoder needs the master 1920×1080 frame without the preview's editing chrome (SelectionOverlay,
DimRects, output-rect outline, badge, placeholder). WPF's RenderTargetBitmap is software-rendered and
captures the visual tree including chrome, so the preview can't be captured — the output is a second,
parallel software compositor over the VideoFrame (BGRA8) seam, and the XAML preview
(MainWindow.xaml CanvasGrid + element DataTemplate) is the rendering contract it replicates. Two
renderers must agree: geometry, UniformToFill cover-crop, round clip, mirror, border, z-order. Preview
stays XAML (editing view); the compositor is the output view.
- Render the active tier's output rect directly (
CompositorOptions {SourceRectX/Y/W/H, OutputWidth, OutputHeight}, fed fromMainViewModel.OutputRect*): 16:9 = full 1920×1080 1:1; the vertical 9:16 tier = composite the centered 607×1080 crop then bilinear-upscale to 1080×1920. - CPU posture: with FFmpeg as a subprocess the master crosses a CPU readback to the pipe every frame anyway, so GPU compositing buys little at this layer count (2-3 live layers; static layers pre-composite once into a cached base). GPU effort belongs to NVENC (the encoder), not composition; if composition grows (wipes, filters, many layers), a D3D11 compositor can replace this one behind the same seam — the CPU master buffer stays the contract.
- Branding flash is composited by the output path too (it's on the live output, per Monetization),
passed in as a pre-rendered
VideoFrame?— the compositor core stays pure byte-math, no WPF. Likely a bundled asset rather than runtime text rendering (deterministic, no font/layout risk). - Frame sources are injected via a
Func<SceneElement, VideoFrame?>resolver (SceneCompositor.Render(scene, frameFor, flashFrame, options)) — the caller maps each element to its frame (webcam →DeviceId, image →AssetIdviaStaticPixelCache, backdrop →CaptureKey), so the compositor is pure, WPF-free, and hermetic to test. The capture managers wire into that resolver in the encoder step, not the compositor step. The master buffer (the compositor's return value) is the seam a future D3D11 compositor would honor identically.
FFmpeg locator (TASK 4 ship step 2 — shipped 2026-08-10, plan in TASKS.md)
The encoder's one external dependency is ffmpeg.exe; it's never shipped in the repo. IFfmpegLocator
resolves an absolute path on demand: PATH probe first (the user's own install wins — their choice,
their responsibility), then the cache (%APPDATA%\ytLlive\tools\ffmpeg.exe), then a pinned BtbN
LGPL-shared win64 zip (~75 MB) from which ffmpeg.exe and the libav*.dll family are extracted
(staged temp-write + move so a crash never corrupts the cache; Windows resolves the DLLs from the exe's
own directory). BtbN LGPL-shared (not gyan.dev, not static): it drops GPL-only libx264/x265 while keeping
NVENC/QSV/AMF + libopenh264 + native AAC, and dynamic linking means LGPL compliance is "license text +
source offer" with no static-relink (§6) material — see the Licensing guardrails below. The pin is a
dated autobuild tag (immutable); BtbN retention keeps the last 14 daily + each month-end for 2 years, so
a cold cache can outlive the pin → the seam throws a clear, logged error (recoverable; the pin is one
const). Constructor-injected search dirs / tools dir / downloader (Func<string, CancellationToken, Task<byte[]>>) keep it hermetic: tests fake the network with a real in-memory zip. Constructed in the
encoder step (not yet — this PR ships the seam + impl + tests only).
Live encoder + RTMP push (TASK 4 ship step 3 — shipped 2026-08-12, plan in TASKS.md)
The encoder is a thin orchestrator over ffmpeg.exe — no H.264/AAC code in the app. It spawns the
subprocess (path from IFfmpegLocator), feeds raw BGRA master frames into stdin, and parses -stats
stderr lines into StreamHealth (bitrate/FPS/duration, dropped-from-frame-count). FfmpegEncoder
(IFfmpegEncoder seam) holds: StartAsync (locate → probe -encoders → spawn → stderr loop →
drain loop), SubmitFrameAsync (slice 10: bounded-queue ENQUEUE — never a pipe write; the drain task
owns stdin writes), StopAsync (flush queue → stdin EOF → ffmpeg
finalizes + exits by itself; a 10s watchdog kills it), Dispose (force-kill + wait), and the
HealthUpdated/ProcessFailed events. Pattern: the encoder never touches Process — it drives the
IEncoderProcess seam (FfmpegEncoderProcess wraps the real Process, redirected stdin/stdout/stderr
- exit control); a
Func<IEncoderProcess>factory + the locator are constructor-injected, so the integration test fakes the whole subprocess (probe + encoder) with a Channel-backedTextReaderwhoseComplete()is EOF (null), never aChannelClosedException.
Decisions (locked): args are pure (FfmpegArgs.Build, no string building in the encoder):
-f rawvideo -pix_fmt bgra -video_size WxH -framerate FPS -i pipe:0 (NO -re — the FramePump
is the pacer since slice 9, 2026-09-10; -re added a second, fighting clock on the rawvideo demux)
- a real audio input — the
mixer writes IEEE-float stereo to a Windows named pipe (
-f f32le -ar 48000 -ac 2 -i \\.\pipe\ytllive_audio, name viaEncoderOptions.AudioPipeName; replaced the old-f lavfi -i anullsrcsilence in the TASK 8 audio milestone) + explicit-map 0:v -map 1:a+-c:v <enc> -b:v K -maxrate K -bufsize 2K+-g fps×4 -keyint_min fps×4 -sc_threshold 0 -bf 0 -pix_fmt yuv420p(≤4s keyframes, closed GOP, H.264 compliance) +-c:a aac -ar 48000 -ac 2 -f flv <rtmpUrl>. Record output (TASK 18):FfmpegArgs.Buildemits one self-contained block per output, each its own-map 0:v -map 1:a+ codec tags (AddVideoTagshelper). Stream block =-f flv <rtmpUrl>; record block =-f mp4 <RecordPath>.EncoderOptions.StreamEnabled/RecordEnabled/RecordPathgate each block, so the engine runs record-only (no RTMP) or stream-only — stream+record simultaneously is OUT by creator ruling (2026-09-01): the VOD is already the copy, and dual-encoding drags mid-range chassis and degrades BOTH outputs ("we're not them"). The old "cheap on NVENC" claim was an unverified assumption; the UI constraint is landed (2026-09-20): the REC/ON-AIR pills are radio-exclusive — arming one clears the other — while the dual-block machinery stays generic.FfmpegEncoder.StartAsyncthrows unless at least one output is enabled. Encoder choice is probed from the binary's-encoderslisting (FfmpegEncoderPicker, pure): NVENC → QSV → AMF → OpenH264 fallback, never libx264 (GPL; see Licensing).EncoderOptions.VideoEncoderforces one and skips the probe.EncoderOptionsalso carries W×H/FPS/bitrate from the quality tier andGopSize= FPS×4. Constructed byMainViewModelsince ship step 5 (new FfmpegEncoder(new FfmpegLocator())), driven by theFramePumpbelow.
Local recording (TASK 18 — shipped 2026-08-29, VM + top-bar UX; plan in TASKS.md)
Recording is independent from streaming and needs no YouTube sign-in. A single segmented
REC|ON-AIR switch in the top bar picks the mode (redesigned 2026-09-22 from the old two pills):
REC (local file, works signed-out) / ON-AIR (streaming; armable signed-out — intent may
light while offline, 2026-09-20: a greyed pill was the dead end the guard shipped to forbid). The switch
selection is intent; IsRecording/IsLive are reality — the reality line (dot + word + elapsed) only
lights when a session is actually running.
- State model (
MainViewModel): pillsRecordPillOn/OnAirPillOnstay the routing model (radio-exclusive — arming one clears the other, record-OR-live 2026-09-01; the ON-AIR setter stays armable signed-out (2026-09-20: intent must be able to light while offline)). Top bar redesign (2026-09-22): the bar is ONE surface rendered by the FIRST decision — Record or Stream — and each world is the whole bar (ShowIdleWorld/ShowRecordAction/ShowStreamActions, wired byRaiseTopBarModes()). The old sign-lights + two pill toggles + status dot + PRIVATE/TEST chips are gone; the mode is one segmented REC|ON-AIR switch (SegmentToggle/SegmentLabelstyles), and "going back" is one tap on the other segment. Idle worlds: Record = switch + "Start Recording" (PrimaryStartButtonLabelnames the armed world) and no YouTube identity at all; Stream = switch- "Go Live" + Test (Test is a child of Stream — visible only stream-armed AND signed-in) plus the
account zone on the right (
ShowSignInButton= stream world signed-out;ShowAvatarButton= the avatar, right-click Change Account/Logout). Running retires the switch and worlds; one reality line● word elapsed+ End replace them (RunningVisible/RunningText/RunningDotBrush— TEST/LIVE/REC, gold/red/green). The account status dot is gone — the avatar/Sign In pair carries connectedness.ShowPrimaryStartButtonremains the always-idle face.StopStreamclears both pills (see "Stop ends everything; failures roll back" below).
- "Go Live" + Test (Test is a child of Stream — visible only stream-armed AND signed-in) plus the
account zone on the right (
- Filenames (pure
Services/RecordingFile): auto-namety-<yyyyMMdd>-<HHmm start>-0000.mp4at start; rename-on-stop toty-…-<hh2mm2 actual length>.mp4(FinalizeRecordingAsyncafter the pump stops & the file closes), numeric-2/-3suffix on collision (UniquePath). Length comes from_liveElapsed, which the session timer walks whileIsLive || IsRecording. - Folder: default
%APPDATA%\ytLlive\recordings\, user-overridable viaChooseRecordFolderCommand(OpenFolderDialog), persisted throughLayoutStore.Load/SaveRecordFolder(RecordFolderkey). - Explicit sign-out only:
StopStreamno longer clears the session/token. Sign out via Logout / Change Account. Stopping a recording leaves the creator signed in. - Stop ends everything; failures roll back (2026-09-01):
StopStreamalso clears both intent pills (RecordPillOn/OnAirPillOn) — a lit pill with no session behind it is a lie. Every frame-pump death and every go-live prep failure now runs the fullStopStreamteardown instead of leaving a limbo (StreamStatus.ErrorwithIsRecording=trueover a dead encoder — the first-launch zombie recording). Toast copy distinguishes "Recording stopped" from "stream pipeline stopped";AudioMixer.StopLiveis idempotent-safe for rollbacks that never reached StartLive. Seams:OnFramePumpFailed+IsRecordingsetter made internal (test-only, InternalsVisibleTo). Test:SessionTeardownTests. EncoderOptions.StreamEnabled/RecordEnabled/RecordPathgate outputs; record+stream is one ffmpeg with two output blocks (the dual-block machinery is unreachable from the UI since the pills are radio-exclusive). Rest of the engine (FramePump/encoder) is unchanged —BuildEncoderOptionsalways returns an options (flags from the pills +_activeRecordPath), so record-only reaches the pump.
Live audio capture (TASK 4 ship step 4 — shipped 2026-08-12; game audio bar 2026-08-13; TASK 8 audio milestone: real stream audio + filters + duck + TRAX, shipped 2026-08-14, plan in TASKS.md)
Capture runs for the app's lifetime and is KISS by rule: desktop/game audio is automatic (WASAPI
loopback), the mic is the creator's only audio control (meter/mute/volume already shipped). The whole
layer sits behind an IAudioSource seam (Services/Audio/: Start/Stop/Started/SampleReady/
Failed, IDisposable) so the app never touches NAudio directly and tests inject hermetic fakes (no
devices, no timers).
- Sources (NAudio 2.2.1 —
NAudio.Wasapi+NAudio.WinMM(forWaveOutEvent), MIT — item 9 inTHIRD-PARTY-NOTICES.txt):WasapiLoopbackAudioSource=WasapiLoopbackCaptureon the default render device;WasapiMicAudioSource=WasapiCapturewith the device resolved byFriendlyNamematchingMicSourceName(the app only persists the DisplayName), falling back to the default capture endpoint. Both run at the device's own mix format — NAudio 2.2.1 exposes no overridableGetDefaultMixFormatonWasapiCapture, so there is no forced 48 kHz path; the mixer'sTinyResamplernormalizes any rate/channel layout to 48 kHz stereo (this replaced the earlier "force IEEE-float 48k" plan — the resampler IS the design). Mic device resolution re-reads the name providerFunc<string?>at eachStart, so a mic picked mid-session takes effect immediately (the mixer restarts the mic on pick). Both sources raiseStartedonce their capture loop actually begins — the mixer turns that intoMicConnected. AudioMixerowns both sources;StartMicCaptureAsyncstarts the mixer once at startup andShutdowndisposes it — NOT go-live — so the meters preview live. Mic samples: downmix to mono →TinyResampler→VoiceFilterChainper sample (bassLowShelfFilter120 Hz +4 dB → trebleHighShelfFilter8 kHz +3 dB →NoiseGate(threshold 0.005, hysteresis 0.5, attack 0.5, release 0.0005) →Compressor(threshold 0.5, ratio 4:1); all pure TDF2 DSP) → ring buffer → the post-filterAudioLevelMeter(RMS, 0.2 exponential smoothing) →MicLevelChanged→ marshalled to the UI thread →AudioLevel. Loopback samples: resample → ring buffer → second meter →LoopbackLevelChanged→ the game bar'sGameAudioLevel. The raw linear level maps to the meter's display scale byAudioLevelMeter.ToDisplay(−60..0 dBFS → 0..1 with +10 dB input amplification). Mic connection state surfaces as events:MicConnected(sourceStarted),MicFailed(sourceFailed), andRestartMic()re-resolves + restarts just the mic (loopback keeps running). Failures log viaAppLog; a mic failure zeroes the meter, a loopback failure never kills the mic. MeterPushis unconditional (a?.on the event would skip the meter update when nothing is subscribed yet).- Go-live audio (TASK 8):
BeginGoLive→StartLive(pipeName)(idempotent) creates the named pipe server (NamedPipeAudioWriter, behind theIAudioPipeWriterseam) and a live loop task; everymixInterval(default 10 ms)FillAndMixreads a chunk from each ring buffer (silence-fills underruns), computes the post-filter mic RMS →AutoDucker(threshold 0.02, duck ×0.25 / −12 dB, attack 0.05, release 0.005, always on), applies the honest gains viaFunc<double>seams (micGain = MicMuted ? 0 : MicVolume;loopbackGain = GameMuted ? 0 : GameAudioVolume, × duck), mixes mono mic + stereo loopback into interleaved stereo, runs theMasterLimiter(a −1 dBFS ceiling so the honest gains can never sum past 0 dBFS and clip the AAC encode — instant attack per frame, smoothed release toward unity), and writes the floats to the pipe. The gains are wired throughAudioGainProvider(Services/Audio/AudioGainProvider.cs) — the VM hands itsMicGain/LoopbackGainFuncs to the mixer at construction, read live each tick, so the volume sliders and mute buttons are stream-honest. (Before this wiring the seams defaulted to unity and the controls were decorative.)StopStream→StopLive()(closes the pipe → ffmpeg audio EOF) BEFORE stopping the frame pump (video EOF) — the reverse order stalls on pipe backpressure.FfmpegEncoderitself is untouched; the VM owns the pipe lifecycle. - Audio sync offset (TASK 22): after the limiter,
FillAndMixroutes the whole interleaved stereo mix through aAudioSyncDelay(Services/Audio/AudioSyncDelay.cs) — a pure delay line whose offset comes from aFunc<int> syncOffsetMsseam (the VM's globalAudioSyncOffsetMs, −500..+500 ms, persisted asAudio.SyncOffsetMs). This is OBS's documented fix for lip-sync: positive delays audio when it runs ahead of video; negative advances audio when it runs behind by eating the first|N|ms of the live stream head (seeAudioMixer.StartLive; the delay line stays clamped 0..500 internally — negative bypasses it entirely). The advance is armed once atStartLive(eating the head mid-stream is impossible); positive is live-reactive (re-read per tick). A slider on the mic bar ("AUDIO SYNC", locked while live/recording viaIsEditMode) + a status dot (IntToSyncBrushConverter) surface it. Changing the delay flushes the line (a live change clicks rather than smears). - TRAX — free background music (TASK 8):
MusicPlayer= NAudioMediaFoundationReader(mp3/wav/m4a) →VolumeWaveProvider16at the hardcoded 0.20 bed (no slider) →WaveOutEventon the default device, looping on any clean natural end (PlaybackStoppedwithe.Exception == null→ rewind + replay; the earlierPosition >= Lengthcheck was unreliable forMediaFoundationReaderand let tracks play once then stop). The desktop/game bar's mute togglesLocalGain(0 or 1;UpdateTraxLocalGainruns on everyGameMutedchange and on TRAX load), so muting kills TRAX in the headphones — matching the stream, where it rides the loopback channel. The volume slider drives system volume (PushSystemVolume) — which scales the HEADPHONES; the loopback capture is pre-endpoint-volume (disproven-then-fixed 2026-09-01), so the STREAM follows the knob viaAudioGainProvider.LoopbackGain = GameMuted ? 0 : GameAudioVolume, and the game meter multiplies its display by the same. One knob, three honest paths. No third mixer input. The footer TRAX button lives inside the game audio bar (in the preview overlay), left of the "Desktop Audio" label (the creator's pick — it rides the desktop channel, so the music control sits with the desktop audio controls): status dot (red no track / yellow loaded stopped / green playing) + "TRAX"; left-click toggles play/pause (opens the in-app picker viaOpenFileDialogwhen no track is loaded); right-click always opens the picker (TraxButton_PreviewMouseRightButtonUp,e.Handled = true, code-behind pattern); tooltip shows the loaded/playing track name or "No track — right-click to choose background music". The picked track persists via schema v9 single-rowMusic(TrackPath/IsEnabled) inLayoutStore. Known wrinkle (out of scope, future feature): YouTube mutes VODs carrying copyrighted music — "music on live, off VOD" is tabled. - Mic status dot (
Models/MicStatus.cs) on the preview-bottom MIC button: green =MicConnected, yellow =MicFailed(in use/unplugged), red = no mic device at startup (the mixer is never started, so loopback and the game bar can't run either — no capture devices at all). The picked mic persists:LayoutStore'sSettingskey/value table storesMicSourceName(saved on pick viaSaveMicSourceName), and the VM restores it at construction BEFORE the mixer's firstStart— so a restart reconnects the same already-vetted device (green) or reports it missing (yellow), instead of silently falling back to the default endpoint. - Game audio bar (desktop/game, always visible — TASK 4's
IGameAudioDetectorshow/hide gating was removed 2026-08-15: the bar used to appear only while a full-screen game with sound was up, which kept hiding the creator's desktop meter; the detector stack is gone, desktop audio is just automatic WASAPI loopback): overlaid at the bottom of the preview window (bottom-center, dark translucent chip, a mirror of the mic bar: meter + mute + volume slider). It's monitoring UI in the preview grid — the meter is display-only (fill =ToDisplay(GameAudioLevel) × GameAudioVolume— the ×volume term was missing until 2026-09-01: loopback capture does NOT shrink with the endpoint, so an unscaled meter left it pegged at 20% volume;GameMeterHonestyTestsguards it), but its volume slider drives system volume (PushSystemVolume— live headphones follow the knob; the capture side is scaled in the mix, see TRAX note above) and mute kills TRAX (MusicPlayer.LocalGain= 0). Relabelled "Desktop Audio" (TASK 8) since TRAX rides the same channel. WaveToFloat(pure, shared): WASAPI mix formats → interleaved float — IEEE float 32-bit direct, PCM 16-bit normalized to -1..1,WaveFormatExtensiblewith the IEEE-float subformat GUID (NAudio.Dmo.AudioMediaSubtypes.MEDIASUBTYPE_IEEE_FLOAT), trailing partial samples ignored.- Build 0 warnings; 197 passing (DSP/ring-buffer/ducker/resampler/pipe + mixer/hysteresis/
game-detector/meter-scale unit tests + the TASK 8 integration test
AudioPipelineTests.Mix_WithFiltersDuckAndGain_Lands_On_AudioPipe+ the polish-batch integration testMix_HonorsProviderGains_AndGameMute_KillsTheLoopbackproving the provider gains reach the pipe and mute silences the loopback).
Live frame pipeline (TASK 4 ship step 5 — shipped 2026-08-12, health stats 2026-08-13, plan in TASKS.md)
The FramePump (Services/Encoder/) is the live frame producer: while live it snapshots the active
scene each tick, resolves every element to its latest frame, composites it into the tier's output frame,
and paces frames into the encoder at the tier's FPS. rawvideo is stamped by ARRIVAL: ffmpeg assigns
pts from frame order at the declared fps — supply rate = output speed. A starved producer (pump < fps)
ships a time-lapse, truncated file with NO error (take-2 lesson, 2026-09-01); the pump therefore logs
FramePump stats: n/target frames per 5s, avg render Xms (resolve R), avg submit Yms so a slow stage names itself — and since 2026-09-04 so does the BUILD: every build gets a GUID stamped
into Helpers/BuildStamp (csproj GenerateBuildStamp target, fresh per compile — no lying incremental build), logged at startup
("Build xxxxxxxx (compiled ...)"). Take 6 (stamp-less, ambiguous): render 35-41ms with slice 3 built or not unknown — attribution is why the stamp exists; takes 7+ are
readable. If the breakdown says resolve dominates, the suspects are pre-cached chat/config thrash or the web frame; if blit dominates, the general-path element sizes. Pattern — everything is a constructor-injected
seam: Func<Scene?>, Func<SceneElement, VideoFrame?> resolver, Func<CompositorOptions>,
Func<EncoderOptions?>, Func<IFfmpegEncoder>, Action<string> log, and an injectable pacing delay
(default Task.Delay; tests inject Task.Yield). The pump is free of WPF and of the capture managers.
StartAsyncnever throws — the VM fires-and-forgets it from the sync command handler; failures log- surface via the
Failedevent.BuildEncoderOptionsnow always returns options (TASK 18) withStreamEnabled/RecordEnabled/RecordPathbaked from the pills +_activeRecordPath; the pump skips the encoder only whenoptions == null, i.e. "no output configured".MainViewModel._rtmpUrlProvideris that seam — aFunc<string?>that now yields the reusable stream's ingest URL (TASK 9, shipped 2026-08-16): loaded from theLayoutStorecache at startup and set fresh byPrepareAndStartLiveAsyncbefore the pump starts. The pump reads the URL once at startup, which is why go-live ensures the stream BEFOREStartAsync.
- surface via the
- Deadline pacing + row-blit render (take-3 starvation fix, 2026-09-03): take 3 recorded 30s into a
2.1s file and the 5s stats line named the cause in one number —
17/300 frames per 5s, avg render 258.1ms, avg submit 1.5ms. Two defects, both fixed: (1) the pump slept the FULL interval after each render, so period = render + interval — OBS'svideo_thread(libobs/media-io/video-io.c) pattern replaces it: absolutenextTick += intervalTicksdeadline, sleep only the remainder, and on overrun skip the wait. (NOTE, slice 9: the original "skip the missed ticks (rebase)" half of this fix was wrong — see the slice 9 correction below.) (2) the compositor did per-pixel float sampling +Math.Roundblending over all 2.07M master pixels (backdrop) and scanned the whole destination per overlay (a 64px social-bar strip cost 2M iterations).SceneCompositornow follows the libyuv pattern (BSD-3, chromium.googlesource.com/libyuv/libyuv — cited per the derivative-work rule): 1:1 aligned blits take a row-walk fast path (bilinear at scale 1 + integer offset is the identity) with per-pixel alpha branch and integer fixed-point blend;BlitOverlayclips to the intersection rect; a full-cover live backdrop skips the opaque-black pre-fill. The general per-pixel path handled scaled/round/mirrored elements. Test:Pump_Paces_To_The_Deadline_Compensating_Render_Costrecords the requested wait via the pacing seam (a seam fake must genuinely await — a synchronous completed task runs the whole pump loop onStartAsync's continuation and hangs the run; MyMistakes). - Slice 2 of the render fix (2026-09-04, take 4): the pacing held (file no longer truncated at
loop level) but render stayed at 58.9ms — the per-pixel row walk was still 2M managed iterations and
every tick allocated a fresh 8.3MB master buffer. Three changes: (1)
VideoFrame.IsOpaque— a producer-contract flag (only the screen-capture and webcam paths set it; DWM/MediaCapture fill alpha 255 by contract); a full-canvas, aligned, opacity-1 blit of an opaque frame is now ONEBuffer.BlockCopy(~1.5ms) instead of the loop, and the black pre-fill is skipped when the backdrop covers. (2) integer fixed-point bilinear in the generalBlitContentpath (row-hoisted invariants, no divisions, noMath.Round) — within ±1 of the float reference, inside the ±2 test tolerance. (3) scratch pool in the pump —AcquireScratch/ReleaseScratchrecycle the master buffer (max 4, keyed by length, owned-by-reference so bake-cache/social-bar/static-art arrays can never be captured); release happens strictly AFTERSubmitFrameAsyncreturns (the write to stdin copies), and the free-list Contains guard makes the transition Cut path safe (BlendFramereturns the to-frame itself, aliasing the scratch). The per-tickfromScenerender in the transition branch was dead weight (BlendFrame usesTransitionService.FromFramecaptured atStart, never the pump's) — removed, and the pump'sfromSceneProviderseam +MainViewModelcall site went with it. Tests:Pump_Pools_ScratchBuffers_Across_Frames_Without_Stale_Pixels(the ONE: alternating backdrop colors pin every frame's content, repeated backing-array identity proves the pool recycles),Composite_OpaqueFullCover_Backdrop_CopiesEveryPixel_Into_Scratch(memcpy branch + scratch sentinel).FakeEncoder.SubmitFrameAsyncnow snapshots bytes like the real stdin write — holding the reference would race legitimate recycling. Take 5 must showavg render ≤ ~10ms, ≈300/300; the vertical tier's final 1080×1920BilinearScalestill allocates fresh per frame (same GC lesson when someone streams vertical — recorded as a follow-up, not silently "done"). - Slice 3 — chat raster-on-change (2026-09-04, take 5): take 5 measured
138/300, avg render 25.5ms— the per-tick blits were cheap now, butResolveOutputFrame→RenderChatBoxran a FULL WPF raster (ChatBoxRenderer: FormattedText +RenderTargetBitmap+ CopyPixels + channel swap) EVERY tick whenever the message buffer was non-empty — and the buffer survives between sessions, so even a signed-out record-only take paid it. Established answer, same as OBS text sources: raster on change, blit the cache every tick.ChatOverlayLayernow keeps a content version (Messages.CollectionChanged→_contentVersion++) plus a config key (box size + all Chat* props);RenderFramereturns the cachedVideoFrameby identity until either changes (the compositor only ever reads a cached frame). Test:ChatOverlayLayerCacheTests(RealApp, real renderer —Same()for unchanged inputs,NotSame()on message/config change, null on empty). Accepted cost pending take 6: one slow tick (~15-25ms) per arriving message; if chat-burst frame loss shows up, the next slice moves the re-render off-tick (debounced, dispatcher-side). - Slice 5 — paste cache for non-opaque layers (2026-09-04, take 7/8 data): the render/resolve
split (added in the stamp commit) finally named the last thing honestly:
resolve ≈ 0butrender 26-27ms(124-135/300, ≈2.2x) — the chat fix HAD worked; the remaining cost was the compositor re-rasterizing EVERY layer per tick even when its pixels never changed (this creator's Live scene: chat 159k + web widget 271k + image 95k + webcam 156k px ≈ 680k samples @ ~38ns each). OBS's actual shape: sources cache their surface, the compositor pastes.SceneCompositornow routes non-opaque layers throughBlitCachedLayer: a layer rasterizes ONCE into an element-space, TRANSPARENT-based frame keyed by (source-array identity, source W×H, ceil'd dst rect, round, mirror), then every later tick PASTES it (integer position, row alpha-blend, opacity applied at paste). Correction (2026-09-12, the web-widget "black box"): the raster builds on a TRANSPARENT base, but the sampler's PARTIAL-alpha branch used the opaque-dst blend — it premultiplied the color into RGB and forcedalpha=255. A translucent widget pixel then pasted as opaque darkened ink (the box in recordings) while the raw-bitmap preview stayed correct.BlitContentRawnow takestransparentDst(the raster call passestrueand writes straight color + straight alpha; the paste rows do the source-over). Master paths are bit-identical. Guard:PasteCache_SemiTransparentLayer_RevealsBackdrop_NotOpaqueInk. Producers hand out fresh immutable arrays, so array-identity keys can never serve stale content; dict bounded at 48, cleared wholesale on overflow. Only changing content (webcam device frames, web capture ticks, chat messages) resamples; the opaque backdrop keeps its memcpy path. Position/opacity drags are now near-free (no resample — paste params, not cache keys). ONE integration test:PasteCache_RepeatRender_IsByteIdentical_And_ContentChangePropagates(raster-vs-paste byte equality incl. round-clip margins + new-array propagation); the whole existing pixel suite guards sampler semantics. Take 9 must show≈300/300, avg render ≤ ~8ms; if it lands there, the recording saga closes. Follow-ups unchanged: vertical-tier scale alloc, debounced chat re-render under message bursts. - Slice 6 — the sleep quantum WAS the ceiling (take 9, 2026-09-04): period measured ~37ms while
work (render+submit) was ~25 — the missing ~12ms is
Task.Delayrounding EVERY request up to the Windows clock tick (documented ~15.6ms default; learn.microsoft.com/en-us/dotnet/api/system.threading.tasks.task.delay). A frame finishing 3ms early requests a 3ms wait and sleeps a full 15.6 — capping the producer at ~27fps NO MATTER how fast the compositor ran. This is why takes 7→9 showed zero playback change despite real render wins: the sleep floor dominated everything above it. Established fix (game-loop canon — stackoverflow.com/questions/5441464; timeBeginPeriod — learn.microsoft.com/en-us/windows/win32/api/timeapi/nf-timeapi-timebeginperiod):timeBeginPeriod(1)for the pump's lifetime (pairedtimeEndPeriodin the finally), sleep only the BULK of the remainder (request minus 2ms),Thread.SpinWaitthe last ~2ms across the deadline; blown deadlines still rebase. (SUPERSEDED in part: slice 9 deleted the rebase — deadlines only advance; slice 10 replaced the burst re-write with one fresh composite per iteration.) Stats gainedavg wait Xmsso render+submit+wait must ≈ period — the accounting is closed, no stage can hide again. Same slice: the webcam'sIsOpaquepaste-cache bypass was removed (it re-sampled ~156k px every tick even between identical device frames; the cached paste beats the sampler on hits and costs the same on misses). Take 10 verdict:≈300/300honest fps — if short, the wait/render split names the remaining term with no ambiguity left. - Slice 7 — the loop was ON the UI thread the whole time (take-8 numbers, 2026-09-04): with
the wait finally measured, take 8 (59a02a5b) showed the impossible pair —
render 22ms, wait 10msagainst a 16.7ms deadline: a blown deadline rebases with NO wait. The wait was the producer queued behind the live preview on the dispatcher:PumpAsyncstarts from a UI command handler and itsawaitcontinuations inherit the UISynchronizationContext, so the "WPF-free" compositor actually rendered on the UI thread and ran only when WPF let it. OBS runs its media loops on dedicated threads for exactly this reason. Fix:_pumpTask = Task.Run(() => PumpAsync(...))(no sync context inside → continuations stay on the pool). Side effects handled, not ignored:StaticPixelCache.Getnow locks (pool miss-decodes race UI callers);ChatOverlayLayer.RenderFramechecks its cache off-thread but MARSHALS the rare raster miss to the dispatcher (DrawingVisual/ RenderTargetBitmap are UI-thread objects) and re-validates there; pump events already marshalled. Also:GCLatencyMode.SustainedLowLatencyfor the pump's life, stats gainedworst render Xms(spike visibility — bimodal averages hid them), webcam now routes through the paste cache (bypass re-sampled 156k px even between identical device frames). Tests:Pump_Produces_OffTheStartingContext(the ONE — inline-pumping SyncContext proves continuations never return to the starting thread) + 70/70. Take 9 verdict: wait ≈ true remainder (period → 16.7, n → ~300);worst rendernames any remaining raster-miss spikes; render+submit+wait still == period — the accounting holds. - Slice 8 — capture buffer ring + paste-cache Epoch (take 11, 2026-09-04): take 11 validated
the architecture — typical frames now land
work ~10ms + wait ~6.8ms = 16.7, exactly the deadline (212/300). The whole remaining gap is PERIODIC 35-65ms worst-render spikes that worsened across the take (189→147 frames/5s) — the signature of gen2 GC pauses, now provable viagen2 +Nin every stats window. Biggest churn was structural:ScreenCaptureFrameSourceminted a fresh ~8.3MBbyte[]per DWM frame (~500MB/s LOH). It now rotates a shared-frame ring (4-deep at slice 8, deepened to 8-deep after the "flash" finding — consumer holds must never outlive depth × source period), size-matched per slot, with reused downscale row scratch. Recycled arrays would poison the paste cache (it keys on array IDENTITY), soVideoFrame.Epoch— monotonic per producer frame, 0 for fresh-array producers — joins the key. Regression testPasteCache_RecycledArrayWithNewEpoch_ReRasterizes_NotStaleHitsfails on the old key by construction. The camera producer carried the same fresh-array churn (110-220MB/s at 30-60fps): it now rotates its own 8-deep ring + Epoch (same identity rule), and the WebView2 capture reuses a canvas scratch + 8-deep output ring + a reused WriteableBitmap instead of minting two fresh arrays + a new bitmap per 10Hz tick. Next suspect if gen2 stays hot: the WPF preview load itself driving gen2 — recorded as follow-up, untouched this slice. - Slice 9 — the "rebase" WAS the recording time-lapse (2026-09-10, the recording playback-timing
take): slice 6's "blown deadlines still rebase" was the recording-timing bug all along. In a real
take the pump emitted
215-219/300 per 5s(~43fps) whilerender+wait == periodstill looked closed — the rebase ERASED every missed slot (deadline = wall-now again), so the missed ticks never showed in the stats. rawvideo has no per-frame timestamps: ffmpeg muxes by frame count at-framerate 60, so a 43fps reality was authored into a 60fps container and every recording played ~1.4x fast (takes confirmed with a WSL ticker visible in the recording: file duration 11.44s vs ~15.5s wall). Two defects, both fixed the OBS way (libobs/media-io/video-io.c— the video thread NEVER resets its deadline; every interval tick outputs ONE frame, and a late render means repeated content — judder — never a skipped timestamp): (1) count-based CFR emission:while (GetTimestamp() >= nextTick) { SubmitFrame(frame); nextTick += intervalTicks; statFrames++; }submits exactly one frame per crossed slot, re-writing the current composite when the render overruns (duplicated footage = correct duration, not a time-lapse), andnextTickis NEVER reset to wall-now; (2)-reremoved fromFfmpegArgs— it was a second, fighting pacer on the rawvideo demux (its "Resumed reading … after a lag" grew 0.79s→4.82s across the take, leaving the pump behind its own honest-stats count). One fix covers the game background AND the webcam — both flow through this one pump. Muxed duration is now frame-count ÷ fps == wall time by construction. Test: the pump suite stays green unchanged — the pacing test's<intervalwait assertion still holds because the frame emits at the next slot boundary, not immediately. (Unrelated pre-existing failure found the same day:Composite_FullScene_MasterPixelspixel (1380,700) cyan-vs-magenta — reproduces with this fix stashed, untouched by it, recorded as a follow-up.) - Slice 10 — bounded encoder queue + drop policy + burned-in frame counter (2026-09-10, the
playback-pacing take): slice 9 made the DURATION right but the CREATOR still read the recorded
ticker as "1...23...4...56..." — variable pacing. The aggregates (301/300, uniform PTS, 15.6s wall vs
15.74s file) could not see it, and the cause was finally measured in
FfmpegEncoder.SubmitFrameAsync: it BLOCKED onWriteAsync(8.3MB) + FlushAsyncwhenever ffmpeg lagged the pipe, and slice 9's burstwhileloop then re-wrote that SAME composite for every slot that ticked past — frozen content runs (the "smeared ticker"). Fixed the OBS way (the encoder queue inlibobs/obs-encoder.c: the encoder's thread NEVER couples back into the video thread; overload = dropped content, never a frozen video thread): (1)FfmpegEncoder.SubmitFrameAsyncis now an ENQUEUE into a boundedChannel<byte[]>(cap 120 ≈ 2s at 60fps) owned by a dedicated drain task with the stdin write + flush; the caller NEVER blocks on the pipe. Pixels are copied into anArrayPoolbuffer before enqueue (the write now happens later on the drain thread, so the pump's scratch reusable the moment submit returns — same contract as before). (2) drop-on-overflow (creator's choice: freshness over coverage): when the queue is full the NEWEST frame is dropped and counted (IFfmpegEncoder.DroppedFrames,Interlocked); the session never freezes or smears — it drops. Stop flushes the whole queue then EOF (Channel.TryComplete→ drain writes leftovers → closes stdin → ffmpeg finalizes+exits), so no accepted frame is ever lost at stop. (3) pump loops ONE submit per iteration (replacing slice 9's burst): every due slot gets a FRESH composite — no catch-up slot ever repeats frozen content; missed slots vanish as a count-based gap. The deadline counter is still never reset to wall-now (slice 9 ruling). (4) burned-in frame counter (the new judge):_outputIndexis burned into a 6-digit dot-matrix strip (white box + black 5×7 glyphs, bottom-right corner) of EVERY composite before submit. The WSL ticker is demoted because its own timers smear under Windows host load; decoding the recording and reading the strip is clock-independent: the number advances +1 per frame and jumps by exactly the counted drops (queue overflow OR skipped catch-up slots). Stats gainedworst submit Xms,dropped N,stalls K; a stall logger names any iteration > 2× interval with its render/submit split — with the queue, submit is ~1ms, so a stall means render/resolve. The strip sits above the social bar (bar is composited, then burned over) — a debug judge, tiny at 1080p. Test:Backpressure_QueueOverflow_DropsFrames_AndNeverBlocks(the ONE: a 40ms-per-frame sink fake makes every enqueue fill the queue; submit must return instantly, drops are counted, stop flushes exactly submitted − dropped bytes). Full suite 289/290 (the pre-existing compositor pixel failure unchanged). Audio untouched (follow-up). WSL ticker-under-load reliability check still to run (informational — the burned counter is the judge). - Slice 11 — web-layer animation ran at ~1/6 speed; capture cadence 10Hz→~30Hz + de-throttle
(2026-09-10, the "widget animates too slow" take): the recording is 60fps but the WebView2
capture loop was a blind 100ms
DispatcherTimer= 10Hz — a 60fps-designed widget was sampled 6× under its native rate (repeated-footage slow-mo). Two stacked throttles:Superseded by Slice 14 (below):
CaptureSchedulerand the cadence hooks were the polling design; frame-driven composition capture replaced them — nothing here is current architecture. (1) the 10Hz cap (unconditional, code-proven) and (2) Chromium hidden-page throttling (requestAnimationFrameparked, JS timers clamped to ~1s per WebView2Feedback#1172/#3070 + Chrome-88 timer throttling) whenever the app window is unfocused or covered — the page is off-screen at (-5000,-5000), so it is a hidden page the moment the host window loses occlusion. Fixed: (1)CaptureScheduler(new,Services/CaptureScheduler.cs) replaces the per-sessionDispatcherTimer: a dispatcher timer that DROPS ticks while a capture is in flight (latest-wins, never queues) — the in-flight drop is what made raising the cadence safe (concurrent full-HD PNGCapturePreviewAsynccalls would stack ~10-30ms encodes and publish stale-after-fresh). Interval is owner-configurable: recording 33ms (~30Hz), idle 200ms. Unit-tested without a WebView2 runtime (the ONE test:CaptureScheduler_Drops_Ticks_While_Capture_InFlight_And_Resumes— overlapping ticks dropped, capture resumes when idle, driven by a TCS so it is deterministic). (2) de-throttle browser args: the sharedCoreWebView2Environmentis created with--disable-backgrounding-occluded-windows --disable-renderer-backgrounding --disable-features=CalculateNativeWinOcclusion(the Electron/Streamlabs-class embedder answer for occluded-window animation throttling) BEFOREEnsureCoreWebView2Async. (3) cadence hook:MainViewModelsetsSetCaptureInterval(33)on record/stream start and(200)on stop (MainViewModel.Streaming.Operations.cs). (4) capture-cost telemetry: first 30 captures per session log elapsed ms to startup.log — PNG-encode+decode cost decides whether ~30Hz stays or drops to ~20Hz; the FramePump drops frames (never time-lapses, slice 10) if UI-thread GC churn starves it, so the cost is observable. Full suite 290/291 passing, the sole failure the pre-existing compositor pixel test. The web-overlay transparency verification (the first-capture diagnostic bound to about:blank) and the audio-silence item remain open push-gate items — both untouched by this slice. - Slice 12 (audio diagnostics, 2026-09-10): the recorded audio is full-length silent AAC
(−91dB, 1124 frames / 23.95s in the latest take) — the named-pipe delivered ~9.1MB of zeros to
ffmpeg for the entire recording, so the loop ran and connected, but both WASAPI capture sources
delivered nothing. Sources started fine (
Audio: using system default mic...logged), no failure callbacks fired, and the mixer's existing integration test (Mix_WithFiltersDuckAndGain_Lands_On_AudioPipe) proves the loop→pipe math is sound. The fault is capture-side: either the default render endpoint carried nothing (audio played on a non-default device — common), or both endpoints were held exclusive, or genuinely nothing played. Added permanent per-5s live-loop telemetry to startup.log (Audio live: pipe connected=, dropped writes=, micLevel=, loopLevel=, drained, peakMix) andNamedPipeAudioWriter.DroppedWrites— names the exact stage on the next take without new code.AudioMixer.FillAndMixnow returns(MicRms, MicDrained, LoopDrained)for the telemetry accumulation. Full suite 290/291, same pre-existing sole failure. - Slice 13 (web transparency diagnostic re-point, 2026-09-10): the "?" in MyMistakes'
"RESOLVED (? VERIFY)" is being closed. The one-shot widget dump + alpha stats bound to the
FIRST capture ever = the initial about:blank document — a blind instrument. NavigationCompleted
for the REAL widget URL now arms
WidgetDumpRemaining = 5; those 5 post-paint captures dump%TEMP%\ytLive-web-<id>-w1..5.png+ the sharedAlphaStats(...)line (alpha[min/max/mean/zero%])- FindContentBounds result to startup.log. That line alone decides the fix branch: zero% ⇒ the
pre-parse injection did not hold for this widget (opaque capture → chroma-key / stronger DOM
injection); large zero% + tight contentBounds ⇒ the capture IS transparent and the recording's
black block lives downstream (compositor blend/underlay).
AlphaStatsextracted as the shared sampler. No new tests (WebView2 runtime not instantiable in tests; the re-point is log-only).
- FindContentBounds result to startup.log. That line alone decides the fix branch: zero% ⇒ the
pre-parse injection did not hold for this widget (opaque capture → chroma-key / stronger DOM
injection); large zero% + tight contentBounds ⇒ the capture IS transparent and the recording's
black block lives downstream (compositor blend/underlay).
- Slice 14 — web frames are now COMPOSITION-CAPTURED; the PNG poll +
CaptureSchedulerare gone (2026-09-14, the "still not 60fps" take): slice 11's ~30Hz was still aCapturePreviewAsyncPNG poll — every 1920×1080 full-HD encode+decode cost 35–165ms, so the session budget capped real cadence at ~20Hz for a 60fps-designed widget. Root fix = frame-driven capture, not faster polling: the WebView2 renderer now feedsWindows.Graphics.Capturedirectly through aCoreWebView2CompositionController(the mechanismWebView2CompositionControland Flutter'swebview_windowsuse — seegraphics_context.ccCreateGraphicsCaptureItemFromVisual, which captures the rootsurface_visual).- Pipeline (per session):
CreateCoreWebView2CompositionControllerAsync(WindowHandle)— the parent HWND must exist, soMainWindow.InitWebView2()moved from the ctor to Loaded (wasInitWebView2(WebViewHostPanel); the hidden XAMLWebViewHostPaneloverlay is deleted).controller.RootVisualTarget= a childContainerVisual(RelativeSizeAdjustment = 1,1) under a rootContainerVisual(1920×1080,IsVisible=true) built on ONEWindows.UI.Composition.Compositor(created once on the UI thread);Bounds = 0,0,1920,1080,BoundsMode=UseRawPixels,ShouldDetectMonitorScaleChanges=false,RasterizationScale=1.0,IsVisible=true,DefaultBackgroundColor=Transparent; thenGraphicsCaptureItem.CreateFromVisual(root). Frames pull from a free-threadedDirect3D11CaptureFramePool(2 buffers) →SoftwareBitmap.CreateCopyFromSurfaceAsync(BGRA,BitmapAlphaMode.Straight— the compositor blends STRAIGHT alpha, so premultiplied readback would wreck corner anti-aliasing) → per-frameFindContentBoundsalpha-bbox on the capture worker →VideoFrame.CropBoundsstamped → ring up to 2 fresh + Epoch (reuse; GC lesson slice 8) → dispatcher-coalesced copy (Render priority) into ONE crop-sized sharedWriteableBitmap;PreviewBitmapChangedraised only on bitmap (re)creation. Also unpumped: the OBS-side de-throttle flags (slice 11) stay — they fix the hidden-page JS throttling; the capture path itself no longer polls. - Implemented in: new
Services/WebCaptureFrameSource.cs(owns item+pool+session — mirrorsScreenCaptureFrameSource, which servers as the copy template) + reworkedServices/WebView2Manager.cs(owns controllers, the visual tree, nav/script wiring, preview; internal seam ctor(Dispatcher, Func<string, IScreenCaptureSource>?)so the tests instantiate zero WinRT; public ctor(Dispatcher)). - CoreMessaging DQ recipe (record-once): the 19041 CsWinRT projection has NO
DispatcherQueueController.CreateOnCurrentThread()(CS0117 — onlyCreateOnDedicatedThread+FromAbi(IntPtr)). P/InvokecoreMessaging.dll!CreateDispatcherQueueControllerwith structDispatcherQueueOptions { DwSize, ThreadType = 2 (DQTYPE_THREAD_CURRENT), ApartmentType = 2 (DQTAT_COM_STA) }, wrap viaDispatcherQueueController.FromAbi(ptr)(mirror ofCaptureInterop), only thennew Compositor()— all once on the WPF UI thread (the app dispatches Render there). - Compile notes (each cost a build cycle):
Compositorcollides with the repo's OWNytLive.Services.Compositornamespace → fully-qualifyWindows.UI.Composition.Compositor;CoreWebView2CompositionControllerexposesClose(), notDispose();Coloris ambiguous (System.DrawingvsSystem.Windows.Media) →System.Drawing.Color.Transparent. - Dropped:
Services/CaptureScheduler.cs(DELETED — no cadence to schedule), all threeSetCaptureIntervalhooks inMainViewModel.Streaming.Operations.cs, the XAMLWebViewHostPanel.TransparentBackgroundScriptconst is unchanged (a test pins it). - Test model (Good Dog): reworked
ytLive.Tests/WebView2ManagerTests.cs— dropped the 4 control-size tests + theCaptureScheduler_Drops…test;FindContentBoundstests moved toWebCaptureFrameSource.FindContentBounds; the ONE integration testFrames_PublishCroppedPreview_And_CoalesceToLatest_CarryingCropBoundsdrives the internal seam withFakeWebSource : IScreenCaptureSource+ a real background-STADispatcherPump(copied fromScreenCaptureManagerTests): register → one crop-sized preview bitmap published →CropBounds+IsOpaque=falseonGetLatestFrame→ back-to-back pumps coalesce to latest. Byte assertion compares the DENSE 2×2 crop (CropByteshelper — slices source rows with stride gaps, a contiguous range spans rows wrongly). - Full suite 293/293 green, 0 warnings. Audio untouched. NOT YET VERIFIED ON DEVICE — the composition path needs a real 60fps-widget run (open item; see HANDOFF). Web work commits stay LOCAL (no push) until the user greenlights.
- Pipeline (per session):
- Slice 15 — one frame per deadline SLOT: duplicate-on-lag, never skip (2026-09-14, device takes
ty-1723/1726): slice 10's freshness choice — a render overrun VANISHES the missed slots from the
file — authored ACCELERATED playback: ~35ms render cost vs the 16.6ms slot, one fresh frame per
35ms (log wall-1726:
FramePump stall: iteration 33ms (> 2× the 17ms interval): worst render 33-46ms), and a 60fps container muxed on arrival → 1723: 697 frames = 11.62s video vs 11.84s audio; 1726: 163 frames = 2.72s vs 2.93s. The video also ended 0.21–0.24s before the audio ("audio speeds up, cuts off at the end"). OBS's answer is duplicate-on-lag:libobs/media-io/video-io.cemits one frame per tick and a lagger DUPLICATES the newest frame, counted as "lagged frames due to rendering lag/stalls" (obs-output.c) — a wall-time hole never exists (docs.obsproject.com/ backend-design: "If the video frame queue is full, it will duplicate the last frame"). The pump's submit is now a bounded catch-up:while (now >= nextTick) { submit; nextTick += intervalTicks; }over adeadlineNowcaptured once per iteration — the LATEST composite, fresh on the first missed slot, repeated (OBS's duplication) for the rest. Duration == wall under any render load, at the cost of a short judder during a stall. The burst is safe becauseChannel.TryWritenever blocks (slice 10's queue — the take-9 smear was the BLOCKING pipe-write re-copying a stale buffer during a long freeze; here each emit is nanoseconds). The burned_outputIndex(slice 10 judge) moved INSIDE the submit loop so every emitted slot carries its own +1 (and the old unconditional pre-gate bump no longer gaps the sequence on non-submitting fast-render iterations). Good Dog testPump_Overrun_Renders_EmitsEverySlot_NotSkipped: 60fps, resolver sleeps 35ms, asserts ≥0.65 of the wall slots are emitted (the skip-pump wrote ~1/35ms ≈ 200 in 7s; the slot pump ~415). Full suite 294/294 green, 0 warnings. The +0.6s audio-late clap reading on 1726 was confounded by the 1.1x acceleration — re-measure on device; if a real residual remains it is the audio pipeline. No push (web/A/V work stays local). - Slice 16 — the desktop capture conversion was the bottleneck: fast downscale +
cadence throttle + reuse-distance ring (2026-09-14, device take ty-1742): slice 15
fixed pacing but the 1742 desktop layer was still jerky/frozen with a horizontal
tear. Decoded to raw frames and audited: the desktop band was frozen 21s of 23.35s
(90%) — ~6.1 content updates/s, freeze runs up to 2.28-2.78s, dup-run max 90
frames (1.5s). The render stat (
worst render 33-36ms) was real but MOOT: the capture CONVERSION was the wall. The monitor delivers at the 240Hz DWM cadence (~4.2ms); with one-in-flight conversions and each 2560×1440→1920×1080DownscaleBgraat ~150ms under load,LatestFrameupdated a handful of times/s — the desktop feed inside a 60fps file read ~90% frozen. (Webcam + audio were their own paths — fine, matching the report.) Three changes, all inServices/ScreenCaptureFrameSource.cs(+Services/FrameRingBuffer.cs): (1)DownscaleBgrais now integer 8.8 fixed-point, "shift only at the end" — the exact two-stage math ofSceneCompositor.Bilinear(MyMistakes take-4 rule), dropping double-per-pixel to a row-walk of integer ops (the capture downscale had stayed the naive float twin of the 258ms disaster); (2) a 10ms conversion floor (MinConvertInterval) so the 240Hz tail stops queuing ~150ms of serialized conversion per second — the open edge sits just above the ~60/s the pump can use; (3) the ring is a reuse-distance pool (FrameRingBuffer, depth 8, redLine 4): a buffer is only rewritten ≥4 rents after its last hand-out else a fresh allocation, replacing the blind round-robin — sinceScreenCaptureManagerkeepssession.LatestFrameacross conversions and the dispatcher preview copy lags, "who released the buffer" needs a consumer API that doesn't exist; a reuse DISTANCE needs no cooperation (the 1742 new-top/old-bottom tear is that read-under-write, structural now). Telemetry added: a startup.log line every 2s (frames/s, conv avg/max ms, skip busy/cadence, ring allocs) so the device take can be judged numerically. Good Dog testRing_NoLap_ReusesOnlyAfterRedLineRents(depth 3/redLine 4 exercises the red-line skip; the 8/4 config settles at 8 buffers and never grows). Full suite 295/295 green, 0 warnings. C4 (compositor Epoch-cached downscale / blit-on-change) was DEFERRED: the slice-15 approved plan assumed a native-res relocation; the measurement recast it — relocating a ~30ms float downscale to the render thread just moves the same cost into the slot budget. Re-measure on device; if render still >16.6ms slots after capture feeds ≤60 real updates/s, add C4. No push. - Slice 17 — the OS readback was the real wall: overlapping conversions + monotonic
publish + deeper pool (2026-09-14, device take ty-1824): slice 16's downscale fix
landed but the desktop was still ~90% frozen on the 1824 take (band 6.8/s updates, max
freeze 4.85s). The new 2s telemetry was decisive:
conv avg 46-50ms max ~61mswithskip busy 37-83— the GPU→CPU readback (CreateCopyFromSurfaceAsync), notDownscaleBgra, is the ~47ms wall (240Hz HDR compositing + encoder + 2 capture devices share the GPU); at one-in-flight that caps the desktop feed at ~17-20 updates/s, which is the file's whole-frame ~7 content-moments/s. Two facts reshaped the fix: (a) shrinking the pool size does NOT scale the desktop (Microsoft docs: "If content is larger than the frame, the contents are clipped") — readback stays at native 2560×1440; (b) delivery is healthy (60-100 arrivals/s), so the lever is conversion throughput, not the pool size. Changes inServices/ScreenCaptureFrameSource.cs: conversions overlap up to MaxConcurrentConversions = 3 (pool deepened to 5 buffers so in-flight frames fit), each completion publishes ONLY if its Epoch is strictly newer than the last published (MonotonicGate— a slow older completion must never overwrite a newer LatestFrame), Epoch increments viaInterlocked, and the DownscaleBgra row-scratch became per-conversion locals (concurrent callers). The render side (33-42ms → ~30 unique composites/s) is the NEXT cap after capture speeds up — that's the C4 slice, queued right after this re-measure. Good Dog testPublishGate_TryPublish_OnlyStrictlyNewerWins. Full suite 296/296 green, 0 warnings. NOT YET DEVICE-VERIFIED; target: telemetry frames/s jumps ≥ ~30-40 and the band audit drops below ~50% frozen. No push. - Slice 18 — C4: the FramePump blit-on-change composite cache (2026-09-15, take ty-1841
verdict): the 1841 take proved capture fixed (band ~20 fresh updates/s, no tears, pacing
clean) but logged a FramePump stall on EVERY iteration (
totalMs 21-44,render=full-render split=0 elements=6 dynamic=4, worst render 166ms startup spike) → ~22-28 composites/s hard-caps the desktop inside the 60fps file. The SceneGraph split CANNOT fix this wired:GetSplitPointreturns 0 because the live-capture backdrop is element 0 and must stay dynamic — a baked capture goes stale — so every tick runs the full re-composite even when no input changed (the compositor's per-layer paste cache already makes identical frames identical per-element; the waste is re-COMPOSITING the whole 8.3MB frame). Same shape as OBS (sources cache their surface, the compositor renders on update only — the pattern this file has cited since the 2026-09-04 paste-cache slice) at the FRAME level.FramePump.RenderFullnow wraps both full-render call sites (no-SceneGraph + split=0 fallback): it computesBuildFullRenderSignature(a deterministic hash of the crop/output dims, the social bar identity + top, and — per element, MIRRORING the compositor's own resolution — the element ref, layout/visual bits, and the frame each element would resolve through the SAME resolver seam, using buffer-array identity + Epoch + CropBounds; a changed frame forces a re-composite, an unchanged one never serves stale bytes) and, when it matches the last composite, reuses the cache with ONEBuffer.BlockCopy(~3ms) instead. The cache buffer is a separate long-lived array — never the scratch pool — because the caller burns the frame counter and recycles the scratch AFTERRenderScenereturns; the cache is written from the rendered scratch BEFORE returning (pre-burn, pre-recycle). Engagement is gated on the 1:1 config (SourceRect == Output, the only deployed tier); the vertical tier's cached fresh-scale-buffer follow-up stays exactly where it was. Telemetry: internalCacheHits/CacheRenderscounters surface ascache {renders}R/{hits}Hon the 5s stats line, and the ticks' key is an internalOutputIndex(burned counter, stable across one tick's resolver passes). Good Dog testFullRenderCache_StaticInputs_RenderOnce_Then_Reuse_UntilInputChanges: static scene → exactly ONE composite (renders==1, hits>0, byte-identical content above the burn strip), then a new frame (new array + monotonic Epoch) invalidates and propagates. Existing pool test now passes a STABLE scene (production handsStagedScene; a fresh scene per tick churns the element refs inside the signature and hides the cache — the tests that care about cache behavior must mirror production) and keys its red/blue alternation onOutputIndex/frame count so the tick's two resolver passes (signature + render) don't double-advance a call-count flip. Full suite 297/297 green, 0 warnings; verify.sh scope-locked to FramePump.cs + FramePumpTests.cs + docs. Target next take:cachehits dominate on TV holds,avg renderdrops toward the BlockCopy, stalls vanish, then the band audit + clap re-measure decide push. No push. - Stop ordering matters:
StopAsyncstops the encoder — since slice 10 it FLUSHES the pending queue (Channel.TryComplete→ drain writes the leftovers, closes stdin → EOF → ffmpeg finalizes+exits; an accepted frame is never lost) — before awaiting the loop. The old reverse-order deadlock was a submit stuck on pipe backpressure; the drain thread owns that wait now, and stopping the encoder first keeps the pump's own (non-blocking) submits from racing the completed queue as a spurious drop.ProcessFailedself-stops the pump.Failedwhile live flipsStreamStatus.Error(minimal). - Health stats (ship step 6, shipped 2026-08-13):
HealthUpdatedis bound to the bottom bar —MainViewModel.OnFramePumpHealthUpdatedmarshals to the UI thread (the encoder's stderr loop raises on a background thread) and copies intoCurrentHealth(the bottom bar's existingCurrentHealth.*bindings).ResetHealth(status)zeroes dropped/duration on go-live and on End so stats never linger from a previous session; bitrate/FPS stay on the tier's targets (ApplyStreamQuality). The bar shows real encoder values since TASK 9 (2026-08-16) wired the reusable stream URL into_rtmpUrlProvider. MainViewModelowns the resolver (ResolveOutputFrame):WebcamSceneConfig→CameraManager.GetLatestFrame(DeviceKeyForWebcam(WebcamId))— device-keyed (configs carry the identity GUID; sessions key on the Windows device id — the old direct-GUID lookup returned null forever and the webcam was structurally absent from output; take-2 fix 2026-09-01,WebcamOutputKeyTests),Source { IsLiveCapture, CaptureKey }→ScreenCaptureManager.GetLatestFrame(CaptureKey)(the new accessor mirroringCameraManager), image/ background →StaticPixelCache.Get(AssetId).BuildCompositorOptionsrounds the VM'sOutputRect*doubles to ints — the vertical 607.5 half-pixel crop rounds to a perfectly-centered 608 (Math.Round, ToEven);BuildEncoderOptionsfills W×H/FPS/bitrate from the tier once the URL provider yields one.- Media source decoder (TASK 21, Increment B — 2026-08-31):
MediaVideoSource(Services/MediaVideoSource.cs) decodes a local file toVideoFrames by spawning ffmpeg with-f rawvideo -pix_fmt bgra -vf scale=WxH -anand draining the raw BGRA stdout pipe through the pureRawVideoFrameReader(Services/RawVideoFrameReader.cs), raisingFrameAvailableper frame. ImplementsIMediaFrameSource(Services/IMediaFrameSource.cs:Key,FrameAvailable,Completed,StartAsync/StopAsync). The subprocess sits behindIDecodeProcess/FfmpegDecodeProcess(binary-stdout mirror of the encoder'sIEncoderProcess), and the path comes fromIFfmpegLocator— codec-agnostic, uses the ffmpeg already shipped, testable with a fake process. Decode sessions are owned app-wide byMediaVideoSourceManager(Services/MediaVideoSourceManager.cs): refcounted byMediaPath,Func<string, IMediaFrameSource?>factory seam (mirror ofScreenCaptureManager),AcquireAsync/ReleaseAsync/ReleaseAllAsync/GetLatestFrame, coalescing each file's frames onto the UI dispatcher onto one sharedWriteableBitmap. Frames emit as fast as the pipe produces them. Wired into the live pipeline (slice 1 step 4):ResolveOutputFramereads_mediaManager.GetLatestFrame(MediaPath)for aMediaSource, andSource.DisplaySource(Models/Source.cs) routesVideoImageSourceforMediaSourceso the preview/canvas show decoded video. Slice 2a shipped the probe seam:FfmpegFrameRateParser(pure,avg_frame_rate=thenr_frame_rate=rational N/N/M, unknown→null) +IFrameRateProbe/FfmpegFrameRateProbe(derives siblingffprobe.exefrom the located ffmpeg dir, reuses theIDecodeProcessseam for the ffprobe subprocess text; null if ffprobe absent) +FfmpegLocator.ProbeFileNamenow also extractsffprobe.exefrom the pinned archive (conditional — old caches without it just get no pacing). Slice 2b wired pacing:MediaVideoSourcetakes optionalIFrameRateProbe?+Func<TimeSpan,CancellationToken,Task>? delay(defaultTask.Delay) seams, probes FPS once inRunAsync, and delays by 1/fps after each emitted frame; no probe/unknown → no pacing (ffmpeg's own pipe backpressure already throttles the decode). Slice 3 added loop control:IMediaFrameSource.Looping(bool);MediaVideoSourcetakes aFunc<IDecodeProcess>process factory (a singleProcesscan't be re-Start()ed, so each loop pass creates a fresh decoder) and wraps the decode in ado…while (Looping)— restart on natural EOF instead of raisingCompleted. Production wiring (MainViewModel media factory): passes() => new FfmpegDecodeProcess()as the factory,FfmpegFrameRateProbe(new FfmpegLocator(), () => new FfmpegDecodeProcess())as the probe. Still open: session acquisition on add/remove (UI picker) + wiringSource.MediaIsLoopingintoIMediaFrameSource.Looping(needs a manager-level per-path loop provider, comes with the picker slice). - Social bar on the output (bar bug-fix branch): the
FramePumptakes an optionalsocialBar: Func<(VideoFrame? Frame, SocialBarPosition Position)>?seam, re-read every frame (so a mid-stream position flip applies immediately). The strip is pre-rasterized byCompositor/SocialBarRenderer.cs(WPF glue — WPF glue here is fine because the strip is rendered once per config change on the UI thread, the resulting immutable frame is then composited pure-CPU bySceneCompositor), andSceneCompositor.Renderblits it last — above the branding flash atsocialBarTop(0 = top,SourceRectHeight − barHeight= bottom) in master space.MainViewModelowns the frame (_socialBarFrame, rebuilt byRenderSocialBarFrameon load/save/NotifySocialsChanged). - Known consideration: the pump reads the active scene on a background thread while the UI can still edit
it; a concurrent-mutation exception is contained (logged +
Failed+ the pump stops) rather than crashing. The background thread + video pipeline is the new reality since this step.
Social bar (TASK 6 — shipped 2026-08-12, plan in TASKS.md; bar bug-fix branch 2026-08-13)
A global bar layer (never a Source, no sources-list row) that sits over the bottom or top of the
output and carries the creator's social links — content-sized, centered, GREEN glow when ON, position is a
click-toggle top ⇄ bottom (default BOTTOM, persisted SocialBarPosition). Models/Socials.cs: SocialService enum
(YouTube/Twitch/X/Instagram/TikTok/Facebook/Discord/Kick/Threads/Bluesky/GitHub/LinkedIn/Pinterest/
Snapchat/Reddit/WhatsApp/Telegram/Link/Website/Fediverse), SocialEntry (Service/Handle/ProfileUrl/
FediverseSoftware), SocialsConfig (Entries + BarPosition + BarEnabled; BarJustify dropped,
column back-compat). Configured in the "Social Media Site Promotion" dialog (SocialsDialog.xaml +
ViewModels/SocialsDialogViewModel, WPF-free, injected ISocialValidator + sign-in/sign-out fakes):
ON/OFF bar switch (schema v8), 6 fixed slots — row 1 always YouTube (signed-in → channel handle;
signed-out → sign-in gate → OAuth; delete → confirm sign-out), row 2 free, rows 3–6 lock icons on
freemium. Service detection (SocialServiceIcons.DetectService): URL domain / fediverse @user@domain →
Fediverse / bare→Website. Validation (Services/SocialValidator.cs, ISocialValidator seam +
HttpSocialValidator default): async GET of the canonical profile URL; 200/redirect = valid, 404/failure =
rejected. Fediverse additionally does a best-effort nodeinfo lookup (/.well-known/nodeinfo →
software.name, stored in SocialEntry.FediverseSoftware / the SocialEntry.Software column,
column-presence migration, no version bump) so the entry shows the instance's real logo
(LogoDataForFediverse: mastodon/peertube/pixelfed/misskey/lemmy/pleroma/firefish, generic fediverse
honeycomb fallback — Simple Icons CC0 path data, initials badges gone); nodeinfo failure still validates
(glyph falls back). If the identity domain's nodeinfo is blocked (YunoHost SSO gates
/.well-known/nodeinfo behind the login page) but the bare root 302s to the real instance, the lookup
follows the root redirect and asks the resolved host. Cancel is a hard stop: LookupAsync takes a
CancellationToken (dialog VM owns a CTS; Cancel/X/Save abort in-flight lookups, canceled continuations
never touch slot state).
Bar bug-fix branch (2026-08-13) — three changes:
- Positioning is a click-toggle (KISS) — the original drag set a local
Canvas.SetTopvalue that permanently overrides the{Binding SocialBarTop}(a binding can never win over a local value), so the bar stayed wherever it was dropped. The first drag-snap fix (SocialBarSnap.Decide+ClearValueon release) still failed for shaky hands — jitter around the ±6px deadzone snapped the bar up but wouldn't let it come back down. Superseded by the user's click-toggle: clicking the bar flips it top ⇄ bottom (MainViewModel.ToggleSocialBarPosition), the bar rides the binding alone, andSocialBarSnapis gone. - Fediverse software self-heal — the DB row
@gramps@llamachile.tubehadSoftware = NULLbecause nodeinfo was only ever asked of the identity domain (a landing page; the real instance ismastodon.llamachile.tube).HttpSocialValidator.ResolveFediverseSoftwareAsyncnow probes well-known subdomains (mastodon.→social.→ …FediverseSubdomainCandidates) when the identity domain and its redirect both come up empty, under a ~15s linked-CTS budget.MainViewModelruns the staticHealFediverseSoftwareAsyncoff the UI thread on layout load, applies matches via the dispatcher, and saves;SocialEntry.FediverseSoftwareis settable and raisesLogoData, so the icon updates in place. - The bar renders on the live output —
Compositor/SocialBarRenderer.csrasterizes the entries into a transparent straight-alpha BGRA strip (1920-wide, 40px content + 24px glow pad, the green glow baked in, Pbgra32→straight-alpha unpremultiply) and the compositor blits it above the flash (see "Live frame pipeline").
Toast notifications (TASK 24 — shipped 2026-08-23, plan in TASKS.md)
Non-blocking user-facing messages replace blocking MessageBoxes. Seam: INotificationService.Show(title, message, AppNotificationSeverity) (Services/INotificationService.cs) + Info/Success/Warning/Error
extension methods; MainViewModel news up the real NotificationService at its field initializer
(no DI container — the VM is built from XAML). Library: Notification.Wpf 11.0.0 (Platonenkov fork,
MIT — notice #10 in THIRD-PARTY-NOTICES.txt). Facts that cost a hunt:
- The
NotificationAreaXAML control registers itself with the manager onLoaded; routing matches the area'sNameagainst the request'sAreaName— our host isx:Name="ToastArea"in MainWindow's root grid (last child = topmost z-order,Grid.RowSpan=4, bottom-right above the footer viaMargin="0,0,12,96", outside the preview Viewbox). An unknown AreaName silently drops the toast. NeverExpires()setsExpirationTime = TimeSpan.MaxValue(NOT null);NotificationColor.ToHex()returns#AARRGGBB(alpha prefix).- Lifetimes: Info/Success ~4s, Warning ~8s auto-dismiss, Error sticky until dismissed (the pure mapping
lives in
NotificationService.BuildRequest+ theCardstint table: slate info#3a3b52, green success#1d5c38, amber warning#b8860b, dark-red error#8f1f1f— matching the health-banner language). - Calls marshal to the captured Dispatcher (
_dispatcher.Invoke) — encoder/pump events fire off-thread.
Migrated MessageBoxes: both webcam-acquire warnings, image-read warning, sign-in unsuccessful/failed. Promoted from log-only (each keeps its AppLog line): go-live prep failures ×3 (reusable stream, broadcast insert, prep exception), frame-pump death while live, mic-missing at startup, premium lapse, saved-session refresh failure (sign-out notice). Deliberately left log-only: offline license re-validation skip (would spam every launch). Still modal by design: SocialsDialog's slot-delete YesNo confirm.
Licensing — do not violate (GA = paid product; see THIRD-PARTY-NOTICES.txt)
This product is closed-source and paid. Every third-party component must stay inside the LGPL/BSD/MIT guardrails below — written down so a future "quick fix" never reintroduces a GPL binary. NEVER:
- Use a GPL FFmpeg build — gyan.dev's builds are GPLv3 and ship libx264; BtbN's
gplvariant is GPL too. GPL in a distributed paid product is the #1 lawsuit risk. Only BtbNlgpl/lgpl-sharedbuilds are allowed. - Distribute the static lgpl build — LGPLv2.1 §6 wants relinkable object files for static linking.
The shared (dynamic-DLL) build sidesteps that: compliance is "license text + source offer +
unmodified binaries". The pin is
lgpl-shared; when the pin is refreshed, keep the shared variant. - Use BtbN's
nonfreevariant — it adds fdk-aac (Fraunhofer code licensing). The native FFmpeg AAC encoder is fine (no Fraunhofer code) but grants no AAC patent license — accepted low-risk posture for RTMP→YouTube, since encoder vendors cover their implementations (Cisco OpenH264, NVIDIA NVENC, Intel QSV, AMD AMF). - Link FFmpeg into the app — it stays a separate subprocess fed frames over a pipe; that separation keeps the app's own code out of LGPL reach.
- Drop
THIRD-PARTY-NOTICES.txtfrom the shipped app or the About screen, or alter the FFmpeg copyright/LGPL notices inside the downloaded binaries. Automating the download counts as distribution — the obligations are not optional. - Pin to a moving target — the
latestBtbN release tag floats. Only immutable autobuild tags give a reproducible source offer. Record the tag + variant beside the URL (TASKS.md) every time the pin moves. - Use non-CC0 icon art — the social bar's bundled SVG logo path data comes from Simple Icons
(CC0 1.0, public domain — see
THIRD-PARTY-NOTICES.txt). Replacing or adding logos must stay CC0 or another public-domain source; a logo asset under a copyleft or attribution license would contaminate the paid product. The About hub's logo is the creator's own art ("llama fortnite superman logo",Assets/llama-logo.png) — owned by us, so it carries no third-party license either. - Forget the v1 license-texts gate —
THIRD-PARTY-NOTICES.txtlinks the canonical license texts; at v1 (GA) the full texts of every license it names MUST ship alongside it (TASK 4 requirement 9 is the release blocker). Queued early is wrong; the release pass owns it. - Break the in-app About requirement — since TASK 11 the About overlay is the app's in-app creator
hub: the real logo, the creator links (YouTube channel, Mastodon, Buy me a coffee, and the greyed
"Unlock Premium" seam whose billing URL is tabled), and a Licenses & legal sub-panel that loads
THIRD-PARTY-NOTICES.txtfrom the executable directory and renders it in a scrolling panel — never the OS viewer, no Notepad, no browser tab. The premium/coffee link constants live inMainViewModel(PremiumUrl,CoffeeUrl,ChannelUrl,MastodonUrl); filling in the billing URL lights up the premium button. Do not reintroduce an "open the notices in Notepad" button.
Design Principle
This software is so intuitive that even the most right-brained person can easily intuit and use it.
Apply this to every UI decision:
- One-click go-live with working defaults
- Prefilled YouTube defaults (RTMP URL, bitrate, resolution, latency)
- Visual/drag-and-drop scene building over property panels
- Every action produces a visible outcome — no dead ends
- "We're not them" (2026-09-01): assume the creator's hardware is mediocre, because it is. Every feature spends the machine's budget once — never twice for the same result (record OR stream, not both; one reusable stream; one webcam; two mixer inputs). If a feature only sings on a high-end chassis, it doesn't ship — the OBS escape hatch is open by design.
Monetization (design decision — the branding flash is the sword)
Full pricing, discount codes, and Polar product details in MONETIZATION.md (gitignored).
Brand palette and assets in MARCOM.md (gitignored). This section covers the in-app model.
Free forever: all features unlocked for everyone — no feature lock between free and paid. The only difference is watermarking. This is deliberate: no creator will tolerate a watermark, and as a good-guy developer, we give them complete access to every feature so no one can call us crooked.
- Free: a periodic full-frame branding flash — "made with LlamaCasty!" rendered big and centered
at ~25% opacity for about one second (soft 250ms fade in/out), repeated every 300s, on the live
output (and on local recordings). Implemented as
BrandFlashLayerin the preview compositor (MainWindow.xamlCanvasGrid) +BrandFlashTimerinMainViewModel— cadence 300s, first flash ~5s after go-live, only while live or recording. An always-on watermark can be cropped or covered; an intermittent full-frame flash can't be cropped and is impractical to edit around on a live feed. The flash is also the free tier's billboard — every free stream advertises LlamaCasty to its own viewers; the free tier is distribution, not compromise. Escalation model (2026-09-01, creator decision): the cadence is obnoxiously self-promoting — intervals shorten with use, starting at the 300s cadence and creeping toward a floor (the exact curve is a build-time design knob). License activation still flips exactly one bit:IsPremium→ flash off. Nothing else changes between free and paid, ever. Pre-GA posture: while the app is unreleased the flash renders in the preview only and is never composited onto the live output or local recording — test VODs stay clean (same channel- protection stance as the visibility lock), and creators can be shown what free looks like without it ever touching a real broadcast. Flipping the flash live-on is a TASK 36 unlock item. - Paid (one-time perpetual license): branding flash removed (flips
BrandFlashEnabledoff). That's it. No feature gating. Alerts, social bar slots, voice filters, TRAX, recording — everything is free. - Pricing (2026-09-21 — switched from subscription to one-time "own it"): $29 lifetime founder's
price (launch → 90 days) → $49 lifetime list at GA. The key is perpetual (
IsPremiumnever lapses; the old renewal/lapse path is dead). Rationale: a local app with no per-user server cost and a renewal-averse, free-surrounded audience — minimize commitment size, don't charge rent. Accessibility is the driver: the free tier is the full app (only the watermark differs), so a broke new streamer pays $0; the low one-time price is the "no recurring bill" entry. Full reasoning, and the dropped $49.99/yr → $99/yr subscription model, inMONETIZATION.md. - Billing: Polar (polar.sh) — open-source MoR (Apache 2.0), handles payments, one-time orders,
license keys, and global tax compliance. Startup Program gives Scale plan free for 12 months.
Product:
d105dfa1-497e-423b-8cd4-e0ee2e3abbc0(⚠️ currently a $99/yr subscription product — must be re-created as a one-time product for the new model). Checkout:llamacasty.com→ Polar hosted page. Org ID:c05fb364-b967-4f6c-adf2-8a144e46d085(org-scoped OAT —organization_idomitted from API calls). Key prefix:LCYT-. Discounts:LLAMAFOUNDER(100% off, 50 uses),LLAMA50(50% off, 12 months). Details inMONETIZATION.md. - Support (creator's model, corrected 2026-09-01): support = email + GitHub issues — the once-planned in-app bug-reporter is out of product (TASKS.md → closed list). Most queries are how-tos / feature requests / manual-skimmers. Maintenance cadence = "when I get around to it" with emergency patches; not a 24/7 service promise. The only license chatter is the paid-user expiry reminder — now moot (perpetual keys never expire, one-time model 2026-09-21); paid users see zero license UI. TASK 36 item 5 (expiry reminder) is superseded.
Monetization awareness (built-in, ungated, free — 2026-09-01; v1 SCOPE per the complete-v1 ruling — build order: capture → report → journey → Alerts): the app is monetization-aware by design, for every creator, free and ungated (the only subscriber swap remains the branding-flash removal above). It has three layers, all free — this is the product's differentiator, not a paid tier:
- Reward-event capture (the data foundation). The live-chat poll (
YouTubeChatService.Poll) already receives every reward event overliveChatMessagesbut today only decodes two (superChatEvent,newSponsorEvent); the other five —superStickerEvent,membershipGiftingEvent,giftMembershipReceivedEvent,memberMilestoneChatEvent,giftEvent— are silently dropped. The plan: decode all seven into a canonicalRewardEventand persist each to a SQLite table (RewardEvents: broadcastId, type, timestamp, amountMicros, currency, tier, memberLevelName, gifter/recipient channelIds).superChatEvents.list(30-day lookback) backfills prior broadcasts. This single feed is the foundation that Alerts, the session report, and the journey tracker all read. - Session / broadcast report. A session is created at Go Live; the poll + End roll that session's
reward totals (new members, Super Chat
, stickers, gifts, milestones). This is the in-app replacement for the emailed external "stream activity report" — the incumbents (Streamlabs et al.) email near-useless post-stream summaries; we track the same data natively and maintain it. - Journey tracker (TASK 39, slice 1 shipped 2026-09-22). A "YPP" tab below the
Stream Settings pull-out (Live screen right edge) shows the connected creator's position
toward each YPP tier threshold. Slice 1 = current-scope data only:
channels.list?mine=true &part=statistics,auditDetails,contentDetails(subs/views/videos + uploads-playlist id + live standing flags) +playlistItems.listfor 90-day public uploads. Every refresh appends aYppSnapshotto SQLite (schema v10) — the history a later slice needs for velocity/ETA. YPP thresholds are versioned, date-aware DATA, never constants — the Tier-2 bar doubles for new applicants on 2027-02-01 (long-form 4,000 → 8,000 qualified hrs / 365d; Shorts 10M → 20M / 90d), while the Tier-1 fan-funding bar (500 subs / 3,000 hrs / 3M Shorts) is unchanged. Hardcoding the pre-2027 numbers would ship the wrong ETA. Watch-hours / Shorts / velocity / ETA are slice 2: Analytics-API scopes (yt-analytics.readonly, additiveyt-analytics-monetary.readonly) behind theIChannelStatsProviderseam (additive capability, requires an OAuth re-consent). Compliance items no API exposes (2FA, AdSense linkage) are in-app self-reported checkboxes with deep links; standing checks are live fromauditDetails. Slice 2's once-daily refresh gate + snapshot history were baked into slice 1.
What was rejected: always-on watermark (obscurable — replaced by the flash), hard stream-time cutoffs (the worst dead end — a stream dying mid-broadcast reads as broken, and YouTube streams routinely run 2-4 hours), soft-limit nagging, freemium feature tiers, and donation-only (relies on the kindness of strangers). Resolution/quality ceilings stay rejected (fixed 2026-09-01, no longer "deferred"): the free tier never loses quality — auto step-down (TASK 33) is a protect the stream feature, never a monetization penalty.
Auth gates Go Live, but not exploration
The app is fully usable without authentication: users can build scenes, add sources, compose previews, and audition the software with zero commitment. But going live requires authentication — it's the one capability gated behind YouTube sign-in. The sign-in should never pressure the user ("sign in (optional)", not a modal wall): the two-state top bar shows Start Stream (offline) / End Stream (live), and the Start Stream dialog hosts the account — a saved session appears as the default with "Change Account"; with none saved, a "Sign in to YouTube" button starts OAuth and the Start button stays disabled until signed in.
Account assumption (do not build an account setup flow)
Connecting uses Google OAuth ("Sign in with Google") to link an existing YouTube creator account. ytLlive never creates or sets up accounts — that is YouTube's job. If the creator has no YouTube channel, they go to YouTube first. This assumption is explicit and must never be silently replaced by an in-app account-creation step. Zero state = the Start Stream dialog's "Sign in to YouTube" button; going live is unreachable until an account is connected.
YouTube Live API — design constraints (do not violate)
These are the hard facts behind every decision. Full list in TASKS.md.
- One-click go-live — never call
transition(live). Insert the broadcast withenableAutoStart=true,enableAutoStop=true,enableMonitorStream=false,selfDeclaredMadeForKids=false,latencyPreference=low. The encoder starting brings YouTube live.enableMonitorStream=falseis what lets us skip the testing stage. - Visibility picker (TASK 9 item 6) — DELIBERATE TEST-PHASE LOCK, not drift (2026-09-01).
The "always Private" enforcement in
YouTubeStreamService.CreateBroadcaststays while the creator runs multi-month real-world testing: non-private test streams would clutter the channel with VODs highlighting where the product breaks.recordFromStart/DVR stay ON — Private VODs are invisible to subscribers and serve as post-mortem review tapes; bulk-delete pre-GA. The unlock is a deliberate final-pass item in TASK 36 (gold pass), wired with the dialog selection — never opportunistic. The PRIVATE badge keeps showing when the stream is actually private. - Full broadcast form (TASK 9 item 7) — RESCOPED 2026-09-01 — the core editable fields ship (since 2026-08-24) as the always-visible Text drawer: title, description, tags, visibility, made-for-kids, live-editable; scheduling ships as TASK 34 (drawer ☑ Scheduled + adoption at Start). The old Advanced tab is permanently out of product (the 10% margin — see TASKS.md → "Out of product"): latency locked
low, DVR/record-from-start locked on, embed/projection/CC/region fixed at sane defaults, invisible — every exposed field is a support ticket.categoryIdis removed fromCreateBroadcast(not aliveBroadcastfield, silently ignored). Monetization enablement (if ever needed) rides the reward-events chain vialiveBroadcasts.update, not a form field. Go-live order (TASK 9, shipped 2026-08-16):BeginGoLive→PrepareAndStartLiveAsync— ensure the reusable stream (GetOrCreateReusableStreamAsync, cache it), create the broadcast bound to it (CreateBroadcast(..., stream.Id)→boundStreamId), THEN start the pump (the URL must exist beforeFramePump.StartAsync, which reads it once). Failure →StreamStatus.Error, never a crash;StopStreamclears_currentBroadcastId. - Variable reusable stream (shipped 2026-08-16) —
GetOrCreateReusableStreamAsynclistsliveStreams?mine=trueand reuses the existingcdn.isReusablestream, inserting once per channel (cdn.resolution=variable,cdn.frameRate=variable,isReusable=true) only on first use; the ingestion URL is cached viaLayoutStoreSettings (SaveReusableStream/LoadReusableStream) and bound to each broadcast at insert (boundStreamId). Any quality tier works without recreating the stream, and auto step-down rides the same property — we drop bitrate/resolution on the fly, zero API calls — but the deciding governor is not built: the old present tense there was a map-lie, corrected 2026-09-01; step-down is TASK 33 (v1 scope). - Quality is greyed out while live — resolution/frameRate/ingestionType are immutable after stream creation; editing title/description/privacy is fine at any time.
- Report-by-exception health (SHIPPED 2026-08-16, TASK 9 item 3) —
YouTubeStreamService.GetStreamHealthAsync(streamId)pollsliveStreams.list?part=status; render nothing ongood/ok/noData, surface a banner only onconfigurationIssues[]withwarning/errorseverity. The decision is the pureServices/StreamHealthReporter.BannerFor(null text = no banner; error beats warning). The VM polls every 30s while live (DispatcherTimer_healthPollTimer, first poll right after go-live) and clears on End viaResetHealth; poll failures log only. UI = a full-width banner strip under the top bar,HealthIssueBannertext +HealthIssueBackground(amber#b8860bwarning / dark red#8f1f1ferror), hidden byNotNullToVis. The design's bottom-strip YouTube logo + green/red dot (clickable → dialog) is still queued. - One dialog, three states — not connected / connected-offline (all editable) / live (title + description + visibility editable; quality + account greyed out). Both entry points (Start Stream button + bottom strip) open it; prefilled from saved session profile.
- End stream (SHIPPED 2026-09-01) — stop encoder →
YouTubeStreamService.EndBroadcastAsyncPOSTsliveBroadcasts/transition?broadcastStatus=complete&id=…&part=status(AFTER RTMP EOF, so no frames post-date the end — VOD finalizes immediately instead of ~1min of frozen "stream offline"), log-only on failure (invalidTransitionwhen autoStop already fired — never throws, never toasts a finished session);enableAutoStopremains the safety net. The design had always called for this call; it was never built until the recording-verification session caught the gap. Record-only stops make ZERO API calls (guard:wasLive && _currentBroadcastId != null). - Encoder compliance — keyframes ≤ 4s (gopSizeLong), closed GOP, H.264, AAC/MP3 @ 44.1/48kHz, mono/stereo only. YouTube flags violations via health status.
- Broadcast ID == Video ID — one ID tracks status, health, and the auto-created VOD
(
recordFromStart+enableDvrdefault true). - Test Stream mode (TASK 41, SHIPPED 2026-09-22) — every broadcast is private-only anyway
(see the visibility lock above), so "Test" is a session variant over the real go-live
pipeline, not a new mode:
BeginTestStream→StartStreamingSession(alsoRecord:false)(the dialog path was extracted into this shared boot — no Go Live dialog for a Test, noBroadcastForm.CaptureGoLive"last live" stamp, no recording).IsTestStreamflips the face: gold top bar + glow (#9c6f1c), a TEST badge, and the End button reads "End Test";StopStreamclears it in every path (including pump-failure rollback). Once the liveChatId resolves,TestSession.BeginTest(broadcastId, liveChatId)opens the TEST drawer (third right-rail pull-out, three-way one-open-at-a-time exclusivity wired inMainViewModel.Ypp.cs). Drawer tooling: Mock Chat Input = a REALliveChat/messages.insert(YouTubeStreamService.InsertChatMessageAsync) that round-trips through the real ~2s poll and renders through the live overlay path; simulated Subscriber / New Member / Super Chat =YouTubeChatService.InjectSimulatedMessage, which raises the poller'sMessageReceivedseam, markedChatMessage.IsSimulated. API fact (researched + hard constraint):liveChat/messages.insertonly createstextMessageEvents — SuperChats / memberships / sponsorships cannot be inserted via any API; simulated non-text events are local-only by design and never reach YouTube. Scopesyoutube+youtube.force-sslalready cover insert. Real stinger/TTS alert widgets remain TASK 3 items 16/17/20.