585 lines
54 KiB
Markdown
585 lines
54 KiB
Markdown
# ytLlive — AI Guide
|
||
|
||
> Memory map entry point. Conventions live in [`schema.md`](schema.md); task
|
||
> status and YouTube API research in [`TASKS.md`](TASKS.md); directory maps in
|
||
> each folder's `index.md`. Reading order: this file → `TASKS.md` → `<dir>/index.md` → source.
|
||
|
||
## Response style
|
||
|
||
No default "Plans & Pitfalls" / planning boilerplate. Respond directly and
|
||
concisely: **do the queued work, then report what changed and what's next.**
|
||
Skip feature pitch, step-by-step implementation plans, pros/cons tables, and
|
||
"potential pitfalls" sections unless the user explicitly asks for a plan first.
|
||
A short diff-style summary beats a proposal document every time.
|
||
|
||
### No-Fluff Mode (on demand)
|
||
|
||
Invoke with "no-fluff mode" (or similar) when you want ruthless review instead
|
||
of reassurance. In that mode:
|
||
|
||
- Strip all polite pleasantries, emojis, transitions, and conversational padding.
|
||
- Treat the user's input as a draft to be methodically deconstructed or
|
||
strengthened — argue, correct, and sharpen rather than agree.
|
||
- Give unvarnished truth, not reassurance.
|
||
|
||
This is an occasional, explicitly-invoked mode — **never the default**. The
|
||
default response style above stays in effect unless invoked.
|
||
|
||
## Run
|
||
|
||
```bash
|
||
# From WSL, ALWAYS use the Windows dotnet host — never Linux `dotnet` for this project:
|
||
"/mnt/c/Program Files/dotnet/dotnet.exe" build "C:\Users\gramp\Documents\Code\projects\ytLive\ytLive.csproj"
|
||
"/mnt/c/Program Files/dotnet/dotnet.exe" run
|
||
```
|
||
|
||
`EnableWindowsTargeting=true` in `ytLive.csproj` lets a cold restore work from WSL, but a Linux
|
||
`dotnet run`/`build` re-downloads 100M+ of `windowsdesktop.app.*` packs into the Linux NuGet cache
|
||
(which lacks them) over the slow 9p `/mnt/c` bridge — twice, because the WPF `_wpftmp` generated
|
||
project triggers a second restore (203s observed). The Windows cache has the SQLite packages and the
|
||
packs resolve from `C:\Program Files\dotnet\packs`, so the Windows host never re-downloads.
|
||
Never use `--no-restore` right after an interrupted restore — the stale `project.assets.json`
|
||
produces misleading `NETSDK1064` "package not found" errors. Running requires Windows anyway.
|
||
|
||
## Tests
|
||
|
||
xUnit in `ytLive.Tests` (net8.0-windows10.0.19041.0, matches the app TFM). Run on Windows —
|
||
WSL can't run net8.0-windows tests:
|
||
|
||
```bash
|
||
dotnet.exe vstest "C:\...\ytLive.Tests\bin\Debug\net8.0-windows10.0.19041.0\ytLive.Tests.dll"
|
||
```
|
||
|
||
Good Dog Rule: ONE integration test per branch, ONE test per PR. Current: TokenStore DPAPI
|
||
roundtrip/corrupt/missing/clear, OAuth exchange/refresh/ClearSession, CameraManager refcount +
|
||
frame pump + failure handling (fakes for the WinRT seams), real-`MainWindow` round-clip
|
||
interaction test, LayoutStore delete roundtrip, LayoutStore pre-round-rect-dims roundtrip,
|
||
LayoutStore backdrop roundtrip, LayoutStore HasBackdrop roundtrip + v5→v6 non-Live backfill,
|
||
ScreenCaptureManager refcount + shared-bitmap + coalescing
|
||
(fake `IScreenCaptureSource` + a real background-STA `Dispatcher`), BackdropTests (EnsureBackdrop
|
||
insert/idempotent/heal + HasBackdrop gate, IsLiveCapture, DisplaySource, INPC), SceneCatalogTests
|
||
(the five canonical scenes, Live-only backdrop policy, EnforceBackdropPolicy), WebcamSafeguardTests
|
||
(the per-scene size clamp incl. the Chat half-screen-area cap), SceneCompositorTests (the full-scene
|
||
composite integration test: backdrop + round webcam + mirrored/bordered images + flash; the vertical
|
||
tier 1080×1920 crop/scale), StretchMathTests (UniformToFill cover-crop + bilinear), FfmpegLocatorTests
|
||
(the PATH → cache → download decision ladder with a fake downloader serving a real in-memory zip; shared
|
||
build DLL extraction) —
|
||
78 passing.
|
||
|
||
### Real-MainWindow tests MUST be hermetic (DB pollution bug)
|
||
|
||
The integration test boots a real `MainWindow` → `MainViewModel` → real `LayoutStore`
|
||
(`%APPDATA%\ytLlive\ytLlive.db`). `Shutdown()` on close **saves the layout** (full rewrite:
|
||
DELETE all scenes/sources, re-insert), so any source a test adds would be persisted over the
|
||
user's real ones — this happened and wiped the real webcam source (DeviceId replaced by the
|
||
test's fake `test-camera`). Rule: a test that constructs `MainWindow` MUST first set
|
||
`MainViewModel.LayoutPathOverride` to a temp DB path and reset it (plus
|
||
`SqliteConnection.ClearAllPools()` + delete) in `finally`. The seam is
|
||
`internal static string? LayoutPathOverride` (line ~529 in `MainViewModel.cs`),
|
||
`ytLive.csproj` has `InternalsVisibleTo("ytLive.Tests")`.
|
||
|
||
The layout DB is a **full rewrite per save** (delete all, re-insert from memory), so
|
||
save/load round trips are exact: an element removed in the UI (`RemoveElement` →
|
||
`scene.Elements.Remove` → `OnElementsChanged` → debounced `ScheduleSave`, plus `Shutdown` on
|
||
close) does **not** come back after reload (`LayoutStorePersistenceTests` guards this).
|
||
|
||
## Architecture
|
||
|
||
C# / WPF (.NET 8) following MVVM:
|
||
|
||
| Path | Role |
|
||
|------|------|
|
||
| `Models/` | Plain data types — Scene, Source (incl. `ClipShape`, `IsMirrored`, `VideoImageSource`), QualityOption, StreamConfig, StreamHealth, YouTubeChannel, ChatMessage, **Socials (`SocialService` enum + `SocialEntry`/`SocialsConfig` + `SocialServiceIcons`) — the social bar** |
|
||
| `ViewModels/` | MainViewModel — exposes collections + commands for the UI; GoLiveViewModel, ReuseImageViewModel, CameraPickerViewModel, **SocialsDialogViewModel** |
|
||
| `Services/` | YouTube OAuth2, stream/broadcast management, live chat polling, LayoutStore (SQLite), **SocialValidator (`ISocialValidator` seam + `HttpSocialValidator` default)**, **webcam: `VideoFrame` seam + `CameraDeviceInfo`/`ICameraEnumerator`/`ICameraFrameSource` interfaces + `MediaCaptureCameraEnumerator`/`MediaCaptureFrameSource` (WinRT) + `CameraManager`**, **screen capture: `IFullScreenDetector`/`Win32FullScreenDetector` + `IScreenCaptureSource`/`ScreenCaptureFrameSource` (WinRT GraphicsCapture) + `ScreenCaptureManager` + `ScreenCaptureSourceFactory` + `Direct3D11Helper`/`CaptureInterop` (COM bridges)**, **compositor: `SceneCompositor` + `CompositorOptions` + pure `StretchMath` + `StaticPixelCache` (see "Scene compositor")**, **audio: `IAudioSource` seam + `WasapiLoopbackAudioSource`/`WasapiMicAudioSource` (NAudio WASAPI) + `AudioMixer` + pure `AudioLevelMeter`/`WaveToFloat` + game bar: `IGameAudioDetector`/`GameAudioHysteresis`/`GameAudioDetector` (see "Live audio capture")**, **encoder: `IFfmpegEncoder`/`FfmpegEncoder` + `IEncoderProcess`/`FfmpegEncoderProcess` + `IFfmpegLocator`/`FfmpegLocator` + pure `FfmpegArgs`/`FfmpegProgressParser`/`FfmpegEncoderPicker` + the `FramePump` frame producer (see "Live encoder" + "Live frame pipeline")** |
|
||
| `Helpers/` | ViewModelBase (INotifyPropertyChanged), RelayCommand, ImageCache, AppLog (file logger), FocusPreservingListBox, OAuthCredentials, **TokenStore (DPAPI session persistence)**, visibility converters |
|
||
| `Themes/` | `Controls.xaml` — the single dark-theme source, merged once in `App.xaml` (see `Themes/index.md`) |
|
||
| `MainWindow.xaml` | Dark theme; layout: top bar (controls), center (preview), left (scenes/sources), right (chat), bottom (health) |
|
||
|
||
### Key patterns
|
||
|
||
- `ViewModelBase.SetProperty<T>()` for property change notifications
|
||
- `RelayCommand` for all button actions; commands gate on state (e.g. Start only when Offline). **Typed `CommandParameter`s — no stringly-typed command tokens:** menu items that pick a source type pass the enum value itself (`CommandParameter="{x:Static models:SourceType.DisplayCapture}"`), so a typo breaks the build instead of silently adding an Image; `AddSource` still falls back to `Enum.TryParse<SourceType>(..., true)` for safety. The webcam item is its own `AddWebcamCommand` (it greys out via `CanAddWebcamToActiveScene` and isn't a `SourceType` — webcams are `WebcamSceneConfig`, not `Source` rows)
|
||
- **Audio is KISS by rule** — the whole of audio is *one knob*: **desktop/game audio is automatic** (WASAPI loopback from the default output at unity, zero UI — "it just is"); the **mic is the creator's only audio control** — sound meter + mute button + volume slider (`MicVolume`, defaults to 0.8) all sit together on the footer's top line, CENTERED beneath the preview panel. Meter: 288px, muted slate track (`#3a3b52`) with ruler graduations and muted yellow/red zone tints at 60%/80%; fill = green → yellow → red via `MeterFillWidth`/`MeterBrush`; the meter is a **READ-ONLY realtime level display** — it shows the live input level scaled by the volume (raising the volume moves ambient noise up the bar), NOT the volume setting: the fill is `Math.Min(1, AudioLevel * MicVolume)` (`AudioLevel` is fed by the audio mixer once capture lands, 0 with no input) and 0 while muted. While the volume slider is being dragged the bar previews the slider position (`SetVolumeAdjusting`, from `PreviewMouseLeftButtonDown/Up` + `LostMouseCapture` handlers) so the creator sees where they're setting it; on release it returns to the live level — with no input it bounces back to 0, exactly as it does today. Clicking the meter does nothing; **clicking the MIC label opens the mic picker** (`OpenMicPickerCommand`), and the picked voice source name (`MicSourceName`) is shown left-justified INSIDE the meter bar (FontSize 10, ellipsized to the bar) — the fill runs at 75% opacity so the text and the ruler markings stay visible through it. Mute (`ToggleMicMuteCommand`/`MicMuted`) is a plain clickable speaker icon (`MicSpeaker_MouseLeftButtonUp` code-behind handler — not a Button, `Stretch="Uniform"` so the glyph is never clipped) that swaps to a red do-not-symbol (slashed speaker) when muted. **The slider and the speaker can never disagree:** `MicMuted` is read-only, derived from `MicVolume == 0` — sliding the volume off flips the speaker to muted (storing the prior level in `_volumeBeforeMute`), sliding it up from 0 clears the mute indicator (and the stored level); the speaker button just runs the volume to 0 or restores it (default 0.8 if unknown). Muting zeroes the meter; **unmuting flashes the meter to the restored position for ~300ms** (`BeginVolumeFlash`/`EndVolumeFlash` on a DispatcherTimer, cancelled if the slider is grabbed) before it returns to the live level. Line 2 of the footer holds everything else: stream stats (bitrate/fps/dropped/duration/health) on the left, quality dropdown + gear on the right. The slider is a slim dimensional style in `Themes/Controls.xaml` (gradient track, beveled green fill on a 5px pill, gloss-sphere thumb with drop shadow — deliberately NOT flat). No device pickers (never show device names — no "install a device you didn't know existed"), no filter stacks, no monitoring, no routing — OBS's confusion (dynamic mixer, unintuitive names, four required filters) is deliberately absent. A production-ready mic chain (high-pass → noise gate → compressor) will be applied invisibly in the mixer, unconfigurable. Capture runs only while live (privacy indicator stays off otherwise). Capture pipeline = `IAudioSource` seam + NAudio `WasapiCapture`/`WasapiLoopbackCapture` + `AudioMixer` (pending — the UI is in place now). The connected account's avatar/name shows in the top bar next to Start Stream (`AccountAvatarUrl`/`AccountDisplayName` via `SyncConnectedAccount`), so the creator always sees WHICH account will go live.
|
||
- ViewModels are constructed in XAML (`<vm:MainViewModel/>` as DataContext)
|
||
- Services are currently instantiated in MainViewModel's constructor — no DI container yet
|
||
- Layout persists to SQLite (`Microsoft.Data.Sqlite`); scenes/sources/asset bytes stored in the DB, asset identity is a SHA-256 content hash (1:M reuse, no file paths — assets are always available)
|
||
- **Five-scene catalog (`Models/SceneCatalog.cs`):** the product is exactly Starting/Live/BRB/Chat/Ending — work with less, never more (the escape hatch for "more" is OBS). Scenes are matched **by name** (`SceneCatalog.Is`, case-insensitive trim). Empty DBs seed all five; the scenes-header "+" (`ShowAddScene`/`MissingScenes` on `MainViewModel`) only appears while ≥1 canonical scene is missing and its menu lists only the missing ones, re-adding them by name (`AddSceneCommand`). Renaming a canonical scene makes it missing again; `AddScene` rejects non-canonical names.
|
||
- Theming: all custom styles live in `Themes/Controls.xaml`, merged in `App.xaml` — never duplicate styles per-window (dialog duplicates were consolidated into this dictionary)
|
||
- Resolution tiers (bottom bar): 1080p60@8 (default) → 1080p30@8 → 720p60@6 → 720p30@6 → **Vertical 1080p60@8 (9:16, 1080×1920)**. The composition master frame is **always 1920×1080** — a tier is an output rect + target resolution over that master, so source geometry is never rewritten (no rounding drift). 16:9 tiers use the full frame; the vertical tier uses a centered **607×1080** window and the preview dims the cropped side strips at 55% black with an accent outline (semi-crop — the cut area stays visible). A resolution badge in the preview corner shows the active tier; the bottom bar shows bitrate/FPS. A **tooltip** explains finding upload bandwidth — an in-app speed test was deliberately dropped (unreliable). The future encoder crops the master to the rect and scales to the tier's Width×Height
|
||
- Crash diagnosis: `AppLog` writes startup checkpoints to `%APPDATA%\ytLlive\startup.log`; `App.xaml.cs` logs `DispatcherUnhandledException`/`AppDomain.UnhandledException`. When WPF won't run from WSL, this log is how you find the failure (it caught the `MenuItemRole.Separator` XAML crash and the ComboBox SelectionBoxItem bug)
|
||
|
||
### Current limitations / TODOs
|
||
|
||
- `Helpers/OAuthCredentials.cs` contains the real ClientId/ClientSecret. Auth is complete and the session **persists via Windows DPAPI** (`Helpers/TokenStore.cs` → `%APPDATA%\ytLlive\ytLlive.auth`, CurrentUser scope), reloaded best-effort at startup with a proactive refresh of a near-expiry access token. Sign-in/Change Account lives **inside the Start Stream dialog** (two-state flow — no separate Connect button). A **graceful End Livestream signs out**: `StopStream()` clears the session + token, so the next go-live needs a fresh sign-in; a crash never runs End, so the token survives and the creator stays signed in. `YouTubeAuthService` takes an optional `HttpClient` + `sessionChanged` callback (test seam + save hook; services are still constructed in `MainViewModel`)
|
||
- Scene/source/asset layout + the social bar persist (SQLite, schema v8); the OAuth session persists (DPAPI); the paid-unlock state does not (yet — itch.io key verification pending)
|
||
- `YouTubeStreamService` uses hardcoded `1080p`/`60fps` and per-broadcast streams — must switch to the v3 `variable` reusable stream
|
||
- Webcam capture is shipped (milestone 1); the live desktop/game backdrop is shipped (ship task #1); **the output compositor (TASK 4 ship step 1) is SHIPPED**, **the FFmpeg locator (TASK 4 ship step 2) is SHIPPED**, **the encoder + RTMP push (TASK 4 ship step 3) is SHIPPED**, **WASAPI audio capture (TASK 4 ship step 4) is SHIPPED** — full plan in `TASKS.md`; the frame-pipeline wiring follows (its own PR)
|
||
- `StreamConfig` defaults (`TargetBitrate=6000`, `Resolution="1920x1080"`) are stale — the live dropdown drives `StreamHealth.CurrentBitrate`/`FPS` instead
|
||
|
||
### Screen backdrop capture (TASK 3 ship task #1)
|
||
|
||
The backdrop is the **live desktop/game capture as a permanent, non-deletable bottom layer**
|
||
rendered in every scene — the "Screen" source from the minimal set, done as content-swap
|
||
instead of a normal draggable source.
|
||
|
||
- **Model (schema v6):** `Source.IsBackdrop` (persisted) marks the one backdrop per scene; `Source.CaptureKey`
|
||
(persisted) names the target — `monitor:<n>`, `window:<hwnd>`, or `picker:<displayname>`. The backdrop is
|
||
a real `Source` of `Type DisplayCapture`, inserted **first** (`MainViewModel.EnsureBackdrop(scene)`,
|
||
internal static — runs on layout load + every `AddScene`, healing any scene missing one), fixed at
|
||
X=0/Y=0/1920×1080, and excluded from drag/hit-test/remove/reorder (remove is guarded in `RemoveElement`;
|
||
`IsDraggableElement` never matches live types; the element template sets `IsHitTestVisible=false` for
|
||
backdrops; the row's remove button and "Remove Source" menu item are hidden). **`Scene.HasBackdrop`
|
||
(persisted, default off) is the Live-only policy flag** — the backdrop belongs to the canonical Live
|
||
scene alone (see `SceneCatalog`). `EnsureBackdrop` returns null for a flag-less scene, so
|
||
Starting/BRB/Chat/Ending compose their own layers. The one-time v5→v6 backfill turns those four scenes
|
||
off and drops their backdrop sources, and **`EnforceBackdropPolicy` (internal static, runs after every
|
||
load)** re-normalizes the flag by scene name and strips any backdrop that lingers in a non-Live scene —
|
||
the flag is owned by policy, never the user. There is no scene-list "Backdrop" checkbox anymore
|
||
(the old `ToggleSceneBackdropCommand` is gone); "Change Capture…"/"Refresh Capture"/"Capture Display"
|
||
only show in the Live scene's preview menu (`CanChangeBackdrop`). The static Background, if any,
|
||
renders **above** the backdrop.
|
||
- **Detection (launch + focus only, no live session listener):** `Win32FullScreenDetector` =
|
||
`GetForegroundWindow` + `DwmGetWindowAttribute(DWMWA_EXTENDED_FRAME_BOUNDS)` + `MonitorFromWindow` +
|
||
`GetMonitorInfo`; a window is full-screen when its frame covers all four monitor edges; own process is
|
||
excluded; monitor **index = `EnumDisplayMonitors` enumeration order** (the same order
|
||
`ScreenCaptureSourceFactory` uses to map index→HMONITOR via `Win32FullScreenDetector.GetMonitorHandle`).
|
||
`IFullScreenDetector` also exposes `GetDisplays()` (`DisplayInfo`: index/name/resolution/bounds/`IsPrimary`,
|
||
friendly name via `EnumDisplayDevices`) + `PrimaryMonitorIndex()` for the in-app "Capture Display"
|
||
submenu. At launch `ReacquireScreenCaptures` keys the backdrop to the full-screen game's monitor (or the
|
||
**primary** display — never assumed monitor 0).
|
||
On **Deactivated**, `NoteBackgroundWindow` samples the foreground ~250ms later (so an alt-tab to a game
|
||
lands before the app re-activates); on the next **Activated**, `RefreshBackdropAutoCapture` re-runs
|
||
detection against that sample and re-designates. Null detection (our app, the desktop, a normal window)
|
||
leaves the current capture alone.
|
||
- **Capture (WinRT GraphicsCapture):** `ScreenCaptureFrameSource` creates a free-threaded
|
||
`Direct3D11CaptureFramePool` (2 buffers, `B8G8R8A8UIntNormalized`) + `GraphicsCaptureSession`; frames →
|
||
`SoftwareBitmap.CreateCopyFromSurfaceAsync` (alpha ignored) → `VideoFrame` (BGRA8), bytes read via
|
||
`WindowsRuntimeMarshal.TryGetDataUnsafe` (the same CsWinRT-safe read the webcam path uses) — the
|
||
`IMemoryBufferByteAccess` ComImport cast threw `Invalid cast` on **every frame** under CsWinRT, which
|
||
flooded `startup.log` (~5 MB in a session) and burned CPU, so it is gone. Surfaces larger than the
|
||
1920×1080 master are downscaled bilinearly to the master (`DownscaleBgra`) before the copy, and
|
||
per-frame conversion failures are logged at most once per 5 s (`ErrorLogThrottle`). DRM-protected
|
||
content delivers black frames (OS limitation, documented). Frame pool
|
||
pauses while the app is minimized — capture keeps running, the pool just stops delivering.
|
||
- **Ownership:** `ScreenCaptureManager` mirrors `CameraManager` — refcounted by target key, one shared
|
||
`WriteableBitmap` per key, dispatcher-coalesced latest-frame copies, `PreviewBitmapChanged`/`CaptureFailed`
|
||
events, `ReleaseAllAsync` on re-designation. `ScreenCaptureSourceFactory.Resolve(key)` parses the key into
|
||
a source; `PickAsync()` shows the OS `GraphicsCapturePicker` ("Change Capture…", owner window set via the
|
||
`IInitializeWithWindow` ComImport) and returns a **transient** `picker:` key — a reload falls back to
|
||
auto-detection.
|
||
- **CsWinRT projection gaps hand-rolled:** `Windows.Graphics.Direct3D11.Direct3D11Helper` is not projected,
|
||
so `Direct3D11Helper` P/Invokes `d3d11.dll!D3D11CreateDevice` (hardware, BGRA_SUPPORT, explicit 11.1-first
|
||
feature array) → QI `IDXGIDevice` → the WinRT interop export
|
||
`CreateDirect3D11DeviceFromDXGIDevice` → `MarshalInterface<IDirect3DDevice>.FromAbi` (one shared device per
|
||
process). Do **not** switch back to the QI-for-`IDirect3DDxgiInterfaceAccess` trick: the raw D3D11 device
|
||
no longer exposes that interface on newer Windows (verified E_NOINTERFACE on build 26200, hardware and
|
||
WARP alike) while `CreateDirect3D11DeviceFromDXGIDevice` keeps working.
|
||
`IInitializeWithWindow` are ComImports in `CaptureInterop.cs`. All WinRT projections were verified by
|
||
reflection against the built `Microsoft.Windows.SDK.NET.dll` before writing the interop.
|
||
- **Known v1 limits:** full-desktop captures are CPU-copied at native resolution (GPU downscale = encoder
|
||
task); window capture (`window:<hwnd>`) and multi-monitor live re-targeting beyond the auto-detected
|
||
game are behind the picker; picker-based captures don't survive reload.
|
||
- **GPU posture:** same as webcam — CPU frames, WPF hardware-presents; D3DImage GPU compositing deferred
|
||
to the encoder task.
|
||
- **Preview watermark:** the "Preview" placeholder hides while a backdrop renders —
|
||
`ShowPreviewPlaceholder` now also checks `BackdropImage` (raised on backdrop change), so a scene with
|
||
live capture shows the feed instead of the "nothing here" label.
|
||
|
||
### Webcam capture (TASK 3 milestone 1)
|
||
|
||
- **Seam-first:** everything above the WinRT layer speaks only `VideoFrame` (normalized tightly-packed
|
||
BGRA8) + `CameraDeviceInfo`/`ICameraEnumerator`/`ICameraFrameSource` interfaces. Tests inject fakes;
|
||
screen capture and background removal later feed the same seam.
|
||
- **CPU-first:** `MediaCaptureInitializationSettings { MemoryPreference = Cpu, StreamingCaptureMode = Video,
|
||
SharingMode = SharedReadOnly }`, frames pulled via
|
||
`CreateFrameReaderAsync(colorSource, MediaEncodingSubtypes.Bgra8)` — the pipeline does any format
|
||
conversion, so every `FrameArrived` yields a ready BGRA8 `SoftwareBitmap` (bytes read via
|
||
`WindowsRuntimeMarshal.TryGetDataUnsafe`, not marshalled copies).
|
||
- **Source pick, not first hit:** the frame reader is bound to the first source that is `VideoPreview`
|
||
(preferred) or `VideoRecord`, not blindly the first preview source. If a camera exposes neither, the
|
||
failure names the device and the stream types it *does* expose. `SharedReadOnly` lets the capture
|
||
coexist with other apps that share the camera.
|
||
- **Known failure: NVIDIA Broadcast** — it opens the physical webcam exclusively, so `InitializeAsync`
|
||
fails with "camera in use" (or, if init slips through, the device exposes no preview source). Fix:
|
||
quit Broadcast while streaming, or pick its virtual "NVIDIA Broadcast" device from the picker and the
|
||
app captures the processed feed. This is a real-device finding (Logitech `VID_046D&PID_082D`).
|
||
- **TFM:** `net8.0-windows10.0.19041.0` (app + tests) pulls the WinRT projection from the SDK reference
|
||
packs — no NuGet package, no capability manifest (unpackaged desktop app works; the Windows privacy
|
||
camera toggle still applies). `EnableWindowsTargeting` keeps WSL builds working.
|
||
- **One webcam, many scenes (schema v3):** a singleton `Webcam` row holds the identity
|
||
(`Id`/`DeviceId`/`Name`); each scene gets its own `WebcamSceneConfig` (position/size/clip/mirror/
|
||
border/`IsVisible`). `Scene.Elements` holds images (`Source`) and, at most once, the webcam
|
||
(`WebcamSceneConfig`); `Scene.WebcamConfig` is the accessor. `CameraManager` refcounts capture
|
||
sessions by `DeviceId` (a session starts at `RefCount = 1`; repeat acquire bumps it; the last
|
||
release stops + disposes). **"Add Webcam" ALWAYS opens the Windows camera picker** — the creator is
|
||
never silently handed the previous camera (which used to happen after deleting one scene's webcam
|
||
while another scene still used it; that identity survived, so re-adding bypassed the choice).
|
||
Picking a different camera than the current app-wide one swaps it everywhere via
|
||
`SwapWebcamIdentityAsync` (the same path "Change Webcam…" uses), keeping the singleton honest;
|
||
picking the same one just places the config. The Add Webcam menu greys out when the **active**
|
||
scene already has a config (`CanAddWebcamToActiveScene`); showing a hidden webcam reuses the
|
||
existing config (`CanShowWebcamInActiveScene` / empty-canvas right-click "Show Webcam" — which, on
|
||
a config-less canvas, delegates to Add Webcam and so also picks). **Removing the last webcam
|
||
config anywhere clears the identity** (`_webcam = null`), which also drops "Change Webcam…".
|
||
- **Round→rect restores the aspect (persisted, schema v4):** `SceneElement.ToggleClipShape()`
|
||
snapshots the rectangular Width/Height into public `RectWidth`/`RectHeight` before going Round and
|
||
restores them when switching back — otherwise the Round resize lock (square) would leave a square
|
||
behind. The rect dims are **persisted** (`WebcamSceneConfig.RectWidth`/`RectHeight`, nullable), so a
|
||
reloaded Round webcam still restores its pre-Round aspect instead of staying square. A one-time
|
||
`HealLegacySquareRect` (load only) widens a pre-v4 `Traditional` config that ended up square to 16:9
|
||
(keeps height; Round and explicit rect dims are untouched).
|
||
- **Device swap / layout reload:** `ChangeWebcamAsync` (picker) and `ReacquireWebcam` (after load)
|
||
release the old device with `ReleaseAllAsync` — a forced full drop that zeroes the refcount and
|
||
stops the source regardless of how many scenes held it (the per-config count isn't known once the
|
||
scenes are replaced) — then `AcquireAsync` the new device once per config.
|
||
- **Shared bitmap, coalesced updates:** one `WriteableBitmap` per active camera, created on the UI thread
|
||
at the device's frame size (first frame), forwarded to every `WebcamSceneConfig.VideoImageSource` via
|
||
`PreviewBitmapChanged`. Frames arrive on a worker thread; `CameraManager` coalesces onto the dispatcher
|
||
(at most one pending copy per session, at `Render` priority, always copying the latest frame) so a 60fps
|
||
device never drowns the render thread.
|
||
- **Webcam added mid-session must get the live frames:** `PreviewBitmapChanged` fires **once** (the first
|
||
frame creates the shared bitmap); later frames only mutate that bitmap in place, so a config that didn't
|
||
exist at first-frame time would never receive it — the empty/transparent container you'd see adding a
|
||
webcam to Chat while Live already had the camera. `CameraManager.GetPreviewBitmap(deviceId)` exposes the
|
||
current shared bitmap; `AddWebcamToActiveSceneAsync` assigns it to the new config right before
|
||
`AcquireAsync`, and `ReacquireWebcam` re-propagates it to every config after a reload.
|
||
- **Clip/mirror/border:** per-element `ClipShape` (Traditional rectangle / Round ellipse) + `IsMirrored`
|
||
(`ScaleX = -1`) + the OSB-standard static border (`BorderColor` `#RRGGBB` or `""`=none, `BorderOpacity`
|
||
0–1, `BorderWidth` 0–20, `BorderAnimation` `None|Pulse|Chase|Rainbow|Shimmer|MarchingAnts|Glow|
|
||
Electricity|Sparkles`). Rendered in the preview DataTemplate; toggled from the element's right-click
|
||
context menu (webcam menu: Change Webcam…, Border Effect submenu — all 9 items enabled, values persist,
|
||
rendering stays static until the animation tier ships — Border Color, Opacity/Thickness sliders, Hide in
|
||
this scene, Remove); persisted in the layout DB. The Add menu shows when no webcam exists; the empty
|
||
preview canvas has its own Show Webcam entry.
|
||
- The Round webcam is an `Image Stretch="UniformToFill"` with an `EllipseGeometry` clip
|
||
(`Center=0.5,0.5` `RadiusX/Y=0.5`), inside a `Viewbox Stretch="Uniform"` holding a `1x1` Grid, so it
|
||
renders as a true circle (diameter = the shorter element dimension) instead of an oval stretched to
|
||
the element rect — the traditional `Image` keeps `UniformToFill` over the full rect. The clip is
|
||
geometry, **not an `ImageBrush`**: a brush re-rasterizes the frequently-updated `WriteableBitmap` per
|
||
frame on the render thread, which is what made the live webcam crawl while round. The Round border is
|
||
a centered `Ellipse` at `Width/Height = RoundBorderSize`.
|
||
- Resizing locks to a square (`_resizeAspect = 1`) while `ClipShape == Round`.
|
||
- **Webcam size clamp:** `ClampWebcamToBounds(config, sceneName)` (internal — test seam) enforces the
|
||
max per dimension at resize + load and no less than 10% of the master (192×108). The cap is picked
|
||
**by canonical scene name**: 50% per dimension (960×540) everywhere except the **Chat scene**, which
|
||
may reach half the screen **area** (~1358×764 @16:9) so the viewer sees the creator better
|
||
(`MaxWebcamWidthFor`/`MaxWebcamHeightFor`; a renamed Chat loses the bigger cap).
|
||
`RoundBorderSize` follows the clamped height. `WebcamSafeguardTests` guards the clamp.
|
||
- **Hit-testing:** a `Grid` without `Background` only hit-tests where its children draw, so clicks in
|
||
the empty corners of a round clip fell through to `Window_PreviewMouseLeftButtonDown` and deselected
|
||
the element — making the corner handle ungrabbable. The element Grid carries
|
||
`Background="Transparent"` (whole rect draggable; the empty canvas Grid uses the same trick for
|
||
right-click Show Webcam) and the `SelectionOverlay` (dashed border + corner dot) is
|
||
`IsHitTestVisible="False"` so it never intercepts the click. Two things make the webcam menu work:
|
||
(1) the `ContextMenu` pins its own `DataContext` to `PlacementTarget.DataContext` — a `ContextMenu`
|
||
isn't in the visual tree, so without it the Click-handler `DataContext:` patterns (and the
|
||
IsChecked/slider bindings) silently fail; (2) `Themes/Controls.xaml` ships a full dark `MenuItem`
|
||
template — `PART_Popup` (submenu popups), a popup `ItemsPresenter` (the Border Opacity/Thickness
|
||
sliders live in Items, so they render in a hover flyout), a `✓` checkmark column, and a `›` arrow
|
||
driven by `HasItems`. An earlier bare `Border + Header` template dropped all three: submenus never
|
||
opened, sliders never rendered, checkmarks never showed — the menu looked dead even though the
|
||
Click handlers were fine.
|
||
- **GPU posture:** webcam frames are CPU (GPU-agnostic; WPF hardware-presents the preview anyway). Hardware
|
||
encoders (NVENC/AMF/QSV) matter for the encoder task, not capture. D3DImage GPU compositing is deferred
|
||
to the encoder task.
|
||
- **Background removal = milestone 2** — ONNX Runtime + DirectML (CPU fallback), MediaPipe Selfie
|
||
Segmentation (Apache-2.0), wired into the same `VideoFrame` seam. Not part of milestone 1.
|
||
|
||
### Scene compositor (TASK 4 ship step 1 — shipped 2026-08-10, plan in TASKS.md)
|
||
|
||
The encoder needs the master 1920×1080 frame **without** the preview's editing chrome (SelectionOverlay,
|
||
DimRects, output-rect outline, badge, placeholder). WPF's `RenderTargetBitmap` is software-rendered and
|
||
captures the visual tree *including* chrome, so the preview can't be captured — the output is a **second,
|
||
parallel software compositor** over the `VideoFrame` (BGRA8) seam, and the XAML preview
|
||
(`MainWindow.xaml` CanvasGrid + element DataTemplate) is the rendering contract it replicates. Two
|
||
renderers must agree: geometry, `UniformToFill` cover-crop, round clip, mirror, border, z-order. Preview
|
||
stays XAML (editing view); the compositor is the output view.
|
||
|
||
- **Render the active tier's output rect directly** (`CompositorOptions {SourceRectX/Y/W/H,
|
||
OutputWidth, OutputHeight}`, fed from `MainViewModel.OutputRect*`): 16:9 = full 1920×1080 1:1; the
|
||
vertical 9:16 tier = composite the centered 607×1080 crop then bilinear-upscale to 1080×1920.
|
||
- **CPU posture:** with FFmpeg as a subprocess the master crosses a CPU readback to the pipe every frame
|
||
anyway, so GPU compositing buys little at this layer count (2-3 live layers; static layers
|
||
pre-composite once into a cached base). GPU effort belongs to **NVENC** (the encoder), not composition;
|
||
if composition grows (wipes, filters, many layers), a D3D11 compositor can replace this one **behind
|
||
the same seam** — the CPU master buffer stays the contract.
|
||
- **Branding flash is composited by the output path too** (it's on the live output, per Monetization),
|
||
passed in as a pre-rendered `VideoFrame?` — the compositor core stays pure byte-math, no WPF. Likely a
|
||
bundled asset rather than runtime text rendering (deterministic, no font/layout risk).
|
||
- **Frame sources are injected** via a `Func<SceneElement, VideoFrame?>` resolver
|
||
(`SceneCompositor.Render(scene, frameFor, flashFrame, options)`) — the caller maps each element to
|
||
its frame (webcam → `DeviceId`, image → `AssetId` via `StaticPixelCache`, backdrop → `CaptureKey`),
|
||
so the compositor is pure, WPF-free, and hermetic to test. The capture managers wire into that
|
||
resolver in the encoder step, not the compositor step. The master buffer (the compositor's return
|
||
value) is the seam a future D3D11 compositor would honor identically.
|
||
|
||
### FFmpeg locator (TASK 4 ship step 2 — shipped 2026-08-10, plan in TASKS.md)
|
||
|
||
The encoder's one external dependency is `ffmpeg.exe`; it's never shipped in the repo. `IFfmpegLocator`
|
||
resolves an absolute path on demand: **PATH probe first** (the user's own install wins — their choice,
|
||
their responsibility), then the cache (`%APPDATA%\ytLlive\tools\ffmpeg.exe`), then a **pinned** BtbN
|
||
LGPL-**shared** win64 zip (~75 MB) from which `ffmpeg.exe` **and the `libav*.dll` family** are extracted
|
||
(staged temp-write + move so a crash never corrupts the cache; Windows resolves the DLLs from the exe's
|
||
own directory). BtbN LGPL-shared (not gyan.dev, not static): it drops GPL-only libx264/x265 while keeping
|
||
NVENC/QSV/AMF + libopenh264 + native AAC, and dynamic linking means LGPL compliance is "license text +
|
||
source offer" with no static-relink (§6) material — see the Licensing guardrails below. The pin is a
|
||
dated autobuild tag (immutable); BtbN retention keeps the last 14 daily + each month-end for 2 years, so
|
||
a cold cache can outlive the pin → the seam throws a clear, logged error (recoverable; the pin is one
|
||
const). Constructor-injected search dirs / tools dir / downloader (`Func<string, CancellationToken,
|
||
Task<byte[]>>`) keep it hermetic: tests fake the network with a real in-memory zip. Constructed in the
|
||
encoder step (not yet — this PR ships the seam + impl + tests only).
|
||
|
||
### Live encoder + RTMP push (TASK 4 ship step 3 — shipped 2026-08-12, plan in TASKS.md)
|
||
|
||
The encoder is a **thin orchestrator over `ffmpeg.exe`** — no H.264/AAC code in the app. It spawns the
|
||
subprocess (path from `IFfmpegLocator`), feeds raw BGRA master frames into stdin, and parses `-stats`
|
||
stderr lines into `StreamHealth` (bitrate/FPS/duration, dropped-from-frame-count). `FfmpegEncoder`
|
||
(`IFfmpegEncoder` seam) holds: `StartAsync` (locate → probe `-encoders` → spawn → stderr loop),
|
||
`SubmitFrameAsync` (serialized stdin writes under `SemaphoreSlim`), `StopAsync` (stdin EOF → ffmpeg
|
||
finalizes + exits by itself; a 10s watchdog kills it), `Dispose` (force-kill + wait), and the
|
||
`HealthUpdated`/`ProcessFailed` events. **Pattern:** the encoder never touches `Process` — it drives the
|
||
`IEncoderProcess` seam (`FfmpegEncoderProcess` wraps the real `Process`, redirected stdin/stdout/stderr
|
||
+ exit control); a `Func<IEncoderProcess>` factory + the locator are constructor-injected, so the
|
||
integration test fakes the whole subprocess (probe + encoder) with a Channel-backed `TextReader` whose
|
||
`Complete()` is EOF (`null`), never a `ChannelClosedException`.
|
||
|
||
**Decisions (locked):** args are pure (`FfmpegArgs.Build`, no string building in the encoder):
|
||
`-re -f rawvideo -pix_fmt bgra -video_size WxH -framerate FPS -i pipe:0` + a **silent placeholder
|
||
`-f lavfi -i anullsrc`** track (WASAPI capture, ship step 4, replaces it) + `-c:v <enc> -b:v K
|
||
-maxrate K -bufsize 2K` + **`-g fps×4 -keyint_min fps×4 -sc_threshold 0 -bf 0 -pix_fmt yuv420p`**
|
||
(≤4s keyframes, closed GOP, H.264 compliance) + `-c:a aac -ar 48000 -ac 2 -f flv <rtmpUrl>`.
|
||
**Encoder choice is probed from the binary's `-encoders` listing** (`FfmpegEncoderPicker`, pure):
|
||
NVENC → QSV → AMF → OpenH264 fallback, **never libx264** (GPL; see Licensing). `EncoderOptions.VideoEncoder`
|
||
forces one and skips the probe. `EncoderOptions` also carries W×H/FPS/bitrate from the quality tier and
|
||
`GopSize` = FPS×4. Constructed by `MainViewModel` since ship step 5 (`new FfmpegEncoder(new FfmpegLocator())`),
|
||
driven by the `FramePump` below.
|
||
|
||
### Live audio capture (TASK 4 ship step 4 — shipped 2026-08-12, game audio bar follow-up 2026-08-13, plans in TASKS.md)
|
||
|
||
**Capture runs for the app's lifetime and is KISS by rule**: desktop/game audio is automatic (WASAPI
|
||
loopback), the mic is the creator's only audio control (meter/mute/volume already shipped). The whole
|
||
layer sits behind an **`IAudioSource` seam** (`Services/Audio/`: `Start`/`Stop`/`Started`/`SampleReady`/
|
||
`Failed`, IDisposable) so the app never touches NAudio directly and tests inject hermetic fakes (no
|
||
devices, no timers).
|
||
|
||
- **Sources (NAudio `NAudio.Wasapi` 2.2.1, MIT — item 9 in `THIRD-PARTY-NOTICES.txt`):**
|
||
`WasapiLoopbackAudioSource` = `WasapiLoopbackCapture` on the default render device;
|
||
`WasapiMicAudioSource` = `WasapiCapture` with the device resolved by `FriendlyName` matching
|
||
`MicSourceName` (the app only persists the **DisplayName**), falling back to the default capture
|
||
endpoint. Mic device resolution re-reads the name provider `Func<string?>` at each `Start`, so a mic
|
||
picked mid-session takes effect **immediately** (the mixer restarts the mic on pick). Both sources
|
||
raise `Started` once their capture loop actually begins — the mixer turns that into `MicConnected`.
|
||
- **`AudioMixer`** owns both sources; **`StartMicCaptureAsync` starts the mixer once at startup** and
|
||
`Shutdown` disposes it — NOT go-live — so both footer meters preview live (`BeginGoLive`/`StopStream`
|
||
no longer touch the mixer). Mic samples feed a pure **`AudioLevelMeter`** (RMS with 0.2 exponential
|
||
smoothing) → `MicLevelChanged` → marshalled to the UI thread → `AudioLevel`; loopback samples feed a
|
||
second meter → `LoopbackLevelChanged` → the game bar's `GameAudioLevel`. **Mic connection state
|
||
surfaces as events:** `MicConnected` (source `Started`), `MicFailed` (source `Failed`), and
|
||
`RestartMic()` re-resolves + restarts just the mic (loopback keeps running). Failures log via
|
||
`AppLog`; a mic failure zeroes the meter, a loopback failure never kills the mic. Meter `Push` is
|
||
unconditional (a `?.` on the event would skip the argument — and the meter update — when nothing is
|
||
subscribed yet).
|
||
- **Mic status dot (`Models/MicStatus.cs`)** on the footer's MIC button: green = `MicConnected`,
|
||
yellow = `MicFailed` (in use/unplugged), red = no mic device at startup (the mixer is never started,
|
||
so loopback and the game bar can't run either — no capture devices at all).
|
||
- **Game audio bar** (desktop/game, only while a full-screen game is up in the preview):
|
||
`IGameAudioDetector` seam (`Services/IGameAudioDetector.cs`), pure `GameAudioHysteresis` (SHOW after
|
||
~500ms of fullscreen + sound, HIDE after ~1s away from fullscreen, **silence never hides an active
|
||
bar**), and the default `GameAudioDetector` composing `IFullScreenDetector` + the live loopback level
|
||
(floor 0.5%). WPF-free — the VM owns a 250ms `DispatcherTimer` that polls it and flips
|
||
`IsGameAudioBarVisible`.
|
||
- **`WaveToFloat`** (pure, shared): WASAPI mix formats → interleaved float — IEEE float 32-bit direct,
|
||
PCM 16-bit normalized to -1..1, `WaveFormatExtensible` with the IEEE-float subformat GUID
|
||
(`NAudio.Dmo.AudioMediaSubtypes.MEDIASUBTYPE_IEEE_FLOAT`), trailing partial samples ignored.
|
||
- Build **0 warnings**; **167 passing** (mixer/hysteresis/game-detector unit tests).
|
||
|
||
**Deferred:** the loopback→encoder AAC mix wiring (a later step — the encoder construction + frame
|
||
pipeline shipped in ship step 5).
|
||
|
||
### Live frame pipeline (TASK 4 ship step 5 — shipped 2026-08-12, plan in TASKS.md)
|
||
|
||
The **`FramePump`** (`Services/Encoder/`) is the live frame producer: while live it snapshots the active
|
||
scene each tick, resolves every element to its latest frame, composites it into the tier's output frame,
|
||
and paces frames into the encoder at the tier's FPS. **Pattern — everything is a constructor-injected
|
||
seam:** `Func<Scene?>`, `Func<SceneElement, VideoFrame?>` resolver, `Func<CompositorOptions>`,
|
||
`Func<EncoderOptions?>`, `Func<IFfmpegEncoder>`, `Action<string>` log, and an injectable pacing delay
|
||
(default `Task.Delay`; tests inject `Task.Yield`). The pump is free of WPF and of the capture managers.
|
||
|
||
- **`StartAsync` never throws** — the VM fires-and-forgets it from the sync command handler; failures log
|
||
+ surface via the `Failed` event. `EncoderOptions == null` means "no RTMP URL": the pump logs and skips
|
||
the encoder entirely. `MainViewModel._rtmpUrlProvider` is that seam — a `Func<string?>` returning null
|
||
until TASK 5 supplies the reusable stream's ingest URL, so go-live runs the current visual flow.
|
||
- **Stop ordering matters:** `StopAsync` stops the encoder (closes stdin → EOF → ffmpeg finalizes+exits)
|
||
**before** awaiting the loop, because closing stdin unblocks a write stuck on pipe backpressure — the
|
||
reverse order would deadlock. `ProcessFailed` self-stops the pump. `HealthUpdated` is forwarded
|
||
(ship step 6 binds it to the bottom bar); `Failed` while live flips `StreamStatus.Error` (minimal).
|
||
- **`MainViewModel` owns the resolver** (`ResolveOutputFrame`): `WebcamSceneConfig` →
|
||
`CameraManager.GetLatestFrame(WebcamId)`, `Source { IsLiveCapture, CaptureKey }` →
|
||
`ScreenCaptureManager.GetLatestFrame(CaptureKey)` (the new accessor mirroring `CameraManager`), image/
|
||
background → `StaticPixelCache.Get(AssetId)`. `BuildCompositorOptions` rounds the VM's `OutputRect*`
|
||
doubles to ints — the vertical 607.5 half-pixel crop rounds to a perfectly-centered **608** (`Math.Round`,
|
||
ToEven); `BuildEncoderOptions` fills W×H/FPS/bitrate from the tier once the URL provider yields one.
|
||
- **Social bar on the output (bar bug-fix branch):** the `FramePump` takes an optional
|
||
`socialBar: Func<(VideoFrame? Frame, SocialBarPosition Position)>?` seam, re-read **every frame** (so a
|
||
mid-stream position flip applies immediately). The strip is pre-rasterized by `Compositor/SocialBarRenderer.cs`
|
||
(WPF glue — WPF glue here is fine because the strip is rendered once per config change on the UI thread,
|
||
the resulting immutable frame is then composited pure-CPU by `SceneCompositor`), and `SceneCompositor.Render`
|
||
blits it **last — above the branding flash** at `socialBarTop` (0 = top, `SourceRectHeight − barHeight` =
|
||
bottom) in master space. `MainViewModel` owns the frame (`_socialBarFrame`, rebuilt by `RenderSocialBarFrame`
|
||
on load/save/`NotifySocialsChanged`).
|
||
- Known consideration: the pump reads the active scene on a background thread while the UI can still edit
|
||
it; a concurrent-mutation exception is contained (logged + `Failed` + the pump stops) rather than
|
||
crashing. The background thread + video pipeline is the new reality since this step.
|
||
|
||
### Social bar (TASK 14 — shipped 2026-08-12, plan in TASKS.md; bar bug-fix branch 2026-08-13)
|
||
|
||
A **global bar layer** (never a Source, no sources-list row) that sits over the bottom or top of the
|
||
output and carries the creator's social links — content-sized, centered, GREEN glow when ON, position is a
|
||
**click-toggle top ⇄ bottom** (default BOTTOM, persisted `SocialBarPosition`). `Models/Socials.cs`: `SocialService` enum
|
||
(YouTube/Twitch/X/Instagram/TikTok/Facebook/Discord/Kick/Threads/Bluesky/GitHub/LinkedIn/Pinterest/
|
||
Snapchat/Reddit/WhatsApp/Telegram/Link/Website/**Fediverse**), `SocialEntry` (Service/Handle/ProfileUrl/
|
||
`FediverseSoftware`), `SocialsConfig` (Entries + `BarPosition` + `BarEnabled`; `BarJustify` dropped,
|
||
column back-compat). Configured in the "Social Media Site Promotion" dialog (`SocialsDialog.xaml` +
|
||
`ViewModels/SocialsDialogViewModel`, WPF-free, injected `ISocialValidator` + sign-in/sign-out fakes):
|
||
ON/OFF bar switch (schema v8), 6 fixed slots — row 1 always YouTube (signed-in → channel handle;
|
||
signed-out → sign-in gate → OAuth; delete → confirm sign-out), row 2 free, rows 3–6 lock icons on
|
||
freemium. Service detection (`SocialServiceIcons.DetectService`): URL domain / fediverse `@user@domain` →
|
||
**Fediverse** / bare→Website. Validation (`Services/SocialValidator.cs`, `ISocialValidator` seam +
|
||
`HttpSocialValidator` default): async GET of the canonical profile URL; 200/redirect = valid, 404/failure =
|
||
rejected. Fediverse additionally does a **best-effort nodeinfo lookup** (`/.well-known/nodeinfo` →
|
||
`software.name`, stored in `SocialEntry.FediverseSoftware` / the `SocialEntry.Software` column,
|
||
column-presence migration, no version bump) so the entry shows the **instance's real logo**
|
||
(`LogoDataForFediverse`: mastodon/peertube/pixelfed/misskey/lemmy/pleroma/firefish, generic fediverse
|
||
honeycomb fallback — Simple Icons CC0 path data, initials badges gone); nodeinfo failure still validates
|
||
(glyph falls back). If the identity domain's nodeinfo is blocked (YunoHost SSO gates
|
||
`/.well-known/nodeinfo` behind the login page) but the bare root 302s to the real instance, the lookup
|
||
**follows the root redirect** and asks the resolved host. Cancel is a hard stop: `LookupAsync` takes a
|
||
`CancellationToken` (dialog VM owns a CTS; Cancel/X/Save abort in-flight lookups, canceled continuations
|
||
never touch slot state).
|
||
|
||
**Bar bug-fix branch (2026-08-13) — three changes:**
|
||
|
||
1. **Positioning is a click-toggle (KISS)** — the original drag set a local `Canvas.SetTop` value that
|
||
permanently overrides the `{Binding SocialBarTop}` (a binding can never win over a local value), so the
|
||
bar stayed wherever it was dropped. The first drag-snap fix (`SocialBarSnap.Decide` + `ClearValue` on
|
||
release) still failed for shaky hands — jitter around the ±6px deadzone snapped the bar up but wouldn't
|
||
let it come back down. **Superseded by the user's click-toggle:** clicking the bar flips it top ⇄ bottom
|
||
(`MainViewModel.ToggleSocialBarPosition`), the bar rides the binding alone, and `SocialBarSnap` is gone.
|
||
2. **Fediverse software self-heal** — the DB row `@gramps@llamachile.tube` had `Software = NULL` because
|
||
nodeinfo was only ever asked of the identity domain (a landing page; the real instance is
|
||
`mastodon.llamachile.tube`). `HttpSocialValidator.ResolveFediverseSoftwareAsync` now **probes
|
||
well-known subdomains** (`mastodon.` → `social.` → … `FediverseSubdomainCandidates`) when the identity
|
||
domain and its redirect both come up empty, under a ~15s linked-CTS budget. `MainViewModel` runs the
|
||
static `HealFediverseSoftwareAsync` off the UI thread on layout load, applies matches via the dispatcher,
|
||
and saves; `SocialEntry.FediverseSoftware` is settable and raises `LogoData`, so the icon updates in place.
|
||
3. **The bar renders on the live output** — `Compositor/SocialBarRenderer.cs` rasterizes the entries into a
|
||
transparent straight-alpha BGRA strip (1920-wide, 40px content + 24px glow pad, the green glow baked in,
|
||
Pbgra32→straight-alpha unpremultiply) and the compositor blits it above the flash (see "Live frame
|
||
pipeline").
|
||
|
||
### Licensing — do not violate (GA = paid product; see `THIRD-PARTY-NOTICES.txt`)
|
||
|
||
This product is closed-source and paid. Every third-party component must stay inside the LGPL/BSD/MIT
|
||
guardrails below — written down so a future "quick fix" never reintroduces a GPL binary. **NEVER:**
|
||
|
||
- **Use a GPL FFmpeg build** — gyan.dev's builds are GPLv3 and ship libx264; BtbN's `gpl` variant is
|
||
GPL too. GPL in a distributed paid product is the #1 lawsuit risk. Only BtbN `lgpl` / `lgpl-shared`
|
||
builds are allowed.
|
||
- **Distribute the static lgpl build** — LGPLv2.1 §6 wants relinkable object files for static linking.
|
||
The **shared** (dynamic-DLL) build sidesteps that: compliance is "license text + source offer +
|
||
unmodified binaries". The pin is `lgpl-shared`; when the pin is refreshed, keep the shared variant.
|
||
- **Use BtbN's `nonfree` variant** — it adds fdk-aac (Fraunhofer code licensing). The native FFmpeg AAC
|
||
encoder is fine (no Fraunhofer code) but grants no AAC patent license — accepted low-risk posture for
|
||
RTMP→YouTube, since encoder vendors cover their implementations (Cisco OpenH264, NVIDIA NVENC, Intel
|
||
QSV, AMD AMF).
|
||
- **Link FFmpeg into the app** — it stays a separate subprocess fed frames over a pipe; that separation
|
||
keeps the app's own code out of LGPL reach.
|
||
- **Drop `THIRD-PARTY-NOTICES.txt`** from the shipped app or the About screen, or alter the FFmpeg
|
||
copyright/LGPL notices inside the downloaded binaries. Automating the download counts as distribution
|
||
— the obligations are not optional.
|
||
- **Pin to a moving target** — the `latest` BtbN release tag floats. Only immutable autobuild tags give
|
||
a reproducible source offer. Record the tag + variant beside the URL (TASKS.md) every time the pin moves.
|
||
- **Use non-CC0 icon art** — the social bar's bundled SVG logo path data comes from **Simple Icons**
|
||
(CC0 1.0, public domain — see `THIRD-PARTY-NOTICES.txt`). Replacing or adding logos must stay CC0 or
|
||
another public-domain source; a logo asset under a copyleft or attribution license would contaminate
|
||
the paid product.
|
||
- **Forget the v1 license-texts gate** — `THIRD-PARTY-NOTICES.txt` links the canonical license texts; at
|
||
**v1 (GA)** the full texts of every license it names MUST ship alongside it (TASK 4 requirement 9 is the
|
||
release blocker). Queued early is wrong; the release pass owns it.
|
||
|
||
## Design Principle
|
||
|
||
> This software is so intuitive that even the most right-brained person can easily intuit and use it.
|
||
|
||
Apply this to every UI decision:
|
||
- One-click go-live with working defaults
|
||
- Prefilled YouTube defaults (RTMP URL, bitrate, resolution, latency)
|
||
- Visual/drag-and-drop scene building over property panels
|
||
- Every action produces a visible outcome — no dead ends
|
||
|
||
## Monetization (design decision — the branding flash is the sword)
|
||
|
||
Free forever: all streams unlimited, no time caps, no subscription, no per-feature paywalls. The
|
||
**one paid line is a one-time unlock** (delivered via itch.io — they handle hosting, payment, and key
|
||
delivery; we never own a server or a key shop):
|
||
|
||
- **Free:** a periodic full-frame branding flash — "made with ytLlive!" rendered big and centered at
|
||
~25% opacity for about one second (soft 250ms fade in/out), repeated every 300s, on the live output
|
||
(and on v0.2 local recordings). Implemented as `BrandFlashLayer` in the preview compositor
|
||
(`MainWindow.xaml` CanvasGrid) + `BrandFlashTimer` in `MainViewModel` — cadence 300s, first flash
|
||
~5s after go-live, only while live or recording. An always-on watermark can be cropped or covered;
|
||
an intermittent full-frame flash can't be cropped and is impractical to edit around on a live feed.
|
||
- **Paid (one-time):** branding flash removed (flips `BrandFlashEnabled` off) + **Alerts**
|
||
(Super Chat / membership / subscribe pop-ins).
|
||
|
||
Deliberately rejected: always-on watermark (obscurable — replaced by the flash), hard stream-time
|
||
cutoffs (the worst dead end — a stream dying mid-broadcast reads as broken, and YouTube streams
|
||
routinely run 2-4 hours), soft-limit nagging, freemium tiers, and donation-only (relies on the
|
||
kindness of strangers). Resolution/quality ceilings are **deferred** — that decision belongs to the
|
||
resolution & streaming-constraints conversation, not monetization.
|
||
|
||
## Auth gates Go Live, but not exploration
|
||
|
||
The app is fully usable without authentication: users can build scenes, add sources, compose
|
||
previews, and audition the software with zero commitment. But **going live requires authentication** —
|
||
it's the one capability gated behind YouTube sign-in. The sign-in should never pressure the user
|
||
("sign in (optional)", not a modal wall): the two-state top bar shows **Start Stream** (offline) /
|
||
**End Stream** (live), and the Start Stream dialog hosts the account — a saved session appears as
|
||
the default with "Change Account"; with none saved, a "Sign in to YouTube" button starts OAuth and
|
||
the Start button stays disabled until signed in.
|
||
|
||
## Account assumption (do not build an account setup flow)
|
||
|
||
Connecting uses Google OAuth ("Sign in with Google") to link an **existing** YouTube creator
|
||
account. ytLlive **never creates or sets up accounts** — that is YouTube's job. If the creator has no
|
||
YouTube channel, they go to YouTube first. This assumption is explicit and must never be silently
|
||
replaced by an in-app account-creation step. Zero state = the Start Stream dialog's "Sign in to
|
||
YouTube" button; going live is unreachable until an account is connected.
|
||
|
||
## YouTube Live API — design constraints (do not violate)
|
||
|
||
These are the hard facts behind every decision. Full list in `TASKS.md`.
|
||
|
||
- **One-click go-live** — never call `transition(live)`. Insert the broadcast with
|
||
`enableAutoStart=true`, `enableAutoStop=true`, `enableMonitorStream=false`,
|
||
`selfDeclaredMadeForKids=false`, `latencyPreference=low`. The encoder starting brings YouTube live.
|
||
`enableMonitorStream=false` is what lets us skip the testing stage.
|
||
- **Variable reusable stream** — `liveStreams.insert` once per channel with
|
||
`cdn.resolution=variable`, `cdn.frameRate=variable`, `isReusable=true`; cache the ingestion URL +
|
||
stream name and reuse for every broadcast. Any quality tier works without recreating the stream,
|
||
and auto step-down is done by us dropping bitrate on the fly (zero API calls).
|
||
- **Quality is greyed out while live** — resolution/frameRate/ingestionType are immutable after
|
||
stream creation; editing title/description/privacy is fine at any time.
|
||
- **Report-by-exception health** — poll `liveStreams.list`; render nothing on `good`/`ok`, surface a
|
||
banner only on `configurationIssues[]` with `warning`/`error` severity. Bottom strip = YouTube logo
|
||
+ green/red connection dot (clickable → opens the dialog).
|
||
- **One dialog, three states** — not connected / connected-offline (all editable) / live
|
||
(title + description + visibility editable; quality + account greyed out). Both entry points
|
||
(Start Stream button + bottom strip) open it; prefilled from saved session profile.
|
||
- **End stream** — stop encoder → `transition(complete)`, `enableAutoStop` as the safety net.
|
||
- **Encoder compliance** — keyframes ≤ 4s (gopSizeLong), closed GOP, H.264, AAC/MP3 @ 44.1/48kHz,
|
||
mono/stereo only. YouTube flags violations via health status.
|
||
- **Broadcast ID == Video ID** — one ID tracks status, health, and the auto-created VOD
|
||
(`recordFromStart` + `enableDvr` default true).
|