359 lines
31 KiB
Markdown
359 lines
31 KiB
Markdown
# ytLlive — AI Guide
|
||
|
||
> Memory map entry point. Conventions live in [`schema.md`](schema.md); task
|
||
> status and YouTube API research in [`TASKS.md`](TASKS.md); directory maps in
|
||
> each folder's `index.md`. Reading order: this file → `TASKS.md` → `<dir>/index.md` → source.
|
||
|
||
## Response style
|
||
|
||
No default "Plans & Pitfalls" / planning boilerplate. Respond directly and
|
||
concisely: **do the queued work, then report what changed and what's next.**
|
||
Skip feature pitch, step-by-step implementation plans, pros/cons tables, and
|
||
"potential pitfalls" sections unless the user explicitly asks for a plan first.
|
||
A short diff-style summary beats a proposal document every time.
|
||
|
||
### No-Fluff Mode (on demand)
|
||
|
||
Invoke with "no-fluff mode" (or similar) when you want ruthless review instead
|
||
of reassurance. In that mode:
|
||
|
||
- Strip all polite pleasantries, emojis, transitions, and conversational padding.
|
||
- Treat the user's input as a draft to be methodically deconstructed or
|
||
strengthened — argue, correct, and sharpen rather than agree.
|
||
- Give unvarnished truth, not reassurance.
|
||
|
||
This is an occasional, explicitly-invoked mode — **never the default**. The
|
||
default response style above stays in effect unless invoked.
|
||
|
||
## Run
|
||
|
||
```bash
|
||
# From WSL, ALWAYS use the Windows dotnet host — never Linux `dotnet` for this project:
|
||
"/mnt/c/Program Files/dotnet/dotnet.exe" build "C:\Users\gramp\Documents\Code\projects\ytLive\ytLive.csproj"
|
||
"/mnt/c/Program Files/dotnet/dotnet.exe" run
|
||
```
|
||
|
||
`EnableWindowsTargeting=true` in `ytLive.csproj` lets a cold restore work from WSL, but a Linux
|
||
`dotnet run`/`build` re-downloads 100M+ of `windowsdesktop.app.*` packs into the Linux NuGet cache
|
||
(which lacks them) over the slow 9p `/mnt/c` bridge — twice, because the WPF `_wpftmp` generated
|
||
project triggers a second restore (203s observed). The Windows cache has the SQLite packages and the
|
||
packs resolve from `C:\Program Files\dotnet\packs`, so the Windows host never re-downloads.
|
||
Never use `--no-restore` right after an interrupted restore — the stale `project.assets.json`
|
||
produces misleading `NETSDK1064` "package not found" errors. Running requires Windows anyway.
|
||
|
||
## Tests
|
||
|
||
xUnit in `ytLive.Tests` (net8.0-windows10.0.19041.0, matches the app TFM). Run on Windows —
|
||
WSL can't run net8.0-windows tests:
|
||
|
||
```bash
|
||
dotnet.exe vstest "C:\...\ytLive.Tests\bin\Debug\net8.0-windows10.0.19041.0\ytLive.Tests.dll"
|
||
```
|
||
|
||
Good Dog Rule: ONE integration test per branch, ONE test per PR. Current: TokenStore DPAPI
|
||
roundtrip/corrupt/missing/clear, OAuth exchange/refresh/ClearSession, CameraManager refcount +
|
||
frame pump + failure handling (fakes for the WinRT seams), real-`MainWindow` round-clip
|
||
interaction test, LayoutStore delete roundtrip, LayoutStore pre-round-rect-dims roundtrip,
|
||
LayoutStore backdrop roundtrip, LayoutStore HasBackdrop roundtrip + v5→v6 non-Live backfill,
|
||
ScreenCaptureManager refcount + shared-bitmap + coalescing
|
||
(fake `IScreenCaptureSource` + a real background-STA `Dispatcher`), BackdropTests (EnsureBackdrop
|
||
insert/idempotent/heal + HasBackdrop gate, IsLiveCapture, DisplaySource, INPC), SceneCatalogTests
|
||
(the five canonical scenes, Live-only backdrop policy, EnforceBackdropPolicy), WebcamSafeguardTests
|
||
(the per-scene size clamp incl. the Chat half-screen-area cap) —
|
||
65 passing.
|
||
|
||
### Real-MainWindow tests MUST be hermetic (DB pollution bug)
|
||
|
||
The integration test boots a real `MainWindow` → `MainViewModel` → real `LayoutStore`
|
||
(`%APPDATA%\ytLlive\ytLlive.db`). `Shutdown()` on close **saves the layout** (full rewrite:
|
||
DELETE all scenes/sources, re-insert), so any source a test adds would be persisted over the
|
||
user's real ones — this happened and wiped the real webcam source (DeviceId replaced by the
|
||
test's fake `test-camera`). Rule: a test that constructs `MainWindow` MUST first set
|
||
`MainViewModel.LayoutPathOverride` to a temp DB path and reset it (plus
|
||
`SqliteConnection.ClearAllPools()` + delete) in `finally`. The seam is
|
||
`internal static string? LayoutPathOverride` (line ~529 in `MainViewModel.cs`),
|
||
`ytLive.csproj` has `InternalsVisibleTo("ytLive.Tests")`.
|
||
|
||
The layout DB is a **full rewrite per save** (delete all, re-insert from memory), so
|
||
save/load round trips are exact: an element removed in the UI (`RemoveElement` →
|
||
`scene.Elements.Remove` → `OnElementsChanged` → debounced `ScheduleSave`, plus `Shutdown` on
|
||
close) does **not** come back after reload (`LayoutStorePersistenceTests` guards this).
|
||
|
||
## Architecture
|
||
|
||
C# / WPF (.NET 8) following MVVM:
|
||
|
||
| Path | Role |
|
||
|------|------|
|
||
| `Models/` | Plain data types — Scene, Source (incl. `ClipShape`, `IsMirrored`, `VideoImageSource`), QualityOption, StreamConfig, StreamHealth, YouTubeChannel, ChatMessage |
|
||
| `ViewModels/` | MainViewModel — exposes collections + commands for the UI; GoLiveViewModel, ReuseImageViewModel, CameraPickerViewModel |
|
||
| `Services/` | YouTube OAuth2, stream/broadcast management, live chat polling, LayoutStore (SQLite), **webcam: `VideoFrame` seam + `CameraDeviceInfo`/`ICameraEnumerator`/`ICameraFrameSource` interfaces + `MediaCaptureCameraEnumerator`/`MediaCaptureFrameSource` (WinRT) + `CameraManager`**, **screen capture: `IFullScreenDetector`/`Win32FullScreenDetector` + `IScreenCaptureSource`/`ScreenCaptureFrameSource` (WinRT GraphicsCapture) + `ScreenCaptureManager` + `ScreenCaptureSourceFactory` + `Direct3D11Helper`/`CaptureInterop` (COM bridges)** |
|
||
| `Helpers/` | ViewModelBase (INotifyPropertyChanged), RelayCommand, ImageCache, AppLog (file logger), FocusPreservingListBox, OAuthCredentials, **TokenStore (DPAPI session persistence)**, visibility converters |
|
||
| `Themes/` | `Controls.xaml` — the single dark-theme source, merged once in `App.xaml` (see `Themes/index.md`) |
|
||
| `MainWindow.xaml` | Dark theme; layout: top bar (controls), center (preview), left (scenes/sources), right (chat), bottom (health) |
|
||
|
||
### Key patterns
|
||
|
||
- `ViewModelBase.SetProperty<T>()` for property change notifications
|
||
- `RelayCommand` for all button actions; commands gate on state (e.g. Start only when Offline). **Typed `CommandParameter`s — no stringly-typed command tokens:** menu items that pick a source type pass the enum value itself (`CommandParameter="{x:Static models:SourceType.DisplayCapture}"`), so a typo breaks the build instead of silently adding an Image; `AddSource` still falls back to `Enum.TryParse<SourceType>(..., true)` for safety. The webcam item is its own `AddWebcamCommand` (it greys out via `CanAddWebcamToActiveScene` and isn't a `SourceType` — webcams are `WebcamSceneConfig`, not `Source` rows)
|
||
- **Audio is KISS by rule** — the whole of audio is *one knob*: **desktop/game audio is automatic** (WASAPI loopback from the default output at unity, zero UI — "it just is"); the **mic is the creator's only audio control**, a single bottom-bar MIC chip: sound meter (blank → green → yellow → red with zone markers at 60%/80%, `MeterFillWidth`/`MeterBrush` from `MainViewModel.AudioLevel`) + volume slider (`MicVolume`) + mute (`MicMuteText`). No device pickers (never show device names — no "install a device you didn't know existed"), no filter stacks, no monitoring, no routing — OBS's confusion (dynamic mixer, unintuitive names, four required filters) is deliberately absent. A production-ready mic chain (high-pass → noise gate → compressor) will be applied invisibly in the mixer, unconfigurable. Capture runs only while live (privacy indicator stays off otherwise). Capture pipeline = `IAudioSource` seam + NAudio `WasapiCapture`/`WasapiLoopbackCapture` + `AudioMixer` (pending — the UI is in place now). The connected account's avatar/name shows in the top bar next to Start Stream (`AccountAvatarUrl`/`AccountDisplayName` via `SyncConnectedAccount`), so the creator always sees WHICH account will go live.
|
||
- ViewModels are constructed in XAML (`<vm:MainViewModel/>` as DataContext)
|
||
- Services are currently instantiated in MainViewModel's constructor — no DI container yet
|
||
- Layout persists to SQLite (`Microsoft.Data.Sqlite`); scenes/sources/asset bytes stored in the DB, asset identity is a SHA-256 content hash (1:M reuse, no file paths — assets are always available)
|
||
- **Five-scene catalog (`Models/SceneCatalog.cs`):** the product is exactly Starting/Live/BRB/Chat/Ending — work with less, never more (the escape hatch for "more" is OBS). Scenes are matched **by name** (`SceneCatalog.Is`, case-insensitive trim). Empty DBs seed all five; the scenes-header "+" (`ShowAddScene`/`MissingScenes` on `MainViewModel`) only appears while ≥1 canonical scene is missing and its menu lists only the missing ones, re-adding them by name (`AddSceneCommand`). Renaming a canonical scene makes it missing again; `AddScene` rejects non-canonical names.
|
||
- Theming: all custom styles live in `Themes/Controls.xaml`, merged in `App.xaml` — never duplicate styles per-window (dialog duplicates were consolidated into this dictionary)
|
||
- Resolution tiers (bottom bar): 1080p60@8 (default) → 1080p30@8 → 720p60@6 → 720p30@6 → **Vertical 1080p60@8 (9:16, 1080×1920)**. The composition master frame is **always 1920×1080** — a tier is an output rect + target resolution over that master, so source geometry is never rewritten (no rounding drift). 16:9 tiers use the full frame; the vertical tier uses a centered **607×1080** window and the preview dims the cropped side strips at 55% black with an accent outline (semi-crop — the cut area stays visible). A resolution badge in the preview corner shows the active tier; the bottom bar shows bitrate/FPS. A **tooltip** explains finding upload bandwidth — an in-app speed test was deliberately dropped (unreliable). The future encoder crops the master to the rect and scales to the tier's Width×Height
|
||
- Crash diagnosis: `AppLog` writes startup checkpoints to `%APPDATA%\ytLlive\startup.log`; `App.xaml.cs` logs `DispatcherUnhandledException`/`AppDomain.UnhandledException`. When WPF won't run from WSL, this log is how you find the failure (it caught the `MenuItemRole.Separator` XAML crash and the ComboBox SelectionBoxItem bug)
|
||
|
||
### Current limitations / TODOs
|
||
|
||
- `Helpers/OAuthCredentials.cs` contains the real ClientId/ClientSecret. Auth is complete and the session **persists via Windows DPAPI** (`Helpers/TokenStore.cs` → `%APPDATA%\ytLlive\ytLlive.auth`, CurrentUser scope), reloaded best-effort at startup with a proactive refresh of a near-expiry access token. Sign-in/Change Account lives **inside the Start Stream dialog** (two-state flow — no separate Connect button). A **graceful End Livestream signs out**: `StopStream()` clears the session + token, so the next go-live needs a fresh sign-in; a crash never runs End, so the token survives and the creator stays signed in. `YouTubeAuthService` takes an optional `HttpClient` + `sessionChanged` callback (test seam + save hook; services are still constructed in `MainViewModel`)
|
||
- Scene/source/asset layout persists (SQLite, schema v6); the OAuth session persists (DPAPI); the paid-unlock state does not (yet — itch.io key verification pending)
|
||
- `YouTubeStreamService` uses hardcoded `1080p`/`60fps` and per-broadcast streams — must switch to the v3 `variable` reusable stream
|
||
- Webcam capture is shipped (milestone 1); the live desktop/game backdrop is shipped (ship task #1); **window capture (non-backdrop), scene compositing/encoding, RTMP are next**
|
||
- `StreamConfig` defaults (`TargetBitrate=6000`, `Resolution="1920x1080"`) are stale — the live dropdown drives `StreamHealth.CurrentBitrate`/`FPS` instead
|
||
|
||
### Screen backdrop capture (TASK 3 ship task #1)
|
||
|
||
The backdrop is the **live desktop/game capture as a permanent, non-deletable bottom layer**
|
||
rendered in every scene — the "Screen" source from the minimal set, done as content-swap
|
||
instead of a normal draggable source.
|
||
|
||
- **Model (schema v6):** `Source.IsBackdrop` (persisted) marks the one backdrop per scene; `Source.CaptureKey`
|
||
(persisted) names the target — `monitor:<n>`, `window:<hwnd>`, or `picker:<displayname>`. The backdrop is
|
||
a real `Source` of `Type DisplayCapture`, inserted **first** (`MainViewModel.EnsureBackdrop(scene)`,
|
||
internal static — runs on layout load + every `AddScene`, healing any scene missing one), fixed at
|
||
X=0/Y=0/1920×1080, and excluded from drag/hit-test/remove/reorder (remove is guarded in `RemoveElement`;
|
||
`IsDraggableElement` never matches live types; the element template sets `IsHitTestVisible=false` for
|
||
backdrops; the row's remove button and "Remove Source" menu item are hidden). **`Scene.HasBackdrop`
|
||
(persisted, default off) is the Live-only policy flag** — the backdrop belongs to the canonical Live
|
||
scene alone (see `SceneCatalog`). `EnsureBackdrop` returns null for a flag-less scene, so
|
||
Starting/BRB/Chat/Ending compose their own layers. The one-time v5→v6 backfill turns those four scenes
|
||
off and drops their backdrop sources, and **`EnforceBackdropPolicy` (internal static, runs after every
|
||
load)** re-normalizes the flag by scene name and strips any backdrop that lingers in a non-Live scene —
|
||
the flag is owned by policy, never the user. There is no scene-list "Backdrop" checkbox anymore
|
||
(the old `ToggleSceneBackdropCommand` is gone); "Change Capture…"/"Refresh Capture"/"Capture Display"
|
||
only show in the Live scene's preview menu (`CanChangeBackdrop`). The static Background, if any,
|
||
renders **above** the backdrop.
|
||
- **Detection (launch + focus only, no live session listener):** `Win32FullScreenDetector` =
|
||
`GetForegroundWindow` + `DwmGetWindowAttribute(DWMWA_EXTENDED_FRAME_BOUNDS)` + `MonitorFromWindow` +
|
||
`GetMonitorInfo`; a window is full-screen when its frame covers all four monitor edges; own process is
|
||
excluded; monitor **index = `EnumDisplayMonitors` enumeration order** (the same order
|
||
`ScreenCaptureSourceFactory` uses to map index→HMONITOR via `Win32FullScreenDetector.GetMonitorHandle`).
|
||
`IFullScreenDetector` also exposes `GetDisplays()` (`DisplayInfo`: index/name/resolution/bounds/`IsPrimary`,
|
||
friendly name via `EnumDisplayDevices`) + `PrimaryMonitorIndex()` for the in-app "Capture Display"
|
||
submenu. At launch `ReacquireScreenCaptures` keys the backdrop to the full-screen game's monitor (or the
|
||
**primary** display — never assumed monitor 0).
|
||
On **Deactivated**, `NoteBackgroundWindow` samples the foreground ~250ms later (so an alt-tab to a game
|
||
lands before the app re-activates); on the next **Activated**, `RefreshBackdropAutoCapture` re-runs
|
||
detection against that sample and re-designates. Null detection (our app, the desktop, a normal window)
|
||
leaves the current capture alone.
|
||
- **Capture (WinRT GraphicsCapture):** `ScreenCaptureFrameSource` creates a free-threaded
|
||
`Direct3D11CaptureFramePool` (2 buffers, `B8G8R8A8UIntNormalized`) + `GraphicsCaptureSession`; frames →
|
||
`SoftwareBitmap.CreateCopyFromSurfaceAsync` (alpha ignored) → `VideoFrame` (BGRA8), bytes read via
|
||
`WindowsRuntimeMarshal.TryGetDataUnsafe` (the same CsWinRT-safe read the webcam path uses) — the
|
||
`IMemoryBufferByteAccess` ComImport cast threw `Invalid cast` on **every frame** under CsWinRT, which
|
||
flooded `startup.log` (~5 MB in a session) and burned CPU, so it is gone. Surfaces larger than the
|
||
1920×1080 master are downscaled bilinearly to the master (`DownscaleBgra`) before the copy, and
|
||
per-frame conversion failures are logged at most once per 5 s (`ErrorLogThrottle`). DRM-protected
|
||
content delivers black frames (OS limitation, documented). Frame pool
|
||
pauses while the app is minimized — capture keeps running, the pool just stops delivering.
|
||
- **Ownership:** `ScreenCaptureManager` mirrors `CameraManager` — refcounted by target key, one shared
|
||
`WriteableBitmap` per key, dispatcher-coalesced latest-frame copies, `PreviewBitmapChanged`/`CaptureFailed`
|
||
events, `ReleaseAllAsync` on re-designation. `ScreenCaptureSourceFactory.Resolve(key)` parses the key into
|
||
a source; `PickAsync()` shows the OS `GraphicsCapturePicker` ("Change Capture…", owner window set via the
|
||
`IInitializeWithWindow` ComImport) and returns a **transient** `picker:` key — a reload falls back to
|
||
auto-detection.
|
||
- **CsWinRT projection gaps hand-rolled:** `Windows.Graphics.Direct3D11.Direct3D11Helper` is not projected,
|
||
so `Direct3D11Helper` P/Invokes `d3d11.dll!D3D11CreateDevice` (hardware, BGRA_SUPPORT, explicit 11.1-first
|
||
feature array) → QI `IDXGIDevice` → the WinRT interop export
|
||
`CreateDirect3D11DeviceFromDXGIDevice` → `MarshalInterface<IDirect3DDevice>.FromAbi` (one shared device per
|
||
process). Do **not** switch back to the QI-for-`IDirect3DDxgiInterfaceAccess` trick: the raw D3D11 device
|
||
no longer exposes that interface on newer Windows (verified E_NOINTERFACE on build 26200, hardware and
|
||
WARP alike) while `CreateDirect3D11DeviceFromDXGIDevice` keeps working.
|
||
`IInitializeWithWindow` are ComImports in `CaptureInterop.cs`. All WinRT projections were verified by
|
||
reflection against the built `Microsoft.Windows.SDK.NET.dll` before writing the interop.
|
||
- **Known v1 limits:** full-desktop captures are CPU-copied at native resolution (GPU downscale = encoder
|
||
task); window capture (`window:<hwnd>`) and multi-monitor live re-targeting beyond the auto-detected
|
||
game are behind the picker; picker-based captures don't survive reload.
|
||
- **GPU posture:** same as webcam — CPU frames, WPF hardware-presents; D3DImage GPU compositing deferred
|
||
to the encoder task.
|
||
- **Preview watermark:** the "Preview" placeholder hides while a backdrop renders —
|
||
`ShowPreviewPlaceholder` now also checks `BackdropImage` (raised on backdrop change), so a scene with
|
||
live capture shows the feed instead of the "nothing here" label.
|
||
|
||
### Webcam capture (TASK 3 milestone 1)
|
||
|
||
- **Seam-first:** everything above the WinRT layer speaks only `VideoFrame` (normalized tightly-packed
|
||
BGRA8) + `CameraDeviceInfo`/`ICameraEnumerator`/`ICameraFrameSource` interfaces. Tests inject fakes;
|
||
screen capture and background removal later feed the same seam.
|
||
- **CPU-first:** `MediaCaptureInitializationSettings { MemoryPreference = Cpu, StreamingCaptureMode = Video,
|
||
SharingMode = SharedReadOnly }`, frames pulled via
|
||
`CreateFrameReaderAsync(colorSource, MediaEncodingSubtypes.Bgra8)` — the pipeline does any format
|
||
conversion, so every `FrameArrived` yields a ready BGRA8 `SoftwareBitmap` (bytes read via
|
||
`WindowsRuntimeMarshal.TryGetDataUnsafe`, not marshalled copies).
|
||
- **Source pick, not first hit:** the frame reader is bound to the first source that is `VideoPreview`
|
||
(preferred) or `VideoRecord`, not blindly the first preview source. If a camera exposes neither, the
|
||
failure names the device and the stream types it *does* expose. `SharedReadOnly` lets the capture
|
||
coexist with other apps that share the camera.
|
||
- **Known failure: NVIDIA Broadcast** — it opens the physical webcam exclusively, so `InitializeAsync`
|
||
fails with "camera in use" (or, if init slips through, the device exposes no preview source). Fix:
|
||
quit Broadcast while streaming, or pick its virtual "NVIDIA Broadcast" device from the picker and the
|
||
app captures the processed feed. This is a real-device finding (Logitech `VID_046D&PID_082D`).
|
||
- **TFM:** `net8.0-windows10.0.19041.0` (app + tests) pulls the WinRT projection from the SDK reference
|
||
packs — no NuGet package, no capability manifest (unpackaged desktop app works; the Windows privacy
|
||
camera toggle still applies). `EnableWindowsTargeting` keeps WSL builds working.
|
||
- **One webcam, many scenes (schema v3):** a singleton `Webcam` row holds the identity
|
||
(`Id`/`DeviceId`/`Name`); each scene gets its own `WebcamSceneConfig` (position/size/clip/mirror/
|
||
border/`IsVisible`). `Scene.Elements` holds images (`Source`) and, at most once, the webcam
|
||
(`WebcamSceneConfig`); `Scene.WebcamConfig` is the accessor. `CameraManager` refcounts capture
|
||
sessions by `DeviceId` (a session starts at `RefCount = 1`; repeat acquire bumps it; the last
|
||
release stops + disposes). **"Add Webcam" ALWAYS opens the Windows camera picker** — the creator is
|
||
never silently handed the previous camera (which used to happen after deleting one scene's webcam
|
||
while another scene still used it; that identity survived, so re-adding bypassed the choice).
|
||
Picking a different camera than the current app-wide one swaps it everywhere via
|
||
`SwapWebcamIdentityAsync` (the same path "Change Webcam…" uses), keeping the singleton honest;
|
||
picking the same one just places the config. The Add Webcam menu greys out when the **active**
|
||
scene already has a config (`CanAddWebcamToActiveScene`); showing a hidden webcam reuses the
|
||
existing config (`CanShowWebcamInActiveScene` / empty-canvas right-click "Show Webcam" — which, on
|
||
a config-less canvas, delegates to Add Webcam and so also picks). **Removing the last webcam
|
||
config anywhere clears the identity** (`_webcam = null`), which also drops "Change Webcam…".
|
||
- **Round→rect restores the aspect (persisted, schema v4):** `SceneElement.ToggleClipShape()`
|
||
snapshots the rectangular Width/Height into public `RectWidth`/`RectHeight` before going Round and
|
||
restores them when switching back — otherwise the Round resize lock (square) would leave a square
|
||
behind. The rect dims are **persisted** (`WebcamSceneConfig.RectWidth`/`RectHeight`, nullable), so a
|
||
reloaded Round webcam still restores its pre-Round aspect instead of staying square. A one-time
|
||
`HealLegacySquareRect` (load only) widens a pre-v4 `Traditional` config that ended up square to 16:9
|
||
(keeps height; Round and explicit rect dims are untouched).
|
||
- **Device swap / layout reload:** `ChangeWebcamAsync` (picker) and `ReacquireWebcam` (after load)
|
||
release the old device with `ReleaseAllAsync` — a forced full drop that zeroes the refcount and
|
||
stops the source regardless of how many scenes held it (the per-config count isn't known once the
|
||
scenes are replaced) — then `AcquireAsync` the new device once per config.
|
||
- **Shared bitmap, coalesced updates:** one `WriteableBitmap` per active camera, created on the UI thread
|
||
at the device's frame size (first frame), forwarded to every `WebcamSceneConfig.VideoImageSource` via
|
||
`PreviewBitmapChanged`. Frames arrive on a worker thread; `CameraManager` coalesces onto the dispatcher
|
||
(at most one pending copy per session, at `Render` priority, always copying the latest frame) so a 60fps
|
||
device never drowns the render thread.
|
||
- **Webcam added mid-session must get the live frames:** `PreviewBitmapChanged` fires **once** (the first
|
||
frame creates the shared bitmap); later frames only mutate that bitmap in place, so a config that didn't
|
||
exist at first-frame time would never receive it — the empty/transparent container you'd see adding a
|
||
webcam to Chat while Live already had the camera. `CameraManager.GetPreviewBitmap(deviceId)` exposes the
|
||
current shared bitmap; `AddWebcamToActiveSceneAsync` assigns it to the new config right before
|
||
`AcquireAsync`, and `ReacquireWebcam` re-propagates it to every config after a reload.
|
||
- **Clip/mirror/border:** per-element `ClipShape` (Traditional rectangle / Round ellipse) + `IsMirrored`
|
||
(`ScaleX = -1`) + the OSB-standard static border (`BorderColor` `#RRGGBB` or `""`=none, `BorderOpacity`
|
||
0–1, `BorderWidth` 0–20, `BorderAnimation` `None|Pulse|Chase|Rainbow|Shimmer|MarchingAnts|Glow|
|
||
Electricity|Sparkles`). Rendered in the preview DataTemplate; toggled from the element's right-click
|
||
context menu (webcam menu: Change Webcam…, Border Effect submenu — all 9 items enabled, values persist,
|
||
rendering stays static until the animation tier ships — Border Color, Opacity/Thickness sliders, Hide in
|
||
this scene, Remove); persisted in the layout DB. The Add menu shows when no webcam exists; the empty
|
||
preview canvas has its own Show Webcam entry.
|
||
- The Round webcam is an `Image Stretch="UniformToFill"` with an `EllipseGeometry` clip
|
||
(`Center=0.5,0.5` `RadiusX/Y=0.5`), inside a `Viewbox Stretch="Uniform"` holding a `1x1` Grid, so it
|
||
renders as a true circle (diameter = the shorter element dimension) instead of an oval stretched to
|
||
the element rect — the traditional `Image` keeps `UniformToFill` over the full rect. The clip is
|
||
geometry, **not an `ImageBrush`**: a brush re-rasterizes the frequently-updated `WriteableBitmap` per
|
||
frame on the render thread, which is what made the live webcam crawl while round. The Round border is
|
||
a centered `Ellipse` at `Width/Height = RoundBorderSize`.
|
||
- Resizing locks to a square (`_resizeAspect = 1`) while `ClipShape == Round`.
|
||
- **Webcam size clamp:** `ClampWebcamToBounds(config, sceneName)` (internal — test seam) enforces the
|
||
max per dimension at resize + load and no less than 10% of the master (192×108). The cap is picked
|
||
**by canonical scene name**: 50% per dimension (960×540) everywhere except the **Chat scene**, which
|
||
may reach half the screen **area** (~1358×764 @16:9) so the viewer sees the creator better
|
||
(`MaxWebcamWidthFor`/`MaxWebcamHeightFor`; a renamed Chat loses the bigger cap).
|
||
`RoundBorderSize` follows the clamped height. `WebcamSafeguardTests` guards the clamp.
|
||
- **Hit-testing:** a `Grid` without `Background` only hit-tests where its children draw, so clicks in
|
||
the empty corners of a round clip fell through to `Window_PreviewMouseLeftButtonDown` and deselected
|
||
the element — making the corner handle ungrabbable. The element Grid carries
|
||
`Background="Transparent"` (whole rect draggable; the empty canvas Grid uses the same trick for
|
||
right-click Show Webcam) and the `SelectionOverlay` (dashed border + corner dot) is
|
||
`IsHitTestVisible="False"` so it never intercepts the click. Two things make the webcam menu work:
|
||
(1) the `ContextMenu` pins its own `DataContext` to `PlacementTarget.DataContext` — a `ContextMenu`
|
||
isn't in the visual tree, so without it the Click-handler `DataContext:` patterns (and the
|
||
IsChecked/slider bindings) silently fail; (2) `Themes/Controls.xaml` ships a full dark `MenuItem`
|
||
template — `PART_Popup` (submenu popups), a popup `ItemsPresenter` (the Border Opacity/Thickness
|
||
sliders live in Items, so they render in a hover flyout), a `✓` checkmark column, and a `›` arrow
|
||
driven by `HasItems`. An earlier bare `Border + Header` template dropped all three: submenus never
|
||
opened, sliders never rendered, checkmarks never showed — the menu looked dead even though the
|
||
Click handlers were fine.
|
||
- **GPU posture:** webcam frames are CPU (GPU-agnostic; WPF hardware-presents the preview anyway). Hardware
|
||
encoders (NVENC/AMF/QSV) matter for the encoder task, not capture. D3DImage GPU compositing is deferred
|
||
to the encoder task.
|
||
- **Background removal = milestone 2** — ONNX Runtime + DirectML (CPU fallback), MediaPipe Selfie
|
||
Segmentation (Apache-2.0), wired into the same `VideoFrame` seam. Not part of milestone 1.
|
||
|
||
## Design Principle
|
||
|
||
> This software is so intuitive that even the most right-brained person can easily intuit and use it.
|
||
|
||
Apply this to every UI decision:
|
||
- One-click go-live with working defaults
|
||
- Prefilled YouTube defaults (RTMP URL, bitrate, resolution, latency)
|
||
- Visual/drag-and-drop scene building over property panels
|
||
- Every action produces a visible outcome — no dead ends
|
||
|
||
## Monetization (design decision — the branding flash is the sword)
|
||
|
||
Free forever: all streams unlimited, no time caps, no subscription, no per-feature paywalls. The
|
||
**one paid line is a one-time unlock** (delivered via itch.io — they handle hosting, payment, and key
|
||
delivery; we never own a server or a key shop):
|
||
|
||
- **Free:** a periodic full-frame branding flash — "made with ytLlive!" rendered big and centered at
|
||
~25% opacity for about one second (soft 250ms fade in/out), repeated every 300s, on the live output
|
||
(and on v0.2 local recordings). Implemented as `BrandFlashLayer` in the preview compositor
|
||
(`MainWindow.xaml` CanvasGrid) + `BrandFlashTimer` in `MainViewModel` — cadence 300s, first flash
|
||
~5s after go-live, only while live or recording. An always-on watermark can be cropped or covered;
|
||
an intermittent full-frame flash can't be cropped and is impractical to edit around on a live feed.
|
||
- **Paid (one-time):** branding flash removed (flips `BrandFlashEnabled` off) + **Alerts**
|
||
(Super Chat / membership / subscribe pop-ins).
|
||
|
||
Deliberately rejected: always-on watermark (obscurable — replaced by the flash), hard stream-time
|
||
cutoffs (the worst dead end — a stream dying mid-broadcast reads as broken, and YouTube streams
|
||
routinely run 2-4 hours), soft-limit nagging, freemium tiers, and donation-only (relies on the
|
||
kindness of strangers). Resolution/quality ceilings are **deferred** — that decision belongs to the
|
||
resolution & streaming-constraints conversation, not monetization.
|
||
|
||
## Auth gates Go Live, but not exploration
|
||
|
||
The app is fully usable without authentication: users can build scenes, add sources, compose
|
||
previews, and audition the software with zero commitment. But **going live requires authentication** —
|
||
it's the one capability gated behind YouTube sign-in. The sign-in should never pressure the user
|
||
("sign in (optional)", not a modal wall): the two-state top bar shows **Start Stream** (offline) /
|
||
**End Stream** (live), and the Start Stream dialog hosts the account — a saved session appears as
|
||
the default with "Change Account"; with none saved, a "Sign in to YouTube" button starts OAuth and
|
||
the Start button stays disabled until signed in.
|
||
|
||
## Account assumption (do not build an account setup flow)
|
||
|
||
Connecting uses Google OAuth ("Sign in with Google") to link an **existing** YouTube creator
|
||
account. ytLlive **never creates or sets up accounts** — that is YouTube's job. If the creator has no
|
||
YouTube channel, they go to YouTube first. This assumption is explicit and must never be silently
|
||
replaced by an in-app account-creation step. Zero state = the Start Stream dialog's "Sign in to
|
||
YouTube" button; going live is unreachable until an account is connected.
|
||
|
||
## YouTube Live API — design constraints (do not violate)
|
||
|
||
These are the hard facts behind every decision. Full list in `TASKS.md`.
|
||
|
||
- **One-click go-live** — never call `transition(live)`. Insert the broadcast with
|
||
`enableAutoStart=true`, `enableAutoStop=true`, `enableMonitorStream=false`,
|
||
`selfDeclaredMadeForKids=false`, `latencyPreference=low`. The encoder starting brings YouTube live.
|
||
`enableMonitorStream=false` is what lets us skip the testing stage.
|
||
- **Variable reusable stream** — `liveStreams.insert` once per channel with
|
||
`cdn.resolution=variable`, `cdn.frameRate=variable`, `isReusable=true`; cache the ingestion URL +
|
||
stream name and reuse for every broadcast. Any quality tier works without recreating the stream,
|
||
and auto step-down is done by us dropping bitrate on the fly (zero API calls).
|
||
- **Quality is greyed out while live** — resolution/frameRate/ingestionType are immutable after
|
||
stream creation; editing title/description/privacy is fine at any time.
|
||
- **Report-by-exception health** — poll `liveStreams.list`; render nothing on `good`/`ok`, surface a
|
||
banner only on `configurationIssues[]` with `warning`/`error` severity. Bottom strip = YouTube logo
|
||
+ green/red connection dot (clickable → opens the dialog).
|
||
- **One dialog, three states** — not connected / connected-offline (all editable) / live
|
||
(title + description + visibility editable; quality + account greyed out). Both entry points
|
||
(Start Stream button + bottom strip) open it; prefilled from saved session profile.
|
||
- **End stream** — stop encoder → `transition(complete)`, `enableAutoStop` as the safety net.
|
||
- **Encoder compliance** — keyframes ≤ 4s (gopSizeLong), closed GOP, H.264, AAC/MP3 @ 44.1/48kHz,
|
||
mono/stereo only. YouTube flags violations via health status.
|
||
- **Broadcast ID == Video ID** — one ID tracks status, health, and the auto-created VOD
|
||
(`recordFromStart` + `enableDvr` default true).
|