ytLlive/ai.md

# ytLlive — AI Guide

> Memory map entry point. Conventions live in [`schema.md`](schema.md); task
> status and YouTube API research in [`TASKS.md`](TASKS.md); directory maps in
> each folder's `index.md`. Reading order: this file → `TASKS.md` → `<dir>/index.md` → source.

## Response style

No default "Plans & Pitfalls" / planning boilerplate. Respond directly and
concisely: **do the queued work, then report what changed and what's next.**
Skip feature pitch, step-by-step implementation plans, pros/cons tables, and
"potential pitfalls" sections unless the user explicitly asks for a plan first.
A short diff-style summary beats a proposal document every time.

### No-Fluff Mode (on demand)

Invoke with "no-fluff mode" (or similar) when you want ruthless review instead
of reassurance. In that mode:

- Strip all polite pleasantries, emojis, transitions, and conversational padding.
- Treat the user's input as a draft to be methodically deconstructed or
  strengthened — argue, correct, and sharpen rather than agree.
- Give unvarnished truth, not reassurance.

This is an occasional, explicitly-invoked mode — **never the default**. The
default response style above stays in effect unless invoked.

## Run

```bash
# From WSL, ALWAYS use the Windows dotnet host — never Linux `dotnet` for this project:
"/mnt/c/Program Files/dotnet/dotnet.exe" build "C:\Users\gramp\Documents\Code\projects\ytLive\ytLive.csproj"
"/mnt/c/Program Files/dotnet/dotnet.exe" run
```

`EnableWindowsTargeting=true` in `ytLive.csproj` lets a cold restore work from WSL, but a Linux
`dotnet run`/`build` re-downloads 100M+ of `windowsdesktop.app.*` packs into the Linux NuGet cache
(which lacks them) over the slow 9p `/mnt/c` bridge — twice, because the WPF `_wpftmp` generated
project triggers a second restore (203s observed). The Windows cache has the SQLite packages and the
packs resolve from `C:\Program Files\dotnet\packs`, so the Windows host never re-downloads.
Never use `--no-restore` right after an interrupted restore — the stale `project.assets.json`
produces misleading `NETSDK1064` "package not found" errors. Running requires Windows anyway.

## Tests

xUnit in `ytLive.Tests` (net8.0-windows10.0.19041.0, matches the app TFM). Run on Windows —
WSL can't run net8.0-windows tests:

```bash
dotnet.exe vstest "C:\...\ytLive.Tests\bin\Debug\net8.0-windows10.0.19041.0\ytLive.Tests.dll"
```

Good Dog Rule: ONE integration test per branch, ONE test per PR. Current: TokenStore DPAPI
roundtrip/corrupt/missing/clear, OAuth exchange/refresh/ClearSession, CameraManager refcount +
frame pump + failure handling (fakes for the WinRT seams), real-`MainWindow` round-clip
interaction test, LayoutStore delete roundtrip, LayoutStore pre-round-rect-dims roundtrip —
25 passing.

### Real-MainWindow tests MUST be hermetic (DB pollution bug)

The integration test boots a real `MainWindow` → `MainViewModel` → real `LayoutStore`
(`%APPDATA%\ytLlive\ytLlive.db`). `Shutdown()` on close **saves the layout** (full rewrite:
DELETE all scenes/sources, re-insert), so any source a test adds would be persisted over the
user's real ones — this happened and wiped the real webcam source (DeviceId replaced by the
test's fake `test-camera`). Rule: a test that constructs `MainWindow` MUST first set
`MainViewModel.LayoutPathOverride` to a temp DB path and reset it (plus
`SqliteConnection.ClearAllPools()` + delete) in `finally`. The seam is
`internal static string? LayoutPathOverride` (line ~529 in `MainViewModel.cs`),
`ytLive.csproj` has `InternalsVisibleTo("ytLive.Tests")`.

The layout DB is a **full rewrite per save** (delete all, re-insert from memory), so
save/load round trips are exact: an element removed in the UI (`RemoveElement` →
`scene.Elements.Remove` → `OnElementsChanged` → debounced `ScheduleSave`, plus `Shutdown` on
close) does **not** come back after reload (`LayoutStorePersistenceTests` guards this).

## Architecture

C# / WPF (.NET 8) following MVVM:

| Path | Role |
|------|------|
| `Models/` | Plain data types — Scene, Source (incl. `ClipShape`, `IsMirrored`, `VideoImageSource`), QualityOption, StreamConfig, StreamHealth, YouTubeChannel, ChatMessage |
| `ViewModels/` | MainViewModel — exposes collections + commands for the UI; GoLiveViewModel, ReuseImageViewModel, CameraPickerViewModel |
| `Services/` | YouTube OAuth2, stream/broadcast management, live chat polling, LayoutStore (SQLite), **webcam: `VideoFrame` seam + `CameraDeviceInfo`/`ICameraEnumerator`/`ICameraFrameSource` interfaces + `MediaCaptureCameraEnumerator`/`MediaCaptureFrameSource` (WinRT) + `CameraManager`** |
| `Helpers/` | ViewModelBase (INotifyPropertyChanged), RelayCommand, ImageCache, AppLog (file logger), FocusPreservingListBox, OAuthCredentials, **TokenStore (DPAPI session persistence)**, visibility converters |
| `Themes/` | `Controls.xaml` — the single dark-theme source, merged once in `App.xaml` (see `Themes/index.md`) |
| `MainWindow.xaml` | Dark theme; layout: top bar (controls), center (preview), left (scenes/sources), right (chat), bottom (health) |

### Key patterns

- `ViewModelBase.SetProperty<T>()` for property change notifications
- `RelayCommand` for all button actions; commands gate on state (e.g. Start only when Offline)
- ViewModels are constructed in XAML (`<vm:MainViewModel/>` as DataContext)
- Services are currently instantiated in MainViewModel's constructor — no DI container yet
- Layout persists to SQLite (`Microsoft.Data.Sqlite`); scenes/sources/asset bytes stored in the DB, asset identity is a SHA-256 content hash (1:M reuse, no file paths — assets are always available)
- Theming: all custom styles live in `Themes/Controls.xaml`, merged in `App.xaml` — never duplicate styles per-window (dialog duplicates were consolidated into this dictionary)
- Resolution tiers (bottom bar): 1080p60@8 (default) → 1080p30@8 → 720p60@6 → 720p30@6 → **Vertical 1080p60@8 (9:16, 1080×1920)**. The composition master frame is **always 1920×1080** — a tier is an output rect + target resolution over that master, so source geometry is never rewritten (no rounding drift). 16:9 tiers use the full frame; the vertical tier uses a centered **607×1080** window and the preview dims the cropped side strips at 55% black with an accent outline (semi-crop — the cut area stays visible). A resolution badge in the preview corner shows the active tier; the bottom bar shows bitrate/FPS. A **tooltip** explains finding upload bandwidth — an in-app speed test was deliberately dropped (unreliable). The future encoder crops the master to the rect and scales to the tier's Width×Height
- Crash diagnosis: `AppLog` writes startup checkpoints to `%APPDATA%\ytLlive\startup.log`; `App.xaml.cs` logs `DispatcherUnhandledException`/`AppDomain.UnhandledException`. When WPF won't run from WSL, this log is how you find the failure (it caught the `MenuItemRole.Separator` XAML crash and the ComboBox SelectionBoxItem bug)

### Current limitations / TODOs

- `Helpers/OAuthCredentials.cs` contains the real ClientId/ClientSecret. Auth is complete and the session **persists via Windows DPAPI** (`Helpers/TokenStore.cs` → `%APPDATA%\ytLlive\ytLlive.auth`, CurrentUser scope), reloaded best-effort at startup with a proactive refresh of a near-expiry access token. Sign-in/Change Account lives **inside the Start Stream dialog** (two-state flow — no separate Connect button). A **graceful End Livestream signs out**: `StopStream()` clears the session + token, so the next go-live needs a fresh sign-in; a crash never runs End, so the token survives and the creator stays signed in. `YouTubeAuthService` takes an optional `HttpClient` + `sessionChanged` callback (test seam + save hook; services are still constructed in `MainViewModel`)
- Scene/source/asset layout persists (SQLite, schema v4); the OAuth session persists (DPAPI); the paid-unlock state does not (yet — itch.io key verification pending)
- `YouTubeStreamService` uses hardcoded `1080p`/`60fps` and per-broadcast streams — must switch to the v3 `variable` reusable stream
- Webcam capture is shipped (milestone 1); **screen capture, scene compositing/encoding, RTMP are next**
- `StreamConfig` defaults (`TargetBitrate=6000`, `Resolution="1920x1080"`) are stale — the live dropdown drives `StreamHealth.CurrentBitrate`/`FPS` instead

### Webcam capture (TASK 3 milestone 1)

- **Seam-first:** everything above the WinRT layer speaks only `VideoFrame` (normalized tightly-packed
  BGRA8) + `CameraDeviceInfo`/`ICameraEnumerator`/`ICameraFrameSource` interfaces. Tests inject fakes;
  screen capture and background removal later feed the same seam.
- **CPU-first:** `MediaCaptureInitializationSettings { MemoryPreference = Cpu, StreamingCaptureMode = Video,
  SharingMode = SharedReadOnly }`, frames pulled via
  `CreateFrameReaderAsync(colorSource, MediaEncodingSubtypes.Bgra8)` — the pipeline does any format
  conversion, so every `FrameArrived` yields a ready BGRA8 `SoftwareBitmap` (bytes read via
  `WindowsRuntimeMarshal.TryGetDataUnsafe`, not marshalled copies).
- **Source pick, not first hit:** the frame reader is bound to the first source that is `VideoPreview`
  (preferred) or `VideoRecord`, not blindly the first preview source. If a camera exposes neither, the
  failure names the device and the stream types it *does* expose. `SharedReadOnly` lets the capture
  coexist with other apps that share the camera.
- **Known failure: NVIDIA Broadcast** — it opens the physical webcam exclusively, so `InitializeAsync`
  fails with "camera in use" (or, if init slips through, the device exposes no preview source). Fix:
  quit Broadcast while streaming, or pick its virtual "NVIDIA Broadcast" device from the picker and the
  app captures the processed feed. This is a real-device finding (Logitech `VID_046D&PID_082D`).
- **TFM:** `net8.0-windows10.0.19041.0` (app + tests) pulls the WinRT projection from the SDK reference
  packs — no NuGet package, no capability manifest (unpackaged desktop app works; the Windows privacy
  camera toggle still applies). `EnableWindowsTargeting` keeps WSL builds working.
- **One webcam, many scenes (schema v3):** a singleton `Webcam` row holds the identity
  (`Id`/`DeviceId`/`Name`); each scene gets its own `WebcamSceneConfig` (position/size/clip/mirror/
  border/`IsVisible`). `Scene.Elements` holds images (`Source`) and, at most once, the webcam
  (`WebcamSceneConfig`); `Scene.WebcamConfig` is the accessor. `CameraManager` refcounts capture
  sessions by `DeviceId` (a session starts at `RefCount = 1`; repeat acquire bumps it; the last
  release stops + disposes). The Add Webcam menu greys out when the **active** scene already has a
  config (`CanAddWebcamToActiveScene`); showing a hidden webcam reuses the existing config
  (`CanShowWebcamInActiveScene` / empty-canvas right-click "Show Webcam"). **Removing the last webcam
  config anywhere clears the identity** (`_webcam = null`), so re-adding opens the picker again
  instead of resurrecting the old camera.
- **Round→rect restores the aspect (persisted, schema v4):** `SceneElement.ToggleClipShape()`
  snapshots the rectangular Width/Height into public `RectWidth`/`RectHeight` before going Round and
  restores them when switching back — otherwise the Round resize lock (square) would leave a square
  behind. The rect dims are **persisted** (`WebcamSceneConfig.RectWidth`/`RectHeight`, nullable), so a
  reloaded Round webcam still restores its pre-Round aspect instead of staying square. A one-time
  `HealLegacySquareRect` (load only) widens a pre-v4 `Traditional` config that ended up square to 16:9
  (keeps height; Round and explicit rect dims are untouched).
- **Device swap / layout reload:** `ChangeWebcamAsync` (picker) and `ReacquireWebcam` (after load)
  release the old device with `ReleaseAllAsync` — a forced full drop that zeroes the refcount and
  stops the source regardless of how many scenes held it (the per-config count isn't known once the
  scenes are replaced) — then `AcquireAsync` the new device once per config.
- **Shared bitmap, coalesced updates:** one `WriteableBitmap` per active camera, created on the UI thread
  at the device's frame size (first frame), forwarded to every `WebcamSceneConfig.VideoImageSource` via
  `PreviewBitmapChanged`. Frames arrive on a worker thread; `CameraManager` coalesces onto the dispatcher
  (at most one pending copy per session, at `Render` priority, always copying the latest frame) so a 60fps
  device never drowns the render thread.
- **Clip/mirror/border:** per-element `ClipShape` (Traditional rectangle / Round ellipse) + `IsMirrored`
  (`ScaleX = -1`) + the OSB-standard static border (`BorderColor` `#RRGGBB` or `""`=none, `BorderOpacity`
  0–1, `BorderWidth` 0–20, `BorderAnimation` `None|Pulse|Chase|Rainbow|Shimmer|MarchingAnts|Glow|
  Electricity|Sparkles`). Rendered in the preview DataTemplate; toggled from the element's right-click
  context menu (webcam menu: Change Webcam…, Border Effect submenu — all 9 items enabled, values persist,
  rendering stays static until the animation tier ships — Border Color, Opacity/Thickness sliders, Hide in
  this scene, Remove); persisted in the layout DB. The Add menu shows when no webcam exists; the empty
  preview canvas has its own Show Webcam entry.
  - The Round `Ellipse` is wrapped in a `Viewbox Stretch="Uniform"` holding a `1x1` Grid, so it renders
    as a true circle (diameter = the shorter element dimension) instead of an oval stretched to the
    element rect — and the traditional `Image` keeps `UniformToFill` over the full rect. The Round
    border is a centered `Ellipse` at `Width/Height = RoundBorderSize`.
  - Resizing locks to a square (`_resizeAspect = 1`) while `ClipShape == Round`.
  - **Webcam size clamp:** `ClampWebcamToBounds` (internal — test seam) enforces 50% of the 1920×1080
    master per dimension (960×540 max) and no less than 10% (192×108) at resize + load;
    `RoundBorderSize` follows the clamped height. `WebcamSafeguardTests` guards the clamp.
  - **Hit-testing:** a `Grid` without `Background` only hit-tests where its children draw, so clicks in
    the empty corners of a round clip fell through to `Window_PreviewMouseLeftButtonDown` and deselected
    the element — making the corner handle ungrabbable. The element Grid carries
    `Background="Transparent"` (whole rect draggable; the empty canvas Grid uses the same trick for
    right-click Show Webcam) and the `SelectionOverlay` (dashed border + corner dot) is
  `IsHitTestVisible="False"` so it never intercepts the click. Two things make the webcam menu work:
  (1) the `ContextMenu` pins its own `DataContext` to `PlacementTarget.DataContext` — a `ContextMenu`
  isn't in the visual tree, so without it the Click-handler `DataContext:` patterns (and the
  IsChecked/slider bindings) silently fail; (2) `Themes/Controls.xaml` ships a full dark `MenuItem`
  template — `PART_Popup` (submenu popups), a popup `ItemsPresenter` (the Border Opacity/Thickness
  sliders live in Items, so they render in a hover flyout), a `✓` checkmark column, and a `›` arrow
  driven by `HasItems`. An earlier bare `Border + Header` template dropped all three: submenus never
  opened, sliders never rendered, checkmarks never showed — the menu looked dead even though the
  Click handlers were fine.
- **GPU posture:** webcam frames are CPU (GPU-agnostic; WPF hardware-presents the preview anyway). Hardware
  encoders (NVENC/AMF/QSV) matter for the encoder task, not capture. D3DImage GPU compositing is deferred
  to the encoder task.
- **Background removal = milestone 2** — ONNX Runtime + DirectML (CPU fallback), MediaPipe Selfie
  Segmentation (Apache-2.0), wired into the same `VideoFrame` seam. Not part of milestone 1.

## Design Principle

> This software is so intuitive that even the most right-brained person can easily intuit and use it.

Apply this to every UI decision:
- One-click go-live with working defaults
- Prefilled YouTube defaults (RTMP URL, bitrate, resolution, latency)
- Visual/drag-and-drop scene building over property panels
- Every action produces a visible outcome — no dead ends

## Monetization (design decision — the branding flash is the sword)

Free forever: all streams unlimited, no time caps, no subscription, no per-feature paywalls. The
**one paid line is a one-time unlock** (delivered via itch.io — they handle hosting, payment, and key
delivery; we never own a server or a key shop):

- **Free:** a periodic full-frame branding flash — "made with ytLlive!" rendered big and centered at
  ~25% opacity for about one second (soft 250ms fade in/out), repeated every 300s, on the live output
  (and on v0.2 local recordings). Implemented as `BrandFlashLayer` in the preview compositor
  (`MainWindow.xaml` CanvasGrid) + `BrandFlashTimer` in `MainViewModel` — cadence 300s, first flash
  ~5s after go-live, only while live or recording. An always-on watermark can be cropped or covered;
  an intermittent full-frame flash can't be cropped and is impractical to edit around on a live feed.
- **Paid (one-time):** branding flash removed (flips `BrandFlashEnabled` off) + **Alerts**
  (Super Chat / membership / subscribe pop-ins).

Deliberately rejected: always-on watermark (obscurable — replaced by the flash), hard stream-time
cutoffs (the worst dead end — a stream dying mid-broadcast reads as broken, and YouTube streams
routinely run 2-4 hours), soft-limit nagging, freemium tiers, and donation-only (relies on the
kindness of strangers). Resolution/quality ceilings are **deferred** — that decision belongs to the
resolution & streaming-constraints conversation, not monetization.

## Auth gates Go Live, but not exploration

The app is fully usable without authentication: users can build scenes, add sources, compose
previews, and audition the software with zero commitment. But **going live requires authentication** —
it's the one capability gated behind YouTube sign-in. The sign-in should never pressure the user
("sign in (optional)", not a modal wall): the two-state top bar shows **Start Stream** (offline) /
**End Stream** (live), and the Start Stream dialog hosts the account — a saved session appears as
the default with "Change Account"; with none saved, a "Sign in to YouTube" button starts OAuth and
the Start button stays disabled until signed in.

## Account assumption (do not build an account setup flow)

Connecting uses Google OAuth ("Sign in with Google") to link an **existing** YouTube creator
account. ytLlive **never creates or sets up accounts** — that is YouTube's job. If the creator has no
YouTube channel, they go to YouTube first. This assumption is explicit and must never be silently
replaced by an in-app account-creation step. Zero state = the Start Stream dialog's "Sign in to
YouTube" button; going live is unreachable until an account is connected.

## YouTube Live API — design constraints (do not violate)

These are the hard facts behind every decision. Full list in `TASKS.md`.

- **One-click go-live** — never call `transition(live)`. Insert the broadcast with
  `enableAutoStart=true`, `enableAutoStop=true`, `enableMonitorStream=false`,
  `selfDeclaredMadeForKids=false`, `latencyPreference=low`. The encoder starting brings YouTube live.
  `enableMonitorStream=false` is what lets us skip the testing stage.
- **Variable reusable stream** — `liveStreams.insert` once per channel with
  `cdn.resolution=variable`, `cdn.frameRate=variable`, `isReusable=true`; cache the ingestion URL +
  stream name and reuse for every broadcast. Any quality tier works without recreating the stream,
  and auto step-down is done by us dropping bitrate on the fly (zero API calls).
- **Quality is greyed out while live** — resolution/frameRate/ingestionType are immutable after
  stream creation; editing title/description/privacy is fine at any time.
- **Report-by-exception health** — poll `liveStreams.list`; render nothing on `good`/`ok`, surface a
  banner only on `configurationIssues[]` with `warning`/`error` severity. Bottom strip = YouTube logo
  + green/red connection dot (clickable → opens the dialog).
- **One dialog, three states** — not connected / connected-offline (all editable) / live
  (title + description + visibility editable; quality + account greyed out). Both entry points
  (Start Stream button + bottom strip) open it; prefilled from saved session profile.
- **End stream** — stop encoder → `transition(complete)`, `enableAutoStop` as the safety net.
- **Encoder compliance** — keyframes ≤ 4s (gopSizeLong), closed GOP, H.264, AAC/MP3 @ 44.1/48kHz,
  mono/stereo only. YouTube flags violations via health status.
- **Broadcast ID == Video ID** — one ID tracks status, health, and the auto-created VOD
  (`recordFromStart` + `enableDvr` default true).