TASK 3 milestone 1: webcam capture (MediaCapture CPU-first, CameraManager refcount, picker, clip/mirror, schema v2, tests)
This commit is contained in:
@@ -36,7 +36,16 @@ Note: `EnableWindowsTargeting=true` is set in `ytLive.csproj`, so the project ca
|
||||
|
||||
## Tests
|
||||
|
||||
No test framework set up yet. When added: `dotnet test`.
|
||||
xUnit in `ytLive.Tests` (net8.0-windows10.0.19041.0, matches the app TFM). Run on Windows —
|
||||
WSL can't run net8.0-windows tests:
|
||||
|
||||
```bash
|
||||
dotnet.exe vstest "C:\...\ytLive.Tests\bin\Debug\net8.0-windows10.0.19041.0\ytLive.Tests.dll"
|
||||
```
|
||||
|
||||
Good Dog Rule: ONE integration test per branch, ONE test per PR. Current: TokenStore DPAPI
|
||||
roundtrip/corrupt/missing/clear, OAuth exchange/refresh/ClearSession, CameraManager refcount +
|
||||
frame pump + failure handling (fakes for the WinRT seams) — 11 passing.
|
||||
|
||||
## Architecture
|
||||
|
||||
@@ -44,9 +53,9 @@ C# / WPF (.NET 8) following MVVM:
|
||||
|
||||
| Path | Role |
|
||||
|------|------|
|
||||
| `Models/` | Plain data types — Scene, Source, QualityOption, StreamConfig, StreamHealth, YouTubeChannel, ChatMessage |
|
||||
| `ViewModels/` | MainViewModel — exposes collections + commands for the UI; GoLiveViewModel, ReuseImageViewModel |
|
||||
| `Services/` | YouTube OAuth2, stream/broadcast management, live chat polling, LayoutStore (SQLite) |
|
||||
| `Models/` | Plain data types — Scene, Source (incl. `ClipShape`, `IsMirrored`, `VideoImageSource`), QualityOption, StreamConfig, StreamHealth, YouTubeChannel, ChatMessage |
|
||||
| `ViewModels/` | MainViewModel — exposes collections + commands for the UI; GoLiveViewModel, ReuseImageViewModel, CameraPickerViewModel |
|
||||
| `Services/` | YouTube OAuth2, stream/broadcast management, live chat polling, LayoutStore (SQLite), **webcam: `VideoFrame` seam + `CameraDeviceInfo`/`ICameraEnumerator`/`ICameraFrameSource` interfaces + `MediaCaptureCameraEnumerator`/`MediaCaptureFrameSource` (WinRT) + `CameraManager`** |
|
||||
| `Helpers/` | ViewModelBase (INotifyPropertyChanged), RelayCommand, ImageCache, AppLog (file logger), FocusPreservingListBox, OAuthCredentials, **TokenStore (DPAPI session persistence)**, visibility converters |
|
||||
| `Themes/` | `Controls.xaml` — the single dark-theme source, merged once in `App.xaml` (see `Themes/index.md`) |
|
||||
| `MainWindow.xaml` | Dark theme; layout: top bar (controls), center (preview), left (scenes/sources), right (chat), bottom (health) |
|
||||
@@ -65,11 +74,41 @@ C# / WPF (.NET 8) following MVVM:
|
||||
### Current limitations / TODOs
|
||||
|
||||
- `Helpers/OAuthCredentials.cs` contains the real ClientId/ClientSecret. Auth is complete and the session **persists via Windows DPAPI** (`Helpers/TokenStore.cs` → `%APPDATA%\ytLlive\ytLlive.auth`, CurrentUser scope), reloaded best-effort at startup with a proactive refresh of a near-expiry access token. Sign-in/Change Account lives **inside the Start Stream dialog** (two-state flow — no separate Connect button). A **graceful End Livestream signs out**: `StopStream()` clears the session + token, so the next go-live needs a fresh sign-in; a crash never runs End, so the token survives and the creator stays signed in. `YouTubeAuthService` takes an optional `HttpClient` + `sessionChanged` callback (test seam + save hook; services are still constructed in `MainViewModel`)
|
||||
- Scene/source/asset layout persists (SQLite); the OAuth session persists (DPAPI); the paid-unlock state does not (yet — itch.io key verification pending)
|
||||
- Scene/source/asset layout persists (SQLite, schema v2); the OAuth session persists (DPAPI); the paid-unlock state does not (yet — itch.io key verification pending)
|
||||
- `YouTubeStreamService` uses hardcoded `1080p`/`60fps` and per-broadcast streams — must switch to the v3 `variable` reusable stream
|
||||
- No capture/encoding/RTMP yet
|
||||
- Webcam capture is shipped (milestone 1); **screen capture, scene compositing/encoding, RTMP are next**
|
||||
- `StreamConfig` defaults (`TargetBitrate=6000`, `Resolution="1920x1080"`) are stale — the live dropdown drives `StreamHealth.CurrentBitrate`/`FPS` instead
|
||||
|
||||
### Webcam capture (TASK 3 milestone 1)
|
||||
|
||||
- **Seam-first:** everything above the WinRT layer speaks only `VideoFrame` (normalized tightly-packed
|
||||
BGRA8) + `CameraDeviceInfo`/`ICameraEnumerator`/`ICameraFrameSource` interfaces. Tests inject fakes;
|
||||
screen capture and background removal later feed the same seam.
|
||||
- **CPU-first:** `MediaCaptureInitializationSettings { MemoryPreference = Cpu, StreamingCaptureMode = Video }`,
|
||||
frames pulled via `CreateFrameReaderAsync(colorSource, MediaEncodingSubtypes.Bgra8)` — the pipeline does
|
||||
any format conversion, so every `FrameArrived` yields a ready BGRA8 `SoftwareBitmap` (bytes read via
|
||||
`WindowsRuntimeMarshal.TryGetDataUnsafe`, not marshalled copies).
|
||||
- **TFM:** `net8.0-windows10.0.19041.0` (app + tests) pulls the WinRT projection from the SDK reference
|
||||
packs — no NuGet package, no capability manifest (unpackaged desktop app works; the Windows privacy
|
||||
camera toggle still applies). `EnableWindowsTargeting` keeps WSL builds working.
|
||||
- **One camera app-wide:** `CameraManager` refcounts sessions by `DeviceId` (a session is created with
|
||||
`RefCount = 1`; repeat acquire bumps it; the last release stops + disposes). The Add menu greys Webcam
|
||||
out once a webcam source exists anywhere (`CanAddWebcam`); the "OBS time" story is a one-camera limit.
|
||||
- **Shared bitmap, coalesced updates:** one `WriteableBitmap` per active camera, created on the UI thread
|
||||
at the device's frame size (first frame), forwarded to the single webcam `Source.VideoImageSource` via
|
||||
`PreviewBitmapChanged`. Frames arrive on a worker thread; `CameraManager` coalesces onto the dispatcher
|
||||
(at most one pending copy per session, at `Render` priority, always copying the latest frame) so a 60fps
|
||||
device never drowns the render thread.
|
||||
- **Clip/mirror:** per-Source `ClipShape` (Traditional rectangle / Round ellipse) + `IsMirrored`
|
||||
(`ScaleX = -1`). Rendered in the preview DataTemplate (Image for Traditional, `ImageBrush` inside an
|
||||
`Ellipse` for Round); toggled from the source chip; persisted in the layout DB. Round hit-testing is the
|
||||
same rectangle as Traditional (selection overlay is rectangular) — acceptable for now.
|
||||
- **GPU posture:** webcam frames are CPU (GPU-agnostic; WPF hardware-presents the preview anyway). Hardware
|
||||
encoders (NVENC/AMF/QSV) matter for the encoder task, not capture. D3DImage GPU compositing is deferred
|
||||
to the encoder task.
|
||||
- **Background removal = milestone 2** — ONNX Runtime + DirectML (CPU fallback), MediaPipe Selfie
|
||||
Segmentation (Apache-2.0), wired into the same `VideoFrame` seam. Not part of milestone 1.
|
||||
|
||||
## Design Principle
|
||||
|
||||
> This software is so intuitive that even the most right-brained person can easily intuit and use it.
|
||||
|
||||
Reference in New Issue
Block a user