Compare commits

..

3 Commits

13 changed files with 438 additions and 88 deletions
+12 -1
View File
@@ -30,7 +30,12 @@ conventions live here and in `ai.md`.
1. [`schema.md`](schema.md) — the memory-map conventions (what lives where, how to keep it true).
2. [`ai.md`](ai.md) — the AI guide: architecture, patterns, decisions, current state.
3. [`TASKS.md`](TASKS.md) — the task queue + authoritative YouTube API research.
4. `<dir>/index.md` — the index for any directory you're about to touch.
4. [`HANDOFF.md`](HANDOFF.md) — current operational state: what's in flight, landmines, next step.
5. `<dir>/index.md` — the index for any directory you're about to touch.
**If `HANDOFF.md` exists, trust it as current state** — no `fsck`, no branch
hunting, no file-scanning to re-derive what it already states, unless it points
at a problem.
## Working rules
@@ -38,6 +43,12 @@ conventions live here and in `ai.md`.
and the map gets fixed in the same change (stale facts are corrected, not appended).
- **Every feature change ships with its memory update:** `ai.md` for
architecture/patterns, `TASKS.md` for status, index files when layout changes.
- **Rewrite `HANDOFF.md` at session end, compaction, or any interruption.**
Never end a session with uncommitted work unrecorded — the handoff names the
branch, the dirty files, and why it stopped.
- **The first time a fact costs a hunt (secrets path, DB path, port, recovery
source), record it** in `ai.md`/indexes/`HANDOFF.md` so the next session never
re-hunts it.
- **Follow existing conventions** — MVVM, `RelayCommand` for actions,
`ViewModelBase.SetProperty<T>()`, all styles in `Themes/Controls.xaml`
(merged once in `App.xaml`; never duplicate per-window).
+14
View File
@@ -0,0 +1,14 @@
# HANDOFF — session state
> Current operational state, read right after `TASKS.md`. Trust this file as the
> truth of what is in flight — do not re-derive from git/fs unless it points at
> a problem. Conventions: [`schema.md`](schema.md). Rewrite this file at session
> end, compaction, or any interruption.
## Session state (last updated: 2026-08-12)
- **Finished:** Webcam resource validation + first-frame proof — merged to `main` and pushed. `MediaCaptureFrameSource` validates post-init (VideoDeviceId match, stream properties, reader StartAsync status), subscribes `Failed`/`CameraStreamStateChanged``SourceFailed`; `CameraManager.AcquireAsync` requires first-frame proof (4s timeout); `MainViewModel` surfaces `WebcamError` chip + MessageBox with suspect-app names (`CameraConflictProbe`). Tested locally — NVIDIA Broadcast identified as the camera hog, killed it, webcam works. 81 tests passing, 0 warnings.
- **In flight:** Nothing. Clean tree on `main`.
- **Landmine:** 19041 SDK projection gaps: `MediaCaptureSharingMode.Exclusive` + `MediaCapture.DeviceLost` not projected; `CameraStreamState` enum member names omitted (compare by `(int)` — 2=Failed). The Aug 11 social-bar work is **gone** (git gc-pruned, opencode.db reinitialized, gitea refuses unadvertised fetch) — rebuild from scratch per TASKS.md item 14.
- **Next step:** Branch B — social bar (global resource, validated lookups, top/bottom + LCR, freemium YT+1 / premium unlimited, +/- scene toggle). Spec in TASKS.md item 14.
- **Secret/DB/port facts live:** OAuth client id/secret in `Helpers/OAuthCredentials.cs`; OAuth session token in `Helpers/TokenStore.cs` (DPAPI → `%APPDATA%\ytLlive\ytLlive.auth`); layout DB `%APPDATA%\ytLlive\ytLlive.db` (schema v6); OAuth callback `http://localhost:8765/oauth2/callback`; crash log `%APPDATA%\ytLlive\startup.log`.
+47
View File
@@ -0,0 +1,47 @@
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
namespace ytLive.Helpers;
/// <summary>
/// Best-effort diagnostic: when a camera won't start, lists other running
/// processes known to hold cameras (OBS, Zoom, Teams, NVIDIA Broadcast, etc.).
/// Windows doesn't expose "which process has this device" via any public API,
/// so this is a suspect list, not a verdict — but it's far better than
/// "maybe in use by another app" with no idea which one.
/// </summary>
public static class CameraConflictProbe
{
private static readonly HashSet<string> KnownCameraApps = new()
{
"obs64", "obs32", "zoom", "teams", "ms-teams", "discord",
"nvidia broadcast", "skype", "webex", "slack",
"streamlabs obs", "restream studio", "manycam", "snap camera",
"logitech capture", "logitune", "facerig", "animaze",
"camera", "vmix", "xsplit broadcaster", "xsplit gamecaster",
"droidcam", "ivcam", "epoccam", "camo",
"chrome", "msedge", "firefox", "brave",
};
/// <summary>
/// Returns the friendly names of known camera apps currently running, or an
/// empty list if none are found (or the probe itself fails).
/// </summary>
public static List<string> GetRunningCameraApps()
{
try
{
return Process.GetProcesses()
.Where(p => KnownCameraApps.Contains(p.ProcessName.ToLowerInvariant()))
.Select(p => p.ProcessName)
.Distinct()
.OrderBy(n => n)
.ToList();
}
catch
{
return new List<string>();
}
}
}
+1
View File
@@ -12,6 +12,7 @@ Cross-cutting utilities. See [`schema.md`](../schema.md) for the memory-map conv
| `TokenStore.cs` | DPAPI-protected OAuth session persistence (`%APPDATA%\ytLlive\ytLlive.auth`, CurrentUser scope); `Save`/`Load`/`Clear` — sign-in survives restarts |
| `ImageCache.cs` | Image byte caching (assets live in the DB) |
| `InverseBoolToVisibilityConverter.cs` / `NotNullToVisibilityConverter.cs` | XAML value converters for visibility bindings |
| `CameraConflictProbe.cs` | Best-effort diagnostic: when a camera won't start, enumerates running processes known to hold cameras (OBS, Zoom, Teams, NVIDIA Broadcast, browsers, etc.) — Windows doesn't expose "which process has this device" via any public API, so this is a suspect list, not a verdict |
Related: [`Themes/Controls.xaml`](../Themes/Controls.xaml) styles the lists this
class backs; [`Models/index.md`](../Models/index.md) and
+6
View File
@@ -658,6 +658,12 @@
Background="#CC16213e" CornerRadius="4" Padding="8,3" IsHitTestVisible="False">
<TextBlock Text="{Binding ResolutionBadgeText}" Foreground="#e0e0e0" FontSize="12"/>
</Border>
<Border HorizontalAlignment="Left" VerticalAlignment="Top" Margin="8,8,0,0"
Background="#CCe94560" CornerRadius="4" Padding="8,3" IsHitTestVisible="False"
Visibility="{Binding WebcamError, Converter={StaticResource NotNullToVis}}">
<TextBlock Text="{Binding WebcamError}" Foreground="White" FontSize="12"
MaxWidth="520" TextTrimming="CharacterEllipsis"/>
</Border>
<TextBlock Text="Preview" Foreground="#333" FontSize="24"
HorizontalAlignment="Center" VerticalAlignment="Center"
Visibility="{Binding ShowPreviewPlaceholder, Converter={StaticResource BoolToVis}}"/>
+73 -8
View File
@@ -15,6 +15,12 @@ namespace ytLive.Services;
/// disposes the source. Frames arrive on a worker thread and are coalesced onto
/// the UI dispatcher (at most one pending copy per session, using the latest
/// frame) so a 60fps device doesn't drown the render thread.
///
/// Allocation is proven, never assumed: <see cref="AcquireAsync"/> treats a
/// started reader as success only once the first frame actually arrives (within
/// <paramref name="firstFrameTimeout"/>), and an async <see cref="SourceFailed"/>
/// tears the session down and surfaces <see cref="CameraFailed"/> — no silent
/// empty box.
/// </summary>
public sealed class CameraManager : IDisposable
{
@@ -23,11 +29,13 @@ public sealed class CameraManager : IDisposable
public string DeviceId { get; }
public ICameraFrameSource Source { get; }
public Action<VideoFrame>? FrameHandler;
public Action<string>? FailureHandler;
public int RefCount;
public bool Started;
public WriteableBitmap? PreviewBitmap;
public VideoFrame? LatestFrame;
public bool FramePending;
public TaskCompletionSource<bool>? FirstFrame;
public CameraSession(string deviceId, ICameraFrameSource source)
{
@@ -40,21 +48,26 @@ public sealed class CameraManager : IDisposable
private readonly ICameraEnumerator _enumerator;
private readonly Func<string, ICameraFrameSource> _frameSourceFactory;
private readonly Dispatcher? _uiDispatcher;
private readonly TimeSpan _firstFrameTimeout;
private readonly Dictionary<string, CameraSession> _sessions = new();
private readonly object _gate = new();
/// <summary>Raised on the UI thread when a camera's shared preview bitmap is first created.</summary>
public event Action<string, WriteableBitmap>? PreviewBitmapChanged;
/// <summary>Raised when a capture fails to start (device in use, access denied, no preview source).</summary>
/// <summary>
/// Raised when a capture fails: device in use, access denied, no preview source,
/// or a started reader that never delivers a first frame within the timeout.
/// </summary>
public event Action<string, string>? CameraFailed;
public CameraManager(ICameraEnumerator enumerator, Func<string, ICameraFrameSource> frameSourceFactory,
Dispatcher? uiDispatcher = null)
Dispatcher? uiDispatcher = null, TimeSpan? firstFrameTimeout = null)
{
_enumerator = enumerator;
_frameSourceFactory = frameSourceFactory;
_uiDispatcher = uiDispatcher;
_firstFrameTimeout = firstFrameTimeout ?? TimeSpan.FromSeconds(4);
}
public ICameraEnumerator Enumerator => _enumerator;
@@ -79,6 +92,8 @@ public sealed class CameraManager : IDisposable
session = new CameraSession(deviceId, _frameSourceFactory(deviceId));
session.FrameHandler = frame => OnFrameAvailable(session, frame);
session.Source.FrameAvailable += session.FrameHandler;
session.FailureHandler = message => OnSourceFailed(session, message);
session.Source.SourceFailed += session.FailureHandler;
_sessions[deviceId] = session;
shouldStart = true;
}
@@ -86,20 +101,32 @@ public sealed class CameraManager : IDisposable
if (!shouldStart) return session.Started;
session.FirstFrame = new TaskCompletionSource<bool>(TaskCreationOptions.RunContinuationsAsynchronously);
try
{
await session.Source.StartAsync();
// First-frame proof: "the reader started" is not "the stream is live".
// Success requires an actual frame within the timeout, else the session
// is rolled back and reported — never left as a silent empty preview.
var proven = _firstFrameTimeout <= TimeSpan.Zero
|| await WaitForFirstFrameAsync(session, _firstFrameTimeout);
if (!proven)
{
var reason = $"Camera '{deviceId}' started but produced no frames within {_firstFrameTimeout.TotalSeconds:0.#}s.";
RollbackSession(session, reason);
return false;
}
// Keep the SourceFailed handler subscribed: an async failure after a
// successful start (device lost, stream state Failed) must still surface.
session.FirstFrame = null;
session.Started = true;
return true;
}
catch (Exception ex)
{
lock (_gate)
_sessions.Remove(deviceId);
session.Source.FrameAvailable -= session.FrameHandler;
AppLog.Write($"CameraManager: failed to start camera '{deviceId}': {ex.Message}");
CameraFailed?.Invoke(deviceId, ex.Message);
await SafeStopAsync(session.Source);
RollbackSession(session, ex.Message);
return false;
}
}
@@ -174,10 +201,47 @@ public sealed class CameraManager : IDisposable
foreach (var session in sessions)
{
session.Source.FrameAvailable -= session.FrameHandler;
if (session.FailureHandler != null)
session.Source.SourceFailed -= session.FailureHandler;
_ = SafeStopAsync(session.Source);
}
}
private static async Task<bool> WaitForFirstFrameAsync(CameraSession session, TimeSpan timeout)
{
var first = session.FirstFrame;
if (first == null) return true;
var completed = await Task.WhenAny(first.Task, Task.Delay(timeout));
return ReferenceEquals(completed, first.Task) && first.Task.Result;
}
private void RollbackSession(CameraSession session, string message)
{
lock (_gate)
{
if (TryGetActiveSession(session))
_sessions.Remove(session.DeviceId);
}
session.Source.FrameAvailable -= session.FrameHandler;
if (session.FailureHandler != null)
session.Source.SourceFailed -= session.FailureHandler;
session.FirstFrame?.TrySetResult(false);
var enriched = message;
var suspects = CameraConflictProbe.GetRunningCameraApps();
if (suspects.Count > 0)
enriched += $" Other camera apps running: {string.Join(", ", suspects)}.";
AppLog.Write($"CameraManager: camera '{session.DeviceId}' failed: {enriched}");
CameraFailed?.Invoke(session.DeviceId, enriched);
_ = SafeStopAsync(session.Source);
}
private void OnSourceFailed(CameraSession session, string message)
{
RollbackSession(session, message);
}
private static async Task SafeStopAsync(ICameraFrameSource source)
{
try
@@ -200,6 +264,7 @@ public sealed class CameraManager : IDisposable
{
if (!TryGetActiveSession(session)) return;
session.LatestFrame = frame;
session.FirstFrame?.TrySetResult(true);
if (session.PreviewBitmap == null)
{
+9
View File
@@ -8,7 +8,16 @@ namespace ytLive.Services;
public interface ICameraFrameSource
{
string DeviceId { get; }
/// <summary>Normalized BGRA frames from a worker thread; callers marshal to the UI thread.</summary>
event Action<VideoFrame>? FrameAvailable;
/// <summary>
/// Raised when the capture session fails asynchronously after a successful start
/// (device lost, stream state Failed, media capture failure). Carries the reason.
/// </summary>
event Action<string>? SourceFailed;
Task StartAsync();
Task StopAsync();
}
+148 -40
View File
@@ -1,4 +1,5 @@
using System;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.InteropServices;
using System.Runtime.InteropServices.WindowsRuntime;
@@ -6,6 +7,7 @@ using System.Threading.Tasks;
using Windows.Graphics.Imaging;
using Windows.Media.Capture;
using Windows.Media.Capture.Frames;
using Windows.Media.Devices;
using Windows.Media.MediaProperties;
using ytLive.Helpers;
@@ -16,17 +18,28 @@ namespace ytLive.Services;
/// preview source; the capture pipeline does any format conversion, so every
/// frame arrives as a normalized <see cref="VideoFrame"/>. Frames arrive on a
/// worker thread — marshal before touching WPF.
///
/// Allocation is validated, never taken on faith: after <c>InitializeAsync</c>
/// the bound device, its frame sources, and its stream properties are checked,
/// the frame reader's start status is read (not swallowed), and the live
/// state signals (<c>Failed</c>, <c>CameraStreamStateChanged</c>) are subscribed
/// so an async death surfaces as <see cref="SourceFailed"/> instead of a silent
/// empty preview.
/// </summary>
public sealed class MediaCaptureFrameSource : ICameraFrameSource
{
private readonly string _deviceId;
private MediaCapture? _capture;
private MediaFrameReader? _frameReader;
private string? _lastError;
private bool _isFailed;
public string DeviceId => _deviceId;
public event Action<VideoFrame>? FrameAvailable;
public event Action<string>? SourceFailed;
public MediaCaptureFrameSource(string deviceId)
{
_deviceId = deviceId;
@@ -34,47 +47,15 @@ public sealed class MediaCaptureFrameSource : ICameraFrameSource
public async Task StartAsync()
{
var capture = new MediaCapture();
MediaFrameReader? reader = null;
try
// Fallback ladder: prefer the camera's VideoPreview stream, but if the
// reader refuses to start there (NoVideoFrameAvailable), retry against
// its VideoRecord stream — some devices only deliver through it.
// (MediaCaptureSharingMode.Exclusive is not projected by the 19041 SDK.)
if (!await TryStartAsync(preferVideoRecord: false))
{
var settings = new MediaCaptureInitializationSettings
{
VideoDeviceId = _deviceId,
StreamingCaptureMode = StreamingCaptureMode.Video,
MemoryPreference = MediaCaptureMemoryPreference.Cpu,
SharingMode = MediaCaptureSharingMode.SharedReadOnly,
};
await capture.InitializeAsync(settings);
var colorSource = capture.FrameSources.Values
.OrderBy(s => s.Info.MediaStreamType == MediaStreamType.VideoPreview ? 0 : 1)
.FirstOrDefault(s => s.Info.MediaStreamType is MediaStreamType.VideoPreview or MediaStreamType.VideoRecord);
if (colorSource == null)
{
var kinds = string.Join(", ", capture.FrameSources.Values
.Select(s => s.Info.MediaStreamType).Distinct());
if (!await TryStartAsync(preferVideoRecord: true))
throw new InvalidOperationException(
$"Camera '{_deviceId}' exposes no video source (found: {kinds}). " +
"It may be locked by another app (e.g. NVIDIA Broadcast).");
}
reader = await capture.CreateFrameReaderAsync(colorSource, MediaEncodingSubtypes.Bgra8);
reader.FrameArrived += OnFrameArrived;
await reader.StartAsync();
_capture = capture;
_frameReader = reader;
}
catch
{
if (reader != null)
{
reader.FrameArrived -= OnFrameArrived;
reader.Dispose();
}
capture.Dispose();
throw;
_lastError ?? $"Camera '{_deviceId}' could not be started.");
}
}
@@ -98,7 +79,134 @@ public sealed class MediaCaptureFrameSource : ICameraFrameSource
var capture = _capture;
_capture = null;
capture?.Dispose();
if (capture != null)
{
UnsubscribeCaptureEvents(capture);
capture.Dispose();
}
}
private async Task<bool> TryStartAsync(bool preferVideoRecord)
{
MediaCapture? capture = null;
MediaFrameReader? reader = null;
try
{
capture = new MediaCapture();
var settings = new MediaCaptureInitializationSettings
{
VideoDeviceId = _deviceId,
StreamingCaptureMode = StreamingCaptureMode.Video,
MemoryPreference = MediaCaptureMemoryPreference.Cpu,
SharingMode = MediaCaptureSharingMode.SharedReadOnly,
};
await capture.InitializeAsync(settings);
// Post-init resource validation: the capture must actually be bound to
// the device we asked for and must expose a live video controller.
if (!string.Equals(capture.MediaCaptureSettings?.VideoDeviceId, _deviceId,
StringComparison.OrdinalIgnoreCase))
throw new InvalidOperationException(
$"Camera '{_deviceId}' initialized but bound a different device.");
var colorSource = capture.FrameSources.Values
.OrderBy(s => preferVideoRecord
? (s.Info.MediaStreamType == MediaStreamType.VideoRecord ? 0 : 1)
: (s.Info.MediaStreamType == MediaStreamType.VideoPreview ? 0 : 1))
.FirstOrDefault(s => s.Info.MediaStreamType is MediaStreamType.VideoPreview or MediaStreamType.VideoRecord);
if (colorSource == null)
{
var kinds = string.Join(", ", capture.FrameSources.Values
.Select(s => s.Info.MediaStreamType).Distinct());
throw new InvalidOperationException(
$"Camera '{_deviceId}' exposes no video source (found: {kinds}). " +
"It may be locked by another app (e.g. NVIDIA Broadcast).");
}
// The device must answer for its preview stream — a dead, suspended, or
// locked device returns no stream properties even after "successful" init.
IReadOnlyList<IMediaEncodingProperties>? previewProps;
try
{
previewProps = capture.VideoDeviceController?
.GetAvailableMediaStreamProperties(MediaStreamType.VideoPreview);
}
catch
{
previewProps = null;
}
if (previewProps == null || previewProps.Count == 0)
throw new InvalidOperationException(
$"Camera '{_deviceId}' answered no video stream properties (offline, suspended, or locked).");
// The OS's live state signals: async failures must surface, not vanish.
capture.Failed += OnCaptureFailed;
capture.CameraStreamStateChanged += OnCameraStreamStateChanged;
reader = await capture.CreateFrameReaderAsync(colorSource, MediaEncodingSubtypes.Bgra8);
if (reader == null)
throw new InvalidOperationException($"Camera '{_deviceId}' created no frame reader.");
reader.FrameArrived += OnFrameArrived;
var status = await reader.StartAsync();
if (status != MediaFrameReaderStartStatus.Success)
throw new InvalidOperationException(
$"Camera '{_deviceId}' frame reader refused to start: {status}.");
_capture = capture;
_frameReader = reader;
_lastError = null;
_isFailed = false;
return true;
}
catch (Exception ex)
{
_lastError = ex.Message;
if (reader != null)
{
reader.FrameArrived -= OnFrameArrived;
reader.Dispose();
}
if (capture != null)
{
UnsubscribeCaptureEvents(capture);
capture.Dispose();
}
return false;
}
}
private void UnsubscribeCaptureEvents(MediaCapture capture)
{
capture.Failed -= OnCaptureFailed;
capture.CameraStreamStateChanged -= OnCameraStreamStateChanged;
}
private void OnCaptureFailed(MediaCapture sender, MediaCaptureFailedEventArgs args)
=> RaiseFailure($"capture failed ({args.Code}): {args.Message}");
private void OnCameraStreamStateChanged(MediaCapture sender, object args)
{
// CameraStreamState enum (Windows.Media.Devices): 0=NotStreaming, 1=Streaming,
// 2=Failed, 3=Shutdown. The 19041 SDK projection omits member names, so
// compare by value — the enum type itself resolves via the property return.
if ((int)sender.CameraStreamState == 2) // Failed
RaiseFailure("camera stream state is Failed");
}
private void RaiseFailure(string message)
{
if (_isFailed) return;
_isFailed = true;
AppLog.Write($"MediaCaptureFrameSource: {message}");
try
{
SourceFailed?.Invoke(message);
}
catch (Exception ex)
{
AppLog.Write($"MediaCaptureFrameSource: SourceFailed handler threw: {ex.Message}");
}
}
private void OnFrameArrived(MediaFrameReader sender, MediaFrameArrivedEventArgs args)
+3 -3
View File
@@ -12,13 +12,13 @@ External-facing logic: YouTube API, persistence. See
| `VideoFrame.cs` | Normalized CPU frame seam (`Width`/`Height`/tightly-packed BGRA `byte[]`) — the only pixel type the rest of the app knows about; future capture sources (screen, background-removed webcam) feed the same seam |
| `CameraDeviceInfo.cs` | `(Id, DisplayName)` for a physical capture device |
| `ICameraEnumerator.cs` | `GetCamerasAsync()` — seam so the picker/`CameraManager` never touch WinRT (tests inject fakes) |
| `ICameraFrameSource.cs` | `StartAsync`/`StopAsync`/`FrameAvailable(VideoFrame)` — seam for a running capture source |
| `ICameraFrameSource.cs` | `StartAsync`/`StopAsync`/`FrameAvailable(VideoFrame)` + **`SourceFailed(string)`** — seam for a running capture source (tests inject fakes) |
| `MediaCaptureCameraEnumerator.cs` | WinRT enumeration via `DeviceInformation.FindAllAsync(DeviceClass.VideoCapture)` |
| `MicrophoneDeviceInfo.cs` | `(Id, DisplayName)` for an audio capture (mic) device |
| `IMicrophoneEnumerator.cs` | `GetMicrophonesAsync()` — seam so the mic picker never touches WinRT (tests inject fakes) |
| `WinRtMicrophoneEnumerator.cs` | WinRT mic enumeration via `DeviceInformation.FindAllAsync(DeviceClass.AudioCapture)` (no NAudio needed — capture libs stay deferred to the audio pipeline milestone) |
| `MediaCaptureFrameSource.cs` | CPU-first MediaCapture source (`MemoryPreference = Cpu`, BGRA8 via `CreateFrameReaderAsync`); frames arrive on a worker thread |
| `CameraManager.cs` | Webcam capture ownership: refcounted by `DeviceId`, one shared `WriteableBitmap` per camera, dispatcher-coalesced ~render-rate UI updates (latest-frame drop); `PreviewBitmapChanged`/`CameraFailed` events. `ReleaseAsync` decrements a ref (stops at zero); `ReleaseAllAsync` force-drops every ref + stops the source (webcam swap / layout reload). `GetPreviewBitmap(deviceId)` returns the current shared bitmap so a `WebcamSceneConfig` added mid-session (after the first frame already created the bitmap) still receives the live frames |
| `MediaCaptureFrameSource.cs` | CPU-first MediaCapture source (`MemoryPreference = Cpu`, BGRA8 via `CreateFrameReaderAsync`). **Validated, not trusted:** post-init checks (`VideoDeviceId` match, `FrameSources` non-empty, `VideoDeviceController.GetAvailableMediaStreamProperties` ≥ 1); `reader.StartAsync()` status read (throws on non-`Success`); `capture.Failed` + `CameraStreamStateChanged` subscribed → `SourceFailed` event. Fallback ladder: VideoPreview → VideoRecord retry. `SharingMode.Exclusive` + `DeviceLost` not in 19041 SDK projection; `CameraStreamState.Failed` compared by `(int)2` (projection omits member names) |
| `CameraManager.cs` | Webcam capture ownership: refcounted by `DeviceId`, one shared `WriteableBitmap` per camera, dispatcher-coalesced ~render-rate UI updates (latest-frame drop); `PreviewBitmapChanged`/`CameraFailed` events. **First-frame proof** (4s timeout, configurable via ctor `TimeSpan?`): `AcquireAsync` returns true only after a real frame arrives — a reader that starts but never delivers (locked, suspended, dead) fails and surfaces `CameraFailed` with suspect-app names (`CameraConflictProbe`) instead of a silent empty box. `SourceFailed` forwarded from the frame source as async death signal |
| `IFullScreenDetector.cs` | Seam for the win32 full-screen detector: `int? GetForegroundFullScreenMonitorIndex()`, `int PrimaryMonitorIndex()`, `IReadOnlyList<DisplayInfo> GetDisplays()` |
| `Win32FullScreenDetector.cs` | `GetForegroundWindow` + `DwmGetWindowAttribute(DWMWA_EXTENDED_FRAME_BOUNDS)` + `MonitorFromWindow` + `GetMonitorInfo`; monitor covers all four edges → full-screen; own process excluded; monitor order = `EnumDisplayMonitors` order (static `GetMonitorHandle(int)` maps index→HMONITOR for the capture factory). `GetDisplays()` returns `DisplayInfo` (index/name/resolution/bounds/`IsPrimary`, friendly name via `EnumDisplayDevices`) for the in-app "Capture Display" picker; `PrimaryMonitorIndex()` is the auto-key fallback when no full-screen game is detected |
| `IScreenCaptureSource.cs` | `Key` + `StartAsync`/`StopAsync`/`FrameAvailable(VideoFrame)` — seam so `ScreenCaptureManager` never touches WinRT (tests inject fakes) |
+11 -9
View File
@@ -104,15 +104,17 @@ Lifecycle: `created → ready → [testing] → live → complete` (transitional
7.**Webcam-after-session-start fix** — a webcam added to a scene after the camera was already running (e.g. Chat) previously rendered a transparent container — `CameraManager.GetPreviewBitmap` + propagation in `AddWebcamToActiveSceneAsync`/`ReacquireWebcam` now hands the running shared frames to any newly added `WebcamSceneConfig` — 65 tests passing
8.**Chat scene webcam size cap** — raised from 50%-per-dimension (960×540) to half the screen AREA (~1358×764 @16:9, `MaxWebcamWidthFor`/`MaxWebcamHeightFor` keyed by canonical name) so the viewer sees the creator better
9.**"Add Webcam" always opens the camera picker** — deleting one scene's webcam then re-adding used to resurrect the old camera when another scene still used it — `SwapWebcamIdentityAsync` now swaps the app-wide identity if a different camera is chosen, same path as "Change Webcam…"
10.**Scenes/sources UI** — add/reorder/rename, image + background overlays with move/resize/opacity/reuse
11.**Audio UX shipped (UI)** — the bottom-bar footer is now two lines (dropped/duration moved under bitrate/fps), with the mic's sound meter + mute button + volume slider grouped CENTERED on the footer's top line, beneath the preview panel (meter: 288px, muted slate track with ruler graduations + muted yellow/red zone tints, green→yellow→red fill; mute = speaker icon → red do-not-symbol when muted, and the slider and speaker stay in sync (volume 0 ⇔ muted — sliding off flips the speaker to muted, sliding up from 0 clears it); mic volume defaults to 80%, muting zeroes the meter and restores the prior volume on unmute (which flashes the meter to the restored position ~300ms before it returns to the live level); the meter is a READ-ONLY realtime level display (fill = live level × volume — volume is a gain on ambient noise; while the slider is dragged the bar previews the slider position and bounces back to the live level on release, which is 0 with no input — clicking the meter does nothing), clicking the MIC label opens a microphone picker whose chosen source shows left-justified inside the meter bar (fill at 75% opacity so the name + ruler markings show through); slim dimensional slider — gradient track/fill, gloss-sphere thumb; the old flat pink 18px-filled one is gone), everything else on line 2 (bitrate/fps/dropped/duration/health left, quality + gear right) — the creator's only audio control, desktop/game audio is automatic (KISS rule)
12.The connected YouTube account's avatar/name shows in the top bar next to Start Stream (`SyncConnectedAccount`); the scenes list is content-height now (no dead space before SOURCES)
13. **Window capture** — absorbed into the Screen picker (no separate source type); dedicated window-as-source work is pending
14.**Scene compositing** — the D3DImage/MediaElement preview compositor (this task's requirement 5; the output compositor ships as TASK 4 ship step 1)
15.**Text source**live text ("Starting soon", "Back in 5", handle, callout)
16.**Chat box** — YouTube live chat rendered *on* the stream so viewers read along in-video
17. **Background removal (milestone 2)** — ONNX Runtime + DirectML, MediaPipe Selfie Segmentation — deliberately NOT in this build
18.**Alerts** — Super Chat / membership / subscribe pop-ins; build after the six; **the one paid feature** (see Monetization in `ai.md`)
10.**Webcam resource validation + first-frame proof**`MediaCaptureFrameSource` validates post-init (VideoDeviceId match, stream properties ≥1, `reader.StartAsync()` status read + throws on non-Success); subscribes `capture.Failed` + `CameraStreamStateChanged``SourceFailed` event on the seam; fallback ladder (VideoPreview → VideoRecord). `CameraManager.AcquireAsync` requires first-frame proof (4s timeout): returns true only after a real frame arrives — silent empty box impossible. `MainViewModel` subscribes `CameraFailed` → red `WebcamError` chip in preview + MessageBox names suspect apps (`CameraConflictProbe`). 19041 SDK projection gaps: `Exclusive`/`DeviceLost` not projected; `CameraStreamState.Failed` compared by `(int)2`. 81 tests passing
11.**Scenes/sources UI** — add/reorder/rename, image + background overlays with move/resize/opacity/reuse
12.**Audio UX shipped (UI)** — the bottom-bar footer is now two lines (dropped/duration moved under bitrate/fps), with the mic's sound meter + mute button + volume slider grouped CENTERED on the footer's top line, beneath the preview panel (meter: 288px, muted slate track with ruler graduations + muted yellow/red zone tints, green→yellow→red fill; mute = speaker icon → red do-not-symbol when muted, and the slider and speaker stay in sync (volume 0 ⇔ muted — sliding off flips the speaker to muted, sliding up from 0 clears it); mic volume defaults to 80%, muting zeroes the meter and restores the prior volume on unmute (which flashes the meter to the restored position ~300ms before it returns to the live level); the meter is a READ-ONLY realtime level display (fill = live level × volume — volume is a gain on ambient noise; while the slider is dragged the bar previews the slider position and bounces back to the live level on release, which is 0 with no input — clicking the meter does nothing), clicking the MIC label opens a microphone picker whose chosen source shows left-justified inside the meter bar (fill at 75% opacity so the name + ruler markings show through); slim dimensional slider — gradient track/fill, gloss-sphere thumb; the old flat pink 18px-filled one is gone), everything else on line 2 (bitrate/fps/dropped/duration/health left, quality + gear right) — the creator's only audio control, desktop/game audio is automatic (KISS rule)
13. ✅ The connected YouTube account's avatar/name shows in the top bar next to Start Stream (`SyncConnectedAccount`); the scenes list is content-height now (no dead space before SOURCES)
14.**Social bar** — global resource like the audio meter; user provides handle/URL → app validates by looking up the social page → adds icon + handle to a single-line horizontal bar (top/bottom, LCR justify). Freemium: YT + 1 other. Premium: unlimited (no wrapping past 1 line). [+] adds to scene, [-] removes. Creates a socials resource per scene. (Work started Aug 11, lost in git incident — rebuild from scratch.)
15.**Window capture**absorbed into the Screen picker (no separate source type); dedicated window-as-source work is pending
16.**Scene compositing** — the D3DImage/MediaElement preview compositor (this task's requirement 5; the output compositor ships as TASK 4 ship step 1)
17. **Text source** — live text ("Starting soon", "Back in 5", handle, callout)
18.**Chat box** — YouTube live chat rendered *on* the stream so viewers read along in-video
19.**Background removal (milestone 2)** — ONNX Runtime + DirectML, MediaPipe Selfie Segmentation — deliberately NOT in this build
20.**Alerts** — Super Chat / membership / subscribe pop-ins; build after the six; **the one paid feature** (see Monetization in `ai.md`)
### The Minimal Source Set (design decision — do not expand casually)
+26 -4
View File
@@ -74,7 +74,7 @@ public class MainViewModel : ViewModelBase
private readonly ICameraEnumerator _cameraEnumerator;
private readonly CameraManager _cameraManager;
private Webcam? _webcam;
private string? _webcamError;
private readonly IMicrophoneEnumerator _microphoneEnumerator;
// Screen backdrop: a permanent live capture (desktop/game) that every scene
@@ -171,6 +171,17 @@ public class MainViewModel : ViewModelBase
/// <summary>Shows the mirror / clip-shape row in the source chip when a webcam is selected.</summary>
public bool IsWebcamSelected => SelectedElement is WebcamSceneConfig;
/// <summary>
/// The reason the webcam feed is down (device in use, offline, locked, no frames),
/// or null when it's alive. Surfaced as a red chip so a dead camera is never a
/// silent empty box. Cleared the moment a real frame arrives.
/// </summary>
public string? WebcamError
{
get => _webcamError;
private set => SetProperty(ref _webcamError, value);
}
public StreamStatus StreamStatus
{
get => _streamStatus;
@@ -783,6 +794,7 @@ public class MainViewModel : ViewModelBase
id => new MediaCaptureFrameSource(id),
System.Windows.Application.Current?.Dispatcher);
_cameraManager.PreviewBitmapChanged += OnCameraPreviewBitmapChanged;
_cameraManager.CameraFailed += OnCameraFailed;
_microphoneEnumerator = new WinRtMicrophoneEnumerator();
@@ -921,15 +933,25 @@ public class MainViewModel : ViewModelBase
}
// CameraManager creates the shared WriteableBitmap on the UI thread at the
// device's frame size; every scene's webcam config picks it up from here.
// device's frame size; every scene's webcam config picks it up from here. A
// bitmap means a real frame arrived — the camera is provably alive.
private void OnCameraPreviewBitmapChanged(string deviceId, WriteableBitmap bitmap)
{
if (_webcam?.DeviceId != deviceId) return;
WebcamError = null;
foreach (var scene in Scenes)
foreach (var config in scene.Elements.OfType<WebcamSceneConfig>())
config.VideoImageSource = bitmap;
}
// The camera failed to start or died asynchronously (in use, offline, locked,
// no frames within the proof timeout) — surface it instead of a silent box.
private void OnCameraFailed(string deviceId, string message)
{
if (_webcam?.DeviceId != deviceId) return;
WebcamError = $"Webcam offline: {message}";
}
// ─── Screen backdrop capture (live desktop/game) ───
private const string MonitorKeyPrefix = "monitor:";
@@ -1348,7 +1370,7 @@ public class MainViewModel : ViewModelBase
if (!started)
{
MessageBox.Show(
"Couldn't start that camera. It may be in use by another app, or Windows camera access may be turned off.",
WebcamError ?? "Couldn't start that camera. It may be in use by another app, or Windows camera access may be turned off.",
"ytLlive", MessageBoxButton.OK, MessageBoxImage.Warning);
}
}
@@ -1372,7 +1394,7 @@ public class MainViewModel : ViewModelBase
if (!started)
{
MessageBox.Show(
"Couldn't start that camera. It may be in use by another app, or Windows camera access may be turned off.",
WebcamError ?? "Couldn't start that camera. It may be in use by another app, or Windows camera access may be turned off.",
"ytLlive", MessageBoxButton.OK, MessageBoxImage.Warning);
break;
}
+1
View File
@@ -20,6 +20,7 @@ that every other memory file follows. Read `ai.md` first; this file explains
| `README.md` | Human-facing intro: what the app is, how to run it, roadmap | no |
| `ai.md` | **AI guide + session handoff** — architecture, patterns, decisions, the cognitive map home | **yes — start here** |
| `TASKS.md` | Task queue + authoritative YouTube API research facts + task statuses | yes — for status |
| `HANDOFF.md` | Current operational state: what's in flight, landmines, next step, secret/DB/port locations | yes — trust it as current state |
| `schema.md` | This file: the conventions below | when in doubt |
| `<dir>/index.md` | Per-directory map (progressive disclosure): what lives there + links | when diving into code |
| `Views/` | Reserved for Views; currently empty | — |
+87 -23
View File
@@ -1,3 +1,6 @@
using System;
using System.Collections.Generic;
using System.Threading.Tasks;
using Xunit;
using ytLive.Services;
@@ -9,20 +12,28 @@ public class CameraManagerTests
{
private readonly List<string> _started;
private readonly List<string> _stopped;
private readonly VideoFrame? _pumpOnStart;
private readonly string? _failOnStart;
public string DeviceId { get; }
public event Action<VideoFrame>? FrameAvailable;
public event Action<string>? SourceFailed;
public FakeFrameSource(string deviceId, List<string> started, List<string> stopped)
public FakeFrameSource(string deviceId, List<string> started, List<string> stopped,
VideoFrame? pumpOnStart = null, string? failOnStart = null)
{
DeviceId = deviceId;
_started = started;
_stopped = stopped;
_pumpOnStart = pumpOnStart;
_failOnStart = failOnStart;
}
public Task StartAsync()
{
if (_failOnStart != null) throw new InvalidOperationException(_failOnStart);
_started.Add(DeviceId);
if (_pumpOnStart != null) FrameAvailable?.Invoke(_pumpOnStart);
return Task.CompletedTask;
}
@@ -33,29 +44,15 @@ public class CameraManagerTests
}
public void Pump(VideoFrame frame) => FrameAvailable?.Invoke(frame);
public void Fail(string message) => SourceFailed?.Invoke(message);
}
private sealed class FailingFrameSource : ICameraFrameSource
{
public string DeviceId { get; }
public event Action<VideoFrame>? FrameAvailable;
public bool Stopped;
public FailingFrameSource(string deviceId) => DeviceId = deviceId;
public Task StartAsync() => throw new InvalidOperationException("camera in use");
public Task StopAsync()
{
Stopped = true;
return Task.CompletedTask;
}
public void Pump(VideoFrame frame) => FrameAvailable?.Invoke(frame);
}
// The refcount/coalescing tests model a camera whose frame pump is driven by
// the test after acquire, so they skip the first-frame proof (TimeSpan.Zero);
// the proof itself is exercised by the dedicated tests below.
private static CameraManager CreateManager(List<string> started, List<string> stopped)
=> new(new FakeEnumerator(), id => new FakeFrameSource(id, started, stopped));
=> new(new FakeEnumerator(), id => new FakeFrameSource(id, started, stopped),
null, TimeSpan.Zero);
private sealed class FakeEnumerator : ICameraEnumerator
{
@@ -93,7 +90,8 @@ public class CameraManagerTests
FakeFrameSource? captured = null;
var manager = new CameraManager(
new FakeEnumerator(),
id => captured = new FakeFrameSource(id, started, stopped));
id => captured = new FakeFrameSource(id, started, stopped),
null, TimeSpan.Zero);
await manager.AcquireAsync("dev1");
var frame = new VideoFrame(2, 2, new byte[16]);
@@ -105,7 +103,10 @@ public class CameraManagerTests
[Fact]
public async Task Acquire_FailingCamera_ReturnsFalseAndStops()
{
var manager = new CameraManager(new FakeEnumerator(), id => new FailingFrameSource(id));
var manager = new CameraManager(
new FakeEnumerator(),
id => new FakeFrameSource(id, new List<string>(), new List<string>(), failOnStart: "camera in use"),
null, TimeSpan.Zero);
string? failedDevice = null;
manager.CameraFailed += (device, _) => failedDevice = device;
@@ -119,4 +120,67 @@ public class CameraManagerTests
var manager = CreateManager(new List<string>(), new List<string>());
Assert.False(await manager.AcquireAsync(" "));
}
// ─── First-frame proof (the integration test for this change) ───
[Fact]
public async Task Acquire_SilentCamera_NoFirstFrame_FailsAndSurfacesCameraFailed()
{
var stopped = new List<string>();
var manager = new CameraManager(
new FakeEnumerator(),
id => new FakeFrameSource(id, new List<string>(), stopped),
null, TimeSpan.FromMilliseconds(150));
string? failedDevice = null;
string? failedMessage = null;
manager.CameraFailed += (device, message) =>
{
failedDevice = device;
failedMessage = message;
};
// The reader "starts" fine but never delivers a frame — the exact
// silent-empty-box scenario. Must fail, be reported, and be rolled back.
Assert.False(await manager.AcquireAsync("dev1"));
Assert.Equal("dev1", failedDevice);
Assert.Contains("no frames", failedMessage);
Assert.Null(manager.GetLatestFrame("dev1"));
Assert.Contains("dev1", stopped);
}
[Fact]
public async Task Acquire_FirstFrameProvesAlive_ReturnsTrueWithoutWaitingTimeout()
{
var manager = new CameraManager(
new FakeEnumerator(),
id => new FakeFrameSource(id, new List<string>(), new List<string>(),
pumpOnStart: new VideoFrame(2, 2, new byte[16])),
null, TimeSpan.FromSeconds(10));
var started = await manager.AcquireAsync("dev1");
Assert.True(started);
Assert.NotNull(manager.GetLatestFrame("dev1"));
}
[Fact]
public async Task Acquire_SourceFailureAfterStart_SurfacesCameraFailedAndRollsBack()
{
var stopped = new List<string>();
FakeFrameSource? source = null;
var manager = new CameraManager(
new FakeEnumerator(),
id => source = new FakeFrameSource(id, new List<string>(), stopped,
pumpOnStart: new VideoFrame(2, 2, new byte[16])),
null, TimeSpan.Zero);
string? failedMessage = null;
manager.CameraFailed += (_, message) => failedMessage = message;
Assert.True(await manager.AcquireAsync("dev1"));
source!.Fail("capture failed (0x8007001F): device not available");
Assert.Contains("device not available", failedMessage);
Assert.Null(manager.GetLatestFrame("dev1"));
Assert.Contains("dev1", stopped);
}
}