Compare commits
3 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| e1d9e1388d | |||
| 3f7f1880c5 | |||
| 60259c08c2 |
@@ -30,7 +30,12 @@ conventions live here and in `ai.md`.
|
|||||||
1. [`schema.md`](schema.md) — the memory-map conventions (what lives where, how to keep it true).
|
1. [`schema.md`](schema.md) — the memory-map conventions (what lives where, how to keep it true).
|
||||||
2. [`ai.md`](ai.md) — the AI guide: architecture, patterns, decisions, current state.
|
2. [`ai.md`](ai.md) — the AI guide: architecture, patterns, decisions, current state.
|
||||||
3. [`TASKS.md`](TASKS.md) — the task queue + authoritative YouTube API research.
|
3. [`TASKS.md`](TASKS.md) — the task queue + authoritative YouTube API research.
|
||||||
4. `<dir>/index.md` — the index for any directory you're about to touch.
|
4. [`HANDOFF.md`](HANDOFF.md) — current operational state: what's in flight, landmines, next step.
|
||||||
|
5. `<dir>/index.md` — the index for any directory you're about to touch.
|
||||||
|
|
||||||
|
**If `HANDOFF.md` exists, trust it as current state** — no `fsck`, no branch
|
||||||
|
hunting, no file-scanning to re-derive what it already states, unless it points
|
||||||
|
at a problem.
|
||||||
|
|
||||||
## Working rules
|
## Working rules
|
||||||
|
|
||||||
@@ -38,6 +43,12 @@ conventions live here and in `ai.md`.
|
|||||||
and the map gets fixed in the same change (stale facts are corrected, not appended).
|
and the map gets fixed in the same change (stale facts are corrected, not appended).
|
||||||
- **Every feature change ships with its memory update:** `ai.md` for
|
- **Every feature change ships with its memory update:** `ai.md` for
|
||||||
architecture/patterns, `TASKS.md` for status, index files when layout changes.
|
architecture/patterns, `TASKS.md` for status, index files when layout changes.
|
||||||
|
- **Rewrite `HANDOFF.md` at session end, compaction, or any interruption.**
|
||||||
|
Never end a session with uncommitted work unrecorded — the handoff names the
|
||||||
|
branch, the dirty files, and why it stopped.
|
||||||
|
- **The first time a fact costs a hunt (secrets path, DB path, port, recovery
|
||||||
|
source), record it** in `ai.md`/indexes/`HANDOFF.md` so the next session never
|
||||||
|
re-hunts it.
|
||||||
- **Follow existing conventions** — MVVM, `RelayCommand` for actions,
|
- **Follow existing conventions** — MVVM, `RelayCommand` for actions,
|
||||||
`ViewModelBase.SetProperty<T>()`, all styles in `Themes/Controls.xaml`
|
`ViewModelBase.SetProperty<T>()`, all styles in `Themes/Controls.xaml`
|
||||||
(merged once in `App.xaml`; never duplicate per-window).
|
(merged once in `App.xaml`; never duplicate per-window).
|
||||||
|
|||||||
+14
@@ -0,0 +1,14 @@
|
|||||||
|
# HANDOFF — session state
|
||||||
|
|
||||||
|
> Current operational state, read right after `TASKS.md`. Trust this file as the
|
||||||
|
> truth of what is in flight — do not re-derive from git/fs unless it points at
|
||||||
|
> a problem. Conventions: [`schema.md`](schema.md). Rewrite this file at session
|
||||||
|
> end, compaction, or any interruption.
|
||||||
|
|
||||||
|
## Session state (last updated: 2026-08-12)
|
||||||
|
|
||||||
|
- **Finished:** Webcam resource validation + first-frame proof — merged to `main` and pushed. `MediaCaptureFrameSource` validates post-init (VideoDeviceId match, stream properties, reader StartAsync status), subscribes `Failed`/`CameraStreamStateChanged` → `SourceFailed`; `CameraManager.AcquireAsync` requires first-frame proof (4s timeout); `MainViewModel` surfaces `WebcamError` chip + MessageBox with suspect-app names (`CameraConflictProbe`). Tested locally — NVIDIA Broadcast identified as the camera hog, killed it, webcam works. 81 tests passing, 0 warnings.
|
||||||
|
- **In flight:** Nothing. Clean tree on `main`.
|
||||||
|
- **Landmine:** 19041 SDK projection gaps: `MediaCaptureSharingMode.Exclusive` + `MediaCapture.DeviceLost` not projected; `CameraStreamState` enum member names omitted (compare by `(int)` — 2=Failed). The Aug 11 social-bar work is **gone** (git gc-pruned, opencode.db reinitialized, gitea refuses unadvertised fetch) — rebuild from scratch per TASKS.md item 14.
|
||||||
|
- **Next step:** Branch B — social bar (global resource, validated lookups, top/bottom + LCR, freemium YT+1 / premium unlimited, +/- scene toggle). Spec in TASKS.md item 14.
|
||||||
|
- **Secret/DB/port facts live:** OAuth client id/secret in `Helpers/OAuthCredentials.cs`; OAuth session token in `Helpers/TokenStore.cs` (DPAPI → `%APPDATA%\ytLlive\ytLlive.auth`); layout DB `%APPDATA%\ytLlive\ytLlive.db` (schema v6); OAuth callback `http://localhost:8765/oauth2/callback`; crash log `%APPDATA%\ytLlive\startup.log`.
|
||||||
@@ -0,0 +1,47 @@
|
|||||||
|
using System.Collections.Generic;
|
||||||
|
using System.Diagnostics;
|
||||||
|
using System.Linq;
|
||||||
|
|
||||||
|
namespace ytLive.Helpers;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Best-effort diagnostic: when a camera won't start, lists other running
|
||||||
|
/// processes known to hold cameras (OBS, Zoom, Teams, NVIDIA Broadcast, etc.).
|
||||||
|
/// Windows doesn't expose "which process has this device" via any public API,
|
||||||
|
/// so this is a suspect list, not a verdict — but it's far better than
|
||||||
|
/// "maybe in use by another app" with no idea which one.
|
||||||
|
/// </summary>
|
||||||
|
public static class CameraConflictProbe
|
||||||
|
{
|
||||||
|
private static readonly HashSet<string> KnownCameraApps = new()
|
||||||
|
{
|
||||||
|
"obs64", "obs32", "zoom", "teams", "ms-teams", "discord",
|
||||||
|
"nvidia broadcast", "skype", "webex", "slack",
|
||||||
|
"streamlabs obs", "restream studio", "manycam", "snap camera",
|
||||||
|
"logitech capture", "logitune", "facerig", "animaze",
|
||||||
|
"camera", "vmix", "xsplit broadcaster", "xsplit gamecaster",
|
||||||
|
"droidcam", "ivcam", "epoccam", "camo",
|
||||||
|
"chrome", "msedge", "firefox", "brave",
|
||||||
|
};
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Returns the friendly names of known camera apps currently running, or an
|
||||||
|
/// empty list if none are found (or the probe itself fails).
|
||||||
|
/// </summary>
|
||||||
|
public static List<string> GetRunningCameraApps()
|
||||||
|
{
|
||||||
|
try
|
||||||
|
{
|
||||||
|
return Process.GetProcesses()
|
||||||
|
.Where(p => KnownCameraApps.Contains(p.ProcessName.ToLowerInvariant()))
|
||||||
|
.Select(p => p.ProcessName)
|
||||||
|
.Distinct()
|
||||||
|
.OrderBy(n => n)
|
||||||
|
.ToList();
|
||||||
|
}
|
||||||
|
catch
|
||||||
|
{
|
||||||
|
return new List<string>();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -12,6 +12,7 @@ Cross-cutting utilities. See [`schema.md`](../schema.md) for the memory-map conv
|
|||||||
| `TokenStore.cs` | DPAPI-protected OAuth session persistence (`%APPDATA%\ytLlive\ytLlive.auth`, CurrentUser scope); `Save`/`Load`/`Clear` — sign-in survives restarts |
|
| `TokenStore.cs` | DPAPI-protected OAuth session persistence (`%APPDATA%\ytLlive\ytLlive.auth`, CurrentUser scope); `Save`/`Load`/`Clear` — sign-in survives restarts |
|
||||||
| `ImageCache.cs` | Image byte caching (assets live in the DB) |
|
| `ImageCache.cs` | Image byte caching (assets live in the DB) |
|
||||||
| `InverseBoolToVisibilityConverter.cs` / `NotNullToVisibilityConverter.cs` | XAML value converters for visibility bindings |
|
| `InverseBoolToVisibilityConverter.cs` / `NotNullToVisibilityConverter.cs` | XAML value converters for visibility bindings |
|
||||||
|
| `CameraConflictProbe.cs` | Best-effort diagnostic: when a camera won't start, enumerates running processes known to hold cameras (OBS, Zoom, Teams, NVIDIA Broadcast, browsers, etc.) — Windows doesn't expose "which process has this device" via any public API, so this is a suspect list, not a verdict |
|
||||||
|
|
||||||
Related: [`Themes/Controls.xaml`](../Themes/Controls.xaml) styles the lists this
|
Related: [`Themes/Controls.xaml`](../Themes/Controls.xaml) styles the lists this
|
||||||
class backs; [`Models/index.md`](../Models/index.md) and
|
class backs; [`Models/index.md`](../Models/index.md) and
|
||||||
|
|||||||
@@ -658,6 +658,12 @@
|
|||||||
Background="#CC16213e" CornerRadius="4" Padding="8,3" IsHitTestVisible="False">
|
Background="#CC16213e" CornerRadius="4" Padding="8,3" IsHitTestVisible="False">
|
||||||
<TextBlock Text="{Binding ResolutionBadgeText}" Foreground="#e0e0e0" FontSize="12"/>
|
<TextBlock Text="{Binding ResolutionBadgeText}" Foreground="#e0e0e0" FontSize="12"/>
|
||||||
</Border>
|
</Border>
|
||||||
|
<Border HorizontalAlignment="Left" VerticalAlignment="Top" Margin="8,8,0,0"
|
||||||
|
Background="#CCe94560" CornerRadius="4" Padding="8,3" IsHitTestVisible="False"
|
||||||
|
Visibility="{Binding WebcamError, Converter={StaticResource NotNullToVis}}">
|
||||||
|
<TextBlock Text="{Binding WebcamError}" Foreground="White" FontSize="12"
|
||||||
|
MaxWidth="520" TextTrimming="CharacterEllipsis"/>
|
||||||
|
</Border>
|
||||||
<TextBlock Text="Preview" Foreground="#333" FontSize="24"
|
<TextBlock Text="Preview" Foreground="#333" FontSize="24"
|
||||||
HorizontalAlignment="Center" VerticalAlignment="Center"
|
HorizontalAlignment="Center" VerticalAlignment="Center"
|
||||||
Visibility="{Binding ShowPreviewPlaceholder, Converter={StaticResource BoolToVis}}"/>
|
Visibility="{Binding ShowPreviewPlaceholder, Converter={StaticResource BoolToVis}}"/>
|
||||||
|
|||||||
@@ -15,6 +15,12 @@ namespace ytLive.Services;
|
|||||||
/// disposes the source. Frames arrive on a worker thread and are coalesced onto
|
/// disposes the source. Frames arrive on a worker thread and are coalesced onto
|
||||||
/// the UI dispatcher (at most one pending copy per session, using the latest
|
/// the UI dispatcher (at most one pending copy per session, using the latest
|
||||||
/// frame) so a 60fps device doesn't drown the render thread.
|
/// frame) so a 60fps device doesn't drown the render thread.
|
||||||
|
///
|
||||||
|
/// Allocation is proven, never assumed: <see cref="AcquireAsync"/> treats a
|
||||||
|
/// started reader as success only once the first frame actually arrives (within
|
||||||
|
/// <paramref name="firstFrameTimeout"/>), and an async <see cref="SourceFailed"/>
|
||||||
|
/// tears the session down and surfaces <see cref="CameraFailed"/> — no silent
|
||||||
|
/// empty box.
|
||||||
/// </summary>
|
/// </summary>
|
||||||
public sealed class CameraManager : IDisposable
|
public sealed class CameraManager : IDisposable
|
||||||
{
|
{
|
||||||
@@ -23,11 +29,13 @@ public sealed class CameraManager : IDisposable
|
|||||||
public string DeviceId { get; }
|
public string DeviceId { get; }
|
||||||
public ICameraFrameSource Source { get; }
|
public ICameraFrameSource Source { get; }
|
||||||
public Action<VideoFrame>? FrameHandler;
|
public Action<VideoFrame>? FrameHandler;
|
||||||
|
public Action<string>? FailureHandler;
|
||||||
public int RefCount;
|
public int RefCount;
|
||||||
public bool Started;
|
public bool Started;
|
||||||
public WriteableBitmap? PreviewBitmap;
|
public WriteableBitmap? PreviewBitmap;
|
||||||
public VideoFrame? LatestFrame;
|
public VideoFrame? LatestFrame;
|
||||||
public bool FramePending;
|
public bool FramePending;
|
||||||
|
public TaskCompletionSource<bool>? FirstFrame;
|
||||||
|
|
||||||
public CameraSession(string deviceId, ICameraFrameSource source)
|
public CameraSession(string deviceId, ICameraFrameSource source)
|
||||||
{
|
{
|
||||||
@@ -40,21 +48,26 @@ public sealed class CameraManager : IDisposable
|
|||||||
private readonly ICameraEnumerator _enumerator;
|
private readonly ICameraEnumerator _enumerator;
|
||||||
private readonly Func<string, ICameraFrameSource> _frameSourceFactory;
|
private readonly Func<string, ICameraFrameSource> _frameSourceFactory;
|
||||||
private readonly Dispatcher? _uiDispatcher;
|
private readonly Dispatcher? _uiDispatcher;
|
||||||
|
private readonly TimeSpan _firstFrameTimeout;
|
||||||
private readonly Dictionary<string, CameraSession> _sessions = new();
|
private readonly Dictionary<string, CameraSession> _sessions = new();
|
||||||
private readonly object _gate = new();
|
private readonly object _gate = new();
|
||||||
|
|
||||||
/// <summary>Raised on the UI thread when a camera's shared preview bitmap is first created.</summary>
|
/// <summary>Raised on the UI thread when a camera's shared preview bitmap is first created.</summary>
|
||||||
public event Action<string, WriteableBitmap>? PreviewBitmapChanged;
|
public event Action<string, WriteableBitmap>? PreviewBitmapChanged;
|
||||||
|
|
||||||
/// <summary>Raised when a capture fails to start (device in use, access denied, no preview source).</summary>
|
/// <summary>
|
||||||
|
/// Raised when a capture fails: device in use, access denied, no preview source,
|
||||||
|
/// or a started reader that never delivers a first frame within the timeout.
|
||||||
|
/// </summary>
|
||||||
public event Action<string, string>? CameraFailed;
|
public event Action<string, string>? CameraFailed;
|
||||||
|
|
||||||
public CameraManager(ICameraEnumerator enumerator, Func<string, ICameraFrameSource> frameSourceFactory,
|
public CameraManager(ICameraEnumerator enumerator, Func<string, ICameraFrameSource> frameSourceFactory,
|
||||||
Dispatcher? uiDispatcher = null)
|
Dispatcher? uiDispatcher = null, TimeSpan? firstFrameTimeout = null)
|
||||||
{
|
{
|
||||||
_enumerator = enumerator;
|
_enumerator = enumerator;
|
||||||
_frameSourceFactory = frameSourceFactory;
|
_frameSourceFactory = frameSourceFactory;
|
||||||
_uiDispatcher = uiDispatcher;
|
_uiDispatcher = uiDispatcher;
|
||||||
|
_firstFrameTimeout = firstFrameTimeout ?? TimeSpan.FromSeconds(4);
|
||||||
}
|
}
|
||||||
|
|
||||||
public ICameraEnumerator Enumerator => _enumerator;
|
public ICameraEnumerator Enumerator => _enumerator;
|
||||||
@@ -79,6 +92,8 @@ public sealed class CameraManager : IDisposable
|
|||||||
session = new CameraSession(deviceId, _frameSourceFactory(deviceId));
|
session = new CameraSession(deviceId, _frameSourceFactory(deviceId));
|
||||||
session.FrameHandler = frame => OnFrameAvailable(session, frame);
|
session.FrameHandler = frame => OnFrameAvailable(session, frame);
|
||||||
session.Source.FrameAvailable += session.FrameHandler;
|
session.Source.FrameAvailable += session.FrameHandler;
|
||||||
|
session.FailureHandler = message => OnSourceFailed(session, message);
|
||||||
|
session.Source.SourceFailed += session.FailureHandler;
|
||||||
_sessions[deviceId] = session;
|
_sessions[deviceId] = session;
|
||||||
shouldStart = true;
|
shouldStart = true;
|
||||||
}
|
}
|
||||||
@@ -86,20 +101,32 @@ public sealed class CameraManager : IDisposable
|
|||||||
|
|
||||||
if (!shouldStart) return session.Started;
|
if (!shouldStart) return session.Started;
|
||||||
|
|
||||||
|
session.FirstFrame = new TaskCompletionSource<bool>(TaskCreationOptions.RunContinuationsAsynchronously);
|
||||||
try
|
try
|
||||||
{
|
{
|
||||||
await session.Source.StartAsync();
|
await session.Source.StartAsync();
|
||||||
|
|
||||||
|
// First-frame proof: "the reader started" is not "the stream is live".
|
||||||
|
// Success requires an actual frame within the timeout, else the session
|
||||||
|
// is rolled back and reported — never left as a silent empty preview.
|
||||||
|
var proven = _firstFrameTimeout <= TimeSpan.Zero
|
||||||
|
|| await WaitForFirstFrameAsync(session, _firstFrameTimeout);
|
||||||
|
if (!proven)
|
||||||
|
{
|
||||||
|
var reason = $"Camera '{deviceId}' started but produced no frames within {_firstFrameTimeout.TotalSeconds:0.#}s.";
|
||||||
|
RollbackSession(session, reason);
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Keep the SourceFailed handler subscribed: an async failure after a
|
||||||
|
// successful start (device lost, stream state Failed) must still surface.
|
||||||
|
session.FirstFrame = null;
|
||||||
session.Started = true;
|
session.Started = true;
|
||||||
return true;
|
return true;
|
||||||
}
|
}
|
||||||
catch (Exception ex)
|
catch (Exception ex)
|
||||||
{
|
{
|
||||||
lock (_gate)
|
RollbackSession(session, ex.Message);
|
||||||
_sessions.Remove(deviceId);
|
|
||||||
session.Source.FrameAvailable -= session.FrameHandler;
|
|
||||||
AppLog.Write($"CameraManager: failed to start camera '{deviceId}': {ex.Message}");
|
|
||||||
CameraFailed?.Invoke(deviceId, ex.Message);
|
|
||||||
await SafeStopAsync(session.Source);
|
|
||||||
return false;
|
return false;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -174,10 +201,47 @@ public sealed class CameraManager : IDisposable
|
|||||||
foreach (var session in sessions)
|
foreach (var session in sessions)
|
||||||
{
|
{
|
||||||
session.Source.FrameAvailable -= session.FrameHandler;
|
session.Source.FrameAvailable -= session.FrameHandler;
|
||||||
|
if (session.FailureHandler != null)
|
||||||
|
session.Source.SourceFailed -= session.FailureHandler;
|
||||||
_ = SafeStopAsync(session.Source);
|
_ = SafeStopAsync(session.Source);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
private static async Task<bool> WaitForFirstFrameAsync(CameraSession session, TimeSpan timeout)
|
||||||
|
{
|
||||||
|
var first = session.FirstFrame;
|
||||||
|
if (first == null) return true;
|
||||||
|
var completed = await Task.WhenAny(first.Task, Task.Delay(timeout));
|
||||||
|
return ReferenceEquals(completed, first.Task) && first.Task.Result;
|
||||||
|
}
|
||||||
|
|
||||||
|
private void RollbackSession(CameraSession session, string message)
|
||||||
|
{
|
||||||
|
lock (_gate)
|
||||||
|
{
|
||||||
|
if (TryGetActiveSession(session))
|
||||||
|
_sessions.Remove(session.DeviceId);
|
||||||
|
}
|
||||||
|
session.Source.FrameAvailable -= session.FrameHandler;
|
||||||
|
if (session.FailureHandler != null)
|
||||||
|
session.Source.SourceFailed -= session.FailureHandler;
|
||||||
|
session.FirstFrame?.TrySetResult(false);
|
||||||
|
|
||||||
|
var enriched = message;
|
||||||
|
var suspects = CameraConflictProbe.GetRunningCameraApps();
|
||||||
|
if (suspects.Count > 0)
|
||||||
|
enriched += $" Other camera apps running: {string.Join(", ", suspects)}.";
|
||||||
|
|
||||||
|
AppLog.Write($"CameraManager: camera '{session.DeviceId}' failed: {enriched}");
|
||||||
|
CameraFailed?.Invoke(session.DeviceId, enriched);
|
||||||
|
_ = SafeStopAsync(session.Source);
|
||||||
|
}
|
||||||
|
|
||||||
|
private void OnSourceFailed(CameraSession session, string message)
|
||||||
|
{
|
||||||
|
RollbackSession(session, message);
|
||||||
|
}
|
||||||
|
|
||||||
private static async Task SafeStopAsync(ICameraFrameSource source)
|
private static async Task SafeStopAsync(ICameraFrameSource source)
|
||||||
{
|
{
|
||||||
try
|
try
|
||||||
@@ -200,6 +264,7 @@ public sealed class CameraManager : IDisposable
|
|||||||
{
|
{
|
||||||
if (!TryGetActiveSession(session)) return;
|
if (!TryGetActiveSession(session)) return;
|
||||||
session.LatestFrame = frame;
|
session.LatestFrame = frame;
|
||||||
|
session.FirstFrame?.TrySetResult(true);
|
||||||
|
|
||||||
if (session.PreviewBitmap == null)
|
if (session.PreviewBitmap == null)
|
||||||
{
|
{
|
||||||
|
|||||||
@@ -8,7 +8,16 @@ namespace ytLive.Services;
|
|||||||
public interface ICameraFrameSource
|
public interface ICameraFrameSource
|
||||||
{
|
{
|
||||||
string DeviceId { get; }
|
string DeviceId { get; }
|
||||||
|
|
||||||
|
/// <summary>Normalized BGRA frames from a worker thread; callers marshal to the UI thread.</summary>
|
||||||
event Action<VideoFrame>? FrameAvailable;
|
event Action<VideoFrame>? FrameAvailable;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Raised when the capture session fails asynchronously after a successful start
|
||||||
|
/// (device lost, stream state Failed, media capture failure). Carries the reason.
|
||||||
|
/// </summary>
|
||||||
|
event Action<string>? SourceFailed;
|
||||||
|
|
||||||
Task StartAsync();
|
Task StartAsync();
|
||||||
Task StopAsync();
|
Task StopAsync();
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,4 +1,5 @@
|
|||||||
using System;
|
using System;
|
||||||
|
using System.Collections.Generic;
|
||||||
using System.Linq;
|
using System.Linq;
|
||||||
using System.Runtime.InteropServices;
|
using System.Runtime.InteropServices;
|
||||||
using System.Runtime.InteropServices.WindowsRuntime;
|
using System.Runtime.InteropServices.WindowsRuntime;
|
||||||
@@ -6,6 +7,7 @@ using System.Threading.Tasks;
|
|||||||
using Windows.Graphics.Imaging;
|
using Windows.Graphics.Imaging;
|
||||||
using Windows.Media.Capture;
|
using Windows.Media.Capture;
|
||||||
using Windows.Media.Capture.Frames;
|
using Windows.Media.Capture.Frames;
|
||||||
|
using Windows.Media.Devices;
|
||||||
using Windows.Media.MediaProperties;
|
using Windows.Media.MediaProperties;
|
||||||
using ytLive.Helpers;
|
using ytLive.Helpers;
|
||||||
|
|
||||||
@@ -16,17 +18,28 @@ namespace ytLive.Services;
|
|||||||
/// preview source; the capture pipeline does any format conversion, so every
|
/// preview source; the capture pipeline does any format conversion, so every
|
||||||
/// frame arrives as a normalized <see cref="VideoFrame"/>. Frames arrive on a
|
/// frame arrives as a normalized <see cref="VideoFrame"/>. Frames arrive on a
|
||||||
/// worker thread — marshal before touching WPF.
|
/// worker thread — marshal before touching WPF.
|
||||||
|
///
|
||||||
|
/// Allocation is validated, never taken on faith: after <c>InitializeAsync</c>
|
||||||
|
/// the bound device, its frame sources, and its stream properties are checked,
|
||||||
|
/// the frame reader's start status is read (not swallowed), and the live
|
||||||
|
/// state signals (<c>Failed</c>, <c>CameraStreamStateChanged</c>) are subscribed
|
||||||
|
/// so an async death surfaces as <see cref="SourceFailed"/> instead of a silent
|
||||||
|
/// empty preview.
|
||||||
/// </summary>
|
/// </summary>
|
||||||
public sealed class MediaCaptureFrameSource : ICameraFrameSource
|
public sealed class MediaCaptureFrameSource : ICameraFrameSource
|
||||||
{
|
{
|
||||||
private readonly string _deviceId;
|
private readonly string _deviceId;
|
||||||
private MediaCapture? _capture;
|
private MediaCapture? _capture;
|
||||||
private MediaFrameReader? _frameReader;
|
private MediaFrameReader? _frameReader;
|
||||||
|
private string? _lastError;
|
||||||
|
private bool _isFailed;
|
||||||
|
|
||||||
public string DeviceId => _deviceId;
|
public string DeviceId => _deviceId;
|
||||||
|
|
||||||
public event Action<VideoFrame>? FrameAvailable;
|
public event Action<VideoFrame>? FrameAvailable;
|
||||||
|
|
||||||
|
public event Action<string>? SourceFailed;
|
||||||
|
|
||||||
public MediaCaptureFrameSource(string deviceId)
|
public MediaCaptureFrameSource(string deviceId)
|
||||||
{
|
{
|
||||||
_deviceId = deviceId;
|
_deviceId = deviceId;
|
||||||
@@ -34,47 +47,15 @@ public sealed class MediaCaptureFrameSource : ICameraFrameSource
|
|||||||
|
|
||||||
public async Task StartAsync()
|
public async Task StartAsync()
|
||||||
{
|
{
|
||||||
var capture = new MediaCapture();
|
// Fallback ladder: prefer the camera's VideoPreview stream, but if the
|
||||||
MediaFrameReader? reader = null;
|
// reader refuses to start there (NoVideoFrameAvailable), retry against
|
||||||
try
|
// its VideoRecord stream — some devices only deliver through it.
|
||||||
|
// (MediaCaptureSharingMode.Exclusive is not projected by the 19041 SDK.)
|
||||||
|
if (!await TryStartAsync(preferVideoRecord: false))
|
||||||
{
|
{
|
||||||
var settings = new MediaCaptureInitializationSettings
|
if (!await TryStartAsync(preferVideoRecord: true))
|
||||||
{
|
|
||||||
VideoDeviceId = _deviceId,
|
|
||||||
StreamingCaptureMode = StreamingCaptureMode.Video,
|
|
||||||
MemoryPreference = MediaCaptureMemoryPreference.Cpu,
|
|
||||||
SharingMode = MediaCaptureSharingMode.SharedReadOnly,
|
|
||||||
};
|
|
||||||
await capture.InitializeAsync(settings);
|
|
||||||
|
|
||||||
var colorSource = capture.FrameSources.Values
|
|
||||||
.OrderBy(s => s.Info.MediaStreamType == MediaStreamType.VideoPreview ? 0 : 1)
|
|
||||||
.FirstOrDefault(s => s.Info.MediaStreamType is MediaStreamType.VideoPreview or MediaStreamType.VideoRecord);
|
|
||||||
if (colorSource == null)
|
|
||||||
{
|
|
||||||
var kinds = string.Join(", ", capture.FrameSources.Values
|
|
||||||
.Select(s => s.Info.MediaStreamType).Distinct());
|
|
||||||
throw new InvalidOperationException(
|
throw new InvalidOperationException(
|
||||||
$"Camera '{_deviceId}' exposes no video source (found: {kinds}). " +
|
_lastError ?? $"Camera '{_deviceId}' could not be started.");
|
||||||
"It may be locked by another app (e.g. NVIDIA Broadcast).");
|
|
||||||
}
|
|
||||||
|
|
||||||
reader = await capture.CreateFrameReaderAsync(colorSource, MediaEncodingSubtypes.Bgra8);
|
|
||||||
reader.FrameArrived += OnFrameArrived;
|
|
||||||
await reader.StartAsync();
|
|
||||||
|
|
||||||
_capture = capture;
|
|
||||||
_frameReader = reader;
|
|
||||||
}
|
|
||||||
catch
|
|
||||||
{
|
|
||||||
if (reader != null)
|
|
||||||
{
|
|
||||||
reader.FrameArrived -= OnFrameArrived;
|
|
||||||
reader.Dispose();
|
|
||||||
}
|
|
||||||
capture.Dispose();
|
|
||||||
throw;
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -98,7 +79,134 @@ public sealed class MediaCaptureFrameSource : ICameraFrameSource
|
|||||||
|
|
||||||
var capture = _capture;
|
var capture = _capture;
|
||||||
_capture = null;
|
_capture = null;
|
||||||
capture?.Dispose();
|
if (capture != null)
|
||||||
|
{
|
||||||
|
UnsubscribeCaptureEvents(capture);
|
||||||
|
capture.Dispose();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private async Task<bool> TryStartAsync(bool preferVideoRecord)
|
||||||
|
{
|
||||||
|
MediaCapture? capture = null;
|
||||||
|
MediaFrameReader? reader = null;
|
||||||
|
try
|
||||||
|
{
|
||||||
|
capture = new MediaCapture();
|
||||||
|
var settings = new MediaCaptureInitializationSettings
|
||||||
|
{
|
||||||
|
VideoDeviceId = _deviceId,
|
||||||
|
StreamingCaptureMode = StreamingCaptureMode.Video,
|
||||||
|
MemoryPreference = MediaCaptureMemoryPreference.Cpu,
|
||||||
|
SharingMode = MediaCaptureSharingMode.SharedReadOnly,
|
||||||
|
};
|
||||||
|
await capture.InitializeAsync(settings);
|
||||||
|
|
||||||
|
// Post-init resource validation: the capture must actually be bound to
|
||||||
|
// the device we asked for and must expose a live video controller.
|
||||||
|
if (!string.Equals(capture.MediaCaptureSettings?.VideoDeviceId, _deviceId,
|
||||||
|
StringComparison.OrdinalIgnoreCase))
|
||||||
|
throw new InvalidOperationException(
|
||||||
|
$"Camera '{_deviceId}' initialized but bound a different device.");
|
||||||
|
|
||||||
|
var colorSource = capture.FrameSources.Values
|
||||||
|
.OrderBy(s => preferVideoRecord
|
||||||
|
? (s.Info.MediaStreamType == MediaStreamType.VideoRecord ? 0 : 1)
|
||||||
|
: (s.Info.MediaStreamType == MediaStreamType.VideoPreview ? 0 : 1))
|
||||||
|
.FirstOrDefault(s => s.Info.MediaStreamType is MediaStreamType.VideoPreview or MediaStreamType.VideoRecord);
|
||||||
|
if (colorSource == null)
|
||||||
|
{
|
||||||
|
var kinds = string.Join(", ", capture.FrameSources.Values
|
||||||
|
.Select(s => s.Info.MediaStreamType).Distinct());
|
||||||
|
throw new InvalidOperationException(
|
||||||
|
$"Camera '{_deviceId}' exposes no video source (found: {kinds}). " +
|
||||||
|
"It may be locked by another app (e.g. NVIDIA Broadcast).");
|
||||||
|
}
|
||||||
|
|
||||||
|
// The device must answer for its preview stream — a dead, suspended, or
|
||||||
|
// locked device returns no stream properties even after "successful" init.
|
||||||
|
IReadOnlyList<IMediaEncodingProperties>? previewProps;
|
||||||
|
try
|
||||||
|
{
|
||||||
|
previewProps = capture.VideoDeviceController?
|
||||||
|
.GetAvailableMediaStreamProperties(MediaStreamType.VideoPreview);
|
||||||
|
}
|
||||||
|
catch
|
||||||
|
{
|
||||||
|
previewProps = null;
|
||||||
|
}
|
||||||
|
if (previewProps == null || previewProps.Count == 0)
|
||||||
|
throw new InvalidOperationException(
|
||||||
|
$"Camera '{_deviceId}' answered no video stream properties (offline, suspended, or locked).");
|
||||||
|
|
||||||
|
// The OS's live state signals: async failures must surface, not vanish.
|
||||||
|
capture.Failed += OnCaptureFailed;
|
||||||
|
capture.CameraStreamStateChanged += OnCameraStreamStateChanged;
|
||||||
|
|
||||||
|
reader = await capture.CreateFrameReaderAsync(colorSource, MediaEncodingSubtypes.Bgra8);
|
||||||
|
if (reader == null)
|
||||||
|
throw new InvalidOperationException($"Camera '{_deviceId}' created no frame reader.");
|
||||||
|
|
||||||
|
reader.FrameArrived += OnFrameArrived;
|
||||||
|
var status = await reader.StartAsync();
|
||||||
|
if (status != MediaFrameReaderStartStatus.Success)
|
||||||
|
throw new InvalidOperationException(
|
||||||
|
$"Camera '{_deviceId}' frame reader refused to start: {status}.");
|
||||||
|
|
||||||
|
_capture = capture;
|
||||||
|
_frameReader = reader;
|
||||||
|
_lastError = null;
|
||||||
|
_isFailed = false;
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
catch (Exception ex)
|
||||||
|
{
|
||||||
|
_lastError = ex.Message;
|
||||||
|
if (reader != null)
|
||||||
|
{
|
||||||
|
reader.FrameArrived -= OnFrameArrived;
|
||||||
|
reader.Dispose();
|
||||||
|
}
|
||||||
|
if (capture != null)
|
||||||
|
{
|
||||||
|
UnsubscribeCaptureEvents(capture);
|
||||||
|
capture.Dispose();
|
||||||
|
}
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private void UnsubscribeCaptureEvents(MediaCapture capture)
|
||||||
|
{
|
||||||
|
capture.Failed -= OnCaptureFailed;
|
||||||
|
capture.CameraStreamStateChanged -= OnCameraStreamStateChanged;
|
||||||
|
}
|
||||||
|
|
||||||
|
private void OnCaptureFailed(MediaCapture sender, MediaCaptureFailedEventArgs args)
|
||||||
|
=> RaiseFailure($"capture failed ({args.Code}): {args.Message}");
|
||||||
|
|
||||||
|
private void OnCameraStreamStateChanged(MediaCapture sender, object args)
|
||||||
|
{
|
||||||
|
// CameraStreamState enum (Windows.Media.Devices): 0=NotStreaming, 1=Streaming,
|
||||||
|
// 2=Failed, 3=Shutdown. The 19041 SDK projection omits member names, so
|
||||||
|
// compare by value — the enum type itself resolves via the property return.
|
||||||
|
if ((int)sender.CameraStreamState == 2) // Failed
|
||||||
|
RaiseFailure("camera stream state is Failed");
|
||||||
|
}
|
||||||
|
|
||||||
|
private void RaiseFailure(string message)
|
||||||
|
{
|
||||||
|
if (_isFailed) return;
|
||||||
|
_isFailed = true;
|
||||||
|
AppLog.Write($"MediaCaptureFrameSource: {message}");
|
||||||
|
try
|
||||||
|
{
|
||||||
|
SourceFailed?.Invoke(message);
|
||||||
|
}
|
||||||
|
catch (Exception ex)
|
||||||
|
{
|
||||||
|
AppLog.Write($"MediaCaptureFrameSource: SourceFailed handler threw: {ex.Message}");
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
private void OnFrameArrived(MediaFrameReader sender, MediaFrameArrivedEventArgs args)
|
private void OnFrameArrived(MediaFrameReader sender, MediaFrameArrivedEventArgs args)
|
||||||
|
|||||||
+3
-3
@@ -12,13 +12,13 @@ External-facing logic: YouTube API, persistence. See
|
|||||||
| `VideoFrame.cs` | Normalized CPU frame seam (`Width`/`Height`/tightly-packed BGRA `byte[]`) — the only pixel type the rest of the app knows about; future capture sources (screen, background-removed webcam) feed the same seam |
|
| `VideoFrame.cs` | Normalized CPU frame seam (`Width`/`Height`/tightly-packed BGRA `byte[]`) — the only pixel type the rest of the app knows about; future capture sources (screen, background-removed webcam) feed the same seam |
|
||||||
| `CameraDeviceInfo.cs` | `(Id, DisplayName)` for a physical capture device |
|
| `CameraDeviceInfo.cs` | `(Id, DisplayName)` for a physical capture device |
|
||||||
| `ICameraEnumerator.cs` | `GetCamerasAsync()` — seam so the picker/`CameraManager` never touch WinRT (tests inject fakes) |
|
| `ICameraEnumerator.cs` | `GetCamerasAsync()` — seam so the picker/`CameraManager` never touch WinRT (tests inject fakes) |
|
||||||
| `ICameraFrameSource.cs` | `StartAsync`/`StopAsync`/`FrameAvailable(VideoFrame)` — seam for a running capture source |
|
| `ICameraFrameSource.cs` | `StartAsync`/`StopAsync`/`FrameAvailable(VideoFrame)` + **`SourceFailed(string)`** — seam for a running capture source (tests inject fakes) |
|
||||||
| `MediaCaptureCameraEnumerator.cs` | WinRT enumeration via `DeviceInformation.FindAllAsync(DeviceClass.VideoCapture)` |
|
| `MediaCaptureCameraEnumerator.cs` | WinRT enumeration via `DeviceInformation.FindAllAsync(DeviceClass.VideoCapture)` |
|
||||||
| `MicrophoneDeviceInfo.cs` | `(Id, DisplayName)` for an audio capture (mic) device |
|
| `MicrophoneDeviceInfo.cs` | `(Id, DisplayName)` for an audio capture (mic) device |
|
||||||
| `IMicrophoneEnumerator.cs` | `GetMicrophonesAsync()` — seam so the mic picker never touches WinRT (tests inject fakes) |
|
| `IMicrophoneEnumerator.cs` | `GetMicrophonesAsync()` — seam so the mic picker never touches WinRT (tests inject fakes) |
|
||||||
| `WinRtMicrophoneEnumerator.cs` | WinRT mic enumeration via `DeviceInformation.FindAllAsync(DeviceClass.AudioCapture)` (no NAudio needed — capture libs stay deferred to the audio pipeline milestone) |
|
| `WinRtMicrophoneEnumerator.cs` | WinRT mic enumeration via `DeviceInformation.FindAllAsync(DeviceClass.AudioCapture)` (no NAudio needed — capture libs stay deferred to the audio pipeline milestone) |
|
||||||
| `MediaCaptureFrameSource.cs` | CPU-first MediaCapture source (`MemoryPreference = Cpu`, BGRA8 via `CreateFrameReaderAsync`); frames arrive on a worker thread |
|
| `MediaCaptureFrameSource.cs` | CPU-first MediaCapture source (`MemoryPreference = Cpu`, BGRA8 via `CreateFrameReaderAsync`). **Validated, not trusted:** post-init checks (`VideoDeviceId` match, `FrameSources` non-empty, `VideoDeviceController.GetAvailableMediaStreamProperties` ≥ 1); `reader.StartAsync()` status read (throws on non-`Success`); `capture.Failed` + `CameraStreamStateChanged` subscribed → `SourceFailed` event. Fallback ladder: VideoPreview → VideoRecord retry. `SharingMode.Exclusive` + `DeviceLost` not in 19041 SDK projection; `CameraStreamState.Failed` compared by `(int)2` (projection omits member names) |
|
||||||
| `CameraManager.cs` | Webcam capture ownership: refcounted by `DeviceId`, one shared `WriteableBitmap` per camera, dispatcher-coalesced ~render-rate UI updates (latest-frame drop); `PreviewBitmapChanged`/`CameraFailed` events. `ReleaseAsync` decrements a ref (stops at zero); `ReleaseAllAsync` force-drops every ref + stops the source (webcam swap / layout reload). `GetPreviewBitmap(deviceId)` returns the current shared bitmap so a `WebcamSceneConfig` added mid-session (after the first frame already created the bitmap) still receives the live frames |
|
| `CameraManager.cs` | Webcam capture ownership: refcounted by `DeviceId`, one shared `WriteableBitmap` per camera, dispatcher-coalesced ~render-rate UI updates (latest-frame drop); `PreviewBitmapChanged`/`CameraFailed` events. **First-frame proof** (4s timeout, configurable via ctor `TimeSpan?`): `AcquireAsync` returns true only after a real frame arrives — a reader that starts but never delivers (locked, suspended, dead) fails and surfaces `CameraFailed` with suspect-app names (`CameraConflictProbe`) instead of a silent empty box. `SourceFailed` forwarded from the frame source as async death signal |
|
||||||
| `IFullScreenDetector.cs` | Seam for the win32 full-screen detector: `int? GetForegroundFullScreenMonitorIndex()`, `int PrimaryMonitorIndex()`, `IReadOnlyList<DisplayInfo> GetDisplays()` |
|
| `IFullScreenDetector.cs` | Seam for the win32 full-screen detector: `int? GetForegroundFullScreenMonitorIndex()`, `int PrimaryMonitorIndex()`, `IReadOnlyList<DisplayInfo> GetDisplays()` |
|
||||||
| `Win32FullScreenDetector.cs` | `GetForegroundWindow` + `DwmGetWindowAttribute(DWMWA_EXTENDED_FRAME_BOUNDS)` + `MonitorFromWindow` + `GetMonitorInfo`; monitor covers all four edges → full-screen; own process excluded; monitor order = `EnumDisplayMonitors` order (static `GetMonitorHandle(int)` maps index→HMONITOR for the capture factory). `GetDisplays()` returns `DisplayInfo` (index/name/resolution/bounds/`IsPrimary`, friendly name via `EnumDisplayDevices`) for the in-app "Capture Display" picker; `PrimaryMonitorIndex()` is the auto-key fallback when no full-screen game is detected |
|
| `Win32FullScreenDetector.cs` | `GetForegroundWindow` + `DwmGetWindowAttribute(DWMWA_EXTENDED_FRAME_BOUNDS)` + `MonitorFromWindow` + `GetMonitorInfo`; monitor covers all four edges → full-screen; own process excluded; monitor order = `EnumDisplayMonitors` order (static `GetMonitorHandle(int)` maps index→HMONITOR for the capture factory). `GetDisplays()` returns `DisplayInfo` (index/name/resolution/bounds/`IsPrimary`, friendly name via `EnumDisplayDevices`) for the in-app "Capture Display" picker; `PrimaryMonitorIndex()` is the auto-key fallback when no full-screen game is detected |
|
||||||
| `IScreenCaptureSource.cs` | `Key` + `StartAsync`/`StopAsync`/`FrameAvailable(VideoFrame)` — seam so `ScreenCaptureManager` never touches WinRT (tests inject fakes) |
|
| `IScreenCaptureSource.cs` | `Key` + `StartAsync`/`StopAsync`/`FrameAvailable(VideoFrame)` — seam so `ScreenCaptureManager` never touches WinRT (tests inject fakes) |
|
||||||
|
|||||||
@@ -104,15 +104,17 @@ Lifecycle: `created → ready → [testing] → live → complete` (transitional
|
|||||||
7. ✅ **Webcam-after-session-start fix** — a webcam added to a scene after the camera was already running (e.g. Chat) previously rendered a transparent container — `CameraManager.GetPreviewBitmap` + propagation in `AddWebcamToActiveSceneAsync`/`ReacquireWebcam` now hands the running shared frames to any newly added `WebcamSceneConfig` — 65 tests passing
|
7. ✅ **Webcam-after-session-start fix** — a webcam added to a scene after the camera was already running (e.g. Chat) previously rendered a transparent container — `CameraManager.GetPreviewBitmap` + propagation in `AddWebcamToActiveSceneAsync`/`ReacquireWebcam` now hands the running shared frames to any newly added `WebcamSceneConfig` — 65 tests passing
|
||||||
8. ✅ **Chat scene webcam size cap** — raised from 50%-per-dimension (960×540) to half the screen AREA (~1358×764 @16:9, `MaxWebcamWidthFor`/`MaxWebcamHeightFor` keyed by canonical name) so the viewer sees the creator better
|
8. ✅ **Chat scene webcam size cap** — raised from 50%-per-dimension (960×540) to half the screen AREA (~1358×764 @16:9, `MaxWebcamWidthFor`/`MaxWebcamHeightFor` keyed by canonical name) so the viewer sees the creator better
|
||||||
9. ✅ **"Add Webcam" always opens the camera picker** — deleting one scene's webcam then re-adding used to resurrect the old camera when another scene still used it — `SwapWebcamIdentityAsync` now swaps the app-wide identity if a different camera is chosen, same path as "Change Webcam…"
|
9. ✅ **"Add Webcam" always opens the camera picker** — deleting one scene's webcam then re-adding used to resurrect the old camera when another scene still used it — `SwapWebcamIdentityAsync` now swaps the app-wide identity if a different camera is chosen, same path as "Change Webcam…"
|
||||||
10. ✅ **Scenes/sources UI** — add/reorder/rename, image + background overlays with move/resize/opacity/reuse
|
10. ✅ **Webcam resource validation + first-frame proof** — `MediaCaptureFrameSource` validates post-init (VideoDeviceId match, stream properties ≥1, `reader.StartAsync()` status read + throws on non-Success); subscribes `capture.Failed` + `CameraStreamStateChanged` → `SourceFailed` event on the seam; fallback ladder (VideoPreview → VideoRecord). `CameraManager.AcquireAsync` requires first-frame proof (4s timeout): returns true only after a real frame arrives — silent empty box impossible. `MainViewModel` subscribes `CameraFailed` → red `WebcamError` chip in preview + MessageBox names suspect apps (`CameraConflictProbe`). 19041 SDK projection gaps: `Exclusive`/`DeviceLost` not projected; `CameraStreamState.Failed` compared by `(int)2`. 81 tests passing
|
||||||
11. ✅ **Audio UX shipped (UI)** — the bottom-bar footer is now two lines (dropped/duration moved under bitrate/fps), with the mic's sound meter + mute button + volume slider grouped CENTERED on the footer's top line, beneath the preview panel (meter: 288px, muted slate track with ruler graduations + muted yellow/red zone tints, green→yellow→red fill; mute = speaker icon → red do-not-symbol when muted, and the slider and speaker stay in sync (volume 0 ⇔ muted — sliding off flips the speaker to muted, sliding up from 0 clears it); mic volume defaults to 80%, muting zeroes the meter and restores the prior volume on unmute (which flashes the meter to the restored position ~300ms before it returns to the live level); the meter is a READ-ONLY realtime level display (fill = live level × volume — volume is a gain on ambient noise; while the slider is dragged the bar previews the slider position and bounces back to the live level on release, which is 0 with no input — clicking the meter does nothing), clicking the MIC label opens a microphone picker whose chosen source shows left-justified inside the meter bar (fill at 75% opacity so the name + ruler markings show through); slim dimensional slider — gradient track/fill, gloss-sphere thumb; the old flat pink 18px-filled one is gone), everything else on line 2 (bitrate/fps/dropped/duration/health left, quality + gear right) — the creator's only audio control, desktop/game audio is automatic (KISS rule)
|
11. ✅ **Scenes/sources UI** — add/reorder/rename, image + background overlays with move/resize/opacity/reuse
|
||||||
12. ✅ The connected YouTube account's avatar/name shows in the top bar next to Start Stream (`SyncConnectedAccount`); the scenes list is content-height now (no dead space before SOURCES)
|
12. ✅ **Audio UX shipped (UI)** — the bottom-bar footer is now two lines (dropped/duration moved under bitrate/fps), with the mic's sound meter + mute button + volume slider grouped CENTERED on the footer's top line, beneath the preview panel (meter: 288px, muted slate track with ruler graduations + muted yellow/red zone tints, green→yellow→red fill; mute = speaker icon → red do-not-symbol when muted, and the slider and speaker stay in sync (volume 0 ⇔ muted — sliding off flips the speaker to muted, sliding up from 0 clears it); mic volume defaults to 80%, muting zeroes the meter and restores the prior volume on unmute (which flashes the meter to the restored position ~300ms before it returns to the live level); the meter is a READ-ONLY realtime level display (fill = live level × volume — volume is a gain on ambient noise; while the slider is dragged the bar previews the slider position and bounces back to the live level on release, which is 0 with no input — clicking the meter does nothing), clicking the MIC label opens a microphone picker whose chosen source shows left-justified inside the meter bar (fill at 75% opacity so the name + ruler markings show through); slim dimensional slider — gradient track/fill, gloss-sphere thumb; the old flat pink 18px-filled one is gone), everything else on line 2 (bitrate/fps/dropped/duration/health left, quality + gear right) — the creator's only audio control, desktop/game audio is automatic (KISS rule)
|
||||||
13. ☐ **Window capture** — absorbed into the Screen picker (no separate source type); dedicated window-as-source work is pending
|
13. ✅ The connected YouTube account's avatar/name shows in the top bar next to Start Stream (`SyncConnectedAccount`); the scenes list is content-height now (no dead space before SOURCES)
|
||||||
14. ☐ **Scene compositing** — the D3DImage/MediaElement preview compositor (this task's requirement 5; the output compositor ships as TASK 4 ship step 1)
|
14. ☐ **Social bar** — global resource like the audio meter; user provides handle/URL → app validates by looking up the social page → adds icon + handle to a single-line horizontal bar (top/bottom, LCR justify). Freemium: YT + 1 other. Premium: unlimited (no wrapping past 1 line). [+] adds to scene, [-] removes. Creates a socials resource per scene. (Work started Aug 11, lost in git incident — rebuild from scratch.)
|
||||||
15. ☐ **Text source** — live text ("Starting soon", "Back in 5", handle, callout)
|
15. ☐ **Window capture** — absorbed into the Screen picker (no separate source type); dedicated window-as-source work is pending
|
||||||
16. ☐ **Chat box** — YouTube live chat rendered *on* the stream so viewers read along in-video
|
16. ☐ **Scene compositing** — the D3DImage/MediaElement preview compositor (this task's requirement 5; the output compositor ships as TASK 4 ship step 1)
|
||||||
17. ❌ **Background removal (milestone 2)** — ONNX Runtime + DirectML, MediaPipe Selfie Segmentation — deliberately NOT in this build
|
17. ☐ **Text source** — live text ("Starting soon", "Back in 5", handle, callout)
|
||||||
18. ☐ **Alerts** — Super Chat / membership / subscribe pop-ins; build after the six; **the one paid feature** (see Monetization in `ai.md`)
|
18. ☐ **Chat box** — YouTube live chat rendered *on* the stream so viewers read along in-video
|
||||||
|
19. ❌ **Background removal (milestone 2)** — ONNX Runtime + DirectML, MediaPipe Selfie Segmentation — deliberately NOT in this build
|
||||||
|
20. ☐ **Alerts** — Super Chat / membership / subscribe pop-ins; build after the six; **the one paid feature** (see Monetization in `ai.md`)
|
||||||
|
|
||||||
### The Minimal Source Set (design decision — do not expand casually)
|
### The Minimal Source Set (design decision — do not expand casually)
|
||||||
|
|
||||||
|
|||||||
@@ -74,7 +74,7 @@ public class MainViewModel : ViewModelBase
|
|||||||
private readonly ICameraEnumerator _cameraEnumerator;
|
private readonly ICameraEnumerator _cameraEnumerator;
|
||||||
private readonly CameraManager _cameraManager;
|
private readonly CameraManager _cameraManager;
|
||||||
private Webcam? _webcam;
|
private Webcam? _webcam;
|
||||||
|
private string? _webcamError;
|
||||||
private readonly IMicrophoneEnumerator _microphoneEnumerator;
|
private readonly IMicrophoneEnumerator _microphoneEnumerator;
|
||||||
|
|
||||||
// Screen backdrop: a permanent live capture (desktop/game) that every scene
|
// Screen backdrop: a permanent live capture (desktop/game) that every scene
|
||||||
@@ -171,6 +171,17 @@ public class MainViewModel : ViewModelBase
|
|||||||
/// <summary>Shows the mirror / clip-shape row in the source chip when a webcam is selected.</summary>
|
/// <summary>Shows the mirror / clip-shape row in the source chip when a webcam is selected.</summary>
|
||||||
public bool IsWebcamSelected => SelectedElement is WebcamSceneConfig;
|
public bool IsWebcamSelected => SelectedElement is WebcamSceneConfig;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// The reason the webcam feed is down (device in use, offline, locked, no frames),
|
||||||
|
/// or null when it's alive. Surfaced as a red chip so a dead camera is never a
|
||||||
|
/// silent empty box. Cleared the moment a real frame arrives.
|
||||||
|
/// </summary>
|
||||||
|
public string? WebcamError
|
||||||
|
{
|
||||||
|
get => _webcamError;
|
||||||
|
private set => SetProperty(ref _webcamError, value);
|
||||||
|
}
|
||||||
|
|
||||||
public StreamStatus StreamStatus
|
public StreamStatus StreamStatus
|
||||||
{
|
{
|
||||||
get => _streamStatus;
|
get => _streamStatus;
|
||||||
@@ -783,6 +794,7 @@ public class MainViewModel : ViewModelBase
|
|||||||
id => new MediaCaptureFrameSource(id),
|
id => new MediaCaptureFrameSource(id),
|
||||||
System.Windows.Application.Current?.Dispatcher);
|
System.Windows.Application.Current?.Dispatcher);
|
||||||
_cameraManager.PreviewBitmapChanged += OnCameraPreviewBitmapChanged;
|
_cameraManager.PreviewBitmapChanged += OnCameraPreviewBitmapChanged;
|
||||||
|
_cameraManager.CameraFailed += OnCameraFailed;
|
||||||
|
|
||||||
_microphoneEnumerator = new WinRtMicrophoneEnumerator();
|
_microphoneEnumerator = new WinRtMicrophoneEnumerator();
|
||||||
|
|
||||||
@@ -921,15 +933,25 @@ public class MainViewModel : ViewModelBase
|
|||||||
}
|
}
|
||||||
|
|
||||||
// CameraManager creates the shared WriteableBitmap on the UI thread at the
|
// CameraManager creates the shared WriteableBitmap on the UI thread at the
|
||||||
// device's frame size; every scene's webcam config picks it up from here.
|
// device's frame size; every scene's webcam config picks it up from here. A
|
||||||
|
// bitmap means a real frame arrived — the camera is provably alive.
|
||||||
private void OnCameraPreviewBitmapChanged(string deviceId, WriteableBitmap bitmap)
|
private void OnCameraPreviewBitmapChanged(string deviceId, WriteableBitmap bitmap)
|
||||||
{
|
{
|
||||||
if (_webcam?.DeviceId != deviceId) return;
|
if (_webcam?.DeviceId != deviceId) return;
|
||||||
|
WebcamError = null;
|
||||||
foreach (var scene in Scenes)
|
foreach (var scene in Scenes)
|
||||||
foreach (var config in scene.Elements.OfType<WebcamSceneConfig>())
|
foreach (var config in scene.Elements.OfType<WebcamSceneConfig>())
|
||||||
config.VideoImageSource = bitmap;
|
config.VideoImageSource = bitmap;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// The camera failed to start or died asynchronously (in use, offline, locked,
|
||||||
|
// no frames within the proof timeout) — surface it instead of a silent box.
|
||||||
|
private void OnCameraFailed(string deviceId, string message)
|
||||||
|
{
|
||||||
|
if (_webcam?.DeviceId != deviceId) return;
|
||||||
|
WebcamError = $"Webcam offline: {message}";
|
||||||
|
}
|
||||||
|
|
||||||
// ─── Screen backdrop capture (live desktop/game) ───
|
// ─── Screen backdrop capture (live desktop/game) ───
|
||||||
|
|
||||||
private const string MonitorKeyPrefix = "monitor:";
|
private const string MonitorKeyPrefix = "monitor:";
|
||||||
@@ -1348,7 +1370,7 @@ public class MainViewModel : ViewModelBase
|
|||||||
if (!started)
|
if (!started)
|
||||||
{
|
{
|
||||||
MessageBox.Show(
|
MessageBox.Show(
|
||||||
"Couldn't start that camera. It may be in use by another app, or Windows camera access may be turned off.",
|
WebcamError ?? "Couldn't start that camera. It may be in use by another app, or Windows camera access may be turned off.",
|
||||||
"ytLlive", MessageBoxButton.OK, MessageBoxImage.Warning);
|
"ytLlive", MessageBoxButton.OK, MessageBoxImage.Warning);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -1372,7 +1394,7 @@ public class MainViewModel : ViewModelBase
|
|||||||
if (!started)
|
if (!started)
|
||||||
{
|
{
|
||||||
MessageBox.Show(
|
MessageBox.Show(
|
||||||
"Couldn't start that camera. It may be in use by another app, or Windows camera access may be turned off.",
|
WebcamError ?? "Couldn't start that camera. It may be in use by another app, or Windows camera access may be turned off.",
|
||||||
"ytLlive", MessageBoxButton.OK, MessageBoxImage.Warning);
|
"ytLlive", MessageBoxButton.OK, MessageBoxImage.Warning);
|
||||||
break;
|
break;
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -20,6 +20,7 @@ that every other memory file follows. Read `ai.md` first; this file explains
|
|||||||
| `README.md` | Human-facing intro: what the app is, how to run it, roadmap | no |
|
| `README.md` | Human-facing intro: what the app is, how to run it, roadmap | no |
|
||||||
| `ai.md` | **AI guide + session handoff** — architecture, patterns, decisions, the cognitive map home | **yes — start here** |
|
| `ai.md` | **AI guide + session handoff** — architecture, patterns, decisions, the cognitive map home | **yes — start here** |
|
||||||
| `TASKS.md` | Task queue + authoritative YouTube API research facts + task statuses | yes — for status |
|
| `TASKS.md` | Task queue + authoritative YouTube API research facts + task statuses | yes — for status |
|
||||||
|
| `HANDOFF.md` | Current operational state: what's in flight, landmines, next step, secret/DB/port locations | yes — trust it as current state |
|
||||||
| `schema.md` | This file: the conventions below | when in doubt |
|
| `schema.md` | This file: the conventions below | when in doubt |
|
||||||
| `<dir>/index.md` | Per-directory map (progressive disclosure): what lives there + links | when diving into code |
|
| `<dir>/index.md` | Per-directory map (progressive disclosure): what lives there + links | when diving into code |
|
||||||
| `Views/` | Reserved for Views; currently empty | — |
|
| `Views/` | Reserved for Views; currently empty | — |
|
||||||
|
|||||||
@@ -1,3 +1,6 @@
|
|||||||
|
using System;
|
||||||
|
using System.Collections.Generic;
|
||||||
|
using System.Threading.Tasks;
|
||||||
using Xunit;
|
using Xunit;
|
||||||
using ytLive.Services;
|
using ytLive.Services;
|
||||||
|
|
||||||
@@ -9,20 +12,28 @@ public class CameraManagerTests
|
|||||||
{
|
{
|
||||||
private readonly List<string> _started;
|
private readonly List<string> _started;
|
||||||
private readonly List<string> _stopped;
|
private readonly List<string> _stopped;
|
||||||
|
private readonly VideoFrame? _pumpOnStart;
|
||||||
|
private readonly string? _failOnStart;
|
||||||
|
|
||||||
public string DeviceId { get; }
|
public string DeviceId { get; }
|
||||||
public event Action<VideoFrame>? FrameAvailable;
|
public event Action<VideoFrame>? FrameAvailable;
|
||||||
|
public event Action<string>? SourceFailed;
|
||||||
|
|
||||||
public FakeFrameSource(string deviceId, List<string> started, List<string> stopped)
|
public FakeFrameSource(string deviceId, List<string> started, List<string> stopped,
|
||||||
|
VideoFrame? pumpOnStart = null, string? failOnStart = null)
|
||||||
{
|
{
|
||||||
DeviceId = deviceId;
|
DeviceId = deviceId;
|
||||||
_started = started;
|
_started = started;
|
||||||
_stopped = stopped;
|
_stopped = stopped;
|
||||||
|
_pumpOnStart = pumpOnStart;
|
||||||
|
_failOnStart = failOnStart;
|
||||||
}
|
}
|
||||||
|
|
||||||
public Task StartAsync()
|
public Task StartAsync()
|
||||||
{
|
{
|
||||||
|
if (_failOnStart != null) throw new InvalidOperationException(_failOnStart);
|
||||||
_started.Add(DeviceId);
|
_started.Add(DeviceId);
|
||||||
|
if (_pumpOnStart != null) FrameAvailable?.Invoke(_pumpOnStart);
|
||||||
return Task.CompletedTask;
|
return Task.CompletedTask;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -33,29 +44,15 @@ public class CameraManagerTests
|
|||||||
}
|
}
|
||||||
|
|
||||||
public void Pump(VideoFrame frame) => FrameAvailable?.Invoke(frame);
|
public void Pump(VideoFrame frame) => FrameAvailable?.Invoke(frame);
|
||||||
|
public void Fail(string message) => SourceFailed?.Invoke(message);
|
||||||
}
|
}
|
||||||
|
|
||||||
private sealed class FailingFrameSource : ICameraFrameSource
|
// The refcount/coalescing tests model a camera whose frame pump is driven by
|
||||||
{
|
// the test after acquire, so they skip the first-frame proof (TimeSpan.Zero);
|
||||||
public string DeviceId { get; }
|
// the proof itself is exercised by the dedicated tests below.
|
||||||
public event Action<VideoFrame>? FrameAvailable;
|
|
||||||
public bool Stopped;
|
|
||||||
|
|
||||||
public FailingFrameSource(string deviceId) => DeviceId = deviceId;
|
|
||||||
|
|
||||||
public Task StartAsync() => throw new InvalidOperationException("camera in use");
|
|
||||||
|
|
||||||
public Task StopAsync()
|
|
||||||
{
|
|
||||||
Stopped = true;
|
|
||||||
return Task.CompletedTask;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void Pump(VideoFrame frame) => FrameAvailable?.Invoke(frame);
|
|
||||||
}
|
|
||||||
|
|
||||||
private static CameraManager CreateManager(List<string> started, List<string> stopped)
|
private static CameraManager CreateManager(List<string> started, List<string> stopped)
|
||||||
=> new(new FakeEnumerator(), id => new FakeFrameSource(id, started, stopped));
|
=> new(new FakeEnumerator(), id => new FakeFrameSource(id, started, stopped),
|
||||||
|
null, TimeSpan.Zero);
|
||||||
|
|
||||||
private sealed class FakeEnumerator : ICameraEnumerator
|
private sealed class FakeEnumerator : ICameraEnumerator
|
||||||
{
|
{
|
||||||
@@ -93,7 +90,8 @@ public class CameraManagerTests
|
|||||||
FakeFrameSource? captured = null;
|
FakeFrameSource? captured = null;
|
||||||
var manager = new CameraManager(
|
var manager = new CameraManager(
|
||||||
new FakeEnumerator(),
|
new FakeEnumerator(),
|
||||||
id => captured = new FakeFrameSource(id, started, stopped));
|
id => captured = new FakeFrameSource(id, started, stopped),
|
||||||
|
null, TimeSpan.Zero);
|
||||||
|
|
||||||
await manager.AcquireAsync("dev1");
|
await manager.AcquireAsync("dev1");
|
||||||
var frame = new VideoFrame(2, 2, new byte[16]);
|
var frame = new VideoFrame(2, 2, new byte[16]);
|
||||||
@@ -105,7 +103,10 @@ public class CameraManagerTests
|
|||||||
[Fact]
|
[Fact]
|
||||||
public async Task Acquire_FailingCamera_ReturnsFalseAndStops()
|
public async Task Acquire_FailingCamera_ReturnsFalseAndStops()
|
||||||
{
|
{
|
||||||
var manager = new CameraManager(new FakeEnumerator(), id => new FailingFrameSource(id));
|
var manager = new CameraManager(
|
||||||
|
new FakeEnumerator(),
|
||||||
|
id => new FakeFrameSource(id, new List<string>(), new List<string>(), failOnStart: "camera in use"),
|
||||||
|
null, TimeSpan.Zero);
|
||||||
string? failedDevice = null;
|
string? failedDevice = null;
|
||||||
manager.CameraFailed += (device, _) => failedDevice = device;
|
manager.CameraFailed += (device, _) => failedDevice = device;
|
||||||
|
|
||||||
@@ -119,4 +120,67 @@ public class CameraManagerTests
|
|||||||
var manager = CreateManager(new List<string>(), new List<string>());
|
var manager = CreateManager(new List<string>(), new List<string>());
|
||||||
Assert.False(await manager.AcquireAsync(" "));
|
Assert.False(await manager.AcquireAsync(" "));
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ─── First-frame proof (the integration test for this change) ───
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public async Task Acquire_SilentCamera_NoFirstFrame_FailsAndSurfacesCameraFailed()
|
||||||
|
{
|
||||||
|
var stopped = new List<string>();
|
||||||
|
var manager = new CameraManager(
|
||||||
|
new FakeEnumerator(),
|
||||||
|
id => new FakeFrameSource(id, new List<string>(), stopped),
|
||||||
|
null, TimeSpan.FromMilliseconds(150));
|
||||||
|
string? failedDevice = null;
|
||||||
|
string? failedMessage = null;
|
||||||
|
manager.CameraFailed += (device, message) =>
|
||||||
|
{
|
||||||
|
failedDevice = device;
|
||||||
|
failedMessage = message;
|
||||||
|
};
|
||||||
|
|
||||||
|
// The reader "starts" fine but never delivers a frame — the exact
|
||||||
|
// silent-empty-box scenario. Must fail, be reported, and be rolled back.
|
||||||
|
Assert.False(await manager.AcquireAsync("dev1"));
|
||||||
|
Assert.Equal("dev1", failedDevice);
|
||||||
|
Assert.Contains("no frames", failedMessage);
|
||||||
|
Assert.Null(manager.GetLatestFrame("dev1"));
|
||||||
|
Assert.Contains("dev1", stopped);
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public async Task Acquire_FirstFrameProvesAlive_ReturnsTrueWithoutWaitingTimeout()
|
||||||
|
{
|
||||||
|
var manager = new CameraManager(
|
||||||
|
new FakeEnumerator(),
|
||||||
|
id => new FakeFrameSource(id, new List<string>(), new List<string>(),
|
||||||
|
pumpOnStart: new VideoFrame(2, 2, new byte[16])),
|
||||||
|
null, TimeSpan.FromSeconds(10));
|
||||||
|
|
||||||
|
var started = await manager.AcquireAsync("dev1");
|
||||||
|
|
||||||
|
Assert.True(started);
|
||||||
|
Assert.NotNull(manager.GetLatestFrame("dev1"));
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public async Task Acquire_SourceFailureAfterStart_SurfacesCameraFailedAndRollsBack()
|
||||||
|
{
|
||||||
|
var stopped = new List<string>();
|
||||||
|
FakeFrameSource? source = null;
|
||||||
|
var manager = new CameraManager(
|
||||||
|
new FakeEnumerator(),
|
||||||
|
id => source = new FakeFrameSource(id, new List<string>(), stopped,
|
||||||
|
pumpOnStart: new VideoFrame(2, 2, new byte[16])),
|
||||||
|
null, TimeSpan.Zero);
|
||||||
|
string? failedMessage = null;
|
||||||
|
manager.CameraFailed += (_, message) => failedMessage = message;
|
||||||
|
|
||||||
|
Assert.True(await manager.AcquireAsync("dev1"));
|
||||||
|
source!.Fail("capture failed (0x8007001F): device not available");
|
||||||
|
|
||||||
|
Assert.Contains("device not available", failedMessage);
|
||||||
|
Assert.Null(manager.GetLatestFrame("dev1"));
|
||||||
|
Assert.Contains("dev1", stopped);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
Reference in New Issue
Block a user