diff --git a/HANDOFF.md b/HANDOFF.md index b7a5543..636b7e5 100644 --- a/HANDOFF.md +++ b/HANDOFF.md @@ -7,41 +7,57 @@ ## Session state (last updated: 2026-08-13) -- **Branch:** `main`. TASK 4 **ship step 5.5** is **committed and pushed** (`ac60a26`, - "TASK 4 ship step 5.5: social bar bug fixes + bar on the live output"). -- **This session (follow-up):** the drag-snap from 5.5 failed in practice — it snapped - up but wouldn't come back down (jitter around the deadzone, per the user). **Superseded - by a click-toggle** (user decision, KISS): `MainWindow.SocialBar_MouseLeftButtonDown` - → `MainViewModel.ToggleSocialBarPosition()` flips the bar top ⇄ bottom; the bar rides - `{Binding SocialBarTop}` alone; `SocialBarSnap` + its 2 tests removed. Build 0 warnings, - **153 tests passing** (155 − 2 snap units). **Uncommitted:** this click-toggle change + - its memory corrections (TASKS.md / ai.md / HANDOFF). Next action: commit + push. -- **Verified:** 0 warnings; 153/153 tests pass. +- **Branch:** `main`. The **game audio bar + mic status dot + always-on capture** + work is **uncommitted** (see list below). The last committed+pushed baseline is + `ea250c0` (social bar click-toggle). +- **This session (audio UX follow-up):** per the creator's requests — + 1. The MIC label is now a **button with a status dot** (`Models/MicStatus`: + green = capturing, yellow = requested mic problem, red = no device). + 2. **Mic capture runs for the app's lifetime** (started at startup via + `StartMicCaptureAsync`, disposed in `Shutdown`; `BeginGoLive`/`StopStream` + no longer start/stop the mixer) so both meters preview live. + 3. A **game audio bar** (desktop/game — meter + mute + volume, mirror of the + mic bar) sits centered beneath the preview and appears only while a + **full-screen game is producing sound** (decision: fullscreen + loopback + sound; **silence never hides an active bar** — the creator's final rule). + Show after ~500ms of fullscreen+sound, hide ~1s after leaving fullscreen. +- **Uncommitted files:** `MainWindow.xaml`/`.cs`, `ViewModels/MainViewModel.cs`, + `Services/Audio/{IAudioSource,WasapiMicAudioSource,WasapiLoopbackAudioSource,AudioMixer}.cs`, + new `Models/MicStatus.cs`, new `Services/{IGameAudioDetector,GameAudioHysteresis,GameAudioDetector}.cs`, + `ytLive.Tests/AudioMixerTests.cs`, new `GameAudioHysteresisTests.cs` + + `GameAudioDetectorTests.cs`, memory docs (TASKS.md, ai.md, Services/index.md, + ViewModels/index.md, Models/index.md, HANDOFF). +- **Verified:** build 0 warnings, 0 errors; **167 tests passing** (full suite, + after the final rebuild). - **Landmines:** - Never set a local `Canvas.SetTop` on the social bar — a local value permanently overrides `{Binding SocialBarTop}` (the `ClearValue` lesson from 5.5). + - `AudioMixer` meter `Push` is unconditional **by design now**: `OnMicSample`/ + `OnLoopbackSample` compute the level first, then raise the event — a + `?.Invoke(meter.Push(...))` short-circuit skipped the meter update when + nothing was subscribed (found by `RestartMic_ResetsLevel`, fixed). + - `MicConnected` comes from the source `Started` event, raised right after + `StartRecording()` succeeds — tests must `MarkStarted()` the fake source + before asserting connection state. + - Zero mic devices at startup = red dot AND the mixer is never started, so + loopback + the game bar can't run either (no capture at all) — acceptable. + - The game detector is polled on the UI thread via a 250ms `DispatcherTimer`; + `OnGameAudioPollTick` wraps `Poll()` in try/catch + `AppLog`. - The pump reads the active scene on a background thread while the UI can still edit it — a concurrent-mutation exception is contained (logged + `Failed` + - the pump stops), not a crash. The background thread + video pipeline is the - new reality since ship step 5. + the pump stops), not a crash. - `StopAsync` must stop the encoder (closes stdin) **before** awaiting the pump loop — closing stdin unblocks a write stuck on pipe backpressure; the reverse order deadlocks. - - `FramePump.IsRunning` must be set true before the loop starts (a completed-task - delay can run the first iteration synchronously on the caller's thread). - - `StartAsync` never throws; the VM fires-and-forgets it. `Failed` while live - flips `StreamStatus.Error` (minimal — real health surfacing is ship step 6). - - The heal runs off the UI thread and applies via the dispatcher; a config swap - mid-heal (dialog Save) can orphan the healed values on the old entries — harmless - (best-effort, re-healed next load). - Tests never instantiate `MainViewModel` directly except the round-clip integration test (a real `MainWindow`), which never goes live — keep it that way. - - Sandbox can't reach outbound HTTPS — the subdomain-probe logic is verified via - stub-handler tests only, not against the real `mastodon.llamachile.tube`. -- **Next step:** commit + push the click-toggle follow-up. Then TASK 4 ship step 6 — - health stats: bind `FramePump.HealthUpdated` (bitrate/FPS/duration) into the bottom - bar. Nothing else queued — do not expand the task queue on your own. Optional, not - queued: rewriting the healed entry's `ProfileUrl` to + - Sandbox can't reach outbound HTTPS — `HttpSocialValidator` stub-handler tests + only, never the real instance. +- **Next step:** commit + push the audio UX follow-up (the whole uncommitted set + above in one commit). Then TASK 4 ship step 6 — health stats: bind + `FramePump.HealthUpdated` (bitrate/FPS/duration) into the bottom bar. Nothing + else queued — do not expand the task queue on your own. Optional, not queued: + rewriting the healed entry's `ProfileUrl` to `https://mastodon.llamachile.tube/@gramps` (user must say the word). - **Secret/DB/port facts live:** OAuth client id/secret in `Helpers/OAuthCredentials.cs`; diff --git a/MainWindow.xaml b/MainWindow.xaml index 3f8103f..06a41b1 100644 --- a/MainWindow.xaml +++ b/MainWindow.xaml @@ -795,13 +795,15 @@ + + Audio is KISS: desktop/game audio is automatic (WASAPI loopback + at unity); the creator's controls are the two meters — the mic + (always visible, with its status dot) and the game bar (appears + only while a full-screen game is producing sound). --> @@ -824,13 +826,19 @@ - + - + @@ -943,6 +951,58 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/MainWindow.xaml.cs b/MainWindow.xaml.cs index 6b86b6e..b8356cd 100644 --- a/MainWindow.xaml.cs +++ b/MainWindow.xaml.cs @@ -83,12 +83,6 @@ public partial class MainWindow : Window } } - private void MicLabel_MouseLeftButtonUp(object sender, MouseButtonEventArgs e) - { - if (sender is FrameworkElement { DataContext: MainViewModel vm }) - vm.OpenMicPickerCommand.Execute(null); - } - private void VolumeSlider_PreviewMouseLeftButtonDown(object sender, MouseButtonEventArgs e) { if (sender is FrameworkElement { DataContext: MainViewModel vm }) @@ -107,6 +101,24 @@ public partial class MainWindow : Window vm.SetVolumeAdjusting(false); } + private void GameVolumeSlider_PreviewMouseLeftButtonDown(object sender, MouseButtonEventArgs e) + { + if (sender is FrameworkElement { DataContext: MainViewModel vm }) + vm.SetGameVolumeAdjusting(true); + } + + private void GameVolumeSlider_PreviewMouseLeftButtonUp(object sender, MouseButtonEventArgs e) + { + if (sender is FrameworkElement { DataContext: MainViewModel vm }) + vm.SetGameVolumeAdjusting(false); + } + + private void GameVolumeSlider_LostMouseCapture(object sender, MouseEventArgs e) + { + if (sender is FrameworkElement { DataContext: MainViewModel vm }) + vm.SetGameVolumeAdjusting(false); + } + private void MicSpeaker_MouseLeftButtonUp(object sender, MouseButtonEventArgs e) { if (sender is FrameworkElement { DataContext: MainViewModel vm }) @@ -115,6 +127,14 @@ public partial class MainWindow : Window } } + private void GameSpeaker_MouseLeftButtonUp(object sender, MouseButtonEventArgs e) + { + if (sender is FrameworkElement { DataContext: MainViewModel vm }) + { + vm.ToggleGameMuteCommand.Execute(null); + } + } + private void GearButton_Click(object sender, RoutedEventArgs e) { if (sender is Button { ContextMenu: { } menu } button) diff --git a/Models/MicStatus.cs b/Models/MicStatus.cs new file mode 100644 index 0000000..bac3269 --- /dev/null +++ b/Models/MicStatus.cs @@ -0,0 +1,13 @@ +namespace ytLive.Models; + +/// Footer mic indicator state (the dot beside the MIC button): +/// = mic capture is running; = a +/// mic was requested but the connection failed (device in use, unplugged, or +/// unavailable); = no capture is active (e.g. no +/// mic device present). +public enum MicStatus +{ + NotConnected, + Connected, + Problem +} diff --git a/Models/index.md b/Models/index.md index 7f8717a..5d74324 100644 --- a/Models/index.md +++ b/Models/index.md @@ -13,6 +13,7 @@ Plain data types. No logic beyond what a property can carry. See | `WebcamSceneConfig.cs` | Per-scene webcam placement (subclass of `SceneElement`): geometry + `IsVisible` + border (`BorderColor`/`BorderOpacity`/`BorderWidth`/`BorderAnimation`) + `VideoImageSource`; `WebcamId` links to `Webcam` | | `QualityOption.cs` | Resolution tier record `(Display, Fps, Bitrate, Width, Height)`; `Label` formats as `1080p60 (8Mbps)` — drives the bottom-bar dropdown and the output-rect math | | `StreamConfig.cs` | `StreamConfig` (note: `TargetBitrate`/`Resolution` defaults are stale — not yet wired to the dropdown), `StreamHealth` (bitrate/FPS/dropped/duration), `StreamStatus` enum | +| `MicStatus.cs` | Footer mic indicator: `NotConnected` / `Connected` / `Problem` — the state behind the MIC button's status dot (red = no device/capture off, green = capturing, yellow = a requested mic failed: in use/unplugged) | | `YouTube.cs` | `YouTubeChannel` and `ChatMessage` (chat feed) | | `Socials.cs` | **Social bar** (global layer, never a Source): `SocialService` enum (YouTube/Twitch/X/Instagram/TikTok/Facebook/Discord/Kick/Threads/Bluesky/GitHub/LinkedIn/Pinterest/Snapchat/Reddit/WhatsApp/Telegram/Link/Website/**Fediverse**), `SocialEntry` (Service/Handle/ProfileUrl/`FediverseSoftware`), `SocialsConfig` (Entries + `BarPosition` Top/Bottom + `BarEnabled` on/off — `BarJustify` dropped, column back-compat), `SocialServiceIcons` (canonical URL builder + `DetectService` (URL domain / fediverse `@user@domain` → **Fediverse** / bare→Website) + bundled SVG logo path data per service (`LogoDataFor`) + `LogoDataForFediverse(software)` mapping nodeinfo software names (mastodon/peertube/pixelfed/misskey/lemmy/pleroma/firefish) to logos with a generic fediverse glyph fallback (`FediverseIconData`) + `LockedIconData`/`DoNotIconData` — initials/colors gone) | diff --git a/Services/Audio/AudioMixer.cs b/Services/Audio/AudioMixer.cs index 61fc0d7..d650164 100644 --- a/Services/Audio/AudioMixer.cs +++ b/Services/Audio/AudioMixer.cs @@ -4,15 +4,18 @@ namespace ytLive.Services.Audio; /// /// Owns the mic + desktop/game capture sources (TASK 4 ship step 4) and drives -/// the live mic meter. Runs only while live: started when go-live succeeds, -/// stopped on end-stream. Mic samples are level-metered and forwarded; loopback -/// samples are currently dropped (consumed by the encoder mix in a later step). +/// the footer meters. Capture runs for the app's lifetime (started once at +/// startup, stopped on shutdown) so both bars stay live in preview: mic samples +/// are level-metered and forwarded, loopback samples feed the game bar's meter. +/// The mixer surfaces mic connection state (Connected/Failed) for the status +/// dot and can restart the mic source mid-session when a device is re-picked. /// public sealed class AudioMixer : IDisposable { private readonly IAudioSource _mic; private readonly IAudioSource _loopback; private readonly AudioLevelMeter _meter; + private readonly AudioLevelMeter _loopbackMeter; private readonly Action? _log; private bool _started; @@ -21,8 +24,10 @@ public sealed class AudioMixer : IDisposable _mic = mic; _loopback = loopback; _meter = new AudioLevelMeter(); + _loopbackMeter = new AudioLevelMeter(); _log = log; + _mic.Started += OnMicStarted; _mic.SampleReady += OnMicSample; _loopback.SampleReady += OnLoopbackSample; _mic.Failed += OnMicFailed; @@ -32,9 +37,21 @@ public sealed class AudioMixer : IDisposable /// Current smoothed mic level (0..1). public float MicLevel => _meter.Level; + /// Current smoothed desktop/game level (0..1). + public float LoopbackLevel => _loopbackMeter.Level; + /// Raised whenever the smoothed mic level changes. public event Action? MicLevelChanged; + /// Raised whenever the smoothed desktop/game level changes. + public event Action? LoopbackLevelChanged; + + /// Raised when the mic capture comes up (the status dot goes green). + public event Action? MicConnected; + + /// Raised when the mic capture fails or dies (the status dot goes yellow). + public event Action? MicFailed; + public void Start() { if (_started) @@ -46,6 +63,17 @@ public sealed class AudioMixer : IDisposable _mic.Start(); } + /// Swaps the mic source without touching loopback — used when the + /// creator picks a different device mid-session. The level resets and the + /// new source raises or . + public void RestartMic() + { + _mic.Stop(); + _meter.Reset(); + MicLevelChanged?.Invoke(0); + _mic.Start(); + } + public void Stop() { if (!_started) @@ -55,12 +83,15 @@ public sealed class AudioMixer : IDisposable _mic.Stop(); _loopback.Stop(); _meter.Reset(); + _loopbackMeter.Reset(); MicLevelChanged?.Invoke(0); + LoopbackLevelChanged?.Invoke(0); } public void Dispose() { Stop(); + _mic.Started -= OnMicStarted; _mic.SampleReady -= OnMicSample; _loopback.SampleReady -= OnLoopbackSample; _mic.Failed -= OnMicFailed; @@ -69,20 +100,30 @@ public sealed class AudioMixer : IDisposable _loopback.Dispose(); } + private void OnMicStarted() + { + MicConnected?.Invoke(); + } + private void OnMicSample(AudioSample sample) { - MicLevelChanged?.Invoke(_meter.Push(sample)); + // Push unconditionally: the ?. on the event would otherwise skip the + // argument (and the meter update) when nothing is subscribed yet. + var level = _meter.Push(sample); + MicLevelChanged?.Invoke(level); } private void OnLoopbackSample(AudioSample sample) { - // Desktop/game audio: captured for the future encoder mix; no UI yet. + var level = _loopbackMeter.Push(sample); + LoopbackLevelChanged?.Invoke(level); } private void OnMicFailed(Exception ex) { _log?.Invoke($"Mic capture failed: {ex.Message}"); MicLevelChanged?.Invoke(0); + MicFailed?.Invoke(ex); } private void OnLoopbackFailed(Exception ex) diff --git a/Services/Audio/IAudioSource.cs b/Services/Audio/IAudioSource.cs index c2257e4..71cdc2e 100644 --- a/Services/Audio/IAudioSource.cs +++ b/Services/Audio/IAudioSource.cs @@ -2,9 +2,10 @@ namespace ytLive.Services.Audio; /// /// A live audio capture source (TASK 4 ship step 4 seam): produces PCM float -/// chunks, runs only while live. The default implementations wrap NAudio's -/// WASAPI capture (mic) / loopback (desktop/game); the mixer and the tests -/// consume this interface, never NAudio directly. +/// chunks. The default implementations wrap NAudio's WASAPI capture (mic) / +/// loopback (desktop/game); the mixer and the tests consume this interface, +/// never NAudio directly. Capture now runs for the app's lifetime so the +/// footer meters stay live in preview — the mixer owns start/stop. /// public interface IAudioSource : IDisposable { @@ -14,6 +15,10 @@ public interface IAudioSource : IDisposable /// Stops capturing; a later Start begins a fresh session. void Stop(); + /// Raises once capture is live (right after recording starts). + /// Never raised when Start fails — fires instead. + event Action? Started; + /// Raises each captured chunk (interleaved PCM float, -1..1). event Action? SampleReady; diff --git a/Services/Audio/WasapiLoopbackAudioSource.cs b/Services/Audio/WasapiLoopbackAudioSource.cs index 620ef5c..b15fb58 100644 --- a/Services/Audio/WasapiLoopbackAudioSource.cs +++ b/Services/Audio/WasapiLoopbackAudioSource.cs @@ -4,12 +4,14 @@ namespace ytLive.Services.Audio; /// /// Captures the user's desktop/game audio (TASK 4 ship step 4) via WASAPI -/// loopback on the default render device. Starts/stops with go-live only. +/// loopback on the default render device. Runs for the app's lifetime so the +/// game audio bar stays live in preview. /// public sealed class WasapiLoopbackAudioSource : IAudioSource { private WasapiLoopbackCapture? _capture; + public event Action? Started; public event Action? SampleReady; public event Action? Failed; @@ -24,6 +26,7 @@ public sealed class WasapiLoopbackAudioSource : IAudioSource _capture.DataAvailable += OnDataAvailable; _capture.RecordingStopped += OnRecordingStopped; _capture.StartRecording(); + Started?.Invoke(); } catch (Exception ex) { diff --git a/Services/Audio/WasapiMicAudioSource.cs b/Services/Audio/WasapiMicAudioSource.cs index 313b14a..0d3df73 100644 --- a/Services/Audio/WasapiMicAudioSource.cs +++ b/Services/Audio/WasapiMicAudioSource.cs @@ -14,12 +14,14 @@ public sealed class WasapiMicAudioSource : IAudioSource private WasapiCapture? _capture; /// Returns the current mic DisplayName; read - /// at each Start so a device picked mid-session takes effect next go-live. + /// at each Start so a device picked mid-session takes effect immediately + /// (the mixer restarts the mic on pick). public WasapiMicAudioSource(Func micNameProvider) { _micNameProvider = micNameProvider; } + public event Action? Started; public event Action? SampleReady; public event Action? Failed; @@ -35,6 +37,7 @@ public sealed class WasapiMicAudioSource : IAudioSource _capture.DataAvailable += OnDataAvailable; _capture.RecordingStopped += OnRecordingStopped; _capture.StartRecording(); + Started?.Invoke(); } catch (Exception ex) { diff --git a/Services/GameAudioDetector.cs b/Services/GameAudioDetector.cs new file mode 100644 index 0000000..e6620c3 --- /dev/null +++ b/Services/GameAudioDetector.cs @@ -0,0 +1,46 @@ +namespace ytLive.Services; + +/// +/// Default : samples a full-screen monitor +/// provider (the existing IFullScreenDetector) and the live loopback +/// level and advances a . The pure transitions +/// live in GameAudioHysteresis (unit-tested); this wrapper owns the providers +/// and the raised-changed event. WPF-free — the VM owns the poll timer. +/// +public sealed class GameAudioDetector : IGameAudioDetector +{ + private const float SoundFloor = 0.005f; + + private readonly Func _foregroundFullScreenMonitorProvider; + private readonly Func _loopbackLevelProvider; + private readonly Func _now; + private readonly GameAudioHysteresis _hysteresis = new(); + + /// Returns the monitor a + /// full-screen foreground window covers, or null when windowed/none. + /// Current smoothed desktop/game level (0..1). + /// Clock for the hysteresis windows; injectable for tests. + public GameAudioDetector( + Func foregroundFullScreenMonitorProvider, + Func loopbackLevelProvider, + Func? now = null) + { + _foregroundFullScreenMonitorProvider = foregroundFullScreenMonitorProvider; + _loopbackLevelProvider = loopbackLevelProvider; + _now = now ?? (() => DateTime.Now); + } + + public bool IsGameAudioActive => _hysteresis.IsActive; + + public event Action? IsGameAudioActiveChanged; + + public void Poll() + { + var before = _hysteresis.IsActive; + var isFullScreen = _foregroundFullScreenMonitorProvider() != null; + var hasSound = _loopbackLevelProvider() > SoundFloor; + _hysteresis.Update(isFullScreen, hasSound, _now()); + if (before != _hysteresis.IsActive) + IsGameAudioActiveChanged?.Invoke(_hysteresis.IsActive); + } +} diff --git a/Services/GameAudioHysteresis.cs b/Services/GameAudioHysteresis.cs new file mode 100644 index 0000000..c39ae7f --- /dev/null +++ b/Services/GameAudioHysteresis.cs @@ -0,0 +1,57 @@ +namespace ytLive.Services; + +/// +/// Pure show/hide state machine for the game audio bar (TASK 4 game audio). +/// SHOW: a full-screen foreground app has been producing desktop audio for at +/// least — "a working game with sound". HIDE: the +/// app leaves fullscreen/foreground for — the +/// game is no longer up in the preview. Silence NEVER hides an active bar; it +/// only matters for the initial show. No timers; is fed by +/// the caller with a clock. +/// +public sealed class GameAudioHysteresis +{ + private readonly TimeSpan _showAfterSound = TimeSpan.FromMilliseconds(500); + private readonly TimeSpan _hideAfterWindowed = TimeSpan.FromSeconds(1); + + private DateTime? _soundSince; + private DateTime? _windowedSince; + + public bool IsActive { get; private set; } + + public void Update(bool isFullScreen, bool hasSound, DateTime now) + { + if (isFullScreen) + { + _windowedSince = null; + if (IsActive) + return; + + if (!hasSound) + { + _soundSince = null; + return; + } + + _soundSince ??= now; + if (now - _soundSince >= _showAfterSound) + IsActive = true; + } + else + { + _soundSince = null; + if (!IsActive) + { + _windowedSince = null; + return; + } + + _windowedSince ??= now; + if (now - _windowedSince >= _hideAfterWindowed) + { + IsActive = false; + _windowedSince = null; + } + } + } +} diff --git a/Services/IGameAudioDetector.cs b/Services/IGameAudioDetector.cs new file mode 100644 index 0000000..c33af26 --- /dev/null +++ b/Services/IGameAudioDetector.cs @@ -0,0 +1,20 @@ +namespace ytLive.Services; + +/// +/// Detects "a working game with sound is up in the preview" (TASK 4 game audio +/// bar): becomes active once a full-screen foreground app is producing desktop +/// audio, and stays active as long as that full-screen app remains — silence +/// never hides the bar, only the game leaving fullscreen/foreground does. Seam +/// so the VM and tests never touch Win32 or audio interop directly. +/// +public interface IGameAudioDetector +{ + /// True while the game audio bar should be visible. + bool IsGameAudioActive { get; } + + /// Raised when the bar should appear or disappear. + event Action? IsGameAudioActiveChanged; + + /// Samples the injected providers and advances the state machine. + void Poll(); +} diff --git a/Services/index.md b/Services/index.md index 830c58d..4e84ecb 100644 --- a/Services/index.md +++ b/Services/index.md @@ -44,13 +44,16 @@ External-facing logic: YouTube API, persistence. See | `Encoder/FfmpegEncoderPicker.cs` | Pure H.264 encoder picker from `-encoders` output: NVENC → QSV → AMF → OpenH264; **never returns libx264** (GPL) | | `Encoder/FramePump.cs` | **TASK 4 ship step 5**: the live frame producer — while live it snapshots the active scene each tick, resolves every element to its latest frame (`Func` resolver), composites it into the tier's output frame, and paces frames into the encoder at the tier's FPS. All collaborators constructor-injected seams; free of WPF and the capture managers. `StartAsync` never throws (failures log + `Failed`); no RTMP URL = encoder skipped; `StopAsync` stops the encoder before awaiting the loop (backpressure deadlock); `ProcessFailed` self-stops. Since the bar bug-fix branch it takes an optional `socialBar: Func<(VideoFrame? Frame, SocialBarPosition Position)>?` seam — a pre-rendered bar strip composited last (above the flash) at the top edge or `SourceRectHeight − bar height`. See `ai.md` "Live frame pipeline" | | `Encoder/FfmpegLocator.cs` | The default locator: PATH probe wins, then the cache, then pull+extract (`ffmpeg.exe` **+ the `libav*.dll` family** via a staging dir, failures via `AppLog`). Pinned to BtbN `autobuild-2026-08-09-13-03` **lgpl-shared** (dynamic linking = LGPL compliance without static-relink §6 material; no GPL libx264; NVENC/QSV/AMF + libopenh264 + native AAC — see `ai.md` → Licensing). Search dirs / tools dir / downloader constructor-injected for hermetic tests. Wired into the encoder since ship step 5 | -| `Audio/IAudioSource.cs` | **TASK 4 ship step 4 seam**: live capture source — `Start`/`Stop`/`SampleReady(AudioSample)`/`Failed(Exception)`, `IDisposable`; runs only while live. The app consumes this seam; tests inject hermetic fakes | +| `Audio/IAudioSource.cs` | **TASK 4 ship step 4 seam**: live capture source — `Start`/`Stop`/`Started`/`SampleReady(AudioSample)`/`Failed(Exception)`, `IDisposable`. Capture now runs for the app's lifetime (started at startup) so the footer meters preview live. The app consumes this seam; tests inject hermetic fakes | | `Audio/AudioSample.cs` | One captured chunk: interleaved PCM float (-1..1) + sample rate + channels | -| `Audio/WasapiLoopbackAudioSource.cs` | Desktop/game capture: NAudio `WasapiLoopbackCapture` on the default render device — automatic at unity, zero UI | -| `Audio/WasapiMicAudioSource.cs` | Mic capture: NAudio `WasapiCapture` with the device resolved by `FriendlyName` matching `MicSourceName` (DisplayName only), default capture endpoint fallback; name provider `Func` re-read at each `Start` so a mic picked mid-session takes effect next go-live | -| `Audio/AudioMixer.cs` | Owns both sources; `Start`/`Stop` follow go-live (`MainViewModel.BeginGoLive`/`StopStream`). Mic samples → `AudioLevelMeter` → `MicLevelChanged`; loopback samples currently dropped (the future encoder AAC mix consumes them). Failures log via `AppLog`; mic failure zeroes the meter, loopback failure doesn't kill the mic | -| `Audio/AudioLevelMeter.cs` | Pure smoothed RMS level (0..1): `Push(AudioSample)` + `Reset` — the unit-tested math behind `AudioLevel` | +| `Audio/WasapiLoopbackAudioSource.cs` | Desktop/game capture: NAudio `WasapiLoopbackCapture` on the default render device — automatic at unity; the game bar's meter consumes it | +| `Audio/WasapiMicAudioSource.cs` | Mic capture: NAudio `WasapiCapture` with the device resolved by `FriendlyName` matching `MicSourceName` (DisplayName only), default capture endpoint fallback; name provider `Func` re-read at each `Start` so a device picked mid-session takes effect immediately (the mixer restarts the mic on pick) | +| `Audio/AudioMixer.cs` | Owns both sources; capture starts once at startup (`MainViewModel.StartMicCaptureAsync`) and stops on `Shutdown` — NOT go-live (preview monitoring). Mic samples → `AudioLevelMeter` → `MicLevelChanged`; loopback samples → the game bar's meter via `LoopbackLevelChanged`. Surfaces mic connection state: `MicConnected`/`MicFailed` (drives the status dot) + `RestartMic()` (device swap mid-session, keeps loopback). Failures log via `AppLog`; mic failure zeroes the meter, loopback failure doesn't kill the mic. Note: the meter `Push` is unconditional (the `?.` on the event would otherwise skip the argument when nothing is subscribed) | +| `Audio/AudioLevelMeter.cs` | Pure smoothed RMS level (0..1): `Push(AudioSample)` + `Reset` — the unit-tested math behind `AudioLevel` and `GameAudioLevel` | | `Audio/WaveToFloat.cs` | Pure WASAPI buffer → float conversion: IEEE float 32-bit direct, PCM 16-bit normalized, `WaveFormatExtensible` IEEE-float subformat GUID, trailing partial samples ignored | +| `IGameAudioDetector.cs` | **TASK 4 game audio bar seam**: `IsGameAudioActive` + `IsGameAudioActiveChanged` + `Poll()` — detects "a working game with sound is up in the preview" | +| `GameAudioHysteresis.cs` | Pure show/hide state machine for the game bar: SHOW = full-screen app holds sound ~500ms; HIDE = the app leaves fullscreen ~1s. **Silence never hides an active bar** — only the game leaving the preview does (creator's rule). No timers; `Update(isFullScreen, hasSound, now)` | +| `GameAudioDetector.cs` | Default `IGameAudioDetector`: composes a full-screen monitor provider (`IFullScreenDetector.GetForegroundFullScreenMonitorIndex`) + the live loopback level (floor 0.5%) into a `GameAudioHysteresis`. WPF-free; the VM owns the 250ms poll timer | Related: constructed in [`ViewModels/MainViewModel.cs`](../ViewModels/MainViewModel.cs) (no DI container yet). Models in [`Models/index.md`](../Models/index.md). diff --git a/TASKS.md b/TASKS.md index 32c3beb..f245558 100644 --- a/TASKS.md +++ b/TASKS.md @@ -106,7 +106,7 @@ Lifecycle: `created → ready → [testing] → live → complete` (transitional 9. ✅ **"Add Webcam" always opens the camera picker** — deleting one scene's webcam then re-adding used to resurrect the old camera when another scene still used it — `SwapWebcamIdentityAsync` now swaps the app-wide identity if a different camera is chosen, same path as "Change Webcam…" 10. ✅ **Webcam resource validation + first-frame proof** — `MediaCaptureFrameSource` validates post-init (VideoDeviceId match, stream properties ≥1, `reader.StartAsync()` status read + throws on non-Success); subscribes `capture.Failed` + `CameraStreamStateChanged` → `SourceFailed` event on the seam; fallback ladder (VideoPreview → VideoRecord). `CameraManager.AcquireAsync` requires first-frame proof (4s timeout): returns true only after a real frame arrives — silent empty box impossible. `MainViewModel` subscribes `CameraFailed` → red `WebcamError` chip in preview + MessageBox names suspect apps (`CameraConflictProbe`). 19041 SDK projection gaps: `Exclusive`/`DeviceLost` not projected; `CameraStreamState.Failed` compared by `(int)2`. 81 tests passing 11. ✅ **Scenes/sources UI** — add/reorder/rename, image + background overlays with move/resize/opacity/reuse -12. ✅ **Audio UX shipped (UI)** — the bottom-bar footer is now two lines (dropped/duration moved under bitrate/fps), with the mic's sound meter + mute button + volume slider grouped CENTERED on the footer's top line, beneath the preview panel (meter: 288px, muted slate track with ruler graduations + muted yellow/red zone tints, green→yellow→red fill; mute = speaker icon → red do-not-symbol when muted, and the slider and speaker stay in sync (volume 0 ⇔ muted — sliding off flips the speaker to muted, sliding up from 0 clears it); mic volume defaults to 80%, muting zeroes the meter and restores the prior volume on unmute (which flashes the meter to the restored position ~300ms before it returns to the live level); the meter is a READ-ONLY realtime level display (fill = live level × volume — volume is a gain on ambient noise; while the slider is dragged the bar previews the slider position and bounces back to the live level on release, which is 0 with no input — clicking the meter does nothing), clicking the MIC label opens a microphone picker whose chosen source shows left-justified inside the meter bar (fill at 75% opacity so the name + ruler markings show through); slim dimensional slider — gradient track/fill, gloss-sphere thumb; the old flat pink 18px-filled one is gone), everything else on line 2 (bitrate/fps/dropped/duration/health left, quality + gear right) — the creator's only audio control, desktop/game audio is automatic (KISS rule) +12. ✅ **Audio UX shipped (UI)** — the bottom-bar footer is now two lines (dropped/duration moved under bitrate/fps), with the mic's sound meter + mute button + volume slider grouped CENTERED on the footer's top line, beneath the preview panel (meter: 288px, muted slate track with ruler graduations + muted yellow/red zone tints, green→yellow→red fill; mute = speaker icon → red do-not-symbol when muted, and the slider and speaker stay in sync (volume 0 ⇔ muted — sliding off flips the speaker to muted, sliding up from 0 clears it); mic volume defaults to 80%, muting zeroes the meter and restores the prior volume on unmute (which flashes the meter to the restored position ~300ms before it returns to the live level); the meter is a READ-ONLY realtime level display (fill = live level × volume — volume is a gain on ambient noise; while the slider is dragged the bar previews the slider position and bounces back to the live level on release, which is 0 with no input — clicking the meter does nothing), clicking the MIC label opens a microphone picker whose chosen source shows left-justified inside the meter bar (fill at 75% opacity so the name + ruler markings show through); slim dimensional slider — gradient track/fill, gloss-sphere thumb; the old flat pink 18px-filled one is gone), everything else on line 2 (bitrate/fps/dropped/duration/health left, quality + gear right) — the creator's only audio control, desktop/game audio is automatic (KISS rule). **Post-test follow-up (2026-08-13, game audio bar branch):** the footer is now THREE lines — a **game audio bar** (desktop/game, a mirror of the mic bar: meter + mute + volume) sits centered beneath the mic bar and appears only while a **full-screen game is producing sound** in the preview (`IGameAudioDetector` seam + default `GameAudioDetector` polling `IFullScreenDetector` + the live loopback level, floor 0.5%, into a pure `GameAudioHysteresis`: SHOW after ~500ms of fullscreen+sound, HIDE ~1s after leaving fullscreen, **silence never hides an active bar**; the VM polls it on a 250ms `DispatcherTimer`); **capture now runs for the app's lifetime** (mixer started once at startup via `StartMicCaptureAsync`, disposed in `Shutdown` — not go-live) so both meters preview live; the **MIC label became a button with a status dot** (`Models/MicStatus`: green = connected via the source `Started` event, yellow = mic problem, red = no device); picking a mic takes effect immediately (`AudioMixer.RestartMic`, loopback keeps running); fixed a latent `?.Invoke(meter.Push(...))` short-circuit that skipped the meter update when nothing was subscribed. 167 tests passing, 0 warnings 13. ✅ The connected YouTube account's avatar/name shows in the top bar next to Start Stream (`SyncConnectedAccount`); the scenes list is content-height now (no dead space before SOURCES) 14. ✅ **Social bar v2 (six-slot dialog, sign-in gate, real logos)** — global bar layer (never a Source, no sources-list row), content-sized, centered, GREEN glow when ON, top/bottom snap-drag (default BOTTOM, persisted `SocialBarPosition`; drag clamps to 0/1040, tie→bottom). Footer Social button gets a state dot (green=ON). Dialog "Social Media Site Promotion" (`SocialsDialog` + `ViewModels/SocialsDialogViewModel`, WPF-free + injected `ISocialValidator`/sign-in/sign-out fakes): ON/OFF bar switch (`SocialsConfig.BarEnabled`, schema v8), 6 fixed slots — row 1 always YouTube (signed-in → channel handle; signed-out → sign-in gate → OAuth; delete → confirm sign-out, mirrors `StopStream`), row 2 free, rows 3-6 lock icons on freemium (Premium seam: all six). Validation: `DetectService` (URL domain / fediverse `@user@domain` / bare→Website) → async `ISocialValidator` on confirm/Save; valid snaps to text + real service logo (bundled SVG path data via `LogoDataFor`, Simple Icons CC0 — initials badges gone); invalid → red do-not, stays editable, Save blocked. LCR justify dropped (`BarJustify` unread), per-scene toggle dropped (`Scene.HasSocialBar` back-compat). **Post-test fixes (2026-08-12):** footer label "Socials" (not "Social"); fediverse `@user@domain` validates — the full handle is the identity end-to-end (`DetectService`/`CanonicalUrlFor`/`HttpSocialValidator` build `https://domain/@user`, no domain loss); **Cancel is a hard stop** — `ISocialValidator.LookupAsync` takes a `CancellationToken`, dialog VM owns a CTS, Cancel/X/Save abort in-flight lookups (HTTP request killed, canceled continuations never touch slot state), and `ConfirmEdit` skips re-submitting identical text (LostFocus on dismiss never re-fires a lookup). `HttpSocialValidator` now has real tests (fake `HttpMessageHandler`). 105 tests passing. **Post-test fixes (2026-08-12, round 2):** fediverse `@user@domain` no longer shows a generic chain — it resolves to the instance's actual software via nodeinfo (`/.well-known/nodeinfo` → `software.name`; `SocialService.Fediverse` enum member + `SocialEntry.FediverseSoftware` persisted in a new `SocialEntry.Software` column, schema migration by column-presence) and renders that software's bundled logo (`LogoDataForFediverse`: mastodon/peertube/pixelfed/misskey/lemmy/pleroma/firefish, generic fediverse honeycomb fallback). Dialog row-2 edit/trash icons were too dark — `IconButton` style gains `Foreground=#d0d0d0`; trash overrides `#e94560` (app red). 112 tests passing. **Post-test fixes (2026-08-12, round 3):** a fediverse handle whose identity domain is itself a redirect (e.g. YunoHost default-app subdomains — `@user@llamachile.tube` where the mastodon instance lives at `mastodon.llamachile.tube`) now still resolves its software: nodeinfo on the identity domain is SSO-blocked, so `HttpSocialValidator` follows the bare root `https://domain/` 302 to the real instance host and re-runs the nodeinfo lookup there. 15. ☐ **Window capture** — absorbed into the Screen picker (no separate source type); dedicated window-as-source work is pending @@ -423,8 +423,10 @@ lifecycle/forwarding/failure against fakes, meter RMS/smoothing/reset, `WaveToFl extensible/truncation) — **139 passing**. **Deferred (later ship steps):** wiring the desktop-capture samples into the encoder's AAC mix (replaces -the `-f lavfi -i anullsrc` placeholder; the encoder construction itself shipped in ship step 5), WASAPI -capture while not live, and any audio UI beyond the existing mic controls. +the `-f lavfi -i anullsrc` placeholder; the encoder construction itself shipped in ship step 5). +*(The "capture while not live" + "audio UI beyond the mic controls" deferrals were SHIPPED on the +2026-08-13 game audio bar branch — capture is now always-on for preview and the game bar is the second +audio UI. The mixer's short-circuit meter fix + `Started`/`RestartMic` seams live in the same branch.)* #### Ship step 5 — Frame-pipeline wiring (the encoder gets a frame source) diff --git a/ViewModels/MainViewModel.cs b/ViewModels/MainViewModel.cs index f945fcb..a3f5f9d 100644 --- a/ViewModels/MainViewModel.cs +++ b/ViewModels/MainViewModel.cs @@ -27,6 +27,8 @@ public class MainViewModel : ViewModelBase private readonly YouTubeChatService _youtubeChat; private readonly DispatcherTimer _liveTimer; private readonly DispatcherTimer _volumeFlashTimer; + private readonly DispatcherTimer _gameVolumeFlashTimer; + private readonly DispatcherTimer _gameAudioTimer; private Scene? _activeScene; private SceneElement? _selectedElement; @@ -41,6 +43,14 @@ public class MainViewModel : ViewModelBase private bool _micMuted; private double? _volumeBeforeMute; private string? _micSourceName; + private MicStatus _micStatus = MicStatus.NotConnected; + private double _gameAudioLevel; + private bool _gameVolumeAdjusting; + private bool _gameVolumeFlash; + private double _gameVolume = 1.0; + private bool _gameMuted; + private double? _gameVolumeBeforeMute; + private bool _isGameAudioBarVisible; private StreamStatus _streamStatus = StreamStatus.Offline; private StreamHealth _currentHealth = new(); private string _streamTitle = string.Empty; @@ -84,9 +94,11 @@ public class MainViewModel : ViewModelBase private readonly IMicrophoneEnumerator _microphoneEnumerator; // Live audio capture (TASK 4 ship step 4): desktop/game via WASAPI loopback, - // mic via WASAPI capture, both owned by the mixer and running only while - // live. Mic level feeds AudioLevel (the meter); loopback is for the future - // encoder mix. Private by design — no UI beyond the existing mic controls. + // mic via WASAPI capture, both owned by the mixer. Capture runs for the app's + // lifetime (started at startup, stopped on shutdown) so the footer meters + // stay live in preview. Mic level feeds AudioLevel (the meter); loopback + // feeds the game audio bar's meter. Private by design — the mixer surfaces + // the levels + mic connection state to the UI. private readonly AudioMixer _audioMixer; // Live frame pipeline (TASK 4 ship step 5): composites the active scene at the @@ -101,6 +113,7 @@ public class MainViewModel : ViewModelBase private readonly IFullScreenDetector _fullScreenDetector; private readonly ScreenCaptureManager _screenCaptureManager; private readonly ScreenCaptureSourceFactory _screenCaptureFactory; + private readonly IGameAudioDetector _gameAudioDetector; private int? _lastForegroundFullScreenMonitor; private CancellationTokenSource? _deactivateCts; private ImageSource? _backdropImage; @@ -272,8 +285,9 @@ public class MainViewModel : ViewModelBase private set => SetProperty(ref _accountDisplayName, value); } - // ─── Audio (KISS: desktop/game audio is automatic — zero UI. The creator's - // ─── only audio control is the mic: meter + volume + mute.) ─── + // ─── Audio: the mic bar (meter + volume + mute + status dot) is always + // ─── visible; the game bar (meter + volume + mute) appears only while a + // ─── full-screen game is producing sound. Both meters preview live. ─── /// Live mic input level (0..1) — fed by the audio mixer once capture /// lands; 0 with no input. Read by the meter, scaled by MicVolume. @@ -409,7 +423,170 @@ public class MainViewModel : ViewModelBase Owner = System.Windows.Application.Current?.MainWindow }; if (dialog.ShowDialog() == true && dialog.PickedDevice != null) + { MicSourceName = dialog.PickedDevice.DisplayName; + _audioMixer.RestartMic(); // swap the live device immediately + } + } + + // ─── Mic status dot (the MIC button) ─── + + private static readonly SolidColorBrush MicProblemBrush = CreateBrush("#f1c40f"); + + /// Mic connection state, driven by the mixer's MicConnected/MicFailed + /// events and the startup device check (see StartMicCaptureAsync). + public MicStatus MicStatus + { + get => _micStatus; + private set + { + if (SetProperty(ref _micStatus, value)) + { + OnPropertyChanged(nameof(MicStatusBrush)); + OnPropertyChanged(nameof(MicStatusToolTip)); + } + } + } + + /// Status dot: green = connected, yellow = problem with the requested + /// connection, red = not connected (no device / not started). + public SolidColorBrush MicStatusBrush => MicStatus switch + { + MicStatus.Connected => BarOnBrush, + MicStatus.Problem => MicProblemBrush, + _ => BarOffBrush, + }; + + public string MicStatusToolTip => MicStatus switch + { + MicStatus.Connected => "Mic connected — click to change", + MicStatus.Problem => "Mic problem — the requested microphone is unavailable (in use or unplugged). Click to change", + _ => "No mic connected — click to choose a microphone", + }; + + // ─── Game audio bar (desktop/game): visible only while a full-screen game + // ─── is producing sound (IGameAudioDetector). Meter + mute + volume mirror + // ─── the mic bar. ─── + + /// True while the game audio bar should be shown (driven by + /// IGameAudioDetector via the poll timer). + public bool IsGameAudioBarVisible + { + get => _isGameAudioBarVisible; + private set => SetProperty(ref _isGameAudioBarVisible, value); + } + + /// Live desktop/game input level (0..1), fed by the mixer's loopback + /// capture. Read by the game meter, scaled by GameAudioVolume. + public double GameAudioLevel + { + get => _gameAudioLevel; + set + { + if (SetProperty(ref _gameAudioLevel, Math.Clamp(value, 0, 1))) + { + OnPropertyChanged(nameof(GameMeterFillWidth)); + OnPropertyChanged(nameof(GameMeterBrush)); + } + } + } + + /// Displayed game meter level: 0 while muted; the volume position + /// while the slider is dragged (or briefly after an unmute flash); otherwise + /// the realtime live level scaled by volume. + private double GameMeterLevel => GameMuted ? 0 : _gameVolumeAdjusting || _gameVolumeFlash ? GameAudioVolume : Math.Min(1, GameAudioLevel * GameAudioVolume); + + public double GameMeterFillWidth => GameMeterLevel * 288; + + public string GameMeterBrush => GameMeterLevel switch + { + < 0.6 => "#22c55e", + < 0.8 => "#eab308", + _ => "#ef4444", + }; + + public void SetGameVolumeAdjusting(bool adjusting) + { + if (adjusting) + CancelGameVolumeFlash(); + if (SetProperty(ref _gameVolumeAdjusting, adjusting)) + { + OnPropertyChanged(nameof(GameMeterFillWidth)); + OnPropertyChanged(nameof(GameMeterBrush)); + } + } + + private void BeginGameVolumeFlash() + { + _gameVolumeFlash = true; + OnPropertyChanged(nameof(GameMeterFillWidth)); + OnPropertyChanged(nameof(GameMeterBrush)); + _gameVolumeFlashTimer.Stop(); + _gameVolumeFlashTimer.Start(); + } + + private void EndGameVolumeFlash() + { + _gameVolumeFlashTimer.Stop(); + if (_gameVolumeFlash) + { + _gameVolumeFlash = false; + OnPropertyChanged(nameof(GameMeterFillWidth)); + OnPropertyChanged(nameof(GameMeterBrush)); + } + } + + private void CancelGameVolumeFlash() + { + _gameVolumeFlashTimer.Stop(); + _gameVolumeFlash = false; + } + + /// Game audio gain (0..1, unity default). Running to 0 mutes; the + /// prior level is remembered so the speaker button can restore it. + public double GameAudioVolume + { + get => _gameVolume; + set + { + var clamped = Math.Clamp(value, 0, 1); + if (clamped == 0 && !_gameMuted) + _gameVolumeBeforeMute ??= _gameVolume; + if (SetProperty(ref _gameVolume, clamped)) + { + var muted = clamped == 0; + if (_gameMuted != muted) + { + _gameMuted = muted; + OnPropertyChanged(nameof(GameMuted)); + OnPropertyChanged(nameof(GameMuteText)); + } + if (!muted) + _gameVolumeBeforeMute = null; + OnPropertyChanged(nameof(GameMeterFillWidth)); + OnPropertyChanged(nameof(GameMeterBrush)); + } + } + } + + /// Read-only: true whenever the volume is 0 (the slider and the + /// speaker can never disagree). + public bool GameMuted => _gameMuted; + + public string GameMuteText => GameMuted ? "Unmute" : "Mute"; + + private void ToggleGameMute() + { + if (GameMuted) + { + GameAudioVolume = _gameVolumeBeforeMute ?? 1.0; + BeginGameVolumeFlash(); + } + else + { + _gameVolumeBeforeMute = GameAudioVolume; + GameAudioVolume = 0; + } } public bool IsOffline => StreamStatus == StreamStatus.Offline; @@ -759,6 +936,7 @@ public class MainViewModel : ViewModelBase public ICommand RefreshCaptureCommand { get; } public ICommand SetBackdropDisplayCommand { get; } public ICommand ToggleMicMuteCommand { get; } + public ICommand ToggleGameMuteCommand { get; } public ICommand OpenMicPickerCommand { get; } public ICommand OpenSocialDialogCommand { get; } public ICommand StartStreamCommand { get; } @@ -795,6 +973,9 @@ public class MainViewModel : ViewModelBase _volumeFlashTimer = new DispatcherTimer { Interval = TimeSpan.FromMilliseconds(300) }; _volumeFlashTimer.Tick += (_, _) => EndVolumeFlash(); + _gameVolumeFlashTimer = new DispatcherTimer { Interval = TimeSpan.FromMilliseconds(300) }; + _gameVolumeFlashTimer.Tick += (_, _) => EndGameVolumeFlash(); + _saveDebounce = new DispatcherTimer { Interval = TimeSpan.FromMilliseconds(1500) }; _saveDebounce.Tick += (_, _) => SaveLayoutNow(); @@ -820,6 +1001,7 @@ public class MainViewModel : ViewModelBase RefreshCaptureCommand = new RelayCommand(_ => RefreshBackdropAutoCapture()); SetBackdropDisplayCommand = new RelayCommand(display => SetBackdropCapture(display as DisplayInfo)); ToggleMicMuteCommand = new RelayCommand(_ => ToggleMicMute()); + ToggleGameMuteCommand = new RelayCommand(_ => ToggleGameMute()); OpenMicPickerCommand = new RelayCommand(_ => PickMicrophone()); OpenSettingsCommand = new RelayCommand(_ => ShowOverlay(nameof(IsSettingsOpen), "App Settings")); OpenBugCommand = new RelayCommand(_ => ShowOverlay(nameof(IsBugOpen), "Report Bug")); @@ -853,10 +1035,22 @@ public class MainViewModel : ViewModelBase new WasapiLoopbackAudioSource(), message => AppLog.Write(message)); _audioMixer.MicLevelChanged += OnMicLevelChanged; + _audioMixer.LoopbackLevelChanged += OnLoopbackLevelChanged; + _audioMixer.MicConnected += OnMicConnected; + _audioMixer.MicFailed += OnMicFailed; + _ = StartMicCaptureAsync(); _fullScreenDetector = new Win32FullScreenDetector(); foreach (var display in _fullScreenDetector.GetDisplays()) Displays.Add(display); + + _gameAudioDetector = new GameAudioDetector( + () => _fullScreenDetector.GetForegroundFullScreenMonitorIndex(), + () => (float)GameAudioLevel); + _gameAudioDetector.IsGameAudioActiveChanged += OnGameAudioActiveChanged; + _gameAudioTimer = new DispatcherTimer { Interval = TimeSpan.FromMilliseconds(250) }; + _gameAudioTimer.Tick += OnGameAudioPollTick; + _gameAudioTimer.Start(); _screenCaptureFactory = new ScreenCaptureSourceFactory( () => new WindowInteropHelper(System.Windows.Application.Current?.MainWindow).Handle); _screenCaptureManager = new ScreenCaptureManager( @@ -1191,6 +1385,8 @@ public class MainViewModel : ViewModelBase { _saveDebounce?.Stop(); SaveLayoutNow(); + _gameAudioTimer.Stop(); + _audioMixer.Dispose(); _framePump.Dispose(); _cameraManager.Dispose(); _screenCaptureManager.Dispose(); @@ -1761,7 +1957,6 @@ public class MainViewModel : ViewModelBase ? "ytLlive" : $"{dialog.StreamTitle} — ytLlive"; StreamStatus = StreamStatus.Streaming; - _audioMixer.Start(); _ = _framePump.StartAsync(); // never throws; failures log + surface via Failed } } @@ -1770,7 +1965,8 @@ public class MainViewModel : ViewModelBase { StreamStatus = StreamStatus.Offline; WindowTitle = "ytLlive"; - _audioMixer.Stop(); + // Audio capture is always-on (preview monitoring); only the frame pump + // and the session stop here. _ = _framePump.StopAsync(); // Graceful end completes the session = signs out (the DPAPI token is // cleared so the next Start Stream requires a fresh sign-in). A crash @@ -1793,6 +1989,70 @@ public class MainViewModel : ViewModelBase AudioLevel = level; } + private void OnLoopbackLevelChanged(float level) + { + var dispatcher = System.Windows.Application.Current?.Dispatcher; + if (dispatcher != null && !dispatcher.CheckAccess()) + dispatcher.BeginInvoke(() => GameAudioLevel = level); + else + GameAudioLevel = level; + } + + private void OnMicConnected() + { + var dispatcher = System.Windows.Application.Current?.Dispatcher; + if (dispatcher != null && !dispatcher.CheckAccess()) + dispatcher.BeginInvoke(() => MicStatus = MicStatus.Connected); + else + MicStatus = MicStatus.Connected; + } + + private void OnMicFailed(Exception ex) + { + // The mixer logs the failure detail; here we only flip the dot to yellow. + var dispatcher = System.Windows.Application.Current?.Dispatcher; + if (dispatcher != null && !dispatcher.CheckAccess()) + dispatcher.BeginInvoke(() => MicStatus = MicStatus.Problem); + else + MicStatus = MicStatus.Problem; + } + + /// Starts capture once at startup: with no mic device present the + /// dot stays red and capture never starts; otherwise the mixer starts and + /// raises MicConnected (green) or MicFailed (yellow). + private async Task StartMicCaptureAsync() + { + try + { + var mics = await _microphoneEnumerator.GetMicrophonesAsync(); + if (mics.Count == 0) + { + MicStatus = MicStatus.NotConnected; + AppLog.Write("Mic: no capture devices found — mic capture not started"); + return; + } + _audioMixer.Start(); + } + catch (Exception ex) + { + AppLog.Write($"Mic: device check failed: {ex.Message}"); + } + } + + private void OnGameAudioActiveChanged(bool active) => IsGameAudioBarVisible = active; + + private void OnGameAudioPollTick(object? sender, EventArgs e) + { + try + { + _gameAudioDetector.Poll(); + } + catch (Exception ex) + { + AppLog.Write($"Game audio detection failed: {ex.Message}"); + } + } + // Scene-element → latest frame, for the live compositor. The map mirrors the // preview: webcam by DeviceId, live captures (the backdrop) by CaptureKey, // images/background by AssetId. A null frame leaves the element transparent. diff --git a/ViewModels/index.md b/ViewModels/index.md index d3c1938..dd36b39 100644 --- a/ViewModels/index.md +++ b/ViewModels/index.md @@ -4,7 +4,7 @@ MVVM layer. See [`schema.md`](../schema.md) for the memory-map conventions. | File | Purpose | |------|---------| -| `MainViewModel.cs` | The app brain: scenes/elements collections + commands, stream state (`IsLive`/`IsOffline`/`IsConnected`), chat feed, overlays, layout save/open, **resolution dropdown** (`QualityOptions`, `SelectedQuality`, `ResolutionHelp`, `ApplyStreamQuality`) that computes the output rect over the 1920×1080 master (`OutputRectX/Y/W/H`, `DimRects`, `IsOutputCropped`, `ResolutionBadgeText`). Auth: loads the saved DPAPI session at startup (`LoadSavedSessionAsync`), `SignInAsync` feeds the GoLive dialog; **End Livestream signs out** (`StopStream` clears session + token — crash-safe). **Connected account:** top-bar avatar/name (`AccountAvatarUrl`/`AccountDisplayName` via `SyncConnectedAccount`, cleared on End). **Audio (KISS — the mic is the creator's only audio control; capture pipeline still pending):** the sound meter is a **READ-ONLY realtime level display**: fill = `Math.Min(1, AudioLevel * MicVolume)` (`MeterFillWidth`/`MeterBrush`, green→yellow→red, zone markers at 60%/80%), where `AudioLevel` (live input, 0 with no input) is fed by the audio mixer once capture lands and `MicVolume` acts as a gain on ambient noise; while the slider is dragged the bar previews the slider position (`SetVolumeAdjusting`), returning to the live level on release (0 with no input — clicking the meter does nothing); `MicVolume` (default 0.8) + read-only `MicMuted`/`MicMuteText`/`ToggleMicMuteCommand` — **MicVolume drives MicMuted** (muted ⇔ volume 0): sliding to 0 flips the speaker to muted, sliding up from 0 clears it; muting stores the prior volume, unmuting restores it (default 0.8 if unknown) and flashes the meter to the restored position ~300ms (`BeginVolumeFlash`/`EndVolumeFlash`); `MicSourceName` = picked voice source, shown left-justified inside the meter bar (the fill runs at 75% opacity so the text + ruler markings show through); `OpenMicPickerCommand`/`PickMicrophone()` open the `MicPickerDialog`; desktop/game audio is automatic (WASAPI loopback at unity, zero UI). **Webcam:** owns `CameraManager` (MediaCapture), `AddWebcamToActiveSceneAsync` (picker → default 480×270 bottom-right placement → acquire; the Windows camera picker ALWAYS opens so the creator chooses — never silently reuses the previous camera; picking a different camera swaps the app-wide identity via `SwapWebcamIdentityAsync`, same path as `ChangeWebcamAsync`; propagates the running shared bitmap via `CameraManager.GetPreviewBitmap` first, so a webcam added mid-session never renders a transparent container; greys out when the active scene already has a config via `CanAddWebcamToActiveScene`), `ChangeWebcamAsync` (device swap — `ReleaseAllAsync` old + re-acquire), `ShowWebcamInActiveScene` (reveal hidden config / empty-canvas right-click), `ReacquireWebcam` after layout load (re-propagates the shared bitmap to every config), `OnCameraPreviewBitmapChanged` forwards the shared bitmap into every `WebcamSceneConfig.VideoImageSource`; `SelectedElement` drives the preview overlay + `internal ClampWebcamToBounds(config, sceneName)` (per-scene size cap seam: 50%-per-dimension everywhere, half-screen-AREA in Chat via `MaxWebcamWidthFor`/`MaxWebcamHeightFor`). **Scenes (five-scene catalog):** empty DB seeds Starting/Live/BRB/Chat/Ending (`SceneCatalog.All`); `AddScene(name)` accepts only canonical, missing names; `MissingScenes`/`ShowAddScene` drive the "+" button (hidden once all five exist; its menu lists only the missing scenes via `AddSceneCommand`). **Screen backdrop (Live-only by policy):** owns `ScreenCaptureManager` + `ScreenCaptureSourceFactory` (WinRT GraphicsCapture), `internal static EnsureBackdrop` heals one per backdrop-enabled scene (gated on `Scene.HasBackdrop`), `ReacquireScreenCaptures`/`RefreshBackdropAutoCapture`/`NoteBackgroundWindow` key by full-screen game monitor or the **primary display** via `Win32FullScreenDetector` (`GetDisplays()`/`PrimaryMonitorIndex()`), `ChangeBackdropCaptureAsync` (OS picker) + `SetBackdropCapture(DisplayInfo)` (in-app "Capture Display"), `RedesignateBackdropAsync` re-targets all backdrops and releases orphaned sessions, `internal static EnforceBackdropPolicy` normalizes `Scene.HasBackdrop` by name on every load + strips lingering non-Live backdrops, `CanChangeBackdrop` gates the capture menu to Live (the only scene with a backdrop), `BackdropImage` hides the preview watermark (`ShowPreviewPlaceholder`) | +| `MainViewModel.cs` | The app brain: scenes/elements collections + commands, stream state (`IsLive`/`IsOffline`/`IsConnected`), chat feed, overlays, layout save/open, **resolution dropdown** (`QualityOptions`, `SelectedQuality`, `ResolutionHelp`, `ApplyStreamQuality`) that computes the output rect over the 1920×1080 master (`OutputRectX/Y/W/H`, `DimRects`, `IsOutputCropped`, `ResolutionBadgeText`). Auth: loads the saved DPAPI session at startup (`LoadSavedSessionAsync`), `SignInAsync` feeds the GoLive dialog; **End Livestream signs out** (`StopStream` clears session + token — crash-safe). **Connected account:** top-bar avatar/name (`AccountAvatarUrl`/`AccountDisplayName` via `SyncConnectedAccount`, cleared on End). **Audio (KISS — the mic is the creator's only audio control; capture SHIPPED, runs for the app's lifetime so both meters preview live):** the sound meter is a **READ-ONLY realtime level display**: fill = `Math.Min(1, AudioLevel * MicVolume)` (`MeterFillWidth`/`MeterBrush`, green→yellow→red, zone markers at 60%/80%), where `AudioLevel` (live input, 0 with no input) is fed by the audio mixer once capture lands and `MicVolume` acts as a gain on ambient noise; while the slider is dragged the bar previews the slider position (`SetVolumeAdjusting`), returning to the live level on release (0 with no input — clicking the meter does nothing); `MicVolume` (default 0.8) + read-only `MicMuted`/`MicMuteText`/`ToggleMicMuteCommand` — **MicVolume drives MicMuted** (muted ⇔ volume 0): sliding to 0 flips the speaker to muted, sliding up from 0 clears it; muting stores the prior volume, unmuting restores it (default 0.8 if unknown) and flashes the meter to the restored position ~300ms (`BeginVolumeFlash`/`EndVolumeFlash`); `MicSourceName` = picked voice source, shown left-justified inside the meter bar (the fill runs at 75% opacity so the text + ruler markings show through); `OpenMicPickerCommand`/`PickMicrophone()` open the `MicPickerDialog` (a picked device takes effect immediately — `PickMicrophone` swaps the live source via `_audioMixer.RestartMic()`, loopback keeps running); the **MIC label is a button** (`OpenMicPickerCommand`) with a **status dot** (`MicStatus`, `Models/MicStatus`: green = `MicConnected` via the source's `Started` event, yellow = `MicFailed` — in use/unplugged, red = no mic device at startup; `MicStatusBrush`/`MicStatusToolTip`); capture starts once at startup (`StartMicCaptureAsync`) — NOT go-live (`BeginGoLive`/`StopStream` no longer touch the mixer) — zero devices = red dot + the mixer never starts. **Game audio bar** (desktop/game, beneath the preview — a mirror of the mic bar): `IsGameAudioBarVisible` (shown only while a full-screen game is producing sound — the VM polls `IGameAudioDetector` via the default `GameAudioDetector` every 250ms (`_gameAudioTimer`); the pure `GameAudioHysteresis` SHOWs after ~500ms of fullscreen+sound, HIDEs ~1s after leaving fullscreen, and **silence never hides an active bar**), `GameAudioLevel` (loopback meter via `LoopbackLevelChanged`, scaled by volume), `GameMuted`/`GameMuteText`/`ToggleGameMuteCommand`, `GameAudioVolume` (0..1 volume slider), `Begin/End/CancelGameVolumeFlash` (mirrors the mic bar's volume flash); the game speaker + slider share the mic bar's `PreviewMouseLeftButtonDown/Up` + `LostMouseCapture` code-behind pattern. **Webcam:** owns `CameraManager` (MediaCapture), `AddWebcamToActiveSceneAsync` (picker → default 480×270 bottom-right placement → acquire; the Windows camera picker ALWAYS opens so the creator chooses — never silently reuses the previous camera; picking a different camera swaps the app-wide identity via `SwapWebcamIdentityAsync`, same path as `ChangeWebcamAsync`; propagates the running shared bitmap via `CameraManager.GetPreviewBitmap` first, so a webcam added mid-session never renders a transparent container; greys out when the active scene already has a config via `CanAddWebcamToActiveScene`), `ChangeWebcamAsync` (device swap — `ReleaseAllAsync` old + re-acquire), `ShowWebcamInActiveScene` (reveal hidden config / empty-canvas right-click), `ReacquireWebcam` after layout load (re-propagates the shared bitmap to every config), `OnCameraPreviewBitmapChanged` forwards the shared bitmap into every `WebcamSceneConfig.VideoImageSource`; `SelectedElement` drives the preview overlay + `internal ClampWebcamToBounds(config, sceneName)` (per-scene size cap seam: 50%-per-dimension everywhere, half-screen-AREA in Chat via `MaxWebcamWidthFor`/`MaxWebcamHeightFor`). **Scenes (five-scene catalog):** empty DB seeds Starting/Live/BRB/Chat/Ending (`SceneCatalog.All`); `AddScene(name)` accepts only canonical, missing names; `MissingScenes`/`ShowAddScene` drive the "+" button (hidden once all five exist; its menu lists only the missing scenes via `AddSceneCommand`). **Screen backdrop (Live-only by policy):** owns `ScreenCaptureManager` + `ScreenCaptureSourceFactory` (WinRT GraphicsCapture), `internal static EnsureBackdrop` heals one per backdrop-enabled scene (gated on `Scene.HasBackdrop`), `ReacquireScreenCaptures`/`RefreshBackdropAutoCapture`/`NoteBackgroundWindow` key by full-screen game monitor or the **primary display** via `Win32FullScreenDetector` (`GetDisplays()`/`PrimaryMonitorIndex()`), `ChangeBackdropCaptureAsync` (OS picker) + `SetBackdropCapture(DisplayInfo)` (in-app "Capture Display"), `RedesignateBackdropAsync` re-targets all backdrops and releases orphaned sessions, `internal static EnforceBackdropPolicy` normalizes `Scene.HasBackdrop` by name on every load + strips lingering non-Live backdrops, `CanChangeBackdrop` gates the capture menu to Live (the only scene with a backdrop), `BackdropImage` hides the preview watermark (`ShowPreviewPlaceholder`) | | `GoLiveViewModel.cs` | Start Stream dialog: **account row** (saved channel shown with Change Account, or Sign in to YouTube; Start gated on `IsSignedIn`/`IsBusy`) + title/description/visibility, start/cancel requests | | `ReuseImageViewModel.cs` | Add Image dialog: candidate list (`ReuseImageCandidate`), reuse/new/cancel | | `CameraPickerViewModel.cs` | Add Webcam dialog: async camera list (loading / has / none states), `UseRequested(CameraDeviceInfo)`/`CancelRequested` | diff --git a/ai.md b/ai.md index f33c505..c7275b9 100644 --- a/ai.md +++ b/ai.md @@ -91,7 +91,7 @@ C# / WPF (.NET 8) following MVVM: |------|------| | `Models/` | Plain data types — Scene, Source (incl. `ClipShape`, `IsMirrored`, `VideoImageSource`), QualityOption, StreamConfig, StreamHealth, YouTubeChannel, ChatMessage, **Socials (`SocialService` enum + `SocialEntry`/`SocialsConfig` + `SocialServiceIcons`) — the social bar** | | `ViewModels/` | MainViewModel — exposes collections + commands for the UI; GoLiveViewModel, ReuseImageViewModel, CameraPickerViewModel, **SocialsDialogViewModel** | -| `Services/` | YouTube OAuth2, stream/broadcast management, live chat polling, LayoutStore (SQLite), **SocialValidator (`ISocialValidator` seam + `HttpSocialValidator` default)**, **webcam: `VideoFrame` seam + `CameraDeviceInfo`/`ICameraEnumerator`/`ICameraFrameSource` interfaces + `MediaCaptureCameraEnumerator`/`MediaCaptureFrameSource` (WinRT) + `CameraManager`**, **screen capture: `IFullScreenDetector`/`Win32FullScreenDetector` + `IScreenCaptureSource`/`ScreenCaptureFrameSource` (WinRT GraphicsCapture) + `ScreenCaptureManager` + `ScreenCaptureSourceFactory` + `Direct3D11Helper`/`CaptureInterop` (COM bridges)**, **compositor: `SceneCompositor` + `CompositorOptions` + pure `StretchMath` + `StaticPixelCache` (see "Scene compositor")**, **audio: `IAudioSource` seam + `WasapiLoopbackAudioSource`/`WasapiMicAudioSource` (NAudio WASAPI) + `AudioMixer` + pure `AudioLevelMeter`/`WaveToFloat` (see "Live audio capture")**, **encoder: `IFfmpegEncoder`/`FfmpegEncoder` + `IEncoderProcess`/`FfmpegEncoderProcess` + `IFfmpegLocator`/`FfmpegLocator` + pure `FfmpegArgs`/`FfmpegProgressParser`/`FfmpegEncoderPicker` + the `FramePump` frame producer (see "Live encoder" + "Live frame pipeline")** | +| `Services/` | YouTube OAuth2, stream/broadcast management, live chat polling, LayoutStore (SQLite), **SocialValidator (`ISocialValidator` seam + `HttpSocialValidator` default)**, **webcam: `VideoFrame` seam + `CameraDeviceInfo`/`ICameraEnumerator`/`ICameraFrameSource` interfaces + `MediaCaptureCameraEnumerator`/`MediaCaptureFrameSource` (WinRT) + `CameraManager`**, **screen capture: `IFullScreenDetector`/`Win32FullScreenDetector` + `IScreenCaptureSource`/`ScreenCaptureFrameSource` (WinRT GraphicsCapture) + `ScreenCaptureManager` + `ScreenCaptureSourceFactory` + `Direct3D11Helper`/`CaptureInterop` (COM bridges)**, **compositor: `SceneCompositor` + `CompositorOptions` + pure `StretchMath` + `StaticPixelCache` (see "Scene compositor")**, **audio: `IAudioSource` seam + `WasapiLoopbackAudioSource`/`WasapiMicAudioSource` (NAudio WASAPI) + `AudioMixer` + pure `AudioLevelMeter`/`WaveToFloat` + game bar: `IGameAudioDetector`/`GameAudioHysteresis`/`GameAudioDetector` (see "Live audio capture")**, **encoder: `IFfmpegEncoder`/`FfmpegEncoder` + `IEncoderProcess`/`FfmpegEncoderProcess` + `IFfmpegLocator`/`FfmpegLocator` + pure `FfmpegArgs`/`FfmpegProgressParser`/`FfmpegEncoderPicker` + the `FramePump` frame producer (see "Live encoder" + "Live frame pipeline")** | | `Helpers/` | ViewModelBase (INotifyPropertyChanged), RelayCommand, ImageCache, AppLog (file logger), FocusPreservingListBox, OAuthCredentials, **TokenStore (DPAPI session persistence)**, visibility converters | | `Themes/` | `Controls.xaml` — the single dark-theme source, merged once in `App.xaml` (see `Themes/index.md`) | | `MainWindow.xaml` | Dark theme; layout: top bar (controls), center (preview), left (scenes/sources), right (chat), bottom (health) | @@ -355,34 +355,47 @@ forces one and skips the probe. `EncoderOptions` also carries W×H/FPS/bitrate f `GopSize` = FPS×4. Constructed by `MainViewModel` since ship step 5 (`new FfmpegEncoder(new FfmpegLocator())`), driven by the `FramePump` below. -### Live audio capture (TASK 4 ship step 4 — shipped 2026-08-12, plan in TASKS.md) +### Live audio capture (TASK 4 ship step 4 — shipped 2026-08-12, game audio bar follow-up 2026-08-13, plans in TASKS.md) -**Capture runs only while live and is KISS by rule**: desktop/game audio is automatic (WASAPI loopback, -zero UI), the mic is the creator's only audio control (meter/mute/volume already shipped). The whole -layer sits behind an **`IAudioSource` seam** (`Services/Audio/`: `Start`/`Stop`/`SampleReady`/`Failed`, -IDisposable) so the app never touches NAudio directly and tests inject hermetic fakes (no devices, no -timers). +**Capture runs for the app's lifetime and is KISS by rule**: desktop/game audio is automatic (WASAPI +loopback), the mic is the creator's only audio control (meter/mute/volume already shipped). The whole +layer sits behind an **`IAudioSource` seam** (`Services/Audio/`: `Start`/`Stop`/`Started`/`SampleReady`/ +`Failed`, IDisposable) so the app never touches NAudio directly and tests inject hermetic fakes (no +devices, no timers). - **Sources (NAudio `NAudio.Wasapi` 2.2.1, MIT — item 9 in `THIRD-PARTY-NOTICES.txt`):** `WasapiLoopbackAudioSource` = `WasapiLoopbackCapture` on the default render device; `WasapiMicAudioSource` = `WasapiCapture` with the device resolved by `FriendlyName` matching `MicSourceName` (the app only persists the **DisplayName**), falling back to the default capture endpoint. Mic device resolution re-reads the name provider `Func` at each `Start`, so a mic - picked mid-session takes effect next go-live. -- **`AudioMixer`** owns both sources; `Start`/`Stop` follow go-live (`MainViewModel.BeginGoLive` success - → `_audioMixer.Start()`, `StopStream` → `Stop()`). Mic samples feed a pure **`AudioLevelMeter`** (RMS - with 0.2 exponential smoothing) → `MicLevelChanged` → marshalled to the UI thread → `AudioLevel`. - Desktop samples are currently **dropped** — a later step's AAC mix consumes them, replacing the - `-f lavfi -i anullsrc` placeholder (the encoder construction itself shipped in ship step 5). Failures log via `AppLog`; a mic failure zeroes the - meter, a loopback failure never kills the mic. + picked mid-session takes effect **immediately** (the mixer restarts the mic on pick). Both sources + raise `Started` once their capture loop actually begins — the mixer turns that into `MicConnected`. +- **`AudioMixer`** owns both sources; **`StartMicCaptureAsync` starts the mixer once at startup** and + `Shutdown` disposes it — NOT go-live — so both footer meters preview live (`BeginGoLive`/`StopStream` + no longer touch the mixer). Mic samples feed a pure **`AudioLevelMeter`** (RMS with 0.2 exponential + smoothing) → `MicLevelChanged` → marshalled to the UI thread → `AudioLevel`; loopback samples feed a + second meter → `LoopbackLevelChanged` → the game bar's `GameAudioLevel`. **Mic connection state + surfaces as events:** `MicConnected` (source `Started`), `MicFailed` (source `Failed`), and + `RestartMic()` re-resolves + restarts just the mic (loopback keeps running). Failures log via + `AppLog`; a mic failure zeroes the meter, a loopback failure never kills the mic. Meter `Push` is + unconditional (a `?.` on the event would skip the argument — and the meter update — when nothing is + subscribed yet). +- **Mic status dot (`Models/MicStatus.cs`)** on the footer's MIC button: green = `MicConnected`, + yellow = `MicFailed` (in use/unplugged), red = no mic device at startup (the mixer is never started, + so loopback and the game bar can't run either — no capture devices at all). +- **Game audio bar** (desktop/game, only while a full-screen game is up in the preview): + `IGameAudioDetector` seam (`Services/IGameAudioDetector.cs`), pure `GameAudioHysteresis` (SHOW after + ~500ms of fullscreen + sound, HIDE after ~1s away from fullscreen, **silence never hides an active + bar**), and the default `GameAudioDetector` composing `IFullScreenDetector` + the live loopback level + (floor 0.5%). WPF-free — the VM owns a 250ms `DispatcherTimer` that polls it and flips + `IsGameAudioBarVisible`. - **`WaveToFloat`** (pure, shared): WASAPI mix formats → interleaved float — IEEE float 32-bit direct, PCM 16-bit normalized to -1..1, `WaveFormatExtensible` with the IEEE-float subformat GUID (`NAudio.Dmo.AudioMediaSubtypes.MEDIASUBTYPE_IEEE_FLOAT`), trailing partial samples ignored. -- `FfmpegEncoder.cs:139` pre-existing CS8602 fixed (`process!`) — build **0 warnings**; **139 passing**. +- Build **0 warnings**; **167 passing** (mixer/hysteresis/game-detector unit tests). **Deferred:** the loopback→encoder AAC mix wiring (a later step — the encoder construction + frame -pipeline shipped in ship step 5); capture while not live is deliberately not shipped (privacy indicator -otherwise). +pipeline shipped in ship step 5). ### Live frame pipeline (TASK 4 ship step 5 — shipped 2026-08-12, plan in TASKS.md) diff --git a/ytLive.Tests/AudioMixerTests.cs b/ytLive.Tests/AudioMixerTests.cs index d613b90..917a5fd 100644 --- a/ytLive.Tests/AudioMixerTests.cs +++ b/ytLive.Tests/AudioMixerTests.cs @@ -18,6 +18,7 @@ public class AudioMixerTests public int StartCount { get; private set; } public int StopCount { get; private set; } public bool Disposed { get; private set; } + public event Action? Started; public event Action? SampleReady; public event Action? Failed; @@ -25,6 +26,7 @@ public class AudioMixerTests public void Stop() => StopCount++; public void Dispose() => Disposed = true; + public void MarkStarted() => Started?.Invoke(); public void Emit(AudioSample sample) => SampleReady?.Invoke(sample); public void Fail(Exception ex) => Failed?.Invoke(ex); } @@ -115,6 +117,98 @@ public class AudioMixerTests Assert.Equal(0f, mixer.MicLevel); } + [Fact] + public void MicSamples_DoNotChangeLoopbackLevel() + { + var mic = new FakeSource(); + var mixer = new AudioMixer(mic, new FakeSource()); + var levels = new List(); + mixer.LoopbackLevelChanged += l => levels.Add(l); + + mixer.Start(); + mic.Emit(new AudioSample(new[] { 1f, 1f, 1f, 1f }, 48000, 1)); + + Assert.Empty(levels); + Assert.Equal(0f, mixer.LoopbackLevel); + } + + [Fact] + public void LoopbackSamples_DriveLoopbackLevelChanged() + { + var loopback = new FakeSource(); + var mixer = new AudioMixer(new FakeSource(), loopback); + var levels = new List(); + mixer.LoopbackLevelChanged += l => levels.Add(l); + + mixer.Start(); + loopback.Emit(new AudioSample(new[] { 1f, 1f, 1f, 1f }, 48000, 2)); + loopback.Emit(new AudioSample(new[] { -1f, -1f, -1f, -1f }, 48000, 2)); + + Assert.NotEmpty(levels); + Assert.All(levels, l => Assert.InRange(l, 0f, 1f)); + } + + [Fact] + public void MicStarted_RaisesMicConnected() + { + var mic = new FakeSource(); + var mixer = new AudioMixer(mic, new FakeSource()); + var connected = 0; + mixer.MicConnected += () => connected++; + + mixer.Start(); + mic.MarkStarted(); + + Assert.Equal(1, connected); + } + + [Fact] + public void MicFailure_RaisesMicFailed() + { + var mic = new FakeSource(); + var mixer = new AudioMixer(mic, new FakeSource()); + Exception? failed = null; + mixer.MicFailed += ex => failed = ex; + + mixer.Start(); + mic.Fail(new InvalidOperationException("boom")); + + Assert.NotNull(failed); + Assert.Equal("boom", failed!.Message); + } + + [Fact] + public void RestartMic_StopsAndRestartsMic_KeepsLoopbackRunning() + { + var mic = new FakeSource(); + var loopback = new FakeSource(); + var mixer = new AudioMixer(mic, loopback); + mixer.Start(); + + mixer.RestartMic(); + + Assert.Equal(2, mic.StartCount); + Assert.Equal(1, mic.StopCount); + Assert.Equal(1, loopback.StartCount); + Assert.Equal(0, loopback.StopCount); + } + + [Fact] + public void RestartMic_ResetsLevel() + { + var mic = new FakeSource(); + var mixer = new AudioMixer(mic, new FakeSource()); + mixer.Start(); + mic.Emit(new AudioSample(new[] { 1f, 1f, 1f, 1f }, 48000, 1)); + Assert.True(mixer.MicLevel > 0); + + float? reset = null; + mixer.MicLevelChanged += l => reset = l; + mixer.RestartMic(); + + Assert.Equal(0f, reset); + } + [Fact] public void MicFailure_LogsAndResetsLevel() { diff --git a/ytLive.Tests/GameAudioDetectorTests.cs b/ytLive.Tests/GameAudioDetectorTests.cs new file mode 100644 index 0000000..818d879 --- /dev/null +++ b/ytLive.Tests/GameAudioDetectorTests.cs @@ -0,0 +1,76 @@ +using Xunit; +using ytLive.Services; + +namespace ytLive.Tests; + +/// +/// TASK 4 game audio bar: the default detector's provider wiring — it composes +/// the full-screen monitor + loopback level into the hysteresis and raises +/// IsGameAudioActiveChanged on transitions. The transition math itself lives in +/// GameAudioHysteresisTests. +/// +public class GameAudioDetectorTests +{ + [Fact] + public void Poll_RaisesChanged_WhenGameAppearsAndLeaves() + { + var now = new DateTime(2026, 8, 13, 12, 0, 0); + int? monitor = 0; + var level = 0f; + var detector = new GameAudioDetector(() => monitor, () => level, () => now); + var changes = new List(); + detector.IsGameAudioActiveChanged += a => changes.Add(a); + + level = 0.9f; + detector.Poll(); + Assert.False(detector.IsGameAudioActive); + + now = now.AddMilliseconds(600); + detector.Poll(); + Assert.True(detector.IsGameAudioActive); + Assert.Equal(new[] { true }, changes); + + level = 0f; + now = now.AddSeconds(2); + detector.Poll(); + Assert.True(detector.IsGameAudioActive); + Assert.Equal(new[] { true }, changes); + + monitor = null; + detector.Poll(); + Assert.True(detector.IsGameAudioActive); + + now = now.AddSeconds(2); + detector.Poll(); + Assert.False(detector.IsGameAudioActive); + Assert.Equal(new[] { true, false }, changes); + } + + [Fact] + public void Poll_StaysInactive_WhenNoFullScreenMonitor() + { + var detector = new GameAudioDetector(() => null, () => 0.9f); + var changes = 0; + detector.IsGameAudioActiveChanged += _ => changes++; + + for (var i = 0; i < 5; i++) + detector.Poll(); + + Assert.False(detector.IsGameAudioActive); + Assert.Equal(0, changes); + } + + [Fact] + public void Poll_StaysInactive_WhileLoopbackSilent() + { + var detector = new GameAudioDetector(() => 0, () => 0f); + var changes = 0; + detector.IsGameAudioActiveChanged += _ => changes++; + + for (var i = 0; i < 5; i++) + detector.Poll(); + + Assert.False(detector.IsGameAudioActive); + Assert.Equal(0, changes); + } +} diff --git a/ytLive.Tests/GameAudioHysteresisTests.cs b/ytLive.Tests/GameAudioHysteresisTests.cs new file mode 100644 index 0000000..7dc61a0 --- /dev/null +++ b/ytLive.Tests/GameAudioHysteresisTests.cs @@ -0,0 +1,82 @@ +using Xunit; +using ytLive.Services; + +namespace ytLive.Tests; + +/// +/// TASK 4 game audio bar: the pure show/hide state machine. Show = a full-screen +/// app holds sound for half a second; hide = the app leaves fullscreen for a +/// second. Silence never hides an active bar — only the game leaving the preview +/// does (per the creator's rule). +/// +public class GameAudioHysteresisTests +{ + private static readonly DateTime T0 = new(2026, 8, 13, 12, 0, 0); + + [Fact] + public void StaysInactive_WhileSilent() + { + var h = new GameAudioHysteresis(); + + for (var i = 0; i < 30; i++) + { + h.Update(true, false, T0.AddSeconds(i)); + Assert.False(h.IsActive); + } + } + + [Fact] + public void StaysInactive_UntilSoundHoldsHalfSecond() + { + var h = new GameAudioHysteresis(); + h.Update(true, true, T0); + Assert.False(h.IsActive); + h.Update(true, true, T0.AddMilliseconds(400)); + Assert.False(h.IsActive); + + h.Update(true, true, T0.AddMilliseconds(600)); + Assert.True(h.IsActive); + } + + [Fact] + public void NeverHides_WhileGameStillFullScreen_EvenWhenSilent() + { + var h = new GameAudioHysteresis(); + h.Update(true, true, T0); + h.Update(true, true, T0.AddSeconds(1)); + Assert.True(h.IsActive); + + for (var i = 2; i < 40; i++) + { + h.Update(true, false, T0.AddSeconds(i)); + Assert.True(h.IsActive); + } + } + + [Fact] + public void Hides_AfterAppLeavesFullScreen() + { + var h = new GameAudioHysteresis(); + h.Update(true, true, T0); + h.Update(true, true, T0.AddSeconds(1)); + Assert.True(h.IsActive); + + h.Update(false, true, T0.AddSeconds(2)); + Assert.True(h.IsActive); + + h.Update(false, true, T0.AddSeconds(4)); + Assert.False(h.IsActive); + } + + [Fact] + public void StaysInactive_WhenNoFullScreen_EvenWithSound() + { + var h = new GameAudioHysteresis(); + + for (var i = 0; i < 30; i++) + { + h.Update(false, true, T0.AddSeconds(i)); + Assert.False(h.IsActive); + } + } +}