TASK 4 audio follow-up: game audio bar + mic status dot + always-on capture — the footer's second audio control (desktop/game, a mirror of the mic bar: meter + mute + volume) appears only while a full-screen game is producing sound (IGameAudioDetector seam + GameAudioHysteresis: show ~500ms of fullscreen+sound, hide ~1s after leaving fullscreen, silence never hides an active bar; VM polls on a 250ms timer); capture now runs for the app's lifetime so both meters preview live (started at startup via StartMicCaptureAsync, disposed in Shutdown — no longer go-live driven); MIC label is a button with a status dot (Models/MicStatus: green via the source Started event, yellow = mic problem, red = no device); PickMicrophone swaps the live device immediately via AudioMixer.RestartMic; fixed a latent ?.Invoke(meter.Push(...)) short-circuit that skipped the meter update when nothing subscribed — 167 tests passing, 0 warnings

This commit is contained in:
2026-08-13 11:29:44 -07:00
parent ea250c02a6
commit 9f9ed34627
20 changed files with 897 additions and 82 deletions
+41 -25
View File
@@ -7,41 +7,57 @@
## Session state (last updated: 2026-08-13) ## Session state (last updated: 2026-08-13)
- **Branch:** `main`. TASK 4 **ship step 5.5** is **committed and pushed** (`ac60a26`, - **Branch:** `main`. The **game audio bar + mic status dot + always-on capture**
"TASK 4 ship step 5.5: social bar bug fixes + bar on the live output"). work is **uncommitted** (see list below). The last committed+pushed baseline is
- **This session (follow-up):** the drag-snap from 5.5 failed in practice — it snapped `ea250c0` (social bar click-toggle).
up but wouldn't come back down (jitter around the deadzone, per the user). **Superseded - **This session (audio UX follow-up):** per the creator's requests —
by a click-toggle** (user decision, KISS): `MainWindow.SocialBar_MouseLeftButtonDown` 1. The MIC label is now a **button with a status dot** (`Models/MicStatus`:
`MainViewModel.ToggleSocialBarPosition()` flips the bar top ⇄ bottom; the bar rides green = capturing, yellow = requested mic problem, red = no device).
`{Binding SocialBarTop}` alone; `SocialBarSnap` + its 2 tests removed. Build 0 warnings, 2. **Mic capture runs for the app's lifetime** (started at startup via
**153 tests passing** (155 2 snap units). **Uncommitted:** this click-toggle change + `StartMicCaptureAsync`, disposed in `Shutdown`; `BeginGoLive`/`StopStream`
its memory corrections (TASKS.md / ai.md / HANDOFF). Next action: commit + push. no longer start/stop the mixer) so both meters preview live.
- **Verified:** 0 warnings; 153/153 tests pass. 3. A **game audio bar** (desktop/game — meter + mute + volume, mirror of the
mic bar) sits centered beneath the preview and appears only while a
**full-screen game is producing sound** (decision: fullscreen + loopback
sound; **silence never hides an active bar** — the creator's final rule).
Show after ~500ms of fullscreen+sound, hide ~1s after leaving fullscreen.
- **Uncommitted files:** `MainWindow.xaml`/`.cs`, `ViewModels/MainViewModel.cs`,
`Services/Audio/{IAudioSource,WasapiMicAudioSource,WasapiLoopbackAudioSource,AudioMixer}.cs`,
new `Models/MicStatus.cs`, new `Services/{IGameAudioDetector,GameAudioHysteresis,GameAudioDetector}.cs`,
`ytLive.Tests/AudioMixerTests.cs`, new `GameAudioHysteresisTests.cs` +
`GameAudioDetectorTests.cs`, memory docs (TASKS.md, ai.md, Services/index.md,
ViewModels/index.md, Models/index.md, HANDOFF).
- **Verified:** build 0 warnings, 0 errors; **167 tests passing** (full suite,
after the final rebuild).
- **Landmines:** - **Landmines:**
- Never set a local `Canvas.SetTop` on the social bar — a local value permanently - Never set a local `Canvas.SetTop` on the social bar — a local value permanently
overrides `{Binding SocialBarTop}` (the `ClearValue` lesson from 5.5). overrides `{Binding SocialBarTop}` (the `ClearValue` lesson from 5.5).
- `AudioMixer` meter `Push` is unconditional **by design now**: `OnMicSample`/
`OnLoopbackSample` compute the level first, then raise the event — a
`?.Invoke(meter.Push(...))` short-circuit skipped the meter update when
nothing was subscribed (found by `RestartMic_ResetsLevel`, fixed).
- `MicConnected` comes from the source `Started` event, raised right after
`StartRecording()` succeeds — tests must `MarkStarted()` the fake source
before asserting connection state.
- Zero mic devices at startup = red dot AND the mixer is never started, so
loopback + the game bar can't run either (no capture at all) — acceptable.
- The game detector is polled on the UI thread via a 250ms `DispatcherTimer`;
`OnGameAudioPollTick` wraps `Poll()` in try/catch + `AppLog`.
- The pump reads the active scene on a background thread while the UI can still - The pump reads the active scene on a background thread while the UI can still
edit it — a concurrent-mutation exception is contained (logged + `Failed` + edit it — a concurrent-mutation exception is contained (logged + `Failed` +
the pump stops), not a crash. The background thread + video pipeline is the the pump stops), not a crash.
new reality since ship step 5.
- `StopAsync` must stop the encoder (closes stdin) **before** awaiting the pump - `StopAsync` must stop the encoder (closes stdin) **before** awaiting the pump
loop — closing stdin unblocks a write stuck on pipe backpressure; the reverse loop — closing stdin unblocks a write stuck on pipe backpressure; the reverse
order deadlocks. order deadlocks.
- `FramePump.IsRunning` must be set true before the loop starts (a completed-task
delay can run the first iteration synchronously on the caller's thread).
- `StartAsync` never throws; the VM fires-and-forgets it. `Failed` while live
flips `StreamStatus.Error` (minimal — real health surfacing is ship step 6).
- The heal runs off the UI thread and applies via the dispatcher; a config swap
mid-heal (dialog Save) can orphan the healed values on the old entries — harmless
(best-effort, re-healed next load).
- Tests never instantiate `MainViewModel` directly except the round-clip - Tests never instantiate `MainViewModel` directly except the round-clip
integration test (a real `MainWindow`), which never goes live — keep it that way. integration test (a real `MainWindow`), which never goes live — keep it that way.
- Sandbox can't reach outbound HTTPS — the subdomain-probe logic is verified via - Sandbox can't reach outbound HTTPS — `HttpSocialValidator` stub-handler tests
stub-handler tests only, not against the real `mastodon.llamachile.tube`. only, never the real instance.
- **Next step:** commit + push the click-toggle follow-up. Then TASK 4 ship step 6 — - **Next step:** commit + push the audio UX follow-up (the whole uncommitted set
health stats: bind `FramePump.HealthUpdated` (bitrate/FPS/duration) into the bottom above in one commit). Then TASK 4 ship step 6 — health stats: bind
bar. Nothing else queued — do not expand the task queue on your own. Optional, not `FramePump.HealthUpdated` (bitrate/FPS/duration) into the bottom bar. Nothing
queued: rewriting the healed entry's `ProfileUrl` to else queued — do not expand the task queue on your own. Optional, not queued:
rewriting the healed entry's `ProfileUrl` to
`https://mastodon.llamachile.tube/@gramps` (user must say the word). `https://mastodon.llamachile.tube/@gramps` (user must say the word).
- **Secret/DB/port facts live:** OAuth client id/secret in `Helpers/OAuthCredentials.cs`; - **Secret/DB/port facts live:** OAuth client id/secret in `Helpers/OAuthCredentials.cs`;
+68 -8
View File
@@ -795,13 +795,15 @@
<Grid.RowDefinitions> <Grid.RowDefinitions>
<RowDefinition Height="Auto"/> <RowDefinition Height="Auto"/>
<RowDefinition Height="Auto"/> <RowDefinition Height="Auto"/>
<RowDefinition Height="Auto"/>
</Grid.RowDefinitions> </Grid.RowDefinitions>
<!-- Line 1: Social controls on the LEFT (under scenes/sources), <!-- Line 1: Social controls on the LEFT (under scenes/sources),
sound meter + volume control CENTERED beneath the preview panel. sound meter + volume control CENTERED beneath the preview panel.
Audio is KISS: desktop/game audio is automatic ("it just is" — Audio is KISS: desktop/game audio is automatic (WASAPI loopback
WASAPI loopback at unity, zero UI); the creator's only audio at unity); the creator's controls are the two meters — the mic
control is the mic — meter, volume, mute. --> (always visible, with its status dot) and the game bar (appears
only while a full-screen game is producing sound). -->
<Grid Grid.Row="0"> <Grid Grid.Row="0">
<Grid.ColumnDefinitions> <Grid.ColumnDefinitions>
<ColumnDefinition Width="220"/> <ColumnDefinition Width="220"/>
@@ -824,13 +826,19 @@
</Button> </Button>
</StackPanel> </StackPanel>
<!-- Meter + volume: centered under the preview panel --> <!-- Mic meter + volume: centered under the preview panel -->
<StackPanel Grid.Column="1" Orientation="Horizontal" <StackPanel Grid.Column="1" Orientation="Horizontal"
HorizontalAlignment="Center" VerticalAlignment="Center"> HorizontalAlignment="Center" VerticalAlignment="Center">
<TextBlock Text="MIC" Foreground="#a0a0b0" FontSize="11" FontWeight="SemiBold" <Button Command="{Binding OpenMicPickerCommand}"
VerticalAlignment="Center" Margin="0,0,8,0" Cursor="Hand" Style="{StaticResource YtButtonSecondary}" Padding="12,4"
ToolTip="Choose the microphone (voice source)" FontSize="11" VerticalAlignment="Center" Margin="0,0,8,0"
MouseLeftButtonUp="MicLabel_MouseLeftButtonUp"/> ToolTip="{Binding MicStatusToolTip}">
<StackPanel Orientation="Horizontal">
<Ellipse Width="8" Height="8" Fill="{Binding MicStatusBrush}"
Margin="0,0,6,0" VerticalAlignment="Center"/>
<TextBlock Text="MIC"/>
</StackPanel>
</Button>
<!-- Sound meter: muted track with a ruler scale, zone tints at <!-- Sound meter: muted track with a ruler scale, zone tints at
the yellow (60%) and red (80%) starts. READ-ONLY realtime the yellow (60%) and red (80%) starts. READ-ONLY realtime
level display (fill = live level scaled by volume). --> level display (fill = live level scaled by volume). -->
@@ -943,6 +951,58 @@
</Button> </Button>
</StackPanel> </StackPanel>
</Grid> </Grid>
<!-- Line 3: game/desktop audio — meter + volume + mute, centered
beneath the preview (a mirror of the mic bar). Appears only
while a full-screen game is producing sound (the game audio
detector); desktop audio itself is automatic WASAPI loopback. -->
<StackPanel Grid.Row="2" Orientation="Horizontal" Margin="0,6,0,0"
HorizontalAlignment="Center" VerticalAlignment="Center"
Visibility="{Binding IsGameAudioBarVisible, Converter={StaticResource BoolToVis}}">
<TextBlock Text="Game Audio Capture" Foreground="#a0a0b0" FontSize="11"
FontWeight="SemiBold" VerticalAlignment="Center" Margin="0,0,8,0"
ToolTip="Desktop/game audio via WASAPI loopback (automatic)"/>
<Border Width="288" Height="14" CornerRadius="7" Background="#3a3b52" Margin="0,0,8,0"
ClipToBounds="True" VerticalAlignment="Center">
<Grid>
<Rectangle Width="57" Fill="#4a4520" HorizontalAlignment="Left" Margin="173,0,0,0"
VerticalAlignment="Stretch"/>
<Rectangle Width="58" Fill="#4a2222" HorizontalAlignment="Left" Margin="230,0,0,0"
VerticalAlignment="Stretch"/>
<Border HorizontalAlignment="Left" Width="{Binding GameMeterFillWidth}"
Background="{Binding GameMeterBrush}" Opacity="0.75"/>
<Rectangle Width="1" Height="10" Fill="#26000000" VerticalAlignment="Center" HorizontalAlignment="Left" Margin="36,0,0,0"/>
<Rectangle Width="1" Height="10" Fill="#26000000" VerticalAlignment="Center" HorizontalAlignment="Left" Margin="72,0,0,0"/>
<Rectangle Width="1" Height="10" Fill="#26000000" VerticalAlignment="Center" HorizontalAlignment="Left" Margin="108,0,0,0"/>
<Rectangle Width="1" Height="10" Fill="#26000000" VerticalAlignment="Center" HorizontalAlignment="Left" Margin="144,0,0,0"/>
<Rectangle Width="1" Height="10" Fill="#26000000" VerticalAlignment="Center" HorizontalAlignment="Left" Margin="180,0,0,0"/>
<Rectangle Width="1" Height="10" Fill="#26000000" VerticalAlignment="Center" HorizontalAlignment="Left" Margin="216,0,0,0"/>
<Rectangle Width="1" Height="10" Fill="#26000000" VerticalAlignment="Center" HorizontalAlignment="Left" Margin="252,0,0,0"/>
<Rectangle Width="2" Fill="#00000080" HorizontalAlignment="Left" Margin="173,0,0,0"
VerticalAlignment="Stretch"/>
<Rectangle Width="2" Fill="#00000080" HorizontalAlignment="Left" Margin="230,0,0,0"
VerticalAlignment="Stretch"/>
</Grid>
</Border>
<Grid Width="16" Height="16" Cursor="Hand" Margin="0,0,8,0"
VerticalAlignment="Center" ToolTip="{Binding GameMuteText}"
MouseLeftButtonUp="GameSpeaker_MouseLeftButtonUp">
<Path Fill="#d0d0d0" Stretch="Uniform"
Data="M3,9v6h4l5,5V4L7,9H3zM16.5,12c0,-1.77 -1,-3.29 -2.5,-4.03v8.05c1.5,-0.73 2.5,-2.25 2.5,-4.02zM14,3.23v2.06c2.89,0.86 5,3.54 5,6.71s-2.11,5.85 -5,6.71v2.06c4.01,-0.91 7,-4.49 7,-8.77s-2.99,-7.86 -7,-8.77z"
Visibility="{Binding GameMuted, Converter={StaticResource InverseBoolToVis}}"/>
<Path Fill="#ef4444" Stretch="Uniform"
Data="M3,9v6h4l5,5V4L7,9H3zM16.5,12c0,-1.77 -1,-3.29 -2.5,-4.03v8.05c1.5,-0.73 2.5,-2.25 2.5,-4.02zM14,3.23v2.06c2.89,0.86 5,3.54 5,6.71s-2.11,5.85 -5,6.71v2.06c4.01,-0.91 7,-4.49 7,-8.77s-2.99,-7.86 -7,-8.77z"
Visibility="{Binding GameMuted, Converter={StaticResource BoolToVis}}"/>
<Path Stroke="#ef4444" StrokeThickness="2.5" StrokeEndLineCap="Round" Stretch="Uniform"
Data="M21,3 L3,21"
Visibility="{Binding GameMuted, Converter={StaticResource BoolToVis}}"/>
</Grid>
<Slider Width="130" Minimum="0" Maximum="1"
Value="{Binding GameAudioVolume, Mode=TwoWay}" VerticalAlignment="Center"
PreviewMouseLeftButtonDown="GameVolumeSlider_PreviewMouseLeftButtonDown"
PreviewMouseLeftButtonUp="GameVolumeSlider_PreviewMouseLeftButtonUp"
LostMouseCapture="GameVolumeSlider_LostMouseCapture"/>
</StackPanel>
</Grid> </Grid>
</Border> </Border>
+26 -6
View File
@@ -83,12 +83,6 @@ public partial class MainWindow : Window
} }
} }
private void MicLabel_MouseLeftButtonUp(object sender, MouseButtonEventArgs e)
{
if (sender is FrameworkElement { DataContext: MainViewModel vm })
vm.OpenMicPickerCommand.Execute(null);
}
private void VolumeSlider_PreviewMouseLeftButtonDown(object sender, MouseButtonEventArgs e) private void VolumeSlider_PreviewMouseLeftButtonDown(object sender, MouseButtonEventArgs e)
{ {
if (sender is FrameworkElement { DataContext: MainViewModel vm }) if (sender is FrameworkElement { DataContext: MainViewModel vm })
@@ -107,6 +101,24 @@ public partial class MainWindow : Window
vm.SetVolumeAdjusting(false); vm.SetVolumeAdjusting(false);
} }
private void GameVolumeSlider_PreviewMouseLeftButtonDown(object sender, MouseButtonEventArgs e)
{
if (sender is FrameworkElement { DataContext: MainViewModel vm })
vm.SetGameVolumeAdjusting(true);
}
private void GameVolumeSlider_PreviewMouseLeftButtonUp(object sender, MouseButtonEventArgs e)
{
if (sender is FrameworkElement { DataContext: MainViewModel vm })
vm.SetGameVolumeAdjusting(false);
}
private void GameVolumeSlider_LostMouseCapture(object sender, MouseEventArgs e)
{
if (sender is FrameworkElement { DataContext: MainViewModel vm })
vm.SetGameVolumeAdjusting(false);
}
private void MicSpeaker_MouseLeftButtonUp(object sender, MouseButtonEventArgs e) private void MicSpeaker_MouseLeftButtonUp(object sender, MouseButtonEventArgs e)
{ {
if (sender is FrameworkElement { DataContext: MainViewModel vm }) if (sender is FrameworkElement { DataContext: MainViewModel vm })
@@ -115,6 +127,14 @@ public partial class MainWindow : Window
} }
} }
private void GameSpeaker_MouseLeftButtonUp(object sender, MouseButtonEventArgs e)
{
if (sender is FrameworkElement { DataContext: MainViewModel vm })
{
vm.ToggleGameMuteCommand.Execute(null);
}
}
private void GearButton_Click(object sender, RoutedEventArgs e) private void GearButton_Click(object sender, RoutedEventArgs e)
{ {
if (sender is Button { ContextMenu: { } menu } button) if (sender is Button { ContextMenu: { } menu } button)
+13
View File
@@ -0,0 +1,13 @@
namespace ytLive.Models;
/// <summary>Footer mic indicator state (the dot beside the MIC button):
/// <see cref="Connected"/> = mic capture is running; <see cref="Problem"/> = a
/// mic was requested but the connection failed (device in use, unplugged, or
/// unavailable); <see cref="NotConnected"/> = no capture is active (e.g. no
/// mic device present).</summary>
public enum MicStatus
{
NotConnected,
Connected,
Problem
}
+1
View File
@@ -13,6 +13,7 @@ Plain data types. No logic beyond what a property can carry. See
| `WebcamSceneConfig.cs` | Per-scene webcam placement (subclass of `SceneElement`): geometry + `IsVisible` + border (`BorderColor`/`BorderOpacity`/`BorderWidth`/`BorderAnimation`) + `VideoImageSource`; `WebcamId` links to `Webcam` | | `WebcamSceneConfig.cs` | Per-scene webcam placement (subclass of `SceneElement`): geometry + `IsVisible` + border (`BorderColor`/`BorderOpacity`/`BorderWidth`/`BorderAnimation`) + `VideoImageSource`; `WebcamId` links to `Webcam` |
| `QualityOption.cs` | Resolution tier record `(Display, Fps, Bitrate, Width, Height)`; `Label` formats as `1080p60 (8Mbps)` — drives the bottom-bar dropdown and the output-rect math | | `QualityOption.cs` | Resolution tier record `(Display, Fps, Bitrate, Width, Height)`; `Label` formats as `1080p60 (8Mbps)` — drives the bottom-bar dropdown and the output-rect math |
| `StreamConfig.cs` | `StreamConfig` (note: `TargetBitrate`/`Resolution` defaults are stale — not yet wired to the dropdown), `StreamHealth` (bitrate/FPS/dropped/duration), `StreamStatus` enum | | `StreamConfig.cs` | `StreamConfig` (note: `TargetBitrate`/`Resolution` defaults are stale — not yet wired to the dropdown), `StreamHealth` (bitrate/FPS/dropped/duration), `StreamStatus` enum |
| `MicStatus.cs` | Footer mic indicator: `NotConnected` / `Connected` / `Problem` — the state behind the MIC button's status dot (red = no device/capture off, green = capturing, yellow = a requested mic failed: in use/unplugged) |
| `YouTube.cs` | `YouTubeChannel` and `ChatMessage` (chat feed) | | `YouTube.cs` | `YouTubeChannel` and `ChatMessage` (chat feed) |
| `Socials.cs` | **Social bar** (global layer, never a Source): `SocialService` enum (YouTube/Twitch/X/Instagram/TikTok/Facebook/Discord/Kick/Threads/Bluesky/GitHub/LinkedIn/Pinterest/Snapchat/Reddit/WhatsApp/Telegram/Link/Website/**Fediverse**), `SocialEntry` (Service/Handle/ProfileUrl/`FediverseSoftware`), `SocialsConfig` (Entries + `BarPosition` Top/Bottom + `BarEnabled` on/off — `BarJustify` dropped, column back-compat), `SocialServiceIcons` (canonical URL builder + `DetectService` (URL domain / fediverse `@user@domain`**Fediverse** / bare→Website) + bundled SVG logo path data per service (`LogoDataFor`) + `LogoDataForFediverse(software)` mapping nodeinfo software names (mastodon/peertube/pixelfed/misskey/lemmy/pleroma/firefish) to logos with a generic fediverse glyph fallback (`FediverseIconData`) + `LockedIconData`/`DoNotIconData` — initials/colors gone) | | `Socials.cs` | **Social bar** (global layer, never a Source): `SocialService` enum (YouTube/Twitch/X/Instagram/TikTok/Facebook/Discord/Kick/Threads/Bluesky/GitHub/LinkedIn/Pinterest/Snapchat/Reddit/WhatsApp/Telegram/Link/Website/**Fediverse**), `SocialEntry` (Service/Handle/ProfileUrl/`FediverseSoftware`), `SocialsConfig` (Entries + `BarPosition` Top/Bottom + `BarEnabled` on/off — `BarJustify` dropped, column back-compat), `SocialServiceIcons` (canonical URL builder + `DetectService` (URL domain / fediverse `@user@domain`**Fediverse** / bare→Website) + bundled SVG logo path data per service (`LogoDataFor`) + `LogoDataForFediverse(software)` mapping nodeinfo software names (mastodon/peertube/pixelfed/misskey/lemmy/pleroma/firefish) to logos with a generic fediverse glyph fallback (`FediverseIconData`) + `LockedIconData`/`DoNotIconData` — initials/colors gone) |
+46 -5
View File
@@ -4,15 +4,18 @@ namespace ytLive.Services.Audio;
/// <summary> /// <summary>
/// Owns the mic + desktop/game capture sources (TASK 4 ship step 4) and drives /// Owns the mic + desktop/game capture sources (TASK 4 ship step 4) and drives
/// the live mic meter. Runs only while live: started when go-live succeeds, /// the footer meters. Capture runs for the app's lifetime (started once at
/// stopped on end-stream. Mic samples are level-metered and forwarded; loopback /// startup, stopped on shutdown) so both bars stay live in preview: mic samples
/// samples are currently dropped (consumed by the encoder mix in a later step). /// are level-metered and forwarded, loopback samples feed the game bar's meter.
/// The mixer surfaces mic connection state (Connected/Failed) for the status
/// dot and can restart the mic source mid-session when a device is re-picked.
/// </summary> /// </summary>
public sealed class AudioMixer : IDisposable public sealed class AudioMixer : IDisposable
{ {
private readonly IAudioSource _mic; private readonly IAudioSource _mic;
private readonly IAudioSource _loopback; private readonly IAudioSource _loopback;
private readonly AudioLevelMeter _meter; private readonly AudioLevelMeter _meter;
private readonly AudioLevelMeter _loopbackMeter;
private readonly Action<string>? _log; private readonly Action<string>? _log;
private bool _started; private bool _started;
@@ -21,8 +24,10 @@ public sealed class AudioMixer : IDisposable
_mic = mic; _mic = mic;
_loopback = loopback; _loopback = loopback;
_meter = new AudioLevelMeter(); _meter = new AudioLevelMeter();
_loopbackMeter = new AudioLevelMeter();
_log = log; _log = log;
_mic.Started += OnMicStarted;
_mic.SampleReady += OnMicSample; _mic.SampleReady += OnMicSample;
_loopback.SampleReady += OnLoopbackSample; _loopback.SampleReady += OnLoopbackSample;
_mic.Failed += OnMicFailed; _mic.Failed += OnMicFailed;
@@ -32,9 +37,21 @@ public sealed class AudioMixer : IDisposable
/// <summary>Current smoothed mic level (0..1).</summary> /// <summary>Current smoothed mic level (0..1).</summary>
public float MicLevel => _meter.Level; public float MicLevel => _meter.Level;
/// <summary>Current smoothed desktop/game level (0..1).</summary>
public float LoopbackLevel => _loopbackMeter.Level;
/// <summary>Raised whenever the smoothed mic level changes.</summary> /// <summary>Raised whenever the smoothed mic level changes.</summary>
public event Action<float>? MicLevelChanged; public event Action<float>? MicLevelChanged;
/// <summary>Raised whenever the smoothed desktop/game level changes.</summary>
public event Action<float>? LoopbackLevelChanged;
/// <summary>Raised when the mic capture comes up (the status dot goes green).</summary>
public event Action? MicConnected;
/// <summary>Raised when the mic capture fails or dies (the status dot goes yellow).</summary>
public event Action<Exception>? MicFailed;
public void Start() public void Start()
{ {
if (_started) if (_started)
@@ -46,6 +63,17 @@ public sealed class AudioMixer : IDisposable
_mic.Start(); _mic.Start();
} }
/// <summary>Swaps the mic source without touching loopback — used when the
/// creator picks a different device mid-session. The level resets and the
/// new source raises <see cref="MicConnected"/> or <see cref="MicFailed"/>.</summary>
public void RestartMic()
{
_mic.Stop();
_meter.Reset();
MicLevelChanged?.Invoke(0);
_mic.Start();
}
public void Stop() public void Stop()
{ {
if (!_started) if (!_started)
@@ -55,12 +83,15 @@ public sealed class AudioMixer : IDisposable
_mic.Stop(); _mic.Stop();
_loopback.Stop(); _loopback.Stop();
_meter.Reset(); _meter.Reset();
_loopbackMeter.Reset();
MicLevelChanged?.Invoke(0); MicLevelChanged?.Invoke(0);
LoopbackLevelChanged?.Invoke(0);
} }
public void Dispose() public void Dispose()
{ {
Stop(); Stop();
_mic.Started -= OnMicStarted;
_mic.SampleReady -= OnMicSample; _mic.SampleReady -= OnMicSample;
_loopback.SampleReady -= OnLoopbackSample; _loopback.SampleReady -= OnLoopbackSample;
_mic.Failed -= OnMicFailed; _mic.Failed -= OnMicFailed;
@@ -69,20 +100,30 @@ public sealed class AudioMixer : IDisposable
_loopback.Dispose(); _loopback.Dispose();
} }
private void OnMicStarted()
{
MicConnected?.Invoke();
}
private void OnMicSample(AudioSample sample) private void OnMicSample(AudioSample sample)
{ {
MicLevelChanged?.Invoke(_meter.Push(sample)); // Push unconditionally: the ?. on the event would otherwise skip the
// argument (and the meter update) when nothing is subscribed yet.
var level = _meter.Push(sample);
MicLevelChanged?.Invoke(level);
} }
private void OnLoopbackSample(AudioSample sample) private void OnLoopbackSample(AudioSample sample)
{ {
// Desktop/game audio: captured for the future encoder mix; no UI yet. var level = _loopbackMeter.Push(sample);
LoopbackLevelChanged?.Invoke(level);
} }
private void OnMicFailed(Exception ex) private void OnMicFailed(Exception ex)
{ {
_log?.Invoke($"Mic capture failed: {ex.Message}"); _log?.Invoke($"Mic capture failed: {ex.Message}");
MicLevelChanged?.Invoke(0); MicLevelChanged?.Invoke(0);
MicFailed?.Invoke(ex);
} }
private void OnLoopbackFailed(Exception ex) private void OnLoopbackFailed(Exception ex)
+8 -3
View File
@@ -2,9 +2,10 @@ namespace ytLive.Services.Audio;
/// <summary> /// <summary>
/// A live audio capture source (TASK 4 ship step 4 seam): produces PCM float /// A live audio capture source (TASK 4 ship step 4 seam): produces PCM float
/// chunks, runs only while live. The default implementations wrap NAudio's /// chunks. The default implementations wrap NAudio's WASAPI capture (mic) /
/// WASAPI capture (mic) / loopback (desktop/game); the mixer and the tests /// loopback (desktop/game); the mixer and the tests consume this interface,
/// consume this interface, never NAudio directly. /// never NAudio directly. Capture now runs for the app's lifetime so the
/// footer meters stay live in preview — the mixer owns start/stop.
/// </summary> /// </summary>
public interface IAudioSource : IDisposable public interface IAudioSource : IDisposable
{ {
@@ -14,6 +15,10 @@ public interface IAudioSource : IDisposable
/// <summary>Stops capturing; a later Start begins a fresh session.</summary> /// <summary>Stops capturing; a later Start begins a fresh session.</summary>
void Stop(); void Stop();
/// <summary>Raises once capture is live (right after recording starts).
/// Never raised when Start fails — <see cref="Failed"/> fires instead.</summary>
event Action? Started;
/// <summary>Raises each captured chunk (interleaved PCM float, -1..1).</summary> /// <summary>Raises each captured chunk (interleaved PCM float, -1..1).</summary>
event Action<AudioSample>? SampleReady; event Action<AudioSample>? SampleReady;
+4 -1
View File
@@ -4,12 +4,14 @@ namespace ytLive.Services.Audio;
/// <summary> /// <summary>
/// Captures the user's desktop/game audio (TASK 4 ship step 4) via WASAPI /// Captures the user's desktop/game audio (TASK 4 ship step 4) via WASAPI
/// loopback on the default render device. Starts/stops with go-live only. /// loopback on the default render device. Runs for the app's lifetime so the
/// game audio bar stays live in preview.
/// </summary> /// </summary>
public sealed class WasapiLoopbackAudioSource : IAudioSource public sealed class WasapiLoopbackAudioSource : IAudioSource
{ {
private WasapiLoopbackCapture? _capture; private WasapiLoopbackCapture? _capture;
public event Action? Started;
public event Action<AudioSample>? SampleReady; public event Action<AudioSample>? SampleReady;
public event Action<Exception>? Failed; public event Action<Exception>? Failed;
@@ -24,6 +26,7 @@ public sealed class WasapiLoopbackAudioSource : IAudioSource
_capture.DataAvailable += OnDataAvailable; _capture.DataAvailable += OnDataAvailable;
_capture.RecordingStopped += OnRecordingStopped; _capture.RecordingStopped += OnRecordingStopped;
_capture.StartRecording(); _capture.StartRecording();
Started?.Invoke();
} }
catch (Exception ex) catch (Exception ex)
{ {
+4 -1
View File
@@ -14,12 +14,14 @@ public sealed class WasapiMicAudioSource : IAudioSource
private WasapiCapture? _capture; private WasapiCapture? _capture;
/// <param name="micNameProvider">Returns the current mic DisplayName; read /// <param name="micNameProvider">Returns the current mic DisplayName; read
/// at each Start so a device picked mid-session takes effect next go-live.</param> /// at each Start so a device picked mid-session takes effect immediately
/// (the mixer restarts the mic on pick).</param>
public WasapiMicAudioSource(Func<string?> micNameProvider) public WasapiMicAudioSource(Func<string?> micNameProvider)
{ {
_micNameProvider = micNameProvider; _micNameProvider = micNameProvider;
} }
public event Action? Started;
public event Action<AudioSample>? SampleReady; public event Action<AudioSample>? SampleReady;
public event Action<Exception>? Failed; public event Action<Exception>? Failed;
@@ -35,6 +37,7 @@ public sealed class WasapiMicAudioSource : IAudioSource
_capture.DataAvailable += OnDataAvailable; _capture.DataAvailable += OnDataAvailable;
_capture.RecordingStopped += OnRecordingStopped; _capture.RecordingStopped += OnRecordingStopped;
_capture.StartRecording(); _capture.StartRecording();
Started?.Invoke();
} }
catch (Exception ex) catch (Exception ex)
{ {
+46
View File
@@ -0,0 +1,46 @@
namespace ytLive.Services;
/// <summary>
/// Default <see cref="IGameAudioDetector"/>: samples a full-screen monitor
/// provider (the existing <c>IFullScreenDetector</c>) and the live loopback
/// level and advances a <see cref="GameAudioHysteresis"/>. The pure transitions
/// live in GameAudioHysteresis (unit-tested); this wrapper owns the providers
/// and the raised-changed event. WPF-free — the VM owns the poll timer.
/// </summary>
public sealed class GameAudioDetector : IGameAudioDetector
{
private const float SoundFloor = 0.005f;
private readonly Func<int?> _foregroundFullScreenMonitorProvider;
private readonly Func<float> _loopbackLevelProvider;
private readonly Func<DateTime> _now;
private readonly GameAudioHysteresis _hysteresis = new();
/// <param name="foregroundFullScreenMonitorProvider">Returns the monitor a
/// full-screen foreground window covers, or null when windowed/none.</param>
/// <param name="loopbackLevelProvider">Current smoothed desktop/game level (0..1).</param>
/// <param name="now">Clock for the hysteresis windows; injectable for tests.</param>
public GameAudioDetector(
Func<int?> foregroundFullScreenMonitorProvider,
Func<float> loopbackLevelProvider,
Func<DateTime>? now = null)
{
_foregroundFullScreenMonitorProvider = foregroundFullScreenMonitorProvider;
_loopbackLevelProvider = loopbackLevelProvider;
_now = now ?? (() => DateTime.Now);
}
public bool IsGameAudioActive => _hysteresis.IsActive;
public event Action<bool>? IsGameAudioActiveChanged;
public void Poll()
{
var before = _hysteresis.IsActive;
var isFullScreen = _foregroundFullScreenMonitorProvider() != null;
var hasSound = _loopbackLevelProvider() > SoundFloor;
_hysteresis.Update(isFullScreen, hasSound, _now());
if (before != _hysteresis.IsActive)
IsGameAudioActiveChanged?.Invoke(_hysteresis.IsActive);
}
}
+57
View File
@@ -0,0 +1,57 @@
namespace ytLive.Services;
/// <summary>
/// Pure show/hide state machine for the game audio bar (TASK 4 game audio).
/// SHOW: a full-screen foreground app has been producing desktop audio for at
/// least <see cref="ShowAfterSound"/> — "a working game with sound". HIDE: the
/// app leaves fullscreen/foreground for <see cref="HideAfterWindowed"/> — the
/// game is no longer up in the preview. Silence NEVER hides an active bar; it
/// only matters for the initial show. No timers; <see cref="Update"/> is fed by
/// the caller with a clock.
/// </summary>
public sealed class GameAudioHysteresis
{
private readonly TimeSpan _showAfterSound = TimeSpan.FromMilliseconds(500);
private readonly TimeSpan _hideAfterWindowed = TimeSpan.FromSeconds(1);
private DateTime? _soundSince;
private DateTime? _windowedSince;
public bool IsActive { get; private set; }
public void Update(bool isFullScreen, bool hasSound, DateTime now)
{
if (isFullScreen)
{
_windowedSince = null;
if (IsActive)
return;
if (!hasSound)
{
_soundSince = null;
return;
}
_soundSince ??= now;
if (now - _soundSince >= _showAfterSound)
IsActive = true;
}
else
{
_soundSince = null;
if (!IsActive)
{
_windowedSince = null;
return;
}
_windowedSince ??= now;
if (now - _windowedSince >= _hideAfterWindowed)
{
IsActive = false;
_windowedSince = null;
}
}
}
}
+20
View File
@@ -0,0 +1,20 @@
namespace ytLive.Services;
/// <summary>
/// Detects "a working game with sound is up in the preview" (TASK 4 game audio
/// bar): becomes active once a full-screen foreground app is producing desktop
/// audio, and stays active as long as that full-screen app remains — silence
/// never hides the bar, only the game leaving fullscreen/foreground does. Seam
/// so the VM and tests never touch Win32 or audio interop directly.
/// </summary>
public interface IGameAudioDetector
{
/// <summary>True while the game audio bar should be visible.</summary>
bool IsGameAudioActive { get; }
/// <summary>Raised when the bar should appear or disappear.</summary>
event Action<bool>? IsGameAudioActiveChanged;
/// <summary>Samples the injected providers and advances the state machine.</summary>
void Poll();
}
+8 -5
View File
@@ -44,13 +44,16 @@ External-facing logic: YouTube API, persistence. See
| `Encoder/FfmpegEncoderPicker.cs` | Pure H.264 encoder picker from `-encoders` output: NVENC → QSV → AMF → OpenH264; **never returns libx264** (GPL) | | `Encoder/FfmpegEncoderPicker.cs` | Pure H.264 encoder picker from `-encoders` output: NVENC → QSV → AMF → OpenH264; **never returns libx264** (GPL) |
| `Encoder/FramePump.cs` | **TASK 4 ship step 5**: the live frame producer — while live it snapshots the active scene each tick, resolves every element to its latest frame (`Func<SceneElement, VideoFrame?>` resolver), composites it into the tier's output frame, and paces frames into the encoder at the tier's FPS. All collaborators constructor-injected seams; free of WPF and the capture managers. `StartAsync` never throws (failures log + `Failed`); no RTMP URL = encoder skipped; `StopAsync` stops the encoder before awaiting the loop (backpressure deadlock); `ProcessFailed` self-stops. Since the bar bug-fix branch it takes an optional `socialBar: Func<(VideoFrame? Frame, SocialBarPosition Position)>?` seam — a pre-rendered bar strip composited last (above the flash) at the top edge or `SourceRectHeight bar height`. See `ai.md` "Live frame pipeline" | | `Encoder/FramePump.cs` | **TASK 4 ship step 5**: the live frame producer — while live it snapshots the active scene each tick, resolves every element to its latest frame (`Func<SceneElement, VideoFrame?>` resolver), composites it into the tier's output frame, and paces frames into the encoder at the tier's FPS. All collaborators constructor-injected seams; free of WPF and the capture managers. `StartAsync` never throws (failures log + `Failed`); no RTMP URL = encoder skipped; `StopAsync` stops the encoder before awaiting the loop (backpressure deadlock); `ProcessFailed` self-stops. Since the bar bug-fix branch it takes an optional `socialBar: Func<(VideoFrame? Frame, SocialBarPosition Position)>?` seam — a pre-rendered bar strip composited last (above the flash) at the top edge or `SourceRectHeight bar height`. See `ai.md` "Live frame pipeline" |
| `Encoder/FfmpegLocator.cs` | The default locator: PATH probe wins, then the cache, then pull+extract (`ffmpeg.exe` **+ the `libav*.dll` family** via a staging dir, failures via `AppLog`). Pinned to BtbN `autobuild-2026-08-09-13-03` **lgpl-shared** (dynamic linking = LGPL compliance without static-relink §6 material; no GPL libx264; NVENC/QSV/AMF + libopenh264 + native AAC — see `ai.md` → Licensing). Search dirs / tools dir / downloader constructor-injected for hermetic tests. Wired into the encoder since ship step 5 | | `Encoder/FfmpegLocator.cs` | The default locator: PATH probe wins, then the cache, then pull+extract (`ffmpeg.exe` **+ the `libav*.dll` family** via a staging dir, failures via `AppLog`). Pinned to BtbN `autobuild-2026-08-09-13-03` **lgpl-shared** (dynamic linking = LGPL compliance without static-relink §6 material; no GPL libx264; NVENC/QSV/AMF + libopenh264 + native AAC — see `ai.md` → Licensing). Search dirs / tools dir / downloader constructor-injected for hermetic tests. Wired into the encoder since ship step 5 |
| `Audio/IAudioSource.cs` | **TASK 4 ship step 4 seam**: live capture source — `Start`/`Stop`/`SampleReady(AudioSample)`/`Failed(Exception)`, `IDisposable`; runs only while live. The app consumes this seam; tests inject hermetic fakes | | `Audio/IAudioSource.cs` | **TASK 4 ship step 4 seam**: live capture source — `Start`/`Stop`/`Started`/`SampleReady(AudioSample)`/`Failed(Exception)`, `IDisposable`. Capture now runs for the app's lifetime (started at startup) so the footer meters preview live. The app consumes this seam; tests inject hermetic fakes |
| `Audio/AudioSample.cs` | One captured chunk: interleaved PCM float (-1..1) + sample rate + channels | | `Audio/AudioSample.cs` | One captured chunk: interleaved PCM float (-1..1) + sample rate + channels |
| `Audio/WasapiLoopbackAudioSource.cs` | Desktop/game capture: NAudio `WasapiLoopbackCapture` on the default render device — automatic at unity, zero UI | | `Audio/WasapiLoopbackAudioSource.cs` | Desktop/game capture: NAudio `WasapiLoopbackCapture` on the default render device — automatic at unity; the game bar's meter consumes it |
| `Audio/WasapiMicAudioSource.cs` | Mic capture: NAudio `WasapiCapture` with the device resolved by `FriendlyName` matching `MicSourceName` (DisplayName only), default capture endpoint fallback; name provider `Func<string?>` re-read at each `Start` so a mic picked mid-session takes effect next go-live | | `Audio/WasapiMicAudioSource.cs` | Mic capture: NAudio `WasapiCapture` with the device resolved by `FriendlyName` matching `MicSourceName` (DisplayName only), default capture endpoint fallback; name provider `Func<string?>` re-read at each `Start` so a device picked mid-session takes effect immediately (the mixer restarts the mic on pick) |
| `Audio/AudioMixer.cs` | Owns both sources; `Start`/`Stop` follow go-live (`MainViewModel.BeginGoLive`/`StopStream`). Mic samples → `AudioLevelMeter``MicLevelChanged`; loopback samples currently dropped (the future encoder AAC mix consumes them). Failures log via `AppLog`; mic failure zeroes the meter, loopback failure doesn't kill the mic | | `Audio/AudioMixer.cs` | Owns both sources; capture starts once at startup (`MainViewModel.StartMicCaptureAsync`) and stops on `Shutdown` — NOT go-live (preview monitoring). Mic samples → `AudioLevelMeter``MicLevelChanged`; loopback samples → the game bar's meter via `LoopbackLevelChanged`. Surfaces mic connection state: `MicConnected`/`MicFailed` (drives the status dot) + `RestartMic()` (device swap mid-session, keeps loopback). Failures log via `AppLog`; mic failure zeroes the meter, loopback failure doesn't kill the mic. Note: the meter `Push` is unconditional (the `?.` on the event would otherwise skip the argument when nothing is subscribed) |
| `Audio/AudioLevelMeter.cs` | Pure smoothed RMS level (0..1): `Push(AudioSample)` + `Reset` — the unit-tested math behind `AudioLevel` | | `Audio/AudioLevelMeter.cs` | Pure smoothed RMS level (0..1): `Push(AudioSample)` + `Reset` — the unit-tested math behind `AudioLevel` and `GameAudioLevel` |
| `Audio/WaveToFloat.cs` | Pure WASAPI buffer → float conversion: IEEE float 32-bit direct, PCM 16-bit normalized, `WaveFormatExtensible` IEEE-float subformat GUID, trailing partial samples ignored | | `Audio/WaveToFloat.cs` | Pure WASAPI buffer → float conversion: IEEE float 32-bit direct, PCM 16-bit normalized, `WaveFormatExtensible` IEEE-float subformat GUID, trailing partial samples ignored |
| `IGameAudioDetector.cs` | **TASK 4 game audio bar seam**: `IsGameAudioActive` + `IsGameAudioActiveChanged` + `Poll()` — detects "a working game with sound is up in the preview" |
| `GameAudioHysteresis.cs` | Pure show/hide state machine for the game bar: SHOW = full-screen app holds sound ~500ms; HIDE = the app leaves fullscreen ~1s. **Silence never hides an active bar** — only the game leaving the preview does (creator's rule). No timers; `Update(isFullScreen, hasSound, now)` |
| `GameAudioDetector.cs` | Default `IGameAudioDetector`: composes a full-screen monitor provider (`IFullScreenDetector.GetForegroundFullScreenMonitorIndex`) + the live loopback level (floor 0.5%) into a `GameAudioHysteresis`. WPF-free; the VM owns the 250ms poll timer |
Related: constructed in [`ViewModels/MainViewModel.cs`](../ViewModels/MainViewModel.cs) Related: constructed in [`ViewModels/MainViewModel.cs`](../ViewModels/MainViewModel.cs)
(no DI container yet). Models in [`Models/index.md`](../Models/index.md). (no DI container yet). Models in [`Models/index.md`](../Models/index.md).
+5 -3
View File
@@ -106,7 +106,7 @@ Lifecycle: `created → ready → [testing] → live → complete` (transitional
9.**"Add Webcam" always opens the camera picker** — deleting one scene's webcam then re-adding used to resurrect the old camera when another scene still used it — `SwapWebcamIdentityAsync` now swaps the app-wide identity if a different camera is chosen, same path as "Change Webcam…" 9.**"Add Webcam" always opens the camera picker** — deleting one scene's webcam then re-adding used to resurrect the old camera when another scene still used it — `SwapWebcamIdentityAsync` now swaps the app-wide identity if a different camera is chosen, same path as "Change Webcam…"
10.**Webcam resource validation + first-frame proof**`MediaCaptureFrameSource` validates post-init (VideoDeviceId match, stream properties ≥1, `reader.StartAsync()` status read + throws on non-Success); subscribes `capture.Failed` + `CameraStreamStateChanged``SourceFailed` event on the seam; fallback ladder (VideoPreview → VideoRecord). `CameraManager.AcquireAsync` requires first-frame proof (4s timeout): returns true only after a real frame arrives — silent empty box impossible. `MainViewModel` subscribes `CameraFailed` → red `WebcamError` chip in preview + MessageBox names suspect apps (`CameraConflictProbe`). 19041 SDK projection gaps: `Exclusive`/`DeviceLost` not projected; `CameraStreamState.Failed` compared by `(int)2`. 81 tests passing 10.**Webcam resource validation + first-frame proof**`MediaCaptureFrameSource` validates post-init (VideoDeviceId match, stream properties ≥1, `reader.StartAsync()` status read + throws on non-Success); subscribes `capture.Failed` + `CameraStreamStateChanged``SourceFailed` event on the seam; fallback ladder (VideoPreview → VideoRecord). `CameraManager.AcquireAsync` requires first-frame proof (4s timeout): returns true only after a real frame arrives — silent empty box impossible. `MainViewModel` subscribes `CameraFailed` → red `WebcamError` chip in preview + MessageBox names suspect apps (`CameraConflictProbe`). 19041 SDK projection gaps: `Exclusive`/`DeviceLost` not projected; `CameraStreamState.Failed` compared by `(int)2`. 81 tests passing
11.**Scenes/sources UI** — add/reorder/rename, image + background overlays with move/resize/opacity/reuse 11.**Scenes/sources UI** — add/reorder/rename, image + background overlays with move/resize/opacity/reuse
12.**Audio UX shipped (UI)** — the bottom-bar footer is now two lines (dropped/duration moved under bitrate/fps), with the mic's sound meter + mute button + volume slider grouped CENTERED on the footer's top line, beneath the preview panel (meter: 288px, muted slate track with ruler graduations + muted yellow/red zone tints, green→yellow→red fill; mute = speaker icon → red do-not-symbol when muted, and the slider and speaker stay in sync (volume 0 ⇔ muted — sliding off flips the speaker to muted, sliding up from 0 clears it); mic volume defaults to 80%, muting zeroes the meter and restores the prior volume on unmute (which flashes the meter to the restored position ~300ms before it returns to the live level); the meter is a READ-ONLY realtime level display (fill = live level × volume — volume is a gain on ambient noise; while the slider is dragged the bar previews the slider position and bounces back to the live level on release, which is 0 with no input — clicking the meter does nothing), clicking the MIC label opens a microphone picker whose chosen source shows left-justified inside the meter bar (fill at 75% opacity so the name + ruler markings show through); slim dimensional slider — gradient track/fill, gloss-sphere thumb; the old flat pink 18px-filled one is gone), everything else on line 2 (bitrate/fps/dropped/duration/health left, quality + gear right) — the creator's only audio control, desktop/game audio is automatic (KISS rule) 12.**Audio UX shipped (UI)** — the bottom-bar footer is now two lines (dropped/duration moved under bitrate/fps), with the mic's sound meter + mute button + volume slider grouped CENTERED on the footer's top line, beneath the preview panel (meter: 288px, muted slate track with ruler graduations + muted yellow/red zone tints, green→yellow→red fill; mute = speaker icon → red do-not-symbol when muted, and the slider and speaker stay in sync (volume 0 ⇔ muted — sliding off flips the speaker to muted, sliding up from 0 clears it); mic volume defaults to 80%, muting zeroes the meter and restores the prior volume on unmute (which flashes the meter to the restored position ~300ms before it returns to the live level); the meter is a READ-ONLY realtime level display (fill = live level × volume — volume is a gain on ambient noise; while the slider is dragged the bar previews the slider position and bounces back to the live level on release, which is 0 with no input — clicking the meter does nothing), clicking the MIC label opens a microphone picker whose chosen source shows left-justified inside the meter bar (fill at 75% opacity so the name + ruler markings show through); slim dimensional slider — gradient track/fill, gloss-sphere thumb; the old flat pink 18px-filled one is gone), everything else on line 2 (bitrate/fps/dropped/duration/health left, quality + gear right) — the creator's only audio control, desktop/game audio is automatic (KISS rule). **Post-test follow-up (2026-08-13, game audio bar branch):** the footer is now THREE lines — a **game audio bar** (desktop/game, a mirror of the mic bar: meter + mute + volume) sits centered beneath the mic bar and appears only while a **full-screen game is producing sound** in the preview (`IGameAudioDetector` seam + default `GameAudioDetector` polling `IFullScreenDetector` + the live loopback level, floor 0.5%, into a pure `GameAudioHysteresis`: SHOW after ~500ms of fullscreen+sound, HIDE ~1s after leaving fullscreen, **silence never hides an active bar**; the VM polls it on a 250ms `DispatcherTimer`); **capture now runs for the app's lifetime** (mixer started once at startup via `StartMicCaptureAsync`, disposed in `Shutdown` — not go-live) so both meters preview live; the **MIC label became a button with a status dot** (`Models/MicStatus`: green = connected via the source `Started` event, yellow = mic problem, red = no device); picking a mic takes effect immediately (`AudioMixer.RestartMic`, loopback keeps running); fixed a latent `?.Invoke(meter.Push(...))` short-circuit that skipped the meter update when nothing was subscribed. 167 tests passing, 0 warnings
13. ✅ The connected YouTube account's avatar/name shows in the top bar next to Start Stream (`SyncConnectedAccount`); the scenes list is content-height now (no dead space before SOURCES) 13. ✅ The connected YouTube account's avatar/name shows in the top bar next to Start Stream (`SyncConnectedAccount`); the scenes list is content-height now (no dead space before SOURCES)
14.**Social bar v2 (six-slot dialog, sign-in gate, real logos)** — global bar layer (never a Source, no sources-list row), content-sized, centered, GREEN glow when ON, top/bottom snap-drag (default BOTTOM, persisted `SocialBarPosition`; drag clamps to 0/1040, tie→bottom). Footer Social button gets a state dot (green=ON). Dialog "Social Media Site Promotion" (`SocialsDialog` + `ViewModels/SocialsDialogViewModel`, WPF-free + injected `ISocialValidator`/sign-in/sign-out fakes): ON/OFF bar switch (`SocialsConfig.BarEnabled`, schema v8), 6 fixed slots — row 1 always YouTube (signed-in → channel handle; signed-out → sign-in gate → OAuth; delete → confirm sign-out, mirrors `StopStream`), row 2 free, rows 3-6 lock icons on freemium (Premium seam: all six). Validation: `DetectService` (URL domain / fediverse `@user@domain` / bare→Website) → async `ISocialValidator` on confirm/Save; valid snaps to text + real service logo (bundled SVG path data via `LogoDataFor`, Simple Icons CC0 — initials badges gone); invalid → red do-not, stays editable, Save blocked. LCR justify dropped (`BarJustify` unread), per-scene toggle dropped (`Scene.HasSocialBar` back-compat). **Post-test fixes (2026-08-12):** footer label "Socials" (not "Social"); fediverse `@user@domain` validates — the full handle is the identity end-to-end (`DetectService`/`CanonicalUrlFor`/`HttpSocialValidator` build `https://domain/@user`, no domain loss); **Cancel is a hard stop**`ISocialValidator.LookupAsync` takes a `CancellationToken`, dialog VM owns a CTS, Cancel/X/Save abort in-flight lookups (HTTP request killed, canceled continuations never touch slot state), and `ConfirmEdit` skips re-submitting identical text (LostFocus on dismiss never re-fires a lookup). `HttpSocialValidator` now has real tests (fake `HttpMessageHandler`). 105 tests passing. **Post-test fixes (2026-08-12, round 2):** fediverse `@user@domain` no longer shows a generic chain — it resolves to the instance's actual software via nodeinfo (`/.well-known/nodeinfo``software.name`; `SocialService.Fediverse` enum member + `SocialEntry.FediverseSoftware` persisted in a new `SocialEntry.Software` column, schema migration by column-presence) and renders that software's bundled logo (`LogoDataForFediverse`: mastodon/peertube/pixelfed/misskey/lemmy/pleroma/firefish, generic fediverse honeycomb fallback). Dialog row-2 edit/trash icons were too dark — `IconButton` style gains `Foreground=#d0d0d0`; trash overrides `#e94560` (app red). 112 tests passing. **Post-test fixes (2026-08-12, round 3):** a fediverse handle whose identity domain is itself a redirect (e.g. YunoHost default-app subdomains — `@user@llamachile.tube` where the mastodon instance lives at `mastodon.llamachile.tube`) now still resolves its software: nodeinfo on the identity domain is SSO-blocked, so `HttpSocialValidator` follows the bare root `https://domain/` 302 to the real instance host and re-runs the nodeinfo lookup there. 14.**Social bar v2 (six-slot dialog, sign-in gate, real logos)** — global bar layer (never a Source, no sources-list row), content-sized, centered, GREEN glow when ON, top/bottom snap-drag (default BOTTOM, persisted `SocialBarPosition`; drag clamps to 0/1040, tie→bottom). Footer Social button gets a state dot (green=ON). Dialog "Social Media Site Promotion" (`SocialsDialog` + `ViewModels/SocialsDialogViewModel`, WPF-free + injected `ISocialValidator`/sign-in/sign-out fakes): ON/OFF bar switch (`SocialsConfig.BarEnabled`, schema v8), 6 fixed slots — row 1 always YouTube (signed-in → channel handle; signed-out → sign-in gate → OAuth; delete → confirm sign-out, mirrors `StopStream`), row 2 free, rows 3-6 lock icons on freemium (Premium seam: all six). Validation: `DetectService` (URL domain / fediverse `@user@domain` / bare→Website) → async `ISocialValidator` on confirm/Save; valid snaps to text + real service logo (bundled SVG path data via `LogoDataFor`, Simple Icons CC0 — initials badges gone); invalid → red do-not, stays editable, Save blocked. LCR justify dropped (`BarJustify` unread), per-scene toggle dropped (`Scene.HasSocialBar` back-compat). **Post-test fixes (2026-08-12):** footer label "Socials" (not "Social"); fediverse `@user@domain` validates — the full handle is the identity end-to-end (`DetectService`/`CanonicalUrlFor`/`HttpSocialValidator` build `https://domain/@user`, no domain loss); **Cancel is a hard stop**`ISocialValidator.LookupAsync` takes a `CancellationToken`, dialog VM owns a CTS, Cancel/X/Save abort in-flight lookups (HTTP request killed, canceled continuations never touch slot state), and `ConfirmEdit` skips re-submitting identical text (LostFocus on dismiss never re-fires a lookup). `HttpSocialValidator` now has real tests (fake `HttpMessageHandler`). 105 tests passing. **Post-test fixes (2026-08-12, round 2):** fediverse `@user@domain` no longer shows a generic chain — it resolves to the instance's actual software via nodeinfo (`/.well-known/nodeinfo``software.name`; `SocialService.Fediverse` enum member + `SocialEntry.FediverseSoftware` persisted in a new `SocialEntry.Software` column, schema migration by column-presence) and renders that software's bundled logo (`LogoDataForFediverse`: mastodon/peertube/pixelfed/misskey/lemmy/pleroma/firefish, generic fediverse honeycomb fallback). Dialog row-2 edit/trash icons were too dark — `IconButton` style gains `Foreground=#d0d0d0`; trash overrides `#e94560` (app red). 112 tests passing. **Post-test fixes (2026-08-12, round 3):** a fediverse handle whose identity domain is itself a redirect (e.g. YunoHost default-app subdomains — `@user@llamachile.tube` where the mastodon instance lives at `mastodon.llamachile.tube`) now still resolves its software: nodeinfo on the identity domain is SSO-blocked, so `HttpSocialValidator` follows the bare root `https://domain/` 302 to the real instance host and re-runs the nodeinfo lookup there.
15.**Window capture** — absorbed into the Screen picker (no separate source type); dedicated window-as-source work is pending 15.**Window capture** — absorbed into the Screen picker (no separate source type); dedicated window-as-source work is pending
@@ -423,8 +423,10 @@ lifecycle/forwarding/failure against fakes, meter RMS/smoothing/reset, `WaveToFl
extensible/truncation) — **139 passing**. extensible/truncation) — **139 passing**.
**Deferred (later ship steps):** wiring the desktop-capture samples into the encoder's AAC mix (replaces **Deferred (later ship steps):** wiring the desktop-capture samples into the encoder's AAC mix (replaces
the `-f lavfi -i anullsrc` placeholder; the encoder construction itself shipped in ship step 5), WASAPI the `-f lavfi -i anullsrc` placeholder; the encoder construction itself shipped in ship step 5).
capture while not live, and any audio UI beyond the existing mic controls. *(The "capture while not live" + "audio UI beyond the mic controls" deferrals were SHIPPED on the
2026-08-13 game audio bar branch — capture is now always-on for preview and the game bar is the second
audio UI. The mixer's short-circuit meter fix + `Started`/`RestartMic` seams live in the same branch.)*
#### Ship step 5 — Frame-pipeline wiring (the encoder gets a frame source) #### Ship step 5 — Frame-pipeline wiring (the encoder gets a frame source)
+267 -7
View File
@@ -27,6 +27,8 @@ public class MainViewModel : ViewModelBase
private readonly YouTubeChatService _youtubeChat; private readonly YouTubeChatService _youtubeChat;
private readonly DispatcherTimer _liveTimer; private readonly DispatcherTimer _liveTimer;
private readonly DispatcherTimer _volumeFlashTimer; private readonly DispatcherTimer _volumeFlashTimer;
private readonly DispatcherTimer _gameVolumeFlashTimer;
private readonly DispatcherTimer _gameAudioTimer;
private Scene? _activeScene; private Scene? _activeScene;
private SceneElement? _selectedElement; private SceneElement? _selectedElement;
@@ -41,6 +43,14 @@ public class MainViewModel : ViewModelBase
private bool _micMuted; private bool _micMuted;
private double? _volumeBeforeMute; private double? _volumeBeforeMute;
private string? _micSourceName; private string? _micSourceName;
private MicStatus _micStatus = MicStatus.NotConnected;
private double _gameAudioLevel;
private bool _gameVolumeAdjusting;
private bool _gameVolumeFlash;
private double _gameVolume = 1.0;
private bool _gameMuted;
private double? _gameVolumeBeforeMute;
private bool _isGameAudioBarVisible;
private StreamStatus _streamStatus = StreamStatus.Offline; private StreamStatus _streamStatus = StreamStatus.Offline;
private StreamHealth _currentHealth = new(); private StreamHealth _currentHealth = new();
private string _streamTitle = string.Empty; private string _streamTitle = string.Empty;
@@ -84,9 +94,11 @@ public class MainViewModel : ViewModelBase
private readonly IMicrophoneEnumerator _microphoneEnumerator; private readonly IMicrophoneEnumerator _microphoneEnumerator;
// Live audio capture (TASK 4 ship step 4): desktop/game via WASAPI loopback, // Live audio capture (TASK 4 ship step 4): desktop/game via WASAPI loopback,
// mic via WASAPI capture, both owned by the mixer and running only while // mic via WASAPI capture, both owned by the mixer. Capture runs for the app's
// live. Mic level feeds AudioLevel (the meter); loopback is for the future // lifetime (started at startup, stopped on shutdown) so the footer meters
// encoder mix. Private by design — no UI beyond the existing mic controls. // stay live in preview. Mic level feeds AudioLevel (the meter); loopback
// feeds the game audio bar's meter. Private by design — the mixer surfaces
// the levels + mic connection state to the UI.
private readonly AudioMixer _audioMixer; private readonly AudioMixer _audioMixer;
// Live frame pipeline (TASK 4 ship step 5): composites the active scene at the // Live frame pipeline (TASK 4 ship step 5): composites the active scene at the
@@ -101,6 +113,7 @@ public class MainViewModel : ViewModelBase
private readonly IFullScreenDetector _fullScreenDetector; private readonly IFullScreenDetector _fullScreenDetector;
private readonly ScreenCaptureManager _screenCaptureManager; private readonly ScreenCaptureManager _screenCaptureManager;
private readonly ScreenCaptureSourceFactory _screenCaptureFactory; private readonly ScreenCaptureSourceFactory _screenCaptureFactory;
private readonly IGameAudioDetector _gameAudioDetector;
private int? _lastForegroundFullScreenMonitor; private int? _lastForegroundFullScreenMonitor;
private CancellationTokenSource? _deactivateCts; private CancellationTokenSource? _deactivateCts;
private ImageSource? _backdropImage; private ImageSource? _backdropImage;
@@ -272,8 +285,9 @@ public class MainViewModel : ViewModelBase
private set => SetProperty(ref _accountDisplayName, value); private set => SetProperty(ref _accountDisplayName, value);
} }
// ─── Audio (KISS: desktop/game audio is automatic — zero UI. The creator's // ─── Audio: the mic bar (meter + volume + mute + status dot) is always
// ─── only audio control is the mic: meter + volume + mute.) ─── // ─── visible; the game bar (meter + volume + mute) appears only while a
// ─── full-screen game is producing sound. Both meters preview live. ───
/// <summary>Live mic input level (0..1) — fed by the audio mixer once capture /// <summary>Live mic input level (0..1) — fed by the audio mixer once capture
/// lands; 0 with no input. Read by the meter, scaled by MicVolume.</summary> /// lands; 0 with no input. Read by the meter, scaled by MicVolume.</summary>
@@ -409,7 +423,170 @@ public class MainViewModel : ViewModelBase
Owner = System.Windows.Application.Current?.MainWindow Owner = System.Windows.Application.Current?.MainWindow
}; };
if (dialog.ShowDialog() == true && dialog.PickedDevice != null) if (dialog.ShowDialog() == true && dialog.PickedDevice != null)
{
MicSourceName = dialog.PickedDevice.DisplayName; MicSourceName = dialog.PickedDevice.DisplayName;
_audioMixer.RestartMic(); // swap the live device immediately
}
}
// ─── Mic status dot (the MIC button) ───
private static readonly SolidColorBrush MicProblemBrush = CreateBrush("#f1c40f");
/// <summary>Mic connection state, driven by the mixer's MicConnected/MicFailed
/// events and the startup device check (see StartMicCaptureAsync).</summary>
public MicStatus MicStatus
{
get => _micStatus;
private set
{
if (SetProperty(ref _micStatus, value))
{
OnPropertyChanged(nameof(MicStatusBrush));
OnPropertyChanged(nameof(MicStatusToolTip));
}
}
}
/// <summary>Status dot: green = connected, yellow = problem with the requested
/// connection, red = not connected (no device / not started).</summary>
public SolidColorBrush MicStatusBrush => MicStatus switch
{
MicStatus.Connected => BarOnBrush,
MicStatus.Problem => MicProblemBrush,
_ => BarOffBrush,
};
public string MicStatusToolTip => MicStatus switch
{
MicStatus.Connected => "Mic connected — click to change",
MicStatus.Problem => "Mic problem — the requested microphone is unavailable (in use or unplugged). Click to change",
_ => "No mic connected — click to choose a microphone",
};
// ─── Game audio bar (desktop/game): visible only while a full-screen game
// ─── is producing sound (IGameAudioDetector). Meter + mute + volume mirror
// ─── the mic bar. ───
/// <summary>True while the game audio bar should be shown (driven by
/// IGameAudioDetector via the poll timer).</summary>
public bool IsGameAudioBarVisible
{
get => _isGameAudioBarVisible;
private set => SetProperty(ref _isGameAudioBarVisible, value);
}
/// <summary>Live desktop/game input level (0..1), fed by the mixer's loopback
/// capture. Read by the game meter, scaled by GameAudioVolume.</summary>
public double GameAudioLevel
{
get => _gameAudioLevel;
set
{
if (SetProperty(ref _gameAudioLevel, Math.Clamp(value, 0, 1)))
{
OnPropertyChanged(nameof(GameMeterFillWidth));
OnPropertyChanged(nameof(GameMeterBrush));
}
}
}
/// <summary>Displayed game meter level: 0 while muted; the volume position
/// while the slider is dragged (or briefly after an unmute flash); otherwise
/// the realtime live level scaled by volume.</summary>
private double GameMeterLevel => GameMuted ? 0 : _gameVolumeAdjusting || _gameVolumeFlash ? GameAudioVolume : Math.Min(1, GameAudioLevel * GameAudioVolume);
public double GameMeterFillWidth => GameMeterLevel * 288;
public string GameMeterBrush => GameMeterLevel switch
{
< 0.6 => "#22c55e",
< 0.8 => "#eab308",
_ => "#ef4444",
};
public void SetGameVolumeAdjusting(bool adjusting)
{
if (adjusting)
CancelGameVolumeFlash();
if (SetProperty(ref _gameVolumeAdjusting, adjusting))
{
OnPropertyChanged(nameof(GameMeterFillWidth));
OnPropertyChanged(nameof(GameMeterBrush));
}
}
private void BeginGameVolumeFlash()
{
_gameVolumeFlash = true;
OnPropertyChanged(nameof(GameMeterFillWidth));
OnPropertyChanged(nameof(GameMeterBrush));
_gameVolumeFlashTimer.Stop();
_gameVolumeFlashTimer.Start();
}
private void EndGameVolumeFlash()
{
_gameVolumeFlashTimer.Stop();
if (_gameVolumeFlash)
{
_gameVolumeFlash = false;
OnPropertyChanged(nameof(GameMeterFillWidth));
OnPropertyChanged(nameof(GameMeterBrush));
}
}
private void CancelGameVolumeFlash()
{
_gameVolumeFlashTimer.Stop();
_gameVolumeFlash = false;
}
/// <summary>Game audio gain (0..1, unity default). Running to 0 mutes; the
/// prior level is remembered so the speaker button can restore it.</summary>
public double GameAudioVolume
{
get => _gameVolume;
set
{
var clamped = Math.Clamp(value, 0, 1);
if (clamped == 0 && !_gameMuted)
_gameVolumeBeforeMute ??= _gameVolume;
if (SetProperty(ref _gameVolume, clamped))
{
var muted = clamped == 0;
if (_gameMuted != muted)
{
_gameMuted = muted;
OnPropertyChanged(nameof(GameMuted));
OnPropertyChanged(nameof(GameMuteText));
}
if (!muted)
_gameVolumeBeforeMute = null;
OnPropertyChanged(nameof(GameMeterFillWidth));
OnPropertyChanged(nameof(GameMeterBrush));
}
}
}
/// <summary>Read-only: true whenever the volume is 0 (the slider and the
/// speaker can never disagree).</summary>
public bool GameMuted => _gameMuted;
public string GameMuteText => GameMuted ? "Unmute" : "Mute";
private void ToggleGameMute()
{
if (GameMuted)
{
GameAudioVolume = _gameVolumeBeforeMute ?? 1.0;
BeginGameVolumeFlash();
}
else
{
_gameVolumeBeforeMute = GameAudioVolume;
GameAudioVolume = 0;
}
} }
public bool IsOffline => StreamStatus == StreamStatus.Offline; public bool IsOffline => StreamStatus == StreamStatus.Offline;
@@ -759,6 +936,7 @@ public class MainViewModel : ViewModelBase
public ICommand RefreshCaptureCommand { get; } public ICommand RefreshCaptureCommand { get; }
public ICommand SetBackdropDisplayCommand { get; } public ICommand SetBackdropDisplayCommand { get; }
public ICommand ToggleMicMuteCommand { get; } public ICommand ToggleMicMuteCommand { get; }
public ICommand ToggleGameMuteCommand { get; }
public ICommand OpenMicPickerCommand { get; } public ICommand OpenMicPickerCommand { get; }
public ICommand OpenSocialDialogCommand { get; } public ICommand OpenSocialDialogCommand { get; }
public ICommand StartStreamCommand { get; } public ICommand StartStreamCommand { get; }
@@ -795,6 +973,9 @@ public class MainViewModel : ViewModelBase
_volumeFlashTimer = new DispatcherTimer { Interval = TimeSpan.FromMilliseconds(300) }; _volumeFlashTimer = new DispatcherTimer { Interval = TimeSpan.FromMilliseconds(300) };
_volumeFlashTimer.Tick += (_, _) => EndVolumeFlash(); _volumeFlashTimer.Tick += (_, _) => EndVolumeFlash();
_gameVolumeFlashTimer = new DispatcherTimer { Interval = TimeSpan.FromMilliseconds(300) };
_gameVolumeFlashTimer.Tick += (_, _) => EndGameVolumeFlash();
_saveDebounce = new DispatcherTimer { Interval = TimeSpan.FromMilliseconds(1500) }; _saveDebounce = new DispatcherTimer { Interval = TimeSpan.FromMilliseconds(1500) };
_saveDebounce.Tick += (_, _) => SaveLayoutNow(); _saveDebounce.Tick += (_, _) => SaveLayoutNow();
@@ -820,6 +1001,7 @@ public class MainViewModel : ViewModelBase
RefreshCaptureCommand = new RelayCommand(_ => RefreshBackdropAutoCapture()); RefreshCaptureCommand = new RelayCommand(_ => RefreshBackdropAutoCapture());
SetBackdropDisplayCommand = new RelayCommand(display => SetBackdropCapture(display as DisplayInfo)); SetBackdropDisplayCommand = new RelayCommand(display => SetBackdropCapture(display as DisplayInfo));
ToggleMicMuteCommand = new RelayCommand(_ => ToggleMicMute()); ToggleMicMuteCommand = new RelayCommand(_ => ToggleMicMute());
ToggleGameMuteCommand = new RelayCommand(_ => ToggleGameMute());
OpenMicPickerCommand = new RelayCommand(_ => PickMicrophone()); OpenMicPickerCommand = new RelayCommand(_ => PickMicrophone());
OpenSettingsCommand = new RelayCommand(_ => ShowOverlay(nameof(IsSettingsOpen), "App Settings")); OpenSettingsCommand = new RelayCommand(_ => ShowOverlay(nameof(IsSettingsOpen), "App Settings"));
OpenBugCommand = new RelayCommand(_ => ShowOverlay(nameof(IsBugOpen), "Report Bug")); OpenBugCommand = new RelayCommand(_ => ShowOverlay(nameof(IsBugOpen), "Report Bug"));
@@ -853,10 +1035,22 @@ public class MainViewModel : ViewModelBase
new WasapiLoopbackAudioSource(), new WasapiLoopbackAudioSource(),
message => AppLog.Write(message)); message => AppLog.Write(message));
_audioMixer.MicLevelChanged += OnMicLevelChanged; _audioMixer.MicLevelChanged += OnMicLevelChanged;
_audioMixer.LoopbackLevelChanged += OnLoopbackLevelChanged;
_audioMixer.MicConnected += OnMicConnected;
_audioMixer.MicFailed += OnMicFailed;
_ = StartMicCaptureAsync();
_fullScreenDetector = new Win32FullScreenDetector(); _fullScreenDetector = new Win32FullScreenDetector();
foreach (var display in _fullScreenDetector.GetDisplays()) foreach (var display in _fullScreenDetector.GetDisplays())
Displays.Add(display); Displays.Add(display);
_gameAudioDetector = new GameAudioDetector(
() => _fullScreenDetector.GetForegroundFullScreenMonitorIndex(),
() => (float)GameAudioLevel);
_gameAudioDetector.IsGameAudioActiveChanged += OnGameAudioActiveChanged;
_gameAudioTimer = new DispatcherTimer { Interval = TimeSpan.FromMilliseconds(250) };
_gameAudioTimer.Tick += OnGameAudioPollTick;
_gameAudioTimer.Start();
_screenCaptureFactory = new ScreenCaptureSourceFactory( _screenCaptureFactory = new ScreenCaptureSourceFactory(
() => new WindowInteropHelper(System.Windows.Application.Current?.MainWindow).Handle); () => new WindowInteropHelper(System.Windows.Application.Current?.MainWindow).Handle);
_screenCaptureManager = new ScreenCaptureManager( _screenCaptureManager = new ScreenCaptureManager(
@@ -1191,6 +1385,8 @@ public class MainViewModel : ViewModelBase
{ {
_saveDebounce?.Stop(); _saveDebounce?.Stop();
SaveLayoutNow(); SaveLayoutNow();
_gameAudioTimer.Stop();
_audioMixer.Dispose();
_framePump.Dispose(); _framePump.Dispose();
_cameraManager.Dispose(); _cameraManager.Dispose();
_screenCaptureManager.Dispose(); _screenCaptureManager.Dispose();
@@ -1761,7 +1957,6 @@ public class MainViewModel : ViewModelBase
? "ytLlive" ? "ytLlive"
: $"{dialog.StreamTitle} — ytLlive"; : $"{dialog.StreamTitle} — ytLlive";
StreamStatus = StreamStatus.Streaming; StreamStatus = StreamStatus.Streaming;
_audioMixer.Start();
_ = _framePump.StartAsync(); // never throws; failures log + surface via Failed _ = _framePump.StartAsync(); // never throws; failures log + surface via Failed
} }
} }
@@ -1770,7 +1965,8 @@ public class MainViewModel : ViewModelBase
{ {
StreamStatus = StreamStatus.Offline; StreamStatus = StreamStatus.Offline;
WindowTitle = "ytLlive"; WindowTitle = "ytLlive";
_audioMixer.Stop(); // Audio capture is always-on (preview monitoring); only the frame pump
// and the session stop here.
_ = _framePump.StopAsync(); _ = _framePump.StopAsync();
// Graceful end completes the session = signs out (the DPAPI token is // Graceful end completes the session = signs out (the DPAPI token is
// cleared so the next Start Stream requires a fresh sign-in). A crash // cleared so the next Start Stream requires a fresh sign-in). A crash
@@ -1793,6 +1989,70 @@ public class MainViewModel : ViewModelBase
AudioLevel = level; AudioLevel = level;
} }
private void OnLoopbackLevelChanged(float level)
{
var dispatcher = System.Windows.Application.Current?.Dispatcher;
if (dispatcher != null && !dispatcher.CheckAccess())
dispatcher.BeginInvoke(() => GameAudioLevel = level);
else
GameAudioLevel = level;
}
private void OnMicConnected()
{
var dispatcher = System.Windows.Application.Current?.Dispatcher;
if (dispatcher != null && !dispatcher.CheckAccess())
dispatcher.BeginInvoke(() => MicStatus = MicStatus.Connected);
else
MicStatus = MicStatus.Connected;
}
private void OnMicFailed(Exception ex)
{
// The mixer logs the failure detail; here we only flip the dot to yellow.
var dispatcher = System.Windows.Application.Current?.Dispatcher;
if (dispatcher != null && !dispatcher.CheckAccess())
dispatcher.BeginInvoke(() => MicStatus = MicStatus.Problem);
else
MicStatus = MicStatus.Problem;
}
/// <summary>Starts capture once at startup: with no mic device present the
/// dot stays red and capture never starts; otherwise the mixer starts and
/// raises MicConnected (green) or MicFailed (yellow).</summary>
private async Task StartMicCaptureAsync()
{
try
{
var mics = await _microphoneEnumerator.GetMicrophonesAsync();
if (mics.Count == 0)
{
MicStatus = MicStatus.NotConnected;
AppLog.Write("Mic: no capture devices found — mic capture not started");
return;
}
_audioMixer.Start();
}
catch (Exception ex)
{
AppLog.Write($"Mic: device check failed: {ex.Message}");
}
}
private void OnGameAudioActiveChanged(bool active) => IsGameAudioBarVisible = active;
private void OnGameAudioPollTick(object? sender, EventArgs e)
{
try
{
_gameAudioDetector.Poll();
}
catch (Exception ex)
{
AppLog.Write($"Game audio detection failed: {ex.Message}");
}
}
// Scene-element → latest frame, for the live compositor. The map mirrors the // Scene-element → latest frame, for the live compositor. The map mirrors the
// preview: webcam by DeviceId, live captures (the backdrop) by CaptureKey, // preview: webcam by DeviceId, live captures (the backdrop) by CaptureKey,
// images/background by AssetId. A null frame leaves the element transparent. // images/background by AssetId. A null frame leaves the element transparent.
+1 -1
View File
File diff suppressed because one or more lines are too long
+30 -17
View File
@@ -91,7 +91,7 @@ C# / WPF (.NET 8) following MVVM:
|------|------| |------|------|
| `Models/` | Plain data types — Scene, Source (incl. `ClipShape`, `IsMirrored`, `VideoImageSource`), QualityOption, StreamConfig, StreamHealth, YouTubeChannel, ChatMessage, **Socials (`SocialService` enum + `SocialEntry`/`SocialsConfig` + `SocialServiceIcons`) — the social bar** | | `Models/` | Plain data types — Scene, Source (incl. `ClipShape`, `IsMirrored`, `VideoImageSource`), QualityOption, StreamConfig, StreamHealth, YouTubeChannel, ChatMessage, **Socials (`SocialService` enum + `SocialEntry`/`SocialsConfig` + `SocialServiceIcons`) — the social bar** |
| `ViewModels/` | MainViewModel — exposes collections + commands for the UI; GoLiveViewModel, ReuseImageViewModel, CameraPickerViewModel, **SocialsDialogViewModel** | | `ViewModels/` | MainViewModel — exposes collections + commands for the UI; GoLiveViewModel, ReuseImageViewModel, CameraPickerViewModel, **SocialsDialogViewModel** |
| `Services/` | YouTube OAuth2, stream/broadcast management, live chat polling, LayoutStore (SQLite), **SocialValidator (`ISocialValidator` seam + `HttpSocialValidator` default)**, **webcam: `VideoFrame` seam + `CameraDeviceInfo`/`ICameraEnumerator`/`ICameraFrameSource` interfaces + `MediaCaptureCameraEnumerator`/`MediaCaptureFrameSource` (WinRT) + `CameraManager`**, **screen capture: `IFullScreenDetector`/`Win32FullScreenDetector` + `IScreenCaptureSource`/`ScreenCaptureFrameSource` (WinRT GraphicsCapture) + `ScreenCaptureManager` + `ScreenCaptureSourceFactory` + `Direct3D11Helper`/`CaptureInterop` (COM bridges)**, **compositor: `SceneCompositor` + `CompositorOptions` + pure `StretchMath` + `StaticPixelCache` (see "Scene compositor")**, **audio: `IAudioSource` seam + `WasapiLoopbackAudioSource`/`WasapiMicAudioSource` (NAudio WASAPI) + `AudioMixer` + pure `AudioLevelMeter`/`WaveToFloat` (see "Live audio capture")**, **encoder: `IFfmpegEncoder`/`FfmpegEncoder` + `IEncoderProcess`/`FfmpegEncoderProcess` + `IFfmpegLocator`/`FfmpegLocator` + pure `FfmpegArgs`/`FfmpegProgressParser`/`FfmpegEncoderPicker` + the `FramePump` frame producer (see "Live encoder" + "Live frame pipeline")** | | `Services/` | YouTube OAuth2, stream/broadcast management, live chat polling, LayoutStore (SQLite), **SocialValidator (`ISocialValidator` seam + `HttpSocialValidator` default)**, **webcam: `VideoFrame` seam + `CameraDeviceInfo`/`ICameraEnumerator`/`ICameraFrameSource` interfaces + `MediaCaptureCameraEnumerator`/`MediaCaptureFrameSource` (WinRT) + `CameraManager`**, **screen capture: `IFullScreenDetector`/`Win32FullScreenDetector` + `IScreenCaptureSource`/`ScreenCaptureFrameSource` (WinRT GraphicsCapture) + `ScreenCaptureManager` + `ScreenCaptureSourceFactory` + `Direct3D11Helper`/`CaptureInterop` (COM bridges)**, **compositor: `SceneCompositor` + `CompositorOptions` + pure `StretchMath` + `StaticPixelCache` (see "Scene compositor")**, **audio: `IAudioSource` seam + `WasapiLoopbackAudioSource`/`WasapiMicAudioSource` (NAudio WASAPI) + `AudioMixer` + pure `AudioLevelMeter`/`WaveToFloat` + game bar: `IGameAudioDetector`/`GameAudioHysteresis`/`GameAudioDetector` (see "Live audio capture")**, **encoder: `IFfmpegEncoder`/`FfmpegEncoder` + `IEncoderProcess`/`FfmpegEncoderProcess` + `IFfmpegLocator`/`FfmpegLocator` + pure `FfmpegArgs`/`FfmpegProgressParser`/`FfmpegEncoderPicker` + the `FramePump` frame producer (see "Live encoder" + "Live frame pipeline")** |
| `Helpers/` | ViewModelBase (INotifyPropertyChanged), RelayCommand, ImageCache, AppLog (file logger), FocusPreservingListBox, OAuthCredentials, **TokenStore (DPAPI session persistence)**, visibility converters | | `Helpers/` | ViewModelBase (INotifyPropertyChanged), RelayCommand, ImageCache, AppLog (file logger), FocusPreservingListBox, OAuthCredentials, **TokenStore (DPAPI session persistence)**, visibility converters |
| `Themes/` | `Controls.xaml` — the single dark-theme source, merged once in `App.xaml` (see `Themes/index.md`) | | `Themes/` | `Controls.xaml` — the single dark-theme source, merged once in `App.xaml` (see `Themes/index.md`) |
| `MainWindow.xaml` | Dark theme; layout: top bar (controls), center (preview), left (scenes/sources), right (chat), bottom (health) | | `MainWindow.xaml` | Dark theme; layout: top bar (controls), center (preview), left (scenes/sources), right (chat), bottom (health) |
@@ -355,34 +355,47 @@ forces one and skips the probe. `EncoderOptions` also carries W×H/FPS/bitrate f
`GopSize` = FPS×4. Constructed by `MainViewModel` since ship step 5 (`new FfmpegEncoder(new FfmpegLocator())`), `GopSize` = FPS×4. Constructed by `MainViewModel` since ship step 5 (`new FfmpegEncoder(new FfmpegLocator())`),
driven by the `FramePump` below. driven by the `FramePump` below.
### Live audio capture (TASK 4 ship step 4 — shipped 2026-08-12, plan in TASKS.md) ### Live audio capture (TASK 4 ship step 4 — shipped 2026-08-12, game audio bar follow-up 2026-08-13, plans in TASKS.md)
**Capture runs only while live and is KISS by rule**: desktop/game audio is automatic (WASAPI loopback, **Capture runs for the app's lifetime and is KISS by rule**: desktop/game audio is automatic (WASAPI
zero UI), the mic is the creator's only audio control (meter/mute/volume already shipped). The whole loopback), the mic is the creator's only audio control (meter/mute/volume already shipped). The whole
layer sits behind an **`IAudioSource` seam** (`Services/Audio/`: `Start`/`Stop`/`SampleReady`/`Failed`, layer sits behind an **`IAudioSource` seam** (`Services/Audio/`: `Start`/`Stop`/`Started`/`SampleReady`/
IDisposable) so the app never touches NAudio directly and tests inject hermetic fakes (no devices, no `Failed`, IDisposable) so the app never touches NAudio directly and tests inject hermetic fakes (no
timers). devices, no timers).
- **Sources (NAudio `NAudio.Wasapi` 2.2.1, MIT — item 9 in `THIRD-PARTY-NOTICES.txt`):** - **Sources (NAudio `NAudio.Wasapi` 2.2.1, MIT — item 9 in `THIRD-PARTY-NOTICES.txt`):**
`WasapiLoopbackAudioSource` = `WasapiLoopbackCapture` on the default render device; `WasapiLoopbackAudioSource` = `WasapiLoopbackCapture` on the default render device;
`WasapiMicAudioSource` = `WasapiCapture` with the device resolved by `FriendlyName` matching `WasapiMicAudioSource` = `WasapiCapture` with the device resolved by `FriendlyName` matching
`MicSourceName` (the app only persists the **DisplayName**), falling back to the default capture `MicSourceName` (the app only persists the **DisplayName**), falling back to the default capture
endpoint. Mic device resolution re-reads the name provider `Func<string?>` at each `Start`, so a mic endpoint. Mic device resolution re-reads the name provider `Func<string?>` at each `Start`, so a mic
picked mid-session takes effect next go-live. picked mid-session takes effect **immediately** (the mixer restarts the mic on pick). Both sources
- **`AudioMixer`** owns both sources; `Start`/`Stop` follow go-live (`MainViewModel.BeginGoLive` success raise `Started` once their capture loop actually begins — the mixer turns that into `MicConnected`.
→ `_audioMixer.Start()`, `StopStream` → `Stop()`). Mic samples feed a pure **`AudioLevelMeter`** (RMS - **`AudioMixer`** owns both sources; **`StartMicCaptureAsync` starts the mixer once at startup** and
with 0.2 exponential smoothing) → `MicLevelChanged` → marshalled to the UI thread → `AudioLevel`. `Shutdown` disposes it — NOT go-live — so both footer meters preview live (`BeginGoLive`/`StopStream`
Desktop samples are currently **dropped** — a later step's AAC mix consumes them, replacing the no longer touch the mixer). Mic samples feed a pure **`AudioLevelMeter`** (RMS with 0.2 exponential
`-f lavfi -i anullsrc` placeholder (the encoder construction itself shipped in ship step 5). Failures log via `AppLog`; a mic failure zeroes the smoothing) → `MicLevelChanged` → marshalled to the UI thread → `AudioLevel`; loopback samples feed a
meter, a loopback failure never kills the mic. second meter → `LoopbackLevelChanged` → the game bar's `GameAudioLevel`. **Mic connection state
surfaces as events:** `MicConnected` (source `Started`), `MicFailed` (source `Failed`), and
`RestartMic()` re-resolves + restarts just the mic (loopback keeps running). Failures log via
`AppLog`; a mic failure zeroes the meter, a loopback failure never kills the mic. Meter `Push` is
unconditional (a `?.` on the event would skip the argument — and the meter update — when nothing is
subscribed yet).
- **Mic status dot (`Models/MicStatus.cs`)** on the footer's MIC button: green = `MicConnected`,
yellow = `MicFailed` (in use/unplugged), red = no mic device at startup (the mixer is never started,
so loopback and the game bar can't run either — no capture devices at all).
- **Game audio bar** (desktop/game, only while a full-screen game is up in the preview):
`IGameAudioDetector` seam (`Services/IGameAudioDetector.cs`), pure `GameAudioHysteresis` (SHOW after
~500ms of fullscreen + sound, HIDE after ~1s away from fullscreen, **silence never hides an active
bar**), and the default `GameAudioDetector` composing `IFullScreenDetector` + the live loopback level
(floor 0.5%). WPF-free — the VM owns a 250ms `DispatcherTimer` that polls it and flips
`IsGameAudioBarVisible`.
- **`WaveToFloat`** (pure, shared): WASAPI mix formats → interleaved float — IEEE float 32-bit direct, - **`WaveToFloat`** (pure, shared): WASAPI mix formats → interleaved float — IEEE float 32-bit direct,
PCM 16-bit normalized to -1..1, `WaveFormatExtensible` with the IEEE-float subformat GUID PCM 16-bit normalized to -1..1, `WaveFormatExtensible` with the IEEE-float subformat GUID
(`NAudio.Dmo.AudioMediaSubtypes.MEDIASUBTYPE_IEEE_FLOAT`), trailing partial samples ignored. (`NAudio.Dmo.AudioMediaSubtypes.MEDIASUBTYPE_IEEE_FLOAT`), trailing partial samples ignored.
- `FfmpegEncoder.cs:139` pre-existing CS8602 fixed (`process!`) — build **0 warnings**; **139 passing**. - Build **0 warnings**; **167 passing** (mixer/hysteresis/game-detector unit tests).
**Deferred:** the loopback→encoder AAC mix wiring (a later step — the encoder construction + frame **Deferred:** the loopback→encoder AAC mix wiring (a later step — the encoder construction + frame
pipeline shipped in ship step 5); capture while not live is deliberately not shipped (privacy indicator pipeline shipped in ship step 5).
otherwise).
### Live frame pipeline (TASK 4 ship step 5 — shipped 2026-08-12, plan in TASKS.md) ### Live frame pipeline (TASK 4 ship step 5 — shipped 2026-08-12, plan in TASKS.md)
+94
View File
@@ -18,6 +18,7 @@ public class AudioMixerTests
public int StartCount { get; private set; } public int StartCount { get; private set; }
public int StopCount { get; private set; } public int StopCount { get; private set; }
public bool Disposed { get; private set; } public bool Disposed { get; private set; }
public event Action? Started;
public event Action<AudioSample>? SampleReady; public event Action<AudioSample>? SampleReady;
public event Action<Exception>? Failed; public event Action<Exception>? Failed;
@@ -25,6 +26,7 @@ public class AudioMixerTests
public void Stop() => StopCount++; public void Stop() => StopCount++;
public void Dispose() => Disposed = true; public void Dispose() => Disposed = true;
public void MarkStarted() => Started?.Invoke();
public void Emit(AudioSample sample) => SampleReady?.Invoke(sample); public void Emit(AudioSample sample) => SampleReady?.Invoke(sample);
public void Fail(Exception ex) => Failed?.Invoke(ex); public void Fail(Exception ex) => Failed?.Invoke(ex);
} }
@@ -115,6 +117,98 @@ public class AudioMixerTests
Assert.Equal(0f, mixer.MicLevel); Assert.Equal(0f, mixer.MicLevel);
} }
[Fact]
public void MicSamples_DoNotChangeLoopbackLevel()
{
var mic = new FakeSource();
var mixer = new AudioMixer(mic, new FakeSource());
var levels = new List<float>();
mixer.LoopbackLevelChanged += l => levels.Add(l);
mixer.Start();
mic.Emit(new AudioSample(new[] { 1f, 1f, 1f, 1f }, 48000, 1));
Assert.Empty(levels);
Assert.Equal(0f, mixer.LoopbackLevel);
}
[Fact]
public void LoopbackSamples_DriveLoopbackLevelChanged()
{
var loopback = new FakeSource();
var mixer = new AudioMixer(new FakeSource(), loopback);
var levels = new List<float>();
mixer.LoopbackLevelChanged += l => levels.Add(l);
mixer.Start();
loopback.Emit(new AudioSample(new[] { 1f, 1f, 1f, 1f }, 48000, 2));
loopback.Emit(new AudioSample(new[] { -1f, -1f, -1f, -1f }, 48000, 2));
Assert.NotEmpty(levels);
Assert.All(levels, l => Assert.InRange(l, 0f, 1f));
}
[Fact]
public void MicStarted_RaisesMicConnected()
{
var mic = new FakeSource();
var mixer = new AudioMixer(mic, new FakeSource());
var connected = 0;
mixer.MicConnected += () => connected++;
mixer.Start();
mic.MarkStarted();
Assert.Equal(1, connected);
}
[Fact]
public void MicFailure_RaisesMicFailed()
{
var mic = new FakeSource();
var mixer = new AudioMixer(mic, new FakeSource());
Exception? failed = null;
mixer.MicFailed += ex => failed = ex;
mixer.Start();
mic.Fail(new InvalidOperationException("boom"));
Assert.NotNull(failed);
Assert.Equal("boom", failed!.Message);
}
[Fact]
public void RestartMic_StopsAndRestartsMic_KeepsLoopbackRunning()
{
var mic = new FakeSource();
var loopback = new FakeSource();
var mixer = new AudioMixer(mic, loopback);
mixer.Start();
mixer.RestartMic();
Assert.Equal(2, mic.StartCount);
Assert.Equal(1, mic.StopCount);
Assert.Equal(1, loopback.StartCount);
Assert.Equal(0, loopback.StopCount);
}
[Fact]
public void RestartMic_ResetsLevel()
{
var mic = new FakeSource();
var mixer = new AudioMixer(mic, new FakeSource());
mixer.Start();
mic.Emit(new AudioSample(new[] { 1f, 1f, 1f, 1f }, 48000, 1));
Assert.True(mixer.MicLevel > 0);
float? reset = null;
mixer.MicLevelChanged += l => reset = l;
mixer.RestartMic();
Assert.Equal(0f, reset);
}
[Fact] [Fact]
public void MicFailure_LogsAndResetsLevel() public void MicFailure_LogsAndResetsLevel()
{ {
+76
View File
@@ -0,0 +1,76 @@
using Xunit;
using ytLive.Services;
namespace ytLive.Tests;
/// <summary>
/// TASK 4 game audio bar: the default detector's provider wiring — it composes
/// the full-screen monitor + loopback level into the hysteresis and raises
/// IsGameAudioActiveChanged on transitions. The transition math itself lives in
/// GameAudioHysteresisTests.
/// </summary>
public class GameAudioDetectorTests
{
[Fact]
public void Poll_RaisesChanged_WhenGameAppearsAndLeaves()
{
var now = new DateTime(2026, 8, 13, 12, 0, 0);
int? monitor = 0;
var level = 0f;
var detector = new GameAudioDetector(() => monitor, () => level, () => now);
var changes = new List<bool>();
detector.IsGameAudioActiveChanged += a => changes.Add(a);
level = 0.9f;
detector.Poll();
Assert.False(detector.IsGameAudioActive);
now = now.AddMilliseconds(600);
detector.Poll();
Assert.True(detector.IsGameAudioActive);
Assert.Equal(new[] { true }, changes);
level = 0f;
now = now.AddSeconds(2);
detector.Poll();
Assert.True(detector.IsGameAudioActive);
Assert.Equal(new[] { true }, changes);
monitor = null;
detector.Poll();
Assert.True(detector.IsGameAudioActive);
now = now.AddSeconds(2);
detector.Poll();
Assert.False(detector.IsGameAudioActive);
Assert.Equal(new[] { true, false }, changes);
}
[Fact]
public void Poll_StaysInactive_WhenNoFullScreenMonitor()
{
var detector = new GameAudioDetector(() => null, () => 0.9f);
var changes = 0;
detector.IsGameAudioActiveChanged += _ => changes++;
for (var i = 0; i < 5; i++)
detector.Poll();
Assert.False(detector.IsGameAudioActive);
Assert.Equal(0, changes);
}
[Fact]
public void Poll_StaysInactive_WhileLoopbackSilent()
{
var detector = new GameAudioDetector(() => 0, () => 0f);
var changes = 0;
detector.IsGameAudioActiveChanged += _ => changes++;
for (var i = 0; i < 5; i++)
detector.Poll();
Assert.False(detector.IsGameAudioActive);
Assert.Equal(0, changes);
}
}
+82
View File
@@ -0,0 +1,82 @@
using Xunit;
using ytLive.Services;
namespace ytLive.Tests;
/// <summary>
/// TASK 4 game audio bar: the pure show/hide state machine. Show = a full-screen
/// app holds sound for half a second; hide = the app leaves fullscreen for a
/// second. Silence never hides an active bar — only the game leaving the preview
/// does (per the creator's rule).
/// </summary>
public class GameAudioHysteresisTests
{
private static readonly DateTime T0 = new(2026, 8, 13, 12, 0, 0);
[Fact]
public void StaysInactive_WhileSilent()
{
var h = new GameAudioHysteresis();
for (var i = 0; i < 30; i++)
{
h.Update(true, false, T0.AddSeconds(i));
Assert.False(h.IsActive);
}
}
[Fact]
public void StaysInactive_UntilSoundHoldsHalfSecond()
{
var h = new GameAudioHysteresis();
h.Update(true, true, T0);
Assert.False(h.IsActive);
h.Update(true, true, T0.AddMilliseconds(400));
Assert.False(h.IsActive);
h.Update(true, true, T0.AddMilliseconds(600));
Assert.True(h.IsActive);
}
[Fact]
public void NeverHides_WhileGameStillFullScreen_EvenWhenSilent()
{
var h = new GameAudioHysteresis();
h.Update(true, true, T0);
h.Update(true, true, T0.AddSeconds(1));
Assert.True(h.IsActive);
for (var i = 2; i < 40; i++)
{
h.Update(true, false, T0.AddSeconds(i));
Assert.True(h.IsActive);
}
}
[Fact]
public void Hides_AfterAppLeavesFullScreen()
{
var h = new GameAudioHysteresis();
h.Update(true, true, T0);
h.Update(true, true, T0.AddSeconds(1));
Assert.True(h.IsActive);
h.Update(false, true, T0.AddSeconds(2));
Assert.True(h.IsActive);
h.Update(false, true, T0.AddSeconds(4));
Assert.False(h.IsActive);
}
[Fact]
public void StaysInactive_WhenNoFullScreen_EvenWithSound()
{
var h = new GameAudioHysteresis();
for (var i = 0; i < 30; i++)
{
h.Update(false, true, T0.AddSeconds(i));
Assert.False(h.IsActive);
}
}
}