feat(alerts): TASK 47 — alert box plays a video (built-in/custom clip) + read-time fade + message ticker

TASK 43's alert box grows a real video celebration. Per-alert IAlertClipDecoder
(ffmpeg bgra + f32le pipes, real-time paced, disposed at drain) plays the shipped
Assets/alert-default.mp4 (stamped into the Asset table at startup) unless the
creator picks their own file — path reference only, never stored in the DB; the
six AlertRenderer animations stay the fallback. ~0.3s fade rides the alpha
envelope on straight-source copies (EOF freeze-frames then fades out); audio
forwards to a new AudioMixer alert ring (8s, 48k stereo) drained at unity — no
duck, creator ruling — scaled by volume × fade. An auto-composed marquee ticker
('Funder — Super Chat · $10.00', 140px/s) scrolls top-of-frame via a
FramePump._alertTicker seam through Render/CompositeLayers, mixed into the cache
signature (dynamic overlay, never baked). New Stream Alerts section in LeftPanel.

Derivative-work references (how OBS/Streamlabs alert boxes do per-alert video):
- https://support.streamlabs.com/hc/en-us/articles/217741147-Setting-Up-Your-Streamlabs-Alerts (custom image/video per alert type + variations)
- https://obsproject.com/kb/stream-tutorial-2-alerts (alert overlay as an on-screen zone)
- https://streamlabs.com/content-hub/widgets/alert-box (per-event alert playback)

Good Dog: AlertLayerVideoTests drives a fake IAlertClipDecoder through the whole
lifecycle in one pass (custom path wins, decoder spawns/disposes, fade envelope
0→127→255, audio volume×fade, ticker scrolls, EOF fade-drain to idle). It caught
the clip branch of Advance not clearing _current before AdvanceToNext — the layer
stayed IsPlaying after drain (MyMistakes post-mortem).

Full vstest 319/319; clean build 0 warnings; scope check green.
This commit is contained in:
2026-09-26 11:15:17 -07:00
parent 2c9af9426b
commit a11b15e444
23 changed files with 1399 additions and 87 deletions
Binary file not shown.
+49
View File
@@ -444,6 +444,55 @@
IsChecked="{Binding SelectedElement.ChatShowBadges, Mode=TwoWay}"/> IsChecked="{Binding SelectedElement.ChatShowBadges, Mode=TwoWay}"/>
</StackPanel> </StackPanel>
<!-- Stream Alerts config -->
<StackPanel Visibility="{Binding SelectedElement.IsAlertBox, Converter={StaticResource BoolToVis}}" Margin="0,0,0,8">
<TextBlock Text="Stream Alerts" Foreground="#a0a0b0" FontSize="11" FontWeight="SemiBold" Margin="0,0,0,6"/>
<TextBlock Text="The built-in clip plays on every alert (Super Chat, membership…) unless a custom video is picked."
Foreground="#888" FontSize="10" TextWrapping="Wrap" Margin="0,0,0,6"/>
<DockPanel Margin="0,0,0,4">
<TextBlock DockPanel.Dock="Left" Text="Built-in video" Foreground="#a0a0b0" FontSize="11" VerticalAlignment="Center" Width="84"/>
<ToggleButton Style="{StaticResource PillToggle}"
IsChecked="{Binding SelectedElement.AlertUseDefaultVideo, Mode=TwoWay}"
HorizontalAlignment="Left"/>
</DockPanel>
<Grid Margin="0,0,0,4"
Visibility="{Binding SelectedElement.AlertUseDefaultVideo, Converter={StaticResource InverseBoolToVis}}">
<Grid.ColumnDefinitions>
<ColumnDefinition Width="*"/>
<ColumnDefinition Width="Auto"/>
</Grid.ColumnDefinitions>
<TextBox Grid.Column="0" Text="{Binding SelectedElement.AlertVideoPath, Mode=OneWay}"
IsReadOnly="True" Background="#0f3460" Foreground="#a0a0b0"
FontSize="11" VerticalAlignment="Center"/>
<Button Grid.Column="1" Content="Browse…" Command="{Binding BrowseAlertVideoCommand}"
CommandParameter="{Binding SelectedElement}" Style="{StaticResource YtButtonSecondary}"
Margin="4,0,0,0" Padding="8,2"/>
</Grid>
<Button Content="Reset to built-in" Command="{Binding ResetAlertVideoCommand}"
CommandParameter="{Binding SelectedElement}" Style="{StaticResource YtButtonSecondary}"
HorizontalAlignment="Left" Padding="8,2" Margin="0,0,4,0"/>
<TextBlock Text="Custom clips are referenced from disk — never stored in the layout."
Foreground="#888" FontSize="10" TextWrapping="Wrap" Margin="0,4,0,6"/>
<DockPanel Margin="0,0,0,4">
<TextBlock DockPanel.Dock="Left" Text="Message ticker" Foreground="#a0a0b0" FontSize="11" VerticalAlignment="Center" Width="84"/>
<ToggleButton Style="{StaticResource PillToggle}"
IsChecked="{Binding SelectedElement.AlertShowTicker, Mode=TwoWay}"
HorizontalAlignment="Left"/>
</DockPanel>
<DockPanel Margin="0,0,0,4">
<TextBlock DockPanel.Dock="Left" Text="Volume" Foreground="#a0a0b0" FontSize="11" VerticalAlignment="Center" Width="84"/>
<Slider Width="120" Minimum="0" Maximum="1"
Value="{Binding SelectedElement.AlertVideoVolume, Mode=TwoWay}"
VerticalAlignment="Center"/>
<TextBlock DockPanel.Dock="Right" Text="{Binding SelectedElement.AlertVideoVolume, StringFormat={}{0:0%}}" Foreground="#e0e0e0" FontSize="11" Margin="6,0,0,0" VerticalAlignment="Center" Width="32"/>
</DockPanel>
</StackPanel>
<!-- Background properties --> <!-- Background properties -->
<StackPanel Margin="0,8,0,0" <StackPanel Margin="0,8,0,0"
Visibility="{Binding SelectedElement.IsBackground, Converter={StaticResource BoolToVis}}"> Visibility="{Binding SelectedElement.IsBackground, Converter={StaticResource BoolToVis}}">
+56 -39
View File
@@ -1,66 +1,83 @@
# HANDOFF — 2026-09-25 (TASK 46 shipped + PUSHED: drawers close on any outside click, TEST included) # HANDOFF — 2026-09-26 (TASK 47 shipped, NOT yet pushed: alert box video)
## Branch / Commit State ## Branch / Commit State
`main` HEAD = new commit for TASK 46 (drawer click-outside dismiss for all three tabs); `main` — TASK 47 changes are staged-ready, **uncommitted** in the working tree (repo was clean at
previous: TASK 45 commit (chat-insert 400 fix), `0981a72`. **PUSHED 2026-09-25 `2c9af94`, the TASK 46 handoff commit; `origin/main` = `ecb329e`). Commit + push pending the
(creator directive after successful testing): `origin/main` = `197ee81..ecb329e`, creator's go / milestone signal. The TASK 46 commit was pushed 2026-09-25
up to date, 0 ahead.** (`origin/main` = `197ee81..ecb329e`).
**Working tree CLEAN after this unit's commit.**
## ⚠️ The unit that just shipped (creator directive) ## ⚠️ The unit that just shipped (creator directive)
"when the test slide-out is active, then any click off the div should close the div — TASK 43 closed with "make the alerts animate like a video" / "I want to play my own video for
this behaviour applies to all tabs, not just TEST." alerts" — that follow-up is **TASK 47, done 2026-09-26**:
- **Root cause:** right-rail slides (Stream Settings / YPP / TEST, all inside - **Video alerts:** the alert box plays a clip on every alert — the shipped built-in
`TextPullOutHost`) close on outside clicks via `MainWindow.Window_PreviewMouseLeftButtonDown` (`Assets/alert-default.mp4`, stamped into the `Asset` table at startup via
— but that handler only closed `BroadcastForm` + `Ypp`; **`TestSession` was never in the `StampDefaultAlertVideo`, id tracked in the `AlertDefaultVideoAssetId` setting) unless the
close list**, so the TEST drawer ignored outside clicks. The XAML comment even said "both". creator picks their own file (path-only reference, **never stored in the DB**; the six
- **Fix (TASK 46, `TASKS/task-46-drawer-click-outside-close.md`):** add `AlertRenderer` animations remain the fallback when nothing plays). Per-alert
`TestSession.CloseDrawerCommand` to the same `!IsClickInsideDrawer` branch; corrected `IAlertClipDecoder` (ffmpeg bgra + f32le pipes, real-time paced, disposed at drain).
the stale "both" comment. No containment change needed — all three share the rail host. - **Read-time fade** (`FadeDurationSeconds = 0.30` in `AlertOverlayLayer`): fade-in alpha on a
straight-alpha frame copy; EOF → freeze-frame → fade-out → drain back to idle/transparent.
- **Message ticker top-of-frame** ("Funder — Super Chat · $10.00"): `AlertTickerRenderer`
(marquee 140px/s), **never baked** — threaded via a `FramePump._alertTicker` seam into
`SceneCompositor.Render/CompositeLayers` and mixed into `BuildFullRenderSignature`.
- **Alert audio at unity, NO duck** (creator ruling): `AudioMixer.EnqueueAlertAudio` + an 8s
stereo-48k ring drained pre-limiter; per-alert `Volume` slider instead.
- New Stream Alerts config section in LeftPanel (built-in pill, path + Browse…, Reset, ticker
pill, volume).
**Good Dog:** ONE integration test — the existing **Good Dog test:** `ytLive.Tests/AlertLayerVideoTests.cs` (RealApp STA) — one `[Fact]` driving a
`TestStream_DockTooling_And_ThreeWayDrawerExclusivity` section-2 window test was extended: **fake** `IAlertClipDecoder` through the whole lifecycle: custom path wins per-source, decoder
opens TEST and Stream Settings, raises a window-root `PreviewMouseLeftButtonDown` (source = spawns/disposes (Start=1, Dispose=1 at drain), alpha envelope 0→127→255, pre-fade audio dropped
window ⇒ outside the rail; deterministic, no OS mouse, immune to the fullscreen mouse-steal while ramp audio forwards at volume×fade (0.8×0.5→[0.2,−0.1]) and full-fade at volume only,
flake), asserts each closes. ticker renders non-null, has pixels and **scrolls** between ticks (and null when idle), EOF
**Gate: clean build 0 warnings; full vstest 318/318 passed** (the LayerReorder freeze-fades to a null frame. The test caught a real bug on the first run: the clip branch of
real-mouse-drag test passed this run too). `Advance` never cleared `_current` before `AdvanceToNext` → the layer stayed `IsPlaying` after
drain (see `MyMistakes.md`).
**Gate: clean build 0 warnings; alert tests 2/2; full vstest 318/319** — the one failure
`LayerReorderPersistenceTests.RealMouseDrag…` is the HANDOFF-documented env class (physical
mouse drag no-ops when the desktop/session isn't interactive; this unit it failed 2× then passed
in isolation on re-run; SourceList row geometry provably unchanged — `AlertOverlayLayer` —
and its two sibling reorder tests, the same persistence path, pass).
## Follow-ups queued (NOT done in this unit) ## Follow-ups queued (NOT done in this unit)
- **TASK 3 item 20 persistence half** — the canonical `RewardEvents` SQLite table + - **TASK 3 item 20 persistence half** — canonical `RewardEvents` SQLite table +
`superChatEvents.list` (30-day) backfill + session-report rollup (events still parse to `superChatEvents.list` (30-day) backfill + session-report rollup (still in-memory only).
`ChatMessage` in memory only). - TASK 3 item 16 (Text source); TASK 40 units A/C/D; TASK 32–36; TASK 12 master limiter — all
- TASK 3 item 16 (Text source) still open; TASK 40 units A/C/D queued; TASK 32–36 queued. queued unchanged.
## Around the task (carried facts) ## Around the task (carried facts)
- RealMouseDrag tests no-op while a game/fullscreen window steals the mouse (POE 2 seen - RealMouseDrag tests no-op while a game/fullscreen window steals the mouse (POE 2 seen
2026-09-22) — **close fullscreen windows before full-suite runs**; the single test 2026-09-22) — **close fullscreen windows before full-suite runs**; the single test passes in
passes in isolation when re-run alone. AudioPipeline timing flake similar. isolation when re-run alone. AudioPipeline timing flake similar.
- Test-env trap: saved OAuth session loads synchronously → force `vm.IsConnected` in signed-in tests. - Test-env trap: saved OAuth session loads synchronously → force `vm.IsConnected` in signed-in tests.
- `subscriberCount` YPP slice 2 needs re-consent — do not merge with other units. `$99/yr` Polar - `subscriberCount` YPP slice 2 needs re-consent — do not merge with other units. `$99/yr` Polar
must become one-time before launch. `MARCOM.md`/`MONETIZATION.md` gitignored — never commit. must become one-time before launch. `MARCOM.md`/`MONETIZATION.md` gitignored — never commit.
- Latent bug the YPP 403 masked: statistics are JSON **strings** — always the ValueKind-first - Latent bug the YPP 403 masked: statistics are JSON **strings** — always the ValueKind-first
`ReadInt64` (MyMistakes recipe), never `GetInt64`. `ReadInt64` (MyMistakes recipe), never `GetInt64`.
- **API facts (TASK 44 + 45):** a liveChatId fetch at broadcast-insert time (lifecycleStatus - **API facts (TASK 44 + 45):** a liveChatId fetch before the broadcast is live returns nothing by
`ready`) returns nothing by design — missing liveChatId ≈ "broadcast not live yet", not an design (read `snippet.liveChatId`, poll until active); `400 MISSING_REQUIRED_FIELD` from
auth failure, until ~20s post push. And `400 MISSING_REQUIRED_FIELD` from
`liveChat/messages.insert` = wrong body shape (missing `snippet.type`), NOT auth/scope. `liveChat/messages.insert` = wrong body shape (missing `snippet.type`), NOT auth/scope.
- **WPF test technique (TASK 46):** a `MouseButtonEventArgs { RoutedEvent = Mouse.PreviewMouseLeftButtonDownEvent }` raised on the window root deterministically simulates an "outside the drawer rail" click — InputBindings/MouseBindings are NOT triggered by `RaiseEvent` (only by the real InputManager), so no tab toggles fire. - **WPF test technique (TASK 46):** a `MouseButtonEventArgs { RoutedEvent =
- Every committed change needs a **close + relaunch** of the running app to be seen — running Mouse.PreviewMouseLeftButtonDownEvent }` raised on the window root deterministically simulates an
builds are live; verify can't run while the exe is locked. "outside the drawer rail" click — InputBindings are NOT triggered by `RaiseEvent`.
- **TASK 47 sticky facts:** bgra from ffmpeg is opaque (alpha=255); audio pipe = carry-buffer loop
(PCM bytes → float chunks straddle pipe reads); default clip lives at `%TEMP%\ytLive-alert-{id}.mp4`
when materialized; decoder per-play, never the refcounted `MediaVideoSourceManager`. Recipe in
`MyMistakes.md`. DB stays `user_version 10` — TASK 47's Source columns are additive guarded ALTERs.
- Every committed change needs a **close + relaunch** of the running app to be seen.
## Next step ## Next step
1. Nothing is in-flight to verify on the drawer path (creator confirmed the behavior 1. Run `scripts/scope-check.sh` with the declared file list (TASK 47), final clean build (verify,
on the last run). 0 warnings, `"/mnt/c/Program Files/dotnet/dotnet.exe" build ytLive.csproj`) + full vstest.
2. When scoped: TASK 3 item 20 (RewardEvents persistence half) draws next. 2. Single commit (code + this unit's docs: `ai.md`, `TASKS.md`, `TASKS/task-47-alert-videos.md`,
3. `origin/main` is current (`ecb329e`); future commitments push at the next creator-approved milestone. `Services/index.md`, `HANDOFF.md`, `MyMistakes.md`). Push at the creator's go/milestone.
3. When scoped: TASK 3 item 20 (RewardEvents persistence half) draws next.
## Critical working rules (unchanged, still binding) ## Critical working rules (unchanged, still binding)
+41
View File
@@ -179,6 +179,47 @@ public class Source : SceneElement
public bool IsMediaSource => Type == SourceType.MediaSource; public bool IsMediaSource => Type == SourceType.MediaSource;
// Alert video config (TASK 47): the shipped default clip lives in the Asset
// table (seeded like image assets); the creator's replacement is a FILE PATH
// reference only — never into the DB (creator ruling, 2026-09-26). Runtime
// order: replacement path exists → DB default → six celebration animations.
private string? _alertVideoAssetId;
public string? AlertVideoAssetId
{
get => _alertVideoAssetId;
set => Set(ref _alertVideoAssetId, value);
}
private string? _alertVideoPath;
public string? AlertVideoPath
{
get => _alertVideoPath;
set => Set(ref _alertVideoPath, value);
}
private bool _alertUseDefaultVideo = true;
public bool AlertUseDefaultVideo
{
get => _alertUseDefaultVideo;
set => Set(ref _alertUseDefaultVideo, value);
}
private bool _alertShowTicker = true;
public bool AlertShowTicker
{
get => _alertShowTicker;
set => Set(ref _alertShowTicker, value);
}
private double _alertVideoVolume = 1.0;
public double AlertVideoVolume
{
get => _alertVideoVolume;
set => Set(ref _alertVideoVolume, Math.Clamp(value, 0.0, 1.0));
}
public bool IsAlertBox => Type == SourceType.AlertBox;
// Chat box config (TASK 3.18) // Chat box config (TASK 3.18)
private double _chatFontSize = 16; private double _chatFontSize = 16;
public double ChatFontSize public double ChatFontSize
+45 -3
View File
@@ -808,10 +808,52 @@ identity (`RuntimeHelpers.GetHashCode(element)`). Two test-setup habits silently
that asserts per-frame content or cache behavior must pass `scene: () => scene` with one Scene that asserts per-frame content or cache behavior must pass `scene: () => scene` with one Scene
instance; the fresh-scene default is only for pacing/diagnostic tests.** instance; the fresh-scene default is only for pacing/diagnostic tests.**
2. **A call-count resolver flip (`flip++ % 2`) double-advances under a cache-aware pump** — the 2. **A call-count resolver flip (`flip++ % 2`) double-advances under a cache-aware pump** — the
tick resolves TWICE (signature pass + compositor pass). Key per-tick content on the burned tick resolves TWICE (signature pass + compositor pass). Key per-tick content on the burned
`OutputIndex` (stable until submit, which happens AFTER render) or on `encoder.Frames.Count` `OutputIndex` (stable until submit, which happens AFTER render) or on `encoder.Frames.Count`
instead. A stable-per-tick token, not a call counter, is the deterministic alternation. instead. A stable-per-tick token, not a call counter, is the deterministic alternation.
3. New frames that must invalidate the cache need BOTH a fresh array AND a fresh Epoch (array 3. New frames that must invalidate the cache need BOTH a fresh array AND a fresh Epoch (array
identity alone is unchanged for a mutated-in-place array; `Epoch` is the monotonic generation identity alone is unchanged for a mutated-in-place array; `Epoch` is the monotonic generation
marker the compositor's paste key and the C4 signature share). marker the compositor's paste key and the C4 signature share).
---
### ALERT-CLIP DECODE PIPE (RECIPE — TASK 47, 2026-09-26) — read a per-play mp4 into loose frames + stereo 48k audio
Worked out for the alert-box video: a short mp4 played **per alert** (one-shot) and dropped at
drain, NOT the refcounted media-source manager. Recipe:
- **Two ffmpeg pipe outputs, one child:** `ffmpeg -loglevel error -i "{file}" -f rawvideo -pix_fmt
bgra -vf scale={W}:{H} - -f f32le -ac 2 -ar 48000 -` — `-loglevel error` keeps stderr quiet (the
other pipes' ffmpeg rivals read the whole FD so a chatty stderr deadlocks); do **not** drain
stderr, mirror the `MediaVideoSource` split. Video pipe read backs onto `RawVideoFrameReader`
(TASK 21) which already pace-throttles to rtc-frame-rate… earlier live pacing is VFR-driven.
- **Audio = a carry-buffer loop, never a per-read slice.** PCM16 bytes → float32 chunks changes the
per-frame byte count; the naive `ReadFullyAsync(count)` per chunk **drops samples that straddle a
pipe read boundary** (the first draft's bug). The fix: an in-loop buffer that carries leftovers —
read `count - pending` bytes, emit whole float chunks from `pending + fresh`, keep the remainder.
Pacer: 50ms audio chunks → `Thread.Sleep` the (chunk length − read wall time) remainder.
- **bgra from ffmpeg is OPAQUE** (alpha=255). Whoever fades must copy the frame and scale alpha
across the copy (straight-alpha); the decoder never touches alpha. That keeps the decoder pure
and the fade a layer concern.
- **Fake-decoder convention:** the integration test implements `IAlertClipDecoder` inline with
`EmitFrame/EmitAudio/End` helpers — Start/Stop/Dispose are recorded as int counters, frames and
audio are pushed from the test thread, EOF is a test-raised event. No ffmpeg, no threads, still
the full real-time path through the layer.
- Re-grep here before building a second "play a file into the mix" path — snippet above is the
whole non-trivial part.
### A component's drain must reset its OWN anchor state (TASK 47 post-mortem — the test caught what a no-frame path hid)
`AlertOverlayLayer.Advance` gained a clip branch for video (TASK 47). In the ANIMATION branch the
old code did `_current = null; _elapsed = 0;` before `AdvanceToNext()`; the new clip branch called
only `StopClip(); AdvanceToNext();` — so after the EOF fade-out the layer kept `_current` pointing
at the finished message: `IsPlaying` stayed true, `RenderFrame` re-rendered the alert through the
animation fallback, and the layer never returned to idle. The six-animation tests couldn't see it
(they only ever exercise the animation branch); the video test's explicit
`Assert.False(layer.IsPlaying)` after drain nailed it on the first run. Rule: **when a second
execution path is added to a state machine, the drain/precondition contract of the ORIGINAL path is
part of the spec** — assert the component's end-state on the new path, don't assume the old path's
cleanup. The forest for the trip: the fallback path (animation) MASKED the broken path because its
output still looked like "an alert rendering", exactly the flick I'd have shipped if the test
hadn't forced the null-frame idle state.
+208
View File
@@ -0,0 +1,208 @@
using System.Diagnostics;
using ytLive.Services.Audio;
using ytLive.Services.Encoder;
namespace ytLive.Services;
/// <summary>
/// TASK 47: one alert clip's playback — video frames + on-stream audio from the
/// same file, both paced to real time. The alert box owns a decoder per event
/// (start at play, dispose at drain) — unlike MediaSource's app-wide refcounted
/// session, an alert REPLAYS the same clip each event, so a fresh decode per
/// play is required. Video is rawvideo BGRA scaled to the box rect (the
/// <see cref="MediaVideoSource"/> shape); audio is ffmpeg <c>f32le</c> at the
/// mixer's native 48 kHz stereo (so the mixer ring needs no resampler). The
/// fade is applied by the alert layer, not here.
/// </summary>
public interface IAlertClipDecoder : IDisposable
{
/// <summary>Starts the decode loop (video + audio); raises the events off the
/// calling thread. Paced to real time so playback matches wall clock.</summary>
void Start();
/// <summary>Stops and kills any running decode subprocesses.</summary>
void Stop();
/// <summary>One decoded video frame (box-sized, straight-alpha BGRA).</summary>
event Action<VideoFrame>? FrameAvailable;
/// <summary>One chunk of on-stream audio (48 kHz stereo float).</summary>
event Action<AudioSample>? AudioReady;
/// <summary>Raised once when both streams have fully played out.</summary>
event Action? Completed;
}
/// <summary>Default <see cref="IAlertClipDecoder"/>: two ffmpeg subprocesses on
/// the same file. Video via <see cref="RawVideoFrameReader"/> on a <c>rawvideo
/// bgra</c> pipe (-an), audio via an <c>f32le -ac 2 -ar 48000</c> pipe (-vn).
/// The seams (locator, process factory, frame-rate probe, delay) let tests drive
/// a fake decoder without a real codec or Windows.</summary>
public sealed class AlertClipDecoder : IAlertClipDecoder
{
public event Action<VideoFrame>? FrameAvailable;
public event Action<AudioSample>? AudioReady;
public event Action? Completed;
private const int AudioSampleRate = 48000;
private const int AudioChannels = 2;
private const int AudioChunkFloats = 4800; // 50ms of stereo
private readonly string _path;
private readonly int _width;
private readonly int _height;
private readonly IFfmpegLocator _locator;
private readonly Func<IDecodeProcess> _processFactory;
private readonly IFrameRateProbe? _probe;
private readonly Func<TimeSpan, CancellationToken, Task> _delay;
private readonly CancellationTokenSource _cts = new();
private Task? _run;
public AlertClipDecoder(
string path,
int width,
int height,
IFfmpegLocator locator,
Func<IDecodeProcess> processFactory,
IFrameRateProbe? frameRateProbe = null,
Func<TimeSpan, CancellationToken, Task>? delay = null)
{
_path = path;
_width = width;
_height = height;
_locator = locator;
_processFactory = processFactory;
_probe = frameRateProbe;
_delay = delay ?? ((t, ct) => Task.Delay(t, ct));
}
public void Start()
{
if (_run != null) return;
_run = Task.Run(RunAsync, CancellationToken.None);
}
public void Stop() => _cts.Cancel();
public void Dispose()
{
_cts.Cancel();
_cts.Dispose();
}
private async Task RunAsync()
{
var ct = _cts.Token;
try
{
double? frameRate = null;
if (_probe != null)
frameRate = await _probe.ProbeAsync(_path, ct).ConfigureAwait(false);
var frameDuration = frameRate is > 0 ? TimeSpan.FromSeconds(1.0 / frameRate.Value) : TimeSpan.Zero;
var ffmpegPath = await _locator.LocateAsync(ct).ConfigureAwait(false);
var videoTask = RunVideoAsync(ffmpegPath, frameDuration, ct);
var audioTask = RunAudioAsync(ffmpegPath, ct);
await Task.WhenAll(videoTask, audioTask).ConfigureAwait(false);
}
catch (OperationCanceledException)
{
return;
}
catch
{
// A failed decode is not fatal: the alert layer falls back to the six
// animations (or the frame it already holds) exactly as a missing file.
}
if (!ct.IsCancellationRequested)
Completed?.Invoke();
}
private async Task RunVideoAsync(string ffmpegPath, TimeSpan frameDuration, CancellationToken ct)
{
var args =
$"-hide_banner -loglevel error -i \"{_path}\" " +
$"-f rawvideo -pix_fmt bgra -vf scale={_width}:{_height} -an -";
using var process = _processFactory();
process.Start(new ProcessStartInfo
{
FileName = ffmpegPath,
Arguments = args,
UseShellExecute = false,
RedirectStandardOutput = true,
RedirectStandardError = true,
CreateNoWindow = true,
});
var reader = new RawVideoFrameReader(_width, _height);
var buffer = new byte[65536];
var stream = process.StandardOutput;
while (!ct.IsCancellationRequested && !process.HasExited)
{
var read = await stream.ReadAsync(buffer, 0, buffer.Length, ct).ConfigureAwait(false);
if (read <= 0) break;
var chunk = new byte[read];
Buffer.BlockCopy(buffer, 0, chunk, 0, read);
foreach (var frame in reader.Feed(chunk))
{
FrameAvailable?.Invoke(frame);
if (frameDuration > TimeSpan.Zero)
await _delay(frameDuration, ct).ConfigureAwait(false);
}
}
}
private async Task RunAudioAsync(string ffmpegPath, CancellationToken ct)
{
var args =
$"-hide_banner -loglevel error -i \"{_path}\" " +
$"-f f32le -ac {AudioChannels} -ar {AudioSampleRate} -vn -";
using var process = _processFactory();
process.Start(new ProcessStartInfo
{
FileName = ffmpegPath,
Arguments = args,
UseShellExecute = false,
RedirectStandardOutput = true,
RedirectStandardError = true,
CreateNoWindow = true,
});
var stream = process.StandardOutput;
var raw = new byte[65536];
var carry = new byte[0];
var floatScratch = new float[AudioChunkFloats];
var floatIndex = 0;
// The pipe is byte-delimited only: a read may split a float, so leftover
// bytes are carried into the next read instead of dropped (a dropped
// half-float would desync the entire stream).
while (!ct.IsCancellationRequested && !process.HasExited)
{
var read = await stream.ReadAsync(raw, 0, raw.Length, ct).ConfigureAwait(false);
if (read <= 0) break;
var total = carry.Length + read;
var bytes = total <= raw.Length ? raw : new byte[total];
if (carry.Length > 0)
{
Buffer.BlockCopy(carry, 0, bytes, 0, carry.Length);
Buffer.BlockCopy(raw, 0, bytes, carry.Length, read);
}
var floatsTo = total - (total % 4);
for (var i = 0; i < floatsTo && floatIndex < floatScratch.Length; i += 4)
floatScratch[floatIndex++] = BitConverter.ToSingle(bytes, i);
carry = total > floatsTo ? bytes[floatsTo..total] : new byte[0];
while (floatIndex == floatScratch.Length)
{
AudioReady?.Invoke(new AudioSample(floatScratch, AudioSampleRate, AudioChannels));
// Pace the NEXT chunk so the whole clip's audio lands at real-time
// cadence (the fade in the alert layer tracks the same wall clock).
await _delay(TimeSpan.FromSeconds(AudioChunkFloats / (double)(AudioSampleRate * AudioChannels)), ct).ConfigureAwait(false);
floatScratch = new float[AudioChunkFloats];
floatIndex = 0;
}
}
}
}
+223 -20
View File
@@ -5,28 +5,59 @@ using System.Windows.Media;
using System.Windows.Media.Imaging; using System.Windows.Media.Imaging;
using System.Windows.Threading; using System.Windows.Threading;
using ytLive.Models; using ytLive.Models;
using ytLive.Services.Audio;
using ytLive.Services.Compositor;
namespace ytLive.Services; namespace ytLive.Services;
/// <summary> /// <summary>
/// The native alert box (OBS-style celebration zone, no third party). Owns the /// The native alert box (OBS-style celebration zone, no third party). Owns the
/// event queue and the animation clock: <see cref="ChatEventKind"/> events from /// event queue and the animation clock: <see cref="ChatEventKind"/> events from
/// the chat feed are enqueued one-at-a-time and played through their own unique /// the chat feed are enqueued one-at-a-time and played through a celebration
/// animation (<see cref="AlertRenderer"/>), frames served to the stream compositor /// animation (<see cref="AlertRenderer"/>) OR — when the alert-box video is
/// via <see cref="RenderFrame"/> and mirrored to the preview via /// configured and resolves — through a decoded clip. Frames are served to the
/// <see cref="UpdatePreview"/>. Idle = transparent (null frame); playback is /// stream compositor via <see cref="RenderFrame"/> and mirrored to the preview
/// via <see cref="UpdatePreview"/>. Idle = transparent (null frame); playback is
/// advanced by a 33ms dispatch ticker. The clock is exposed as /// advanced by a 33ms dispatch ticker. The clock is exposed as
/// <see cref="Advance"/> so tests drive playback deterministically without the /// <see cref="Advance"/> so tests drive playback deterministically without the
/// timer. Ordinary chat rows (<see cref="ChatEventKind.None"/>) never enqueue — /// timer. Ordinary chat rows (<see cref="ChatEventKind.None"/>) never enqueue —
/// a subscriber mention is a chat row, not an alert. /// a subscriber mention is a chat row, not an alert.
///
/// TASK 47 (Stream Alerts video): a video-backed alert replaces the animation
/// while a clip decodes (<see cref="IAlertClipDecoder"/>, one per play). The
/// clip plays at its own media pace; the layer applies a ~300ms fade-in/out
/// alpha envelope to the frames AND the audio samples (freeze-frame fade-out on
/// the tail), clips the alert to the clip length, and forwards the scaled audio
/// to the mixer's alert ring via <paramref name="audioSink"/> (no duck). The
/// per-source config (path/default/ticker/volume) is resolved through
/// <paramref name="clipPathResolver"/>; the ticker strip is served via
/// <see cref="AlertTickerFrame"/>.
/// </summary> /// </summary>
public sealed class AlertOverlayLayer : IDisposable public sealed class AlertOverlayLayer : IDisposable
{ {
/// <summary>The fade-in/out ramp length (video alpha AND audio gain). Measured
/// against a real clip in the AlertLayerVideoTests unit; keep the ramp short
/// enough to read as a "pop-in" on stream (see ai.md).</summary>
internal const double FadeDurationSeconds = 0.30;
private static readonly TimeSpan TickInterval = TimeSpan.FromMilliseconds(33);
private readonly AlertRenderer _renderer; private readonly AlertRenderer _renderer;
private readonly Func<string, int, int, IAlertClipDecoder>? _clipDecoderFactory;
private readonly Func<Source, string?>? _clipPathResolver;
private readonly Action<AudioSample>? _audioSink;
private readonly Queue<ChatMessage> _queue = new(); private readonly Queue<ChatMessage> _queue = new();
private readonly DispatcherTimer? _ticker; private readonly DispatcherTimer? _ticker;
private ChatMessage? _current; private ChatMessage? _current;
private Source? _currentBox;
private IAlertClipDecoder? _clip;
private VideoFrame? _latestClipFrame;
private bool _clipEof;
private double _clipEofElapsed;
private int _clipWidth;
private int _clipHeight;
private string? _tickerText;
private double _elapsed; private double _elapsed;
private IEnumerable<Scene>? _scenes; private IEnumerable<Scene>? _scenes;
@@ -35,11 +66,16 @@ public sealed class AlertOverlayLayer : IDisposable
private string? _frameKey; private string? _frameKey;
private VideoFrame? _cachedFrame; private VideoFrame? _cachedFrame;
private static readonly TimeSpan TickInterval = TimeSpan.FromMilliseconds(33); public AlertOverlayLayer(
AlertRenderer renderer,
public AlertOverlayLayer(AlertRenderer renderer) Func<string, int, int, IAlertClipDecoder>? clipDecoderFactory = null,
Func<Source, string?>? clipPathResolver = null,
Action<AudioSample>? audioSink = null)
{ {
_renderer = renderer; _renderer = renderer;
_clipDecoderFactory = clipDecoderFactory;
_clipPathResolver = clipPathResolver;
_audioSink = audioSink;
if (Application.Current != null) if (Application.Current != null)
{ {
_ticker = new DispatcherTimer { Interval = TickInterval }; _ticker = new DispatcherTimer { Interval = TickInterval };
@@ -73,6 +109,7 @@ public sealed class AlertOverlayLayer : IDisposable
{ {
_current = _queue.Dequeue(); _current = _queue.Dequeue();
_elapsed = 0; _elapsed = 0;
BeginClip();
} }
_contentVersion++; _contentVersion++;
} }
@@ -82,11 +119,30 @@ public sealed class AlertOverlayLayer : IDisposable
/// nothing stops the ticker and leaves the box transparent.</summary> /// nothing stops the ticker and leaves the box transparent.</summary>
public void Advance(double deltaSeconds) public void Advance(double deltaSeconds)
{ {
// Video-backed alert: duration is clip-driven. The clip plays at media
// pace; once it EOFs the last frame sweeps through the fade-out before
// the alert moves on (a hard cut at EOF would pop, not fade).
if (_clip != null)
{
_elapsed += deltaSeconds;
if (_clipEof && _elapsed - _clipEofElapsed >= FadeDurationSeconds)
{
_clipEof = false;
StopClip();
_current = null;
_elapsed = 0;
AdvanceToNext();
}
_contentVersion++;
return;
}
if (_current == null) if (_current == null)
{ {
if (_queue.Count == 0) return; if (_queue.Count == 0) return;
_current = _queue.Dequeue(); _current = _queue.Dequeue();
_elapsed = 0; _elapsed = 0;
BeginClip();
_contentVersion++; _contentVersion++;
return; return;
} }
@@ -96,15 +152,7 @@ public sealed class AlertOverlayLayer : IDisposable
{ {
_current = null; _current = null;
_elapsed = 0; _elapsed = 0;
if (_queue.Count > 0) AdvanceToNext();
{
_current = _queue.Dequeue();
_elapsed = 0;
}
else
{
_ticker?.Stop();
}
} }
_contentVersion++; _contentVersion++;
} }
@@ -112,10 +160,9 @@ public sealed class AlertOverlayLayer : IDisposable
/// <summary>Whether an alert is currently playing or queued.</summary> /// <summary>Whether an alert is currently playing or queued.</summary>
public bool IsPlaying => _current != null || _queue.Count > 0; public bool IsPlaying => _current != null || _queue.Count > 0;
/// <summary>The live-output frame for one AlertBox element: the current /// <summary>The live-output frame for one AlertBox element: the decoded clip
/// animation frame while playing, null (transparent) when idle. Mirrors the /// frame (faded) while a clip is active, else the animation frame, null
/// chat cache pattern — the raster is WPF/UI-thread, so only the cache miss /// (transparent) when idle.</summary>
/// marshals to the dispatcher and re-validates there.</summary>
public VideoFrame? RenderFrame(Source alertBox) public VideoFrame? RenderFrame(Source alertBox)
{ {
if (_current == null) if (_current == null)
@@ -124,6 +171,9 @@ public sealed class AlertOverlayLayer : IDisposable
return null; return null;
} }
if (_clip != null)
return RenderClipFrame(alertBox);
var width = (int)Math.Max(1, alertBox.Width); var width = (int)Math.Max(1, alertBox.Width);
var height = (int)Math.Max(1, alertBox.Height); var height = (int)Math.Max(1, alertBox.Height);
var key = $"{width}x{height}|{_current.Id}|{Progress()}"; var key = $"{width}x{height}|{_current.Id}|{Progress()}";
@@ -147,6 +197,139 @@ public sealed class AlertOverlayLayer : IDisposable
return RenderCore(alertBox, width, height, key, _contentVersion); return RenderCore(alertBox, width, height, key, _contentVersion);
} }
/// <summary>The global ticker strip for the stream output (top edge, full
/// master width): a marquee of "viewer — event" while an alert is playing
/// and the config has the ticker on, else null. Pure byte-math per call so
/// the frame pump can read it on any thread (the text strip is rasterized
/// once on the UI thread and cached).</summary>
public VideoFrame? AlertTickerFrame
{
get
{
if (_current == null || string.IsNullOrWhiteSpace(_tickerText)) return null;
return AlertTickerRenderer.Render(_tickerText!, _elapsed);
}
}
private VideoFrame? RenderClipFrame(Source alertBox)
{
var frame = _latestClipFrame;
if (frame == null) return null;
var w = (int)Math.Max(1, alertBox.Width);
var h = (int)Math.Max(1, alertBox.Height);
if (frame.Width != w || frame.Height != h)
frame = StretchMath.BilinearScale(frame, w, h);
var fade = FadeAt();
if (fade >= 1.0) return frame;
// Straight-alpha source-over on the compositor side: scale the source
// alpha by the envelope so the clip blends out (a 680x200 copy per fade
// tick is trivial; outside the ramps fade==1 returns the raw frame).
var copy = new byte[frame.BgraPixels.Length];
Buffer.BlockCopy(frame.BgraPixels, 0, copy, 0, copy.Length);
var a = (byte)(fade * 255 + 0.5);
for (var i = 3; i < copy.Length; i += 4)
copy[i] = (byte)((copy[i] * a) >> 8);
return new VideoFrame(frame.Width, frame.Height, copy);
}
private double FadeAt()
{
if (_clip == null) return 1.0;
if (_clipEof)
return Math.Clamp(1.0 - (_elapsed - _clipEofElapsed) / FadeDurationSeconds, 0.0, 1.0);
return Math.Clamp(_elapsed / FadeDurationSeconds, 0.0, 1.0);
}
private void BeginClip()
{
StopClip();
_tickerText = null;
if (_current == null) return;
var box = _scenes?.SelectMany(s => s.Elements).OfType<Source>()
.FirstOrDefault(s => s.Type == SourceType.AlertBox);
_currentBox = box;
if (box != null && box.AlertShowTicker)
_tickerText = ComposeTicker(_current);
if (box == null || _clipDecoderFactory == null) return;
var path = _clipPathResolver?.Invoke(box);
if (string.IsNullOrWhiteSpace(path)) return;
_clipWidth = (int)Math.Max(1, box.Width);
_clipHeight = (int)Math.Max(1, box.Height);
try
{
_clip = _clipDecoderFactory(path, _clipWidth, _clipHeight);
_clip.FrameAvailable += OnClipFrame;
_clip.AudioReady += OnClipAudio;
_clip.Completed += OnClipCompleted;
_clip.Start();
}
catch
{
StopClip(); // decode setup failed (bad path) — fall back to animations
}
}
private void StopClip()
{
var clip = _clip;
_clip = null;
_latestClipFrame = null;
_clipEof = false;
if (clip == null) return;
clip.FrameAvailable -= OnClipFrame;
clip.AudioReady -= OnClipAudio;
clip.Completed -= OnClipCompleted;
try { clip.Stop(); } catch { /* already stopped */ }
clip.Dispose();
}
private void OnClipFrame(VideoFrame frame) => _latestClipFrame = frame;
private void OnClipCompleted()
{
_clipEof = true;
_clipEofElapsed = _elapsed;
}
private void OnClipAudio(AudioSample sample)
{
if (_audioSink == null) return;
var volume = (float)Math.Clamp(_currentBox?.AlertVideoVolume ?? 1.0, 0.0, 1.0);
if (volume <= 0f) return;
var fade = FadeAt();
if (fade <= 0.0) return;
var data = sample.Samples;
if (fade < 1.0 || Math.Abs(volume - 1f) > 0.001f)
{
data = new float[data.Length];
var gain = volume * (float)fade;
for (var i = 0; i < sample.Samples.Length; i++)
data[i] = sample.Samples[i] * gain;
}
_audioSink(new AudioSample(data, sample.SampleRate, sample.Channels));
}
private void AdvanceToNext()
{
if (_queue.Count > 0)
{
_current = _queue.Dequeue();
_elapsed = 0;
BeginClip();
}
else
{
_ticker?.Stop();
}
}
private VideoFrame? RenderCore(Source alertBox, int width, int height, string key, int version) private VideoFrame? RenderCore(Source alertBox, int width, int height, string key, int version)
{ {
_frameKey = key; _frameKey = key;
@@ -172,6 +355,25 @@ public sealed class AlertOverlayLayer : IDisposable
_ => 4.0, _ => 4.0,
}; };
private static string ComposeTicker(ChatMessage m)
{
var label = m.Kind switch
{
ChatEventKind.SuperChat => "Super Chat",
ChatEventKind.SuperSticker => "Super Sticker",
ChatEventKind.NewMember => "New Member",
ChatEventKind.MemberGift => "Membership Gift",
ChatEventKind.GiftReceived => "Gifted Membership",
ChatEventKind.MemberMilestone => "Member Milestone",
_ => "Stream Alert",
};
var amount = m.Kind is ChatEventKind.SuperChat or ChatEventKind.SuperSticker
&& m.SuperChatDisplayString is { Length: > 0 } s
? $" · {s}"
: "";
return $"{m.AuthorName} — {label}{amount}";
}
/// <summary>Render the current state onto one AlertBox preview (mirrors chat): /// <summary>Render the current state onto one AlertBox preview (mirrors chat):
/// the playing frame, or null when idle so the source shows nothing until an /// the playing frame, or null when idle so the source shows nothing until an
/// event actually fires.</summary> /// event actually fires.</summary>
@@ -227,5 +429,6 @@ public sealed class AlertOverlayLayer : IDisposable
_ticker?.Stop(); _ticker?.Stop();
_queue.Clear(); _queue.Clear();
_current = null; _current = null;
StopClip();
} }
} }
+49 -3
View File
@@ -14,7 +14,9 @@ namespace ytLive.Services.Audio;
/// queues, applies the honest gains (mic = MicVolume, loopback = unity /// queues, applies the honest gains (mic = MicVolume, loopback = unity
/// × auto-duck when the mic is hot — the slider drives system volume via mixes them to stereo float, runs the /// × auto-duck when the mic is hot — the slider drives system volume via mixes them to stereo float, runs the
/// master limiter (a -1 dBFS ceiling so the sum never clips the encoder), and /// master limiter (a -1 dBFS ceiling so the sum never clips the encoder), and
/// writes the chunk to the encoder's audio pipe. /// writes the chunk to the encoder's audio pipe. Since TASK 47 the alert layer's
/// clip audio joins the same mix (pre-scaled, unity-added, never ducked) through
/// <see cref="EnqueueAlertAudio"/>.
/// </summary> /// </summary>
public sealed class AudioMixer : IDisposable public sealed class AudioMixer : IDisposable
{ {
@@ -39,6 +41,7 @@ public sealed class AudioMixer : IDisposable
// thread-safe queues → the ducker + gain/mix loop → the encoder's pipe. // thread-safe queues → the ducker + gain/mix loop → the encoder's pipe.
private readonly AudioRingBuffer _micBuffer; private readonly AudioRingBuffer _micBuffer;
private readonly AudioRingBuffer _loopbackBuffer; private readonly AudioRingBuffer _loopbackBuffer;
private readonly AudioRingBuffer _alertBuffer;
private readonly VoiceFilterChain _voiceChain; private readonly VoiceFilterChain _voiceChain;
private readonly AutoDucker _ducker; private readonly AutoDucker _ducker;
private readonly MasterLimiter _masterLimiter = new(); private readonly MasterLimiter _masterLimiter = new();
@@ -48,6 +51,7 @@ public sealed class AudioMixer : IDisposable
private NamedPipeAudioWriter _pipe = new(); private NamedPipeAudioWriter _pipe = new();
private float[]? _micChunk; private float[]? _micChunk;
private float[]? _loopbackChunk; private float[]? _loopbackChunk;
private float[]? _alertChunk;
private float[]? _mixBuffer; private float[]? _mixBuffer;
private float[]? _delayedMix; private float[]? _delayedMix;
private DateTime _lastLoopErrorLogged = DateTime.MinValue; private DateTime _lastLoopErrorLogged = DateTime.MinValue;
@@ -80,10 +84,12 @@ public sealed class AudioMixer : IDisposable
var framesPerTick = Math.Max(1, (int)Math.Round(OutputSampleRate * seconds)); var framesPerTick = Math.Max(1, (int)Math.Round(OutputSampleRate * seconds));
_micBuffer = new AudioRingBuffer(OutputSampleRate * 2); _micBuffer = new AudioRingBuffer(OutputSampleRate * 2);
_loopbackBuffer = new AudioRingBuffer(OutputSampleRate * 2 * 2); _loopbackBuffer = new AudioRingBuffer(OutputSampleRate * 2 * 2);
_alertBuffer = new AudioRingBuffer(OutputSampleRate * 2 * 8);
_voiceChain = new VoiceFilterChain(OutputSampleRate); _voiceChain = new VoiceFilterChain(OutputSampleRate);
_ducker = new AutoDucker(); _ducker = new AutoDucker();
_micChunk = new float[framesPerTick]; _micChunk = new float[framesPerTick];
_loopbackChunk = new float[framesPerTick * 2]; _loopbackChunk = new float[framesPerTick * 2];
_alertChunk = new float[framesPerTick * 2];
_mixBuffer = new float[framesPerTick * 2]; _mixBuffer = new float[framesPerTick * 2];
_delayedMix = new float[framesPerTick * 2]; _delayedMix = new float[framesPerTick * 2];
@@ -168,6 +174,7 @@ public sealed class AudioMixer : IDisposable
// Clear under the ring's lock, so only the in-flight chunk is dropped. // Clear under the ring's lock, so only the in-flight chunk is dropped.
_micBuffer.Clear(); _micBuffer.Clear();
_loopbackBuffer.Clear(); _loopbackBuffer.Clear();
_alertBuffer.Clear();
// Negative sync offset (audio runs BEHIND video): OBS's fix is to "eat // Negative sync offset (audio runs BEHIND video): OBS's fix is to "eat
// the head of the buffer" — discard the first |N| ms of the live stream // the head of the buffer" — discard the first |N| ms of the live stream
@@ -246,6 +253,34 @@ public sealed class AudioMixer : IDisposable
LoopbackLevelChanged?.Invoke(level); LoopbackLevelChanged?.Invoke(level);
} }
/// <summary>Drops the alert clip's audio into its own ring (TASK 47 — the
/// stream-alert video's soundtrack, pre-scaled by the alert layer to
/// volume × fade). Mixed at unity into the live output — NO auto-duck: an
/// alert must be heard over the mic, so the Creator's per-alert volume
/// slider is the honest gain and nothing else is stacked on top. Monno and
/// non-48k sources are normalized (the decoder already emits 48k stereo).</summary>
public void EnqueueAlertAudio(AudioSample sample)
{
var data = sample.Samples;
if (data.Length == 0) return;
if (sample.SampleRate != OutputSampleRate)
data = ResampleLoopback(data, sample.SampleRate);
if (sample.Channels == 1)
data = UpmixToStereo(data);
_alertBuffer.Write(data);
}
private static float[] UpmixToStereo(float[] mono)
{
var stereo = new float[mono.Length * 2];
for (var i = 0; i < mono.Length; i++)
{
stereo[i * 2] = mono[i];
stereo[i * 2 + 1] = mono[i];
}
return stereo;
}
private void OnMicFailed(Exception ex) private void OnMicFailed(Exception ex)
{ {
_log?.Invoke($"Mic capture failed: {ex.Message}"); _log?.Invoke($"Mic capture failed: {ex.Message}");
@@ -386,6 +421,8 @@ public sealed class AudioMixer : IDisposable
{ {
var micCount = _micBuffer.Read(micChunk, micChunk.Length); var micCount = _micBuffer.Read(micChunk, micChunk.Length);
var loopCount = _loopbackBuffer.Read(loopbackChunk, loopbackChunk.Length); var loopCount = _loopbackBuffer.Read(loopbackChunk, loopbackChunk.Length);
var alertChunk = _alertChunk!;
var alertCount = _alertBuffer.Read(alertChunk, alertChunk.Length);
float micRms = 0; float micRms = 0;
if (micCount == 0) if (micCount == 0)
@@ -411,6 +448,15 @@ public sealed class AudioMixer : IDisposable
Array.Clear(loopbackChunk, loopCount, loopbackChunk.Length - loopCount); Array.Clear(loopbackChunk, loopCount, loopbackChunk.Length - loopCount);
} }
if (alertCount == 0)
{
Array.Clear(alertChunk, 0, alertChunk.Length);
}
else if (alertCount < alertChunk.Length)
{
Array.Clear(alertChunk, alertCount, alertChunk.Length - alertCount);
}
var duck = _ducker.Update(micRms); var duck = _ducker.Update(micRms);
var micGain = (float)(_micGain?.Invoke() ?? 1.0); var micGain = (float)(_micGain?.Invoke() ?? 1.0);
var loopGain = (float)(_loopbackGain?.Invoke() ?? 1.0) * duck; var loopGain = (float)(_loopbackGain?.Invoke() ?? 1.0) * duck;
@@ -419,8 +465,8 @@ public sealed class AudioMixer : IDisposable
for (var i = 0; i < frames; i++) for (var i = 0; i < frames; i++)
{ {
var m = micChunk[i] * micGain; var m = micChunk[i] * micGain;
mix[i * 2] = m + loopbackChunk[i * 2] * loopGain; mix[i * 2] = m + loopbackChunk[i * 2] * loopGain + alertChunk[i * 2];
mix[i * 2 + 1] = m + loopbackChunk[i * 2 + 1] * loopGain; mix[i * 2 + 1] = m + loopbackChunk[i * 2 + 1] * loopGain + alertChunk[i * 2 + 1];
} }
_masterLimiter.Process(mix); _masterLimiter.Process(mix);
+139
View File
@@ -0,0 +1,139 @@
using System.Collections.Generic;
using System.Globalization;
using System.Windows;
using System.Windows.Media;
using System.Windows.Media.Imaging;
using System.Windows.Threading;
namespace ytLive.Services.Compositor;
/// <summary>
/// Rasterizes the stream-alert ticker into a transparent BGRA strip that the
/// output compositor overlays along the very top edge of the master frame
/// (TASK 47). The strip is text ("Viewer — Super Chat · $12.34") inside a dark
/// pill with the LlamaCasty accent bar, rendered ONCE per unique text on the UI
/// thread (WPF <c>RenderTargetBitmap</c>, Pbgra32→straight-alpha like
/// <see cref="SocialBarRenderer"/>), then cached. Each <see cref="Render"/> call
/// is pure byte-math on that cached raster producing a scrolling marquee frame,
/// so the frame pump can call it from any thread for every output frame at
/// 30fps without a Dispatcher round-trip.
/// </summary>
public static class AlertTickerRenderer
{
/// <summary>Master-frame width — the ticker spans the full 1920px output.</summary>
public const int Width = 1920;
/// <summary>Strip height (pill + vertical padding).</summary>
public const int Height = 48;
private const int PixelsPerSecond = 140;
private const int RepeatGap = 120;
private const int CacheLimit = 48;
private static readonly object Gate = new();
private static readonly Dictionary<string, VideoFrame> StripCache = new();
/// <summary>A marquee frame of <paramref name="text"/> scrolled
/// <paramref name="elapsedSeconds"/> into its loop, or null for empty text.
/// Thread-safe: reads the cached raster and does row copies only.</summary>
public static VideoFrame? Render(string? text, double elapsedSeconds)
{
if (string.IsNullOrWhiteSpace(text)) return null;
var strip = GetStrip(text!);
if (strip == null) return null;
var sw = strip.Width;
var cycle = Width + sw + RepeatGap;
var pos = (int)(elapsedSeconds * PixelsPerSecond) % cycle;
var x = Width - pos;
var pixels = new byte[Width * Height * 4];
CopyStrip(pixels, strip, x);
if (x < 0)
CopyStrip(pixels, strip, x + Width + RepeatGap); // wrapped tail enters from the right
return new VideoFrame(Width, Height, pixels);
}
private static void CopyStrip(byte[] dst, VideoFrame strip, int dstX)
{
if (dstX >= Width || dstX + strip.Width <= 0) return;
var x0 = Math.Max(0, dstX);
var x1 = Math.Min(Width, dstX + strip.Width);
var sx0 = x0 - dstX;
var copyBytes = (x1 - x0) * 4;
var sw = strip.Width;
for (var y = 0; y < strip.Height; y++)
Buffer.BlockCopy(strip.BgraPixels, (y * sw + sx0) * 4, dst, (y * Width + x0) * 4, copyBytes);
}
private static VideoFrame? GetStrip(string text)
{
lock (Gate)
{
if (StripCache.TryGetValue(text, out var cached)) return cached;
}
var strip = RenderStrip(text);
if (strip == null) return null;
lock (Gate)
{
if (StripCache.Count >= CacheLimit) StripCache.Clear();
if (!StripCache.TryGetValue(text, out var existing))
StripCache[text] = strip;
return StripCache[text];
}
}
private static VideoFrame? RenderStrip(string text)
{
// WPF rendering is single-threaded; the pump thread resolves to the UI
// thread at most once per unique text (then hits the cache).
var app = Application.Current;
if (app != null && !app.Dispatcher.CheckAccess())
{
VideoFrame? result = null;
app.Dispatcher.Invoke(() => result = RenderStrip(text));
return result;
}
const double pillHeight = 34;
const double accentW = 6;
const double padX = 16;
const double gap = 8;
var font = new Typeface(new FontFamily("Segoe UI"), FontStyles.Normal, FontWeights.SemiBold, FontStretches.Normal);
var ft = new FormattedText(text, CultureInfo.InvariantCulture, FlowDirection.LeftToRight,
font, 22, Brushes.White, 1.0);
var pillW = (int)Math.Ceiling(padX + accentW + gap + ft.Width + padX);
var top = (Height - pillHeight) / 2.0;
var textTop = top + (pillHeight - ft.Height) / 2.0;
var visual = new DrawingVisual();
using (var dc = visual.RenderOpen())
{
var pill = new Rect(0, top, pillW, pillHeight);
dc.DrawRoundedRectangle(new SolidColorBrush(Color.FromArgb(0xE6, 0x10, 0x13, 0x24)), null, pill, pillHeight / 2, pillHeight / 2);
dc.DrawRoundedRectangle(new SolidColorBrush(Color.FromRgb(0xe9, 0x45, 0x60)), null,
new Rect(padX, top + (pillHeight - 12) / 2.0, accentW, 12), 2, 2);
dc.DrawText(ft, new Point(padX + accentW + gap, textTop));
}
var bitmap = new RenderTargetBitmap(pillW, Height, 96, 96, PixelFormats.Pbgra32);
bitmap.Render(visual);
var pixels = new byte[pillW * Height * 4];
bitmap.CopyPixels(pixels, pillW * 4, 0);
// Pbgra32 is premultiplied — unpremultiply so the compositor's
// straight-alpha source-over doesn't halo the text edges.
for (var i = 0; i < pixels.Length; i += 4)
{
var a = pixels[i + 3];
if (a == 0 || a == 255) continue;
pixels[i] = (byte)(pixels[i] * 255 / a);
pixels[i + 1] = (byte)(pixels[i + 1] * 255 / a);
pixels[i + 2] = (byte)(pixels[i + 2] * 255 / a);
}
return new VideoFrame(pillW, Height, pixels);
}
}
+17 -4
View File
@@ -37,7 +37,8 @@ public sealed class SceneCompositor
/// front): live background (the scene's <c>IsBackground</c> source) → background image → /// front): live background (the scene's <c>IsBackground</c> source) → background image →
/// visible elements (z-order = <c>Elements</c> order, mirroring the XAML DataTemplate) → /// visible elements (z-order = <c>Elements</c> order, mirroring the XAML DataTemplate) →
/// branding flash → social bar (a global overlay; spans the master width at /// branding flash → social bar (a global overlay; spans the master width at
/// <paramref name="socialBarTop"/>). Transparent regions read opaque black. /// <paramref name="socialBarTop"/>) → stream-alert ticker (a global overlay pinned to
/// the very top, TASK 47). Transparent regions read opaque black.
/// </summary> /// </summary>
public VideoFrame Render( public VideoFrame Render(
Scene scene, Scene scene,
@@ -48,7 +49,8 @@ public sealed class SceneCompositor
int socialBarTop = 0, int socialBarTop = 0,
VideoFrame? staticBase = null, VideoFrame? staticBase = null,
int split = -1, int split = -1,
byte[]? scratch = null) byte[]? scratch = null,
VideoFrame? tickerFrame = null)
{ {
if (scene == null) throw new ArgumentNullException(nameof(scene)); if (scene == null) throw new ArgumentNullException(nameof(scene));
if (frameFor == null) throw new ArgumentNullException(nameof(frameFor)); if (frameFor == null) throw new ArgumentNullException(nameof(frameFor));
@@ -137,6 +139,9 @@ public sealed class SceneCompositor
if (socialBarFrame != null && socialBarFrame.Width > 0 && socialBarFrame.Height > 0) if (socialBarFrame != null && socialBarFrame.Width > 0 && socialBarFrame.Height > 0)
BlitOverlay(buffer, cropW, cropH, options, socialBarFrame, 0, socialBarTop); BlitOverlay(buffer, cropW, cropH, options, socialBarFrame, 0, socialBarTop);
if (tickerFrame != null && tickerFrame.Width > 0 && tickerFrame.Height > 0)
BlitOverlay(buffer, cropW, cropH, options, tickerFrame, 0, 0);
return StretchMath.BilinearScale( return StretchMath.BilinearScale(
new VideoFrame(cropW, cropH, buffer), options.OutputWidth, options.OutputHeight); new VideoFrame(cropW, cropH, buffer), options.OutputWidth, options.OutputHeight);
} }
@@ -212,7 +217,10 @@ public sealed class SceneCompositor
/// <summary>The compositor's dynamic/above-split path: start from an existing frame /// <summary>The compositor's dynamic/above-split path: start from an existing frame
/// buffer and composite the z-layers from <paramref name="split"/> onward on top. /// buffer and composite the z-layers from <paramref name="split"/> onward on top.
/// When <paramref name="baseFrame"/> is null the caller must fall back to full render.</summary> /// When <paramref name="baseFrame"/> is null the caller must fall back to full render.
/// The social bar never reaches here (it IS baked into the static base) but the alert
/// ticker must: it is dynamic by construction — only ever present while an alert plays —
/// so it overlays LAST, pinned to the very top (TASK 47).</summary>
public static VideoFrame CompositeLayers( public static VideoFrame CompositeLayers(
VideoFrame baseFrame, VideoFrame baseFrame,
Scene scene, Scene scene,
@@ -222,7 +230,8 @@ public sealed class SceneCompositor
VideoFrame? socialBarFrame = null, VideoFrame? socialBarFrame = null,
int socialBarTop = 0, int socialBarTop = 0,
VideoFrame? flashFrame = null, VideoFrame? flashFrame = null,
byte[]? scratch = null) byte[]? scratch = null,
VideoFrame? tickerFrame = null)
{ {
var cropW = options.SourceRectWidth; var cropW = options.SourceRectWidth;
var cropH = options.SourceRectHeight; var cropH = options.SourceRectHeight;
@@ -247,6 +256,10 @@ public sealed class SceneCompositor
if (frame == null) continue; if (frame == null) continue;
BlitBakedElement(buffer, cropW, cropH, options, element, frame); BlitBakedElement(buffer, cropW, cropH, options, element, frame);
} }
if (tickerFrame != null && tickerFrame.Width > 0 && tickerFrame.Height > 0)
BlitOverlay(buffer, cropW, cropH, options, tickerFrame, 0, 0);
return StretchMath.BilinearScale(new VideoFrame(cropW, cropH, buffer), options.OutputWidth, options.OutputHeight); return StretchMath.BilinearScale(new VideoFrame(cropW, cropH, buffer), options.OutputWidth, options.OutputHeight);
} }
+19 -6
View File
@@ -29,6 +29,7 @@ public sealed class FramePump : IDisposable
private readonly Action<string>? _log; private readonly Action<string>? _log;
private readonly Func<TimeSpan, CancellationToken, Task> _pacingDelay; private readonly Func<TimeSpan, CancellationToken, Task> _pacingDelay;
private readonly Func<(VideoFrame? Frame, SocialBarPosition Position)>? _socialBar; private readonly Func<(VideoFrame? Frame, SocialBarPosition Position)>? _socialBar;
private readonly Func<VideoFrame?>? _alertTicker;
private readonly TransitionService? _transition; private readonly TransitionService? _transition;
private readonly SceneGraph? _sceneGraph; private readonly SceneGraph? _sceneGraph;
private readonly SceneCompositor _compositor = new(); private readonly SceneCompositor _compositor = new();
@@ -129,6 +130,7 @@ public sealed class FramePump : IDisposable
Action<string>? log = null, Action<string>? log = null,
Func<TimeSpan, CancellationToken, Task>? pacingDelay = null, Func<TimeSpan, CancellationToken, Task>? pacingDelay = null,
Func<(VideoFrame? Frame, SocialBarPosition Position)>? socialBar = null, Func<(VideoFrame? Frame, SocialBarPosition Position)>? socialBar = null,
Func<VideoFrame?>? alertTicker = null,
TransitionService? transition = null, TransitionService? transition = null,
SceneGraph? sceneGraph = null) SceneGraph? sceneGraph = null)
{ {
@@ -140,6 +142,7 @@ public sealed class FramePump : IDisposable
_log = log; _log = log;
_pacingDelay = pacingDelay ?? ((delay, ct) => Task.Delay(delay, ct)); _pacingDelay = pacingDelay ?? ((delay, ct) => Task.Delay(delay, ct));
_socialBar = socialBar; _socialBar = socialBar;
_alertTicker = alertTicker;
_transition = transition; _transition = transition;
_sceneGraph = sceneGraph; _sceneGraph = sceneGraph;
} }
@@ -405,6 +408,11 @@ public sealed class FramePump : IDisposable
: compositorOptions.SourceRectHeight - barFrame.Height; : compositorOptions.SourceRectHeight - barFrame.Height;
} }
// The alert ticker is a per-frame overlay (it scrolls while an alert
// plays), NEVER baked into the static base. Lawn-mower seam: read it
// each tick and let the compositor pin it to the very top.
var tickerFrame = _alertTicker?.Invoke();
VideoFrame frame; VideoFrame frame;
var scratchSize = compositorOptions.SourceRectWidth var scratchSize = compositorOptions.SourceRectWidth
* compositorOptions.SourceRectHeight * 4; * compositorOptions.SourceRectHeight * 4;
@@ -415,7 +423,7 @@ public sealed class FramePump : IDisposable
// pooling change because its buffer's only consumer was its own release. // pooling change because its buffer's only consumer was its own release.
renderSw.Restart(); renderSw.Restart();
var scratch = AcquireScratch(scratchSize); var scratch = AcquireScratch(scratchSize);
frame = RenderScene(scene, compositorOptions, socialBarFrame, socialBarTop, scratch, TimedResolver); frame = RenderScene(scene, compositorOptions, socialBarFrame, socialBarTop, tickerFrame, scratch, TimedResolver);
if (_transition is { Active: true } transition) if (_transition is { Active: true } transition)
{ {
frame = transition.BlendFrame(frame); frame = transition.BlendFrame(frame);
@@ -555,6 +563,7 @@ public sealed class FramePump : IDisposable
CompositorOptions options, CompositorOptions options,
VideoFrame? socialBarFrame, VideoFrame? socialBarFrame,
int socialBarTop, int socialBarTop,
VideoFrame? tickerFrame,
byte[]? scratch = null, byte[]? scratch = null,
Func<SceneElement, VideoFrame?>? resolver = null) Func<SceneElement, VideoFrame?>? resolver = null)
{ {
@@ -562,7 +571,7 @@ public sealed class FramePump : IDisposable
if (_sceneGraph == null) if (_sceneGraph == null)
{ {
var sw = System.Diagnostics.Stopwatch.StartNew(); var sw = System.Diagnostics.Stopwatch.StartNew();
var frame = RenderFull(scene, options, socialBarFrame, socialBarTop, scratch, resolver); var frame = RenderFull(scene, options, socialBarFrame, socialBarTop, tickerFrame, scratch, resolver);
ProbeRender("full-no-graph", scene, -1, sw.ElapsedMilliseconds, 0); ProbeRender("full-no-graph", scene, -1, sw.ElapsedMilliseconds, 0);
return frame; return frame;
} }
@@ -586,14 +595,14 @@ public sealed class FramePump : IDisposable
if (baseFrame != null) if (baseFrame != null)
{ {
var frame = SceneCompositor.CompositeLayers( var frame = SceneCompositor.CompositeLayers(
baseFrame, scene, split, resolver, options, socialBarFrame, socialBarTop, scratch: scratch); baseFrame, scene, split, resolver, options, socialBarFrame, socialBarTop, tickerFrame: tickerFrame, scratch: scratch);
ProbeRender("base+layers", scene, split, swBase.ElapsedMilliseconds, 0); ProbeRender("base+layers", scene, split, swBase.ElapsedMilliseconds, 0);
return frame; return frame;
} }
// No static base (first layer is dynamic or empty scene) — full render. // No static base (first layer is dynamic or empty scene) — full render.
var swFull = System.Diagnostics.Stopwatch.StartNew(); var swFull = System.Diagnostics.Stopwatch.StartNew();
var frame2 = RenderFull(scene, options, socialBarFrame, socialBarTop, scratch, resolver); var frame2 = RenderFull(scene, options, socialBarFrame, socialBarTop, tickerFrame, scratch, resolver);
ProbeRender("full-render", scene, split, swFull.ElapsedMilliseconds, 0); ProbeRender("full-render", scene, split, swFull.ElapsedMilliseconds, 0);
return frame2; return frame2;
} }
@@ -605,6 +614,7 @@ public sealed class FramePump : IDisposable
/// hit costs ~3ms of pure copy and a miss costs exactly the old full render.</summary> /// hit costs ~3ms of pure copy and a miss costs exactly the old full render.</summary>
private VideoFrame RenderFull( private VideoFrame RenderFull(
Scene scene, CompositorOptions options, VideoFrame? socialBarFrame, int socialBarTop, Scene scene, CompositorOptions options, VideoFrame? socialBarFrame, int socialBarTop,
VideoFrame? tickerFrame,
byte[]? scratch, Func<SceneElement, VideoFrame?> resolver) byte[]? scratch, Func<SceneElement, VideoFrame?> resolver)
{ {
// 1:1 guard: the cache stores the OUTPUT bytes and scratch is source-sized — // 1:1 guard: the cache stores the OUTPUT bytes and scratch is source-sized —
@@ -615,7 +625,7 @@ public sealed class FramePump : IDisposable
var useCache = scratch != null && options.SourceRectWidth == options.OutputWidth var useCache = scratch != null && options.SourceRectWidth == options.OutputWidth
&& options.SourceRectHeight == options.OutputHeight; && options.SourceRectHeight == options.OutputHeight;
var signature = useCache var signature = useCache
? BuildFullRenderSignature(scene, options, socialBarFrame, socialBarTop, resolver) ? BuildFullRenderSignature(scene, options, socialBarFrame, socialBarTop, tickerFrame, resolver)
: 0UL; : 0UL;
if (useCache && _fullCacheValid && signature == _lastFullSignature if (useCache && _fullCacheValid && signature == _lastFullSignature
@@ -626,7 +636,7 @@ public sealed class FramePump : IDisposable
return new VideoFrame(options.SourceRectWidth, options.SourceRectHeight, scratch); return new VideoFrame(options.SourceRectWidth, options.SourceRectHeight, scratch);
} }
var rendered = _compositor.Render(scene, resolver, null, options, socialBarFrame, socialBarTop, scratch: scratch); var rendered = _compositor.Render(scene, resolver, null, options, socialBarFrame, socialBarTop, scratch: scratch, tickerFrame: tickerFrame);
CacheRenders++; CacheRenders++;
if (useCache) if (useCache)
{ {
@@ -659,6 +669,7 @@ public sealed class FramePump : IDisposable
/// never trusted to key recycled content.</summary> /// never trusted to key recycled content.</summary>
private static ulong BuildFullRenderSignature( private static ulong BuildFullRenderSignature(
Scene scene, CompositorOptions options, VideoFrame? socialBarFrame, int socialBarTop, Scene scene, CompositorOptions options, VideoFrame? socialBarFrame, int socialBarTop,
VideoFrame? tickerFrame,
Func<SceneElement, VideoFrame?> resolver) Func<SceneElement, VideoFrame?> resolver)
{ {
var h = 14695981039346656037UL; // FNV-1a offset basis var h = 14695981039346656037UL; // FNV-1a offset basis
@@ -671,6 +682,8 @@ public sealed class FramePump : IDisposable
h = Mix(h, (ulong)socialBarTop); h = Mix(h, (ulong)socialBarTop);
if (socialBarFrame != null) h = MixFrame(h, socialBarFrame); if (socialBarFrame != null) h = MixFrame(h, socialBarFrame);
else h = Mix(h, 0xFFFFFFFFFFFFFFFFUL); else h = Mix(h, 0xFFFFFFFFFFFFFFFFUL);
if (tickerFrame != null) h = MixFrame(h, tickerFrame);
else h = Mix(h, 0xFFFFFFFFFFFFFFFFUL);
var backdropResolved = false; var backdropResolved = false;
foreach (var element in scene.Elements) foreach (var element in scene.Elements)
+8 -2
View File
@@ -99,7 +99,8 @@ public partial class LayoutStore : IDisposable
IsBackground, CaptureKey, ShowDesktop, IsBackground, CaptureKey, ShowDesktop,
ChatFontSize, ChatFontColor, ChatBackgroundColor, ChatMaxMessages, ChatFontSize, ChatFontColor, ChatBackgroundColor, ChatMaxMessages,
ChatShowTimestamps, ChatShowBadges, ChatMemberColor, ChatSuperChatColor, ChatShowTimestamps, ChatShowBadges, ChatMemberColor, ChatSuperChatColor,
WebUri, MediaPath, MediaIsLooping, MediaVolume, MediaPlaybackState, SortOrder WebUri, MediaPath, MediaIsLooping, MediaVolume, MediaPlaybackState,
AlertVideoAssetId, AlertVideoPath, AlertUseDefaultVideo, AlertShowTicker, AlertVideoVolume, SortOrder
FROM Source ORDER BY SortOrder FROM Source ORDER BY SortOrder
"""; """;
using var reader = cmd.ExecuteReader(); using var reader = cmd.ExecuteReader();
@@ -145,10 +146,15 @@ public partial class LayoutStore : IDisposable
: Enum.TryParse<MediaPlaybackState>(reader.GetString(29), out var ps) : Enum.TryParse<MediaPlaybackState>(reader.GetString(29), out var ps)
? ps ? ps
: MediaPlaybackState.Stopped, : MediaPlaybackState.Stopped,
AlertVideoAssetId = reader.IsDBNull(30) ? null : reader.GetString(30),
AlertVideoPath = reader.IsDBNull(31) ? null : reader.GetString(31),
AlertUseDefaultVideo = reader.IsDBNull(32) ? true : reader.GetInt32(32) != 0,
AlertShowTicker = reader.IsDBNull(33) ? true : reader.GetInt32(33) != 0,
AlertVideoVolume = reader.IsDBNull(34) ? 1.0 : reader.GetDouble(34),
}; };
if (!sourcesByScene.TryGetValue(sceneId, out var list)) if (!sourcesByScene.TryGetValue(sceneId, out var list))
sourcesByScene[sceneId] = list = new List<(Source, int)>(); sourcesByScene[sceneId] = list = new List<(Source, int)>();
list.Add((source, reader.GetInt32(30))); list.Add((source, reader.GetInt32(35)));
} }
} }
+34
View File
@@ -285,6 +285,40 @@ public partial class LayoutStore : IDisposable
cmd.ExecuteNonQuery(); cmd.ExecuteNonQuery();
} }
// TASK 47: alert video config. The default clip asset id + the creator's
// replacement path (a FILE reference, never a stored BLOB) + the pill,
// ticker toggle and volume live on the row.
if (!columns.Contains("AlertVideoAssetId"))
{
using var cmd = _connection.CreateCommand();
cmd.CommandText = "ALTER TABLE Source ADD COLUMN AlertVideoAssetId TEXT;";
cmd.ExecuteNonQuery();
}
if (!columns.Contains("AlertVideoPath"))
{
using var cmd = _connection.CreateCommand();
cmd.CommandText = "ALTER TABLE Source ADD COLUMN AlertVideoPath TEXT;";
cmd.ExecuteNonQuery();
}
if (!columns.Contains("AlertUseDefaultVideo"))
{
using var cmd = _connection.CreateCommand();
cmd.CommandText = "ALTER TABLE Source ADD COLUMN AlertUseDefaultVideo INTEGER NOT NULL DEFAULT 1;";
cmd.ExecuteNonQuery();
}
if (!columns.Contains("AlertShowTicker"))
{
using var cmd = _connection.CreateCommand();
cmd.CommandText = "ALTER TABLE Source ADD COLUMN AlertShowTicker INTEGER NOT NULL DEFAULT 1;";
cmd.ExecuteNonQuery();
}
if (!columns.Contains("AlertVideoVolume"))
{
using var cmd = _connection.CreateCommand();
cmd.CommandText = "ALTER TABLE Source ADD COLUMN AlertVideoVolume REAL NOT NULL DEFAULT 1;";
cmd.ExecuteNonQuery();
}
// Rename legacy "Backdrop" display name to "Background" in existing rows // Rename legacy "Backdrop" display name to "Background" in existing rows
using (var cmd = _connection.CreateCommand()) using (var cmd = _connection.CreateCommand())
{ {
+23 -3
View File
@@ -88,13 +88,15 @@ public partial class LayoutStore : IDisposable
ClipShape, IsMirrored, IsBackground, CaptureKey, ShowDesktop, SortOrder, ClipShape, IsMirrored, IsBackground, CaptureKey, ShowDesktop, SortOrder,
ChatFontSize, ChatFontColor, ChatBackgroundColor, ChatMaxMessages, ChatFontSize, ChatFontColor, ChatBackgroundColor, ChatMaxMessages,
ChatShowTimestamps, ChatShowBadges, ChatMemberColor, ChatSuperChatColor, ChatShowTimestamps, ChatShowBadges, ChatMemberColor, ChatSuperChatColor,
WebUri, MediaPath, MediaIsLooping, MediaVolume, MediaPlaybackState) WebUri, MediaPath, MediaIsLooping, MediaVolume, MediaPlaybackState,
AlertVideoAssetId, AlertVideoPath, AlertUseDefaultVideo, AlertShowTicker, AlertVideoVolume)
VALUES ($id, $sceneId, $assetId, $type, $name, $isEnabled, VALUES ($id, $sceneId, $assetId, $type, $name, $isEnabled,
$x, $y, $w, $h, $opacity, $monitor, $x, $y, $w, $h, $opacity, $monitor,
$clip, $mirrored, $isBackground, $captureKey, $showDesktop, $sort, $clip, $mirrored, $isBackground, $captureKey, $showDesktop, $sort,
$chatFontSize, $chatFontColor, $chatBgColor, $chatMaxMsgs, $chatFontSize, $chatFontColor, $chatBgColor, $chatMaxMsgs,
$chatTimestamps, $chatBadges, $chatMemberColor, $chatSuperChatColor, $chatTimestamps, $chatBadges, $chatMemberColor, $chatSuperChatColor,
$webUri, $mediaPath, $mediaIsLooping, $mediaVolume, $mediaPlaybackState) $webUri, $mediaPath, $mediaIsLooping, $mediaVolume, $mediaPlaybackState,
$alertVideoAssetId, $alertVideoPath, $alertUseDefaultVideo, $alertShowTicker, $alertVideoVolume)
"""; """;
cmd.Transaction = tx; cmd.Transaction = tx;
var idP = cmd.Parameters.Add("$id", SqliteType.Text); var idP = cmd.Parameters.Add("$id", SqliteType.Text);
@@ -128,6 +130,11 @@ public partial class LayoutStore : IDisposable
var mediaIsLoopingP = cmd.Parameters.Add("$mediaIsLooping", SqliteType.Integer); var mediaIsLoopingP = cmd.Parameters.Add("$mediaIsLooping", SqliteType.Integer);
var mediaVolumeP = cmd.Parameters.Add("$mediaVolume", SqliteType.Real); var mediaVolumeP = cmd.Parameters.Add("$mediaVolume", SqliteType.Real);
var mediaPlaybackStateP = cmd.Parameters.Add("$mediaPlaybackState", SqliteType.Text); var mediaPlaybackStateP = cmd.Parameters.Add("$mediaPlaybackState", SqliteType.Text);
var alertVideoAssetIdP = cmd.Parameters.Add("$alertVideoAssetId", SqliteType.Text);
var alertVideoPathP = cmd.Parameters.Add("$alertVideoPath", SqliteType.Text);
var alertUseDefaultVideoP = cmd.Parameters.Add("$alertUseDefaultVideo", SqliteType.Integer);
var alertShowTickerP = cmd.Parameters.Add("$alertShowTicker", SqliteType.Integer);
var alertVideoVolumeP = cmd.Parameters.Add("$alertVideoVolume", SqliteType.Real);
// Source and WebcamSceneConfig share ONE z-order space per scene: // Source and WebcamSceneConfig share ONE z-order space per scene:
// each element gets its index within scene.Elements regardless of // each element gets its index within scene.Elements regardless of
@@ -170,6 +177,11 @@ public partial class LayoutStore : IDisposable
mediaIsLoopingP.Value = source.MediaIsLooping ? 1 : 0; mediaIsLoopingP.Value = source.MediaIsLooping ? 1 : 0;
mediaVolumeP.Value = source.MediaVolume; mediaVolumeP.Value = source.MediaVolume;
mediaPlaybackStateP.Value = source.MediaPlaybackState.ToString(); mediaPlaybackStateP.Value = source.MediaPlaybackState.ToString();
alertVideoAssetIdP.Value = (object?)source.AlertVideoAssetId ?? DBNull.Value;
alertVideoPathP.Value = (object?)source.AlertVideoPath ?? DBNull.Value;
alertUseDefaultVideoP.Value = source.AlertUseDefaultVideo ? 1 : 0;
alertShowTickerP.Value = source.AlertShowTicker ? 1 : 0;
alertVideoVolumeP.Value = source.AlertVideoVolume;
cmd.ExecuteNonQuery(); cmd.ExecuteNonQuery();
} }
} }
@@ -300,7 +312,15 @@ public partial class LayoutStore : IDisposable
using (var cmd = _connection.CreateCommand()) using (var cmd = _connection.CreateCommand())
{ {
cmd.CommandText = "DELETE FROM Asset WHERE Id NOT IN (SELECT AssetId FROM Source WHERE AssetId IS NOT NULL);"; cmd.CommandText = """
DELETE FROM Asset WHERE Id NOT IN (
SELECT AssetId FROM Source WHERE AssetId IS NOT NULL
UNION
SELECT AlertVideoAssetId FROM Source WHERE AlertVideoAssetId IS NOT NULL
UNION
SELECT Value FROM Settings WHERE Key = 'AlertDefaultVideoAssetId'
);
""";
cmd.Transaction = tx; cmd.Transaction = tx;
cmd.ExecuteNonQuery(); cmd.ExecuteNonQuery();
} }
+22
View File
@@ -304,4 +304,26 @@ public partial class LayoutStore : IDisposable
insert.Parameters.AddWithValue("$value", value); insert.Parameters.AddWithValue("$value", value);
insert.ExecuteNonQuery(); insert.ExecuteNonQuery();
} }
internal const string AlertDefaultVideoKey = "AlertDefaultVideoAssetId";
/// <summary>Stamps the shipped stream-alert placeholder clip (TASK 47): like any
/// source image it lives as a BLOB in the internal Asset table, tracked by this
/// Setting so the Save pruner keeps it alive. The creator's OWN replacement
/// videos are never stored — they are plain path references (the 2026-09-26
/// decision: the file the user picks stays at its real location on disk).
/// <c>UpsertAsset</c> is hash-keyed, so re-stamping the same bytes returns the
/// same id.</summary>
public string EnsureAlertDefaultVideo(byte[] data)
{
var saved = GetSetting(AlertDefaultVideoKey);
if (!string.IsNullOrWhiteSpace(saved) && GetAssetBytes(saved) != null)
return saved;
var id = UpsertAsset(data, 0, 0);
UpsertSetting(AlertDefaultVideoKey, id);
return id;
}
/// <summary>The stored default clip's Asset id, or null when never stamped.</summary>
public string? GetAlertDefaultVideoId() => GetSetting(AlertDefaultVideoKey);
} }
+8 -6
View File
@@ -11,9 +11,10 @@ External-facing logic: YouTube API, persistence. See
| `YouTubeChatService.cs` | Polls `liveChat/messages`, raises `MessageReceived`; `IDisposable`. **TASK 43 rewrite (2026-09-24):** re-arms its one-shot poll on the server's `pollingIntervalMillis` (streamList semantics — reconnect on that cadence with `nextPageToken`, clamp 1000–6000ms, `maxResults=2000`); `ParsePage` (internal static seam) decodes **all six** event types into `ChatMessage.Kind`/fields (superChat / superSticker / newSponsor / membershipGifting / giftMembershipReceived / memberMilestoneChat + milestone months best-effort regex); optional `HttpClient` ctor seam endures; `InjectSimulatedMessage` (TASK 41) preserved | | `YouTubeChatService.cs` | Polls `liveChat/messages`, raises `MessageReceived`; `IDisposable`. **TASK 43 rewrite (2026-09-24):** re-arms its one-shot poll on the server's `pollingIntervalMillis` (streamList semantics — reconnect on that cadence with `nextPageToken`, clamp 1000–6000ms, `maxResults=2000`); `ParsePage` (internal static seam) decodes **all six** event types into `ChatMessage.Kind`/fields (superChat / superSticker / newSponsor / membershipGifting / giftMembershipReceived / memberMilestoneChat + milestone months best-effort regex); optional `HttpClient` ctor seam endures; `InjectSimulatedMessage` (TASK 41) preserved |
| `ChatBoxRenderer.cs` | Chat overlay rasterizer (WPF FormattedText → BGRA `VideoFrame`), live rows + Super Chat/member styling | | `ChatBoxRenderer.cs` | Chat overlay rasterizer (WPF FormattedText → BGRA `VideoFrame`), live rows + Super Chat/member styling |
| `ChatOverlayLayer.cs` | **The chat component (Commit G):** owns the message buffer, `ChatBoxRenderer`, fade/mock timers, per-source preview renders, live `RenderFrame`; raster-on-change cache pattern (OBS text-source contract) | | `ChatOverlayLayer.cs` | **The chat component (Commit G):** owns the message buffer, `ChatBoxRenderer`, fade/mock timers, per-source preview renders, live `RenderFrame`; raster-on-change cache pattern (OBS text-source contract) |
| `AlertRenderer.cs` | **The alert rasterizer (TASK 43, 2026-09-24):** renders one event's celebration animation into a BGRA `VideoFrame` — SIX DISTINCT animations (SuperChat slide-up/shine/count-up, SuperSticker scale-pop, NewMember drop-in/flash-band, MemberGift slide-left/chip-fan, GiftReceived confetti scale-in, MemberMilestone rise/growth-bar), enter/hold/exit phases, brand line on every card | | `AlertRenderer.cs` | **The alert rasterizer (TASK 43, 2026-09-24):** renders one event's celebration animation into a BGRA `VideoFrame` — SIX DISTINCT animations (SuperChat slide-up/shine/count-up, SuperSticker scale-pop, NewMember drop-in/flash-band, MemberGift slide-left/chip-fan, GiftReceived confetti scale-in, MemberMilestone rise/growth-bar), enter/hold/exit phases, brand line on every card. **Still the fallback** when no alert clip is playable (TASK 47) |
| `AlertOverlayLayer.cs` | **The native alert-box component (TASK 43):** event queue (cap 10) + 33ms dispatch ticker + `RenderFrame` cache-first live path + `UpdatePreview`; **`Advance(double)` is the deterministic test clock** (tests never start the timer); `ChatEventKind.None` rows (sub mentions) never enqueue; idle = transparent (null frame) | | `AlertOverlayLayer.cs` | **The native alert-box component (TASK 43; video branch TASK 47):** event queue (cap 10) + 33ms dispatch ticker + `RenderFrame` cache-first live path + `UpdatePreview`; **`Advance(double)` is the deterministic test clock** (tests never start the timer); `ChatEventKind.None` rows (sub mentions) never enqueue; idle = transparent (null frame). Since TASK 47 a clip branch owns `IAlertClipDecoder` per play — ctor seams `(clipDecoderFactory, clipPathResolver, audioSink)`, fade in/out over `FadeDurationSeconds = 0.30` (straight-alpha copies + volume×fade audio), EOF freeze-frame → drain, and `AlertTickerFrame` composes the scrolling message strip |
| `LayoutStore.cs` | SQLite persistence (`Microsoft.Data.Sqlite`) at `%APPDATA%\ytLlive\ytLlive.db`; assets stored as BLOBs keyed by SHA-256 content hash; save/open layout files; schema `user_version` 6 (`Source.ClipShape`/`IsMirrored` via `ALTER TABLE` for pre-v2 DBs; v3 = singleton `Webcam` + per-scene `WebcamSceneConfig`, migrated idempotently **without backfill** — the stale `Source.DeviceId` column remains but is no longer read/written; v4 = `WebcamSceneConfig.RectWidth`/`RectHeight`, the pre-Round rect for the round-to-rect restore; v5 = `Source.IsBackdrop` + `Source.CaptureKey`, the live-capture backdrop; v6 = `Scene.HasBackdrop` — Live-only policy, one-time backfill turns Starting/BRB/Chat/Ending off + drops their backdrop sources; `MainViewModel.NormalizeBackgrounds` re-normalizes on every load — TASK 25: one locked "Background" per canonical screen (static art everywhere, DisplayCapture on Live), duplicates dropped, index 0, orphaned `BackgroundUseDefault_*`/`BackgroundPath_*` Settings keys purged at save). Settings key/value table holds app state: `MicSourceName`, `ReusableStream` (TASK 9 — the cached reusable stream's id/address/name so the pump has its RTMP URL at startup) | | `AlertClipDecoder.cs` | **Per-play alert video decoder (TASK 47):** `IAlertClipDecoder` (Start/Stop/Dispose + `FrameAvailable`/`AudioReady`/`Completed`) + ffmpeg child emitting `rawvideo bgra` (`-vf scale=W:H`, read via `RawVideoFrameReader`) and `f32le -ar 48000 -ac 2` on a **carry-buffer** audio loop (float boundary straddles never drop samples), both real-time paced; bgra is opaque (layer owns alpha). One-shot lifetime — created per alert, disposed at drain; tests inject a fake |
| `LayoutStore.cs` | SQLite persistence (`Microsoft.Data.Sqlite`) at `%APPDATA%\ytLlive\ytLlive.db`; assets stored as BLOBs keyed by SHA-256 content hash; save/open layout files; schema `user_version` 10 (`Source.ClipShape`/`IsMirrored` via `ALTER TABLE` for pre-v2 DBs; v3 = singleton `Webcam` + per-scene `WebcamSceneConfig`, migrated idempotently **without backfill** — the stale `Source.DeviceId` column remains but is no longer read/written; v4 = `WebcamSceneConfig.RectWidth`/`RectHeight`, the pre-Round rect for the round-to-rect restore; v5 = `Source.IsBackdrop` + `Source.CaptureKey`, the live-capture backdrop; v6 = `Scene.HasBackdrop` — Live-only policy, one-time backfill turns Starting/BRB/Chat/Ending off + drops their backdrop sources; v9→v10 (TASK 39) = `YppSnapshot` history table; all later columns stay additive guarded ALTERs — TASK 47 adds the five alert `Source` columns (`AlertVideoAssetId`/`AlertVideoPath`/`AlertUseDefaultVideo`/`AlertShowTicker`/`AlertVideoVolume`) without a version bump; `MainViewModel.NormalizeBackgrounds` re-normalizes on every load — TASK 25: one locked "Background" per canonical screen (static art everywhere, DisplayCapture on Live), duplicates dropped, index 0, orphaned `BackgroundUseDefault_*`/`BackgroundPath_*` Settings keys purged at save). Settings key/value table holds app state: `MicSourceName`, `ReusableStream` (TASK 9 — the cached reusable stream's id/address/name so the pump has its RTMP URL at startup), `AlertDefaultVideoAssetId` (TASK 47 — the stamped built-in alert clip's asset row; save's prune exempts it via UNION) |
| `VideoFrame.cs` | Normalized CPU frame seam (`Width`/`Height`/tightly-packed BGRA `byte[]`) — the only pixel type the rest of the app knows about; future capture sources (screen, background-removed webcam) feed the same seam | | `VideoFrame.cs` | Normalized CPU frame seam (`Width`/`Height`/tightly-packed BGRA `byte[]`) — the only pixel type the rest of the app knows about; future capture sources (screen, background-removed webcam) feed the same seam |
| `CameraDeviceInfo.cs` | `(Id, DisplayName)` for a physical capture device | | `CameraDeviceInfo.cs` | `(Id, DisplayName)` for a physical capture device |
| `ICameraEnumerator.cs` | `GetCamerasAsync()` — seam so the picker/`CameraManager` never touch WinRT (tests inject fakes) | | `ICameraEnumerator.cs` | `GetCamerasAsync()` — seam so the picker/`CameraManager` never touch WinRT (tests inject fakes) |
@@ -34,13 +35,14 @@ External-facing logic: YouTube API, persistence. See
| `ScreenCaptureSourceFactory.cs` | `Resolve(key)` parses `monitor:<n>` / `window:<hwnd>` / `picker:<name>` into a source; `PickAsync()` shows the OS `GraphicsCapturePicker` and returns the `picker:` key (transient — a reload falls back to auto-detection) | | `ScreenCaptureSourceFactory.cs` | `Resolve(key)` parses `monitor:<n>` / `window:<hwnd>` / `picker:<name>` into a source; `PickAsync()` shows the OS `GraphicsCapturePicker` and returns the `picker:` key (transient — a reload falls back to auto-detection) |
| `WebView2Manager.cs` | **Web-source ownership (TASK 17, composition-capture since 2026-09-14):** per registered `Source { Type: WebSource }` it owns a `CoreWebView2CompositionController` + its visual tree (`RootVisualTarget` child under a 1920×1080 root `ContainerVisual` on one `Windows.UI.Composition.Compositor` created on the UI thread via the CoreMessaging `CreateDispatcherQueueController` P/Invoke), a wrapped `WebCaptureFrameSource`, and the shared crop-sized `WriteableBitmap` preview (dispatcher-coalesced copy at Render priority, `PreviewBitmapChanged` raised only on recreation). Nav + `TransparentBackgroundScript` injection unchanged. Public ctor `(Dispatcher)`; internal seam ctor `(Dispatcher, Func<string, IScreenCaptureSource>?)` for hermetic tests (no WinRT). Caller: `MainViewModel.InitWebView2()` (now invoked from `MainWindow_Loaded` — the parent HWND must exist). See `../ai.md` "Slice 14" | | `WebView2Manager.cs` | **Web-source ownership (TASK 17, composition-capture since 2026-09-14):** per registered `Source { Type: WebSource }` it owns a `CoreWebView2CompositionController` + its visual tree (`RootVisualTarget` child under a 1920×1080 root `ContainerVisual` on one `Windows.UI.Composition.Compositor` created on the UI thread via the CoreMessaging `CreateDispatcherQueueController` P/Invoke), a wrapped `WebCaptureFrameSource`, and the shared crop-sized `WriteableBitmap` preview (dispatcher-coalesced copy at Render priority, `PreviewBitmapChanged` raised only on recreation). Nav + `TransparentBackgroundScript` injection unchanged. Public ctor `(Dispatcher)`; internal seam ctor `(Dispatcher, Func<string, IScreenCaptureSource>?)` for hermetic tests (no WinRT). Caller: `MainViewModel.InitWebView2()` (now invoked from `MainWindow_Loaded` — the parent HWND must exist). See `../ai.md` "Slice 14" |
| `WebCaptureFrameSource.cs` | The `IScreenCaptureSource` for a web session: owns the `GraphicsCaptureItem` (`CreateFromVisual`) + a free-threaded `Direct3D11CaptureFramePool` (2 buffers) + session; readback via `SoftwareBitmap.CreateCopyFromSurfaceAsync` (BGRA, **`BitmapAlphaMode.Straight`** — the compositor blends straight alpha); per-frame `FindContentBounds` alpha-bbox (internal static, unit-tested) stamps `VideoFrame.CropBounds`; ring up to 2 fresh frames + Epoch reuse; `IsOpaque=false`. Mirror of `ScreenCaptureFrameSource` | | `WebCaptureFrameSource.cs` | The `IScreenCaptureSource` for a web session: owns the `GraphicsCaptureItem` (`CreateFromVisual`) + a free-threaded `Direct3D11CaptureFramePool` (2 buffers) + session; readback via `SoftwareBitmap.CreateCopyFromSurfaceAsync` (BGRA, **`BitmapAlphaMode.Straight`** — the compositor blends straight alpha); per-frame `FindContentBounds` alpha-bbox (internal static, unit-tested) stamps `VideoFrame.CropBounds`; ring up to 2 fresh frames + Epoch reuse; `IsOpaque=false`. Mirror of `ScreenCaptureFrameSource` |
| `Compositor/SceneCompositor.cs` | **The output compositor (TASK 4 ship step 1)**: renders a scene into the encoder's master `VideoFrame` (tightly-packed BGRA8), mirroring the XAML preview minus editing chrome — backdrop → background → elements (`UniformToFill` cover-crop, round clip, mirror, opacity, border) → branding flash → social bar (optional `socialBarFrame` + `socialBarTop` in master space, blitted last so the bar sits above the flash; the old `BlitFlash` generalized to `BlitOverlay` with source offsets). Pure and WPF-free: frames injected via a `Func<SceneElement, VideoFrame?>` resolver (webcam → DeviceId, image → AssetId, backdrop → CaptureKey); output sized by `CompositorOptions` (16:9 = full master 1:1; vertical 9:16 = 607×1080 crop → 1080×1920 bilinear). Preview stays XAML (editing view); this is the output view — see `ai.md` "Scene compositor" | | `Compositor/SceneCompositor.cs` | **The output compositor (TASK 4 ship step 1)**: renders a scene into the encoder's master `VideoFrame` (tightly-packed BGRA8), mirroring the XAML preview minus editing chrome — backdrop → background → elements (`UniformToFill` cover-crop, round clip, mirror, opacity, border) → branding flash → social bar (optional `socialBarFrame` + `socialBarTop` in master space, blitted last so the bar sits above the flash; the old `BlitFlash` generalized to `BlitOverlay` with source offsets). **TASK 47:** `Render` + `CompositeLayers` take a trailing `tickerFrame` blitted at (0,0) after the bar — the alert ticker is a message-scrolling dynamic overlay, never baked. Pure and WPF-free: frames injected via a `Func<SceneElement, VideoFrame?>` resolver (webcam → DeviceId, image → AssetId, backdrop → CaptureKey); output sized by `CompositorOptions` (16:9 = full master 1:1; vertical 9:16 = 607×1080 crop → 1080×1920 bilinear). Preview stays XAML (editing view); this is the output view — see `ai.md` "Scene compositor" |
| `Compositor/CompositorOptions.cs` | The active tier's output rect (source space over the 1920×1080 master, integer-aligned — `MainViewModel.OutputRectX` can be 656.5) + target W×H | | `Compositor/CompositorOptions.cs` | The active tier's output rect (source space over the 1920×1080 master, integer-aligned — `MainViewModel.OutputRectX` can be 656.5) + target W×H |
| `Compositor/StretchMath.cs` | Pure pixel math: the WPF `UniformToFill` cover-crop, clamped bilinear sample/scale (unit-tested half of the compositor) | | `Compositor/StretchMath.cs` | Pure pixel math: the WPF `UniformToFill` cover-crop, clamped bilinear sample/scale (unit-tested half of the compositor) |
| `Compositor/StaticPixelCache.cs` | Asset bytes → cached BGRA8 `VideoFrame`, decoded once per content hash — the output path's raw-pixel source (the preview uses ImageCache's `BitmapImage`) | | `Compositor/StaticPixelCache.cs` | Asset bytes → cached BGRA8 `VideoFrame`, decoded once per content hash — the output path's raw-pixel source (the preview uses ImageCache's `BitmapImage`) |
| `Encoder/IFfmpegLocator.cs` | **TASK 4 ship step 2 seam**: resolves an absolute path to `ffmpeg.exe` (PATH first → cached `%APPDATA%\ytLlive\tools\ffmpeg.exe` → pinned BtbN LGPL-shared zip download). The encoder step and tests fake it | | `Encoder/IFfmpegLocator.cs` | **TASK 4 ship step 2 seam**: resolves an absolute path to `ffmpeg.exe` (PATH first → cached `%APPDATA%\ytLlive\tools\ffmpeg.exe` → pinned BtbN LGPL-shared zip download). The encoder step and tests fake it |
| `SocialValidator.cs` | `ISocialValidator` seam + `HttpSocialValidator` default: validates a social handle/URL by constructing the canonical profile URL and issuing an HTTP GET — the "act of validation" before adding to the bar. 200/redirect = exists; 404/connection failure = rejected. Fediverse `@user@domain` additionally does a best-effort nodeinfo lookup (`/.well-known/nodeinfo` → `software.name`) so the entry can show the instance's real logo; nodeinfo failure still validates (generic fediverse glyph). If the identity domain's nodeinfo is blocked (SSO) but the bare root 302s to the real instance (YunoHost default-app subdomains), the lookup follows the redirect and asks that host. Since the bar bug-fix branch the resolution **probes well-known subdomains** (`mastodon.`, `social.`, ... order in `FediverseSubdomainCandidates`) when both the identity domain and its redirect come up empty, bounded by a ~15s linked-CTS budget; `ResolveFediverseSoftwareAsync(domain, ct)` on the seam powers the load-time heal. Constructor takes optional `HttpClient` + `Action<string>` log for tests | | `SocialValidator.cs` | `ISocialValidator` seam + `HttpSocialValidator` default: validates a social handle/URL by constructing the canonical profile URL and issuing an HTTP GET — the "act of validation" before adding to the bar. 200/redirect = exists; 404/connection failure = rejected. Fediverse `@user@domain` additionally does a best-effort nodeinfo lookup (`/.well-known/nodeinfo` → `software.name`) so the entry can show the instance's real logo; nodeinfo failure still validates (generic fediverse glyph). If the identity domain's nodeinfo is blocked (SSO) but the bare root 302s to the real instance (YunoHost default-app subdomains), the lookup follows the redirect and asks that host. Since the bar bug-fix branch the resolution **probes well-known subdomains** (`mastodon.`, `social.`, ... order in `FediverseSubdomainCandidates`) when both the identity domain and its redirect come up empty, bounded by a ~15s linked-CTS budget; `ResolveFediverseSoftwareAsync(domain, ct)` on the seam powers the load-time heal. Constructor takes optional `HttpClient` + `Action<string>` log for tests |
| `Compositor/SocialBarRenderer.cs` | **Social bar output strip** (bar bug-fix branch): rasterizes the global social bar into a transparent straight-alpha BGRA8 `VideoFrame` strip (1920 wide, 40px content + 24px glow pad) via `RenderTargetBitmap` — one Path (logo) + one TextBlock (handle) per entry, white on transparent with the preview's green `#2ecc71` glow baked in (Pbgra32 → straight-alpha unpremultiply). UI thread only; the frame is immutable afterwards so the `FramePump` reads it from its own thread | | `Compositor/SocialBarRenderer.cs` | **Social bar output strip** (bar bug-fix branch): rasterizes the global social bar into a transparent straight-alpha BGRA8 `VideoFrame` strip (1920 wide, 40px content + 24px glow pad) via `RenderTargetBitmap` — one Path (logo) + one TextBlock (handle) per entry, white on transparent with the preview's green `#2ecc71` glow baked in (Pbgra32 → straight-alpha unpremultiply). UI thread only; the frame is immutable afterwards so the `FramePump` reads it from its own thread |
| `Compositor/AlertTickerRenderer.cs` | **Alert message ticker strip (TASK 47):** marquee overlay rendered per call by pure byte-math onto a 1920×48 transparent BGRA frame — the text pill ("Author — Kind · amount") is rasterized once per text on the UI thread (cached, unpremultiplied straight alpha) and scrolled at 140px/s with a repeat-gap wrap, so the pump can call `Render(elapsed, text)` from its own thread without locking. **Dynamic overlay — never baked** (unlike the social bar) |
| `Encoder/EncoderOptions.cs` | One go-live's encoder config: full `RtmpUrl` (ingest + stream key), W×H/FPS/bitrate from the quality tier, `VideoEncoder` (null = probe), audio sample rate/channels, `GopSize` = FPS×4 (the ≤4s keyframe bound) | | `Encoder/EncoderOptions.cs` | One go-live's encoder config: full `RtmpUrl` (ingest + stream key), W×H/FPS/bitrate from the quality tier, `VideoEncoder` (null = probe), audio sample rate/channels, `GopSize` = FPS×4 (the ≤4s keyframe bound) |
| `Encoder/IFfmpegEncoder.cs` | **TASK 4 ship step 3 seam**: start/feed/stop the live encoder; `HealthUpdated` (parsed `StreamHealth`), `ProcessFailed` on unexpected non-zero exit. Constructor-injected locator + process factory (tests fake both) | | `Encoder/IFfmpegEncoder.cs` | **TASK 4 ship step 3 seam**: start/feed/stop the live encoder; `HealthUpdated` (parsed `StreamHealth`), `ProcessFailed` on unexpected non-zero exit. Constructor-injected locator + process factory (tests fake both) |
| `Encoder/FfmpegEncoder.cs` | The default encoder: spawn `ffmpeg.exe` (probe `-encoders` first → hardware NVENC/QSV/AMF, OpenH264 fallback — never libx264), feed raw BGRA frames into stdin, parse `-stats` lines into `StreamHealth`, graceful stop via stdin EOF + 10s kill watchdog. Constructed by `MainViewModel` since ship step 5 (`new FfmpegEncoder(new FfmpegLocator())`), driven by the `FramePump` | | `Encoder/FfmpegEncoder.cs` | The default encoder: spawn `ffmpeg.exe` (probe `-encoders` first → hardware NVENC/QSV/AMF, OpenH264 fallback — never libx264), feed raw BGRA frames into stdin, parse `-stats` lines into `StreamHealth`, graceful stop via stdin EOF + 10s kill watchdog. Constructed by `MainViewModel` since ship step 5 (`new FfmpegEncoder(new FfmpegLocator())`), driven by the `FramePump` |
@@ -49,7 +51,7 @@ External-facing logic: YouTube API, persistence. See
| `Encoder/FfmpegArgs.cs` | Pure FFmpeg command-line builder: rawvideo `pipe:0` input, `anullsrc` silent audio (WASAPI replaces it), H.264+AAC, closed GOP (`-g fps×4 -keyint_min -sc_threshold 0 -bf 0`, `yuv420p`), FLV → RTMP | | `Encoder/FfmpegArgs.cs` | Pure FFmpeg command-line builder: rawvideo `pipe:0` input, `anullsrc` silent audio (WASAPI replaces it), H.264+AAC, closed GOP (`-g fps×4 -keyint_min -sc_threshold 0 -bf 0`, `yuv420p`), FLV → RTMP |
| `Encoder/FfmpegProgressParser.cs` | Pure parser for `frame=/fps=/size=/time=/bitrate=` stats lines → `FfmpegProgress` | | `Encoder/FfmpegProgressParser.cs` | Pure parser for `frame=/fps=/size=/time=/bitrate=` stats lines → `FfmpegProgress` |
| `Encoder/FfmpegEncoderPicker.cs` | Pure H.264 encoder picker from `-encoders` output: NVENC → QSV → AMF → OpenH264; **never returns libx264** (GPL) | | `Encoder/FfmpegEncoderPicker.cs` | Pure H.264 encoder picker from `-encoders` output: NVENC → QSV → AMF → OpenH264; **never returns libx264** (GPL) |
| `Encoder/FramePump.cs` | **TASK 4 ship step 5**: the live frame producer — while live it snapshots the active scene each tick, resolves every element to its latest frame (`Func<SceneElement, VideoFrame?>` resolver), composites it into the tier's output frame, and paces frames into the encoder at the tier's FPS. All collaborators constructor-injected seams; free of WPF and the capture managers. `StartAsync` never throws (failures log + `Failed`); no RTMP URL = encoder skipped; `StopAsync` stops the encoder before awaiting the loop (backpressure deadlock); `ProcessFailed` self-stops. Since the bar bug-fix branch it takes an optional `socialBar: Func<(VideoFrame? Frame, SocialBarPosition Position)>?` seam — a pre-rendered bar strip composited last (above the flash) at the top edge or `SourceRectHeight − bar height`. See `ai.md` "Live frame pipeline" | | `Encoder/FramePump.cs` | **TASK 4 ship step 5**: the live frame producer — while live it snapshots the active scene each tick, resolves every element to its latest frame (`Func<SceneElement, VideoFrame?>` resolver), composites it into the tier's output frame, and paces frames into the encoder at the tier's FPS. All collaborators constructor-injected seams; free of WPF and the capture managers. `StartAsync` never throws (failures log + `Failed`); no RTMP URL = encoder skipped; `StopAsync` stops the encoder before awaiting the loop (backpressure deadlock); `ProcessFailed` self-stops. Since the bar bug-fix branch it takes an optional `socialBar: Func<(VideoFrame? Frame, SocialBarPosition Position)>?` seam — a pre-rendered bar strip composited last (above the flash) at the top edge or `SourceRectHeight − bar height`. **TASK 47:** adds an `alertTicker: Func<VideoFrame?>?` ctor seam (parallel to `socialBar`), read per tick and threaded through `RenderScene`/`RenderFull` + **mixed into `BuildFullRenderSignature`** so the ticker scroll changes the cache key. See `ai.md` "Live frame pipeline" |
| `Encoder/FfmpegLocator.cs` | The default locator: PATH probe wins, then the cache, then pull+extract (`ffmpeg.exe` **+ the `libav*.dll` family** via a staging dir, failures via `AppLog`). Pinned to BtbN `autobuild-2026-08-09-13-03` **lgpl-shared** (dynamic linking = LGPL compliance without static-relink §6 material; no GPL libx264; NVENC/QSV/AMF + libopenh264 + native AAC — see `ai.md` → Licensing). Search dirs / tools dir / downloader constructor-injected for hermetic tests. Wired into the encoder since ship step 5 | | `Encoder/FfmpegLocator.cs` | The default locator: PATH probe wins, then the cache, then pull+extract (`ffmpeg.exe` **+ the `libav*.dll` family** via a staging dir, failures via `AppLog`). Pinned to BtbN `autobuild-2026-08-09-13-03` **lgpl-shared** (dynamic linking = LGPL compliance without static-relink §6 material; no GPL libx264; NVENC/QSV/AMF + libopenh264 + native AAC — see `ai.md` → Licensing). Search dirs / tools dir / downloader constructor-injected for hermetic tests. Wired into the encoder since ship step 5 |
| `INotificationService.cs` | **Toast seam (TASK 24)**: `Show(title, message, AppNotificationSeverity)` + `AppNotificationSeverity { Info, Success, Warning, Error }` + `Info/Success/Warning/Error` extension methods — non-blocking user-facing messages; VMs/services raise through this so failures surface without MessageBox blocking the UI thread | | `INotificationService.cs` | **Toast seam (TASK 24)**: `Show(title, message, AppNotificationSeverity)` + `AppNotificationSeverity { Info, Success, Warning, Error }` + `Info/Success/Warning/Error` extension methods — non-blocking user-facing messages; VMs/services raise through this so failures surface without MessageBox blocking the UI thread |
| `NotificationService.cs` | The default seam impl (Notification.Wpf 11.0.0, MIT): wraps a library `NotificationManager`; routes to the in-window `NotificationArea` named `ToastArea` (MainWindow bottom-right above the footer); lifetimes via the pure `BuildRequest` mapping + `Cards` tint table — Info/Success ~4s, Warning ~8s auto-dismiss, Error sticky (`NeverExpires()` = `TimeSpan.MaxValue`, never null); dark card tints matching the health-banner language; all Show calls marshal to the captured Dispatcher. See [`../ai.md`](../ai.md) "Toast notifications" | | `NotificationService.cs` | The default seam impl (Notification.Wpf 11.0.0, MIT): wraps a library `NotificationManager`; routes to the in-window `NotificationArea` named `ToastArea` (MainWindow bottom-right above the footer); lifetimes via the pure `BuildRequest` mapping + `Cards` tint table — Info/Success ~4s, Warning ~8s auto-dismiss, Error sticky (`NeverExpires()` = `TimeSpan.MaxValue`, never null); dark card tints matching the health-banner language; all Show calls marshal to the captured Dispatcher. See [`../ai.md`](../ai.md) "Toast notifications" |
@@ -57,7 +59,7 @@ External-facing logic: YouTube API, persistence. See
| `Audio/AudioSample.cs` | One captured chunk: interleaved PCM float (-1..1) + sample rate + channels | | `Audio/AudioSample.cs` | One captured chunk: interleaved PCM float (-1..1) + sample rate + channels |
| `Audio/WasapiLoopbackAudioSource.cs` | Desktop/game capture: NAudio `WasapiLoopbackCapture` on the default render device — automatic at unity; the game bar's meter consumes it | | `Audio/WasapiLoopbackAudioSource.cs` | Desktop/game capture: NAudio `WasapiLoopbackCapture` on the default render device — automatic at unity; the game bar's meter consumes it |
| `Audio/WasapiMicAudioSource.cs` | Mic capture: NAudio `WasapiCapture` with the device resolved by `FriendlyName` matching `MicSourceName` (DisplayName only), default capture endpoint fallback; name provider `Func<string?>` re-read at each `Start` so a device picked mid-session takes effect immediately (the mixer restarts the mic on pick) | | `Audio/WasapiMicAudioSource.cs` | Mic capture: NAudio `WasapiCapture` with the device resolved by `FriendlyName` matching `MicSourceName` (DisplayName only), default capture endpoint fallback; name provider `Func<string?>` re-read at each `Start` so a device picked mid-session takes effect immediately (the mixer restarts the mic on pick) |
| `Audio/AudioMixer.cs` | Owns both sources; capture starts once at startup (`MainViewModel.StartMicCaptureAsync`) and stops on `Shutdown` — NOT go-live (preview monitoring). Mic samples → `AudioLevelMeter` → `MicLevelChanged`; loopback samples → the game bar's meter via `LoopbackLevelChanged`. Surfaces mic connection state: `MicConnected`/`MicFailed` (drives the status dot) + `RestartMic()` (device swap mid-session, keeps loopback). Failures log via `AppLog`; mic failure zeroes the meter, loopback failure doesn't kill the mic. Note: the meter `Push` is unconditional (the `?.` on the event would otherwise skip the argument when nothing is subscribed) | | `Audio/AudioMixer.cs` | Owns both sources; capture starts once at startup (`MainViewModel.StartMicCaptureAsync`) and stops on `Shutdown` — NOT go-live (preview monitoring). Mic samples → `AudioLevelMeter` → `MicLevelChanged`; loopback samples → the game bar's meter via `LoopbackLevelChanged`. Surfaces mic connection state: `MicConnected`/`MicFailed` (drives the status dot) + `RestartMic()` (device swap mid-session, keeps loopback). Failures log via `AppLog`; mic failure zeroes the meter, loopback failure doesn't kill the mic. Note: the meter `Push` is unconditional (the `?.` on the event would otherwise skip the argument when nothing is subscribed). **TASK 47 — alert audio:** `EnqueueAlertAudio` writes a dedicated 8s stereo-48k ring (`AlertBufferSeconds`), drained in `FillAndMix` pre-limiter and added at **unity (never ducked)**; `StartLive` clears the ring |
| `Audio/AudioLevelMeter.cs` | Pure smoothed RMS level (0..1): `Push(AudioSample)` + `Reset` — the unit-tested math behind `AudioLevel` and `GameAudioLevel`. `ToDisplay(float)` maps the raw linear RMS onto the meter's display scale (−60..0 dBFS spread across 0..1, with **+10 dB input amplification** so real speech peaks hit the red zone at maxed volume) — real speech/game RMS (~0.01..0.1) would otherwise leave a flat scale looking dead; ≤0.001 linear reads as zero (never idles on background noise) | | `Audio/AudioLevelMeter.cs` | Pure smoothed RMS level (0..1): `Push(AudioSample)` + `Reset` — the unit-tested math behind `AudioLevel` and `GameAudioLevel`. `ToDisplay(float)` maps the raw linear RMS onto the meter's display scale (−60..0 dBFS spread across 0..1, with **+10 dB input amplification** so real speech peaks hit the red zone at maxed volume) — real speech/game RMS (~0.01..0.1) would otherwise leave a flat scale looking dead; ≤0.001 linear reads as zero (never idles on background noise) |
| `Audio/WaveToFloat.cs` | Pure WASAPI buffer → float conversion: IEEE float 32-bit direct, PCM 16-bit normalized, `WaveFormatExtensible` IEEE-float subformat GUID, trailing partial samples ignored | | `Audio/WaveToFloat.cs` | Pure WASAPI buffer → float conversion: IEEE float 32-bit direct, PCM 16-bit normalized, `WaveFormatExtensible` IEEE-float subformat GUID, trailing partial samples ignored |
| `Audio/MasterLimiter.cs` | **Master output limiter**: a **−1 dBFS ceiling** (`Ceiling = 0.891`) applied after the live mix so the honest gains can never sum past 0 dBFS and clip the AAC encode. Instant attack per frame (a hot frame is scaled exactly to the ceiling — no overshoot), smoothed release toward unity so loud passages don't pump. Pure — unit-tested | | `Audio/MasterLimiter.cs` | **Master output limiter**: a **−1 dBFS ceiling** (`Ceiling = 0.891`) applied after the live mix so the honest gains can never sum past 0 dBFS and clip the AAC encode. Instant attack per frame (a hot frame is scaled exactly to the ceiling — no overshoot), smoothed release toward unity so loud passages don't pump. Pure — unit-tested |
+12
View File
@@ -62,6 +62,7 @@
| 44 | TEST-tab chat fix: resolve liveChatId from snippet (not contentDetails) + poll until the broadcast is live | ✅ Done (2026-09-25); full suite 318/318 | [`TASKS/task-44-test-chat-fix.md`](TASKS/task-44-test-chat-fix.md) | | 44 | TEST-tab chat fix: resolve liveChatId from snippet (not contentDetails) + poll until the broadcast is live | ✅ Done (2026-09-25); full suite 318/318 | [`TASKS/task-44-test-chat-fix.md`](TASKS/task-44-test-chat-fix.md) |
| 45 | TEST-tab chat fix #2: insert body must declare `snippet.type` (400 MISSING_REQUIRED_FIELD) | ✅ Done (2026-09-25) | [`TASKS/task-45-chat-insert-type.md`](TASKS/task-45-chat-insert-type.md) | | 45 | TEST-tab chat fix #2: insert body must declare `snippet.type` (400 MISSING_REQUIRED_FIELD) | ✅ Done (2026-09-25) | [`TASKS/task-45-chat-insert-type.md`](TASKS/task-45-chat-insert-type.md) |
| 46 | Drawers: click outside the rail closes whichever is open — TEST added to the existing Stream Settings + YPP dismiss behavior | ✅ Done (2026-09-25) | [`TASKS/task-46-drawer-click-outside-close.md`](TASKS/task-46-drawer-click-outside-close.md) | | 46 | Drawers: click outside the rail closes whichever is open — TEST added to the existing Stream Settings + YPP dismiss behavior | ✅ Done (2026-09-25) | [`TASKS/task-46-drawer-click-outside-close.md`](TASKS/task-46-drawer-click-outside-close.md) |
| 47 | Alert box video: built-in/custom alert clip (+ six-animation fallback) with read-time fade in/out + ticker + alert volume in the live mix | ✅ Done (2026-09-26) | [`TASKS/task-47-alert-videos.md`](TASKS/task-47-alert-videos.md) |
--- ---
@@ -120,6 +121,17 @@
enableAutoStart; YouTube only populates the id on live broadcasts — official GetLiveChatId.java enableAutoStart; YouTube only populates the id on live broadcasts — official GetLiveChatId.java
sample lists broadcastStatus=active). Fixed by reading `snippet.liveChatId` and polling with a sample lists broadcastStatus=active). Fixed by reading `snippet.liveChatId` and polling with a
bounded retry after the frame pump starts; chat stays non-fatal. bounded retry after the frame pump starts; chat stays non-fatal.
- **TASK 47 — Alert box video** — **DONE 2026-09-26** (`TASKS/task-47-alert-videos.md`):
the TASK 43 alert box now plays a **video** on every alert — a shipped built-in mp4
(`Assets/alert-default.mp4`, stamped into the `Asset` table at startup) that the creator can
swap for their own file via the new Stream Alerts section (path only, never stored in the DB;
six `AlertRenderer` animations remain the fallback). Per-alert decode via a new
`IAlertClipDecoder` fx (ffmpeg rawvideo bgra + f32le pipes, real-time paced, disposed at
drain); ~0.3s fade-in/out rides the alpha envelope + scales the audio (mixed into the live
stream at unity — **no duck**, a per-alert Volume slider instead); an auto-composed message
**ticker** ("Funder — Super Chat · $10.00") scrolls along the very top of the frame
(never-baked dynamic overlay threaded through SceneCompositor + FramePump). Creator rulings:
custom + fallback (not either/or), ticker on top, no ducking.
--- ---
+138
View File
@@ -0,0 +1,138 @@
# TASK 47 — Alert box video (TASK 43 follow-up): creator-replaceable alert clip + read-time fade + auto ticker
> Catalog: [`TASKS.md`](../TASKS.md). Status: ✅ **SHIPPED 2026-09-26** — the TASK 43
> alert box grows a real **video** celebration: a built-in mp4 loops on every alert unless
> the creator picks their own clip (referenced from disk, never stored in the DB), the
> clip fades in/out in ~0.3s (alpha envelope on the video + volume on the audio, mixed
> into the live stream at unity — no duck), and an auto-composed **message ticker**
> ("Funder — Super Chat · $10.00") scrolls across the very top of the frame, toggleable.
> The six TASK 43 animations remain the intrinsic fallback when no asset is playable.
## Provenance (recorded per the feature-provenance rule)
- **2026-09-24, the creator asked (right after TASK 43 shipped)** — "make the alerts
animate like a video" / "I want to play my own video for alerts." The TASK 43 unit
closed with this as the explicit next step.
- **Lineage:** Research (websearch: how do OBS/StreamElements/Streamlabs alert boxes show
per-alert videos?) settled the shape, cited in the commit message:
- Alert videos play **per-alert** (a fresh decoder per event), never a resident media
player — the OBS "media source reset on activation" model, not a playlist.
- The clip **must not block**: decode is a background pipe (ffmpeg child, rawvideo
frames + f32le audio), paced real-time, so alerts never stall the 60fps frame pump.
- A **freeze-frame at clip EOF fades out** rather than hard-dropping (Streamlabs'
"end by fade" option) — the tail of a long clip dissolves without a new spawn.
- The custom file is served by **path, not bytes** (OBS media-source semantics; a
read-at-play clip lets the creator re-edit the file without a re-import). Decided
with the creator: "never store the video in the layout." The built-in default is the
one BLOB, stamped into the internal `Asset` table at startup.
- **Creator rulings (asked once, held):** custom video + six-animation fallback
(not either/or); ticker on top of the canvas (very top, full width); **no ducking** —
"don't lower my game audio for a tip" (the TASK 43 planner's auto-duck idea was
rejected on the floor); a per-alert **Volume** slider instead.
## Product model (as shipped)
- **Five new `Source` props** (schema v10, additive guarded ALTERs): `AlertVideoPath`
(custom file), `AlertUseDefaultVideo` (bool, default true — the built-in clip),
`AlertShowTicker` (bool, default true), `AlertVideoVolume` (0–1, default 1).
`AlertVideoAssetId` (internal reference to the stamped default asset row).
- **Built-in clip:** `Assets/alert-default.mp4` (~4.9MB, branded 2.5s loop, included as a
csproj Resource). At startup `MainViewModel.StampDefaultAlertVideo()` upserts its bytes
into the `Asset` table and records the row id in the settings key
`AlertDefaultVideoAssetId` (`AlertDefaultVideoKey`). The **prune** step exempts that
key via a UNION so the default never gets cleaned.
- **Per-alert resolver** (`ResolveAlertClipPath` in `MainViewModel.Chat.cs`): custom path
wins ONLY when `AlertUseDefaultVideo == false` AND the file exists; otherwise the
thumbnailed built-in is materialized once to `%TEMP%\ytLive-alert-{id}.mp4`; failure →
the six `AlertRenderer` animations play (fallback, not an error).
- **`Services/AlertClipDecoder.cs`** — `IAlertClipDecoder` (Start/Stop/Dispose +
`FrameAvailable(Action<VideoFrame>)` / `AudioReady(Action<AudioSample>)` /
`Completed(Action)`) + `AlertClipDecoder` impl: ffmpeg `-loglevel error` child
emitting `rawvideo bgra` (`-vf scale=W:H`) on one pipe and `f32le -ar 48000 -ac 2` on
another; `RawVideoFrameReader` (the TASK 21 media-source reader) on the video pipe,
a **carry-buffer** loop on the audio pipe (float-boundary straddles never drop a
sample — the 3-byte PCM16↔4-byte-f32 mismatch that broke the first draft), both
real-time paced. Per-play lifetime: the layer creates one per alert and disposes it at
drain. Prefer `Start`-synced `TrackFrameRate` — the frame pipe throttles to vfr
timestamps; audio paces itself. ffmpeg bgra is opaque (alpha=255) — the layer owns all
alpha.
- **`Services/AlertOverlayLayer.cs`** — clip branch beside the animation path:
`Enqueue` → `BeginClip` (factory + path + box dims seam), `Advance` advances `_elapsed`
(fade-in frame copies scale alpha 0→255 straight-source over the box), events forward
to the audio sink scaled by `volume × fade`, EOF → **freeze-frame** + fade-out over the
same `FadeDurationSeconds = 0.30` envelope → drain (StopClip + dispose, idle null
again). Ticker: `AlertTickerFrame` composes "Author — Kind · amount" from the live
message and scrolls it.
- **`Services/Compositor/AlertTickerRenderer.cs`** — marquee strip: 1920×48 transparent
BGRA, text-pill rasterized once per text on the UI thread (cached, unpremultiplied),
then pure byte-math per call (scroll `pos = elapsed × 140px/s % cycle`, pill drawn
straight-alpha over transparent with a repeat-gap copy), so the pump can call it from
its own thread every tick with no locking.
- **Ticker threading** — dynamic overlay like the social bar, but **never baked**
(the social bar IS baked into `BakeStaticBase`; the ticker is a per-frame marquee):
`SceneCompositor.Render` + `CompositeLayers` take a trailing `tickerFrame` param
(blitted at 0,0 after the social bar), threaded by `FramePump._alertTicker`
(`Func<VideoFrame?>` seam, ctor param) through `RenderScene`/`RenderFull`/
`BuildFullRenderSignature` (mixed into the cache signature so a ticker change
invalidates — ticker is distinct from static-cache-eligible content).
- **Audio path** — `AudioMixer.EnqueueAlertAudio` uses a dedicated ring
(`AlertBufferSeconds = 8` at 48k stereo): resample/upmix to stereo 48k, drained
pre-limiter in `FillAndMix` and added at **unity** (never ducked); `StartLive` clears
it. The layer applies `volume × fade` per sample before forwarding.
- **UI — Stream Alerts section** (LeftPanel, visible only on an `AlertBox`):
"Built-in video" pill; custom path box + Browse…; "Reset to built-in"; "Message
ticker" pill; "Volume" slider with % readout. Wiring: `BrowseAlertVideoCommand` +
`ResetAlertVideoCommand` (CanExecute gates on `Source { Type: AlertBox }`); browse =
OpenFileDialog (mp4/mov/webm), result writes `AlertVideoPath` + switches the pill off.
## Test (Good Dog — ONE integration test per change)
`ytLive.Tests/AlertLayerVideoTests.cs` (RealApp STA host, real WPF raster) —
`AlertVideo_PlaysCustomClip_FadesReadTime_TickerScrolls_ForwardsScaledAudio_DrainsIdle`
drives a **fake `IAlertClipDecoder`** (no ffmpeg in tests) through the whole lifecycle
in one pass: the custom path wins per-source (resolution seam asserts box W/H on the
factory + same source to the resolver); the decoder spawns on play (StartCount 1) and
dies at drain (DisposeCount 1); fade-in rides the 0.3s envelope (alpha 0 → 127 at 0.15s
→ 255, straight-alpha copy); pre-fade audio is dropped while ramp audio forwards at
`volume × fade` (0.8 × 0.5 → [0.2, −0.1]) and full-fade at volume only; the ticker
renders non-null with pixels and **scrolls** between two pump ticks while idle returns
null; EOF freeze → fade-out 127 → drain to `IsPlaying false` + null frame + disposed
decoder.
> Catches a REAL bug the animation path never had: the clip branch of `Advance` failed
> to clear `_current` before `AdvanceToNext`, so after the fade-out the layer stayed
> `IsPlaying` and re-rendered the finished alert via the animation fallback. The clean
> frame literally forced the `Assert.False(layer.IsPlaying)` to see it.
## Files (declared scope)
- `Models/Source.cs` (five alert props + `IsAlertBox`), `Services/LayoutStore.Migrations.cs`
(guarded ALTERs), `Services/LayoutStore.Save.cs` (INSERT + prune UNION),
`Services/LayoutStore.Load.cs` (reader indices),
`Services/LayoutStore.Settings.cs` (`AlertDefaultVideoKey` + asset stamps)
- `Assets/alert-default.mp4` + `ytLive.csproj` (Resource include)
- **NEW** `Services/AlertClipDecoder.cs`, **NEW** `Services/Compositor/AlertTickerRenderer.cs`
- `Services/AlertOverlayLayer.cs` (clip branch + fade + ticker + sink), `Services/Audio/AudioMixer.cs`
(alert ring + `EnqueueAlertAudio` + unity drain), `Services/Compositor/SceneCompositor.cs` +
`Services/Encoder/FramePump.cs` (ticker seam)
- `ViewModels/MainViewModel.cs` + `ViewModels/MainViewModel.Chat.cs` (seams, stamp,
resolver, materialize, browse/reset), `Controls/LeftPanel.xaml` (Stream Alerts section)
- **NEW** `ytLive.Tests/AlertLayerVideoTests.cs`
- Docs: `ai.md`, `TASKS.md`, this file, `Services/index.md`, `HANDOFF.md`, `MyMistakes.md`
## Gate
Clean build 0 warnings (Windows dotnet host); alert tests green (`AlertLayer*
FullyQualifiedName` filter = 2/2). Full suite 318/319 — the one failure
`LayerReorderPersistenceTests.RealMouseDrag…` is the HANDOFF-documented environmental
class (physical-mouse drag no-ops when the desktop/session isn't interactive; it passed
in isolation on the third re-run of this unit; the two sibling reorder tests — identical
persistence path — pass). `scripts/scope-check.sh` green.
## Open follow-ups (NOT this unit)
- TASK 3 item 20 (RewardEvent SQLite persistence half) and item 16 (Text source) remain
queued, unchanged.
- The alert-store folder (`Apps_Commands`-style per-clip assets) stays out of scope —
the current model is read-at-play from a path, per the creator's "never store the
video" ruling.
+110
View File
@@ -1,8 +1,13 @@
using System;
using System.Collections.ObjectModel; using System.Collections.ObjectModel;
using System.IO;
using System.Linq; using System.Linq;
using System.Windows;
using Microsoft.Win32;
using ytLive.Helpers; using ytLive.Helpers;
using ytLive.Models; using ytLive.Models;
using ytLive.Services; using ytLive.Services;
using ytLive.Services.Encoder;
namespace ytLive.ViewModels; namespace ytLive.ViewModels;
@@ -60,4 +65,109 @@ public partial class MainViewModel : ViewModelBase
private void UpdateActiveAlertBoxPreviews() private void UpdateActiveAlertBoxPreviews()
=> _alertLayer.UpdateConfiguredBoxes(Scenes); => _alertLayer.UpdateConfiguredBoxes(Scenes);
// ─── TASK 47: the alert clip (video) ───
private string? _alertDefaultVideoId;
private string? _defaultAlertClipScratch;
/// <summary>Runtime clip resolution for one alert box: the Creator's OWN video
/// is a plain path on disk (never stored — the 2026-09-26 decision); the shipped
/// llama placeholder lives as a BLOB in the internal Asset table and is
/// materialized to a temp file once. Null → the six celebratory animations.
/// Called from the output pump's alert frame path.</summary>
private string? ResolveAlertClipPath(Source box)
{
if (!box.AlertUseDefaultVideo
&& !string.IsNullOrWhiteSpace(box.AlertVideoPath)
&& File.Exists(box.AlertVideoPath))
return box.AlertVideoPath;
return MaterializeDefaultAlertClip();
}
private string? MaterializeDefaultAlertClip()
{
if (_defaultAlertClipScratch != null && File.Exists(_defaultAlertClipScratch))
return _defaultAlertClipScratch;
_alertDefaultVideoId ??= _layoutStore.GetAlertDefaultVideoId();
if (_alertDefaultVideoId == null) return null;
var bytes = _layoutStore.GetAssetBytes(_alertDefaultVideoId);
if (bytes == null) return null;
var scratch = Path.Combine(Path.GetTempPath(), $"ytLive-alert-{_alertDefaultVideoId}.mp4");
try
{
File.WriteAllBytes(scratch, bytes);
_defaultAlertClipScratch = scratch;
return scratch;
}
catch (Exception ex)
{
AppLog.Write(ex, "Alert default materialization failed");
return null;
}
}
/// <summary>One decoder per play (the refcounted MediaVideoSourceManager sessions
/// can't restart per replay). The clip is full-box in video + 48k stereo audio;
/// both steams pace themselves to real time inside the decoder.</summary>
private IAlertClipDecoder AlertClipDecoderFor(string path, int width, int height) =>
new AlertClipDecoder(path, width, height, new FfmpegLocator(), () => new FfmpegDecodeProcess());
/// <summary>Stamp the shipped llama placeholder into the Asset table at startup
/// (idempotent — hash-keyed UpsertAsset dedupes). Best-effort: if it fails the
/// alert box simply falls back to the six animations.</summary>
private void StampDefaultAlertVideo()
{
try
{
_alertDefaultVideoId = _layoutStore.GetAlertDefaultVideoId();
if (!string.IsNullOrWhiteSpace(_alertDefaultVideoId)) return;
var bytes = ReadAlertDefaultVideoResource();
if (bytes is null or { Length: 0 }) return;
_alertDefaultVideoId = _layoutStore.EnsureAlertDefaultVideo(bytes);
}
catch (Exception ex)
{
AppLog.Write(ex, "Alert default video stamp failed");
}
}
private static byte[]? ReadAlertDefaultVideoResource()
{
try
{
var uri = new Uri("pack://application:,,,/Assets/alert-default.mp4", UriKind.Absolute);
using var stream = Application.GetResourceStream(uri)?.Stream;
if (stream == null) return null;
using var ms = new MemoryStream();
stream.CopyTo(ms);
return ms.ToArray();
}
catch (Exception ex)
{
AppLog.Write(ex, "Alert default video resource missing");
return null;
}
}
private void BrowseAlertVideo(Source? box)
{
if (box == null || box.Type != SourceType.AlertBox) return;
var dialog = new OpenFileDialog
{
Title = "Choose the alert clip (mp4)",
Filter = "Video files (*.mp4;*.mov;*.webm;*.mkv)|*.mp4;*.mov;*.webm;*.mkv|All files (*.*)|*.*"
};
if (dialog.ShowDialog() != true) return;
box.AlertVideoPath = dialog.FileName;
box.AlertUseDefaultVideo = false;
}
private void ResetAlertVideo(Source? box)
{
if (box == null || box.Type != SourceType.AlertBox) return;
box.AlertVideoPath = null;
box.AlertShowTicker = true;
box.AlertUseDefaultVideo = true;
}
} }
+11 -1
View File
@@ -139,6 +139,8 @@ public partial class MainViewModel : ViewModelBase
public ICommand RemoveSourceCommand { get; } public ICommand RemoveSourceCommand { get; }
public ICommand EditElementCommand { get; } public ICommand EditElementCommand { get; }
public ICommand ToggleElementVisibilityCommand { get; } public ICommand ToggleElementVisibilityCommand { get; }
public ICommand BrowseAlertVideoCommand { get; }
public ICommand ResetAlertVideoCommand { get; }
public ICommand ActivateLicenseCommand { get; } public ICommand ActivateLicenseCommand { get; }
public ICommand OpenLicenseEntryCommand { get; } public ICommand OpenLicenseEntryCommand { get; }
public ICommand OpenHotkeyConfigCommand { get; } public ICommand OpenHotkeyConfigCommand { get; }
@@ -187,6 +189,8 @@ public partial class MainViewModel : ViewModelBase
RemoveSourceCommand = new RelayCommand(element => RemoveElement(element as SceneElement)); RemoveSourceCommand = new RelayCommand(element => RemoveElement(element as SceneElement));
EditElementCommand = new RelayCommand(element => BeginEditElement(element as SceneElement)); EditElementCommand = new RelayCommand(element => BeginEditElement(element as SceneElement));
ToggleElementVisibilityCommand = new RelayCommand(element => ToggleElementVisibility(element as SceneElement)); ToggleElementVisibilityCommand = new RelayCommand(element => ToggleElementVisibility(element as SceneElement));
BrowseAlertVideoCommand = new RelayCommand(element => BrowseAlertVideo(element as Source), element => element is Source { Type: SourceType.AlertBox });
ResetAlertVideoCommand = new RelayCommand(element => ResetAlertVideo(element as Source), element => element is Source { Type: SourceType.AlertBox });
ChangeWebcamCommand = new RelayCommand(_ => _ = ChangeWebcamAsync(), _ => CanChangeWebcam); ChangeWebcamCommand = new RelayCommand(_ => _ = ChangeWebcamAsync(), _ => CanChangeWebcam);
ShowWebcamCommand = new RelayCommand(_ => ShowWebcamInStagedScene(), _ => CanShowWebcamInStagedScene); ShowWebcamCommand = new RelayCommand(_ => ShowWebcamInStagedScene(), _ => CanShowWebcamInStagedScene);
OpenSocialDialogCommand = new RelayCommand(_ => OpenSocialDialog()); OpenSocialDialogCommand = new RelayCommand(_ => OpenSocialDialog());
@@ -308,7 +312,11 @@ public partial class MainViewModel : ViewModelBase
_mediaManager.MediaFailed += OnMediaFailed; _mediaManager.MediaFailed += OnMediaFailed;
_chatLayer = new ChatOverlayLayer(new ChatBoxRenderer()); _chatLayer = new ChatOverlayLayer(new ChatBoxRenderer());
_alertLayer = new AlertOverlayLayer(new AlertRenderer()); _alertLayer = new AlertOverlayLayer(
new AlertRenderer(),
clipDecoderFactory: AlertClipDecoderFor,
clipPathResolver: ResolveAlertClipPath,
audioSink: _audioMixer.EnqueueAlertAudio);
_rtmpUrlProvider = () => _reusableStreamUrl; // TASK 5: the reusable stream's ingest URL _rtmpUrlProvider = () => _reusableStreamUrl; // TASK 5: the reusable stream's ingest URL
_framePump = new FramePump( _framePump = new FramePump(
@@ -319,12 +327,14 @@ public partial class MainViewModel : ViewModelBase
encoderFactory: () => new FfmpegEncoder(new FfmpegLocator()), encoderFactory: () => new FfmpegEncoder(new FfmpegLocator()),
log: message => AppLog.Write(message), log: message => AppLog.Write(message),
socialBar: () => (_socialBarFrame, _socials?.BarPosition ?? SocialBarPosition.Bottom), socialBar: () => (_socialBarFrame, _socials?.BarPosition ?? SocialBarPosition.Bottom),
alertTicker: () => _alertLayer.AlertTickerFrame,
transition: _transition, transition: _transition,
sceneGraph: _sceneGraph); sceneGraph: _sceneGraph);
_framePump.Failed += OnFramePumpFailed; _framePump.Failed += OnFramePumpFailed;
_framePump.HealthUpdated += OnFramePumpHealthUpdated; _framePump.HealthUpdated += OnFramePumpHealthUpdated;
LoadLayout(); LoadLayout();
StampDefaultAlertVideo();
// Resource validation runs as soon as the layout is up: enumerate the // Resource validation runs as soon as the layout is up: enumerate the
// OS once, and if exactly one webcam exists, lock it for the session // OS once, and if exactly one webcam exists, lock it for the session
// (the saved identity's scene configs may already hold it — refcounted). // (the saved identity's scene configs may already hold it — refcounted).
+28
View File
@@ -1616,3 +1616,31 @@ published decisions (recorded in `TASKS/task-43-native-alerts.md`):
**never enqueue** — per the creator ruling, a subscriber mention is sufficient; there is **never enqueue** — per the creator ruling, a subscriber mention is sufficient; there is
no free-sub alert (YouTube emits none) and no viewer count (vetoed — demotivating, no free-sub alert (YouTube emits none) and no viewer count (vetoed — demotivating,
weaponizable). weaponizable).
- **Alert box VIDEO (TASK 47, 2026-09-26):** since TASK 47 the box plays a clip on every
alert instead of (only) an animation — `Enqueue`→`BeginClip` spawns a **per-play**
`IAlertClipDecoder` (ffmpeg `rawvideo bgra -vf scale=W:H` frame pipe + `f32le` audio
pipe, both real-time paced via `RawVideoFrameReader`; **not** the refcounted
`MediaVideoSourceManager` — alerts are one-shot plays that must be disposed at drain).
Custom clip wins per-source over the built-in ONLY when `AlertUseDefaultVideo==false`
AND the file exists; the shipped `Assets/alert-default.mp4` is stamped into the `Asset`
table at startup (`AlertDefaultVideoKey` setting; the prune keeps it via UNION) and
read-at-play; any decode failure falls back to the six `AlertRenderer` animations
(deliberate, not an error). Fades in/out via **`FadeDurationSeconds = 0.30`**
(`AlertOverlayLayer` const): fade-in by `elapsed` on straight-alpha frame copies;
EOF → freeze-frame → fade-out → drain (the clip branch of `Advance` **clears
`_current` before `AdvanceToNext`** — the miss that the TASK 47 test caught). Audio
forwards to the mixer sink scaled by `volume × fade`.
- **Alert ticker:** `AlertTickerFrame` (in the layer) composes "Author — Kind · amount"
and hands it to `Services/Compositor/AlertTickerRenderer.cs` — a 1920×48 marquee strip
rasterized **on the UI thread once per text** (cached, unpremultiplied straight-alpha
pill) then scrolled by pure byte-math per call (`140px/s`, repeat-gap wrap), so the
pump calls it from its own thread with no locking. **It is a dynamic overlay, NEVER
baked** (the social bar IS baked into `BakeStaticBase`): `SceneCompositor.Render` +
`CompositeLayers` take a trailing `tickerFrame` (blitted last at 0,0) threaded from
`FramePump._alertTicker` (`Func<VideoFrame?>` ctor seam) through
`RenderScene`/`RenderFull` and **mixed into `BuildFullRenderSignature`** so a scroll
changes the cache key. Toggled by `AlertShowTicker`.
- **Alert audio under the master limiter:** `AudioMixer.EnqueueAlertAudio` writes a
dedicated 8s stereo-48k ring (`AlertBufferSeconds`), drained in `FillAndMix` before the
limiter and added at **unity — never ducked** (creator ruling: "don't lower my game
audio for a tip"); `StartLive` clears it. Per-alert loudness = the Volume slider.
+158
View File
@@ -0,0 +1,158 @@
using System;
using System.Collections.Generic;
using System.Linq;
using Xunit;
using ytLive.Models;
using ytLive.Services;
using ytLive.Services.Audio;
namespace ytLive.Tests;
/// <summary>
/// TASK 47 Good Dog: the video-backed alert unit — the SAME good-dog single test
/// pinned on the RealApp STA host, driven through a FAKE <see cref="IAlertClipDecoder"/>
/// (the repo never runs a real codec in tests). It pins the whole lifecycle in one
/// pass: the custom clip path wins per-source over the built-in (resolution
/// seam), the decoder is created at play + torn down at drain, the ~300ms fade-in/out
/// alpha ENVELOPE rides the frames, the audio forwards at volume × fade (unity-add,
/// never ducked elsewhere), the EOF freeze-frame fades the tail out, the ticker strip
/// renders and scrolls during play and vanishes when idle — and the alert box always
/// returns to the transparent/idle state. The six-animation fallback stays covered by
/// AlertLayerTests.
/// </summary>
[Collection("RealApp")]
public sealed class AlertLayerVideoTests
{
private readonly RealAppHost _app;
public AlertLayerVideoTests(RealAppHost app) => _app = app;
[Fact]
public void AlertVideo_PlaysCustomClip_FadesReadTime_TickerScrolls_ForwardsScaledAudio_DrainsIdle()
{
_app.Run(Run);
}
private void Run()
{
const string clipPath = @"C:\fake\thank-you.mp4";
var fake = new FakeAlertClipDecoder();
var forwarded = new List<AudioSample>();
string? resolvedPath = null;
var box = new Source
{
Type = SourceType.AlertBox,
Name = "Stream Alerts",
Width = 600,
Height = 200,
AlertVideoPath = clipPath,
AlertUseDefaultVideo = false,
AlertShowTicker = true,
AlertVideoVolume = 0.8,
};
var scenes = new List<Scene> { new Scene { Name = "Main", Elements = { box } } };
var layer = new AlertOverlayLayer(
new AlertRenderer(),
clipDecoderFactory: (path, width, height) =>
{
resolvedPath = path;
Assert.Equal((int)box.Width, width);
Assert.Equal((int)box.Height, height);
return fake;
},
clipPathResolver: source =>
{
Assert.Same(box, source);
return source.AlertVideoPath;
},
audioSink: sample => forwarded.Add(sample));
layer.UpdateConfiguredBoxes(scenes);
var msg = new ChatMessage
{
Kind = ChatEventKind.SuperChat,
AuthorName = "Funder",
SuperChatDisplayString = "$10.00",
};
// ── Play: the custom path feeds a per-play decoder ──
layer.Enqueue(msg);
Assert.True(layer.IsPlaying);
Assert.Equal(1, fake.StartCount);
Assert.Equal(clipPath, resolvedPath);
fake.EmitFrame(600, 200);
// Audio inside the fade-in ramp (gain 0) must not leak into the mix.
fake.EmitAudio(new[] { 0.5f, -0.25f });
Assert.Empty(forwarded);
// ── Fade-in: alpha rides the 0.3s envelope measured against a real clip ──
Assert.Equal(0, AlphaAt(layer.RenderFrame(box)!)); // t=0
layer.Advance(0.15); // mid-ramp
Assert.Equal(127, AlphaAt(layer.RenderFrame(box)!));
fake.EmitAudio(new[] { 0.5f, -0.25f });
var midSample = Assert.Single(forwarded).Samples;
Assert.Equal(0.2, midSample[0], 2); // 0.8 × 0.5 mid-ramp
Assert.Equal(-0.1, midSample[1], 2);
forwarded.Clear();
layer.Advance(0.20); // past the ramp → opaque
Assert.Equal(255, AlphaAt(layer.RenderFrame(box)!));
fake.EmitAudio(new[] { 1.0f, 0.0f });
Assert.Equal(0.8, forwarded[^1].Samples[0], 2); // full fade: volume only
Assert.Equal(0.0, forwarded[^1].Samples[1], 2);
// ── Ticker: on during play, scrolls between frames ──
layer.Advance(1.0);
var tickerA = layer.AlertTickerFrame;
layer.Advance(0.1);
var tickerB = layer.AlertTickerFrame;
Assert.False(tickerA is null);
Assert.True(HasPixels(tickerA!), "the ticker must be visible while an alert plays");
Assert.False(tickerA!.BgraPixels.SequenceEqual(tickerB!.BgraPixels), "the ticker must scroll");
// ── EOF → freeze-frame fade-out over the same envelope, then drain ──
fake.End();
layer.Advance(0.15);
Assert.Equal(127, AlphaAt(layer.RenderFrame(box)!)); // halfway out
layer.Advance(0.20); // past the 0.3s tail
Assert.False(layer.IsPlaying);
Assert.Null(layer.RenderFrame(box));
Assert.Null(layer.AlertTickerFrame);
Assert.True(fake.DisposeCount == 1, "the per-play decoder must be torn down at drain");
}
private static byte AlphaAt(VideoFrame frame) => frame.BgraPixels[3];
private static bool HasPixels(VideoFrame frame) => frame.BgraPixels.Any(b => b != 0);
/// <summary>A decoder seam the test drives synchronously (no ffmpeg, no threads):
/// the layer calls Start/Stop/Dispose and the test feeds frames/audio/EOF.</summary>
private sealed class FakeAlertClipDecoder : IAlertClipDecoder
{
public int StartCount;
public int StopCount;
public int DisposeCount;
public event Action<VideoFrame>? FrameAvailable;
public event Action<AudioSample>? AudioReady;
public event Action? Completed;
public void Start() => StartCount++;
public void Stop() => StopCount++;
public void Dispose() => DisposeCount++;
public void EmitFrame(int width, int height)
{
var pixels = new byte[width * height * 4];
for (var i = 3; i < pixels.Length; i += 4)
pixels[i] = 255; // opaque straight-alpha BGRA, blue body
FrameAvailable?.Invoke(new VideoFrame(width, height, pixels));
}
public void EmitAudio(float[] samples) => AudioReady?.Invoke(new AudioSample(samples, 48000, 2));
public void End() => Completed?.Invoke();
}
}
+1
View File
@@ -68,6 +68,7 @@
<Resource Include="Assets\live-background.jpg" /> <Resource Include="Assets\live-background.jpg" />
<Resource Include="Assets\ending-background.jpg" /> <Resource Include="Assets\ending-background.jpg" />
<Resource Include="Assets\chat-background.jpg" /> <Resource Include="Assets\chat-background.jpg" />
<Resource Include="Assets\alert-default.mp4" />
</ItemGroup> </ItemGroup>
<ItemGroup> <ItemGroup>