feat(alerts): TASK 47 — alert box plays a video (built-in/custom clip) + read-time fade + message ticker

TASK 43's alert box grows a real video celebration. Per-alert IAlertClipDecoder
(ffmpeg bgra + f32le pipes, real-time paced, disposed at drain) plays the shipped
Assets/alert-default.mp4 (stamped into the Asset table at startup) unless the
creator picks their own file — path reference only, never stored in the DB; the
six AlertRenderer animations stay the fallback. ~0.3s fade rides the alpha
envelope on straight-source copies (EOF freeze-frames then fades out); audio
forwards to a new AudioMixer alert ring (8s, 48k stereo) drained at unity — no
duck, creator ruling — scaled by volume × fade. An auto-composed marquee ticker
('Funder — Super Chat · $10.00', 140px/s) scrolls top-of-frame via a
FramePump._alertTicker seam through Render/CompositeLayers, mixed into the cache
signature (dynamic overlay, never baked). New Stream Alerts section in LeftPanel.

Derivative-work references (how OBS/Streamlabs alert boxes do per-alert video):
- https://support.streamlabs.com/hc/en-us/articles/217741147-Setting-Up-Your-Streamlabs-Alerts (custom image/video per alert type + variations)
- https://obsproject.com/kb/stream-tutorial-2-alerts (alert overlay as an on-screen zone)
- https://streamlabs.com/content-hub/widgets/alert-box (per-event alert playback)

Good Dog: AlertLayerVideoTests drives a fake IAlertClipDecoder through the whole
lifecycle in one pass (custom path wins, decoder spawns/disposes, fade envelope
0→127→255, audio volume×fade, ticker scrolls, EOF fade-drain to idle). It caught
the clip branch of Advance not clearing _current before AdvanceToNext — the layer
stayed IsPlaying after drain (MyMistakes post-mortem).

Full vstest 319/319; clean build 0 warnings; scope check green.
This commit is contained in:
2026-09-26 11:15:17 -07:00
parent 2c9af9426b
commit a11b15e444
23 changed files with 1399 additions and 87 deletions
+208
View File
@@ -0,0 +1,208 @@
using System.Diagnostics;
using ytLive.Services.Audio;
using ytLive.Services.Encoder;
namespace ytLive.Services;
/// <summary>
/// TASK 47: one alert clip's playback — video frames + on-stream audio from the
/// same file, both paced to real time. The alert box owns a decoder per event
/// (start at play, dispose at drain) — unlike MediaSource's app-wide refcounted
/// session, an alert REPLAYS the same clip each event, so a fresh decode per
/// play is required. Video is rawvideo BGRA scaled to the box rect (the
/// <see cref="MediaVideoSource"/> shape); audio is ffmpeg <c>f32le</c> at the
/// mixer's native 48 kHz stereo (so the mixer ring needs no resampler). The
/// fade is applied by the alert layer, not here.
/// </summary>
public interface IAlertClipDecoder : IDisposable
{
/// <summary>Starts the decode loop (video + audio); raises the events off the
/// calling thread. Paced to real time so playback matches wall clock.</summary>
void Start();
/// <summary>Stops and kills any running decode subprocesses.</summary>
void Stop();
/// <summary>One decoded video frame (box-sized, straight-alpha BGRA).</summary>
event Action<VideoFrame>? FrameAvailable;
/// <summary>One chunk of on-stream audio (48 kHz stereo float).</summary>
event Action<AudioSample>? AudioReady;
/// <summary>Raised once when both streams have fully played out.</summary>
event Action? Completed;
}
/// <summary>Default <see cref="IAlertClipDecoder"/>: two ffmpeg subprocesses on
/// the same file. Video via <see cref="RawVideoFrameReader"/> on a <c>rawvideo
/// bgra</c> pipe (-an), audio via an <c>f32le -ac 2 -ar 48000</c> pipe (-vn).
/// The seams (locator, process factory, frame-rate probe, delay) let tests drive
/// a fake decoder without a real codec or Windows.</summary>
public sealed class AlertClipDecoder : IAlertClipDecoder
{
public event Action<VideoFrame>? FrameAvailable;
public event Action<AudioSample>? AudioReady;
public event Action? Completed;
private const int AudioSampleRate = 48000;
private const int AudioChannels = 2;
private const int AudioChunkFloats = 4800; // 50ms of stereo
private readonly string _path;
private readonly int _width;
private readonly int _height;
private readonly IFfmpegLocator _locator;
private readonly Func<IDecodeProcess> _processFactory;
private readonly IFrameRateProbe? _probe;
private readonly Func<TimeSpan, CancellationToken, Task> _delay;
private readonly CancellationTokenSource _cts = new();
private Task? _run;
public AlertClipDecoder(
string path,
int width,
int height,
IFfmpegLocator locator,
Func<IDecodeProcess> processFactory,
IFrameRateProbe? frameRateProbe = null,
Func<TimeSpan, CancellationToken, Task>? delay = null)
{
_path = path;
_width = width;
_height = height;
_locator = locator;
_processFactory = processFactory;
_probe = frameRateProbe;
_delay = delay ?? ((t, ct) => Task.Delay(t, ct));
}
public void Start()
{
if (_run != null) return;
_run = Task.Run(RunAsync, CancellationToken.None);
}
public void Stop() => _cts.Cancel();
public void Dispose()
{
_cts.Cancel();
_cts.Dispose();
}
private async Task RunAsync()
{
var ct = _cts.Token;
try
{
double? frameRate = null;
if (_probe != null)
frameRate = await _probe.ProbeAsync(_path, ct).ConfigureAwait(false);
var frameDuration = frameRate is > 0 ? TimeSpan.FromSeconds(1.0 / frameRate.Value) : TimeSpan.Zero;
var ffmpegPath = await _locator.LocateAsync(ct).ConfigureAwait(false);
var videoTask = RunVideoAsync(ffmpegPath, frameDuration, ct);
var audioTask = RunAudioAsync(ffmpegPath, ct);
await Task.WhenAll(videoTask, audioTask).ConfigureAwait(false);
}
catch (OperationCanceledException)
{
return;
}
catch
{
// A failed decode is not fatal: the alert layer falls back to the six
// animations (or the frame it already holds) exactly as a missing file.
}
if (!ct.IsCancellationRequested)
Completed?.Invoke();
}
private async Task RunVideoAsync(string ffmpegPath, TimeSpan frameDuration, CancellationToken ct)
{
var args =
$"-hide_banner -loglevel error -i \"{_path}\" " +
$"-f rawvideo -pix_fmt bgra -vf scale={_width}:{_height} -an -";
using var process = _processFactory();
process.Start(new ProcessStartInfo
{
FileName = ffmpegPath,
Arguments = args,
UseShellExecute = false,
RedirectStandardOutput = true,
RedirectStandardError = true,
CreateNoWindow = true,
});
var reader = new RawVideoFrameReader(_width, _height);
var buffer = new byte[65536];
var stream = process.StandardOutput;
while (!ct.IsCancellationRequested && !process.HasExited)
{
var read = await stream.ReadAsync(buffer, 0, buffer.Length, ct).ConfigureAwait(false);
if (read <= 0) break;
var chunk = new byte[read];
Buffer.BlockCopy(buffer, 0, chunk, 0, read);
foreach (var frame in reader.Feed(chunk))
{
FrameAvailable?.Invoke(frame);
if (frameDuration > TimeSpan.Zero)
await _delay(frameDuration, ct).ConfigureAwait(false);
}
}
}
private async Task RunAudioAsync(string ffmpegPath, CancellationToken ct)
{
var args =
$"-hide_banner -loglevel error -i \"{_path}\" " +
$"-f f32le -ac {AudioChannels} -ar {AudioSampleRate} -vn -";
using var process = _processFactory();
process.Start(new ProcessStartInfo
{
FileName = ffmpegPath,
Arguments = args,
UseShellExecute = false,
RedirectStandardOutput = true,
RedirectStandardError = true,
CreateNoWindow = true,
});
var stream = process.StandardOutput;
var raw = new byte[65536];
var carry = new byte[0];
var floatScratch = new float[AudioChunkFloats];
var floatIndex = 0;
// The pipe is byte-delimited only: a read may split a float, so leftover
// bytes are carried into the next read instead of dropped (a dropped
// half-float would desync the entire stream).
while (!ct.IsCancellationRequested && !process.HasExited)
{
var read = await stream.ReadAsync(raw, 0, raw.Length, ct).ConfigureAwait(false);
if (read <= 0) break;
var total = carry.Length + read;
var bytes = total <= raw.Length ? raw : new byte[total];
if (carry.Length > 0)
{
Buffer.BlockCopy(carry, 0, bytes, 0, carry.Length);
Buffer.BlockCopy(raw, 0, bytes, carry.Length, read);
}
var floatsTo = total - (total % 4);
for (var i = 0; i < floatsTo && floatIndex < floatScratch.Length; i += 4)
floatScratch[floatIndex++] = BitConverter.ToSingle(bytes, i);
carry = total > floatsTo ? bytes[floatsTo..total] : new byte[0];
while (floatIndex == floatScratch.Length)
{
AudioReady?.Invoke(new AudioSample(floatScratch, AudioSampleRate, AudioChannels));
// Pace the NEXT chunk so the whole clip's audio lands at real-time
// cadence (the fade in the alert layer tracks the same wall clock).
await _delay(TimeSpan.FromSeconds(AudioChunkFloats / (double)(AudioSampleRate * AudioChannels)), ct).ConfigureAwait(false);
floatScratch = new float[AudioChunkFloats];
floatIndex = 0;
}
}
}
}
+223 -20
View File
@@ -5,28 +5,59 @@ using System.Windows.Media;
using System.Windows.Media.Imaging;
using System.Windows.Threading;
using ytLive.Models;
using ytLive.Services.Audio;
using ytLive.Services.Compositor;
namespace ytLive.Services;
/// <summary>
/// The native alert box (OBS-style celebration zone, no third party). Owns the
/// event queue and the animation clock: <see cref="ChatEventKind"/> events from
/// the chat feed are enqueued one-at-a-time and played through their own unique
/// animation (<see cref="AlertRenderer"/>), frames served to the stream compositor
/// via <see cref="RenderFrame"/> and mirrored to the preview via
/// <see cref="UpdatePreview"/>. Idle = transparent (null frame); playback is
/// the chat feed are enqueued one-at-a-time and played through a celebration
/// animation (<see cref="AlertRenderer"/>) OR — when the alert-box video is
/// configured and resolves — through a decoded clip. Frames are served to the
/// stream compositor via <see cref="RenderFrame"/> and mirrored to the preview
/// via <see cref="UpdatePreview"/>. Idle = transparent (null frame); playback is
/// advanced by a 33ms dispatch ticker. The clock is exposed as
/// <see cref="Advance"/> so tests drive playback deterministically without the
/// timer. Ordinary chat rows (<see cref="ChatEventKind.None"/>) never enqueue —
/// a subscriber mention is a chat row, not an alert.
///
/// TASK 47 (Stream Alerts video): a video-backed alert replaces the animation
/// while a clip decodes (<see cref="IAlertClipDecoder"/>, one per play). The
/// clip plays at its own media pace; the layer applies a ~300ms fade-in/out
/// alpha envelope to the frames AND the audio samples (freeze-frame fade-out on
/// the tail), clips the alert to the clip length, and forwards the scaled audio
/// to the mixer's alert ring via <paramref name="audioSink"/> (no duck). The
/// per-source config (path/default/ticker/volume) is resolved through
/// <paramref name="clipPathResolver"/>; the ticker strip is served via
/// <see cref="AlertTickerFrame"/>.
/// </summary>
public sealed class AlertOverlayLayer : IDisposable
{
/// <summary>The fade-in/out ramp length (video alpha AND audio gain). Measured
/// against a real clip in the AlertLayerVideoTests unit; keep the ramp short
/// enough to read as a "pop-in" on stream (see ai.md).</summary>
internal const double FadeDurationSeconds = 0.30;
private static readonly TimeSpan TickInterval = TimeSpan.FromMilliseconds(33);
private readonly AlertRenderer _renderer;
private readonly Func<string, int, int, IAlertClipDecoder>? _clipDecoderFactory;
private readonly Func<Source, string?>? _clipPathResolver;
private readonly Action<AudioSample>? _audioSink;
private readonly Queue<ChatMessage> _queue = new();
private readonly DispatcherTimer? _ticker;
private ChatMessage? _current;
private Source? _currentBox;
private IAlertClipDecoder? _clip;
private VideoFrame? _latestClipFrame;
private bool _clipEof;
private double _clipEofElapsed;
private int _clipWidth;
private int _clipHeight;
private string? _tickerText;
private double _elapsed;
private IEnumerable<Scene>? _scenes;
@@ -35,11 +66,16 @@ public sealed class AlertOverlayLayer : IDisposable
private string? _frameKey;
private VideoFrame? _cachedFrame;
private static readonly TimeSpan TickInterval = TimeSpan.FromMilliseconds(33);
public AlertOverlayLayer(AlertRenderer renderer)
public AlertOverlayLayer(
AlertRenderer renderer,
Func<string, int, int, IAlertClipDecoder>? clipDecoderFactory = null,
Func<Source, string?>? clipPathResolver = null,
Action<AudioSample>? audioSink = null)
{
_renderer = renderer;
_clipDecoderFactory = clipDecoderFactory;
_clipPathResolver = clipPathResolver;
_audioSink = audioSink;
if (Application.Current != null)
{
_ticker = new DispatcherTimer { Interval = TickInterval };
@@ -73,6 +109,7 @@ public sealed class AlertOverlayLayer : IDisposable
{
_current = _queue.Dequeue();
_elapsed = 0;
BeginClip();
}
_contentVersion++;
}
@@ -82,11 +119,30 @@ public sealed class AlertOverlayLayer : IDisposable
/// nothing stops the ticker and leaves the box transparent.</summary>
public void Advance(double deltaSeconds)
{
// Video-backed alert: duration is clip-driven. The clip plays at media
// pace; once it EOFs the last frame sweeps through the fade-out before
// the alert moves on (a hard cut at EOF would pop, not fade).
if (_clip != null)
{
_elapsed += deltaSeconds;
if (_clipEof && _elapsed - _clipEofElapsed >= FadeDurationSeconds)
{
_clipEof = false;
StopClip();
_current = null;
_elapsed = 0;
AdvanceToNext();
}
_contentVersion++;
return;
}
if (_current == null)
{
if (_queue.Count == 0) return;
_current = _queue.Dequeue();
_elapsed = 0;
BeginClip();
_contentVersion++;
return;
}
@@ -96,15 +152,7 @@ public sealed class AlertOverlayLayer : IDisposable
{
_current = null;
_elapsed = 0;
if (_queue.Count > 0)
{
_current = _queue.Dequeue();
_elapsed = 0;
}
else
{
_ticker?.Stop();
}
AdvanceToNext();
}
_contentVersion++;
}
@@ -112,10 +160,9 @@ public sealed class AlertOverlayLayer : IDisposable
/// <summary>Whether an alert is currently playing or queued.</summary>
public bool IsPlaying => _current != null || _queue.Count > 0;
/// <summary>The live-output frame for one AlertBox element: the current
/// animation frame while playing, null (transparent) when idle. Mirrors the
/// chat cache pattern — the raster is WPF/UI-thread, so only the cache miss
/// marshals to the dispatcher and re-validates there.</summary>
/// <summary>The live-output frame for one AlertBox element: the decoded clip
/// frame (faded) while a clip is active, else the animation frame, null
/// (transparent) when idle.</summary>
public VideoFrame? RenderFrame(Source alertBox)
{
if (_current == null)
@@ -124,6 +171,9 @@ public sealed class AlertOverlayLayer : IDisposable
return null;
}
if (_clip != null)
return RenderClipFrame(alertBox);
var width = (int)Math.Max(1, alertBox.Width);
var height = (int)Math.Max(1, alertBox.Height);
var key = $"{width}x{height}|{_current.Id}|{Progress()}";
@@ -147,6 +197,139 @@ public sealed class AlertOverlayLayer : IDisposable
return RenderCore(alertBox, width, height, key, _contentVersion);
}
/// <summary>The global ticker strip for the stream output (top edge, full
/// master width): a marquee of "viewer — event" while an alert is playing
/// and the config has the ticker on, else null. Pure byte-math per call so
/// the frame pump can read it on any thread (the text strip is rasterized
/// once on the UI thread and cached).</summary>
public VideoFrame? AlertTickerFrame
{
get
{
if (_current == null || string.IsNullOrWhiteSpace(_tickerText)) return null;
return AlertTickerRenderer.Render(_tickerText!, _elapsed);
}
}
private VideoFrame? RenderClipFrame(Source alertBox)
{
var frame = _latestClipFrame;
if (frame == null) return null;
var w = (int)Math.Max(1, alertBox.Width);
var h = (int)Math.Max(1, alertBox.Height);
if (frame.Width != w || frame.Height != h)
frame = StretchMath.BilinearScale(frame, w, h);
var fade = FadeAt();
if (fade >= 1.0) return frame;
// Straight-alpha source-over on the compositor side: scale the source
// alpha by the envelope so the clip blends out (a 680x200 copy per fade
// tick is trivial; outside the ramps fade==1 returns the raw frame).
var copy = new byte[frame.BgraPixels.Length];
Buffer.BlockCopy(frame.BgraPixels, 0, copy, 0, copy.Length);
var a = (byte)(fade * 255 + 0.5);
for (var i = 3; i < copy.Length; i += 4)
copy[i] = (byte)((copy[i] * a) >> 8);
return new VideoFrame(frame.Width, frame.Height, copy);
}
private double FadeAt()
{
if (_clip == null) return 1.0;
if (_clipEof)
return Math.Clamp(1.0 - (_elapsed - _clipEofElapsed) / FadeDurationSeconds, 0.0, 1.0);
return Math.Clamp(_elapsed / FadeDurationSeconds, 0.0, 1.0);
}
private void BeginClip()
{
StopClip();
_tickerText = null;
if (_current == null) return;
var box = _scenes?.SelectMany(s => s.Elements).OfType<Source>()
.FirstOrDefault(s => s.Type == SourceType.AlertBox);
_currentBox = box;
if (box != null && box.AlertShowTicker)
_tickerText = ComposeTicker(_current);
if (box == null || _clipDecoderFactory == null) return;
var path = _clipPathResolver?.Invoke(box);
if (string.IsNullOrWhiteSpace(path)) return;
_clipWidth = (int)Math.Max(1, box.Width);
_clipHeight = (int)Math.Max(1, box.Height);
try
{
_clip = _clipDecoderFactory(path, _clipWidth, _clipHeight);
_clip.FrameAvailable += OnClipFrame;
_clip.AudioReady += OnClipAudio;
_clip.Completed += OnClipCompleted;
_clip.Start();
}
catch
{
StopClip(); // decode setup failed (bad path) — fall back to animations
}
}
private void StopClip()
{
var clip = _clip;
_clip = null;
_latestClipFrame = null;
_clipEof = false;
if (clip == null) return;
clip.FrameAvailable -= OnClipFrame;
clip.AudioReady -= OnClipAudio;
clip.Completed -= OnClipCompleted;
try { clip.Stop(); } catch { /* already stopped */ }
clip.Dispose();
}
private void OnClipFrame(VideoFrame frame) => _latestClipFrame = frame;
private void OnClipCompleted()
{
_clipEof = true;
_clipEofElapsed = _elapsed;
}
private void OnClipAudio(AudioSample sample)
{
if (_audioSink == null) return;
var volume = (float)Math.Clamp(_currentBox?.AlertVideoVolume ?? 1.0, 0.0, 1.0);
if (volume <= 0f) return;
var fade = FadeAt();
if (fade <= 0.0) return;
var data = sample.Samples;
if (fade < 1.0 || Math.Abs(volume - 1f) > 0.001f)
{
data = new float[data.Length];
var gain = volume * (float)fade;
for (var i = 0; i < sample.Samples.Length; i++)
data[i] = sample.Samples[i] * gain;
}
_audioSink(new AudioSample(data, sample.SampleRate, sample.Channels));
}
private void AdvanceToNext()
{
if (_queue.Count > 0)
{
_current = _queue.Dequeue();
_elapsed = 0;
BeginClip();
}
else
{
_ticker?.Stop();
}
}
private VideoFrame? RenderCore(Source alertBox, int width, int height, string key, int version)
{
_frameKey = key;
@@ -172,6 +355,25 @@ public sealed class AlertOverlayLayer : IDisposable
_ => 4.0,
};
private static string ComposeTicker(ChatMessage m)
{
var label = m.Kind switch
{
ChatEventKind.SuperChat => "Super Chat",
ChatEventKind.SuperSticker => "Super Sticker",
ChatEventKind.NewMember => "New Member",
ChatEventKind.MemberGift => "Membership Gift",
ChatEventKind.GiftReceived => "Gifted Membership",
ChatEventKind.MemberMilestone => "Member Milestone",
_ => "Stream Alert",
};
var amount = m.Kind is ChatEventKind.SuperChat or ChatEventKind.SuperSticker
&& m.SuperChatDisplayString is { Length: > 0 } s
? $" · {s}"
: "";
return $"{m.AuthorName} — {label}{amount}";
}
/// <summary>Render the current state onto one AlertBox preview (mirrors chat):
/// the playing frame, or null when idle so the source shows nothing until an
/// event actually fires.</summary>
@@ -227,5 +429,6 @@ public sealed class AlertOverlayLayer : IDisposable
_ticker?.Stop();
_queue.Clear();
_current = null;
StopClip();
}
}
+49 -3
View File
@@ -14,7 +14,9 @@ namespace ytLive.Services.Audio;
/// queues, applies the honest gains (mic = MicVolume, loopback = unity
/// × auto-duck when the mic is hot — the slider drives system volume via mixes them to stereo float, runs the
/// master limiter (a -1 dBFS ceiling so the sum never clips the encoder), and
/// writes the chunk to the encoder's audio pipe.
/// writes the chunk to the encoder's audio pipe. Since TASK 47 the alert layer's
/// clip audio joins the same mix (pre-scaled, unity-added, never ducked) through
/// <see cref="EnqueueAlertAudio"/>.
/// </summary>
public sealed class AudioMixer : IDisposable
{
@@ -39,6 +41,7 @@ public sealed class AudioMixer : IDisposable
// thread-safe queues → the ducker + gain/mix loop → the encoder's pipe.
private readonly AudioRingBuffer _micBuffer;
private readonly AudioRingBuffer _loopbackBuffer;
private readonly AudioRingBuffer _alertBuffer;
private readonly VoiceFilterChain _voiceChain;
private readonly AutoDucker _ducker;
private readonly MasterLimiter _masterLimiter = new();
@@ -48,6 +51,7 @@ public sealed class AudioMixer : IDisposable
private NamedPipeAudioWriter _pipe = new();
private float[]? _micChunk;
private float[]? _loopbackChunk;
private float[]? _alertChunk;
private float[]? _mixBuffer;
private float[]? _delayedMix;
private DateTime _lastLoopErrorLogged = DateTime.MinValue;
@@ -80,10 +84,12 @@ public sealed class AudioMixer : IDisposable
var framesPerTick = Math.Max(1, (int)Math.Round(OutputSampleRate * seconds));
_micBuffer = new AudioRingBuffer(OutputSampleRate * 2);
_loopbackBuffer = new AudioRingBuffer(OutputSampleRate * 2 * 2);
_alertBuffer = new AudioRingBuffer(OutputSampleRate * 2 * 8);
_voiceChain = new VoiceFilterChain(OutputSampleRate);
_ducker = new AutoDucker();
_micChunk = new float[framesPerTick];
_loopbackChunk = new float[framesPerTick * 2];
_alertChunk = new float[framesPerTick * 2];
_mixBuffer = new float[framesPerTick * 2];
_delayedMix = new float[framesPerTick * 2];
@@ -168,6 +174,7 @@ public sealed class AudioMixer : IDisposable
// Clear under the ring's lock, so only the in-flight chunk is dropped.
_micBuffer.Clear();
_loopbackBuffer.Clear();
_alertBuffer.Clear();
// Negative sync offset (audio runs BEHIND video): OBS's fix is to "eat
// the head of the buffer" — discard the first |N| ms of the live stream
@@ -246,6 +253,34 @@ public sealed class AudioMixer : IDisposable
LoopbackLevelChanged?.Invoke(level);
}
/// <summary>Drops the alert clip's audio into its own ring (TASK 47 — the
/// stream-alert video's soundtrack, pre-scaled by the alert layer to
/// volume × fade). Mixed at unity into the live output — NO auto-duck: an
/// alert must be heard over the mic, so the Creator's per-alert volume
/// slider is the honest gain and nothing else is stacked on top. Monno and
/// non-48k sources are normalized (the decoder already emits 48k stereo).</summary>
public void EnqueueAlertAudio(AudioSample sample)
{
var data = sample.Samples;
if (data.Length == 0) return;
if (sample.SampleRate != OutputSampleRate)
data = ResampleLoopback(data, sample.SampleRate);
if (sample.Channels == 1)
data = UpmixToStereo(data);
_alertBuffer.Write(data);
}
private static float[] UpmixToStereo(float[] mono)
{
var stereo = new float[mono.Length * 2];
for (var i = 0; i < mono.Length; i++)
{
stereo[i * 2] = mono[i];
stereo[i * 2 + 1] = mono[i];
}
return stereo;
}
private void OnMicFailed(Exception ex)
{
_log?.Invoke($"Mic capture failed: {ex.Message}");
@@ -386,6 +421,8 @@ public sealed class AudioMixer : IDisposable
{
var micCount = _micBuffer.Read(micChunk, micChunk.Length);
var loopCount = _loopbackBuffer.Read(loopbackChunk, loopbackChunk.Length);
var alertChunk = _alertChunk!;
var alertCount = _alertBuffer.Read(alertChunk, alertChunk.Length);
float micRms = 0;
if (micCount == 0)
@@ -411,6 +448,15 @@ public sealed class AudioMixer : IDisposable
Array.Clear(loopbackChunk, loopCount, loopbackChunk.Length - loopCount);
}
if (alertCount == 0)
{
Array.Clear(alertChunk, 0, alertChunk.Length);
}
else if (alertCount < alertChunk.Length)
{
Array.Clear(alertChunk, alertCount, alertChunk.Length - alertCount);
}
var duck = _ducker.Update(micRms);
var micGain = (float)(_micGain?.Invoke() ?? 1.0);
var loopGain = (float)(_loopbackGain?.Invoke() ?? 1.0) * duck;
@@ -419,8 +465,8 @@ public sealed class AudioMixer : IDisposable
for (var i = 0; i < frames; i++)
{
var m = micChunk[i] * micGain;
mix[i * 2] = m + loopbackChunk[i * 2] * loopGain;
mix[i * 2 + 1] = m + loopbackChunk[i * 2 + 1] * loopGain;
mix[i * 2] = m + loopbackChunk[i * 2] * loopGain + alertChunk[i * 2];
mix[i * 2 + 1] = m + loopbackChunk[i * 2 + 1] * loopGain + alertChunk[i * 2 + 1];
}
_masterLimiter.Process(mix);
+139
View File
@@ -0,0 +1,139 @@
using System.Collections.Generic;
using System.Globalization;
using System.Windows;
using System.Windows.Media;
using System.Windows.Media.Imaging;
using System.Windows.Threading;
namespace ytLive.Services.Compositor;
/// <summary>
/// Rasterizes the stream-alert ticker into a transparent BGRA strip that the
/// output compositor overlays along the very top edge of the master frame
/// (TASK 47). The strip is text ("Viewer — Super Chat · $12.34") inside a dark
/// pill with the LlamaCasty accent bar, rendered ONCE per unique text on the UI
/// thread (WPF <c>RenderTargetBitmap</c>, Pbgra32→straight-alpha like
/// <see cref="SocialBarRenderer"/>), then cached. Each <see cref="Render"/> call
/// is pure byte-math on that cached raster producing a scrolling marquee frame,
/// so the frame pump can call it from any thread for every output frame at
/// 30fps without a Dispatcher round-trip.
/// </summary>
public static class AlertTickerRenderer
{
/// <summary>Master-frame width — the ticker spans the full 1920px output.</summary>
public const int Width = 1920;
/// <summary>Strip height (pill + vertical padding).</summary>
public const int Height = 48;
private const int PixelsPerSecond = 140;
private const int RepeatGap = 120;
private const int CacheLimit = 48;
private static readonly object Gate = new();
private static readonly Dictionary<string, VideoFrame> StripCache = new();
/// <summary>A marquee frame of <paramref name="text"/> scrolled
/// <paramref name="elapsedSeconds"/> into its loop, or null for empty text.
/// Thread-safe: reads the cached raster and does row copies only.</summary>
public static VideoFrame? Render(string? text, double elapsedSeconds)
{
if (string.IsNullOrWhiteSpace(text)) return null;
var strip = GetStrip(text!);
if (strip == null) return null;
var sw = strip.Width;
var cycle = Width + sw + RepeatGap;
var pos = (int)(elapsedSeconds * PixelsPerSecond) % cycle;
var x = Width - pos;
var pixels = new byte[Width * Height * 4];
CopyStrip(pixels, strip, x);
if (x < 0)
CopyStrip(pixels, strip, x + Width + RepeatGap); // wrapped tail enters from the right
return new VideoFrame(Width, Height, pixels);
}
private static void CopyStrip(byte[] dst, VideoFrame strip, int dstX)
{
if (dstX >= Width || dstX + strip.Width <= 0) return;
var x0 = Math.Max(0, dstX);
var x1 = Math.Min(Width, dstX + strip.Width);
var sx0 = x0 - dstX;
var copyBytes = (x1 - x0) * 4;
var sw = strip.Width;
for (var y = 0; y < strip.Height; y++)
Buffer.BlockCopy(strip.BgraPixels, (y * sw + sx0) * 4, dst, (y * Width + x0) * 4, copyBytes);
}
private static VideoFrame? GetStrip(string text)
{
lock (Gate)
{
if (StripCache.TryGetValue(text, out var cached)) return cached;
}
var strip = RenderStrip(text);
if (strip == null) return null;
lock (Gate)
{
if (StripCache.Count >= CacheLimit) StripCache.Clear();
if (!StripCache.TryGetValue(text, out var existing))
StripCache[text] = strip;
return StripCache[text];
}
}
private static VideoFrame? RenderStrip(string text)
{
// WPF rendering is single-threaded; the pump thread resolves to the UI
// thread at most once per unique text (then hits the cache).
var app = Application.Current;
if (app != null && !app.Dispatcher.CheckAccess())
{
VideoFrame? result = null;
app.Dispatcher.Invoke(() => result = RenderStrip(text));
return result;
}
const double pillHeight = 34;
const double accentW = 6;
const double padX = 16;
const double gap = 8;
var font = new Typeface(new FontFamily("Segoe UI"), FontStyles.Normal, FontWeights.SemiBold, FontStretches.Normal);
var ft = new FormattedText(text, CultureInfo.InvariantCulture, FlowDirection.LeftToRight,
font, 22, Brushes.White, 1.0);
var pillW = (int)Math.Ceiling(padX + accentW + gap + ft.Width + padX);
var top = (Height - pillHeight) / 2.0;
var textTop = top + (pillHeight - ft.Height) / 2.0;
var visual = new DrawingVisual();
using (var dc = visual.RenderOpen())
{
var pill = new Rect(0, top, pillW, pillHeight);
dc.DrawRoundedRectangle(new SolidColorBrush(Color.FromArgb(0xE6, 0x10, 0x13, 0x24)), null, pill, pillHeight / 2, pillHeight / 2);
dc.DrawRoundedRectangle(new SolidColorBrush(Color.FromRgb(0xe9, 0x45, 0x60)), null,
new Rect(padX, top + (pillHeight - 12) / 2.0, accentW, 12), 2, 2);
dc.DrawText(ft, new Point(padX + accentW + gap, textTop));
}
var bitmap = new RenderTargetBitmap(pillW, Height, 96, 96, PixelFormats.Pbgra32);
bitmap.Render(visual);
var pixels = new byte[pillW * Height * 4];
bitmap.CopyPixels(pixels, pillW * 4, 0);
// Pbgra32 is premultiplied — unpremultiply so the compositor's
// straight-alpha source-over doesn't halo the text edges.
for (var i = 0; i < pixels.Length; i += 4)
{
var a = pixels[i + 3];
if (a == 0 || a == 255) continue;
pixels[i] = (byte)(pixels[i] * 255 / a);
pixels[i + 1] = (byte)(pixels[i + 1] * 255 / a);
pixels[i + 2] = (byte)(pixels[i + 2] * 255 / a);
}
return new VideoFrame(pillW, Height, pixels);
}
}
+17 -4
View File
@@ -37,7 +37,8 @@ public sealed class SceneCompositor
/// front): live background (the scene's <c>IsBackground</c> source) → background image →
/// visible elements (z-order = <c>Elements</c> order, mirroring the XAML DataTemplate) →
/// branding flash → social bar (a global overlay; spans the master width at
/// <paramref name="socialBarTop"/>). Transparent regions read opaque black.
/// <paramref name="socialBarTop"/>) → stream-alert ticker (a global overlay pinned to
/// the very top, TASK 47). Transparent regions read opaque black.
/// </summary>
public VideoFrame Render(
Scene scene,
@@ -48,7 +49,8 @@ public sealed class SceneCompositor
int socialBarTop = 0,
VideoFrame? staticBase = null,
int split = -1,
byte[]? scratch = null)
byte[]? scratch = null,
VideoFrame? tickerFrame = null)
{
if (scene == null) throw new ArgumentNullException(nameof(scene));
if (frameFor == null) throw new ArgumentNullException(nameof(frameFor));
@@ -137,6 +139,9 @@ public sealed class SceneCompositor
if (socialBarFrame != null && socialBarFrame.Width > 0 && socialBarFrame.Height > 0)
BlitOverlay(buffer, cropW, cropH, options, socialBarFrame, 0, socialBarTop);
if (tickerFrame != null && tickerFrame.Width > 0 && tickerFrame.Height > 0)
BlitOverlay(buffer, cropW, cropH, options, tickerFrame, 0, 0);
return StretchMath.BilinearScale(
new VideoFrame(cropW, cropH, buffer), options.OutputWidth, options.OutputHeight);
}
@@ -212,7 +217,10 @@ public sealed class SceneCompositor
/// <summary>The compositor's dynamic/above-split path: start from an existing frame
/// buffer and composite the z-layers from <paramref name="split"/> onward on top.
/// When <paramref name="baseFrame"/> is null the caller must fall back to full render.</summary>
/// When <paramref name="baseFrame"/> is null the caller must fall back to full render.
/// The social bar never reaches here (it IS baked into the static base) but the alert
/// ticker must: it is dynamic by construction — only ever present while an alert plays —
/// so it overlays LAST, pinned to the very top (TASK 47).</summary>
public static VideoFrame CompositeLayers(
VideoFrame baseFrame,
Scene scene,
@@ -222,7 +230,8 @@ public sealed class SceneCompositor
VideoFrame? socialBarFrame = null,
int socialBarTop = 0,
VideoFrame? flashFrame = null,
byte[]? scratch = null)
byte[]? scratch = null,
VideoFrame? tickerFrame = null)
{
var cropW = options.SourceRectWidth;
var cropH = options.SourceRectHeight;
@@ -247,6 +256,10 @@ public sealed class SceneCompositor
if (frame == null) continue;
BlitBakedElement(buffer, cropW, cropH, options, element, frame);
}
if (tickerFrame != null && tickerFrame.Width > 0 && tickerFrame.Height > 0)
BlitOverlay(buffer, cropW, cropH, options, tickerFrame, 0, 0);
return StretchMath.BilinearScale(new VideoFrame(cropW, cropH, buffer), options.OutputWidth, options.OutputHeight);
}
+19 -6
View File
@@ -29,6 +29,7 @@ public sealed class FramePump : IDisposable
private readonly Action<string>? _log;
private readonly Func<TimeSpan, CancellationToken, Task> _pacingDelay;
private readonly Func<(VideoFrame? Frame, SocialBarPosition Position)>? _socialBar;
private readonly Func<VideoFrame?>? _alertTicker;
private readonly TransitionService? _transition;
private readonly SceneGraph? _sceneGraph;
private readonly SceneCompositor _compositor = new();
@@ -129,6 +130,7 @@ public sealed class FramePump : IDisposable
Action<string>? log = null,
Func<TimeSpan, CancellationToken, Task>? pacingDelay = null,
Func<(VideoFrame? Frame, SocialBarPosition Position)>? socialBar = null,
Func<VideoFrame?>? alertTicker = null,
TransitionService? transition = null,
SceneGraph? sceneGraph = null)
{
@@ -140,6 +142,7 @@ public sealed class FramePump : IDisposable
_log = log;
_pacingDelay = pacingDelay ?? ((delay, ct) => Task.Delay(delay, ct));
_socialBar = socialBar;
_alertTicker = alertTicker;
_transition = transition;
_sceneGraph = sceneGraph;
}
@@ -405,6 +408,11 @@ public sealed class FramePump : IDisposable
: compositorOptions.SourceRectHeight - barFrame.Height;
}
// The alert ticker is a per-frame overlay (it scrolls while an alert
// plays), NEVER baked into the static base. Lawn-mower seam: read it
// each tick and let the compositor pin it to the very top.
var tickerFrame = _alertTicker?.Invoke();
VideoFrame frame;
var scratchSize = compositorOptions.SourceRectWidth
* compositorOptions.SourceRectHeight * 4;
@@ -415,7 +423,7 @@ public sealed class FramePump : IDisposable
// pooling change because its buffer's only consumer was its own release.
renderSw.Restart();
var scratch = AcquireScratch(scratchSize);
frame = RenderScene(scene, compositorOptions, socialBarFrame, socialBarTop, scratch, TimedResolver);
frame = RenderScene(scene, compositorOptions, socialBarFrame, socialBarTop, tickerFrame, scratch, TimedResolver);
if (_transition is { Active: true } transition)
{
frame = transition.BlendFrame(frame);
@@ -555,6 +563,7 @@ public sealed class FramePump : IDisposable
CompositorOptions options,
VideoFrame? socialBarFrame,
int socialBarTop,
VideoFrame? tickerFrame,
byte[]? scratch = null,
Func<SceneElement, VideoFrame?>? resolver = null)
{
@@ -562,7 +571,7 @@ public sealed class FramePump : IDisposable
if (_sceneGraph == null)
{
var sw = System.Diagnostics.Stopwatch.StartNew();
var frame = RenderFull(scene, options, socialBarFrame, socialBarTop, scratch, resolver);
var frame = RenderFull(scene, options, socialBarFrame, socialBarTop, tickerFrame, scratch, resolver);
ProbeRender("full-no-graph", scene, -1, sw.ElapsedMilliseconds, 0);
return frame;
}
@@ -586,14 +595,14 @@ public sealed class FramePump : IDisposable
if (baseFrame != null)
{
var frame = SceneCompositor.CompositeLayers(
baseFrame, scene, split, resolver, options, socialBarFrame, socialBarTop, scratch: scratch);
baseFrame, scene, split, resolver, options, socialBarFrame, socialBarTop, tickerFrame: tickerFrame, scratch: scratch);
ProbeRender("base+layers", scene, split, swBase.ElapsedMilliseconds, 0);
return frame;
}
// No static base (first layer is dynamic or empty scene) — full render.
var swFull = System.Diagnostics.Stopwatch.StartNew();
var frame2 = RenderFull(scene, options, socialBarFrame, socialBarTop, scratch, resolver);
var frame2 = RenderFull(scene, options, socialBarFrame, socialBarTop, tickerFrame, scratch, resolver);
ProbeRender("full-render", scene, split, swFull.ElapsedMilliseconds, 0);
return frame2;
}
@@ -605,6 +614,7 @@ public sealed class FramePump : IDisposable
/// hit costs ~3ms of pure copy and a miss costs exactly the old full render.</summary>
private VideoFrame RenderFull(
Scene scene, CompositorOptions options, VideoFrame? socialBarFrame, int socialBarTop,
VideoFrame? tickerFrame,
byte[]? scratch, Func<SceneElement, VideoFrame?> resolver)
{
// 1:1 guard: the cache stores the OUTPUT bytes and scratch is source-sized —
@@ -615,7 +625,7 @@ public sealed class FramePump : IDisposable
var useCache = scratch != null && options.SourceRectWidth == options.OutputWidth
&& options.SourceRectHeight == options.OutputHeight;
var signature = useCache
? BuildFullRenderSignature(scene, options, socialBarFrame, socialBarTop, resolver)
? BuildFullRenderSignature(scene, options, socialBarFrame, socialBarTop, tickerFrame, resolver)
: 0UL;
if (useCache && _fullCacheValid && signature == _lastFullSignature
@@ -626,7 +636,7 @@ public sealed class FramePump : IDisposable
return new VideoFrame(options.SourceRectWidth, options.SourceRectHeight, scratch);
}
var rendered = _compositor.Render(scene, resolver, null, options, socialBarFrame, socialBarTop, scratch: scratch);
var rendered = _compositor.Render(scene, resolver, null, options, socialBarFrame, socialBarTop, scratch: scratch, tickerFrame: tickerFrame);
CacheRenders++;
if (useCache)
{
@@ -659,6 +669,7 @@ public sealed class FramePump : IDisposable
/// never trusted to key recycled content.</summary>
private static ulong BuildFullRenderSignature(
Scene scene, CompositorOptions options, VideoFrame? socialBarFrame, int socialBarTop,
VideoFrame? tickerFrame,
Func<SceneElement, VideoFrame?> resolver)
{
var h = 14695981039346656037UL; // FNV-1a offset basis
@@ -671,6 +682,8 @@ public sealed class FramePump : IDisposable
h = Mix(h, (ulong)socialBarTop);
if (socialBarFrame != null) h = MixFrame(h, socialBarFrame);
else h = Mix(h, 0xFFFFFFFFFFFFFFFFUL);
if (tickerFrame != null) h = MixFrame(h, tickerFrame);
else h = Mix(h, 0xFFFFFFFFFFFFFFFFUL);
var backdropResolved = false;
foreach (var element in scene.Elements)
+8 -2
View File
@@ -99,7 +99,8 @@ public partial class LayoutStore : IDisposable
IsBackground, CaptureKey, ShowDesktop,
ChatFontSize, ChatFontColor, ChatBackgroundColor, ChatMaxMessages,
ChatShowTimestamps, ChatShowBadges, ChatMemberColor, ChatSuperChatColor,
WebUri, MediaPath, MediaIsLooping, MediaVolume, MediaPlaybackState, SortOrder
WebUri, MediaPath, MediaIsLooping, MediaVolume, MediaPlaybackState,
AlertVideoAssetId, AlertVideoPath, AlertUseDefaultVideo, AlertShowTicker, AlertVideoVolume, SortOrder
FROM Source ORDER BY SortOrder
""";
using var reader = cmd.ExecuteReader();
@@ -145,10 +146,15 @@ public partial class LayoutStore : IDisposable
: Enum.TryParse<MediaPlaybackState>(reader.GetString(29), out var ps)
? ps
: MediaPlaybackState.Stopped,
AlertVideoAssetId = reader.IsDBNull(30) ? null : reader.GetString(30),
AlertVideoPath = reader.IsDBNull(31) ? null : reader.GetString(31),
AlertUseDefaultVideo = reader.IsDBNull(32) ? true : reader.GetInt32(32) != 0,
AlertShowTicker = reader.IsDBNull(33) ? true : reader.GetInt32(33) != 0,
AlertVideoVolume = reader.IsDBNull(34) ? 1.0 : reader.GetDouble(34),
};
if (!sourcesByScene.TryGetValue(sceneId, out var list))
sourcesByScene[sceneId] = list = new List<(Source, int)>();
list.Add((source, reader.GetInt32(30)));
list.Add((source, reader.GetInt32(35)));
}
}
+34
View File
@@ -285,6 +285,40 @@ public partial class LayoutStore : IDisposable
cmd.ExecuteNonQuery();
}
// TASK 47: alert video config. The default clip asset id + the creator's
// replacement path (a FILE reference, never a stored BLOB) + the pill,
// ticker toggle and volume live on the row.
if (!columns.Contains("AlertVideoAssetId"))
{
using var cmd = _connection.CreateCommand();
cmd.CommandText = "ALTER TABLE Source ADD COLUMN AlertVideoAssetId TEXT;";
cmd.ExecuteNonQuery();
}
if (!columns.Contains("AlertVideoPath"))
{
using var cmd = _connection.CreateCommand();
cmd.CommandText = "ALTER TABLE Source ADD COLUMN AlertVideoPath TEXT;";
cmd.ExecuteNonQuery();
}
if (!columns.Contains("AlertUseDefaultVideo"))
{
using var cmd = _connection.CreateCommand();
cmd.CommandText = "ALTER TABLE Source ADD COLUMN AlertUseDefaultVideo INTEGER NOT NULL DEFAULT 1;";
cmd.ExecuteNonQuery();
}
if (!columns.Contains("AlertShowTicker"))
{
using var cmd = _connection.CreateCommand();
cmd.CommandText = "ALTER TABLE Source ADD COLUMN AlertShowTicker INTEGER NOT NULL DEFAULT 1;";
cmd.ExecuteNonQuery();
}
if (!columns.Contains("AlertVideoVolume"))
{
using var cmd = _connection.CreateCommand();
cmd.CommandText = "ALTER TABLE Source ADD COLUMN AlertVideoVolume REAL NOT NULL DEFAULT 1;";
cmd.ExecuteNonQuery();
}
// Rename legacy "Backdrop" display name to "Background" in existing rows
using (var cmd = _connection.CreateCommand())
{
+23 -3
View File
@@ -88,13 +88,15 @@ public partial class LayoutStore : IDisposable
ClipShape, IsMirrored, IsBackground, CaptureKey, ShowDesktop, SortOrder,
ChatFontSize, ChatFontColor, ChatBackgroundColor, ChatMaxMessages,
ChatShowTimestamps, ChatShowBadges, ChatMemberColor, ChatSuperChatColor,
WebUri, MediaPath, MediaIsLooping, MediaVolume, MediaPlaybackState)
WebUri, MediaPath, MediaIsLooping, MediaVolume, MediaPlaybackState,
AlertVideoAssetId, AlertVideoPath, AlertUseDefaultVideo, AlertShowTicker, AlertVideoVolume)
VALUES ($id, $sceneId, $assetId, $type, $name, $isEnabled,
$x, $y, $w, $h, $opacity, $monitor,
$clip, $mirrored, $isBackground, $captureKey, $showDesktop, $sort,
$chatFontSize, $chatFontColor, $chatBgColor, $chatMaxMsgs,
$chatTimestamps, $chatBadges, $chatMemberColor, $chatSuperChatColor,
$webUri, $mediaPath, $mediaIsLooping, $mediaVolume, $mediaPlaybackState)
$webUri, $mediaPath, $mediaIsLooping, $mediaVolume, $mediaPlaybackState,
$alertVideoAssetId, $alertVideoPath, $alertUseDefaultVideo, $alertShowTicker, $alertVideoVolume)
""";
cmd.Transaction = tx;
var idP = cmd.Parameters.Add("$id", SqliteType.Text);
@@ -128,6 +130,11 @@ public partial class LayoutStore : IDisposable
var mediaIsLoopingP = cmd.Parameters.Add("$mediaIsLooping", SqliteType.Integer);
var mediaVolumeP = cmd.Parameters.Add("$mediaVolume", SqliteType.Real);
var mediaPlaybackStateP = cmd.Parameters.Add("$mediaPlaybackState", SqliteType.Text);
var alertVideoAssetIdP = cmd.Parameters.Add("$alertVideoAssetId", SqliteType.Text);
var alertVideoPathP = cmd.Parameters.Add("$alertVideoPath", SqliteType.Text);
var alertUseDefaultVideoP = cmd.Parameters.Add("$alertUseDefaultVideo", SqliteType.Integer);
var alertShowTickerP = cmd.Parameters.Add("$alertShowTicker", SqliteType.Integer);
var alertVideoVolumeP = cmd.Parameters.Add("$alertVideoVolume", SqliteType.Real);
// Source and WebcamSceneConfig share ONE z-order space per scene:
// each element gets its index within scene.Elements regardless of
@@ -170,6 +177,11 @@ public partial class LayoutStore : IDisposable
mediaIsLoopingP.Value = source.MediaIsLooping ? 1 : 0;
mediaVolumeP.Value = source.MediaVolume;
mediaPlaybackStateP.Value = source.MediaPlaybackState.ToString();
alertVideoAssetIdP.Value = (object?)source.AlertVideoAssetId ?? DBNull.Value;
alertVideoPathP.Value = (object?)source.AlertVideoPath ?? DBNull.Value;
alertUseDefaultVideoP.Value = source.AlertUseDefaultVideo ? 1 : 0;
alertShowTickerP.Value = source.AlertShowTicker ? 1 : 0;
alertVideoVolumeP.Value = source.AlertVideoVolume;
cmd.ExecuteNonQuery();
}
}
@@ -300,7 +312,15 @@ public partial class LayoutStore : IDisposable
using (var cmd = _connection.CreateCommand())
{
cmd.CommandText = "DELETE FROM Asset WHERE Id NOT IN (SELECT AssetId FROM Source WHERE AssetId IS NOT NULL);";
cmd.CommandText = """
DELETE FROM Asset WHERE Id NOT IN (
SELECT AssetId FROM Source WHERE AssetId IS NOT NULL
UNION
SELECT AlertVideoAssetId FROM Source WHERE AlertVideoAssetId IS NOT NULL
UNION
SELECT Value FROM Settings WHERE Key = 'AlertDefaultVideoAssetId'
);
""";
cmd.Transaction = tx;
cmd.ExecuteNonQuery();
}
+22
View File
@@ -304,4 +304,26 @@ public partial class LayoutStore : IDisposable
insert.Parameters.AddWithValue("$value", value);
insert.ExecuteNonQuery();
}
internal const string AlertDefaultVideoKey = "AlertDefaultVideoAssetId";
/// <summary>Stamps the shipped stream-alert placeholder clip (TASK 47): like any
/// source image it lives as a BLOB in the internal Asset table, tracked by this
/// Setting so the Save pruner keeps it alive. The creator's OWN replacement
/// videos are never stored — they are plain path references (the 2026-09-26
/// decision: the file the user picks stays at its real location on disk).
/// <c>UpsertAsset</c> is hash-keyed, so re-stamping the same bytes returns the
/// same id.</summary>
public string EnsureAlertDefaultVideo(byte[] data)
{
var saved = GetSetting(AlertDefaultVideoKey);
if (!string.IsNullOrWhiteSpace(saved) && GetAssetBytes(saved) != null)
return saved;
var id = UpsertAsset(data, 0, 0);
UpsertSetting(AlertDefaultVideoKey, id);
return id;
}
/// <summary>The stored default clip's Asset id, or null when never stamped.</summary>
public string? GetAlertDefaultVideoId() => GetSetting(AlertDefaultVideoKey);
}
+8 -6
View File
@@ -11,9 +11,10 @@ External-facing logic: YouTube API, persistence. See
| `YouTubeChatService.cs` | Polls `liveChat/messages`, raises `MessageReceived`; `IDisposable`. **TASK 43 rewrite (2026-09-24):** re-arms its one-shot poll on the server's `pollingIntervalMillis` (streamList semantics — reconnect on that cadence with `nextPageToken`, clamp 1000–6000ms, `maxResults=2000`); `ParsePage` (internal static seam) decodes **all six** event types into `ChatMessage.Kind`/fields (superChat / superSticker / newSponsor / membershipGifting / giftMembershipReceived / memberMilestoneChat + milestone months best-effort regex); optional `HttpClient` ctor seam endures; `InjectSimulatedMessage` (TASK 41) preserved |
| `ChatBoxRenderer.cs` | Chat overlay rasterizer (WPF FormattedText → BGRA `VideoFrame`), live rows + Super Chat/member styling |
| `ChatOverlayLayer.cs` | **The chat component (Commit G):** owns the message buffer, `ChatBoxRenderer`, fade/mock timers, per-source preview renders, live `RenderFrame`; raster-on-change cache pattern (OBS text-source contract) |
| `AlertRenderer.cs` | **The alert rasterizer (TASK 43, 2026-09-24):** renders one event's celebration animation into a BGRA `VideoFrame` — SIX DISTINCT animations (SuperChat slide-up/shine/count-up, SuperSticker scale-pop, NewMember drop-in/flash-band, MemberGift slide-left/chip-fan, GiftReceived confetti scale-in, MemberMilestone rise/growth-bar), enter/hold/exit phases, brand line on every card |
| `AlertOverlayLayer.cs` | **The native alert-box component (TASK 43):** event queue (cap 10) + 33ms dispatch ticker + `RenderFrame` cache-first live path + `UpdatePreview`; **`Advance(double)` is the deterministic test clock** (tests never start the timer); `ChatEventKind.None` rows (sub mentions) never enqueue; idle = transparent (null frame) |
| `LayoutStore.cs` | SQLite persistence (`Microsoft.Data.Sqlite`) at `%APPDATA%\ytLlive\ytLlive.db`; assets stored as BLOBs keyed by SHA-256 content hash; save/open layout files; schema `user_version` 6 (`Source.ClipShape`/`IsMirrored` via `ALTER TABLE` for pre-v2 DBs; v3 = singleton `Webcam` + per-scene `WebcamSceneConfig`, migrated idempotently **without backfill** — the stale `Source.DeviceId` column remains but is no longer read/written; v4 = `WebcamSceneConfig.RectWidth`/`RectHeight`, the pre-Round rect for the round-to-rect restore; v5 = `Source.IsBackdrop` + `Source.CaptureKey`, the live-capture backdrop; v6 = `Scene.HasBackdrop` — Live-only policy, one-time backfill turns Starting/BRB/Chat/Ending off + drops their backdrop sources; `MainViewModel.NormalizeBackgrounds` re-normalizes on every load — TASK 25: one locked "Background" per canonical screen (static art everywhere, DisplayCapture on Live), duplicates dropped, index 0, orphaned `BackgroundUseDefault_*`/`BackgroundPath_*` Settings keys purged at save). Settings key/value table holds app state: `MicSourceName`, `ReusableStream` (TASK 9 — the cached reusable stream's id/address/name so the pump has its RTMP URL at startup) |
| `AlertRenderer.cs` | **The alert rasterizer (TASK 43, 2026-09-24):** renders one event's celebration animation into a BGRA `VideoFrame` — SIX DISTINCT animations (SuperChat slide-up/shine/count-up, SuperSticker scale-pop, NewMember drop-in/flash-band, MemberGift slide-left/chip-fan, GiftReceived confetti scale-in, MemberMilestone rise/growth-bar), enter/hold/exit phases, brand line on every card. **Still the fallback** when no alert clip is playable (TASK 47) |
| `AlertOverlayLayer.cs` | **The native alert-box component (TASK 43; video branch TASK 47):** event queue (cap 10) + 33ms dispatch ticker + `RenderFrame` cache-first live path + `UpdatePreview`; **`Advance(double)` is the deterministic test clock** (tests never start the timer); `ChatEventKind.None` rows (sub mentions) never enqueue; idle = transparent (null frame). Since TASK 47 a clip branch owns `IAlertClipDecoder` per play — ctor seams `(clipDecoderFactory, clipPathResolver, audioSink)`, fade in/out over `FadeDurationSeconds = 0.30` (straight-alpha copies + volume×fade audio), EOF freeze-frame → drain, and `AlertTickerFrame` composes the scrolling message strip |
| `AlertClipDecoder.cs` | **Per-play alert video decoder (TASK 47):** `IAlertClipDecoder` (Start/Stop/Dispose + `FrameAvailable`/`AudioReady`/`Completed`) + ffmpeg child emitting `rawvideo bgra` (`-vf scale=W:H`, read via `RawVideoFrameReader`) and `f32le -ar 48000 -ac 2` on a **carry-buffer** audio loop (float boundary straddles never drop samples), both real-time paced; bgra is opaque (layer owns alpha). One-shot lifetime — created per alert, disposed at drain; tests inject a fake |
| `LayoutStore.cs` | SQLite persistence (`Microsoft.Data.Sqlite`) at `%APPDATA%\ytLlive\ytLlive.db`; assets stored as BLOBs keyed by SHA-256 content hash; save/open layout files; schema `user_version` 10 (`Source.ClipShape`/`IsMirrored` via `ALTER TABLE` for pre-v2 DBs; v3 = singleton `Webcam` + per-scene `WebcamSceneConfig`, migrated idempotently **without backfill** — the stale `Source.DeviceId` column remains but is no longer read/written; v4 = `WebcamSceneConfig.RectWidth`/`RectHeight`, the pre-Round rect for the round-to-rect restore; v5 = `Source.IsBackdrop` + `Source.CaptureKey`, the live-capture backdrop; v6 = `Scene.HasBackdrop` — Live-only policy, one-time backfill turns Starting/BRB/Chat/Ending off + drops their backdrop sources; v9→v10 (TASK 39) = `YppSnapshot` history table; all later columns stay additive guarded ALTERs — TASK 47 adds the five alert `Source` columns (`AlertVideoAssetId`/`AlertVideoPath`/`AlertUseDefaultVideo`/`AlertShowTicker`/`AlertVideoVolume`) without a version bump; `MainViewModel.NormalizeBackgrounds` re-normalizes on every load — TASK 25: one locked "Background" per canonical screen (static art everywhere, DisplayCapture on Live), duplicates dropped, index 0, orphaned `BackgroundUseDefault_*`/`BackgroundPath_*` Settings keys purged at save). Settings key/value table holds app state: `MicSourceName`, `ReusableStream` (TASK 9 — the cached reusable stream's id/address/name so the pump has its RTMP URL at startup), `AlertDefaultVideoAssetId` (TASK 47 — the stamped built-in alert clip's asset row; save's prune exempts it via UNION) |
| `VideoFrame.cs` | Normalized CPU frame seam (`Width`/`Height`/tightly-packed BGRA `byte[]`) — the only pixel type the rest of the app knows about; future capture sources (screen, background-removed webcam) feed the same seam |
| `CameraDeviceInfo.cs` | `(Id, DisplayName)` for a physical capture device |
| `ICameraEnumerator.cs` | `GetCamerasAsync()` — seam so the picker/`CameraManager` never touch WinRT (tests inject fakes) |
@@ -34,13 +35,14 @@ External-facing logic: YouTube API, persistence. See
| `ScreenCaptureSourceFactory.cs` | `Resolve(key)` parses `monitor:<n>` / `window:<hwnd>` / `picker:<name>` into a source; `PickAsync()` shows the OS `GraphicsCapturePicker` and returns the `picker:` key (transient — a reload falls back to auto-detection) |
| `WebView2Manager.cs` | **Web-source ownership (TASK 17, composition-capture since 2026-09-14):** per registered `Source { Type: WebSource }` it owns a `CoreWebView2CompositionController` + its visual tree (`RootVisualTarget` child under a 1920×1080 root `ContainerVisual` on one `Windows.UI.Composition.Compositor` created on the UI thread via the CoreMessaging `CreateDispatcherQueueController` P/Invoke), a wrapped `WebCaptureFrameSource`, and the shared crop-sized `WriteableBitmap` preview (dispatcher-coalesced copy at Render priority, `PreviewBitmapChanged` raised only on recreation). Nav + `TransparentBackgroundScript` injection unchanged. Public ctor `(Dispatcher)`; internal seam ctor `(Dispatcher, Func<string, IScreenCaptureSource>?)` for hermetic tests (no WinRT). Caller: `MainViewModel.InitWebView2()` (now invoked from `MainWindow_Loaded` — the parent HWND must exist). See `../ai.md` "Slice 14" |
| `WebCaptureFrameSource.cs` | The `IScreenCaptureSource` for a web session: owns the `GraphicsCaptureItem` (`CreateFromVisual`) + a free-threaded `Direct3D11CaptureFramePool` (2 buffers) + session; readback via `SoftwareBitmap.CreateCopyFromSurfaceAsync` (BGRA, **`BitmapAlphaMode.Straight`** — the compositor blends straight alpha); per-frame `FindContentBounds` alpha-bbox (internal static, unit-tested) stamps `VideoFrame.CropBounds`; ring up to 2 fresh frames + Epoch reuse; `IsOpaque=false`. Mirror of `ScreenCaptureFrameSource` |
| `Compositor/SceneCompositor.cs` | **The output compositor (TASK 4 ship step 1)**: renders a scene into the encoder's master `VideoFrame` (tightly-packed BGRA8), mirroring the XAML preview minus editing chrome — backdrop → background → elements (`UniformToFill` cover-crop, round clip, mirror, opacity, border) → branding flash → social bar (optional `socialBarFrame` + `socialBarTop` in master space, blitted last so the bar sits above the flash; the old `BlitFlash` generalized to `BlitOverlay` with source offsets). Pure and WPF-free: frames injected via a `Func<SceneElement, VideoFrame?>` resolver (webcam → DeviceId, image → AssetId, backdrop → CaptureKey); output sized by `CompositorOptions` (16:9 = full master 1:1; vertical 9:16 = 607×1080 crop → 1080×1920 bilinear). Preview stays XAML (editing view); this is the output view — see `ai.md` "Scene compositor" |
| `Compositor/SceneCompositor.cs` | **The output compositor (TASK 4 ship step 1)**: renders a scene into the encoder's master `VideoFrame` (tightly-packed BGRA8), mirroring the XAML preview minus editing chrome — backdrop → background → elements (`UniformToFill` cover-crop, round clip, mirror, opacity, border) → branding flash → social bar (optional `socialBarFrame` + `socialBarTop` in master space, blitted last so the bar sits above the flash; the old `BlitFlash` generalized to `BlitOverlay` with source offsets). **TASK 47:** `Render` + `CompositeLayers` take a trailing `tickerFrame` blitted at (0,0) after the bar — the alert ticker is a message-scrolling dynamic overlay, never baked. Pure and WPF-free: frames injected via a `Func<SceneElement, VideoFrame?>` resolver (webcam → DeviceId, image → AssetId, backdrop → CaptureKey); output sized by `CompositorOptions` (16:9 = full master 1:1; vertical 9:16 = 607×1080 crop → 1080×1920 bilinear). Preview stays XAML (editing view); this is the output view — see `ai.md` "Scene compositor" |
| `Compositor/CompositorOptions.cs` | The active tier's output rect (source space over the 1920×1080 master, integer-aligned — `MainViewModel.OutputRectX` can be 656.5) + target W×H |
| `Compositor/StretchMath.cs` | Pure pixel math: the WPF `UniformToFill` cover-crop, clamped bilinear sample/scale (unit-tested half of the compositor) |
| `Compositor/StaticPixelCache.cs` | Asset bytes → cached BGRA8 `VideoFrame`, decoded once per content hash — the output path's raw-pixel source (the preview uses ImageCache's `BitmapImage`) |
| `Encoder/IFfmpegLocator.cs` | **TASK 4 ship step 2 seam**: resolves an absolute path to `ffmpeg.exe` (PATH first → cached `%APPDATA%\ytLlive\tools\ffmpeg.exe` → pinned BtbN LGPL-shared zip download). The encoder step and tests fake it |
| `SocialValidator.cs` | `ISocialValidator` seam + `HttpSocialValidator` default: validates a social handle/URL by constructing the canonical profile URL and issuing an HTTP GET — the "act of validation" before adding to the bar. 200/redirect = exists; 404/connection failure = rejected. Fediverse `@user@domain` additionally does a best-effort nodeinfo lookup (`/.well-known/nodeinfo` → `software.name`) so the entry can show the instance's real logo; nodeinfo failure still validates (generic fediverse glyph). If the identity domain's nodeinfo is blocked (SSO) but the bare root 302s to the real instance (YunoHost default-app subdomains), the lookup follows the redirect and asks that host. Since the bar bug-fix branch the resolution **probes well-known subdomains** (`mastodon.`, `social.`, ... order in `FediverseSubdomainCandidates`) when both the identity domain and its redirect come up empty, bounded by a ~15s linked-CTS budget; `ResolveFediverseSoftwareAsync(domain, ct)` on the seam powers the load-time heal. Constructor takes optional `HttpClient` + `Action<string>` log for tests |
| `Compositor/SocialBarRenderer.cs` | **Social bar output strip** (bar bug-fix branch): rasterizes the global social bar into a transparent straight-alpha BGRA8 `VideoFrame` strip (1920 wide, 40px content + 24px glow pad) via `RenderTargetBitmap` — one Path (logo) + one TextBlock (handle) per entry, white on transparent with the preview's green `#2ecc71` glow baked in (Pbgra32 → straight-alpha unpremultiply). UI thread only; the frame is immutable afterwards so the `FramePump` reads it from its own thread |
| `Compositor/AlertTickerRenderer.cs` | **Alert message ticker strip (TASK 47):** marquee overlay rendered per call by pure byte-math onto a 1920×48 transparent BGRA frame — the text pill ("Author — Kind · amount") is rasterized once per text on the UI thread (cached, unpremultiplied straight alpha) and scrolled at 140px/s with a repeat-gap wrap, so the pump can call `Render(elapsed, text)` from its own thread without locking. **Dynamic overlay — never baked** (unlike the social bar) |
| `Encoder/EncoderOptions.cs` | One go-live's encoder config: full `RtmpUrl` (ingest + stream key), W×H/FPS/bitrate from the quality tier, `VideoEncoder` (null = probe), audio sample rate/channels, `GopSize` = FPS×4 (the ≤4s keyframe bound) |
| `Encoder/IFfmpegEncoder.cs` | **TASK 4 ship step 3 seam**: start/feed/stop the live encoder; `HealthUpdated` (parsed `StreamHealth`), `ProcessFailed` on unexpected non-zero exit. Constructor-injected locator + process factory (tests fake both) |
| `Encoder/FfmpegEncoder.cs` | The default encoder: spawn `ffmpeg.exe` (probe `-encoders` first → hardware NVENC/QSV/AMF, OpenH264 fallback — never libx264), feed raw BGRA frames into stdin, parse `-stats` lines into `StreamHealth`, graceful stop via stdin EOF + 10s kill watchdog. Constructed by `MainViewModel` since ship step 5 (`new FfmpegEncoder(new FfmpegLocator())`), driven by the `FramePump` |
@@ -49,7 +51,7 @@ External-facing logic: YouTube API, persistence. See
| `Encoder/FfmpegArgs.cs` | Pure FFmpeg command-line builder: rawvideo `pipe:0` input, `anullsrc` silent audio (WASAPI replaces it), H.264+AAC, closed GOP (`-g fps×4 -keyint_min -sc_threshold 0 -bf 0`, `yuv420p`), FLV → RTMP |
| `Encoder/FfmpegProgressParser.cs` | Pure parser for `frame=/fps=/size=/time=/bitrate=` stats lines → `FfmpegProgress` |
| `Encoder/FfmpegEncoderPicker.cs` | Pure H.264 encoder picker from `-encoders` output: NVENC → QSV → AMF → OpenH264; **never returns libx264** (GPL) |
| `Encoder/FramePump.cs` | **TASK 4 ship step 5**: the live frame producer — while live it snapshots the active scene each tick, resolves every element to its latest frame (`Func<SceneElement, VideoFrame?>` resolver), composites it into the tier's output frame, and paces frames into the encoder at the tier's FPS. All collaborators constructor-injected seams; free of WPF and the capture managers. `StartAsync` never throws (failures log + `Failed`); no RTMP URL = encoder skipped; `StopAsync` stops the encoder before awaiting the loop (backpressure deadlock); `ProcessFailed` self-stops. Since the bar bug-fix branch it takes an optional `socialBar: Func<(VideoFrame? Frame, SocialBarPosition Position)>?` seam — a pre-rendered bar strip composited last (above the flash) at the top edge or `SourceRectHeight − bar height`. See `ai.md` "Live frame pipeline" |
| `Encoder/FramePump.cs` | **TASK 4 ship step 5**: the live frame producer — while live it snapshots the active scene each tick, resolves every element to its latest frame (`Func<SceneElement, VideoFrame?>` resolver), composites it into the tier's output frame, and paces frames into the encoder at the tier's FPS. All collaborators constructor-injected seams; free of WPF and the capture managers. `StartAsync` never throws (failures log + `Failed`); no RTMP URL = encoder skipped; `StopAsync` stops the encoder before awaiting the loop (backpressure deadlock); `ProcessFailed` self-stops. Since the bar bug-fix branch it takes an optional `socialBar: Func<(VideoFrame? Frame, SocialBarPosition Position)>?` seam — a pre-rendered bar strip composited last (above the flash) at the top edge or `SourceRectHeight − bar height`. **TASK 47:** adds an `alertTicker: Func<VideoFrame?>?` ctor seam (parallel to `socialBar`), read per tick and threaded through `RenderScene`/`RenderFull` + **mixed into `BuildFullRenderSignature`** so the ticker scroll changes the cache key. See `ai.md` "Live frame pipeline" |
| `Encoder/FfmpegLocator.cs` | The default locator: PATH probe wins, then the cache, then pull+extract (`ffmpeg.exe` **+ the `libav*.dll` family** via a staging dir, failures via `AppLog`). Pinned to BtbN `autobuild-2026-08-09-13-03` **lgpl-shared** (dynamic linking = LGPL compliance without static-relink §6 material; no GPL libx264; NVENC/QSV/AMF + libopenh264 + native AAC — see `ai.md` → Licensing). Search dirs / tools dir / downloader constructor-injected for hermetic tests. Wired into the encoder since ship step 5 |
| `INotificationService.cs` | **Toast seam (TASK 24)**: `Show(title, message, AppNotificationSeverity)` + `AppNotificationSeverity { Info, Success, Warning, Error }` + `Info/Success/Warning/Error` extension methods — non-blocking user-facing messages; VMs/services raise through this so failures surface without MessageBox blocking the UI thread |
| `NotificationService.cs` | The default seam impl (Notification.Wpf 11.0.0, MIT): wraps a library `NotificationManager`; routes to the in-window `NotificationArea` named `ToastArea` (MainWindow bottom-right above the footer); lifetimes via the pure `BuildRequest` mapping + `Cards` tint table — Info/Success ~4s, Warning ~8s auto-dismiss, Error sticky (`NeverExpires()` = `TimeSpan.MaxValue`, never null); dark card tints matching the health-banner language; all Show calls marshal to the captured Dispatcher. See [`../ai.md`](../ai.md) "Toast notifications" |
@@ -57,7 +59,7 @@ External-facing logic: YouTube API, persistence. See
| `Audio/AudioSample.cs` | One captured chunk: interleaved PCM float (-1..1) + sample rate + channels |
| `Audio/WasapiLoopbackAudioSource.cs` | Desktop/game capture: NAudio `WasapiLoopbackCapture` on the default render device — automatic at unity; the game bar's meter consumes it |
| `Audio/WasapiMicAudioSource.cs` | Mic capture: NAudio `WasapiCapture` with the device resolved by `FriendlyName` matching `MicSourceName` (DisplayName only), default capture endpoint fallback; name provider `Func<string?>` re-read at each `Start` so a device picked mid-session takes effect immediately (the mixer restarts the mic on pick) |
| `Audio/AudioMixer.cs` | Owns both sources; capture starts once at startup (`MainViewModel.StartMicCaptureAsync`) and stops on `Shutdown` — NOT go-live (preview monitoring). Mic samples → `AudioLevelMeter` → `MicLevelChanged`; loopback samples → the game bar's meter via `LoopbackLevelChanged`. Surfaces mic connection state: `MicConnected`/`MicFailed` (drives the status dot) + `RestartMic()` (device swap mid-session, keeps loopback). Failures log via `AppLog`; mic failure zeroes the meter, loopback failure doesn't kill the mic. Note: the meter `Push` is unconditional (the `?.` on the event would otherwise skip the argument when nothing is subscribed) |
| `Audio/AudioMixer.cs` | Owns both sources; capture starts once at startup (`MainViewModel.StartMicCaptureAsync`) and stops on `Shutdown` — NOT go-live (preview monitoring). Mic samples → `AudioLevelMeter` → `MicLevelChanged`; loopback samples → the game bar's meter via `LoopbackLevelChanged`. Surfaces mic connection state: `MicConnected`/`MicFailed` (drives the status dot) + `RestartMic()` (device swap mid-session, keeps loopback). Failures log via `AppLog`; mic failure zeroes the meter, loopback failure doesn't kill the mic. Note: the meter `Push` is unconditional (the `?.` on the event would otherwise skip the argument when nothing is subscribed). **TASK 47 — alert audio:** `EnqueueAlertAudio` writes a dedicated 8s stereo-48k ring (`AlertBufferSeconds`), drained in `FillAndMix` pre-limiter and added at **unity (never ducked)**; `StartLive` clears the ring |
| `Audio/AudioLevelMeter.cs` | Pure smoothed RMS level (0..1): `Push(AudioSample)` + `Reset` — the unit-tested math behind `AudioLevel` and `GameAudioLevel`. `ToDisplay(float)` maps the raw linear RMS onto the meter's display scale (−60..0 dBFS spread across 0..1, with **+10 dB input amplification** so real speech peaks hit the red zone at maxed volume) — real speech/game RMS (~0.01..0.1) would otherwise leave a flat scale looking dead; ≤0.001 linear reads as zero (never idles on background noise) |
| `Audio/WaveToFloat.cs` | Pure WASAPI buffer → float conversion: IEEE float 32-bit direct, PCM 16-bit normalized, `WaveFormatExtensible` IEEE-float subformat GUID, trailing partial samples ignored |
| `Audio/MasterLimiter.cs` | **Master output limiter**: a **−1 dBFS ceiling** (`Ceiling = 0.891`) applied after the live mix so the honest gains can never sum past 0 dBFS and clip the AAC encode. Instant attack per frame (a hot frame is scaled exactly to the ceiling — no overshoot), smoothed release toward unity so loud passages don't pump. Pure — unit-tested |