feat(alerts): TASK 47 — alert box plays a video (built-in/custom clip) + read-time fade + message ticker
TASK 43's alert box grows a real video celebration. Per-alert IAlertClipDecoder
(ffmpeg bgra + f32le pipes, real-time paced, disposed at drain) plays the shipped
Assets/alert-default.mp4 (stamped into the Asset table at startup) unless the
creator picks their own file — path reference only, never stored in the DB; the
six AlertRenderer animations stay the fallback. ~0.3s fade rides the alpha
envelope on straight-source copies (EOF freeze-frames then fades out); audio
forwards to a new AudioMixer alert ring (8s, 48k stereo) drained at unity — no
duck, creator ruling — scaled by volume × fade. An auto-composed marquee ticker
('Funder — Super Chat · $10.00', 140px/s) scrolls top-of-frame via a
FramePump._alertTicker seam through Render/CompositeLayers, mixed into the cache
signature (dynamic overlay, never baked). New Stream Alerts section in LeftPanel.
Derivative-work references (how OBS/Streamlabs alert boxes do per-alert video):
- https://support.streamlabs.com/hc/en-us/articles/217741147-Setting-Up-Your-Streamlabs-Alerts (custom image/video per alert type + variations)
- https://obsproject.com/kb/stream-tutorial-2-alerts (alert overlay as an on-screen zone)
- https://streamlabs.com/content-hub/widgets/alert-box (per-event alert playback)
Good Dog: AlertLayerVideoTests drives a fake IAlertClipDecoder through the whole
lifecycle in one pass (custom path wins, decoder spawns/disposes, fade envelope
0→127→255, audio volume×fade, ticker scrolls, EOF fade-drain to idle). It caught
the clip branch of Advance not clearing _current before AdvanceToNext — the layer
stayed IsPlaying after drain (MyMistakes post-mortem).
Full vstest 319/319; clean build 0 warnings; scope check green.
This commit is contained in:
@@ -0,0 +1,208 @@
|
||||
using System.Diagnostics;
|
||||
using ytLive.Services.Audio;
|
||||
using ytLive.Services.Encoder;
|
||||
|
||||
namespace ytLive.Services;
|
||||
|
||||
/// <summary>
|
||||
/// TASK 47: one alert clip's playback — video frames + on-stream audio from the
|
||||
/// same file, both paced to real time. The alert box owns a decoder per event
|
||||
/// (start at play, dispose at drain) — unlike MediaSource's app-wide refcounted
|
||||
/// session, an alert REPLAYS the same clip each event, so a fresh decode per
|
||||
/// play is required. Video is rawvideo BGRA scaled to the box rect (the
|
||||
/// <see cref="MediaVideoSource"/> shape); audio is ffmpeg <c>f32le</c> at the
|
||||
/// mixer's native 48 kHz stereo (so the mixer ring needs no resampler). The
|
||||
/// fade is applied by the alert layer, not here.
|
||||
/// </summary>
|
||||
public interface IAlertClipDecoder : IDisposable
|
||||
{
|
||||
/// <summary>Starts the decode loop (video + audio); raises the events off the
|
||||
/// calling thread. Paced to real time so playback matches wall clock.</summary>
|
||||
void Start();
|
||||
|
||||
/// <summary>Stops and kills any running decode subprocesses.</summary>
|
||||
void Stop();
|
||||
|
||||
/// <summary>One decoded video frame (box-sized, straight-alpha BGRA).</summary>
|
||||
event Action<VideoFrame>? FrameAvailable;
|
||||
|
||||
/// <summary>One chunk of on-stream audio (48 kHz stereo float).</summary>
|
||||
event Action<AudioSample>? AudioReady;
|
||||
|
||||
/// <summary>Raised once when both streams have fully played out.</summary>
|
||||
event Action? Completed;
|
||||
}
|
||||
|
||||
/// <summary>Default <see cref="IAlertClipDecoder"/>: two ffmpeg subprocesses on
|
||||
/// the same file. Video via <see cref="RawVideoFrameReader"/> on a <c>rawvideo
|
||||
/// bgra</c> pipe (-an), audio via an <c>f32le -ac 2 -ar 48000</c> pipe (-vn).
|
||||
/// The seams (locator, process factory, frame-rate probe, delay) let tests drive
|
||||
/// a fake decoder without a real codec or Windows.</summary>
|
||||
public sealed class AlertClipDecoder : IAlertClipDecoder
|
||||
{
|
||||
public event Action<VideoFrame>? FrameAvailable;
|
||||
public event Action<AudioSample>? AudioReady;
|
||||
public event Action? Completed;
|
||||
|
||||
private const int AudioSampleRate = 48000;
|
||||
private const int AudioChannels = 2;
|
||||
private const int AudioChunkFloats = 4800; // 50ms of stereo
|
||||
|
||||
private readonly string _path;
|
||||
private readonly int _width;
|
||||
private readonly int _height;
|
||||
private readonly IFfmpegLocator _locator;
|
||||
private readonly Func<IDecodeProcess> _processFactory;
|
||||
private readonly IFrameRateProbe? _probe;
|
||||
private readonly Func<TimeSpan, CancellationToken, Task> _delay;
|
||||
private readonly CancellationTokenSource _cts = new();
|
||||
private Task? _run;
|
||||
|
||||
public AlertClipDecoder(
|
||||
string path,
|
||||
int width,
|
||||
int height,
|
||||
IFfmpegLocator locator,
|
||||
Func<IDecodeProcess> processFactory,
|
||||
IFrameRateProbe? frameRateProbe = null,
|
||||
Func<TimeSpan, CancellationToken, Task>? delay = null)
|
||||
{
|
||||
_path = path;
|
||||
_width = width;
|
||||
_height = height;
|
||||
_locator = locator;
|
||||
_processFactory = processFactory;
|
||||
_probe = frameRateProbe;
|
||||
_delay = delay ?? ((t, ct) => Task.Delay(t, ct));
|
||||
}
|
||||
|
||||
public void Start()
|
||||
{
|
||||
if (_run != null) return;
|
||||
_run = Task.Run(RunAsync, CancellationToken.None);
|
||||
}
|
||||
|
||||
public void Stop() => _cts.Cancel();
|
||||
|
||||
public void Dispose()
|
||||
{
|
||||
_cts.Cancel();
|
||||
_cts.Dispose();
|
||||
}
|
||||
|
||||
private async Task RunAsync()
|
||||
{
|
||||
var ct = _cts.Token;
|
||||
try
|
||||
{
|
||||
double? frameRate = null;
|
||||
if (_probe != null)
|
||||
frameRate = await _probe.ProbeAsync(_path, ct).ConfigureAwait(false);
|
||||
var frameDuration = frameRate is > 0 ? TimeSpan.FromSeconds(1.0 / frameRate.Value) : TimeSpan.Zero;
|
||||
|
||||
var ffmpegPath = await _locator.LocateAsync(ct).ConfigureAwait(false);
|
||||
|
||||
var videoTask = RunVideoAsync(ffmpegPath, frameDuration, ct);
|
||||
var audioTask = RunAudioAsync(ffmpegPath, ct);
|
||||
await Task.WhenAll(videoTask, audioTask).ConfigureAwait(false);
|
||||
}
|
||||
catch (OperationCanceledException)
|
||||
{
|
||||
return;
|
||||
}
|
||||
catch
|
||||
{
|
||||
// A failed decode is not fatal: the alert layer falls back to the six
|
||||
// animations (or the frame it already holds) exactly as a missing file.
|
||||
}
|
||||
|
||||
if (!ct.IsCancellationRequested)
|
||||
Completed?.Invoke();
|
||||
}
|
||||
|
||||
private async Task RunVideoAsync(string ffmpegPath, TimeSpan frameDuration, CancellationToken ct)
|
||||
{
|
||||
var args =
|
||||
$"-hide_banner -loglevel error -i \"{_path}\" " +
|
||||
$"-f rawvideo -pix_fmt bgra -vf scale={_width}:{_height} -an -";
|
||||
using var process = _processFactory();
|
||||
process.Start(new ProcessStartInfo
|
||||
{
|
||||
FileName = ffmpegPath,
|
||||
Arguments = args,
|
||||
UseShellExecute = false,
|
||||
RedirectStandardOutput = true,
|
||||
RedirectStandardError = true,
|
||||
CreateNoWindow = true,
|
||||
});
|
||||
|
||||
var reader = new RawVideoFrameReader(_width, _height);
|
||||
var buffer = new byte[65536];
|
||||
var stream = process.StandardOutput;
|
||||
while (!ct.IsCancellationRequested && !process.HasExited)
|
||||
{
|
||||
var read = await stream.ReadAsync(buffer, 0, buffer.Length, ct).ConfigureAwait(false);
|
||||
if (read <= 0) break;
|
||||
var chunk = new byte[read];
|
||||
Buffer.BlockCopy(buffer, 0, chunk, 0, read);
|
||||
foreach (var frame in reader.Feed(chunk))
|
||||
{
|
||||
FrameAvailable?.Invoke(frame);
|
||||
if (frameDuration > TimeSpan.Zero)
|
||||
await _delay(frameDuration, ct).ConfigureAwait(false);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private async Task RunAudioAsync(string ffmpegPath, CancellationToken ct)
|
||||
{
|
||||
var args =
|
||||
$"-hide_banner -loglevel error -i \"{_path}\" " +
|
||||
$"-f f32le -ac {AudioChannels} -ar {AudioSampleRate} -vn -";
|
||||
using var process = _processFactory();
|
||||
process.Start(new ProcessStartInfo
|
||||
{
|
||||
FileName = ffmpegPath,
|
||||
Arguments = args,
|
||||
UseShellExecute = false,
|
||||
RedirectStandardOutput = true,
|
||||
RedirectStandardError = true,
|
||||
CreateNoWindow = true,
|
||||
});
|
||||
|
||||
var stream = process.StandardOutput;
|
||||
var raw = new byte[65536];
|
||||
var carry = new byte[0];
|
||||
var floatScratch = new float[AudioChunkFloats];
|
||||
var floatIndex = 0;
|
||||
// The pipe is byte-delimited only: a read may split a float, so leftover
|
||||
// bytes are carried into the next read instead of dropped (a dropped
|
||||
// half-float would desync the entire stream).
|
||||
while (!ct.IsCancellationRequested && !process.HasExited)
|
||||
{
|
||||
var read = await stream.ReadAsync(raw, 0, raw.Length, ct).ConfigureAwait(false);
|
||||
if (read <= 0) break;
|
||||
var total = carry.Length + read;
|
||||
var bytes = total <= raw.Length ? raw : new byte[total];
|
||||
if (carry.Length > 0)
|
||||
{
|
||||
Buffer.BlockCopy(carry, 0, bytes, 0, carry.Length);
|
||||
Buffer.BlockCopy(raw, 0, bytes, carry.Length, read);
|
||||
}
|
||||
var floatsTo = total - (total % 4);
|
||||
for (var i = 0; i < floatsTo && floatIndex < floatScratch.Length; i += 4)
|
||||
floatScratch[floatIndex++] = BitConverter.ToSingle(bytes, i);
|
||||
carry = total > floatsTo ? bytes[floatsTo..total] : new byte[0];
|
||||
|
||||
while (floatIndex == floatScratch.Length)
|
||||
{
|
||||
AudioReady?.Invoke(new AudioSample(floatScratch, AudioSampleRate, AudioChannels));
|
||||
// Pace the NEXT chunk so the whole clip's audio lands at real-time
|
||||
// cadence (the fade in the alert layer tracks the same wall clock).
|
||||
await _delay(TimeSpan.FromSeconds(AudioChunkFloats / (double)(AudioSampleRate * AudioChannels)), ct).ConfigureAwait(false);
|
||||
floatScratch = new float[AudioChunkFloats];
|
||||
floatIndex = 0;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
+223
-20
@@ -5,28 +5,59 @@ using System.Windows.Media;
|
||||
using System.Windows.Media.Imaging;
|
||||
using System.Windows.Threading;
|
||||
using ytLive.Models;
|
||||
using ytLive.Services.Audio;
|
||||
using ytLive.Services.Compositor;
|
||||
|
||||
namespace ytLive.Services;
|
||||
|
||||
/// <summary>
|
||||
/// The native alert box (OBS-style celebration zone, no third party). Owns the
|
||||
/// event queue and the animation clock: <see cref="ChatEventKind"/> events from
|
||||
/// the chat feed are enqueued one-at-a-time and played through their own unique
|
||||
/// animation (<see cref="AlertRenderer"/>), frames served to the stream compositor
|
||||
/// via <see cref="RenderFrame"/> and mirrored to the preview via
|
||||
/// <see cref="UpdatePreview"/>. Idle = transparent (null frame); playback is
|
||||
/// the chat feed are enqueued one-at-a-time and played through a celebration
|
||||
/// animation (<see cref="AlertRenderer"/>) OR — when the alert-box video is
|
||||
/// configured and resolves — through a decoded clip. Frames are served to the
|
||||
/// stream compositor via <see cref="RenderFrame"/> and mirrored to the preview
|
||||
/// via <see cref="UpdatePreview"/>. Idle = transparent (null frame); playback is
|
||||
/// advanced by a 33ms dispatch ticker. The clock is exposed as
|
||||
/// <see cref="Advance"/> so tests drive playback deterministically without the
|
||||
/// timer. Ordinary chat rows (<see cref="ChatEventKind.None"/>) never enqueue —
|
||||
/// a subscriber mention is a chat row, not an alert.
|
||||
///
|
||||
/// TASK 47 (Stream Alerts video): a video-backed alert replaces the animation
|
||||
/// while a clip decodes (<see cref="IAlertClipDecoder"/>, one per play). The
|
||||
/// clip plays at its own media pace; the layer applies a ~300ms fade-in/out
|
||||
/// alpha envelope to the frames AND the audio samples (freeze-frame fade-out on
|
||||
/// the tail), clips the alert to the clip length, and forwards the scaled audio
|
||||
/// to the mixer's alert ring via <paramref name="audioSink"/> (no duck). The
|
||||
/// per-source config (path/default/ticker/volume) is resolved through
|
||||
/// <paramref name="clipPathResolver"/>; the ticker strip is served via
|
||||
/// <see cref="AlertTickerFrame"/>.
|
||||
/// </summary>
|
||||
public sealed class AlertOverlayLayer : IDisposable
|
||||
{
|
||||
/// <summary>The fade-in/out ramp length (video alpha AND audio gain). Measured
|
||||
/// against a real clip in the AlertLayerVideoTests unit; keep the ramp short
|
||||
/// enough to read as a "pop-in" on stream (see ai.md).</summary>
|
||||
internal const double FadeDurationSeconds = 0.30;
|
||||
|
||||
private static readonly TimeSpan TickInterval = TimeSpan.FromMilliseconds(33);
|
||||
|
||||
private readonly AlertRenderer _renderer;
|
||||
private readonly Func<string, int, int, IAlertClipDecoder>? _clipDecoderFactory;
|
||||
private readonly Func<Source, string?>? _clipPathResolver;
|
||||
private readonly Action<AudioSample>? _audioSink;
|
||||
private readonly Queue<ChatMessage> _queue = new();
|
||||
private readonly DispatcherTimer? _ticker;
|
||||
|
||||
private ChatMessage? _current;
|
||||
private Source? _currentBox;
|
||||
private IAlertClipDecoder? _clip;
|
||||
private VideoFrame? _latestClipFrame;
|
||||
private bool _clipEof;
|
||||
private double _clipEofElapsed;
|
||||
private int _clipWidth;
|
||||
private int _clipHeight;
|
||||
private string? _tickerText;
|
||||
private double _elapsed;
|
||||
private IEnumerable<Scene>? _scenes;
|
||||
|
||||
@@ -35,11 +66,16 @@ public sealed class AlertOverlayLayer : IDisposable
|
||||
private string? _frameKey;
|
||||
private VideoFrame? _cachedFrame;
|
||||
|
||||
private static readonly TimeSpan TickInterval = TimeSpan.FromMilliseconds(33);
|
||||
|
||||
public AlertOverlayLayer(AlertRenderer renderer)
|
||||
public AlertOverlayLayer(
|
||||
AlertRenderer renderer,
|
||||
Func<string, int, int, IAlertClipDecoder>? clipDecoderFactory = null,
|
||||
Func<Source, string?>? clipPathResolver = null,
|
||||
Action<AudioSample>? audioSink = null)
|
||||
{
|
||||
_renderer = renderer;
|
||||
_clipDecoderFactory = clipDecoderFactory;
|
||||
_clipPathResolver = clipPathResolver;
|
||||
_audioSink = audioSink;
|
||||
if (Application.Current != null)
|
||||
{
|
||||
_ticker = new DispatcherTimer { Interval = TickInterval };
|
||||
@@ -73,6 +109,7 @@ public sealed class AlertOverlayLayer : IDisposable
|
||||
{
|
||||
_current = _queue.Dequeue();
|
||||
_elapsed = 0;
|
||||
BeginClip();
|
||||
}
|
||||
_contentVersion++;
|
||||
}
|
||||
@@ -82,11 +119,30 @@ public sealed class AlertOverlayLayer : IDisposable
|
||||
/// nothing stops the ticker and leaves the box transparent.</summary>
|
||||
public void Advance(double deltaSeconds)
|
||||
{
|
||||
// Video-backed alert: duration is clip-driven. The clip plays at media
|
||||
// pace; once it EOFs the last frame sweeps through the fade-out before
|
||||
// the alert moves on (a hard cut at EOF would pop, not fade).
|
||||
if (_clip != null)
|
||||
{
|
||||
_elapsed += deltaSeconds;
|
||||
if (_clipEof && _elapsed - _clipEofElapsed >= FadeDurationSeconds)
|
||||
{
|
||||
_clipEof = false;
|
||||
StopClip();
|
||||
_current = null;
|
||||
_elapsed = 0;
|
||||
AdvanceToNext();
|
||||
}
|
||||
_contentVersion++;
|
||||
return;
|
||||
}
|
||||
|
||||
if (_current == null)
|
||||
{
|
||||
if (_queue.Count == 0) return;
|
||||
_current = _queue.Dequeue();
|
||||
_elapsed = 0;
|
||||
BeginClip();
|
||||
_contentVersion++;
|
||||
return;
|
||||
}
|
||||
@@ -96,15 +152,7 @@ public sealed class AlertOverlayLayer : IDisposable
|
||||
{
|
||||
_current = null;
|
||||
_elapsed = 0;
|
||||
if (_queue.Count > 0)
|
||||
{
|
||||
_current = _queue.Dequeue();
|
||||
_elapsed = 0;
|
||||
}
|
||||
else
|
||||
{
|
||||
_ticker?.Stop();
|
||||
}
|
||||
AdvanceToNext();
|
||||
}
|
||||
_contentVersion++;
|
||||
}
|
||||
@@ -112,10 +160,9 @@ public sealed class AlertOverlayLayer : IDisposable
|
||||
/// <summary>Whether an alert is currently playing or queued.</summary>
|
||||
public bool IsPlaying => _current != null || _queue.Count > 0;
|
||||
|
||||
/// <summary>The live-output frame for one AlertBox element: the current
|
||||
/// animation frame while playing, null (transparent) when idle. Mirrors the
|
||||
/// chat cache pattern — the raster is WPF/UI-thread, so only the cache miss
|
||||
/// marshals to the dispatcher and re-validates there.</summary>
|
||||
/// <summary>The live-output frame for one AlertBox element: the decoded clip
|
||||
/// frame (faded) while a clip is active, else the animation frame, null
|
||||
/// (transparent) when idle.</summary>
|
||||
public VideoFrame? RenderFrame(Source alertBox)
|
||||
{
|
||||
if (_current == null)
|
||||
@@ -124,6 +171,9 @@ public sealed class AlertOverlayLayer : IDisposable
|
||||
return null;
|
||||
}
|
||||
|
||||
if (_clip != null)
|
||||
return RenderClipFrame(alertBox);
|
||||
|
||||
var width = (int)Math.Max(1, alertBox.Width);
|
||||
var height = (int)Math.Max(1, alertBox.Height);
|
||||
var key = $"{width}x{height}|{_current.Id}|{Progress()}";
|
||||
@@ -147,6 +197,139 @@ public sealed class AlertOverlayLayer : IDisposable
|
||||
return RenderCore(alertBox, width, height, key, _contentVersion);
|
||||
}
|
||||
|
||||
/// <summary>The global ticker strip for the stream output (top edge, full
|
||||
/// master width): a marquee of "viewer — event" while an alert is playing
|
||||
/// and the config has the ticker on, else null. Pure byte-math per call so
|
||||
/// the frame pump can read it on any thread (the text strip is rasterized
|
||||
/// once on the UI thread and cached).</summary>
|
||||
public VideoFrame? AlertTickerFrame
|
||||
{
|
||||
get
|
||||
{
|
||||
if (_current == null || string.IsNullOrWhiteSpace(_tickerText)) return null;
|
||||
return AlertTickerRenderer.Render(_tickerText!, _elapsed);
|
||||
}
|
||||
}
|
||||
|
||||
private VideoFrame? RenderClipFrame(Source alertBox)
|
||||
{
|
||||
var frame = _latestClipFrame;
|
||||
if (frame == null) return null;
|
||||
|
||||
var w = (int)Math.Max(1, alertBox.Width);
|
||||
var h = (int)Math.Max(1, alertBox.Height);
|
||||
if (frame.Width != w || frame.Height != h)
|
||||
frame = StretchMath.BilinearScale(frame, w, h);
|
||||
|
||||
var fade = FadeAt();
|
||||
if (fade >= 1.0) return frame;
|
||||
|
||||
// Straight-alpha source-over on the compositor side: scale the source
|
||||
// alpha by the envelope so the clip blends out (a 680x200 copy per fade
|
||||
// tick is trivial; outside the ramps fade==1 returns the raw frame).
|
||||
var copy = new byte[frame.BgraPixels.Length];
|
||||
Buffer.BlockCopy(frame.BgraPixels, 0, copy, 0, copy.Length);
|
||||
var a = (byte)(fade * 255 + 0.5);
|
||||
for (var i = 3; i < copy.Length; i += 4)
|
||||
copy[i] = (byte)((copy[i] * a) >> 8);
|
||||
return new VideoFrame(frame.Width, frame.Height, copy);
|
||||
}
|
||||
|
||||
private double FadeAt()
|
||||
{
|
||||
if (_clip == null) return 1.0;
|
||||
if (_clipEof)
|
||||
return Math.Clamp(1.0 - (_elapsed - _clipEofElapsed) / FadeDurationSeconds, 0.0, 1.0);
|
||||
return Math.Clamp(_elapsed / FadeDurationSeconds, 0.0, 1.0);
|
||||
}
|
||||
|
||||
private void BeginClip()
|
||||
{
|
||||
StopClip();
|
||||
_tickerText = null;
|
||||
if (_current == null) return;
|
||||
|
||||
var box = _scenes?.SelectMany(s => s.Elements).OfType<Source>()
|
||||
.FirstOrDefault(s => s.Type == SourceType.AlertBox);
|
||||
_currentBox = box;
|
||||
if (box != null && box.AlertShowTicker)
|
||||
_tickerText = ComposeTicker(_current);
|
||||
|
||||
if (box == null || _clipDecoderFactory == null) return;
|
||||
var path = _clipPathResolver?.Invoke(box);
|
||||
if (string.IsNullOrWhiteSpace(path)) return;
|
||||
|
||||
_clipWidth = (int)Math.Max(1, box.Width);
|
||||
_clipHeight = (int)Math.Max(1, box.Height);
|
||||
try
|
||||
{
|
||||
_clip = _clipDecoderFactory(path, _clipWidth, _clipHeight);
|
||||
_clip.FrameAvailable += OnClipFrame;
|
||||
_clip.AudioReady += OnClipAudio;
|
||||
_clip.Completed += OnClipCompleted;
|
||||
_clip.Start();
|
||||
}
|
||||
catch
|
||||
{
|
||||
StopClip(); // decode setup failed (bad path) — fall back to animations
|
||||
}
|
||||
}
|
||||
|
||||
private void StopClip()
|
||||
{
|
||||
var clip = _clip;
|
||||
_clip = null;
|
||||
_latestClipFrame = null;
|
||||
_clipEof = false;
|
||||
if (clip == null) return;
|
||||
clip.FrameAvailable -= OnClipFrame;
|
||||
clip.AudioReady -= OnClipAudio;
|
||||
clip.Completed -= OnClipCompleted;
|
||||
try { clip.Stop(); } catch { /* already stopped */ }
|
||||
clip.Dispose();
|
||||
}
|
||||
|
||||
private void OnClipFrame(VideoFrame frame) => _latestClipFrame = frame;
|
||||
|
||||
private void OnClipCompleted()
|
||||
{
|
||||
_clipEof = true;
|
||||
_clipEofElapsed = _elapsed;
|
||||
}
|
||||
|
||||
private void OnClipAudio(AudioSample sample)
|
||||
{
|
||||
if (_audioSink == null) return;
|
||||
var volume = (float)Math.Clamp(_currentBox?.AlertVideoVolume ?? 1.0, 0.0, 1.0);
|
||||
if (volume <= 0f) return;
|
||||
var fade = FadeAt();
|
||||
if (fade <= 0.0) return;
|
||||
|
||||
var data = sample.Samples;
|
||||
if (fade < 1.0 || Math.Abs(volume - 1f) > 0.001f)
|
||||
{
|
||||
data = new float[data.Length];
|
||||
var gain = volume * (float)fade;
|
||||
for (var i = 0; i < sample.Samples.Length; i++)
|
||||
data[i] = sample.Samples[i] * gain;
|
||||
}
|
||||
_audioSink(new AudioSample(data, sample.SampleRate, sample.Channels));
|
||||
}
|
||||
|
||||
private void AdvanceToNext()
|
||||
{
|
||||
if (_queue.Count > 0)
|
||||
{
|
||||
_current = _queue.Dequeue();
|
||||
_elapsed = 0;
|
||||
BeginClip();
|
||||
}
|
||||
else
|
||||
{
|
||||
_ticker?.Stop();
|
||||
}
|
||||
}
|
||||
|
||||
private VideoFrame? RenderCore(Source alertBox, int width, int height, string key, int version)
|
||||
{
|
||||
_frameKey = key;
|
||||
@@ -172,6 +355,25 @@ public sealed class AlertOverlayLayer : IDisposable
|
||||
_ => 4.0,
|
||||
};
|
||||
|
||||
private static string ComposeTicker(ChatMessage m)
|
||||
{
|
||||
var label = m.Kind switch
|
||||
{
|
||||
ChatEventKind.SuperChat => "Super Chat",
|
||||
ChatEventKind.SuperSticker => "Super Sticker",
|
||||
ChatEventKind.NewMember => "New Member",
|
||||
ChatEventKind.MemberGift => "Membership Gift",
|
||||
ChatEventKind.GiftReceived => "Gifted Membership",
|
||||
ChatEventKind.MemberMilestone => "Member Milestone",
|
||||
_ => "Stream Alert",
|
||||
};
|
||||
var amount = m.Kind is ChatEventKind.SuperChat or ChatEventKind.SuperSticker
|
||||
&& m.SuperChatDisplayString is { Length: > 0 } s
|
||||
? $" · {s}"
|
||||
: "";
|
||||
return $"{m.AuthorName} — {label}{amount}";
|
||||
}
|
||||
|
||||
/// <summary>Render the current state onto one AlertBox preview (mirrors chat):
|
||||
/// the playing frame, or null when idle so the source shows nothing until an
|
||||
/// event actually fires.</summary>
|
||||
@@ -227,5 +429,6 @@ public sealed class AlertOverlayLayer : IDisposable
|
||||
_ticker?.Stop();
|
||||
_queue.Clear();
|
||||
_current = null;
|
||||
StopClip();
|
||||
}
|
||||
}
|
||||
@@ -14,7 +14,9 @@ namespace ytLive.Services.Audio;
|
||||
/// queues, applies the honest gains (mic = MicVolume, loopback = unity
|
||||
/// × auto-duck when the mic is hot — the slider drives system volume via mixes them to stereo float, runs the
|
||||
/// master limiter (a -1 dBFS ceiling so the sum never clips the encoder), and
|
||||
/// writes the chunk to the encoder's audio pipe.
|
||||
/// writes the chunk to the encoder's audio pipe. Since TASK 47 the alert layer's
|
||||
/// clip audio joins the same mix (pre-scaled, unity-added, never ducked) through
|
||||
/// <see cref="EnqueueAlertAudio"/>.
|
||||
/// </summary>
|
||||
public sealed class AudioMixer : IDisposable
|
||||
{
|
||||
@@ -39,6 +41,7 @@ public sealed class AudioMixer : IDisposable
|
||||
// thread-safe queues → the ducker + gain/mix loop → the encoder's pipe.
|
||||
private readonly AudioRingBuffer _micBuffer;
|
||||
private readonly AudioRingBuffer _loopbackBuffer;
|
||||
private readonly AudioRingBuffer _alertBuffer;
|
||||
private readonly VoiceFilterChain _voiceChain;
|
||||
private readonly AutoDucker _ducker;
|
||||
private readonly MasterLimiter _masterLimiter = new();
|
||||
@@ -48,6 +51,7 @@ public sealed class AudioMixer : IDisposable
|
||||
private NamedPipeAudioWriter _pipe = new();
|
||||
private float[]? _micChunk;
|
||||
private float[]? _loopbackChunk;
|
||||
private float[]? _alertChunk;
|
||||
private float[]? _mixBuffer;
|
||||
private float[]? _delayedMix;
|
||||
private DateTime _lastLoopErrorLogged = DateTime.MinValue;
|
||||
@@ -80,10 +84,12 @@ public sealed class AudioMixer : IDisposable
|
||||
var framesPerTick = Math.Max(1, (int)Math.Round(OutputSampleRate * seconds));
|
||||
_micBuffer = new AudioRingBuffer(OutputSampleRate * 2);
|
||||
_loopbackBuffer = new AudioRingBuffer(OutputSampleRate * 2 * 2);
|
||||
_alertBuffer = new AudioRingBuffer(OutputSampleRate * 2 * 8);
|
||||
_voiceChain = new VoiceFilterChain(OutputSampleRate);
|
||||
_ducker = new AutoDucker();
|
||||
_micChunk = new float[framesPerTick];
|
||||
_loopbackChunk = new float[framesPerTick * 2];
|
||||
_alertChunk = new float[framesPerTick * 2];
|
||||
_mixBuffer = new float[framesPerTick * 2];
|
||||
_delayedMix = new float[framesPerTick * 2];
|
||||
|
||||
@@ -168,6 +174,7 @@ public sealed class AudioMixer : IDisposable
|
||||
// Clear under the ring's lock, so only the in-flight chunk is dropped.
|
||||
_micBuffer.Clear();
|
||||
_loopbackBuffer.Clear();
|
||||
_alertBuffer.Clear();
|
||||
|
||||
// Negative sync offset (audio runs BEHIND video): OBS's fix is to "eat
|
||||
// the head of the buffer" — discard the first |N| ms of the live stream
|
||||
@@ -246,6 +253,34 @@ public sealed class AudioMixer : IDisposable
|
||||
LoopbackLevelChanged?.Invoke(level);
|
||||
}
|
||||
|
||||
/// <summary>Drops the alert clip's audio into its own ring (TASK 47 — the
|
||||
/// stream-alert video's soundtrack, pre-scaled by the alert layer to
|
||||
/// volume × fade). Mixed at unity into the live output — NO auto-duck: an
|
||||
/// alert must be heard over the mic, so the Creator's per-alert volume
|
||||
/// slider is the honest gain and nothing else is stacked on top. Monno and
|
||||
/// non-48k sources are normalized (the decoder already emits 48k stereo).</summary>
|
||||
public void EnqueueAlertAudio(AudioSample sample)
|
||||
{
|
||||
var data = sample.Samples;
|
||||
if (data.Length == 0) return;
|
||||
if (sample.SampleRate != OutputSampleRate)
|
||||
data = ResampleLoopback(data, sample.SampleRate);
|
||||
if (sample.Channels == 1)
|
||||
data = UpmixToStereo(data);
|
||||
_alertBuffer.Write(data);
|
||||
}
|
||||
|
||||
private static float[] UpmixToStereo(float[] mono)
|
||||
{
|
||||
var stereo = new float[mono.Length * 2];
|
||||
for (var i = 0; i < mono.Length; i++)
|
||||
{
|
||||
stereo[i * 2] = mono[i];
|
||||
stereo[i * 2 + 1] = mono[i];
|
||||
}
|
||||
return stereo;
|
||||
}
|
||||
|
||||
private void OnMicFailed(Exception ex)
|
||||
{
|
||||
_log?.Invoke($"Mic capture failed: {ex.Message}");
|
||||
@@ -386,6 +421,8 @@ public sealed class AudioMixer : IDisposable
|
||||
{
|
||||
var micCount = _micBuffer.Read(micChunk, micChunk.Length);
|
||||
var loopCount = _loopbackBuffer.Read(loopbackChunk, loopbackChunk.Length);
|
||||
var alertChunk = _alertChunk!;
|
||||
var alertCount = _alertBuffer.Read(alertChunk, alertChunk.Length);
|
||||
|
||||
float micRms = 0;
|
||||
if (micCount == 0)
|
||||
@@ -411,6 +448,15 @@ public sealed class AudioMixer : IDisposable
|
||||
Array.Clear(loopbackChunk, loopCount, loopbackChunk.Length - loopCount);
|
||||
}
|
||||
|
||||
if (alertCount == 0)
|
||||
{
|
||||
Array.Clear(alertChunk, 0, alertChunk.Length);
|
||||
}
|
||||
else if (alertCount < alertChunk.Length)
|
||||
{
|
||||
Array.Clear(alertChunk, alertCount, alertChunk.Length - alertCount);
|
||||
}
|
||||
|
||||
var duck = _ducker.Update(micRms);
|
||||
var micGain = (float)(_micGain?.Invoke() ?? 1.0);
|
||||
var loopGain = (float)(_loopbackGain?.Invoke() ?? 1.0) * duck;
|
||||
@@ -419,8 +465,8 @@ public sealed class AudioMixer : IDisposable
|
||||
for (var i = 0; i < frames; i++)
|
||||
{
|
||||
var m = micChunk[i] * micGain;
|
||||
mix[i * 2] = m + loopbackChunk[i * 2] * loopGain;
|
||||
mix[i * 2 + 1] = m + loopbackChunk[i * 2 + 1] * loopGain;
|
||||
mix[i * 2] = m + loopbackChunk[i * 2] * loopGain + alertChunk[i * 2];
|
||||
mix[i * 2 + 1] = m + loopbackChunk[i * 2 + 1] * loopGain + alertChunk[i * 2 + 1];
|
||||
}
|
||||
_masterLimiter.Process(mix);
|
||||
|
||||
|
||||
@@ -0,0 +1,139 @@
|
||||
using System.Collections.Generic;
|
||||
using System.Globalization;
|
||||
using System.Windows;
|
||||
using System.Windows.Media;
|
||||
using System.Windows.Media.Imaging;
|
||||
using System.Windows.Threading;
|
||||
|
||||
namespace ytLive.Services.Compositor;
|
||||
|
||||
/// <summary>
|
||||
/// Rasterizes the stream-alert ticker into a transparent BGRA strip that the
|
||||
/// output compositor overlays along the very top edge of the master frame
|
||||
/// (TASK 47). The strip is text ("Viewer — Super Chat · $12.34") inside a dark
|
||||
/// pill with the LlamaCasty accent bar, rendered ONCE per unique text on the UI
|
||||
/// thread (WPF <c>RenderTargetBitmap</c>, Pbgra32→straight-alpha like
|
||||
/// <see cref="SocialBarRenderer"/>), then cached. Each <see cref="Render"/> call
|
||||
/// is pure byte-math on that cached raster producing a scrolling marquee frame,
|
||||
/// so the frame pump can call it from any thread for every output frame at
|
||||
/// 30fps without a Dispatcher round-trip.
|
||||
/// </summary>
|
||||
public static class AlertTickerRenderer
|
||||
{
|
||||
/// <summary>Master-frame width — the ticker spans the full 1920px output.</summary>
|
||||
public const int Width = 1920;
|
||||
|
||||
/// <summary>Strip height (pill + vertical padding).</summary>
|
||||
public const int Height = 48;
|
||||
|
||||
private const int PixelsPerSecond = 140;
|
||||
private const int RepeatGap = 120;
|
||||
private const int CacheLimit = 48;
|
||||
|
||||
private static readonly object Gate = new();
|
||||
private static readonly Dictionary<string, VideoFrame> StripCache = new();
|
||||
|
||||
/// <summary>A marquee frame of <paramref name="text"/> scrolled
|
||||
/// <paramref name="elapsedSeconds"/> into its loop, or null for empty text.
|
||||
/// Thread-safe: reads the cached raster and does row copies only.</summary>
|
||||
public static VideoFrame? Render(string? text, double elapsedSeconds)
|
||||
{
|
||||
if (string.IsNullOrWhiteSpace(text)) return null;
|
||||
var strip = GetStrip(text!);
|
||||
if (strip == null) return null;
|
||||
|
||||
var sw = strip.Width;
|
||||
var cycle = Width + sw + RepeatGap;
|
||||
var pos = (int)(elapsedSeconds * PixelsPerSecond) % cycle;
|
||||
var x = Width - pos;
|
||||
|
||||
var pixels = new byte[Width * Height * 4];
|
||||
CopyStrip(pixels, strip, x);
|
||||
if (x < 0)
|
||||
CopyStrip(pixels, strip, x + Width + RepeatGap); // wrapped tail enters from the right
|
||||
return new VideoFrame(Width, Height, pixels);
|
||||
}
|
||||
|
||||
private static void CopyStrip(byte[] dst, VideoFrame strip, int dstX)
|
||||
{
|
||||
if (dstX >= Width || dstX + strip.Width <= 0) return;
|
||||
var x0 = Math.Max(0, dstX);
|
||||
var x1 = Math.Min(Width, dstX + strip.Width);
|
||||
var sx0 = x0 - dstX;
|
||||
var copyBytes = (x1 - x0) * 4;
|
||||
var sw = strip.Width;
|
||||
for (var y = 0; y < strip.Height; y++)
|
||||
Buffer.BlockCopy(strip.BgraPixels, (y * sw + sx0) * 4, dst, (y * Width + x0) * 4, copyBytes);
|
||||
}
|
||||
|
||||
private static VideoFrame? GetStrip(string text)
|
||||
{
|
||||
lock (Gate)
|
||||
{
|
||||
if (StripCache.TryGetValue(text, out var cached)) return cached;
|
||||
}
|
||||
var strip = RenderStrip(text);
|
||||
if (strip == null) return null;
|
||||
lock (Gate)
|
||||
{
|
||||
if (StripCache.Count >= CacheLimit) StripCache.Clear();
|
||||
if (!StripCache.TryGetValue(text, out var existing))
|
||||
StripCache[text] = strip;
|
||||
return StripCache[text];
|
||||
}
|
||||
}
|
||||
|
||||
private static VideoFrame? RenderStrip(string text)
|
||||
{
|
||||
// WPF rendering is single-threaded; the pump thread resolves to the UI
|
||||
// thread at most once per unique text (then hits the cache).
|
||||
var app = Application.Current;
|
||||
if (app != null && !app.Dispatcher.CheckAccess())
|
||||
{
|
||||
VideoFrame? result = null;
|
||||
app.Dispatcher.Invoke(() => result = RenderStrip(text));
|
||||
return result;
|
||||
}
|
||||
|
||||
const double pillHeight = 34;
|
||||
const double accentW = 6;
|
||||
const double padX = 16;
|
||||
const double gap = 8;
|
||||
|
||||
var font = new Typeface(new FontFamily("Segoe UI"), FontStyles.Normal, FontWeights.SemiBold, FontStretches.Normal);
|
||||
var ft = new FormattedText(text, CultureInfo.InvariantCulture, FlowDirection.LeftToRight,
|
||||
font, 22, Brushes.White, 1.0);
|
||||
|
||||
var pillW = (int)Math.Ceiling(padX + accentW + gap + ft.Width + padX);
|
||||
var top = (Height - pillHeight) / 2.0;
|
||||
var textTop = top + (pillHeight - ft.Height) / 2.0;
|
||||
|
||||
var visual = new DrawingVisual();
|
||||
using (var dc = visual.RenderOpen())
|
||||
{
|
||||
var pill = new Rect(0, top, pillW, pillHeight);
|
||||
dc.DrawRoundedRectangle(new SolidColorBrush(Color.FromArgb(0xE6, 0x10, 0x13, 0x24)), null, pill, pillHeight / 2, pillHeight / 2);
|
||||
dc.DrawRoundedRectangle(new SolidColorBrush(Color.FromRgb(0xe9, 0x45, 0x60)), null,
|
||||
new Rect(padX, top + (pillHeight - 12) / 2.0, accentW, 12), 2, 2);
|
||||
dc.DrawText(ft, new Point(padX + accentW + gap, textTop));
|
||||
}
|
||||
|
||||
var bitmap = new RenderTargetBitmap(pillW, Height, 96, 96, PixelFormats.Pbgra32);
|
||||
bitmap.Render(visual);
|
||||
|
||||
var pixels = new byte[pillW * Height * 4];
|
||||
bitmap.CopyPixels(pixels, pillW * 4, 0);
|
||||
|
||||
// Pbgra32 is premultiplied — unpremultiply so the compositor's
|
||||
// straight-alpha source-over doesn't halo the text edges.
|
||||
for (var i = 0; i < pixels.Length; i += 4)
|
||||
{
|
||||
var a = pixels[i + 3];
|
||||
if (a == 0 || a == 255) continue;
|
||||
pixels[i] = (byte)(pixels[i] * 255 / a);
|
||||
pixels[i + 1] = (byte)(pixels[i + 1] * 255 / a);
|
||||
pixels[i + 2] = (byte)(pixels[i + 2] * 255 / a);
|
||||
}
|
||||
return new VideoFrame(pillW, Height, pixels);
|
||||
}
|
||||
}
|
||||
@@ -37,7 +37,8 @@ public sealed class SceneCompositor
|
||||
/// front): live background (the scene's <c>IsBackground</c> source) → background image →
|
||||
/// visible elements (z-order = <c>Elements</c> order, mirroring the XAML DataTemplate) →
|
||||
/// branding flash → social bar (a global overlay; spans the master width at
|
||||
/// <paramref name="socialBarTop"/>). Transparent regions read opaque black.
|
||||
/// <paramref name="socialBarTop"/>) → stream-alert ticker (a global overlay pinned to
|
||||
/// the very top, TASK 47). Transparent regions read opaque black.
|
||||
/// </summary>
|
||||
public VideoFrame Render(
|
||||
Scene scene,
|
||||
@@ -48,7 +49,8 @@ public sealed class SceneCompositor
|
||||
int socialBarTop = 0,
|
||||
VideoFrame? staticBase = null,
|
||||
int split = -1,
|
||||
byte[]? scratch = null)
|
||||
byte[]? scratch = null,
|
||||
VideoFrame? tickerFrame = null)
|
||||
{
|
||||
if (scene == null) throw new ArgumentNullException(nameof(scene));
|
||||
if (frameFor == null) throw new ArgumentNullException(nameof(frameFor));
|
||||
@@ -137,6 +139,9 @@ public sealed class SceneCompositor
|
||||
if (socialBarFrame != null && socialBarFrame.Width > 0 && socialBarFrame.Height > 0)
|
||||
BlitOverlay(buffer, cropW, cropH, options, socialBarFrame, 0, socialBarTop);
|
||||
|
||||
if (tickerFrame != null && tickerFrame.Width > 0 && tickerFrame.Height > 0)
|
||||
BlitOverlay(buffer, cropW, cropH, options, tickerFrame, 0, 0);
|
||||
|
||||
return StretchMath.BilinearScale(
|
||||
new VideoFrame(cropW, cropH, buffer), options.OutputWidth, options.OutputHeight);
|
||||
}
|
||||
@@ -212,7 +217,10 @@ public sealed class SceneCompositor
|
||||
|
||||
/// <summary>The compositor's dynamic/above-split path: start from an existing frame
|
||||
/// buffer and composite the z-layers from <paramref name="split"/> onward on top.
|
||||
/// When <paramref name="baseFrame"/> is null the caller must fall back to full render.</summary>
|
||||
/// When <paramref name="baseFrame"/> is null the caller must fall back to full render.
|
||||
/// The social bar never reaches here (it IS baked into the static base) but the alert
|
||||
/// ticker must: it is dynamic by construction — only ever present while an alert plays —
|
||||
/// so it overlays LAST, pinned to the very top (TASK 47).</summary>
|
||||
public static VideoFrame CompositeLayers(
|
||||
VideoFrame baseFrame,
|
||||
Scene scene,
|
||||
@@ -222,7 +230,8 @@ public sealed class SceneCompositor
|
||||
VideoFrame? socialBarFrame = null,
|
||||
int socialBarTop = 0,
|
||||
VideoFrame? flashFrame = null,
|
||||
byte[]? scratch = null)
|
||||
byte[]? scratch = null,
|
||||
VideoFrame? tickerFrame = null)
|
||||
{
|
||||
var cropW = options.SourceRectWidth;
|
||||
var cropH = options.SourceRectHeight;
|
||||
@@ -247,6 +256,10 @@ public sealed class SceneCompositor
|
||||
if (frame == null) continue;
|
||||
BlitBakedElement(buffer, cropW, cropH, options, element, frame);
|
||||
}
|
||||
|
||||
if (tickerFrame != null && tickerFrame.Width > 0 && tickerFrame.Height > 0)
|
||||
BlitOverlay(buffer, cropW, cropH, options, tickerFrame, 0, 0);
|
||||
|
||||
return StretchMath.BilinearScale(new VideoFrame(cropW, cropH, buffer), options.OutputWidth, options.OutputHeight);
|
||||
}
|
||||
|
||||
|
||||
@@ -29,6 +29,7 @@ public sealed class FramePump : IDisposable
|
||||
private readonly Action<string>? _log;
|
||||
private readonly Func<TimeSpan, CancellationToken, Task> _pacingDelay;
|
||||
private readonly Func<(VideoFrame? Frame, SocialBarPosition Position)>? _socialBar;
|
||||
private readonly Func<VideoFrame?>? _alertTicker;
|
||||
private readonly TransitionService? _transition;
|
||||
private readonly SceneGraph? _sceneGraph;
|
||||
private readonly SceneCompositor _compositor = new();
|
||||
@@ -129,6 +130,7 @@ public sealed class FramePump : IDisposable
|
||||
Action<string>? log = null,
|
||||
Func<TimeSpan, CancellationToken, Task>? pacingDelay = null,
|
||||
Func<(VideoFrame? Frame, SocialBarPosition Position)>? socialBar = null,
|
||||
Func<VideoFrame?>? alertTicker = null,
|
||||
TransitionService? transition = null,
|
||||
SceneGraph? sceneGraph = null)
|
||||
{
|
||||
@@ -140,6 +142,7 @@ public sealed class FramePump : IDisposable
|
||||
_log = log;
|
||||
_pacingDelay = pacingDelay ?? ((delay, ct) => Task.Delay(delay, ct));
|
||||
_socialBar = socialBar;
|
||||
_alertTicker = alertTicker;
|
||||
_transition = transition;
|
||||
_sceneGraph = sceneGraph;
|
||||
}
|
||||
@@ -405,6 +408,11 @@ public sealed class FramePump : IDisposable
|
||||
: compositorOptions.SourceRectHeight - barFrame.Height;
|
||||
}
|
||||
|
||||
// The alert ticker is a per-frame overlay (it scrolls while an alert
|
||||
// plays), NEVER baked into the static base. Lawn-mower seam: read it
|
||||
// each tick and let the compositor pin it to the very top.
|
||||
var tickerFrame = _alertTicker?.Invoke();
|
||||
|
||||
VideoFrame frame;
|
||||
var scratchSize = compositorOptions.SourceRectWidth
|
||||
* compositorOptions.SourceRectHeight * 4;
|
||||
@@ -415,7 +423,7 @@ public sealed class FramePump : IDisposable
|
||||
// pooling change because its buffer's only consumer was its own release.
|
||||
renderSw.Restart();
|
||||
var scratch = AcquireScratch(scratchSize);
|
||||
frame = RenderScene(scene, compositorOptions, socialBarFrame, socialBarTop, scratch, TimedResolver);
|
||||
frame = RenderScene(scene, compositorOptions, socialBarFrame, socialBarTop, tickerFrame, scratch, TimedResolver);
|
||||
if (_transition is { Active: true } transition)
|
||||
{
|
||||
frame = transition.BlendFrame(frame);
|
||||
@@ -555,6 +563,7 @@ public sealed class FramePump : IDisposable
|
||||
CompositorOptions options,
|
||||
VideoFrame? socialBarFrame,
|
||||
int socialBarTop,
|
||||
VideoFrame? tickerFrame,
|
||||
byte[]? scratch = null,
|
||||
Func<SceneElement, VideoFrame?>? resolver = null)
|
||||
{
|
||||
@@ -562,7 +571,7 @@ public sealed class FramePump : IDisposable
|
||||
if (_sceneGraph == null)
|
||||
{
|
||||
var sw = System.Diagnostics.Stopwatch.StartNew();
|
||||
var frame = RenderFull(scene, options, socialBarFrame, socialBarTop, scratch, resolver);
|
||||
var frame = RenderFull(scene, options, socialBarFrame, socialBarTop, tickerFrame, scratch, resolver);
|
||||
ProbeRender("full-no-graph", scene, -1, sw.ElapsedMilliseconds, 0);
|
||||
return frame;
|
||||
}
|
||||
@@ -586,14 +595,14 @@ public sealed class FramePump : IDisposable
|
||||
if (baseFrame != null)
|
||||
{
|
||||
var frame = SceneCompositor.CompositeLayers(
|
||||
baseFrame, scene, split, resolver, options, socialBarFrame, socialBarTop, scratch: scratch);
|
||||
baseFrame, scene, split, resolver, options, socialBarFrame, socialBarTop, tickerFrame: tickerFrame, scratch: scratch);
|
||||
ProbeRender("base+layers", scene, split, swBase.ElapsedMilliseconds, 0);
|
||||
return frame;
|
||||
}
|
||||
|
||||
// No static base (first layer is dynamic or empty scene) — full render.
|
||||
var swFull = System.Diagnostics.Stopwatch.StartNew();
|
||||
var frame2 = RenderFull(scene, options, socialBarFrame, socialBarTop, scratch, resolver);
|
||||
var frame2 = RenderFull(scene, options, socialBarFrame, socialBarTop, tickerFrame, scratch, resolver);
|
||||
ProbeRender("full-render", scene, split, swFull.ElapsedMilliseconds, 0);
|
||||
return frame2;
|
||||
}
|
||||
@@ -605,6 +614,7 @@ public sealed class FramePump : IDisposable
|
||||
/// hit costs ~3ms of pure copy and a miss costs exactly the old full render.</summary>
|
||||
private VideoFrame RenderFull(
|
||||
Scene scene, CompositorOptions options, VideoFrame? socialBarFrame, int socialBarTop,
|
||||
VideoFrame? tickerFrame,
|
||||
byte[]? scratch, Func<SceneElement, VideoFrame?> resolver)
|
||||
{
|
||||
// 1:1 guard: the cache stores the OUTPUT bytes and scratch is source-sized —
|
||||
@@ -615,7 +625,7 @@ public sealed class FramePump : IDisposable
|
||||
var useCache = scratch != null && options.SourceRectWidth == options.OutputWidth
|
||||
&& options.SourceRectHeight == options.OutputHeight;
|
||||
var signature = useCache
|
||||
? BuildFullRenderSignature(scene, options, socialBarFrame, socialBarTop, resolver)
|
||||
? BuildFullRenderSignature(scene, options, socialBarFrame, socialBarTop, tickerFrame, resolver)
|
||||
: 0UL;
|
||||
|
||||
if (useCache && _fullCacheValid && signature == _lastFullSignature
|
||||
@@ -626,7 +636,7 @@ public sealed class FramePump : IDisposable
|
||||
return new VideoFrame(options.SourceRectWidth, options.SourceRectHeight, scratch);
|
||||
}
|
||||
|
||||
var rendered = _compositor.Render(scene, resolver, null, options, socialBarFrame, socialBarTop, scratch: scratch);
|
||||
var rendered = _compositor.Render(scene, resolver, null, options, socialBarFrame, socialBarTop, scratch: scratch, tickerFrame: tickerFrame);
|
||||
CacheRenders++;
|
||||
if (useCache)
|
||||
{
|
||||
@@ -659,6 +669,7 @@ public sealed class FramePump : IDisposable
|
||||
/// never trusted to key recycled content.</summary>
|
||||
private static ulong BuildFullRenderSignature(
|
||||
Scene scene, CompositorOptions options, VideoFrame? socialBarFrame, int socialBarTop,
|
||||
VideoFrame? tickerFrame,
|
||||
Func<SceneElement, VideoFrame?> resolver)
|
||||
{
|
||||
var h = 14695981039346656037UL; // FNV-1a offset basis
|
||||
@@ -671,6 +682,8 @@ public sealed class FramePump : IDisposable
|
||||
h = Mix(h, (ulong)socialBarTop);
|
||||
if (socialBarFrame != null) h = MixFrame(h, socialBarFrame);
|
||||
else h = Mix(h, 0xFFFFFFFFFFFFFFFFUL);
|
||||
if (tickerFrame != null) h = MixFrame(h, tickerFrame);
|
||||
else h = Mix(h, 0xFFFFFFFFFFFFFFFFUL);
|
||||
|
||||
var backdropResolved = false;
|
||||
foreach (var element in scene.Elements)
|
||||
|
||||
@@ -99,7 +99,8 @@ public partial class LayoutStore : IDisposable
|
||||
IsBackground, CaptureKey, ShowDesktop,
|
||||
ChatFontSize, ChatFontColor, ChatBackgroundColor, ChatMaxMessages,
|
||||
ChatShowTimestamps, ChatShowBadges, ChatMemberColor, ChatSuperChatColor,
|
||||
WebUri, MediaPath, MediaIsLooping, MediaVolume, MediaPlaybackState, SortOrder
|
||||
WebUri, MediaPath, MediaIsLooping, MediaVolume, MediaPlaybackState,
|
||||
AlertVideoAssetId, AlertVideoPath, AlertUseDefaultVideo, AlertShowTicker, AlertVideoVolume, SortOrder
|
||||
FROM Source ORDER BY SortOrder
|
||||
""";
|
||||
using var reader = cmd.ExecuteReader();
|
||||
@@ -145,10 +146,15 @@ public partial class LayoutStore : IDisposable
|
||||
: Enum.TryParse<MediaPlaybackState>(reader.GetString(29), out var ps)
|
||||
? ps
|
||||
: MediaPlaybackState.Stopped,
|
||||
AlertVideoAssetId = reader.IsDBNull(30) ? null : reader.GetString(30),
|
||||
AlertVideoPath = reader.IsDBNull(31) ? null : reader.GetString(31),
|
||||
AlertUseDefaultVideo = reader.IsDBNull(32) ? true : reader.GetInt32(32) != 0,
|
||||
AlertShowTicker = reader.IsDBNull(33) ? true : reader.GetInt32(33) != 0,
|
||||
AlertVideoVolume = reader.IsDBNull(34) ? 1.0 : reader.GetDouble(34),
|
||||
};
|
||||
if (!sourcesByScene.TryGetValue(sceneId, out var list))
|
||||
sourcesByScene[sceneId] = list = new List<(Source, int)>();
|
||||
list.Add((source, reader.GetInt32(30)));
|
||||
list.Add((source, reader.GetInt32(35)));
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -285,6 +285,40 @@ public partial class LayoutStore : IDisposable
|
||||
cmd.ExecuteNonQuery();
|
||||
}
|
||||
|
||||
// TASK 47: alert video config. The default clip asset id + the creator's
|
||||
// replacement path (a FILE reference, never a stored BLOB) + the pill,
|
||||
// ticker toggle and volume live on the row.
|
||||
if (!columns.Contains("AlertVideoAssetId"))
|
||||
{
|
||||
using var cmd = _connection.CreateCommand();
|
||||
cmd.CommandText = "ALTER TABLE Source ADD COLUMN AlertVideoAssetId TEXT;";
|
||||
cmd.ExecuteNonQuery();
|
||||
}
|
||||
if (!columns.Contains("AlertVideoPath"))
|
||||
{
|
||||
using var cmd = _connection.CreateCommand();
|
||||
cmd.CommandText = "ALTER TABLE Source ADD COLUMN AlertVideoPath TEXT;";
|
||||
cmd.ExecuteNonQuery();
|
||||
}
|
||||
if (!columns.Contains("AlertUseDefaultVideo"))
|
||||
{
|
||||
using var cmd = _connection.CreateCommand();
|
||||
cmd.CommandText = "ALTER TABLE Source ADD COLUMN AlertUseDefaultVideo INTEGER NOT NULL DEFAULT 1;";
|
||||
cmd.ExecuteNonQuery();
|
||||
}
|
||||
if (!columns.Contains("AlertShowTicker"))
|
||||
{
|
||||
using var cmd = _connection.CreateCommand();
|
||||
cmd.CommandText = "ALTER TABLE Source ADD COLUMN AlertShowTicker INTEGER NOT NULL DEFAULT 1;";
|
||||
cmd.ExecuteNonQuery();
|
||||
}
|
||||
if (!columns.Contains("AlertVideoVolume"))
|
||||
{
|
||||
using var cmd = _connection.CreateCommand();
|
||||
cmd.CommandText = "ALTER TABLE Source ADD COLUMN AlertVideoVolume REAL NOT NULL DEFAULT 1;";
|
||||
cmd.ExecuteNonQuery();
|
||||
}
|
||||
|
||||
// Rename legacy "Backdrop" display name to "Background" in existing rows
|
||||
using (var cmd = _connection.CreateCommand())
|
||||
{
|
||||
|
||||
@@ -88,13 +88,15 @@ public partial class LayoutStore : IDisposable
|
||||
ClipShape, IsMirrored, IsBackground, CaptureKey, ShowDesktop, SortOrder,
|
||||
ChatFontSize, ChatFontColor, ChatBackgroundColor, ChatMaxMessages,
|
||||
ChatShowTimestamps, ChatShowBadges, ChatMemberColor, ChatSuperChatColor,
|
||||
WebUri, MediaPath, MediaIsLooping, MediaVolume, MediaPlaybackState)
|
||||
WebUri, MediaPath, MediaIsLooping, MediaVolume, MediaPlaybackState,
|
||||
AlertVideoAssetId, AlertVideoPath, AlertUseDefaultVideo, AlertShowTicker, AlertVideoVolume)
|
||||
VALUES ($id, $sceneId, $assetId, $type, $name, $isEnabled,
|
||||
$x, $y, $w, $h, $opacity, $monitor,
|
||||
$clip, $mirrored, $isBackground, $captureKey, $showDesktop, $sort,
|
||||
$chatFontSize, $chatFontColor, $chatBgColor, $chatMaxMsgs,
|
||||
$chatTimestamps, $chatBadges, $chatMemberColor, $chatSuperChatColor,
|
||||
$webUri, $mediaPath, $mediaIsLooping, $mediaVolume, $mediaPlaybackState)
|
||||
$webUri, $mediaPath, $mediaIsLooping, $mediaVolume, $mediaPlaybackState,
|
||||
$alertVideoAssetId, $alertVideoPath, $alertUseDefaultVideo, $alertShowTicker, $alertVideoVolume)
|
||||
""";
|
||||
cmd.Transaction = tx;
|
||||
var idP = cmd.Parameters.Add("$id", SqliteType.Text);
|
||||
@@ -128,6 +130,11 @@ public partial class LayoutStore : IDisposable
|
||||
var mediaIsLoopingP = cmd.Parameters.Add("$mediaIsLooping", SqliteType.Integer);
|
||||
var mediaVolumeP = cmd.Parameters.Add("$mediaVolume", SqliteType.Real);
|
||||
var mediaPlaybackStateP = cmd.Parameters.Add("$mediaPlaybackState", SqliteType.Text);
|
||||
var alertVideoAssetIdP = cmd.Parameters.Add("$alertVideoAssetId", SqliteType.Text);
|
||||
var alertVideoPathP = cmd.Parameters.Add("$alertVideoPath", SqliteType.Text);
|
||||
var alertUseDefaultVideoP = cmd.Parameters.Add("$alertUseDefaultVideo", SqliteType.Integer);
|
||||
var alertShowTickerP = cmd.Parameters.Add("$alertShowTicker", SqliteType.Integer);
|
||||
var alertVideoVolumeP = cmd.Parameters.Add("$alertVideoVolume", SqliteType.Real);
|
||||
|
||||
// Source and WebcamSceneConfig share ONE z-order space per scene:
|
||||
// each element gets its index within scene.Elements regardless of
|
||||
@@ -170,6 +177,11 @@ public partial class LayoutStore : IDisposable
|
||||
mediaIsLoopingP.Value = source.MediaIsLooping ? 1 : 0;
|
||||
mediaVolumeP.Value = source.MediaVolume;
|
||||
mediaPlaybackStateP.Value = source.MediaPlaybackState.ToString();
|
||||
alertVideoAssetIdP.Value = (object?)source.AlertVideoAssetId ?? DBNull.Value;
|
||||
alertVideoPathP.Value = (object?)source.AlertVideoPath ?? DBNull.Value;
|
||||
alertUseDefaultVideoP.Value = source.AlertUseDefaultVideo ? 1 : 0;
|
||||
alertShowTickerP.Value = source.AlertShowTicker ? 1 : 0;
|
||||
alertVideoVolumeP.Value = source.AlertVideoVolume;
|
||||
cmd.ExecuteNonQuery();
|
||||
}
|
||||
}
|
||||
@@ -300,7 +312,15 @@ public partial class LayoutStore : IDisposable
|
||||
|
||||
using (var cmd = _connection.CreateCommand())
|
||||
{
|
||||
cmd.CommandText = "DELETE FROM Asset WHERE Id NOT IN (SELECT AssetId FROM Source WHERE AssetId IS NOT NULL);";
|
||||
cmd.CommandText = """
|
||||
DELETE FROM Asset WHERE Id NOT IN (
|
||||
SELECT AssetId FROM Source WHERE AssetId IS NOT NULL
|
||||
UNION
|
||||
SELECT AlertVideoAssetId FROM Source WHERE AlertVideoAssetId IS NOT NULL
|
||||
UNION
|
||||
SELECT Value FROM Settings WHERE Key = 'AlertDefaultVideoAssetId'
|
||||
);
|
||||
""";
|
||||
cmd.Transaction = tx;
|
||||
cmd.ExecuteNonQuery();
|
||||
}
|
||||
|
||||
@@ -304,4 +304,26 @@ public partial class LayoutStore : IDisposable
|
||||
insert.Parameters.AddWithValue("$value", value);
|
||||
insert.ExecuteNonQuery();
|
||||
}
|
||||
|
||||
internal const string AlertDefaultVideoKey = "AlertDefaultVideoAssetId";
|
||||
|
||||
/// <summary>Stamps the shipped stream-alert placeholder clip (TASK 47): like any
|
||||
/// source image it lives as a BLOB in the internal Asset table, tracked by this
|
||||
/// Setting so the Save pruner keeps it alive. The creator's OWN replacement
|
||||
/// videos are never stored — they are plain path references (the 2026-09-26
|
||||
/// decision: the file the user picks stays at its real location on disk).
|
||||
/// <c>UpsertAsset</c> is hash-keyed, so re-stamping the same bytes returns the
|
||||
/// same id.</summary>
|
||||
public string EnsureAlertDefaultVideo(byte[] data)
|
||||
{
|
||||
var saved = GetSetting(AlertDefaultVideoKey);
|
||||
if (!string.IsNullOrWhiteSpace(saved) && GetAssetBytes(saved) != null)
|
||||
return saved;
|
||||
var id = UpsertAsset(data, 0, 0);
|
||||
UpsertSetting(AlertDefaultVideoKey, id);
|
||||
return id;
|
||||
}
|
||||
|
||||
/// <summary>The stored default clip's Asset id, or null when never stamped.</summary>
|
||||
public string? GetAlertDefaultVideoId() => GetSetting(AlertDefaultVideoKey);
|
||||
}
|
||||
|
||||
+8
-6
@@ -11,9 +11,10 @@ External-facing logic: YouTube API, persistence. See
|
||||
| `YouTubeChatService.cs` | Polls `liveChat/messages`, raises `MessageReceived`; `IDisposable`. **TASK 43 rewrite (2026-09-24):** re-arms its one-shot poll on the server's `pollingIntervalMillis` (streamList semantics — reconnect on that cadence with `nextPageToken`, clamp 1000–6000ms, `maxResults=2000`); `ParsePage` (internal static seam) decodes **all six** event types into `ChatMessage.Kind`/fields (superChat / superSticker / newSponsor / membershipGifting / giftMembershipReceived / memberMilestoneChat + milestone months best-effort regex); optional `HttpClient` ctor seam endures; `InjectSimulatedMessage` (TASK 41) preserved |
|
||||
| `ChatBoxRenderer.cs` | Chat overlay rasterizer (WPF FormattedText → BGRA `VideoFrame`), live rows + Super Chat/member styling |
|
||||
| `ChatOverlayLayer.cs` | **The chat component (Commit G):** owns the message buffer, `ChatBoxRenderer`, fade/mock timers, per-source preview renders, live `RenderFrame`; raster-on-change cache pattern (OBS text-source contract) |
|
||||
| `AlertRenderer.cs` | **The alert rasterizer (TASK 43, 2026-09-24):** renders one event's celebration animation into a BGRA `VideoFrame` — SIX DISTINCT animations (SuperChat slide-up/shine/count-up, SuperSticker scale-pop, NewMember drop-in/flash-band, MemberGift slide-left/chip-fan, GiftReceived confetti scale-in, MemberMilestone rise/growth-bar), enter/hold/exit phases, brand line on every card |
|
||||
| `AlertOverlayLayer.cs` | **The native alert-box component (TASK 43):** event queue (cap 10) + 33ms dispatch ticker + `RenderFrame` cache-first live path + `UpdatePreview`; **`Advance(double)` is the deterministic test clock** (tests never start the timer); `ChatEventKind.None` rows (sub mentions) never enqueue; idle = transparent (null frame) |
|
||||
| `LayoutStore.cs` | SQLite persistence (`Microsoft.Data.Sqlite`) at `%APPDATA%\ytLlive\ytLlive.db`; assets stored as BLOBs keyed by SHA-256 content hash; save/open layout files; schema `user_version` 6 (`Source.ClipShape`/`IsMirrored` via `ALTER TABLE` for pre-v2 DBs; v3 = singleton `Webcam` + per-scene `WebcamSceneConfig`, migrated idempotently **without backfill** — the stale `Source.DeviceId` column remains but is no longer read/written; v4 = `WebcamSceneConfig.RectWidth`/`RectHeight`, the pre-Round rect for the round-to-rect restore; v5 = `Source.IsBackdrop` + `Source.CaptureKey`, the live-capture backdrop; v6 = `Scene.HasBackdrop` — Live-only policy, one-time backfill turns Starting/BRB/Chat/Ending off + drops their backdrop sources; `MainViewModel.NormalizeBackgrounds` re-normalizes on every load — TASK 25: one locked "Background" per canonical screen (static art everywhere, DisplayCapture on Live), duplicates dropped, index 0, orphaned `BackgroundUseDefault_*`/`BackgroundPath_*` Settings keys purged at save). Settings key/value table holds app state: `MicSourceName`, `ReusableStream` (TASK 9 — the cached reusable stream's id/address/name so the pump has its RTMP URL at startup) |
|
||||
| `AlertRenderer.cs` | **The alert rasterizer (TASK 43, 2026-09-24):** renders one event's celebration animation into a BGRA `VideoFrame` — SIX DISTINCT animations (SuperChat slide-up/shine/count-up, SuperSticker scale-pop, NewMember drop-in/flash-band, MemberGift slide-left/chip-fan, GiftReceived confetti scale-in, MemberMilestone rise/growth-bar), enter/hold/exit phases, brand line on every card. **Still the fallback** when no alert clip is playable (TASK 47) |
|
||||
| `AlertOverlayLayer.cs` | **The native alert-box component (TASK 43; video branch TASK 47):** event queue (cap 10) + 33ms dispatch ticker + `RenderFrame` cache-first live path + `UpdatePreview`; **`Advance(double)` is the deterministic test clock** (tests never start the timer); `ChatEventKind.None` rows (sub mentions) never enqueue; idle = transparent (null frame). Since TASK 47 a clip branch owns `IAlertClipDecoder` per play — ctor seams `(clipDecoderFactory, clipPathResolver, audioSink)`, fade in/out over `FadeDurationSeconds = 0.30` (straight-alpha copies + volume×fade audio), EOF freeze-frame → drain, and `AlertTickerFrame` composes the scrolling message strip |
|
||||
| `AlertClipDecoder.cs` | **Per-play alert video decoder (TASK 47):** `IAlertClipDecoder` (Start/Stop/Dispose + `FrameAvailable`/`AudioReady`/`Completed`) + ffmpeg child emitting `rawvideo bgra` (`-vf scale=W:H`, read via `RawVideoFrameReader`) and `f32le -ar 48000 -ac 2` on a **carry-buffer** audio loop (float boundary straddles never drop samples), both real-time paced; bgra is opaque (layer owns alpha). One-shot lifetime — created per alert, disposed at drain; tests inject a fake |
|
||||
| `LayoutStore.cs` | SQLite persistence (`Microsoft.Data.Sqlite`) at `%APPDATA%\ytLlive\ytLlive.db`; assets stored as BLOBs keyed by SHA-256 content hash; save/open layout files; schema `user_version` 10 (`Source.ClipShape`/`IsMirrored` via `ALTER TABLE` for pre-v2 DBs; v3 = singleton `Webcam` + per-scene `WebcamSceneConfig`, migrated idempotently **without backfill** — the stale `Source.DeviceId` column remains but is no longer read/written; v4 = `WebcamSceneConfig.RectWidth`/`RectHeight`, the pre-Round rect for the round-to-rect restore; v5 = `Source.IsBackdrop` + `Source.CaptureKey`, the live-capture backdrop; v6 = `Scene.HasBackdrop` — Live-only policy, one-time backfill turns Starting/BRB/Chat/Ending off + drops their backdrop sources; v9→v10 (TASK 39) = `YppSnapshot` history table; all later columns stay additive guarded ALTERs — TASK 47 adds the five alert `Source` columns (`AlertVideoAssetId`/`AlertVideoPath`/`AlertUseDefaultVideo`/`AlertShowTicker`/`AlertVideoVolume`) without a version bump; `MainViewModel.NormalizeBackgrounds` re-normalizes on every load — TASK 25: one locked "Background" per canonical screen (static art everywhere, DisplayCapture on Live), duplicates dropped, index 0, orphaned `BackgroundUseDefault_*`/`BackgroundPath_*` Settings keys purged at save). Settings key/value table holds app state: `MicSourceName`, `ReusableStream` (TASK 9 — the cached reusable stream's id/address/name so the pump has its RTMP URL at startup), `AlertDefaultVideoAssetId` (TASK 47 — the stamped built-in alert clip's asset row; save's prune exempts it via UNION) |
|
||||
| `VideoFrame.cs` | Normalized CPU frame seam (`Width`/`Height`/tightly-packed BGRA `byte[]`) — the only pixel type the rest of the app knows about; future capture sources (screen, background-removed webcam) feed the same seam |
|
||||
| `CameraDeviceInfo.cs` | `(Id, DisplayName)` for a physical capture device |
|
||||
| `ICameraEnumerator.cs` | `GetCamerasAsync()` — seam so the picker/`CameraManager` never touch WinRT (tests inject fakes) |
|
||||
@@ -34,13 +35,14 @@ External-facing logic: YouTube API, persistence. See
|
||||
| `ScreenCaptureSourceFactory.cs` | `Resolve(key)` parses `monitor:<n>` / `window:<hwnd>` / `picker:<name>` into a source; `PickAsync()` shows the OS `GraphicsCapturePicker` and returns the `picker:` key (transient — a reload falls back to auto-detection) |
|
||||
| `WebView2Manager.cs` | **Web-source ownership (TASK 17, composition-capture since 2026-09-14):** per registered `Source { Type: WebSource }` it owns a `CoreWebView2CompositionController` + its visual tree (`RootVisualTarget` child under a 1920×1080 root `ContainerVisual` on one `Windows.UI.Composition.Compositor` created on the UI thread via the CoreMessaging `CreateDispatcherQueueController` P/Invoke), a wrapped `WebCaptureFrameSource`, and the shared crop-sized `WriteableBitmap` preview (dispatcher-coalesced copy at Render priority, `PreviewBitmapChanged` raised only on recreation). Nav + `TransparentBackgroundScript` injection unchanged. Public ctor `(Dispatcher)`; internal seam ctor `(Dispatcher, Func<string, IScreenCaptureSource>?)` for hermetic tests (no WinRT). Caller: `MainViewModel.InitWebView2()` (now invoked from `MainWindow_Loaded` — the parent HWND must exist). See `../ai.md` "Slice 14" |
|
||||
| `WebCaptureFrameSource.cs` | The `IScreenCaptureSource` for a web session: owns the `GraphicsCaptureItem` (`CreateFromVisual`) + a free-threaded `Direct3D11CaptureFramePool` (2 buffers) + session; readback via `SoftwareBitmap.CreateCopyFromSurfaceAsync` (BGRA, **`BitmapAlphaMode.Straight`** — the compositor blends straight alpha); per-frame `FindContentBounds` alpha-bbox (internal static, unit-tested) stamps `VideoFrame.CropBounds`; ring up to 2 fresh frames + Epoch reuse; `IsOpaque=false`. Mirror of `ScreenCaptureFrameSource` |
|
||||
| `Compositor/SceneCompositor.cs` | **The output compositor (TASK 4 ship step 1)**: renders a scene into the encoder's master `VideoFrame` (tightly-packed BGRA8), mirroring the XAML preview minus editing chrome — backdrop → background → elements (`UniformToFill` cover-crop, round clip, mirror, opacity, border) → branding flash → social bar (optional `socialBarFrame` + `socialBarTop` in master space, blitted last so the bar sits above the flash; the old `BlitFlash` generalized to `BlitOverlay` with source offsets). Pure and WPF-free: frames injected via a `Func<SceneElement, VideoFrame?>` resolver (webcam → DeviceId, image → AssetId, backdrop → CaptureKey); output sized by `CompositorOptions` (16:9 = full master 1:1; vertical 9:16 = 607×1080 crop → 1080×1920 bilinear). Preview stays XAML (editing view); this is the output view — see `ai.md` "Scene compositor" |
|
||||
| `Compositor/SceneCompositor.cs` | **The output compositor (TASK 4 ship step 1)**: renders a scene into the encoder's master `VideoFrame` (tightly-packed BGRA8), mirroring the XAML preview minus editing chrome — backdrop → background → elements (`UniformToFill` cover-crop, round clip, mirror, opacity, border) → branding flash → social bar (optional `socialBarFrame` + `socialBarTop` in master space, blitted last so the bar sits above the flash; the old `BlitFlash` generalized to `BlitOverlay` with source offsets). **TASK 47:** `Render` + `CompositeLayers` take a trailing `tickerFrame` blitted at (0,0) after the bar — the alert ticker is a message-scrolling dynamic overlay, never baked. Pure and WPF-free: frames injected via a `Func<SceneElement, VideoFrame?>` resolver (webcam → DeviceId, image → AssetId, backdrop → CaptureKey); output sized by `CompositorOptions` (16:9 = full master 1:1; vertical 9:16 = 607×1080 crop → 1080×1920 bilinear). Preview stays XAML (editing view); this is the output view — see `ai.md` "Scene compositor" |
|
||||
| `Compositor/CompositorOptions.cs` | The active tier's output rect (source space over the 1920×1080 master, integer-aligned — `MainViewModel.OutputRectX` can be 656.5) + target W×H |
|
||||
| `Compositor/StretchMath.cs` | Pure pixel math: the WPF `UniformToFill` cover-crop, clamped bilinear sample/scale (unit-tested half of the compositor) |
|
||||
| `Compositor/StaticPixelCache.cs` | Asset bytes → cached BGRA8 `VideoFrame`, decoded once per content hash — the output path's raw-pixel source (the preview uses ImageCache's `BitmapImage`) |
|
||||
| `Encoder/IFfmpegLocator.cs` | **TASK 4 ship step 2 seam**: resolves an absolute path to `ffmpeg.exe` (PATH first → cached `%APPDATA%\ytLlive\tools\ffmpeg.exe` → pinned BtbN LGPL-shared zip download). The encoder step and tests fake it |
|
||||
| `SocialValidator.cs` | `ISocialValidator` seam + `HttpSocialValidator` default: validates a social handle/URL by constructing the canonical profile URL and issuing an HTTP GET — the "act of validation" before adding to the bar. 200/redirect = exists; 404/connection failure = rejected. Fediverse `@user@domain` additionally does a best-effort nodeinfo lookup (`/.well-known/nodeinfo` → `software.name`) so the entry can show the instance's real logo; nodeinfo failure still validates (generic fediverse glyph). If the identity domain's nodeinfo is blocked (SSO) but the bare root 302s to the real instance (YunoHost default-app subdomains), the lookup follows the redirect and asks that host. Since the bar bug-fix branch the resolution **probes well-known subdomains** (`mastodon.`, `social.`, ... order in `FediverseSubdomainCandidates`) when both the identity domain and its redirect come up empty, bounded by a ~15s linked-CTS budget; `ResolveFediverseSoftwareAsync(domain, ct)` on the seam powers the load-time heal. Constructor takes optional `HttpClient` + `Action<string>` log for tests |
|
||||
| `Compositor/SocialBarRenderer.cs` | **Social bar output strip** (bar bug-fix branch): rasterizes the global social bar into a transparent straight-alpha BGRA8 `VideoFrame` strip (1920 wide, 40px content + 24px glow pad) via `RenderTargetBitmap` — one Path (logo) + one TextBlock (handle) per entry, white on transparent with the preview's green `#2ecc71` glow baked in (Pbgra32 → straight-alpha unpremultiply). UI thread only; the frame is immutable afterwards so the `FramePump` reads it from its own thread |
|
||||
| `Compositor/AlertTickerRenderer.cs` | **Alert message ticker strip (TASK 47):** marquee overlay rendered per call by pure byte-math onto a 1920×48 transparent BGRA frame — the text pill ("Author — Kind · amount") is rasterized once per text on the UI thread (cached, unpremultiplied straight alpha) and scrolled at 140px/s with a repeat-gap wrap, so the pump can call `Render(elapsed, text)` from its own thread without locking. **Dynamic overlay — never baked** (unlike the social bar) |
|
||||
| `Encoder/EncoderOptions.cs` | One go-live's encoder config: full `RtmpUrl` (ingest + stream key), W×H/FPS/bitrate from the quality tier, `VideoEncoder` (null = probe), audio sample rate/channels, `GopSize` = FPS×4 (the ≤4s keyframe bound) |
|
||||
| `Encoder/IFfmpegEncoder.cs` | **TASK 4 ship step 3 seam**: start/feed/stop the live encoder; `HealthUpdated` (parsed `StreamHealth`), `ProcessFailed` on unexpected non-zero exit. Constructor-injected locator + process factory (tests fake both) |
|
||||
| `Encoder/FfmpegEncoder.cs` | The default encoder: spawn `ffmpeg.exe` (probe `-encoders` first → hardware NVENC/QSV/AMF, OpenH264 fallback — never libx264), feed raw BGRA frames into stdin, parse `-stats` lines into `StreamHealth`, graceful stop via stdin EOF + 10s kill watchdog. Constructed by `MainViewModel` since ship step 5 (`new FfmpegEncoder(new FfmpegLocator())`), driven by the `FramePump` |
|
||||
@@ -49,7 +51,7 @@ External-facing logic: YouTube API, persistence. See
|
||||
| `Encoder/FfmpegArgs.cs` | Pure FFmpeg command-line builder: rawvideo `pipe:0` input, `anullsrc` silent audio (WASAPI replaces it), H.264+AAC, closed GOP (`-g fps×4 -keyint_min -sc_threshold 0 -bf 0`, `yuv420p`), FLV → RTMP |
|
||||
| `Encoder/FfmpegProgressParser.cs` | Pure parser for `frame=/fps=/size=/time=/bitrate=` stats lines → `FfmpegProgress` |
|
||||
| `Encoder/FfmpegEncoderPicker.cs` | Pure H.264 encoder picker from `-encoders` output: NVENC → QSV → AMF → OpenH264; **never returns libx264** (GPL) |
|
||||
| `Encoder/FramePump.cs` | **TASK 4 ship step 5**: the live frame producer — while live it snapshots the active scene each tick, resolves every element to its latest frame (`Func<SceneElement, VideoFrame?>` resolver), composites it into the tier's output frame, and paces frames into the encoder at the tier's FPS. All collaborators constructor-injected seams; free of WPF and the capture managers. `StartAsync` never throws (failures log + `Failed`); no RTMP URL = encoder skipped; `StopAsync` stops the encoder before awaiting the loop (backpressure deadlock); `ProcessFailed` self-stops. Since the bar bug-fix branch it takes an optional `socialBar: Func<(VideoFrame? Frame, SocialBarPosition Position)>?` seam — a pre-rendered bar strip composited last (above the flash) at the top edge or `SourceRectHeight − bar height`. See `ai.md` "Live frame pipeline" |
|
||||
| `Encoder/FramePump.cs` | **TASK 4 ship step 5**: the live frame producer — while live it snapshots the active scene each tick, resolves every element to its latest frame (`Func<SceneElement, VideoFrame?>` resolver), composites it into the tier's output frame, and paces frames into the encoder at the tier's FPS. All collaborators constructor-injected seams; free of WPF and the capture managers. `StartAsync` never throws (failures log + `Failed`); no RTMP URL = encoder skipped; `StopAsync` stops the encoder before awaiting the loop (backpressure deadlock); `ProcessFailed` self-stops. Since the bar bug-fix branch it takes an optional `socialBar: Func<(VideoFrame? Frame, SocialBarPosition Position)>?` seam — a pre-rendered bar strip composited last (above the flash) at the top edge or `SourceRectHeight − bar height`. **TASK 47:** adds an `alertTicker: Func<VideoFrame?>?` ctor seam (parallel to `socialBar`), read per tick and threaded through `RenderScene`/`RenderFull` + **mixed into `BuildFullRenderSignature`** so the ticker scroll changes the cache key. See `ai.md` "Live frame pipeline" |
|
||||
| `Encoder/FfmpegLocator.cs` | The default locator: PATH probe wins, then the cache, then pull+extract (`ffmpeg.exe` **+ the `libav*.dll` family** via a staging dir, failures via `AppLog`). Pinned to BtbN `autobuild-2026-08-09-13-03` **lgpl-shared** (dynamic linking = LGPL compliance without static-relink §6 material; no GPL libx264; NVENC/QSV/AMF + libopenh264 + native AAC — see `ai.md` → Licensing). Search dirs / tools dir / downloader constructor-injected for hermetic tests. Wired into the encoder since ship step 5 |
|
||||
| `INotificationService.cs` | **Toast seam (TASK 24)**: `Show(title, message, AppNotificationSeverity)` + `AppNotificationSeverity { Info, Success, Warning, Error }` + `Info/Success/Warning/Error` extension methods — non-blocking user-facing messages; VMs/services raise through this so failures surface without MessageBox blocking the UI thread |
|
||||
| `NotificationService.cs` | The default seam impl (Notification.Wpf 11.0.0, MIT): wraps a library `NotificationManager`; routes to the in-window `NotificationArea` named `ToastArea` (MainWindow bottom-right above the footer); lifetimes via the pure `BuildRequest` mapping + `Cards` tint table — Info/Success ~4s, Warning ~8s auto-dismiss, Error sticky (`NeverExpires()` = `TimeSpan.MaxValue`, never null); dark card tints matching the health-banner language; all Show calls marshal to the captured Dispatcher. See [`../ai.md`](../ai.md) "Toast notifications" |
|
||||
@@ -57,7 +59,7 @@ External-facing logic: YouTube API, persistence. See
|
||||
| `Audio/AudioSample.cs` | One captured chunk: interleaved PCM float (-1..1) + sample rate + channels |
|
||||
| `Audio/WasapiLoopbackAudioSource.cs` | Desktop/game capture: NAudio `WasapiLoopbackCapture` on the default render device — automatic at unity; the game bar's meter consumes it |
|
||||
| `Audio/WasapiMicAudioSource.cs` | Mic capture: NAudio `WasapiCapture` with the device resolved by `FriendlyName` matching `MicSourceName` (DisplayName only), default capture endpoint fallback; name provider `Func<string?>` re-read at each `Start` so a device picked mid-session takes effect immediately (the mixer restarts the mic on pick) |
|
||||
| `Audio/AudioMixer.cs` | Owns both sources; capture starts once at startup (`MainViewModel.StartMicCaptureAsync`) and stops on `Shutdown` — NOT go-live (preview monitoring). Mic samples → `AudioLevelMeter` → `MicLevelChanged`; loopback samples → the game bar's meter via `LoopbackLevelChanged`. Surfaces mic connection state: `MicConnected`/`MicFailed` (drives the status dot) + `RestartMic()` (device swap mid-session, keeps loopback). Failures log via `AppLog`; mic failure zeroes the meter, loopback failure doesn't kill the mic. Note: the meter `Push` is unconditional (the `?.` on the event would otherwise skip the argument when nothing is subscribed) |
|
||||
| `Audio/AudioMixer.cs` | Owns both sources; capture starts once at startup (`MainViewModel.StartMicCaptureAsync`) and stops on `Shutdown` — NOT go-live (preview monitoring). Mic samples → `AudioLevelMeter` → `MicLevelChanged`; loopback samples → the game bar's meter via `LoopbackLevelChanged`. Surfaces mic connection state: `MicConnected`/`MicFailed` (drives the status dot) + `RestartMic()` (device swap mid-session, keeps loopback). Failures log via `AppLog`; mic failure zeroes the meter, loopback failure doesn't kill the mic. Note: the meter `Push` is unconditional (the `?.` on the event would otherwise skip the argument when nothing is subscribed). **TASK 47 — alert audio:** `EnqueueAlertAudio` writes a dedicated 8s stereo-48k ring (`AlertBufferSeconds`), drained in `FillAndMix` pre-limiter and added at **unity (never ducked)**; `StartLive` clears the ring |
|
||||
| `Audio/AudioLevelMeter.cs` | Pure smoothed RMS level (0..1): `Push(AudioSample)` + `Reset` — the unit-tested math behind `AudioLevel` and `GameAudioLevel`. `ToDisplay(float)` maps the raw linear RMS onto the meter's display scale (−60..0 dBFS spread across 0..1, with **+10 dB input amplification** so real speech peaks hit the red zone at maxed volume) — real speech/game RMS (~0.01..0.1) would otherwise leave a flat scale looking dead; ≤0.001 linear reads as zero (never idles on background noise) |
|
||||
| `Audio/WaveToFloat.cs` | Pure WASAPI buffer → float conversion: IEEE float 32-bit direct, PCM 16-bit normalized, `WaveFormatExtensible` IEEE-float subformat GUID, trailing partial samples ignored |
|
||||
| `Audio/MasterLimiter.cs` | **Master output limiter**: a **−1 dBFS ceiling** (`Ceiling = 0.891`) applied after the live mix so the honest gains can never sum past 0 dBFS and clip the AAC encode. Instant attack per frame (a hot frame is scaled exactly to the ceiling — no overshoot), smoothed release toward unity so loud passages don't pump. Pure — unit-tested |
|
||||
|
||||
Reference in New Issue
Block a user