LlamaCasty/TASKS/task-47-alert-videos.md

TASK 47 — Alert box video (TASK 43 follow-up): creator-replaceable alert clip + read-time fade + auto ticker

Catalog: TASKS.md. Status: ✅ SHIPPED 2026-09-26 — the TASK 43 alert box grows a real video celebration: a built-in mp4 loops on every alert unless the creator picks their own clip (referenced from disk, never stored in the DB), the clip fades in/out in ~0.3s (alpha envelope on the video + volume on the audio, mixed into the live stream at unity — no duck), and an auto-composed message ticker ("Funder — Super Chat · $10.00") scrolls across the very top of the frame, toggleable. The six TASK 43 animations remain the intrinsic fallback when no asset is playable.

Provenance (recorded per the feature-provenance rule)

• 2026-09-24, the creator asked (right after TASK 43 shipped) — "make the alerts animate like a video" / "I want to play my own video for alerts." The TASK 43 unit closed with this as the explicit next step.
• Lineage: Research (websearch: how do OBS/StreamElements/Streamlabs alert boxes show per-alert videos?) settled the shape, cited in the commit message:
  • Alert videos play per-alert (a fresh decoder per event), never a resident media player — the OBS "media source reset on activation" model, not a playlist.
  • The clip must not block: decode is a background pipe (ffmpeg child, rawvideo frames + f32le audio), paced real-time, so alerts never stall the 60fps frame pump.
  • A freeze-frame at clip EOF fades out rather than hard-dropping (Streamlabs' "end by fade" option) — the tail of a long clip dissolves without a new spawn.
  • The custom file is served by path, not bytes (OBS media-source semantics; a read-at-play clip lets the creator re-edit the file without a re-import). Decided with the creator: "never store the video in the layout." The built-in default is the one BLOB, stamped into the internal Asset table at startup.
• Creator rulings (asked once, held): custom video + six-animation fallback (not either/or); ticker on top of the canvas (very top, full width); no ducking — "don't lower my game audio for a tip" (the TASK 43 planner's auto-duck idea was rejected on the floor); a per-alert Volume slider instead.

Product model (as shipped)

• Five new Source props (schema v10, additive guarded ALTERs): AlertVideoPath (custom file), AlertUseDefaultVideo (bool, default true — the built-in clip), AlertShowTicker (bool, default true), AlertVideoVolume (0–1, default 1). AlertVideoAssetId (internal reference to the stamped default asset row).
• Built-in clip: Assets/alert-default.mp4 (~4.9MB, branded 2.5s loop, included as a csproj Resource). At startup MainViewModel.StampDefaultAlertVideo() upserts its bytes into the Asset table and records the row id in the settings key AlertDefaultVideoAssetId (AlertDefaultVideoKey). The prune step exempts that key via a UNION so the default never gets cleaned.
• Per-alert resolver (ResolveAlertClipPath in MainViewModel.Chat.cs): custom path wins ONLY when AlertUseDefaultVideo == false AND the file exists; otherwise the thumbnailed built-in is materialized once to %TEMP%\ytLive-alert-{id}.mp4; failure → the six AlertRenderer animations play (fallback, not an error).
• Services/AlertClipDecoder.cs — IAlertClipDecoder (Start/Stop/Dispose + FrameAvailable(Action<VideoFrame>) / AudioReady(Action<AudioSample>) / Completed(Action)) + AlertClipDecoder impl: ffmpeg -loglevel error child emitting rawvideo bgra (-vf scale=W:H) on one pipe and f32le -ar 48000 -ac 2 on another; RawVideoFrameReader (the TASK 21 media-source reader) on the video pipe, a carry-buffer loop on the audio pipe (float-boundary straddles never drop a sample — the 3-byte PCM16↔4-byte-f32 mismatch that broke the first draft), both real-time paced. Per-play lifetime: the layer creates one per alert and disposes it at drain. Prefer Start-synced TrackFrameRate — the frame pipe throttles to vfr timestamps; audio paces itself. ffmpeg bgra is opaque (alpha=255) — the layer owns all alpha.
• Services/AlertOverlayLayer.cs — clip branch beside the animation path: Enqueue → BeginClip (factory + path + box dims seam), Advance advances _elapsed (fade-in frame copies scale alpha 0→255 straight-source over the box), events forward to the audio sink scaled by volume × fade, EOF → freeze-frame + fade-out over the same FadeDurationSeconds = 0.30 envelope → drain (StopClip + dispose, idle null again). Ticker: AlertTickerFrame composes "Author — Kind · amount" from the live message and scrolls it.
• Services/Compositor/AlertTickerRenderer.cs — marquee strip: 1920×48 transparent BGRA, text-pill rasterized once per text on the UI thread (cached, unpremultiplied), then pure byte-math per call (scroll pos = elapsed × 140px/s % cycle, pill drawn straight-alpha over transparent with a repeat-gap copy), so the pump can call it from its own thread every tick with no locking.
• Ticker threading — dynamic overlay like the social bar, but never baked (the social bar IS baked into BakeStaticBase; the ticker is a per-frame marquee): SceneCompositor.Render + CompositeLayers take a trailing tickerFrame param (blitted at 0,0 after the social bar), threaded by FramePump._alertTicker (Func<VideoFrame?> seam, ctor param) through RenderScene/RenderFull/ BuildFullRenderSignature (mixed into the cache signature so a ticker change invalidates — ticker is distinct from static-cache-eligible content).
• Audio path — AudioMixer.EnqueueAlertAudio uses a dedicated ring (AlertBufferSeconds = 8 at 48k stereo): resample/upmix to stereo 48k, drained pre-limiter in FillAndMix and added at unity (never ducked); StartLive clears it. The layer applies volume × fade per sample before forwarding.
• UI — Stream Alerts section (LeftPanel, visible only on an AlertBox): "Built-in video" pill; custom path box + Browse…; "Reset to built-in"; "Message ticker" pill; "Volume" slider with % readout. Wiring: BrowseAlertVideoCommand + ResetAlertVideoCommand (CanExecute gates on Source { Type: AlertBox }); browse = OpenFileDialog (mp4/mov/webm), result writes AlertVideoPath + switches the pill off.

Test (Good Dog — ONE integration test per change)

ytLive.Tests/AlertLayerVideoTests.cs (RealApp STA host, real WPF raster) — AlertVideo_PlaysCustomClip_FadesReadTime_TickerScrolls_ForwardsScaledAudio_DrainsIdle drives a fake IAlertClipDecoder (no ffmpeg in tests) through the whole lifecycle in one pass: the custom path wins per-source (resolution seam asserts box W/H on the factory + same source to the resolver); the decoder spawns on play (StartCount 1) and dies at drain (DisposeCount 1); fade-in rides the 0.3s envelope (alpha 0 → 127 at 0.15s → 255, straight-alpha copy); pre-fade audio is dropped while ramp audio forwards at volume × fade (0.8 × 0.5 → [0.2, −0.1]) and full-fade at volume only; the ticker renders non-null with pixels and scrolls between two pump ticks while idle returns null; EOF freeze → fade-out 127 → drain to IsPlaying false + null frame + disposed decoder.

Catches a REAL bug the animation path never had: the clip branch of Advance failed to clear _current before AdvanceToNext, so after the fade-out the layer stayed IsPlaying and re-rendered the finished alert via the animation fallback. The clean frame literally forced the Assert.False(layer.IsPlaying) to see it.

Files (declared scope)

• Models/Source.cs (five alert props + IsAlertBox), Services/LayoutStore.Migrations.cs (guarded ALTERs), Services/LayoutStore.Save.cs (INSERT + prune UNION), Services/LayoutStore.Load.cs (reader indices), Services/LayoutStore.Settings.cs (AlertDefaultVideoKey + asset stamps)
• Assets/alert-default.mp4 + ytLive.csproj (Resource include)
• NEW Services/AlertClipDecoder.cs, NEW Services/Compositor/AlertTickerRenderer.cs
• Services/AlertOverlayLayer.cs (clip branch + fade + ticker + sink), Services/Audio/AudioMixer.cs (alert ring + EnqueueAlertAudio + unity drain), Services/Compositor/SceneCompositor.cs + Services/Encoder/FramePump.cs (ticker seam)
• ViewModels/MainViewModel.cs + ViewModels/MainViewModel.Chat.cs (seams, stamp, resolver, materialize, browse/reset), Controls/LeftPanel.xaml (Stream Alerts section)
• NEW ytLive.Tests/AlertLayerVideoTests.cs
• Docs: ai.md, TASKS.md, this file, Services/index.md, HANDOFF.md, MyMistakes.md

Gate

Clean build 0 warnings (Windows dotnet host); alert tests green (AlertLayer* FullyQualifiedName filter = 2/2). Full suite 318/319 — the one failure LayerReorderPersistenceTests.RealMouseDrag… is the HANDOFF-documented environmental class (physical-mouse drag no-ops when the desktop/session isn't interactive; it passed in isolation on the third re-run of this unit; the two sibling reorder tests — identical persistence path — pass). scripts/scope-check.sh green.

Post-ship crash fix (2026-09-26)

First live run found a crash the sims never could: test-tab alerts work (button commands run on the UI thread — startup.log shows the alert clip's audio in the live mix, peakMix 0.375 → 0.891 with mic and loop at 0.000), but the first real message through the chat poller killed the app at 11:22:01.301:

System.ArgumentException: Must create DependencySource on same Thread as the DependencyObject.
at ...MS.Internal.Data.DataBindEngine.ProcessCrossThreadRequests()

AlertOverlayLayer.OnMessageReceived lacked the UI-thread marshal ChatOverlayLayer.OnMessageReceived has: a polled message ran RefreshAlertPreviews → UpdatePreview on the MTA poller thread and stamped alertBox.VideoImageSource (INPC-raised and WPF-bound) with a WriteableBitmap created there — the binding engine's cross-thread re-bind crashed. Fix: marshal the whole ingest (Dispatcher.Invoke, Application.Current-null guard) so enqueue, ticker, and preview writes stay on the UI thread in lockstep. Good Dog test OnMessageReceived_FromPollerThread_MarshalsPreviewWritesToTheUiThread (red on the old seam: the WriteableBitmap's Dispatcher was the poller's) — committed with the fix.

Silent-decoder fallback + decode diagnostics (2026-09-26)

Creator repro after the crash fix: On-Air → test → remove and re-add the Stream Alerts layer → TEST tab event → procs in the chat windows but NOTHING in the alert box. Startup.log 11:44 (the re-add) shows Test chat simulation 3 injected but peakMix 0.105 (no alert audio), vs 11:21's 0.375–0.891 (alert audio in the mix) — the re-added box's alert never reached the mix. No exception anywhere: every failure stage in the clip path was silent by design. Hunted facts: LiveScene is never assigned (the encoder always renders StagedScene, MainViewModel.cs:323) so the re-added box IS in the render graph; the bake is invalidated per-element on add; the box's DB row (c85b46ac-…, X=602 Y=26 W=680 H=200) is healthy; ffmpeg tools incl. the pinned binary decode the default clip to scale=680:200 rawvideo correctly to a file. The silent void: a decoder whose Start() succeeds but whose task yields no frames AND no audio leaves _clip != null with _latestClipFrame == null → RenderClipFrame returns null forever → permanently blank box, no animation fallback (the fallback only ran on setup-time throws) and no log trail.

Fix (commit …): a no-frame grace in AlertOverlayLayer.Advance (NoFrameFallbackSeconds = 1.0) — a started clip that has produced neither a frame nor an audio chunk inside the grace is torn down (StopClip), _elapsed resets, and the SAME alert continues as the six-animation render (never a blank box, never drained early). Diagnostics stop the lying silence: BeginClip logs the box id + resolved path + size and the setup failure that used to only Debug.WriteLine; AlertClipDecoder.RunAsync now logs frame/chunk end-state and the exception it used to swallow. Good Dog test SilentDecoder_FallsBackToTheAnimationAfterTheNoFrameGrace pins the whole contract — transparent inside the grace, decoder disposed past it, animation rendering, drains to idle. This lands the corpus at 321, with the one known-env flake (RealMouseDrag reorder, passes with no fullscreen windows up) — unrelated to this change. Unit was the direct lineage of bug surface: a decoder booth production failure surfaces as dead air.

Video-frame pacing fix (2026-09-26)

Second creator repro after the fallback commit (12:35 session): decoder HEALTHY — Alert clip start: box … → Alert clip end: … frames=240 audioChunks=156 failed=False twice, alert audio in the live mix (peakMix 0.277 → 0.733, mic/loop 0.000) — yet "no video plays in the web-alert box", "like you lost the video". The hole was never decode: the video pipe was unpaced. AlertClipDecoderFor (MainViewModel.Chat.cs:113) built the decoder with no frameRateProbe, so RunAsync computed frameDuration = TimeSpan.Zero and RunVideoAsync's pace (if (frameDuration > 0)) was skipped — all 240 frames (a 10s, 680×200 clip, ~130MB) dumped through the pipe in the first ~1-2s as fast as ffmpeg read them, then the box sat on the LAST frame frozen for the remaining ~6-8s while the audio chunk-pacer (50ms/chunk × 156) ran out its real-time cadence. To the eye: a blur then a dead frame = "no video / you lost the video". The media path already wired exactly this seam (MediaVideoSource gets frameRateProbe: new FfmpegFrameRateProbe(…), MainViewModel.cs:308) — the alert factory just never supplied one.

Fix (commit …): AlertClipDecoderFor now passes frameRateProbe: new FfmpegFrameRateProbe(new FfmpegLocator(), () => new FfmpegDecodeProcess()) so the 240 frames pace at the clip's native ~24fps (10s real-time), matching the audio cadence. Good Dog test AlertClipDecoderTests.AlertClipDecoder_PacesVideoFramesToTheProbedFrameRate drives the REAL AlertClipDecoder through the same fake-process/fake-probe seam the media test uses (3 frames, probe 100fps → one ~10ms pacing delay per frame) — red on the old factory (no delay), green with a probe. Full suite 321/322, one known-env flake (RealMouseDrag reorder). Derivative lesson → MyMistakes.md: a decoder that reads faster than wall-clock needs an explicit pace; "plays but you don't see it" is pacing, not decode.

Open follow-ups (NOT this unit)

• TASK 3 item 20 (RewardEvent SQLite persistence half) and item 16 (Text source) remain queued, unchanged.
• The alert-store folder (Apps_Commands-style per-clip assets) stays out of scope — the current model is read-at-play from a path, per the creator's "never store the video" ruling.