gramps
a11b15e444
TASK 43's alert box grows a real video celebration. Per-alert IAlertClipDecoder
(ffmpeg bgra + f32le pipes, real-time paced, disposed at drain) plays the shipped
Assets/alert-default.mp4 (stamped into the Asset table at startup) unless the
creator picks their own file — path reference only, never stored in the DB; the
six AlertRenderer animations stay the fallback. ~0.3s fade rides the alpha
envelope on straight-source copies (EOF freeze-frames then fades out); audio
forwards to a new AudioMixer alert ring (8s, 48k stereo) drained at unity — no
duck, creator ruling — scaled by volume × fade. An auto-composed marquee ticker
('Funder — Super Chat · $10.00', 140px/s) scrolls top-of-frame via a
FramePump._alertTicker seam through Render/CompositeLayers, mixed into the cache
signature (dynamic overlay, never baked). New Stream Alerts section in LeftPanel.
Derivative-work references (how OBS/Streamlabs alert boxes do per-alert video):
- https://support.streamlabs.com/hc/en-us/articles/217741147-Setting-Up-Your-Streamlabs-Alerts (custom image/video per alert type + variations)
- https://obsproject.com/kb/stream-tutorial-2-alerts (alert overlay as an on-screen zone)
- https://streamlabs.com/content-hub/widgets/alert-box (per-event alert playback)
Good Dog: AlertLayerVideoTests drives a fake IAlertClipDecoder through the whole
lifecycle in one pass (custom path wins, decoder spawns/disposes, fade envelope
0→127→255, audio volume×fade, ticker scrolls, EOF fade-drain to idle). It caught
the clip branch of Advance not clearing _current before AdvanceToNext — the layer
stayed IsPlaying after drain (MyMistakes post-mortem).
Full vstest 319/319; clean build 0 warnings; scope check green.
138 lines
9.4 KiB
Markdown
138 lines
9.4 KiB
Markdown
# TASK 47 — Alert box video (TASK 43 follow-up): creator-replaceable alert clip + read-time fade + auto ticker
|
||
|
||
> Catalog: [`TASKS.md`](../TASKS.md). Status: ✅ **SHIPPED 2026-09-26** — the TASK 43
|
||
> alert box grows a real **video** celebration: a built-in mp4 loops on every alert unless
|
||
> the creator picks their own clip (referenced from disk, never stored in the DB), the
|
||
> clip fades in/out in ~0.3s (alpha envelope on the video + volume on the audio, mixed
|
||
> into the live stream at unity — no duck), and an auto-composed **message ticker**
|
||
> ("Funder — Super Chat · $10.00") scrolls across the very top of the frame, toggleable.
|
||
> The six TASK 43 animations remain the intrinsic fallback when no asset is playable.
|
||
|
||
## Provenance (recorded per the feature-provenance rule)
|
||
|
||
- **2026-09-24, the creator asked (right after TASK 43 shipped)** — "make the alerts
|
||
animate like a video" / "I want to play my own video for alerts." The TASK 43 unit
|
||
closed with this as the explicit next step.
|
||
- **Lineage:** Research (websearch: how do OBS/StreamElements/Streamlabs alert boxes show
|
||
per-alert videos?) settled the shape, cited in the commit message:
|
||
- Alert videos play **per-alert** (a fresh decoder per event), never a resident media
|
||
player — the OBS "media source reset on activation" model, not a playlist.
|
||
- The clip **must not block**: decode is a background pipe (ffmpeg child, rawvideo
|
||
frames + f32le audio), paced real-time, so alerts never stall the 60fps frame pump.
|
||
- A **freeze-frame at clip EOF fades out** rather than hard-dropping (Streamlabs'
|
||
"end by fade" option) — the tail of a long clip dissolves without a new spawn.
|
||
- The custom file is served by **path, not bytes** (OBS media-source semantics; a
|
||
read-at-play clip lets the creator re-edit the file without a re-import). Decided
|
||
with the creator: "never store the video in the layout." The built-in default is the
|
||
one BLOB, stamped into the internal `Asset` table at startup.
|
||
- **Creator rulings (asked once, held):** custom video + six-animation fallback
|
||
(not either/or); ticker on top of the canvas (very top, full width); **no ducking** —
|
||
"don't lower my game audio for a tip" (the TASK 43 planner's auto-duck idea was
|
||
rejected on the floor); a per-alert **Volume** slider instead.
|
||
|
||
## Product model (as shipped)
|
||
|
||
- **Five new `Source` props** (schema v10, additive guarded ALTERs): `AlertVideoPath`
|
||
(custom file), `AlertUseDefaultVideo` (bool, default true — the built-in clip),
|
||
`AlertShowTicker` (bool, default true), `AlertVideoVolume` (0–1, default 1).
|
||
`AlertVideoAssetId` (internal reference to the stamped default asset row).
|
||
- **Built-in clip:** `Assets/alert-default.mp4` (~4.9MB, branded 2.5s loop, included as a
|
||
csproj Resource). At startup `MainViewModel.StampDefaultAlertVideo()` upserts its bytes
|
||
into the `Asset` table and records the row id in the settings key
|
||
`AlertDefaultVideoAssetId` (`AlertDefaultVideoKey`). The **prune** step exempts that
|
||
key via a UNION so the default never gets cleaned.
|
||
- **Per-alert resolver** (`ResolveAlertClipPath` in `MainViewModel.Chat.cs`): custom path
|
||
wins ONLY when `AlertUseDefaultVideo == false` AND the file exists; otherwise the
|
||
thumbnailed built-in is materialized once to `%TEMP%\ytLive-alert-{id}.mp4`; failure →
|
||
the six `AlertRenderer` animations play (fallback, not an error).
|
||
- **`Services/AlertClipDecoder.cs`** — `IAlertClipDecoder` (Start/Stop/Dispose +
|
||
`FrameAvailable(Action<VideoFrame>)` / `AudioReady(Action<AudioSample>)` /
|
||
`Completed(Action)`) + `AlertClipDecoder` impl: ffmpeg `-loglevel error` child
|
||
emitting `rawvideo bgra` (`-vf scale=W:H`) on one pipe and `f32le -ar 48000 -ac 2` on
|
||
another; `RawVideoFrameReader` (the TASK 21 media-source reader) on the video pipe,
|
||
a **carry-buffer** loop on the audio pipe (float-boundary straddles never drop a
|
||
sample — the 3-byte PCM16↔4-byte-f32 mismatch that broke the first draft), both
|
||
real-time paced. Per-play lifetime: the layer creates one per alert and disposes it at
|
||
drain. Prefer `Start`-synced `TrackFrameRate` — the frame pipe throttles to vfr
|
||
timestamps; audio paces itself. ffmpeg bgra is opaque (alpha=255) — the layer owns all
|
||
alpha.
|
||
- **`Services/AlertOverlayLayer.cs`** — clip branch beside the animation path:
|
||
`Enqueue` → `BeginClip` (factory + path + box dims seam), `Advance` advances `_elapsed`
|
||
(fade-in frame copies scale alpha 0→255 straight-source over the box), events forward
|
||
to the audio sink scaled by `volume × fade`, EOF → **freeze-frame** + fade-out over the
|
||
same `FadeDurationSeconds = 0.30` envelope → drain (StopClip + dispose, idle null
|
||
again). Ticker: `AlertTickerFrame` composes "Author — Kind · amount" from the live
|
||
message and scrolls it.
|
||
- **`Services/Compositor/AlertTickerRenderer.cs`** — marquee strip: 1920×48 transparent
|
||
BGRA, text-pill rasterized once per text on the UI thread (cached, unpremultiplied),
|
||
then pure byte-math per call (scroll `pos = elapsed × 140px/s % cycle`, pill drawn
|
||
straight-alpha over transparent with a repeat-gap copy), so the pump can call it from
|
||
its own thread every tick with no locking.
|
||
- **Ticker threading** — dynamic overlay like the social bar, but **never baked**
|
||
(the social bar IS baked into `BakeStaticBase`; the ticker is a per-frame marquee):
|
||
`SceneCompositor.Render` + `CompositeLayers` take a trailing `tickerFrame` param
|
||
(blitted at 0,0 after the social bar), threaded by `FramePump._alertTicker`
|
||
(`Func<VideoFrame?>` seam, ctor param) through `RenderScene`/`RenderFull`/
|
||
`BuildFullRenderSignature` (mixed into the cache signature so a ticker change
|
||
invalidates — ticker is distinct from static-cache-eligible content).
|
||
- **Audio path** — `AudioMixer.EnqueueAlertAudio` uses a dedicated ring
|
||
(`AlertBufferSeconds = 8` at 48k stereo): resample/upmix to stereo 48k, drained
|
||
pre-limiter in `FillAndMix` and added at **unity** (never ducked); `StartLive` clears
|
||
it. The layer applies `volume × fade` per sample before forwarding.
|
||
- **UI — Stream Alerts section** (LeftPanel, visible only on an `AlertBox`):
|
||
"Built-in video" pill; custom path box + Browse…; "Reset to built-in"; "Message
|
||
ticker" pill; "Volume" slider with % readout. Wiring: `BrowseAlertVideoCommand` +
|
||
`ResetAlertVideoCommand` (CanExecute gates on `Source { Type: AlertBox }`); browse =
|
||
OpenFileDialog (mp4/mov/webm), result writes `AlertVideoPath` + switches the pill off.
|
||
|
||
## Test (Good Dog — ONE integration test per change)
|
||
|
||
`ytLive.Tests/AlertLayerVideoTests.cs` (RealApp STA host, real WPF raster) —
|
||
`AlertVideo_PlaysCustomClip_FadesReadTime_TickerScrolls_ForwardsScaledAudio_DrainsIdle`
|
||
drives a **fake `IAlertClipDecoder`** (no ffmpeg in tests) through the whole lifecycle
|
||
in one pass: the custom path wins per-source (resolution seam asserts box W/H on the
|
||
factory + same source to the resolver); the decoder spawns on play (StartCount 1) and
|
||
dies at drain (DisposeCount 1); fade-in rides the 0.3s envelope (alpha 0 → 127 at 0.15s
|
||
→ 255, straight-alpha copy); pre-fade audio is dropped while ramp audio forwards at
|
||
`volume × fade` (0.8 × 0.5 → [0.2, −0.1]) and full-fade at volume only; the ticker
|
||
renders non-null with pixels and **scrolls** between two pump ticks while idle returns
|
||
null; EOF freeze → fade-out 127 → drain to `IsPlaying false` + null frame + disposed
|
||
decoder.
|
||
|
||
> Catches a REAL bug the animation path never had: the clip branch of `Advance` failed
|
||
> to clear `_current` before `AdvanceToNext`, so after the fade-out the layer stayed
|
||
> `IsPlaying` and re-rendered the finished alert via the animation fallback. The clean
|
||
> frame literally forced the `Assert.False(layer.IsPlaying)` to see it.
|
||
|
||
## Files (declared scope)
|
||
|
||
- `Models/Source.cs` (five alert props + `IsAlertBox`), `Services/LayoutStore.Migrations.cs`
|
||
(guarded ALTERs), `Services/LayoutStore.Save.cs` (INSERT + prune UNION),
|
||
`Services/LayoutStore.Load.cs` (reader indices),
|
||
`Services/LayoutStore.Settings.cs` (`AlertDefaultVideoKey` + asset stamps)
|
||
- `Assets/alert-default.mp4` + `ytLive.csproj` (Resource include)
|
||
- **NEW** `Services/AlertClipDecoder.cs`, **NEW** `Services/Compositor/AlertTickerRenderer.cs`
|
||
- `Services/AlertOverlayLayer.cs` (clip branch + fade + ticker + sink), `Services/Audio/AudioMixer.cs`
|
||
(alert ring + `EnqueueAlertAudio` + unity drain), `Services/Compositor/SceneCompositor.cs` +
|
||
`Services/Encoder/FramePump.cs` (ticker seam)
|
||
- `ViewModels/MainViewModel.cs` + `ViewModels/MainViewModel.Chat.cs` (seams, stamp,
|
||
resolver, materialize, browse/reset), `Controls/LeftPanel.xaml` (Stream Alerts section)
|
||
- **NEW** `ytLive.Tests/AlertLayerVideoTests.cs`
|
||
- Docs: `ai.md`, `TASKS.md`, this file, `Services/index.md`, `HANDOFF.md`, `MyMistakes.md`
|
||
|
||
## Gate
|
||
|
||
Clean build 0 warnings (Windows dotnet host); alert tests green (`AlertLayer*
|
||
FullyQualifiedName` filter = 2/2). Full suite 318/319 — the one failure
|
||
`LayerReorderPersistenceTests.RealMouseDrag…` is the HANDOFF-documented environmental
|
||
class (physical-mouse drag no-ops when the desktop/session isn't interactive; it passed
|
||
in isolation on the third re-run of this unit; the two sibling reorder tests — identical
|
||
persistence path — pass). `scripts/scope-check.sh` green.
|
||
|
||
## Open follow-ups (NOT this unit)
|
||
|
||
- TASK 3 item 20 (RewardEvent SQLite persistence half) and item 16 (Text source) remain
|
||
queued, unchanged.
|
||
- The alert-store folder (`Apps_Commands`-style per-clip assets) stays out of scope —
|
||
the current model is read-at-play from a path, per the creator's "never store the
|
||
video" ruling. |