gramps
7b940b6a5a
Live test session proved the native alert box DID play (the alert ring was the only source in the mix: micLevel/loopLevel 0.000 while peakMix went 0.375->0.733->0.891 after the sim injections) but the app crashed at 11:22:01.301 the moment a REAL message round-tripped through the poller: System.ArgumentException: Must create DependencySource on same Thread as the DependencyObject at ...MS.Internal.Data.DataBindEngine.ProcessCrossThreadRequests() OnMessageReceived ran RefreshAlertPreviews on the MTA poller thread and stamped the WPF-bound VideoImageSource with a WriteableBitmap created there; the binding engine's cross-thread re-bind killed the process. ChatOverlayLayer already marshals this exact seam (Dispatcher.Invoke) - mirror it, guarded for Application.Current null so the pure test seams still run inline. Good Dog test: AlertLayerVideoTests.OnMessageReceived_FromPollerThread_MarshalsPreviewWritesToTheUiThread calls the seam from a raw MTA Thread while the RealApp loop runs, then asserts on the UI thread that the preview bitmap's Dispatcher is the App's (red on the old seam, green on the fix). WPF cross-thread rule recorded in MyMistakes.md. Full suite 320/320, clean build 0 warnings, scope-check green.
157 lines
11 KiB
Markdown
157 lines
11 KiB
Markdown
# TASK 47 — Alert box video (TASK 43 follow-up): creator-replaceable alert clip + read-time fade + auto ticker
|
||
|
||
> Catalog: [`TASKS.md`](../TASKS.md). Status: ✅ **SHIPPED 2026-09-26** — the TASK 43
|
||
> alert box grows a real **video** celebration: a built-in mp4 loops on every alert unless
|
||
> the creator picks their own clip (referenced from disk, never stored in the DB), the
|
||
> clip fades in/out in ~0.3s (alpha envelope on the video + volume on the audio, mixed
|
||
> into the live stream at unity — no duck), and an auto-composed **message ticker**
|
||
> ("Funder — Super Chat · $10.00") scrolls across the very top of the frame, toggleable.
|
||
> The six TASK 43 animations remain the intrinsic fallback when no asset is playable.
|
||
|
||
## Provenance (recorded per the feature-provenance rule)
|
||
|
||
- **2026-09-24, the creator asked (right after TASK 43 shipped)** — "make the alerts
|
||
animate like a video" / "I want to play my own video for alerts." The TASK 43 unit
|
||
closed with this as the explicit next step.
|
||
- **Lineage:** Research (websearch: how do OBS/StreamElements/Streamlabs alert boxes show
|
||
per-alert videos?) settled the shape, cited in the commit message:
|
||
- Alert videos play **per-alert** (a fresh decoder per event), never a resident media
|
||
player — the OBS "media source reset on activation" model, not a playlist.
|
||
- The clip **must not block**: decode is a background pipe (ffmpeg child, rawvideo
|
||
frames + f32le audio), paced real-time, so alerts never stall the 60fps frame pump.
|
||
- A **freeze-frame at clip EOF fades out** rather than hard-dropping (Streamlabs'
|
||
"end by fade" option) — the tail of a long clip dissolves without a new spawn.
|
||
- The custom file is served by **path, not bytes** (OBS media-source semantics; a
|
||
read-at-play clip lets the creator re-edit the file without a re-import). Decided
|
||
with the creator: "never store the video in the layout." The built-in default is the
|
||
one BLOB, stamped into the internal `Asset` table at startup.
|
||
- **Creator rulings (asked once, held):** custom video + six-animation fallback
|
||
(not either/or); ticker on top of the canvas (very top, full width); **no ducking** —
|
||
"don't lower my game audio for a tip" (the TASK 43 planner's auto-duck idea was
|
||
rejected on the floor); a per-alert **Volume** slider instead.
|
||
|
||
## Product model (as shipped)
|
||
|
||
- **Five new `Source` props** (schema v10, additive guarded ALTERs): `AlertVideoPath`
|
||
(custom file), `AlertUseDefaultVideo` (bool, default true — the built-in clip),
|
||
`AlertShowTicker` (bool, default true), `AlertVideoVolume` (0–1, default 1).
|
||
`AlertVideoAssetId` (internal reference to the stamped default asset row).
|
||
- **Built-in clip:** `Assets/alert-default.mp4` (~4.9MB, branded 2.5s loop, included as a
|
||
csproj Resource). At startup `MainViewModel.StampDefaultAlertVideo()` upserts its bytes
|
||
into the `Asset` table and records the row id in the settings key
|
||
`AlertDefaultVideoAssetId` (`AlertDefaultVideoKey`). The **prune** step exempts that
|
||
key via a UNION so the default never gets cleaned.
|
||
- **Per-alert resolver** (`ResolveAlertClipPath` in `MainViewModel.Chat.cs`): custom path
|
||
wins ONLY when `AlertUseDefaultVideo == false` AND the file exists; otherwise the
|
||
thumbnailed built-in is materialized once to `%TEMP%\ytLive-alert-{id}.mp4`; failure →
|
||
the six `AlertRenderer` animations play (fallback, not an error).
|
||
- **`Services/AlertClipDecoder.cs`** — `IAlertClipDecoder` (Start/Stop/Dispose +
|
||
`FrameAvailable(Action<VideoFrame>)` / `AudioReady(Action<AudioSample>)` /
|
||
`Completed(Action)`) + `AlertClipDecoder` impl: ffmpeg `-loglevel error` child
|
||
emitting `rawvideo bgra` (`-vf scale=W:H`) on one pipe and `f32le -ar 48000 -ac 2` on
|
||
another; `RawVideoFrameReader` (the TASK 21 media-source reader) on the video pipe,
|
||
a **carry-buffer** loop on the audio pipe (float-boundary straddles never drop a
|
||
sample — the 3-byte PCM16↔4-byte-f32 mismatch that broke the first draft), both
|
||
real-time paced. Per-play lifetime: the layer creates one per alert and disposes it at
|
||
drain. Prefer `Start`-synced `TrackFrameRate` — the frame pipe throttles to vfr
|
||
timestamps; audio paces itself. ffmpeg bgra is opaque (alpha=255) — the layer owns all
|
||
alpha.
|
||
- **`Services/AlertOverlayLayer.cs`** — clip branch beside the animation path:
|
||
`Enqueue` → `BeginClip` (factory + path + box dims seam), `Advance` advances `_elapsed`
|
||
(fade-in frame copies scale alpha 0→255 straight-source over the box), events forward
|
||
to the audio sink scaled by `volume × fade`, EOF → **freeze-frame** + fade-out over the
|
||
same `FadeDurationSeconds = 0.30` envelope → drain (StopClip + dispose, idle null
|
||
again). Ticker: `AlertTickerFrame` composes "Author — Kind · amount" from the live
|
||
message and scrolls it.
|
||
- **`Services/Compositor/AlertTickerRenderer.cs`** — marquee strip: 1920×48 transparent
|
||
BGRA, text-pill rasterized once per text on the UI thread (cached, unpremultiplied),
|
||
then pure byte-math per call (scroll `pos = elapsed × 140px/s % cycle`, pill drawn
|
||
straight-alpha over transparent with a repeat-gap copy), so the pump can call it from
|
||
its own thread every tick with no locking.
|
||
- **Ticker threading** — dynamic overlay like the social bar, but **never baked**
|
||
(the social bar IS baked into `BakeStaticBase`; the ticker is a per-frame marquee):
|
||
`SceneCompositor.Render` + `CompositeLayers` take a trailing `tickerFrame` param
|
||
(blitted at 0,0 after the social bar), threaded by `FramePump._alertTicker`
|
||
(`Func<VideoFrame?>` seam, ctor param) through `RenderScene`/`RenderFull`/
|
||
`BuildFullRenderSignature` (mixed into the cache signature so a ticker change
|
||
invalidates — ticker is distinct from static-cache-eligible content).
|
||
- **Audio path** — `AudioMixer.EnqueueAlertAudio` uses a dedicated ring
|
||
(`AlertBufferSeconds = 8` at 48k stereo): resample/upmix to stereo 48k, drained
|
||
pre-limiter in `FillAndMix` and added at **unity** (never ducked); `StartLive` clears
|
||
it. The layer applies `volume × fade` per sample before forwarding.
|
||
- **UI — Stream Alerts section** (LeftPanel, visible only on an `AlertBox`):
|
||
"Built-in video" pill; custom path box + Browse…; "Reset to built-in"; "Message
|
||
ticker" pill; "Volume" slider with % readout. Wiring: `BrowseAlertVideoCommand` +
|
||
`ResetAlertVideoCommand` (CanExecute gates on `Source { Type: AlertBox }`); browse =
|
||
OpenFileDialog (mp4/mov/webm), result writes `AlertVideoPath` + switches the pill off.
|
||
|
||
## Test (Good Dog — ONE integration test per change)
|
||
|
||
`ytLive.Tests/AlertLayerVideoTests.cs` (RealApp STA host, real WPF raster) —
|
||
`AlertVideo_PlaysCustomClip_FadesReadTime_TickerScrolls_ForwardsScaledAudio_DrainsIdle`
|
||
drives a **fake `IAlertClipDecoder`** (no ffmpeg in tests) through the whole lifecycle
|
||
in one pass: the custom path wins per-source (resolution seam asserts box W/H on the
|
||
factory + same source to the resolver); the decoder spawns on play (StartCount 1) and
|
||
dies at drain (DisposeCount 1); fade-in rides the 0.3s envelope (alpha 0 → 127 at 0.15s
|
||
→ 255, straight-alpha copy); pre-fade audio is dropped while ramp audio forwards at
|
||
`volume × fade` (0.8 × 0.5 → [0.2, −0.1]) and full-fade at volume only; the ticker
|
||
renders non-null with pixels and **scrolls** between two pump ticks while idle returns
|
||
null; EOF freeze → fade-out 127 → drain to `IsPlaying false` + null frame + disposed
|
||
decoder.
|
||
|
||
> Catches a REAL bug the animation path never had: the clip branch of `Advance` failed
|
||
> to clear `_current` before `AdvanceToNext`, so after the fade-out the layer stayed
|
||
> `IsPlaying` and re-rendered the finished alert via the animation fallback. The clean
|
||
> frame literally forced the `Assert.False(layer.IsPlaying)` to see it.
|
||
|
||
## Files (declared scope)
|
||
|
||
- `Models/Source.cs` (five alert props + `IsAlertBox`), `Services/LayoutStore.Migrations.cs`
|
||
(guarded ALTERs), `Services/LayoutStore.Save.cs` (INSERT + prune UNION),
|
||
`Services/LayoutStore.Load.cs` (reader indices),
|
||
`Services/LayoutStore.Settings.cs` (`AlertDefaultVideoKey` + asset stamps)
|
||
- `Assets/alert-default.mp4` + `ytLive.csproj` (Resource include)
|
||
- **NEW** `Services/AlertClipDecoder.cs`, **NEW** `Services/Compositor/AlertTickerRenderer.cs`
|
||
- `Services/AlertOverlayLayer.cs` (clip branch + fade + ticker + sink), `Services/Audio/AudioMixer.cs`
|
||
(alert ring + `EnqueueAlertAudio` + unity drain), `Services/Compositor/SceneCompositor.cs` +
|
||
`Services/Encoder/FramePump.cs` (ticker seam)
|
||
- `ViewModels/MainViewModel.cs` + `ViewModels/MainViewModel.Chat.cs` (seams, stamp,
|
||
resolver, materialize, browse/reset), `Controls/LeftPanel.xaml` (Stream Alerts section)
|
||
- **NEW** `ytLive.Tests/AlertLayerVideoTests.cs`
|
||
- Docs: `ai.md`, `TASKS.md`, this file, `Services/index.md`, `HANDOFF.md`, `MyMistakes.md`
|
||
|
||
## Gate
|
||
|
||
Clean build 0 warnings (Windows dotnet host); alert tests green (`AlertLayer*
|
||
FullyQualifiedName` filter = 2/2). Full suite 318/319 — the one failure
|
||
`LayerReorderPersistenceTests.RealMouseDrag…` is the HANDOFF-documented environmental
|
||
class (physical-mouse drag no-ops when the desktop/session isn't interactive; it passed
|
||
in isolation on the third re-run of this unit; the two sibling reorder tests — identical
|
||
persistence path — pass). `scripts/scope-check.sh` green.
|
||
|
||
## Post-ship crash fix (2026-09-26)
|
||
|
||
First live run found a crash the sims never could: test-tab alerts work (button commands run on
|
||
the UI thread — startup.log shows the alert clip's audio in the live mix, `peakMix 0.375 → 0.891`
|
||
with mic and loop at 0.000), but the first **real** message through the chat poller killed the app
|
||
at `11:22:01.301`:
|
||
|
||
System.ArgumentException: Must create DependencySource on same Thread as the DependencyObject.
|
||
at ...MS.Internal.Data.DataBindEngine.ProcessCrossThreadRequests()
|
||
|
||
`AlertOverlayLayer.OnMessageReceived` lacked the UI-thread marshal `ChatOverlayLayer.OnMessageReceived`
|
||
has: a polled message ran `RefreshAlertPreviews` → `UpdatePreview` on the MTA poller thread and
|
||
stamped `alertBox.VideoImageSource` (INPC-raised and WPF-bound) with a `WriteableBitmap` created
|
||
there — the binding engine's cross-thread re-bind crashed. Fix: marshal the whole ingest
|
||
(`Dispatcher.Invoke`, `Application.Current`-null guard) so enqueue, ticker, and preview writes stay
|
||
on the UI thread in lockstep. Good Dog test
|
||
`OnMessageReceived_FromPollerThread_MarshalsPreviewWritesToTheUiThread` (red on the old seam: the
|
||
WriteableBitmap's `Dispatcher` was the poller's) — committed with the fix.
|
||
|
||
## Open follow-ups (NOT this unit)
|
||
|
||
- TASK 3 item 20 (RewardEvent SQLite persistence half) and item 16 (Text source) remain
|
||
queued, unchanged.
|
||
- The alert-store folder (`Apps_Commands`-style per-clip assets) stays out of scope —
|
||
the current model is read-at-play from a path, per the creator's "never store the
|
||
video" ruling. |