gramps
b00bf48b9d
Creator request: a sound-byte browse + label under Built-in video, with an option to loop the sound while the clip plays. The video itself is always one pass to completion. - Source gains AlertAudioPath and AlertAudioLoop, persisted (column + migration + INSERT + SELECT + reader, the whole chain — that chain is where the blur crashed once). - The sound byte plays through MediaPlayer on the default output device; MediaEnded re-seeks to zero while looping, and StopClip kills it so a tail cannot bleed into the next alert. - The clip's OWN soundtrack moved off the mixer and onto the device as well (Services/Audio/AlertAudioOutput: one WaveOutEvent + buffered float provider). Why the move: the creator asked that alert audio — the clip's track or the added sound byte — appear in the desktop sound bar. Only audio that reaches the default device is picked up by the WASAPI loopback that feeds both the meter and the stream. The old EnqueueAlertAudio route put the sound in the stream but left the meter flat and the room silent. Tests: the sound byte and its loop flag round-trip through SQLite, and a fresh alert layer starts with no sound byte and no loop. Release 0 warnings.
164 lines
13 KiB
Markdown
164 lines
13 KiB
Markdown
# Output overlays: social bar, alerts, chat
|
||
|
||
Everything composited *over* the frame: the social bar, the native alert box (six event
|
||
animations, alert video, ticker), and the chat box conventions. Read before touching any of
|
||
them or anything that composites a global layer.
|
||
|
||
**Invariant:** every global overlay must be composited on **every** render path — pump,
|
||
preview/thumbnails, scene dump (`SceneCompositor.Overlay` is shared by all three). A feature
|
||
wired into one path is silently absent from the output. That bug ("preview-only forever",
|
||
2026-09-26) is why the rule exists.
|
||
|
||
## Social bar (TASK 6 — shipped 2026-08-12, plan in TASKS.md; bar bug-fix branch 2026-08-13)
|
||
|
||
A **global bar layer** (never a Source, no sources-list row) that sits over the bottom or top of the
|
||
output and carries the creator's social links — content-sized, centered, GREEN glow when ON, position is a
|
||
**click-toggle top ⇄ bottom** (default BOTTOM, persisted `SocialBarPosition`). `Models/Socials.cs`: `SocialService` enum
|
||
(YouTube/Twitch/X/Instagram/TikTok/Facebook/Discord/Kick/Threads/Bluesky/GitHub/LinkedIn/Pinterest/
|
||
Snapchat/Reddit/WhatsApp/Telegram/Link/Website/**Fediverse**), `SocialEntry` (Service/Handle/ProfileUrl/
|
||
`FediverseSoftware`), `SocialsConfig` (Entries + `BarPosition` + `BarEnabled`; `BarJustify` dropped,
|
||
column back-compat). Configured in the "Social Media Site Promotion" dialog (`SocialsDialog.xaml` +
|
||
`ViewModels/SocialsDialogViewModel`, WPF-free, injected `ISocialValidator` + sign-in/sign-out fakes):
|
||
ON/OFF bar switch (schema v8), 6 fixed slots — row 1 always YouTube (signed-in → channel handle;
|
||
signed-out → sign-in gate → OAuth; delete → confirm sign-out), row 2 free, rows 3–6 lock icons on
|
||
freemium. Service detection (`SocialServiceIcons.DetectService`): URL domain / fediverse `@user@domain` →
|
||
**Fediverse** / bare→Website. Validation (`Services/SocialValidator.cs`, `ISocialValidator` seam +
|
||
`HttpSocialValidator` default): async GET of the canonical profile URL; 200/redirect = valid, 404/failure =
|
||
rejected. Fediverse additionally does a **best-effort nodeinfo lookup** (`/.well-known/nodeinfo` →
|
||
`software.name`, stored in `SocialEntry.FediverseSoftware` / the `SocialEntry.Software` column,
|
||
column-presence migration, no version bump) so the entry shows the **instance's real logo**
|
||
(`LogoDataForFediverse`: mastodon/peertube/pixelfed/misskey/lemmy/pleroma/firefish, generic fediverse
|
||
honeycomb fallback — Simple Icons CC0 path data, initials badges gone); nodeinfo failure still validates
|
||
(glyph falls back). If the identity domain's nodeinfo is blocked (YunoHost SSO gates
|
||
`/.well-known/nodeinfo` behind the login page) but the bare root 302s to the real instance, the lookup
|
||
**follows the root redirect** and asks the resolved host. Cancel is a hard stop: `LookupAsync` takes a
|
||
`CancellationToken` (dialog VM owns a CTS; Cancel/X/Save abort in-flight lookups, canceled continuations
|
||
never touch slot state).
|
||
|
||
**Bar bug-fix branch (2026-08-13) — three changes:**
|
||
|
||
1. **Positioning is a click-toggle (KISS)** — the original drag set a local `Canvas.SetTop` value that
|
||
permanently overrides the `{Binding SocialBarTop}` (a binding can never win over a local value), so the
|
||
bar stayed wherever it was dropped. The first drag-snap fix (`SocialBarSnap.Decide` + `ClearValue` on
|
||
release) still failed for shaky hands — jitter around the ±6px deadzone snapped the bar up but wouldn't
|
||
let it come back down. **Superseded by the user's click-toggle:** clicking the bar flips it top ⇄ bottom
|
||
(`MainViewModel.ToggleSocialBarPosition`), the bar rides the binding alone, and `SocialBarSnap` is gone.
|
||
2. **Fediverse software self-heal** — the DB row `@gramps@llamachile.tube` had `Software = NULL` because
|
||
nodeinfo was only ever asked of the identity domain (a landing page; the real instance is
|
||
`mastodon.llamachile.tube`). `HttpSocialValidator.ResolveFediverseSoftwareAsync` now **probes
|
||
well-known subdomains** (`mastodon.` → `social.` → … `FediverseSubdomainCandidates`) when the identity
|
||
domain and its redirect both come up empty, under a ~15s linked-CTS budget. `MainViewModel` runs the
|
||
static `HealFediverseSoftwareAsync` off the UI thread on layout load, applies matches via the dispatcher,
|
||
and saves; `SocialEntry.FediverseSoftware` is settable and raises `LogoData`, so the icon updates in place.
|
||
3. **The bar renders on the live output** — `Compositor/SocialBarRenderer.cs` rasterizes the entries into a
|
||
transparent straight-alpha BGRA strip (1920-wide, 40px content + 24px glow pad, the green glow baked in,
|
||
Pbgra32→straight-alpha unpremultiply) and the compositor blits it above the flash (see "Live frame
|
||
pipeline").
|
||
|
||
## Native events & alerts (TASK 43, shipped 2026-09-24 — StreamElements replaced)
|
||
|
||
The creator asked to replace the external StreamElements feed with native YouTube events;
|
||
the answer is **native events + a native alert box**, and the product decisions are
|
||
published decisions (recorded in `TASKS/task-43-native-alerts.md`):
|
||
|
||
- **No third party.** The chat feed IS the event feed: `liveChat/messages` carries all six
|
||
monetization events (superChat / superSticker / newSponsor / membershipGifting /
|
||
giftMembershipReceived / memberMilestoneChat). `streamList` is the *connection semantics*
|
||
of that same endpoint (pollingIntervalMillis-driven), not a separate call — the service
|
||
re-arms its one-shot poll on the server's cadence (clamp 1000–6000ms), `maxResults=2000`.
|
||
Cite: https://developers.google.com/youtube/v3/live/docs/liveChatMessages/streamList
|
||
- **`SourceType.AlertBox` — one "Stream Alerts" celebration zone per layout**, OBS
|
||
alert-box style: a designated draggable canvas area, idle = transparent. Mirrors the chat
|
||
gate (`CanAddAlerts`; `AddSource` refuses a second). Defaults 620×430 → 680×200.
|
||
- **Six events → six unique animations** (`Services/AlertRenderer.cs`): SuperChat slide-up
|
||
+ gold count-up + shine sweep (green), SuperSticker scale-pop on a tilted chip (purple),
|
||
NewMember drop-in + flash band (violet), MemberGift slide-left + gift-chip `×N` fan
|
||
(amber), GiftReceived confetti scale-in (green→purple), MemberMilestone rise + tenure
|
||
growth bar (orange). Phases enter/hold/exit (~4–5s); every card draws the **brand line
|
||
"made with LlamaCasty!"** — each broadcasted event is free product placement.
|
||
- **`Services/AlertOverlayLayer.cs`** — the component: enqueues events one-at-a-time,
|
||
advances on a 33ms dispatch ticker, serves the live frame through `RenderFrame` (same
|
||
cache-first pattern as chat) and the preview through `UpdatePreview`; **`Advance(double)`
|
||
is the deterministic test clock** (tests drive playback with no timer). Queue cap 10
|
||
(floods drop the tail, never stall). `ChatEventKind.None` rows (plain chat, sub mentions)
|
||
**never enqueue** — per the creator ruling, a subscriber mention is sufficient; there is
|
||
no free-sub alert (YouTube emits none) and no viewer count (vetoed — demotivating,
|
||
weaponizable).
|
||
- **Alert box VIDEO (TASK 47, 2026-09-26):** since TASK 47 the box plays a clip on every
|
||
alert instead of (only) an animation — `Enqueue`→`BeginClip` spawns a **per-play**
|
||
`IAlertClipDecoder` (ffmpeg `rawvideo bgra -vf scale=W:H` frame pipe + `f32le` audio
|
||
pipe, both real-time paced via `RawVideoFrameReader`; **not** the refcounted
|
||
`MediaVideoSourceManager` — alerts are one-shot plays that must be disposed at drain).
|
||
Custom clip wins per-source over the built-in ONLY when `AlertUseDefaultVideo==false`
|
||
AND the file exists; the shipped `Assets/alert-default.mp4` is stamped into the `Asset`
|
||
table at startup (`AlertDefaultVideoKey` setting; the prune keeps it via UNION) and
|
||
read-at-play; any decode failure falls back to the six `AlertRenderer` animations
|
||
(deliberate, not an error). Fades in/out via **`FadeDurationSeconds = 0.30`**
|
||
(`AlertOverlayLayer` const): fade-in by `elapsed` on straight-alpha frame copies;
|
||
EOF → freeze-frame → fade-out → drain (the clip branch of `Advance` **clears
|
||
`_current` before `AdvanceToNext`** — the miss that the TASK 47 test caught). Audio
|
||
forwards to the mixer sink scaled by `volume × fade`.
|
||
- **Alert ticker (announcement strip):** `AlertTickerFrame` (in the layer) composes
|
||
"Author — Kind · amount" and hands it to `Services/Compositor/AlertTickerRenderer.cs` —
|
||
a strip rasterized **on the UI thread once per text** (cached, unpremultiplied
|
||
straight-alpha pill) then positioned by pure byte-math per call, so the pump calls it
|
||
from its own thread with no locking. **It is a dynamic overlay, NEVER baked** (the
|
||
social bar IS baked into `BakeStaticBase`): `SceneCompositor.Render` +
|
||
`CompositeLayers` take a trailing `tickerFrame` threaded from
|
||
`FramePump._alertTicker` (`Func<VideoFrame?>` ctor seam) through
|
||
`RenderScene`/`RenderFull` and **mixed into `BuildFullRenderSignature`** so a scroll
|
||
changes the cache key. Toggled by `AlertShowTicker`.
|
||
- **It draws INSIDE the Stream Alerts box (creator ruling 2026-09-26).** Was a global
|
||
1920px bar pinned to the top edge: *"the ticker should appear over the stream alerts
|
||
video, not over the entire preview window"*. The strip is now rendered at the
|
||
**alert box's own size** (`AlertTickerRenderer.Render(..., width, height)`, defaults
|
||
still 1920×48) and the box's origin rides on **`VideoFrame.Placement`**
|
||
(`(int X, int Y)?`, with `OriginX`/`OriginY` = 0 when unset). Every blit site reads
|
||
`tickerFrame.OriginX/OriginY` instead of a literal `0, 0` — `SceneCompositor` ×2 and
|
||
`FramePump`'s static-bake `Overlay` path. The preview binds the same numbers
|
||
(`AlertTickerLeft/Top/Width/Height` → `AlertTickerElement`'s `Canvas.Left/Top` +
|
||
`Width`/`Height`), so preview and output cannot drift — the same guarantee as the
|
||
branding credit. Product default box is **680×200 at (620, 430)**
|
||
(`MainViewModel.Sources.cs`); height is clamped to the natural 48px strip.
|
||
**No alert box in the scene ⇒ no ticker at all** — there is no global position left.
|
||
- **Why the position travels on the frame, not in a full-canvas frame:** a 1920×1080
|
||
overlay is 8.3MB of large-object-heap garbage per tick — ~2.5GB churned over one 10s
|
||
alert at 30fps. A box-sized strip is ~370KB. The branding flash *does* render
|
||
full-canvas, but it **recycles** one 8MB buffer and bumps `Epoch`, so it never
|
||
allocates per frame; the ticker allocates per call, so it must stay small.
|
||
- `CopyStrip` clips **rows** to the target height: the pill rasterises at its natural
|
||
48px, so a box shorter than 48px would otherwise write past the end of the buffer.
|
||
- **Three display methods** (`Source.AlertDisplayMethod`, DB column, panel "Display"
|
||
selector): `TickerScroll` (marquee), `Flash` (0.5s on / 0.5s off), `Solid` (centred,
|
||
still). The marquee is paced in **reads per alert** — `TickerReadsPerAlert = 3`
|
||
passes inside the alert's own length, speed derived from that — **never px/s**; the
|
||
old fixed `140px/s` took ~17s per pass, so a 10s alert showed the message once.
|
||
Same convention as the incumbent (Streamlabs: "Alert Duration" + "Text Delay", not a
|
||
scroll-speed slider). Flash's off half returns null.
|
||
- **A ticker still needs its own preview element, not a `Source`.** It is a frame-pump
|
||
producer, so it never rides a per-element `Image`; `AlertOverlayLayer` takes a
|
||
`tickerPreviewSink` and publishes from `RefreshAlertPreviews()` (UI thread only —
|
||
the pump thread must never touch a `WriteableBitmap`); the VM writes it into one
|
||
reused `WriteableBitmap` (`AlertTickerImageSource`/`AlertTickerVisible`) bound to
|
||
`AlertTickerElement` in `PreviewPane.xaml`, mirroring `SocialBarElement`. The rect
|
||
is re-raised on **every** published frame, not just on a bitmap resize, so a box
|
||
moved or resized mid-alert tracks live.
|
||
- **Two measurement traps here:** a seamless marquee never goes blank (a wrapped copy
|
||
enters as the pill clears), so count a pass by the pill's leading edge resetting, not
|
||
by an empty frame; and the run is phase-started half a frame in, or the first
|
||
published frame is blank (the pill sits exactly off the right edge).
|
||
- **Alert audio under the master limiter:** `AudioMixer.EnqueueAlertAudio` writes a
|
||
dedicated 8s stereo-48k ring (`AlertBufferSeconds`), drained in `FillAndMix` before the
|
||
limiter and added at **unity — never ducked** (creator ruling: "don't lower my game
|
||
audio for a tip"); `StartLive` clears it. Per-alert loudness = the Volume slider.
|
||
|
||
- **Alert sound byte (2026-11-01, creator ruling):** the alert layer can carry an audio file
|
||
(`AlertAudioPath`) that plays with the clip, plus `AlertAudioLoop` to repeat it under the clip
|
||
("Loop the sound until the video ends"). The video is always one pass to completion.
|
||
⛔ **Alert audio plays on the DEFAULT OUTPUT DEVICE, not into the mixer** (`Services/Audio/
|
||
AlertAudioOutput` for the clip's own PCM, `MediaPlayer` for the sound byte). That is deliberate:
|
||
only audio that reaches the speakers is captured by the WASAPI loopback, which is what makes it
|
||
audible to the creator *and* visible in the desktop sound bar. The old `EnqueueAlertAudio` path
|
||
produced sound that was in the stream but invisible in the meter and inaudible in the room.
|
||
Failure to open a device is logged and swallowed — a silent alert still renders.
|