LlamaCasty/docs/ai/overlays.md

Output overlays: social bar, alerts, chat

Everything composited over the frame: the social bar, the native alert box (six event animations, alert video, ticker), and the chat box conventions. Read before touching any of them or anything that composites a global layer.

Invariant: every global overlay must be composited on every render path — pump, preview/thumbnails, scene dump (SceneCompositor.Overlay is shared by all three). A feature wired into one path is silently absent from the output. That bug ("preview-only forever", 2026-09-26) is why the rule exists.

Social bar (TASK 6 — shipped 2026-08-12, plan in TASKS.md; bar bug-fix branch 2026-08-13)

A global bar layer (never a Source, no sources-list row) that sits over the bottom or top of the output and carries the creator's social links — content-sized, centered, GREEN glow when ON, position is a click-toggle top ⇄ bottom (default BOTTOM, persisted SocialBarPosition). Models/Socials.cs: SocialService enum (YouTube/Twitch/X/Instagram/TikTok/Facebook/Discord/Kick/Threads/Bluesky/GitHub/LinkedIn/Pinterest/ Snapchat/Reddit/WhatsApp/Telegram/Link/Website/Fediverse), SocialEntry (Service/Handle/ProfileUrl/ FediverseSoftware), SocialsConfig (Entries + BarPosition + BarEnabled; BarJustify dropped, column back-compat). Configured in the "Social Media Site Promotion" dialog (SocialsDialog.xaml + ViewModels/SocialsDialogViewModel, WPF-free, injected ISocialValidator + sign-in/sign-out fakes): ON/OFF bar switch (schema v8), 6 fixed slots — row 1 always YouTube (signed-in → channel handle; signed-out → sign-in gate → OAuth; delete → confirm sign-out), row 2 free, rows 3–6 lock icons on freemium. Service detection (SocialServiceIcons.DetectService): URL domain / fediverse @user@domain → Fediverse / bare→Website. Validation (Services/SocialValidator.cs, ISocialValidator seam + HttpSocialValidator default): async GET of the canonical profile URL; 200/redirect = valid, 404/failure = rejected. Fediverse additionally does a best-effort nodeinfo lookup (/.well-known/nodeinfo → software.name, stored in SocialEntry.FediverseSoftware / the SocialEntry.Software column, column-presence migration, no version bump) so the entry shows the instance's real logo (LogoDataForFediverse: mastodon/peertube/pixelfed/misskey/lemmy/pleroma/firefish, generic fediverse honeycomb fallback — Simple Icons CC0 path data, initials badges gone); nodeinfo failure still validates (glyph falls back). If the identity domain's nodeinfo is blocked (YunoHost SSO gates /.well-known/nodeinfo behind the login page) but the bare root 302s to the real instance, the lookup follows the root redirect and asks the resolved host. Cancel is a hard stop: LookupAsync takes a CancellationToken (dialog VM owns a CTS; Cancel/X/Save abort in-flight lookups, canceled continuations never touch slot state).

Bar bug-fix branch (2026-08-13) — three changes:

1. Positioning is a click-toggle (KISS) — the original drag set a local Canvas.SetTop value that permanently overrides the {Binding SocialBarTop} (a binding can never win over a local value), so the bar stayed wherever it was dropped. The first drag-snap fix (SocialBarSnap.Decide + ClearValue on release) still failed for shaky hands — jitter around the ±6px deadzone snapped the bar up but wouldn't let it come back down. Superseded by the user's click-toggle: clicking the bar flips it top ⇄ bottom (MainViewModel.ToggleSocialBarPosition), the bar rides the binding alone, and SocialBarSnap is gone.
2. Fediverse software self-heal — the DB row @gramps@llamachile.tube had Software = NULL because nodeinfo was only ever asked of the identity domain (a landing page; the real instance is mastodon.llamachile.tube). HttpSocialValidator.ResolveFediverseSoftwareAsync now probes well-known subdomains (mastodon. → social. → … FediverseSubdomainCandidates) when the identity domain and its redirect both come up empty, under a ~15s linked-CTS budget. MainViewModel runs the static HealFediverseSoftwareAsync off the UI thread on layout load, applies matches via the dispatcher, and saves; SocialEntry.FediverseSoftware is settable and raises LogoData, so the icon updates in place.
3. The bar renders on the live output — Compositor/SocialBarRenderer.cs rasterizes the entries into a transparent straight-alpha BGRA strip (1920-wide, 40px content + 24px glow pad, the green glow baked in, Pbgra32→straight-alpha unpremultiply) and the compositor blits it above the flash (see "Live frame pipeline").

Native events & alerts (TASK 43, shipped 2026-09-24 — StreamElements replaced)

The creator asked to replace the external StreamElements feed with native YouTube events; the answer is native events + a native alert box, and the product decisions are published decisions (recorded in TASKS/task-43-native-alerts.md):

• No third party. The chat feed IS the event feed: liveChat/messages carries all six monetization events (superChat / superSticker / newSponsor / membershipGifting / giftMembershipReceived / memberMilestoneChat). streamList is the connection semantics of that same endpoint (pollingIntervalMillis-driven), not a separate call — the service re-arms its one-shot poll on the server's cadence (clamp 1000–6000ms), maxResults=2000. Cite: https://developers.google.com/youtube/v3/live/docs/liveChatMessages/streamList

• SourceType.AlertBox — one "Stream Alerts" celebration zone per layout, OBS alert-box style: a designated draggable canvas area, idle = transparent. Mirrors the chat gate (CanAddAlerts; AddSource refuses a second). Defaults 620×430 → 680×200.

• Six events → six unique animations (Services/AlertRenderer.cs): SuperChat slide-up

  • gold count-up + shine sweep (green), SuperSticker scale-pop on a tilted chip (purple), NewMember drop-in + flash band (violet), MemberGift slide-left + gift-chip ×N fan (amber), GiftReceived confetti scale-in (green→purple), MemberMilestone rise + tenure growth bar (orange). Phases enter/hold/exit (~4–5s); every card draws the brand line "made with LlamaCasty!" — each broadcasted event is free product placement.

• Services/AlertOverlayLayer.cs — the component: enqueues events one-at-a-time, advances on a 33ms dispatch ticker, serves the live frame through RenderFrame (same cache-first pattern as chat) and the preview through UpdatePreview; Advance(double) is the deterministic test clock (tests drive playback with no timer). Queue cap 10 (floods drop the tail, never stall). ChatEventKind.None rows (plain chat, sub mentions) never enqueue — per the creator ruling, a subscriber mention is sufficient; there is no free-sub alert (YouTube emits none) and no viewer count (vetoed — demotivating, weaponizable).

• Alert box VIDEO (TASK 47, 2026-09-26): since TASK 47 the box plays a clip on every alert instead of (only) an animation — Enqueue→BeginClip spawns a per-play IAlertClipDecoder (ffmpeg rawvideo bgra -vf scale=W:H frame pipe + f32le audio pipe, both real-time paced via RawVideoFrameReader; not the refcounted MediaVideoSourceManager — alerts are one-shot plays that must be disposed at drain). Custom clip wins per-source over the built-in ONLY when AlertUseDefaultVideo==false AND the file exists; the shipped Assets/alert-default.mp4 is stamped into the Asset table at startup (AlertDefaultVideoKey setting; the prune keeps it via UNION) and read-at-play; any decode failure falls back to the six AlertRenderer animations (deliberate, not an error). Known overlap (recorded 2026-11-05): the video half of AlertClipDecoder.RunVideoAsync and MediaVideoSource.RunAsync are near-duplicates — same ffmpeg args (-f rawvideo -pix_fmt bgra -vf scale=W:H -an -), same RawVideoFrameReader feed loop, same IFrameRateProbe + delay pacing, both behind IDecodeProcess. What differs is ownership and role, not decoding: alerts are per-play and disposed at drain, media is a refcounted per-path session; alerts also run an f32le audio pipe into the mixer, media runs none at all (-an) — so a media file's soundtrack is silent today, and task-21 item 10 holds the two routes for changing that; alerts play once, media loops. A shared raw-video pump is the obvious consolidation if a third consumer appears — do not do it for two. Fades in/out via FadeDurationSeconds = 0.30 (AlertOverlayLayer const): fade-in by elapsed on straight-alpha frame copies; EOF → freeze-frame → fade-out → drain (the clip branch of Advance clears _current before AdvanceToNext — the miss that the TASK 47 test caught). Audio forwards to the mixer sink scaled by volume × fade.

• Ticker settings and alert feed: Models/Ticker.cs is shared by the AlertBox and standalone SourceType.Ticker layers. Each source owns a distinct settings instance (IsEnabled + DisplayMethod); the existing AlertShowTicker and AlertDisplayMethod columns persist those settings on their own Source row, so this is additive without a schema migration. The standalone ticker can be added once in each scene and shows the in-memory active event plus queued events. Completed events are not retained as history. The AlertBox ticker uses the same renderer/settings shape but shows only the active event.

• AlertBox ticker (announcement strip): AlertBoxTickerFrame composes "Author — Kind · amount" and hands it to Services/Compositor/AlertTickerRenderer.cs — a strip rasterized on the UI thread once per text (cached, unpremultiplied straight-alpha pill) then positioned by pure byte-math per call, so the pump calls it from its own thread with no locking. It is a dynamic overlay, NEVER baked (the social bar IS baked into BakeStaticBase): SceneCompositor.Render + CompositeLayers take a trailing tickerFrame threaded from FramePump._alertTicker (Func<Scene, VideoFrame?> ctor seam) through RenderScene/RenderFull and mixed into BuildFullRenderSignature so a scroll changes the cache key. It reads the active scene's AlertBox settings.

  • It draws INSIDE the Stream Alerts box (creator ruling 2026-09-26). Was a global 1920px bar pinned to the top edge: "the ticker should appear over the stream alerts video, not over the entire preview window". The strip is now rendered at the alert box's own size (AlertTickerRenderer.Render(..., width, height), defaults still 1920×48) and the box's origin rides on VideoFrame.Placement ((int X, int Y)?, with OriginX/OriginY = 0 when unset). Every blit site reads tickerFrame.OriginX/OriginY instead of a literal 0, 0 — SceneCompositor ×2 and FramePump's static-bake Overlay path. The preview binds the same numbers (AlertTickerLeft/Top/Width/Height → AlertTickerElement's Canvas.Left/Top + Width/Height), so preview and output cannot drift — the same guarantee as the branding credit. Product default box is 680×200 at (620, 430) (MainViewModel.Sources.cs); height is clamped to the natural 48px strip. No alert box in the scene ⇒ no ticker at all — there is no global position left.
  • Why the position travels on the frame, not in a full-canvas frame: a 1920×1080 overlay is 8.3MB of large-object-heap garbage per tick — ~2.5GB churned over one 10s alert at 30fps. A box-sized strip is ~370KB. The branding flash does render full-canvas, but it recycles one 8MB buffer and bumps Epoch, so it never allocates per frame; the ticker allocates per call, so it must stay small.
  • CopyStrip clips rows to the target height: the pill rasterises at its natural 48px, so a box shorter than 48px would otherwise write past the end of the buffer.
  • Three display methods (Source.AlertDisplayMethod, DB column, panel "Display" selector): TickerScroll (marquee), Flash (0.5s on / 0.5s off), Solid (centred, still). The marquee is paced in reads per alert — TickerReadsPerAlert = 3 passes inside the alert's own length, speed derived from that — never px/s; the old fixed 140px/s took ~17s per pass, so a 10s alert showed the message once. Same convention as the incumbent (Streamlabs: "Alert Duration" + "Text Delay", not a scroll-speed slider). Flash's off half returns null.
  • AlertBox preview: tickerPreviewSink publishes from RefreshAlertPreviews() (UI thread only — the pump thread must never touch a WriteableBitmap); the VM writes the box-sized frame into the reused bitmap bound to AlertTickerElement in PreviewPane.xaml. The rect is re-raised on every published frame so a box moved or resized mid-alert tracks live.
  • Standalone ticker preview/output: it is an ordinary dynamic Source with a per-element frame. AlertOverlayLayer updates each scene ticker's reusable VideoImageSource on its UI timer for preview; the output resolver renders the same source with that source's Ticker settings. It is not the AlertBox global ticker callback, so each scene's layer and settings remain independent.
  • Two measurement traps here: a seamless marquee never goes blank (a wrapped copy enters as the pill clears), so count a pass by the pill's leading edge resetting, not by an empty frame; and the run is phase-started half a frame in, or the first published frame is blank (the pill sits exactly off the right edge).

• Alert audio under the master limiter: AudioMixer.EnqueueAlertAudio writes a dedicated 8s stereo-48k ring (AlertBufferSeconds), drained in FillAndMix before the limiter and added at unity — never ducked (creator ruling: "don't lower my game audio for a tip"); StartLive clears it. Per-alert loudness = the Volume slider.

• Alert sound byte (2026-11-01, creator ruling): the alert layer can carry an audio file (AlertAudioPath) that plays with the clip, plus AlertAudioLoop to repeat it under the clip ("Loop the sound until the video ends"). The video is always one pass to completion. ⛔ Alert audio plays on the DEFAULT OUTPUT DEVICE, not into the mixer (Services/Audio/ AlertAudioOutput for the clip's own PCM, MediaPlayer for the sound byte). That is deliberate: only audio that reaches the speakers is captured by the WASAPI loopback, which is what makes it audible to the creator and visible in the desktop sound bar. The old EnqueueAlertAudio path produced sound that was in the stream but invisible in the meter and inaudible in the room. Failure to open a device is logged and swallowed — a silent alert still renders.