gramps
a311c8feb1
ai.md 1,308 → 228 lines. No prose rewritten: every block was moved as-is so nothing could be lost in translation, then replaced by a pointer. New files: - docs/ai/environment.md — LAN/SSH + Gitea push recipe, Windows dotnet build/run, test-suite map, hermetic real-MainWindow test rule - docs/ai/product.md — brand/naming, AI transparency, GPL licensing prohibitions, free-forever monetization, auth/account assumptions, packaging & distribution - docs/ai/youtube-api.md — go-live ordering, the broadcast-binding fix, the private-only lock, health parsing, end-stream pre-flight, Test mode, chat insert shape - docs/ai/overlays.md — social bar, native alert box (animations, clip, ticker), chat box conventions - docs/ai/ui-surface.md — control surface, scene strip, preview rail, hotkeys, broadcast drawer, toasts, transitions, brand logo, launch geometry, theme/properties conventions ai.md keeps the response/derivation rules, the architecture table, the four existing invariant stubs, and one pointer per moved subsystem. docs/ai/index.md routes every file with a "read when" column. References falsified by the move were fixed in the same commit: AGENTS.md build note → environment.md → Run, and the "Related" footers in frame-pipeline.md / output.md / capture.md now point at the owning files. Docs-only unit: scope-check run, no build (no source touched).
155 lines
12 KiB
Markdown
155 lines
12 KiB
Markdown
# Output overlays: social bar, alerts, chat
|
||
|
||
Everything composited *over* the frame: the social bar, the native alert box (six event
|
||
animations, alert video, ticker), and the chat box conventions. Read before touching any of
|
||
them or anything that composites a global layer.
|
||
|
||
**Invariant:** every global overlay must be composited on **every** render path — pump,
|
||
preview/thumbnails, scene dump (`SceneCompositor.Overlay` is shared by all three). A feature
|
||
wired into one path is silently absent from the output. That bug ("preview-only forever",
|
||
2026-09-26) is why the rule exists.
|
||
|
||
## Social bar (TASK 6 — shipped 2026-08-12, plan in TASKS.md; bar bug-fix branch 2026-08-13)
|
||
|
||
A **global bar layer** (never a Source, no sources-list row) that sits over the bottom or top of the
|
||
output and carries the creator's social links — content-sized, centered, GREEN glow when ON, position is a
|
||
**click-toggle top ⇄ bottom** (default BOTTOM, persisted `SocialBarPosition`). `Models/Socials.cs`: `SocialService` enum
|
||
(YouTube/Twitch/X/Instagram/TikTok/Facebook/Discord/Kick/Threads/Bluesky/GitHub/LinkedIn/Pinterest/
|
||
Snapchat/Reddit/WhatsApp/Telegram/Link/Website/**Fediverse**), `SocialEntry` (Service/Handle/ProfileUrl/
|
||
`FediverseSoftware`), `SocialsConfig` (Entries + `BarPosition` + `BarEnabled`; `BarJustify` dropped,
|
||
column back-compat). Configured in the "Social Media Site Promotion" dialog (`SocialsDialog.xaml` +
|
||
`ViewModels/SocialsDialogViewModel`, WPF-free, injected `ISocialValidator` + sign-in/sign-out fakes):
|
||
ON/OFF bar switch (schema v8), 6 fixed slots — row 1 always YouTube (signed-in → channel handle;
|
||
signed-out → sign-in gate → OAuth; delete → confirm sign-out), row 2 free, rows 3–6 lock icons on
|
||
freemium. Service detection (`SocialServiceIcons.DetectService`): URL domain / fediverse `@user@domain` →
|
||
**Fediverse** / bare→Website. Validation (`Services/SocialValidator.cs`, `ISocialValidator` seam +
|
||
`HttpSocialValidator` default): async GET of the canonical profile URL; 200/redirect = valid, 404/failure =
|
||
rejected. Fediverse additionally does a **best-effort nodeinfo lookup** (`/.well-known/nodeinfo` →
|
||
`software.name`, stored in `SocialEntry.FediverseSoftware` / the `SocialEntry.Software` column,
|
||
column-presence migration, no version bump) so the entry shows the **instance's real logo**
|
||
(`LogoDataForFediverse`: mastodon/peertube/pixelfed/misskey/lemmy/pleroma/firefish, generic fediverse
|
||
honeycomb fallback — Simple Icons CC0 path data, initials badges gone); nodeinfo failure still validates
|
||
(glyph falls back). If the identity domain's nodeinfo is blocked (YunoHost SSO gates
|
||
`/.well-known/nodeinfo` behind the login page) but the bare root 302s to the real instance, the lookup
|
||
**follows the root redirect** and asks the resolved host. Cancel is a hard stop: `LookupAsync` takes a
|
||
`CancellationToken` (dialog VM owns a CTS; Cancel/X/Save abort in-flight lookups, canceled continuations
|
||
never touch slot state).
|
||
|
||
**Bar bug-fix branch (2026-08-13) — three changes:**
|
||
|
||
1. **Positioning is a click-toggle (KISS)** — the original drag set a local `Canvas.SetTop` value that
|
||
permanently overrides the `{Binding SocialBarTop}` (a binding can never win over a local value), so the
|
||
bar stayed wherever it was dropped. The first drag-snap fix (`SocialBarSnap.Decide` + `ClearValue` on
|
||
release) still failed for shaky hands — jitter around the ±6px deadzone snapped the bar up but wouldn't
|
||
let it come back down. **Superseded by the user's click-toggle:** clicking the bar flips it top ⇄ bottom
|
||
(`MainViewModel.ToggleSocialBarPosition`), the bar rides the binding alone, and `SocialBarSnap` is gone.
|
||
2. **Fediverse software self-heal** — the DB row `@gramps@llamachile.tube` had `Software = NULL` because
|
||
nodeinfo was only ever asked of the identity domain (a landing page; the real instance is
|
||
`mastodon.llamachile.tube`). `HttpSocialValidator.ResolveFediverseSoftwareAsync` now **probes
|
||
well-known subdomains** (`mastodon.` → `social.` → … `FediverseSubdomainCandidates`) when the identity
|
||
domain and its redirect both come up empty, under a ~15s linked-CTS budget. `MainViewModel` runs the
|
||
static `HealFediverseSoftwareAsync` off the UI thread on layout load, applies matches via the dispatcher,
|
||
and saves; `SocialEntry.FediverseSoftware` is settable and raises `LogoData`, so the icon updates in place.
|
||
3. **The bar renders on the live output** — `Compositor/SocialBarRenderer.cs` rasterizes the entries into a
|
||
transparent straight-alpha BGRA strip (1920-wide, 40px content + 24px glow pad, the green glow baked in,
|
||
Pbgra32→straight-alpha unpremultiply) and the compositor blits it above the flash (see "Live frame
|
||
pipeline").
|
||
|
||
## Native events & alerts (TASK 43, shipped 2026-09-24 — StreamElements replaced)
|
||
|
||
The creator asked to replace the external StreamElements feed with native YouTube events;
|
||
the answer is **native events + a native alert box**, and the product decisions are
|
||
published decisions (recorded in `TASKS/task-43-native-alerts.md`):
|
||
|
||
- **No third party.** The chat feed IS the event feed: `liveChat/messages` carries all six
|
||
monetization events (superChat / superSticker / newSponsor / membershipGifting /
|
||
giftMembershipReceived / memberMilestoneChat). `streamList` is the *connection semantics*
|
||
of that same endpoint (pollingIntervalMillis-driven), not a separate call — the service
|
||
re-arms its one-shot poll on the server's cadence (clamp 1000–6000ms), `maxResults=2000`.
|
||
Cite: https://developers.google.com/youtube/v3/live/docs/liveChatMessages/streamList
|
||
- **`SourceType.AlertBox` — one "Stream Alerts" celebration zone per layout**, OBS
|
||
alert-box style: a designated draggable canvas area, idle = transparent. Mirrors the chat
|
||
gate (`CanAddAlerts`; `AddSource` refuses a second). Defaults 620×430 → 680×200.
|
||
- **Six events → six unique animations** (`Services/AlertRenderer.cs`): SuperChat slide-up
|
||
+ gold count-up + shine sweep (green), SuperSticker scale-pop on a tilted chip (purple),
|
||
NewMember drop-in + flash band (violet), MemberGift slide-left + gift-chip `×N` fan
|
||
(amber), GiftReceived confetti scale-in (green→purple), MemberMilestone rise + tenure
|
||
growth bar (orange). Phases enter/hold/exit (~4–5s); every card draws the **brand line
|
||
"made with LlamaCasty!"** — each broadcasted event is free product placement.
|
||
- **`Services/AlertOverlayLayer.cs`** — the component: enqueues events one-at-a-time,
|
||
advances on a 33ms dispatch ticker, serves the live frame through `RenderFrame` (same
|
||
cache-first pattern as chat) and the preview through `UpdatePreview`; **`Advance(double)`
|
||
is the deterministic test clock** (tests drive playback with no timer). Queue cap 10
|
||
(floods drop the tail, never stall). `ChatEventKind.None` rows (plain chat, sub mentions)
|
||
**never enqueue** — per the creator ruling, a subscriber mention is sufficient; there is
|
||
no free-sub alert (YouTube emits none) and no viewer count (vetoed — demotivating,
|
||
weaponizable).
|
||
- **Alert box VIDEO (TASK 47, 2026-09-26):** since TASK 47 the box plays a clip on every
|
||
alert instead of (only) an animation — `Enqueue`→`BeginClip` spawns a **per-play**
|
||
`IAlertClipDecoder` (ffmpeg `rawvideo bgra -vf scale=W:H` frame pipe + `f32le` audio
|
||
pipe, both real-time paced via `RawVideoFrameReader`; **not** the refcounted
|
||
`MediaVideoSourceManager` — alerts are one-shot plays that must be disposed at drain).
|
||
Custom clip wins per-source over the built-in ONLY when `AlertUseDefaultVideo==false`
|
||
AND the file exists; the shipped `Assets/alert-default.mp4` is stamped into the `Asset`
|
||
table at startup (`AlertDefaultVideoKey` setting; the prune keeps it via UNION) and
|
||
read-at-play; any decode failure falls back to the six `AlertRenderer` animations
|
||
(deliberate, not an error). Fades in/out via **`FadeDurationSeconds = 0.30`**
|
||
(`AlertOverlayLayer` const): fade-in by `elapsed` on straight-alpha frame copies;
|
||
EOF → freeze-frame → fade-out → drain (the clip branch of `Advance` **clears
|
||
`_current` before `AdvanceToNext`** — the miss that the TASK 47 test caught). Audio
|
||
forwards to the mixer sink scaled by `volume × fade`.
|
||
- **Alert ticker (announcement strip):** `AlertTickerFrame` (in the layer) composes
|
||
"Author — Kind · amount" and hands it to `Services/Compositor/AlertTickerRenderer.cs` —
|
||
a strip rasterized **on the UI thread once per text** (cached, unpremultiplied
|
||
straight-alpha pill) then positioned by pure byte-math per call, so the pump calls it
|
||
from its own thread with no locking. **It is a dynamic overlay, NEVER baked** (the
|
||
social bar IS baked into `BakeStaticBase`): `SceneCompositor.Render` +
|
||
`CompositeLayers` take a trailing `tickerFrame` threaded from
|
||
`FramePump._alertTicker` (`Func<VideoFrame?>` ctor seam) through
|
||
`RenderScene`/`RenderFull` and **mixed into `BuildFullRenderSignature`** so a scroll
|
||
changes the cache key. Toggled by `AlertShowTicker`.
|
||
- **It draws INSIDE the Stream Alerts box (creator ruling 2026-09-26).** Was a global
|
||
1920px bar pinned to the top edge: *"the ticker should appear over the stream alerts
|
||
video, not over the entire preview window"*. The strip is now rendered at the
|
||
**alert box's own size** (`AlertTickerRenderer.Render(..., width, height)`, defaults
|
||
still 1920×48) and the box's origin rides on **`VideoFrame.Placement`**
|
||
(`(int X, int Y)?`, with `OriginX`/`OriginY` = 0 when unset). Every blit site reads
|
||
`tickerFrame.OriginX/OriginY` instead of a literal `0, 0` — `SceneCompositor` ×2 and
|
||
`FramePump`'s static-bake `Overlay` path. The preview binds the same numbers
|
||
(`AlertTickerLeft/Top/Width/Height` → `AlertTickerElement`'s `Canvas.Left/Top` +
|
||
`Width`/`Height`), so preview and output cannot drift — the same guarantee as the
|
||
branding credit. Product default box is **680×200 at (620, 430)**
|
||
(`MainViewModel.Sources.cs`); height is clamped to the natural 48px strip.
|
||
**No alert box in the scene ⇒ no ticker at all** — there is no global position left.
|
||
- **Why the position travels on the frame, not in a full-canvas frame:** a 1920×1080
|
||
overlay is 8.3MB of large-object-heap garbage per tick — ~2.5GB churned over one 10s
|
||
alert at 30fps. A box-sized strip is ~370KB. The branding flash *does* render
|
||
full-canvas, but it **recycles** one 8MB buffer and bumps `Epoch`, so it never
|
||
allocates per frame; the ticker allocates per call, so it must stay small.
|
||
- `CopyStrip` clips **rows** to the target height: the pill rasterises at its natural
|
||
48px, so a box shorter than 48px would otherwise write past the end of the buffer.
|
||
- **Three display methods** (`Source.AlertDisplayMethod`, DB column, panel "Display"
|
||
selector): `TickerScroll` (marquee), `Flash` (0.5s on / 0.5s off), `Solid` (centred,
|
||
still). The marquee is paced in **reads per alert** — `TickerReadsPerAlert = 3`
|
||
passes inside the alert's own length, speed derived from that — **never px/s**; the
|
||
old fixed `140px/s` took ~17s per pass, so a 10s alert showed the message once.
|
||
Same convention as the incumbent (Streamlabs: "Alert Duration" + "Text Delay", not a
|
||
scroll-speed slider). Flash's off half returns null.
|
||
- **A ticker still needs its own preview element, not a `Source`.** It is a frame-pump
|
||
producer, so it never rides a per-element `Image`; `AlertOverlayLayer` takes a
|
||
`tickerPreviewSink` and publishes from `RefreshAlertPreviews()` (UI thread only —
|
||
the pump thread must never touch a `WriteableBitmap`); the VM writes it into one
|
||
reused `WriteableBitmap` (`AlertTickerImageSource`/`AlertTickerVisible`) bound to
|
||
`AlertTickerElement` in `PreviewPane.xaml`, mirroring `SocialBarElement`. The rect
|
||
is re-raised on **every** published frame, not just on a bitmap resize, so a box
|
||
moved or resized mid-alert tracks live.
|
||
- **Two measurement traps here:** a seamless marquee never goes blank (a wrapped copy
|
||
enters as the pill clears), so count a pass by the pill's leading edge resetting, not
|
||
by an empty frame; and the run is phase-started half a frame in, or the first
|
||
published frame is blank (the pill sits exactly off the right edge).
|
||
- **Alert audio under the master limiter:** `AudioMixer.EnqueueAlertAudio` writes a
|
||
dedicated 8s stereo-48k ring (`AlertBufferSeconds`), drained in `FillAndMix` before the
|
||
limiter and added at **unity — never ducked** (creator ruling: "don't lower my game
|
||
audio for a tip"); `StartLive` clears it. Per-alert loudness = the Volume slider.
|
||
|