Compare commits

...
20 Commits
Author SHA1 Message Date
gramps 931ab46783 Pinned chat message + brand wordmark assets. Chat can pin one message to a banner above the feed: ChatMessage.IsPinned, MainViewModel.PinnedChatMessage + Pin/ClearPinnedChatMessageCommand (Helpers/RelayCommandGeneric.cs), LeftPanel pinned-message border bound via NotNullToVis with the feed nudged down. 2026-10-10 14:54:08 -07:00
gramps 8e1d6f44db Scene strip tile artwork + on-air indicator (creator ruling 2026-10-10). Top-bar tiles now carry their own art: each canonical scene's tile is a per-scene CRT-styled image stored in the internal DB (Scene.ThumbnailAssetId - additive migration, seeded at heal from the creator's five Downloads JPGs via SceneThumbnailSeedPaths), and the preview pane keeps rendering the scene's ORIGINAL background - never the tiles. One shared on-air image (Gemini 4zajyl...jpg) replaces the staged scene's tile and follows the scene swap ('the on-air screen jumps around the top bar as the user swaps scenes'): stored as an Asset BLOB tracked by the OnAirImageAssetId Setting (EnsureOnAirImage, same contract as the alert clip) with MainViewModel.OnAirSeedPath override; RefreshSnapshotsAsync shows it for the staged scene, otherwise thumbnail-or-background art, and a staged Live with no on-air art still falls back to the green placeholder (minis never render live captures). Removed the 6px #22c55e rectangle in PreviewPane's backdrop ('green outline around the preview screen' that surfaced when a scene had no background) - the outer PreviewGlow red border is the real live signal. Doubled the scene-tile labels (FontSize 10 -> 20). Fixed a real bug this surfaced: Save()'s Asset pruner only kept Source/alert-clip rows, so the first save would have DELETED the seeded tile + on-air assets - the keep-set now includes Scene.ThumbnailAssetId and the OnAirImageAssetId Setting. New Settings/Load/Save migrations + SceneThumbnailTests (4 tests: seeded tile in strip, preview keeps original bg, Live-staged tile, no-seed fallback, on-air swap + persistence). 427 pass, 0 build warnings. (Unit: strip tiles + on-air) 2026-10-10 13:27:26 -07:00
gramps 06f88f4142 Scene strip now renders in canonical F-key order regardless of persisted SortOrder. The strip loads Scenes ORDER BY SortOrder while each tab's F-key label is name-derived, so a session that persisted Chat before BRB (found on the creator's DB: SortOrder 2/3) showed tabs 'Chat F4' then 'BRB F3'. MainViewModel.LoadLayout now calls the new ReorderScenesToCanonical() - the five canonical scenes reorder to Starting, Live, BRB, Chat, Ending (F1-F5, stable among renamed non-canonical scenes), and the next auto-save re-stamps SortOrder so the heal persists. One integration test (SceneOrderTests) seeds the swapped-order DB and asserts loaded order, F1-F5 labels, and that a save persists the corrected ordering. 423 pass, 0 build warnings. (Unit: scene order) 2026-10-10 12:09:07 -07:00
gramps 63bf8444bd Desktop 'lock' fix: the window no longer steals focus when a full-screen game owns the foreground at launch. Opening after an exclusive-fullscreen game (creator's THE DIVISION 2 gameplay-stream start) the window ACTIVATED by default, Windows dropped the game to the taskbar and the 1920x1040 window covered the whole work area - 'the desktop is locked and I can't reach the game with the mouse'. MainWindow ctor now sets ShowActivated = !MainViewModel.FullscreenGameRunningNow (GetForegroundFullScreenMonitorIndex() != null), so an in-game launch keeps the game focused and live; a normal launch is unchanged. New FullScreenDetectorOverride test seam (mirrors WindowEnumeratorOverride - deterministic, no real Win32 foreground state) + DesktopLockTests: game at launch -> ShowActivated=false, normal launch -> true, FullscreenGameRunningNow reflects the detector. 422 pass / 1 known env flake, 0 build warnings. (Unit: launch focus) 2026-10-10 11:52:17 -07:00
gramps c2d64dcd6d TASK 51 blur 'heavy' fix: quadratic blur radius. BlurRadius(strength) = 1 + 5s^2, MaxBlurRadius 12 -> 6 downscaled px (~24 full px max). Default 0.5 is now radius 2 small px (was the old linear 6) - ~3x lighter perceived spread, matching the creator's real-camera finding. Range from the established tools: StreamFX defaults its Box blur Size to 5 and its author stops downscale-based blurs around 8px. One integration test pins the mapping (0 -> 1, 0.5 -> 2, 1 -> 6) and step-edge behaviour (light reaches 9px at default vs 26px at full); the existing slice-2 blur test passes unchanged. 419/419 pass. Refs: https://github.com/Vhonowslend/StreamFX-Public/wiki/Filter-Blur https://xaymar.com/articles/2020/04/26/optimizing-streamfx-in-your-obs-scenes/ (Unit: bg-blur heavy fix) 2026-10-10 07:08:01 -07:00
gramps 305706956e TASK 51 jitter fix: temporal mask smoothing. MaskSmoother blends each raw segmentation 0.75xnew + 0.25xprevious at the one CameraManager worker (state per CameraSession, not in the shared segmenter); null resets history, first frame passes through, input array never aliased, ping-pong buffers (no LOH alloc for a 147KB 256x144 mask). Creator's real-camera check: Remove was 'jittery' (per-frame independent segmentation). Weights from the OBS/Jitsi consensus. One integration test asserts the second frame keeps partial alpha (x=6 -> 58) where the raw mask cuts to 0. 416 pass / 2 known env flakes. Refs: https://deepwiki.com/royshil/obs-backgroundremoval/2-core-features https://github.com/jitsi/jitsi-meet/issues/17080 (Unit: bg-removal jitter fix) 2026-10-09 22:15:59 -07:00
gramps 5685c5dc3a TRAX: toggle refreshes the status light on play/pause, terse scrolling error on failure, fixed quiet volume (0.5), ITraxPlayer seam. Deadlocked test fixed via RunAsync (was GetAwaiter().GetResult on the UI thread). 416 pass / 1 known env flake. Refs: https://learn.microsoft.com/en-us/dotnet/standard/asynchronous-programming-patterns/async-programming-model-tap#dont-block-on-async-code (Unit: trax toggle) 2026-10-09 19:05:04 -07:00
gramps 87f05ba2ce Private broadcasts end like recordings: Enter keeps, Escape deletes (KeepBroadcastDialog). Privacy read from YouTube; delete = videos.delete (fallback liveBroadcasts.delete on 404). Refs: https://developers.google.com/youtube/v3/docs/videos/delete https://developers.google.com/youtube/v3/live/docs/liveBroadcasts/delete (Unit: private broadcast close-out) 2026-10-09 13:32:38 -07:00
gramps b140024ebf Audio: speech leveler (auto gain to -20 dBFS speech, -6..+20 dB, gated) between gate and compressor; honest mic meter (drop the +10 dB display boost and x1.2). Creator upload measured -28.5 LUFS; same audio through the new chain: -17.9 LUFS / -0.7 dBFS peak (Unit: audio loudness) 2026-10-09 13:27:53 -07:00
gramps 8303b1cf7f Brand flash OFF by default: MainViewModel.ShowBrandFlash master switch (default false) gates BrandFlashEnabled; feature kept, tests opt in; new test guards the default (Unit: brand flash off) 2026-10-09 13:22:52 -07:00
gramps 352f117cb2 TASK 51 slice 5: webcam Background section in the Layers properties — Remove/Blur PillToggles (radio-exclusive) + Strength slider; strength changes never blank the webcam (new variant inherits the same-effect frame + bitmap). TASK 51 built (Unit: bg-effects UI) 2026-10-09 10:31:11 -07:00
gramps de22983b36 TASK 51 slice 4: camera-path hookup — CameraManager renders per-scene effect variants (one segmentation per raw frame on a latest-frame-drop worker, raw path untouched), VM declares/adopts variants per config; separable upsampling cuts Remove 46->3 ms, Blur 80->18 ms (Debug, 720p) (Unit: bg-effects hookup) 2026-10-09 10:27:38 -07:00
gramps 55210b9a61 TASK 51 slice 3: per-scene webcam BackgroundEffect + BackgroundStrength on WebcamSceneConfig, persisted via additive guarded columns; radio-pill bools RemoveBackground/BlurBackground (Unit: bg-effect config) 2026-10-09 10:19:48 -07:00
gramps 227ee8fca6 TASK 51 slice 2: Blur — background at 1/4 res, 3 separable box passes (~gaussian, radius by strength), mask-blended; person untouched; Remove gains strength = edge feather; mask computed once, rendered per effect (Unit: bg blur) 2026-10-09 10:18:00 -07:00
gramps fbeb63ba79 Docs: queue TASK 56 — pre-publication secret scan of the full git history (gate before the repo goes public) (Unit: task catalog) 2026-10-09 10:15:33 -07:00
gramps 4e42871fae Docs: README rewritten in the creator's voice (why it was built, generic comparison, free + open, real build-from-source steps incl. ffmpeg fetch + own OAuth client); source offer URL llgit.llamachile.tube/gramps/LlamaCasty in README + notices; TASK 51 keeps the strength slider (Unit: README + source offer) 2026-10-09 10:13:43 -07:00
gramps db2fdbf68d Docs: GPL-3.0-or-later (creator ruling); AI-generated scene backgrounds + alert video are GPL placeholders, not reserved; TASK 51 UI spec — Remove/Blur as radio-exclusive pill switches in the webcam's Layers properties (Unit: license follow-up + bg-effects UI spec) 2026-10-09 10:11:11 -07:00
gramps c6569a7ccb License: LlamaCasty is GPL-3.0-only with section 7 terms (attribution, rename modified versions, no trademark rights); icon + logo all rights reserved; LICENSE + ADDITIONAL-TERMS ship beside the exe; notices + ai.md rewritten off the paid/closed-source framing (Unit: GPL-3.0 licensing) 2026-10-09 10:05:12 -07:00
gramps f529d47f42 Build: require 64-bit Windows — PlatformTarget x64 in app + tests (bundled ffmpeg is win64, ONNX Runtime has no win-x86, 32-bit address space can't hold 1080p buffers) (Unit: 64-bit requirement) 2026-10-09 10:01:38 -07:00
gramps 9f3c154d84 TASK 51 slice 1: background-effects engine — bundled MediaPipe Selfie Segmenter (ONNX) + Remove (Unit: bg-effects engine)
- Model: Google's official MediaPipe Selfie Segmenter (landscape, Apache-2.0 per model card),
  converted to ONNX by tools/selfie-seg/convert_tflite_to_onnx.py (custom op
  Convolution2DTransposeBias -> ConvTranspose) and verified against MediaPipe's own runtime
  (verify_against_mediapipe.py: max |diff| 5e-5, 100% binary agreement). Embedded resource, 430 KB.
- Services/BackgroundEffects: IPersonSegmenter seam, OnnxPersonSegmenter (onnxruntime 1.31.0 CPU,
  never throws), BackgroundEffectProcessor (None passthrough, Remove = alpha mask). Not wired yet.
- Tests: Remove integration test (fake segmenter) + bundled-model contract test vs MediaPipe
  reference mask. Full suite 407/408 — the one failure is the documented interactive-desktop flake
  RealMouseDrag_OnTheLayerList_PersistsTheReorder (passed in the run before; code path untouched).
- Docs: ai.md reversal + section + licensing guard, THIRD-PARTY-NOTICES 11-12, TASK 51, indexes,
  MyMistakes recipe, HANDOFF.

References: obs-backgroundremoval (approach only — ONNX Runtime on CPU, 2 threads; GPL-3.0, no code
used) https://github.com/royshil/obs-backgroundremoval ; MediaPipe image segmenter
https://ai.google.dev/edge/mediapipe/solutions/vision/image_segmenter
2026-10-09 09:24:13 -07:00
75 changed files with 4542 additions and 243 deletions
+54
View File
@@ -0,0 +1,54 @@
# LlamaCasty — copyright, license, and additional terms
Copyright (C) 2026 Gramps — https://llamacasty.com
LlamaCasty is free software: you can redistribute it and/or modify it under the terms of the
**GNU General Public License as published by the Free Software Foundation, either version 3 of
the License, or (at your option) any later version** (GPL-3.0-or-later), together with the
additional terms below, which section 7 of that license permits.
The full license text is in [`LICENSE`](LICENSE).
LlamaCasty is distributed in the hope that it will be useful, but **WITHOUT ANY WARRANTY**;
without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
Third-party components bundled with LlamaCasty keep their own licenses — see
[`THIRD-PARTY-NOTICES.txt`](THIRD-PARTY-NOTICES.txt).
---
## Additional terms (GPL-3.0 section 7)
1. **Attribution — section 7(b).** Every conveyed copy of this program, modified or not, must keep
the copyright notice above and must show, in its About screen (or the equivalent place in its
user interface), the notice:
> Based on LlamaCasty by Gramps — https://llamacasty.com
2. **Modified versions — section 7(c).** A modified version must be clearly marked as modified,
must use a name other than "LlamaCasty", and must not be represented as the original LlamaCasty
or as made or endorsed by Gramps.
3. **Trademarks — section 7(e).** This license grants no right to use the LlamaCasty name, the
llama logo, or any other LlamaCasty brand mark, except for the factual attribution required by
term 1.
## Brand artwork — not licensed under the GPL
The following files are Copyright (C) 2026 Gramps, **all rights reserved**, and are **not**
covered by GPL-3.0:
- `Assets/llamacasty-icon.png`
- `Assets/llamacasty-icon.ico`
- `Assets/llamacasty-logo.jpg`
Unmodified copies of LlamaCasty may be redistributed with these files included. Modified versions
must replace them with their own artwork. No other use — including on merchandise — is permitted
without written permission.
## Placeholder artwork — licensed with the code
The scene background images (`Assets/*-background.jpg`) and the default alert video
(`Assets/alert-default.mp4`) are AI-generated placeholders, there so the app works the moment it
opens. Creators are expected to replace them with their own. They are distributed with the code
under GPL-3.0-or-later and carry none of the restrictions above.
Binary file not shown.
Binary file not shown.

After

Width:  |  Height:  |  Size: 660 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 717 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 560 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 553 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 378 KiB

+42 -1
View File
@@ -284,6 +284,37 @@
<TextBlock Text="{Binding SelectedElement.BorderWidth}" Foreground="#a0a0b0" FontSize="11"
VerticalAlignment="Center" Margin="8,0,0,0" Width="20"/>
</DockPanel>
<!-- TASK 51: background effects — two pills acting as radio buttons
(one on turns the other off; both off = no effect). -->
<TextBlock Text="Background" Foreground="#a0a0b0" FontSize="11" FontWeight="SemiBold" Margin="0,6,0,6"/>
<DockPanel Margin="0,0,0,4">
<TextBlock DockPanel.Dock="Left" Text="Remove" Foreground="#a0a0b0" FontSize="11" VerticalAlignment="Center" Width="84"/>
<ToggleButton x:Name="RemoveBackgroundPill" Style="{StaticResource PillToggle}"
IsChecked="{Binding SelectedElement.RemoveBackground, Mode=TwoWay}"
AutomationProperties.Name="Remove background"
ToolTip="Remove the background behind you — the layers below show through"
HorizontalAlignment="Left"/>
</DockPanel>
<DockPanel Margin="0,0,0,4">
<TextBlock DockPanel.Dock="Left" Text="Blur" Foreground="#a0a0b0" FontSize="11" VerticalAlignment="Center" Width="84"/>
<ToggleButton x:Name="BlurBackgroundPill" Style="{StaticResource PillToggle}"
IsChecked="{Binding SelectedElement.BlurBackground, Mode=TwoWay}"
AutomationProperties.Name="Blur background"
ToolTip="Blur the background behind you"
HorizontalAlignment="Left"/>
</DockPanel>
<DockPanel Margin="0,0,0,6"
Visibility="{Binding SelectedElement.HasBackgroundEffect, Converter={StaticResource BoolToVis}}">
<TextBlock DockPanel.Dock="Left" Text="Strength" Foreground="#a0a0b0" FontSize="11" VerticalAlignment="Center" Width="84"/>
<TextBlock DockPanel.Dock="Right" Text="{Binding SelectedElement.BackgroundStrength, StringFormat={}{0:P0}}"
Foreground="#a0a0b0" FontSize="11" VerticalAlignment="Center" Margin="8,0,0,0" Width="34"/>
<Slider x:Name="BackgroundStrengthSlider" Minimum="0" Maximum="1"
Value="{Binding SelectedElement.BackgroundStrength, Mode=TwoWay}"
TickFrequency="0.05" IsSnapToTickEnabled="True" VerticalAlignment="Center"
AutomationProperties.Name="Background effect strength"
ToolTip="Blur: how strong the blur is · Remove: how soft the edge is"/>
</DockPanel>
</StackPanel>
<!-- Countdown config -->
@@ -565,8 +596,18 @@
HorizontalAlignment="Center" TextAlignment="Center" MaxWidth="240"/>
</StackPanel>
<!-- Pinned message -->
<Border Grid.Row="1" VerticalAlignment="Top" Margin="4,4,4,0" Padding="6,4" CornerRadius="4"
Background="#1a2a3a" BorderBrush="#38bdf8" BorderThickness="1"
Visibility="{Binding PinnedChatMessage, Converter={StaticResource NotNullToVis}}">
<StackPanel>
<TextBlock Text="{Binding PinnedChatMessage.AuthorName}" Foreground="#38bdf8" FontWeight="SemiBold" FontSize="11"/>
<TextBlock Text="{Binding PinnedChatMessage.Message}" Foreground="#e0e0e0" FontSize="11" TextWrapping="Wrap"/>
</StackPanel>
</Border>
<!-- Message feed (shown when live) -->
<ListBox Grid.Row="1" Background="Transparent" BorderThickness="0"
<ListBox Grid.Row="1" Background="Transparent" BorderThickness="0" Margin="0,40,0,0"
ItemsSource="{Binding ChatMessages}"
ItemContainerStyle="{StaticResource YtListBoxItem}"
Visibility="{Binding ShowChatInactiveMessage, Converter={StaticResource InverseBoolToVis}}"
+22 -7
View File
@@ -69,7 +69,6 @@
<Image Source="{Binding BackgroundImage}" Stretch="UniformToFill"
Visibility="{Binding BackgroundVisible, Converter={StaticResource BoolToVis}}"/>
<Image Source="{Binding ActiveBackgroundImage}" Stretch="UniformToFill"/>
<Rectangle Stroke="#22c55e" StrokeThickness="6"/>
</Grid>
<Canvas x:Name="OverlayCanvas" Width="1920" Height="1080">
<ItemsControl ItemsSource="{Binding StagedScene.Elements}"
@@ -899,12 +898,28 @@
VerticalAlignment="Stretch"/>
<Rectangle Width="2" Fill="#00000080" HorizontalAlignment="Left" Margin="230,0,0,0"
VerticalAlignment="Stretch"/>
<ScrollViewer HorizontalScrollBarVisibility="Auto" VerticalScrollBarVisibility="Disabled"
Visibility="{Binding TraxError, Converter={StaticResource NotNullToVis}}"
VerticalAlignment="Center" Margin="4,0">
<TextBlock Text="{Binding TraxError}" Foreground="#e94560" FontSize="9"
FontWeight="SemiBold" VerticalAlignment="Center"/>
</ScrollViewer>
<!-- TRAX error: a terse message that scrolls across the bar
(marquee) while the TRAX dot is red. -->
<Canvas x:Name="TraxErrorMarquee" ClipToBounds="True"
Visibility="{Binding TraxError, Converter={StaticResource NotNullToVis}}">
<TextBlock x:Name="TraxErrorText" Text="{Binding TraxError}" Foreground="#ff6b81" FontSize="10"
FontWeight="SemiBold" Canvas.Top="0">
<TextBlock.RenderTransform>
<TranslateTransform x:Name="TraxErrorScroll"/>
</TextBlock.RenderTransform>
<TextBlock.Triggers>
<EventTrigger RoutedEvent="FrameworkElement.Loaded">
<BeginStoryboard>
<Storyboard RepeatBehavior="Forever">
<DoubleAnimation Storyboard.TargetName="TraxErrorScroll"
Storyboard.TargetProperty="X"
From="288" To="-220" Duration="0:0:7"/>
</Storyboard>
</BeginStoryboard>
</EventTrigger>
</TextBlock.Triggers>
</TextBlock>
</Canvas>
</Grid>
</Border>
<Grid Width="16" Height="16" Cursor="Hand" Margin="0,0,8,0"
+1 -1
View File
@@ -39,7 +39,7 @@
Stretch="UniformToFill"
RenderOptions.BitmapScalingMode="LowQuality"/>
</Border>
<TextBlock Grid.Row="1" Foreground="#a0a0b0" FontSize="10"
<TextBlock Grid.Row="1" Foreground="#a0a0b0" FontSize="20"
HorizontalAlignment="Center"
Margin="0,2,0,0"
TextTrimming="CharacterEllipsis">
+1 -1
View File
@@ -12,7 +12,7 @@ See [`schema.md`](../schema.md) for the memory-map conventions.
| `TopBar.xaml(.cs)` | Row 0 session surface (2026-09-22 redesign): brand + gear menu (left), one segmented REC|ON-AIR mode switch + world actions (Record: Start Recording · Stream: Go Live + Test) + reality line while running (dot/word/elapsed) + End (center), account zone — Sign In or avatar w/ Change Account/Logout context menu (right). `RefreshAvatar()`, `GearButton_Click` |
| `PreviewPane.xaml(.cs)` | Row 1 Col 1 center: the 1920×1080 `OverlayCanvas`/`SelectionOverlay`/`BrandFlashLayer`/`SocialBarElement`, text pull-out drawer, and the live-controls row (TRAX / game / mic meters + sliders + speakers + socials) |
| `SceneThumbnailStrip.xaml(.cs)` | Row 0 scene-picker strip (`ItemsControl` over `Scenes`), staged-red / live-green border highlight |
| `LeftPanel.xaml(.cs)` | Row 1 Col 0 left panel: edit mode → layers+properties; live → YouTube chat. Source add/edit/remove, list drag-to-reorder, snapshot revert-accept |
| `LeftPanel.xaml(.cs)` | Row 1 Col 0 left panel: edit mode → layers+properties; live → YouTube chat. Source add/edit/remove, list drag-to-reorder, snapshot revert-accept. **TASK 51:** the webcam properties carry the Background section — `RemoveBackgroundPill` / `BlurBackgroundPill` (`PillToggle`, radio-exclusive via `WebcamSceneConfig.RemoveBackground`/`BlurBackground`) + `BackgroundStrengthSlider` (0..1, 5% snaps, shown only while an effect is on) |
| `OverlayHost.xaml(.cs)` | Root-grid overlay dialogs: Settings / Report Bug / Feature Request / About (+ licenses + license-key entry). `BackToAbout_Click` lives here |
| `SceneThumbnailStrip.xaml.cs` | Code-behind for the strip above |
| `LeftPanel.xaml.cs` | Code-behind for the panel above |
+68 -50
View File
@@ -1,60 +1,78 @@
# HANDOFF — 2026-10-08 (zoom box shipped, READY FOR PUSH)
# HANDOFF — 2026-10-10 (strip tile artwork + on-air tile unit; uncommitted)
## At a glance
- **Branch** main (no feature branch). Pushed through `5685c5d`; 19 commits local, **not pushed**
(push at the creator's call).
- Scene-order unit committed `06f88f4`; desktop-lock committed `63bf844`. The **strip tile
artwork unit below is uncommitted**.
## Branch
- main (no feature branch)
- main (no feature branch). Pushed through `5685c5d` (TRAX toggle); everything after is local,
**not pushed** (push at the creator's call).
## Push readiness
- **5 commits ahead of origin/main, clean to push** (fetch confirmed origin has not moved):
`966e718` zoom box visuals → `4b3d82f` zoom as layered + TASK 51 booking → `6bff21b`
anchored-capture rework → `948dd08` dims-shading fix → (docs unit: TASK 52 record +
numbering fixes + this HANDOFF).
- The pinned-chat work unit below is **uncommitted on purpose** — `git push` sends only
commits, so pushing is safe; the dirty files stay local.
## New work unit (this handoff) — VERIFIED, ready to commit
Strip tile artwork + on-air indicator (creator ruling 2026-10-10). Three sub-parts:
## Dirty files (NOT mine — the in-flight "pinned chat" work unit; do NOT commit or co-edit)
- `Models/YouTube.cs`
- `ViewModels/MainViewModel.Chat.cs`
- `Controls/LeftPanel.xaml` (pinned-message UI)
- Untracked: `Helpers/RelayCommandGeneric.cs`
1. **Per-scene CRT tile art** — each canonical scene's top-bar tile is its OWN image stored in the
internal DB (`Scene.ThumbnailAssetId`, additive migration, seeded at heal from the creator's five
Downloads JPGs via `SceneThumbnailSeedPaths`). The preview pane keeps rendering the scene's original
background — never the tile. `Save()` Asset prune now keeps `Scene.ThumbnailAssetId` rows
(this was a real bug: the first save would have deleted the seeded tiles).
2. **On-air image** — one shared image (`Gemini_Generated_Image_4zajyl4zajyl4zaj.jpg`) that REPLACES the
staged scene's tile ("the on-air screen jumps around the top bar as the creator swaps scenes").
Stored as an Asset BLOB tracked by the `OnAirImageAssetId` Setting (`EnsureOnAirImage`, same contract
as the alert clip; prune keep-set extended). Seed path override: `MainViewModel.OnAirSeedPath`.
`RefreshSnapshotsAsync`: staged scene → on-air art; unstaged → own thumbnail or background art;
staged Live without on-air art → green placeholder (unchanged invariant: minis never render captures).
3. **Preview + labels polish** — removed the 6px `#22c55e` rectangle in `BackdropLayer`
(`Controls/PreviewPane.xaml`) — the ONLY live signal is the outer `PreviewGlow` red border
(`MainViewModel.Streaming.cs`, transparent otherwise); the redundant rect was only visible when a
scene had no background ("green outline around the preview screen" the creator saw). Scene-tile
labels doubled: `Controls/SceneThumbnailStrip.xaml` FontSize 10 → 20.
## What just happened (2026-10-08, third zoom pass — dims shading)
The zoom box shipped as a **loupe** (`4b3d82f`), then was corrected to an **anchored-capture
magnifier over the full composite drawn last** (commit `6bff21b`). The final correction: as the box
is dragged, the dim rectangles (55% black "div shading" around the box) were sliding under the
capture region — shown inside the box because its `VisualBrush` samples all of `CanvasGrid`, and the
`DimRects` ItemsControl previously lived INSIDE `CanvasGrid`.
- Files: `Models/Scene.cs`, `Services/LayoutStore.{Migrations,Load,Save,Settings}.cs`,
`ViewModels/MainViewModel.Background.cs`, `Controls/PreviewPane.xaml` (rect removed),
`Controls/SceneThumbnailStrip.xaml` (label), `ytLive.Tests/SceneThumbnailTests.cs` (4 tests).
- Full suite **427 passed / 0 failed / 0 warnings** (one run flaked on the known
`RealMouseDrag_…PersistsTheReorder` env test — needs an idle desktop — and passed on re-run;
nothing in this unit touches layer reorder). Existing 3 tile tests stub seed paths + `OnAirSeedPath`
so the suite is hermetic; the new 4th test covers the on-air swap + persistence.
- **To commit later (this unit only):** the files above. Then the working tree is back to just the
creator's pinned-chat unit (NOT mine to commit).
Fix: the `DimRects` overlay is now a **sibling of `CanvasGrid`** (renders above the composed
content, below the `ZoomBox`), never a descendant — so the zoom sample can never pick up the shading
and the captured area stays clear no matter where the box is dragged. Behavior consequence to note:
in non-zoom ("quality dims") mode the dims now also shade the global overlays (brand flash, alert
ticker, social bar) that sit above them in the old tree — consistent with "chrome outside the crop
is dimmed". Structural invariant added to `ZoomBoxInteractionTests`: `DimRectsLayer` is a direct
child of `PreviewCanvas`, not inside `CanvasGrid`, and renders below the `ZoomBox`.
## Desktop-lock unit (COMMITTED 63bf844)
- `ShowActivated = !FullscreenGameRunningNow` + `FullScreenDetectorOverride` seam + `DesktopLockTests`.
Creator still to verify in-game (launch while an exclusive-fullscreen game owns the foreground).
## Zoom feature state
- **TASK 52 (zoom box) is now a first-class record** — see [`TASKS/task-52-zoom-box.md`](TASKS/task-52-zoom-box.md):
feature provenance + the three creator corrections, shapes, implementation map, tests.
TASK 51 (webcam background Remove + Blur) is unblocked for the next unit.
- Models: `ZoomState.cs`, `ZoomShape.cs` (Circle/VerticalRectangle/HorizontalRectangle), `ZoomSceneElement.cs`.
- ViewModel (`MainViewModel.Zoom.cs`): `ActivateZoom`/`ToggleZoom`/`DismissZoom`/`StartZoomMove`/`UpdateZoomMove`/`EndZoomMove` + `RebuildDimRectsForZoom`; transient element healed/mirrored by `HealZoomLayers`/`EnsureZoomLayer`.
- Output: box composited last via `BlitZoom`/`ZoomMagnifier`; output crop = tier (box never the crop).
- Preview: `ZoomBox` + `DimRectsLayer` are both SIBLINGS of `CanvasGrid` (box above dims above content); box samples the full composite at the fixed capture, 2x; circle→ellipse clip, rect/square outline per shape; `LeftPanel` hides the placeholder row via `IsZoomLayer`.
- Rectangle shape = 4:3; Circle square; Phone 9:16. Capture anchored on click (`CaptureX/Y`); dragging the box never changes it.
## Scene-order unit (COMMITTED 06f88f4)
- `ReorderScenesToCanonical()` + `SceneOrderTests`. The creator's DB (Chat SortOrder 2, BRB 3) heals to
canonical F1-F5 order at load; next save re-stamps SortOrder.
## Tests
- Full suite: **406 passed / 0 failed** on a clean full run (the pre-existing audio-loopback timing
flake and `RealMouseDrag`'s interactive-desktop requirement recur occasionally — not zoom
regressions). Zoom tests 3/3 green, 0 build warnings.
## Known issues / follow-ups
- **Background "Use default background" toggle still not working** (carried from 2026-10-07):
binding exists (`UseDefaultBackground`), VM handler invalidates bake; investigate when time permits.
- The 3px selection / crop-outline strokes could still show in the zoom box IF the capture overlaps
them in edit mode — not shading, thin outlines only; the dims (the user's complaint) are gone.
## Dirty files — current reality (2026-10-10)
- **NOT mine — do NOT commit:** the creator's in-flight "pinned chat" unit (`Models/YouTube.cs`,
`ViewModels/MainViewModel.Chat.cs`, untracked `Helpers/RelayCommandGeneric.cs`, and **its hunks in
`Controls/LeftPanel.xaml`**), plus the creator's untracked `Assets/llamacasty*` art.
- ⚠️ Pre-commit: `scripts/verify.sh` runs `scripts/scope-check.sh` against ALL working-tree changes —
the not-mine pinned-chat + art files WILL flag. Pass them as declared-known.
## Next step
- User runs the app live to eyeball: drag the zoom box anywhere and the captured content stays clean
(no dim shading in the box; the scene outside the box stays dimmed). Then TASK 51 (webcam
background Remove + Blur) per `TASKS/task-51-webcam-bg-effects.md`, and the background-toggle
investigation if still open.
- Commit this unit (files listed above; gate already green at 427/427, 0 warnings). Then the creator
launches the app on the real DB: the five CRT tiles seed from `C:\Users\gramp\Downloads\Gemini_…`,
the staged scene's tile shows the on-air image, the preview shows the scene's real background with no
green frame, and the tile labels are legible (FontSize 20). Push is still at the creator's call.
## Tests
- Full suite **427 passed / 0 failed / 0 skipped / 0 warnings** (2026-10-10). Known env flakes:
`Mix_HonorsProviderGains…` (fails while system audio plays), `RealMouseDrag…` (needs an idle desktop),
`Round_Clip_Corner…` (grab/geometry under active mouse). The verify gate whitelists only the latter two —
`RealMouseDrag…` failing mid-session needs one re-run when the desktop idles.
## Also still open
- **Vertical-format regression (STALE, not fixed):** 1080p60 vertical selection no longer shrinks the
capture border in the preview. Mechanism mapped (see prior summary): the crop Rectangle
`Controls/PreviewPane.xaml:319-322` bound to `OutputRectX/Y/W/H` + `IsOutputCropped`, driven by
`MainViewModel.Streaming.cs` `ApplyStreamQuality` from `SelectedQuality` (binding + ctor only,
`new("Vertical 1080p60", 60, 8.0, 1080, 1920)`), `DimRects` in `MainViewModel.cs:91-94`. Root cause
unknown — paused.
- ⛔ Sudo password rotation pending (user; then update both password lines in `CREDENTIALS.md`).
- TASK 56 secret scan before `gramps/LlamaCasty` goes public. TASK 55 disk hygiene queued.
+34
View File
@@ -0,0 +1,34 @@
using System;
using System.Windows.Input;
namespace ytLive.Helpers;
public class RelayCommand<T> : ICommand
{
private readonly Action<T?> _execute;
private readonly Predicate<T?>? _canExecute;
public RelayCommand(Action<T?> execute, Predicate<T?>? canExecute = null)
{
_execute = execute;
_canExecute = canExecute;
}
public event EventHandler? CanExecuteChanged
{
add => CommandManager.RequerySuggested += value;
remove => CommandManager.RequerySuggested -= value;
}
public bool CanExecute(object? parameter)
{
if (parameter is T t) return _canExecute?.Invoke(t) ?? true;
return _canExecute?.Invoke(default) ?? true;
}
public void Execute(object? parameter)
{
if (parameter is T t) _execute(t);
else _execute(default);
}
}
+31
View File
@@ -0,0 +1,31 @@
<Window x:Class="ytLive.KeepBroadcastDialog"
xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
Title="Keep this broadcast?" Height="240" Width="460"
Icon="/ytLive;component/Assets/llamacasty-icon.png"
WindowStartupLocation="CenterOwner" ResizeMode="NoResize"
ShowInTaskbar="False" Background="#1a1a2e">
<Grid Margin="24">
<Grid.RowDefinitions>
<RowDefinition Height="Auto"/>
<RowDefinition Height="Auto"/>
<RowDefinition Height="Auto"/>
</Grid.RowDefinitions>
<TextBlock Grid.Row="0" Text="Keep this broadcast?" FontSize="18" FontWeight="Bold"
Foreground="#e0e0e0" Margin="0,0,0,12"/>
<TextBlock Grid.Row="1" x:Name="MessageText" TextWrapping="Wrap" Foreground="#a0a0b0" FontSize="13"
Margin="0,0,0,16"/>
<StackPanel Grid.Row="2" Orientation="Horizontal" HorizontalAlignment="Right">
<Button x:Name="DeleteBroadcastButton" Content="Delete" IsCancel="True" Click="Delete_Click"
Style="{StaticResource YtButtonSecondary}" Margin="0,0,8,0"
ToolTip="Esc — delete the recording from YouTube"/>
<Button x:Name="KeepBroadcastButton" Content="Keep" IsDefault="True" Click="Keep_Click"
Style="{StaticResource YtButton}" MinWidth="90"
ToolTip="Enter — keep it on YouTube as a private video"/>
</StackPanel>
</Grid>
</Window>
+23
View File
@@ -0,0 +1,23 @@
using System.Windows;
namespace ytLive;
/// <summary>Asked when a PRIVATE broadcast ends — the same contract as the recording's save
/// dialog (creator ruling 2026-10-09): <b>Enter keeps</b> it on YouTube as a private video,
/// <b>Escape deletes</b> it. <c>DialogResult == true</c> = keep; false (Delete, Escape, the X)
/// = delete.</summary>
public partial class KeepBroadcastDialog : Window
{
public KeepBroadcastDialog(string title)
{
InitializeComponent();
MessageText.Text = string.IsNullOrWhiteSpace(title)
? "Your private broadcast is on YouTube as a private video. Keep it, or delete it?"
: $"\u201c{title}\u201d is on YouTube as a private video. Keep it, or delete it?";
Loaded += (_, _) => KeepBroadcastButton.Focus();
}
private void Keep_Click(object sender, RoutedEventArgs e) => DialogResult = true;
private void Delete_Click(object sender, RoutedEventArgs e) => DialogResult = false;
}
+674
View File
@@ -0,0 +1,674 @@
GNU GENERAL PUBLIC LICENSE
Version 3, 29 June 2007
Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
Preamble
The GNU General Public License is a free, copyleft license for
software and other kinds of works.
The licenses for most software and other practical works are designed
to take away your freedom to share and change the works. By contrast,
the GNU General Public License is intended to guarantee your freedom to
share and change all versions of a program--to make sure it remains free
software for all its users. We, the Free Software Foundation, use the
GNU General Public License for most of our software; it applies also to
any other work released this way by its authors. You can apply it to
your programs, too.
When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
them if you wish), that you receive source code or can get it if you
want it, that you can change the software or use pieces of it in new
free programs, and that you know you can do these things.
To protect your rights, we need to prevent others from denying you
these rights or asking you to surrender the rights. Therefore, you have
certain responsibilities if you distribute copies of the software, or if
you modify it: responsibilities to respect the freedom of others.
For example, if you distribute copies of such a program, whether
gratis or for a fee, you must pass on to the recipients the same
freedoms that you received. You must make sure that they, too, receive
or can get the source code. And you must show them these terms so they
know their rights.
Developers that use the GNU GPL protect your rights with two steps:
(1) assert copyright on the software, and (2) offer you this License
giving you legal permission to copy, distribute and/or modify it.
For the developers' and authors' protection, the GPL clearly explains
that there is no warranty for this free software. For both users' and
authors' sake, the GPL requires that modified versions be marked as
changed, so that their problems will not be attributed erroneously to
authors of previous versions.
Some devices are designed to deny users access to install or run
modified versions of the software inside them, although the manufacturer
can do so. This is fundamentally incompatible with the aim of
protecting users' freedom to change the software. The systematic
pattern of such abuse occurs in the area of products for individuals to
use, which is precisely where it is most unacceptable. Therefore, we
have designed this version of the GPL to prohibit the practice for those
products. If such problems arise substantially in other domains, we
stand ready to extend this provision to those domains in future versions
of the GPL, as needed to protect the freedom of users.
Finally, every program is threatened constantly by software patents.
States should not allow patents to restrict development and use of
software on general-purpose computers, but in those that do, we wish to
avoid the special danger that patents applied to a free program could
make it effectively proprietary. To prevent this, the GPL assures that
patents cannot be used to render the program non-free.
The precise terms and conditions for copying, distribution and
modification follow.
TERMS AND CONDITIONS
0. Definitions.
"This License" refers to version 3 of the GNU General Public License.
"Copyright" also means copyright-like laws that apply to other kinds of
works, such as semiconductor masks.
"The Program" refers to any copyrightable work licensed under this
License. Each licensee is addressed as "you". "Licensees" and
"recipients" may be individuals or organizations.
To "modify" a work means to copy from or adapt all or part of the work
in a fashion requiring copyright permission, other than the making of an
exact copy. The resulting work is called a "modified version" of the
earlier work or a work "based on" the earlier work.
A "covered work" means either the unmodified Program or a work based
on the Program.
To "propagate" a work means to do anything with it that, without
permission, would make you directly or secondarily liable for
infringement under applicable copyright law, except executing it on a
computer or modifying a private copy. Propagation includes copying,
distribution (with or without modification), making available to the
public, and in some countries other activities as well.
To "convey" a work means any kind of propagation that enables other
parties to make or receive copies. Mere interaction with a user through
a computer network, with no transfer of a copy, is not conveying.
An interactive user interface displays "Appropriate Legal Notices"
to the extent that it includes a convenient and prominently visible
feature that (1) displays an appropriate copyright notice, and (2)
tells the user that there is no warranty for the work (except to the
extent that warranties are provided), that licensees may convey the
work under this License, and how to view a copy of this License. If
the interface presents a list of user commands or options, such as a
menu, a prominent item in the list meets this criterion.
1. Source Code.
The "source code" for a work means the preferred form of the work
for making modifications to it. "Object code" means any non-source
form of a work.
A "Standard Interface" means an interface that either is an official
standard defined by a recognized standards body, or, in the case of
interfaces specified for a particular programming language, one that
is widely used among developers working in that language.
The "System Libraries" of an executable work include anything, other
than the work as a whole, that (a) is included in the normal form of
packaging a Major Component, but which is not part of that Major
Component, and (b) serves only to enable use of the work with that
Major Component, or to implement a Standard Interface for which an
implementation is available to the public in source code form. A
"Major Component", in this context, means a major essential component
(kernel, window system, and so on) of the specific operating system
(if any) on which the executable work runs, or a compiler used to
produce the work, or an object code interpreter used to run it.
The "Corresponding Source" for a work in object code form means all
the source code needed to generate, install, and (for an executable
work) run the object code and to modify the work, including scripts to
control those activities. However, it does not include the work's
System Libraries, or general-purpose tools or generally available free
programs which are used unmodified in performing those activities but
which are not part of the work. For example, Corresponding Source
includes interface definition files associated with source files for
the work, and the source code for shared libraries and dynamically
linked subprograms that the work is specifically designed to require,
such as by intimate data communication or control flow between those
subprograms and other parts of the work.
The Corresponding Source need not include anything that users
can regenerate automatically from other parts of the Corresponding
Source.
The Corresponding Source for a work in source code form is that
same work.
2. Basic Permissions.
All rights granted under this License are granted for the term of
copyright on the Program, and are irrevocable provided the stated
conditions are met. This License explicitly affirms your unlimited
permission to run the unmodified Program. The output from running a
covered work is covered by this License only if the output, given its
content, constitutes a covered work. This License acknowledges your
rights of fair use or other equivalent, as provided by copyright law.
You may make, run and propagate covered works that you do not
convey, without conditions so long as your license otherwise remains
in force. You may convey covered works to others for the sole purpose
of having them make modifications exclusively for you, or provide you
with facilities for running those works, provided that you comply with
the terms of this License in conveying all material for which you do
not control copyright. Those thus making or running the covered works
for you must do so exclusively on your behalf, under your direction
and control, on terms that prohibit them from making any copies of
your copyrighted material outside their relationship with you.
Conveying under any other circumstances is permitted solely under
the conditions stated below. Sublicensing is not allowed; section 10
makes it unnecessary.
3. Protecting Users' Legal Rights From Anti-Circumvention Law.
No covered work shall be deemed part of an effective technological
measure under any applicable law fulfilling obligations under article
11 of the WIPO copyright treaty adopted on 20 December 1996, or
similar laws prohibiting or restricting circumvention of such
measures.
When you convey a covered work, you waive any legal power to forbid
circumvention of technological measures to the extent such circumvention
is effected by exercising rights under this License with respect to
the covered work, and you disclaim any intention to limit operation or
modification of the work as a means of enforcing, against the work's
users, your or third parties' legal rights to forbid circumvention of
technological measures.
4. Conveying Verbatim Copies.
You may convey verbatim copies of the Program's source code as you
receive it, in any medium, provided that you conspicuously and
appropriately publish on each copy an appropriate copyright notice;
keep intact all notices stating that this License and any
non-permissive terms added in accord with section 7 apply to the code;
keep intact all notices of the absence of any warranty; and give all
recipients a copy of this License along with the Program.
You may charge any price or no price for each copy that you convey,
and you may offer support or warranty protection for a fee.
5. Conveying Modified Source Versions.
You may convey a work based on the Program, or the modifications to
produce it from the Program, in the form of source code under the
terms of section 4, provided that you also meet all of these conditions:
a) The work must carry prominent notices stating that you modified
it, and giving a relevant date.
b) The work must carry prominent notices stating that it is
released under this License and any conditions added under section
7. This requirement modifies the requirement in section 4 to
"keep intact all notices".
c) You must license the entire work, as a whole, under this
License to anyone who comes into possession of a copy. This
License will therefore apply, along with any applicable section 7
additional terms, to the whole of the work, and all its parts,
regardless of how they are packaged. This License gives no
permission to license the work in any other way, but it does not
invalidate such permission if you have separately received it.
d) If the work has interactive user interfaces, each must display
Appropriate Legal Notices; however, if the Program has interactive
interfaces that do not display Appropriate Legal Notices, your
work need not make them do so.
A compilation of a covered work with other separate and independent
works, which are not by their nature extensions of the covered work,
and which are not combined with it such as to form a larger program,
in or on a volume of a storage or distribution medium, is called an
"aggregate" if the compilation and its resulting copyright are not
used to limit the access or legal rights of the compilation's users
beyond what the individual works permit. Inclusion of a covered work
in an aggregate does not cause this License to apply to the other
parts of the aggregate.
6. Conveying Non-Source Forms.
You may convey a covered work in object code form under the terms
of sections 4 and 5, provided that you also convey the
machine-readable Corresponding Source under the terms of this License,
in one of these ways:
a) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by the
Corresponding Source fixed on a durable physical medium
customarily used for software interchange.
b) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by a
written offer, valid for at least three years and valid for as
long as you offer spare parts or customer support for that product
model, to give anyone who possesses the object code either (1) a
copy of the Corresponding Source for all the software in the
product that is covered by this License, on a durable physical
medium customarily used for software interchange, for a price no
more than your reasonable cost of physically performing this
conveying of source, or (2) access to copy the
Corresponding Source from a network server at no charge.
c) Convey individual copies of the object code with a copy of the
written offer to provide the Corresponding Source. This
alternative is allowed only occasionally and noncommercially, and
only if you received the object code with such an offer, in accord
with subsection 6b.
d) Convey the object code by offering access from a designated
place (gratis or for a charge), and offer equivalent access to the
Corresponding Source in the same way through the same place at no
further charge. You need not require recipients to copy the
Corresponding Source along with the object code. If the place to
copy the object code is a network server, the Corresponding Source
may be on a different server (operated by you or a third party)
that supports equivalent copying facilities, provided you maintain
clear directions next to the object code saying where to find the
Corresponding Source. Regardless of what server hosts the
Corresponding Source, you remain obligated to ensure that it is
available for as long as needed to satisfy these requirements.
e) Convey the object code using peer-to-peer transmission, provided
you inform other peers where the object code and Corresponding
Source of the work are being offered to the general public at no
charge under subsection 6d.
A separable portion of the object code, whose source code is excluded
from the Corresponding Source as a System Library, need not be
included in conveying the object code work.
A "User Product" is either (1) a "consumer product", which means any
tangible personal property which is normally used for personal, family,
or household purposes, or (2) anything designed or sold for incorporation
into a dwelling. In determining whether a product is a consumer product,
doubtful cases shall be resolved in favor of coverage. For a particular
product received by a particular user, "normally used" refers to a
typical or common use of that class of product, regardless of the status
of the particular user or of the way in which the particular user
actually uses, or expects or is expected to use, the product. A product
is a consumer product regardless of whether the product has substantial
commercial, industrial or non-consumer uses, unless such uses represent
the only significant mode of use of the product.
"Installation Information" for a User Product means any methods,
procedures, authorization keys, or other information required to install
and execute modified versions of a covered work in that User Product from
a modified version of its Corresponding Source. The information must
suffice to ensure that the continued functioning of the modified object
code is in no case prevented or interfered with solely because
modification has been made.
If you convey an object code work under this section in, or with, or
specifically for use in, a User Product, and the conveying occurs as
part of a transaction in which the right of possession and use of the
User Product is transferred to the recipient in perpetuity or for a
fixed term (regardless of how the transaction is characterized), the
Corresponding Source conveyed under this section must be accompanied
by the Installation Information. But this requirement does not apply
if neither you nor any third party retains the ability to install
modified object code on the User Product (for example, the work has
been installed in ROM).
The requirement to provide Installation Information does not include a
requirement to continue to provide support service, warranty, or updates
for a work that has been modified or installed by the recipient, or for
the User Product in which it has been modified or installed. Access to a
network may be denied when the modification itself materially and
adversely affects the operation of the network or violates the rules and
protocols for communication across the network.
Corresponding Source conveyed, and Installation Information provided,
in accord with this section must be in a format that is publicly
documented (and with an implementation available to the public in
source code form), and must require no special password or key for
unpacking, reading or copying.
7. Additional Terms.
"Additional permissions" are terms that supplement the terms of this
License by making exceptions from one or more of its conditions.
Additional permissions that are applicable to the entire Program shall
be treated as though they were included in this License, to the extent
that they are valid under applicable law. If additional permissions
apply only to part of the Program, that part may be used separately
under those permissions, but the entire Program remains governed by
this License without regard to the additional permissions.
When you convey a copy of a covered work, you may at your option
remove any additional permissions from that copy, or from any part of
it. (Additional permissions may be written to require their own
removal in certain cases when you modify the work.) You may place
additional permissions on material, added by you to a covered work,
for which you have or can give appropriate copyright permission.
Notwithstanding any other provision of this License, for material you
add to a covered work, you may (if authorized by the copyright holders of
that material) supplement the terms of this License with terms:
a) Disclaiming warranty or limiting liability differently from the
terms of sections 15 and 16 of this License; or
b) Requiring preservation of specified reasonable legal notices or
author attributions in that material or in the Appropriate Legal
Notices displayed by works containing it; or
c) Prohibiting misrepresentation of the origin of that material, or
requiring that modified versions of such material be marked in
reasonable ways as different from the original version; or
d) Limiting the use for publicity purposes of names of licensors or
authors of the material; or
e) Declining to grant rights under trademark law for use of some
trade names, trademarks, or service marks; or
f) Requiring indemnification of licensors and authors of that
material by anyone who conveys the material (or modified versions of
it) with contractual assumptions of liability to the recipient, for
any liability that these contractual assumptions directly impose on
those licensors and authors.
All other non-permissive additional terms are considered "further
restrictions" within the meaning of section 10. If the Program as you
received it, or any part of it, contains a notice stating that it is
governed by this License along with a term that is a further
restriction, you may remove that term. If a license document contains
a further restriction but permits relicensing or conveying under this
License, you may add to a covered work material governed by the terms
of that license document, provided that the further restriction does
not survive such relicensing or conveying.
If you add terms to a covered work in accord with this section, you
must place, in the relevant source files, a statement of the
additional terms that apply to those files, or a notice indicating
where to find the applicable terms.
Additional terms, permissive or non-permissive, may be stated in the
form of a separately written license, or stated as exceptions;
the above requirements apply either way.
8. Termination.
You may not propagate or modify a covered work except as expressly
provided under this License. Any attempt otherwise to propagate or
modify it is void, and will automatically terminate your rights under
this License (including any patent licenses granted under the third
paragraph of section 11).
However, if you cease all violation of this License, then your
license from a particular copyright holder is reinstated (a)
provisionally, unless and until the copyright holder explicitly and
finally terminates your license, and (b) permanently, if the copyright
holder fails to notify you of the violation by some reasonable means
prior to 60 days after the cessation.
Moreover, your license from a particular copyright holder is
reinstated permanently if the copyright holder notifies you of the
violation by some reasonable means, this is the first time you have
received notice of violation of this License (for any work) from that
copyright holder, and you cure the violation prior to 30 days after
your receipt of the notice.
Termination of your rights under this section does not terminate the
licenses of parties who have received copies or rights from you under
this License. If your rights have been terminated and not permanently
reinstated, you do not qualify to receive new licenses for the same
material under section 10.
9. Acceptance Not Required for Having Copies.
You are not required to accept this License in order to receive or
run a copy of the Program. Ancillary propagation of a covered work
occurring solely as a consequence of using peer-to-peer transmission
to receive a copy likewise does not require acceptance. However,
nothing other than this License grants you permission to propagate or
modify any covered work. These actions infringe copyright if you do
not accept this License. Therefore, by modifying or propagating a
covered work, you indicate your acceptance of this License to do so.
10. Automatic Licensing of Downstream Recipients.
Each time you convey a covered work, the recipient automatically
receives a license from the original licensors, to run, modify and
propagate that work, subject to this License. You are not responsible
for enforcing compliance by third parties with this License.
An "entity transaction" is a transaction transferring control of an
organization, or substantially all assets of one, or subdividing an
organization, or merging organizations. If propagation of a covered
work results from an entity transaction, each party to that
transaction who receives a copy of the work also receives whatever
licenses to the work the party's predecessor in interest had or could
give under the previous paragraph, plus a right to possession of the
Corresponding Source of the work from the predecessor in interest, if
the predecessor has it or can get it with reasonable efforts.
You may not impose any further restrictions on the exercise of the
rights granted or affirmed under this License. For example, you may
not impose a license fee, royalty, or other charge for exercise of
rights granted under this License, and you may not initiate litigation
(including a cross-claim or counterclaim in a lawsuit) alleging that
any patent claim is infringed by making, using, selling, offering for
sale, or importing the Program or any portion of it.
11. Patents.
A "contributor" is a copyright holder who authorizes use under this
License of the Program or a work on which the Program is based. The
work thus licensed is called the contributor's "contributor version".
A contributor's "essential patent claims" are all patent claims
owned or controlled by the contributor, whether already acquired or
hereafter acquired, that would be infringed by some manner, permitted
by this License, of making, using, or selling its contributor version,
but do not include claims that would be infringed only as a
consequence of further modification of the contributor version. For
purposes of this definition, "control" includes the right to grant
patent sublicenses in a manner consistent with the requirements of
this License.
Each contributor grants you a non-exclusive, worldwide, royalty-free
patent license under the contributor's essential patent claims, to
make, use, sell, offer for sale, import and otherwise run, modify and
propagate the contents of its contributor version.
In the following three paragraphs, a "patent license" is any express
agreement or commitment, however denominated, not to enforce a patent
(such as an express permission to practice a patent or covenant not to
sue for patent infringement). To "grant" such a patent license to a
party means to make such an agreement or commitment not to enforce a
patent against the party.
If you convey a covered work, knowingly relying on a patent license,
and the Corresponding Source of the work is not available for anyone
to copy, free of charge and under the terms of this License, through a
publicly available network server or other readily accessible means,
then you must either (1) cause the Corresponding Source to be so
available, or (2) arrange to deprive yourself of the benefit of the
patent license for this particular work, or (3) arrange, in a manner
consistent with the requirements of this License, to extend the patent
license to downstream recipients. "Knowingly relying" means you have
actual knowledge that, but for the patent license, your conveying the
covered work in a country, or your recipient's use of the covered work
in a country, would infringe one or more identifiable patents in that
country that you have reason to believe are valid.
If, pursuant to or in connection with a single transaction or
arrangement, you convey, or propagate by procuring conveyance of, a
covered work, and grant a patent license to some of the parties
receiving the covered work authorizing them to use, propagate, modify
or convey a specific copy of the covered work, then the patent license
you grant is automatically extended to all recipients of the covered
work and works based on it.
A patent license is "discriminatory" if it does not include within
the scope of its coverage, prohibits the exercise of, or is
conditioned on the non-exercise of one or more of the rights that are
specifically granted under this License. You may not convey a covered
work if you are a party to an arrangement with a third party that is
in the business of distributing software, under which you make payment
to the third party based on the extent of your activity of conveying
the work, and under which the third party grants, to any of the
parties who would receive the covered work from you, a discriminatory
patent license (a) in connection with copies of the covered work
conveyed by you (or copies made from those copies), or (b) primarily
for and in connection with specific products or compilations that
contain the covered work, unless you entered into that arrangement,
or that patent license was granted, prior to 28 March 2007.
Nothing in this License shall be construed as excluding or limiting
any implied license or other defenses to infringement that may
otherwise be available to you under applicable patent law.
12. No Surrender of Others' Freedom.
If conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot convey a
covered work so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you may
not convey it at all. For example, if you agree to terms that obligate you
to collect a royalty for further conveying from those to whom you convey
the Program, the only way you could satisfy both those terms and this
License would be to refrain entirely from conveying the Program.
13. Use with the GNU Affero General Public License.
Notwithstanding any other provision of this License, you have
permission to link or combine any covered work with a work licensed
under version 3 of the GNU Affero General Public License into a single
combined work, and to convey the resulting work. The terms of this
License will continue to apply to the part which is the covered work,
but the special requirements of the GNU Affero General Public License,
section 13, concerning interaction through a network will apply to the
combination as such.
14. Revised Versions of this License.
The Free Software Foundation may publish revised and/or new versions of
the GNU General Public License from time to time. Such new versions will
be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.
Each version is given a distinguishing version number. If the
Program specifies that a certain numbered version of the GNU General
Public License "or any later version" applies to it, you have the
option of following the terms and conditions either of that numbered
version or of any later version published by the Free Software
Foundation. If the Program does not specify a version number of the
GNU General Public License, you may choose any version ever published
by the Free Software Foundation.
If the Program specifies that a proxy can decide which future
versions of the GNU General Public License can be used, that proxy's
public statement of acceptance of a version permanently authorizes you
to choose that version for the Program.
Later license versions may give you additional or different
permissions. However, no additional obligations are imposed on any
author or copyright holder as a result of your choosing to follow a
later version.
15. Disclaimer of Warranty.
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
16. Limitation of Liability.
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
SUCH DAMAGES.
17. Interpretation of Sections 15 and 16.
If the disclaimer of warranty and limitation of liability provided
above cannot be given local legal effect according to their terms,
reviewing courts shall apply local law that most closely approximates
an absolute waiver of all civil liability in connection with the
Program, unless a warranty or assumption of liability accompanies a
copy of the Program in return for a fee.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Programs
If you develop a new program, and you want it to be of the greatest
possible use to the public, the best way to achieve this is to make it
free software which everyone can redistribute and change under these terms.
To do so, attach the following notices to the program. It is safest
to attach them to the start of each source file to most effectively
state the exclusion of warranty; and each file should have at least
the "copyright" line and a pointer to where the full notice is found.
<one line to give the program's name and a brief idea of what it does.>
Copyright (C) <year> <name of author>
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
Also add information on how to contact you by electronic and paper mail.
If the program does terminal interaction, make it output a short
notice like this when it starts in an interactive mode:
<program> Copyright (C) <year> <name of author>
This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
This is free software, and you are welcome to redistribute it
under certain conditions; type `show c' for details.
The hypothetical commands `show w' and `show c' should show the appropriate
parts of the General Public License. Of course, your program's commands
might be different; for a GUI interface, you would use an "about box".
You should also get your employer (if you work as a programmer) or school,
if any, to sign a "copyright disclaimer" for the program, if necessary.
For more information on this, and how to apply and follow the GNU GPL, see
<https://www.gnu.org/licenses/>.
The GNU General Public License does not permit incorporating your program
into proprietary programs. If your program is a subroutine library, you
may consider it more useful to permit linking proprietary applications with
the library. If this is what you want to do, use the GNU Lesser General
Public License instead of this License. But first, please read
<https://www.gnu.org/licenses/why-not-lgpl.html>.
+4
View File
@@ -23,6 +23,10 @@ public partial class MainWindow : Window
AppLog.Write("MainWindow ctor: after InitializeComponent");
_viewModel = (MainViewModel)DataContext;
_viewModel.PropertyChanged += OnViewModelPropertyChanged;
// Don't steal focus from an exclusive-fullscreen game the creator is already
// playing (e.g. while starting a gameplay stream): activating our window would
// drop that game to the taskbar and make the desktop look "locked".
ShowActivated = !_viewModel.FullscreenGameRunningNow;
_topBar = (Controls.TopBar)FindName("TopBar")!;
_previewPane = (Controls.PreviewPane)FindName("PreviewPane")!;
_leftPanel = (Controls.LeftPanel)FindName("LeftPanel")!;
+11
View File
@@ -0,0 +1,11 @@
namespace ytLive.Models;
/// <summary>Per-webcam background effect (TASK 51). <see cref="Remove"/> makes everything
/// outside the person transparent so the layers below show through; <see cref="Blur"/>
/// blurs the background behind the person.</summary>
public enum BackgroundEffect
{
None,
Remove,
Blur,
}
+8
View File
@@ -72,6 +72,14 @@ public class Scene : INotifyPropertyChanged
public bool IsChatScene { get; init; }
/// <summary>
/// Asset-table id of this scene's top-strip tile artwork. The tile is a
/// separate image from the scene's background: the preview pane keeps
/// rendering the scene's original background, the strip shows this instead.
/// Null = the strip falls back to the background art (creator ruling 2026-10-10).
/// </summary>
public string? ThumbnailAssetId { get; set; }
private BitmapSource? _snapshot;
/// <summary>
+42
View File
@@ -15,4 +15,46 @@ public class WebcamSceneConfig : SceneElement
public override bool IsDraggable => true;
public override ElementKind Kind => ElementKind.Dynamic;
public override ImageSource? DisplaySource => VideoImageSource;
private BackgroundEffect _backgroundEffect = BackgroundEffect.None;
/// <summary>TASK 51: this scene's background effect for the webcam (persisted).</summary>
public BackgroundEffect BackgroundEffect
{
get => _backgroundEffect;
set
{
if (!Set(ref _backgroundEffect, value)) return;
Raise(nameof(RemoveBackground));
Raise(nameof(BlurBackground));
Raise(nameof(HasBackgroundEffect));
}
}
/// <summary>True while Remove or Blur is on — shows the strength slider.</summary>
public bool HasBackgroundEffect => BackgroundEffect != BackgroundEffect.None;
private double _backgroundStrength = 0.5;
/// <summary>TASK 51: 0..1 — Blur amount, or Remove's edge softness (persisted).</summary>
public double BackgroundStrength
{
get => _backgroundStrength;
set => Set(ref _backgroundStrength, Math.Clamp(value, 0, 1));
}
/// <summary>Pill-switch surface: Remove and Blur behave like radio buttons — turning one
/// on turns the other off; turning the active one off means <see cref="BackgroundEffect.None"/>.</summary>
public bool RemoveBackground
{
get => BackgroundEffect == BackgroundEffect.Remove;
set => BackgroundEffect = value ? BackgroundEffect.Remove
: BackgroundEffect == BackgroundEffect.Remove ? BackgroundEffect.None : BackgroundEffect;
}
/// <inheritdoc cref="RemoveBackground"/>
public bool BlurBackground
{
get => BackgroundEffect == BackgroundEffect.Blur;
set => BackgroundEffect = value ? BackgroundEffect.Blur
: BackgroundEffect == BackgroundEffect.Blur ? BackgroundEffect.None : BackgroundEffect;
}
}
+1
View File
@@ -70,4 +70,5 @@ public class ChatMessage
/// subscriber events locally (YouTube's insert API only supports text), so the
/// buffer can label them honestly — simulated events never reach YouTube.</summary>
public bool IsSimulated { get; set; }
public bool IsPinned { get; set; }
}
+2 -1
View File
@@ -11,7 +11,8 @@ Plain data types. No logic beyond what a property can carry. See
| `SceneElement.cs` | Base for anything placeable in a scene: shared layout/clip/mirror/border surface, `virtual IsWebcam`/`virtual IsImageSource`/`virtual IsZoomLayer` (TASK 52); `ToggleClipShape` + persisted `RectWidth`/`RectHeight` (pre-Round rect so round→rect restores after reload) |
| `ZoomSceneElement.cs` | **Transient zoom box (TASK 52, creator-corrected 2026-10-08):** carries the box rect AND the anchored `CaptureX`/`CaptureY` (the fixed box/2 capture window set on `ActivateZoom`; moving the box never touches it). `IsZoomActive` flips `Kind` Static→Dynamic (leaves the baked base while active); never persisted (LayoutStore save skips non-Source/non-WebcamSceneConfig rows), never shown in the layers list or the element template (`IsZoomLayer` gates both). The compositor SKIPS the active element in every layer loop and draws its 2x content LAST (layer n+1) via `ZoomMagnifier`. |
| `Webcam.cs` | Singleton webcam identity: `Id`, `DeviceId`, `Name` (one row app-wide) |
| `WebcamSceneConfig.cs` | Per-scene webcam placement (subclass of `SceneElement`): geometry + `IsVisible` + border (`BorderColor`/`BorderOpacity`/`BorderWidth`/`BorderAnimation`) + `VideoImageSource`; `WebcamId` links to `Webcam` |
| `BackgroundEffect.cs` | **TASK 51:** per-webcam background effect enum `None` / `Remove` / `Blur`, carried by `WebcamSceneConfig` |
| `WebcamSceneConfig.cs` | Per-scene webcam placement (subclass of `SceneElement`): geometry + `IsVisible` + border (`BorderColor`/`BorderOpacity`/`BorderWidth`/`BorderAnimation`) + `VideoImageSource`; `WebcamId` links to `Webcam`; **TASK 51:** `BackgroundEffect` + `BackgroundStrength` (0..1, default 0.5) + the radio-pill surface `RemoveBackground`/`BlurBackground` (one on turns the other off; both off = None) + `HasBackgroundEffect` (shows the strength slider) |
| `QualityOption.cs` | Resolution tier record `(Display, Fps, Bitrate, Width, Height)`; `Label` formats as `1080p60 (8Mbps)` — drives the bottom-bar dropdown and the output-rect math |
| `StreamConfig.cs` | `StreamConfig` (note: `TargetBitrate`/`Resolution` defaults are stale — not yet wired to the dropdown), `StreamHealth` (bitrate/FPS/dropped/duration), `StreamStatus` enum |
| `MicStatus.cs` | Footer mic indicator: `NotConnected` / `Connected` / `Problem` — the state behind the MIC button's status dot (red = no device/capture off, green = capturing, yellow = a requested mic failed: in use/unplugged) |
+16
View File
@@ -15,6 +15,22 @@
## 🔬 Recipes registry
### CONVERTING A MEDIAPIPE .tflite (WITH CUSTOM OPS) TO ONNX — AND PROVING IT (RECIPE, TASK 51)
**Problem:** MediaPipe segmentation models end in the custom op `Convolution2DTransposeBias`;
tf2onnx and tflite2onnx refuse it. **Recipe:** `tools/selfie-seg/convert_tflite_to_onnx.py` —
parse the flatbuffer with the `tflite` pip package, emit ONNX with `onnx.helper`, transpose
NHWC→NCHW once at input/output, fold every `DEQUANTIZE` (fp16 weights) into fp32 initializers.
Weight layouts: CONV OHWI→OIHW; DEPTHWISE [1,H,W,C]→[C,1,H,W] with `group=C`; transpose-conv
[O,H,W,I]→[I,O,H,W]. TF `SAME` padding = ONNX `auto_pad=SAME_UPPER`.
- ⚠️ **Enum trap:** the custom op's options are the C-API `TfLiteTransposeConvParams`, whose
`padding` uses `TfLitePadding` (Unknown=0, **SAME=1**, VALID=2) — NOT the flatbuffer `Padding`
enum (SAME=0). Comparing against `Padding.SAME` rejects the real model.
- **Proof, not hope:** `pip install mediapipe` ships the TFLite runtime WITH the custom op. Feed
an image already at the model's exact input size (no resampling on either side) to MediaPipe's
`ImageSegmenter` and to onnxruntime; compare masks (`verify_against_mediapipe.py`, exit 0 at
max |diff| < 1e-3 — we got 5e-5). Work in a throwaway venv under `/tmp/opencode/`.
### ⛔ NEVER PRINT CREDENTIALS.md — NOT EVEN "FILTERED" (2026-10-09 incident)
**What happened:** to look for an API key, a `sed -n '1,40p' CREDENTIALS.md` piped through a
+102 -42
View File
@@ -1,74 +1,134 @@
# ytLlive
# LlamaCasty
<p align="center">
<img src="Assets/llamacasty-logo.jpg" alt="LlamaCasty — livestream smarter" height="120"/>
</p>
A creator-proof live-streaming and recording app for YouTube — native Windows (C# / WPF / .NET 8).
A free, open-source Windows app for streaming live to YouTube and recording to your PC.
Native Windows (C# / WPF / .NET 8). Website: https://llamacasty.com
## Screenshot
![LlamaCasty — main window with scenes, sources, and live preview](docs/ytLlive-preview.jpg)
![ytLlive — main window with scenes, sources, and live preview](docs/ytLlive-preview.jpg)
## Why I built it
## Design Principle
I'm Gramps. I've been gaming since 1981, and these days I stream *The Division 2* on YouTube.
> This software is so intuitive that even the most right-brained person can easily intuit and use it.
When I started streaming, I did what everybody does: I installed the big, general-purpose
streaming tools. They're excellent software — powerful, flexible, and built to handle every
platform, every camera rig, and every plugin anyone has ever dreamed up. That breadth is exactly
what makes them great, and it's also what makes them a lot to learn. Before my first stream I was
reading setup guides, picking encoders, balancing audio by hand, and wiring up scenes from
scratch. The tools were ready for anything. I just wanted to go live.
Every feature is measured against this. One-click go-live, sane YouTube defaults, visual scene
building, and no dead ends — every action has a visible outcome.
Most streamers I talk to are in the same spot. They don't need a tool for every platform and every
setup — they need one that's ready for *theirs*. So I built LlamaCasty around three choices:
## Why ytLlive
- **One platform, done properly.** LlamaCasty streams to YouTube and only YouTube. That focus lets
it be built around how YouTube Live actually works — your channel, your chat, Super Chats and
memberships, Shorts-style vertical video, and the YouTube Partner Program — instead of treating
YouTube as one destination among many.
- **Ready the moment you open it.** Five scenes (Starting, Live, BRB, Chat, Ending) are set up,
your game and voice are balanced with auto-ducking and voice polish built in, and YouTube's
settings are filled in. You arrange things by dragging them, then press Go Live.
- **Calm while you're live.** A private Test Stream to practice, a layout that locks while you're
on air, and warnings only when YouTube reports a real problem.
OBS treats YouTube as an afterthought; SLOBS is Twitch-first with YouTube bolted on. ytLlive is
designed and dedicated to YouTube livestreaming and recording, and to YouTube's specific quirks.
The general-purpose tools are the right choice when you need everything. LlamaCasty is for when you
just want to get your show on air.
## Account Assumption
## Free, and staying that way
ytLlive assumes you already have a YouTube creator account — connecting uses "Sign in with Google"
to link that existing account. **ytLlive does not create accounts.** If you don't have a YouTube
channel yet, set one up on YouTube first, then connect it here.
LlamaCasty is free forever: no watermark, no branding on your stream, no subscription, no
LlamaCasty account. It's also open source (see [License](#license)) — enjoy it, explore it,
experiment with it. If it earns a spot in your setup and you'd like to say thanks, there's a
coffee jar and a merch store linked from https://llamacasty.com. Completely optional, never
expected, and the app will never ask.
## Audio Assumption
## How it's built
ytLlive uses your **default** microphone and desktop/game output — it never hunts for a device or
contends for one. **It assumes your audio is already configured and working** (default input and
output set, not hijacked by another app). Audio issues are a Windows/device matter outside ytLlive's
scope: we don't debug, detect, or try to fix them. If it doesn't sound right, fix the system's audio
settings first — it's on the OS, not on us.
LlamaCasty is built through AI-assisted pair programming, under the direction of a developer with
decades of experience. Architecture, product direction, and quality gates are human-owned; AI
speeds up the implementation. The repo's memory map (below) is how that collaboration stays honest
from one session to the next.
## Run
## Requirements
```bash
dotnet build # Windows only — WPF
dotnet run
```
- **64-bit Windows 10 or 11.** (The bundled FFmpeg and ONNX Runtime are 64-bit only.)
- **A YouTube channel, to stream.** LlamaCasty links an existing channel with "Sign in with
Google" — it doesn't create accounts. Recording to an .mp4 works without signing in.
- **Working sound in Windows.** Desktop and game audio come from your default output
automatically; you choose your mic in the app. LlamaCasty doesn't diagnose or repair Windows
audio — if it doesn't sound right in Windows, fix it there first.
## Version Roadmap
## Build from source
**v1 is the finished product** (creator ruling, 2026-09-01 — there are no v1.x releases).
Everything still queueing lands in v1; anything not on the feature list is on the closed
**"Out of product — permanently"** list in `TASKS.md` — read that page before asking for a feature.
The internal project, assembly, and namespace names are still `ytLive` (the app was renamed to
LlamaCasty; the internals weren't). Building needs Windows and the .NET 8 SDK.
What v1 ships: the five-scene director's control surface, webcam + screen capture, media + web +
text sources, chat on-stream, the social bar, mic/desktop audio with auto-duck and TRAX, local
recording, variable quality tiers with auto step-down, one-click go-live + scheduling, stream
resilience (auto-reconnect inside YouTube's grace, one-click back on air), built-in monetization
awareness (reward capture, session reports, YPP journey tracker), Alerts — all free and ungated;
the paid tier only removes the branding flash.
1. **Fetch the bundled FFmpeg** (not in git — ~135 MB, pinned and SHA-256-verified):
`scripts/fetch-ffmpeg.sh`
2. **Add your own Google OAuth client.** `Helpers/OAuthCredentials.cs` is gitignored. Create a
"Desktop app" OAuth client with the YouTube Data API v3 enabled in Google Cloud Console, then:
```csharp
namespace ytLive.Helpers;
public static class OAuthCredentials
{
public const string ClientId = "your-client-id.apps.googleusercontent.com";
public const string ClientSecret = "your-client-secret";
}
```
3. **Build and run:**
```bash
dotnet build ytLive.csproj
dotnet run --project ytLive.csproj
```
## Version roadmap
**v1 is the finished product** (no v1.x releases are planned). Everything still in the queue lands
in v1; anything not on the feature list is on the **"Out of product — permanently"** list in
`TASKS.md` — read that before asking for a feature.
What v1 ships: the five-scene control surface, webcam + screen capture, media + web + text
sources, chat on stream, the social bar, mic/desktop audio with auto-ducking and TRAX background
music, local recording, quality tiers including vertical 9:16, one-click go-live + scheduling,
stream resilience, monetization awareness (YPP tracker), animated alerts, and webcam background
removal and blur — all free and ungated.
## Structure
| Path | Role |
|------|------|
| `Models/` | Scene, Source, WebcamSceneConfig, SceneElement, SceneCatalog, QualityOption, StreamConfig/Health, chat, Socials, BroadcastMetadata, Music, MicStatus |
| `Models/` | Scene, Source, WebcamSceneConfig, SceneElement, SceneCatalog, QualityOption, StreamConfig/Health, chat, Socials, BroadcastMetadata, Music, MicStatus, BackgroundEffect |
| `ViewModels/` | MainViewModel (partial classes per area) + dialogs: GoLive, CameraPicker, MicPicker, HotkeyConfig, Socials, LiveBroadcastForm |
| `Services/` | YouTube (OAuth, streams/broadcasts, chat polling), LayoutStore (SQLite), capture (camera/screen-capture/media-file via FFmpeg), SceneCompositor + FramePump, FFmpeg encoder + RTMP, audio pipeline (WASAPI loopback + mic, mixer, filters, ducker, limiter, TRAX), SocialValidator, notifications, hotkeys, SceneGraph, ChatOverlayLayer |
| `Helpers/` | ViewModelBase, RelayCommand, ImageCache, AppLog, TokenStore (DPAPI), OAuthCredentials, converters |
| `Services/` | YouTube (OAuth, streams/broadcasts, chat polling), LayoutStore (SQLite), capture (camera/screen-capture/media-file via FFmpeg), SceneCompositor + FramePump, FFmpeg encoder + RTMP, audio pipeline (WASAPI loopback + mic, mixer, filters, ducker, limiter, TRAX), background effects (ONNX person segmentation), SocialValidator, notifications, hotkeys, SceneGraph, ChatOverlayLayer |
| `Helpers/` | ViewModelBase, RelayCommand, ImageCache, AppLog, TokenStore (DPAPI), OAuthCredentials (gitignored), converters |
| `Themes/` | `Controls.xaml` — the dark-theme control styles, merged once in `App.xaml` |
| `MainWindow.xaml` | Dark-theme control surface: thumbnails + central preview, two-state left panel (layers/properties ↔ live chat), top bar, footer |
| `Distribution.md` | GA-time plan: signing, installer, EULA, hardening — executed as TASK 36 |
| `tools/` | Offline helper tools (e.g. `selfie-seg/` — converts and verifies the background-effects model) |
## Docs (memory map)
`schema.md` defines the conventions; `ai.md` is the AI/session guide;
`TASKS.md` is the task queue. Each code directory carries an `index.md`.
`schema.md` defines the conventions; `ai.md` is the AI/session guide; `TASKS.md` is the task
queue; `HANDOFF.md` is the current state. Each code directory carries an `index.md`.
## License
LlamaCasty is free software, released under the **GNU General Public License v3.0 or later**
([`LICENSE`](LICENSE)) with three additional terms permitted by GPL-3.0 section 7
([`ADDITIONAL-TERMS.md`](ADDITIONAL-TERMS.md)):
- **Keep the credit** — any copy, modified or not, shows "Based on LlamaCasty by Gramps —
https://llamacasty.com" in its About screen.
- **Rename your fork** — modified versions are marked as modified and use a different name.
- **No trademark rights** — the LlamaCasty name and llama logo aren't licensed. The icon and logo
files are all rights reserved and aren't covered by the GPL.
The scene backgrounds and default alert video are AI-generated placeholders, licensed with the
code — replace them with your own. Bundled third-party components keep their own licenses: see
[`THIRD-PARTY-NOTICES.txt`](THIRD-PARTY-NOTICES.txt).
Source code: https://llgit.llamachile.tube/gramps/LlamaCasty
+9 -7
View File
@@ -16,18 +16,20 @@ public sealed class AudioLevelMeter
/// <summary>
/// Maps a linear level (0..1) onto the meter's display scale: -60 dBFS..0 dBFS
/// spread linearly across 0..1, with +10 dB of input amplification. Linear RMS
/// of real speech or game audio is ~0.01..0.1 (-40..-20 dBFS), which leaves a
/// flat (linear) meter looking dead; the log scale makes typical levels occupy
/// the bar and the amplification pushes real speech peaks into the red zone
/// (0.8+) at maxed volume instead of hovering at its edge. Inputs at or below
/// -60 dBFS (0.001 linear) read as zero — the meter never idles on background noise.
/// spread linearly across 0..1 — an HONEST dBFS scale. Linear RMS of real speech is
/// ~0.01..0.1 (-40..-20 dBFS), which leaves a flat (linear) meter looking dead; the
/// log scale makes typical levels occupy the bar. Inputs at or below -60 dBFS
/// (0.001 linear) read as zero — the meter never idles on background noise.
/// <para>2026-10-09: the +10 dB "input amplification" that used to sit here is gone.
/// It made a −30 dBFS voice look dialed in, and a creator's upload came out at
/// −28.5 LUFS. The speech leveler now puts voice at −20 dBFS, which lands at 2/3 of
/// the bar (the green/yellow edge); loud peaks reach the red.</para>
/// </summary>
public static float ToDisplay(float linear)
{
if (linear <= 0.001f)
return 0f;
var db = 20f * MathF.Log10(linear) + 10f;
var db = 20f * MathF.Log10(linear);
return Math.Clamp(1f + db / 60f, 0f, 1f);
}
+14
View File
@@ -0,0 +1,14 @@
namespace ytLive.Services.Audio;
/// <summary>Seam over the TRAX background-music player (<see cref="MusicPlayer"/>) so the
/// toggle/status-light logic is testable without a real audio device.</summary>
public interface ITraxPlayer : IDisposable
{
event Action<Exception?>? PlaybackEnded;
string? TrackName { get; }
bool IsPlaying { get; }
float LocalGain { get; set; }
void Load(string path);
void Play();
void Pause();
}
+10 -5
View File
@@ -11,12 +11,17 @@ namespace ytLive.Services.Audio;
/// The desktop/game bar's gain and mute scale <see cref="LocalGain"/>, so the
/// creator hears the game-audio controls work on the music too.
/// </summary>
public sealed class MusicPlayer : IDisposable
public sealed class MusicPlayer : ITraxPlayer
{
/// <summary>The one and only music bed — quiet, no slider (the creator's
/// voice must stay on top; game/music loudness is the ducker's job). The
/// desktop audio bar scales this via <see cref="LocalGain"/>.</summary>
public const float MusicVolume = 0.20f;
/// <summary>The one and only music bed — no slider (the creator's voice must stay on
/// top; that's the ducker's job, −12 dB while they talk). The desktop audio bar scales
/// this via <see cref="LocalGain"/>.
/// <para>2026-10-09: was 0.20 (−14 dB) — "even at max volume I can barely discern it".
/// The creator's track masters at −11.3 LUFS (typical: −9..−14), so 0.20 put the bed at
/// ~−25 LUFS, and ducked under speech ~−37 LUFS: inaudible on stream too, because the
/// loopback carries exactly what plays locally. 0.5 (−6 dB) lands a typical master at
/// ~−15..−20 LUFS — beside the leveled voice (≈ −17 LUFS), and well under it when ducked.</para></summary>
public const float MusicVolume = 0.5f;
private MediaFoundationReader? _reader;
private VolumeWaveProvider16? _volume;
+92 -4
View File
@@ -3,15 +3,21 @@ namespace ytLive.Services.Audio;
/// <summary>
/// The always-on voice processing chain for the mic (TASK 9 audio milestone).
/// Pure, per-sample, unit-tested: bass boost → treble lift → noise gate →
/// compressor. Applied BEFORE the meter and the stream mix so what the creator
/// hears on the meter is what viewers hear — and so background hum is gated
/// out before it reaches the encoder.
/// <b>speech leveler</b> → compressor. Applied BEFORE the meter and the stream mix so
/// what the creator sees on the meter is what viewers hear — and so background hum
/// is gated out before it reaches the encoder.
/// <para>The leveler (2026-10-09) is the gain stage the chain was missing: a creator's
/// upload measured −28.5 LUFS against YouTube's −14 reference because nothing ever
/// brought quiet speech UP — the compressor only folds peaks above −6 dBFS and the
/// master limiter only caps at −1 dBFS. Compressor → gain → limiter is the standard
/// streaming voice chain.</para>
/// </summary>
public sealed class VoiceFilterChain
{
private readonly LowShelfFilter _bass;
private readonly HighShelfFilter _treble;
private readonly NoiseGate _gate;
private readonly SpeechLeveler _leveler;
private readonly Compressor _compressor;
/// <param name="sampleRate">The (post-resample) mic rate — the RBJ shelf
@@ -21,15 +27,20 @@ public sealed class VoiceFilterChain
_bass = new LowShelfFilter(sampleRate, 120f, 4f);
_treble = new HighShelfFilter(sampleRate, 8000f, 3f);
_gate = new NoiseGate();
_leveler = new SpeechLeveler(sampleRate);
_compressor = new Compressor();
}
/// <summary>The leveler's current gain (linear) — for diagnostics and tests.</summary>
public float LevelerGain => _leveler.Gain;
public float Process(float sample)
{
var bass = _bass.Process(sample);
var treble = _treble.Process(bass);
var gated = _gate.Process(treble);
return _compressor.Process(gated);
var leveled = _leveler.Process(gated, _gate.IsOpen);
return _compressor.Process(leveled);
}
public void Reset()
@@ -37,10 +48,87 @@ public sealed class VoiceFilterChain
_bass.Reset();
_treble.Reset();
_gate.Reset();
_leveler.Reset();
_compressor.Reset();
}
}
/// <summary>
/// Automatic gain for speech: tracks the voice's RMS over ~0.3 s and slowly steers a
/// gain so speech sits at <see cref="TargetRms"/> (−20 dBFS mono ≈ −17 LUFS once the mic
/// is mixed to stereo — close to YouTube's −14 reference while leaving the limiter room).
/// Different mics differ by 20 dB+ in sensitivity; this is what makes "it just sounds
/// right" true for all of them.
/// <list type="bullet">
/// <item>Only learns while the noise gate is open and the voice is above a silence floor —
/// pauses and background hum never get pumped up.</item>
/// <item>Gain rises at most <see cref="RiseDbPerSecond"/> and falls at most
/// <see cref="FallDbPerSecond"/> (faster down than up, so a sudden shout is tamed quickly
/// and a quiet stretch isn't chased), within <see cref="MinGain"/>..<see cref="MaxGain"/>.</item>
/// <item>Peaks it pushes up are caught by the <see cref="Compressor"/> after it and the
/// master limiter at the end of the mix.</item>
/// </list>
/// </summary>
public sealed class SpeechLeveler
{
public const float TargetRms = 0.1f; // −20 dBFS
public const float MaxGain = 10f; // +20 dB
public const float MinGain = 0.5f; // −6 dB
public const float SilenceFloorRms = 0.002f; // −54 dBFS: below this, hold the gain
public const float RiseDbPerSecond = 10f;
public const float FallDbPerSecond = 30f;
private const float WindowSeconds = 0.3f;
private const float UpdateSeconds = 0.01f;
private readonly float _meanAlpha;
private readonly int _updateEvery;
private readonly float _maxRiseDb;
private readonly float _maxFallDb;
private float _meanSquare;
private float _gainDb;
private float _gain = 1f;
private int _sinceUpdate;
public SpeechLeveler(int sampleRate)
{
_meanAlpha = 1f / (WindowSeconds * sampleRate);
_updateEvery = Math.Max(1, (int)(UpdateSeconds * sampleRate));
_maxRiseDb = RiseDbPerSecond * UpdateSeconds;
_maxFallDb = FallDbPerSecond * UpdateSeconds;
}
public float Gain => _gain;
public float Process(float sample, bool voiceActive)
{
if (voiceActive)
_meanSquare += (sample * sample - _meanSquare) * _meanAlpha;
if (++_sinceUpdate >= _updateEvery)
{
_sinceUpdate = 0;
var rms = MathF.Sqrt(_meanSquare);
if (voiceActive && rms > SilenceFloorRms)
{
var desiredDb = 20f * MathF.Log10(Math.Clamp(TargetRms / rms, MinGain, MaxGain));
var step = Math.Clamp(desiredDb - _gainDb, -_maxFallDb, _maxRiseDb);
_gainDb += step;
_gain = MathF.Pow(10f, _gainDb / 20f);
}
}
return sample * _gain;
}
public void Reset()
{
_meanSquare = 0;
_gainDb = 0;
_gain = 1f;
_sinceUpdate = 0;
}
}
/// <summary>
/// RBJ audio EQ cookbook low-shelf biquad (bass boost). Transposed direct
/// form II — stable, cheap, standard.
@@ -0,0 +1,356 @@
using ytLive.Helpers;
using ytLive.Models;
namespace ytLive.Services.BackgroundEffects;
/// <summary>
/// Applies a webcam's <see cref="BackgroundEffect"/> to one captured frame (TASK 51).
/// Pure pixel work over the <see cref="VideoFrame"/> seam — WPF-free and thread-agnostic,
/// so the camera's processing worker can call it before the frame reaches the preview bitmap
/// and the compositor.
///
/// <list type="bullet">
/// <item><see cref="BackgroundEffect.None"/> returns the SAME frame instance — a true
/// passthrough, no copy.</item>
/// <item><see cref="BackgroundEffect.Remove"/> returns a copy whose alpha is the person mask,
/// RGB untouched, <c>IsOpaque = false</c> — the compositor's straight-alpha blend then shows
/// the layers below the webcam. Strength = edge softness (0 = crisp cut-out, 1 = the model's
/// raw soft mask).</item>
/// <item><see cref="BackgroundEffect.Blur"/> returns a copy where the background is replaced
/// by a blurred version of itself and the person stays sharp. Strength = blur amount.</item>
/// </list>
///
/// The mask is computed ONCE per camera frame (<see cref="TrySegment"/>) and then rendered for
/// every effect variant in use (<see cref="Render"/>). A segmenter that returns no mask (or
/// throws) never drops the layer: the frame passes through and the failure is logged once.
/// </summary>
public sealed class BackgroundEffectProcessor
{
public const double DefaultStrength = 0.5;
/// <summary>The background is blurred at 1/<see cref="BlurDownscale"/> resolution and
/// upsampled while blending — a strong blur at a fraction of the cost.</summary>
internal const int BlurDownscale = 4;
/// <summary>Box-blur radius (in downscaled pixels) at strength 0 and 1. The maximum was halved
/// from 12 (2026-10-09): the creator found full range too heavy — the old max spread ≈48 full
/// px. The established tools blur far less: StreamFX defaults its Box blur size to 5 and its
/// author stops downscale-based blurs around 8 px.</summary>
internal const int MinBlurRadius = 1;
internal const int MaxBlurRadius = 6;
private readonly IPersonSegmenter _segmenter;
private bool _loggedFailure;
public BackgroundEffectProcessor(IPersonSegmenter segmenter)
{
_segmenter = segmenter;
}
public VideoFrame Apply(VideoFrame frame, BackgroundEffect effect, double strength = DefaultStrength)
{
if (effect == BackgroundEffect.None) return frame;
var mask = TrySegment(frame);
return mask == null ? frame : Render(frame, mask, effect, strength);
}
/// <summary>Runs the segmenter once; null (logged once) when it can't produce a usable mask.</summary>
public SegmentationMask? TrySegment(VideoFrame frame)
{
SegmentationMask? mask;
try
{
mask = _segmenter.Segment(frame);
}
catch (Exception ex)
{
LogFailureOnce($"{ex.GetType().Name}: {ex.Message}");
return null;
}
if (mask == null || mask.Width < 1 || mask.Height < 1 || mask.Values.Length < mask.Width * mask.Height)
{
LogFailureOnce("segmenter returned no usable mask");
return null;
}
return mask;
}
public static VideoFrame Render(VideoFrame frame, SegmentationMask mask, BackgroundEffect effect, double strength)
{
var s = Math.Clamp(strength, 0, 1);
return effect switch
{
BackgroundEffect.Remove => ApplyRemove(frame, mask, s),
BackgroundEffect.Blur => ApplyBlur(frame, mask, s),
_ => frame,
};
}
internal static VideoFrame ApplyRemove(VideoFrame frame, SegmentationMask mask, double strength)
{
var src = frame.BgraPixels;
var dst = new byte[src.Length];
Buffer.BlockCopy(src, 0, dst, 0, src.Length);
// Edge feather: map the mask's [0.5 - w, 0.5 + w] band to alpha 0..1, as a 256-entry LUT
// over the quantized mask value (so the per-pixel work is a lookup + one multiply).
var w = 0.05 + 0.45 * strength;
var lo = 0.5 - w;
var span = 2 * w;
var alphaLut = new int[256];
for (var i = 0; i < 256; i++)
alphaLut[i] = (int)Math.Round(Math.Clamp((i / 255.0 - lo) / span, 0, 1) * 256);
var width = frame.Width;
var height = frame.Height;
var stride = frame.Stride;
var maskRows = UpsampleMaskRows(mask, width);
var (r0, r1, fy) = AxisLut(mask.Height, height);
Parallel.For(0, height, RowParallelism, y =>
{
var a0 = maskRows[r0[y]];
var a1 = maskRows[r1[y]];
var t = fy[y];
var row = y * stride + 3;
for (var x = 0; x < width; x++)
{
var m = a0[x] + (a1[x] - a0[x]) * t;
var q = (int)(m * 255f + 0.5f);
q = q < 0 ? 0 : q > 255 ? 255 : q;
var p = row + x * 4;
dst[p] = (byte)((src[p] * alphaLut[q] + 128) >> 8);
}
});
return new VideoFrame(width, height, dst) { Epoch = frame.Epoch, IsOpaque = false };
}
internal static VideoFrame ApplyBlur(VideoFrame frame, SegmentationMask mask, double strength)
{
var width = frame.Width;
var height = frame.Height;
var src = frame.BgraPixels;
var stride = frame.Stride;
// 1. Downscale by box-averaging BlurDownscale×BlurDownscale blocks (3 float channels).
var f = BlurDownscale;
var sw = (width + f - 1) / f;
var sh = (height + f - 1) / f;
var small = new float[sw * sh * 3];
Parallel.For(0, sh, RowParallelism, sy =>
{
var y0 = sy * f;
var y1 = Math.Min(height, y0 + f);
for (var sx = 0; sx < sw; sx++)
{
var x0 = sx * f;
var x1 = Math.Min(width, x0 + f);
float b = 0, g = 0, r = 0;
var n = 0;
for (var y = y0; y < y1; y++)
{
var p = y * stride + x0 * 4;
for (var x = x0; x < x1; x++, p += 4)
{
b += src[p]; g += src[p + 1]; r += src[p + 2];
n++;
}
}
var o = (sy * sw + sx) * 3;
small[o] = b / n; small[o + 1] = g / n; small[o + 2] = r / n;
}
});
// 2. Three separable box-blur passes ≈ a gaussian.
var radius = BlurRadius(strength);
var scratch = new float[small.Length];
for (var pass = 0; pass < 3; pass++)
{
BoxBlurHorizontal(small, scratch, sw, sh, radius);
BoxBlurVertical(scratch, small, sw, sh, radius);
}
// 3. Upsample: horizontal interpolation precomputed once per small row (and per mask row),
// then each output row lerps two precomputed rows — no per-pixel 2-D bilinear.
var blurRows = UpsampleColorRows(small, sw, sh, width);
var (b0, b1, bfy) = AxisLut(sh, height);
var maskRows = UpsampleMaskRows(mask, width);
var (m0, m1, mfy) = AxisLut(mask.Height, height);
// 4. Blend: person (mask) keeps the source pixel, background takes the blur.
var dst = new byte[src.Length];
Parallel.For(0, height, RowParallelism, y =>
{
var ma = maskRows[m0[y]];
var mb = maskRows[m1[y]];
var mt = mfy[y];
var ba = blurRows[b0[y]];
var bb = blurRows[b1[y]];
var bt = bfy[y];
var row = y * stride;
for (var x = 0; x < width; x++)
{
var p = row + x * 4;
var m = ma[x] + (mb[x] - ma[x]) * mt;
if (m >= 0.999f)
{
dst[p] = src[p]; dst[p + 1] = src[p + 1]; dst[p + 2] = src[p + 2]; dst[p + 3] = src[p + 3];
continue;
}
if (m < 0f) m = 0f;
var k = 1f - m;
var c = x * 3;
var vb = ba[c] + (bb[c] - ba[c]) * bt;
var vg = ba[c + 1] + (bb[c + 1] - ba[c + 1]) * bt;
var vr = ba[c + 2] + (bb[c + 2] - ba[c + 2]) * bt;
dst[p] = ClampByte(src[p] * m + vb * k);
dst[p + 1] = ClampByte(src[p + 1] * m + vg * k);
dst[p + 2] = ClampByte(src[p + 2] * m + vr * k);
dst[p + 3] = src[p + 3];
}
});
return new VideoFrame(width, height, dst) { Epoch = frame.Epoch, IsOpaque = frame.IsOpaque };
}
/// <summary>Blur radius in downscaled pixels for a strength. Quadratic (s²), not linear, so the
/// default 0.5 stays gentle (2 small px ≈ ~8–10 full px of perceived spread, ~3× lighter than
/// the old linear 6) while only the top of the slider approaches the full range.</summary>
internal static int BlurRadius(double strength)
{
var s = Math.Clamp(strength, 0, 1);
return (int)Math.Round(MinBlurRadius + (MaxBlurRadius - MinBlurRadius) * s * s);
}
/// <summary>Row-parallel pixel loops, capped so the effect worker can't starve the compositor
/// and encoder threads.</summary>
private static readonly ParallelOptions RowParallelism = new()
{
MaxDegreeOfParallelism = Math.Clamp(Environment.ProcessorCount / 2, 1, 4),
};
private static byte ClampByte(float v)
{
var i = (int)(v + 0.5f);
return (byte)(i < 0 ? 0 : i > 255 ? 255 : i);
}
/// <summary>For each destination index along one axis: the two source indices and the
/// fractional weight — pixel-center aligned, edge-clamped (the same mapping a bilinear
/// sample uses).</summary>
private static (int[] I0, int[] I1, float[] F) AxisLut(int srcSize, int dstSize)
{
var i0 = new int[dstSize];
var i1 = new int[dstSize];
var fr = new float[dstSize];
for (var d = 0; d < dstSize; d++)
{
var s = (d + 0.5f) * srcSize / dstSize - 0.5f;
var a = (int)MathF.Floor(s);
var t = s - a;
i0[d] = Math.Clamp(a, 0, srcSize - 1);
i1[d] = Math.Clamp(a + 1, 0, srcSize - 1);
fr[d] = a < 0 ? 0f : a + 1 > srcSize - 1 ? 0f : t;
}
return (i0, i1, fr);
}
/// <summary>Each mask row, horizontally interpolated (and clamped to 0..1) to the frame width.</summary>
private static float[][] UpsampleMaskRows(SegmentationMask mask, int width)
{
var (x0, x1, fx) = AxisLut(mask.Width, width);
var rows = new float[mask.Height][];
var v = mask.Values;
for (var r = 0; r < mask.Height; r++)
{
var row = new float[width];
var o = r * mask.Width;
for (var x = 0; x < width; x++)
{
var a = v[o + x0[x]];
var m = a + (v[o + x1[x]] - a) * fx[x];
row[x] = m < 0f ? 0f : m > 1f ? 1f : m;
}
rows[r] = row;
}
return rows;
}
/// <summary>Each row of a 3-channel float image, horizontally interpolated to <paramref name="width"/>.</summary>
private static float[][] UpsampleColorRows(float[] img, int w, int h, int width)
{
var (x0, x1, fx) = AxisLut(w, width);
var rows = new float[h][];
Parallel.For(0, h, RowParallelism, r =>
{
var row = new float[width * 3];
var o = r * w;
for (var x = 0; x < width; x++)
{
var a = (o + x0[x]) * 3;
var b = (o + x1[x]) * 3;
var t = fx[x];
row[x * 3] = img[a] + (img[b] - img[a]) * t;
row[x * 3 + 1] = img[a + 1] + (img[b + 1] - img[a + 1]) * t;
row[x * 3 + 2] = img[a + 2] + (img[b + 2] - img[a + 2]) * t;
}
rows[r] = row;
});
return rows;
}
private static void BoxBlurHorizontal(float[] input, float[] output, int w, int h, int r)
{
var norm = 1f / (2 * r + 1);
for (var y = 0; y < h; y++)
{
var row = y * w;
for (var c = 0; c < 3; c++)
{
float sum = 0;
for (var i = -r; i <= r; i++)
sum += input[(row + Math.Clamp(i, 0, w - 1)) * 3 + c];
for (var x = 0; x < w; x++)
{
output[(row + x) * 3 + c] = sum * norm;
var add = Math.Min(w - 1, x + r + 1);
var sub = Math.Max(0, x - r);
sum += input[(row + add) * 3 + c] - input[(row + sub) * 3 + c];
}
}
}
}
private static void BoxBlurVertical(float[] input, float[] output, int w, int h, int r)
{
var norm = 1f / (2 * r + 1);
for (var x = 0; x < w; x++)
{
for (var c = 0; c < 3; c++)
{
float sum = 0;
for (var i = -r; i <= r; i++)
sum += input[(Math.Clamp(i, 0, h - 1) * w + x) * 3 + c];
for (var y = 0; y < h; y++)
{
output[(y * w + x) * 3 + c] = sum * norm;
var add = Math.Min(h - 1, y + r + 1);
var sub = Math.Max(0, y - r);
sum += input[(add * w + x) * 3 + c] - input[(sub * w + x) * 3 + c];
}
}
}
}
private void LogFailureOnce(string reason)
{
if (_loggedFailure) return;
_loggedFailure = true;
AppLog.Write($"BackgroundEffectProcessor: passing webcam through — {reason}");
}
}
@@ -0,0 +1,13 @@
namespace ytLive.Services.BackgroundEffects;
/// <summary>A person-probability mask at the segmenter's own resolution: row-major,
/// one value per pixel, 0 = background, 1 = person.</summary>
public sealed record SegmentationMask(int Width, int Height, float[] Values);
/// <summary>Seam over the person-segmentation model (TASK 51) so tests inject a fake and
/// never run real inference. Returns null when no mask can be produced; callers then pass
/// the frame through untouched.</summary>
public interface IPersonSegmenter
{
SegmentationMask? Segment(VideoFrame frame);
}
@@ -0,0 +1,68 @@
namespace ytLive.Services.BackgroundEffects;
/// <summary>
/// Temporal smoothing for the person mask (TASK 51): each raw frame's segmentation is blended
/// with the previous smoothed mask so the cut-out edge doesn't flutter from frame to frame when
/// the model's per-frame verdict wavers.
///
/// Blend = <see cref="CurrentWeight"/> × new + (1 − <see cref="CurrentWeight"/>) × previous.
/// Weights come from the established tools rather than a guess: OBS's obs-backgroundremoval
/// smoke-tests `temporal_smooth_factor` at its 0.85 default, and Jitsi's webcam background
/// filter (jitsi/jitsi-meet#17080) found 0.7/0.3 transformed best — a midpoint that still
/// reacts to real movement. 0.5/0.5 is worse: on motion it leaves a smearing edge ghost.
///
/// State lives per camera session (the <see cref="CameraManager"/>'s one-per-frame worker), NOT
/// in the segmenter — the segmenter is shared app-wide and its contract test pins the raw
/// per-frame mask.
/// </summary>
public sealed class MaskSmoother
{
/// <summary>Weight of the newest raw mask; the rest is the previous smoothed mask.</summary>
internal const double CurrentWeight = 0.75;
private float[]? _current;
private float[]? _scratch;
private int _width;
private int _height;
private bool _hasHistory;
/// <summary>
/// Returns the temporally-smoothed mask. A null input (segmenter hiccup) resets history so
/// the next real mask starts moving again; the first frame and any size change pass through
/// un-blended, and the returned mask never aliases the caller's array (the first frame copies
/// its history). The two buffers ping-pong so no per-frame allocation is needed — a 256×144
/// mask is 147 KB, past the 85 KB LOH threshold.
/// </summary>
public SegmentationMask? Smooth(SegmentationMask? raw)
{
if (raw == null)
{
_hasHistory = false;
_width = 0;
_height = 0;
return null;
}
if (!_hasHistory || _width != raw.Width || _height != raw.Height
|| _current == null || _scratch == null)
{
_width = raw.Width;
_height = raw.Height;
_current = new float[raw.Values.Length];
_scratch = new float[raw.Values.Length];
Array.Copy(raw.Values, _current, raw.Values.Length);
_hasHistory = true;
return raw;
}
var v = raw.Values;
var next = _scratch;
var keep = 1f - (float)CurrentWeight;
for (var i = 0; i < v.Length; i++)
next[i] = keep * _current[i] + (float)CurrentWeight * v[i];
_scratch = _current;
_current = next;
return new SegmentationMask(_width, _height, next);
}
}
@@ -0,0 +1,107 @@
using System.IO;
using Microsoft.ML.OnnxRuntime;
using ytLive.Helpers;
namespace ytLive.Services.BackgroundEffects;
/// <summary>
/// The default <see cref="IPersonSegmenter"/>: Google's MediaPipe Selfie Segmenter
/// (landscape, Apache-2.0) converted to ONNX by <c>tools/selfie-seg/</c> and shipped as an
/// embedded resource — nothing is downloaded at runtime and no GPU is required.
///
/// Contract: input [1,144,256,3] RGB in [0,1] (NHWC), output [1,144,256,1] person
/// probability. The whole frame is resampled to 256×144 (stretched, not cropped), so the
/// mask maps back to the frame by normalized coordinates.
///
/// Never throws from <see cref="Segment"/>: a model that fails to load or run logs once
/// and returns null, and the caller passes the webcam frame through.
/// </summary>
public sealed class OnnxPersonSegmenter : IPersonSegmenter, IDisposable
{
public const int ModelWidth = 256;
public const int ModelHeight = 144;
internal const string ModelResourceName = "ytLive.Assets.Models.selfie_segmenter_landscape.onnx";
private static readonly long[] InputShape = { 1, ModelHeight, ModelWidth, 3 };
private readonly object _gate = new();
private readonly float[] _input = new float[ModelWidth * ModelHeight * 3];
private readonly RunOptions _runOptions = new();
private InferenceSession? _session;
private bool _failed;
private bool _disposed;
public SegmentationMask? Segment(VideoFrame frame)
{
lock (_gate)
{
if (_failed || _disposed) return null;
try
{
_session ??= CreateSession();
FillInput(frame, _input);
using var input = OrtValue.CreateTensorValueFromMemory(_input, InputShape);
using var outputs = _session.Run(_runOptions, new[] { "input" }, new[] { input }, new[] { "mask" });
var mask = outputs[0].GetTensorDataAsSpan<float>().ToArray();
return new SegmentationMask(ModelWidth, ModelHeight, mask);
}
catch (Exception ex)
{
_failed = true;
AppLog.Write($"OnnxPersonSegmenter: background effects disabled — {ex.GetType().Name}: {ex.Message}");
return null;
}
}
}
public void Dispose()
{
lock (_gate)
{
_disposed = true;
_session?.Dispose();
_session = null;
_runOptions.Dispose();
}
}
private static InferenceSession CreateSession()
{
using var stream = typeof(OnnxPersonSegmenter).Assembly.GetManifestResourceStream(ModelResourceName)
?? throw new InvalidOperationException($"embedded model '{ModelResourceName}' is missing");
using var buffer = new MemoryStream();
stream.CopyTo(buffer);
// Two intra-op threads: the model is tiny, and the webcam must not starve the
// compositor/encoder threads (obs-backgroundremoval reports 2 as the sweet spot).
using var options = new SessionOptions
{
IntraOpNumThreads = 2,
InterOpNumThreads = 1,
GraphOptimizationLevel = GraphOptimizationLevel.ORT_ENABLE_ALL,
};
return new InferenceSession(buffer.ToArray(), options);
}
/// <summary>BGRA frame → model input: nearest-sample stretch to 256×144, RGB order,
/// values /255. Nearest is enough — the model input is ~5× smaller than a 720p frame
/// and the mask is the only thing that comes back.</summary>
internal static void FillInput(VideoFrame frame, float[] input)
{
var src = frame.BgraPixels;
var stride = frame.Stride;
var o = 0;
for (var y = 0; y < ModelHeight; y++)
{
var sy = Math.Min(frame.Height - 1, (int)((y + 0.5) * frame.Height / ModelHeight));
var row = sy * stride;
for (var x = 0; x < ModelWidth; x++)
{
var sx = Math.Min(frame.Width - 1, (int)((x + 0.5) * frame.Width / ModelWidth));
var p = row + sx * 4;
input[o++] = src[p + 2] / 255f;
input[o++] = src[p + 1] / 255f;
input[o++] = src[p] / 255f;
}
}
}
}
@@ -0,0 +1,16 @@
using ytLive.Models;
namespace ytLive.Services.BackgroundEffects;
/// <summary>Identifies one rendered variant of a camera's frames (TASK 51): the effect plus its
/// strength, rounded to 2 decimals so a slider drag doesn't spawn a variant per pixel of travel.
/// <see cref="BackgroundEffect.None"/> always normalizes to strength 0 — the raw path.</summary>
public readonly record struct WebcamEffectKey(BackgroundEffect Effect, double Strength)
{
public static WebcamEffectKey None => new(BackgroundEffect.None, 0);
public static WebcamEffectKey For(BackgroundEffect effect, double strength)
=> effect == BackgroundEffect.None
? None
: new WebcamEffectKey(effect, Math.Round(Math.Clamp(strength, 0, 1), 2));
}
+211 -1
View File
@@ -6,6 +6,8 @@ using System.Windows.Media;
using System.Windows.Media.Imaging;
using System.Windows.Threading;
using ytLive.Helpers;
using ytLive.Models;
using ytLive.Services.BackgroundEffects;
namespace ytLive.Services;
@@ -21,9 +23,24 @@ namespace ytLive.Services;
/// <paramref name="firstFrameTimeout"/>), and an async <see cref="SourceFailed"/>
/// tears the session down and surfaces <see cref="CameraFailed"/> — no silent
/// empty box.
///
/// Background effects (TASK 51) are the ONE processing point on the camera path: per device,
/// the set of requested <see cref="WebcamEffectKey"/>s (<see cref="SetRequestedEffects"/>) is
/// rendered from each raw frame on a background worker — one segmentation per frame, smoothed
/// per camera across frames (see <see cref="MaskSmoother"/>) and shared by every variant,
/// latest-frame-drop when the worker is busy — and each variant keeps its own
/// latest frame (the compositor) and shared WriteableBitmap (the preview). The raw path is
/// untouched, so a slow or failing model never stalls the plain webcam.
/// </summary>
public sealed class CameraManager : IDisposable
{
private sealed class EffectVariant
{
public WriteableBitmap? PreviewBitmap;
public VideoFrame? LatestFrame;
public bool FramePending;
}
private sealed class CameraSession
{
public string DeviceId { get; }
@@ -36,6 +53,10 @@ public sealed class CameraManager : IDisposable
public VideoFrame? LatestFrame;
public bool FramePending;
public TaskCompletionSource<bool>? FirstFrame;
public readonly Dictionary<WebcamEffectKey, EffectVariant> Effects = new();
public bool EffectWorkerRunning;
public VideoFrame? LastEffectSource;
public readonly MaskSmoother Smoother = new();
public CameraSession(string deviceId, ICameraFrameSource source)
{
@@ -50,11 +71,18 @@ public sealed class CameraManager : IDisposable
private readonly Dispatcher? _uiDispatcher;
private readonly TimeSpan _firstFrameTimeout;
private readonly Dictionary<string, CameraSession> _sessions = new();
private readonly Dictionary<string, HashSet<WebcamEffectKey>> _requestedEffects = new();
private readonly object _gate = new();
private readonly BackgroundEffectProcessor? _effects;
private readonly EffectTimer _effectTimer = new();
/// <summary>Raised on the UI thread when a camera's shared preview bitmap is first created.</summary>
public event Action<string, WriteableBitmap>? PreviewBitmapChanged;
/// <summary>Raised on the UI thread when an effect variant's preview bitmap is first created
/// (device id, effect key, bitmap) — the effect counterpart of <see cref="PreviewBitmapChanged"/>.</summary>
public event Action<string, WebcamEffectKey, WriteableBitmap>? EffectPreviewBitmapChanged;
/// <summary>
/// Raised when a capture fails: device in use, access denied, no preview source,
/// or a started reader that never delivers a first frame within the timeout.
@@ -62,12 +90,55 @@ public sealed class CameraManager : IDisposable
public event Action<string, string>? CameraFailed;
public CameraManager(ICameraEnumerator enumerator, Func<string, ICameraFrameSource> frameSourceFactory,
Dispatcher? uiDispatcher = null, TimeSpan? firstFrameTimeout = null)
Dispatcher? uiDispatcher = null, TimeSpan? firstFrameTimeout = null,
BackgroundEffectProcessor? effects = null)
{
_enumerator = enumerator;
_frameSourceFactory = frameSourceFactory;
_uiDispatcher = uiDispatcher;
_firstFrameTimeout = firstFrameTimeout ?? TimeSpan.FromSeconds(4);
_effects = effects;
}
/// <summary>
/// Declares which background-effect variants a device must render (the union over every
/// scene's webcam config). <see cref="BackgroundEffect.None"/> keys are ignored — that's the
/// raw path. Variants no longer requested are dropped (their worker cost stops); new ones
/// start rendering from the next camera frame. Survives the session starting later.
/// </summary>
public void SetRequestedEffects(string deviceId, IEnumerable<WebcamEffectKey> keys)
{
if (string.IsNullOrWhiteSpace(deviceId)) return;
lock (_gate)
{
var set = new HashSet<WebcamEffectKey>();
foreach (var key in keys)
if (key.Effect != BackgroundEffect.None) set.Add(key);
_requestedEffects[deviceId] = set;
if (_sessions.TryGetValue(deviceId, out var session))
SyncVariants(session, set);
}
}
/// <summary>The latest frame for a variant: the raw frame for <see cref="BackgroundEffect.None"/>,
/// otherwise the processed frame — null until the first one is rendered (an un-processed
/// frame is never shown in its place, so Remove never flashes the real room).</summary>
public VideoFrame? GetLatestFrame(string deviceId, WebcamEffectKey key)
{
if (key.Effect == BackgroundEffect.None) return GetLatestFrame(deviceId);
lock (_gate)
return _sessions.TryGetValue(deviceId, out var session)
&& session.Effects.TryGetValue(key, out var variant) ? variant.LatestFrame : null;
}
/// <summary>The shared preview bitmap for a variant (raw bitmap for None), or null until
/// its first frame — then <see cref="EffectPreviewBitmapChanged"/> fires.</summary>
public WriteableBitmap? GetPreviewBitmap(string deviceId, WebcamEffectKey key)
{
if (key.Effect == BackgroundEffect.None) return GetPreviewBitmap(deviceId);
lock (_gate)
return _sessions.TryGetValue(deviceId, out var session)
&& session.Effects.TryGetValue(key, out var variant) ? variant.PreviewBitmap : null;
}
public ICameraEnumerator Enumerator => _enumerator;
@@ -94,6 +165,8 @@ public sealed class CameraManager : IDisposable
session.Source.FrameAvailable += session.FrameHandler;
session.FailureHandler = message => OnSourceFailed(session, message);
session.Source.SourceFailed += session.FailureHandler;
if (_requestedEffects.TryGetValue(deviceId, out var requested))
SyncVariants(session, requested);
_sessions[deviceId] = session;
shouldStart = true;
}
@@ -277,6 +350,7 @@ public sealed class CameraManager : IDisposable
if (!TryGetActiveSession(session)) return;
session.LatestFrame = frame;
session.FirstFrame?.TrySetResult(true);
ScheduleEffects(session);
if (session.PreviewBitmap == null)
{
@@ -308,6 +382,142 @@ public sealed class CameraManager : IDisposable
}, DispatcherPriority.Render);
}
// Caller holds _gate. A new key inherits the last frame + bitmap of a dropped variant with
// the SAME effect (a strength change), so neither the stream nor the preview blanks for a
// frame while the creator drags the strength slider.
private static void SyncVariants(CameraSession session, HashSet<WebcamEffectKey> keys)
{
var dropped = session.Effects.Where(kv => !keys.Contains(kv.Key)).ToList();
foreach (var (stale, _) in dropped)
session.Effects.Remove(stale);
foreach (var key in keys)
{
if (session.Effects.ContainsKey(key)) continue;
var heir = dropped.FirstOrDefault(kv => kv.Key.Effect == key.Effect).Value;
session.Effects[key] = new EffectVariant
{
LatestFrame = heir?.LatestFrame,
PreviewBitmap = heir?.PreviewBitmap,
};
}
}
private void ScheduleEffects(CameraSession session)
{
if (_effects == null) return;
lock (_gate)
{
if (session.Effects.Count == 0 || session.EffectWorkerRunning) return;
session.EffectWorkerRunning = true;
}
_ = Task.Run(() => RunEffectWorker(session));
}
/// <summary>Renders every requested variant from the newest raw frame, then loops while a
/// newer frame has arrived; exits (and re-arms on the next frame) when caught up. One
/// segmentation per processed frame, whatever the number of variants.</summary>
private void RunEffectWorker(CameraSession session)
{
try
{
while (true)
{
VideoFrame? source;
List<WebcamEffectKey> keys;
lock (_gate)
{
source = session.LatestFrame;
if (!TryGetActiveSessionLocked(session) || session.Effects.Count == 0
|| source == null || ReferenceEquals(source, session.LastEffectSource))
{
session.EffectWorkerRunning = false;
return;
}
session.LastEffectSource = source;
keys = session.Effects.Keys.ToList();
}
var started = System.Diagnostics.Stopwatch.GetTimestamp();
var mask = session.Smoother.Smooth(_effects!.TrySegment(source));
var rendered = new List<(WebcamEffectKey Key, VideoFrame Frame)>(keys.Count);
if (mask != null)
foreach (var key in keys)
rendered.Add((key, BackgroundEffectProcessor.Render(source, mask, key.Effect, key.Strength)));
_effectTimer.Record(System.Diagnostics.Stopwatch.GetElapsedTime(started), source, keys.Count);
foreach (var (key, frame) in rendered)
{
EffectVariant? variant;
lock (_gate)
{
if (session.Effects.TryGetValue(key, out variant))
variant.LatestFrame = frame;
}
if (variant != null) PublishVariantPreview(session, key, variant);
}
}
}
catch (Exception ex)
{
lock (_gate) session.EffectWorkerRunning = false;
AppLog.Write($"CameraManager: background effects worker failed: {ex.GetType().Name}: {ex.Message}");
}
}
private void PublishVariantPreview(CameraSession session, WebcamEffectKey key, EffectVariant variant)
{
if (_uiDispatcher == null) return;
lock (_gate)
{
if (variant.FramePending) return;
variant.FramePending = true;
}
_uiDispatcher.BeginInvoke(() =>
{
VideoFrame? frame;
lock (_gate)
{
variant.FramePending = false;
if (!TryGetActiveSessionLocked(session) || !session.Effects.TryGetValue(key, out var current)
|| !ReferenceEquals(current, variant)) return;
frame = variant.LatestFrame;
}
if (frame == null) return;
var bitmap = variant.PreviewBitmap;
if (bitmap == null || bitmap.PixelWidth != frame.Width || bitmap.PixelHeight != frame.Height)
{
bitmap = new WriteableBitmap(frame.Width, frame.Height, 96, 96, PixelFormats.Bgra32, null);
bitmap.WritePixels(new Int32Rect(0, 0, frame.Width, frame.Height), frame.BgraPixels, frame.Stride, 0);
variant.PreviewBitmap = bitmap;
EffectPreviewBitmapChanged?.Invoke(session.DeviceId, key, bitmap);
return;
}
bitmap.WritePixels(new Int32Rect(0, 0, frame.Width, frame.Height), frame.BgraPixels, frame.Stride, 0);
}, DispatcherPriority.Render);
}
private bool TryGetActiveSessionLocked(CameraSession session)
=> _sessions.TryGetValue(session.DeviceId, out var current) && ReferenceEquals(current, session);
/// <summary>Perf record for the creator's-box check (TASK 51): logs the average
/// segment + render cost once per session after 120 processed frames.</summary>
private sealed class EffectTimer
{
private int _count;
private double _totalMs;
private bool _logged;
public void Record(TimeSpan elapsed, VideoFrame frame, int variants)
{
if (_logged) return;
_totalMs += elapsed.TotalMilliseconds;
if (++_count < 120) return;
_logged = true;
AppLog.Write($"CameraManager: background effects avg {_totalMs / _count:0.0} ms/frame " +
$"({frame.Width}x{frame.Height}, {variants} variant(s), {_count} frames)");
}
}
private void EnsurePreviewBitmap(CameraSession session)
{
if (session.PreviewBitmap != null || session.LatestFrame == null) return;
+5 -2
View File
@@ -16,7 +16,7 @@ public partial class LayoutStore : IDisposable
using (var cmd = _connection.CreateCommand())
{
cmd.CommandText = "SELECT Id, Name, IsHidden, IsChatScene, HasBackground, HasSocialBar FROM Scene ORDER BY SortOrder";
cmd.CommandText = "SELECT Id, Name, IsHidden, IsChatScene, HasBackground, HasSocialBar, ThumbnailAssetId FROM Scene ORDER BY SortOrder";
using var reader = cmd.ExecuteReader();
while (reader.Read())
{
@@ -28,6 +28,7 @@ public partial class LayoutStore : IDisposable
IsChatScene = reader.GetInt32(3) != 0,
HasBackground = reader.GetInt32(4) != 0,
HasSocialBar = reader.FieldCount > 5 && !reader.IsDBNull(5) && reader.GetInt32(5) != 0,
ThumbnailAssetId = reader.FieldCount > 6 && !reader.IsDBNull(6) ? reader.GetString(6) : null,
});
}
}
@@ -165,7 +166,7 @@ public partial class LayoutStore : IDisposable
cmd.CommandText = """
SELECT SceneId, WebcamId, IsVisible, X, Y, Width, Height, Opacity,
ClipShape, IsMirrored, BorderColor, BorderOpacity, BorderWidth, BorderAnimation,
RectWidth, RectHeight, SortOrder
RectWidth, RectHeight, SortOrder, BackgroundEffect, BackgroundStrength
FROM WebcamSceneConfig ORDER BY SortOrder
""";
using var reader = cmd.ExecuteReader();
@@ -190,6 +191,8 @@ public partial class LayoutStore : IDisposable
BorderAnimation = Enum.TryParse<BorderAnimation>(reader.GetString(13), out var anim) ? anim : BorderAnimation.None,
RectWidth = reader.IsDBNull(14) ? null : reader.GetDouble(14),
RectHeight = reader.IsDBNull(15) ? null : reader.GetDouble(15),
BackgroundEffect = Enum.TryParse<BackgroundEffect>(reader.GetString(17), out var effect) ? effect : BackgroundEffect.None,
BackgroundStrength = reader.GetDouble(18),
};
if (!configsByScene.TryGetValue(sceneId, out var list))
configsByScene[sceneId] = list = new List<(WebcamSceneConfig, int)>();
+39
View File
@@ -75,6 +75,8 @@ public partial class LayoutStore : IDisposable
BorderOpacity REAL NOT NULL DEFAULT 1,
BorderWidth INTEGER NOT NULL DEFAULT 0,
BorderAnimation TEXT NOT NULL DEFAULT 'None',
BackgroundEffect TEXT NOT NULL DEFAULT 'None',
BackgroundStrength REAL NOT NULL DEFAULT 0.5,
SortOrder INTEGER NOT NULL DEFAULT 0,
PRIMARY KEY (SceneId, WebcamId)
);
@@ -136,6 +138,7 @@ public partial class LayoutStore : IDisposable
MigrateWebcamConfigTable();
MigrateSceneTable();
MigrateSceneSocialBarColumn();
MigrateSceneThumbnailColumn();
MigrateSocialsTable();
MigrateSocialEntryTable();
if (GetUserVersion() < 3)
@@ -360,6 +363,21 @@ public partial class LayoutStore : IDisposable
cmd.CommandText = "ALTER TABLE WebcamSceneConfig ADD COLUMN RectHeight REAL;";
cmd.ExecuteNonQuery();
}
// TASK 51: per-webcam background effect + strength (additive, no version bump).
if (!columns.Contains("BackgroundEffect"))
{
using var cmd = _connection.CreateCommand();
cmd.CommandText = "ALTER TABLE WebcamSceneConfig ADD COLUMN BackgroundEffect TEXT NOT NULL DEFAULT 'None';";
cmd.ExecuteNonQuery();
}
if (!columns.Contains("BackgroundStrength"))
{
using var cmd = _connection.CreateCommand();
cmd.CommandText = "ALTER TABLE WebcamSceneConfig ADD COLUMN BackgroundStrength REAL NOT NULL DEFAULT 0.5;";
cmd.ExecuteNonQuery();
}
}
// v2 → v3: the webcam leaves the Source table. Any webcam Source rows become
@@ -510,6 +528,27 @@ public partial class LayoutStore : IDisposable
alter.ExecuteNonQuery();
}
// 2026-10-10: Scene gains ThumbnailAssetId (the top-strip tile artwork, stored
// as an Asset row like every image). Additive, no version bump — old DBs just
// get a NULL column and the strip falls back to the scene's background art.
private void MigrateSceneThumbnailColumn()
{
var columns = new HashSet<string>(StringComparer.OrdinalIgnoreCase);
using (var cmd = _connection.CreateCommand())
{
cmd.CommandText = "PRAGMA table_info(Scene);";
using var reader = cmd.ExecuteReader();
while (reader.Read())
columns.Add(reader.GetString(1));
}
if (columns.Contains("ThumbnailAssetId")) return;
using var alter = _connection.CreateCommand();
alter.CommandText = "ALTER TABLE Scene ADD COLUMN ThumbnailAssetId TEXT;";
alter.ExecuteNonQuery();
}
// v7 → v8: Socials gains BarEnabled (the dialog's show/hide switch). The old
// BarJustify column stays for back-compat but is no longer read or written.
private void MigrateSocialsTable()
+14 -4
View File
@@ -54,8 +54,8 @@ public partial class LayoutStore : IDisposable
using (var cmd = _connection.CreateCommand())
{
cmd.CommandText = """
INSERT INTO Scene (Id, Name, IsHidden, IsChatScene, HasBackground, HasSocialBar, SortOrder)
VALUES ($id, $name, $isHidden, $isChat, $hasBackground, $hasSocialBar, $sort)
INSERT INTO Scene (Id, Name, IsHidden, IsChatScene, HasBackground, HasSocialBar, ThumbnailAssetId, SortOrder)
VALUES ($id, $name, $isHidden, $isChat, $hasBackground, $hasSocialBar, $thumbnail, $sort)
""";
cmd.Transaction = tx;
var idP = cmd.Parameters.Add("$id", SqliteType.Text);
@@ -64,6 +64,7 @@ public partial class LayoutStore : IDisposable
var chatP = cmd.Parameters.Add("$isChat", SqliteType.Integer);
var backgroundP = cmd.Parameters.Add("$hasBackground", SqliteType.Integer);
var socialBarP = cmd.Parameters.Add("$hasSocialBar", SqliteType.Integer);
var thumbnailP = cmd.Parameters.Add("$thumbnail", SqliteType.Text);
var sortP = cmd.Parameters.Add("$sort", SqliteType.Integer);
var sort = 0;
@@ -75,6 +76,7 @@ public partial class LayoutStore : IDisposable
chatP.Value = scene.IsChatScene ? 1 : 0;
backgroundP.Value = scene.HasBackground ? 1 : 0;
socialBarP.Value = scene.HasSocialBar ? 1 : 0;
thumbnailP.Value = (object?)scene.ThumbnailAssetId ?? DBNull.Value;
sortP.Value = sort++;
cmd.ExecuteNonQuery();
}
@@ -209,10 +211,10 @@ public partial class LayoutStore : IDisposable
INSERT INTO WebcamSceneConfig
(SceneId, WebcamId, IsVisible, X, Y, Width, Height, Opacity,
ClipShape, IsMirrored, BorderColor, BorderOpacity, BorderWidth, BorderAnimation,
RectWidth, RectHeight, SortOrder)
RectWidth, RectHeight, SortOrder, BackgroundEffect, BackgroundStrength)
VALUES ($sceneId, $webcamId, $isVisible, $x, $y, $w, $h, $opacity,
$clip, $mirrored, $borderColor, $borderOpacity, $borderWidth, $borderAnimation,
$rectWidth, $rectHeight, $sort)
$rectWidth, $rectHeight, $sort, $bgEffect, $bgStrength)
""";
cmd.Transaction = tx;
var sceneIdP = cmd.Parameters.Add("$sceneId", SqliteType.Text);
@@ -232,6 +234,8 @@ public partial class LayoutStore : IDisposable
var rectWidthP = cmd.Parameters.Add("$rectWidth", SqliteType.Real);
var rectHeightP = cmd.Parameters.Add("$rectHeight", SqliteType.Real);
var sortP = cmd.Parameters.Add("$sort", SqliteType.Integer);
var bgEffectP = cmd.Parameters.Add("$bgEffect", SqliteType.Text);
var bgStrengthP = cmd.Parameters.Add("$bgStrength", SqliteType.Real);
foreach (var scene in scenes)
{
@@ -239,6 +243,8 @@ public partial class LayoutStore : IDisposable
foreach (var element in scene.Elements)
{
if (element is not WebcamSceneConfig config) { z++; continue; }
bgEffectP.Value = config.BackgroundEffect.ToString();
bgStrengthP.Value = config.BackgroundStrength;
sceneIdP.Value = scene.Id;
webcamIdP.Value = config.WebcamId;
visibleP.Value = config.IsVisible ? 1 : 0;
@@ -320,9 +326,13 @@ public partial class LayoutStore : IDisposable
DELETE FROM Asset WHERE Id NOT IN (
SELECT AssetId FROM Source WHERE AssetId IS NOT NULL
UNION
SELECT ThumbnailAssetId FROM Scene WHERE ThumbnailAssetId IS NOT NULL
UNION
SELECT AlertVideoAssetId FROM Source WHERE AlertVideoAssetId IS NOT NULL
UNION
SELECT Value FROM Settings WHERE Key = 'AlertDefaultVideoAssetId'
UNION
SELECT Value FROM Settings WHERE Key = 'OnAirImageAssetId'
);
""";
cmd.Transaction = tx;
+22
View File
@@ -326,4 +326,26 @@ public partial class LayoutStore : IDisposable
/// <summary>The stored default clip's Asset id, or null when never stamped.</summary>
public string? GetAlertDefaultVideoId() => GetSetting(AlertDefaultVideoKey);
internal const string OnAirImageKey = "OnAirImageAssetId";
/// <summary>The strip tile that labels the scene currently PREVIEWED (creator
/// ruling 2026-10-10 — the "on-air background screen"). One shared image, not
/// per-scene: whichever scene is staged shows it in the top bar, so it "jumps
/// around" as the creator swaps scenes. Lives as a BLOB in the internal Asset
/// table, tracked by this Setting so the Save pruner keeps it alive — the same
/// contract as the alert default clip. <c>UpsertAsset</c> is hash-keyed, so
/// re-stamping the same bytes returns the same id.</summary>
public string EnsureOnAirImage(byte[] data)
{
var saved = GetSetting(OnAirImageKey);
if (!string.IsNullOrWhiteSpace(saved) && GetAssetBytes(saved) != null)
return saved;
var id = UpsertAsset(data, 0, 0);
UpsertSetting(OnAirImageKey, id);
return id;
}
/// <summary>The stored on-air image's Asset id, or null when never stamped.</summary>
public string? GetOnAirImageId() => GetSetting(OnAirImageKey);
}
+60
View File
@@ -174,6 +174,66 @@ public class YouTubeStreamService
return $"YouTube rejected the end transition ({(int)response.StatusCode})";
}
/// <summary>The broadcast's current <c>status.privacyStatus</c> ("private", "unlisted",
/// "public"), read from YouTube rather than trusted from local state — the creator can
/// change visibility mid-stream from the broadcast form. Null when signed out or the
/// read fails (the caller then does nothing destructive).</summary>
public async Task<string?> GetBroadcastPrivacyAsync(string broadcastId)
{
if (!await EnsureToken()) return null;
_http.DefaultRequestHeaders.Authorization = new("Bearer", _auth.CurrentChannel!.AccessToken);
try
{
var response = await _http.GetAsync(
$"{ApiBase}/liveBroadcasts?part=status&id={Uri.EscapeDataString(broadcastId)}");
if (!response.IsSuccessStatusCode) return null;
var json = JsonSerializer.Deserialize<JsonElement>(await response.Content.ReadAsStringAsync());
if (!json.TryGetProperty("items", out var items) || items.GetArrayLength() == 0) return null;
if (!items[0].TryGetProperty("status", out var status)) return null;
return status.TryGetProperty("privacyStatus", out var privacy) ? privacy.GetString() : null;
}
catch (Exception ex)
{
AppLog.Write($"Broadcast privacy read failed: {ex.Message}");
return null;
}
}
/// <summary>Deletes a finished broadcast's recording from YouTube (the creator pressed
/// Escape / Delete on the keep-or-delete prompt). A live broadcast's id IS its video's
/// id, so this is <c>videos.delete</c> (Data API, 204 on success); if YouTube hasn't
/// created the video yet (404) it falls back to <c>liveBroadcasts.delete</c>. Returns
/// null on success, or an error string — never throws.
/// Docs: https://developers.google.com/youtube/v3/docs/videos/delete ,
/// https://developers.google.com/youtube/v3/live/docs/liveBroadcasts/delete</summary>
public async Task<string?> DeleteFinishedBroadcastAsync(string broadcastId)
{
if (!await EnsureToken()) return "not signed in";
_http.DefaultRequestHeaders.Authorization = new("Bearer", _auth.CurrentChannel!.AccessToken);
try
{
var id = Uri.EscapeDataString(broadcastId);
var video = await _http.DeleteAsync($"{ApiBase}/videos?id={id}");
if (video.IsSuccessStatusCode) return null;
if (video.StatusCode == System.Net.HttpStatusCode.NotFound)
{
var broadcast = await _http.DeleteAsync($"{ApiBase}/liveBroadcasts?id={id}");
if (broadcast.IsSuccessStatusCode) return null;
var bBody = await broadcast.Content.ReadAsStringAsync();
AppLog.Write($"liveBroadcasts.delete failed ({(int)broadcast.StatusCode}): {bBody}");
return $"YouTube refused to delete the broadcast ({(int)broadcast.StatusCode})";
}
var body = await video.Content.ReadAsStringAsync();
AppLog.Write($"videos.delete failed ({(int)video.StatusCode}): {body}");
return $"YouTube refused to delete the video ({(int)video.StatusCode})";
}
catch (Exception ex)
{
AppLog.Write($"Broadcast delete failed: {ex.Message}");
return ex.Message;
}
}
/// <summary>One liveBroadcasts.list (part=status) read of the broadcast's own
/// lifeCycleStatus — the only reliable "can we complete?" signal. Null when the
/// list fails or the status is missing (caller degrades gracefully).</summary>
+9 -3
View File
@@ -7,6 +7,7 @@ External-facing logic: YouTube API, persistence. See
|------|---------|
| `YouTubeAuthService.cs` | OAuth2 via Google: loopback callback (`http://localhost:8765/oauth2/callback`), token exchange, refresh, channel fetch. Constructor takes optional `HttpClient` + `sessionChanged` callback (test seam + save hook); session persists via `Helpers/TokenStore` (DPAPI); `ClearSession()` signs out (called by `MainViewModel.StopStream` on End Livestream) |
| `YouTubeStreamService.cs` | Broadcast/stream management via the v3 API. **`CreateBroadcast` always sends `privacyStatus = "private"`** (ship step 7 — private-only until v1) and the one-click v3 flags (`enableAutoStart/Stop`, `enableMonitorStream=false`, `latencyPreference=low`); injectable `HttpClient? http = null` ctor seam so tests can fake the API. **`GetOrCreateReusableStreamAsync` (TASK 9, shipped 2026-08-16)** returns the channel's `ReusableStream` — lists `liveStreams?mine=true` and reuses the existing `cdn.isReusable` stream, inserting once only on first use (`resolution=variable`, `frameRate=variable`, `isReusable=true`); a stream id given to `CreateBroadcast` binds at insert via `contentDetails.boundStreamId`. RTMP URL = `ingestionAddress + "/" + streamName`. **`GetStreamHealthAsync(streamId)` (TASK 9 item 3, shipped 2026-08-16)** polls `liveStreams?part=status` → `StreamHealth` with parsed `configurationIssues[]` (NOTE 2026-09-22: the real API nests `status.healthStatus = {status, lastUpdateTimeSeconds, configurationIssues[]}` as an OBJECT — the parse reads the nested `.status` and tolerates the old flat-string shape; the object shape used to crash every poll) | **`EndBroadcastAsync` (2026-09-01; hardened 2026-09-22)** posts `liveBroadcasts/transition?broadcastStatus=complete&part=status` AFTER RTMP EOF with a pre-flight `liveBroadcasts.list` read of the broadcast's OWN `lifeCycleStatus`: only transitions from `live`/`testing`, skips already-complete broadcasts silently (2026-09-22: every blind end logged `invalidTransition` — autoStop races us to complete); `enableAutoStop` finishes every skip |
| `YouTubeStreamService.cs` (2026-10-09 additions) | `GetBroadcastPrivacyAsync(id)` (`liveBroadcasts.list part=status` → privacyStatus) and `DeleteFinishedBroadcastAsync(id)` (`videos.delete`, fallback `liveBroadcasts.delete` on 404; never throws) — the private-broadcast keep-or-delete close-out |
| `StreamHealthReporter.cs` | Pure report-by-exception decision (TASK 9 item 3): `BannerFor(issues)` → `HealthIssueReport(Text?, IsError)` — null text on good/ok/noData/info-only, first warning/error issue produces its type text, error beats warning for color |
| `YouTubeChatService.cs` | Polls `liveChat/messages`, raises `MessageReceived`; `IDisposable`. **TASK 43 rewrite (2026-09-24):** re-arms its one-shot poll on the server's `pollingIntervalMillis` (streamList semantics — reconnect on that cadence with `nextPageToken`, clamp 1000–6000ms, `maxResults=2000`); `ParsePage` (internal static seam) decodes **all six** event types into `ChatMessage.Kind`/fields (superChat / superSticker / newSponsor / membershipGifting / giftMembershipReceived / memberMilestoneChat + milestone months best-effort regex); optional `HttpClient` ctor seam endures; `InjectSimulatedMessage` (TASK 41) preserved |
| `ChatBoxRenderer.cs` | Chat overlay rasterizer (WPF FormattedText → BGRA `VideoFrame`), live rows + Super Chat/member styling |
@@ -14,8 +15,13 @@ External-facing logic: YouTube API, persistence. See
| `AlertRenderer.cs` | **The alert rasterizer (TASK 43, 2026-09-24):** renders one event's celebration animation into a BGRA `VideoFrame` — SIX DISTINCT animations (SuperChat slide-up/shine/count-up, SuperSticker scale-pop, NewMember drop-in/flash-band, MemberGift slide-left/chip-fan, GiftReceived confetti scale-in, MemberMilestone rise/growth-bar), enter/hold/exit phases, brand line on every card. **Still the fallback** when no alert clip is playable (TASK 47) |
| `AlertOverlayLayer.cs` | **The native alert-box component (TASK 43; video branch TASK 47):** event queue (cap 10) + 33ms dispatch ticker + `RenderFrame` cache-first live path + `UpdatePreview`; **`Advance(double)` is the deterministic test clock** (tests never start the timer); `ChatEventKind.None` rows (sub mentions) never enqueue; idle = transparent (null frame). Since TASK 47 a clip branch owns `IAlertClipDecoder` per play — ctor seams `(clipDecoderFactory, clipPathResolver, audioSink)`, fade in/out over `FadeDurationSeconds = 0.30` (straight-alpha copies + volume×fade audio), EOF freeze-frame → drain, and `AlertTickerFrame` composes the scrolling message strip |
| `AlertClipDecoder.cs` | **Per-play alert video decoder (TASK 47):** `IAlertClipDecoder` (Start/Stop/Dispose + `FrameAvailable`/`AudioReady`/`Completed`) + ffmpeg child emitting `rawvideo bgra` (`-vf scale=W:H`, read via `RawVideoFrameReader`) and `f32le -ar 48000 -ac 2` on a **carry-buffer** audio loop (float boundary straddles never drop samples), both real-time paced; bgra is opaque (layer owns alpha). One-shot lifetime — created per alert, disposed at drain; tests inject a fake |
| `LayoutStore.cs` | SQLite persistence (`Microsoft.Data.Sqlite`) at `%APPDATA%\ytLlive\ytLlive.db`; assets stored as BLOBs keyed by SHA-256 content hash; save/open layout files; schema `user_version` 10 (`Source.ClipShape`/`IsMirrored` via `ALTER TABLE` for pre-v2 DBs; v3 = singleton `Webcam` + per-scene `WebcamSceneConfig`, migrated idempotently **without backfill** — the stale `Source.DeviceId` column remains but is no longer read/written; v4 = `WebcamSceneConfig.RectWidth`/`RectHeight`, the pre-Round rect for the round-to-rect restore; v5 = `Source.IsBackdrop` + `Source.CaptureKey`, the live-capture backdrop; v6 = `Scene.HasBackdrop` — Live-only policy, one-time backfill turns Starting/BRB/Chat/Ending off + drops their backdrop sources; v9→v10 (TASK 39) = `YppSnapshot` history table; all later columns stay additive guarded ALTERs — TASK 47 adds the five alert `Source` columns (`AlertVideoAssetId`/`AlertVideoPath`/`AlertUseDefaultVideo`/`AlertShowTicker`/`AlertVideoVolume`) without a version bump; `MainViewModel.NormalizeBackgrounds` re-normalizes on every load — TASK 25: one locked "Background" per canonical screen (static art everywhere, DisplayCapture on Live), duplicates dropped, index 0, orphaned `BackgroundUseDefault_*`/`BackgroundPath_*` Settings keys purged at save). Settings key/value table holds app state: `MicSourceName`, `ReusableStream` (TASK 9 — the cached reusable stream's id/address/name so the pump has its RTMP URL at startup), `AlertDefaultVideoAssetId` (TASK 47 — the stamped built-in alert clip's asset row; save's prune exempts it via UNION) |
| `LayoutStore.cs` | SQLite persistence (`Microsoft.Data.Sqlite`) at `%APPDATA%\ytLlive\ytLlive.db`; assets stored as BLOBs keyed by SHA-256 content hash; save/open layout files; schema `user_version` 10 (`Source.ClipShape`/`IsMirrored` via `ALTER TABLE` for pre-v2 DBs; v3 = singleton `Webcam` + per-scene `WebcamSceneConfig`, migrated idempotently **without backfill** — the stale `Source.DeviceId` column remains but is no longer read/written; v4 = `WebcamSceneConfig.RectWidth`/`RectHeight`, the pre-Round rect for the round-to-rect restore; v5 = `Source.IsBackdrop` + `Source.CaptureKey`, the live-capture backdrop; v6 = `Scene.HasBackdrop` — Live-only policy, one-time backfill turns Starting/BRB/Chat/Ending off + drops their backdrop sources; v9→v10 (TASK 39) = `YppSnapshot` history table; all later columns stay additive guarded ALTERs — TASK 51 adds `WebcamSceneConfig.BackgroundEffect`/`BackgroundStrength`; TASK 47 adds the five alert `Source` columns (`AlertVideoAssetId`/`AlertVideoPath`/`AlertUseDefaultVideo`/`AlertShowTicker`/`AlertVideoVolume`) without a version bump; `MainViewModel.NormalizeBackgrounds` re-normalizes on every load — TASK 25: one locked "Background" per canonical screen (static art everywhere, DisplayCapture on Live), duplicates dropped, index 0, orphaned `BackgroundUseDefault_*`/`BackgroundPath_*` Settings keys purged at save). Settings key/value table holds app state: `MicSourceName`, `ReusableStream` (TASK 9 — the cached reusable stream's id/address/name so the pump has its RTMP URL at startup), `AlertDefaultVideoAssetId` (TASK 47 — the stamped built-in alert clip's asset row; save's prune exempts it via UNION) |
| `VideoFrame.cs` | Normalized CPU frame seam (`Width`/`Height`/tightly-packed BGRA `byte[]`) — the only pixel type the rest of the app knows about; future capture sources (screen, background-removed webcam) feed the same seam |
| `BackgroundEffects/IPersonSegmenter.cs` | **TASK 51 seam:** `Segment(VideoFrame) → SegmentationMask?` (`SegmentationMask(Width, Height, float[] Values)`, 0 = background, 1 = person); null = no mask → caller passes through. Tests inject fakes |
| `BackgroundEffects/OnnxPersonSegmenter.cs` | **TASK 51 default segmenter:** MediaPipe Selfie Segmenter (landscape, Apache-2.0) as an embedded ONNX resource, run by onnxruntime on the CPU (2 intra-op threads); frame stretched to 256×144 RGB/255 NHWC; lazy session; never throws (fail → log once, null forever) |
| `BackgroundEffects/WebcamEffectKey.cs` | **TASK 51:** `(Effect, Strength)` variant key, strength rounded to 0.01; `None` normalizes to strength 0 (the raw path) |
| `BackgroundEffects/BackgroundEffectProcessor.cs` | **TASK 51 pure pixel stage:** `Apply(frame, BackgroundEffect)` — None = same instance, Remove = copy with alpha = feathered mask × source alpha (`IsOpaque=false`), Blur = background blurred at ¼ res (3 box passes) and mask-blended, person untouched; strength 0..1; blur radius is **quadratic** via `BlurRadius(strength)` — 1→6 small px (default 0.5 = 2, was a linear 1→12 ≈ 48 full px; the "blur heavy" fix 2026-10-09); `TrySegment` once + `Render` per effect; no mask/exception → passthrough + one log |
| `BackgroundEffects/MaskSmoother.cs` | **TASK 51 jitter fix (2026-10-09):** temporally blends each raw mask with the previous smoothed one — `CurrentWeight` 0.75 × new + 0.25 × previous — so a per-frame segmentation verdict that wavers at an edge doesn't flutter the cut-out. Null input resets history; first frame/size change pass through; the returned mask never aliases the caller's array and the two buffers ping-pong (no per-frame LOH alloc: a 256×144 mask is 147 KB). State is **per camera session**, not in the segmenter (which is shared app-wide and contract-pinned to raw output). Weights from the OBS/Jitsi consensus — see `ai.md` "Webcam background effects" |
| `CameraDeviceInfo.cs` | `(Id, DisplayName)` for a physical capture device |
| `ICameraEnumerator.cs` | `GetCamerasAsync()` — seam so the picker/`CameraManager` never touch WinRT (tests inject fakes) |
| `ICameraFrameSource.cs` | `StartAsync`/`StopAsync`/`FrameAvailable(VideoFrame)` + **`SourceFailed(string)`** — seam for a running capture source (tests inject fakes) |
@@ -24,7 +30,7 @@ External-facing logic: YouTube API, persistence. See
| `IMicrophoneEnumerator.cs` | `GetMicrophonesAsync()` — seam so the mic picker never touches WinRT (tests inject fakes) |
| `WinRtMicrophoneEnumerator.cs` | WinRT mic enumeration via `DeviceInformation.FindAllAsync(DeviceClass.AudioCapture)` (no NAudio needed — capture libs stay deferred to the audio pipeline milestone) |
| `MediaCaptureFrameSource.cs` | CPU-first MediaCapture source (`MemoryPreference = Cpu`, BGRA8 via `CreateFrameReaderAsync`). **Validated, not trusted:** post-init checks (`VideoDeviceId` match, `FrameSources` non-empty, `VideoDeviceController.GetAvailableMediaStreamProperties` ≥ 1); `reader.StartAsync()` status read (throws on non-`Success`); `capture.Failed` + `CameraStreamStateChanged` subscribed → `SourceFailed` event. Fallback ladder: VideoPreview → VideoRecord retry. `SharingMode.Exclusive` + `DeviceLost` not in 19041 SDK projection; `CameraStreamState.Failed` compared by `(int)2` (projection omits member names) |
| `CameraManager.cs` | Webcam capture ownership: refcounted by `DeviceId`, one shared `WriteableBitmap` per camera, dispatcher-coalesced ~render-rate UI updates (latest-frame drop); `PreviewBitmapChanged`/`CameraFailed` events. **First-frame proof** (4s timeout, configurable via ctor `TimeSpan?`): `AcquireAsync` returns true only after a real frame arrives — a reader that starts but never delivers (locked, suspended, dead) fails and surfaces `CameraFailed` with suspect-app names (`CameraConflictProbe`) instead of a silent empty box. `SourceFailed` forwarded from the frame source as async death signal |
| `CameraManager.cs` | Webcam capture ownership: refcounted by `DeviceId`, one shared `WriteableBitmap` per camera, dispatcher-coalesced ~render-rate UI updates (latest-frame drop); `PreviewBitmapChanged`/`CameraFailed` events. **First-frame proof** (4s timeout, configurable via ctor `TimeSpan?`): `AcquireAsync` returns true only after a real frame arrives — a reader that starts but never delivers (locked, suspended, dead) fails and surfaces `CameraFailed` with suspect-app names (`CameraConflictProbe`) instead of a silent empty box. `SourceFailed` forwarded from the frame source as async death signal. **TASK 51:** the one background-effects processing point — `SetRequestedEffects(device, keys)` + per-variant `GetLatestFrame`/`GetPreviewBitmap(device, key)` + `EffectPreviewBitmapChanged`; a background worker segments once per raw frame (mask temporally smoothed per camera — `MaskSmoother` — see `ai.md` "Webcam background effects") and renders every variant (latest-frame-drop); raw path untouched |
| `IFullScreenDetector.cs` | Seam for the win32 full-screen detector: `int? GetForegroundFullScreenMonitorIndex()`, `int PrimaryMonitorIndex()`, `IReadOnlyList<DisplayInfo> GetDisplays()` |
| `Win32FullScreenDetector.cs` | `GetForegroundWindow` + `DwmGetWindowAttribute(DWMWA_EXTENDED_FRAME_BOUNDS)` + `MonitorFromWindow` + `GetMonitorInfo`; monitor covers all four edges → full-screen; own process excluded; monitor order = `EnumDisplayMonitors` order (static `GetMonitorHandle(int)` maps index→HMONITOR for the capture factory). `GetDisplays()` returns `DisplayInfo` (index/name/resolution/bounds/`IsPrimary`, friendly name via `EnumDisplayDevices`) for the in-app "Capture Display" picker; `PrimaryMonitorIndex()` is the auto-key fallback when no full-screen game is detected |
| `IScreenCaptureSource.cs` | `Key` + `StartAsync`/`StopAsync`/`FrameAvailable(VideoFrame)` — seam so `ScreenCaptureManager` never touches WinRT (tests inject fakes) |
@@ -60,7 +66,7 @@ External-facing logic: YouTube API, persistence. See
| `Audio/WasapiLoopbackAudioSource.cs` | Desktop/game capture: NAudio `WasapiLoopbackCapture` on the default render device — automatic at unity; the game bar's meter consumes it |
| `Audio/WasapiMicAudioSource.cs` | Mic capture: NAudio `WasapiCapture` with the device resolved by `FriendlyName` matching `MicSourceName` (DisplayName only), default capture endpoint fallback; name provider `Func<string?>` re-read at each `Start` so a device picked mid-session takes effect immediately (the mixer restarts the mic on pick) |
| `Audio/AudioMixer.cs` | Owns both sources; capture starts once at startup (`MainViewModel.StartMicCaptureAsync`) and stops on `Shutdown` — NOT go-live (preview monitoring). Mic samples → `AudioLevelMeter` → `MicLevelChanged`; loopback samples → the game bar's meter via `LoopbackLevelChanged`. Surfaces mic connection state: `MicConnected`/`MicFailed` (drives the status dot) + `RestartMic()` (device swap mid-session, keeps loopback). Failures log via `AppLog`; mic failure zeroes the meter, loopback failure doesn't kill the mic. Note: the meter `Push` is unconditional (the `?.` on the event would otherwise skip the argument when nothing is subscribed). **TASK 47 — alert audio:** `EnqueueAlertAudio` writes a dedicated 8s stereo-48k ring (`AlertBufferSeconds`), drained in `FillAndMix` pre-limiter and added at **unity (never ducked)**; `StartLive` clears the ring |
| `Audio/AudioLevelMeter.cs` | Pure smoothed RMS level (0..1): `Push(AudioSample)` + `Reset` — the unit-tested math behind `AudioLevel` and `GameAudioLevel`. `ToDisplay(float)` maps the raw linear RMS onto the meter's display scale (−60..0 dBFS spread across 0..1, with **+10 dB input amplification** so real speech peaks hit the red zone at maxed volume) — real speech/game RMS (~0.01..0.1) would otherwise leave a flat scale looking dead; ≤0.001 linear reads as zero (never idles on background noise) |
| `Audio/AudioLevelMeter.cs` | Pure smoothed RMS level (0..1): `Push(AudioSample)` + `Reset` — the unit-tested math behind `AudioLevel` and `GameAudioLevel`. `ToDisplay(float)` maps the raw linear RMS onto the meter's display scale (−60..0 dBFS spread across 0..1 — **honest since 2026-10-09**: the old +10 dB display boost made a −30 dBFS voice look dialed in; leveled speech (−20 dBFS) now sits at 2/3 of the bar) — real speech/game RMS (~0.01..0.1) would otherwise leave a flat scale looking dead; ≤0.001 linear reads as zero (never idles on background noise) |
| `Audio/WaveToFloat.cs` | Pure WASAPI buffer → float conversion: IEEE float 32-bit direct, PCM 16-bit normalized, `WaveFormatExtensible` IEEE-float subformat GUID, trailing partial samples ignored |
| `Audio/MasterLimiter.cs` | **Master output limiter**: a **−1 dBFS ceiling** (`Ceiling = 0.891`) applied after the live mix so the honest gains can never sum past 0 dBFS and clip the AAC encode. Instant attack per frame (a hot frame is scaled exactly to the ceiling — no overshoot), smoothed release toward unity so loud passages don't pump. Pure — unit-tested |
+2 -1
View File
@@ -66,11 +66,12 @@
| 48 | Distribution & packaging: **MSIX + Store IAP** | 🔶 In progress — **✅ route decided 2026-09-27 (Store MSIX + Store IAP)**; **item 1 (bundle ffmpeg) SHIPPED 2026-09-28** | [`TASKS/task-48-distribution-msix.md`](TASKS/task-48-distribution-msix.md) |
| 49 | Chat profanity filter (local, opt-in, non-persistent, user word list) | ☐ Queued (2026-09-27) | [`TASKS/task-49-chat-profanity-filter.md`](TASKS/task-49-chat-profanity-filter.md) |
| 50 | One-button "Publish this video" (resumable upload of the local recording) | ☐ Queued (2026-09-28) | [`TASKS/task-50-publish-recording.md`](TASKS/task-50-publish-recording.md) |
| 51 | **Webcam background effects: Remove + Blur** | ☐ **v1 requirement** — booked 2026-10-08; next up now that the zoom box shipped (TASK 52) | [`TASKS/task-51-webcam-bg-effects.md`](TASKS/task-51-webcam-bg-effects.md) |
| 51 | **Webcam background effects: Remove + Blur** | ✅ **Built 2026-10-09** (v1 requirement) — bundled MediaPipe Selfie Segmenter → ONNX (verified vs Google's runtime), Remove + Blur with strength, per-scene config persisted, one processing point in CameraManager, radio-pill UI in the webcam's Layers properties. Creator check 2026-10-09: **Remove jitter FIXED** (temporal mask smoothing 0.75/0.25, OBS/Jitsi-derived) and **Blur "heavy" FIXED** (quadratic radius 1→6 small px, default 0.5 = 2, StreamFX-consensus); recheck on the real camera pending | [`TASKS/task-51-webcam-bg-effects.md`](TASKS/task-51-webcam-bg-effects.md) |
| 52 | **Zoom box (anchored 2× magnifier)** | ✅ Done — shipped 2026-10-08 (3 creator corrections during the build: loupe → anchored capture over the full composite, drawn last → never shows the dim shading) | [`TASKS/task-52-zoom-box.md`](TASKS/task-52-zoom-box.md) |
| 53 | **Output-format verification: Shorts/TikTok geometry (9:16)** | ☐ Queued (2026-10-08) — output formats never tested end-to-end; pin + prove the vertical geometry | [`TASKS/task-53-output-formats.md`](TASKS/task-53-output-formats.md) |
| 54 | **Marcom round: free-forever positioning + merch storefront** | 🔶 In progress — llamacasty.com copy live 2026-10-08 (pricing stripped); merch POD + app About links queued | [`TASKS/task-54-marcom-free-merch.md`](TASKS/task-54-marcom-free-merch.md) |
| 55 | **Disk hygiene: log rotation + low-disk guard** | ☐ Queued (2026-10-09) — `startup.log` appears uncapped; also temp alert-clip cleanup + free-space check before recording | [`TASKS/task-55-disk-hygiene.md`](TASKS/task-55-disk-hygiene.md) |
| 56 | **Pre-publication secret scan (full git history)** | ☐ Queued (2026-10-09) — gate before the Gitea repo goes public; report commit/path only, never values | [`TASKS/task-56-secret-scan.md`](TASKS/task-56-secret-scan.md) |
---
+4
View File
@@ -151,6 +151,10 @@ There is currently **no `.manifest` file in the repo at all**, so this is purely
- Declare **English only** (10.7 — otherwise the description must be localized into every
declared language)
- Block map SHA2-256, `StoreManifest`, under the 25 GB cap
- **x64 only** (64-bit requirement, creator ruling 2026-10-09 — `PlatformTarget=x64` already in
both csproj files): package architecture `x64`, publish with `-r win-x64` so only the win-x64
native folders ship (an RID-less build copies every RID's ONNX Runtime native, ~17 MB each).
Store listing system requirements: "64-bit Windows 10 or 11".
**Do NOT use `appContainer`** — see the invariant in item 6.
+75 -25
View File
@@ -4,7 +4,12 @@
> **Booked 2026-10-08 by creator ruling** — *"we're doing the bg removal and blur for v1 - book it, Danno"*.
> This **reverses** the 2026-10-05 "background removal: out of product" exclusion (TASK 3 item 19).
**Status:** ☐ Booked — **v1 requirement**. Queued behind the zoom build (current in-flight work unit).
**Status:** ✅ **Built 2026-10-09 — all five slices shipped** (engine, Blur, config + persistence,
camera hookup, UI). **Creator's real-camera check done 2026-10-09 and both findings addressed:**
🎯 **Remove "jittery" — FIXED** (temporal mask smoothing, `MaskSmoother` — 0.75×new + 0.25×previous
momentum, weights from the OBS/Jitsi consensus); 🎯 **Blur "heavy" — FIXED** (quadratic radius curve,
max 12→6 small px, default 0.5 = 2 — StreamFX-consensus scale). **Pending: creator's recheck of both
on a real camera.**
**Goal:** per-webcam **Background Remove** and **Background Blur** on the live webcam layer,
selected per element in its properties panel, composited into stream + recording like any
@@ -34,7 +39,8 @@ delivery shape changed:
same posture as the ffmpeg bundling in TASK 48 item 1.)
2. **No hardware requirement.** CPU path must work; DirectML is an optional accelerator.
3. **Per-element (Meld-style), not global.** Each webcam element carries its own effect.
4. **No feature gating.** Works for free users; only the branding flash is paid.
4. **No feature gating.** Everything is free (free-forever ruling 2026-10-08, TASK 54 — the old
"only the branding flash is paid" line here was superseded).
5. **Scene change / webcam add/remove must not crash or stall the pump** — a failing model
run skips to passthrough, never drops the layer.
@@ -44,17 +50,22 @@ delivery shape changed:
- ❌ **Super resolution / denoise / prettify passes**
- ❌ **Virtual camera output** — permanently out of product (no vcam)
## 1. ☐ Engine
## 1. Engine
- ☐ Bundle an ONNX person-segmentation model (MediaPipe Selfie-Segmentation class, ~2–5 MB)
into the app (csproj copy like `ffmpeg/`, or as an `Asset`). Verify license + THIRD-PARTY
notices (TASK 36 item 6 gate).
- ☐ `onnxruntime` via NuGet. CPU EP by default; optionally DirectML EP later. Failing EP
init → passthrough, log once, never throw from the pump.
- ☐ Blur = **separable in-app blur** (box/gaussian) over the masked background region. No
model needed for blur once the mask exists.
- ☐ Remove = **alpha 0 outside the person region.** The compositor already alpha-blends per
element (`SceneCompositor`, source-over) — no compositor change expected.
- ✅ **Model bundled (slice 1, 2026-10-09):** Google's official MediaPipe Selfie Segmenter
**landscape** (144×256, Apache-2.0 per the model card — license + changes recorded in
THIRD-PARTY-NOTICES item 12). Converted to ONNX by `tools/selfie-seg/convert_tflite_to_onnx.py`
(custom op `Convolution2DTransposeBias` → `ConvTranspose`); verified against MediaPipe's own
runtime by `tools/selfie-seg/verify_against_mediapipe.py` (max |diff| 5e-5). 430 KB,
`EmbeddedResource` in the exe. Rejected sources: PINTO #109 (ML Kit-extracted, unclear license),
obs-backgroundremoval code (GPL-3.0) — referenced for approach only (ONNX Runtime, CPU, 2 threads).
- ✅ `Microsoft.ML.OnnxRuntime` 1.31.0, CPU EP (`OnnxPersonSegmenter`). Failing load/run →
logs once, returns null, caller passes through. DirectML still optional/later.
- ✅ **Blur (slice 2, 2026-10-09):** background box-averaged to ¼ res, 3 separable box-blur passes
(≈ gaussian, radius by strength), mask-blended back; person pixels untouched (`BackgroundEffectProcessor.ApplyBlur`).
- ✅ Remove = alpha = mask outside the person (`BackgroundEffectProcessor`), `IsOpaque = false`.
The compositor already alpha-blends per element — no compositor change expected (confirm in
the hookup slice).
- ☐ Feed the **VideoFrame seam** (`Services/VideoFrame.cs` doc explicitly anticipates
*"every future capture source (screen, background-removed webcam) feeds the same seam"*):
the frame handed to BOTH the preview bitmap path and the compositor's
@@ -62,28 +73,67 @@ delivery shape changed:
## 2. ☐ Config surface + persistence
- ☐ `WebcamSceneConfig` gains `BackgroundEffect` (`None | Remove | Blur`) + `Strength`.
- ☐ Persist via the existing `WebcamSceneConfig` table path in `LayoutStore.Save.cs`/
- ✅ `WebcamSceneConfig` gains `BackgroundEffect` (`None | Remove | Blur`) + `BackgroundStrength` (slice 3).
- ✅ Persist via the existing `WebcamSceneConfig` table path in `LayoutStore.Save.cs`/
`Load.cs` (new columns + migration). Follow `LayoutStore.Migrations.cs` conventions.
- ☐ Properties panel: effect dropdown + strength slider, per webcam element.
- ✅ **Properties UI (creator spec 2026-10-09, slice 5):** in the webcam's properties in the **Layers
panel**, two **pill switches** — "Remove background" and "Blur background" — using the existing
`PillToggle` style (`Themes/Controls.xaml`, already used in `Controls/LeftPanel.xaml`). They act
as **radio buttons**: turning one on turns the other off; at most one is active; both off =
`None`. Bound to the element's `BackgroundEffect`, per webcam element. (Supersedes the old
"effect dropdown" idea.) **Plus a strength slider** (creator: "keep", 2026-10-09) — Blur
radius / Remove edge softness via `Strength`.
- ☐ Replace-background affordance: **none in v1** — the creator drops any layer behind the
webcam. Effect preview must not obscure the fact that layers below still show through.
## 3. ☐ Test seam (Good Dog — one integration test per unit)
- ☐ Model-runner interface seam (like `CameraEnumeratorOverride`) so tests inject a fake
segmenter — no real ONNX inference in the test suite.
- ☐ Assert the **processed frame reaches both consumers** (preview bitmap + compositor frame).
- ☐ `None` = byte-identical passthrough of the raw frame.
- ☐ `Remove` = alpha 0 outside person rect; `Blur` = background region blurred, person sharp.
- ☐ Config → persistence roundtrip (per webcam element).
- ☐ Engine failure → passthrough + single log, layer never dropped.
- ✅ Model-runner interface seam (`IPersonSegmenter`) so tests inject a fake segmenter.
- ✅ The **processed frame reaches both consumers** (preview bitmap + compositor frame), raw stays raw (`CameraManagerTests`, slice 4).
- ✅ `None` = same-instance passthrough of the raw frame (`BackgroundEffectEngineTests`).
- ✅ `Remove` = alpha 0 outside the person, RGB untouched, non-opaque (`BackgroundEffectEngineTests`);
✅ `Blur` = background smoothed, person pixel-identical (`BackgroundEffectEngineTests`).
- ✅ **Bundled-model contract:** the embedded ONNX model through the C# preprocessing reproduces
MediaPipe's reference mask on a fixture (≥99.5% agreement). This is a fixed, deterministic
contract check of the shipped bytes — not a camera/lighting quality test.
- ✅ Config → persistence roundtrip per scene + legacy-DB migration (`LayoutStorePersistenceTests`).
- ☐ Engine failure → passthrough + single log, layer never dropped (code path exists; dedicated
test lands with the hookup slice).
- ☐ Real inference: **manual on the creator's box** (camera + lighting), not in CI.
- ✅ **Perf, synthetic 720p (Debug):** segment 1.6 ms, Remove 3.3 ms, Blur 17.5 ms (after making the
upsampling separable — naive was 46/80 ms). Runs on its own worker, off the raw path.
☐ **Real numbers on the creator's box:** `startup.log` → "background effects avg … ms/frame".
## 4. ☐ Sequencing
## 4. Sequencing
- ☐ **Separate work unit AFTER the zoom build.** This file is booked + scoped; do not
co-commit it with the zoom work.
- ✅ Separate work unit after the zoom build (zoom shipped 2026-10-08).
- Slice order: 1 engine ✅ → 2 Blur ✅ → 3 config + persistence ✅ → 4 camera-path hookup ✅ (one
processing point, both consumers) + perf check → 5 properties-panel UI ✅.
## 5. Creator's real-camera check (2026-10-09) — jitter FIXED, blur heavy deferred
- 🎯 **Finding: "removal is jittery"** (the edge flickers frame to frame). Root cause: the segmenter
classifies each frame independently — 256×144 mask, bilinear upsample — zero temporal coherence.
**Fix shipped: `MaskSmoother`** blends each raw mask with the previous smoothed one
(`CurrentWeight` = 0.75 × new + 0.25 × previous), wired at the ONE production point
(`CameraManager.RunEffectWorker`, per `CameraSession`). Why per-session and not in the segmenter:
the `OnnxPersonSegmenter` is shared app-wide (`MainViewModel:284-285`, one processor), and the
bundled-model contract test pins its raw per-frame output. Why 0.75/0.25: OBS's
obs-backgroundremoval smoke-tests `temporal_smooth_factor` at 0.85 default; Jitsi's webcam
background filter (jitsi/jitsi-meet#17080) found 0.7/0.3 best (0.5/0.5 smears on motion). Null mask
resets history; first frame passes through; no per-frame allocation (two ping-pong buffers — a
256×144 mask is 147 KB > the 85 KB LOH). Integration test: `CameraManagerTests.Temporal_Smoothing_…`
(two consecutive masks disagree at an edge → second frame keeps partial alpha where raw would cut).
- 🎯 **Finding: "blur is heavy"** — the OLD curve was linear 1→12 downscaled px (≈ 26–50 full-res px),
far past any moderate background blur. **Fix shipped: quadratic radius** —
`BackgroundEffectProcessor.BlurRadius(strength)` = 1 + 5·s²: default 0.5 = 2 small px (≈ a third of
the old default's spread), max = 6 small px (≈ 24 full px). Why these numbers: StreamFX defaults its
Box blur Size to 5 and its author stops downscale-based blurs around 8 px (cited). Integration test
`BackgroundEffectEngineTests.Blur_default_strength_is_gentle_and_full_strength_is_strong` pins the
mapping (BlurRadius(0)=1, 0.5→2, 1→6) and the step-edge behaviour: default smears light ~9px into
the dark half vs ~26px at full strength. The existing slice-2 blur test (person sharp + checkerboard
smoothed) passes unchanged.
- ✅ `startup.log` perf line present on the real box ("background effects avg … ms/frame").
## Related (not duplicated here)
+23
View File
@@ -0,0 +1,23 @@
# TASK 56 — Pre-publication secret scan of the full git history (queued 2026-10-09)
> Catalog: [`TASKS.md`](../TASKS.md) — status and requirements live here.
**Why:** the creator is switching the Gitea repo (`https://llgit.llamachile.tube/gramps/LlamaCasty`)
to **public** (GPL-3.0-or-later release). The working tree is clean — `CREDENTIALS.md`,
`MARCOM.md`, and `Helpers/OAuthCredentials.cs` are gitignored and were never committed — but every
past commit becomes readable too. Anything ever committed by mistake would be exposed.
**Gate:** do this BEFORE the repo is toggled public.
**Slices:**
1. ☐ **Scan every commit, every branch** for secret-shaped content: Google OAuth client secrets
(`GOCSPX-`), API keys / tokens (Polar OAT, `ghp_`, `AKIA`, private-key headers), passwords,
stream keys, and any once-tracked `CREDENTIALS.md` / `OAuthCredentials.cs`.
⛔ **Report commit + path + rule only — never print a matched value** (see `MyMistakes.md` →
"NEVER PRINT CREDENTIALS.md"). A tool like gitleaks with `--redact`, or `git log -p` piped to a
counter, never to the screen.
2. ☐ **If anything is found:** rotate that secret first (it's burned regardless), then decide with
the creator: rewrite history (`git filter-repo`) vs. accept the rotated value staying in history.
3. ☐ Record the result (clean / what was rotated) here and in `HANDOFF.md`, then the creator flips
the repo public.
+48 -11
View File
@@ -1,10 +1,24 @@
ytLlive — Third-Party Notices
================================
LlamaCasty — License and Third-Party Notices
=============================================
ytLlive is a paid, closed-source product. This file lists every third-party
component the product distributes, its license, and where to get
its source, so the LGPL/BSD/MIT obligations are met. Distribution obligations
are NOT optional: they attach because this product ships these components.
LlamaCasty
Copyright (C) 2026 Gramps — https://llamacasty.com
License: GNU General Public License v3.0 or later (GPL-3.0-or-later), with
additional terms under GPL-3.0 section 7 (attribution, marking of
modified versions, no trademark rights). See LICENSE and
ADDITIONAL-TERMS.md, shipped beside the application.
Source: https://llgit.llamachile.tube/gramps/LlamaCasty (complete
corresponding source, GPL-3.0 section 6)
LlamaCasty is free software: you can redistribute it and/or modify it
under those terms. It comes with ABSOLUTELY NO WARRANTY.
Brand artwork (the LlamaCasty name, icon, and logo files) is NOT covered
by the GPL — all rights reserved; see ADDITIONAL-TERMS.md.
The rest of this file lists every third-party component LlamaCasty
distributes, its license, and where to get its source, so the
LGPL/BSD/MIT/Apache obligations are met. Distribution obligations are NOT
optional: they attach because LlamaCasty ships these components. Every
license below is compatible with GPL-3.0.
Note on FFmpeg (items 1-2): it used to be DOWNLOADED at first run. As of
TASK 48 item 1 (2026-09-28) it is BUNDLED — the exact pinned build is fetched
@@ -30,8 +44,11 @@ there on purpose.
single `ffmpeg\` folder beside the application executable. They are the
lgpl-SHARED build, so the DLLs must stay siblings of the exe — Windows
resolves them from the executable's own directory.
Why LGPL (not GPL): a GPL build (e.g. gyan.dev, or BtbN's "gpl" variant)
would contaminate this proprietary product. Do NOT use one.
Why LGPL-shared: this is the pinned, known-good build (no libx264). Since
LlamaCasty became GPL-3.0 (2026-10-09) a GPL FFmpeg build would also
be license-compatible, but the pin stays: changing builds means
re-verifying the encoders, and the shared build keeps compliance to
this notice plus the source offer below.
Why "shared" (not static): LGPL v2.1 §6 requires "relinkable" materials for
statically-linked libraries. The shared build links dynamically, so
the user can replace the DLLs — compliance is this notice plus the
@@ -64,8 +81,8 @@ there on purpose.
openh264 binary to ship). So item 1's LGPL obligations apply to the FFmpeg
code and Cisco's BSD terms plus its patent grant apply to the OpenH264
code inside it. This is the product's software H.264 fallback when no
hardware encoder (NVENC/QSV/AMF) is available — it is deliberately never
libx264, which would be GPL.
hardware encoder (NVENC/QSV/AMF) is available — it is deliberately not
libx264, which carries no H.264 patent grant (Cisco's covers OpenH264).
4. SQLite (bundled via SQLitePCLRaw's e_sqlite3 native bundle)
License: public domain (no rights reserved)
@@ -115,7 +132,27 @@ there on purpose.
Transitive dependencies (Microsoft.Xaml.Behaviors.Wpf,
Microsoft.Extensions.DependencyInjection.Abstractions) are MIT,
covered by item 7's .NET notice policy; this notice is kept per
this file's "list everything" policy.
this file's "list everything" policy.
11. ONNX Runtime (Microsoft.ML.OnnxRuntime — runs the webcam background model, TASK 51)
Copyright (c) Microsoft Corporation
License: MIT
Home: https://github.com/microsoft/onnxruntime
Used as: the CPU inference runtime (managed package + native onnxruntime.dll)
behind the `IPersonSegmenter` seam. MIT imposes no source offer.
12. MediaPipe Selfie Segmenter (landscape) (webcam background Remove/Blur, TASK 51)
Copyright (c) Google LLC (authors: Tingbo Hou, Siargey Pisarchyk, Karthik
Raveendran — model card "MediaPipe Selfie Segmentation", 2021-05-06)
License: Apache License 2.0
Home: https://ai.google.dev/edge/mediapipe/solutions/vision/image_segmenter
Source: https://storage.googleapis.com/mediapipe-models/image_segmenter/selfie_segmenter_landscape/float16/latest/selfie_segmenter_landscape.tflite
Changes (Apache-2.0 §4b): converted from TensorFlow Lite to ONNX format by
tools/selfie-seg/convert_tflite_to_onnx.py — float16 weights widened
to float32, NHWC graph re-expressed in NCHW; the weights and the
network are otherwise unmodified. Embedded in the application as
selfie_segmenter_landscape.onnx. Runs entirely on the user's PC.
--------------------------------------------------------------------------------
+1 -1
View File
@@ -62,7 +62,7 @@ public partial class MainViewModel
/// level flashes on the bar) so the creator sees where they're setting it;
/// otherwise the realtime live level scaled by volume (raising the volume
/// moves ambient noise up the bar). Read-only meter.</summary>
private double MeterLevel => MicMuted ? 0 : _volumeAdjusting || _volumeFlash ? MicVolume : Math.Min(1, AudioLevelMeter.ToDisplay((float)AudioLevel) * 1.2 * MicVolume);
private double MeterLevel => MicMuted ? 0 : _volumeAdjusting || _volumeFlash ? MicVolume : AudioLevelMeter.ToDisplay((float)(AudioLevel * MicVolume));
public double MeterFillWidth => MeterLevel * 288;
+110 -7
View File
@@ -24,6 +24,12 @@ public partial class MainViewModel
/// swaps in a fake enumerator to drive the "window dies -> re-heal" flow.</summary>
internal static IWindowEnumerator? WindowEnumeratorOverride { get; set; }
/// <summary>Test seam for the launch focus decision: a fake detector simulates
/// a full-screen game owning the foreground without touching any real window,
/// so the <c>ShowActivated = !FullscreenGameRunningNow</c> rule (desktop-"lock"
/// fix 2026-10-10) is pinned without Win32 state.</summary>
internal static IFullScreenDetector? FullScreenDetectorOverride { get; set; }
private ImageSource? _activeBackgroundImage;
private ImageSource? _backgroundImage;
private bool _backgroundSizeMismatch;
@@ -138,19 +144,34 @@ public partial class MainViewModel
{
foreach (var scene in Scenes)
{
// Minis never render live captures. While the Live screen is staged
// (being previewed) its mini shows the green placeholder; unstaged,
// it shows the stored default art. Real-time surface = center only.
var stagedLive = ReferenceEquals(scene, StagedScene)
&& SceneCatalog.Is(scene.Name, SceneCatalog.Live);
var img = stagedLive ? null : LoadBackgroundImage(scene);
// The scene currently shown in the preview labels its top-bar tile with
// the on-air image (creator ruling 2026-10-10) — it jumps around the
// strip as the creator swaps scenes. Every other tile keeps its own
// artwork (ThumbnailAssetId) or falls back to the background art; minis
// never render live captures: a staged Live without a stamped on-air art
// still falls back to the green placeholder. Real-time surface = center only.
var isStaged = ReferenceEquals(scene, StagedScene);
var img = isStaged && OnAirImage != null
? OnAirImage
: SceneThumbnailImage(scene);
if (img == null)
{
var stagedLive = isStaged
&& SceneCatalog.Is(scene.Name, SceneCatalog.Live);
img = stagedLive ? null : LoadBackgroundImage(scene);
}
if (img != null)
scene.Snapshot = img;
else if (scene.Snapshot == null || stagedLive)
else if (scene.Snapshot == null || (isStaged && OnAirImage == null))
scene.Snapshot = CreatePlaceholderSnapshot(scene.Name);
}
}
private static System.Windows.Media.Imaging.BitmapSource? SceneThumbnailImage(Scene scene)
=> string.IsNullOrWhiteSpace(scene.ThumbnailAssetId)
? null
: ImageCache.Get(scene.ThumbnailAssetId);
// Full-screen game on its monitor, else null (static bg fallback — no
// infinity mirror). Called at launch (from ReacquireScreenCaptures, before
@@ -168,6 +189,15 @@ public partial class MainViewModel
return null;
}
/// <summary>True when a full-screen game/app owns the foreground right now.
/// MainWindow reads this at launch to decide whether to steal focus: opening
/// LlamaCasty while the creator is playing an exclusive-fullscreen game would
/// minimize that game out of fullscreen — the "desktop is locked and I can't
/// get back into the game" bug (2026-10-10) — so the window opens without
/// activating instead.</summary>
public bool FullscreenGameRunningNow
=> _fullScreenDetector.GetForegroundFullScreenMonitorIndex() != null;
// After a layout load / file open: make sure every background-enabled scene
// has a background, give any background without a persisted key the auto-detected
// one, then acquire one capture per unique background key. ScreenCaptureManager
@@ -411,9 +441,82 @@ public partial class MainViewModel
if (background != null)
StampDefaultBackgroundAsset(background, scene.Name);
}
SeedSceneThumbnails();
SeedOnAirImage();
OnPropertyChanged(nameof(ShowSourcesEmptyHint));
}
// ─── Scene strip tile artwork (creator ruling 2026-10-10) ───
// Each canonical scene's tile in the top strip shows its own image, stored as
// an Asset row like any image. The five files are the creator's CRT-styled
// renders (each scene's original background seen through a CRT screen). They
// are seeded into the layout's Asset table once per scene; the preview pane
// keeps rendering each scene's ORIGINAL background — never these tiles.
internal static readonly Dictionary<string, string> SceneThumbnailSeedPaths = new(StringComparer.OrdinalIgnoreCase)
{
[SceneCatalog.Starting] = @"C:\Users\gramp\Downloads\Gemini_Generated_Image_uvq3i3uvq3i3uvq3.jpg",
[SceneCatalog.Live] = @"C:\Users\gramp\Downloads\Gemini_Generated_Image_y27sgqy27sgqy27s.jpg",
[SceneCatalog.Brb] = @"C:\Users\gramp\Downloads\Gemini_Generated_Image_lxcq0klxcq0klxcq.jpg",
[SceneCatalog.Chat] = @"C:\Users\gramp\Downloads\Gemini_Generated_Image_nqosdenqosdenqos.jpg",
[SceneCatalog.Ending] = @"C:\Users\gramp\Downloads\Gemini_Generated_Image_8je5n68je5n68je5.jpg",
};
private void SeedSceneThumbnails()
{
var imported = false;
foreach (var scene in Scenes)
{
if (!string.IsNullOrWhiteSpace(scene.ThumbnailAssetId)) continue;
if (!SceneThumbnailSeedPaths.TryGetValue(scene.Name, out var path)) continue;
if (!System.IO.File.Exists(path)) continue;
try
{
// UpsertAsset is SHA256-keyed, so a re-seed never duplicates rows
// and returns the same id. Only the Scene mapping is new state.
var assetId = AddAsset(System.IO.File.ReadAllBytes(path));
if (assetId == null) continue;
scene.ThumbnailAssetId = assetId;
imported = true;
}
catch { /* the strip keeps the scene's background-art fallback */ }
}
if (!imported) return;
RefreshSnapshotsAsync();
ScheduleSave();
}
// ─── On-air strip tile (creator ruling 2026-10-10) ───
// A single shared image ("on-air background screen") labels whichever scene is
// CURRENTLY PREVIEWED in the top bar — it replaces that scene's tile and jumps
// around as the creator swaps scenes. Stored as an Asset row like any image,
// tracked by the OnAirImageAssetId setting so the Save pruner keeps it alive.
internal static string OnAirSeedPath = @"C:\Users\gramp\Downloads\Gemini_Generated_Image_4zajyl4zajyl4zaj.jpg";
private System.Windows.Media.Imaging.BitmapImage? OnAirImage;
private void SeedOnAirImage()
{
var saved = _layoutStore.GetOnAirImageId();
if (!string.IsNullOrWhiteSpace(saved))
{
var cached = ImageCache.Get(saved);
if (cached != null)
{
OnAirImage = cached;
return;
}
}
if (string.IsNullOrWhiteSpace(OnAirSeedPath) || !System.IO.File.Exists(OnAirSeedPath)) return;
try
{
var id = _layoutStore.EnsureOnAirImage(System.IO.File.ReadAllBytes(OnAirSeedPath));
OnAirImage = ImageCache.Get(id);
RefreshSnapshotsAsync();
}
catch { /* the strip keeps per-scene tiles when the on-air art can't import */ }
}
/// <summary>Points a background row at its scene's shipped default art when it
/// carries none. A custom image (Browse…) always wins and is never overwritten.</summary>
private void StampDefaultBackgroundAsset(Source background, string sceneName)
+10
View File
@@ -22,6 +22,16 @@ public partial class MainViewModel : ViewModelBase
/// left panel can subscribe to its changes.</summary>
public ObservableCollection<ChatMessage> ChatMessages => _chatLayer.Messages;
private ChatMessage? _pinnedChatMessage;
public ChatMessage? PinnedChatMessage
{
get => _pinnedChatMessage;
set => SetProperty(ref _pinnedChatMessage, value);
}
public RelayCommand<ChatMessage> PinChatMessageCommand => new((m) => { PinnedChatMessage = m; });
public RelayCommand ClearPinnedChatMessageCommand => new(_ => { PinnedChatMessage = null; });
/// <summary>The Add → YouTube Chat menu item: enabled only while no chat layer exists
/// anywhere — one chat overlay per layout.</summary>
public bool CanAddYouTubeChat
+1 -1
View File
@@ -23,7 +23,7 @@ public partial class MainViewModel
if (!SetProperty(ref _isPremium, value)) return;
// The flash is composited into the broadcast, so the gate has to be pushed
// all the way into the renderer — not just the preview.
_brandFlash.Enabled = !value;
_brandFlash.Enabled = ShowBrandFlash && !value;
}
}
+26
View File
@@ -196,6 +196,32 @@ public partial class MainViewModel
public IEnumerable<string> MissingScenes
=> SceneCatalog.All.Where(canonical => Scenes.All(s => !SceneCatalog.Is(s.Name, canonical)));
/// <summary>
/// The five canonical scenes render in F-key order (Starting, Live, BRB, Chat,
/// Ending = F1-F5), whichever order the persisted SortOrder rows ended up in —
/// an older session can leave Chat before BRB (seen on the creator's DB).
/// Renamed (non-canonical) scenes keep their loaded order AFTER the canonical
/// five. Stable <see cref="Enumerable.OrderBy"/>, so one pass; the next
/// auto-save re-stamps each SortOrder to its corrected index.
/// </summary>
private void ReorderScenesToCanonical()
{
var target = Scenes.OrderBy(s => CanonicalIndex(s.Name)).ToList();
for (var i = 0; i < target.Count; i++)
{
var current = Scenes.IndexOf(target[i]);
if (current != i) Scenes.Move(current, i);
}
}
private static int CanonicalIndex(string? name)
{
if (name == null) return SceneCatalog.All.Length;
for (var i = 0; i < SceneCatalog.All.Length; i++)
if (SceneCatalog.Is(name, SceneCatalog.All[i])) return i;
return SceneCatalog.All.Length;
}
/// <summary>Only show the "+" button when one of the five canonical scenes is missing.</summary>
public bool ShowAddScene => MissingScenes.Any();
+1
View File
@@ -252,6 +252,7 @@ public partial class MainViewModel
if (element is WebcamSceneConfig)
{
SyncWebcamEffects(); // drop a variant nobody shows any more
OnPropertyChanged(nameof(CanAddWebcam));
OnPropertyChanged(nameof(CanShowWebcamInStagedScene));
OnPropertyChanged(nameof(WebcamAddToolTip));
@@ -360,9 +360,10 @@ public partial class MainViewModel : ViewModelBase
// finalizes the VOD immediately instead of leaving viewers on a frozen
// "stream offline" until enableAutoStop fires (~1 min). Log-only failure:
// the backstop still ends the broadcast; a stop never fails over cosmetics.
if (wasLive && _currentBroadcastId != null)
var endedBroadcastId = wasLive ? _currentBroadcastId : null;
if (endedBroadcastId != null)
{
var closeOutError = await _youtubeStream.EndBroadcastAsync(_currentBroadcastId);
var closeOutError = await _youtubeStream.EndBroadcastAsync(endedBroadcastId);
if (closeOutError != null)
AppLog.Write($"End close-out: {closeOutError} — enableAutoStop will finish the broadcast");
}
@@ -370,6 +371,11 @@ public partial class MainViewModel : ViewModelBase
_currentBroadcastId = null;
OnPropertyChanged(nameof(CurrentBroadcastId));
// Creator ruling 2026-10-09: a PRIVATE broadcast ends like a recording — Enter keeps
// it on YouTube, Escape deletes it.
if (endedBroadcastId != null)
await OfferToKeepPrivateBroadcastAsync(endedBroadcastId);
if (wasRecording) await FinalizeRecordingAsync();
// Explicit sign-out only (TASK 18): stopping the stream does NOT clear
@@ -519,4 +525,49 @@ public partial class MainViewModel : ViewModelBase
_recDotPulse = _recDotPulse > 0.5 ? 0.35 : 1.0;
OnPropertyChanged(nameof(RecDotOpacity));
}
/// <summary>What happened to a finished broadcast at the keep-or-delete prompt.</summary>
internal enum PrivateBroadcastOutcome { NotPrivate, Kept, Deleted, DeleteFailed }
/// <summary>Test seam: answers the keep-or-delete prompt (title → keep?) instead of showing
/// <see cref="KeepBroadcastDialog"/>. Mirrors the other static overrides.</summary>
internal static Func<string, bool>? KeepBroadcastPromptOverride { get; set; }
private async Task OfferToKeepPrivateBroadcastAsync(string broadcastId)
{
var title = BroadcastForm.Title;
var (outcome, error) = await ClosePrivateBroadcastAsync(_youtubeStream, broadcastId,
() => KeepBroadcastPromptOverride?.Invoke(title) ?? new KeepBroadcastDialog(title).ShowDialog() == true);
switch (outcome)
{
case PrivateBroadcastOutcome.Kept:
AppLog.Write($"Private broadcast kept: {broadcastId}");
_notifications.Info("Broadcast kept", "It's on YouTube as a private video.");
break;
case PrivateBroadcastOutcome.Deleted:
AppLog.Write($"Private broadcast deleted by creator: {broadcastId}");
_notifications.Info("Broadcast deleted", "The broadcast was removed from YouTube.");
break;
case PrivateBroadcastOutcome.DeleteFailed:
AppLog.Write($"Private broadcast delete failed: {broadcastId}: {error}");
_notifications.Error("Couldn't delete the broadcast",
$"{error}. It's still on YouTube as a private video — you can delete it in YouTube Studio.");
break;
}
}
/// <summary>The keep-or-delete decision for a finished broadcast. Asks ONLY when YouTube
/// itself reports the broadcast as private (visibility can change mid-stream, so local state
/// isn't trusted); an unreadable status asks nothing and deletes nothing. Keep = leave it;
/// otherwise delete the video from YouTube.</summary>
internal static async Task<(PrivateBroadcastOutcome Outcome, string? Error)> ClosePrivateBroadcastAsync(
YouTubeStreamService youtube, string broadcastId, Func<bool> askKeep)
{
var privacy = await youtube.GetBroadcastPrivacyAsync(broadcastId);
if (!string.Equals(privacy, "private", StringComparison.OrdinalIgnoreCase))
return (PrivateBroadcastOutcome.NotPrivate, null);
if (askKeep()) return (PrivateBroadcastOutcome.Kept, null);
var error = await youtube.DeleteFinishedBroadcastAsync(broadcastId);
return error == null ? (PrivateBroadcastOutcome.Deleted, null) : (PrivateBroadcastOutcome.DeleteFailed, error);
}
}
+60 -15
View File
@@ -10,9 +10,13 @@ namespace ytLive.ViewModels;
public partial class MainViewModel
{
// TRAX (TASK 9): free background music at a fixed quiet volume, playing to
// the default device so it rides the loopback into the stream. The dot is
// red (no track) / yellow (loaded, stopped) / green (playing).
private readonly MusicPlayer _musicPlayer = new();
// the default device so it rides the loopback into the stream. The button is a
// toggle; its dot is green (playing) / red (error — a terse message scrolls in the
// desktop-audio bar) / gray (idle or no track).
private readonly ITraxPlayer _musicPlayer = TraxPlayerOverride?.Invoke() ?? new MusicPlayer();
/// <summary>Test seam: a fake TRAX player (no audio device). Set before constructing.</summary>
internal static Func<ITraxPlayer>? TraxPlayerOverride { get; set; }
private Music? _music;
private bool _musicPlaying;
private bool _traxLoadInFlight;
@@ -46,13 +50,18 @@ public partial class MainViewModel
public string? TraxError
{
get => _traxError;
private set => SetProperty(ref _traxError, value);
private set
{
if (SetProperty(ref _traxError, value))
OnPropertyChanged(nameof(TraxDotBrush));
}
}
/// <summary>TRAX status dot: green = playing, gray = idle or no track.</summary>
/// <summary>TRAX status dot: green = playing, red = the last load/play failed (the
/// message scrolls in the desktop-audio bar), gray = idle or no track.</summary>
public SolidColorBrush TraxDotBrush => _musicPlayer.IsPlaying
? BarOnBrush
: TraxIdleBrush;
: TraxError != null ? BarOffBrush : TraxIdleBrush;
public string TraxToolTip => _musicPlayer.TrackName != null
? $"TRAX: {_musicPlayer.TrackName} — left-click {(IsMusicPlaying ? "pauses" : "plays")}, right-click chooses a track"
@@ -82,8 +91,15 @@ public partial class MainViewModel
if (dialog.ShowDialog() != true)
return;
_music = new Music { TrackPath = dialog.FileName, IsEnabled = true };
LoadTrax(play: true);
_ = SetTraxTrackAsync(dialog.FileName, play: true);
}
/// <summary>Points TRAX at a track and loads it (optionally playing). The picker's path;
/// internal so tests can load a track without the file dialog.</summary>
internal Task SetTraxTrackAsync(string path, bool play)
{
_music = new Music { TrackPath = path, IsEnabled = true };
return LoadTraxAsync(play);
}
/// <summary>Probes a path off the UI thread and gives up at <paramref name="budget"/>.
@@ -128,20 +144,20 @@ public partial class MainViewModel
UpdateTraxLocalGain();
music.IsEnabled = true;
TraxError = null;
if (play)
_musicPlayer.Play();
loaded = true;
if (play)
PlayTrax();
}
else
{
AppLog.Write($"TRAX: '{path}' unreachable within {TraxProbeBudget.TotalSeconds:0}s — not loading");
TraxError = $"Can't reach {Path.GetFileName(path)} — is the drive on?";
TraxError = "TRAX: can't reach the track";
}
}
catch (Exception ex)
{
AppLog.Write($"TRAX: failed to load '{music.TrackPath}': {ex.Message}");
TraxError = $"Failed to load: {Path.GetFileName(music.TrackPath)} — {ex.Message}";
TraxError = "TRAX: can't load the track";
}
finally
{
@@ -152,16 +168,45 @@ public partial class MainViewModel
}
}
private void PlayTrax() => _musicPlayer.Play();
// Both toggle halves MUST refresh the UI: the dot and IsMusicPlaying used to update only
// on load/playback-ended, so pressing TRAX started the music with the light still off —
// and because IsMusicPlaying stayed false, the next press tried to play again instead of
// pausing (2026-10-09 bug).
private void PlayTrax()
{
try
{
_musicPlayer.Play();
TraxError = _musicPlayer.IsPlaying ? null : "TRAX: no audio output";
if (TraxError != null) AppLog.Write("TRAX: Play() returned but playback did not start (no output device?)");
}
catch (Exception ex)
{
AppLog.Write($"TRAX: play failed: {ex.Message}");
TraxError = "TRAX: can't play the track";
}
RefreshTraxUi();
}
private void PauseTrax() => _musicPlayer.Pause();
private void PauseTrax()
{
try
{
_musicPlayer.Pause();
}
catch (Exception ex)
{
AppLog.Write($"TRAX: pause failed: {ex.Message}");
}
RefreshTraxUi();
}
private void OnTraxPlaybackEnded(Exception? ex)
{
if (ex != null)
{
AppLog.Write($"TRAX: playback error: {ex.Message}");
TraxError = $"Playback error: {ex.Message}";
TraxError = "TRAX: playback stopped";
}
RefreshTraxUi();
}
+61 -7
View File
@@ -7,6 +7,7 @@ using ytLive;
using ytLive.Helpers;
using ytLive.Models;
using ytLive.Services;
using ytLive.Services.BackgroundEffects;
namespace ytLive.ViewModels;
@@ -193,9 +194,8 @@ public partial class MainViewModel
// A session already running (same device, or one that produced a first
// frame) has its shared bitmap; freshly loaded configs must adopt it here,
// because PreviewBitmapChanged never re-fires for an existing bitmap.
if (_cameraManager.GetPreviewBitmap(newDevice) is { } running)
foreach (var config in Scenes.SelectMany(s => s.Elements.OfType<WebcamSceneConfig>()))
config.VideoImageSource = running;
// SyncWebcamEffects also declares each config's background-effect variant.
SyncWebcamEffects();
}
// CameraManager creates the shared WriteableBitmap on the UI thread at the
@@ -209,7 +209,59 @@ public partial class MainViewModel
RaiseWebcamGateChanged();
foreach (var scene in Scenes)
foreach (var config in scene.Elements.OfType<WebcamSceneConfig>())
config.VideoImageSource = bitmap;
if (EffectKeyFor(config).Effect == BackgroundEffect.None)
config.VideoImageSource = bitmap;
}
// TASK 51: an effect variant's first processed frame created its bitmap — every config
// showing that variant adopts it (later frames write into it in place).
private void OnCameraEffectPreviewBitmapChanged(string deviceId, WebcamEffectKey key, WriteableBitmap bitmap)
{
if (_webcam?.DeviceId != deviceId) return;
foreach (var scene in Scenes)
foreach (var config in scene.Elements.OfType<WebcamSceneConfig>())
if (EffectKeyFor(config) == key)
config.VideoImageSource = bitmap;
}
internal static WebcamEffectKey EffectKeyFor(WebcamSceneConfig config)
=> WebcamEffectKey.For(config.BackgroundEffect, config.BackgroundStrength);
private readonly HashSet<WebcamSceneConfig> _effectWatchedConfigs = new();
/// <summary>TASK 51: declares the union of every scene's webcam effect variant to the camera
/// (unused variants stop costing CPU) and points each config's preview at its variant's
/// bitmap. An effect variant with no processed frame yet shows nothing rather than the raw
/// room. Re-run whenever a config is added/removed/loaded or its effect changes.</summary>
internal void SyncWebcamEffects()
{
if (_webcam == null || string.IsNullOrWhiteSpace(_webcam.DeviceId)) return;
var configs = Scenes.SelectMany(s => s.Elements.OfType<WebcamSceneConfig>()).ToList();
_cameraManager.SetRequestedEffects(_webcam.DeviceId, configs.Select(EffectKeyFor));
foreach (var config in configs)
{
if (_effectWatchedConfigs.Add(config))
config.PropertyChanged += OnWebcamConfigEffectChanged;
var key = EffectKeyFor(config);
var bitmap = _cameraManager.GetPreviewBitmap(_webcam.DeviceId, key);
// A strength change within the same effect keeps showing the previous variant's
// last frame until the new one renders (no blank flicker while dragging the slider);
// switching INTO an effect shows nothing first, so Remove never flashes the room.
var sameEffect = _shownEffect.TryGetValue(config, out var shown) && shown.Effect == key.Effect;
_shownEffect[config] = key;
if (bitmap != null) config.VideoImageSource = bitmap;
else if (key.Effect != BackgroundEffect.None && !sameEffect) config.VideoImageSource = null;
}
}
private readonly Dictionary<WebcamSceneConfig, WebcamEffectKey> _shownEffect = new();
private void OnWebcamConfigEffectChanged(object? sender, System.ComponentModel.PropertyChangedEventArgs e)
{
if (e.PropertyName is not (nameof(WebcamSceneConfig.BackgroundEffect) or nameof(WebcamSceneConfig.BackgroundStrength)))
return;
SyncWebcamEffects();
ScheduleSave();
}
// The camera failed to start or died asynchronously (in use, offline, locked,
@@ -321,9 +373,9 @@ public partial class MainViewModel
// PreviewBitmapChanged fires only on the camera's first frame, so a config
// added while the session is already running must pick up the shared bitmap
// directly (it's written in place from then on).
if (_cameraManager.GetPreviewBitmap(_webcam.DeviceId) is { } running)
config.VideoImageSource = running;
// directly (it's written in place from then on) — SyncWebcamEffects does that,
// per the config's background-effect variant.
SyncWebcamEffects();
var started = await _cameraManager.AcquireAsync(_webcam.DeviceId);
if (!started)
@@ -346,6 +398,8 @@ public partial class MainViewModel
if (!string.IsNullOrWhiteSpace(oldDevice) && oldDevice != device.Id)
await _cameraManager.ReleaseAllAsync(oldDevice);
SyncWebcamEffects(); // the new device inherits the scenes' background effects
foreach (var unused in Scenes.SelectMany(s => s.Elements.OfType<WebcamSceneConfig>()))
{
var started = await _cameraManager.AcquireAsync(device.Id);
+26 -14
View File
@@ -51,11 +51,9 @@ public partial class MainViewModel : ViewModelBase
private WebView2Manager? _webView2Manager;
// Branding flash (monetization): a full-frame "made with LlamaCasty!" shown
// for ~1s every 300s on the live output, ~25% opacity. Paid unlock sets
// BrandFlashEnabled = false. See ai.md "Monetization".
// The free-tier branding credit. Composed into the OUTPUT by FramePump, not just
// previewed — see BrandFlashPresenter. Gated on IsPremium (Polar), never a toggle.
// Branding credit: "made with LlamaCasty!" composed into the OUTPUT by FramePump, not
// just previewed — see BrandFlashPresenter. OFF by default since 2026-10-09
// (ShowBrandFlash = false, free-forever: no branding on the stream); the code stays.
private readonly BrandFlashPresenter _brandFlash;
public SceneElement? SelectedElement
@@ -122,11 +120,17 @@ public partial class MainViewModel : ViewModelBase
public bool ShowPreviewPlaceholder => !ShowEmptySceneHint && ActiveBackgroundImage == null && BackgroundImage == null;
// Paid unlock flips this off (see ai.md "Monetization"). When disabled the
// presenter stops emitting, so any active credit dies on the next tick — and a
// credit already in flight is cut, not run out. Derived, never assignable: the
// whole point is that nobody can turn the advertising on a paying stream.
public bool BrandFlashEnabled => !IsPremium;
/// <summary>Master switch for the "made with LlamaCasty!" credit (creator ruling
/// 2026-10-09: turn it OFF, but keep the feature). <c>false</c> = the credit never
/// shows — not in the preview, the stream, or recordings. Read when the view model is
/// constructed (and on every licence change), so set it before a <c>MainWindow</c> is
/// created; the tests that exercise the credit opt in that way.</summary>
internal static bool ShowBrandFlash { get; set; } = false;
// When disabled the presenter stops emitting, so any active credit dies on the next
// tick — and a credit already in flight is cut, not run out. Derived, never
// assignable from the UI: the master switch AND the licence must both allow it.
public bool BrandFlashEnabled => ShowBrandFlash && !IsPremium;
// Window geometry pass-through (code-behind owns the WPF specifics).
public (double Left, double Top, double Width, double Height, string State)? RestoreWindowState()
@@ -184,7 +188,7 @@ public partial class MainViewModel : ViewModelBase
// — go-live and local recording are the same FramePump, so one start covers
// output, recording and the preview pane alike. Enabled is seeded from the
// current licence state; ValidateLicenseAtStartupAsync re-pushes it later.
_brandFlash.Enabled = !IsPremium;
_brandFlash.Enabled = BrandFlashEnabled;
_brandFlash.Start();
_healthPollTimer = new DispatcherTimer { Interval = TimeSpan.FromSeconds(30) };
@@ -276,8 +280,11 @@ public partial class MainViewModel : ViewModelBase
_cameraManager = new CameraManager(
_cameraEnumerator,
CameraFrameSourceFactoryOverride ?? (id => new MediaCaptureFrameSource(id)),
System.Windows.Application.Current?.Dispatcher);
System.Windows.Application.Current?.Dispatcher,
effects: new Services.BackgroundEffects.BackgroundEffectProcessor(
PersonSegmenterOverride ?? new Services.BackgroundEffects.OnnxPersonSegmenter()));
_cameraManager.PreviewBitmapChanged += OnCameraPreviewBitmapChanged;
_cameraManager.EffectPreviewBitmapChanged += OnCameraEffectPreviewBitmapChanged;
_cameraManager.CameraFailed += OnCameraFailed;
_microphoneEnumerator = new WinRtMicrophoneEnumerator();
@@ -305,7 +312,7 @@ public partial class MainViewModel : ViewModelBase
_musicPlayer.PlaybackEnded += OnTraxPlaybackEnded;
_ = StartMicCaptureAsync();
_fullScreenDetector = new Win32FullScreenDetector();
_fullScreenDetector = FullScreenDetectorOverride ?? new Win32FullScreenDetector();
_fullScreenDetector.FullscreenMonitorChanged += OnFullscreenMonitorChanged;
foreach (var display in _fullScreenDetector.GetDisplays())
Displays.Add(display);
@@ -384,6 +391,10 @@ public partial class MainViewModel : ViewModelBase
internal static ICameraEnumerator? CameraEnumeratorOverride { get; set; }
internal static Func<string, ICameraFrameSource>? CameraFrameSourceFactoryOverride { get; set; }
/// <summary>Test seam (TASK 51): a fake person segmenter so hermetic tests never load the
/// real ONNX model. Mirrors <see cref="CameraFrameSourceFactoryOverride"/>.</summary>
internal static Services.BackgroundEffects.IPersonSegmenter? PersonSegmenterOverride { get; set; }
private void LoadLayout()
{
_isLoading = true;
@@ -400,6 +411,7 @@ public partial class MainViewModel : ViewModelBase
foreach (var name in SceneCatalog.All)
AddScene(name);
}
ReorderScenesToCanonical();
HealBackgrounds();
HealZoomLayers();
RefreshHotkeyLabels();
@@ -570,7 +582,7 @@ public partial class MainViewModel : ViewModelBase
// Looking up by the GUID returned null forever — the webcam was structurally
// invisible on the output while the preview (bitmap path) looked fine
// (caught by the resolver logging, 2026-09-01 take two).
WebcamSceneConfig webcam => _cameraManager.GetLatestFrame(DeviceKeyForWebcam(webcam.WebcamId)),
WebcamSceneConfig webcam => _cameraManager.GetLatestFrame(DeviceKeyForWebcam(webcam.WebcamId), EffectKeyFor(webcam)),
Source { IsLiveCapture: true, CaptureKey: not null } live => _screenCaptureManager.GetLatestFrame(live.CaptureKey),
Source { Type: SourceType.ChatBox } chatBox => RenderChatBox(chatBox),
Source { Type: SourceType.AlertBox } alertBox => RenderAlertBox(alertBox),
+164 -19
View File
@@ -239,7 +239,7 @@ C# / WPF (.NET 8) following MVVM:
|------|------|
| `Models/` | Plain data types — Scene, Source (incl. `ClipShape`, `IsMirrored`, `VideoImageSource`), QualityOption, StreamConfig, StreamHealth, YouTubeChannel, ChatMessage, **Socials (`SocialService` enum + `SocialEntry`/`SocialsConfig` + `SocialServiceIcons`) — the social bar** |
| `ViewModels/` | MainViewModel — `public partial class`, one file per functional area (Scenes, Background, Webcam, Audio, Trax, Socials, Streaming, Chat, Overlays, Account, License, Recording — split complete, see `ViewModels/index.md`); **Chat.cs is a thin delegating facade over `Services/ChatOverlayLayer.cs` (Commit G, first true decomposition), and now also over `Services/AlertOverlayLayer.cs` (TASK 43 — the native alert box, same true-component pattern)**; GoLiveViewModel, ReuseImageViewModel, CameraPickerViewModel, **SocialsDialogViewModel** |
| `Services/` | YouTube OAuth2, stream/broadcast management, live chat polling, LayoutStore (SQLite), **SocialValidator (`ISocialValidator` seam + `HttpSocialValidator` default)**, **webcam: `VideoFrame` seam + `CameraDeviceInfo`/`ICameraEnumerator`/`ICameraFrameSource` interfaces + `MediaCaptureCameraEnumerator`/`MediaCaptureFrameSource` (WinRT) + `CameraManager`**, **screen capture: `IFullScreenDetector`/`Win32FullScreenDetector` + `IScreenCaptureSource`/`ScreenCaptureFrameSource` (WinRT GraphicsCapture) + `ScreenCaptureManager` + `ScreenCaptureSourceFactory` + `Direct3D11Helper`/`CaptureInterop` (COM bridges)**, **media source: `IMediaFrameSource` + `MediaVideoSource` (spawns ffmpeg `rawvideo` BGRA decode) + `MediaVideoSourceManager` (refcount-by-path session owner) + pure `RawVideoFrameReader` + `IDecodeProcess`/`FfmpegDecodeProcess` process seam (binary-stdout mirror of `IEncoderProcess`; see "Media source")**, **compositor: `SceneCompositor` + `CompositorOptions` + pure `StretchMath` + `StaticPixelCache` (see "Scene compositor")**, **alerts: `ChatOverlayLayer` + `AlertOverlayLayer`/`AlertRenderer` (the native celebration-zone overlay, TASK 43 — see "Native events & alerts")**, **audio: `IAudioSource` seam + `WasapiLoopbackAudioSource`/`WasapiMicAudioSource` (NAudio WASAPI) + `AudioMixer` + pure `AudioLevelMeter`/`WaveToFloat`/`VoiceFilterChain`/`LowShelfFilter`/`HighShelfFilter`/`NoiseGate`/`Compressor`/`AutoDucker`/`AudioRingBuffer`/`TinyResampler`/`AudioSyncDelay` + `MusicPlayer` + `IAudioPipeWriter`/`NamedPipeAudioWriter` (see "Live audio capture")**, **encoder: `IFfmpegEncoder`/`FfmpegEncoder` + `IEncoderProcess`/`FfmpegEncoderProcess` + `IFfmpegLocator`/`FfmpegLocator` + pure `FfmpegArgs`/`FfmpegProgressParser`/`FfmpegEncoderPicker` + the `FramePump` frame producer (see "Live encoder" + "Live frame pipeline")**, **notifications: `INotificationService` seam (`AppNotificationSeverity` Info/Success/Warning/Error) + `NotificationService` (Notification.Wpf toasts, see "Toast notifications")** |
| `Services/` | YouTube OAuth2, stream/broadcast management, live chat polling, LayoutStore (SQLite), **SocialValidator (`ISocialValidator` seam + `HttpSocialValidator` default)**, **webcam: `VideoFrame` seam + `CameraDeviceInfo`/`ICameraEnumerator`/`ICameraFrameSource` interfaces + `MediaCaptureCameraEnumerator`/`MediaCaptureFrameSource` (WinRT) + `CameraManager`**, **screen capture: `IFullScreenDetector`/`Win32FullScreenDetector` + `IScreenCaptureSource`/`ScreenCaptureFrameSource` (WinRT GraphicsCapture) + `ScreenCaptureManager` + `ScreenCaptureSourceFactory` + `Direct3D11Helper`/`CaptureInterop` (COM bridges)**, **media source: `IMediaFrameSource` + `MediaVideoSource` (spawns ffmpeg `rawvideo` BGRA decode) + `MediaVideoSourceManager` (refcount-by-path session owner) + pure `RawVideoFrameReader` + `IDecodeProcess`/`FfmpegDecodeProcess` process seam (binary-stdout mirror of `IEncoderProcess`; see "Media source")**, **compositor: `SceneCompositor` + `CompositorOptions` + pure `StretchMath` + `StaticPixelCache` (see "Scene compositor")**, **alerts: `ChatOverlayLayer` + `AlertOverlayLayer`/`AlertRenderer` (the native celebration-zone overlay, TASK 43 — see "Native events & alerts")**, **audio: `IAudioSource` seam + `WasapiLoopbackAudioSource`/`WasapiMicAudioSource` (NAudio WASAPI) + `AudioMixer` + pure `AudioLevelMeter`/`WaveToFloat`/`VoiceFilterChain`/`LowShelfFilter`/`HighShelfFilter`/`NoiseGate`/`Compressor`/`AutoDucker`/`AudioRingBuffer`/`TinyResampler`/`AudioSyncDelay` + `MusicPlayer` + `ITraxPlayer` (TRAX toggle seam — FakePlayer in tests; the dot is green/red/gray and the error scrolls terse in the desktop-audio bar; MusicVolume fixed 0.5) + `IAudioPipeWriter`/`NamedPipeAudioWriter` (see "Live audio capture")**, **encoder: `IFfmpegEncoder`/`FfmpegEncoder` + `IEncoderProcess`/`FfmpegEncoderProcess` + `IFfmpegLocator`/`FfmpegLocator` + pure `FfmpegArgs`/`FfmpegProgressParser`/`FfmpegEncoderPicker` + the `FramePump` frame producer (see "Live encoder" + "Live frame pipeline")**, **notifications: `INotificationService` seam (`AppNotificationSeverity` Info/Success/Warning/Error) + `NotificationService` (Notification.Wpf toasts, see "Toast notifications")** |
| `Helpers/` | ViewModelBase (INotifyPropertyChanged), RelayCommand, ImageCache, AppLog (file logger), FocusPreservingListBox, OAuthCredentials, **TokenStore (DPAPI session persistence)**, **BuildStamp (wordmark release counter `#N`, +1 per commit, generated by `GenerateBuildStamp` in ytLive.csproj via git rev-list; the per-build GUID now logs to startup.log only — 2026-09-04)**, visibility converters |
| `Themes/` | `Controls.xaml` — the single dark-theme source, merged once in `App.xaml` (see `Themes/index.md`) |
| `MainWindow.xaml` | Dark theme; layout: top bar (controls), center (preview + live controls below), left (scenes/sources), right (chat), bottom (gear + stream stats + resolution) |
@@ -299,7 +299,7 @@ The app is a **director's control surface**, not an editor. This replaces the OB
**Thumbnails:** `Scene.Snapshot` property. `SceneCompositor.Render()` at 320×180 → `WriteableBitmap`. Only staged scene has live backdrop render; other 4 = static snapshots.
**Thumbnails:** `Scene.Snapshot` property. `RefreshSnapshotsAsync` fills minis from each scene's background art (`LoadBackgroundImage`); **minis never render live captures** — while the Live screen is staged its mini shows the green placeholder, unstaged it shows `live-background.jpg` (real-time surface = center monitor only; user decision 2026-08-23).
**Thumbnails:** `Scene.Snapshot` property. Fill = per-scene tile artwork first — each canonical scene's `ThumbnailAssetId` (an Asset-table image, the creator's CRT-styled renders seeded at heal into the internal DB), else background art; **minis never render live captures**. The scene currently staged shows the shared **on-air image** (`OnAirImageAssetId` setting → Asset row, seed path `MainViewModel.OnAirSeedPath`) in its tile and it follows the scene swap — "the on-air screen jumps around the top bar" (user ruling 2026-10-10). The Live screen's tile only shows the green placeholder when it is staged and no on-air art is stamped. The strip labels are FontSize 20 (user ruling 2026-10-10); border = `#e94560` when staged, `#22c55e` when live. The preview's own live signal is the outer `PreviewGlow` red border — NOT the 6px green `#22c55e` rectangle that previously sat in the BackdropLayer (removed 2026-10-10).
### Global hotkeys (TASK 20 — step 1 shipped 2026-08-24)
@@ -602,12 +602,100 @@ This replaces the old five-seeder cluster (`Seed{Starting,Brb,Ending,Chat}Backgr
- **GPU posture:** webcam frames are CPU (GPU-agnostic; WPF hardware-presents the preview anyway). Hardware
encoders (NVENC/AMF/QSV) matter for the encoder task, not capture. D3DImage GPU compositing is deferred
to the encoder task.
- **Background removal = PERMANENTLY OUT** (creator ruling 2026-10-05, confirmed: "no - not doing
it. the note is correct and accurate"). The line here used to read "milestone 2", which was stale —
it was never a plan, only a leftover from TASK 3's original scoping. The rationale that stands:
a vcam app (NVIDIA Broadcast, XSplit VCam) does it free, and an in-app model means a runtime, a
GPU/driver matrix, and per-scene lighting tuning — support hell for a solo dev
(`TASKS/task-36-gold-pass.md`, `TASKS/task-03-capture-pipeline.md` item 19). Don't reopen it.
- **Background removal — REVERSED, now IN for v1 (creator ruling 2026-10-08, TASK 51).** The
2026-10-05 "permanently out" ruling stood on "an in-app model means a runtime, a GPU/driver matrix,
and per-scene tuning". The delivery shape changed: a ~430 KB Apache-2.0 model embedded in the exe,
run on the CPU by onnxruntime — no download, no GPU requirement, no driver matrix. NVIDIA's Video
Effects SDK stays **ruled out** (RTX-only + ~1 GB user-side download). See "Webcam background
effects" below and `TASKS/task-51-webcam-bg-effects.md`.
### Webcam background effects (TASK 51 — built 2026-10-09; jitter fix 2026-10-09)
- **Model:** Google's **MediaPipe Selfie Segmenter, landscape** (Apache-2.0, model card 2021-05-06),
official file `storage.googleapis.com/mediapipe-models/image_segmenter/selfie_segmenter_landscape/
float16/latest/selfie_segmenter_landscape.tflite`. Converted to ONNX by our own
`tools/selfie-seg/convert_tflite_to_onnx.py` (the graph ends in MediaPipe's custom op
`Convolution2DTransposeBias`, which tf2onnx/tflite2onnx can't convert — 11 op types, so a direct
translation was smaller than any workaround). **Verified against Google's own runtime:**
`verify_against_mediapipe.py` feeds the same frame to MediaPipe's `ImageSegmenter` (original
.tflite) and onnxruntime (our .onnx): max |diff| 5e-5, 100% binary agreement on two images.
⛔ Do NOT swap in PINTO model-zoo #109 (it's ML Kit's model, extracted — not Apache provenance).
obs-backgroundremoval is GPL-3.0 — license-compatible since LlamaCasty went GPL-3.0 (2026-10-09),
but none of its code is used; if any is ever borrowed, it needs its copyright notice kept.
- **Contract:** input `[1,144,256,3]` RGB /255 NHWC → output `[1,144,256,1]` person probability. The
whole frame is stretched (not cropped) to 256×144, so the mask maps back by normalized coordinates.
- **Shipping:** `Assets/Models/selfie_segmenter_landscape.onnx` is an **EmbeddedResource** (in git,
inside the exe — nothing to lose next to it, nothing fetched at runtime). Runtime =
`Microsoft.ML.OnnxRuntime` 1.31.0 (MIT), CPU execution provider, 2 intra-op threads. Note the native
`onnxruntime.dll` is ~17 MB (win-x64); an RID-less build copies every RID's native folder — the MSIX
publish (TASK 48) must pin the RID so only win-x64 ships.
- **Code (`Services/BackgroundEffects/`):** `IPersonSegmenter` seam (`Segment(VideoFrame) →
SegmentationMask?`, tests inject fakes); `OnnxPersonSegmenter` (lazy session from the embedded
resource; **never throws** — a load/run failure logs once and returns null forever after);
`BackgroundEffectProcessor.Apply(frame, BackgroundEffect)` — `None` = same instance (true
passthrough), `Remove` = copy with alpha = bilinear-upscaled mask × source alpha, RGB untouched,
`IsOpaque = false` (the compositor's straight-alpha blend shows the layers below), strength =
edge feather (mask band 0.5±(0.05+0.45·s) → alpha); `Blur` (slice 2) = background downscaled 4×
by box average, 3 separable box-blur passes and bilinear-upsampled, blended by the mask — person
pixels (mask = 1) are copied
untouched, opacity preserved. **Blur radius is quadratic** (`BlurRadius(strength)`: 1→6 small px,
default 0.5 = 2 — the "blur heavy" fix 2026-10-09; the old LINEAR 1→12 small px ≈ 48 full px was
far beyond what the established tools use: StreamFX defaults its Box blur Size to 5 and its author
stops downscale-based blurs around 8 px). The mask is computed once per frame (`TrySegment`) and
rendered per effect (`Render`), so several variants share one inference. No mask / segmenter
exception → passthrough + one log line, never a dropped layer.
- **Config (slice 3):** per scene, on `WebcamSceneConfig` — `BackgroundEffect` + `BackgroundStrength`
(0..1, default 0.5), persisted as two additive guarded columns on `WebcamSceneConfig`
(`BackgroundEffect TEXT DEFAULT 'None'`, `BackgroundStrength REAL DEFAULT 0.5`, no version bump —
the TASK 47 convention). `RemoveBackground`/`BlurBackground` are the radio-pill bools the UI binds.
- **Hookup (slice 4) — the ONE processing point is `CameraManager`.** Effects are per scene but the
camera session is shared, so the manager renders **variants** keyed by `WebcamEffectKey(effect,
strength rounded to 0.01)`. `MainViewModel.SyncWebcamEffects()` declares the union of every
config's key (`SetRequestedEffects`; unused variants are dropped so their CPU stops) and points each
config's `VideoImageSource` at its variant's bitmap; it re-runs on load, add, remove, device swap,
and any `BackgroundEffect`/`BackgroundStrength` change (configs are watched via `PropertyChanged`,
which also `ScheduleSave`s). Per raw frame a background worker (latest-frame-drop, one at a time)
segments ONCE and renders every variant; each variant keeps its own latest frame (the resolver:
`GetLatestFrame(device, key)`) and WriteableBitmap (`EffectPreviewBitmapChanged` on first
creation). **The raw path is untouched** — a slow/failed model never stalls the plain webcam — and
an effect variant shows **nothing** until its first processed frame (Remove never flashes the
real room). The compositor needed no change: its sampler already honours source alpha.
- **Temporal smoothing (2026-10-09, creator check: "removal is jittery"):** the segmenter classifies
every frame independently, so a verdict that wavers at an edge makes the cut-out flutter. Fix:
`MaskSmoother` blends each raw mask with the previous smoothed one — **0.75 × new + 0.25 × previous**
(`CurrentWeight`, internal const). State is **per `CameraSession`** (one per-camera worker), NOT in
the segmenter — the `OnnxPersonSegmenter` is shared app-wide and its bundled-model contract test
pins the raw per-frame output. A null mask (model hiccup) resets history; the first frame and any
size change pass through un-blended; the returned mask never aliases the caller's array and two
buffers ping-pong (no per-frame allocation — a 256×144 mask is 147 KB, past the 85 KB LOH). Weights
come from the established tools, not a guess: OBS's obs-backgroundremoval smoke-tests
`temporal_smooth_factor` at its 0.85 default, Jitsi's webcam background filter (jitsi/jitsi-meet#17080)
found 0.7/0.3 transformed best — a midpoint that still reacts to real movement; 0.5/0.5 leaves a
smearing edge ghost. The blur "heavy" follow-up was fixed the same day: quadratic radius, 1→6
small px (see the `Blur` bullet above).
- **Perf (synthetic 1280×720, Debug build):** segment 1.6 ms, Remove 3.3 ms, Blur 17.5 ms per frame
(Release is several × faster). Pixel loops are row-parallel, capped at min(4, cores/2); the
bilinear upsampling is separable (horizontal rows precomputed per mask/small row, one vertical lerp
per pixel) — the first naive per-pixel 2-D bilinear cost 46/80 ms. `CameraManager` logs the real
average once per session after 120 processed frames (`startup.log`: "background effects avg …").
`MainViewModel.PersonSegmenterOverride` is the hermetic-test seam.
- **UI (slice 5, creator spec 2026-10-09):** the webcam's properties in the Layers panel
(`Controls/LeftPanel.xaml`) gain a **Background** section — two `PillToggle` switches, Remove and
Blur, bound to `RemoveBackground`/`BlurBackground` (radio-exclusive: one on turns the other off,
turning the active one off = None) and a Strength slider (0..1, 5% snaps, visible only while an
effect is on; Blur = amount, Remove = edge softness). Dragging the slider never blanks the webcam:
a new strength variant inherits the dropped same-effect variant's last frame + bitmap
(`CameraManager.SyncVariants`), and the VM keeps the shown bitmap until the new one renders.
- **Tests:** `BackgroundEffectEngineTests` — the Remove integration test (fake segmenter), and a
model contract test that runs the BUNDLED model through the C# preprocessing on a fixture and
matches the MediaPipe-produced reference mask (≥99.5% agreement; fixture = MediaPipe's public test
image `mozart_square.jpg`, public-domain painting, in `ytLive.Tests/Fixtures/`). `CameraManagerTests`
— the ONE integration test for the jitter fix (a `SteppingMaskSegmenter` whose two consecutive
masks disagree at an edge; asserts the second frame keeps a partial alpha where the raw mask would
have cut to 0: x=6 → 58 upstairs of the raw 0, person core 255, background 0).
`Blur_default_strength_is_gentle_and_full_strength_is_strong` pins the quadratic radius mapping
(`BlurRadius(0.5)` = 2, `BlurRadius(1)` = 6) and the behaviour at a step edge: default smears
light ~9px into the dark half vs ~26px at full strength.
### Frame cadence — how a dropped frame became measurable (UNIT 0, 2026-10-05)
@@ -916,6 +1004,15 @@ lights when a session is actually running.
survives the rename, and without it two instances started in the same minute compute the *same* start
path and both ffmpeg processes write one file — which is exactly the dev multi-instance case
(`Helpers/InstanceProfile`, one instance recording the other).
- **Private broadcasts end the same way (creator ruling 2026-10-09).** After `StopStream`'s
transition(complete), `OfferToKeepPrivateBroadcastAsync` reads the broadcast's privacy **from YouTube**
(`GetBroadcastPrivacyAsync` — visibility can change mid-stream, local state isn't trusted) and, only if
it is `private`, shows `KeepBroadcastDialog` (repo root): **Enter = Keep** (`IsDefault`), **Escape /
Delete / the X = delete** via `DeleteFinishedBroadcastAsync` — `videos.delete?id=<broadcastId>` (a live
broadcast's id IS its video id; 204), falling back to `liveBroadcasts.delete` on 404. An unreadable
status asks nothing and deletes nothing; a failed delete toasts that the video is still on YouTube.
Decision logic = `MainViewModel.ClosePrivateBroadcastAsync` (internal static, tested in
`PrivateBroadcastCloseOutTests`); `KeepBroadcastPromptOverride` is the dialog seam.
- **Save dialog: Cancel DISCARDS (creator ruling 2026-09-26).** The `RenameRecordingDialog` result is a
two-outcome decision, and it lives in the extracted `MainViewModel.CompleteRecordingSave(startPath, dir,
autoStem, chosenStem, creatorSaved)` (internal, so `ytLive.Tests` can drive it against real files without
@@ -972,11 +1069,21 @@ devices, no timers).
`Shutdown` disposes it — NOT go-live — so the meters preview live. Mic samples: downmix to mono →
`TinyResampler` → **`VoiceFilterChain` per sample** (bass `LowShelfFilter` 120 Hz +4 dB → treble
`HighShelfFilter` 8 kHz +3 dB → `NoiseGate` (threshold 0.005, hysteresis 0.5, attack 0.5, release
0.0005) → `Compressor` (threshold 0.5, ratio 4:1); all pure TDF2 DSP) → ring buffer → the
0.0005) → **`SpeechLeveler`** (2026-10-09: automatic gain steering speech RMS to −20 dBFS over a
0.3 s window, gain −6..+20 dB, rises ≤10 dB/s / falls ≤30 dB/s, learns only while the gate is open
and above −54 dBFS so pauses/hum are never pumped) → `Compressor` (threshold 0.5, ratio 4:1); all
pure DSP) → ring buffer → the
**post-filter** `AudioLevelMeter` (RMS, 0.2 exponential smoothing) → `MicLevelChanged` →
marshalled to the UI thread → `AudioLevel`. Loopback samples: resample → ring buffer → second meter
→ `LoopbackLevelChanged` → the game bar's `GameAudioLevel`. The raw linear level maps to the meter's
display scale by `AudioLevelMeter.ToDisplay` (−60..0 dBFS → 0..1 with **+10 dB input amplification**).
display scale by `AudioLevelMeter.ToDisplay` (−60..0 dBFS → 0..1, **honest** — the +10 dB display boost
was removed 2026-10-09; the mic meter is `ToDisplay(AudioLevel × MicVolume)`, true dB math, no ×1.2).
**Why (2026-10-09):** a creator's YouTube upload measured −28.5 LUFS / −8.9 dBFS peak against YouTube's
−14 LUFS reference (YouTube only turns loud videos down, never quiet ones up) while the boosted meter
looked "dialed in". Nothing in the chain added gain — the compressor folds only above −6 dBFS and the
limiter only caps. Re-running that same audio through the new chain (mic volume 0.8): **−17.9 LUFS,
−0.7 dBFS peak**. Recording and streaming share this one mix, so both were (and now are) the same.
Measure with the bundled ffmpeg: `ffmpeg -i file -af ebur128=peak=true:framelog=quiet -f null -`.
**Mic connection state surfaces as events:** `MicConnected` (source `Started`), `MicFailed` (source
`Failed`), and `RestartMic()` re-resolves + restarts just the mic (loopback keeps running). Failures
log via `AppLog`; a mic failure zeroes the meter, a loopback failure never kills the mic. Meter `Push`
@@ -1625,14 +1732,31 @@ broadcast insert, prep exception), frame-pump death while live, mic-missing at s
saved-session refresh failure (sign-out notice). Deliberately left log-only: offline license re-validation
skip (would spam every launch). Still modal by design: SocialsDialog's slot-delete YesNo confirm.
### Licensing — do not violate (GA = paid product; see `THIRD-PARTY-NOTICES.txt`)
### Licensing — do not violate (GPL-3.0 open source since 2026-10-09; see `LICENSE`, `ADDITIONAL-TERMS.md`, `THIRD-PARTY-NOTICES.txt`)
This product is closed-source and paid. Every third-party component must stay inside the LGPL/BSD/MIT
guardrails below — written down so a future "quick fix" never reintroduces a GPL binary. **NEVER:**
**LlamaCasty is free software under GPL-3.0-or-later** (creator rulings 2026-10-09 — free forever, public
repo, "GPL-3 with attribution"). Copyright (C) 2026 Gramps. `ADDITIONAL-TERMS.md` carries the three
§7 terms: **7(b)** every copy keeps "Based on LlamaCasty by Gramps — https://llamacasty.com" in its
About screen; **7(c)** modified versions are marked and renamed; **7(e)** no trademark rights. The
**icon + logo files** (`Assets/llamacasty-icon.png/.ico`, `Assets/llamacasty-logo.jpg`) are **all
rights reserved, outside the GPL** — verbatim copies may carry them, forks must replace them (this is
what keeps the merch brand protected). The scene backgrounds + default alert video are AI-generated
**placeholders** (creator: made "with the intent of individual creators to immediately replace with
their own images") — they go out under the GPL with the code, no reservation. `LICENSE` + `ADDITIONAL-TERMS.md` ship beside the exe
(csproj `None` items); the About panel shows `THIRD-PARTY-NOTICES.txt`, whose header carries the
LlamaCasty license notice (GPL §5(d) "Appropriate Legal Notices"). **Source offer (§6):**
`https://llgit.llamachile.tube/gramps/LlamaCasty` (the Gitea repo; creator toggles it public — the
old `gramps/ytLlive` path 301-redirects there), recorded in the notices header + README.
- **Use a GPL FFmpeg build** — gyan.dev's builds are GPLv3 and ship libx264; BtbN's `gpl` variant is
GPL too. GPL in a distributed paid product is the #1 lawsuit risk. Only BtbN `lgpl` / `lgpl-shared`
builds are allowed.
Every bundled component must stay GPL-3.0-compatible (MIT/BSD/Apache-2.0/LGPL/public domain all are;
**GPL-2.0-only and proprietary are not**; Apache-2.0 is fine with GPL-3 but NOT with GPL-2). The
rules below predate the GPL switch; where their *reason* was "proprietary product" it is noted, but
the rule itself still stands for its other reasons. **NEVER:**
- **Swap the FFmpeg pin casually** — the BtbN `lgpl-shared` pin is the verified build. A GPL FFmpeg
build would now be license-compatible (the old "GPL contaminates the paid product" reason is gone),
but changing builds means re-verifying encoders, and libx264 carries no H.264 patent grant (Cisco's
covers OpenH264). Keep `lgpl-shared` unless a task says otherwise.
- **Distribute the static lgpl build** — LGPLv2.1 §6 wants relinkable object files for static linking.
The **shared** (dynamic-DLL) build sidesteps that: compliance is "license text + source offer +
unmodified binaries". The pin is `lgpl-shared`; when the pin is refreshed, keep the shared variant.
@@ -1649,9 +1773,14 @@ guardrails below — written down so a future "quick fix" never reintroduces a G
a reproducible source offer. Record the tag + variant beside the URL (TASKS.md) every time the pin moves.
- **Use non-CC0 icon art** — the social bar's bundled SVG logo path data comes from **Simple Icons**
(CC0 1.0, public domain — see `THIRD-PARTY-NOTICES.txt`). Replacing or adding logos must stay CC0 or
another public-domain source; a logo asset under a copyleft or attribution license would contaminate
the paid product. The About hub's logo is the creator's own art ("llama fortnite superman logo",
`Assets/llama-logo.png`) — owned by us, so it carries no third-party license either.
another public-domain source; a logo asset under an incompatible or attribution-heavy license
would complicate every redistribution. The app's icon + logo (`Assets/llamacasty-icon.*`,
`Assets/llamacasty-logo.jpg`) are the creator's own art — all rights reserved, outside the GPL
(`ADDITIONAL-TERMS.md`).
- **Ship an ML model without clean provenance** — the background-effects model (TASK 51) is Google's
official MediaPipe Selfie Segmenter download (Apache-2.0, model card), converted by our own script.
Model-zoo re-hosts (e.g. PINTO #109 = ML Kit's model, extracted) are off-limits. Apache-2.0 §4
obligations (license notice + "we converted it") live in `THIRD-PARTY-NOTICES.txt`.
- **Forget the v1 license-texts gate** — `THIRD-PARTY-NOTICES.txt` links the canonical license texts; at
**v1 (GA)** the full texts of every license it names MUST ship alongside it (TASK 4 requirement 9 is the
release blocker). Queued early is wrong; the release pass owns it.
@@ -1679,6 +1808,15 @@ Apply this to every UI decision:
## Monetization (design decision — the branding flash is the sword)
> **⛔ Superseded 2026-10-08/09 — free forever, no branding on the stream.** The credit is now
> **OFF by default**: `MainViewModel.ShowBrandFlash` (internal static, default `false`) is the
> master switch, and `BrandFlashEnabled => ShowBrandFlash && !IsPremium` feeds the presenter at
> construction and on every licence change. The code (`BrandFlashPresenter`, the FramePump
> `brandFlash:` seam, the preview layer) is kept, not deleted — creator ruling: "don't delete it,
> just make it conditionally displayed". `BrandFlashOutputTests` opt in by setting the switch
> before constructing a `MainWindow`; `Credit_IsOffByDefault_EvenOnTheFreeTier` guards the default.
> Everything below is the historical paid-tier design.
Full pricing and the Year 1 revenue forecast in `MONETIZATION.md` (gitignored). ⚠️ **No Polar
product, discount code, or license key exists any more** — billing is Microsoft Store IAP.
Brand palette and assets in `MARCOM.md` (gitignored). This section covers the in-app model.
@@ -2205,6 +2343,13 @@ before re-deriving it. Executable checklist: `TASKS/task-48-distribution-msix.md
thinks I'm expired" bug class along with the offline-grace machine.
- **⛔ Bundle ffmpeg — no runtime downloads** (Store policy 10.2.2; also the 2026-09-01 404).
See the FFmpeg locator section above. **This is the first code unit.**
- **⛔ 64-bit Windows only (creator ruling 2026-10-09).** Both csproj files set
`<PlatformTarget>x64</PlatformTarget>` (the test project too — an AnyCPU test project
referencing an x64 app raises MSB3270 and breaks the 0-warning gate). Why: the bundled ffmpeg
is the win64 build, ONNX Runtime 1.31 ships no win-x86 native (only win-x64/win-arm64), and a
32-bit address space can't hold the 1080p frame buffers. Windows 11 is 64-bit-only anyway. The
MSIX publish targets x64 only (ARM64 is a possible later add: both ffmpeg and ONNX Runtime have
arm64 builds). Public copy says "64-bit Windows 10 or 11".
**What does NOT change: the monetization posture.** Free gets everything; the branding flash
stays the **only** paid delta (`TASK 36` item 2). Store IAP changes how `IsPremium` is
+40
View File
@@ -1,5 +1,45 @@
# Bugs
## Scene strip tile + on-air artwork assets deleted on the next Save (FIXED 2026-10-10)
The scene tiles and the shared on-air image are stored as Asset-table BLOBs (per-scene
`Scene.ThumbnailAssetId` + the `OnAirImageAssetId` Setting). `LayoutStore.Save()` runs an Asset
pruner that only keeps rows referenced by `Source`, the alert clip setting, and — after the fix —
the two new references: it `DELETE FROM Asset WHERE Id NOT IN (SELECT ThumbnailAssetId FROM Scene …,
SELECT Value FROM Settings WHERE Key = 'OnAirImageAssetId')`. Without that, the very first save after
seeding would have wiped the freshly-imported tile art, and every tile would have silently fallen
back to the scene's background until the next launch re-seeded it.
## Scene strip / F-keys out of sequence (FIXED 2026-10-10)
Report: "the ordering of the 5 screens and their corresponding F-key is out of sequence." The strip
renders `Scenes ORDER BY SortOrder` but each tab's F-key label is name-derived, so the two can drift
when a session persists a non-canonical order (on the creator's DB: Chat SortOrder 2, BRB 3 — tabs
showed "Chat F4" then "BRB F3").
Fix (same commit): `MainViewModel.LoadLayout` calls the new
`ReorderScenesToCanonical()` after loading — the five canonical scenes render in F-key order
(Starting, Live, BRB, Chat, Ending = F1-F5), renamed (non-canonical) scenes keep their loaded order
after them; the next auto-save re-stamps SortOrder to the corrected index. One integration test
(`SceneOrderTests`) seeds a Chat-before-BRB DB and asserts the loaded order + F1-F5 labels, then that
a save persists the corrected SortOrder (fresh load keeps canonical order).
## Desktop "locked" after the app starts while a fullscreen game is running (FIXED 2026-10-10)
Report: once the app is started, the desktop is "locked" — the mouse can no longer interact with it —
which made streaming live gameplay impossible.
Root cause: at launch the main window **activates** (WPF default `ShowActivated=true`). If the creator
is already playing an exclusive-fullscreen game (e.g. THE DIVISION 2 while starting a gameplay stream),
activating LlamaCasty makes Windows drop that game to the taskbar; the app's 1920×1040 window then sits
over the whole work area, so the desktop looks locked and the game can't be reached with the mouse.
Fix (same commit as this entry): the window no longer steals focus when a full-screen game owns the
foreground at launch — `MainWindow.xaml.cs` ctor sets `ShowActivated = !_viewModel.FullscreenGameRunningNow`,
and the new `MainViewModel.FullscreenGameRunningNow` (initially added in `MainViewModel.Background.cs`)
wraps `GetForegroundFullScreenMonitorIndex() != null`. Result: opening the app while in-game leaves the
game focused and alive; opening it normally behaves exactly as before.
Verification: build clean (0 warnings). Real-app reproduction/test launch deferred until the creator is
off-mouse — the real-mouse + fullscreen-detector tests must not run while a game owns the screen.
## iTunes music showing on mic meter
Playing music via iTunes causes the mic meter to show audio levels. The WASAPI
mic capture (`DataFlow.Capture`) and loopback capture (`DataFlow.Loopback`) are
+218
View File
@@ -0,0 +1,218 @@
"""Convert Google's MediaPipe Selfie Segmenter (.tflite) to ONNX for LlamaCasty (TASK 51).
Why a hand-rolled converter: the model ends in MediaPipe's custom op
`Convolution2DTransposeBias`, which tf2onnx / tflite2onnx don't support. The graph only
uses 11 op types, so a direct, auditable translation is smaller than any workaround.
Source model (Apache-2.0, model card "MediaPipe Selfie Segmentation", Google, 2021-05-06):
https://storage.googleapis.com/mediapipe-models/image_segmenter/selfie_segmenter_landscape/float16/latest/selfie_segmenter_landscape.tflite
Layout: TFLite is NHWC. The ONNX graph keeps the SAME external contract
(input [1,144,256,3] float RGB in [0,1], output [1,144,256,1] person probability in [0,1])
and transposes to NCHW internally.
Usage: python convert_tflite_to_onnx.py in.tflite out.onnx
Requires: tflite, onnx, numpy. Verify with verify_against_mediapipe.py.
"""
import sys
import numpy as np
import onnx
import tflite
from onnx import TensorProto, helper, numpy_helper
from tflite.ActivationFunctionType import ActivationFunctionType as Act
from tflite.BuiltinOperator import BuiltinOperator
from tflite.Padding import Padding
from tflite.TensorType import TensorType
OPSET = 17
OP_NAMES = {v: k for k, v in BuiltinOperator.__dict__.items() if not k.startswith("_")}
NP_TYPES = {TensorType.FLOAT32: np.float32, TensorType.FLOAT16: np.float16, TensorType.INT32: np.int32}
def main(src, dst):
model = tflite.Model.GetRootAsModel(open(src, "rb").read(), 0)
g = model.Subgraphs(0)
def shape(ix):
return g.Tensors(ix).ShapeAsNumpy().tolist()
def const(ix):
"""Constant tensor value, or None for runtime tensors."""
t = g.Tensors(ix)
buf = model.Buffers(t.Buffer())
if buf is None or buf.DataLength() == 0:
return None
raw = buf.DataAsNumpy().tobytes()
return np.frombuffer(raw, dtype=NP_TYPES[t.Type()]).reshape(shape(ix))
nodes, inits = [], []
consts = {} # tensor index -> float32 numpy (after DEQUANTIZE folding)
uid = [0]
def name(ix):
return f"t{ix}"
def fresh(prefix):
uid[0] += 1
return f"{prefix}_{uid[0]}"
def add_init(arr, nm=None):
nm = nm or fresh("c")
inits.append(numpy_helper.from_array(np.ascontiguousarray(arr), nm))
return nm
def value(ix):
if ix in consts:
return consts[ix]
c = const(ix)
if c is not None:
consts[ix] = c.astype(np.float32) if c.dtype == np.float16 else c
return consts[ix]
return None
def activation(out_name, act, final_name):
if act == Act.NONE:
return out_name
if act == Act.RELU:
nodes.append(helper.make_node("Relu", [out_name], [final_name]))
elif act == Act.RELU6:
nodes.append(helper.make_node("Clip", [out_name, add_init(np.float32(0)), add_init(np.float32(6))], [final_name]))
else:
raise NotImplementedError(f"fused activation {act}")
return final_name
def operand(ix):
"""Runtime tensor name, or a constant reshaped from NHWC broadcast form to NCHW."""
v = value(ix)
if v is None:
return name(ix)
if v.ndim == 1:
v = v.reshape(1, -1, 1, 1)
elif v.ndim == 4:
v = v.transpose(0, 3, 1, 2)
return add_init(v.astype(np.float32))
# NHWC -> NCHW at the input.
inp = g.Inputs(0)
nodes.append(helper.make_node("Transpose", ["input"], [name(inp)], perm=[0, 3, 1, 2]))
for i in range(g.OperatorsLength()):
op = g.Operators(i)
oc = model.OperatorCodes(op.OpcodeIndex())
code = max(oc.BuiltinCode(), oc.DeprecatedBuiltinCode())
kind = "CUSTOM:" + oc.CustomCode().decode() if code == BuiltinOperator.CUSTOM else OP_NAMES[code]
ins = [op.Inputs(j) for j in range(op.InputsLength())]
out = op.Outputs(0)
o = name(out)
if kind == "DEQUANTIZE":
consts[out] = const(ins[0]).astype(np.float32)
continue
if kind in ("CONV_2D", "DEPTHWISE_CONV_2D"):
opt = (tflite.Conv2DOptions() if kind == "CONV_2D" else tflite.DepthwiseConv2DOptions())
bo = op.BuiltinOptions()
opt.Init(bo.Bytes, bo.Pos)
w = value(ins[1])
b = value(ins[2]) if len(ins) > 2 and ins[2] >= 0 else None
if kind == "CONV_2D":
W = w.transpose(0, 3, 1, 2) # OHWI -> OIHW
group = 1
else:
# [1,H,W,C*mult] -> [C*mult,1,H,W], group = input channels
W = w.transpose(3, 0, 1, 2)
group = shape(ins[0])[3]
attrs = dict(
strides=[opt.StrideH(), opt.StrideW()],
dilations=[opt.DilationHFactor(), opt.DilationWFactor()],
kernel_shape=list(W.shape[2:]),
group=group,
auto_pad="SAME_UPPER" if opt.Padding() == Padding.SAME else "VALID",
)
conv_in = [name(ins[0]), add_init(W)] + ([add_init(b)] if b is not None else [])
tmp = fresh("conv") if opt.FusedActivationFunction() != Act.NONE else o
nodes.append(helper.make_node("Conv", conv_in, [tmp], **attrs))
activation(tmp, opt.FusedActivationFunction(), o)
continue
if kind in ("ADD", "MUL"):
opt = tflite.AddOptions() if kind == "ADD" else tflite.MulOptions()
bo = op.BuiltinOptions()
act = Act.NONE
if bo is not None:
opt.Init(bo.Bytes, bo.Pos)
act = opt.FusedActivationFunction()
tmp = fresh("ew") if act != Act.NONE else o
nodes.append(helper.make_node("Add" if kind == "ADD" else "Mul", [operand(ins[0]), operand(ins[1])], [tmp]))
activation(tmp, act, o)
continue
if kind == "RELU":
nodes.append(helper.make_node("Relu", [name(ins[0])], [o]))
continue
if kind == "HARD_SWISH":
nodes.append(helper.make_node("HardSwish", [name(ins[0])], [o]))
continue
if kind == "LOGISTIC":
nodes.append(helper.make_node("Sigmoid", [name(ins[0])], [o]))
continue
if kind == "MEAN":
opt = tflite.ReducerOptions()
bo = op.BuiltinOptions()
opt.Init(bo.Bytes, bo.Pos)
nhwc_to_nchw = {0: 0, 1: 2, 2: 3, 3: 1}
axes = [nhwc_to_nchw[int(a)] for a in value(ins[1]).reshape(-1)]
nodes.append(helper.make_node("ReduceMean", [name(ins[0])], [o], axes=axes, keepdims=1 if opt.KeepDims() else 0))
continue
if kind == "RESIZE_BILINEAR":
opt = tflite.ResizeBilinearOptions()
bo = op.BuiltinOptions()
opt.Init(bo.Bytes, bo.Pos)
h, w = (int(x) for x in value(ins[1]).reshape(-1))
c = shape(ins[0])[3]
mode = "align_corners" if opt.AlignCorners() else ("half_pixel" if opt.HalfPixelCenters() else "asymmetric")
sizes = add_init(np.array([1, c, h, w], dtype=np.int64))
nodes.append(helper.make_node("Resize", [name(ins[0]), "", "", sizes], [o], mode="linear", coordinate_transformation_mode=mode))
continue
if kind == "CUSTOM:Convolution2DTransposeBias":
# Custom options = TfLiteTransposeConvParams {padding, stride_w, stride_h} (int32 each).
# NOTE: `padding` is the C-API TfLitePadding enum (Unknown=0, SAME=1, VALID=2),
# NOT the flatbuffer Padding enum (SAME=0, VALID=1).
padding, stride_w, stride_h = np.frombuffer(bytes(op.CustomOptionsAsNumpy().tolist()), dtype=np.int32)[:3]
w = value(ins[1]) # [O, kH, kW, I]
b = value(ins[2])
W = w.transpose(3, 0, 1, 2) # -> [I, O, kH, kW] (ONNX ConvTranspose)
kh, kw = w.shape[1], w.shape[2]
if padding != 1 or kh != stride_h or kw != stride_w:
raise NotImplementedError("only SAME, kernel == stride transpose conv is supported")
nodes.append(helper.make_node("ConvTranspose", [name(ins[0]), add_init(W), add_init(b)], [o],
strides=[int(stride_h), int(stride_w)], kernel_shape=[kh, kw]))
continue
raise NotImplementedError(kind)
# NCHW -> NHWC at the output.
outp = g.Outputs(0)
nodes.append(helper.make_node("Transpose", [name(outp)], ["mask"], perm=[0, 2, 3, 1]))
graph = helper.make_graph(
nodes, "mediapipe_selfie_segmenter_landscape",
[helper.make_tensor_value_info("input", TensorProto.FLOAT, shape(inp))],
[helper.make_tensor_value_info("mask", TensorProto.FLOAT, shape(outp))],
initializer=inits)
m = helper.make_model(graph, opset_imports=[helper.make_opsetid("", OPSET)],
producer_name="llamacasty-tflite2onnx",
doc_string="MediaPipe Selfie Segmenter (landscape), Apache-2.0, Google. "
"Converted for LlamaCasty by tools/selfie-seg/convert_tflite_to_onnx.py")
m.ir_version = 8
onnx.checker.check_model(m)
onnx.save(m, dst)
print(f"wrote {dst}: {len(nodes)} nodes, {len(inits)} initializers")
if __name__ == "__main__":
main(sys.argv[1], sys.argv[2])
@@ -0,0 +1,59 @@
"""Verify the converted ONNX model against Google's own MediaPipe runtime (TASK 51).
Feeds the SAME 256x144 RGB frame to (a) MediaPipe's ImageSegmenter running the original
.tflite (with its custom op) and (b) onnxruntime running our converted .onnx, then compares
the person-probability masks. Also writes side-by-side PNGs for an eyeball check.
Usage: python verify_against_mediapipe.py model.tflite model.onnx photo.jpg [out_prefix]
Requires: mediapipe, onnxruntime, numpy, pillow.
"""
import sys
import mediapipe as mp
import numpy as np
import onnxruntime as ort
from mediapipe.tasks import python as mp_python
from mediapipe.tasks.python import vision
from PIL import Image
W, H = 256, 144
def main(tfl, onx, photo, prefix="verify"):
img = Image.open(photo).convert("RGB")
# Cover-crop to 16:9, then resize to the model's exact input so neither runtime resamples.
sw, sh = img.size
target = W / H
if sw / sh > target:
nw = int(sh * target); x = (sw - nw) // 2; img = img.crop((x, 0, x + nw, sh))
else:
nh = int(sw / target); y = (sh - nh) // 2; img = img.crop((0, y, sw, y + nh))
img = img.resize((W, H), Image.BILINEAR)
rgb = np.asarray(img, dtype=np.uint8)
seg = vision.ImageSegmenter.create_from_options(vision.ImageSegmenterOptions(
base_options=mp_python.BaseOptions(model_asset_path=tfl), output_confidence_masks=True))
res = seg.segment(mp.Image(image_format=mp.ImageFormat.SRGB, data=rgb))
ref = res.confidence_masks[0].numpy_view().astype(np.float32).reshape(H, W)
sess = ort.InferenceSession(onx, providers=["CPUExecutionProvider"])
x = (rgb.astype(np.float32) / 255.0)[None]
got = sess.run(None, {"input": x})[0].reshape(H, W)
diff = np.abs(ref - got)
agree = np.mean((ref > 0.5) == (got > 0.5))
print(f"ref range {ref.min():.3f}..{ref.max():.3f} person%={np.mean(ref > 0.5) * 100:.1f}")
print(f"onnx range {got.min():.3f}..{got.max():.3f} person%={np.mean(got > 0.5) * 100:.1f}")
print(f"max|diff|={diff.max():.5f} mean|diff|={diff.mean():.6f} binary-agreement={agree * 100:.2f}%")
def to_img(m):
return Image.fromarray((np.clip(m, 0, 1) * 255).astype(np.uint8)).convert("RGB")
sheet = Image.new("RGB", (W * 3, H))
sheet.paste(img, (0, 0)); sheet.paste(to_img(ref), (W, 0)); sheet.paste(to_img(got), (W * 2, 0))
sheet.resize((W * 6, H * 2), Image.NEAREST).save(f"{prefix}_sheet.png")
return diff.max()
if __name__ == "__main__":
md = main(*sys.argv[1:])
sys.exit(0 if md < 1e-3 else 1)
+9 -11
View File
@@ -331,22 +331,20 @@ public class AudioLevelMeterTests
Assert.Equal(0f, AudioLevelMeter.ToDisplay(0f));
Assert.Equal(0f, AudioLevelMeter.ToDisplay(0.001f), 2);
// -60 dBFS floor → 0, 0 dBFS → 1, and +10 dB amplification: a decade
// (~-20 dB) is 2/3 up the bar.
// -60 dBFS floor → 0, 0 dBFS → 1, honest in between (no hidden boost).
Assert.Equal(1f, AudioLevelMeter.ToDisplay(1f), 3);
Assert.InRange(AudioLevelMeter.ToDisplay(0.0316f), 0.66f, 0.67f);
Assert.InRange(AudioLevelMeter.ToDisplay(0.1f), 0.83f, 0.84f);
Assert.InRange(AudioLevelMeter.ToDisplay(0.01f), 0.49f, 0.51f);
Assert.InRange(AudioLevelMeter.ToDisplay(0.0316f), 0.49f, 0.51f);
Assert.InRange(AudioLevelMeter.ToDisplay(0.1f), 0.66f, 0.67f);
Assert.InRange(AudioLevelMeter.ToDisplay(0.01f), 0.33f, 0.34f);
}
[Fact]
public void ToDisplay_Pushes_Speech_Peaks_Into_Red_At_Maxed_Volume()
public void ToDisplay_Puts_Leveled_Speech_At_The_Green_Yellow_Edge_And_Peaks_In_Red()
{
// ~0.2 linear RMS (-14 dBFS) is a loud speech peak; at maxed volume it
// must land in the red zone (0.8+) and normal speech (~0.05, -26 dBFS)
// in the yellow — the meter uses the whole bar proportionately.
Assert.InRange(AudioLevelMeter.ToDisplay(0.2f), 0.92f, 0.95f);
Assert.InRange(AudioLevelMeter.ToDisplay(0.05f), 0.72f, 0.75f);
// The speech leveler targets -20 dBFS: that sits at 2/3 of the bar (the yellow
// zone starts at 60%); a hot -6 dBFS peak reaches the red (80%+).
Assert.InRange(AudioLevelMeter.ToDisplay(SpeechLeveler.TargetRms), 0.66f, 0.67f);
Assert.InRange(AudioLevelMeter.ToDisplay(0.5f), 0.89f, 0.91f);
}
[Fact]
+45
View File
@@ -98,6 +98,51 @@ public class AudioPipelineTests
}
}
/// <summary>2026-10-09 — a creator's upload measured −28.5 LUFS (YouTube's reference is
/// −14): nothing in the chain brought quiet speech up. THE test for the speech leveler:
/// syllable-shaped "speech" fed through the REAL voice chain lands at the leveler's
/// −20 dBFS target (±3 dB) whether the mic is quiet (−38 dBFS) or hot (−12 dBFS), and the
/// pauses between phrases stay silent (the gain is never pumped into the noise).</summary>
[Theory]
[InlineData(-38.0)]
[InlineData(-12.0)]
public void VoiceChain_LevelsQuietAndHotSpeech_ToTheTarget_AndKeepsPausesSilent(double inputDbfs)
{
const int rate = 48000;
var chain = new VoiceFilterChain(rate);
// Unit-RMS "voice": three partials, 4 Hz syllable envelope, then scaled to the input level.
float Voice(int n)
{
var t = n / (float)rate;
var tone = MathF.Sin(2 * MathF.PI * 210 * t) + 0.6f * MathF.Sin(2 * MathF.PI * 420 * t) + 0.3f * MathF.Sin(2 * MathF.PI * 840 * t);
var syllable = MathF.Abs(MathF.Sin(2 * MathF.PI * 4 * t));
return tone * syllable;
}
double unitSq = 0;
for (var n = 0; n < rate; n++) unitSq += Voice(n) * (double)Voice(n);
var scale = (float)(Math.Pow(10, inputDbfs / 20) / Math.Sqrt(unitSq / rate));
// 12 s: speak 2 s, pause 0.5 s, repeat. Measure the last 4 s.
double speechSq = 0; long speechN = 0;
var pauseMax = 0f;
var total = 12 * rate;
for (var n = 0; n < total; n++)
{
var phase = n % (int)(2.5 * rate);
var speaking = phase < 2 * rate;
var output = chain.Process(speaking ? Voice(n) * scale : 0f);
if (n < total - 4 * rate) continue;
if (speaking) { speechSq += output * (double)output; speechN++; }
else if (phase > 2 * rate + rate / 10) pauseMax = MathF.Max(pauseMax, MathF.Abs(output)); // after the gate's release tail
}
var speechDb = 10 * Math.Log10(speechSq / speechN);
var targetDb = 20 * Math.Log10(SpeechLeveler.TargetRms);
Assert.InRange(speechDb, targetDb - 3, targetDb + 3);
Assert.InRange(pauseMax, 0f, 1e-4f); // < -80 dBFS: only the EQ's decaying tail, never boosted noise
}
// ─── Ring buffer ───
[Fact]
+224
View File
@@ -0,0 +1,224 @@
using System.IO;
using System.Windows.Media;
using System.Windows.Media.Imaging;
using Xunit;
using ytLive.Models;
using ytLive.Services;
using ytLive.Services.BackgroundEffects;
namespace ytLive.Tests;
/// <summary>TASK 51 slice 1 — the background-effects engine.</summary>
public class BackgroundEffectEngineTests
{
private sealed class FakeSegmenter : IPersonSegmenter
{
private readonly SegmentationMask? _mask;
public int Calls;
public FakeSegmenter(SegmentationMask? mask) => _mask = mask;
public SegmentationMask? Segment(VideoFrame frame)
{
Calls++;
return _mask;
}
}
/// <summary>The integration test for this unit: a webcam frame run through the
/// processor with Remove comes back with the person kept and the background
/// transparent, colors untouched, flagged non-opaque so the compositor blends it.
/// None is a true passthrough (same instance, segmenter never called).</summary>
[Fact]
public void Remove_keeps_the_person_and_makes_the_background_transparent()
{
const int w = 64, h = 32;
var pixels = new byte[w * h * 4];
for (var i = 0; i < pixels.Length; i += 4)
{
pixels[i] = 10; pixels[i + 1] = 20; pixels[i + 2] = 30; pixels[i + 3] = 255;
}
var frame = new VideoFrame(w, h, pixels) { IsOpaque = true };
// 4×2 mask: left half person, right half background.
var mask = new SegmentationMask(4, 2, new[] { 1f, 1f, 0f, 0f, 1f, 1f, 0f, 0f });
var segmenter = new FakeSegmenter(mask);
var processor = new BackgroundEffectProcessor(segmenter);
Assert.Same(frame, processor.Apply(frame, BackgroundEffect.None));
Assert.Equal(0, segmenter.Calls);
var result = processor.Apply(frame, BackgroundEffect.Remove);
Assert.NotSame(frame, result);
Assert.False(result.IsOpaque);
Assert.Equal(w, result.Width);
Assert.Equal(h, result.Height);
for (var y = 0; y < h; y++)
{
Assert.Equal(255, Alpha(result, 4, y)); // well inside the person half
Assert.Equal(0, Alpha(result, w - 5, y)); // well inside the background half
}
for (var i = 0; i < pixels.Length; i += 4)
{
Assert.Equal(pixels[i], result.BgraPixels[i]);
Assert.Equal(pixels[i + 1], result.BgraPixels[i + 1]);
Assert.Equal(pixels[i + 2], result.BgraPixels[i + 2]);
}
Assert.Equal(255, frame.BgraPixels[(w - 5) * 4 + 3]); // source frame never mutated
}
/// <summary>Model contract: the BUNDLED (embedded) ONNX model, driven through the C#
/// preprocessing, reproduces Google's own MediaPipe runtime on a fixed fixture.
/// The expected mask was produced by tools/selfie-seg/ running the original .tflite in
/// MediaPipe (fixture: MediaPipe's public test image mozart_square.jpg, a public-domain
/// painting, cropped to 256×144). Guards the embedding, the input layout (RGB, NHWC,
/// /255) and the conversion itself.</summary>
[Fact]
public void Bundled_model_matches_the_MediaPipe_reference_mask()
{
var frame = LoadBgra(Fixture("selfie-fixture-256x144.png"));
var expected = LoadBgra(Fixture("selfie-fixture-expected-mask.png"));
using var segmenter = new OnnxPersonSegmenter();
var mask = segmenter.Segment(frame);
Assert.NotNull(mask);
Assert.Equal(OnnxPersonSegmenter.ModelWidth, mask!.Width);
Assert.Equal(OnnxPersonSegmenter.ModelHeight, mask.Height);
var agree = 0;
var person = 0;
for (var i = 0; i < mask.Width * mask.Height; i++)
{
var expectPerson = expected.BgraPixels[i * 4] > 127;
var gotPerson = mask.Values[i] > 0.5f;
if (expectPerson == gotPerson) agree++;
if (gotPerson) person++;
}
var total = mask.Width * mask.Height;
Assert.True(agree >= total * 0.995, $"binary agreement {100.0 * agree / total:0.00}% < 99.5%");
Assert.InRange(100.0 * person / total, 45, 60); // reference: 52.5% person
}
/// <summary>Slice 2 integration test: Blur keeps the person pixel-for-pixel and smooths the
/// background — a 1px black/white checkerboard behind the "person" comes back as mid-tones,
/// opaque, same size.</summary>
[Fact]
public void Blur_keeps_the_person_sharp_and_smooths_the_background()
{
const int w = 64, h = 32;
var pixels = new byte[w * h * 4];
for (var y = 0; y < h; y++)
for (var x = 0; x < w; x++)
{
var p = (y * w + x) * 4;
if (x < w / 2)
{
pixels[p] = 100; pixels[p + 1] = 150; pixels[p + 2] = 200;
}
else
{
var v = (byte)(((x + y) & 1) == 0 ? 0 : 255);
pixels[p] = v; pixels[p + 1] = v; pixels[p + 2] = v;
}
pixels[p + 3] = 255;
}
var frame = new VideoFrame(w, h, pixels) { IsOpaque = true };
var mask = new SegmentationMask(4, 2, new[] { 1f, 1f, 0f, 0f, 1f, 1f, 0f, 0f });
var processor = new BackgroundEffectProcessor(new FakeSegmenter(mask));
var result = processor.Apply(frame, BackgroundEffect.Blur);
Assert.NotSame(frame, result);
Assert.True(result.IsOpaque);
Assert.Equal(w, result.Width);
Assert.Equal(h, result.Height);
for (var y = 0; y < h; y++)
{
for (var x = 0; x < 24; x++) // person: untouched
for (var c = 0; c < 4; c++)
Assert.Equal(pixels[(y * w + x) * 4 + c], result.BgraPixels[(y * w + x) * 4 + c]);
for (var x = 44; x < w; x++) // background: checkerboard smoothed away
{
var p = (y * w + x) * 4;
for (var c = 0; c < 3; c++)
Assert.InRange(result.BgraPixels[p + c], (byte)30, (byte)225);
Assert.Equal(255, result.BgraPixels[p + 3]);
}
}
}
/// <summary>TASK 51 blur-heavy fix — the integration test: an all-background frame with one
/// bright pixel. The default 0.5 strength must keep the halo tight (quadratic curve: radius 2
/// small px) while full strength spreads it far (radius 6) — the maker complained the default
/// was heavy under the old linear 1..12 range.</summary>
[Fact]
public void Blur_default_strength_is_gentle_and_full_strength_is_strong()
{
const int w = 128, h = 64;
var pixels = new byte[w * h * 4];
for (var y = 0; y < h; y++)
for (var x = 0; x < w; x++)
{
var p = (y * w + x) * 4;
if (x < w / 2) pixels[p] = 200;
pixels[p + 1] = 180;
pixels[p + 2] = 160;
pixels[p + 3] = 255;
}
var frame = new VideoFrame(w, h, pixels) { IsOpaque = true };
// All background — the whole frame blurs, no person boundary to confuse the edge spread.
var mask = new SegmentationMask(4, 2, new float[8]);
var processor = new BackgroundEffectProcessor(new FakeSegmenter(mask));
var gentle = processor.Apply(frame, BackgroundEffect.Blur, BackgroundEffectProcessor.DefaultStrength);
var strong = processor.Apply(frame, BackgroundEffect.Blur, 1.0);
// The radius mapping itself: quadratic, not the old linear 1..12. Default 0.5 = 2 small px
// (was 6), max = 6 small px (was 12) — the reason the default felt heavy before.
Assert.Equal(1, BackgroundEffectProcessor.BlurRadius(0.0));
Assert.Equal(2, BackgroundEffectProcessor.BlurRadius(BackgroundEffectProcessor.DefaultStrength));
Assert.Equal(BackgroundEffectProcessor.MinBlurRadius, BackgroundEffectProcessor.BlurRadius(0.0));
Assert.Equal(BackgroundEffectProcessor.MaxBlurRadius, BackgroundEffectProcessor.BlurRadius(1.0));
// Behaviour at a step edge: the default smears light only ~9px into the dark half (measured),
// full strength smears ~26px — a gentle default with a real range above it.
var (reachGentle, sumGentle) = EdgeReach(gentle);
var (reachStrong, sumStrong) = EdgeReach(strong);
Assert.True(reachGentle <= 12, $"default blur reach {reachGentle}px (expected <= 12)");
Assert.True(reachStrong >= 20, $"max blur reach {reachStrong}px (expected >= 20)");
Assert.True(reachStrong > reachGentle);
Assert.True(sumStrong > sumGentle * 2, "max strength should smear visibly more light than default");
}
/// <summary>Largest x into the dark half (x = w/2) still carrying light, and the total light
/// in the dark half — the blur's smearing distance at a step edge.</summary>
private static (int Reach, double Sum) EdgeReach(VideoFrame f)
{
var edge = f.Width / 2;
var reach = 0;
double sum = 0;
for (var y = 0; y < f.Height; y++)
for (var x = edge; x < f.Width; x++)
{
var v = f.BgraPixels[(y * f.Width + x) * 4];
if (v >= 32) reach = Math.Max(reach, x - edge);
sum += v;
}
return (reach, sum);
}
private static int Alpha(VideoFrame f, int x, int y) => f.BgraPixels[(y * f.Width + x) * 4 + 3];
private static string Fixture(string name) => Path.Combine(AppContext.BaseDirectory, "Fixtures", name);
private static VideoFrame LoadBgra(string path)
{
using var stream = File.OpenRead(path);
var decoded = BitmapDecoder.Create(stream, BitmapCreateOptions.PreservePixelFormat, BitmapCacheOption.OnLoad).Frames[0];
var bgra = new FormatConvertedBitmap(decoded, PixelFormats.Bgra32, null, 0);
var pixels = new byte[bgra.PixelWidth * bgra.PixelHeight * 4];
bgra.CopyPixels(pixels, bgra.PixelWidth * 4, 0);
return new VideoFrame(bgra.PixelWidth, bgra.PixelHeight, pixels);
}
}
+34
View File
@@ -164,6 +164,7 @@ public sealed class BrandFlashOutputTests
{
var tempDb = Path.Combine(Path.GetTempPath(), $"ytLlive-test-{Guid.NewGuid():N}.db");
MainViewModel.LayoutPathOverride = tempDb;
MainViewModel.ShowBrandFlash = true; // the credit is off by default (2026-10-09)
var window = new MainWindow();
var vm = (MainViewModel)window.DataContext;
try
@@ -202,6 +203,7 @@ public sealed class BrandFlashOutputTests
{
window.Close();
MainViewModel.LayoutPathOverride = null;
MainViewModel.ShowBrandFlash = false;
SqliteConnection.ClearAllPools();
try { File.Delete(tempDb); } catch { /* best-effort cleanup */ }
}
@@ -381,6 +383,36 @@ public sealed class BrandFlashOutputTests
Assert.True(prop.CanRead, "the preview needs to read the gate state");
}
/// <summary>Creator ruling 2026-10-09: the credit is OFF by default — a freshly
/// constructed app with no licence still emits nothing to the preview or the output.</summary>
[Fact]
public void Credit_IsOffByDefault_EvenOnTheFreeTier()
{
_app.Run(() =>
{
Assert.False(MainViewModel.ShowBrandFlash);
var tempDb = Path.Combine(Path.GetTempPath(), $"ytLlive-test-{Guid.NewGuid():N}.db");
MainViewModel.LayoutPathOverride = tempDb;
var window = new MainWindow();
var vm = (MainViewModel)window.DataContext;
try
{
Assert.False(vm.IsPremium);
Assert.False(vm.BrandFlashEnabled);
vm.AdvanceBrandFlash(120); // past several cadence windows
Assert.Null(vm.BrandFlashFrame());
Assert.False(vm.BrandFlashVisible);
}
finally
{
window.Close();
MainViewModel.LayoutPathOverride = null;
SqliteConnection.ClearAllPools();
try { File.Delete(tempDb); } catch { /* best-effort cleanup */ }
}
});
}
// ---------- the credit is NOT a go-live-only behaviour ----------
/// <summary>Reported 2026-09-26: "the made with llamacasty flash should appear in
@@ -400,6 +432,7 @@ public sealed class BrandFlashOutputTests
{
var tempDb = Path.Combine(Path.GetTempPath(), $"ytLlive-test-{Guid.NewGuid():N}.db");
MainViewModel.LayoutPathOverride = tempDb;
MainViewModel.ShowBrandFlash = true; // the credit is off by default (2026-10-09)
var window = new MainWindow();
var vm = (MainViewModel)window.DataContext;
try
@@ -441,6 +474,7 @@ public sealed class BrandFlashOutputTests
{
window.Close();
MainViewModel.LayoutPathOverride = null;
MainViewModel.ShowBrandFlash = false;
SqliteConnection.ClearAllPools();
try { File.Delete(tempDb); } catch { /* best-effort cleanup */ }
}
+167
View File
@@ -1,8 +1,13 @@
using System;
using System.Collections.Generic;
using System.Threading;
using System.Threading.Tasks;
using System.Windows.Media.Imaging;
using System.Windows.Threading;
using Xunit;
using ytLive.Models;
using ytLive.Services;
using ytLive.Services.BackgroundEffects;
namespace ytLive.Tests;
@@ -183,4 +188,166 @@ public class CameraManagerTests
Assert.Null(manager.GetLatestFrame("dev1"));
Assert.Contains("dev1", stopped);
}
private sealed class HalfMaskSegmenter : IPersonSegmenter
{
public int Calls;
public SegmentationMask? Segment(VideoFrame frame)
{
Interlocked.Increment(ref Calls);
return new SegmentationMask(2, 1, new[] { 1f, 0f }); // left person, right background
}
}
/// <summary>Returns a FRESH mask per call — like the real segmenter, whose contract test
/// pins the raw per-frame output: {person, person, bg, bg} then {person, 0.2, bg, bg}.</summary>
private sealed class SteppingMaskSegmenter : IPersonSegmenter
{
public int Calls;
public SegmentationMask? Segment(VideoFrame _)
{
var n = Interlocked.Increment(ref Calls);
var v = n == 1 ? new[] { 1f, 1f, 0f, 0f } : new[] { 1f, 0.2f, 0f, 0f };
return new SegmentationMask(4, 1, v);
}
}
/// <summary>TASK 51 slice 4 integration test — the one processing point: with a Remove
/// variant requested, a raw camera frame reaches BOTH consumers processed — the
/// compositor's latest frame for that variant and the variant's preview bitmap on the UI
/// dispatcher — while the raw path keeps the untouched frame.</summary>
[Fact]
public async Task Requested_Remove_Variant_Reaches_Compositor_And_Preview_Raw_Stays_Raw()
{
Dispatcher? ui = null;
using var ready = new ManualResetEventSlim();
var uiThread = new Thread(() =>
{
ui = Dispatcher.CurrentDispatcher;
ready.Set();
Dispatcher.Run();
});
uiThread.SetApartmentState(ApartmentState.STA);
uiThread.IsBackground = true;
uiThread.Start();
ready.Wait();
try
{
FakeFrameSource? source = null;
var segmenter = new HalfMaskSegmenter();
var manager = new CameraManager(
new FakeEnumerator(),
id => source = new FakeFrameSource(id, new List<string>(), new List<string>()),
ui, TimeSpan.Zero, new BackgroundEffectProcessor(segmenter));
var key = WebcamEffectKey.For(BackgroundEffect.Remove, 0.5);
manager.SetRequestedEffects("dev1", new[] { key, WebcamEffectKey.None });
var previewArrived = new TaskCompletionSource<WriteableBitmap>(TaskCreationOptions.RunContinuationsAsynchronously);
manager.EffectPreviewBitmapChanged += (device, k, bitmap) =>
{
if (device == "dev1" && k == key) previewArrived.TrySetResult(bitmap);
};
Assert.True(await manager.AcquireAsync("dev1"));
const int w = 16, h = 4;
var pixels = new byte[w * h * 4];
for (var i = 0; i < pixels.Length; i += 4) { pixels[i] = 50; pixels[i + 1] = 60; pixels[i + 2] = 70; pixels[i + 3] = 255; }
var raw = new VideoFrame(w, h, pixels) { IsOpaque = true };
source!.Pump(raw);
var bitmap = await previewArrived.Task.WaitAsync(TimeSpan.FromSeconds(10));
// Raw consumer: the very same frame, untouched.
Assert.Same(raw, manager.GetLatestFrame("dev1"));
Assert.Same(raw, manager.GetLatestFrame("dev1", WebcamEffectKey.None));
// Compositor consumer: the processed variant.
var processed = manager.GetLatestFrame("dev1", key);
Assert.NotNull(processed);
Assert.False(processed!.IsOpaque);
Assert.Equal(255, processed.BgraPixels[(1 * w + 1) * 4 + 3]); // person side
Assert.Equal(0, processed.BgraPixels[(1 * w + (w - 2)) * 4 + 3]); // background side
// Preview consumer: the variant's bitmap carries the same cut-out.
var previewPixels = new byte[w * h * 4];
var previewWidth = await ui!.InvokeAsync(() =>
{
bitmap.CopyPixels(previewPixels, w * 4, 0);
return bitmap.PixelWidth;
});
Assert.Equal(w, previewWidth);
Assert.Equal(255, previewPixels[(1 * w + 1) * 4 + 3]);
Assert.Equal(0, previewPixels[(1 * w + (w - 2)) * 4 + 3]);
Assert.Same(bitmap, manager.GetPreviewBitmap("dev1", key));
Assert.True(segmenter.Calls >= 1);
// Dropping the request stops the variant.
manager.SetRequestedEffects("dev1", new[] { WebcamEffectKey.None });
Assert.Null(manager.GetLatestFrame("dev1", key));
}
finally
{
ui?.InvokeShutdown();
}
}
/// <summary>TASK 51 jitter fix integration test — the one processing point: two consecutive
/// frames whose raw masks disagree at an edge. The first frame passes through untouched; the
/// second's blend with history (0.75/0.25) keeps a partial alpha where the raw mask would
/// have snapped to 0 — the per-frame judder the smoother damps.</summary>
[Fact]
public async Task Temporal_Smoothing_Damps_Segmentation_Jitter_Across_Frames()
{
FakeFrameSource? source = null;
var segmenter = new SteppingMaskSegmenter();
var manager = new CameraManager(
new FakeEnumerator(),
id => source = new FakeFrameSource(id, new List<string>(), new List<string>()),
null, TimeSpan.Zero, new BackgroundEffectProcessor(segmenter));
var key = WebcamEffectKey.For(BackgroundEffect.Remove, 0.5);
manager.SetRequestedEffects("dev1", new[] { key });
Assert.True(await manager.AcquireAsync("dev1"));
const int w = 16, h = 4;
var pixels = new byte[w * h * 4];
for (var i = 0; i < pixels.Length; i += 4) { pixels[i] = 50; pixels[i + 1] = 60; pixels[i + 2] = 70; pixels[i + 3] = 255; }
var frame = new VideoFrame(w, h, pixels) { IsOpaque = true };
var edge = (0 * w + 6) * 4 + 3; // row 0, x 6: the contested mask boundary
source!.Pump(frame);
var first = await PollLatestFrameAsync(manager, key, distinctFrom: null);
Assert.NotNull(first);
Assert.Equal(255, first!.BgraPixels[edge]); // maiden frame: raw mask, untouched
// A NEW instance — the worker's latest-frame-drop guard treats a re-queued
// identical reference as "no new frame", which is the real camera behavior.
source.Pump(new VideoFrame(w, h, pixels) { IsOpaque = true });
var second = await PollLatestFrameAsync(manager, key, distinctFrom: first);
Assert.NotNull(second);
Assert.Equal(2, segmenter.Calls);
Assert.False(second!.IsOpaque);
Assert.Equal(58, second.BgraPixels[edge]);
Assert.Equal(255, second.BgraPixels[(0 * w + 1) * 4 + 3]); // person core stays opaque
Assert.Equal(0, second.BgraPixels[(0 * w + 14) * 4 + 3]); // background stays cut
// The raw second mask alone would have cut the same pixel to 0 — exactly the flutter
// the smoother removes.
var rawSecond = BackgroundEffectProcessor.Render(frame,
new SegmentationMask(4, 1, new[] { 1f, 0.2f, 0f, 0f }), BackgroundEffect.Remove, 0.5);
Assert.Equal(0, rawSecond.BgraPixels[edge]);
}
private static async Task<VideoFrame?> PollLatestFrameAsync(CameraManager manager,
WebcamEffectKey key, VideoFrame? distinctFrom)
{
var deadline = DateTime.UtcNow + TimeSpan.FromSeconds(10);
while (DateTime.UtcNow < deadline)
{
var frame = manager.GetLatestFrame("dev1", key);
if (frame != null && !ReferenceEquals(frame, distinctFrom)) return frame;
await Task.Delay(10);
}
return null;
}
}
+152
View File
@@ -0,0 +1,152 @@
using System;
using System.Collections.Generic;
using System.IO;
using System.Windows;
using Microsoft.Data.Sqlite;
using Xunit;
using ytLive.Services;
using ytLive.ViewModels;
namespace ytLive.Tests;
/// <summary>
/// The desktop-"lock" bug (2026-10-10): at launch the main window ACTIVES
/// (WPF default <c>ShowActivated = true</c>), and when the creator is already
/// playing an exclusive-fullscreen game, activating LlamaCasty makes Windows
/// drop that game to the taskbar; the app's 1920×1040 window then sits over the
/// whole work area, so the desktop looks locked and the game can't be reached
/// with the mouse. Fix: the window opens WITHOUT activating when a full-screen
/// game owns the foreground at launch — <c>ShowActivated = !FullscreenGameRunningNow</c>.
/// These tests pin that rule using the <c>FullScreenDetectorOverride</c> seam,
/// so no real Win32 foreground state is touched.
/// </summary>
[Collection("RealApp")]
public sealed class DesktopLockTests
{
private readonly RealAppHost _app;
public DesktopLockTests(RealAppHost app) => _app = app;
[Fact]
public void MainWindow_Does_Not_Steal_Focus_When_A_Fullscreen_Game_Is_Foreground()
{
_app.Run(() =>
{
using var _ = new TempLayout();
MainViewModel.FullScreenDetectorOverride = new StubFullScreenDetector(foregroundMonitor: 0);
try
{
var window = new MainWindow();
try
{
// The regression: a full-screen game already owns the screen, so
// the window must open WITHOUT activating to keep the game live.
Assert.False(window.ShowActivated,
"launching while a full-screen game owns the foreground must not steal focus");
}
finally
{
window.Close();
}
}
finally
{
MainViewModel.FullScreenDetectorOverride = null;
}
});
}
[Fact]
public void MainWindow_Normal_Launch_Still_Activates()
{
_app.Run(() =>
{
using var _ = new TempLayout();
MainViewModel.FullScreenDetectorOverride = new StubFullScreenDetector(foregroundMonitor: null);
try
{
var window = new MainWindow();
try
{
// No game running: behaviour is exactly as before the fix.
Assert.True(window.ShowActivated,
"a normal launch (no full-screen game) must keep the default activate-on-show");
}
finally
{
window.Close();
}
}
finally
{
MainViewModel.FullScreenDetectorOverride = null;
}
});
}
[Fact]
public void FullscreenGameRunningNow_Reflects_The_Foreground_Monitor()
{
_app.Run(() =>
{
MainViewModel.FullScreenDetectorOverride = new StubFullScreenDetector(foregroundMonitor: 0);
try
{
var gameVm = new MainViewModel();
Assert.True(gameVm.FullscreenGameRunningNow);
}
finally
{
MainViewModel.FullScreenDetectorOverride = null;
}
MainViewModel.FullScreenDetectorOverride = new StubFullScreenDetector(foregroundMonitor: null);
try
{
var idleVm = new MainViewModel();
Assert.False(idleVm.FullscreenGameRunningNow);
}
finally
{
MainViewModel.FullScreenDetectorOverride = null;
}
});
}
private sealed class TempLayout : IDisposable
{
private readonly string _path;
public TempLayout()
{
_path = Path.Combine(Path.GetTempPath(), $"ytLive-desktoplock-{Guid.NewGuid():N}.db");
using (var schema = new LayoutStore(_path)) { }
SqliteConnection.ClearAllPools();
MainViewModel.LayoutPathOverride = _path;
}
public void Dispose()
{
MainViewModel.LayoutPathOverride = null;
SqliteConnection.ClearAllPools();
try { File.Delete(_path); } catch { /* best-effort cleanup */ }
}
}
/// <summary>A deterministic stand-in for <see cref="Win32FullScreenDetector"/>:
/// claims a full-screen window owns the given foreground monitor (or none).</summary>
#pragma warning disable CS0067 // fullscreen-changed wiring is not part of this seam
private sealed class StubFullScreenDetector : IFullScreenDetector
{
private readonly int? _foregroundMonitor;
public StubFullScreenDetector(int? foregroundMonitor) => _foregroundMonitor = foregroundMonitor;
public int? GetForegroundFullScreenMonitorIndex() => _foregroundMonitor;
public int PrimaryMonitorIndex() => 0;
public IReadOnlyList<DisplayInfo> GetDisplays() => Array.Empty<DisplayInfo>();
public event Action<int?>? FullscreenMonitorChanged;
public void StartWatching() { }
public void StopWatching() { }
}
}
Binary file not shown.

After

Width:  |  Height:  |  Size: 67 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1019 B

@@ -128,6 +128,57 @@ public class LayoutStorePersistenceTests
}
}
/// <summary>TASK 51 slice 3: each scene's webcam background effect + strength survives a
/// save and a reopen; a pre-TASK-51 DB (no columns) migrates in place and reads None/0.5.</summary>
[Fact]
public void Webcam_Background_Effect_Round_Trips_Per_Scene_And_Legacy_Db_Migrates()
{
var path = Path.Combine(Path.GetTempPath(), $"ytLlive-layout-{Guid.NewGuid():N}.db");
try
{
var webcam = new Webcam { DeviceId = "real-device", Name = "Logitech" };
var live = new Scene { Name = "Live" };
live.Elements.Add(new WebcamSceneConfig { WebcamId = webcam.Id, Width = 480, Height = 270, BackgroundEffect = BackgroundEffect.Remove, BackgroundStrength = 0.2 });
var chat = new Scene { Name = "Chat" };
chat.Elements.Add(new WebcamSceneConfig { WebcamId = webcam.Id, Width = 480, Height = 270, BackgroundEffect = BackgroundEffect.Blur, BackgroundStrength = 0.8 });
using (var store = new LayoutStore(path))
store.Save(new[] { live, chat }, webcam, null);
using (var reopened = new LayoutStore(path))
{
var scenes = reopened.Load();
var liveCfg = scenes.Single(s => s.Name == "Live").Elements.OfType<WebcamSceneConfig>().Single();
var chatCfg = scenes.Single(s => s.Name == "Chat").Elements.OfType<WebcamSceneConfig>().Single();
Assert.Equal(BackgroundEffect.Remove, liveCfg.BackgroundEffect);
Assert.Equal(0.2, liveCfg.BackgroundStrength, 6);
Assert.Equal(BackgroundEffect.Blur, chatCfg.BackgroundEffect);
Assert.Equal(0.8, chatCfg.BackgroundStrength, 6);
}
// Simulate a DB written before TASK 51: drop the two columns, reopen.
SqliteConnection.ClearAllPools();
using (var raw = new SqliteConnection($"Data Source={path}"))
{
raw.Open();
using var cmd = raw.CreateCommand();
cmd.CommandText = "ALTER TABLE WebcamSceneConfig DROP COLUMN BackgroundEffect; ALTER TABLE WebcamSceneConfig DROP COLUMN BackgroundStrength;";
cmd.ExecuteNonQuery();
}
SqliteConnection.ClearAllPools();
using (var legacy = new LayoutStore(path))
{
var cfg = legacy.Load().SelectMany(s => s.Elements).OfType<WebcamSceneConfig>().First();
Assert.Equal(BackgroundEffect.None, cfg.BackgroundEffect);
Assert.Equal(0.5, cfg.BackgroundStrength, 6);
}
}
finally
{
SqliteConnection.ClearAllPools();
try { File.Delete(path); } catch { /* best-effort cleanup */ }
}
}
// The background (schema v5): permanent bottom layer holding live desktop/game
// capture. IsBackground + CaptureKey must survive a save + reload so a
// re-designated target isn't lost on restart.
@@ -0,0 +1,71 @@
using System.Net;
using System.Text;
using System.Xml.Linq;
using Xunit;
using ytLive.Models;
using ytLive.Services;
using ytLive.ViewModels;
namespace ytLive.Tests;
/// <summary>Creator ruling 2026-10-09: a PRIVATE broadcast ends like a recording — Enter keeps
/// it on YouTube, Escape deletes it.</summary>
public class PrivateBroadcastCloseOutTests
{
private sealed class Handler : HttpMessageHandler
{
public string Privacy = "private";
public readonly List<string> Requests = new();
protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken ct)
{
Requests.Add($"{request.Method.Method} {request.RequestUri!.PathAndQuery}");
if (request.Method == HttpMethod.Delete)
return Task.FromResult(new HttpResponseMessage(HttpStatusCode.NoContent));
var body = "{\"items\":[{\"id\":\"BC1\",\"status\":{\"privacyStatus\":\"" + Privacy + "\",\"lifeCycleStatus\":\"complete\"}}]}";
return Task.FromResult(new HttpResponseMessage(HttpStatusCode.OK)
{
Content = new StringContent(body, Encoding.UTF8, "application/json"),
});
}
}
private static YouTubeStreamService Service(Handler handler)
{
var auth = new YouTubeAuthService("test-id", "test-secret");
auth.SetSession(new YouTubeChannel { AccessToken = "acc", TokenExpiry = DateTime.UtcNow.AddHours(1) });
return new YouTubeStreamService(auth, new HttpClient(handler));
}
[Fact]
public async Task Private_Broadcast_Escape_Deletes_Enter_Keeps_Public_Is_Never_Asked()
{
// Escape / Delete → the video is deleted from YouTube.
var h = new Handler();
var (outcome, error) = await MainViewModel.ClosePrivateBroadcastAsync(Service(h), "BC1", askKeep: () => false);
Assert.Equal(MainViewModel.PrivateBroadcastOutcome.Deleted, outcome);
Assert.Null(error);
Assert.Contains(h.Requests, r => r.StartsWith("DELETE /youtube/v3/videos?id=BC1"));
// Enter / Keep → nothing is deleted.
h = new Handler();
(outcome, _) = await MainViewModel.ClosePrivateBroadcastAsync(Service(h), "BC1", askKeep: () => true);
Assert.Equal(MainViewModel.PrivateBroadcastOutcome.Kept, outcome);
Assert.DoesNotContain(h.Requests, r => r.StartsWith("DELETE"));
// Not private (changed mid-stream) → no prompt, nothing deleted.
h = new Handler { Privacy = "public" };
var asked = false;
(outcome, _) = await MainViewModel.ClosePrivateBroadcastAsync(Service(h), "BC1", askKeep: () => asked = true);
Assert.Equal(MainViewModel.PrivateBroadcastOutcome.NotPrivate, outcome);
Assert.False(asked);
Assert.DoesNotContain(h.Requests, r => r.StartsWith("DELETE"));
// The prompt itself: Enter = Keep (IsDefault), Escape = Delete (IsCancel).
XNamespace x = "http://schemas.microsoft.com/winfx/2006/xaml";
var doc = XDocument.Load(SourceAssert.RepoPath("KeepBroadcastDialog.xaml"));
XElement Named(string name) => doc.Descendants().Single(e => (string?)e.Attribute(x + "Name") == name);
Assert.Equal("True", (string?)Named("KeepBroadcastButton").Attribute("IsDefault"));
Assert.Equal("True", (string?)Named("DeleteBroadcastButton").Attribute("IsCancel"));
}
}
+89
View File
@@ -0,0 +1,89 @@
using System;
using System.IO;
using System.Linq;
using Microsoft.Data.Sqlite;
using Xunit;
using ytLive.ViewModels;
namespace ytLive.Tests;
/// <summary>
/// Scene-strip ordering vs F-keys (creator's report 2026-10-10). The five
/// canonical scenes are loaded <c>ORDER BY SortOrder</c>, and a session can
/// persist Chat before BRB, so the thumbnail strip renders in the wrong order
/// while each tab's F-key label is name-derived — "the ordering of the 5
/// screens and their corresponding F-key is out of sequence". Fix: the VM
/// reorders Scenes to the canonical F1-F5 order (Starting, Live, BRB, Chat,
/// Ending) at load; the next save re-stamps SortOrder to the corrected order.
/// </summary>
[Collection("RealApp")]
public sealed class SceneOrderTests
{
private readonly RealAppHost _app;
public SceneOrderTests(RealAppHost app) => _app = app;
[Fact]
public void Scenes_Load_Into_Canonical_FKey_Order_Whatever_The_SortOrder_Rows()
{
_app.Run(() =>
{
using var layout = new TempSwappedOrderLayout();
var vm = new MainViewModel();
var names = vm.Scenes.Select(s => s.Name).ToArray();
Assert.Equal(new[] { "Starting", "Live", "BRB", "Chat", "Ending" }, names);
var keys = vm.Scenes.Select(s => s.HotkeyLabel).ToArray();
Assert.Equal(new[] { "F1", "F2", "F3", "F4", "F5" }, keys);
// The heal persists: a save re-stamps SortOrder, so a fresh load (the
// thumbnail strip's rendering source) keeps the canonical order.
vm.SaveLayoutNow();
using var reloaded = new Services.LayoutStore(layout.DbPath);
Assert.Equal(new[] { "Starting", "Live", "BRB", "Chat", "Ending" },
reloaded.Load().Select(s => s.Name).ToArray());
});
}
/// <summary>Creates a DB whose five scenes are persisted with Chat BEFORE
/// BRB — the exact stale state found on the creator's box.</summary>
private sealed class TempSwappedOrderLayout : IDisposable
{
public string DbPath { get; }
private readonly string _path;
public TempSwappedOrderLayout()
{
_path = Path.Combine(Path.GetTempPath(), $"ytLive-sceneorder-{Guid.NewGuid():N}.db");
DbPath = _path;
using (var schema = new Services.LayoutStore(_path)) { }
SqliteConnection.ClearAllPools();
MainViewModel.LayoutPathOverride = _path;
using var connection = new SqliteConnection($"Data Source={_path}");
connection.Open();
using (var cmd = connection.CreateCommand())
{
cmd.CommandText = """
INSERT INTO Scene (Id, Name, IsHidden, IsChatScene, HasBackground, SortOrder)
VALUES
('scene-starting', 'Starting', 0, 0, 1, 0),
('scene-live', 'Live', 0, 0, 1, 1),
('scene-chat', 'Chat', 0, 1, 1, 2),
('scene-brb', 'BRB', 0, 0, 1, 3),
('scene-ending', 'Ending', 0, 0, 1, 4);
""";
cmd.ExecuteNonQuery();
}
}
public void Dispose()
{
MainViewModel.LayoutPathOverride = null;
SqliteConnection.ClearAllPools();
try { File.Delete(_path); } catch { /* best-effort cleanup */ }
}
}
}
+358
View File
@@ -0,0 +1,358 @@
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Windows;
using System.Windows.Media;
using System.Windows.Media.Imaging;
using Microsoft.Data.Sqlite;
using Xunit;
using ytLive.Helpers;
using ytLive.Models;
using ytLive.Services;
using ytLive.ViewModels;
namespace ytLive.Tests;
/// <summary>
/// Scene strip tile artwork (creator ruling 2026-10-10). Each canonical scene's
/// top-strip tile is its own image, stored as an Asset row and seeded from the
/// creator's CRT-styled renders. The preview pane keeps rendering each scene's
/// ORIGINAL background — the tiles never appear as preview backgrounds.
/// </summary>
[Collection("RealApp")]
public sealed class SceneThumbnailTests
{
private readonly RealAppHost _app;
public SceneThumbnailTests(RealAppHost app) => _app = app;
[Fact]
public void Strip_Shows_The_Seeded_Tile_And_The_Preview_Keeps_The_Original_Background()
{
_app.Run(() =>
{
EnvGuard? env = null;
try
{
env = ArrangedDb(out var file);
env.SeedPaths[SceneCatalog.Starting] = file.TilePath;
env.SeedPaths[SceneCatalog.Live] = MissingPath();
env.Apply();
var vm = new MainViewModel();
var starting = SceneByName(vm, SceneCatalog.Starting);
Assert.False(string.IsNullOrWhiteSpace(starting.ThumbnailAssetId));
// The strip tile is the seeded file (identity + dims prove it).
var tile = ImageCache.Get(starting.ThumbnailAssetId);
Assert.NotNull(tile);
Assert.Same(tile, starting.Snapshot);
Assert.Equal(320, starting.Snapshot!.PixelWidth);
Assert.Equal(180, starting.Snapshot.PixelHeight);
// The preview pane renders the scene's ORIGINAL background — never
// the tile artwork.
var originalBackground = ImageCache.Get(env.BackgroundIds[SceneCatalog.Starting]);
Assert.NotNull(originalBackground);
Assert.Same(originalBackground, vm.BackgroundImage);
Assert.Same(originalBackground, vm.ActiveBackgroundImage);
Assert.Same(originalBackground, MainViewModel.LoadBackgroundImage(starting));
Assert.NotSame(tile, vm.BackgroundImage);
// A scene without its own tile keeps its background art as the strip.
var ending = SceneByName(vm, SceneCatalog.Ending);
Assert.Null(ending.ThumbnailAssetId);
Assert.Same(ImageCache.Get(env.BackgroundIds[SceneCatalog.Ending]), ending.Snapshot);
// The mapping and the tile asset survive a save + reload (the
// Save-time Asset prune must keep Scene.ThumbnailAssetId rows), and
// a second load re-seeds nothing new (UpsertAsset is SHA256-keyed).
vm.SaveLayoutNow();
using (var reloaded = new LayoutStore(env.DbPath))
{
var reloadedStarting = reloaded.Load().Single(s => s.Name == SceneCatalog.Starting);
Assert.Equal(starting.ThumbnailAssetId, reloadedStarting.ThumbnailAssetId);
Assert.NotNull(reloaded.GetAssetBytes(starting.ThumbnailAssetId));
Assert.Equal(1, CountAssetRows(env.DbPath, starting.ThumbnailAssetId));
}
var secondVm = new MainViewModel();
Assert.Equal(starting.ThumbnailAssetId,
SceneByName(secondVm, SceneCatalog.Starting).ThumbnailAssetId);
}
finally
{
env?.Restore();
env?.Dispose();
}
});
}
[Fact]
public void Live_Staged_Mini_Shows_Its_Tile_Not_The_Capture_Placeholder()
{
_app.Run(() =>
{
EnvGuard? env = null;
try
{
env = ArrangedDb(out var file);
env.SeedPaths[SceneCatalog.Live] = file.TilePath;
env.Apply();
var vm = new MainViewModel();
var live = SceneByName(vm, SceneCatalog.Live);
Assert.False(string.IsNullOrWhiteSpace(live.ThumbnailAssetId));
vm.StagedScene = live;
// Minis never render captures; a seeded Live tile shows instead of
// the green placeholder, while the preview still shows the background.
Assert.Same(ImageCache.Get(live.ThumbnailAssetId), live.Snapshot);
Assert.Same(ImageCache.Get(env.BackgroundIds[SceneCatalog.Live]), vm.BackgroundImage);
}
finally
{
env?.Restore();
env?.Dispose();
}
});
}
[Fact]
public void Staged_Scene_Tile_Shows_The_On_Air_Art_And_It_Follows_The_Scene_Swap()
{
_app.Run(() =>
{
EnvGuard? env = null;
try
{
env = ArrangedDb(out var file);
MainViewModel.OnAirSeedPath = file.OnAirPath;
env.Apply();
var vm = new MainViewModel();
var starting = SceneByName(vm, SceneCatalog.Starting);
var live = SceneByName(vm, SceneCatalog.Live);
var onAir = ImageCache.Get(Services.LayoutStore.Instance!.GetOnAirImageId()!);
Assert.NotNull(onAir);
// The staged scene's tile (Starting at launch) shows the on-air art.
Assert.Same(onAir, starting.Snapshot);
// Unstaged scenes keep their own art, never the on-air image.
Assert.NotSame(onAir, live.Snapshot);
// Swapping scenes moves the on-air tile: staging Live swaps it across,
// and Starting's tile returns to its own background art (no thumbnail).
vm.StagedScene = live;
Assert.Same(onAir, live.Snapshot);
Assert.Same(ImageCache.Get(env.BackgroundIds[SceneCatalog.Starting]), starting.Snapshot);
// The preview pane still renders the scene's ORIGINAL background —
// the on-air art never leaks into the preview.
Assert.Same(ImageCache.Get(env.BackgroundIds[SceneCatalog.Live]), vm.BackgroundImage);
// Persistence: the on-air asset survives a save + reload (the Save
// pruner must keep the OnAirImageAssetId setting's row), and a fresh
// VM re-stamps it from the DB instead of re-importing.
vm.SaveLayoutNow();
using (var reloaded = new LayoutStore(env.DbPath))
{
var id = reloaded.GetOnAirImageId();
Assert.NotNull(id);
Assert.NotNull(reloaded.GetAssetBytes(id));
}
var secondVm = new MainViewModel();
Assert.Same(onAir, SceneByName(secondVm, SceneCatalog.Starting).Snapshot);
}
finally
{
env?.Restore();
env?.Dispose();
}
});
}
[Fact]
public void No_Seed_File_Means_No_Thumbnail_And_Background_Art_Fallback()
{
_app.Run(() =>
{
EnvGuard? env = null;
try
{
env = ArrangedDb(out _);
env.SeedPaths.Clear();
env.Apply();
var vm = new MainViewModel();
foreach (var scene in vm.Scenes)
{
Assert.Null(scene.ThumbnailAssetId);
Assert.Same(ImageCache.Get(env.BackgroundIds[scene.Name]), scene.Snapshot);
}
}
finally
{
env?.Restore();
env?.Dispose();
}
});
}
// ─── Fixture plumbing ───
private static Scene SceneByName(MainViewModel vm, string name)
=> vm.Scenes.Single(s => s.Name == name);
private static int CountAssetRows(string dbPath, string assetId)
{
using var connection = new SqliteConnection($"Data Source={dbPath}");
connection.Open();
using var cmd = connection.CreateCommand();
cmd.CommandText = "SELECT COUNT(*) FROM Asset WHERE Id = $id;";
cmd.Parameters.AddWithValue("$id", assetId);
return Convert.ToInt32(cmd.ExecuteScalar());
}
/// <summary>Builds a temp DB whose every scene has a REAL background asset row.
/// One throwaway VM creates the canonical scenes + heals their Backgrounds
/// (the pack:// default-art stamp silently fails under the test host, so the
/// contract is tested with an explicit background — the tile must never
/// replace it). Seed paths start all-missing; the caller picks which scenes
/// get a tile.</summary>
private static EnvGuard ArrangedDb(out Files files)
{
files = new Files();
var dbPath = Path.Combine(Path.GetTempPath(), $"ytLive-scenethumb-{Guid.NewGuid():N}.db");
using (new LayoutStore(dbPath)) { }
SqliteConnection.ClearAllPools();
var env = new EnvGuard(dbPath, files);
try
{
env.Apply();
var builder = new MainViewModel();
var backgroundIds = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
var i = 0;
foreach (var scene in builder.Scenes)
{
var bg = scene.Elements.OfType<Source>().FirstOrDefault(s => s.IsBackground);
Assert.NotNull(bg);
var bytes = files.NewBackgroundBytes(i++);
var id = LayoutStore.Instance!.UpsertAsset(bytes, 320, 180);
Assert.NotNull(id);
bg!.AssetId = id;
backgroundIds[scene.Name] = id;
}
builder.SaveLayoutNow();
env.BackgroundIds = backgroundIds;
return env;
}
catch
{
env.Restore();
env.Dispose();
throw;
}
}
private sealed record EnvGuard : IDisposable
{
public string DbPath { get; }
public Files Files { get; }
public Dictionary<string, string> BackgroundIds { get; set; } = new(StringComparer.OrdinalIgnoreCase);
public Dictionary<string, string> SeedPaths { get; }
private readonly Dictionary<string, string> _originalPaths;
private readonly string _originalOnAirPath;
public EnvGuard(string dbPath, Files files)
{
DbPath = dbPath;
Files = files;
_originalPaths = new Dictionary<string, string>(MainViewModel.SceneThumbnailSeedPaths, StringComparer.OrdinalIgnoreCase);
_originalOnAirPath = MainViewModel.OnAirSeedPath;
SeedPaths = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
MainViewModel.LayoutPathOverride = dbPath;
MainViewModel.OnAirSeedPath = MissingPath();
foreach (var path in _originalPaths)
SeedPaths[path.Key] = MissingPath();
}
/// <summary>Writes the tests' chosen SeedPaths into the live map; any
/// canonical scene not overridden stays pointing at a missing file.</summary>
public void Apply()
{
MainViewModel.SceneThumbnailSeedPaths.Clear();
foreach (var (key, value) in SeedPaths)
MainViewModel.SceneThumbnailSeedPaths[key] = value;
}
public void Restore()
{
MainViewModel.SceneThumbnailSeedPaths.Clear();
foreach (var (key, value) in _originalPaths)
MainViewModel.SceneThumbnailSeedPaths[key] = value;
MainViewModel.OnAirSeedPath = _originalOnAirPath;
MainViewModel.LayoutPathOverride = null;
SqliteConnection.ClearAllPools();
}
public void Dispose()
{
Files.Dispose();
try { File.Delete(DbPath); } catch { }
}
}
private sealed class Files : IDisposable
{
public string TilePath { get; }
public string OnAirPath { get; }
private readonly List<string> _paths = new();
public Files()
{
TilePath = NewImage(320, 180, Brushes.Teal, Brushes.Red);
OnAirPath = NewImage(320, 180, Brushes.SlateBlue, Brushes.White);
}
/// <summary>Distinct per-scene solid-color "original background".</summary>
public byte[] NewBackgroundBytes(int index)
{
var colors = new[] { Brushes.ForestGreen, Brushes.OrangeRed, Brushes.DodgerBlue, Brushes.DarkOrchid, Brushes.SaddleBrown };
var path = NewImage(320, 180, colors[index % colors.Length], Brushes.Black);
return File.ReadAllBytes(path);
}
private string NewImage(int w, int h, Brush fill, Brush accent)
{
var path = Path.Combine(Path.GetTempPath(), $"ytLive-scenethumb-img-{Guid.NewGuid():N}.png");
var dv = new DrawingVisual();
using (var dc = dv.RenderOpen())
{
dc.DrawRectangle(fill, null, new Rect(0, 0, w, h));
dc.DrawRectangle(accent, null, new Rect(24, 24, 64, 64));
}
var rtb = new RenderTargetBitmap(w, h, 96, 96, PixelFormats.Pbgra32);
rtb.Render(dv);
var encoder = new PngBitmapEncoder();
encoder.Frames.Add(BitmapFrame.Create(rtb));
using var fs = File.OpenWrite(path);
encoder.Save(fs);
_paths.Add(path);
return path;
}
public void Dispose()
{
foreach (var path in _paths)
try { File.Delete(path); } catch { }
}
}
private static string MissingPath()
=> Path.Combine(Path.GetTempPath(), $"ytLive-no-such-{Guid.NewGuid():N}.png");
}
+92
View File
@@ -0,0 +1,92 @@
using System.IO;
using System.Threading.Tasks;
using Microsoft.Data.Sqlite;
using Xunit;
using ytLive.Services.Audio;
using ytLive.ViewModels;
namespace ytLive.Tests;
/// <summary>2026-10-09 bug: pressing TRAX started the music but the status light stayed off —
/// and the toggle couldn't pause, because the playing state was never refreshed after Play.
/// TRAX is a toggle: the light is green while playing, red with a terse scrolling message on
/// failure, gray when idle.</summary>
[Collection("RealApp")]
public sealed class TraxToggleTests
{
private sealed class FakePlayer : ITraxPlayer
{
public bool Starts = true;
public event Action<Exception?>? PlaybackEnded;
public string? TrackName { get; private set; }
public bool IsPlaying { get; private set; }
public float LocalGain { get; set; } = 1f;
public void Load(string path) => TrackName = Path.GetFileNameWithoutExtension(path);
public void Play() => IsPlaying = Starts;
public void Pause() => IsPlaying = false;
public void Dispose() { }
public void Fail() { IsPlaying = false; PlaybackEnded?.Invoke(new IOException("device lost")); }
}
private readonly RealAppHost _app;
public TraxToggleTests(RealAppHost app) => _app = app;
[Fact]
public async Task Trax_Toggle_Lights_Green_While_Playing_And_Red_With_A_Terse_Error()
{
await _app.RunAsync(async () =>
{
var tempDb = Path.Combine(Path.GetTempPath(), $"ytLlive-test-{Guid.NewGuid():N}.db");
var track = Path.Combine(Path.GetTempPath(), $"trax-{Guid.NewGuid():N}.mp3");
File.WriteAllBytes(track, new byte[16]);
var player = new FakePlayer();
MainViewModel.LayoutPathOverride = tempDb;
MainViewModel.TraxPlayerOverride = () => player;
var window = new MainWindow();
var vm = (MainViewModel)window.DataContext;
try
{
await vm.SetTraxTrackAsync(track, play: false);
Assert.False(vm.IsMusicPlaying);
var idle = vm.TraxDotBrush.Color;
vm.TraxLeftClick(); // press: plays → light on
Assert.True(vm.IsMusicPlaying);
var playing = vm.TraxDotBrush.Color;
Assert.NotEqual(idle, playing);
Assert.Null(vm.TraxError);
vm.TraxLeftClick(); // press again: pauses → light off
Assert.False(vm.IsMusicPlaying);
Assert.Equal(idle, vm.TraxDotBrush.Color);
player.Starts = false; // press: playback can't start
vm.TraxLeftClick();
Assert.False(vm.IsMusicPlaying);
Assert.NotNull(vm.TraxError);
Assert.True(vm.TraxError!.Length <= 32, $"error not terse: '{vm.TraxError}'");
var red = vm.TraxDotBrush.Color;
Assert.NotEqual(idle, red);
Assert.NotEqual(playing, red);
player.Starts = true; // recovery clears the error
vm.TraxLeftClick();
Assert.True(vm.IsMusicPlaying);
Assert.Null(vm.TraxError);
Assert.Equal(playing, vm.TraxDotBrush.Color);
player.Fail(); // device lost mid-play
Assert.False(vm.IsMusicPlaying);
Assert.Equal(red, vm.TraxDotBrush.Color);
}
finally
{
window.Close();
MainViewModel.LayoutPathOverride = null;
MainViewModel.TraxPlayerOverride = null;
SqliteConnection.ClearAllPools();
try { File.Delete(tempDb); File.Delete(track); } catch { /* best-effort cleanup */ }
}
});
}
}
+63
View File
@@ -0,0 +1,63 @@
using System.Linq;
using System.Xml.Linq;
using Xunit;
using ytLive.Models;
namespace ytLive.Tests;
/// <summary>TASK 51 slice 5 — the creator's UI spec (2026-10-09): Remove and Blur are two pill
/// switches in the webcam's Layers properties that act as radio buttons (one on turns the other
/// off; both off = no effect), plus a strength slider.</summary>
public class WebcamBackgroundPillTests
{
[Fact]
public void Remove_and_Blur_pills_are_radio_exclusive_and_wired_in_the_webcam_properties()
{
// Behaviour: the bools the pills bind to.
var config = new WebcamSceneConfig();
var raised = new System.Collections.Generic.List<string?>();
config.PropertyChanged += (_, e) => raised.Add(e.PropertyName);
config.RemoveBackground = true;
Assert.Equal(BackgroundEffect.Remove, config.BackgroundEffect);
Assert.True(config.HasBackgroundEffect);
raised.Clear();
config.BlurBackground = true; // turning Blur on turns Remove off
Assert.Equal(BackgroundEffect.Blur, config.BackgroundEffect);
Assert.False(config.RemoveBackground);
Assert.Contains(nameof(WebcamSceneConfig.RemoveBackground), raised); // Remove pill un-checks
Assert.Contains(nameof(WebcamSceneConfig.BlurBackground), raised);
config.RemoveBackground = false; // turning the INACTIVE pill off changes nothing
Assert.Equal(BackgroundEffect.Blur, config.BackgroundEffect);
config.BlurBackground = false; // turning the active pill off = None
Assert.Equal(BackgroundEffect.None, config.BackgroundEffect);
Assert.False(config.HasBackgroundEffect);
// Wiring: the two PillToggle switches + the strength slider live in the webcam's
// properties section of the Layers panel, bound to those properties.
XNamespace x = "http://schemas.microsoft.com/winfx/2006/xaml";
var doc = XDocument.Load(SourceAssert.RepoPath("Controls", "LeftPanel.xaml"));
var webcamPanel = doc.Descendants()
.Single(e => e.Name.LocalName == "StackPanel"
&& ((string?)e.Attribute("Visibility") ?? "").Contains("SelectedElement.IsWebcam"));
XElement Named(string name) => webcamPanel.Descendants().Single(e => (string?)e.Attribute(x + "Name") == name);
foreach (var (pill, property) in new[] { ("RemoveBackgroundPill", "RemoveBackground"), ("BlurBackgroundPill", "BlurBackground") })
{
var toggle = Named(pill);
Assert.Equal("ToggleButton", toggle.Name.LocalName);
Assert.Contains("PillToggle", (string?)toggle.Attribute("Style"));
Assert.Contains($"SelectedElement.{property}", (string?)toggle.Attribute("IsChecked"));
Assert.Contains("TwoWay", (string?)toggle.Attribute("IsChecked"));
}
var slider = Named("BackgroundStrengthSlider");
Assert.Contains("SelectedElement.BackgroundStrength", (string?)slider.Attribute("Value"));
Assert.Equal("0", (string?)slider.Attribute("Minimum"));
Assert.Equal("1", (string?)slider.Attribute("Maximum"));
}
}
+7
View File
@@ -6,6 +6,9 @@
<ImplicitUsings>enable</ImplicitUsings>
<IsPackable>false</IsPackable>
<EnableWindowsTargeting>true</EnableWindowsTargeting>
<!-- Matches the app (64-bit only) — an AnyCPU test project referencing an x64 app
raises MSB3270 (architecture mismatch) and breaks the 0-warning gate. -->
<PlatformTarget>x64</PlatformTarget>
</PropertyGroup>
<ItemGroup>
@@ -18,4 +21,8 @@
<ProjectReference Include="..\ytLive.csproj"/>
</ItemGroup>
<ItemGroup>
<None Include="Fixtures\**" CopyToOutputDirectory="PreserveNewest"/>
</ItemGroup>
</Project>
+17
View File
@@ -10,6 +10,11 @@
<ApplicationIcon>Assets\llamacasty-icon.ico</ApplicationIcon>
<AssemblyName>ytLive</AssemblyName>
<RootNamespace>ytLive</RootNamespace>
<!-- 64-bit Windows only (creator ruling 2026-10-09). The bundled ffmpeg is the win64 build
and ONNX Runtime ships no win-x86 native, so a 32-bit process could never stream; a
32-bit address space (2–4 GB) would also choke on 1080p frame buffers. x64 makes the
exe refuse to start on 32-bit Windows instead of failing at Go Live. -->
<PlatformTarget>x64</PlatformTarget>
</PropertyGroup>
<ItemGroup>
@@ -73,6 +78,17 @@
<ItemGroup>
<None Include="THIRD-PARTY-NOTICES.txt" CopyToOutputDirectory="PreserveNewest" />
<!-- GPL-3.0 §4/§6: the license text (and our §7 additional terms) travel with every copy. -->
<None Include="LICENSE" CopyToOutputDirectory="PreserveNewest" CopyToPublishDirectory="Always" />
<None Include="ADDITIONAL-TERMS.md" CopyToOutputDirectory="PreserveNewest" CopyToPublishDirectory="Always" />
</ItemGroup>
<!-- Webcam background effects (TASK 51): Google's MediaPipe Selfie Segmenter (landscape,
Apache-2.0), converted to ONNX by tools/selfie-seg/ and embedded in the assembly —
~430 KB, so it lives in git, ships inside the exe, and is never downloaded at runtime
(Store policy 10.2.2, same posture as the bundled ffmpeg). -->
<ItemGroup>
<EmbeddedResource Include="Assets\Models\selfie_segmenter_landscape.onnx" LogicalName="ytLive.Assets.Models.selfie_segmenter_landscape.onnx" />
</ItemGroup>
<!-- Bundled FFmpeg (TASK 48 item 1, Store policy 10.2.2 dynamic code inclusion).
@@ -102,6 +118,7 @@
<ItemGroup>
<PackageReference Include="Microsoft.Data.Sqlite" Version="8.0.10" />
<PackageReference Include="Microsoft.ML.OnnxRuntime" Version="1.31.0" />
<PackageReference Include="NAudio.Wasapi" Version="2.2.1" />
<PackageReference Include="NAudio.WinMM" Version="2.2.1" />
<PackageReference Include="Notification.Wpf" Version="11.0.0" />