TASK 31: SceneGraph design + baked-crust compositor optimization

Design captured in TASKS.md, pattern note in ai.md, handoff updated.
2026-08-31 11:38:12 -07:00
parent 84a038c498
commit 3bf053a3b1
3 changed files with 132 additions and 0 deletions
@@ -1221,6 +1221,134 @@ and a full layout restructure moving all live controls into the preview pane.

 ---

+## TASK 31 — SceneGraph component + static/dynamic compositor optimization
+
+**Goal:** extract the scene collection, element inventory, and mutation surface into a standalone
+`SceneGraph` component. Classify elements as static or dynamic. Enable the compositor to bake
+static layers once and only recompose dynamic layers per frame.
+
+### Status: ☐ Not started
+
+### Design
+
+**Element classification:**
+
+Every `SceneElement` exposes `ElementKind Kind` — `Static` or `Dynamic`:
+
+| Type | Kind | Why |
+|------|------|-----|
+| `Source` (Background art) | Static | Pixels don't change at runtime |
+| `Source` (Image) | Static | Pixels don't change at runtime |
+| `Source` (DisplayCapture) | Dynamic | Live game/desktop feed |
+| `Source` (ChatBox) | Dynamic | Live chat messages arrive continuously |
+| `Source` (WebSource) | Dynamic | WebView content can change |
+| `WebcamSceneConfig` | Dynamic | Camera feed, changes every frame |
+
+**Split point:**
+
+The **split point** is the index of the first dynamic element in a scene's z-ordered `Elements`.
+Everything below it = baked base. Everything from it upward (including static layers above dynamic
+elements) = composited per frame. If zero dynamic elements exist: entire scene is baked,
+compositor skipped entirely.
+
+**BakedSceneCache:**
+
+```
+BakedSceneCache {
+    VideoFrame BaseFrame           // composited static layers below split point
+    int Version                    // incremented on mutation
+    SceneElement[] BakedElements   // for invalidation tracking
+}
+```
+
+Invalidation triggers (re-bake): element added/removed/reordered, static element moved/resized/
+opacity changed, `IsVisible` toggled on a static element, background asset changed, scene switch.
+
+No invalidation when: only a dynamic element's pixels changed (webcam frame, chat message),
+or a dynamic element moved/resized (per-frame compositing concern, not a base rebake).
+
+**Compositor integration:**
+
+```
+Render(scene, frameFor, options):
+    1. Find split point (first Dynamic element index)
+    2. If split == scene.Elements.Count:
+        → return cached base (or bake if stale)
+    3. Otherwise:
+        → start from cached base (or bake if stale)
+        → composite elements[split..] on top using frameFor
+```
+
+**SceneGraph interface:**
+
+```
+SceneGraph {
+    // Collection
+    ObservableCollection<Scene> Scenes
+    Scene? StagedScene
+    Scene? LiveScene
+
+    // Mutation surface (single owner of element operations)
+    AddElement(scene, element)
+    RemoveElement(scene, element)
+    MoveElement(scene, element, newIndex)
+
+    // Queries (replace scattered LINQ)
+    GetBackground(scene): Source?
+    GetWebcam(scene): WebcamSceneConfig?
+    GetChatBoxes(): IEnumerable<Source>
+    GetSplitPoint(scene): int
+    IsStatic(scene): bool
+
+    // Events
+    SceneChanged
+    ElementAdded/Removed/Moved
+    SplitPointChanged
+}
+```
+
+**What moves into SceneGraph:**
+
+| Current location | Moves to |
+|-----------------|----------|
+| `MainViewModel.Scenes.cs` — Scenes collection, StagedScene, scene switching | `SceneGraph` |
+| `MainViewModel.Background.cs` — `NormalizeBackgrounds`, `EnsureBackground`, background queries | `SceneGraph` (background helpers) |
+| `MainViewModel.Sources.cs` — `AddSource`, `RemoveElement`, element inventory | `SceneGraph` (mutation surface) |
+| `MainViewModel.Webcam.cs` — webcam queries across scenes | `SceneGraph.GetWebcam(scene)` |
+| `ChatOverlayLayer` — `scenes.SelectMany(...).OfType<Source>().Where(ChatBox)` | `SceneGraph.GetChatBoxes()` |
+
+**What stays on the ViewModel:**
+
+The binding surface — `StagedScene` setter still raises `OnPropertyChanged` for
+`ShowEmptySceneHint`, `CanAddWebcam`, etc. But now it delegates to `SceneGraph` for actual
+state queries. ViewModel becomes a thin binding facade over the SceneGraph (same pattern as
+`MainViewModel.Chat.cs` over `ChatOverlayLayer`).
+
+The compositor resolver (`frameFor` callback) stays in the ViewModel/encoder layer — it bridges
+WPF concepts (CameraManager, ScreenCaptureManager, ImageCache) into the pure `VideoFrame` seam.
+SceneGraph doesn't know about capture sessions; it just knows element kinds.
+
+**Migration path:**
+
+1. Extract `SceneGraph` as a standalone class in `Services/`
+2. Move `Scenes` collection + `StagedScene`/`LiveScene` + wiring
+3. Move element mutation surface (add/remove/move)
+4. Move background normalization/queries
+5. Add `ElementKind` to `SceneElement` base
+6. Add `GetSplitPoint` + `BakedSceneCache`
+7. Update compositor to use split point
+8. Wire ViewModel as thin facade
+9. One integration test: baked static base + dynamic layer composite
+
+### Tests
+
+ONE integration test: `SceneGraphTests.BakedStaticBase_WithDynamicLayer_CompositesCorrectly`
+— a scene with static background + image + dynamic webcam; assert the baked base is cached
+(renders once), dynamic layer composited on top, static element mutation invalidates the cache,
+dynamic-only pixel change does not.
+
+---
+
 ## Backlog (future versions)

 1. v1.1 — Stream Deck / Loupedeck integration (requires hotkey foundation from TASK 20)