LlamaCasty/TASKS/task-31-scenegraph-optimization.md

# TASK 31 — SceneGraph component + static/dynamic compositor optimization

> Catalog: [`TASKS.md`](../TASKS.md) — status and requirements live here.


**Goal:** extract the scene collection, element inventory, and mutation surface into a standalone
`SceneGraph` component. Classify elements as static or dynamic. Enable the compositor to bake
static layers once and only recompose dynamic layers per frame.

### Status: ✅ Done (2026-08-31)

What landed — core optimization + SceneGraph component, all in one commit:

- `ElementKind` (Static/Dynamic) on `SceneElement` base, overridden in `Source` and `WebcamSceneConfig`
- `Services/SceneGraph.cs`: owns `Scenes` collection (ViewModel's `Scenes` property delegates to it);
  mutation surface `AddElement`/`InsertElement`/`RemoveElement`/`MoveElement` (each invalidates the bake);
  queries `GetBackground`/`GetWebcam`/`GetChatBoxes`/`GetSplitPoint`/`IsStatic`; and the
  `GetBakedBase` bake-cache (keyed by scene id + static element identities below the split)
- `SceneCompositor.BakeStaticBase`/`CompositeLayers` + `Render(.., staticBase, split)` overload —
  split-aware, builds the cached base in source-rect space, composites dynamic/above-split per frame
- `FramePump` takes an optional `SceneGraph` and uses the optimized path when wired (falls back to full render)
- `MainViewModel`: routes element mutations through the graph; invalidates the bake on static
  layout/opacity/visibility/useDefaultBackground changes and after background heal / EnsureBackground
- Integration test `SceneGraphTests.BakedStaticBase_WithDynamicLayer_CompositesCorrectly` (audio-free,
  verifies bake-once + cache hit + dynamic-on-top pixels + static-mutation invalidation +
  dynamic-only non-invalidation)

**Defensive deviations from this spec (decision 2026-08-31):**
1. `ChatOverlayLayer` keeps taking `IEnumerable<Scene>` rather than `SceneGraph.GetChatBoxes()` — it is
   deliberately decoupled from the graph (`// without owning the scene graph`). Forcing the graph in would
   couple a WPF-bound layer to it and violate that documented seam.
2. Background static helpers (`EnsureBackground`/`CreateBackground`/`NormalizeBackgrounds`) stay on the
   ViewModel because `BackgroundTests.cs` unit-tests `MainViewModel.EnsureBackground` directly. Queries
   moved to SceneGraph; helpers + their invalidation wiring stayed on the VM.
3. The full ViewModel-as-facade extraction (moving the ~all-binding-surface off Scenes/StagedScene/
   LiveScene) was deliberately NOT done in this unsupervised pass — it is the "touch 11 files across 4
   layers" regression risk the handoff flagged. SceneGraph owns the collection + mutation surface now;
   the remaining VM call sites binding to those still work because `Scenes` delegates. Revisit after 1.0.

**Verification:** SceneCompositorTests 4, StretchMathTests 4, BackgroundTests 16, SceneCatalogTests 18,
LayoutStorePersistenceTests 12, FramePumpTests 9, SceneGraphTests 1 all green. RealAppHost GUI/collection
tests (SourceNaming, RoundClip, BackgroundHeal, ...) construct a real `MainWindow` — **CORRECTED
2026-09-01:** they DO run from WSL when invoked per-class through the Windows `dotnet.exe` vstest host
(the old "cannot run headless" claim conflated them with the full-suite WASAPI hang). Two first-launch
crashes this refactor shipped with were caught on the first real native launch and fixed same day:
ctor-order `SceneGraph` NRE (`null!` field assigned after first ctor use — now field-initialized) and
window-scope `EyeButton`/`EyeIconStyle` consumed via `StaticResource` from the extracted `LeftPanel`
(UserControl namescopes can't see window resources — styles moved to `Themes/Controls.xaml`, the
app-scope rule honored). The RoundClip "known failure" was then root-caused to stale test code
(`window.FindName` across namescopes + `VisualTreeHelper.HitTest` where `UIElement.InputHitTest`
models input) — test green 2026-09-01, the sole remaining known failure is the audio one.

### Design

**Element classification:**

Every `SceneElement` exposes `ElementKind Kind` — `Static` or `Dynamic`:

| Type | Kind | Why |
|------|------|-----|
| `Source` (Background art) | Static | Pixels don't change at runtime |
| `Source` (Image) | Static | Pixels don't change at runtime |
| `Source` (DisplayCapture) | Dynamic | Live game/desktop feed |
| `Source` (ChatBox) | Dynamic | Live chat messages arrive continuously |
| `Source` (WebSource) | Dynamic | WebView content can change |
| `WebcamSceneConfig` | Dynamic | Camera feed, changes every frame |

**Split point:**

The **split point** is the index of the first dynamic element in a scene's z-ordered `Elements`.
Everything below it = baked base. Everything from it upward (including static layers above dynamic
elements) = composited per frame. If zero dynamic elements exist: entire scene is baked,
compositor skipped entirely.

**BakedSceneCache:**

```
BakedSceneCache {
    VideoFrame BaseFrame           // composited static layers below split point
    int Version                    // incremented on mutation
    SceneElement[] BakedElements   // for invalidation tracking
}
```

Invalidation triggers (re-bake): element added/removed/reordered, static element moved/resized/
opacity changed, `IsVisible` toggled on a static element, background asset changed, scene switch.

No invalidation when: only a dynamic element's pixels changed (webcam frame, chat message),
or a dynamic element moved/resized (per-frame compositing concern, not a base rebake).

**Compositor integration:**

```
Render(scene, frameFor, options):
    1. Find split point (first Dynamic element index)
    2. If split == scene.Elements.Count:
        → return cached base (or bake if stale)
    3. Otherwise:
        → start from cached base (or bake if stale)
        → composite elements[split..] on top using frameFor
```

**SceneGraph interface:**

```
SceneGraph {
    // Collection
    ObservableCollection<Scene> Scenes
    Scene? StagedScene
    Scene? LiveScene

    // Mutation surface (single owner of element operations)
    AddElement(scene, element)
    RemoveElement(scene, element)
    MoveElement(scene, element, newIndex)

    // Queries (replace scattered LINQ)
    GetBackground(scene): Source?
    GetWebcam(scene): WebcamSceneConfig?
    GetChatBoxes(): IEnumerable<Source>
    GetSplitPoint(scene): int
    IsStatic(scene): bool

    // Events
    SceneChanged
    ElementAdded/Removed/Moved
    SplitPointChanged
}
```

**What moves into SceneGraph:**

| Current location | Moves to |
|-----------------|----------|
| `MainViewModel.Scenes.cs` — Scenes collection, StagedScene, scene switching | `SceneGraph` |
| `MainViewModel.Background.cs` — `NormalizeBackgrounds`, `EnsureBackground`, background queries | `SceneGraph` (background helpers) |
| `MainViewModel.Sources.cs` — `AddSource`, `RemoveElement`, element inventory | `SceneGraph` (mutation surface) |
| `MainViewModel.Webcam.cs` — webcam queries across scenes | `SceneGraph.GetWebcam(scene)` |
| `ChatOverlayLayer` — `scenes.SelectMany(...).OfType<Source>().Where(ChatBox)` | `SceneGraph.GetChatBoxes()` |

**What stays on the ViewModel:**

The binding surface — `StagedScene` setter still raises `OnPropertyChanged` for
`ShowEmptySceneHint`, `CanAddWebcam`, etc. But now it delegates to `SceneGraph` for actual
state queries. ViewModel becomes a thin binding facade over the SceneGraph (same pattern as
`MainViewModel.Chat.cs` over `ChatOverlayLayer`).

The compositor resolver (`frameFor` callback) stays in the ViewModel/encoder layer — it bridges
WPF concepts (CameraManager, ScreenCaptureManager, ImageCache) into the pure `VideoFrame` seam.
SceneGraph doesn't know about capture sessions; it just knows element kinds.

**Migration path:**

1. Extract `SceneGraph` as a standalone class in `Services/`
2. Move `Scenes` collection + `StagedScene`/`LiveScene` + wiring
3. Move element mutation surface (add/remove/move)
4. Move background normalization/queries
5. Add `ElementKind` to `SceneElement` base
6. Add `GetSplitPoint` + `BakedSceneCache`
7. Update compositor to use split point
8. Wire ViewModel as thin facade
9. One integration test: baked static base + dynamic layer composite

### Tests

ONE integration test: `SceneGraphTests.BakedStaticBase_WithDynamicLayer_CompositesCorrectly`
— a scene with static background + image + dynamic webcam; assert the baked base is cached
(renders once), dynamic layer composited on top, static element mutation invalidates the cache,
dynamic-only pixel change does not.