TASK 31: SceneGraph design + baked-crust compositor optimization

Design captured in TASKS.md, pattern note in ai.md, handoff updated.
This commit is contained in:
2026-08-31 11:38:12 -07:00
parent 84a038c498
commit 3bf053a3b1
3 changed files with 132 additions and 0 deletions
+3
View File
@@ -27,6 +27,9 @@ docs commit `cb54637` + **Phase 3 splits A–E** (tags `refactor-commit-A..E`) +
Nothing code-in-flight — working tree clean.
**TASK 31 designed:** SceneGraph component + baked-crust compositor optimization.
Design captured in `TASKS.md`, pattern note in `ai.md`. Ready to implement.
**The directive (2026-08-31, user):** rewrite the project, breaking files into
**functional components to compliment AI retrieval/processing** — NOT line-count
chasing. Line count is a guideline for context management, not a design goal.
+128
View File
@@ -1221,6 +1221,134 @@ and a full layout restructure moving all live controls into the preview pane.
---
## TASK 31 — SceneGraph component + static/dynamic compositor optimization
**Goal:** extract the scene collection, element inventory, and mutation surface into a standalone
`SceneGraph` component. Classify elements as static or dynamic. Enable the compositor to bake
static layers once and only recompose dynamic layers per frame.
### Status: ☐ Not started
### Design
**Element classification:**
Every `SceneElement` exposes `ElementKind Kind` — `Static` or `Dynamic`:
| Type | Kind | Why |
|------|------|-----|
| `Source` (Background art) | Static | Pixels don't change at runtime |
| `Source` (Image) | Static | Pixels don't change at runtime |
| `Source` (DisplayCapture) | Dynamic | Live game/desktop feed |
| `Source` (ChatBox) | Dynamic | Live chat messages arrive continuously |
| `Source` (WebSource) | Dynamic | WebView content can change |
| `WebcamSceneConfig` | Dynamic | Camera feed, changes every frame |
**Split point:**
The **split point** is the index of the first dynamic element in a scene's z-ordered `Elements`.
Everything below it = baked base. Everything from it upward (including static layers above dynamic
elements) = composited per frame. If zero dynamic elements exist: entire scene is baked,
compositor skipped entirely.
**BakedSceneCache:**
```
BakedSceneCache {
VideoFrame BaseFrame // composited static layers below split point
int Version // incremented on mutation
SceneElement[] BakedElements // for invalidation tracking
}
```
Invalidation triggers (re-bake): element added/removed/reordered, static element moved/resized/
opacity changed, `IsVisible` toggled on a static element, background asset changed, scene switch.
No invalidation when: only a dynamic element's pixels changed (webcam frame, chat message),
or a dynamic element moved/resized (per-frame compositing concern, not a base rebake).
**Compositor integration:**
```
Render(scene, frameFor, options):
1. Find split point (first Dynamic element index)
2. If split == scene.Elements.Count:
→ return cached base (or bake if stale)
3. Otherwise:
→ start from cached base (or bake if stale)
→ composite elements[split..] on top using frameFor
```
**SceneGraph interface:**
```
SceneGraph {
// Collection
ObservableCollection<Scene> Scenes
Scene? StagedScene
Scene? LiveScene
// Mutation surface (single owner of element operations)
AddElement(scene, element)
RemoveElement(scene, element)
MoveElement(scene, element, newIndex)
// Queries (replace scattered LINQ)
GetBackground(scene): Source?
GetWebcam(scene): WebcamSceneConfig?
GetChatBoxes(): IEnumerable<Source>
GetSplitPoint(scene): int
IsStatic(scene): bool
// Events
SceneChanged
ElementAdded/Removed/Moved
SplitPointChanged
}
```
**What moves into SceneGraph:**
| Current location | Moves to |
|-----------------|----------|
| `MainViewModel.Scenes.cs` — Scenes collection, StagedScene, scene switching | `SceneGraph` |
| `MainViewModel.Background.cs` — `NormalizeBackgrounds`, `EnsureBackground`, background queries | `SceneGraph` (background helpers) |
| `MainViewModel.Sources.cs` — `AddSource`, `RemoveElement`, element inventory | `SceneGraph` (mutation surface) |
| `MainViewModel.Webcam.cs` — webcam queries across scenes | `SceneGraph.GetWebcam(scene)` |
| `ChatOverlayLayer` — `scenes.SelectMany(...).OfType<Source>().Where(ChatBox)` | `SceneGraph.GetChatBoxes()` |
**What stays on the ViewModel:**
The binding surface — `StagedScene` setter still raises `OnPropertyChanged` for
`ShowEmptySceneHint`, `CanAddWebcam`, etc. But now it delegates to `SceneGraph` for actual
state queries. ViewModel becomes a thin binding facade over the SceneGraph (same pattern as
`MainViewModel.Chat.cs` over `ChatOverlayLayer`).
The compositor resolver (`frameFor` callback) stays in the ViewModel/encoder layer — it bridges
WPF concepts (CameraManager, ScreenCaptureManager, ImageCache) into the pure `VideoFrame` seam.
SceneGraph doesn't know about capture sessions; it just knows element kinds.
**Migration path:**
1. Extract `SceneGraph` as a standalone class in `Services/`
2. Move `Scenes` collection + `StagedScene`/`LiveScene` + wiring
3. Move element mutation surface (add/remove/move)
4. Move background normalization/queries
5. Add `ElementKind` to `SceneElement` base
6. Add `GetSplitPoint` + `BakedSceneCache`
7. Update compositor to use split point
8. Wire ViewModel as thin facade
9. One integration test: baked static base + dynamic layer composite
### Tests
ONE integration test: `SceneGraphTests.BakedStaticBase_WithDynamicLayer_CompositesCorrectly`
— a scene with static background + image + dynamic webcam; assert the baked base is cached
(renders once), dynamic layer composited on top, static element mutation invalidates the cache,
dynamic-only pixel change does not.
---
## Backlog (future versions)
1. v1.1 — Stream Deck / Loupedeck integration (requires hotkey foundation from TASK 20)
+1
View File
File diff suppressed because one or more lines are too long