TASK 31: SceneGraph design + baked-crust compositor optimization
Design captured in TASKS.md, pattern note in ai.md, handoff updated.
This commit is contained in:
@@ -27,6 +27,9 @@ docs commit `cb54637` + **Phase 3 splits A–E** (tags `refactor-commit-A..E`) +
|
|||||||
|
|
||||||
Nothing code-in-flight — working tree clean.
|
Nothing code-in-flight — working tree clean.
|
||||||
|
|
||||||
|
**TASK 31 designed:** SceneGraph component + baked-crust compositor optimization.
|
||||||
|
Design captured in `TASKS.md`, pattern note in `ai.md`. Ready to implement.
|
||||||
|
|
||||||
**The directive (2026-08-31, user):** rewrite the project, breaking files into
|
**The directive (2026-08-31, user):** rewrite the project, breaking files into
|
||||||
**functional components to compliment AI retrieval/processing** — NOT line-count
|
**functional components to compliment AI retrieval/processing** — NOT line-count
|
||||||
chasing. Line count is a guideline for context management, not a design goal.
|
chasing. Line count is a guideline for context management, not a design goal.
|
||||||
|
|||||||
@@ -1221,6 +1221,134 @@ and a full layout restructure moving all live controls into the preview pane.
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## TASK 31 — SceneGraph component + static/dynamic compositor optimization
|
||||||
|
|
||||||
|
**Goal:** extract the scene collection, element inventory, and mutation surface into a standalone
|
||||||
|
`SceneGraph` component. Classify elements as static or dynamic. Enable the compositor to bake
|
||||||
|
static layers once and only recompose dynamic layers per frame.
|
||||||
|
|
||||||
|
### Status: ☐ Not started
|
||||||
|
|
||||||
|
### Design
|
||||||
|
|
||||||
|
**Element classification:**
|
||||||
|
|
||||||
|
Every `SceneElement` exposes `ElementKind Kind` — `Static` or `Dynamic`:
|
||||||
|
|
||||||
|
| Type | Kind | Why |
|
||||||
|
|------|------|-----|
|
||||||
|
| `Source` (Background art) | Static | Pixels don't change at runtime |
|
||||||
|
| `Source` (Image) | Static | Pixels don't change at runtime |
|
||||||
|
| `Source` (DisplayCapture) | Dynamic | Live game/desktop feed |
|
||||||
|
| `Source` (ChatBox) | Dynamic | Live chat messages arrive continuously |
|
||||||
|
| `Source` (WebSource) | Dynamic | WebView content can change |
|
||||||
|
| `WebcamSceneConfig` | Dynamic | Camera feed, changes every frame |
|
||||||
|
|
||||||
|
**Split point:**
|
||||||
|
|
||||||
|
The **split point** is the index of the first dynamic element in a scene's z-ordered `Elements`.
|
||||||
|
Everything below it = baked base. Everything from it upward (including static layers above dynamic
|
||||||
|
elements) = composited per frame. If zero dynamic elements exist: entire scene is baked,
|
||||||
|
compositor skipped entirely.
|
||||||
|
|
||||||
|
**BakedSceneCache:**
|
||||||
|
|
||||||
|
```
|
||||||
|
BakedSceneCache {
|
||||||
|
VideoFrame BaseFrame // composited static layers below split point
|
||||||
|
int Version // incremented on mutation
|
||||||
|
SceneElement[] BakedElements // for invalidation tracking
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Invalidation triggers (re-bake): element added/removed/reordered, static element moved/resized/
|
||||||
|
opacity changed, `IsVisible` toggled on a static element, background asset changed, scene switch.
|
||||||
|
|
||||||
|
No invalidation when: only a dynamic element's pixels changed (webcam frame, chat message),
|
||||||
|
or a dynamic element moved/resized (per-frame compositing concern, not a base rebake).
|
||||||
|
|
||||||
|
**Compositor integration:**
|
||||||
|
|
||||||
|
```
|
||||||
|
Render(scene, frameFor, options):
|
||||||
|
1. Find split point (first Dynamic element index)
|
||||||
|
2. If split == scene.Elements.Count:
|
||||||
|
→ return cached base (or bake if stale)
|
||||||
|
3. Otherwise:
|
||||||
|
→ start from cached base (or bake if stale)
|
||||||
|
→ composite elements[split..] on top using frameFor
|
||||||
|
```
|
||||||
|
|
||||||
|
**SceneGraph interface:**
|
||||||
|
|
||||||
|
```
|
||||||
|
SceneGraph {
|
||||||
|
// Collection
|
||||||
|
ObservableCollection<Scene> Scenes
|
||||||
|
Scene? StagedScene
|
||||||
|
Scene? LiveScene
|
||||||
|
|
||||||
|
// Mutation surface (single owner of element operations)
|
||||||
|
AddElement(scene, element)
|
||||||
|
RemoveElement(scene, element)
|
||||||
|
MoveElement(scene, element, newIndex)
|
||||||
|
|
||||||
|
// Queries (replace scattered LINQ)
|
||||||
|
GetBackground(scene): Source?
|
||||||
|
GetWebcam(scene): WebcamSceneConfig?
|
||||||
|
GetChatBoxes(): IEnumerable<Source>
|
||||||
|
GetSplitPoint(scene): int
|
||||||
|
IsStatic(scene): bool
|
||||||
|
|
||||||
|
// Events
|
||||||
|
SceneChanged
|
||||||
|
ElementAdded/Removed/Moved
|
||||||
|
SplitPointChanged
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**What moves into SceneGraph:**
|
||||||
|
|
||||||
|
| Current location | Moves to |
|
||||||
|
|-----------------|----------|
|
||||||
|
| `MainViewModel.Scenes.cs` — Scenes collection, StagedScene, scene switching | `SceneGraph` |
|
||||||
|
| `MainViewModel.Background.cs` — `NormalizeBackgrounds`, `EnsureBackground`, background queries | `SceneGraph` (background helpers) |
|
||||||
|
| `MainViewModel.Sources.cs` — `AddSource`, `RemoveElement`, element inventory | `SceneGraph` (mutation surface) |
|
||||||
|
| `MainViewModel.Webcam.cs` — webcam queries across scenes | `SceneGraph.GetWebcam(scene)` |
|
||||||
|
| `ChatOverlayLayer` — `scenes.SelectMany(...).OfType<Source>().Where(ChatBox)` | `SceneGraph.GetChatBoxes()` |
|
||||||
|
|
||||||
|
**What stays on the ViewModel:**
|
||||||
|
|
||||||
|
The binding surface — `StagedScene` setter still raises `OnPropertyChanged` for
|
||||||
|
`ShowEmptySceneHint`, `CanAddWebcam`, etc. But now it delegates to `SceneGraph` for actual
|
||||||
|
state queries. ViewModel becomes a thin binding facade over the SceneGraph (same pattern as
|
||||||
|
`MainViewModel.Chat.cs` over `ChatOverlayLayer`).
|
||||||
|
|
||||||
|
The compositor resolver (`frameFor` callback) stays in the ViewModel/encoder layer — it bridges
|
||||||
|
WPF concepts (CameraManager, ScreenCaptureManager, ImageCache) into the pure `VideoFrame` seam.
|
||||||
|
SceneGraph doesn't know about capture sessions; it just knows element kinds.
|
||||||
|
|
||||||
|
**Migration path:**
|
||||||
|
|
||||||
|
1. Extract `SceneGraph` as a standalone class in `Services/`
|
||||||
|
2. Move `Scenes` collection + `StagedScene`/`LiveScene` + wiring
|
||||||
|
3. Move element mutation surface (add/remove/move)
|
||||||
|
4. Move background normalization/queries
|
||||||
|
5. Add `ElementKind` to `SceneElement` base
|
||||||
|
6. Add `GetSplitPoint` + `BakedSceneCache`
|
||||||
|
7. Update compositor to use split point
|
||||||
|
8. Wire ViewModel as thin facade
|
||||||
|
9. One integration test: baked static base + dynamic layer composite
|
||||||
|
|
||||||
|
### Tests
|
||||||
|
|
||||||
|
ONE integration test: `SceneGraphTests.BakedStaticBase_WithDynamicLayer_CompositesCorrectly`
|
||||||
|
— a scene with static background + image + dynamic webcam; assert the baked base is cached
|
||||||
|
(renders once), dynamic layer composited on top, static element mutation invalidates the cache,
|
||||||
|
dynamic-only pixel change does not.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## Backlog (future versions)
|
## Backlog (future versions)
|
||||||
|
|
||||||
1. v1.1 — Stream Deck / Loupedeck integration (requires hotkey foundation from TASK 20)
|
1. v1.1 — Stream Deck / Loupedeck integration (requires hotkey foundation from TASK 20)
|
||||||
|
|||||||
Reference in New Issue
Block a user