feat(chroma): offer to remove a green or blue screen, and key it in the decode

TASK 64 slices D and E: the feature is usable end to end.

- Add a video file → one frame is probed (new `IChromaProbe`/`FfmpegChromaProbe`, one ffmpeg
  pass at 160x90, off the UI thread) and the layer gets an offer: "Looks like a green screen —
  Remove it", with the measured colour as a swatch.
- Accepting it turns the offer into settings (Amount / Softness / Fringe) that reach ffmpeg as
  `colorkey` + `despill` + `format=bgra`, so the key is computed in the pipe and both consumers
  (the shared preview bitmap and the compositor's frame) carry the same alpha.
- Media sessions are now keyed by **path + look** (`MainViewModel.MediaSessionKey`), so two
  layers with the same settings still share one decoder, and changing the look releases the old
  session and starts the new one. Removal, load reconciliation, loop and audio settings all move
  to the same key.
- Settings persist on the `Source` row (six ALTER TABLE guards + SELECT/INSERT), so a keyed layer
  comes back keyed.
- Panel section is plain creator words: Green/Blue Screen, Amount, Softness, Fringe.

Test: `ChromaKeyApplyTests` — the pipe asks ffmpeg to key (and does not when off); a green-screen
clip is offered, accepting it starts a keyed session, the layer's resolver then reads that session
rather than the old one; the setting survives a save.

Docs: TASK 64 D/E ticked with slice F (auto-apply setting + alert clips) left open; frame-pipeline
records the decode-option design; HANDOFF rewritten. verify.sh: 544 passed, 0 failed, 0 warnings.
This commit is contained in:
2026-11-05 08:15:32 -08:00
parent 21db8a17b8
commit ea24405ba2
19 changed files with 732 additions and 106 deletions
+17 -15
View File
@@ -1,21 +1,24 @@
# HANDOFF — 2026-11-05 (TASK 21 — video file layer + its audio)
# HANDOFF — 2026-11-05 (TASK 64 — green/blue screen for clips)
## State
- `main`; TASK 21 is complete. A video file added from the (+) menu decodes, composites into preview and
stream, loops on toggle, and **plays its own soundtrack** (Mute + Level per layer, mixed into the
desktop/game group so the desktop fader and the ducker apply). Removing it stops the decoder.
- Decoding is now ONE pipe: `Services/VideoClipPipe.cs` behind both `MediaVideoSource` (TASK 21) and
`AlertClipDecoder` (TASK 47) — "a clip is a clip". Alert behaviour is unchanged and its tests stay green.
- Tests: `MediaSourceAcquisitionTests` (add → decoder starts, frames, loop, remove) and
`MediaSourceAudioTests` (chunks, level scaling, mute, sound reaching the desktop meter). App build 0 warnings.
- `main`. TASK 21 complete (video file layer + its soundtrack). TASK 64 shipped through slice E:
a clip with a green or blue backdrop is **offered** a key on add ("Looks like a green screen —
Remove it"), accepting it keys the background **inside the decode pipe** (bundled ffmpeg
`colorkey`/`despill`), with Amount/Softness/Fringe in the panel and the settings persisted.
Sessions are keyed by **path + look**, so identical looks share one decoder.
- Detection stays silent on lawns/pitches (confidence = coverage × hue tightness) and ignores
letterbox bars. It never acts on its own.
- Tests: `ChromaKeyDetectorTests` (pure) + `ChromaKeyApplyTests` (pipe asks ffmpeg to key; offer →
accept → keyed session feeds the resolver; survives save). App build 0 warnings; full suite green
apart from the two documented pre-existing failures.
## Next action
- Next queued unit: TASK 49 (chat profanity filter). Day 3's TASK 32/33 resilience slices are still
outstanding — 32·4 (fragmented MP4) is the smallest.
- Agreed next feature work (designed, not started): green/blue screen detection + keying through the clip
pipe — suggest-only by default, an opt-in "Auto Green/Blue Screen" setting, alert clips included.
- TASK 64 slice F: the **"Auto Green/Blue Screen"** opt-in setting (App Settings, high confidence
only) and wiring alert clips to the same settings block.
- Then TASK 49 (chat profanity filter). Day 3's TASK 32/33 resilience slices remain outstanding —
32·4 (fragmented MP4) is the smallest.
- Do not push absent a checkpoint request.
## Landmines
@@ -23,7 +26,6 @@
- `LayerReorderPersistenceTests.RealMouseDrag_OnTheLayerList_PersistsTheReorder` injects physical
mouse input. Do not run it while the desktop is in use.
- Integration tests constructing `MainWindow` must set a temporary `LayoutPathOverride` and clear it in `finally`.
- Media sessions are keyed per **path**, not per layer: acquire through `EnsureMediaSession`, release
through `ReleaseMediaIfUnused`, never `AcquireAsync` twice for one path. Level/mute are therefore
per session too.
- Media sessions are keyed by **path + look** (`MainViewModel.MediaSessionKey`); never acquire by raw
path, and release through `ReleaseMediaIfUnused`.
- Bundled FFmpeg stays outside the single-file exe (`ExcludeFromSingleFile="true"`).