Commit Graph
7 Commits
Author SHA1 Message Date
gramps 5fcfa545e5 feat(65): the replay layer plays — once in Live, looping everywhere else
608 green, 0 warnings. Completes the layer half of TASK 65.

- **(+) → Replay Clip** adds an ordinary media layer with an app-owned path, one per scene, silent by
  default (`MediaMuted`) and looping only outside Live — the compositor needs no special case.
- **F10 saves → the Live layer plays it once, then hides** (`SessionCompleted` → invisible, guard
  cleared). Every other scene with a replay layer keeps looping the kept set, advancing on each clip's
  completion, wrapping at the end.
- **The guard the creator asked for:** while that Live replay is playing, **F10 is inert** with a plain
  explanation. That is what replaces the impossible "exclude the layer from its own clip" — with one
  encoder and two muxes the clip *is* the program, so the fix is to not clip a clip.

`ReplayLayerTests` drives the real VM: one layer per scene, muted by default, once-through in Live,
looping in BRB, and a stray F10 with clips off writes nothing.
2026-11-10 12:54:54 -08:00
gramps 242a13ca30 fix(media): a played-out clip can play again; play button beside the Video File label
Reported 2026-11-05: "when I click Loop, the video doesn't play in the preview pane". Two
causes, both real:

- `VideoClipPipe.StartAsync` bailed out once it had run once (`_run != null`), so a clip that
  reached EOF could never be started again — it held its last frame forever.
- Loop is only consulted at end-of-file by a LIVE decoder, so flipping it on a finished clip
  did nothing at all.

Now: the pipe reports running state and is restartable after a natural end; the manager marks a
session finished (keeping its last frame) and exposes `IsRunning`/`RestartAsync`; the layer shows
`IsMediaRunning`/`IsMediaFinished`; flipping Loop on a finished clip restarts it.

UI: `▶ Play` sits to the right of the **Video File** label and is enabled only while Loop is off
(creator ruling: with Loop on the clip already runs continuously, so there is nothing to start).
The panel says "Clip finished — holding the last frame." so a frozen picture explains itself.

Also: verify.sh no longer runs physical-input tests by default. That test moves the real cursor
(SetCursorPos + mouse_event); running it mid-gate hijacks the machine someone is using, which is
exactly how it felt. Opt in with PHYSICAL_TESTS=1 when the desktop is idle; the header documents
the scoped-run command for working hours.

Test: `MediaPlaybackRestartTests` (pipe restart, finished state, Play again, Loop-on-finished).
8 scoped tests green. FULL SUITE NOT RUN — the machine was in use; run scripts/verify.sh in a
quiet window before the next milestone.
2026-11-05 09:15:30 -08:00
gramps 7bb1207fb2 feat(chroma): opt-in auto key, and alert clips key like any other clip
TASK 64 slice F — the two things left after the feature worked by hand.

- App Settings → **Video clips → Auto Green/Blue Screen** (off by default). When on, a
  high-confidence detected screen is keyed as soon as the clip is added; the panel says
  "Removed automatically" so an automatic change is never silent. Medium confidence still
  asks, low says nothing — a wrong automatic key eats the picture.
- **A clip is a clip:** the alert box's own replacement clip keys through the same settings
  block (one alert box per layout, so its settings are the ones the next alert uses), and
  choosing an alert clip runs the same detector. The Green/Blue Screen panel now appears for
  a video layer *or* an alert box with a custom clip (`Source.ShowsChromaKey`).
- Spill ships as the creator-facing **Fringe** switch; nothing further to tune behind it.

Test: `ChromaKeyApplyTests` gains the auto-apply case (no click, still explicit in the panel)
and an alert-clip case (its pipe asks ffmpeg for colorkey+despill with the blue screen type).

Docs: TASK 64 complete in TASKS.md, known limits recorded in the task file, HANDOFF rewritten.
verify.sh: 546 passed, 0 failed, 0 warnings.
2026-11-05 08:20:33 -08:00
gramps ea24405ba2 feat(chroma): offer to remove a green or blue screen, and key it in the decode
TASK 64 slices D and E: the feature is usable end to end.

- Add a video file → one frame is probed (new `IChromaProbe`/`FfmpegChromaProbe`, one ffmpeg
  pass at 160x90, off the UI thread) and the layer gets an offer: "Looks like a green screen —
  Remove it", with the measured colour as a swatch.
- Accepting it turns the offer into settings (Amount / Softness / Fringe) that reach ffmpeg as
  `colorkey` + `despill` + `format=bgra`, so the key is computed in the pipe and both consumers
  (the shared preview bitmap and the compositor's frame) carry the same alpha.
- Media sessions are now keyed by **path + look** (`MainViewModel.MediaSessionKey`), so two
  layers with the same settings still share one decoder, and changing the look releases the old
  session and starts the new one. Removal, load reconciliation, loop and audio settings all move
  to the same key.
- Settings persist on the `Source` row (six ALTER TABLE guards + SELECT/INSERT), so a keyed layer
  comes back keyed.
- Panel section is plain creator words: Green/Blue Screen, Amount, Softness, Fringe.

Test: `ChromaKeyApplyTests` — the pipe asks ffmpeg to key (and does not when off); a green-screen
clip is offered, accepting it starts a keyed session, the layer's resolver then reads that session
rather than the old one; the setting survives a save.

Docs: TASK 64 D/E ticked with slice F (auto-apply setting + alert clips) left open; frame-pipeline
records the decode-option design; HANDOFF rewritten. verify.sh: 544 passed, 0 failed, 0 warnings.
2026-11-05 08:15:32 -08:00
gramps 2670ac022b feat(media): play the clip's soundtrack and share one decode pipe
A video file was picture-only: the decode pipe passed `-an`, so a starting-soon
or BRB clip with music played silent everywhere. Adds sound to every clip and
collapses the two decoders into one.

- `Services/VideoClipPipe.cs`: the shared pipe behind `MediaVideoSource` (TASK 21)
  and `AlertClipDecoder` (TASK 47) — video (`rawvideo bgra`, scaled) + audio
  (`f32le -ac 2 -ar 48000`, 50 ms chunks, byte-carry across reads), both paced to
  real time, loop restarts both so picture and sound stay together. Alert
  behaviour is unchanged.
- `MediaVideoSourceManager` applies the layer's Level/Mute per session and raises
  `AudioAvailable`; `MainViewModel` feeds it to the new
  `AudioMixer.EnqueueDesktopMediaAudio`, which joins the DESKTOP/GAME ring so the
  desktop fader and the mic auto-ducker apply exactly as they do for game audio
  and TRAX (creator ruling 2026-11-05: audio joins the group).
- Per-layer `MediaMuted` + existing `MediaVolume` surfaced as Mute/Level in the
  properties panel; `MediaMuted` persisted (ALTER TABLE guard + SELECT/INSERT).
- Level/mute are per SESSION because one decoder is shared per file path — the
  last layer touched wins; one decoder per layer would play the file twice.

Test: `MediaSourceAudioTests` — the pipe emits 50 ms chunks at 48k stereo, the
manager scales by level and drops when muted, and a media layer's sound moves the
desktop meter. Alert/decode suites unchanged and green.

Docs: task-21 closed on item 10, the "silent soundtrack" facts corrected in
frame-pipeline/output/overlays, HANDOFF rewritten. verify.sh green (529 passed;
the two documented pre-existing failures only).
2026-11-05 07:54:06 -08:00
gramps 77d7208bfa feat(media): add a video file layer that decodes into the scene
TASK 21's picker slice: the decoder, manager, pacing and loop mechanism were
already shipped but nothing ever acquired a session, so a media Source rendered
nothing. Adds "Video File" to the (+) menu (video-filtered OpenFileDialog),
one decode session per distinct path with a HasSession guard, release when the
last scene reference goes, and a post-load reconcile. Loop is a manager-level
per-path answer ("any live reference loops") applied before StartAsync and
updated live via SetLooping. Properties panel shows the file with Change… and
Loop. Volume stays unwired: media audio rides the desktop loopback, TRAX-style.

Reference: OBS's media source (file → decode → composite into the scene) is the
established answer for this feature; the lifecycle shape mirrors this repo's
WebView2/ScreenCapture registration rather than inventing a new one.

Test: ytLive.Tests/MediaSourceAcquisitionTests.cs — add starts one decoder and
frames reach the layer, loop flip reaches the decoder, remove stops it.

Docs: task-21 closed, TASKS row + Day-4 row + stale 3·17 row corrected,
frame-pipeline acquisition note, HANDOFF rewritten. verify.sh green.
2026-11-05 07:17:43 -08:00
gramps 071ec8be0c TASK 21 slice 1 step 4: wire media into resolver + preview routing
Wire the MediaVideoSourceManager into the live pipeline (new partial
ViewModels/MainViewModel.Media.cs mirrors Webcam/Background partials):
readonly _mediaManager assigned in the core ctor (decode at 1920x1080 master,
dispatcher-coalesced preview), OnMediaPreviewBitmapChanged adopts the shared
bitmap onto every IsMediaSource with a matching MediaPath, a
Source { IsMediaSource, MediaPath } case in ResolveOutputFrame feeds
GetLatestFrame, and the manager is disposed on shutdown.

Source.DisplaySource now routes VideoImageSource for MediaSource (Source.cs),
so media previews/canvas show decoded video.

Compositor needs no change: BlitContent UniformToFill-scales any frame to the
element rect. Decoder/manager/seam derived-work mirrors ScreenCaptureManager
(citation in commit for slice-step-3).

Test (one unit for this change, mirroring the live-capture case):
Source_DisplaySource_IsVideoImageSource_WhenMediaSource. 14/14 media +
DisplaySource tests pass, build 0 warnings. Docs (TASKS/HANDOFF/ai.md) updated.
2026-08-31 19:32:41 -07:00