63 lines
4.8 KiB
Markdown
63 lines
4.8 KiB
Markdown
# HANDOFF — 2026-10-09 (TASK 51 quality pass: jitter FIXED; blur-heavy is the next unit)
|
||
|
||
## Branch
|
||
- main (no feature branch). Pushed through `5685c5d` (TRAX toggle); everything after is local,
|
||
**not pushed** (push at the creator's call).
|
||
|
||
## Dirty files — split by ownership
|
||
- **This work unit (COMMIT these):** `Services/BackgroundEffects/MaskSmoother.cs` (new), the smoother
|
||
wiring in `Services/CameraManager.cs`, `ytLive.Tests/CameraManagerTests.cs` (new integration test),
|
||
and the same-commit memory updates (`Services/index.md`, `ai.md`, `TASKS/task-51-webcam-bg-effects.md`,
|
||
`TASKS.md` row 51, and this file) — all ship in the SAME commit.
|
||
- **NOT mine — do NOT commit:** the in-flight "pinned chat" work unit (`Models/YouTube.cs`,
|
||
`ViewModels/MainViewModel.Chat.cs`, untracked `Helpers/RelayCommandGeneric.cs`, and **its hunks in
|
||
`Controls/LeftPanel.xaml`** — only the webcam-properties region is TASK 51; the pinned-chat hunks
|
||
stayed unstaged/untouched), plus the creator's untracked `Assets/llamacasty*` art.
|
||
- ⚠️ Pre-commit: `scripts/verify.sh` runs `scripts/scope-check.sh` against ALL working-tree changes —
|
||
the not-mine files WILL flag. Pass them to the script as declared-known (checksums unchanged since
|
||
`5685c5d`) or let the script's output be reconciled before committing.
|
||
|
||
## What just happened — temporal jitter fix
|
||
- Creator check: bg Remove is "jittery", Blur is "heavy". User picked **jitter first**.
|
||
- Root cause: `OnnxPersonSegmenter` classifies each frame independently (256×144 mask, bilinear
|
||
upsample) → zero temporal coherence → edge flutter. There is no temporal smoothing upstream.
|
||
- **Fix: `MaskSmoother`** — per-session blend of each raw mask with the previous smoothed one,
|
||
0.75×new + 0.25×previous (`CurrentWeight`, internal). Wired at the ONE production point
|
||
(`CameraManager.RunEffectWorker` → `session.Smoother.Smooth(TrySegment(source))`); state on
|
||
`CameraSession`, NOT the segmenter (segmenter is shared app-wide and contract-pinned to raw output).
|
||
Null mask resets history; first frame/size-change passthrough; never aliases the caller's array; two
|
||
ping-pong buffers, no per-frame LOH alloc. Weights: OBS obs-backgroundremoval 0.85 default test, Jitsi
|
||
jitsi/jitsi-meet#17080 0.7/0.3 best (0.5/0.5 smears on motion).
|
||
- **Test:** `CameraManagerTests.Temporal_Smoothing_Damps_Segmentation_Jitter_Across_Frames` — PASSED.
|
||
A `SteppingMaskSegmenter` returns fresh masks per call: `{1,1,0,0}` then `{1,0.2,0,0}`; 16×4 frame,
|
||
Remove 0.5. Asserts x=6: frame1 = 255 (passthrough), frame2 smoothed = **58**, raw-unsmoothed = 0;
|
||
person core x=1 = 255, background x=14 = 0. ⚠️ Gotcha: the worker's latest-frame-drop guard treats a
|
||
re-queued IDENTICAL frame reference as "no new frame" — pump frame2 as a NEW `VideoFrame` instance.
|
||
- `Assert.Equal(255, first[...])`/`58`/`0` predictions were hand-verified; other pixels differ by
|
||
float rounding (e.g. x=5 Python 34 vs C# 36) — only assert x=6/x=1/x=14.
|
||
|
||
## Next step
|
||
1. Run `./scripts/verify.sh` with the declared file list; reconcile the not-mine dirty files with
|
||
scope-check; commit the unit (message `…(Unit: bg-removal jitter fix)` + OBS/Jitsi citations).
|
||
2. Next unit = **Blur "heavy"** — soften the radius curve in `BackgroundEffectProcessor.ApplyBlur`
|
||
(`MinBlurRadius=1 / MaxBlurRadius=12` at ¼ res ≈ 26–50 full px; strength 0.5 → 6 small px). The
|
||
creator needs a retry pass on the real camera after both.
|
||
|
||
## Also still open
|
||
- **Vertical-format regression (STALE, not fixed):** 1080p60 vertical selection no longer shrinks the
|
||
capture border in the preview. Mechanism mapped (see prior summary): the crop Rectangle
|
||
`Controls/PreviewPane.xaml:319-322` bound to `OutputRectX/Y/W/H` + `IsOutputCropped`, driven by
|
||
`MainViewModel.Streaming.cs` `ApplyStreamQuality` from `SelectedQuality` (binding + ctor only,
|
||
`new("Vertical 1080p60", 60, 8.0, 1080, 1920)`), `DimRects` in `MainViewModel.cs:91-94`. Root cause
|
||
unknown — paused.
|
||
- **Vertical recording file closed:** the recording itself is 1080×1920 60fps correctly; the
|
||
"whitespace" was the app's preview UI, not the file. Audio loudness (`ebur128`) for the private video
|
||
was never run and the thread is closed unless the user reopens it.
|
||
- Blur-heaviness = next unit (above). Yellow-border regression = stale thread.
|
||
- ⛔ Sudo password rotation pending (user; then update both password lines in `CREDENTIALS.md`).
|
||
- TASK 56 secret scan before `gramps/LlamaCasty` goes public. TASK 55 disk hygiene queued.
|
||
|
||
## Tests
|
||
- Full suite **416 passed / 1 known env flake** at `5685c5d`; the new smoothing test passes
|
||
standalone (1 test run, Duration < 1 s after warm). 0 build warnings. Known environment flakes:
|
||
`Mix_HonorsProviderGains…` (fails while system audio plays), `RealMouseDrag…` (needs an idle desktop). |