TASK 51 slice 5: webcam Background section in the Layers properties — Remove/Blur PillToggles (radio-exclusive) + Strength slider; strength changes never blank the webcam (new variant inherits the same-effect frame + bitmap). TASK 51 built (Unit: bg-effects UI)
This commit is contained in:
+33
-36
@@ -1,50 +1,47 @@
|
||||
# HANDOFF — 2026-10-09 (TASK 51 slice 1 shipped: background-effects engine)
|
||||
# HANDOFF — 2026-10-09 (TASK 51 webcam background Remove + Blur — BUILT)
|
||||
|
||||
## Branch
|
||||
- main (no feature branch). Pushed through `f8dd107`; the TASK 51 slice-1 commit sits on top,
|
||||
**not pushed** (push at the user's call).
|
||||
- main (no feature branch). Pushed through `f8dd107`; everything after is local, **not pushed**
|
||||
(push at the creator's call).
|
||||
|
||||
## Dirty files (NOT mine — do NOT commit or co-edit)
|
||||
## Dirty files (NOT mine — do NOT commit)
|
||||
- The in-flight "pinned chat" work unit: `Models/YouTube.cs`, `ViewModels/MainViewModel.Chat.cs`,
|
||||
`Controls/LeftPanel.xaml`, untracked `Helpers/RelayCommandGeneric.cs`.
|
||||
untracked `Helpers/RelayCommandGeneric.cs`, and **its hunks in `Controls/LeftPanel.xaml`**
|
||||
(lines ~599–610, the chat area). TASK 51 slice 5 edited the SAME file in a different region (the
|
||||
webcam properties, ~line 287); only the TASK 51 hunk was staged (`git apply --cached` of that hunk
|
||||
alone) — the pinned-chat hunks are still unstaged and untouched.
|
||||
- The creator's untracked logo art in `Assets/` (`llamacasty text.*`, `llamacasty-text-nobg*.png`,
|
||||
`thin logo wordmark.png`).
|
||||
|
||||
## What just happened — TASK 51 slice 1 (engine)
|
||||
- **Model:** Google's official MediaPipe Selfie Segmenter (landscape, 144×256, Apache-2.0),
|
||||
converted to ONNX by our own `tools/selfie-seg/convert_tflite_to_onnx.py` and **verified against
|
||||
MediaPipe's own runtime** (`verify_against_mediapipe.py`, max |diff| 5e-5). Embedded in the exe
|
||||
(`Assets/Models/selfie_segmenter_landscape.onnx`, 430 KB, in git).
|
||||
- **Code:** `Services/BackgroundEffects/` — `IPersonSegmenter` seam, `OnnxPersonSegmenter`
|
||||
(onnxruntime 1.31.0 CPU, never throws), `BackgroundEffectProcessor` (None passthrough, Remove =
|
||||
alpha mask, Blur = passthrough until slice 2). `Models/BackgroundEffect.cs` enum.
|
||||
- **Not wired:** nothing calls it yet — no config, UI, or camera hookup. Zero risk to the live path.
|
||||
- **Docs:** ai.md (stale "background removal PERMANENTLY OUT" bullet replaced by the reversal + a
|
||||
"Webcam background effects" section + a model-provenance licensing guard), THIRD-PARTY-NOTICES
|
||||
items 11 (ONNX Runtime, MIT) + 12 (MediaPipe model, Apache-2.0 with §4b change note),
|
||||
TASK 51 file + TASKS.md row, Services/Models indexes, MyMistakes conversion recipe.
|
||||
## What just happened — TASK 51, all five slices
|
||||
1. **Engine** — Google's MediaPipe Selfie Segmenter (landscape, Apache-2.0) converted to ONNX by
|
||||
`tools/selfie-seg/` and verified vs MediaPipe's own runtime; embedded resource; onnxruntime CPU.
|
||||
2. **Blur** — background at ¼ res, 3 box passes, mask-blended; Remove gets edge feather.
|
||||
3. **Config** — `WebcamSceneConfig.BackgroundEffect` + `BackgroundStrength`, persisted (additive cols).
|
||||
4. **Hookup** — `CameraManager` renders per-scene effect **variants** (one segmentation per raw frame
|
||||
on a latest-frame-drop worker; raw path untouched); VM `SyncWebcamEffects()` declares/adopts them.
|
||||
Synthetic 720p Debug: segment 1.6 ms, Remove 3.3 ms, Blur 17.5 ms.
|
||||
5. **UI** — Background section in the webcam's Layers properties: Remove / Blur `PillToggle`s,
|
||||
radio-exclusive, + Strength slider (visible while an effect is on).
|
||||
Also this session: 64-bit requirement (`PlatformTarget=x64`), GPL-3.0-or-later licensing, README
|
||||
rewrite, TASK 55 (disk hygiene) + TASK 56 (secret scan before going public) queued.
|
||||
|
||||
## Tests
|
||||
- Full suite **408 passed / 0 failed**, 0 build warnings. New: `BackgroundEffectEngineTests` (Remove
|
||||
integration test with a fake segmenter; bundled-model contract test vs a MediaPipe reference mask,
|
||||
fixtures in `ytLive.Tests/Fixtures/`).
|
||||
- Full suite **412 passed / 0 failed**, 0 build warnings. New: `BackgroundEffectEngineTests` (Remove,
|
||||
Blur, bundled-model contract), `CameraManagerTests` (variant reaches compositor + preview, raw
|
||||
stays raw), `LayoutStorePersistenceTests` (per-scene roundtrip + legacy migration),
|
||||
`WebcamBackgroundPillTests` (radio semantics + XAML wiring).
|
||||
- Known environment flakes (not regressions): `Mix_HonorsProviderGains…` (fails while system audio
|
||||
plays), `RealMouseDrag…` (needs an idle desktop).
|
||||
|
||||
## Known issues / follow-ups
|
||||
- **64-bit only (2026-10-09):** `PlatformTarget=x64` in both csproj files (verified: ytLive.dll PE
|
||||
machine 0x8664, 408/408 green). TASK 48 item 2 carries the x64-only MSIX + `-r win-x64` publish.
|
||||
- `onnxruntime.dll` is ~17 MB; an RID-less build copies every RID's native folder — TASK 48's MSIX
|
||||
publish must pin win-x64.
|
||||
- **License: GPL-3.0-or-later** (2026-10-09: `LICENSE`, `ADDITIONAL-TERMS.md` with §7 b/c/e, icon +
|
||||
logo all rights reserved, AI-generated backgrounds/alert video = GPL placeholders; both files ship
|
||||
beside the exe). Source offer = `https://llgit.llamachile.tube/gramps/LlamaCasty` (Gitea; the
|
||||
creator toggles it public), in the notices header + README. **Before it goes public: scan git
|
||||
history for secrets.** README rewritten (creator's voice, generic comparison, real build steps).
|
||||
Still stale and NOT yet rewritten: ai.md "Windows packaging" (Store IAP / branding-flash paid
|
||||
delta) and "Monetization" sections, MARCOM.md asset table (`llama-logo.png` doesn't exist).
|
||||
- **Creator's real-camera check (TASK 51):** toggle Remove/Blur on a webcam, watch edges/lighting,
|
||||
and read `startup.log` → "background effects avg … ms/frame".
|
||||
- `onnxruntime.dll` ~17 MB; TASK 48's MSIX publish must pin `-r win-x64` (x64-only already enforced).
|
||||
- Still stale, not rewritten: ai.md "Windows packaging" (Store IAP / branding-flash paid delta) and
|
||||
"Monetization" sections; MARCOM.md asset table (`llama-logo.png` doesn't exist).
|
||||
- TASK 56: secret-scan the full history BEFORE the Gitea repo (`gramps/LlamaCasty`) goes public.
|
||||
- Carried: "Use default background" toggle still not working (from 2026-10-07).
|
||||
|
||||
## Next step
|
||||
- TASK 51 slice 2: **Blur** in `BackgroundEffectProcessor` (separable box/gaussian, strength), one
|
||||
integration test. Then slice 3 config + persistence (`BackgroundEffect` + `Strength`), slice 4
|
||||
camera-path hookup + perf check on the creator's box, slice 5 UI: radio-exclusive Remove/Blur pill
|
||||
switches + strength slider in the webcam's Layers properties.
|
||||
- Creator runs the app: webcam → Layers properties → Background → Remove / Blur + Strength.
|
||||
|
||||
Reference in New Issue
Block a user