gramps/LlamaCasty
0
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

TASK 51 slice 1: background-effects engine — bundled MediaPipe Selfie Segmenter (ONNX) + Remove (Unit: bg-effects engine)

Browse Source 
- Model: Google's official MediaPipe Selfie Segmenter (landscape, Apache-2.0 per model card),
  converted to ONNX by tools/selfie-seg/convert_tflite_to_onnx.py (custom op
  Convolution2DTransposeBias -> ConvTranspose) and verified against MediaPipe's own runtime
  (verify_against_mediapipe.py: max |diff| 5e-5, 100% binary agreement). Embedded resource, 430 KB.
- Services/BackgroundEffects: IPersonSegmenter seam, OnnxPersonSegmenter (onnxruntime 1.31.0 CPU,
  never throws), BackgroundEffectProcessor (None passthrough, Remove = alpha mask). Not wired yet.
- Tests: Remove integration test (fake segmenter) + bundled-model contract test vs MediaPipe
  reference mask. Full suite 407/408 — the one failure is the documented interactive-desktop flake
  RealMouseDrag_OnTheLayerList_PersistsTheReorder (passed in the run before; code path untouched).
- Docs: ai.md reversal + section + licensing guard, THIRD-PARTY-NOTICES 11-12, TASK 51, indexes,
  MyMistakes recipe, HANDOFF.

References: obs-backgroundremoval (approach only — ONNX Runtime on CPU, 2 threads; GPL-3.0, no code
used) https://github.com/royshil/obs-backgroundremoval ; MediaPipe image segmenter
https://ai.google.dev/edge/mediapipe/solutions/vision/image_segmenter
This commit is contained in:
gramps
2026-10-09 09:24:13 -07:00
parent f8dd1079f3
commit 9f3c154d84
20 changed files with 796 additions and 78 deletions
Download Patch File Download Diff File Expand all files Collapse all files
TASKS.md
+1 -1
View File
@@ -66,7 +66,7 @@
| 48 | Distribution & packaging: **MSIX + Store IAP** | 🔶 In progress — **✅ route decided 2026-09-27 (Store MSIX + Store IAP)**; **item 1 (bundle ffmpeg) SHIPPED 2026-09-28** | [`TASKS/task-48-distribution-msix.md`](TASKS/task-48-distribution-msix.md) |
| 49 | Chat profanity filter (local, opt-in, non-persistent, user word list) | ☐ Queued (2026-09-27) | [`TASKS/task-49-chat-profanity-filter.md`](TASKS/task-49-chat-profanity-filter.md) |
| 50 | One-button "Publish this video" (resumable upload of the local recording) | ☐ Queued (2026-09-28) | [`TASKS/task-50-publish-recording.md`](TASKS/task-50-publish-recording.md) |
| 51 | **Webcam background effects: Remove + Blur** | ☐ **v1 requirement** — booked 2026-10-08; next up now that the zoom box shipped (TASK 52) | [`TASKS/task-51-webcam-bg-effects.md`](TASKS/task-51-webcam-bg-effects.md) |
| 51 | **Webcam background effects: Remove + Blur** | 🔶 **v1 requirement — in progress**: slice 1 engine shipped 2026-10-09 (bundled MediaPipe Selfie Segmenter → ONNX, verified vs Google's runtime; Remove); next: Blur → config → camera hookup → UI | [`TASKS/task-51-webcam-bg-effects.md`](TASKS/task-51-webcam-bg-effects.md) |
| 52 | **Zoom box (anchored 2× magnifier)** | ✅ Done — shipped 2026-10-08 (3 creator corrections during the build: loupe → anchored capture over the full composite, drawn last → never shows the dim shading) | [`TASKS/task-52-zoom-box.md`](TASKS/task-52-zoom-box.md) |
| 53 | **Output-format verification: Shorts/TikTok geometry (9:16)** | ☐ Queued (2026-10-08) — output formats never tested end-to-end; pin + prove the vertical geometry | [`TASKS/task-53-output-formats.md`](TASKS/task-53-output-formats.md) |
| 54 | **Marcom round: free-forever positioning + merch storefront** | 🔶 In progress — llamacasty.com copy live 2026-10-08 (pricing stripped); merch POD + app About links queued | [`TASKS/task-54-marcom-free-merch.md`](TASKS/task-54-marcom-free-merch.md) |