Commit Graph
4 Commits
Author SHA1 Message Date
gramps 305706956e TASK 51 jitter fix: temporal mask smoothing. MaskSmoother blends each raw segmentation 0.75xnew + 0.25xprevious at the one CameraManager worker (state per CameraSession, not in the shared segmenter); null resets history, first frame passes through, input array never aliased, ping-pong buffers (no LOH alloc for a 147KB 256x144 mask). Creator's real-camera check: Remove was 'jittery' (per-frame independent segmentation). Weights from the OBS/Jitsi consensus. One integration test asserts the second frame keeps partial alpha (x=6 -> 58) where the raw mask cuts to 0. 416 pass / 2 known env flakes. Refs: https://deepwiki.com/royshil/obs-backgroundremoval/2-core-features https://github.com/jitsi/jitsi-meet/issues/17080 (Unit: bg-removal jitter fix) 2026-10-09 22:15:59 -07:00
gramps de22983b36 TASK 51 slice 4: camera-path hookup — CameraManager renders per-scene effect variants (one segmentation per raw frame on a latest-frame-drop worker, raw path untouched), VM declares/adopts variants per config; separable upsampling cuts Remove 46->3 ms, Blur 80->18 ms (Debug, 720p) (Unit: bg-effects hookup) 2026-10-09 10:27:38 -07:00
gramps 227ee8fca6 TASK 51 slice 2: Blur — background at 1/4 res, 3 separable box passes (~gaussian, radius by strength), mask-blended; person untouched; Remove gains strength = edge feather; mask computed once, rendered per effect (Unit: bg blur) 2026-10-09 10:18:00 -07:00
gramps 9f3c154d84 TASK 51 slice 1: background-effects engine — bundled MediaPipe Selfie Segmenter (ONNX) + Remove (Unit: bg-effects engine)
- Model: Google's official MediaPipe Selfie Segmenter (landscape, Apache-2.0 per model card),
  converted to ONNX by tools/selfie-seg/convert_tflite_to_onnx.py (custom op
  Convolution2DTransposeBias -> ConvTranspose) and verified against MediaPipe's own runtime
  (verify_against_mediapipe.py: max |diff| 5e-5, 100% binary agreement). Embedded resource, 430 KB.
- Services/BackgroundEffects: IPersonSegmenter seam, OnnxPersonSegmenter (onnxruntime 1.31.0 CPU,
  never throws), BackgroundEffectProcessor (None passthrough, Remove = alpha mask). Not wired yet.
- Tests: Remove integration test (fake segmenter) + bundled-model contract test vs MediaPipe
  reference mask. Full suite 407/408 — the one failure is the documented interactive-desktop flake
  RealMouseDrag_OnTheLayerList_PersistsTheReorder (passed in the run before; code path untouched).
- Docs: ai.md reversal + section + licensing guard, THIRD-PARTY-NOTICES 11-12, TASK 51, indexes,
  MyMistakes recipe, HANDOFF.

References: obs-backgroundremoval (approach only — ONNX Runtime on CPU, 2 threads; GPL-3.0, no code
used) https://github.com/royshil/obs-backgroundremoval ; MediaPipe image segmenter
https://ai.google.dev/edge/mediapipe/solutions/vision/image_segmenter
2026-10-09 09:24:13 -07:00