TASK 51 slice 4: camera-path hookup — CameraManager renders per-scene effect variants (one segmentation per raw frame on a latest-frame-drop worker, raw path untouched), VM declares/adopts variants per config; separable upsampling cuts Remove 46->3 ms, Blur 80->18 ms (Debug, 720p) (Unit: bg-effects hookup)
This commit is contained in:
@@ -645,6 +645,24 @@ This replaces the old five-seeder cluster (`Seed{Starting,Brb,Ending,Chat}Backgr
|
||||
(0..1, default 0.5), persisted as two additive guarded columns on `WebcamSceneConfig`
|
||||
(`BackgroundEffect TEXT DEFAULT 'None'`, `BackgroundStrength REAL DEFAULT 0.5`, no version bump —
|
||||
the TASK 47 convention). `RemoveBackground`/`BlurBackground` are the radio-pill bools the UI binds.
|
||||
- **Hookup (slice 4) — the ONE processing point is `CameraManager`.** Effects are per scene but the
|
||||
camera session is shared, so the manager renders **variants** keyed by `WebcamEffectKey(effect,
|
||||
strength rounded to 0.01)`. `MainViewModel.SyncWebcamEffects()` declares the union of every
|
||||
config's key (`SetRequestedEffects`; unused variants are dropped so their CPU stops) and points each
|
||||
config's `VideoImageSource` at its variant's bitmap; it re-runs on load, add, remove, device swap,
|
||||
and any `BackgroundEffect`/`BackgroundStrength` change (configs are watched via `PropertyChanged`,
|
||||
which also `ScheduleSave`s). Per raw frame a background worker (latest-frame-drop, one at a time)
|
||||
segments ONCE and renders every variant; each variant keeps its own latest frame (the resolver:
|
||||
`GetLatestFrame(device, key)`) and WriteableBitmap (`EffectPreviewBitmapChanged` on first
|
||||
creation). **The raw path is untouched** — a slow/failed model never stalls the plain webcam — and
|
||||
an effect variant shows **nothing** until its first processed frame (Remove never flashes the
|
||||
real room). The compositor needed no change: its sampler already honours source alpha.
|
||||
- **Perf (synthetic 1280×720, Debug build):** segment 1.6 ms, Remove 3.3 ms, Blur 17.5 ms per frame
|
||||
(Release is several × faster). Pixel loops are row-parallel, capped at min(4, cores/2); the
|
||||
bilinear upsampling is separable (horizontal rows precomputed per mask/small row, one vertical lerp
|
||||
per pixel) — the first naive per-pixel 2-D bilinear cost 46/80 ms. `CameraManager` logs the real
|
||||
average once per session after 120 processed frames (`startup.log`: "background effects avg …").
|
||||
`MainViewModel.PersonSegmenterOverride` is the hermetic-test seam.
|
||||
- **Tests:** `BackgroundEffectEngineTests` — the Remove integration test (fake segmenter), and a
|
||||
model contract test that runs the BUNDLED model through the C# preprocessing on a fixture and
|
||||
matches the MediaPipe-produced reference mask (≥99.5% agreement; fixture = MediaPipe's public test
|
||||
|
||||
Reference in New Issue
Block a user