feat(webcam): resource lifecycle startup slice — poll-on-start, single-cam lock, persistent Layers alert
The creator couldn't add a webcam to Live (grayed app-wide) and nothing in the app explained why. Ground truth from the live DB: one Webcam identity AND one WebcamSceneConfig in the Chat scene — so the gray was the single-identity rule working, but the reason was unobservable. This slice makes the webcam a resource the app validates and locks, mirroring how OBS reserves its devices. At startup we enumerate the OS once (ValidateWebcamResourceStartupAsync, fired after LoadLayout, stored as WebcamStartupValidationTask for tests to await): - 0 webcams -> app runs on, layer inactive, no alarm - exactly 1 -> attempt CameraManager.AcquireAsync as an app-wide lock; on failure show a persistent red alert at the bottom of the Layers panel (WebcamLockAlert + Retry) that re-polls every 5s and clears itself the moment the camera locks, or on any first real frame - >=2 -> deliberately no auto-lock; camera selection belongs to the App Settings dialog (gear) — next slice CameraManager.IsRunning(deviceId) tells the pass a session already exists (started OR still starting) so loaded identity configs count as the lock and the pass never double-acquires. Test seams mirror LayoutPathOverride: CameraEnumeratorOverride / CameraFrameSourceFactoryOverride so the startup probe never touches real hardware under test. Good Dog test: WebcamStartupResourceTests x3 — single-cam locked + app runs on, zero-cams no-alarm/no-lock, lock-fails -> red alert -> Retry -> clears. Full suite 304/304, build 0 warnings, scope-check passed. Docs in-commit: TASKS Open items + ai.md Webcam section + HANDOFF rewrite. Also corrects the record: 'NVIDIA Broadcast opens the webcam exclusively' was a suspect-list claim (CameraConflictProbe reads process names only, no handles) — not restated as fact.
This commit is contained in:
+75
-70
@@ -1,62 +1,68 @@
|
||||
# HANDOFF — 2026-09-15 (webcam take fix 1 of 2 committed locally; slice-18 already local — verify both)
|
||||
# HANDOFF — 2026-09-17 (webcam resource lifecycle, startup slice; committed locally — no push)
|
||||
|
||||
## Branch / Commit State
|
||||
|
||||
`main` HEAD = **webcam reader-ladder fix** (committed LOCALLY with this handoff, NOT pushed — web/A/V
|
||||
work stays commit-local until greenlight). Below it: **slice-18 C4 composite cache** (also local, un-pushed),
|
||||
slice-17 overlapping capture readbacks, slice-16 capture conversion fix, slice-15 pacing fix, `c01206f`
|
||||
(composition capture), `b22d08e` (signed audio-sync, pushed). Working tree clean.
|
||||
`main` HEAD = **`94a934f` webcam reader-ladder fix** (local). This work unit sits un-pushed on top
|
||||
as a NEW commit (same bundle of web/A/V work — stays commit-local until greenlight). Below: slice-18
|
||||
C4 cache (`64a5a6d`), slice-17, slice-16, slice-15, `c01206f`, `b22d08e` (signed audio-sync, pushed).
|
||||
No push after this commit either — still awaiting the device take + user greenlight.
|
||||
|
||||
## ⚠️ Branding (2026-09-14, creator-corrected): product = **llamacasty**, internals = ytLive
|
||||
|
||||
The product is **llamacasty**; repo path, csproj `AssemblyName`/`RootNamespace`, DB/log paths
|
||||
(`%APPDATA%\ytLlive\...`), and most code names are the legacy **ytLive/ytLlive**. User-facing
|
||||
language says "llamacasty"; code/assembly/repo names stay ytLive. See `ai.md` → Brand.
|
||||
Product is **llamacasty**; repo path, csproj `AssemblyName`/`RootNamespace`, DB/log paths
|
||||
(`%APPDATA%\ytLlive\...`), most code names are legacy **ytLive/ytLlive**. User-facing text:
|
||||
"llamacasty" / UI labels use the catalog names ("Web Cam", "YouTube Chat", "Countdown Timer"…).
|
||||
|
||||
## ✅ Committed locally — webcam take fix 1 of 2: reader-output-subtype ladder
|
||||
## ✅ Committed locally — webcam resource lifecycle (startup slice)
|
||||
|
||||
**Incident (2026-09-15):** browser (msedge) grabbed the C920 → app died silently (last log line
|
||||
09:16:47 is the `MediaCapture.Failed` "device no longer present" rollback; then nothing — no
|
||||
AppDomain/Dispatcher handler fired → native WMF death). Fresh launches then showed NO webcam:
|
||||
`CameraManager: camera '…GLOBAL' failed: '… frame reader refused to start: OutputFormatNotSupported'`.
|
||||
**Incident review (the reason this exists):** the creator couldn't add a webcam to the Live scene
|
||||
(WebCam row grayed app-wide). Ground truth from `%APPDATA%\ytLlive\ytLlive.db` (NOT `layout.db`,
|
||||
which is a 0-byte legacy file): one `Webcam` identity (C920 `\?\USB#VID_046D&PID_082D…GLOBAL`) AND
|
||||
one `WebcamSceneConfig` — in the **Chat** scene. Under the single-identity rule that means Live's
|
||||
"+" menu is gray *by design*, but the app couldn't *say why*. Combined with the earlier crash/failure
|
||||
chain, the fix is a proper resource lifecycle:
|
||||
|
||||
**Diagnosis (startup.log, four sessions):** the camera's live media type is the variable. Under
|
||||
msedge + NVIDIA Broadcast contention our `SetMediaStreamPropertiesAsync` refuses ("file is being used
|
||||
by another process / CaptureMode is SharedReadOnly") so the camera keeps whoever's format: YUY2 640x480
|
||||
@07:47-08:52 → reader started fine (starved of frames while they held it; worked at 08:52 when they
|
||||
didn't); **MJPG 1920x1080 @19:01/19:05 → `CreateFrameReaderAsync(.., Bgra8)` + `StartAsync` refused
|
||||
with OutputFormatNotSupported** (MS-documented: an MJPG-active source can't be read as Bgra8 by a
|
||||
MediaFrameReader — it exposes NV12; the MJPG→BGRA converter isn't on the reader pipeline).
|
||||
- **Startup poll + tri-state** (`MainViewModel.ValidateWebcamResourceStartupAsync`, fired
|
||||
fire-and-forget right after `LoadLayout`): 0 webcams → app runs on, layer inactive, no alarm;
|
||||
**1 → attempt `CameraManager.AcquireAsync` as an app-wide LOCK**; **≥2 → no auto-lock** — selection
|
||||
belongs to the App Settings dialog (gear) — that's the next slice.
|
||||
- **Persistent red alert** (`WebcamLockAlert`, bottom of the Layers panel, + Retry button): shown when
|
||||
the one detected camera can't be locked; **re-polls every 5 s** (`_webcamLockPollTimer`) and clears
|
||||
the moment a lock succeeds, or on any first real frame (`OnCameraPreviewBitmapChanged`).
|
||||
- **`CameraManager.IsRunning(deviceId)`** — new accessor (session exists, started OR starting) so the
|
||||
pass never double-acquires a lock the loaded identity's configs already hold; a rolled-back (failed)
|
||||
session leaves the dictionary, so the pass retries cleanly.
|
||||
- **Test seams mirroring `LayoutPathOverride`:** `CameraEnumeratorOverride` /
|
||||
`CameraFrameSourceFactoryOverride` — the startup probe must NEVER touch real hardware under test
|
||||
(the reason the 09:15 tests never asserted camera state). **Good Dog test**
|
||||
`WebcamStartupResourceTests` x3: single-cam locked + app runs on; zero-cams no-alarm no-lock;
|
||||
lock-fails → red alert → Retry → clears. **304/304 green, build 0 warnings.**
|
||||
- **Attribution correction committed to `ai.md`:** "NVIDIA Broadcast opens the webcam
|
||||
exclusively" is a *suspect-list* claim — `CameraConflictProbe` reads running process NAMES only, no
|
||||
device handles; the contention symptoms fit shared-mode/bandwidth just as well. Do not restate it as
|
||||
fact (creator called this out).
|
||||
|
||||
**Fix** (`Services/MediaCaptureFrameSource.cs`): reader creation is now a per-candidate ladder —
|
||||
`ReaderSubtypeCandidates(activeSubtype)`: Bgra8 for uncompressed cameras (unchanged fast path);
|
||||
**NV12 then source-default for MJPG** (converted in `OnFrameArrived`, which already handled non-BGRA).
|
||||
Each candidate is allocated AND started under its **own catch** — WinRT answers an unsupported subtype
|
||||
with a THROW (`E_INVALIDARG`), not a status; the throw degrades to the next candidate instead of
|
||||
aborting acquisition (creator rule: WINRT resource-allocation calls wrap per-call; see
|
||||
`MyMistakes.md`). Rejections are logged + joined into the final error.
|
||||
|
||||
**Good Dog tests:** `ytLive.Tests/MediaCaptureFrameSourceTests.cs` x4 — MJPG never asked as Bgra8,
|
||||
case-insensitive, uncompressed keeps Bgra8 fast path, unknown/null → Bgra8. **301/301 green, clean
|
||||
build 0 warnings, scope-check passed** (2 code files + ai.md + HANDOFF + MyMistakes).
|
||||
**Task docs:** `TASKS.md` Open items + `ai.md` Webcam section updated to match.
|
||||
|
||||
## ⚠️ Open items
|
||||
|
||||
- **Webcam take fix 2 of 2 — the 09:16 crash is UNFIXED (hard, untested):** hypothesis — the app died
|
||||
natively (no managed log line) when `MediaCapture.Failed` fired mid-stream: RollbackSession runs
|
||||
`_ = SafeStopAsync` fire-and-forget → `StopAsync` disposes reader+capture while the frame-reader
|
||||
thread is mid-`TryAcquireLatestFrame`/`Marshal.Copy` (that call region sits OUTSIDE the frame's
|
||||
try/catch). Managed/unwrapped failures there surface via AppDomain handler (absent → native AV).
|
||||
NOT fixed in this change: no repro, no integration test, and a native WMF race isn't catchable --
|
||||
deferred per spin-guard. Record if a second occurrence shows a pattern.
|
||||
- **Can't re-add webcam (3rd symptom):** partly the same root cause (add → AcquireAsync fails → empty
|
||||
chip + toast). Note: if ANOTHER scene still holds a WebcamSceneConfig, `_webcam != null` persists and
|
||||
"Add → Webcam" stays disabled by the single-identity rule — use right-click "Show Webcam" instead.
|
||||
- **Device re-verify (two things, same take session):**
|
||||
1. slice-18 cache (unchanged verdict criteria): `cache NR/WH` hits dominate on TV holds, stalls gone,
|
||||
band fresh updates/s toward ~60 → else residual 24↔60 pulldown, content decision.
|
||||
2. webcam ladder: while msedge holds the camera, a fresh launch must show the webcam (or degrade to
|
||||
a clear chip, never a silent box); after closing the browser it must come up at 1080p30 MJPEG.
|
||||
- **Webcam take fix 2 of 2 — the 09:16 crash is UNFIXED (hard, untested):** hypothesis — native WMF
|
||||
death when `MediaCapture.Failed` fires mid-stream and `SafeStopAsync` (fire-and-forget) disposes
|
||||
reader+capture while the frame thread sits in `TryAcquireLatestFrame`/`Marshal.Copy` (outside the
|
||||
frame's try/catch). No repro, no integration test, native race not catchable — deferred per
|
||||
spin-guard. Record a second occurrence's pattern.
|
||||
- **Multi-camera slice (user-queued):** when ≥2 webcams exist, add the web-cam selection text +
|
||||
control to the **App Settings** dialog (gear icon, `OverlayHost` — left-click opens settings,
|
||||
right-click the context menu). Selecting + successfully allocating/locking activates the WebCam
|
||||
layer. Currently ≥2 cameras → nothing shows in settings.
|
||||
- **Static (+) catalog slice (user-queued):** all catalog rows always listed (Background, "Web Cam",
|
||||
"Countdown Timer", "Web Resource", "YouTube Chat", "YouTube Event"…), gray only for (1) system-level
|
||||
unavailability (persistent alert), (2) max already allocated, (3) not available for the current
|
||||
scene; tooltips must name the reason (e.g. "already in Chat"). Static labels only, no new code-behind.
|
||||
- **Device re-verify (one take session):** 1. slice-18 cache verdicts (NR/WH hits, stalls, band
|
||||
~60); 2. webcam: fresh launch with the browser holding the camera must show the webcam or a NAMED
|
||||
chip (now also the persistent red alert, and the alert must clear after the browser closes + retry);
|
||||
3. verify the Chat scene still owns the webcam (so Live's + is intentionally gray with a reason).
|
||||
- **No push yet** — after the take verdict, re-measure the clap offset (`/tmp/opencode/avsync.py`),
|
||||
then decide push with the user.
|
||||
|
||||
@@ -71,33 +77,32 @@ build 0 warnings, scope-check passed** (2 code files + ai.md + HANDOFF + MyMista
|
||||
## Landmines
|
||||
|
||||
- testhost shares startup.log — filter by time; the app also writes fake-device failures
|
||||
('test-camera', 'dev-physical-9') from the CameraManager unit tests.
|
||||
('test-camera', 'dev-physical-9') from CameraManager unit tests.
|
||||
- `cmd.exe /c "taskkill /F /IM ytLive.exe"` (WSL double-slashes mangle) before rebuilds — a live
|
||||
app process locks `ytLive.exe` and the apphost copy fails (MSB3021).
|
||||
app process locks `ytLive.exe` (MSB3021).
|
||||
- Build/tests: **Windows dotnet host** (`/mnt/c/Program Files/dotnet/dotnet.exe`). 0 warnings —
|
||||
only `./scripts/verify.sh "<files>"`'s clean build counts. Building `ytLive.csproj` alone does
|
||||
NOT rebuild `ytLive.Tests.dll` — run the Tests csproj before `vstest`. Known audio flake:
|
||||
`Mix_HonorsProviderGains_AndGameMute_KillsTheLoopback` (float precision) — re-run if it trips
|
||||
alone; it is unrelated to camera work.
|
||||
- Camera contention failure modes (startup.log): "no frames within 4s" = init OK but starved;
|
||||
"OutputFormatNotSupported" = reader subtype below the MJPG-active camera; "file is being used by
|
||||
another process / CaptureMode is SharedReadOnly" = our stream re-negotiation refused while others
|
||||
hold the device. CameraConflictProbe lists named processes that are merely RUNNING (msedge,
|
||||
NVIDIA Broadcast) — a suspect list, not handle evidence.
|
||||
- FramePump tests that assert per-frame CONTENT must pass a STABLE scene (`() => scene`) — the
|
||||
default NewPump scene is fresh-per-tick (ok for pacing tests, but it churns the C4 render
|
||||
signature and hides the cache). Cache-sensitive assertions also can't use a call-count resolver
|
||||
flip (the tick resolves twice: signature + render) — key alternation on `OutputIndex`.
|
||||
only `./scripts/verify.sh "<files>"`'s clean build counts. Running `ytLive.csproj` alone does NOT
|
||||
rebuild `ytLive.Tests.dll` — run the Tests csproj before `vstest`. Known audio flake:
|
||||
`Mix_HonorsProviderGains_AndGameMute_KillsTheLoopback` (float precision) — re-run if it trips alone;
|
||||
unrelated to camera work.
|
||||
- **Assert footguns in this repo's xUnit:** `Assert.Null(x, "msg")`/`Assert.Single(coll, "msg")` do
|
||||
NOT take a message — the 2-arg overloads mean something else (predicate/item). Empty message = 1 arg.
|
||||
- The webcam's real state lives in `%APPDATA%\ytLlive\ytLlive.db` (`layout.db` there is a 0-byte
|
||||
legacy file). sqlite3 at `/home/gramps/android-sdk/platform-tools/sqlite3`.
|
||||
- Contention failure modes (startup.log): "no frames within 4s" = init OK but starved;
|
||||
"OutputFormatNotSupported" = reader subtype below the MJPG-active camera; "file being used by
|
||||
another process / CaptureMode is SharedReadOnly" = re-negotiation refused while others hold the
|
||||
device. `CameraConflictProbe` = suspect list, not handle evidence.
|
||||
- FramePump tests asserting per-frame CONTENT need a STABLE scene; cache-sensitive assertions can't
|
||||
use a call-count resolver flip (tick resolves twice) — key alternation on `OutputIndex`.
|
||||
- ffmpeg/ffprobe: `/mnt/c/Program Files/Krita (x64)/bin/` with Windows paths.
|
||||
- `MyMistakes.md` has the WINRT resource-allocation RECIPE (new), freeze-audit RECIPE, A/V sync
|
||||
measurement recipe, deadline-pacing lessons, CoreMessaging DQ recipe, WGC-CLIP + slice blocks —
|
||||
grep before re-deriving. ai.md has the webcam reader-ladder note.
|
||||
- sqlite3 at `/home/gramps/android-sdk/platform-tools/sqlite3`.
|
||||
- `C:\tmpout` is for ffmpeg evidence artifacts (raw decodes / PNGs); keep them out of the repo.
|
||||
- `MyMistakes.md`: WINRT resource-allocation RECIPE, freeze-audit RECIPE, A/V sync recipe, deadline
|
||||
pacing, CoreMessaging DQ, WGC-CLIP + slice blocks. Grep before re-deriving.
|
||||
- `C:\tmpout` is for ffmpeg evidence artifacts; keep them out of the repo.
|
||||
|
||||
## Next step
|
||||
|
||||
Creator records a take on this build (backend test): (a) with the browser holding the camera — fresh
|
||||
launch must show the webcam or a named chip, then (b) close the browser and re-add the webcam to the
|
||||
live view — must come back at 1080p30 MJPEG; (c) the slice-18 cache verdicts from the same session
|
||||
(`cache NR/WH` hits, stall frequency, band audit). If (b) works: measure clap offset, decide push.
|
||||
The multi-camera App Settings selector slice (+ static catalog rows after), then the creator's take
|
||||
on this build. The alert is the new expected behavior to eyeball on the take: browser holds the
|
||||
camera → fresh launch → red alert in Layers (unless Chat's config grabbed it) that clears after a
|
||||
Retry once the browser closes.
|
||||
Reference in New Issue
Block a user