Files
LlamaCasty/HANDOFF.md
T
gramps 94a934ffa9 fix(webcam): reader-output-subtype ladder + record WinRT per-call wrap rule
A browser grabbing the C920 kills our reader startup: the camera stays in
MJPG 1920x1080@30 (another app's format) and our OBS-refusal to re-negotiate
under SharedReadOnly contention (`SetMediaStreamPropertiesAsync` throws
'file in use / CaptureMode is SharedReadOnly') leaves it there.
CreateFrameReaderAsync(.., Bgra8) + StartAsync then refuses with
OutputFormatNotSupported — the MJPG-active source only exposes NV12 at the
reader level (clue: startup.log 19:01/19:05 sessions, same hardware that
started YUY2 640x480 fine at 08:52). Fresh launches showed no webcam and
'Add Webcam' failed.

Reader creation is now a per-candidate ladder (ReaderSubtypeCandidates):
Bgra8 for uncompressed cameras (unchanged fast path); NV12 then the
source-default for MJPG cameras — converted in OnFrameArrived like any
non-BGRA frame. Each candidate is allocated AND started under its OWN
catch: WinRT answers an unsupported subtype with a throw (E_INVALIDARG),
not a status, so a single rejected format must degrade to the next
candidate instead of aborting acquisition (creator rule — see MyMistakes
WINRT resource-allocation recipe). Rejections are logged and collected
into the final error.

Good Dog: 4 unit tests lock the candidate ordering (MJPG never Bgra8,
case-insensitive, uncompressed keeps Bgra8 first, unknown/null -> Bgra8).
301/301 green, 0 warnings. [no push]
2026-09-15 19:25:39 -07:00

7.4 KiB
Raw Blame History

HANDOFF — 2026-09-15 (webcam take fix 1 of 2 committed locally; slice-18 already local — verify both)

Branch / Commit State

main HEAD = webcam reader-ladder fix (committed LOCALLY with this handoff, NOT pushed — web/A/V work stays commit-local until greenlight). Below it: slice-18 C4 composite cache (also local, un-pushed), slice-17 overlapping capture readbacks, slice-16 capture conversion fix, slice-15 pacing fix, c01206f (composition capture), b22d08e (signed audio-sync, pushed). Working tree clean.

⚠️ Branding (2026-09-14, creator-corrected): product = llamacasty, internals = ytLive

The product is llamacasty; repo path, csproj AssemblyName/RootNamespace, DB/log paths (%APPDATA%\ytLlive\...), and most code names are the legacy ytLive/ytLlive. User-facing language says "llamacasty"; code/assembly/repo names stay ytLive. See ai.md → Brand.

✅ Committed locally — webcam take fix 1 of 2: reader-output-subtype ladder

Incident (2026-09-15): browser (msedge) grabbed the C920 → app died silently (last log line 09:16:47 is the MediaCapture.Failed "device no longer present" rollback; then nothing — no AppDomain/Dispatcher handler fired → native WMF death). Fresh launches then showed NO webcam: CameraManager: camera '…GLOBAL' failed: '… frame reader refused to start: OutputFormatNotSupported'.

Diagnosis (startup.log, four sessions): the camera's live media type is the variable. Under msedge + NVIDIA Broadcast contention our SetMediaStreamPropertiesAsync refuses ("file is being used by another process / CaptureMode is SharedReadOnly") so the camera keeps whoever's format: YUY2 640x480 @07:47-08:52 → reader started fine (starved of frames while they held it; worked at 08:52 when they didn't); MJPG 1920x1080 @19:01/19:05 → CreateFrameReaderAsync(.., Bgra8) + StartAsync refused with OutputFormatNotSupported (MS-documented: an MJPG-active source can't be read as Bgra8 by a MediaFrameReader — it exposes NV12; the MJPG→BGRA converter isn't on the reader pipeline).

Fix (Services/MediaCaptureFrameSource.cs): reader creation is now a per-candidate ladder — ReaderSubtypeCandidates(activeSubtype): Bgra8 for uncompressed cameras (unchanged fast path); NV12 then source-default for MJPG (converted in OnFrameArrived, which already handled non-BGRA). Each candidate is allocated AND started under its own catch — WinRT answers an unsupported subtype with a THROW (E_INVALIDARG), not a status; the throw degrades to the next candidate instead of aborting acquisition (creator rule: WINRT resource-allocation calls wrap per-call; see MyMistakes.md). Rejections are logged + joined into the final error.

Good Dog tests: ytLive.Tests/MediaCaptureFrameSourceTests.cs x4 — MJPG never asked as Bgra8, case-insensitive, uncompressed keeps Bgra8 fast path, unknown/null → Bgra8. 301/301 green, clean build 0 warnings, scope-check passed (2 code files + ai.md + HANDOFF + MyMistakes).

⚠️ Open items

  • Webcam take fix 2 of 2 — the 09:16 crash is UNFIXED (hard, untested): hypothesis — the app died natively (no managed log line) when MediaCapture.Failed fired mid-stream: RollbackSession runs _ = SafeStopAsync fire-and-forget → StopAsync disposes reader+capture while the frame-reader thread is mid-TryAcquireLatestFrame/Marshal.Copy (that call region sits OUTSIDE the frame's try/catch). Managed/unwrapped failures there surface via AppDomain handler (absent → native AV). NOT fixed in this change: no repro, no integration test, and a native WMF race isn't catchable -- deferred per spin-guard. Record if a second occurrence shows a pattern.
  • Can't re-add webcam (3rd symptom): partly the same root cause (add → AcquireAsync fails → empty chip + toast). Note: if ANOTHER scene still holds a WebcamSceneConfig, _webcam != null persists and "Add → Webcam" stays disabled by the single-identity rule — use right-click "Show Webcam" instead.
  • Device re-verify (two things, same take session):
    1. slice-18 cache (unchanged verdict criteria): cache NR/WH hits dominate on TV holds, stalls gone, band fresh updates/s toward ~60 → else residual 24↔60 pulldown, content decision.
    2. webcam ladder: while msedge holds the camera, a fresh launch must show the webcam (or degrade to a clear chip, never a silent box); after closing the browser it must come up at 1080p30 MJPEG.
  • No push yet — after the take verdict, re-measure the clap offset (/tmp/opencode/avsync.py), then decide push with the user.

Open threads (carried)

  • Audio-silence verification — fixed (724af14); creator heard real audio.
  • Webcam MJPG missing / ~10–14Hz, layer SortOrder, truncation-with-dynamic-scenes — queued.
  • Sync control user-doc tutorial — REQUIRED before 1.0 (creator directive; TASK 22).
  • Signed A/V sync: verify the negative (advance) direction on device.
  • Focus-loss capture lag — closed as NOT the cause (1824: delivery healthy ~60-100/s).

Landmines

  • testhost shares startup.log — filter by time; the app also writes fake-device failures ('test-camera', 'dev-physical-9') from the CameraManager unit tests.
  • cmd.exe /c "taskkill /F /IM ytLive.exe" (WSL double-slashes mangle) before rebuilds — a live app process locks ytLive.exe and the apphost copy fails (MSB3021).
  • Build/tests: Windows dotnet host (/mnt/c/Program Files/dotnet/dotnet.exe). 0 warnings — only ./scripts/verify.sh "<files>"'s clean build counts. Building ytLive.csproj alone does NOT rebuild ytLive.Tests.dll — run the Tests csproj before vstest. Known audio flake: Mix_HonorsProviderGains_AndGameMute_KillsTheLoopback (float precision) — re-run if it trips alone; it is unrelated to camera work.
  • Camera contention failure modes (startup.log): "no frames within 4s" = init OK but starved; "OutputFormatNotSupported" = reader subtype below the MJPG-active camera; "file is being used by another process / CaptureMode is SharedReadOnly" = our stream re-negotiation refused while others hold the device. CameraConflictProbe lists named processes that are merely RUNNING (msedge, NVIDIA Broadcast) — a suspect list, not handle evidence.
  • FramePump tests that assert per-frame CONTENT must pass a STABLE scene (() => scene) — the default NewPump scene is fresh-per-tick (ok for pacing tests, but it churns the C4 render signature and hides the cache). Cache-sensitive assertions also can't use a call-count resolver flip (the tick resolves twice: signature + render) — key alternation on OutputIndex.
  • ffmpeg/ffprobe: /mnt/c/Program Files/Krita (x64)/bin/ with Windows paths.
  • MyMistakes.md has the WINRT resource-allocation RECIPE (new), freeze-audit RECIPE, A/V sync measurement recipe, deadline-pacing lessons, CoreMessaging DQ recipe, WGC-CLIP + slice blocks — grep before re-deriving. ai.md has the webcam reader-ladder note.
  • sqlite3 at /home/gramps/android-sdk/platform-tools/sqlite3.
  • C:\tmpout is for ffmpeg evidence artifacts (raw decodes / PNGs); keep them out of the repo.

Next step

Creator records a take on this build (backend test): (a) with the browser holding the camera — fresh launch must show the webcam or a named chip, then (b) close the browser and re-add the webcam to the live view — must come back at 1080p30 MJPEG; (c) the slice-18 cache verdicts from the same session (cache NR/WH hits, stall frequency, band audit). If (b) works: measure clap offset, decide push.