Commit Graph
12 Commits
Author SHA1 Message Date
gramps 55ca1029a3 UNIT 0: make a dropped webcam frame measurable, not just inferable
The creator is DHH and lipreads their own recordings, so a webcam that
judders is an accessibility defect, not a polish item. Take
ty-20261005-1001-821-0001.mp4 holds 25.6 distinct webcam images/s against
the camera's 30fps with 281 gaps of 50-100ms - roughly one hitch in five -
in a file that muxes flawlessly. Nothing in the app could see that: rawvideo
carries no per-frame timestamps (ffmpeg stamps by ARRIVAL at the declared
60fps), and the stats line divided every average by the number of frames
SUBMITTED, which deadline padding always fills to the full target. So the
pump reported "300/300 frames, avg render 12.4ms" while producing 60
distinct composites.

Helpers/FrameCadence.cs is one pure counter fed from three points:
MediaCaptureFrameSource logs the device's own SystemRelativeTime stamps
against our arrival times (TryAcquireLatestFrame keeps only the LATEST
frame, so falling behind is silent by design), and FramePump logs the
distinct webcam Epoch values that reached a composite. Read together they
name the culprit - device/USB, our reader, or our compositor - instead of
guessing. Render/resolve/wait now divide by the real render count, and
"frames per 5s from R renders" is the duplicate-on-lag ratio stated
honestly.

An out-of-order timestamp counts as a frame but produces no gap and does not
become the next interval's reference, so one bad stamp cannot invent a
multi-second hitch twice.

Also corrects a stale ai.md line (the "webcam renders at ~18fps" figure
came from a different session and counted every newly rendered layer) and
re-records the background-removal ruling as permanently out. Records the
agreed-but-unbuilt Units 1-2, the no-fallback-camera ruling, and the
SetFormatAsync dead end found by probing the 19041 projection.

Suite 402/402, 0 warnings. The one red, LayerReorderPersistenceTests'
RealMouseDrag, fails identically on a stashed clean tree - it needs an
interactive desktop session this shell lacks.
2026-10-05 11:10:22 -07:00
gramps 94a934ffa9 fix(webcam): reader-output-subtype ladder + record WinRT per-call wrap rule
A browser grabbing the C920 kills our reader startup: the camera stays in
MJPG 1920x1080@30 (another app's format) and our OBS-refusal to re-negotiate
under SharedReadOnly contention (`SetMediaStreamPropertiesAsync` throws
'file in use / CaptureMode is SharedReadOnly') leaves it there.
CreateFrameReaderAsync(.., Bgra8) + StartAsync then refuses with
OutputFormatNotSupported — the MJPG-active source only exposes NV12 at the
reader level (clue: startup.log 19:01/19:05 sessions, same hardware that
started YUY2 640x480 fine at 08:52). Fresh launches showed no webcam and
'Add Webcam' failed.

Reader creation is now a per-candidate ladder (ReaderSubtypeCandidates):
Bgra8 for uncompressed cameras (unchanged fast path); NV12 then the
source-default for MJPG cameras — converted in OnFrameArrived like any
non-BGRA frame. Each candidate is allocated AND started under its OWN
catch: WinRT answers an unsupported subtype with a throw (E_INVALIDARG),
not a status, so a single rejected format must degrade to the next
candidate instead of aborting acquisition (creator rule — see MyMistakes
WINRT resource-allocation recipe). Rejections are logged and collected
into the final error.

Good Dog: 4 unit tests lock the candidate ordering (MJPG never Bgra8,
case-insensitive, uncompressed keeps Bgra8 first, unknown/null -> Bgra8).
301/301 green, 0 warnings. [no push]
2026-09-15 19:25:39 -07:00
gramps 724af1499b fix: audio silence (idempotent sync-delay configure), webcam gray block, truncated videos
- AudioSyncDelay.Configure reallocated/zeroed its buffer every ~10ms tick
  (AudioMixer re-reads the UI setting each mix), so any non-zero sync offset
  erased the just-written audio -> total silence. Now early-returns when the
  delay samples are unchanged. Regression test proven both ways.
- SceneCompositor.BlitContentRaw defaulted cbW/cbH=0 when CropBounds is null
  (regression from ed9d7c1) -> webcam blit to an empty rect = gray block.
  Default to src.Width/Height. Regression test proven both ways.
- Truncated recordings: rawvideo mux stamps frames at declared 60fps by
  arrival; a scene whose first layer is dynamic (hidden elements still count)
  kills the bake cache -> full render ~35ms -> ~27fps submitted -> halved
  file length. Static scenes bake once (246ms cold, then <1ms) -> 60fps,
  full-length (probe + 13:53 take, 301/300 per 5s, 12.46s file from 12.3s
  wall). FramePump.ProbeRender names the hot render path on slow frames.
2026-09-12 13:56:55 -07:00
grampsandaider a62a283fd4 fix: resolve build errors with missing using directive and variable name conflict
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2026-09-12 10:33:16 -07:00
grampsandaider 992bb21bdb feat: add webcam frame diagnostics to debug gray block issue
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2026-09-12 10:32:18 -07:00
grampsandaider 58050d9ab1 feat: add alpha probe diagnostics to webcam frame capture
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2026-09-12 10:25:18 -07:00
gramps 70db34455f perf(camera): MJPEG negotiation before frame reader — camera drives at its best rate
Slice 10 from today-slices (d1126dd): UVC cameras default MediaFrameReader to their
FIRST media type (YUY2 at crippled fps — C920 720p = 5fps). A 60fps container
then shows a 5-20fps face cam as "every few frames cut out". Fix:
VideoDeviceController.SetMediaStreamPropertiesAsync negotiated to best MJPEG
>=640x360 @>=30fps (<=1280 wide) BEFORE creating the frame reader, with
device-default fallback. Plus a startup.log truth line of the actual agreed media
type so the next diagnosis starts from facts (no more re-deriving the fps).
Build 0 warnings, 289 tests.
2026-09-05 17:07:24 -07:00
gramps d35823a4a9 feat(ux)+fix(rings): release counter #N in the wordmark; all shared-frame rings 4->8 (#11)
Roll-forward of today-slices 6fd1d9c onto the slice-8 base, two-loop hunk dropped.

Release counter (per-build GUID read as noise; +1 per commit from git rev-list,
baseline 241 -> #13, generated by GenerateBuildStamp; GUID demotes to startup.log).
Tests pinned to Label/#N >= 13; wordmark display test asserts the Label.

Flash fix (take 14 finding): consumer holds must never outlive depth x source
period — 4 slots at high refresh lap ~27ms vs a <=50ms compositor read, so a
recycled slot flashed its new frame over the lagged old one. All shared rings 4->8
(OBS/overlay precedent for ring discipline).

Camera producer now rotates an 8-deep ring + Epoch instead of a fresh ~3.7MB
array per device frame (110-220MB/s LOH churn); WebView2 capture reuses a canvas
scratch + 8-deep output ring + a reused WriteableBitmap instead of two fresh
arrays + a fresh bitmap per 10Hz tick. Paste cache stays identity-keyed (Epoch).
2026-09-05 14:31:03 -07:00
gramps 432adfdaef perf(render): take-4 slice 2 — IsOpaque memcpy, integer bilinear, pump scratch pool
Take 4: pacing held (sync perfect) but avg render stayed 58.9ms — 2M
managed row-walk iterations + a fresh 8.3MB buffer every tick (LOH churn
into GC stalls inside the render measurement).

- VideoFrame.IsOpaque: producer-contract flag (screen capture + webcam —
  DWM/MF fill alpha 255; media/chat/web/static NOT flagged). Full-cover
  aligned opaque backdrop = ONE Buffer.BlockCopy; black pre-fill skipped
  when it covers.
- General BlitContent: integer 8.8 fixed-point bilinear + blend, row
  invariants hoisted, no per-pixel division/Math.Round. Within ±1 of the
  float reference (pixel tests allow ±2). Research per derivative-work
  rule: libyuv row/scale kernels (chromium.googlesource.com/libyuv/libyuv).
- FramePump scratch pool (max 4, length-keyed, owned-by-reference):
  release strictly AFTER SubmitFrameAsync returns (stdin write copies);
  Contains-guard makes the transition Cut alias safe.
- Removed the dead per-tick fromScene render + fromSceneProvider seam —
  BlendFrame consumes TransitionService.FromFrame captured at Start; the
  pump's render fed nothing. MainViewModel call site updated (signature).

Bugs caught by the pixel probes pre-ship (recorded MyMistakes): first
Bilinear double-shifted both stages (solid-255 sampled to ~1 -> general
path drew nothing); sentinel 0xAB collided with an x+y pixel. FakeEncoder
snapshots submitted frames (mirrors real copy semantics under recycling).

ONE integration test: Pump_Pools_ScratchBuffers_Across_Frames_Without_
Stale_Pixels (alternating backdrops + repeated backing identity). Direct
pin: Composite_OpaqueFullCover_Backdrop_CopiesEveryPixel_Into_Scratch.
Clean build 0 warnings; 59/59 per-class + RealApp boot-smoke. take 5
verdict: expect avg render <= ~10ms, ~300/300 frames. Docs same commit:
ai.md pipeline section, TASKS.md TASK 18, HANDOFF rewritten (Unit B spec
+ settled decisions queued).
2026-09-04 09:35:59 -07:00
gramps 60259c08c2 Fix CameraStreamState.Failed compile error (19041 projection omits enum member names — compare by int); fix first-frame proof test (no dispatcher = no bitmap, assert LatestFrame instead) 2026-08-12 07:11:54 -07:00
gramps b64830ed11 Webcam grab: SharedReadOnly mode, VideoPreview/VideoRecord source fallback, name the blocking app in the error (NVIDIA Broadcast finding) 2026-08-06 14:42:54 -07:00
gramps 98cdc4b3f4 TASK 3 milestone 1: webcam capture (MediaCapture CPU-first, CameraManager refcount, picker, clip/mirror, schema v2, tests) 2026-08-06 14:35:34 -07:00