gramps
71932b9756
The 240Hz monitor delivery + one-in-flight conversions + naive double-per-pixel DownscaleBgra (~150ms/frame under load) froze the desktop layer 90% of take ty-1742 (6.1 fresh content updates/s, freeze runs to 2.8s; decoded raw-frame audit). The render stat (33-36ms) was real but moot — the capture CONVERSION was the wall, and the one torn frame was a ring slot rewritten under the consumer's read. Reference: WGC delivers at DWM/monitor cadence (https://learn.microsoft.com/en-us/windows/apps/develop/media-authoring-processing/screen-capture) and libyuv row-simple/fixed-point scaling (https://chromium.googlesource.com/libyuv/libyuv/) — the repo's own take-4 rule. - DownscaleBgra: integer 8.8 fixed-point, shift-only-at-the-end (same two-stage math as SceneCompositor.Bilinear). ~150ms -> ~5ms per 2.5K->1080p frame. - 10ms MinConvertInterval: the ~4.2ms 240Hz tail stopped queuing ~150ms of serialized conversion/s; capacity sits just above the 60/s the pump can use. - FrameRingBuffer (depth 8, redLine 4): reuse-DISTANCE ring — a buffer is only rewritten >=4 rents after its last hand-out else fresh-allocated, so a frame a consumer still holds (session.LatestFrame survives conversions, dispatcher preview lags) is never read-while-overwritten. Needs no consumer Release API. - 2s startup.log telemetry: frames/s, conv avg/max ms, skip busy/cadence, ring allocs — the device take is judgeable numerically. Good Dog test: Ring_NoLap_ReusesOnlyAfterRedLineRents. 295/295 green, 0 warnings. C4 (composite Epoch-cached downscale) deferred pending the device re-measure. Local only, no push.
101 lines
6.2 KiB
Markdown
101 lines
6.2 KiB
Markdown
# HANDOFF — 2026-09-14 (slice-16 capture-conversion fix committed locally — device verify next)
|
||
|
||
## Branch / Commit State
|
||
|
||
`main` HEAD = **slice-16 commit** (capture conversion bottleneck — committed LOCALLY, **NOT
|
||
pushed**; web/A/V work stays commit-local until greenlight). Before it: slice-15 pacing fix
|
||
(FramePump), before that `c01206f` (composition capture), before that `b22d08e` (signed
|
||
audio-sync, pushed). Working tree **clean**.
|
||
|
||
## ⚠️ Branding (2026-09-14, creator-corrected): product = **llamacasty**, internals = ytLive
|
||
|
||
The product is **llamacasty**; repo path, csproj `AssemblyName`/`RootNamespace`, DB/log paths
|
||
(`%APPDATA%\ytLlive\...`), and most code names are the legacy **ytLive/ytLlive**. User-facing
|
||
language says "llamacasty"; code/assembly/repo names stay ytLive. See `ai.md` → Brand.
|
||
|
||
## 🔬 Measured the desktop-capture complaint (take ty-20260914-1742)
|
||
|
||
Slice 15 fixed pacing (video 23.35s ≈ audio 23.52s) but the creator reports the desktop layer
|
||
(tv show) still **jerky / laggy / frozen with a horizontal tear**; webcam + audio are great.
|
||
Decoded `ty-20260914-1742-0000-2.mp4` (1401 frames) to raw gray and audited (`/tmp/opencode/
|
||
tear_audit.py` + `mix_check.py`):
|
||
|
||
- **Desktop band (rows 40-320) frozen 21s of 23.35s (90%)** — ~6.1 content updates/s, freeze
|
||
runs up to **2.28-2.78s**, dup-run max 90 frames (1.5s). Render stat (`worst render 33-36ms`)
|
||
was real but MOOT.
|
||
- Root cause: the **capture CONVERSION** is the wall. Monitor delivers at the **240Hz DWM
|
||
cadence** (~4.2ms); one-in-flight conversions (`_framePending`), and each 2560×1440→1080p
|
||
`DownscaleBgra` (naive double-per-pixel) ≈ 30-45ms quiet / **~150ms under 240Hz-HDR load** →
|
||
`LatestFrame` updated ~6-9×/s. The tear (one new-top/old-bottom frame) = read-under-write on
|
||
a recycled ring buffer / DWM readback race. Webcam+audio are separate paths — fine, as
|
||
reported.
|
||
|
||
## 🔬 Committed locally — slice 16: fast downscale + cadence throttle + reuse-distance ring
|
||
|
||
**What** (`Services/ScreenCaptureFrameSource.cs` + new `Services/FrameRingBuffer.cs`):
|
||
|
||
1. `DownscaleBgra` → **integer 8.8 fixed-point, "shift only at the end"** (the exact two-stage
|
||
math of `SceneCompositor.Bilinear`). Kills the double-per-pixel float cost.
|
||
2. **10ms conversion floor** (`MinConvertInterval`): the 240Hz tail stops queuing ~150ms of
|
||
serialized conversion/s; capacity sits just above the ~60/s the pump can use.
|
||
3. Ring → **`FrameRingBuffer` reuse-distance pool (depth 8, redLine 4)**: a buffer is only
|
||
rewritten ≥4 rents after its last hand-out, else a fresh allocation. Structural no-lap that
|
||
needs no consumer Release API (`session.LatestFrame` survives conversions; dispatcher
|
||
preview lags).
|
||
4. **Telemetry:** a startup.log line every 2s — frames/s, conv avg/max ms, `skip busy/cadence`,
|
||
`ring allocs` — so the next device take is judged numerically.
|
||
|
||
**Good Dog test:** `ScreenCaptureFrameSourceTests.Ring_NoLap_ReusesOnlyAfterRedLineRents`
|
||
(depth 3/redLine 4 exercises the red-line skip → fresh hand-out; 8/4 settles at 8 buffers and
|
||
never grows). **295/295 green, app build 0 warnings.** Files: `Services/ScreenCaptureFrameSource.cs`,
|
||
`Services/FrameRingBuffer.cs`, `ytLive.Tests/ScreenCaptureFrameSourceTests.cs`. Docs in same
|
||
commit: ai.md (Slice 16 + capture bullet + focus-loss clause), MyMistakes (freeze-audit RECIPE +
|
||
lessons + the recast note), this HANDOFF.
|
||
|
||
**Note — approved-plan recast:** the earlier C1 (native-res capture + composite-side
|
||
Epoch-cached downscale) was recast to "fix the downscale in place" after the measurement:
|
||
relocating a 30ms float downscale to the render thread just moves the same cost into the slot
|
||
budget. C4 (compositor optimization) stays conditional on the re-measure.
|
||
No FPS-tier change, no HDR work (creator decisions preserved). Scope-lock list for this commit
|
||
was the 6 files above (+docs); `SceneCompositor.cs` and `FramePump.cs` were NOT touched.
|
||
|
||
## ⚠️ Open items (before PUSHABLE)
|
||
|
||
- **Device re-verify (next step):** the creator records the SAME tv-show scenario on the
|
||
slice-16 build. Judge numerically:
|
||
- startup.log telemetry: conv avg ≤ ~8ms, frames/s ≥ ~60, `ring allocs` ≈ 0 (steady).
|
||
- Decode + `/tmp/opencode/tear_audit.py`: ≥ ~55 content updates/s in the desktop band, frozen
|
||
% in the single digits, no mid-frame split survivors (social-bar strip churn is fine).
|
||
- ffprobe: video ≈ audio ≈ wall.
|
||
- If render still busts the 16.6ms slot after capture feeds real updates → C4 (Epoch-cached
|
||
composite downscale / blit-on-change), still staying 60fps.
|
||
- **No push yet** — commit-locally-until-greenlight for web/A/V work.
|
||
|
||
## Open threads (carried)
|
||
|
||
- Audio-silence verification — fixed (`724af14`); creator heard real audio.
|
||
- Webcam MJPG missing / ~10–14Hz, layer SortOrder, truncation-with-dynamic-scenes — queued.
|
||
- Sync control user-doc tutorial — REQUIRED before 1.0 (creator directive; TASK 22).
|
||
- Signed A/V sync: verify the negative (advance) direction on device.
|
||
- Focus-loss capture lag (OS-level delivery throttle) — deferred, still open.
|
||
|
||
## Landmines
|
||
|
||
- testhost shares startup.log — filter by time.
|
||
- `cmd.exe /c "taskkill /F /IM ytLive.exe"` (WSL double-slashes mangle) before rebuilds — a live
|
||
app process locks `ytLive.exe` and the apphost copy fails (MSB3021).
|
||
- Build/tests: **Windows dotnet host** (`/mnt/c/Program Files/dotnet/dotnet.exe`). 0 warnings —
|
||
only `./scripts/verify.sh "<files>"`'s clean build counts. Building `ytLive.csproj` alone does
|
||
NOT rebuild `ytLive.Tests.dll` — run the Tests csproj before `vstest`.
|
||
- ffmpeg/ffprobe: `/mnt/c/Program Files/Krita (x64)/bin/` with Windows paths.
|
||
- `MyMistakes.md` has the **freeze-audit RECIPE** (ffmpeg→raw-gray→numpy band audit), the
|
||
**A/V sync measurement recipe**, the **deadline-pacing** lessons, and the **CoreMessaging DQ
|
||
recipe** — grep before re-deriving.
|
||
- sqlite3 at `/home/gramps/android-sdk/platform-tools/sqlite3`.
|
||
- `C:\tmpout` is for ffmpeg evidence artifacts (raw decodes / PNGs); keep them out of the repo.
|
||
|
||
## Next step
|
||
|
||
Creator records a tv-show take on the slice-16 build → read the startup.log telemetry line +
|
||
`tear_audit.py` cadence + ffprobe durations. If conv~5ms + ≤60 fresh + no splits: defect closed;
|
||
re-measure the clap offset (`/tmp/opencode/avsync.py`); then decide push with the user. |