96 lines
5.8 KiB
Markdown
96 lines
5.8 KiB
Markdown
# HANDOFF — 2026-09-14 (web capture REDESIGNED: composition-capture, PNG polling + CaptureScheduler deleted — NOT yet verified on device, NOT pushed)
|
||
|
||
## Branch / Commit State
|
||
|
||
`main` HEAD = **`c01206f`** (web-frame-capture redesign — committed LOCALLY, **NOT pushed**; web work
|
||
commit-locally-until-greenlight). Before it: `b22d08e` (signed audio-sync, already on origin/main).
|
||
Working tree **clean**.
|
||
|
||
## ⚠️ Branding (2026-09-14, creator-corrected): product = **llamacasty**, internals = ytLive
|
||
|
||
The product is **llamacasty**; the repo path, csproj `AssemblyName`/`RootNamespace`, DB/log paths
|
||
(`%APPDATA%\ytLlive\...`), and most code names are the legacy **ytLive/ytLlive**. User-facing
|
||
language must say "llamacasty"; code/assembly/repo names stay ytLive. Full detail in
|
||
`ai.md` → Brand → "Product name vs repo/assembly branding".
|
||
|
||
## 💥 The web-capture take history (why this redesign exists)
|
||
|
||
The recording was 60fps but web widgets ran at ~1/6 speed: capture was a `CapturePreviewAsync` PNG
|
||
poll; every full-HD encode+decode cost **35–165ms** so the session budget capped real cadence at
|
||
~20Hz — a 60fps widget still juddered. Slice 11's `CaptureScheduler` (30Hz) was the wrong lever:
|
||
the fix is frame-driven, not faster polling.
|
||
|
||
## 🔬 Committed locally as `c01206f` — composition capture (NOT pushed)
|
||
|
||
**What:** web sources now render through `CoreWebView2CompositionController` into
|
||
`Windows.Graphics.Capture` (the Flutter `webview_windows` / `WebView2CompositionControl`
|
||
mechanism) — frame-driven at the renderer's pace instead of polling PNGs.
|
||
|
||
**Pipeline (per session):** `CreateCoreWebView2CompositionControllerAsync(appHwnd)` →
|
||
`RootVisualTarget` = a `RelativeSizeAdjustment=1,1` child under a 1920×1080 root `ContainerVisual`
|
||
on one `Compositor` (CoreMessaging DQ P/Invoke recipe — see ai.md Slice 14 / MyMistakes) →
|
||
`GraphicsCaptureItem.CreateFromVisual(root)` → free-threaded `Direct3D11CaptureFramePool` (2
|
||
buffers) → `SoftwareBitmap.CreateCopyFromSurfaceAsync` (`BitmapAlphaMode.Straight`) →
|
||
per-frame `FindContentBounds` bbox → `CropBounds` → epoch'd ring → dispatcher-coalesced crop copy
|
||
into ONE shared `WriteableBitmap`.
|
||
|
||
**Files:** NEW `Services/WebCaptureFrameSource.cs`; reworked `Services/WebView2Manager.cs` (internal
|
||
seam ctor `(Dispatcher, Func<string, IScreenCaptureSource>?)` for hermetic tests); DELETED
|
||
`Services/CaptureScheduler.cs`; `MainViewModel.Web.cs` (`InitWebView2()` no-arg),
|
||
`MainViewModel.Streaming.Operations.cs` (three `SetCaptureInterval` hooks gone — verified zero
|
||
remaining), `MainWindow.xaml(.cs)` (`InitWebView2()` moved ctor→**Loaded**; `WebViewHostPanel`
|
||
overlay deleted). `TransparentBackgroundScript` unchanged.
|
||
|
||
**Tests:** reworked `ytLive.Tests/WebView2ManagerTests.cs` — dropped the 4 control-size tests +
|
||
`CaptureScheduler_Drops…`; `FindContentBounds` tests moved to `WebCaptureFrameSource`; the ONE
|
||
integration test (`Frames_PublishCroppedPreview_And_CoalesceToLatest_CarryingCropBounds`) drives the
|
||
seam with `FakeWebSource` + background-STA `DispatcherPump`: crop-sized preview published once →
|
||
`CropBounds` + `IsOpaque=false` → back-to-back frames coalesce to latest. `RoundClipInteractionTests`
|
||
comment updated (WebViewHostPanel gone). Suite **293/293 green**, app build **0 warnings**. Docs
|
||
slant (ai.md Slice 14, HANDOFF, MyMistakes, TASK 17, Controls/ViewModels/Services indexes) updated in
|
||
the same working set.
|
||
|
||
## ⚠️ Open items on this change (before it is PUSHABLE)
|
||
|
||
- **Device verification:** the composition path has never run against a real widget. Verify ~60fps
|
||
web animation in a take (also: the OBS de-throttle flags stay; hidden-page JS throttling is fixed
|
||
by them, capture pacing is now renderer-driven).
|
||
- **No push yet** — web work is commit-local until the user says push.
|
||
|
||
## ✅ SHIPPED + PUSHED — signed audio sync, −500..+500 (`b22d08e`)
|
||
|
||
Positive = delay the mix (audio AHEAD — the `AudioSyncDelay` line, live-reactive). Negative =
|
||
**advance** (audio BEHIND): OBS-style "eat the stream head" — `AudioMixer.StartLive` arms
|
||
`_advanceSamplesRemaining = |N| ms`, `LiveLoopAsync` skips `min(budget, mixBuffer.Length)` off each
|
||
write head while it lasts. Slider relabeled **AUDIO SYNC**, `−500..500`, locked while
|
||
live/recording (`IsEditMode`). One regression test:
|
||
`StartLive_NegativeOffset_AdvancesAudio_ByDroppingTheStreamHead`.
|
||
|
||
Take `ty-20260914-1128-0000-2.mp4`: **offset ≈ +0.54s** (collapsed from +2.11/2.22s; residual was
|
||
the baked-in `SyncOffsetMs=300`, now 0 — confirmed via sqlite3). Both streams `start_time=0`.
|
||
|
||
## Open threads
|
||
|
||
- **Verify signed sync negative direction on device** (clap take with a negative offset).
|
||
- **Audio-silence verification** — fixed (`724af14`); creator heard real audio.
|
||
- **Webcam MJPG missing / ~10–14Hz**, layer SortOrder, truncation-with-dynamic-scenes — queued.
|
||
- **Sync control user-doc tutorial — REQUIRED before 1.0** (creator directive; TASK 22 file).
|
||
- **Device verify the composition web capture (this change).**
|
||
|
||
## Landmines
|
||
|
||
- testhost shares startup.log — filter by time.
|
||
- `cmd.exe /c "taskkill /F /IM ytLive.exe"` (WSL double-slashes mangle) before rebuilds.
|
||
- Build/tests: **Windows dotnet host** (`/mnt/c/Program Files/dotnet/dotnet.exe`). 0 warnings —
|
||
only `./scripts/verify.sh "<files>"`'s clean build counts.
|
||
- ffmpeg/ffprobe: `/mnt/c/Program Files/Krita (x64)/bin/` with Windows paths.
|
||
- `MyMistakes.md` has the **audio/video sync measurement recipe** AND the **composition-capture
|
||
CoreMessaging DQ recipe (2026-09-14)** — grep before re-deriving.
|
||
- sqlite3 lives at `/home/gramps/android-sdk/platform-tools/sqlite3` (WSL) for the DB at
|
||
`/mnt/c/Users/gramp/AppData/Roaming/ytLlive/ytLlive.db`.
|
||
|
||
## Next step
|
||
|
||
From this dirty state: run `./scripts/verify.sh "<all changed files>"`, review the diff, commit
|
||
locally (web work — NO push, per standing directive). Then a device take to verify ~60fps web
|
||
animation, then present for push review. |