The cadence work ended with a question the log could not answer: the compositor
manages 6.8 composites/s at ~141ms while the 10:03 take did 29-34ms on the "same
scene shape" (full-render split=0 elements=6 dynamic=5). Nothing in the stats line
said WHICH of the six elements cost what, and nothing recorded the scene itself, so
the two takes could only be compared by eye. This closes that gap and nothing else:
no fix is attempted until a real take names the culprit.
- Services/Compositor/CompositeTimings.cs (new): per-window collector.
Add(stage, ticks) accumulates; Describe(renders, minMs = 0.1) prints the averaged
breakdown slowest-first, dropping sub-0.1ms noise.
- SceneCompositor.Render / CompositeLayers / BakeStaticBase / BlitBakedElement /
BlitCachedLayer + SceneGraph.GetBakedBase: trailing optional CompositeTimings?
Every stage is timed (backdrop, base, flash, social, ticker, zoom, scale, overlay,
bake backdrop, bake overlays) and EVERY element gets its own entry keyed by
z-index + name ("#2 Webcam"), with the nested "raster" sub-stage recording the
paste-cache rasterize miss - the split between "re-samples every tick" and "slow
paste" without a second session. Bake-pass elements are prefixed "bake #..." so a
one-off rebake cannot masquerade as a per-composite cost. Callers that pass
nothing (thumbnails, existing tests) are unaffected and pay nothing.
- FramePump: owns the collector, threads it through RenderScene/RenderFull/
GetBakedBase/CompositeLayers, and logs "FramePump parts (N renders/5s): ..." once
per 5s window beside the stats line (averaged per render, so it compares directly
with "avg render"), then resets it. Also dumps the scene - "FramePump scene dump:"
header plus one line per element (z-index, name, kind, source type, visibility,
rect, opacity, clip) - at pump start and on every scene change.
- One Good Dog integration test (ytLive.Tests/CompositeTimingTests.cs): a real
composite of a full-canvas element plus a 64x64 one through the real collector
must rank the full-canvas element first, carry the raster sub-stage and the fixed
stages, and the dump must name both elements with z-index/type/rect and echo the
split point.
Derivative work (AGENTS.md): OBS's answer to "which source is slow" is the Source
Profiler - average execution time of each source's tick and render functions
(https://docs.obsproject.com/reference-libobs-util-source-profiler, util/profiler.h).
Same shape here: one timer per element per composite, reported as an average, plus
the stage split this case needs.
Docs in the same commit: ai.md (new subsection "Per-element composite timing + scene
dump", the closed gap corrected in "Frame cadence"), Services/index.md (new file row
+ FramePump/SceneCompositor rows), TASKS.md (unit marked shipped, next = creator
take), HANDOFF.md (rewritten: unpushed commit, next step, verify.sh whitelist hole).
Also recorded this session (from the creator): the launch command is
`dotnet run -c Release` - Release is the DEFAULT, Debug only when explicitly named,
and perf takes must read Release. ai.md -> Run updated.
Full suite 428 tests, 0 warnings; the only failures are the known audio env flakes
(both pass in isolation). Scope check passed.
The prior handoff claimed the strip-tile/on-air unit was uncommitted and the
pinned-chat unit was 'not mine - do NOT commit'. Reality: both are committed
(8e1d6f4, 931ab46) and origin/main is level with HEAD (0 ahead). Corrected to
match the code and recorded the true next step (webcam per-element composite
timing measurement). Removed the stray Assets/llamacasty-text-nobg-2.png~ backup.
(Docs: handoff)
- Model: Google's official MediaPipe Selfie Segmenter (landscape, Apache-2.0 per model card),
converted to ONNX by tools/selfie-seg/convert_tflite_to_onnx.py (custom op
Convolution2DTransposeBias -> ConvTranspose) and verified against MediaPipe's own runtime
(verify_against_mediapipe.py: max |diff| 5e-5, 100% binary agreement). Embedded resource, 430 KB.
- Services/BackgroundEffects: IPersonSegmenter seam, OnnxPersonSegmenter (onnxruntime 1.31.0 CPU,
never throws), BackgroundEffectProcessor (None passthrough, Remove = alpha mask). Not wired yet.
- Tests: Remove integration test (fake segmenter) + bundled-model contract test vs MediaPipe
reference mask. Full suite 407/408 — the one failure is the documented interactive-desktop flake
RealMouseDrag_OnTheLayerList_PersistsTheReorder (passed in the run before; code path untouched).
- Docs: ai.md reversal + section + licensing guard, THIRD-PARTY-NOTICES 11-12, TASK 51, indexes,
MyMistakes recipe, HANDOFF.
References: obs-backgroundremoval (approach only — ONNX Runtime on CPU, 2 threads; GPL-3.0, no code
used) https://github.com/royshil/obs-backgroundremoval ; MediaPipe image segmenter
https://ai.google.dev/edge/mediapipe/solutions/vision/image_segmenter
The zoom feature shipped without a task record while its index rows carried the
"TASK 51" label — which belongs to the webcam background effects (Remove + Blur).
For an honest catalogue and handoff:
- TASKS/task-52-zoom-box.md: feature provenance (creator-requested), the three
2026-10-08 corrections (loupe -> anchored capture over the full composite drawn
last -> never show the dim shading), shapes, implementation map, tests, DONE.
- TASKS.md: row 52 added (shipped); row 51 reworded — unblocked now that the
zoom build shipped.
- TASK 51 -> TASK 52 in Models/index.md, ViewModels/index.md, and the two code
comments that pointed at the zoom as "TASK 51".
- HANDOFF.md: pre-push readiness notes (5 commits ahead, push-safe).
Creator's final zoom correction: as the box is dragged, its capture region slides
under the dim rectangles and the box picks up the 55% black "div shading" inside
itself, because the VisualBrush samples the whole CanvasGrid and DimRects lived
inside CanvasGrid. The captured area must stay clear regardless of box position.
- PreviewPane.xaml: the DimRects ItemsControl is now a SIBLING of CanvasGrid
(above the composed content, below the ZoomBox), never a descendant — the zoom
source is shading-free by construction.
- ZoomBoxInteractionTests: structural invariant — DimRectsLayer is a direct
child of PreviewCanvas (not inside CanvasGrid) and renders below the ZoomBox.
- Side effect (correctness, not a bug): quality dims now also sedate the global
overlays outside the crop, matching "chrome outside the crop is dimmed".
Creator corrected the "loupe" zoom on 2026-10-08: the click anchors a fixed
capture region (box/2 centred on the click) and the box is a live 2x view of
that region — dragging the box never changes its content; it magnifies the
fully-composited frame (every layer, incl. webcam) and draws LAST (n+1).
Rectangle shape is 4:3 ("more of a 640x480 aspect").
- MainViewModel.Zoom.cs rewritten: ActivateZoom anchors CaptureX/CaptureY
(independently clamped); UpdateZoomMove relocates the box only; the per-frame
BuildZoomFrame/ZoomFrameBuilder path is gone.
- SceneCompositor: skips the active ZoomSceneElement in every layer loop and
BlitZooms it last via the new pure ZoomMagnifier (2x bilinear, temp snapshot
via ArrayPool, round clip).
- Preview: ZoomBox hoisted above CanvasGrid as a sibling; its brush samples the
whole composed preview at the fixed capture (2x).
- Tests: ZoomMagnifierTests (box-over-blue shows green capture at 2x incl. an
extra red "layer" — the loupe/backdrop-only regressions both fail it) +
round-clip; ZoomBoxInteractionTests rewritten (anchored capture, move-doesn't-
change-capture, 4:3/9:16/square). 404 pass, 2 known environment flakes.
Right-click now creates a transient ZoomSceneElement at z-index 1 whose pixels are a
genuine 2x magnified sample of the backdrop (live capture under art), composited into
stream/recording like any layer while the output crop stays the tier. All shapes are
centered on the click and clamped to frame; healed on load/scene-create, never persisted,
hidden from the layers list + element template (IsZoomLayer). BuildCompositorOptions is
now internal = always tier crop (test seam). ZoomFrameBuilder is a pure static sampler
(box/2 window centered on the box center, single-slot cache) proven by a banded-backdrop
test; LayerReorderPersistenceTests drag resolves rows by item since the collapsed zoom
row shifts container indices. Also books TASK 51 (webcam background Remove + Blur) and
syncs memory docs (HANDOFF, MyMistakes, indexes). Full suite: 405 passed, 0 warnings.
Right-click in the live preview now procs a box that is actually visible and
populated with the zoomed region, and each Zoom View Shape sculpts the box per
spec. OBS-style derivative: magnifier box shows the composited region and dims
the surroundings (same pattern as OBS "zoom" / preview region highlights).
Root causes of the invisible box (two WPF traps, recorded in MyMistakes.md):
- Empty <VisualBrush Stretch="None"/> used as an OpacityMask paints nothing,
so the mask alpha is 0 and the WHOLE box (border included) went transparent.
- ZoomBox lived inside CanvasGrid while its VisualBrush sampled CanvasGrid — a
circular reference WPF renders as nothing. Hoisted ZoomBox to a sibling of
CanvasGrid inside a new PreviewCanvas wrapper (same 1920x1080 space).
Shape semantics (MainViewModel.ActivateZoom):
- Circle: square, click point = center.
- VerticalRectangle (phone): tall 9:16, click point = top-left corner.
- HorizontalRectangle (tablet): wide 16:9, click point = top-left corner.
- Boxes stay fully inside 1920x1080 via Math.Clamp.
UI: ZoomRectOutline/ZoomCircleOutline shapes replace the square Border;
circle clips the box to an EllipseGeometry; brush Visual released when inactive.
Zoom View Shape dropdown now uses the standard YtComboBox style.
Test (Good Dog rule, one integration test): ZoomBoxInteractionTests verifies
aspect/anchoring/clamp per shape and the matching outline+clip in the real
MainWindow. Full suite 404 passed, 0 warnings.
Memory: HANDOFF.md zoom state updated; WPF VisualBrush traps appended to
MyMistakes.md.
A composite splits into four horizontal row strips across four DEDICATED
background threads; the calling thread takes a strip too (so four workers
never mean five-way oversubscription), and anything under 64 rows runs
inline. BlitContentRaw, BlitBakedElement and BlitOverlay now fan out.
Output is byte-identical — strips already partitioned by row.
The demotion deliberately does NOT come from Parallel.For. The pool is
shared, so Thread.CurrentThread.Priority = BelowNormal inside it leaks
into every unrelated work item that thread later runs — a full test run
after trying it broke AudioPipelineTests.Mix_HonorsProviderGains_AndGameMute_
KillsTheLoopback. Established guidance agrees: use your own producer/consumer
queue with your own Threads and set their priority yourself
(https://stackoverflow.com/questions/524483/net-backgroundworker-objects-thread-priority,
https://stackoverflow.com/questions/1578599/c-sharp-lower-thread-priority-in-thread-pool).
Helpers/StripPool.cs is that queue. Four threads is the measured sweet
spot: fewer leaves render over budget, more steals cycles from the game
and the encoder and made render worse.
The scaled blit's column LUT moves from a shared static buffer to
ArrayPool<float>.Shared with a finally — a single static buffer is a data
race the moment strips run in parallel.
Gate: SceneCompositorTests.StripParallel_RepeatedRenders_AreByteIdentical —
8 renders of a scaled round webcam + mirrored image + master-sized flash over
a short backdrop must be byte-for-byte identical, and must differ from a
backdrop-only render so it cannot pass vacuously. It caught a loop-variable
capture (new Thread(() => Worker(i)) saw i == Workers) that crashed the test
host on the first run.
Handle the case where an opaque source exactly covers the destination
canvas with 1:1 dimensions (even if offsets differ in calculation). This
avoids per-pixel alpha branching and mirrors the existing memcpy fast
path. Byte-identical for opaque sources.
Add timing for social bar, ticker, flash delegates and scratch acquire,
and extend the 5s stats line to report them (ms per render). No behavior
change, only measurement.
The Stream Alerts properties panel was incorrectly showing for the webcam
layer (a WebcamSceneConfig is not a Source) because IsAlertBox only existed
on Source, causing WPF binding to be treated as false-but-invisible context
or the trigger to bind against the wrong type. Adding a virtual IsAlertBox
to the base SceneElement (default false) and overriding on Source prevents
the UI block from rendering under webcam selection. Scope: Models/SceneElement.cs, Models/Source.cs, ytLive.Tests/AlertBoxPreviewVisibilityTests.cs. Build/test: app + 4 alert-box tests pass, other failures unrelated (float precision).
Take ty-20261005-1201-556-0001.mp4 (75.3s) plus its log settle it. The
C920 delivers 30.0/s with ZERO gaps over 50ms and its own device stamps
agree - camera, USB and driver are clean. The pump managed 34 composites per
5s at ~141ms each, and the file holds 6.8 distinct webcam images per second
at 88.7% duplicates with no arrival hitch whatsoever. The judder is us
holding one face for 140ms and padding the rest of a 60fps container, which
is precisely what a lipreader experiences as "cutting out".
So 720p is a SHARPNESS win, not a smoothness win: it triples the bytes the
compositor stretches per render on a path already 4.4x over budget. The
exclusive camera lock stays agreed and stays queued, but it moves after the
compositor work or it makes the judder measurably worse.
The number still unexplained: the 10:01 take hit 25.6 images/s and this one
6.8, with the same scene shape (full-render split=0 elements=6 dynamic=5)
and the same 640x480 camera - composites went 29-34ms to 143-155ms between
sessions and nothing logged says why. There is no per-element composite
breakdown and no scene dump at pump start, so the next unit is measurement,
not a fix. Recorded the pump-counter overcount (13/s vs 6.8/s: one 141ms
composite lets ~4 camera frames arrive, and the counter stamps the first
epoch the resolver sees, not the frame actually drawn) so it is not
re-derived as a bug later.
The creator is DHH and lipreads their own recordings, so a webcam that
judders is an accessibility defect, not a polish item. Take
ty-20261005-1001-821-0001.mp4 holds 25.6 distinct webcam images/s against
the camera's 30fps with 281 gaps of 50-100ms - roughly one hitch in five -
in a file that muxes flawlessly. Nothing in the app could see that: rawvideo
carries no per-frame timestamps (ffmpeg stamps by ARRIVAL at the declared
60fps), and the stats line divided every average by the number of frames
SUBMITTED, which deadline padding always fills to the full target. So the
pump reported "300/300 frames, avg render 12.4ms" while producing 60
distinct composites.
Helpers/FrameCadence.cs is one pure counter fed from three points:
MediaCaptureFrameSource logs the device's own SystemRelativeTime stamps
against our arrival times (TryAcquireLatestFrame keeps only the LATEST
frame, so falling behind is silent by design), and FramePump logs the
distinct webcam Epoch values that reached a composite. Read together they
name the culprit - device/USB, our reader, or our compositor - instead of
guessing. Render/resolve/wait now divide by the real render count, and
"frames per 5s from R renders" is the duplicate-on-lag ratio stated
honestly.
An out-of-order timestamp counts as a frame but produces no gap and does not
become the next interval's reference, so one bad stamp cannot invent a
multi-second hitch twice.
Also corrects a stale ai.md line (the "webcam renders at ~18fps" figure
came from a different session and counted every newly rendered layer) and
re-records the background-removal ruling as permanently out. Records the
agreed-but-unbuilt Units 1-2, the no-fallback-camera ruling, and the
SetFormatAsync dead end found by probing the 19041 projection.
Suite 402/402, 0 warnings. The one red, LayerReorderPersistenceTests'
RealMouseDrag, fails identically on a stashed clean tree - it needs an
interactive desktop session this shell lacks.
Push state was stale (said cc2d1ef, actual origin is 048cab7 with 5 local commits),
the RTMP entry still framed the binding as an open question, and the test count was
381. Also records the frame-pump jitter as unquantified and the creator's future
long-run benchmarking to-do as record-only.
Creator ruling: open in the upper-left of the screen with ~75px padding. The
restore path was placing the window below the fold.
RestoreWindowGeometry now ignores the saved position and calls the pure
TopLeftPlacement(workArea, w, h, padding): work-area origin + LaunchPadding
(75, in DIPs so it scales with DPI), clamped so the window is never pushed
off-screen. A window that fits stays wholly visible; one too big to fit falls
back to the work-area origin.
Size and maximized state are still persisted -- only position was dropped. The
size clamp is unchanged (saved values clamp to [MinWidth, work.Width], which is
why a saved 800x500 returns as 1366x768 per the XAML minimums).
The old clamp tolerated Left as low as work.Left - width + 120 and Top up to
work.Bottom - 80, ranges wide enough to leave the window almost entirely
off-screen while still looking like a successful clamp. A clamp that permits
the window to be unreachable is worse than none: it reads as a guarantee it
does not keep.
BroadcastPullOutTests is renamed ..._Window_Launches_TopLeft_... and now asserts
the constructor applies the derived placement for both an on-screen and a -99999
saved position. Placement math is covered by WindowPlacementTests as a pure
function needing no HWND.
Suite 395/395, 0 warnings.
root cause (proven, not inferred): contentDetails.boundStreamId on
liveBroadcasts.insert does NOT bind the broadcast. Our diagnostic reads
lifeCycle=created privacy=private autoStart=true bound=NONE
boundMatches=NO-BINDING | streamStatus=active
created is Google's "incomplete settings" state, so enableAutoStart has no
bound stream to fire from; close-out then 403s invalidTransition because
created -> complete is not a legal edge. A reusable stream accepts bytes for
a broadcast that isn't watching it, so a perfectly healthy 99MB RTMP session
is exactly what an unbound broadcast looks like -- every observable pointed
away from the binding.
Google's migration guide states the rule outright: a liveBroadcast must be
bound to exactly one stream before it can start, and is not bound at creation.
That was in the docs read during the original design and was not applied.
Shipped:
- BindBroadcastAsync -> POST liveBroadcasts/bind?part=id&id&streamId (no body),
called from PrepareAndStartLiveAsync right after CreateBroadcast, before the pump.
Failure is fatal to the session: an unbound broadcast can never go live, so a
live-looking UI over one is a lie.
- boundStreamId removed from the insert body, and the streamId parameter removed
from CreateBroadcast. A parameter that looks meaningful but cannot affect the
outcome is what made the misdiagnosis possible.
- Replaced CreateBroadcast_With_StreamId_Binds_Reusable_Stream_At_Insert, which
asserted the disproven behaviour and therefore laundered a guess into a
guarantee, with tests for the omitted field and the real bind endpoint.
Error messages (creator ruling 2026-10-04): bind failures are not all "check the
sign-in". Six documented reasons exist and only one is auth; BindFailureMessage
gives each its own text, and the fallback passes the raw reason through verbatim
so an unmapped reason stays greppable. liveBroadcastBindingNotAllowed matters
most -- it means our ordering is wrong, and a generic re-auth message would hide
a bug in our own code behind a plausible excuse. See MyMistakes.md.
ai.md: also corrects the TASK 44 claim that liveChatId is only populated once the
broadcast is live. The diagnostic shows it present while lifecycle is created.
That claim was inferred from a sample's active-status guard, which is not a spec.
Suite 395/395, 0 warnings.
gramps