Two defects made the producer 17x slow (37s record -> 2.1s/127-frame file,
rawvideo stamps by arrival): FramePump slept the FULL interval after each
render (period = render+submit+interval) and SceneCompositor did per-pixel
float sampling + Math.Round blends over all 2.07M master pixels, scanning the
whole destination per overlay (258ms avg render vs 1.5ms submit).
Both solutions are established, not invented — researched before coding per
the derivative-work rule:
- deadline pacing: OBS libobs/media-io/video-io.c video_thread (nextTick +=
intervalTicks, sleep only the remainder, rebase on overrun, never burst)
- row blits: libyuv pattern (BSD-3, chromium.googlesource.com/libyuv/libyuv)
— 1:1 aligned identity fast path, per-pixel alpha branch, integer
fixed-point blend, overlay clipped to the intersection rect, skip the dead
black pre-fill when the backdrop covers
ONE integration test: Pump_Paces_To_The_Deadline_Compensating_Render_Cost
(lands after a fake-seam lesson: pacing fakes must await, not complete
synchronously, or the pump loop runs inline on StartAsync and hangs vstest).
Clean build 0 warnings; FramePumpTests 10/10, SceneCompositor/SceneGraph/
SocialBar/StretchMath 20/20. Docs same commit: ai.md pipeline section,
TASKS.md TASK 18 (webcam-in-output + rename modal verified from take 3),
MyMistakes recipe, HANDOFF rewritten. Take 4 pending on the user's machine.
All headless-testable TASK 21 decoder/mechanism slices shipped; the remaining UI
picker slice (AddMedia command + file dialog + Acquire/Release + Source.MediaIsLooping
-> IMediaFrameSource.Looping) is a GUI feature and is spec'd in HANDOFF for native
Windows build + verification. TASKS status + HANDOFF updated.
- IMediaFrameSource gains bool Looping.
- MediaVideoSource ctor takes Func<IDecodeProcess> processFactory instead of a
single IDecodeProcess: a System.Diagnostics.Process can't be re-Start()ed, so
each loop pass creates a fresh decoder. Decode wrapped in do-while(Looping):
restart on natural EOF instead of raising Completed.
- Production wiring (MainViewModel media factory): passes the process factory
AND FfmpegFrameRateProbe -- closes the slice-2b gap where production had no
probe and therefore no pacing.
- Tests: loop test (single frame re-emits across passes, Completed only when
loop cleared); fakes updated for the new interface member. Media tests 12/12,
build 0 warnings.
Wiring Source.MediaIsLooping into the flag needs a manager-level per-path loop
provider -> lands with the UI-picker (acquisition) slice.
Derivative reference: looping media by restarting decode on EOF, standard in
playback/overlay tooling (OBS media source repeat).
- MediaVideoSource takes optional IFrameRateProbe? + Func<TimeSpan,CancellationToken,Task>? delay
seams (default Task.Delay); probes FPS once in RunAsync, delays by 1/fps after each
emitted frame. No probe/unknown fps -> no pacing (ffmpeg pipe backpressure throttles).
- Test: MediaVideoSource_PacesFramesByProbedFps (fake probe returns 1000fps + recording
delay; one delay per frame ~= 1ms). Media tests 5/5, build 0 warnings.
Derivative reference: per-frame delay pacing of decoded output, standard in media playback.
- FfmpegFrameRateParser (pure): prefers avg_frame_rate= then r_frame_rate=,
rational N/N/M, unknown/0 -> null.
- IFrameRateProbe + FfmpegFrameRateProbe: derives sibling ffprobe.exe from the
located ffmpeg dir, reuses the IDecodeProcess seam for the ffprobe subprocess
text; null if ffprobe absent.
- FfmpegLocator now also extracts ffprobe.exe (ProbeFileName) from the pinned
archive, conditional so old caches without it degrade to no pacing.
- Tests: 6 pure parser units + 1 probe integration via fake locator/process;
FfmpegLocatorTests still green. 15/15, 0 warnings.
Pacing (probe->delay) is slice 2b. Derivative reference: standard ffprobe
avg_frame_rate probing used across OBS/media tooling.
Wire the MediaVideoSourceManager into the live pipeline (new partial
ViewModels/MainViewModel.Media.cs mirrors Webcam/Background partials):
readonly _mediaManager assigned in the core ctor (decode at 1920x1080 master,
dispatcher-coalesced preview), OnMediaPreviewBitmapChanged adopts the shared
bitmap onto every IsMediaSource with a matching MediaPath, a
Source { IsMediaSource, MediaPath } case in ResolveOutputFrame feeds
GetLatestFrame, and the manager is disposed on shutdown.
Source.DisplaySource now routes VideoImageSource for MediaSource (Source.cs),
so media previews/canvas show decoded video.
Compositor needs no change: BlitContent UniformToFill-scales any frame to the
element rect. Decoder/manager/seam derived-work mirrors ScreenCaptureManager
(citation in commit for slice-step-3).
Test (one unit for this change, mirroring the live-capture case):
Source_DisplaySource_IsVideoImageSource_WhenMediaSource. 14/14 media +
DisplaySource tests pass, build 0 warnings. Docs (TASKS/HANDOFF/ai.md) updated.
Adds the codec-agnostic decoder half of the media source. Spawns ffmpeg
with -f rawvideo -pix_fmt bgra (reusing the already-shipped ffmpeg via
IFfmpegLocator) and drains the raw BGRA stdout pipe into VideoFrames.
- Services/RawVideoFrameReader.cs: pure rawvideo BGRA stream -> frames
(partial reads kept across Feed; no ffmpeg needed to test)
- Services/IDecodeProcess.cs + FfmpegDecodeProcess.cs: binary-stdout
subprocess seam, mirror of the encoder's IEncoderProcess
- Services/MediaVideoSource.cs: owns the decode, raises FrameReady/Completed
- MediaVideoSourceTests: 3 pure reader + 1 integration (fake decode
process through the real source loop, frames in order)
Reference (external scan): ffmpeg rawvideo pipe decode is the canonical
codec-agnostic frame feeds pattern (ffmpeg docs -f rawvideo; how OBS/media
pipelines push frames to a compositor). Verified: 4/4 tests, build 0 warnings.
Native-FPS pacing + resolver/compositor wiring are the next slice.
- Add ElementKind (Static/Dynamic) to SceneElement base; Source/WebcamSceneConfig classify
- Services/SceneGraph.cs: owns the Scenes collection (ViewModel's Scenes delegates to it),
the element mutation surface (Add/Insert/Remove/Move, each invalidating the bake), and the
queries that were scattered LINQ (GetBackground/GetWebcam/GetChatBoxes/GetSplitPoint/IsStatic)
- SceneCompositor: split-aware BakeStaticBase + CompositeLayers + Render(.., staticBase, split);
builds/caches the static base below the split point in source-rect space
- FramePump: optional SceneGraph -> optimized bake+composite path; falls back to full render
- MainViewModel: routes element mutations through the graph; invalidates the bake on static
layout/opacity/visibility/useDefaultBackground changes and after background heal/Ensure
- Integration test SceneGraphTests.BakedStaticBase_WithDynamicLayer_CompositesCorrectly
Derivative survey (mandated): tried before writing — OBS does per-source opacity/visibility
caching and static-scene baking; this mirrors OBS's 'cached static source' optimization.
3 documented defensive deviations from the spec (ChatOverlayLayer stays decoupled;
background helpers stay VM-static for direct testability; full facade peel deferred post-1.0)
in TASKS.md + ai.md. Scope check passed; clean build 0 warnings.
ai.md: Key patterns gains the 'true decomposition beats partial-shuffling'
rule + the glue-vs-component diagnostic (count OnPropertyChanged/SetProperty;
extract only zero/low-glue cohesive state blobs like chat's; leave binding glue
over already-extracted services alone; line count is a guideline not a goal).
ViewModels table row notes Chat.cs is now a thin facade over ChatOverlayLayer.
HANDOFF: everything through Commit G is pushed (origin/main=85893ea); the
remaining big decomposition is the scene-graph segment, flagged as design-needed
(not an unsupervised peel).
Rewrite HANDOFF.md: Phase 2 is pushed at 1f4624c (stale claim corrected);
Phase 3 commit rows A-E + Controls/index.md listed as local-only, push pending
on explicit approval. Records the re-structuring principle (split by concern,
500-line count is a ceiling not the goal) and the session's truncation-accident
lesson. MyMistakes.md gains the never-awk-into-the-source-you-read rule.
Working tree clean. Phase 1 (pushed) + Phase 2 (local-only, commits 12-18 with
tags) both complete. Push checkpoint reached: next step is 'git push origin main
&& git push --tags' on explicit user approval. Landmines updated with the XAML
extraction gotchas (control xmlns, name-looked-up host stubs, window-resources
converter rule).
Zero behavior change: \`public class\` -> \`public partial class\`. Each functional area
(Scenes, Background, Webcam, Audio, Trax, Socials, Streaming, Chat, Overlays,
Account, License) will move to its own file next, one commit per partial — each
a roll-back point. Docs updated in the same commit: ViewModels/index.md (split
map), ai.md (architecture table), HANDOFF.md (state). verify.sh: build 0 warnings,
only the 2 known pre-existing test failures.
Encoder dual-output: StreamEnabled/RecordEnabled/RecordPath options; per-output
ffmpeg block (stream ->flv, record ->mp4); throw unless at least one output.
RecordingFile: auto-name ty-<yyyymmdd>-<start hhmm>-0000.mp4, rename-on-stop to
ty-...-<len hhmmss>.mp4 with numeric-suffix on collision. LayoutStore persists
record folder. MainViewModel: REC/ON-AIR pills, status light, single Start<->End
button (replaces Start/Stop variants), StopStream no longer signs out, explicit
session timer. About overlay widened. Sign In label + spacing before Start.
Tests: 244 passed / 246 (2 pre-existing: AudioPipeline gains-mute, RoundClip).
Build 0 warnings. Scope check clean.
Memory-map correction (derived-solution rule, 2026-08-29): image-shrink recipe was
never recorded when first done, so it was re-derived. Added Derived-solution/recipes
rule to AGENTS.md; MyMistakes.md now a permanent recipes registry (records the
verified WPF-imaging shrink recipe); schema.md rows MyMistakes as registry; ai.md
points to the rule. README hero image replaced: 3.2MB screenshot -> 1400x794 ->
188KB docs/ytLlive-preview.jpg (JPEG q82).
Project lesson cited (the 2026-08 web-box saga, 9 commits): browser sources are fixed canvases
per OBS convention — the 'gap' was a widget CSS glow, not a bug.
Root-cause note: the apparent remaining gap was a WIDGET GLOW EFFECT (the widget's own CSS glow pushed
out its perceived borders), not a code bug — the alpha-bbox already hugged the glow halo. Verified
perfect with a second widget without the effect.
- FindContentBounds already trimmed the bitmap; the grid/box still rendered at the uncropped user
size, so Fill stretched the tight widget into a bigger frame = dead-between-content-and-border.
- ContentCanvasSize maps the crop rect to canvas units (1920/pixW per axis, DPI-correct) and the
preview event now carries (bitmap, canvasW, canvasH).
- OnWebView2PreviewBitmapChanged sets the element Width/Height to that size; the grid and the
SelectionOverlay (both bound to the element's Width/Height) now equal the widget exactly.
- IsPreviewDragging (set in MainWindow drag/resize) suppresses auto-fit mid-gesture.
- Tests 10/10 (alpha bounds, canvas-size mapping incl. DPI, control size); 0 warnings.
- FindContentBounds scans the Bgra32 capture for the bounding box of non-transparent pixels
(background is injected transparent, so alpha = widget extent). Crop to that rect + Stretch=Fill
maps the widget flush under the selection box: full-canvas widgets fall through to the full frame
(identical to the confirmed-good image); floating widgets get tight-fitted with no dead space.
- Removes QueryContentBoundsAsync + the JS script entirely — the DOM-union attempt (a5b9952) broke
rendering by measuring an unsettled layout on NavigationCompleted; alpha measurement is immune.
- 3 new unit tests: tight box, no-alpha no-crop, full-alpha no-crop. 7/7 pass, 0 warnings.
- QueryContentBoundsAsync now returns a real JS object (not JSON.stringify — ExecuteScriptAsync
double-encodes strings, which silently killed the old crop) and measures the union of every
visible body element's getBoundingClientRect: the widget's actual rect, top-left offset included.
- CaptureFrame crops AT that offset (x,y) instead of from (0,0), so the widget anchors at origin
and the box is flush on right/bottom. Fill maps the frame flush under the box.
Bundle exception (2026-08-27): this lands prior-session work plus today's
transparency fix as one commit per creator instruction.
WebView2Manager (new): off-screen WebView2 control parked at -5000/-5000 in
the window's visual tree, captures via CapturePreviewAsync every 100 ms into
a BGRA8 VideoFrame. Source Width/Height sync to the control via
PropertyChanged. JS transparent-background injection runs on every
NavigationStarting/NavigationCompleted so the page's own CSS does not paint
white over the capture.
Transparency fix: DefaultBackgroundColor = System.Drawing.Color.Transparent
set on the WPF control before EnsureCoreWebView2Async. The WPF SDK doc says
the value is forwarded to the controller on init; the IDL (ICoreWebView2-
Controller2) confirms alpha=0 makes the capture alpha-preserving. The
prior-session 'CoreWebView2.BackgroundColor' path was the wrong property
name and has been replaced.
LayoutStore: WebUri column added (idempotent ALTER TABLE) and round-tripped
on save/load (schema v11).
MainViewModel: WebView2Manager constructed on Loaded; loaded web sources
re-registered; Source.WebUri PropertyChanged forwards to manager.
MainWindow: off-screen host Canvas added to visual tree.
Tests: 3 scale-sync + WebView2_DefaultBackgroundColor_Is_Transparent +
LayoutStorePersistenceTests.WebSource_WebUri_Persists. 234/236 pass
(2 pre-existing failures unchanged).
Scale bug (DPI/physical-pixel mismatch) still open.
gramps