Take 5: render 58.9 -> 25.5ms (138/300, still ~2.2x). The blits were fixed; the
resolver was not: ResolveOutputFrame -> RenderChatBox ran a FULL WPF raster
(FormattedText + RenderTargetBitmap + CopyPixels + channel swap) EVERY tick
whenever the chat buffer was non-empty — and the buffer survives sessions, so
even a signed-out record-only take paid it. Established answer (OBS text
sources): re-render on change, blit the cache every tick.
ChatOverlayLayer: content version bumped from Messages.CollectionChanged
(covers adds, the 500-cap removal, the fade Clear from any caller) + a config
key (size + all Chat* appearance props); RenderFrame returns the cached
VideoFrame by identity until either changes (compositor only reads cached
frames). Conservative ordering (version latched BEFORE render) makes a
mid-render message re-render next tick, never serve stale.
ONE integration test: ChatOverlayLayerCacheTests (RealApp, real renderer):
Same() for unchanged inputs, NotSame() on message/config change, null on
empty. Full regression green (62 across touched classes), clean build 0
warnings. Accepted cost pending take 6: one ~15-25ms tick per arriving
message; if live-chat bursts sag n/300, next slice = debounced off-tick
re-render. Docs same commit: ai.md pipeline section, TASKS.md TASK 18,
MyMistakes recipe (raster-on-change + session-surviving-buffer trap),
HANDOFF (take 6 -> then Unit B, spec unchanged).
Two defects made the producer 17x slow (37s record -> 2.1s/127-frame file,
rawvideo stamps by arrival): FramePump slept the FULL interval after each
render (period = render+submit+interval) and SceneCompositor did per-pixel
float sampling + Math.Round blends over all 2.07M master pixels, scanning the
whole destination per overlay (258ms avg render vs 1.5ms submit).
Both solutions are established, not invented — researched before coding per
the derivative-work rule:
- deadline pacing: OBS libobs/media-io/video-io.c video_thread (nextTick +=
intervalTicks, sleep only the remainder, rebase on overrun, never burst)
- row blits: libyuv pattern (BSD-3, chromium.googlesource.com/libyuv/libyuv)
— 1:1 aligned identity fast path, per-pixel alpha branch, integer
fixed-point blend, overlay clipped to the intersection rect, skip the dead
black pre-fill when the backdrop covers
ONE integration test: Pump_Paces_To_The_Deadline_Compensating_Render_Cost
(lands after a fake-seam lesson: pacing fakes must await, not complete
synchronously, or the pump loop runs inline on StartAsync and hangs vstest).
Clean build 0 warnings; FramePumpTests 10/10, SceneCompositor/SceneGraph/
SocialBar/StretchMath 20/20. Docs same commit: ai.md pipeline section,
TASKS.md TASK 18 (webcam-in-output + rename modal verified from take 3),
MyMistakes recipe, HANDOFF rewritten. Take 4 pending on the user's machine.
Take-two forensics: rawvideo carries no per-frame timestamps — ffmpeg stamps frames
by ARRIVAL at the declared fps, so a 2fps producer yields a 30x time-lapse, 0.6s-long
'60fps' file with zero errors anywhere (explains the creator's accelerated webcam
motion + truncated length observations). The pump now logs frames-produced-vs-target
per 5s with the avg render/submit split — take three names the slow stage instead
of us theorizing. FramePumpTests 9/9. (Multi-commit working unit: remaining declared
files land in the next commit.)
First real recording (2026-09-01): ffmpeg died at ~0.9s mid-session with ZERO log
lines while the pump fed a corpse for 9 more seconds. Three blind spots closed:
- OnStderrLine discarded everything non-progress: ffmpeg's actual words (startup
config, warnings, the reason it quit) now go to AppLog prefixed 'ffmpeg:' (400-char
cap so a \r-choked blob can't flood).
- Unexpected exit only fired ProcessFailed when code != 0 — a CLEAN exit vanished
silently. Any exit we didn't request is now logged and fired, any code.
- ExitCode read raced StopAsync's Dispose (tonight's 'No process is associated'
line) — guarded.
- VM output resolver logs once/5s when a webcam element resolves to a null frame
(preview showed the cam, recording didn't — the layer skip was invisible).
FfmpegEncoderTests + FramePumpTests 21/21 against the new code, 0 warnings.
The creator-declared known failure (Mix_HonorsProviderGains) and AudioPipelineTests'
standalone hang shared ONE root cause, born in TASK 22: AudioMixer.FillAndMix
dereferenced _delayedMix (declared float[]? , never assigned) as delayed.Length —
every live-mix tick NRE'd before the pipe write, so NO audio ever reached the wire
(tests starved -> hung/fail; app -> swallowed catch logged only ex.Message, 10ms
flood). Now: ctor-allocated + null-check, catch logs WITH stack via AppLog and is
throttled 5s, and a cancelled token breaks out before logging.
Bonus contract fix: AudioGainProvider.LoopbackGain returned unity on a disproven
premise ('loopback scales with endpoint volume'). The creator's tonight observation —
volume at 20%, meter pegged — proves the WASAPI tap is pre-endpoint-volume, so the
mixer must multiply by GameAudioVolume for stream honesty (this is what ai.md always
specified; the 'failing' test encoded the same and was RIGHT).
AudioPipelineTests: 25/25 in 26ms, standalone, no hang. Known-failure count: ZERO.
ai.md tests paragraph rewritten (no more suite-total claims, both ex-'knowns'
explained); TASK 22 regression recorded; MyMistakes: the known-failure-label rules.
Creator ruling 2026-09-01 (stuck REC pill after the failed first recording):
- StopStream clears RecordPillOn + OnAirPillOn — intent resets with reality.
- OnFramePumpFailed: dispatcher-marshalled FULL rollback via StopStream (was:
toast + Error-status only when live — record-only sessions zombied with
IsRecording=true over a dead encoder; the 20:34 attempt proved it). Toast copy
says 'Recording stopped' vs 'stream pipeline stopped'.
- Same zombie class in the three go-live prep failure branches: StreamStatus.Error
limbo replaced by StopStream() (audio loop + recording + pills unwind; a created
broadcast still gets the close-out).
- AudioMixer.StopLive made explicitly idempotent (Stop() twice, rollback paths that
never reached StartLive).
- Seams: OnFramePumpFailed + IsRecording setter internal (InternalsVisibleTo; test
pattern mirrors LayoutPathOverride).
ONE integration test: SessionTeardownTests (real window + temp DB: pump death with
lit pill -> no zombie, no End button, Offline). AudioPipelineTests confirmed to hang
STANDALONE (pre-existing, the declared-known audio class) — ai.md test-count
paragraph corrected to stop claiming a suite total that cannot currently be measured.
The specced 'End stream -> transition(complete)' call never existed: stopping relied
entirely on enableAutoStop (viewers sat on a frozen stream-offline for ~a minute, VOD
finalized late). Found during the 2026-09-01 recording-verification pass while the
creator asked 'if there's proper close-out info yt needs, we'll provide it?'
EndBroadcastAsync POSTs liveBroadcasts/transition?broadcastStatus=complete&id=..&part=status,
called after the pump stops (RTMP EOF first) and only when a live session had a broadcast —
record-only stops stay offline. invalidTransition/410 (autoStop already ended it) is logged
and returned as an error string, never thrown: a stop must never fail over close-out.
ONE integration test (URL shape + never-throws on 403). ai.md/TASKS.md design lines marked
SHIPPED with the map-lie note.
Ref: https://developers.google.com/youtube/v3/live/docs/liveBroadcasts/transition
The 2026-08-09 pin was a DAILY build; BtbN retention aged it out and the cold-cache
download 404'd on the creator's first native recording attempt (2026-09-01,
startup.log). Re-pinned to the MONTH-END build autobuild-2026-08-31-13-27
(N-126342, lgpl-shared win64 — 2-year retention; tag+variant recorded in TASKS.md
per the licensing rule). Verified alive: HEAD 200 + zip contents (bin/ffmpeg.exe,
bin/ffprobe.exe, 7 libav DLLs) match the name-agnostic extractor.
Also closes the wrap-gap: HttpRequestException escaped the locator untouched,
surfacing a raw 'Response status code... 404' from the frame pump; now wrapped in
IOException with the refresh-pin-or-install-ffmpeg message. FfmpegLocatorTests 7/7
(one updated, one added for the 404 case).
- IMediaFrameSource gains bool Looping.
- MediaVideoSource ctor takes Func<IDecodeProcess> processFactory instead of a
single IDecodeProcess: a System.Diagnostics.Process can't be re-Start()ed, so
each loop pass creates a fresh decoder. Decode wrapped in do-while(Looping):
restart on natural EOF instead of raising Completed.
- Production wiring (MainViewModel media factory): passes the process factory
AND FfmpegFrameRateProbe -- closes the slice-2b gap where production had no
probe and therefore no pacing.
- Tests: loop test (single frame re-emits across passes, Completed only when
loop cleared); fakes updated for the new interface member. Media tests 12/12,
build 0 warnings.
Wiring Source.MediaIsLooping into the flag needs a manager-level per-path loop
provider -> lands with the UI-picker (acquisition) slice.
Derivative reference: looping media by restarting decode on EOF, standard in
playback/overlay tooling (OBS media source repeat).
- MediaVideoSource takes optional IFrameRateProbe? + Func<TimeSpan,CancellationToken,Task>? delay
seams (default Task.Delay); probes FPS once in RunAsync, delays by 1/fps after each
emitted frame. No probe/unknown fps -> no pacing (ffmpeg pipe backpressure throttles).
- Test: MediaVideoSource_PacesFramesByProbedFps (fake probe returns 1000fps + recording
delay; one delay per frame ~= 1ms). Media tests 5/5, build 0 warnings.
Derivative reference: per-frame delay pacing of decoded output, standard in media playback.
- FfmpegFrameRateParser (pure): prefers avg_frame_rate= then r_frame_rate=,
rational N/N/M, unknown/0 -> null.
- IFrameRateProbe + FfmpegFrameRateProbe: derives sibling ffprobe.exe from the
located ffmpeg dir, reuses the IDecodeProcess seam for the ffprobe subprocess
text; null if ffprobe absent.
- FfmpegLocator now also extracts ffprobe.exe (ProbeFileName) from the pinned
archive, conditional so old caches without it degrade to no pacing.
- Tests: 6 pure parser units + 1 probe integration via fake locator/process;
FfmpegLocatorTests still green. 15/15, 0 warnings.
Pacing (probe->delay) is slice 2b. Derivative reference: standard ffprobe
avg_frame_rate probing used across OBS/media tooling.
Adds the codec-agnostic decoder half of the media source. Spawns ffmpeg
with -f rawvideo -pix_fmt bgra (reusing the already-shipped ffmpeg via
IFfmpegLocator) and drains the raw BGRA stdout pipe into VideoFrames.
- Services/RawVideoFrameReader.cs: pure rawvideo BGRA stream -> frames
(partial reads kept across Feed; no ffmpeg needed to test)
- Services/IDecodeProcess.cs + FfmpegDecodeProcess.cs: binary-stdout
subprocess seam, mirror of the encoder's IEncoderProcess
- Services/MediaVideoSource.cs: owns the decode, raises FrameReady/Completed
- MediaVideoSourceTests: 3 pure reader + 1 integration (fake decode
process through the real source loop, frames in order)
Reference (external scan): ffmpeg rawvideo pipe decode is the canonical
codec-agnostic frame feeds pattern (ffmpeg docs -f rawvideo; how OBS/media
pipelines push frames to a compositor). Verified: 4/4 tests, build 0 warnings.
Native-FPS pacing + resolver/compositor wiring are the next slice.
- Add ElementKind (Static/Dynamic) to SceneElement base; Source/WebcamSceneConfig classify
- Services/SceneGraph.cs: owns the Scenes collection (ViewModel's Scenes delegates to it),
the element mutation surface (Add/Insert/Remove/Move, each invalidating the bake), and the
queries that were scattered LINQ (GetBackground/GetWebcam/GetChatBoxes/GetSplitPoint/IsStatic)
- SceneCompositor: split-aware BakeStaticBase + CompositeLayers + Render(.., staticBase, split);
builds/caches the static base below the split point in source-rect space
- FramePump: optional SceneGraph -> optimized bake+composite path; falls back to full render
- MainViewModel: routes element mutations through the graph; invalidates the bake on static
layout/opacity/visibility/useDefaultBackground changes and after background heal/Ensure
- Integration test SceneGraphTests.BakedStaticBase_WithDynamicLayer_CompositesCorrectly
Derivative survey (mandated): tried before writing — OBS does per-source opacity/visibility
caching and static-scene baking; this mirrors OBS's 'cached static source' optimization.
3 documented defensive deviations from the spec (ChatOverlayLayer stays decoupled;
background helpers stay VM-static for direct testability; full facade peel deferred post-1.0)
in TASKS.md + ai.md. Scope check passed; clean build 0 warnings.
The first true decomposition of the 4105-line god-object, not another partial
shuffle. MainViewModel.Chat.cs 194 -> 44 lines of thin delegation; all chat
behavior now lives in a self-contained Services/ChatOverlayLayer.cs (199):
- Owns the message buffer (Messages), ChatBoxRenderer, fade + mock-preview
timers, per-source preview renders, and the live-output RenderFrame path.
- The VM keeps only the binding surface: ChatMessages delegates to
_chatLayer.Messages (so XAML ItemsSource + LeftPanel CollectionChanged hold),
CanAddYouTubeChat / ShowChatInactiveMessage stay computed on the VM
(OnPropertyChanged raised from VM setters + XAML-bound), and thin forwards.
- Scenes handed in as args (no Func seam), so the layer owns no scene graph.
An AI reading ChatOverlayLayer.cs now sees the entire chat feature in one
self-contained unit. Zero behavior change; build 0 warnings; 246 pass, only the
2 known failures. Docs: ViewModels/index.md tracker updated in same commit.
LayoutStore.cs 1402 -> 47-line shell + 5 partials, grouped by functionality
(line count is a consequence, not the goal):
- LayoutStore.Migrations.cs: all DB schema migration (EnsureSchema, GetUserVersion,
MigrateSourceTable/WebcamConfigTable/ToV3/SceneTable/SceneSocialBarColumn/
SocialsTable/SocialEntryTable)
- LayoutStore.Load.cs: Load() scene/source read-back
- LayoutStore.Save.cs: Save() full scene tree write
- LayoutStore.Settings.cs: key/value settings + mic/record-folder/reusable-stream
+ license/transition/broadcast/hotkey/window-state GetSetting/UpsertSetting
- LayoutStore.Assets.cs: GetAssetBytes/UpsertAsset blob IO
- LayoutStore.cs: shell — Instance, fields, ctor, Dispose
Converted to public partial class LayoutStore : IDisposable. Pruned per-file
dead usings. Phase 3 DOWN: no production .cs exceeds 500 lines.
Zero behavior change; build 0 warnings; 246 pass, only the 2 known failures.
Docs: ViewModels/index.md Phase 3 tracker updated in same commit.
Encoder dual-output: StreamEnabled/RecordEnabled/RecordPath options; per-output
ffmpeg block (stream ->flv, record ->mp4); throw unless at least one output.
RecordingFile: auto-name ty-<yyyymmdd>-<start hhmm>-0000.mp4, rename-on-stop to
ty-...-<len hhmmss>.mp4 with numeric-suffix on collision. LayoutStore persists
record folder. MainViewModel: REC/ON-AIR pills, status light, single Start<->End
button (replaces Start/Stop variants), StopStream no longer signs out, explicit
session timer. About overlay widened. Sign In label + spacing before Start.
Tests: 244 passed / 246 (2 pre-existing: AudioPipeline gains-mute, RoundClip).
Build 0 warnings. Scope check clean.
Memory-map correction (derived-solution rule, 2026-08-29): image-shrink recipe was
never recorded when first done, so it was re-derived. Added Derived-solution/recipes
rule to AGENTS.md; MyMistakes.md now a permanent recipes registry (records the
verified WPF-imaging shrink recipe); schema.md rows MyMistakes as registry; ai.md
points to the rule. README hero image replaced: 3.2MB screenshot -> 1400x794 ->
188KB docs/ytLlive-preview.jpg (JPEG q82).
- FindContentBounds already trimmed the bitmap; the grid/box still rendered at the uncropped user
size, so Fill stretched the tight widget into a bigger frame = dead-between-content-and-border.
- ContentCanvasSize maps the crop rect to canvas units (1920/pixW per axis, DPI-correct) and the
preview event now carries (bitmap, canvasW, canvasH).
- OnWebView2PreviewBitmapChanged sets the element Width/Height to that size; the grid and the
SelectionOverlay (both bound to the element's Width/Height) now equal the widget exactly.
- IsPreviewDragging (set in MainWindow drag/resize) suppresses auto-fit mid-gesture.
- Tests 10/10 (alpha bounds, canvas-size mapping incl. DPI, control size); 0 warnings.
- FindContentBounds scans the Bgra32 capture for the bounding box of non-transparent pixels
(background is injected transparent, so alpha = widget extent). Crop to that rect + Stretch=Fill
maps the widget flush under the selection box: full-canvas widgets fall through to the full frame
(identical to the confirmed-good image); floating widgets get tight-fitted with no dead space.
- Removes QueryContentBoundsAsync + the JS script entirely — the DOM-union attempt (a5b9952) broke
rendering by measuring an unsettled layout on NavigationCompleted; alpha measurement is immune.
- 3 new unit tests: tight box, no-alpha no-crop, full-alpha no-crop. 7/7 pass, 0 warnings.
The wrapper-excluded content-union crop measured an unsettled layout on NavigationCompleted
(async fonts/images, load animations) and produced a broken/zoomed sliver — the image was gone.
Restoring the exact confirmed-good pipeline; the bounding-box dead-space problem stays open and
will not be touched without explicit approval.
- scrollWidth/scrollHeight returns the whole 1920x1080 canvas for widgets whose body/background
spans it, so the crop kept massive transparent margins and the content floated in the Fill box.
- ContentBoundsScript now unions every visible element's getBoundingClientRect, excluding elements
that span >=98% of the viewport (bodies, full-canvas background layers); full-bleed widgets fall
back to the full rect. Crop taken AT the rect's top-left (l,t) — content pins to bitmap (0,0).
- Web Image explicitly Margin=0 + Left/Top alignment so the frame and widget coincide at the origin.
- Canvas-size render untouched.
- ContentBoundsScript back to scrollWidth/scrollHeight extents, kept as a real JS object
(the JSON.stringify double-encode bug that silently disabled the crop stays fixed).
- Crop again from (0,0), no X/Y offset. The union-of-elements approach (582d1f4) created an
all-sides gap by including invisible full-canvas wrappers; revert to the proven code.
- QueryContentBoundsAsync now returns a real JS object (not JSON.stringify — ExecuteScriptAsync
double-encodes strings, which silently killed the old crop) and measures the union of every
visible body element's getBoundingClientRect: the widget's actual rect, top-left offset included.
- CaptureFrame crops AT that offset (x,y) instead of from (0,0), so the widget anchors at origin
and the box is flush on right/bottom. Fill maps the frame flush under the box.
Bundle exception (2026-08-27): this lands prior-session work plus today's
transparency fix as one commit per creator instruction.
WebView2Manager (new): off-screen WebView2 control parked at -5000/-5000 in
the window's visual tree, captures via CapturePreviewAsync every 100 ms into
a BGRA8 VideoFrame. Source Width/Height sync to the control via
PropertyChanged. JS transparent-background injection runs on every
NavigationStarting/NavigationCompleted so the page's own CSS does not paint
white over the capture.
Transparency fix: DefaultBackgroundColor = System.Drawing.Color.Transparent
set on the WPF control before EnsureCoreWebView2Async. The WPF SDK doc says
the value is forwarded to the controller on init; the IDL (ICoreWebView2-
Controller2) confirms alpha=0 makes the capture alpha-preserving. The
prior-session 'CoreWebView2.BackgroundColor' path was the wrong property
name and has been replaced.
LayoutStore: WebUri column added (idempotent ALTER TABLE) and round-tripped
on save/load (schema v11).
MainViewModel: WebView2Manager constructed on Loaded; loaded web sources
re-registered; Source.WebUri PropertyChanged forwards to manager.
MainWindow: off-screen host Canvas added to visual tree.
Tests: 3 scale-sync + WebView2_DefaultBackgroundColor_Is_Transparent +
LayoutStorePersistenceTests.WebSource_WebUri_Persists. 234/236 pass
(2 pre-existing failures unchanged).
Scale bug (DPI/physical-pixel mismatch) still open.
- Source.ShowDesktop bool persisted in DB (schema migration + LayoutStore)
- ClearBackdropCaptureAsync clears CaptureKey + VideoImageSource for static fallback
- Top bar: red/green/pulsing status light, avatar via BitmapImage, Log In button
- ShowStartStream now requires IsConnected
- RunMockChatPreviewAsync: simple async loop, ChatPreviewEnabled property on Source
- EnsureDefaultBackdrop called on every scene switch, seeds all 5 scenes
- SeedEndingBackdrop replaces existing, SeedChatBackdrop added
- SceneCatalog: Chat scene display name corrected (layer = 'YouTube Chat')
- Left panel spacing consistency, context menu reordered
- Border animation rendering via code-behind Storyboard approach
- 3 new backdrop assets (ending-backdrop.jpg, chat-backdrop.jpg, live-backdrop.jpg)
- Build 0 warnings, 178 tests passing
- Elements panel config UI for ChatBox source (font size slider,
font/member/super-chat/background color inputs, max messages slider,
timestamp/badge toggles)
- Schema migration: 8 new ChatBox columns on Source table (ChatFontSize,
ChatFontColor, ChatBackgroundColor, ChatMaxMessages, ChatShowTimestamps,
ChatShowBadges, ChatMemberColor, ChatSuperChatColor)
- Load/save: Source SELECT and INSERT now include all ChatBox fields
- IsElementsPanelEmpty excludes ChatBox so hint doesn't cover config
- ChatBoxRenderer.cs created: WPF FormattedText + RenderTargetBitmap
renderer for on-stream chat overlay
- Slider bindings set to TwoWay for live propagation
- YouTubeStreamService: added GetBroadcastLiveChatIdAsync to fetch liveChatId from broadcast contentDetails
- YouTubeChatService.Poll(): parse snippet.type to handle textMessageEvent, superChatEvent, newSponsorEvent
- MainViewModel: wire _youtubeChat.Start(liveChatId) in PrepareAndStartLiveAsync after broadcast creation
- MainViewModel: wire _youtubeChat.Stop() in StopStream before encoder stop
- MainViewModel: wire _youtubeChat.Dispose() in Shutdown
- MainWindow.xaml: updated chat DataTemplate with member badge (⭐), Super Chat highlight (💰 + gold border), timestamps (HH:mm)
- MainWindow.xaml.cs: auto-scroll ChatListBox on new messages via CollectionChanged
- Chat lifecycle is non-fatal: failures log and the stream continues without on-panel chat
Issue 3: Desktop meter now decays 2.5x faster than it rises
(DecaySmoothing=0.5 vs Smoothing=0.2). When rms drops below current
level the higher alpha pulls the bar down quickly; when audio returns
it rises smoothly. 207 tests pass, 0 warnings.
gramps