Two defects made the producer 17x slow (37s record -> 2.1s/127-frame file,
rawvideo stamps by arrival): FramePump slept the FULL interval after each
render (period = render+submit+interval) and SceneCompositor did per-pixel
float sampling + Math.Round blends over all 2.07M master pixels, scanning the
whole destination per overlay (258ms avg render vs 1.5ms submit).
Both solutions are established, not invented — researched before coding per
the derivative-work rule:
- deadline pacing: OBS libobs/media-io/video-io.c video_thread (nextTick +=
intervalTicks, sleep only the remainder, rebase on overrun, never burst)
- row blits: libyuv pattern (BSD-3, chromium.googlesource.com/libyuv/libyuv)
— 1:1 aligned identity fast path, per-pixel alpha branch, integer
fixed-point blend, overlay clipped to the intersection rect, skip the dead
black pre-fill when the backdrop covers
ONE integration test: Pump_Paces_To_The_Deadline_Compensating_Render_Cost
(lands after a fake-seam lesson: pacing fakes must await, not complete
synchronously, or the pump loop runs inline on StartAsync and hangs vstest).
Clean build 0 warnings; FramePumpTests 10/10, SceneCompositor/SceneGraph/
SocialBar/StretchMath 20/20. Docs same commit: ai.md pipeline section,
TASKS.md TASK 18 (webcam-in-output + rename modal verified from take 3),
MyMistakes recipe, HANDOFF rewritten. Take 4 pending on the user's machine.
Take-two forensics: rawvideo carries no per-frame timestamps — ffmpeg stamps frames
by ARRIVAL at the declared fps, so a 2fps producer yields a 30x time-lapse, 0.6s-long
'60fps' file with zero errors anywhere (explains the creator's accelerated webcam
motion + truncated length observations). The pump now logs frames-produced-vs-target
per 5s with the avg render/submit split — take three names the slow stage instead
of us theorizing. FramePumpTests 9/9. (Multi-commit working unit: remaining declared
files land in the next commit.)
First real recording (2026-09-01): ffmpeg died at ~0.9s mid-session with ZERO log
lines while the pump fed a corpse for 9 more seconds. Three blind spots closed:
- OnStderrLine discarded everything non-progress: ffmpeg's actual words (startup
config, warnings, the reason it quit) now go to AppLog prefixed 'ffmpeg:' (400-char
cap so a \r-choked blob can't flood).
- Unexpected exit only fired ProcessFailed when code != 0 — a CLEAN exit vanished
silently. Any exit we didn't request is now logged and fired, any code.
- ExitCode read raced StopAsync's Dispose (tonight's 'No process is associated'
line) — guarded.
- VM output resolver logs once/5s when a webcam element resolves to a null frame
(preview showed the cam, recording didn't — the layer skip was invisible).
FfmpegEncoderTests + FramePumpTests 21/21 against the new code, 0 warnings.
The creator-declared known failure (Mix_HonorsProviderGains) and AudioPipelineTests'
standalone hang shared ONE root cause, born in TASK 22: AudioMixer.FillAndMix
dereferenced _delayedMix (declared float[]? , never assigned) as delayed.Length —
every live-mix tick NRE'd before the pipe write, so NO audio ever reached the wire
(tests starved -> hung/fail; app -> swallowed catch logged only ex.Message, 10ms
flood). Now: ctor-allocated + null-check, catch logs WITH stack via AppLog and is
throttled 5s, and a cancelled token breaks out before logging.
Bonus contract fix: AudioGainProvider.LoopbackGain returned unity on a disproven
premise ('loopback scales with endpoint volume'). The creator's tonight observation —
volume at 20%, meter pegged — proves the WASAPI tap is pre-endpoint-volume, so the
mixer must multiply by GameAudioVolume for stream honesty (this is what ai.md always
specified; the 'failing' test encoded the same and was RIGHT).
AudioPipelineTests: 25/25 in 26ms, standalone, no hang. Known-failure count: ZERO.
ai.md tests paragraph rewritten (no more suite-total claims, both ex-'knowns'
explained); TASK 22 regression recorded; MyMistakes: the known-failure-label rules.
Creator ruling 2026-09-01 (stuck REC pill after the failed first recording):
- StopStream clears RecordPillOn + OnAirPillOn — intent resets with reality.
- OnFramePumpFailed: dispatcher-marshalled FULL rollback via StopStream (was:
toast + Error-status only when live — record-only sessions zombied with
IsRecording=true over a dead encoder; the 20:34 attempt proved it). Toast copy
says 'Recording stopped' vs 'stream pipeline stopped'.
- Same zombie class in the three go-live prep failure branches: StreamStatus.Error
limbo replaced by StopStream() (audio loop + recording + pills unwind; a created
broadcast still gets the close-out).
- AudioMixer.StopLive made explicitly idempotent (Stop() twice, rollback paths that
never reached StartLive).
- Seams: OnFramePumpFailed + IsRecording setter internal (InternalsVisibleTo; test
pattern mirrors LayoutPathOverride).
ONE integration test: SessionTeardownTests (real window + temp DB: pump death with
lit pill -> no zombie, no End button, Offline). AudioPipelineTests confirmed to hang
STANDALONE (pre-existing, the declared-known audio class) — ai.md test-count
paragraph corrected to stop claiming a suite total that cannot currently be measured.
The specced 'End stream -> transition(complete)' call never existed: stopping relied
entirely on enableAutoStop (viewers sat on a frozen stream-offline for ~a minute, VOD
finalized late). Found during the 2026-09-01 recording-verification pass while the
creator asked 'if there's proper close-out info yt needs, we'll provide it?'
EndBroadcastAsync POSTs liveBroadcasts/transition?broadcastStatus=complete&id=..&part=status,
called after the pump stops (RTMP EOF first) and only when a live session had a broadcast —
record-only stops stay offline. invalidTransition/410 (autoStop already ended it) is logged
and returned as an error string, never thrown: a stop must never fail over close-out.
ONE integration test (URL shape + never-throws on 403). ai.md/TASKS.md design lines marked
SHIPPED with the map-lie note.
Ref: https://developers.google.com/youtube/v3/live/docs/liveBroadcasts/transition
The 2026-08-09 pin was a DAILY build; BtbN retention aged it out and the cold-cache
download 404'd on the creator's first native recording attempt (2026-09-01,
startup.log). Re-pinned to the MONTH-END build autobuild-2026-08-31-13-27
(N-126342, lgpl-shared win64 — 2-year retention; tag+variant recorded in TASKS.md
per the licensing rule). Verified alive: HEAD 200 + zip contents (bin/ffmpeg.exe,
bin/ffprobe.exe, 7 libav DLLs) match the name-agnostic extractor.
Also closes the wrap-gap: HttpRequestException escaped the locator untouched,
surfacing a raw 'Response status code... 404' from the frame pump; now wrapped in
IOException with the refresh-pin-or-install-ffmpeg message. FfmpegLocatorTests 7/7
(one updated, one added for the 404 case).
- IMediaFrameSource gains bool Looping.
- MediaVideoSource ctor takes Func<IDecodeProcess> processFactory instead of a
single IDecodeProcess: a System.Diagnostics.Process can't be re-Start()ed, so
each loop pass creates a fresh decoder. Decode wrapped in do-while(Looping):
restart on natural EOF instead of raising Completed.
- Production wiring (MainViewModel media factory): passes the process factory
AND FfmpegFrameRateProbe -- closes the slice-2b gap where production had no
probe and therefore no pacing.
- Tests: loop test (single frame re-emits across passes, Completed only when
loop cleared); fakes updated for the new interface member. Media tests 12/12,
build 0 warnings.
Wiring Source.MediaIsLooping into the flag needs a manager-level per-path loop
provider -> lands with the UI-picker (acquisition) slice.
Derivative reference: looping media by restarting decode on EOF, standard in
playback/overlay tooling (OBS media source repeat).
- MediaVideoSource takes optional IFrameRateProbe? + Func<TimeSpan,CancellationToken,Task>? delay
seams (default Task.Delay); probes FPS once in RunAsync, delays by 1/fps after each
emitted frame. No probe/unknown fps -> no pacing (ffmpeg pipe backpressure throttles).
- Test: MediaVideoSource_PacesFramesByProbedFps (fake probe returns 1000fps + recording
delay; one delay per frame ~= 1ms). Media tests 5/5, build 0 warnings.
Derivative reference: per-frame delay pacing of decoded output, standard in media playback.
- FfmpegFrameRateParser (pure): prefers avg_frame_rate= then r_frame_rate=,
rational N/N/M, unknown/0 -> null.
- IFrameRateProbe + FfmpegFrameRateProbe: derives sibling ffprobe.exe from the
located ffmpeg dir, reuses the IDecodeProcess seam for the ffprobe subprocess
text; null if ffprobe absent.
- FfmpegLocator now also extracts ffprobe.exe (ProbeFileName) from the pinned
archive, conditional so old caches without it degrade to no pacing.
- Tests: 6 pure parser units + 1 probe integration via fake locator/process;
FfmpegLocatorTests still green. 15/15, 0 warnings.
Pacing (probe->delay) is slice 2b. Derivative reference: standard ffprobe
avg_frame_rate probing used across OBS/media tooling.
Adds the codec-agnostic decoder half of the media source. Spawns ffmpeg
with -f rawvideo -pix_fmt bgra (reusing the already-shipped ffmpeg via
IFfmpegLocator) and drains the raw BGRA stdout pipe into VideoFrames.
- Services/RawVideoFrameReader.cs: pure rawvideo BGRA stream -> frames
(partial reads kept across Feed; no ffmpeg needed to test)
- Services/IDecodeProcess.cs + FfmpegDecodeProcess.cs: binary-stdout
subprocess seam, mirror of the encoder's IEncoderProcess
- Services/MediaVideoSource.cs: owns the decode, raises FrameReady/Completed
- MediaVideoSourceTests: 3 pure reader + 1 integration (fake decode
process through the real source loop, frames in order)
Reference (external scan): ffmpeg rawvideo pipe decode is the canonical
codec-agnostic frame feeds pattern (ffmpeg docs -f rawvideo; how OBS/media
pipelines push frames to a compositor). Verified: 4/4 tests, build 0 warnings.
Native-FPS pacing + resolver/compositor wiring are the next slice.
- Add ElementKind (Static/Dynamic) to SceneElement base; Source/WebcamSceneConfig classify
- Services/SceneGraph.cs: owns the Scenes collection (ViewModel's Scenes delegates to it),
the element mutation surface (Add/Insert/Remove/Move, each invalidating the bake), and the
queries that were scattered LINQ (GetBackground/GetWebcam/GetChatBoxes/GetSplitPoint/IsStatic)
- SceneCompositor: split-aware BakeStaticBase + CompositeLayers + Render(.., staticBase, split);
builds/caches the static base below the split point in source-rect space
- FramePump: optional SceneGraph -> optimized bake+composite path; falls back to full render
- MainViewModel: routes element mutations through the graph; invalidates the bake on static
layout/opacity/visibility/useDefaultBackground changes and after background heal/Ensure
- Integration test SceneGraphTests.BakedStaticBase_WithDynamicLayer_CompositesCorrectly
Derivative survey (mandated): tried before writing — OBS does per-source opacity/visibility
caching and static-scene baking; this mirrors OBS's 'cached static source' optimization.
3 documented defensive deviations from the spec (ChatOverlayLayer stays decoupled;
background helpers stay VM-static for direct testability; full facade peel deferred post-1.0)
in TASKS.md + ai.md. Scope check passed; clean build 0 warnings.
The first true decomposition of the 4105-line god-object, not another partial
shuffle. MainViewModel.Chat.cs 194 -> 44 lines of thin delegation; all chat
behavior now lives in a self-contained Services/ChatOverlayLayer.cs (199):
- Owns the message buffer (Messages), ChatBoxRenderer, fade + mock-preview
timers, per-source preview renders, and the live-output RenderFrame path.
- The VM keeps only the binding surface: ChatMessages delegates to
_chatLayer.Messages (so XAML ItemsSource + LeftPanel CollectionChanged hold),
CanAddYouTubeChat / ShowChatInactiveMessage stay computed on the VM
(OnPropertyChanged raised from VM setters + XAML-bound), and thin forwards.
- Scenes handed in as args (no Func seam), so the layer owns no scene graph.
An AI reading ChatOverlayLayer.cs now sees the entire chat feature in one
self-contained unit. Zero behavior change; build 0 warnings; 246 pass, only the
2 known failures. Docs: ViewModels/index.md tracker updated in same commit.
LayoutStore.cs 1402 -> 47-line shell + 5 partials, grouped by functionality
(line count is a consequence, not the goal):
- LayoutStore.Migrations.cs: all DB schema migration (EnsureSchema, GetUserVersion,
MigrateSourceTable/WebcamConfigTable/ToV3/SceneTable/SceneSocialBarColumn/
SocialsTable/SocialEntryTable)
- LayoutStore.Load.cs: Load() scene/source read-back
- LayoutStore.Save.cs: Save() full scene tree write
- LayoutStore.Settings.cs: key/value settings + mic/record-folder/reusable-stream
+ license/transition/broadcast/hotkey/window-state GetSetting/UpsertSetting
- LayoutStore.Assets.cs: GetAssetBytes/UpsertAsset blob IO
- LayoutStore.cs: shell — Instance, fields, ctor, Dispose
Converted to public partial class LayoutStore : IDisposable. Pruned per-file
dead usings. Phase 3 DOWN: no production .cs exceeds 500 lines.
Zero behavior change; build 0 warnings; 246 pass, only the 2 known failures.
Docs: ViewModels/index.md Phase 3 tracker updated in same commit.
Encoder dual-output: StreamEnabled/RecordEnabled/RecordPath options; per-output
ffmpeg block (stream ->flv, record ->mp4); throw unless at least one output.
RecordingFile: auto-name ty-<yyyymmdd>-<start hhmm>-0000.mp4, rename-on-stop to
ty-...-<len hhmmss>.mp4 with numeric-suffix on collision. LayoutStore persists
record folder. MainViewModel: REC/ON-AIR pills, status light, single Start<->End
button (replaces Start/Stop variants), StopStream no longer signs out, explicit
session timer. About overlay widened. Sign In label + spacing before Start.
Tests: 244 passed / 246 (2 pre-existing: AudioPipeline gains-mute, RoundClip).
Build 0 warnings. Scope check clean.
Memory-map correction (derived-solution rule, 2026-08-29): image-shrink recipe was
never recorded when first done, so it was re-derived. Added Derived-solution/recipes
rule to AGENTS.md; MyMistakes.md now a permanent recipes registry (records the
verified WPF-imaging shrink recipe); schema.md rows MyMistakes as registry; ai.md
points to the rule. README hero image replaced: 3.2MB screenshot -> 1400x794 ->
188KB docs/ytLlive-preview.jpg (JPEG q82).
- FindContentBounds already trimmed the bitmap; the grid/box still rendered at the uncropped user
size, so Fill stretched the tight widget into a bigger frame = dead-between-content-and-border.
- ContentCanvasSize maps the crop rect to canvas units (1920/pixW per axis, DPI-correct) and the
preview event now carries (bitmap, canvasW, canvasH).
- OnWebView2PreviewBitmapChanged sets the element Width/Height to that size; the grid and the
SelectionOverlay (both bound to the element's Width/Height) now equal the widget exactly.
- IsPreviewDragging (set in MainWindow drag/resize) suppresses auto-fit mid-gesture.
- Tests 10/10 (alpha bounds, canvas-size mapping incl. DPI, control size); 0 warnings.
- FindContentBounds scans the Bgra32 capture for the bounding box of non-transparent pixels
(background is injected transparent, so alpha = widget extent). Crop to that rect + Stretch=Fill
maps the widget flush under the selection box: full-canvas widgets fall through to the full frame
(identical to the confirmed-good image); floating widgets get tight-fitted with no dead space.
- Removes QueryContentBoundsAsync + the JS script entirely — the DOM-union attempt (a5b9952) broke
rendering by measuring an unsettled layout on NavigationCompleted; alpha measurement is immune.
- 3 new unit tests: tight box, no-alpha no-crop, full-alpha no-crop. 7/7 pass, 0 warnings.
The wrapper-excluded content-union crop measured an unsettled layout on NavigationCompleted
(async fonts/images, load animations) and produced a broken/zoomed sliver — the image was gone.
Restoring the exact confirmed-good pipeline; the bounding-box dead-space problem stays open and
will not be touched without explicit approval.
- scrollWidth/scrollHeight returns the whole 1920x1080 canvas for widgets whose body/background
spans it, so the crop kept massive transparent margins and the content floated in the Fill box.
- ContentBoundsScript now unions every visible element's getBoundingClientRect, excluding elements
that span >=98% of the viewport (bodies, full-canvas background layers); full-bleed widgets fall
back to the full rect. Crop taken AT the rect's top-left (l,t) — content pins to bitmap (0,0).
- Web Image explicitly Margin=0 + Left/Top alignment so the frame and widget coincide at the origin.
- Canvas-size render untouched.
- ContentBoundsScript back to scrollWidth/scrollHeight extents, kept as a real JS object
(the JSON.stringify double-encode bug that silently disabled the crop stays fixed).
- Crop again from (0,0), no X/Y offset. The union-of-elements approach (582d1f4) created an
all-sides gap by including invisible full-canvas wrappers; revert to the proven code.
- QueryContentBoundsAsync now returns a real JS object (not JSON.stringify — ExecuteScriptAsync
double-encodes strings, which silently killed the old crop) and measures the union of every
visible body element's getBoundingClientRect: the widget's actual rect, top-left offset included.
- CaptureFrame crops AT that offset (x,y) instead of from (0,0), so the widget anchors at origin
and the box is flush on right/bottom. Fill maps the frame flush under the box.
Bundle exception (2026-08-27): this lands prior-session work plus today's
transparency fix as one commit per creator instruction.
WebView2Manager (new): off-screen WebView2 control parked at -5000/-5000 in
the window's visual tree, captures via CapturePreviewAsync every 100 ms into
a BGRA8 VideoFrame. Source Width/Height sync to the control via
PropertyChanged. JS transparent-background injection runs on every
NavigationStarting/NavigationCompleted so the page's own CSS does not paint
white over the capture.
Transparency fix: DefaultBackgroundColor = System.Drawing.Color.Transparent
set on the WPF control before EnsureCoreWebView2Async. The WPF SDK doc says
the value is forwarded to the controller on init; the IDL (ICoreWebView2-
Controller2) confirms alpha=0 makes the capture alpha-preserving. The
prior-session 'CoreWebView2.BackgroundColor' path was the wrong property
name and has been replaced.
LayoutStore: WebUri column added (idempotent ALTER TABLE) and round-tripped
on save/load (schema v11).
MainViewModel: WebView2Manager constructed on Loaded; loaded web sources
re-registered; Source.WebUri PropertyChanged forwards to manager.
MainWindow: off-screen host Canvas added to visual tree.
Tests: 3 scale-sync + WebView2_DefaultBackgroundColor_Is_Transparent +
LayoutStorePersistenceTests.WebSource_WebUri_Persists. 234/236 pass
(2 pre-existing failures unchanged).
Scale bug (DPI/physical-pixel mismatch) still open.
- Source.ShowDesktop bool persisted in DB (schema migration + LayoutStore)
- ClearBackdropCaptureAsync clears CaptureKey + VideoImageSource for static fallback
- Top bar: red/green/pulsing status light, avatar via BitmapImage, Log In button
- ShowStartStream now requires IsConnected
- RunMockChatPreviewAsync: simple async loop, ChatPreviewEnabled property on Source
- EnsureDefaultBackdrop called on every scene switch, seeds all 5 scenes
- SeedEndingBackdrop replaces existing, SeedChatBackdrop added
- SceneCatalog: Chat scene display name corrected (layer = 'YouTube Chat')
- Left panel spacing consistency, context menu reordered
- Border animation rendering via code-behind Storyboard approach
- 3 new backdrop assets (ending-backdrop.jpg, chat-backdrop.jpg, live-backdrop.jpg)
- Build 0 warnings, 178 tests passing
- Elements panel config UI for ChatBox source (font size slider,
font/member/super-chat/background color inputs, max messages slider,
timestamp/badge toggles)
- Schema migration: 8 new ChatBox columns on Source table (ChatFontSize,
ChatFontColor, ChatBackgroundColor, ChatMaxMessages, ChatShowTimestamps,
ChatShowBadges, ChatMemberColor, ChatSuperChatColor)
- Load/save: Source SELECT and INSERT now include all ChatBox fields
- IsElementsPanelEmpty excludes ChatBox so hint doesn't cover config
- ChatBoxRenderer.cs created: WPF FormattedText + RenderTargetBitmap
renderer for on-stream chat overlay
- Slider bindings set to TwoWay for live propagation
- YouTubeStreamService: added GetBroadcastLiveChatIdAsync to fetch liveChatId from broadcast contentDetails
- YouTubeChatService.Poll(): parse snippet.type to handle textMessageEvent, superChatEvent, newSponsorEvent
- MainViewModel: wire _youtubeChat.Start(liveChatId) in PrepareAndStartLiveAsync after broadcast creation
- MainViewModel: wire _youtubeChat.Stop() in StopStream before encoder stop
- MainViewModel: wire _youtubeChat.Dispose() in Shutdown
- MainWindow.xaml: updated chat DataTemplate with member badge (⭐), Super Chat highlight (💰 + gold border), timestamps (HH:mm)
- MainWindow.xaml.cs: auto-scroll ChatListBox on new messages via CollectionChanged
- Chat lifecycle is non-fatal: failures log and the stream continues without on-panel chat
Issue 3: Desktop meter now decays 2.5x faster than it rises
(DecaySmoothing=0.5 vs Smoothing=0.2). When rms drops below current
level the higher alpha pulls the bar down quickly; when audio returns
it rises smoothly. 207 tests pass, 0 warnings.
1. Error text shows in red inside the desktop audio meter bar (scrollable)
when TRAX load or playback fails — user gets feedback, not just a
silent red dot.
2. LoadTrax no longer nulls _music on error — the track path is retained
so the user sees what failed and can retry.
3. OnTraxPlaybackEnded now receives the NAudio exception and surfaces it.
4. TRAX dot is green (playing) or gray (idle/no track) — no more yellow
or red states.
207 tests, 0 warnings.
gramps