gramps/LlamaCasty
0
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Zoom: anchor the capture, magnify the full composite, draw on top (Unit: zoom rework per creator correction)

Browse Source 
Creator corrected the "loupe" zoom on 2026-10-08: the click anchors a fixed
capture region (box/2 centred on the click) and the box is a live 2x view of
that region — dragging the box never changes its content; it magnifies the
fully-composited frame (every layer, incl. webcam) and draws LAST (n+1).
Rectangle shape is 4:3 ("more of a 640x480 aspect").

- MainViewModel.Zoom.cs rewritten: ActivateZoom anchors CaptureX/CaptureY
  (independently clamped); UpdateZoomMove relocates the box only; the per-frame
  BuildZoomFrame/ZoomFrameBuilder path is gone.
- SceneCompositor: skips the active ZoomSceneElement in every layer loop and
  BlitZooms it last via the new pure ZoomMagnifier (2x bilinear, temp snapshot
  via ArrayPool, round clip).
- Preview: ZoomBox hoisted above CanvasGrid as a sibling; its brush samples the
  whole composed preview at the fixed capture (2x).
- Tests: ZoomMagnifierTests (box-over-blue shows green capture at 2x incl. an
  extra red "layer" — the loupe/backdrop-only regressions both fail it) +
  round-clip; ZoomBoxInteractionTests rewritten (anchored capture, move-doesn't-
  change-capture, 4:3/9:16/square). 404 pass, 2 known environment flakes.
This commit is contained in:
gramps
2026-10-08 08:26:29 -07:00
parent 4b3d82ff0f
commit 6bff21b234
15 changed files with 467 additions and 330 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ViewModels/MainViewModel.Zoom.cs
+32 -182
View File
@@ -2,18 +2,20 @@ using System;
using System.Linq;
using System.Windows;
using ytLive.Models;
using ytLive.Services;
using ytLive.Services.Compositor;
namespace ytLive.ViewModels;
/// <summary>
/// The zoom box as a real composited layer (TASK 51): right-clicking the live
/// preview places a 2x magnifier one z-slot above the background and below every
/// other layer. The box drives a transient <see cref="ZoomSceneElement"/> whose
/// pixels are the magnified backdrop inside the box; the frame pump composites it
/// into the stream/recording like any other layer (output stays the normal tier
/// crop — the box is NEVER the crop source). All shapes are centered on the click.
/// The zoom box (creator-corrected model, 2026-10-08): right-clicking the live
/// preview anchors a CAPTURE window — the box/2 region centred on the click — and
/// parks a movable 2x magnifier box on screen. The box shows a live, always-on,
/// magnified view of that anchored capture region (the CLICKED area, never re-follows
/// the box); dragging it only repositions the box. The content is composited LAST
/// (layer n+1) by <see cref="Services.Compositor.SceneCompositor"/> from the already-
/// composed frame, so it captures every layer — prove it with the ZoomMagnifierTests
/// banded-buffer "box over blue shows green" proof. The transient
/// <see cref="ZoomSceneElement"/> carries the box placement + capture anchor; the
/// output crop stays the normal tier (the box is NEVER the crop source).
/// </summary>
public partial class MainViewModel
{
@@ -34,18 +36,27 @@ public partial class MainViewModel
w = h * 9.0 / 16.0;
break;
default:
w = maxW;
h = w * 9.0 / 16.0;
// HorizontalRectangle: 4:3 per creator ("more of a 640x480 aspect
// ratio") — NOT the 16:9 of the first cut, which read as a square.
h = maxH;
w = h * 4.0 / 3.0;
break;
}
// Option B anchor: the click point is the CENTER of every shape. Clamped
// so the whole box stays inside the master 1920x1080 frame.
// The box AND the capture are both centred on the click; the capture window
// is the box/2 interior and is FIXED after this call. Each is clamped to the
// master 1920x1080 frame independently (an edge click can push the capture
// toward centre while the box still sits right at the edge).
var x = Math.Clamp(clickX - w / 2, 0, MasterFrameWidth - w);
var y = Math.Clamp(clickY - h / 2, 0, MasterFrameHeight - h);
var capX = Math.Clamp(clickX - w / 4, 0, MasterFrameWidth - w / 2);
var capY = Math.Clamp(clickY - h / 4, 0, MasterFrameHeight - h / 2);
_zoomState.X = x;
_zoomState.Y = y;
_zoomState.Width = w;
_zoomState.Height = h;
_zoomState.CaptureX = capX;
_zoomState.CaptureY = capY;
_zoomState.IsActive = true;
_zoomState.IsMoving = false;
@@ -54,6 +65,8 @@ public partial class MainViewModel
zoomTarget.Y = y;
zoomTarget.Width = w;
zoomTarget.Height = h;
zoomTarget.CaptureX = capX;
zoomTarget.CaptureY = capY;
zoomTarget.ClipShape = _zoomShape == ZoomShape.Circle ? ClipShape.Round : ClipShape.Traditional;
zoomTarget.IsZoomActive = true;
zoomTarget.IsVisible = true;
@@ -76,6 +89,8 @@ public partial class MainViewModel
{
zoomTarget.IsZoomActive = false;
zoomTarget.IsVisible = false;
zoomTarget.CaptureX = 0;
zoomTarget.CaptureY = 0;
}
if (StagedScene != null) _sceneGraph.InvalidateBake(StagedScene);
RefreshZoomUi();
@@ -102,6 +117,8 @@ public partial class MainViewModel
{
zoomTarget.X = _zoomState.X;
zoomTarget.Y = _zoomState.Y;
// CaptureX/Y are deliberately NOT touched: dragging relocates the box,
// never the anchored captured area (creator ruling 2026-10-08).
}
OnPropertyChanged(nameof(ZoomState));
RebuildDimRectsForZoom();
@@ -145,9 +162,9 @@ public partial class MainViewModel
}
/// <summary>Heals a missing zoom placeholder on every scene after a layout
/// load: the invisible 0x0 row parked at z-index 1 (above the background,
/// below every other layer), mirroring how backgrounds heal. The zoom box
/// itself is transient — the placeholder survives so the layer has a seat.</summary>
/// load: the invisible 0x0 row mirrored by the healed backgrounds. The zoom box
/// itself is transient — the placeholder survives so the layer has a seat. The
/// compositor ranks the ACTIVE box last regardless of this index.</summary>
private void HealZoomLayers()
{
foreach (var scene in Scenes)
@@ -163,171 +180,4 @@ public partial class MainViewModel
_sceneGraph.InsertElement(scene!, Math.Min(1, scene!.Elements.Count), zoom);
return zoom;
}
// Single-slot zoom frame cache: the frame-pump resolver runs every tick, so
// rebuilding the ~4 MB box raster per tick would churn the LOH even when the
// backdrop and box did not change. One active zoom → one cached frame; the
// key is the backdrop frames' identity+Epoch and the box rect's ints.
private VideoFrame? _zoomFrame;
private (byte[]?, long?, byte[]?, long?, int, int, int, int) _zoomFrameKey;
private VideoFrame? BuildZoomFrame(ZoomSceneElement zoom)
{
var scene = StagedScene;
if (scene == null) return null;
// Mirror the compositor's backdrop ordering: live capture first, static
// background art over it (SceneCompositor.Render, same conditions — live
// only when visible, art whenever present).
var live = scene.Elements.OfType<Source>().FirstOrDefault(s => s.IsBackground);
var liveFrame = live is { IsVisible: true } ? ResolveOutputFrameCore(live) : null;
var art = scene.Elements.OfType<Source>().FirstOrDefault(s => s.Type == SourceType.Background);
var artFrame = art != null ? ResolveOutputFrameCore(art) : null;
if (liveFrame == null && artFrame == null) return null;
var zx = (int)Math.Round(zoom.X);
var zy = (int)Math.Round(zoom.Y);
var zw = (int)Math.Ceiling(zoom.Width);
var zh = (int)Math.Ceiling(zoom.Height);
var key = (liveFrame?.BgraPixels, liveFrame?.Epoch, artFrame?.BgraPixels, artFrame?.Epoch, zx, zy, zw, zh);
if (key != _zoomFrameKey)
{
_zoomFrame = ZoomFrameBuilder.Build(
liveFrame, artFrame, zx, zy, zw, zh, MasterFrameWidth, MasterFrameHeight);
_zoomFrameKey = key;
}
return _zoomFrame;
}
}
/// <summary>Pure pixel builder for the zoom layer: samples the box/2 interior
/// centred on the box centre from the sequenced backdrop (live capture then art)
/// and upscales it 2x — the same region and magnification the preview's
/// VisualBrush shows. Static, so the magnification proof is a plain unit test.</summary>
internal static class ZoomFrameBuilder
{
internal static VideoFrame? Build(
VideoFrame? liveFrame, VideoFrame? artFrame,
int zx, int zy, int zw, int zh,
double masterW, double masterH)
{
if (zw <= 0 || zh <= 0) return null;
if (liveFrame == null && artFrame == null) return null;
var buffer = new byte[zw * zh * 4];
for (var oy = 0; oy < zh; oy++)
{
// Source window = the box/2 interior CENTERED on the box center (the
// same region as the preview's VisualBrush Viewbox), sampled at 2x.
var my = zy + zh / 4.0 + (oy + 0.5) * 0.5;
var di = oy * zw * 4;
for (var ox = 0; ox < zw; ox++, di += 4)
{
var mx = zx + zw / 4.0 + (ox + 0.5) * 0.5;
var b = 0;
var g = 0;
var r = 0;
var a = 0;
if (liveFrame != null)
Sample(liveFrame, mx, my, masterW, masterH, ref b, ref g, ref r, ref a);
if (artFrame != null)
{
var ab = 0;
var ag = 0;
var ar = 0;
var aa = 0;
Sample(artFrame, mx, my, masterW, masterH, ref ab, ref ag, ref ar, ref aa);
if (aa >= 255)
{
b = ab; g = ag; r = ar;
}
else if (aa > 0)
{
var inv = 255 - aa;
b = (b * inv + ab * aa + 127) / 255;
g = (g * inv + ag * aa + 127) / 255;
r = (r * inv + ar * aa + 127) / 255;
}
a = 255;
}
else if (liveFrame != null)
{
a = 255;
}
buffer[di] = (byte)b;
buffer[di + 1] = (byte)g;
buffer[di + 2] = (byte)r;
buffer[di + 3] = (byte)a;
}
}
return new VideoFrame(zw, zh, buffer);
}
/// <summary>Inverse of the compositor's full-frame backdrop blit: maps a master
/// pixel back through <see cref="StretchMath.UniformToFill"/> (or the frame's
/// CropBounds) into the source canvas and bilinear-samples it. Fixed-point
/// bilinear, same precision as SceneCompositor.Bilinear.</summary>
private static void Sample(
VideoFrame src, double mx, double my, double masterW, double masterH,
ref int b, ref int g, ref int r, ref int a)
{
var cbX = 0;
var cbY = 0;
var cbW = src.Width;
var cbH = src.Height;
float scale, ox, oy, drawnW, drawnH;
if (src.CropBounds is { } cb && cb.W > 0 && cb.H > 0)
{
cbX = cb.X;
cbY = cb.Y;
cbW = cb.W;
cbH = cb.H;
scale = Math.Max((float)masterW / cbW, (float)masterH / cbH);
drawnW = cbW * scale;
drawnH = cbH * scale;
ox = (float)((masterW - drawnW) / 2);
oy = (float)((masterH - drawnH) / 2);
}
else
{
var (uScale, uOx, uOy) = StretchMath.UniformToFill((float)masterW, (float)masterH, src.Width, src.Height);
scale = uScale;
drawnW = src.Width * uScale;
drawnH = src.Height * uScale;
ox = uOx;
oy = uOy;
}
var sxFloat = cbX + (mx - ox) / drawnW * cbW;
var syFloat = cbY + (my - oy) / drawnH * cbH;
if (sxFloat < 0f) sxFloat = 0f;
if (sxFloat > src.Width - 1f) sxFloat = src.Width - 1f;
if (syFloat < 0f) syFloat = 0f;
if (syFloat > src.Height - 1f) syFloat = src.Height - 1f;
var sxi = (int)sxFloat;
var syi = (int)syFloat;
var sxInv = Math.Min(sxi + 1, src.Width - 1);
var syInv = Math.Min(syi + 1, src.Height - 1);
var fx8 = (int)((sxFloat - sxi) * 256);
var fxInv = 256 - fx8;
var fy8 = (int)((syFloat - syi) * 256);
var fyInv = 256 - fy8;
var stride = src.Width;
var p00 = (syi * stride + sxi) * 4;
var p10 = (syi * stride + sxInv) * 4;
var p01 = (syInv * stride + sxi) * 4;
var p11 = (syInv * stride + sxInv) * 4;
var px = src.BgraPixels;
b = Bilinear(px[p00], px[p10], px[p01], px[p11], fx8, fxInv, fy8, fyInv);
g = Bilinear(px[p00 + 1], px[p10 + 1], px[p01 + 1], px[p11 + 1], fx8, fxInv, fy8, fyInv);
r = Bilinear(px[p00 + 2], px[p10 + 2], px[p01 + 2], px[p11 + 2], fx8, fxInv, fy8, fyInv);
a = Bilinear(px[p00 + 3], px[p10 + 3], px[p01 + 3], px[p11 + 3], fx8, fxInv, fy8, fyInv);
}
private static int Bilinear(int tl, int tr, int bl, int br, int fx8, int fxInv, int fy8, int fyInv)
{
var top = tl * fxInv + tr * fx8;
var bot = bl * fxInv + br * fx8;
return (top * fyInv + bot * fy8 + 32768) >> 16;
}
}
ViewModels/MainViewModel.cs
-1
View File
@@ -577,7 +577,6 @@ public partial class MainViewModel : ViewModelBase
Source { Type: SourceType.WebSource } web => _webView2Manager?.GetLatestFrame(web.Id),
Source { IsMediaSource: true, MediaPath: not null } media => _mediaManager.GetLatestFrame(media.MediaPath),
Source { AssetId: not null } image => StaticPixelCache.Get(image.AssetId),
ZoomSceneElement zoom when zoom.IsZoomActive => BuildZoomFrame(zoom),
_ => null,
};
}
ViewModels/index.md
+1 -1
View File
@@ -12,7 +12,7 @@ areas' docs in their own rows keeps this index small to read.
| `MainViewModel.Account.cs` | Session + connected account: `LoadSavedSessionAsync` (DPAPI restore at startup), `SignInAsync`/`ChangeAccountAsync`/`SignOutYouTubeAsync`, `IsConnected`/`AccountAvatarUrl`/`AccountDisplayName`/`AccountStatusLightToolTip` via `SyncConnectedAccount` (cleared on End). |
| `MainViewModel.License.cs` | Polar license (freemium): `LicenseKey`/`LicenseMessage`/`IsLicenseValidating`/`IsLicenseEntryOpen`/`IsHubVisible`/`IsPremium`, `OfflineGracePeriod`, `ValidateLicenseAtStartupAsync`/`ActivateLicenseAsync`. |
| `MainViewModel.Scenes.cs` | Five-scene catalog: `Scenes` + `StagedScene`/`LiveScene`/`IsEditMode`, `AddScene` (canonical names only), `MissingScenes`/`ShowAddScene`/`AddSceneCommand`; scene wiring `OnScenesChanged`/`WireScene`/`UnwireScene`; transitions `SelectedTransitionType`/`TransitionDurationMs`/`TransitionToScene`/`ApplyTransitionToLive`; global hotkeys `HandleHotkey`/`OpenHotkeyConfig`/`RefreshHotkeyLabels`; scene-gating hints (`CanChangeBackground`). |
| `MainViewModel.Zoom.cs` | **Zoom layer (TASK 51)** `ActivateZoom`/`ToggleZoom`/`DismissZoom`/`StartZoomMove`/`UpdateZoomMove`/`EndZoomMove` + `RefreshZoomUi`/`RebuildDimRectsForZoom` — right-click in the live preview parks a **2x magnifier as a real composited layer** at z-index 1 (`ZoomSceneElement`, healed by `HealZoomLayers`/`EnsureZoomLayer` on load + scene create; never persisted). All shapes are CENTERED on the click and clamped to the master frame. `BuildZoomFrame` resolves the backdrop (live capture under static art — same frames/conditions as `SceneCompositor.Render`) and calls the pure static `ZoomFrameBuilder.Build` (box/2 interior centred on the box centre, bilinear, single-slot cache keyed on frame identity+Epoch+box ints so the per-tick resolver allocates nothing while idle). `BuildCompositorOptions` (moved internal = test seam) always returns the tier crop — the box is never the crop source. |
| `MainViewModel.Zoom.cs` | **Zoom box (TASK 51, creator-corrected 2026-10-08)** `ActivateZoom`/`ToggleZoom`/`DismissZoom`/`StartZoomMove`/`UpdateZoomMove`/`EndZoomMove` + `RefreshZoomUi`/`RebuildDimRectsForZoom`. Right-clicking the live preview anchors a FIXED **capture window** (the box/2 region centred on the click) and parks a movable **2x magnifier**. The box shows a live always-on 2x view of the anchored capture; **dragging moves only the box — `CaptureX/Y` are deliberately untouched in `UpdateZoomMove`** (creator ruling). Each shape is centred on the click and clamped independently (box to the master, capture to box/2 windows): Circle = square, VerticalRectangle = 9:16, HorizontalRectangle = **4:3** (w=h·4/3 with h=0.7·1080; NOT 16:9 — first cut read as a square). The transient `ZoomSceneElement` mirrors box + capture; the output compositor draws it **LAST (layer n+1)** from the already-composed frame (`SceneCompositor.BlitZoom`→`ZoomMagnifier`) so it captures EVERY layer. No per-frame builder/cache — the resolver case that built `ZoomFrameBuilder` frames was removed. `BuildCompositorOptions` (internal = test seam) always returns the tier crop — the box is never the crop source. |
| `MainViewModel.Background.cs` | Screen backdrop (Live-only by policy): `ScreenCaptureManager` + `ScreenCaptureSourceFactory` (WinRT GraphicsCapture); background model `EnsureBackground`/`CreateBackground`/`NormalizeBackgrounds`/`HealBackgrounds`/`StampDefaultBackgroundAsset`; screen capture `ReacquireScreenCaptures`/`ResolveAutoCaptureKey`/`RedesignateBackgroundAsync`/`ClearBackgroundCaptureAsync`/`SetBackgroundCapture`/`OnScreenPreviewBitmapChanged`/`OnFullscreenMonitorChanged`/`RefreshBackgroundAutoCapture`/`BackgroundShowDesktop` (keys on full-screen-game monitor or primary display via `Win32FullScreenDetector`); display picker `Displays`/`SetBackgroundDisplayCommand`; preview fns `ActiveBackgroundImage`/`BackgroundImage`/`UpdateActiveBackground`/`UpdateBackgroundImage`/`RefreshSnapshotsAsync`/`LoadBackgroundImage`/`CreatePlaceholderSnapshot`; `ChangeCaptureCommand`/`RefreshCaptureCommand`/`ToggleShowDesktopCommand`/`BrowseBackgroundCommand`. `ShowPreviewPlaceholder` stays in core. |
| `MainViewModel.Webcam.cs` | Webcam identity: `_webcam`/`ReacquireWebcam`/`SwapWebcamIdentityAsync`; per-scene configs `AddWebcamToStagedSceneAsync`/`ShowWebcamInStagedScene`/`ChangeWebcamAsync` (Windows picker always opens); capture `OnCameraPreviewBitmapChanged`/`OnCameraFailed`; gating `CanAddWebcam`/`CanChangeWebcam`/`CanShowWebcamInStagedScene`/`IsWebcamSelected`/`WebcamError`; size safeguards `WebcamMax*`/`ClampWebcamToBounds`/`HealLegacySquareRect`; `AddWebcamCommand`/`ChangeWebcamCommand`/`ShowWebcamCommand`. |
| `MainViewModel.Audio.cs` | **Mic bar:** read-only realtime meter `AudioLevel`/`MeterFillWidth`/`MeterBrush` (fill = `Math.Min(1, AudioLevelMeter.ToDisplay((float)AudioLevel) * MicVolume)`, green→yellow→red at 60%/80% zones), `MicVolume` (0.8 default; drives `MicMuted` ⇔ volume 0; unmute flash `BeginVolumeFlash`/`EndVolumeFlash`/`CancelVolumeFlash`), `MicSourceName` (left-justified inside the bar, persists via `LayoutStore.Settings`), `ToggleMicMute`/`PickMicrophone` (`MicPickerDialog`; picked device swaps the live source via `_audioMixer.RestartMic()`, loopback keeps running), status dot `MicStatus`/`MicStatusBrush`/`MicStatusToolTip` (`Models.MicStatus`: green Connected / yellow Problem / red no-device), capture starts once at startup (`StartMicCaptureAsync`) — NOT go-live. **Game audio bar:** overlaid bottom-center, always visible (show/hide gating removed 2026-08-15); `GameAudioLevel` (loopback via `OnLoopbackLevelChanged`), `GameAudioVolume`/`GameMuted`/`GameMuteText`/`ToggleGameMute`/`PushSystemVolume`, mirror volume flash `SetGameVolumeAdjusting`/`BeginGameVolumeFlash`/`EndGameVolumeFlash`/`CancelGameVolumeFlash`. Mixer handlers `OnMicLevelChanged`/`OnLoopbackLevelChanged`/`OnMicConnected`/`OnMicFailed`. |