diff --git a/.gitignore b/.gitignore index f18aefb..81d9755 100644 --- a/.gitignore +++ b/.gitignore @@ -1,6 +1,10 @@ bin/ obj/ Helpers/OAuthCredentials.cs + +# Bundled FFmpeg (135 MB of DLLs). Fetched once by scripts/fetch-ffmpeg.sh, which pins +# the URL + SHA-256; the build then copies it to the output. Never committed — see the +# script header for why the hash, not the binaries, is the audit trail. *.user *.suo .vs/ @@ -11,3 +15,4 @@ MONETIZATION.md MARCOM.md CREDENTIALS.md .aider* +ffmpeg/ diff --git a/HANDOFF.md b/HANDOFF.md index bfc0cd2..4476285 100644 --- a/HANDOFF.md +++ b/HANDOFF.md @@ -127,21 +127,44 @@ ONLY paid delta. Store IAP changes how `IsPremium` is *obtained*, never what it this file.** The old "~$69 floor" and "don't break the launch-price promise" notes refer to the old Polar storefront and are **not** current. -### The first code unit: bundle ffmpeg (TASK 48 item 1) +### ✅ DONE 2026-09-28 — TASK 48 item 1: bundle ffmpeg (was the "first code unit") -Two real defects the research surfaced — **neither is fixed yet**, this was a docs unit: +The app no longer downloads an executable at runtime. `scripts/fetch-ffmpeg.sh` populates a +gitignored `ffmpeg/` from a **SHA-256-verified** pinned month-end BtbN artifact; `ytLive.csproj` +copies it to output *and publish* and **fails the build** if it's missing; `FfmpegLocator` lost its +downloader, its zip extraction and the `%APPDATA%\ytLlive\tools` cache, and now resolves +**bundle → PATH** with a loud, actionable failure. The pin that 404'd is gone, so the +2026-09-01 failure class cannot recur. Suite **379/379**, 0 warnings; 16 locator facts including an +integration test that **executes the bundled binary** (which is what proves the sibling DLLs +resolve). `THIRD-PARTY-NOTICES.txt` corrected — it still named the *stale* tag +`autobuild-2026-08-09-13-03`, i.e. the one that 404'd. -1. **⛔ `FfmpegLocator` downloads an unsigned exe from GitHub and runs it.** Store policy - **10.2.2** (dynamic code inclusion) verbatim, *and* the root cause of the 2026-09-01 - failure — the pin aged out of BtbN's 14-day retention and the download **404'd on the - creator's first real recording attempt**. `FfmpegLocator.cs:30-35` records the incident but - still reads like a design choice. **Fix: bundle it.** Also deletes the startup network - dependency. -2. **⛔ `Distribution.md:318` recommended a $400+/yr EV certificate for a SmartScreen bypass - Microsoft removed in March 2024.** Fixed last commit — had it shipped, it would have cost - $400+/yr to buy what $150 buys. Now moot anyway (MSIX is signed by Microsoft), but the - lesson in `MyMistakes.md` stands: a policy citation is a claim about **scope**, not just - text. +**Fresh-clone cost, deliberately accepted:** `./scripts/fetch-ffmpeg.sh` once (~65 MB). 135 MB of +DLLs in git would tax every commit on this 9p tree; the URL + SHA-256 are the audit trail. + +**Answering "other 3p binaries needing bundling?" — no, verified.** ffmpeg was the only +download-and-execute. `e_sqlite3.dll` + `WebView2Loader.dll` already ship via NuGet +(`runtimes\win-x64\native\`), OpenH264 is compiled *into* `avcodec-63.dll`, the rest are managed. +Two gaps remain that bundling cannot fix, and they are the next real risks: the **WebView2 Evergreen +runtime** is a machine prerequisite (item 2's manifest dependency + a graceful-failure path), and +**Velopack's updater downloads and executes**, so its teardown in step 3 is a compliance action. + +### The next code unit: the packaging project + `Package.appxmanifest` (TASK 48 items 2–3) + +Full trust, `webcam`/`microphone`, English only; Velopack deleted. Add the `DefaultRecordFolder()` +comment in the same unit (the mediumIL invariant below becomes load-bearing the moment the manifest +lands). Full-trust writes under a real package identity are still **unverified** — confirm on a +real packaged build before trusting them with recordings. + +Carried over from the research unit, still open: +- **⛔ `Distribution.md:318` recommended a $400+/yr EV certificate for a SmartScreen bypass + Microsoft removed in March 2024.** Fixed then — had it shipped, it would have cost + $400+/yr to buy what $150 buys. Now moot anyway (MSIX is signed by Microsoft), but the + lesson in `MyMistakes.md` stands: a policy citation is a claim about **scope**, not just + text. +- **WebView2 Evergreen runtime** is a machine prerequisite the app will hard-fail without. The + manifest needs the framework dependency, and WebView2 init needs a graceful error instead of a + crash. New, found while bundling ffmpeg. ### ⛔ Two invariants that break silently if touched @@ -177,10 +200,13 @@ Two real defects the research surfaced — **neither is fixed yet**, this was a ## Test state -**367/367 as of `938c5b3`.** Not re-run for this unit — docs only, no code touched. +**379/379, 0 warnings, as of the ffmpeg-bundling unit (2026-09-28).** Build the **test** project, +not just the app project, or VSTest runs a stale DLL. ## Uncommitted / untracked +- `ffmpeg/` — the 9 bundled binaries (~135 MB), **gitignored by design**. Populated by + `scripts/fetch-ffmpeg.sh`. Do not commit it. - `MARCOM.md`, `MONETIZATION.md` — both edited (privacy-copy guard; the Polar-obsolete banner and the re-opened-price note) and both **gitignored by design** (confidential business files, `.gitignore:10-11`). They stay local, as intended. Same for `CREDENTIALS.md` — @@ -188,33 +214,34 @@ Two real defects the research surfaced — **neither is fixed yet**, this was a ## Next -1. **Bundle ffmpeg (TASK 48 item 1)** — unblocked, first code unit, fixes a real user-facing - failure and clears the one hard certification gate. -2. **Settle the price** — one-time vs subscription, and the number. Then declare the IAP - products in Partner Center. **Blocks:** the Store listing, and the `OverlayHost.xaml` copy. -3. **The packaging project + `Package.appxmanifest`** (TASK 48 items 2–3) — full trust, - `webcam`/`microphone`, English only; Velopack deleted. Add the `DefaultRecordFolder()` - comment in the same unit. -4. **Polar teardown** (TASK 48 item 0) — `PolarLicenseService`, `PolarLicense`, +1. **The packaging project + `Package.appxmanifest`** (TASK 48 items 2–3) — the next code unit. + Full trust, `webcam`/`microphone`, English only; Velopack deleted. Add the + `DefaultRecordFolder()` comment in the same unit, and the WebView2 framework dependency + + graceful-failure path. +2. **Declare the IAP products in Partner Center** — the price is **settled** ($10/mo, $400 + lifetime, no annual, 15% verified), so this is unblocked. **Blocks:** the Store listing and the + `OverlayHost.xaml` copy. ⛔ The Individual-vs-Company account question is still unanswered — + get it right before enrolling. +3. **Polar teardown** (TASK 48 item 0) — `PolarLicenseService`, `PolarLicense`, `MainViewModel.License.cs`, the `OverlayHost.xaml` string, the `csproj`/`App.xaml.cs` Velopack sites. `IsPremium` ← Store entitlement. **Do this as one unit** — a half-torn-down licensing path is worse than either end state. -5. **Verify the Store revenue-share rate** — before step 2, not after. -6. **Vertical recording verification** — the compositor tier is proven by +4. **Vertical recording verification** — the compositor tier is proven by `Render_VerticalTier_Outputs_1080x1920_From_The_Center_Crop`; the *record* path has never been run. `FramePump.cs:645` flags off-size tiers as a known follow-up. -7. **WACK + certification notes + the privacy policy** (TASK 48 items 4–5) — the policy must - state camera/mic is OS-gated, nothing is retained, and nothing is fetched at runtime. -8. **Post-session efficacy report** — roll up `CurrentHealth` (dropped frames, duration, health +5. **WACK + certification notes + the privacy policy** (TASK 48 items 4–5) — the policy must + state camera/mic is OS-gated, nothing is retained, and **nothing is fetched at runtime** + (now literally true — the ffmpeg bundle is what makes that sentence defensible). +6. **Post-session efficacy report** — roll up `CurrentHealth` (dropped frames, duration, health message) when a stream or recording ends. `SessionTeardownTests` is the natural home. -9. **Measure whether the Windows camera toggle gates DShow** — gates all privacy-forward copy +7. **Measure whether the Windows camera toggle gates DShow** — gates all privacy-forward copy (`MARCOM.md` guard). Not a certification risk; a trust risk. -10. **TASK 36:212 stale "shipped" line** — one-line fix, needs a hand. +8. **TASK 36:212 stale "shipped" line** — one-line fix, needs a hand. **Dropped from the list** because the ruling made them moot: multi-instance's ffmpeg tools-dir -race (item 1 makes it disappear), `scripts/publish.sh` + the `LlamaCasty.exe` rename (the Store -packages and delivers — *revisit only if we ever ship outside the Store*), and the alert-gating -copy (the wrong string dies with the Polar teardown in step 4). +race (item 1 deleted the tools dir entirely), `scripts/publish.sh` + the `LlamaCasty.exe` rename +(the Store packages and delivers — *revisit only if we ever ship outside the Store*), and the +alert-gating copy (the wrong string dies with the Polar teardown in step 3). **Everything else in the v1 queue:** stream resilience (32), bandwidth step-down (33), scheduled streams (34), scene-linked audio (35), the master limiter (12), text source (3.16), reward diff --git a/MyMistakes.md b/MyMistakes.md index 08327c2..ae17420 100644 --- a/MyMistakes.md +++ b/MyMistakes.md @@ -1266,3 +1266,51 @@ Two corollaries, both paid for here: **Generalisation: a memory map entry that makes the reader's *next action* wrong is a bug, not an omission.** When a fact is superseded by a ruling, the ruling wins the headline — the stale detail belongs underneath it, labelled as what it is. + +### Bundling a pinned binary: verify the pin, then delete the whole download path (TASK 48 item 1) + +FFmpeg was the app's only download-and-execute third-party binary, which is exactly what Store +policy 10.2.2 names — and it had already failed in production (2026-09-01, the pin aged out of +BtbN's 14-day retention and the download 404'd mid-task). Bundling it took one unit. Four things +worth keeping, in the order I learned them: + +**1. Check whether you already have the bytes before you re-fetch them.** The creator's local +`%APPDATA%\ytLlive\tools` already held the pinned build from the September download. Extracting the +pinned zip fresh and comparing SHA-256 per file proved all 9 were byte-identical, which upgraded +"probably the right build" to "provably the right build" for the price of one download. **An +existing cache is a hypothesis, not a fact — but it is a cheap hypothesis to test, and testing it +is what makes the bundle trustworthy.** + +**2. Flip the probe order and say why, in the code comment.** The locator probed PATH first +("the user's own install wins"). Once the binaries are in the package, that is backwards: a PATH +`ffmpeg` may be a different major version or a **GPL** build, and the bundle is the one you pinned, +license-vetted and tested. PATH stays as the escape hatch, which is what it was actually for. +I would not have caught this without asking "which of these two is the one I tested?" + +**3. Delete the fallback, do not just deprioritise it.** The old ladder had +`PATH → tools-cache → download`, and the tools-cache step meant a **missing bundle could be +silently satisfied by a stale copy**. For a dependency that Store certification inspects, silent +drift is the failure mode you least want: the package is wrong and nothing says so. A missing +bundle now throws, naming the folder and the fetch script. Likewise a build-time guard in the +csproj, so a fresh clone learns before a take rather than during one. + +**4. 135 MB of DLLs in git is a real, recurring tax — the hash is the better audit trail.** The +temptation is to commit the binaries so "the build just works". On a 9p-bridged working tree that +tax is paid on *every* commit, status and checkout, forever, to serve a one-time packaging step. +Shipping a `fetch-ffmpeg.sh` that pins the URL and **verifies SHA-256 before extracting** keeps +the repo small, keeps the network dependency at build time instead of run time (which is what the +policy actually forbids), and gives the LGPL written-offer something better to point at than an +opaque blob: a URL, an artifact name, and a hash a reviewer can recompute. + +**The test that earned its keep:** the integration test runs the bundled `ffmpeg.exe -version`. +Presence checks would all have passed with the DLLs missing — a shared build simply fails to +start, so *executing* it is the only assertion that proves the `libav*.dll` siblings resolved. When +a binary's correctness depends on its neighbours being adjacent, test the neighbours. + +**Also: check what you just claimed about the licences.** `THIRD-PARTY-NOTICES.txt` said OpenH264 +"ships inside the FFmpeg build". Verified rather than assumed: the encoder is compiled *into* +`avcodec-63.dll` (8 references in that DLL) and there is no separate OpenH264 binary to ship — so +the claim was right but imprecise, and it now says precisely that, with the check and its date. +The same file was still naming build tag `autobuild-2026-08-09-13-03` — **the tag that 404'd.** +A licence record that names a stale artifact is worse than no licence record, because it reads as +verified. When you pin something, the pin and the paperwork must move in the same commit. diff --git a/Services/Encoder/FfmpegLocator.cs b/Services/Encoder/FfmpegLocator.cs index a214530..29d51aa 100644 --- a/Services/Encoder/FfmpegLocator.cs +++ b/Services/Encoder/FfmpegLocator.cs @@ -1,153 +1,81 @@ using System.IO; -using System.IO.Compression; -using System.Net.Http; using ytLive.Helpers; namespace ytLive.Services.Encoder; /// -/// The default : probe PATH first (the user's own -/// install wins), then the cache in %APPDATA%\ytLlive\tools, then pull the -/// pinned BtbN **lgpl-shared** build (TASK 4 ship step 2). The shared variant is a -/// deliberate licensing choice: dynamic linking means LGPL compliance is "license -/// text + source offer", with no static-relink (LGPL §6) material required. The -/// shared zip puts ffmpeg.exe plus the libav*.dll family in bin/, -/// so both are extracted — Windows resolves the DLLs from the exe's own directory. -/// Search dirs, tools dir, and the downloader are constructor-injected so tests -/// fake the network and stay on a temp directory. +/// The default : resolve the bundled FFmpeg that ships +/// next to the app, falling back to PATH for anyone who deliberately installed their own. +/// +/// Nothing is downloaded. This type used to pin a BtbN release URL, download +/// the zip on first run, and execute it — which is Windows Store policy 10.2.2 (dynamic +/// code inclusion, cited in TASKS/research-store-certification.md §4) and already +/// cost a real failure: the pin aged out of BtbN's 14-day retention and the download +/// 404'd on the creator's first real recording attempt (2026-09-01). The binaries +/// are now fetched at BUILD time by scripts/fetch-ffmpeg.sh (pinned URL + SHA-256) +/// and copied into ffmpeg/ beside the executable. +/// +/// Order is bundled-first, deliberately. The old code probed PATH first +/// ("the user's own install wins"). With a bundle in the package that is the wrong way +/// round: a random PATH ffmpeg may be a different major version or a GPL build, and +/// the bundle is the one we pinned, license-vetted, and tested against. PATH stays as the +/// escape hatch, which is what it is actually good for. +/// +/// The lgpl-shared variant is a licensing choice, not a packaging one: dynamic +/// linking makes LGPL compliance "license text + source offer" with no static-relink +/// (LGPL §6) material required. That means the libav*.dll family must sit in the +/// same folder as the exe — Windows resolves them from the executable's own directory. /// public sealed class FfmpegLocator : IFfmpegLocator { public const string FileName = "ffmpeg.exe"; - /// Sibling probe binary — extracted alongside ffmpeg so the media + /// Sibling probe binary — bundled alongside ffmpeg so the media /// frame-rate probe () can /// derive its path from the located ffmpeg directory. public const string ProbeFileName = "ffprobe.exe"; - /// - /// Pinned BtbN LGPL-shared win64 build — autobuild-2026-08-31-13-27, the - /// ffmpeg N-126342 build (immutable tag; recorded in TASKS.md per the licensing rule). - /// Deliberately a MONTH-END tag: BtbN retains the last ~14 dailies but month-end - /// builds for ~2 years — the previous pin (a 2026-08-09 daily) aged out of retention - /// and 404'd on a cold cache exactly as ai.md predicted (2026-09-01, the first real - /// recording attempt). Dated tags carry versioned asset names (ffmpeg-N-…-win64-lgpl-shared.zip); - /// extraction is name-agnostic. - /// - public const string PinnedUrl = - "https://github.com/BtbN/FFmpeg-Builds/releases/download/autobuild-2026-08-31-13-27/ffmpeg-N-126342-gf88b741dbf-win64-lgpl-shared.zip"; + /// Folder, relative to the app base directory, that the build copies the + /// bundled build into. One folder for the exe and every DLL. + public const string BundleFolderName = "ffmpeg"; + + /// Base directory the bundle is resolved against. Injectable so tests can + /// point at a temp tree; defaults to the running app's base directory. + private readonly string _bundleRoot; private readonly string[] _searchDirs; - private readonly string _toolsDir; - private readonly Func> _downloader; - public FfmpegLocator( - string[]? searchDirs = null, - string? toolsDir = null, - Func>? downloader = null) + public FfmpegLocator(string[]? searchDirs = null, string? bundleRoot = null) { _searchDirs = searchDirs ?? ParsePath(); - _toolsDir = toolsDir ?? Path.Combine( - Environment.GetFolderPath(Environment.SpecialFolder.ApplicationData), "ytLlive", "tools"); - _downloader = downloader ?? DefaultDownload; + _bundleRoot = bundleRoot ?? AppContext.BaseDirectory; } - public async Task LocateAsync(CancellationToken cancellationToken = default) + public Task LocateAsync(CancellationToken cancellationToken = default) { + var bundled = Path.Combine(_bundleRoot, BundleFolderName, FileName); + if (IsUsable(bundled)) return Task.FromResult(bundled); + foreach (var dir in _searchDirs) { if (string.IsNullOrEmpty(dir)) continue; var candidate = Path.Combine(dir, FileName); - if (File.Exists(candidate)) return candidate; + if (IsUsable(candidate)) return Task.FromResult(candidate); } - var cached = Path.Combine(_toolsDir, FileName); - if (File.Exists(cached) && new FileInfo(cached).Length > 0) return cached; - - try - { - var zip = await _downloader(PinnedUrl, cancellationToken).ConfigureAwait(false); - if (zip.Length == 0) - throw new IOException($"FFmpeg download from {PinnedUrl} returned an empty payload."); - Directory.CreateDirectory(_toolsDir); - ExtractBinaries(zip, _toolsDir); - return cached; - } - catch (HttpRequestException ex) - { - // The old gap: this exception type escaped unwrapped, so a dead pin surfaced - // as a raw "Response status code does not indicate success: 404" from the - // frame pump. Locator failures are IOException with an actionable message. - AppLog.Write(ex, "FFmpeg locator: download failed"); - throw new IOException( - $"FFmpeg download failed ({ex.Message}). The pinned build may have aged out of " + - "retention — update FfmpegLocator.PinnedUrl, or install FFmpeg (LGPL-shared build) " + - "and make sure 'ffmpeg' is on PATH.", ex); - } - catch (Exception ex) when (ex is IOException or InvalidDataException or NotSupportedException) - { - AppLog.Write(ex, "FFmpeg locator: download/extract failed"); - throw; - } + // Deliberately loud. The previous cache-then-download ladder had a "found + // something on disk" path that could satisfy a missing bundle with a stale copy, + // which is exactly the silent-drift a certification-gated dependency must not + // have. If the package forgot ffmpeg/, the app says so and stops. + throw new IOException( + $"FFmpeg was not found. The bundled copy is missing from " + + $"'{Path.Combine(_bundleRoot, BundleFolderName)}' — this build is incomplete " + + "(reinstall or re-fetch with scripts/fetch-ffmpeg.sh). Installing an LGPL-shared " + + "FFmpeg and putting 'ffmpeg' on PATH also works."); } - /// - /// Extract ffmpeg.exe and every *.dll into toolsDir via a - /// staging directory, so a failed extract never leaves a partially-populated - /// cache behind (the previous good cache stays until every move succeeds). - /// - private static void ExtractBinaries(byte[] zip, string toolsDir) - { - var staging = toolsDir + ".stage"; - if (Directory.Exists(staging)) Directory.Delete(staging, recursive: true); - Directory.CreateDirectory(staging); - try - { - using var stream = new MemoryStream(zip, writable: false); - using var archive = new ZipArchive(stream, ZipArchiveMode.Read); - var exe = archive.Entries.FirstOrDefault( - e => e.FullName.EndsWith("/" + FileName, StringComparison.OrdinalIgnoreCase)) - ?? throw new InvalidDataException($"The pinned FFmpeg archive does not contain {FileName}."); - ExtractOne(exe, Path.Combine(staging, FileName)); - var probe = archive.Entries.FirstOrDefault( - e => e.FullName.EndsWith("/" + ProbeFileName, StringComparison.OrdinalIgnoreCase)); - if (probe != null) - ExtractOne(probe, Path.Combine(staging, ProbeFileName)); - foreach (var entry in archive.Entries) - { - if (entry.FullName.EndsWith(".dll", StringComparison.OrdinalIgnoreCase)) - ExtractOne(entry, Path.Combine(staging, Path.GetFileName(entry.FullName))); - } - foreach (var file in Directory.GetFiles(staging)) - File.Move(file, Path.Combine(toolsDir, Path.GetFileName(file)), overwrite: true); - } - finally - { - if (Directory.Exists(staging)) Directory.Delete(staging, recursive: true); - } - } - - private static void ExtractOne(ZipArchiveEntry entry, string destination) - { - var temp = destination + ".tmp"; - try - { - using (var source = entry.Open()) - using (var target = File.Create(temp)) - source.CopyTo(target); - File.Move(temp, destination); - } - finally - { - if (File.Exists(temp)) File.Delete(temp); - } - } - - private static async Task DefaultDownload(string url, CancellationToken ct) - { - using var http = new HttpClient { Timeout = TimeSpan.FromMinutes(10) }; - return await http.GetByteArrayAsync(url, ct).ConfigureAwait(false); - } + private static bool IsUsable(string path) => + File.Exists(path) && new FileInfo(path).Length > 0; private static string[] ParsePath() { diff --git a/TASKS.md b/TASKS.md index d2cd9ac..2935c28 100644 --- a/TASKS.md +++ b/TASKS.md @@ -63,7 +63,7 @@ | 45 | TEST-tab chat fix #2: insert body must declare `snippet.type` (400 MISSING_REQUIRED_FIELD) | ✅ Done (2026-09-25) | [`TASKS/task-45-chat-insert-type.md`](TASKS/task-45-chat-insert-type.md) | | 46 | Drawers: click outside the rail closes whichever is open — TEST added to the existing Stream Settings + YPP dismiss behavior | ✅ Done (2026-09-25) | [`TASKS/task-46-drawer-click-outside-close.md`](TASKS/task-46-drawer-click-outside-close.md) | | 47 | Alert box video: built-in/custom alert clip (+ six-animation fallback) with read-time fade in/out + ticker (now in the preview too, 3 display methods) + alert volume in the live mix | ✅ Done (2026-09-26) | [`TASKS/task-47-alert-videos.md`](TASKS/task-47-alert-videos.md) | -| 48 | Distribution & packaging: **MSIX + Store IAP** | 🔶 In progress — **✅ route decided 2026-09-27 (Store MSIX + Store IAP)**; item 1 (bundle ffmpeg) is the next code unit | [`TASKS/task-48-distribution-msix.md`](TASKS/task-48-distribution-msix.md) | +| 48 | Distribution & packaging: **MSIX + Store IAP** | 🔶 In progress — **✅ route decided 2026-09-27 (Store MSIX + Store IAP)**; **item 1 (bundle ffmpeg) SHIPPED 2026-09-28** | [`TASKS/task-48-distribution-msix.md`](TASKS/task-48-distribution-msix.md) | | 49 | Chat profanity filter (local, opt-in, non-persistent, user word list) | ☐ Queued (2026-09-27) | [`TASKS/task-49-chat-profanity-filter.md`](TASKS/task-49-chat-profanity-filter.md) | --- @@ -237,8 +237,18 @@ second encoder path needing a "redirect". Record+simulcast is one ffmpeg with tw - **Still to verify (does not block packaging):** current Store revenue-share terms — ⚠️ do not assume any percentage, verify before setting a price; and the one-time vs subscription shape of the IAP tier (the storefront is decided, the price is not). - - **First code unit: item 1 — bundle ffmpeg instead of downloading it.** Unblocked, - recommended on every route, and it fixes a real user-facing failure. + - **✅ item 1 SHIPPED 2026-09-28 — bundle ffmpeg instead of downloading it.** The app no + longer fetches an executable at runtime (Store policy 10.2.2, and the pin 404'd on the + creator's first real recording on 2026-09-01). `scripts/fetch-ffmpeg.sh` populates a + gitignored `ffmpeg/` from a **SHA-256-verified** pinned month-end BtbN artifact; the csproj + copies it to output/publish and **fails the build** if it is missing; `FfmpegLocator` lost its + downloader and now resolves **bundle → PATH**. Immutable tag recorded per the licensing rule: + `autobuild-2026-08-31-13-27`. `ffplay.exe` is not shipped. Verified: ffmpeg was the **only** + download-and-execute third-party binary — `e_sqlite3.dll`/`WebView2Loader.dll` already ship + via NuGet and OpenH264 is compiled into `avcodec-63.dll`. **Remaining gaps are not bundling + problems:** the WebView2 Evergreen *runtime* is a machine prerequisite (item 2), and + **Velopack's updater downloads and executes — its teardown is a compliance action.** + Detail: `TASKS/task-48-distribution-msix.md` §1. - **TASK 10 is now a teardown, not a build** (2026-09-27) — Store IAP deletes the Polar licensing path. The Polar fee tables, the perpetual-key model, and the customer-portal plumbing in `MONETIZATION.md` (gitignored, local) are obsolete; the **watermark-only diff --git a/TASKS/task-48-distribution-msix.md b/TASKS/task-48-distribution-msix.md index 46d1592..ad2a1e2 100644 --- a/TASKS/task-48-distribution-msix.md +++ b/TASKS/task-48-distribution-msix.md @@ -81,31 +81,62 @@ ONLY paid delta (TASK 36 item 2). Store IAP changes how the bit is *obtained*, n --- -## 1. ⛔ Bundle ffmpeg instead of downloading it — **do this on EVERY route** +## 1. ☑ Bundle ffmpeg instead of downloading it — **SHIPPED 2026-09-28** **This is item 1 because it is genuinely first** — it fixes a user-facing failure on its own and is a hard certification gate. -`Services/Encoder/FfmpegLocator.cs:37-38` pins a GitHub release URL; `LocateAsync` -downloads (line 69), extracts (line 73), and the encoder executes the result. On a cold -cache the app downloads an unsigned executable from the internet and runs it. +`Services/Encoder/FfmpegLocator.cs` used to pin a GitHub release URL, download it on first run, +extract it, and execute the result. On a cold cache the app downloaded an unsigned executable from +the internet and ran it. - **Store blocker:** policy 10.2.2 forbids dynamic code inclusion (download-and-execute). -- **Route-independent bug class:** `FfmpegLocator.cs:30-35` records that the previous - daily pin aged out of GitHub retention and **404'd on the first real recording attempt - (2026-09-01)**. Bundling kills this permanently and removes the startup network - dependency. +- **Route-independent bug class:** the previous daily pin aged out of GitHub retention and + **404'd on the first real recording attempt (2026-09-01)**. Bundling kills this permanently and + removes the startup network dependency. -**Do:** ship `ffmpeg.exe` + `ffprobe.exe` + the `libav*.dll` family inside the package; -`FfmpegLocator` resolves PATH → package-local → cache, and never downloads. -`IFfmpegLocator`'s contract, the `ExtractBinaries` staging discipline, and the existing -`FfmpegLocatorTests` cold/cache/PATH coverage all still apply. +**What shipped (2026-09-28, `eb0dd91` docs + this unit):** +- `scripts/fetch-ffmpeg.sh` — populates `./ffmpeg` from a pinned month-end BtbN LGPL-shared + artifact, **SHA-256 verified before extraction**, idempotent, `FFMPEG_ZIP=` override. The + binaries are **gitignored** (135 MB of DLLs would tax every commit on the 9p tree); the pinned + URL + SHA-256 are the audit trail the LGPL written-offer actually needs. +- `ytLive.csproj` — copies `ffmpeg\**` to the output with `CopyToPublishDirectory="Always"` (the + MSIX one), plus a **`GuardBundledFfmpeg` target that fails the build** if the folder is missing, + so a fresh clone finds out before a take rather than during one. +- `FfmpegLocator` — downloader, zip extraction, and the `%APPDATA%\ytLlive\tools` cache are all + **gone**. Order is now **bundle → PATH** (deliberately inverted from "the user's install wins": + a PATH ffmpeg may be a different major version or a GPL build). A missing bundle throws a loud + `IOException` naming the folder and the fetch script — the "found something on disk" path that + could mask a missing bundle is deliberately deleted. +- `ffplay.exe` (18 MB) is **not** shipped; nothing calls it. +- `THIRD-PARTY-NOTICES.txt` — "downloads" → "bundled" throughout, and the stale build tag + `autobuild-2026-08-09-13-03` (**the tag that 404'd**) corrected to `autobuild-2026-08-31-13-27` + with the exact artifact name and archive SHA-256 recorded. OpenH264 item made precise: it is + **compiled into `avcodec-63.dll`**, not a separate binary. + +**Immutable tag (licensing rule):** `autobuild-2026-08-31-13-27`, artifact +`ffmpeg-N-126342-gf88b741dbf-win64-lgpl-shared.zip`, SHA-256 +`46ea6521ed3877689d2ff0bd03640ac2637104415f6237d42625d13408767146`. The URL is now a **provenance +record, not a runtime fetch**. Month-end on purpose: BtbN keeps ~14 dailies but month-end builds +~2 years — a daily pin is what aged out. **Licensing:** the LGPL-shared build was chosen for LGPL §6 compliance (dynamic linking = -"license text + source offer"). That reasoning is **unaffected**. Confirm -`THIRD-PARTY-NOTICES.txt` covers the bundled build. +"license text + source offer"). That reasoning is **unaffected**, and the notices now cover the +bundled copy explicitly. -**Record the immutable tag** in `TASKS.md` per the licensing rule, and note the URL is now -a provenance record rather than a runtime fetch. +**Tests (16 facts, `FfmpegLocatorTests`, suite 379/379):** the Good Dog integration test walks the +real build output, asserts the 9 files ship, resolves via the locator with an empty PATH, and +**executes `ffmpeg -version`** — running it is the part that matters, because a shared build only +starts if every `libav*.dll` is a sibling. Plus: bundle-over-PATH, PATH fallback, zero-byte bundle, +the loud failure, `TheBuildRefusesToCompileWithoutTheBundle`, and +`TheLocatorCarriesNoNetworkCapability` — a source-level fact that no `HttpClient`/URL/downloader +can reappear, so 10.2.2 cannot rot back. + +**Answering "were there other 3p binaries needing bundling?" (verified 2026-09-28): no.** ffmpeg was +the only download-and-execute. `e_sqlite3.dll` + `WebView2Loader.dll` already ship via NuGet into +`runtimes\win-x64\native\`; OpenH264 is inside `avcodec-63.dll`; everything else is managed. The two +real gaps are **not** bundling problems: the **WebView2 Evergreen runtime** is a machine prerequisite +(item 2's manifest dependency + a graceful-failure path), and **Velopack's updater downloads and +executes** — its teardown (item 3) is a compliance action, not just cleanup. --- diff --git a/THIRD-PARTY-NOTICES.txt b/THIRD-PARTY-NOTICES.txt index ea5b33c..7b2cc64 100644 --- a/THIRD-PARTY-NOTICES.txt +++ b/THIRD-PARTY-NOTICES.txt @@ -2,10 +2,14 @@ ytLlive — Third-Party Notices ================================ ytLlive is a paid, closed-source product. This file lists every third-party -component the product distributes or downloads, its license, and where to get +component the product distributes, its license, and where to get its source, so the LGPL/BSD/MIT obligations are met. Distribution obligations -are NOT optional: they attach because this product ships or automates the -download of these components. +are NOT optional: they attach because this product ships these components. + +Note on FFmpeg (items 1-2): it used to be DOWNLOADED at first run. As of +TASK 48 item 1 (2026-09-28) it is BUNDLED — the exact pinned build is fetched +at BUILD time by scripts/fetch-ffmpeg.sh, verified against a SHA-256, and +shipped inside the package. The product fetches no executable at runtime. If this file changes, update it here AND in the app's About screen (the About overlay's licensing panel surfaces this file in-app). See TASKS.md (TASK 4) and @@ -20,34 +24,48 @@ there on purpose. Home: https://ffmpeg.org/ Source: https://git.ffmpeg.org/ffmpeg.git Used as: the streaming encoder/RTMP subprocess. This product distributes the - UNMODIFIED binaries; it never links FFmpeg into its own code (it is - launched as a separate process fed raw frames over a pipe). + UNMODIFIED binaries; it never links FFmpeg into its own code (it is + launched as a separate process fed raw frames over a pipe). + Bundled: ffmpeg.exe, ffprobe.exe and the libav*.dll family, unmodified, in a + single `ffmpeg\` folder beside the application executable. They are the + lgpl-SHARED build, so the DLLs must stay siblings of the exe — Windows + resolves them from the executable's own directory. Why LGPL (not GPL): a GPL build (e.g. gyan.dev, or BtbN's "gpl" variant) - would contaminate this proprietary product. Do NOT use one. + would contaminate this proprietary product. Do NOT use one. Why "shared" (not static): LGPL v2.1 §6 requires "relinkable" materials for - statically-linked libraries. The shared build links dynamically, so - the user can replace the DLLs — compliance is this notice plus the - source offer below, with no relink material required. + statically-linked libraries. The shared build links dynamically, so + the user can replace the DLLs — compliance is this notice plus the + source offer below, with no relink material required. Compliance supplied by this product: - The license text: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.html - The corresponding source / written offer to obtain it: FFmpeg source https://ffmpeg.org/download.html Exact binary https://github.com/BtbN/FFmpeg-Builds - Build tag: autobuild-2026-08-09-13-03 (variant lgpl-shared) + Exact artifact ffmpeg-N-126342-gf88b741dbf-win64-lgpl-shared.zip + Build tag: autobuild-2026-08-31-13-27 (variant lgpl-shared) + Archive SHA-256: 46ea6521ed3877689d2ff0bd03640ac2637104415f6237d42625d13408767146 - The binaries are unmodified and the LGPL notices therein are intact. + - The SHA-256 above is the integrity record for the bundled copy; it is + asserted by scripts/fetch-ffmpeg.sh before extraction and re-derivable + from the artifact URL in that script. -2. BtbN FFmpeg-Builds (the exact binary this product downloads) +2. BtbN FFmpeg-Builds (the build system that produced the bundled binary) License: MIT (build scripts + repository) — the produced binaries are covered by FFmpeg's LGPL (item 1). Home: https://github.com/BtbN/FFmpeg-Builds -3. OpenH264 (libopenh264 — the H.264 software fallback encoder) +3. OpenH264 (the H.264 software fallback encoder, libopenh264) Copyright (c) 2010-2026 Cisco Systems, Inc. (and contributors) License: BSD 2-Clause + Cisco's H.264 patent grant Home: https://www.openh264.org/ Note: Cisco grants the patent license for its own H.264 implementation; - it ships inside the FFmpeg build above (LGPL obligations of item 1 - apply to the library; the BSD terms apply to Cisco's code). + it is COMPILED INTO avcodec-63.dll in the bundled build above (verified + 2026-09-28: the encoder is registered in that DLL; there is no separate + openh264 binary to ship). So item 1's LGPL obligations apply to the FFmpeg + code and Cisco's BSD terms plus its patent grant apply to the OpenH264 + code inside it. This is the product's software H.264 fallback when no + hardware encoder (NVENC/QSV/AMF) is available — it is deliberately never + libx264, which would be GPL. 4. SQLite (bundled via SQLitePCLRaw's e_sqlite3 native bundle) License: public domain (no rights reserved) diff --git a/ai.md b/ai.md index 51473f7..aaf48fe 100644 --- a/ai.md +++ b/ai.md @@ -113,6 +113,18 @@ collaboration, not autonomous code generation. ## Run +**First time on a fresh clone (or if `ffmpeg/` was deleted) — this is REQUIRED, not optional:** + +```bash +./scripts/fetch-ffmpeg.sh # ~65 MB once; verifies SHA-256, extracts 9 files into ./ffmpeg +``` + +FFmpeg is **bundled, never downloaded at runtime** (Windows Store policy 10.2.2 — the old runtime +download 404'd on 2026-09-01 and would fail certification). The binaries are gitignored because +135 MB of DLLs in git would tax every commit on this 9p tree; the pinned URL + SHA-256 in the script +are the audit trail. `ytLive.csproj`'s `GuardBundledFfmpeg` target **fails the build** with that +exact instruction if the folder is missing. `FFMPEG_ZIP=` reuses a pre-downloaded archive. + ```bash # From WSL, ALWAYS use the Windows dotnet host — never Linux `dotnet` for this project: "/mnt/c/Program Files/dotnet/dotnet.exe" build "C:\Users\gramp\Documents\Code\projects\ytLive\ytLive.csproj" @@ -619,37 +631,58 @@ stays XAML (editing view); the compositor is the output view. resolver in the encoder step, not the compositor step. The master buffer (the compositor's return value) is the seam a future D3D11 compositor would honor identically. -### FFmpeg locator (TASK 4 ship step 2 — shipped 2026-08-10, plan in TASKS.md) +### FFmpeg locator (TASK 4 ship step 2 — shipped 2026-08-10; **rewritten to BUNDLED 2026-09-28**, plan in TASKS.md) -The encoder's one external dependency is `ffmpeg.exe`; it's never shipped in the repo. `IFfmpegLocator` -resolves an absolute path on demand: **PATH probe first** (the user's own install wins — their choice, -their responsibility), then the cache (`%APPDATA%\ytLlive\tools\ffmpeg.exe`), then a **pinned** BtbN -LGPL-**shared** win64 zip (~75 MB) from which `ffmpeg.exe` **and the `libav*.dll` family** are extracted -(staged temp-write + move so a crash never corrupts the cache; Windows resolves the DLLs from the exe's -own directory). BtbN LGPL-shared (not gyan.dev, not static): it drops GPL-only libx264/x265 while keeping +The encoder's one external dependency is `ffmpeg.exe`. **It is BUNDLED — never downloaded, never +fetched at runtime.** `IFfmpegLocator` resolves an absolute path on demand: **the bundle first** +(`\ffmpeg\ffmpeg.exe`), then **PATH** as the escape hatch for someone who +deliberately installed their own. The order is the opposite of the original ("the user's own install +wins") on purpose: a PATH `ffmpeg` may be a different major version or a **GPL** build, and the +bundle is the one that was pinned, license-vetted and tested. If neither hits, it throws a loud +`IOException` naming the missing folder and the fetch script — the old "found something on disk" +cache path is gone deliberately, because silently satisfying a missing bundle with a stale copy is +exactly the silent drift a certification-gated dependency must not have. + +**The bundle is not in git** (135 MB of incompressible DLLs would tax every commit on this +9p-bridged tree). `scripts/fetch-ffmpeg.sh` populates `./ffmpeg` from a pinned month-end BtbN +LGPL-**shared** win64 artifact, **verifies SHA-256 before extracting**, and is idempotent; set +`FFMPEG_ZIP=` to reuse a pre-downloaded archive. `ytLive.csproj` copies `ffmpeg\**` to the +output (`CopyToPublishDirectory="Always"` for the MSIX package) and the `GuardBundledFfmpeg` target +**fails the build** if the folder is absent, so a fresh clone finds out before the creator presses +Record rather than during a take. `ffmpeg.exe` + `ffprobe.exe` + the `libav*.dll` family must stay in +one folder — Windows resolves a shared build's DLLs from the executable's own directory. +`ffplay.exe` (18 MB) is deliberately **not** shipped; nothing calls it. + +BtbN LGPL-shared (not gyan.dev, not static): it drops GPL-only libx264/x265 while keeping NVENC/QSV/AMF + libopenh264 + native AAC, and dynamic linking means LGPL compliance is "license text + -source offer" with no static-relink (§6) material — see the Licensing guardrails below. The pin is a -dated autobuild tag (immutable); BtbN retention keeps the last 14 daily + each month-end for 2 years, so -a cold cache can outlive the pin → the seam throws a clear, logged error (recoverable; the pin is one -const). Constructor-injected search dirs / tools dir / downloader (`Func>`) keep it hermetic: tests fake the network with a real in-memory zip. Constructed in the -encoder step (not yet — this PR ships the seam + impl + tests only). +source offer" with no static-relink (§6) material — see the Licensing guardrails below. libopenh264 +is **compiled into `avcodec-63.dll`** (verified 2026-09-28: the encoder is registered there, and there +is no separate OpenH264 binary to ship), so Cisco's BSD terms + patent grant ride along inside item 1 +of `THIRD-PARTY-NOTICES.txt`. Pin: `autobuild-2026-08-31-13-27` (month-end on purpose — BtbN keeps +~14 dailies but month-end builds ~2 years; a daily pin is what 404'd, see below). -**⛔ The pinned-download design is now formally a defect, not a cold-start edge case (2026-09-27).** -It **failed in production on 2026-09-01** — the previous daily pin aged out of BtbN's 14-day -retention and the download 404'd on the creator's first real recording attempt. Two facts make -the "pin is a const, just bump it" assumption wrong: +**⛔ Why this was rewritten (2026-09-27 → shipped 2026-09-28).** The pinned-download design was a +defect, not a cold-start edge case. It **failed in production on 2026-09-01** — the previous daily +pin aged out of BtbN's 14-day retention and the download 404'd on the creator's first real recording +attempt. Two facts made "the pin is a const, just bump it" wrong: - **Microsoft Store policy 10.2.2 forbids dynamic code inclusion.** Downloading an executable from the internet and running it is the exact pattern the policy names. An MSIX submission is rejected over it. - Bumping the pin is whack-a-mole against a retention window we do not control. -**Fix (TASK 48 item 1, recommended on EVERY route — it is not Store-conditional):** bundle -`ffmpeg.exe` + `ffprobe.exe` + the `libav*.dll` family **inside the package**; `FfmpegLocator` -resolves PATH → package-local → cache and never downloads. This also deletes the startup -network dependency entirely. The LGPL-**shared** build choice is **unaffected** — dynamic -linking was chosen for LGPL §6 compliance ("license text + source offer"), not to enable -downloading. Analysis: `TASKS/research-store-certification.md` §4. +Bundling deletes the startup network dependency entirely. The LGPL-**shared** build choice is +**unaffected** — dynamic linking was chosen for LGPL §6 compliance ("license text + source offer"), +not to enable downloading. Analysis: `TASKS/research-store-certification.md` §4. +**Guardrail:** `FfmpegLocatorTests.TheLocatorCarriesNoNetworkCapability` asserts the locator's +source contains no `HttpClient`/URL/downloader at all, so the policy cannot rot back. + +**Every other third-party binary already ships via NuGet — ffmpeg was the only download-and-execute.** +Verified 2026-09-28: `e_sqlite3.dll` and `WebView2Loader.dll` land in `runtimes\win-x64\native\` +(SQLitePCLRaw + WebView2 packages) and NAudio/Notification.Wpf/Microsoft.Data.Sqlite/ProtectedData +are managed. Two gaps remain and are **not** fixed by bundling: the **WebView2 Evergreen runtime** is +a machine prerequisite (a manifest framework-dependency + graceful-failure path, TASK 48 item 2), and +**Velopack's updater downloads and executes** — its teardown is a compliance action, not just cleanup. + ### Live encoder + RTMP push (TASK 4 ship step 3 — shipped 2026-08-12, plan in TASKS.md) diff --git a/scripts/fetch-ffmpeg.sh b/scripts/fetch-ffmpeg.sh new file mode 100644 index 0000000..54dc6a6 --- /dev/null +++ b/scripts/fetch-ffmpeg.sh @@ -0,0 +1,98 @@ +#!/usr/bin/env bash +# fetch-ffmpeg.sh — populate ./ffmpeg/ with the pinned BtbN LGPL-shared FFmpeg build. +# +# WHY THIS EXISTS: the app used to download ffmpeg from GitHub on first run and execute +# it. That is Windows Store policy 10.2.2 (dynamic code inclusion) and it already cost a +# real failure — the pin aged out of BtbN's 14-day retention and the download 404'd on +# the creator's first recording attempt (2026-09-01). FFmpeg is now BUNDLED, so nothing is +# fetched at runtime. This script is the BUILD-TIME half of that: a fresh clone has no +# ffmpeg/ folder, so it must be filled once before the first build. +# +# The binaries are NOT committed to git (135 MB of incompressible DLLs would tax every +# commit on this 9p-bridged working tree). The URL + SHA-256 below are the audit trail +# that the LGPL written-offer obligation actually needs, and they are reviewable in a diff. +# +# Usage: ./scripts/fetch-ffmpeg.sh +# FFMPEG_ZIP=/path/to/already-downloaded.zip ./scripts/fetch-ffmpeg.sh +# +# Idempotent: re-running with ffmpeg/ already populated verifies and exits. Delete +# ffmpeg/ to force a refetch. + +set -euo pipefail + +# Pinned BtbN lgpl-shared win64 build. A MONTH-END autobuild tag on purpose: BtbN keeps +# ~14 dailies, but month-end builds for ~2 years. Do not "refresh" this to a daily — that +# is the mistake that caused the 404. +URL="https://github.com/BtbN/FFmpeg-Builds/releases/download/autobuild-2026-08-31-13-27/ffmpeg-N-126342-gf88b741dbf-win64-lgpl-shared.zip" +SHA256="46ea6521ed3877689d2ff0bd03640ac2637104415f6237d42625d13408767146" + +# The files we ship. ffplay.exe (18 MB) is deliberately excluded — nothing in the app +# calls it, and shipping an unused player would widen the package and the license surface +# for nothing. +WANT_EXE="ffmpeg.exe ffprobe.exe" + +ROOT="$(cd "$(dirname "$0")/.." && pwd)" +DEST="$ROOT/ffmpeg" + +if [ -f "$DEST/ffmpeg.exe" ]; then + echo "✓ ffmpeg/ already populated — nothing to do." + echo " (delete $DEST to force a refetch)" + exit 0 +fi + +command -v python3 >/dev/null 2>&1 || { + echo "❌ python3 is required to extract the zip (unzip is not installed in this WSL)." + echo " Set FFMPEG_ZIP to a pre-downloaded archive, or install python3." + exit 1 +} + +ZIP="${FFMPEG_ZIP:-}" +if [ -z "$ZIP" ]; then + ZIP="$(mktemp -t ffmpeg-pin-XXXXXX.zip)" + trap 'rm -f "$ZIP"' EXIT + echo "══ Downloading the pinned FFmpeg build (~65 MB, one time) ══" + curl -fsSL --retry 3 -o "$ZIP" "$URL" +fi + +echo "══ Verifying SHA-256 ══" +GOT="$(sha256sum "$ZIP" | cut -d' ' -f1)" +if [ "$GOT" != "$SHA256" ]; then + echo "❌ SHA-256 mismatch — refusing to extract." + echo " expected $SHA256" + echo " actual $GOT" + exit 1 +fi +echo "✓ $GOT" + +echo "══ Extracting into ffmpeg/ ══" +python3 - "$ZIP" "$DEST" "$WANT_EXE" <<'PY' +import os, sys, zipfile + +archive_path, dest, want = sys.argv[1], sys.argv[2], set(sys.argv[3].split()) +wanted_exe = {n for n in want} + +os.makedirs(dest, exist_ok=True) +found = [] +with zipfile.ZipFile(archive_path) as z: + for name in z.namelist(): + if '/bin/' not in name or name.endswith('/'): + continue + base = os.path.basename(name) + if not (base.endswith('.dll') or base in wanted_exe): + continue + with z.open(name) as src, open(os.path.join(dest, base), 'wb') as out: + out.write(src.read()) + found.append(base) + +missing = wanted_exe - set(found) +if missing: + sys.exit(f"❌ archive is missing required binaries: {sorted(missing)}") + +print(f"✓ extracted {len(found)} files") +for base in sorted(found): + print(f" {base}") +PY + +echo "" +echo "✓ Bundled FFmpeg ready. The build copies ffmpeg/ → bin///ffmpeg/." +echo " Nothing is downloaded at runtime any more (Store policy 10.2.2 satisfied)." diff --git a/ytLive.Tests/FfmpegLocatorTests.cs b/ytLive.Tests/FfmpegLocatorTests.cs index a626e18..59df7dc 100644 --- a/ytLive.Tests/FfmpegLocatorTests.cs +++ b/ytLive.Tests/FfmpegLocatorTests.cs @@ -1,39 +1,30 @@ +using System; +using System.Diagnostics; using System.IO; -using System.IO.Compression; +using System.Linq; using Xunit; using ytLive.Services.Encoder; namespace ytLive.Tests; /// -/// The FFmpeg locator (TASK 4 ship step 2): resolves ffmpeg.exe by probing PATH -/// first, then the cache in the tools directory, then pulling the pinned BtbN -/// LGPL-shared zip. The integration test drives the full ladder against a temp -/// tools dir and a fake downloader that returns a real in-memory zip; the units -/// pin down the failure and edge cases. No network, no real binary. +/// The FFmpeg locator (TASK 4 ship step 2, rewritten by TASK 48 item 1). +/// +/// What changed. The locator used to pin a BtbN release URL, download the zip +/// on first run, extract it, and execute it. That is Windows Store policy 10.2.2 (dynamic +/// code inclusion — TASKS/research-store-certification.md §4) and it had already +/// failed in production: the pin aged out of BtbN's 14-day retention and the download +/// 404'd on the creator's first real recording attempt (2026-09-01). FFmpeg is now +/// bundled, fetched at build time by scripts/fetch-ffmpeg.sh and copied into +/// ffmpeg/ beside the executable. Nothing is downloaded at runtime, ever. +/// +/// Why the order flipped. PATH used to win ("the user's own install wins"). +/// With a bundle in the package that is backwards: a PATH ffmpeg may be a different +/// major version or a GPL build, and the bundle is the one that was pinned, license-vetted +/// and tested. PATH remains the escape hatch for someone who deliberately wants their own. /// public class FfmpegLocatorTests { - private static byte[] MakeZip( - string exePath = "ffmpeg-master-latest-win64-lgpl-shared/bin/ffmpeg.exe", - string[]? dllPaths = null) - { - using var ms = new MemoryStream(); - using (var archive = new ZipArchive(ms, ZipArchiveMode.Create, leaveOpen: true)) - { - var entry = archive.CreateEntry(exePath); - using (var writer = new StreamWriter(entry.Open())) - writer.Write("dummy ffmpeg binary"); - foreach (var dll in dllPaths ?? Array.Empty()) - { - var dllEntry = archive.CreateEntry(dll); - using var dllWriter = new StreamWriter(dllEntry.Open()); - dllWriter.Write("dummy dll"); - } - } - return ms.ToArray(); - } - private static string TempDir() { var dir = Path.Combine(Path.GetTempPath(), "ytllive-tests-" + Guid.NewGuid().ToString("N")); @@ -41,146 +32,215 @@ public class FfmpegLocatorTests return dir; } - private sealed class RecordingDownloader + /// Repo root, found by walking up to a file only the repo has. + private static string RepoRoot() { - public int Calls { get; private set; } - public byte[] Payload { get; set; } = MakeZip(); - public Exception? Error { get; set; } - - public Task DownloadAsync(string url, CancellationToken ct) + var dir = new DirectoryInfo(AppContext.BaseDirectory); + while (dir != null) { - Calls++; - if (Error != null) throw Error; - return Task.FromResult(Payload); + if (File.Exists(Path.Combine(dir.FullName, "ytLive.csproj"))) return dir.FullName; + dir = dir.Parent; } + throw new FileNotFoundException($"could not locate ytLive.csproj above {AppContext.BaseDirectory}"); } - [Fact] - public async Task Locate_Integration_FullDecisionLadder() + private static string RepoFile(params string[] parts) { - var toolsDir = TempDir(); - try - { - // 1. PATH hit wins, downloader never invoked. - var pathDir = TempDir(); - var pathExe = Path.Combine(pathDir, FfmpegLocator.FileName); - File.WriteAllText(pathExe, "user's ffmpeg"); - var downloader = new RecordingDownloader(); - - var fromPath = new FfmpegLocator([pathDir], toolsDir, downloader.DownloadAsync); - Assert.Equal(pathExe, await fromPath.LocateAsync()); - Assert.Equal(0, downloader.Calls); - - // 2. Cache hit skips the network. - var cached = Path.Combine(toolsDir, FfmpegLocator.FileName); - Directory.CreateDirectory(toolsDir); - File.WriteAllText(cached, "cached ffmpeg"); - var fromCache = new FfmpegLocator([], toolsDir, downloader.DownloadAsync); - Assert.Equal(cached, await fromCache.LocateAsync()); - Assert.Equal(0, downloader.Calls); - - // 3. Cold cache downloads exactly once, extracts ffmpeg.exe, and the - // second call serves the cache without re-downloading. - File.Delete(cached); - var coldTools = TempDir(); - var cold = new FfmpegLocator([], coldTools, downloader.DownloadAsync); - var resolved = await cold.LocateAsync(); - Assert.Equal(Path.Combine(coldTools, FfmpegLocator.FileName), resolved); - Assert.Equal(1, downloader.Calls); - Assert.True(new FileInfo(resolved).Length > 0); - Assert.Equal(resolved, await cold.LocateAsync()); - Assert.Equal(1, downloader.Calls); - } - finally - { - Directory.Delete(toolsDir, recursive: true); - } + var candidate = Path.Combine(new[] { RepoRoot() }.Concat(parts).ToArray()); + if (File.Exists(candidate)) return candidate; + throw new FileNotFoundException($"could not locate {Path.Combine(parts)} under {RepoRoot()}"); } + /// + /// The Good Dog integration test for this unit: prove the whole chain end to end — + /// the build actually copied the bundle into the app's output, the locator resolves it + /// without any network, and the located binary executes. Running it is the part + /// that matters most: FFmpeg is an lgpl-shared build, so it only starts if every + /// libav*.dll sits beside the exe. A packaging change that dropped the DLLs would + /// pass every existence check and still hand the user a broken encoder. + /// [Fact] - public async Task Locate_ZeroByteCache_IsRefreshed() + public async Task Locate_Integration_TheBuiltOutputShipsABundleThatRuns() { - var toolsDir = TempDir(); - try - { - File.WriteAllText(Path.Combine(toolsDir, FfmpegLocator.FileName), ""); - var downloader = new RecordingDownloader(); - var locator = new FfmpegLocator([], toolsDir, downloader.DownloadAsync); - var resolved = await locator.LocateAsync(); - Assert.Equal(1, downloader.Calls); - Assert.True(new FileInfo(resolved).Length > 0); - } - finally - { - Directory.Delete(toolsDir, recursive: true); - } - } + // NOTE: the output root is derived from the *project* directory, not from the + // repository folder name. The repo directory happens to also be called "ytLive", + // so naively appending "ytLive" would resolve to a path that only works on this + // machine's checkout. + var projectDir = Path.GetDirectoryName(RepoFile("ytLive.csproj"))!; + var binRoot = Path.Combine(projectDir, "bin"); + Assert.True(Directory.Exists(binRoot), + $"no build output at {binRoot} — build ytLive.csproj before running this test"); - [Fact] - public async Task Locate_SharedBuild_ExtractsDllsAlongsideExe() - { - var toolsDir = TempDir(); - try + var withBundle = Directory + .GetDirectories(binRoot, "*", SearchOption.AllDirectories) + .Where(d => File.Exists(Path.Combine(d, FfmpegLocator.BundleFolderName, FfmpegLocator.FileName))) + .ToArray(); + + Assert.True(withBundle.Length > 0, + $"No build output under {binRoot} carries ffmpeg/{FfmpegLocator.FileName}. " + + "Run ./scripts/fetch-ffmpeg.sh, then rebuild — the csproj GuardBundledFfmpeg " + + "target should have failed the build if the source folder was missing."); + + var outputDir = withBundle[0]; + var bundleDir = Path.Combine(outputDir, FfmpegLocator.BundleFolderName); + + // The sibling DLLs are the shared-build's whole dependency story. + foreach (var required in new[] { FfmpegLocator.ProbeFileName, "avcodec-63.dll", "avformat-63.dll" }) + Assert.True(File.Exists(Path.Combine(bundleDir, required)), + $"{required} must ship beside {FfmpegLocator.FileName} — a shared build resolves DLLs " + + "from the executable's own directory."); + + // The locator resolves it with an empty PATH, i.e. nothing outside the bundle. + var resolved = await new FfmpegLocator(searchDirs: [], bundleRoot: outputDir).LocateAsync(); + + Assert.Equal(Path.Combine(bundleDir, FfmpegLocator.FileName), resolved); + + // And it actually runs. This is what proves the DLLs resolve. + var info = new ProcessStartInfo { - var downloader = new RecordingDownloader - { - Payload = MakeZip(dllPaths: - [ - "ffmpeg-master-latest-win64-lgpl-shared/bin/avcodec-61.dll", - "ffmpeg-master-latest-win64-lgpl-shared/bin/avformat-61.dll", - ]) - }; - var locator = new FfmpegLocator([], toolsDir, downloader.DownloadAsync); - var resolved = await locator.LocateAsync(); - Assert.True(File.Exists(Path.Combine(toolsDir, "avcodec-61.dll"))); - Assert.True(File.Exists(Path.Combine(toolsDir, "avformat-61.dll"))); - Assert.Equal(1, downloader.Calls); - } - finally - { - Directory.Delete(toolsDir, recursive: true); - } - } - - [Fact] - public async Task Locate_EmptyPayload_Throws() - { - var downloader = new RecordingDownloader { Payload = Array.Empty() }; - var locator = new FfmpegLocator([], TempDir(), downloader.DownloadAsync); - await Assert.ThrowsAsync(() => locator.LocateAsync()); - Assert.Equal(1, downloader.Calls); - } - - [Fact] - public async Task Locate_ZipWithoutFfmpegEntry_Throws() - { - var downloader = new RecordingDownloader { Payload = MakeZip(exePath: "readme.txt") }; - var locator = new FfmpegLocator([], TempDir(), downloader.DownloadAsync); - await Assert.ThrowsAsync(() => locator.LocateAsync()); - } - - [Fact] - public async Task Locate_DownloaderFailure_Propagates() - { - var downloader = new RecordingDownloader { Error = new HttpRequestException("offline") }; - var locator = new FfmpegLocator([], TempDir(), downloader.DownloadAsync); - var ex = await Assert.ThrowsAsync(() => locator.LocateAsync()); - Assert.IsType(ex.InnerException); - Assert.Contains("pinned build", ex.Message, StringComparison.OrdinalIgnoreCase); - } - - [Fact] - public async Task Locate_DeadPin404_ThrowsActionableLocatorError() - { - // 2026-09-01 incident: the pinned autobuild tag aged out of BtbN retention; - // the raw HttpRequestException must never surface from the pump again. - var downloader = new RecordingDownloader - { - Error = new HttpRequestException("Response status code does not indicate success: 404 (Not Found)."), + FileName = resolved, + Arguments = "-version", + UseShellExecute = false, + RedirectStandardOutput = true, + RedirectStandardError = true, + CreateNoWindow = true, }; - var locator = new FfmpegLocator([], TempDir(), downloader.DownloadAsync); - var ex = await Assert.ThrowsAsync(() => locator.LocateAsync()); - Assert.Contains("PATH", ex.Message); - Assert.Contains("PinnedUrl", ex.Message); + using var process = Process.Start(info)!; + var stdout = process.StandardOutput.ReadToEnd(); + Assert.True(process.WaitForExit(milliseconds: 30_000), "bundled ffmpeg -version did not exit in 30s"); + Assert.Equal(0, process.ExitCode); + Assert.Contains("ffmpeg version", stdout, StringComparison.OrdinalIgnoreCase); + } + + [Fact] + public async Task Locate_PrefersTheBundle_OverAPathInstall() + { + var root = TempDir(); + try + { + var pathDir = Path.Combine(root, "on-path"); + var bundleDir = Path.Combine(root, FfmpegLocator.BundleFolderName); + Directory.CreateDirectory(pathDir); + Directory.CreateDirectory(bundleDir); + File.WriteAllText(Path.Combine(pathDir, FfmpegLocator.FileName), "user's ffmpeg"); + var bundled = Path.Combine(bundleDir, FfmpegLocator.FileName); + File.WriteAllText(bundled, "bundled ffmpeg"); + + var resolved = await new FfmpegLocator([pathDir], root).LocateAsync(); + Assert.Equal(bundled, resolved); + } + finally + { + Directory.Delete(root, recursive: true); + } + } + + [Fact] + public async Task Locate_FallsBackToPath_WhenNoBundleIsPresent() + { + var root = TempDir(); + try + { + var pathDir = Path.Combine(root, "on-path"); + Directory.CreateDirectory(pathDir); + var onPath = Path.Combine(pathDir, FfmpegLocator.FileName); + File.WriteAllText(onPath, "user's ffmpeg"); + + var resolved = await new FfmpegLocator([pathDir], root).LocateAsync(); + Assert.Equal(onPath, resolved); + } + finally + { + Directory.Delete(root, recursive: true); + } + } + + [Fact] + public async Task Locate_ZeroByteBundle_IsNotUsable() + { + var root = TempDir(); + try + { + var bundleDir = Path.Combine(root, FfmpegLocator.BundleFolderName); + Directory.CreateDirectory(bundleDir); + File.WriteAllText(Path.Combine(bundleDir, FfmpegLocator.FileName), ""); + + var ex = await Assert.ThrowsAsync( + () => new FfmpegLocator([], root).LocateAsync()); + Assert.Contains(FfmpegLocator.BundleFolderName, ex.Message, StringComparison.Ordinal); + } + finally + { + Directory.Delete(root, recursive: true); + } + } + + /// + /// A missing bundle must be LOUD and must say where to look. The old ladder had a + /// "found something on disk" path that could quietly satisfy a missing bundle with a + /// stale cached copy — silent drift in exactly the dependency Store certification + /// inspects, and silent drift is how a packaging bug ships. + /// + [Fact] + public async Task Locate_NoBundleAndNoPath_ThrowsNamingTheBundleFolderAndTheFix() + { + var root = TempDir(); + try + { + var ex = await Assert.ThrowsAsync( + () => new FfmpegLocator([], root).LocateAsync()); + Assert.Contains(FfmpegLocator.BundleFolderName, ex.Message, StringComparison.Ordinal); + Assert.Contains("fetch-ffmpeg.sh", ex.Message, StringComparison.Ordinal); + Assert.Contains("PATH", ex.Message, StringComparison.Ordinal); + } + finally + { + Directory.Delete(root, recursive: true); + } + } + + /// + /// The Store-gate assertion, kept as a source-level fact so it cannot rot: the locator + /// must contain no network capability at all. Policy 10.2.2 forbids download-and-execute, + /// and this is the only type in the app that ever executed a downloaded binary. If a + /// future change reintroduces an HttpClient, a URL, or a downloader here, this fails — + /// which is the entire point. + /// + /// Patterns are call-shaped (new HttpClient, not HttpClient) + /// on purpose. A bare word match flags the file's own doc comment, which legitimately + /// says "nothing is downloaded" — a test that fails on the prose explaining the + /// guarantee is a test that gets deleted instead of obeyed. + /// + [Theory] + [InlineData("new HttpClient")] + [InlineData("HttpClient()")] + [InlineData("GetByteArrayAsync(")] + [InlineData("GetStreamAsync(")] + [InlineData("WebRequest.Create")] + [InlineData("DownloadFileTaskAsync")] + [InlineData("http://")] + [InlineData("https://")] + [InlineData("PinnedUrl =")] + [InlineData("ExtractBinaries(")] + public void TheLocatorCarriesNoNetworkCapability(string forbidden) + { + var source = File.ReadAllText( + RepoFile("Services", "Encoder", "FfmpegLocator.cs")); + + Assert.DoesNotContain(forbidden, source, StringComparison.OrdinalIgnoreCase); + } + + /// + /// The guard that makes a forgotten bundle a build failure rather than a runtime one. + /// Without it, a fresh clone with no ffmpeg/ builds cleanly and the app only + /// discovers the problem when the creator presses Record. + /// + [Fact] + public void TheBuildRefusesToCompileWithoutTheBundle() + { + var csproj = File.ReadAllText(RepoFile("ytLive.csproj")); + Assert.Contains("GuardBundledFfmpeg", csproj, StringComparison.Ordinal); + Assert.Contains("ffmpeg\\ffmpeg.exe", csproj, StringComparison.Ordinal); + Assert.Contains("CopyToPublishDirectory=\"Always\"", csproj, StringComparison.Ordinal); } } diff --git a/ytLive.csproj b/ytLive.csproj index ec53ad9..135b622 100644 --- a/ytLive.csproj +++ b/ytLive.csproj @@ -75,6 +75,25 @@ + + + + + + + + + <_Parameter1>ytLive.Tests