gramps/LlamaCasty
0
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

TASK 48 item 1: bundle ffmpeg — the app no longer downloads an executable

Browse Source 
Store policy 10.2.2 forbids dynamic code inclusion, and this was the only
third-party binary the app ever downloaded and ran. It had already failed in
production: the pin aged out of BtbN's 14-day retention and the download 404'd
on the creator's first real recording attempt (2026-09-01). Bundling kills
that failure class permanently and deletes the startup network dependency.

- scripts/fetch-ffmpeg.sh populates a gitignored ./ffmpeg from a pinned
  month-end BtbN LGPL-shared artifact, SHA-256 verified before extraction,
  idempotent, FFMPEG_ZIP= override. 135 MB of DLLs in git would tax every
  commit on this 9p tree; the URL + hash are the audit trail LGPL §6 wants.
- ytLive.csproj copies ffmpeg\** to output and publish (Always), and a
  GuardBundledFfmpeg target fails the BUILD if the folder is missing, so a
  fresh clone finds out before a take rather than during one.
- FfmpegLocator loses the downloader, the zip extraction and the
  %APPDATA%\ytLlive\tools cache. Order is now bundle -> PATH, deliberately
  inverted: a PATH ffmpeg may be a different major version or a GPL build,
  and the bundle is what was pinned, license-vetted and tested. A missing
  bundle throws loudly; the old "found something on disk" path could satisfy
  a missing bundle with a stale copy, and silent drift is the worst failure
  mode for a dependency certification inspects.
- ffplay.exe (18 MB) is not shipped; nothing calls it.
- THIRD-PARTY-NOTICES.txt: "downloads" -> "bundled" throughout, and the stale
  build tag autobuild-2026-08-09-13-03 — the tag that 404'd — corrected to
  autobuild-2026-08-31-13-27 with the exact artifact name and archive hash.
  OpenH264 made precise: compiled INTO avcodec-63.dll, no separate binary.

Checked whether other third-party binaries needed bundling: no. e_sqlite3.dll
and WebView2Loader.dll already ship via NuGet into runtimes\win-x64\native\,
OpenH264 is inside avcodec-63.dll, the rest are managed. Two gaps remain that
bundling cannot fix, recorded in the map: the WebView2 Evergreen runtime is a
machine prerequisite (TASK 48 item 2), and Velopack's updater downloads and
executes, so its teardown is a compliance action.

Tests: 16 locator facts, suite 379/379, 0 warnings. The integration test runs
the bundled ffmpeg -version, which is the only assertion that proves the
sibling libav*.dll resolved — presence checks pass with the DLLs missing and a
shared build then fails to start. TheLocatorCarriesNoNetworkCapability keeps
10.2.2 from rotting back.

Analysis: TASKS/research-store-certification.md §4.
This commit is contained in:
gramps
2026-09-28 09:38:21 -07:00
parent eb0dd91126
commit 4fcf76289c
11 changed files with 638 additions and 361 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Services/Encoder/FfmpegLocator.cs
+47 -119
View File
@@ -1,153 +1,81 @@
using System.IO;
using System.IO.Compression;
using System.Net.Http;
using ytLive.Helpers;
namespace ytLive.Services.Encoder;
/// <summary>
/// The default <see cref="IFfmpegLocator"/>: probe PATH first (the user's own
/// install wins), then the cache in <c>%APPDATA%\ytLlive\tools</c>, then pull the
/// pinned BtbN **lgpl-shared** build (TASK 4 ship step 2). The shared variant is a
/// deliberate licensing choice: dynamic linking means LGPL compliance is "license
/// text + source offer", with no static-relink (LGPL §6) material required. The
/// shared zip puts <c>ffmpeg.exe</c> plus the <c>libav*.dll</c> family in <c>bin/</c>,
/// so both are extracted — Windows resolves the DLLs from the exe's own directory.
/// Search dirs, tools dir, and the downloader are constructor-injected so tests
/// fake the network and stay on a temp directory.
/// The default <see cref="IFfmpegLocator"/>: resolve the <b>bundled</b> FFmpeg that ships
/// next to the app, falling back to PATH for anyone who deliberately installed their own.
///
/// <para><b>Nothing is downloaded.</b> This type used to pin a BtbN release URL, download
/// the zip on first run, and execute it — which is Windows Store policy 10.2.2 (dynamic
/// code inclusion, cited in <c>TASKS/research-store-certification.md</c> §4) and already
/// cost a real failure: the pin aged out of BtbN's 14-day retention and the download
/// <b>404'd on the creator's first real recording attempt</b> (2026-09-01). The binaries
/// are now fetched at BUILD time by <c>scripts/fetch-ffmpeg.sh</c> (pinned URL + SHA-256)
/// and copied into <c>ffmpeg/</c> beside the executable.</para>
///
/// <para><b>Order is bundled-first, deliberately.</b> The old code probed PATH first
/// ("the user's own install wins"). With a bundle in the package that is the wrong way
/// round: a random PATH <c>ffmpeg</c> may be a different major version or a GPL build, and
/// the bundle is the one we pinned, license-vetted, and tested against. PATH stays as the
/// escape hatch, which is what it is actually good for.</para>
///
/// <para>The lgpl-<b>shared</b> variant is a licensing choice, not a packaging one: dynamic
/// linking makes LGPL compliance "license text + source offer" with no static-relink
/// (LGPL §6) material required. That means the <c>libav*.dll</c> family must sit in the
/// same folder as the exe — Windows resolves them from the executable's own directory.</para>
/// </summary>
public sealed class FfmpegLocator : IFfmpegLocator
{
public const string FileName = "ffmpeg.exe";
/// <summary>Sibling probe binary — extracted alongside ffmpeg so the media
/// <summary>Sibling probe binary — bundled alongside ffmpeg so the media
/// frame-rate probe (<see cref="ytLive.Services.FfmpegFrameRateProbe"/>) can
/// derive its path from the located ffmpeg directory.</summary>
public const string ProbeFileName = "ffprobe.exe";
/// <summary>
/// Pinned BtbN LGPL-shared win64 build — <c>autobuild-2026-08-31-13-27</c>, the
/// ffmpeg N-126342 build (immutable tag; recorded in TASKS.md per the licensing rule).
/// Deliberately a MONTH-END tag: BtbN retains the last ~14 dailies but month-end
/// builds for ~2 years — the previous pin (a 2026-08-09 daily) aged out of retention
/// and 404'd on a cold cache exactly as ai.md predicted (2026-09-01, the first real
/// recording attempt). Dated tags carry versioned asset names (ffmpeg-N-…-win64-lgpl-shared.zip);
/// extraction is name-agnostic.
/// </summary>
public const string PinnedUrl =
"https://github.com/BtbN/FFmpeg-Builds/releases/download/autobuild-2026-08-31-13-27/ffmpeg-N-126342-gf88b741dbf-win64-lgpl-shared.zip";
/// <summary>Folder, relative to the app base directory, that the build copies the
/// bundled build into. One folder for the exe and every DLL.</summary>
public const string BundleFolderName = "ffmpeg";
/// <summary>Base directory the bundle is resolved against. Injectable so tests can
/// point at a temp tree; defaults to the running app's base directory.</summary>
private readonly string _bundleRoot;
private readonly string[] _searchDirs;
private readonly string _toolsDir;
private readonly Func<string, CancellationToken, Task<byte[]>> _downloader;
public FfmpegLocator(
string[]? searchDirs = null,
string? toolsDir = null,
Func<string, CancellationToken, Task<byte[]>>? downloader = null)
public FfmpegLocator(string[]? searchDirs = null, string? bundleRoot = null)
{
_searchDirs = searchDirs ?? ParsePath();
_toolsDir = toolsDir ?? Path.Combine(
Environment.GetFolderPath(Environment.SpecialFolder.ApplicationData), "ytLlive", "tools");
_downloader = downloader ?? DefaultDownload;
_bundleRoot = bundleRoot ?? AppContext.BaseDirectory;
}
public async Task<string> LocateAsync(CancellationToken cancellationToken = default)
public Task<string> LocateAsync(CancellationToken cancellationToken = default)
{
var bundled = Path.Combine(_bundleRoot, BundleFolderName, FileName);
if (IsUsable(bundled)) return Task.FromResult(bundled);
foreach (var dir in _searchDirs)
{
if (string.IsNullOrEmpty(dir)) continue;
var candidate = Path.Combine(dir, FileName);
if (File.Exists(candidate)) return candidate;
if (IsUsable(candidate)) return Task.FromResult(candidate);
}
var cached = Path.Combine(_toolsDir, FileName);
if (File.Exists(cached) && new FileInfo(cached).Length > 0) return cached;
try
{
var zip = await _downloader(PinnedUrl, cancellationToken).ConfigureAwait(false);
if (zip.Length == 0)
throw new IOException($"FFmpeg download from {PinnedUrl} returned an empty payload.");
Directory.CreateDirectory(_toolsDir);
ExtractBinaries(zip, _toolsDir);
return cached;
}
catch (HttpRequestException ex)
{
// The old gap: this exception type escaped unwrapped, so a dead pin surfaced
// as a raw "Response status code does not indicate success: 404" from the
// frame pump. Locator failures are IOException with an actionable message.
AppLog.Write(ex, "FFmpeg locator: download failed");
throw new IOException(
$"FFmpeg download failed ({ex.Message}). The pinned build may have aged out of " +
"retention — update FfmpegLocator.PinnedUrl, or install FFmpeg (LGPL-shared build) " +
"and make sure 'ffmpeg' is on PATH.", ex);
}
catch (Exception ex) when (ex is IOException or InvalidDataException or NotSupportedException)
{
AppLog.Write(ex, "FFmpeg locator: download/extract failed");
throw;
}
// Deliberately loud. The previous cache-then-download ladder had a "found
// something on disk" path that could satisfy a missing bundle with a stale copy,
// which is exactly the silent-drift a certification-gated dependency must not
// have. If the package forgot ffmpeg/, the app says so and stops.
throw new IOException(
$"FFmpeg was not found. The bundled copy is missing from " +
$"'{Path.Combine(_bundleRoot, BundleFolderName)}' — this build is incomplete " +
"(reinstall or re-fetch with scripts/fetch-ffmpeg.sh). Installing an LGPL-shared " +
"FFmpeg and putting 'ffmpeg' on PATH also works.");
}
/// <summary>
/// Extract <c>ffmpeg.exe</c> and every <c>*.dll</c> into <c>toolsDir</c> via a
/// staging directory, so a failed extract never leaves a partially-populated
/// cache behind (the previous good cache stays until every move succeeds).
/// </summary>
private static void ExtractBinaries(byte[] zip, string toolsDir)
{
var staging = toolsDir + ".stage";
if (Directory.Exists(staging)) Directory.Delete(staging, recursive: true);
Directory.CreateDirectory(staging);
try
{
using var stream = new MemoryStream(zip, writable: false);
using var archive = new ZipArchive(stream, ZipArchiveMode.Read);
var exe = archive.Entries.FirstOrDefault(
e => e.FullName.EndsWith("/" + FileName, StringComparison.OrdinalIgnoreCase))
?? throw new InvalidDataException($"The pinned FFmpeg archive does not contain {FileName}.");
ExtractOne(exe, Path.Combine(staging, FileName));
var probe = archive.Entries.FirstOrDefault(
e => e.FullName.EndsWith("/" + ProbeFileName, StringComparison.OrdinalIgnoreCase));
if (probe != null)
ExtractOne(probe, Path.Combine(staging, ProbeFileName));
foreach (var entry in archive.Entries)
{
if (entry.FullName.EndsWith(".dll", StringComparison.OrdinalIgnoreCase))
ExtractOne(entry, Path.Combine(staging, Path.GetFileName(entry.FullName)));
}
foreach (var file in Directory.GetFiles(staging))
File.Move(file, Path.Combine(toolsDir, Path.GetFileName(file)), overwrite: true);
}
finally
{
if (Directory.Exists(staging)) Directory.Delete(staging, recursive: true);
}
}
private static void ExtractOne(ZipArchiveEntry entry, string destination)
{
var temp = destination + ".tmp";
try
{
using (var source = entry.Open())
using (var target = File.Create(temp))
source.CopyTo(target);
File.Move(temp, destination);
}
finally
{
if (File.Exists(temp)) File.Delete(temp);
}
}
private static async Task<byte[]> DefaultDownload(string url, CancellationToken ct)
{
using var http = new HttpClient { Timeout = TimeSpan.FromMinutes(10) };
return await http.GetByteArrayAsync(url, ct).ConfigureAwait(false);
}
private static bool IsUsable(string path) =>
File.Exists(path) && new FileInfo(path).Length > 0;
private static string[] ParsePath()
{