Commit Graph
4 Commits
Author SHA1 Message Date
gramps 2670ac022b feat(media): play the clip's soundtrack and share one decode pipe
A video file was picture-only: the decode pipe passed `-an`, so a starting-soon
or BRB clip with music played silent everywhere. Adds sound to every clip and
collapses the two decoders into one.

- `Services/VideoClipPipe.cs`: the shared pipe behind `MediaVideoSource` (TASK 21)
  and `AlertClipDecoder` (TASK 47) — video (`rawvideo bgra`, scaled) + audio
  (`f32le -ac 2 -ar 48000`, 50 ms chunks, byte-carry across reads), both paced to
  real time, loop restarts both so picture and sound stay together. Alert
  behaviour is unchanged.
- `MediaVideoSourceManager` applies the layer's Level/Mute per session and raises
  `AudioAvailable`; `MainViewModel` feeds it to the new
  `AudioMixer.EnqueueDesktopMediaAudio`, which joins the DESKTOP/GAME ring so the
  desktop fader and the mic auto-ducker apply exactly as they do for game audio
  and TRAX (creator ruling 2026-11-05: audio joins the group).
- Per-layer `MediaMuted` + existing `MediaVolume` surfaced as Mute/Level in the
  properties panel; `MediaMuted` persisted (ALTER TABLE guard + SELECT/INSERT).
- Level/mute are per SESSION because one decoder is shared per file path — the
  last layer touched wins; one decoder per layer would play the file twice.

Test: `MediaSourceAudioTests` — the pipe emits 50 ms chunks at 48k stereo, the
manager scales by level and drops when muted, and a media layer's sound moves the
desktop meter. Alert/decode suites unchanged and green.

Docs: task-21 closed on item 10, the "silent soundtrack" facts corrected in
frame-pipeline/output/overlays, HANDOFF rewritten. verify.sh green (529 passed;
the two documented pre-existing failures only).
2026-11-05 07:54:06 -08:00
gramps 6310f50c27 perf(video): move tier scaling to ffmpeg
The frame pump now submits crop-sized BGRA frames; FFmpeg scales them to final tier dimensions. Bundled scaler benchmark: 300 608x1080 to 1080x1920 frames in 0.435s (scaler-only).

Reference: https://ffmpeg.org/ffmpeg-filters.html#scale-1
2026-11-04 10:52:35 -08:00
gramps a311c8feb1 docs(ai): partition ai.md into five subsystem files (verbatim moves)
ai.md 1,308 → 228 lines. No prose rewritten: every block was moved as-is so
nothing could be lost in translation, then replaced by a pointer.

New files:
- docs/ai/environment.md — LAN/SSH + Gitea push recipe, Windows dotnet
  build/run, test-suite map, hermetic real-MainWindow test rule
- docs/ai/product.md — brand/naming, AI transparency, GPL licensing
  prohibitions, free-forever monetization, auth/account assumptions,
  packaging & distribution
- docs/ai/youtube-api.md — go-live ordering, the broadcast-binding fix, the
  private-only lock, health parsing, end-stream pre-flight, Test mode,
  chat insert shape
- docs/ai/overlays.md — social bar, native alert box (animations, clip,
  ticker), chat box conventions
- docs/ai/ui-surface.md — control surface, scene strip, preview rail,
  hotkeys, broadcast drawer, toasts, transitions, brand logo, launch
  geometry, theme/properties conventions

ai.md keeps the response/derivation rules, the architecture table, the four
existing invariant stubs, and one pointer per moved subsystem.
docs/ai/index.md routes every file with a "read when" column.

References falsified by the move were fixed in the same commit: AGENTS.md
build note → environment.md → Run, and the "Related" footers in
frame-pipeline.md / output.md / capture.md now point at the owning files.

Docs-only unit: scope-check run, no build (no source touched).
2026-10-29 11:05:53 -07:00
gramps a4dfca6b6c docs(memory): split the output path (ffmpeg, encoder, recording, audio) out of ai.md
Unit 3. Verbatim move of 'FFmpeg locator', 'Live encoder + RTMP push',
'Local recording' and 'Live audio capture' into docs/ai/output.md, grouped as
encoder / recording / audio with a 'Related' footer.

ai.md's stub keeps the decisions that must not be re-litigated or rediscovered:
ffmpeg is bundled (never download/fetch; no-network test), args pure and no -re,
probe order NVENC->QSV->AMF->OpenH264 and never libx264, REC/ON-AIR radio-
exclusive by creator ruling, a modal's falsy result must never fall through into
its side effect, full StopStream teardown on every failure, audio pipe closes
BEFORE the video pump, and honest-by-measurement audio (true dBFS meters,
SpeechLeveler + -1 dBFS limiter, live Func gains).

- ai.md Services row and one 'FFmpeg locator section above' pointer repointed;
  MyMistakes.md dead-Pin entry repointed.
- docs/ai/index.md gains the output.md row.

ai.md 1,558 -> 1,289 lines. No source touched; no push.
2026-10-26 10:28:01 -07:00