TASK 9 audio milestone: real stream audio + voice filters + auto-duck + free TRAX music — the mixer now feeds the encoder real audio (ffmpeg reads a Windows named pipe, -f f32le -ar 48000 -ac 2 -i \.\pipe\ytllive_audio, replacing anullsrc silence; explicit -map 0:v -map 1:a via EncoderOptions.AudioPipeName), with honest gains reaching the stream (MicVolume scales mic, GameAudioVolume + mute scale loopback, Func<double> seams on AudioMixer), an always-on pure-C# voice chain on the mic BEFORE the meter and mix (LowShelfFilter 120Hz +4dB → HighShelfFilter 8kHz +3dB → NoiseGate 0.005/hysteresis 0.5 → Compressor 0.5 4:1, all TDF2), AutoDucker (mic RMS>0.02 → loopback ×0.25, attack 0.05/release 0.005), and TRAX free background music (MusicPlayer = MediaFoundationReader → VolumeWaveProvider16 at fixed 0.20 → WaveOutEvent via the new sibling NAudio.WinMM 2.2.1 package; plays to the default device so the existing loopback carries it, ducked with game; footer TRAX button with red/yellow/green status dot, left-click toggles/picks, right-click opens the OpenFileDialog picker, tooltip shows the track name; persisted via schema v9 single-row Music). Build-time deviations from the plan (recorded in ai.md + TASKS.md): WasapiCapture in NAudio 2.2.1 exposes no overridable GetDefaultMixFormat so sources run device mix format and the mixer's TinyResampler normalizes any rate to 48kHz (resampler IS the design); FfmpegEncoder.cs untouched — MainViewModel.StopStream calls _audioMixer.StopLive() (pipe EOF) before the frame pump stops; sound-bar relabelled 'Desktop Audio', IsGameAudioBarVisible = game sound OR music playing. Tests: new AudioPipelineTests (DSP/ring-buffer/ducker/resampler units + the ONE integration test Mix_WithFiltersDuckAndGain_Lands_On_AudioPipe reading real pipe bytes; ring-buffer overwrite bug found + fixed), FfmpegEncoderTests/LayoutStorePersistenceTests updated — 196 tests passing, 0 warnings
This commit is contained in:
+90
-46
@@ -7,49 +7,67 @@
|
||||
|
||||
## Session state (last updated: 2026-08-14)
|
||||
|
||||
- **Branch:** `main`, in sync with `origin/main`.
|
||||
- **TASK 21 — real logo + creator-hub About: SHIPPED** (this session). The About overlay is now the
|
||||
**creator hub**: the real logo ("llama fortnite superman logo" copied from `\\llamavault` → `Assets/llama-logo.png`,
|
||||
the creator's own art — no third-party license), plus hub links — **llama chile shop on YouTube**
|
||||
(`MainViewModel.ChannelUrl`, public now), **Mastodon** (`https://mastodon.llamachile.tube/@gramps`),
|
||||
**Buy me a coffee** (`https://buymeacoffee.com/llamachiley` — LIVE), and **Unlock Premium** (greyed
|
||||
"coming soon"; the itch.io product URL is a **tabled seam**, `MainViewModel.PremiumUrl` — the itch.io
|
||||
account was just created and needs a setup session later). **Licenses & legal** flips the About overlay
|
||||
to an in-app scrolling panel loading the shipped `THIRD-PARTY-NOTICES.txt` (`ShowLicenses()` reads from
|
||||
the executable dir on first open, graceful "not found" fallback — never the OS viewer); "← Back to
|
||||
About" (`BackToAbout_Click`) returns. Integration test (ONE): `AboutHubTests` drives the real window,
|
||||
asserting the hub opens, the link URLs are real, licensing loads the shipped notices text, and back
|
||||
returns to the hub. **174 tests passing, 0 warnings.**
|
||||
- **TASK 8 — meter scaling amplification:** SHIPPED + pushed (`0b71b03`). `AudioLevelMeter.ToDisplay`
|
||||
adds **+10 dB input amplification** before the −60..0 dBFS log map: speech peaks (~0.2 RMS) read
|
||||
~0.93 (red) and normal speech (~0.05) ~0.73 (yellow) at maxed volume; ≤0.001 linear still reads 0
|
||||
(never idles on background noise). One knob shared by the mic + game bars; `× MicVolume` untouched.
|
||||
- **TASK 4 ship step 7 — one-click go live + private-only enforcement: SHIPPED** (this session; the
|
||||
final open TASK 4 requirement). Double-enforced privacy:
|
||||
1. **Dialog locked to Private** — `GoLiveViewModel.Visibility` is a get-only `"Private"`, the
|
||||
visibility ComboBox is gone (replaced by a "Streams always start Private" note); settings' dead
|
||||
"Default Visibility" dropdown + `MainViewModel.Visibilities`/`DefaultStreamVisibility` removed.
|
||||
2. **Service forces private** — `YouTubeStreamService.CreateBroadcast` always sends
|
||||
`privacyStatus="private"`; ctor gained `HttpClient? http = null` seam for tests.
|
||||
3. **Wired into go-live** — `BeginGoLive` fires `CreateBroadcastAsync` (stores `_currentBroadcastId`
|
||||
for TASK 5's bind/transition; failure → `StreamStatus.Error` via AppLog, never a crash) +
|
||||
`_framePump.StartAsync()`; `StopStream` clears `_currentBroadcastId`.
|
||||
4. **PRIVATE badge** — dark-red bordered, next to REC in the top bar, `Visibility="{Binding
|
||||
IsLivePrivate}"` (MainWindow.xaml ~110).
|
||||
5. **Integration test** — `ytLive.Tests/YouTubeStreamServiceTests.cs`: the broadcast-insert request
|
||||
must carry `"privacyStatus":"private"` (RecordingHandler seam) + a no-session → null guard test.
|
||||
- **Verified:** build 0 warnings / 0 errors; **174/174 tests pass** (173 + the new AboutHubTests).
|
||||
- **Uncommitted (this session's TASK 21 batch):** `MainWindow.xaml`, `ViewModels/MainViewModel.cs`,
|
||||
`MainWindow.xaml.cs`, `Assets/llama-logo.png` (replaced with the real logo), `ytLive.Tests/AboutHubTests.cs`
|
||||
(new), `TASKS.md`, `HANDOFF.md`. ai.md + ViewModels/index.md updates pending — **commit + push
|
||||
before continuing.**
|
||||
- **Next step:** commit + push TASK 21, then **TASK 5 — reusable `variable` stream** (create once
|
||||
per channel, cache ingestion URL, bind to broadcast, feed `_rtmpUrlProvider` so the pump actually
|
||||
pushes — `_currentBroadcastId` is already stashed for it). Also queued: **task 22 (voice filters)**,
|
||||
and **setting up the itch.io product page** (the creator just made the account; the premium-unlock
|
||||
URL then lands in `MainViewModel.PremiumUrl` and the About hub's "Unlock Premium" button lights up).
|
||||
Nothing else queued — do not expand the task queue on your own. Optional, not queued: rewriting the
|
||||
healed entry's `ProfileUrl` to `https://mastodon.llamachile.tube/@gramps` (user must say the word).
|
||||
- **Branch:** `main` (audio milestone work is uncommitted, see "Uncommitted").
|
||||
- **AUDIO MILESTONE (TASK 9) — SHIPPED this session.** Real stream audio + voice filters + auto-duck
|
||||
+ free TRAX background music. Build **0 warnings**, full suite **196 passing** (ONE integration
|
||||
test: `AudioPipelineTests.Mix_WithFiltersDuckAndGain_Lands_On_AudioPipe`). Everything below in the
|
||||
milestone section is DONE. **NOT yet committed** — next step is review + commit (with `ai.md`/
|
||||
`TASKS.md`/this file already updated in the same changeset).
|
||||
- **Two build-time deviations from the settled plan (both recorded in ai.md + TASKS.md):**
|
||||
1. `MusicPlayer` uses **`WaveOutEvent`** (new sibling package `NAudio.WinMM` 2.2.1) — not `WasapiOut`
|
||||
as planned; `WaveOutEvent` isn't in `NAudio.Wasapi`/`NAudio.Core`.
|
||||
2. No forced 48 kHz capture — NAudio 2.2.1's `WasapiCapture` has no overridable
|
||||
`GetDefaultMixFormat`, so both sources run the device mix format and the mixer's
|
||||
`TinyResampler` normalizes any rate to 48 kHz (resampler IS the design, not a fallback).
|
||||
3. `FfmpegEncoder.cs` is **untouched** — the VM closes the audio pipe (`_audioMixer.StopLive()`) in
|
||||
`StopStream` BEFORE the frame pump stops (audio EOF then video EOF, in order).
|
||||
- **SERVER RECOVERY (llamachile.tube / YunoHost / DO droplet) — done earlier session.** Side task
|
||||
complete; **rotate the DO API token** (control-panel only) and any password no longer trusted.
|
||||
- **Shipped, all pushed before this session:** TASK 21 (creator-hub About, `3d92bd0`), TASK 8 (meter
|
||||
+10 dB), TASK 4 ship step 7 (one-click go-live + private-only). **174 tests passing, 0 warnings.**
|
||||
- **Uncommitted (whole TASK 9 milestone):** `Services/Audio/` (new `VoiceFilterChain.cs`,
|
||||
`AudioRingBuffer.cs`, `AutoDucker.cs`, `TinyResampler.cs`, `MusicPlayer.cs`, `NamedPipeAudioWriter.cs`,
|
||||
rewritten `AudioMixer.cs`, simplified `WasapiMicAudioSource.cs`), `Models/Music.cs` (new),
|
||||
`FfmpegArgs.cs`, `EncoderOptions.cs`, `LayoutStore.cs` (v9), `MainViewModel.cs`, `MainWindow.xaml`,
|
||||
`MainWindow.xaml.cs`, `ytLive.csproj` (NAudio.WinMM), `ytLive.Tests/` (`AudioPipelineTests.cs` new,
|
||||
`FfmpegEncoderTests.cs` + `LayoutStorePersistenceTests.cs` updated), and docs `ai.md`, `TASKS.md`,
|
||||
`HANDOFF.md`.
|
||||
- **Next step:** commit the milestone (docs already staged in the same changeset), push, then start
|
||||
**TASK 5** (reusable stream → `_rtmpUrlProvider` — the last blocker before go-live actually encodes).
|
||||
Then **itch.io product page setup** (premium URL lands in `MainViewModel.PremiumUrl`, lights up
|
||||
Unlock Premium). Optional, not queued: rewriting the healed entry's `ProfileUrl` to
|
||||
`https://mastodon.llamachile.tube/@gramps` (user must say the word).
|
||||
|
||||
## TASK 9 audio milestone — SHIPPED 2026-08-14 (what changed)
|
||||
|
||||
The creator's feature review settled this as the single next branch ("all the audio issues done and
|
||||
tested — a huge milestone"). Final spec and full shipped-state records live in `TASKS.md` (TASK 9)
|
||||
and `ai.md` ("Live audio capture"). Highlights:
|
||||
|
||||
- **Real audio into the encoder.** `FfmpegArgs` now builds `-f f32le -ar 48000 -ac 2 -i \\.\pipe\ytllive_audio`
|
||||
+ explicit `-map 0:v -map 1:a` (replaces `anullsrc` silence). `MainViewModel.BeginGoLive` →
|
||||
`_audioMixer.StartLive(EncoderOptions.DefaultAudioPipeName)`; `StopStream` → `_audioMixer.StopLive()`
|
||||
before `_framePump.StopAsync()`.
|
||||
- **2-input mix (mic + loopback)**, honest gains: `micGain = MicVolume` (mute = 0),
|
||||
`loopbackGain = GameMuted ? 0 : GameAudioVolume` × duck. No third music channel.
|
||||
- **Voice chain on the mic** (TASK 22), before meter AND mix: bass 120 Hz +4 dB → treble 8 kHz +3 dB →
|
||||
gate (0.005 / hysteresis 0.5) → compressor (0.5, 4:1). Pure TDF2 DSP, per-sample, always on.
|
||||
- **Auto-duck:** mic RMS > 0.02 → loopback ×0.25 (−12 dB), attack 0.05 / release 0.005.
|
||||
- **TRAX (free BGM):** `MusicPlayer` = MediaFoundationReader → `VolumeWaveProvider16` at fixed **0.20** →
|
||||
`WaveOutEvent`. Plays to the default device → rides the loopback into the stream (ducked with game).
|
||||
Footer TRAX button (dot red/yellow/green + "TRAX"), left-click toggles/picks, right-click opens the
|
||||
picker (`OpenFileDialog`), tooltip shows the track name. Track persists via **schema v9** single-row
|
||||
`Music`. Sound-bar label "Game Audio Capture" → **"Desktop Audio"**;
|
||||
`IsGameAudioBarVisible = gameDetectorProducingSound || IsMusicPlaying`.
|
||||
- **Tests:** new `AudioPipelineTests.cs` (DSP/ring-buffer/ducker/resampler units + the ONE integration
|
||||
test reading real pipe bytes via `NamedPipeClientStream`); `FfmpegEncoderTests` + layout persistence
|
||||
updated for pipe args / Music roundtrip. Ring-buffer overwrite bug found by the unit test and fixed
|
||||
(head must NOT advance on eviction — the write itself advances it). Integration test pre-fills the
|
||||
ring buffers before `StartLive` so the first pipe tick already carries audio (deterministic — a
|
||||
start-of-stream silence race was seen and eliminated).
|
||||
|
||||
**Out of scope this branch:** IP webcam, chat box, alt-key crop, credits, bg removal, music-off-VOD
|
||||
track (YouTube mutes VODs with copyrighted music — future feature), itch.io `PremiumUrl`.
|
||||
|
||||
- **Landmines:**
|
||||
- Never add another test that constructs `new App()` — use `RealAppHost.Run(...)` (shared STA host
|
||||
@@ -74,7 +92,8 @@
|
||||
the pump stops), not a crash.
|
||||
- `StopAsync` must stop the encoder (closes stdin) **before** awaiting the pump
|
||||
loop — closing stdin unblocks a write stuck on pipe backpressure; the reverse
|
||||
order deadlocks.
|
||||
order deadlocks. Same rule for audio: `StopLive()` (pipe EOF) before the pump
|
||||
stop, so ffmpeg's two inputs end in order.
|
||||
- Tests never instantiate `MainViewModel` directly except via a real `MainWindow`
|
||||
on the `RealAppHost` STA thread (round-clip + naming), which never go live.
|
||||
- Sandbox can't reach outbound HTTPS — `HttpSocialValidator` stub-handler tests
|
||||
@@ -87,6 +106,31 @@
|
||||
|
||||
- **Secret/DB/port facts live:** OAuth client id/secret in `Helpers/OAuthCredentials.cs`;
|
||||
OAuth session token in `Helpers/TokenStore.cs` (DPAPI → `%APPDATA%\ytLlive\ytLlive.auth`);
|
||||
layout DB `%APPDATA%\ytLlive\ytLlive.db` (schema v8; `SocialEntry.Software`
|
||||
column is a column-presence migration like the others, no version bump); OAuth callback
|
||||
layout DB `%APPDATA%\ytLlive\ytLlive.db` (**schema v9** — single-row `Music`; the
|
||||
`SocialEntry.Software` column is a column-presence migration like the others); OAuth callback
|
||||
`http://localhost:8765/oauth2/callback`; crash log `%APPDATA%\ytLlive\startup.log`.
|
||||
|
||||
## Server recovery (llamachile.tube / YunoHost / DO) — 2026-08-14
|
||||
|
||||
**Facts (hunted this session — do not re-hunt):**
|
||||
- Droplet **llamachile.tube**: DO droplet ID `473190301`, IP `143.244.176.131`, sfo3, 4GB/2vCPU/50GB.
|
||||
**SSH port = 2214** (matches git remote `ssh://llgit.llamachile.tube:2214/gramps/ytLlive.git`);
|
||||
22/2222/2200/etc all closed. SSH as `gramps` works with `~/.ssh/id_ed25519` (key auth, no password).
|
||||
- **Root is YunoHost-locked**: `PermitRootLogin no` in `/etc/ssh/sshd_config` (there's a conflicting
|
||||
cloud-init override section below it, but DO's "Reset root password" email does NOT work on this
|
||||
box — cloud-init `set-passwords` only ran once at install). Root is reachable via `sudo -i` or the
|
||||
DO web Recovery Console (Settings → Recovery console — VNC-based; the droplet page's Console button
|
||||
is SSH-based and fails with "all configured authentication methods failed" when no account has a
|
||||
working password).
|
||||
- **gramps is a YunoHost LDAP account** (local `/etc/passwd` entry has `*`, auth via pam_ldap) — its
|
||||
password is changed with `sudo yunohost user update gramps -p '<pw>'`, NOT `passwd`.
|
||||
- **fail2ban bans the whole IP for 10-12 min** after repeated failed SSH/root logins (all ports
|
||||
refuse; `ping` still works; droplet API still shows `active`). Wait it out — do NOT power-cycle.
|
||||
- Password reset this session: `sudo yunohost tools rootpw -n '<pw>'` sets root (rootpw takes `-n`).
|
||||
Verification: `getent shadow root` shows a `$y$` yescrypt hash + `passwd -S root` = `P`.
|
||||
- Mastodon services run as systemd units `mastodon-web/sidekiq/streaming` (all were `active` after
|
||||
the reboot); gitea/nginx/mariadb/postgres/redis/yunohost-api also systemd units. Disk was **93% full**
|
||||
(3.6G free) — keep an eye on it before any big upgrade.
|
||||
- The original outage was an interrupted Mastodon upgrade + a DO `password_reset` reboot; everything
|
||||
came back on its own after boot. No code/schema damage observed.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user