gramps/cAIc
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

expand B7 with implementation plan for Apple Silicon

Llama Chile Shop
2026-07-13 11:12:22 -07:00
parent 25d3beaa32
commit bedad47fd1
1 changed files with 6 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Current-WiP.md
+6 -1
@@ -18,7 +18,12 @@ All 15 tasks are done. Final commit: `f0689ac feat: Roadmap N — AMQP cluster n
- HTTPS / reverse proxy (Caddy) - HTTPS / reverse proxy (Caddy)
- Conversation search/filter and export tooling - Conversation search/filter and export tooling
- B6 — Waterfall direction toggle — add a setting to flip message order between top-down (newest at top, waterfall) and bottom-up (newest at bottom, standard chat). Needs `localStorage` persistence and a UI toggle (maybe in settings modal or topbar). - B6 — Waterfall direction toggle — add a setting to flip message order between top-down (newest at top, waterfall) and bottom-up (newest at bottom, standard chat). Needs `localStorage` persistence and a UI toggle (maybe in settings modal or topbar).
- B7 — Apple Silicon worker support — document/validate running the node agent + llama-server on macOS with M-series GPUs (Metal backend). May need changes to hardware probing, GPU stats, and node registration. - **B7 — Apple Silicon worker support** — no Apple hardware needed for implementation; llama.cpp Metal backend exposes the same OpenAI-compat API. Changes:
- `gpu.py` — add `get_gpu_stats_macos()` path using `system_profiler SPDisplaysDataType` for VRAM, `sysctl hw.memsize` for unified memory
- `hardware.py` — `platform.system() == "Darwin"` branch skips `rocm-smi`, calls macOS VRAM probe instead
- `node_agent/agent.py` — detect macOS, register with `platform: darwin, backend: metal`
- `routers/models.py` — `get_gpu_stats()` dispatch to Metal stats on macOS
- Tests — add test fixture mocking a darwin worker registration + ping/pong; coordinator treats it identically to CUDA/ROCm workers
- Keyboard shortcuts, retry button, source-link polish - Keyboard shortcuts, retry button, source-link polish
## Maintenance Rules ## Maintenance Rules