B5: default model auto-pull on first start (v0.19.1)

model_pull.py: ensure_model() checks llama-server availability at
startup, falls back to Ollama pull API if model not found.
Integrated into app.py lifespan after assess_hardware().
11 tests cover all paths: available/unreachable/pull success/fail.

AGENTS.md

@@ -40,7 +40,7 @@ Every router has a dedicated test file:
 | `test_error_envelopes.py` | Global exception handler + stream error incidents |
 | `test_upload.py` | `routers/upload.py` — upload, delete, link, by-conversation, attachment_count integration |
 
-Modules that call `httpx.AsyncClient` (chat, completions, models, search_route, upload, ingest)
+Modules that call `httpx.AsyncClient` (chat, completions, models, search_route, upload, ingest, model_pull)
 are mocked via `monkeypatch.setattr` on `AsyncClient.stream`, `.get`, or `.post`.
 CPU stats in `models.py` (`api/stats`) use real `psutil`; GPU stats are
 monkeypatched via `routers.models.get_gpu_stats`.
@@ -61,6 +61,7 @@ Refactored from single-file (`app.py`) into modules under project root:
 | `rag.py` | Qdrant vector search + system prompt assembly + chunk_text() helper |
 | `eviction.py` | Score-based RAG eviction engine |
 | `gpu.py` | GPU stats — `rocm-smi` (AMD/Linux) or `system_profiler` (Apple Silicon/macOS) |
+| `model_pull.py` | Startup model availability check + Ollama pull API |
 | `triage.py` | Phi-4-mini-based query classification + cluster node selection |
 | `cluster.py` | Cluster node registry, event log, coordinator election, ping/pong, model swap handlers |
 | `amqp.py` | AMQP connection manager — connect, disconnect, publish, subscribe, auto-reconnect |
@@ -113,7 +114,7 @@ The upstream request includes `"logprobs": true`. `parse_llama_stream_chunk()` e
 | wttr.in | No | weather shortcut |
 | rocm-smi | No | AMD GPU stats |
 | Qdrant | No | 6333 (coordinator) — RAG vector search |
-| Ollama (worker) | No | 11434 — embeddings only |
+| Ollama (worker) | No | 11434 — embeddings + model pull |
 
 ### Config quirks
 
@@ -136,6 +137,7 @@ All streaming endpoints yield `data: {json}\n\n`. Key shapes:
 
 ### Completed this session
 - **B7 (v0.19.0)** — Apple Silicon worker support. `gpu.py` now detects `sys.platform == "darwin"` and parses `system_profiler SPDisplaysDataType` for GPU model/VRAM instead of `rocm-smi`. `hardware.py` has darwin branch via `_get_vram_darwin()`. `node_agent/agent.py` reports VRAM on macOS via `system_profiler`. 5 new tests cover linux/darwin gpu paths, 3 new hardware tests cover darwin assessment + VRAM parsing.
+- **B5 (v0.19.1)** — Default model auto-pull on first start. `model_pull.py` with `ensure_model()` checks llama-server availability, falls back to Ollama pull API. Integrated into `app.py` lifespan. 11 tests cover all paths.
 - **Scroll-position fighting** — `scrollToTop()` now respects `_userScrolledAway` flag (100px threshold), skips auto-scroll when user is reading older content. `resetScrollLock()` called on new messages.
 - **401 error cascade** — `SESSION_TIMEOUT_SECONDS` bumped 90→3600 (1 hour). All 10 unprotected `authFetch` calls wrapped in try/catch.
 - **Token counter** — removed localStorage persistence; resets to 0 on page refresh.
@@ -155,14 +157,13 @@ All streaming endpoints yield `data: {json}\n\n`. Key shapes:
 
 ### Upcoming (backlog)
 - B4 — RAG Corpus Management UI
-- B5 — default model auto-pull on first start
 - B6 — waterfall direction toggle
 - B8 — **Encryption & PHI readiness** — spec out encryption at rest (SQLCipher for caic.db, Qdrant payload encryption) and in-transit (TLS for inference, AMQP, RAG). Per-user auth, audit logging, log sanitizer, data lifecycle. Document the "personal LAN HIPAA gap."
   - **Preferred approach: Private Chat mode** — a toggle that skips DB persistence, memory/RAG injection, content logging, and **external SearXNG searching** entirely. Zero stored data, zero external queries = zero data to protect. Simpler, more robust, less code to audit than full encryption. Design this as the primary PHI path before reaching for crypto.
   - **In-transit still needs TLS** — Private Chat eliminates at-rest risk, but AMQP (RabbitMQ) and inference (llama-server) traffic between coordinator and workers is still plaintext on the wire. TLS termination on each node is the lightweight fix (self-signed CA, nginx sidecar or RabbitMQ TLS).
 
 ### Key config values (current)
-- `VERSION = "v0.19.0"` in `config.py`
+- `VERSION = "v0.19.1"` in `config.py`
 - `SESSION_TIMEOUT_SECONDS = 3600`
 - `DEFAULT_MODEL = "qwen2.5-7b-instruct"`
 - `LLAMA_SERVER_BASE = "http://192.168.50.108:8081"`

README.md

@@ -1,6 +1,6 @@
 ![cAIc banner](static/readme-banner.png)
 
-# cAIc v0.18.0
+# cAIc v0.19.1
 
 Consumer AI hardware is a wasteland of incompatibility. NVIDIA speaks CUDA, AMD speaks ROCm. Your RTX 5070 Ti lives in one machine with 16 GB VRAM; your RX 6600 XT lives in another with 12 GB. Alone, neither can run a 14B model at usable speed. Together, they could — if the software stack didn't treat heterogeneous hardware as a bug instead of a feature.
 
@@ -42,13 +42,19 @@ At v1.0, this ships with a Docker compose stack and setup wizard that detect CPU
 
 Developer wiki: [Home](https://llgit.llamachile.tube/gramps/cAIc/wiki/Home) — includes [FAQ](https://llgit.llamachile.tube/gramps/cAIc/wiki/FAQ), [Installation Guide](https://llgit.llamachile.tube/gramps/cAIc/wiki/Installation), and [full architecture docs](https://llgit.llamachile.tube/gramps/cAIc/wiki/Developer-Architecture)
 
+## What's New in v0.19.1
+
+### Default Model Auto-Pull on First Start (B5)
+- **`model_pull.py`** — new module that checks if `default_model` is available on llama-server at startup, falls back to Ollama pull API if not found
+- **Startup integration** — `app.py` lifespan calls `ensure_model()` after `assess_hardware()`, pulling the missing model via Ollama's streaming pull API
+- **Idempotent** — skips pull if model already available on llama-server or Ollama
+
 ## What's New in v0.18.0
 
 ### Wiki — Installation Guide, Screenshots Gallery, Full Documentation
 - **New Installation & Configuration page** — bare-metal walkthrough, cluster setup, config reference, security checklist, 12 troubleshooting topics. Everything a new user needs to get cAIc running.
 - **Screenshots gallery** — clickable image gallery on the wiki Screenshots page
 - **Wiki fully populated** — 5 pages linked from Home, renders at root URL
-- **B5 added to backlog** — auto-download of default GGUF model on first start
 
 ### UX Polish — Waterfall Layout, Barcode Stripes, Confidence Badges
 - **Waterfall display** — newest messages at top via `prepend()`, scroll to top

app.py

@@ -16,8 +16,8 @@ from fastapi.templating import Jinja2Templates
 
 from amqp import connect as amqp_connect, disconnect as amqp_disconnect
 from cluster import start_cluster_subscriptions
-from config import VERSION, RATE_WINDOW_SECONDS, UPLOAD_DIR, RAG_MAX_VECTORS, RAG_EVICTION_HIGH_WATER, RAG_EVICTION_LOW_WATER, RAG_EVICTION_BATCH
+from config import VERSION, DEFAULT_MODEL, RATE_WINDOW_SECONDS, UPLOAD_DIR, RAG_MAX_VECTORS, RAG_EVICTION_HIGH_WATER, RAG_EVICTION_LOW_WATER, RAG_EVICTION_BATCH
-from db import init_db
+from db import init_db, get_db, get_setting
 from hardware import assess_hardware
 from memory import get_memory_count
 from security import (
@@ -60,6 +60,11 @@ async def lifespan(app: FastAPI):
     init_db()
     log.info(f"Memory system: {get_memory_count()} memories loaded")
     await assess_hardware()
+    from model_pull import ensure_model
+    _db = get_db()
+    _user_model = get_setting(_db, "default_model", DEFAULT_MODEL)
+    _db.close()
+    await ensure_model(_user_model)
     await amqp_connect()
     await start_cluster_subscriptions()
 

config.py

@@ -9,7 +9,7 @@ import logging
 
 log = logging.getLogger("caic")
 
-VERSION = "v0.19.0"
+VERSION = "v0.19.1"
 OLLAMA_BASE = os.environ.get("OLLAMA_BASE", "http://localhost:11434")
 LLAMA_SERVER_BASE = os.environ.get("LLAMA_SERVER_BASE", "http://192.168.50.108:8081")
 SEARXNG_BASE = "http://localhost:8888"

model_pull.py

@@ -0,0 +1,80 @@
+"""
+cAIc — Model pull/download helper.
+
+Uses Ollama's pull API to download models that aren't available on the
+inference server. Runs synchronously during startup.
+"""
+import asyncio
+import json
+import logging
+
+import httpx
+
+from config import DEFAULT_MODEL, LLAMA_SERVER_BASE, OLLAMA_BASE
+
+log = logging.getLogger("caic")
+
+
+async def _model_available_on_llama(model: str) -> bool:
+    try:
+        async with httpx.AsyncClient(timeout=5) as client:
+            resp = await client.get(f"{LLAMA_SERVER_BASE}/v1/models")
+            if resp.status_code == 200:
+                models = resp.json().get("data", [])
+                return any(m.get("id") == model for m in models)
+    except Exception:
+        pass
+    return False
+
+
+async def _model_available_on_ollama(model: str) -> bool:
+    try:
+        async with httpx.AsyncClient(timeout=5) as client:
+            resp = await client.post(f"{OLLAMA_BASE}/api/show", json={"name": model})
+            return resp.status_code == 200
+    except Exception:
+        pass
+    return False
+
+
+async def _pull_via_ollama(model: str) -> bool:
+    try:
+        async with httpx.AsyncClient(timeout=300) as client:
+            async with client.stream("POST", f"{OLLAMA_BASE}/api/pull", json={"name": model}) as resp:
+                if resp.status_code != 200:
+                    log.warning("ollama pull returned %s for %s", resp.status_code, model)
+                    return False
+                async for line in resp.aiter_lines():
+                    if line.strip():
+                        try:
+                            data = json.loads(line)
+                            status = data.get("status", "")
+                            if status:
+                                log.info("ollama pull %s: %s", model, status)
+                        except json.JSONDecodeError:
+                            pass
+                return True
+    except httpx.ConnectError:
+        log.warning("ollama not reachable at %s — cannot pull %s", OLLAMA_BASE, model)
+    except Exception as e:
+        log.warning("ollama pull error for %s: %s", model, e)
+    return False
+
+
+async def ensure_model(model: str = "") -> bool:
+    """Ensure *model* is available for inference. Pull via Ollama if needed."""
+    model = model or DEFAULT_MODEL
+    if await _model_available_on_llama(model):
+        log.info("model %s already available on llama-server", model)
+        return True
+    log.info("model %s not found on llama-server, checking Ollama", model)
+    if await _model_available_on_ollama(model):
+        log.info("model %s found on Ollama (available for embeddings)", model)
+        return True
+    log.info("model %s not found on Ollama either — pulling", model)
+    ok = await _pull_via_ollama(model)
+    if ok:
+        log.info("model %s pulled successfully", model)
+    else:
+        log.warning("model %s could not be pulled", model)
+    return ok

tests/test_model_pull.py

@@ -0,0 +1,142 @@
+import asyncio
+
+import httpx
+
+from model_pull import ensure_model, _model_available_on_llama, _model_available_on_ollama, _pull_via_ollama
+
+
+class _MockAsyncResponse:
+    def __init__(self, status_code=200, json_data=None):
+        self.status_code = status_code
+        self._json_data = json_data or {}
+
+    def json(self):
+        return self._json_data
+
+
+class _MockStreamResponse:
+    def __init__(self, status_code=200, lines=None):
+        self.status_code = status_code
+        self._lines = lines or []
+
+    async def __aenter__(self):
+        return self
+
+    async def __aexit__(self, *args):
+        pass
+
+    def aiter_lines(self):
+        class _AIter:
+            def __init__(self, lines):
+                self._lines = iter(lines)
+            def __aiter__(self):
+                return self
+            async def __anext__(self):
+                try:
+                    return next(self._lines)
+                except StopIteration:
+                    raise StopAsyncIteration
+        return _AIter(self._lines)
+
+
+def _mock_models_on_llama(*models):
+    async def _get(*args, **kwargs):
+        return _MockAsyncResponse(json_data={
+            "data": [{"id": m} for m in models]
+        })
+    return _get
+
+
+async def _mock_ollama_available(*args, **kwargs):
+    return _MockAsyncResponse(status_code=200, json_data={"name": "qwen2.5:latest"})
+
+
+async def _mock_ollama_unavailable(*args, **kwargs):
+    raise httpx.ConnectError("refused")
+
+
+def _mock_ollama_pull_ok(*args, **kwargs):
+    return _MockStreamResponse(200, [
+        '{"status": "pulling manifest"}',
+        '{"status": "success"}',
+    ])
+
+
+def _mock_ollama_pull_fail(*args, **kwargs):
+    return _MockStreamResponse(500, [])
+
+
+def _mock_ollama_connect_error(*args, **kwargs):
+    raise httpx.ConnectError("refused")
+
+
+def test_available_on_llama(monkeypatch):
+    monkeypatch.setattr(httpx.AsyncClient, "get", _mock_models_on_llama("qwen2.5-7b-instruct"))
+    assert asyncio.run(_model_available_on_llama("qwen2.5-7b-instruct")) is True
+    assert asyncio.run(_model_available_on_llama("nonexistent-model")) is False
+
+
+def test_available_on_llama_unreachable(monkeypatch):
+    async def _connect_error(*args, **kwargs):
+        raise httpx.ConnectError("refused")
+    monkeypatch.setattr(httpx.AsyncClient, "get", _connect_error)
+    assert asyncio.run(_model_available_on_llama("qwen2.5-7b-instruct")) is False
+
+
+def test_available_on_ollama(monkeypatch):
+    monkeypatch.setattr(httpx.AsyncClient, "post", _mock_ollama_available)
+    assert asyncio.run(_model_available_on_ollama("qwen2.5:latest")) is True
+
+
+def test_available_on_ollama_unreachable(monkeypatch):
+    monkeypatch.setattr(httpx.AsyncClient, "post", _mock_ollama_unavailable)
+    assert asyncio.run(_model_available_on_ollama("qwen2.5:latest")) is False
+
+
+def test_pull_via_ollama_success(monkeypatch):
+    monkeypatch.setattr(httpx.AsyncClient, "stream", _mock_ollama_pull_ok)
+    assert asyncio.run(_pull_via_ollama("qwen2.5:latest")) is True
+
+
+def test_pull_via_ollama_fail(monkeypatch):
+    monkeypatch.setattr(httpx.AsyncClient, "stream", _mock_ollama_pull_fail)
+    assert asyncio.run(_pull_via_ollama("qwen2.5:latest")) is False
+
+
+def test_pull_via_ollama_connect_error(monkeypatch):
+    monkeypatch.setattr(httpx.AsyncClient, "stream", _mock_ollama_connect_error)
+    assert asyncio.run(_pull_via_ollama("qwen2.5:latest")) is False
+
+
+def test_ensure_model_already_on_llama(monkeypatch):
+    monkeypatch.setattr(httpx.AsyncClient, "get", _mock_models_on_llama("qwen2.5-7b-instruct"))
+    assert asyncio.run(ensure_model("qwen2.5-7b-instruct")) is True
+
+
+def test_ensure_model_not_on_llama_but_on_ollama(monkeypatch):
+    async def _get(*args, **kwargs):
+        return _MockAsyncResponse(json_data={"data": []})
+
+    monkeypatch.setattr(httpx.AsyncClient, "get", _get)
+    monkeypatch.setattr(httpx.AsyncClient, "post", _mock_ollama_available)
+    assert asyncio.run(ensure_model("qwen2.5-7b-instruct")) is True
+
+
+def test_ensure_model_needs_pull(monkeypatch):
+    async def _get(*args, **kwargs):
+        return _MockAsyncResponse(json_data={"data": []})
+
+    monkeypatch.setattr(httpx.AsyncClient, "get", _get)
+    monkeypatch.setattr(httpx.AsyncClient, "post", _mock_ollama_unavailable)
+    monkeypatch.setattr(httpx.AsyncClient, "stream", _mock_ollama_pull_ok)
+    assert asyncio.run(ensure_model("qwen2.5-7b-instruct")) is True
+
+
+def test_ensure_model_pull_fails(monkeypatch):
+    async def _get(*args, **kwargs):
+        return _MockAsyncResponse(json_data={"data": []})
+
+    monkeypatch.setattr(httpx.AsyncClient, "get", _get)
+    monkeypatch.setattr(httpx.AsyncClient, "post", _mock_ollama_unavailable)
+    monkeypatch.setattr(httpx.AsyncClient, "stream", _mock_ollama_pull_fail)
+    assert asyncio.run(ensure_model("qwen2.5-7b-instruct")) is False