fix: auto-derive model shorthand from any model name following qwen2.5:7B:i convention

- delete memory/preset/conversation/all/gallery/clear
- save/reset profile
- new chat, add preset, edit preset
- clear file selection
- rate up/down thumbs toasts

.gitignore

@@ -5,3 +5,7 @@ __pycache__/
 venv/
 readme.md-
 *.bak
+hardware_state.json
+.env
+secrets/
+searxng/

=9.0.0

@@ -0,0 +1 @@
+/bin/bash: line 1: ./venv/bin/pip: No such file or directory

AGENTS.md

@@ -0,0 +1,168 @@
+# cAIc — Agents Guide
+
+## Run
+
+```bash
+uvicorn app:app --host 0.0.0.0 --port 8080 --reload
+```
+
+## Tests
+
+```bash
+python3 -m pytest tests/ -v
+```
+
+All tests use `tmp_path` fixtures + monkeypatched `httpx.AsyncClient.stream/get/post/put`. No external services needed. Test factories reset `SESSIONS`, `PIN_ATTEMPTS`, `RATE_EVENTS` globals — be careful not to let test state leak. Tests import directly from the correct modules (`db`, `security`, `config`, `search`, `rag`, `memory`, `routers.*`).
+
+Every router has a dedicated test file:
+| File | Covers |
+|------|--------|
+| `test_auth_capabilities.py` | `auth.py` — guest/admin sessions, origin blocking, logout |
+| `test_chat_streaming_and_memory_paths.py` | `routers/chat.py` — streaming, auto-search, remember/forget, upload context injection |
+| `test_completions.py` | `routers/completions.py` — API key auth, FIM, streaming, blocking, errors |
+| `test_conversations.py` | `routers/conversations.py` — full CRUD, guest admin enforcement, attachment_count |
+| `test_ingest.py` | `routers/ingest.py` — Bearer auth, chunk/embed/upsert, validation |
+| `test_memories.py` | `routers/memories.py` — edit, search, stats endpoints |
+| `test_models_router.py` | `routers/models.py` — models list, ps, show, stats, search/status |
+| `test_presets.py` | `routers/presets.py` — full CRUD, default preset protection |
+| `test_profile.py` | `routers/profile.py` — get, update, default, length validation |
+| `test_search_route.py` | `routers/search_route.py` — explicit search flow, no results, errors |
+| `test_search_url_sanitization.py` | `search.py` URL sanitizer |
+| `test_cluster.py` | `cluster.py` — registration, deregistration, pong, events, coordinator query |
+| `test_cluster_heartbeat.py` | `cluster.py` — heartbeat handler, known/unknown node |
+| `test_model_swap.py` | `cluster.py` + `triage.py` — request_model_swap, handle_model_ready/failed, select_node swap triggering |
+| `test_node_agent.py` | `node_agent/agent.py` — registration, ping/pong, model swap |
+| `test_triage.py` | `triage.py` — classify_query, select_node, get_inference_url |
+| `test_settings_allowlist.py` | `routers/settings.py` — allowlisted key enforcement |
+| `test_skills_framework.py` | `routers/skills.py` — list, toggle, unknown skill, prompt injection |
+| `test_ip_allowlist.py` | IP allowlist helper + middleware |
+| `test_rate_and_payload_guardrails.py` | Rate limits + payload size enforcement |
+| `test_error_envelopes.py` | Global exception handler + stream error incidents |
+| `test_upload.py` | `routers/upload.py` — upload, delete, link, by-conversation, attachment_count integration |
+
+Modules that call `httpx.AsyncClient` (chat, completions, models, search_route, upload, ingest)
+are mocked via `monkeypatch.setattr` on `AsyncClient.stream`, `.get`, or `.post`.
+CPU stats in `models.py` (`api/stats`) use real `psutil`; GPU stats are
+monkeypatched via `routers.models.get_gpu_stats`.
+
+## Architecture
+
+Refactored from single-file (`app.py`) into modules under project root:
+
+| File | Role |
+|------|------|
+| `app.py` | FastAPI app, middleware, router registration |
+| `config.py` | Constants, env vars, rate/payload limits, built-in skills registry, upload limits |
+| `db.py` | SQLite schema, connection factory, settings helpers, upload_context CRUD |
+| `auth.py` | PIN-based guest/admin sessions, auth routes |
+| `security.py` | Rate limiting, origin checks, IP allowlist, audit/incident logging |
+| `memory.py` | FTS5 memory CRUD, remember/forget command parsing |
+| `search.py` | SearXNG integration, perplexity scoring, refusal detection |
+| `rag.py` | Qdrant vector search + system prompt assembly + chunk_text() helper |
+| `eviction.py` | Score-based RAG eviction engine |
+| `gpu.py` | AMD GPU stats via `rocm-smi` |
+| `triage.py` | Phi-4-mini-based query classification + cluster node selection |
+| `cluster.py` | Cluster node registry, event log, coordinator election, ping/pong, model swap handlers |
+| `amqp.py` | AMQP connection manager — connect, disconnect, publish, subscribe, auto-reconnect |
+| `node_agent/` | Standalone worker agent — AMQP client for registration, ping/pong, model swap |
+| `routers/` | One module per endpoint group (chat, search, skills, completions, upload, ingest) |
+
+### Entrypoint / API keys
+
+- `app.py` line 148: `uvicorn.run(app, ...)` when called directly
+- `config.py` line 14: `LLAMA_SERVER_BASE` defaults to `http://192.168.50.108:8081` — llama-server on coordinator, RPC-offloads GPU layers to worker :50052
+- `config.py` line 17: `COMPLETIONS_API_KEY` read from `CAIC_COMPLETIONS_API_KEY` env var or auto-generates
+- `config.py` line 13: `OLLAMA_BASE` is legacy/unused — all endpoints use `LLAMA_SERVER_BASE`
+
+### Key flows
+
+1. **`/api/chat`** → `process_remember_command()` intercepts "remember that..." / "forget about..." first → optional `upload_context_id` fetches document text from SQLite → `build_system_prompt()` (profile + FTS5 memory + Qdrant RAG + preset + skills + uploaded doc) → triage classifies query (general/code/search/rag) → `select_node()` picks best worker → stream from chosen node with `logprobs: true` → if perplexity > 15.0 OR `REFUSAL_PATTERNS` match, re-query with SearXNG results
+2. **`/api/search`** → bypasses perplexity/refusal, queries SearXNG directly → summarizes via llama-server
+3. **`/v1/chat/completions`** → OpenAI-compatible for Continue.dev/IDE integration; FIM requests proxied without persistence
+4. **`/api/upload`** → multipart file upload, PDF/text extraction, `mode=(context|ingest|both)`, stores SQLite context (1hr expiry) + Qdrant upsert
+5. **`/api/ingest`** → Bearer token auth, programmatic RAG ingest (terminal hook, external tools)
+
+### Perplexity / auto-search
+
+The upstream request includes `"logprobs": true`. `parse_llama_stream_chunk()` extracts per-token logprobs from each chunk's `choices[0].logprobs.content[].logprob`. The `all_logprobs` list is populated during streaming, so `calculate_perplexity()` and `is_uncertain()` work correctly.
+
+### Auth / lockdown
+
+- Guest session by default (`POST /api/auth/guest`), admin unlock via 4-digit PIN (`POST /api/auth/login`)
+- Admin required for PUT/DELETE/PATCH + all POST except allowlist (`/api/chat`, `/api/search`, `/api/auth/*`)
+- `/api/ingest` is exempt from session auth — self-authenticates via Bearer token
+- IP allowlist, rate limiting, origin checking, payload size limits — all enforced in `app.py` middleware
+- Origin check applies to **all** `/api/` requests; returns `False` when both `Origin` and `Referer` are absent
+- `CAIC_ADMIN_PIN` env var required on first boot (or `CAIC_ALLOW_DEFAULT_PIN=true`)
+
+### Database
+
+- SQLite at `caic.db`, auto-created by `init_db()` on startup via FastAPI `lifespan`
+- `get_db()` opens new connection per request (no pool). Close after use.
+- FTS5 virtual table `memories` for full-text search with BM25 ranking.
+- `upload_context` table: auto-expiring document storage for chat context injection.
+
+### External services
+
+| Service | Required | Port |
+|---------|----------|------|
+| llama-server (coordinator) | Yes | 8081 + RPC :50052 (worker GPU) |
+| Phi-4-mini (triage) | No | 8083 |
+| SearXNG | No | 8888 |
+| RabbitMQ (coordinator) | No | 5672 — AMQP broker |
+| wttr.in | No | weather shortcut |
+| rocm-smi | No | AMD GPU stats |
+| Qdrant | No | 6333 (coordinator) — RAG vector search |
+| Ollama (worker) | No | 11434 — embeddings only |
+
+### Config quirks
+
+- `BODY_LIMIT_UPLOAD_BYTES` = 20MB for `/api/upload`; other paths use smaller limits
+- `SUPPORTED_UPLOAD_TYPES` includes images (png/jpeg/gif/svg/webp) + text + PDF + JSON
+- `UPLOAD_CONTEXT_EXPIRY_HOURS` = 1 hour
+- Rate limits and payload caps in `config.py` — patch `security.RL_*` not `config.RL_*` for tests
+- RAG embedding requests go to `EMBED_URL` at `/api/embeddings` (Ollama on worker :11434)
+
+### SSE Protocol
+
+All streaming endpoints yield `data: {json}\n\n`. Key shapes:
+- `{token, conversation_id}` — streaming token
+- `{searching: true}` — web search triggered
+- `{search_results: N}` — N results (no raw_results payload)
+- `{done: true, perplexity, tokens_per_sec, searched?}` — terminal
+- `{error: "...", error_key: "..."}` — error with incident key
+
+## Work State
+
+### Completed this session
+- **Scroll-position fighting** — `scrollToTop()` now respects `_userScrolledAway` flag (100px threshold), skips auto-scroll when user is reading older content. `resetScrollLock()` called on new messages.
+- **401 error cascade** — `SESSION_TIMEOUT_SECONDS` bumped 90→3600 (1 hour). All 10 unprotected `authFetch` calls wrapped in try/catch.
+- **Token counter** — removed localStorage persistence; resets to 0 on page refresh.
+- **Ctrl+Enter for web search** — Shift+Enter now inserts newline (universal convention), Ctrl+Enter triggers search.
+- **Search button styling** — WEB button matches SEND: same font/size/weight, orange bg with dark navy text (`var(--bg-tertiary)`). Input placeholder updated.
+- **Toast notifications** — `showToast()` helper, every action icon now fires a slide-out notification (copy, save, delete, rate, etc.). Print is the exception (dialog handles it).
+- **Clipboard reliability** — `execCopy()` helper for HTTP fallback (navigator.clipboard fails on plain HTTP). All three copy paths (user inline, toolbar, code blocks) now work on jarvis:8080.
+- **User copy icon** — single 📋 inline at end of user query text (not a full toolbar). Uses `data-content` attr to avoid HTML injection in onclick.
+- **Rating thumbs removed** — 👍👎 cut (no backend, no persistence, privacy concern).
+- **Keybinding fix** — Shift+Enter = newline, Ctrl+Enter = search (universal conventions).
+
+### Active
+- (none)
+
+### Blocked
+- (none)
+
+### Upcoming (backlog)
+- B4 — RAG Corpus Management UI
+- B5 — default model auto-pull on first start
+- B6 — waterfall direction toggle
+- B7 — Apple Silicon worker support (gpu.py Metal, hardware.py Darwin)
+- B8 — **Encryption & PHI readiness** — spec out encryption at rest (SQLCipher for caic.db, Qdrant payload encryption) and in-transit (TLS for inference, AMQP, RAG). Per-user auth, audit logging, log sanitizer, data lifecycle. Document the "personal LAN HIPAA gap."
+  - **Preferred approach: Private Chat mode** — a toggle that skips DB persistence, memory/RAG injection, and content logging entirely. Zero stored data = zero data to protect. Simpler, more robust, less code to audit than full encryption. Design this as the primary PHI path before reaching for crypto.
+  - **In-transit still needs TLS** — Private Chat eliminates at-rest risk, but AMQP (RabbitMQ) and inference (llama-server) traffic between coordinator and workers is still plaintext on the wire. TLS termination on each node is the lightweight fix (self-signed CA, nginx sidecar or RabbitMQ TLS).
+
+### Key config values (current)
+- `VERSION = "v0.18.0"` in `config.py`
+- `SESSION_TIMEOUT_SECONDS = 3600`
+- `DEFAULT_MODEL = "qwen2.5-7b-instruct"`
+- `LLAMA_SERVER_BASE = "http://192.168.50.108:8081"`

CLAUDE.md

@@ -9,7 +9,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
 ./venv/bin/uvicorn app:app --host 0.0.0.0 --port 8080 --reload
 
 # Production (via systemd)
-sudo systemctl restart jarvischat
+sudo systemctl restart caic
 
 # Direct run
 ./venv/bin/python app.py
@@ -19,46 +19,66 @@ sudo systemctl restart jarvischat
 
 ```bash
 ./venv/bin/pip install -r requirements.txt
-# Also requires: psutil jinja2 python-multipart (not in requirements.txt)
+# Also requires: psutil jinja2 python-multipart pypdf (not in requirements.txt)
 ```
 
 ## Architecture
 
-Single-file FastAPI backend (`app.py`) + single-template frontend (`templates/index.html`). No build step. SQLite database auto-created at `jarvischat.db` on first run.
+Modular FastAPI app — `app.py` wires routers, middleware, and lifespan. SQLite database auto-created at `caic.db` on first run. No build step, single `templates/index.html`.
 
 ### Request Flow: `/api/chat`
 
 1. User message saved to DB → conversation created if new
-2. `build_system_prompt()` assembles: profile + FTS5 memory search results + preset prompt
+2. `process_remember_command()` intercepts "remember that..." / "forget about..." first
-3. Streamed to Ollama (`/api/chat`, `stream: true`, `logprobs: true`) via SSE
+3. Optional `upload_context_id` → fetches document text from `upload_context` table, injects `[ATTACHED DOCUMENT]` into system prompt
-4. **Auto web search trigger**: if perplexity > 15.0 OR response matches `REFUSAL_PATTERNS`, re-queries Ollama with SearXNG results prepended to system prompt
+4. `build_system_prompt()` assembles: profile + FTS5 memory search + Qdrant RAG + preset + skills + uploaded doc
-5. Final response saved to DB; SSE `done` event sent with perplexity + tokens/sec
+5. Streamed to llama-server (`/v1/chat/completions`, `stream: true`, `logprobs: true`) via SSE
+6. **Auto web search trigger**: if perplexity > 15.0 OR response matches `REFUSAL_PATTERNS`, re-queries with SearXNG results
+7. Final response saved to DB; SSE `done` event sent with perplexity + tokens/sec
 
 ### Request Flow: `/api/search` (explicit search)
 
-Bypasses perplexity/refusal detection entirely — queries SearXNG directly then asks Ollama to summarize with results as system context.
+Bypasses perplexity/refusal — queries SearXNG directly then asks llama-server to summarize results.
+
+### Request Flow: `/api/upload`
+
+Multipart file upload → PDF/text extraction + chunking → optional Qdrant upsert + SQLite context storage (1hr expiry). Supports `mode=(context|ingest|both)`. Images upload as storage only — model cannot process image content.
+
+### Request Flow: `/api/ingest`
+
+Bearer-token-authenticated terminal RAG hook. Accepts raw text, chunks via `chunk_text()`, embeds via Ollama `/api/embeddings`, upserts to Qdrant.
 
 ### Memory System
 
-FTS5 virtual table (`memories`) in SQLite. `search_memories()` uses BM25 ranking. `process_remember_command()` intercepts "remember that..." / "forget about..." before the message reaches Ollama and returns a confirmation string. Topic auto-detection via keyword matching in `detect_topic()`.
+FTS5 virtual table (`memories`) in SQLite. `search_memories()` uses BM25 ranking. `process_remember_command()` intercepts "remember that..." / "forget about..." before the message reaches the model and returns a confirmation string.
 
-### Key Constants (top of `app.py`)
+### Key Constants (`config.py`)
 
-- `OLLAMA_BASE` — `http://localhost:11434`
+- `LLAMA_SERVER_BASE` — `http://192.168.50.108:8081` (coordinator llama-server, RPC offloads to worker GPU)
 - `SEARXNG_BASE` — `http://localhost:8888`
-- `PERPLEXITY_THRESHOLD` — `15.0` (controls auto-search sensitivity)
+- `QDRANT_URL` — `http://192.168.50.108:6333` (Qdrant on coordinator)
-- `DEFAULT_MODEL` — `llama3.1:latest`
+- `TRIAGE_BASE` — `http://127.0.0.1:8083/v1` (Phi-4-mini)
+- `AMQP_URL` — `amqp://caic:{pw}@localhost:5672/caic` (RabbitMQ, pw read from `~/.caic_amqp_secret`)
+- `PERPLEXITY_THRESHOLD` — `15.0`
+- `EMBED_URL` — `http://192.168.50.210:11434/api/embeddings` (Ollama on worker)
+- `VERSION` — current version string
 
 ### External Services
 
-- **Ollama** — required, must be running on port 11434
+| Service | Required | Port |
-- **SearXNG** — optional, port 8888; `GET /api/search/status` probes availability
+|---------|----------|------|
-- **wttr.in** — weather shortcut in `query_searxng()`, bypasses SearXNG for weather queries
+| **llama-server** (coordinator) | Yes | 8081 + RPC :50052 (worker GPU) |
-- **rocm-smi** — AMD GPU stats via subprocess; gracefully degrades if not available
+| **Phi-4-mini** (triage) | No | 8083 |
+| **SearXNG** | No | 8888 |
+| **RabbitMQ** (coordinator) | No | 5672 — AMQP broker |
+| **wttr.in** | No | weather shortcut |
+| **rocm-smi** | No | AMD GPU stats |
+| **Qdrant** (coordinator) | No | 6333 — RAG vector search |
+| **Ollama** (worker) | No | 11434 — embeddings only |
 
 ### Database
 
-`get_db()` opens a new connection per request (no connection pool). `init_db()` runs at startup via the FastAPI `lifespan` handler. The `profile` table uses a singleton row (`id = 1`). Default settings are seeded but never overwritten by `init_db()`.
+`get_db()` opens a new connection per request (no pool). `init_db()` runs at startup via FastAPI `lifespan`. Tables: `conversations`, `messages`, `settings`, `profile` (singleton id=1), `memories` (FTS5), `upload_context`. Default settings seeded but never overwritten.
 
 ### SSE Protocol
 
@@ -67,8 +87,8 @@ All streaming endpoints yield `data: {json}\n\n`. Key event shapes:
 - `{searching: true}` — web search triggered
 - `{search_results: N}` — N results retrieved
 - `{done: true, perplexity, tokens_per_sec, searched?}` — terminal event
-- `{error: "..."}` — error event
+- `{error: "...", error_key: "..."}` — error with incident key
 
 ### Deployment
 
-Runs as systemd service under user `jarvischat`, working directory `/opt/jarvischat`. Logs via syslog (`journalctl -u jarvischat`).
+Runs as systemd service under user `caic`, working directory `/opt/caic`. Logs via syslog (`journalctl -u caic`). Version bumps via git tag + commit, deployed via `git pull && systemctl restart caic`.

README.md

@@ -1,453 +1,394 @@
-![jarvisChat logo](static/jcscreenie.png)
+![cAIc banner](static/readme-banner.png)
-# ⚡ JarvisChat v1.9.0
 
-**A privacy-first, homelab-native developer knowledge platform.**
+# cAIc v0.18.0
 
-> JarvisChat turns a heterogeneous LAN of budget hardware into a distributed local AI inference cluster — accumulating institutional knowledge over time, keeping all data off the cloud, and squeezing real performance out of modest consumer hardware through architecture rather than dollars.
+Consumer AI hardware is a wasteland of incompatibility. NVIDIA speaks CUDA, AMD speaks ROCm. Your RTX 5070 Ti lives in one machine with 16 GB VRAM; your RX 6600 XT lives in another with 12 GB. Alone, neither can run a 14B model at usable speed. Together, they could — if the software stack didn't treat heterogeneous hardware as a bug instead of a feature.
 
-This is not another AI chat wrapper. jC is the UX and knowledge-management layer for a local AI brain — analogous to what Windows was to DOS, or what the web is to the internet. The intelligence lives in the model and the RAG corpus. jC makes it accessible and keeps feeding it.
+The industry consensus — llama.cpp RPC, vLLM, TensorFlow distributed — all assume a homogeneous cluster: same GPU vendor, same VRAM, same driver stack, reachable over a fast fabric. This assumption works for data centers that buy 64 identical H100s at a time. It does not work for the person who has a gaming PC with an NVIDIA card in the living room, an AMD-powered home server in the closet, and an old MacBook on the desk. That person has more aggregate compute than any single consumer machine, but no software stack can make it cooperate.
 
----
+cAIc is a cluster orchestration layer that fuses mismatched GPUs, CPUs, and machines into a single inference surface. The advantage isn't just compatibility — it's matching each request to the hardware best suited for it. The coordinator (CPU-only, no discrete GPU) handles all CPU-bound work: RAG embedding, query triage, web search, memory, conversation storage, the message broker, and the web UI itself. Workers (discrete GPU) do nothing but inference — no database, no browser sessions, no orchestration overhead stealing VRAM. Triage classifies each query and routes to the node running the optimal model; if the right model isn't loaded, the coordinator requests a swap and the worker handles it asynchronously. Every machine contributes what it does best.
 
-## The Four Pillars
+You might also be doing this with retired office PCs and GPUs from the Obama era. That works too. But the core problem cAIc solves isn't budget reuse — it's making non-homogeneous hardware cooperate.
 
-### 1. Privacy
+### Paired Programming
-Everything runs on your LAN. No API keys, no cloud endpoints, no data leaving your network, no subscription, no terms-of-service surprises. Your conversations, your codebase, your decisions — stay yours.
 
-### 2. Knowledge Retention
+Every line of code in this repository was written by an AI (Claude, via opencode). But AI does not architect, design, test, deploy, or decide what to build — that requires experience, judgement, and the discipline to say "no" to feature creep.
-Unlike stateless chat tools that forget everything when you close the tab, jC accumulates institutional memory. Every solved problem, every architectural decision, every working command gets absorbed into the RAG corpus via Qdrant. The system gets smarter the longer you use it.
 
-### 3. Budget Hardware Maximization
+**Gramps** (BS Computer Science, Oklahoma State, coding since 1981) performed that role — designing the architecture, managing the development process, writing and maintaining the test suite, operating the deployment pipeline, and directing every feature decision across dozens of sessions spanning months. Without that human steering, this would be yet another AI-generated repo that compiles but doesn't solve a real problem. With it, cAIc ships as a functional, tested, deployed system that runs 24/7 on real hardware serving real users.
-You don't need a $10,000 workstation. jC is designed for the developer who has a drawer full of machines and the skills to wire them together. RPC clustering, model splitting across CPU and GPU nodes, dynamic resource negotiation, and smart RAG eviction squeeze real performance out of modest consumer hardware.
 
-### 4. Homelab-Native Architecture
+This is paired programming, elevated: the AI handles the mechanical work of code generation; the human brings decades of systems-level experience, architectural judgment, and the maturity to ship something that lasts.
-Built specifically for the heterogeneous homelab: mixed hardware, mixed OS, consumer GPUs, ARM boards, NAS storage — all working together as a coherent AI platform. A designated master node hosts jC, llama-server, and SearXNG. GPU nodes self-register as RPC inference workers. The architecture scales horizontally across whatever you've got.
 
----
+### Architecture: CPU Coordinator + GPU Workers
 
-## Target Audience
+cAIc splits the workload across two machine roles:
 
-Solo developers and homelab enthusiasts who are:
+**Coordinator** (ultron — Ryzen 7 7840HS, no discrete GPU) runs the FastAPI app, RAG vector search (Qdrant), text embedding (Ollama on CPU), query triage (Phi-4-mini), web search (SearXNG), message broker (RabbitMQ), and all SQLite-backed services — memory, profiles, conversations, settings. Every CPU-bound task stays here.
-- Budget-constrained but hardware-rich (multiple machines, NAS, spare GPUs)
-- Privacy-conscious (no cloud AI subscriptions)
-- Technically capable (if you can install jC, you can designate the master node)
-- Building something over time and want their AI to remember it
 
----
+**Workers** (jarvis — RX 6600 XT 12 GB / corsair — RTX 5070 Ti 16 GB) run only llama-server for GPU inference. The coordinator never touches a model; workers never touch the database. Workers register via AMQP, receive ping/pong health checks, and accept model-swap commands when triage determines a different model is needed for the current query.
 
-## Architecture
+This split keeps the UI responsive during inference (the coordinator isn't blocked by GPU compute) and lets workers focus VRAM entirely on model weights rather than browser sessions or API orchestration.
+
+Under the hood: FastAPI + SQLite + Jinja2 on Python 3.13. AMQP-mediated cluster coordination with an OpenAI-compatible inference endpoint.
+
+#### Query-routing vs. layer-splitting — why it matters
+
+Most distributed inference tools (llama.cpp RPC, vLLM with tensor parallelism, exo) split a *single model* across multiple GPUs. The first GPU runs layers 0–15, the second runs 16–31, and so on. This works well in a homogeneous cluster where every GPU is identical, but in a heterogeneous setup the slowest card sets the pace — every forward pass waits for the straggler. Communication overhead between GPUs (NCCL, RPC) adds latency too.
+
+cAIc takes a different approach: **query-routing**. Each worker runs a complete model on its own GPU. When a query comes in, triage classifies it and routes the *whole request* to the worker best suited for it — code questions go to the worker with a coder model, general chat goes to the instruct model. No layer sharing, no lockstep, no straggler problem. The tradeoff is that no single query can use combined VRAM across multiple GPUs, but the throughput and responsiveness of the cluster as a whole isn't dragged down by the weakest link.
+
+This also means a worker with a slow GPU can still contribute meaningfully — it handles less latency-sensitive queries or batch background work, while the fast GPU handles interactive chat.
+
+At v1.0, this ships with a Docker compose stack and setup wizard that detect CPU vs GPU, probe your hardware, and stand up SearXNG, Qdrant, RabbitMQ, and everything else with a single `docker compose up`. The same install docs work bare-metal for those who prefer to skip containers entirely.
+
+Developer wiki: [Home](https://llgit.llamachile.tube/gramps/cAIc/wiki/Home) — includes [FAQ](https://llgit.llamachile.tube/gramps/cAIc/wiki/FAQ), [Installation Guide](https://llgit.llamachile.tube/gramps/cAIc/wiki/Installation), and [full architecture docs](https://llgit.llamachile.tube/gramps/cAIc/wiki/Developer-Architecture)
+
+## What's New in v0.18.0
+
+### Wiki — Installation Guide, Screenshots Gallery, Full Documentation
+- **New Installation & Configuration page** — bare-metal walkthrough, cluster setup, config reference, security checklist, 12 troubleshooting topics. Everything a new user needs to get cAIc running.
+- **Screenshots gallery** — clickable image gallery on the wiki Screenshots page
+- **Wiki fully populated** — 5 pages linked from Home, renders at root URL
+- **B5 added to backlog** — auto-download of default GGUF model on first start
+
+### UX Polish — Waterfall Layout, Barcode Stripes, Confidence Badges
+- **Waterfall display** — newest messages at top via `prepend()`, scroll to top
+- **Barcode alternating pairs** — each Q&A wrapped in `.msg-pair` with alternating tint + left border accent
+- **Confidence % badge** (`1/ppl * 100`) replaces raw perplexity, color-coded green/orange/red
+- **Cumulative token counter (TOK)** in topbar center, persisted in `localStorage`
+- **TOK reformatted** to `# / %` — `#` is all-time tokens, `%` is last response's context-window percentage, color-coded
+- **Dot-matrix sprocket strips** on left/right edges of `.main` (24px strips, punch-hole pattern)
+- **Paper grain background** on chat container
+- **Timestamps on user messages** (`HH:MM`), later upgraded to `MON dd, YYYY HH:MM:SS.ss` centisecond precision
+- **Shift+Enter** triggers web search
+- **Typing indicator greys out** on abort
+- **Token count badge** on search responses using client-side `tokenCount`
+- **Removed status dots** from input area (no functional purpose)
+- **Removed thumbs** from toolbar, restored only on non-search AI responses
+
+### Version bumped to v0.18.0
+
+## What's New in v0.17.26
+
+### Dynamic Model Swap — `request_model_swap()`, `select_node()` async (Roadmap N Task 14)
+- **`cluster.py`** — `request_model_swap()` publishes `cmd.swap_model` to `jc.admin`; `handle_model_ready()` and `handle_model_failed()` consume `model_ready`/`model_failed` on `jc.system`
+- **`select_node()` async** — Queries worker `inventory` for ideal model; triggers swap if model not active, returns `None` for fallback during swap
+- **`SUBSCRIBE_TABLE`** — 7 AMQP routing key bindings in cluster.py
+
+### Cluster Status UI — Heartbeat + Live Status Panel (Roadmap N Task 15)
+- **`handle_heartbeat()`** — Consumes `node.*.heartbeat` on `jc.system` to update `last_seen` per node
+- **UI cluster panel** — sidebar polls `GET /api/cluster` every 15s; green=active, yellow=swapping, red=error/offline
+- **Version bumped to v0.17.0** — All 179 tests pass
+
+### What's New in v0.14.0
+
+### Cluster Protocol — `GET /api/cluster`, 9 AMQP Message Types (Roadmap N Task 11)
+- **`cluster.py`** — Node registry (`CLUSTER_NODES`), bounded event log (`CLUSTER_EVENTS`, max 1000), coordinator auto-promotion
+- **Ping/pong health** — No passive heartbeats; coordinator pings workers on-demand before routing work. 5s timeout → auto-deregister
+- **9 message types** — register, deregister, admitted, rejected, ping, pong (on `jc.admin`); event, coord_query, coord_response (on `jc.system`)
+- **`amqp.py` subscribe()** — Exclusive anonymous queues bound to routing keys; `_rebind_subscriptions()` recreates them on reconnect
+- **`routers/cluster.py`** — `GET /api/cluster` returns nodes, coordinator, event log
+
+### RAG Corpus Management — `POST /api/rag/flush`, `GET /api/rag/stats` (v0.13.0)
+- **Score-based eviction** with hysteresis (80% high-water, 20% low-water) and pinned sources
+- **Eviction engine** in `eviction.py` — scroll Qdrant, score by retrieval count + age, evict lowest scores first
+- **Grace period** — vectors younger than 1 hour are never evicted
+- **Flush endpoint** — `POST /api/rag/flush` (admin) deletes all non-pinned vectors
+- **Stats endpoint** — `GET /api/rag/stats` (admin) returns vector count, at-risk count, pinned count, eviction rates
+
+### File & Document Attachments (v1.9.0–v1.10.0)
+- **`POST /api/upload`** — multipart file upload with PDF/text extraction; modes: `context` (chat injection), `ingest` (RAG corpus), `both`
+- **`DELETE /api/upload/{id}`** — removes upload from SQLite + Qdrant
+- **`PATCH /api/upload/{id}/link`** — associates upload with a conversation
+- **`GET /api/upload/by-conversation/{id}`** — list attachments per conversation
+- **Paperclip UI** — file picker, preview pill, image thumbnails, gallery overlay
+- **Attachment indicators** — 📎 badge on conversations with attachments
+- **Chat context injection** — `upload_context_id` prepends document text to system prompt
+
+### Terminal RAG Hook — `POST /api/ingest` (v0.11.0)
+- Bearer token auth (same key as `/v1/chat/completions`)
+- Chunking via shared `chunk_text()` helper, embed via Ollama, upsert to Qdrant
+- `caic-ingest.sh` — PROMPT_COMMAND shell script for autonomous terminal history ingestion
+
+### v1.8.0 Foundation (refactor & fixes)
+- **Modular refactor** — single-file `app.py` split into `config.py`, `db.py`, `auth.py`, `security.py`, `memory.py`, `search.py`, `rag.py`, `gpu.py`, and `routers/` package
+- **Perplexity auto-search fixed** — `logprobs: true` now properly extracted from stream chunks
+- **All `/api/models` endpoints** target `LLAMA_SERVER_BASE` (llama-server) not Ollama
+- **RAG embedding** via Ollama at `http://192.168.50.210:11434`
+- **Origin check** applies to all API methods, rejects absent Origin/Referer
+
+## Features
+
+- **Persistent Memory** — SQLite FTS5 full-text search for fast, relevant memory retrieval
+- **Web Search** — SearXNG integration for automatic web lookups when the model is uncertain
+- **Explicit Search** — Search button to force web search without waiting for model uncertainty
+- **Profile Injection** — Custom system prompt injected into every conversation
+- **System Presets** — Save and switch between different system prompts
+- **Real-time Stats** — CPU, RAM, GPU, VRAM monitoring in sidebar
+- **Token Thermometer** — Visual context window usage indicator
+- **Streaming Responses** — Server-sent events for real-time token display
+- **Conversation History** — SQLite-backed chat persistence with mass-delete option
+- **Model Switching** — Change inference models on the fly
+- **Skills Framework** — Built-in skill registry with per-skill enable/disable controls
+
+## File Structure
 
 ```
-┌─────────────────────────────────────────────────────────────┐
+/opt/caic/
-│                        YOUR LAN                             │
+├── amqp.py             # aio-pika AMQP connection manager + subscribe/rebind
-│                                                             │
+├── app.py              # FastAPI app entry point
-│  ┌─────────────────┐         ┌──────────────────────────┐  │
+├── auth.py             # PIN-based guest/admin sessions, auth routes
-│  │   jarvis        │◄──RPC───│   ultron                 │  │
+├── cluster.py          # Cluster protocol: node registry, event log, ping/pong
-│  │   192.168.50.212│  50052  │   192.168.50.108         │  │
+├── config.py           # Constants, env vars, limits, skill registry
-│  │                 │         │                          │  │
+├── db.py               # SQLite schema, connection factory
-│  │  jC :8080       │         │  llama-server :8081      │  │
+├── eviction.py         # Score-based RAG eviction engine
-│  │  SearXNG :8888  │         │  llama-server :8082 (*)  │  │
+├── gpu.py              # AMD GPU stats via rocm-smi
-│  │  RX 6600 XT 8GB │         │  Qdrant :6333            │  │
+├── hardware.py         # Hardware self-assessment (CPU, RAM, VRAM)
-│  │  GPU RPC worker │         │  mxbai-embed :11434      │  │
+├── memory.py           # FTS5 memory CRUD, remember/forget commands
-│  │  Vulkan backend │         │  AMD Ryzen 7 7840HS      │  │
+├── rag.py              # Qdrant vector search + system prompt assembly
-│  └─────────────────┘         │  Radeon 780M iGPU        │  │
+├── search.py           # SearXNG integration, perplexity, refusal detection
-│                              └──────────────────────────┘  │
+├── security.py         # Rate limiting, origin checks, IP allowlist, audit
-│                                                             │
+├── triage.py           # Query classification + cluster node selection
-│  ┌─────────────────┐         ┌──────────────────────────┐  │
+├── routers/
-│  │   pivault       │         │   corsair                │  │
+│   ├── chat.py         # /api/chat streaming endpoint
-│  │   192.168.50.158│         │   192.168.50.132         │  │
+│   ├── cluster.py      # Cluster status endpoint
-│  │                 │         │                          │  │
+│   ├── completions.py  # /v1/chat/completions OpenAI-compat endpoint
-│  │  10.83TB RAID5  │         │  RTX 5070 Ti 16GB        │  │
+│   ├── conversations.py# Conversation CRUD
-│  │  RPi 5 8GB      │         │  Ryzen 7 7800X3D         │  │
+│   ├── ingest.py       # Terminal RAG ingest
-│  │  NAS / Kopia    │         │  Gaming / Streaming      │  │
+│   ├── memories.py     # Memory CRUD API
-│  └─────────────────┘         └──────────────────────────┘  │
+│   ├── models.py       # Model listing, system stats
-│                                                             │
+│   ├── presets.py      # System prompt presets
-│  (*) Planned: Qwen2.5-Coder-14B on :8082                   │
+│   ├── profile.py      # User profile
-└─────────────────────────────────────────────────────────────┘
+│   ├── search_route.py # /api/search explicit search endpoint
+│   ├── settings.py     # Runtime settings
+│   ├── skills.py       # Skills management
+│   └── upload.py       # File attachment endpoints
+├── static/
+│   └── logo.png        # Logo image (optional)
+├── templates/
+│   └── index.html      # Frontend
+├── node_agent/
+│   ├── agent.py        # Standalone worker agent (AMQP client)
+│   └── requirements.txt
+└── tests/              # 179 pytest tests
 ```
 
-**Data flow:**
+## Requirements
-```
-Browser / IDE (Continue.dev)
-    → jC :8080 (FastAPI — auth, RAG, memory, conversation history)
-        → Qdrant :6333 (vector search, mxbai-embed-large for embeddings)
-        → llama-server :8081 (inference)
-            → jarvis RPC :50052 (GPU layer offload — RX 6600 XT)
-```
 
----
+- Python 3.11+ (tested on 3.13)
-
+- llama-server running locally or on network (OpenAI-compatible API on port 8081)
-## The AMD + NVIDIA Cross-Cluster Reality
+- SearXNG (optional, for web search)
-
+- RabbitMQ (optional, for AMQP cluster — coordinator only)
-This cluster intentionally mixes GPU architectures — **AMD RX 6600 XT on jarvis** and **NVIDIA RTX 5070 Ti on corsair**. This is deliberate and it works.
+- Qdrant (optional, for RAG vector search)
-
-The RPC layer in llama.cpp is GPU-vendor-agnostic. jarvis runs llama-rpc with a **Vulkan backend** (not ROCm, not CUDA) which provides hardware-neutral GPU acceleration. ultron's llama-server connects to it over TCP and offloads tensor layers without caring what GPU is on the other end.
-
-This means any machine on your LAN with any GPU (AMD, NVIDIA, Intel Arc) can participate as an RPC worker — as long as it can run llama-rpc with Vulkan support.
-
----
-
-## Cluster Performance Tuning
-
-### The Layer Offloading Trick
-
-The key to squeezing performance out of a CPU+GPU split cluster is `--n-gpu-layers`. This controls how many transformer layers get offloaded to the RPC GPU backend versus staying on the CPU.
-
-**Starting point (before tuning):** ~7 t/s  
-**After initial layer optimization:** ~17 t/s  
-**After full cluster tuning:** 30–35 t/s
-
-The progression that got us there:
-
-1. **Start with `--n-gpu-layers 99`** — tells llama-server to offload as many layers as possible. With Mistral-Nemo-12B Q4_K_M this results in all 41/41 layers offloading to jarvis GPU via RPC.
-
-2. **Verify GPU is actually working** — watch the llama-server startup log for:
-   ```
-   load_tensors: offloaded 41/41 layers to GPU
-   load_tensors: RPC[192.168.50.210:50052] model buffer size = 6763.30 MiB
-   load_tensors: CPU_Mapped model buffer size = 360.00 MiB
-   ```
-   If layers aren't offloading, the RPC connection isn't established.
-
-3. **Check actual throughput** — the timings block in llama-server responses shows real t/s. Tune from there.
-
-**Current llama-server service on ultron (`/etc/systemd/system/llama-server.service`):**
-```ini
-[Unit]
-Description=Llama.cpp Server (RPC frontend — Mistral-Nemo general)
-After=network.target
-
-[Service]
-Type=simple
-User=root
-ExecStart=/root/llama.cpp/build/bin/llama-server \
-  --model /home/gramps/models/Mistral-Nemo-Instruct-2407-Q4_K_M.gguf \
-  --rpc 192.168.50.212:50052 \
-  --host 0.0.0.0 \
-  --port 8081 \
-  --n-gpu-layers 99
-Restart=on-failure
-RestartSec=5
-
-[Install]
-WantedBy=multi-user.target
-```
-
-**llama-rpc service on jarvis (`/etc/systemd/system/llama-rpc.service`):**
-```ini
-[Unit]
-Description=Llama.cpp RPC Server (GPU backend — RX 6600 XT Vulkan)
-After=network.target
-
-[Service]
-Type=simple
-User=root
-ExecStart=/root/llama.cpp/build/bin/llama-rpc-server \
-  --host 0.0.0.0 \
-  --port 50052
-Restart=on-failure
-RestartSec=5
-
-[Install]
-WantedBy=multi-user.target
-```
-
----
-
-## Models
-
-### Current
-| Model | Location | Port | Purpose |
-|-------|----------|------|---------|
-| Mistral-Nemo-Instruct-2407-Q4_K_M | `/home/gramps/models/` on jarvis | ultron:8081 | General assistant, chat |
-| mxbai-embed-large | ultron (Docker/Ollama) | ultron:11434 | RAG embeddings |
-
-### Planned
-| Model | Size | Port | Purpose |
-|-------|------|------|---------|
-| Qwen2.5-Coder-14B-Q5_K_M | ~10GB | ultron:8082 | Code completion, pair programming |
-
-> **Note:** ultron has 16GB RAM. Only one primary inference model can be hot at a time. llama-server instances are swapped via systemd when switching between general and code models.
-
----
-
-## RAG System
-
-jC uses **Qdrant** for vector storage and **mxbai-embed-large** (1024-dim) for embeddings.
-
-### Qdrant Collection
-- **Collection:** `jarvis_rag`
-- **Vector size:** 1024 (mxbai-embed-large output)
-- **Distance:** Cosine
-- **Score threshold:** 0.25 (filters low-relevance chunks)
-- **Chunks retrieved per query:** 3 (configurable)
-
-### RAM Ceiling
-Each vector = 4KB (1024 dims × float32). With ultron's ~4-6GB available to Qdrant after llama-server:
-- Practical ceiling: ~1–1.5M chunks before RAM becomes the bottleneck
-- Current corpus: 219 points (early stage)
-- Storage on disk: negligible against pivault's 10.83TB
-
-### What Gets Ingested
-- Code repositories (your actual codebase)
-- Pair-programming conversation history
-- Architecture decisions and working commands
-- Documentation and URLs (fetched and stripped via beautifulsoup4/httpx)
-
----
-
-## JarvisChat Service (`/etc/systemd/system/jarvischat.service`)
-
-```ini
-[Unit]
-Description=JarvisChat - Local LLM Developer Platform
-After=network.target
-
-[Service]
-Type=simple
-User=root
-WorkingDirectory=/opt/jarvischat
-ExecStart=/opt/jarvischat/venv/bin/uvicorn app:app --host 0.0.0.0 --port 8080
-Restart=always
-RestartSec=5
-Environment=PYTHONUNBUFFERED=1
-Environment=OLLAMA_BASE=http://192.168.50.108:8081
-Environment=LLAMA_SERVER_BASE=http://192.168.50.108:8081
-
-[Install]
-WantedBy=multi-user.target
-```
-
----
 
 ## Installation
 
-### Prerequisites
-- Python 3.11+ (tested on 3.13)
-- llama.cpp built from source on both jarvis (RPC server) and ultron (llama-server)
-- Qdrant running on ultron
-- Ollama on ultron (for mxbai-embed-large embeddings)
-- SearXNG on jarvis:8888 (optional, for web search)
-
 ### Fresh Install
 
 ```bash
-sudo mkdir -p /opt/jarvischat
+# Create directory and venv
-sudo chown $USER:$USER /opt/jarvischat
+sudo mkdir -p /opt/caic
-cd /opt/jarvischat
+sudo chown $USER:$USER /opt/caic
+cd /opt/caic
 python3 -m venv venv
-./venv/bin/pip install fastapi uvicorn httpx psutil jinja2 python-multipart qdrant-client
+
+# Install dependencies
+pip install fastapi uvicorn httpx psutil jinja2 python-multipart pypdf aio-pika
+
+# Set admin PIN before first startup (4 digits)
+export CAIC_ADMIN_PIN=4827
+
+# Create subdirectories
 mkdir -p templates static
+
+# Copy files
+# (copy all .py files to /opt/caic/)
+# (copy routers/ directory to /opt/caic/)
+# (copy templates/index.html to /opt/caic/templates/)
 ```
 
-Copy `app.py` to `/opt/jarvischat/` and `index.html` to `/opt/jarvischat/templates/`.
+WARNING: Do not use `1234` as your admin PIN unless you accept weak local security.
 
-### Bootstrap the PIN
+NOTE: First boot requires `CAIC_ADMIN_PIN` unless you explicitly opt into insecure fallback with `CAIC_ALLOW_DEFAULT_PIN=true`.
+
+## Systemd Service
+
+Create `/etc/systemd/system/caic.service`:
+
+```ini
+[Unit]
+Description=cAIc - Local Inference Web Interface
+After=network.target
+
+[Service]
+Type=simple
+User=caic
+Group=caic
+WorkingDirectory=/opt/caic
+ExecStart=/opt/caic/venv/bin/uvicorn app:app --host 0.0.0.0 --port 8080
+Restart=always
+RestartSec=5
+
+[Install]
+WantedBy=multi-user.target
+```
 
 ```bash
-export JARVISCHAT_ADMIN_PIN=XXXX  # your 4-digit PIN
+sudo systemctl daemon-reload
+sudo systemctl enable caic
+sudo systemctl start caic
 ```
 
-Or allow the insecure default for testing:
+## Memory Commands
-```bash
-export JARVISCHAT_ALLOW_DEFAULT_PIN=true
-```
 
-### Environment Variables
+In chat, natural language triggers memory operations:
 
-| Variable | Default | Description |
+| You say | What happens |
-|----------|---------|-------------|
+|---------|--------------|
-| `OLLAMA_BASE` | `http://localhost:11434` | Ollama-compatible endpoint (legacy) |
+| "remember that I prefer Rust over Go" | Stores as `preference` |
-| `LLAMA_SERVER_BASE` | `http://192.168.50.108:8081` | llama-server OpenAI-compat inference endpoint |
+| "remember that cAIc runs on port 8080" | Stores as `infrastructure` |
-| `JARVISCHAT_ADMIN_PIN` | (none) | 4-digit admin PIN (required on first boot) |
+| "note that the deadline is Friday" | Stores as `general` |
-| `JARVISCHAT_ALLOW_DEFAULT_PIN` | `false` | Allow insecure default PIN 1234 |
+| "forget about the deadline" | Removes matching memories |
-| `JARVISCHAT_TRUSTED_ORIGINS` | (none) | Comma-separated trusted origins for CSRF |
-| `JARVISCHAT_ALLOWED_CIDRS` | RFC1918 + loopback | Allowed client IP CIDRs |
 
----
+Memories are automatically searched based on your message content and injected into the system prompt when relevant.
+
+### Memory Topics
+
+Memories are auto-categorized:
+- `preference` — likes, dislikes, choices
+- `project` — active work, repos, tasks
+- `infrastructure` — servers, services, configs
+- `personal` — name, location, background
+- `general` — everything else
 
 ## API Endpoints
 
-### Auth
+### Completions (OpenAI-compatible)
-| Method | Path | Description |
+
-|--------|------|-------------|
+| Method | Endpoint | Description |
-| POST | `/api/auth/guest` | Create guest session |
+|--------|----------|-------------|
-| POST | `/api/auth/login` | Admin PIN login |
+| POST | `/v1/chat/completions` | OpenAI-compatible chat (requires Bearer API key) |
-| POST | `/api/auth/logout` | Revoke session |
-| GET | `/api/auth/session` | Check session status |
-| POST | `/api/auth/heartbeat` | Keep session alive |
 
 ### Chat & Search
-| Method | Path | Description |
-|--------|------|-------------|
-| POST | `/api/chat` | Streaming chat (SSE) |
-| POST | `/api/search` | Explicit web search via SearXNG |
-| GET | `/api/search/status` | SearXNG health check |
 
-### Models
+| Method | Endpoint | Description |
-| Method | Path | Description |
+|--------|----------|-------------|
-|--------|------|-------------|
+| POST | `/api/chat` | Send message (streaming SSE) |
-| GET | `/api/models` | List available models from llama-server |
+| POST | `/api/search` | Explicit web search (streaming SSE) |
-| GET | `/api/ps` | Running models |
+
-| POST | `/api/show` | Model info |
+### File Upload & Ingest
+
+| Method | Endpoint | Description |
+|--------|----------|-------------|
+| POST | `/api/upload` | Upload file (multipart, admin) |
+| DELETE | `/api/upload/{id}` | Delete upload (admin) |
+| PATCH | `/api/upload/{id}/link` | Link upload to conversation (admin) |
+| GET | `/api/upload/by-conversation/{id}` | List uploads for conversation |
+| POST | `/api/ingest` | Ingest text into RAG (Bearer token auth) |
 
 ### Memory
-| Method | Path | Description |
+
-|--------|------|-------------|
+| Method | Endpoint | Description |
+|--------|----------|-------------|
 | GET | `/api/memories` | List all memories |
 | POST | `/api/memories` | Add memory |
 | PUT | `/api/memories/{rowid}` | Update memory |
 | DELETE | `/api/memories/{rowid}` | Delete memory |
-| GET | `/api/memories/search?q=` | FTS5 search memories |
+| GET | `/api/memories/search?q=term` | Search memories |
-| GET | `/api/memories/stats` | Memory statistics |
+| GET | `/api/memories/stats` | Get counts by topic |
+
+### Cluster
+
+| Method | Endpoint | Description |
+|--------|----------|-------------|
+| GET | `/api/cluster` | Cluster status (nodes, coordinator, event log) |
+
+### RAG Management
+
+| Method | Endpoint | Description |
+|--------|----------|-------------|
+| GET | `/api/rag/stats` | RAG corpus stats (admin) |
+| POST | `/api/rag/flush` | Delete non-pinned vectors (admin) |
+
+### Models & System
+
+| Method | Endpoint | Description |
+|--------|----------|-------------|
+| GET | `/api/models` | List available models |
+| GET | `/api/ps` | List loaded models |
+| POST | `/api/show` | Get model info |
+| GET | `/api/stats` | CPU, RAM, GPU, VRAM stats |
+| GET | `/api/search/status` | SearXNG availability |
+
+### Settings & Profile
+
+| Method | Endpoint | Description |
+|--------|----------|-------------|
+| GET | `/api/profile` | Get profile content |
+| PUT | `/api/profile` | Update profile (admin) |
+| GET | `/api/profile/default` | Get default profile |
+| GET | `/api/settings` | Get settings |
+| PUT | `/api/settings` | Update settings (admin) |
 
 ### Conversations
-| Method | Path | Description |
+
-|--------|------|-------------|
+| Method | Endpoint | Description |
+|--------|----------|-------------|
 | GET | `/api/conversations` | List conversations |
 | POST | `/api/conversations` | Create conversation |
-| GET | `/api/conversations/{id}` | Get conversation + messages |
+| GET | `/api/conversations/{id}` | Get conversation with messages |
-| PUT | `/api/conversations/{id}` | Update title/model |
+| PUT | `/api/conversations/{id}` | Update conversation title/model |
 | DELETE | `/api/conversations/{id}` | Delete conversation |
-| DELETE | `/api/conversations` | Delete all conversations |
+| DELETE | `/api/conversations` | Delete ALL conversations |
 
-### Profile & Settings
+### Presets
-| Method | Path | Description |
+
-|--------|------|-------------|
+| Method | Endpoint | Description |
-| GET | `/api/profile` | Get profile |
+|--------|----------|-------------|
-| PUT | `/api/profile` | Update profile |
+| GET | `/api/presets` | List presets |
-| GET | `/api/settings` | Get settings |
+| POST | `/api/presets` | Create preset |
-| PUT | `/api/settings` | Update settings |
+| PUT | `/api/presets/{id}` | Update preset |
-| GET | `/api/stats` | CPU/RAM/GPU stats |
+| DELETE | `/api/presets/{id}` | Delete preset |
 
 ### Skills
-| Method | Path | Description |
-|--------|------|-------------|
-| GET | `/api/skills` | List all skills |
-| GET | `/api/skills/active` | List enabled skills |
-| PUT | `/api/skills/{key}` | Enable/disable skill |
 
----
+| Method | Endpoint | Description |
+|--------|----------|-------------|
+| GET | `/api/skills` | List all skills with state |
+| GET | `/api/skills/active` | List active skills |
+| PUT | `/api/skills/{key}` | Toggle skill enabled (admin) |
 
-## Memory Commands
+### Auth
 
-Say these in chat to interact with the memory system:
+| Method | Endpoint | Description |
+|--------|----------|-------------|
+| POST | `/api/auth/guest` | Create guest session |
+| POST | `/api/auth/login` | Admin PIN login |
+| POST | `/api/auth/logout` | Revoke session |
+| GET | `/api/auth/session` | Check session validity |
+| POST | `/api/auth/heartbeat` | Extend session TTL |
 
-| Command | Effect |
+## Configuration
-|---------|--------|
-| `remember that [fact]` | Stores fact in FTS5 memory |
-| `please remember [fact]` | Same |
-| `don't forget [fact]` | Same |
-| `forget about [topic]` | Deletes matching memories |
 
----
+Settings are stored in the `settings` table and include:
 
-## Troubleshooting
+- `profile_enabled` — Inject profile into chats (true/false)
+- `search_enabled` — Auto web search (true/false)
+- `memory_enabled` — Memory injection (true/false)
+- `skills_enabled` — Skills framework (true/false)
+- `default_model` — Default inference model
+
+## Testing
 
-### jC starts but inference is slow or failing
-Check that llama-rpc is running on jarvis and llama-server is connected:
 ```bash
-# On jarvis
+python3 -m pytest tests/ -v
-systemctl status llama-rpc
-
-# On ultron — look for "offloaded N/N layers to GPU" in logs
-journalctl -u llama-server -n 50 --no-pager
 ```
 
-### ultron shows no CPU activity during inference
+All 179 tests use `tmp_path` fixtures + monkeypatched `httpx.AsyncClient`/`aio-pika`. No external services needed.
-Inference is being handled entirely by jarvis GPU via RPC — this is correct and expected. ultron's CPU is only involved for non-offloaded tensors (a small fraction of the model).
-
-### RAG not returning results
-Check Qdrant is up and the collection exists:
-```bash
-curl http://192.168.50.108:6333/collections/jarvis_rag
-```
-Verify `points_count` > 0. If zero, the corpus hasn't been seeded yet.
-
-### jC won't start — PIN bootstrap error
-Set the PIN via environment before first boot:
-```bash
-export JARVISCHAT_ADMIN_PIN=XXXX
-systemctl restart jarvischat
-```
-
-### sqlite3 not found
-Use Python instead:
-```bash
-python3 -c "import sqlite3; print(sqlite3.connect('/opt/jarvischat/jarvischat.db').execute('SELECT * FROM settings').fetchall())"
-```
-
----
-
-## Roadmap
-
-### TODO (Priority Order)
-1. **Tool calling** — read_file/write_file with /opt/jarvischat whitelist, tool_calls dispatch loop
-2. **git_tool** — Gitea integration for commit/push from jC
-3. **Audit logging** — structured audit trail to syslog
-4. SearXNG persistence (DONE ✅)
-5. search+ prefix for explicit search
-6. profile.example.md
-7. Conversation search/filter
-8. Export to markdown
-9. Keyboard shortcuts
-10. Retry button
-11. Source links in responses
-12. Rename conversations
-13. Multiple profiles
-14. KWIC auto-tags
-15. Image input (vision)
-16. btop split-screen integration
-17. Containerize
-18. SearXNG health indicator in UI
-19. check_patch_notes tool
-20. GitLab mirror of llgit repo
-
-### ROADMAP (Longer Horizon)
-
-**(A) Modular refactor** — Split monolithic app.py into routers/, services/, config.py, db.py, auth.py. Prerequisite for everything below.
-
-**(B) RAG ingest/manage UI** — File upload, URL ingest (fetch + strip HTML via beautifulsoup4/httpx, store URL as source metadata for citation), delete chunks/collections.
-
-**(C) Backend config panel** — Switch between Ollama/llama-server, endpoint URLs, model switching, restart — all from the UI without touching config files.
-
-**(D) Response metrics display** — tokens/sec, TTFT, context size, RAG chunks retrieved + scores — visible in the UI per response.
-
-**(E) Response quality feedback** — thumbs/stars/tags per response → feedback corpus → future RLHF dataset.
-
-**(F) IDE integration** — Continue.dev + VS Code, pointed at jC:8080 (not direct to inference endpoint). All IDE traffic — including pair-programming conversations — goes through jC so sessions are persisted and become RAG-worthy content. jC needs FIM request format handling to support inline autocomplete.
-
-**(G) Conversation history export → RAG ingest** — Bulk ingest existing conversation history into Qdrant.
-
-**(H) Fine-tuning pipeline** — LoRA on Mistral-Nemo from feedback corpus (item E).
-
-**(I) Autonomous RAG** — At conversation end, jC self-evaluates the transcript, extracts significant chunks (solved problems, working commands, architectural decisions), and ingests them into Qdrant automatically with metadata (date, conversation_id, reason). jC decides what it needs to remember. Closes the loop.
-
-**(J) Startup hardware/resource self-assessment** — On boot, jC queries ultron for available RAM, Qdrant consumption, and llama-server footprint. Derives dynamic high-water marks for RAG chunk limits, context window sizing, retrieval limits, and eviction thresholds. Writes a living config file. Replaces magic numbers with runtime-negotiated values.
-
-**(K) RAG corpus management** — Weighted LRU eviction with composite score (recency + frequency + content age) + manual pin flag for load-bearing knowledge. Prevents corpus bloat from degrading retrieval quality. Analogous to memcache eviction policy.
-
-**(L) Dual inference model architecture** — Mistral-Nemo-12B on ultron:8081 (general assistant), Qwen2.5-Coder-14B-Q5_K_M on ultron:8082 (code/pair programming). jC selects endpoint based on active model. Only one model hot at a time given ultron's 16GB RAM constraint.
-
----
-
-## Primary Cluster Objectives
-
-1. **Generative AI inference** — Local, private, fast enough to be useful
-2. **Agentic functionality** — Autonomous RAG self-management is the canonical first example. The system acts, not just responds.
-
----
-
-## Repository
-
-```
-ssh://gitea@llgit.llamachile.tube:1319/gramps/jarvisChat.git
-```
-
-> SSH username is `gitea`, not `git`. Port 1319.
-
----
 
 ## License
 
 MIT
+
+## Repository
+
+Gitea: `ssh://gitea@llgit.llamachile.tube:1319/gramps/caic.git`

TASKS.md

@@ -0,0 +1,986 @@
+# cAIc — OpenCode Prompt Sequence
+# Generated: 2026-07-01
+# Execute sequentially. Run full test suite after each task before proceeding.
+# Test command: ./venv/bin/python -m pytest tests/ -v
+
+---
+
+## ~~TASK 1 — README Cleanup [DONE]~~
+
+Review README.md in the current repo. Remove any node references other than `coordinator` (192.168.50.108) and `worker` (192.168.50.210). Ensure all references to the project use the exact casing `cAIc` — not `Jarvischat`, `JarvisChat`, or `jarvischat`. Do not change any functional content, endpoint documentation, or architecture descriptions — this is a text cleanup only. After editing, verify the file renders cleanly as markdown. Commit with message: `docs: clean up node references and branding consistency`.
+
+No new tests required for this task.
+
+---
+
+## ~~TASK 2 — Qwen2.5-Coder llama-server Service on Coordinator (Infrastructure) [DONE]~~
+
+**Status: Systemd unit created, verified, and restored.**
+
+This task originally defined creation of `/etc/systemd/system/llama-server-coder.service` (port 8082, Qwen2.5-Coder-14B Q5_K_M) as a prerequisite for dynamic model swapping. That sysadmin work is done.
+
+**The real Task 2 deliverable — the ability to dynamically swap models based on query classification — is delivered by Roadmap N (Tasks 9–15).** The flow:
+
+1. **Task 13** — Phi-4-mini triage (`triage.py`) classifies the query as `general`, `code`, `search`, or `rag`
+2. **Task 13** — `select_node()` picks the best worker node; if the ideal model isn't active, it triggers a swap
+3. **Task 14** — `request_model_swap()` publishes `cmd.swap_model` via AMQP `jc.admin` exchange
+4. **Task 12** — The node agent on worker receives the command, stops the current llama-server, starts the correct one, waits for health, and publishes `model_ready`
+5. **Task 14** — coordinator receives `model_ready`, updates the cluster registry, and routes the query to the node
+
+The swap is async and transparent — the user sees only latency. The UI (Task 15) shows a yellow "swapping" status dot during the transition.
+
+The service unit at `/etc/systemd/system/llama-server-coder.service` is the **target** the node agent starts when swapping to code inference. It is not enabled at boot — the AMQP cluster manages activation.
+
+See Tasks 9–15 for the actual model swap implementation.
+
+No pytest tests required for this infrastructure task.
+
+---
+
+## ~~TASK 3 — Update OpenCode Config to Use Qwen on :8082 [DONE]~~
+
+Update `/home/gramps/.config/opencode/opencode.jsonc` (on this machine, coordinator) to point the configured provider at `http://127.0.0.1:8082/v1` instead of `http://127.0.0.1:8081/v1`. The model name in the config should be updated to reflect `qwen2.5-coder-14b` or whatever model ID the llama-server instance at :8082 reports via `/v1/models`. Verify the endpoint is reachable before writing the config change. Do not restart OpenCode — the config change takes effect on next session start.
+
+No pytest tests required for this task.
+
+---
+
+## ~~TASK 4 — File/Document Attachment: Backend Ingest Endpoint [DONE]~~
+
+**Status: `POST /api/upload` with mode=(context|ingest|both), PDF/text extraction, Qdrant upsert, SQLite context (1hr expiry). Committed `4a891c8` (v1.9.0).**
+
+This task implements the backend half of file/document attachment (TODO #21). The goal is dual-aspect upload: a file can be used as immediate chat context, ingested into the RAG corpus (Qdrant), or both.
+
+**Add to `config.py`:**
+- `UPLOAD_DIR` — path for temporary upload storage, default `/tmp/caic_uploads`
+- `MAX_UPLOAD_BYTES` — max file size, default 20MB
+- `SUPPORTED_UPLOAD_TYPES` — set of MIME types: `text/plain`, `text/markdown`, `application/pdf`, `application/json`, `text/x-python`, `text/html`
+
+**Create `routers/upload.py`:**
+
+Implement `POST /api/upload` (admin required). Accept `multipart/form-data` with:
+- `file` — the uploaded file (required)
+- `mode` — string enum: `context` (inject into next chat only), `ingest` (add to RAG corpus), `both` (default: `both`)
+- `conversation_id` — optional, associates context-mode content with a specific conversation
+
+Behavior:
+- Validate file size against `MAX_UPLOAD_BYTES` — return 413 if exceeded
+- Validate MIME type against `SUPPORTED_UPLOAD_TYPES` — return 415 if unsupported
+- For PDF files, extract text using `pypdf` (add to requirements.txt)
+- For all other types, read as UTF-8 text
+- If mode includes `ingest`: chunk the extracted text into 512-token overlapping chunks (128-token overlap), generate embeddings via `EMBED_URL` (http://192.168.50.108:11434/api/embeddings, model mxbai-embed-large), upsert into Qdrant collection `caic` with metadata `{source: filename, upload_date: iso_timestamp, type: "upload"}`
+- If mode includes `context`: store the full extracted text in a new SQLite table `upload_context` with columns `(id INTEGER PRIMARY KEY, conversation_id TEXT, filename TEXT, content TEXT, created_at TEXT, expires_at TEXT)`. Context entries expire after 1 hour.
+- Return JSON: `{filename, size_bytes, mode, chunks_ingested (if ingest), context_id (if context), message}`
+
+**Add `upload_context` table to `db.py`** `init_db()`.
+
+**Wire `upload.router` into `app.py`** in the router registration block.
+
+**Write `tests/test_upload.py`** covering:
+- Valid text file upload, mode=ingest — assert chunks_ingested > 0, Qdrant upsert called
+- Valid text file upload, mode=context — assert context_id returned, row exists in upload_context
+- Valid text file upload, mode=both — assert both behaviors
+- File exceeds MAX_UPLOAD_BYTES — assert 413
+- Unsupported MIME type — assert 415
+- Guest session attempt — assert 403
+- PDF extraction path — mock pypdf, assert text extracted and processed
+
+Mock Qdrant and EMBED_URL calls via monkeypatch. Do not require live external services in tests.
+
+Run full test suite after implementation. All 26 existing tests must continue to pass.
+
+---
+
+## ~~TASK 5 — File/Document Attachment: UI Integration [DONE]~~
+
+**Status: Paperclip icon, file preview pill, gallery overlay, attachment indicators, DELETE/PATCH link/by-conversation endpoints, chat context injection. Committed `81238c0` (v1.10.0).**
+
+This task implements the frontend half of TODO #21. The UI is a single file at `templates/index.html`.
+
+Add a file attachment button to the chat input area. Requirements:
+- Paperclip icon button adjacent to the send button
+- Clicking opens a file picker filtered to supported types (`.txt`, `.md`, `.pdf`, `.json`, `.py`, `.html`)
+- On file selection, show a pill/badge above the input showing the filename with an X to remove it
+- On send, if a file is attached: POST to `/api/upload` with `mode=both` and the current `conversation_id`, then include the returned `context_id` in the subsequent `/api/chat` POST body as `upload_context_id`
+- If the upload fails, show an inline error and do not send the chat message
+- File attachment state clears after send
+
+**Update `/api/chat` in `routers/chat.py`:**
+- Accept optional `upload_context_id` in the request body
+- If present, look up the content in `upload_context` table and prepend it to the system prompt as: `\n\n[ATTACHED DOCUMENT: {filename}]\n{content}\n[END DOCUMENT]`
+- If the context_id is expired or missing, log a warning and continue without it (do not error)
+
+**Add to `tests/test_chat_streaming_and_memory_paths.py`:**
+- Test that a valid `upload_context_id` results in document content being prepended to the system prompt
+- Test that an expired/missing `upload_context_id` is silently ignored
+
+Run full test suite. All existing tests must continue to pass.
+
+---
+
+## ~~TASK 6 — Roadmap I: Terminal Command RAG Hook [DONE]~~
+
+**Status: `POST /api/ingest` with Bearer token auth, `chunk_text()` shared helper, `caic-ingest.sh` script. Committed `1ac21ad` (v0.11.0).**
+
+This task implements autonomous RAG ingestion of significant terminal activity (TODO #23).
+
+**Create `routers/ingest.py`:**
+
+Implement `POST /api/ingest` (requires Bearer token auth — use same `COMPLETIONS_API_KEY` mechanism as `routers/completions.py`). Accept JSON body:
+- `content` — string, the text to ingest (required)
+- `source` — string, origin label e.g. `terminal`, `file`, `external` (default: `external`)
+- `metadata` — optional dict of additional key/value pairs
+
+Behavior:
+- Chunk `content` into 512-token overlapping chunks (128-token overlap) — extract this logic into a shared helper `chunk_text(text, chunk_size=512, overlap=128)` in `rag.py` if not already present
+- Generate embeddings via `EMBED_URL`
+- Upsert into Qdrant collection `caic` with metadata `{source, ingest_date: iso_timestamp, ...metadata}`
+- Return JSON: `{chunks_ingested, source, message}`
+
+**Wire `ingest.router` into `app.py`.**
+
+**Create `/usr/local/bin/caic-ingest.sh` on worker (192.168.50.210)** — this is a shell script, not a Python file, and lives outside the repo. Write it to stdout/document it clearly so gramps can deploy it manually:
+
+```bash
+#!/bin/bash
+# caic-ingest.sh — pipe terminal commands into cAIc RAG
+# Add to ~/.bashrc: export PROMPT_COMMAND="jc_capture"
+# Function to call after significant commands
+
+JC_URL="http://192.168.50.210:8080/api/ingest"
+JC_TOKEN="${CAIC_COMPLETIONS_API_KEY}"
+
+jc_capture() {
+    local cmd
+    cmd=$(history 1 | sed 's/^[ ]*[0-9]*[ ]*//')
+    # Only ingest significant commands
+    if echo "$cmd" | grep -qE '^(git|pip|systemctl|sudo|vi|vim|curl|wget|apt|python|pytest)'; then
+        curl -s -X POST "$JC_URL" \
+            -H "Authorization: Bearer $JC_TOKEN" \
+            -H "Content-Type: application/json" \
+            -d "{\"content\": $(echo "$cmd" | python3 -c 'import json,sys; print(json.dumps(sys.stdin.read().strip()))'), \"source\": \"terminal\"}" \
+            > /dev/null 2>&1 &
+    fi
+}
+```
+
+**Write `tests/test_ingest.py`** covering:
+- Valid ingest with content — assert chunks_ingested > 0
+- Missing Bearer token — assert 401
+- Wrong Bearer token — assert 403
+- Empty content — assert 422
+- Qdrant and embed calls mocked via monkeypatch
+
+Run full test suite. All existing tests must continue to pass.
+
+---
+
+## ~~TASK 7 — Roadmap J: Startup Hardware Self-Assessment [DONE]~~
+
+**Status: `hardware.py` + `routers/hardware.py` + 4 tests. Committed `7291b8f` (v0.12.0).**
+
+On jC startup, probe available hardware and write a living config snapshot. This replaces hardcoded assumptions about VRAM and RAM.
+
+**Create `hardware.py`** in the project root:
+
+```
+async def assess_hardware() -> dict
+```
+
+Probes:
+- System RAM: `psutil.virtual_memory().total` and `.available`
+- CPU count: `psutil.cpu_count()`
+- GPU VRAM total and free: call `rocm-smi --showmeminfo vram --json` via subprocess, parse output. If rocm-smi absent or fails, set VRAM values to 0 and log a warning.
+- llama-server reachable: GET `LLAMA_SERVER_BASE/v1/models`, timeout 3s. Record True/False and list of available model IDs.
+- Qdrant reachable: GET `http://192.168.50.108:6333/collections`, timeout 3s. Record True/False and collection list.
+- SearXNG reachable: GET `http://localhost:8888`, timeout 3s. Record True/False.
+
+Returns a dict with all of the above. Writes result as JSON to `hardware_state.json` in the working directory.
+
+**Call `assess_hardware()` from the FastAPI `lifespan` context** in `app.py` on startup, after `init_db()`. Log a summary line: `HW: {ram_gb}GB RAM, {vram_mb}MB VRAM, llama={reachable}, qdrant={reachable}, searxng={reachable}`.
+
+**Expose `GET /api/hardware`** in a new `routers/hardware.py` — returns the current `hardware_state.json` content as JSON. No auth required (read-only, non-sensitive aggregate stats).
+
+**Wire `hardware.router` into `app.py`.**
+
+**Write `tests/test_hardware.py`** covering:
+- `assess_hardware()` with all services reachable (mock subprocess and httpx calls) — assert all fields present
+- `assess_hardware()` with rocm-smi absent — assert VRAM=0, no exception raised
+- `assess_hardware()` with llama-server unreachable — assert `llama_reachable=False`, no exception
+- `GET /api/hardware` — assert returns JSON with expected keys
+
+Run full test suite. All existing tests must continue to pass.
+
+---
+
+## ~~TASK 8 — Roadmap K: RAG Corpus Management [DONE]~~
+
+Qdrant collection `caic` currently grows without bound. Implement score-based eviction with hysteresis, pinned sources, operational stats, and a flush command.
+
+### Config — add to `config.py`:
+
+```python
+RAG_MAX_VECTORS = 50000               # absolute ceiling; eviction targets thresholds below it
+RAG_EVICTION_HIGH_WATER = 0.80        # fraction of RAG_MAX_VECTORS that triggers eviction
+RAG_EVICTION_LOW_WATER = 0.20         # fraction where eviction stops
+RAG_EVICTION_BATCH = 1000             # max points to delete per Qdrant scroll/delete cycle
+RAG_PINNED_SOURCES = ["upload", "profile"]  # never evicted
+RAG_GRACE_HOURS = 1                   # new vectors ineligible for eviction until this old
+RAG_ACCESS_WEIGHT = 1.0               # score factor: retrieval_count * ACCESS_WEIGHT
+RAG_AGE_WEIGHT = 0.1                  # score factor: ingest_age_hours * AGE_WEIGHT
+```
+
+Validations on boot: `high_water > low_water`, `batch > 0`, `max_vectors > 0`.
+
+### Eviction algorithm — add to `rag.py`:
+
+```
+score = (retrieval_count * ACCESS_WEIGHT) + (age_hours * AGE_WEIGHT)
+```
+
+Lower score = evicted first. Tiebreak: `last_accessed` ASC (older wins).
+
+```python
+async def get_collection_count() -> int
+    # GET /collections/caic → return vectors_count
+
+async def get_collection_stats() -> dict
+    # Return {vector_count, max_vectors, high_water, low_water, percent_full, pinned_sources}
+
+async def evict_batch(batch_size: int) -> int
+    # Scroll Qdrant for vectors NOT in RAG_PINNED_SOURCES, WHERE ingest_age > RAG_GRACE_HOURS,
+    # ordered by score ASC, last_accessed ASC.
+    # Delete up to batch_size. Return count deleted.
+    # If 0 evictable vectors found: log warning, return 0 (break loop).
+
+async def maybe_evict() -> int
+    # Acquire eviction_lock (asyncio.Lock).
+    # count = get_collection_count()
+    # threshold_high = RAG_MAX_VECTORS * RAG_EVICTION_HIGH_WATER
+    # threshold_low  = RAG_MAX_VECTORS * RAG_EVICTION_LOW_WATER
+    # total_evicted = 0
+    # while count >= threshold_low:
+    #     if total_evicted > 0 and count < threshold_low: break
+    #     deleted = evict_batch(RAG_EVICTION_BATCH)
+    #     if deleted == 0: break  # no more unpinned targets
+    #     total_evicted += deleted
+    #     count -= deleted
+    #     if count < threshold_high and total_evicted > 0: break
+    #     # only one pass if batch spans the full gap
+    #     if count < threshold_low: break
+    # Record total_evicted + timestamp in EVICTION_LOG (list of dicts, kept in memory, max 1000 entries)
+    # Release lock. Return total_evicted.
+
+async def get_rag_operational_stats() -> dict
+    # Returns: vector_count, max_vectors, high_water_pct, low_water_pct,
+    # percent_full, pinned_sources, grace_hours,
+    # eviction_counts_last_1m, eviction_counts_last_5m, eviction_counts_last_30m,
+    # at_risk_count (vectors in bottom 10% by score),
+    # pinned_count, avg_retrieval_count
+```
+
+### Edge cases & guards:
+
+1. **Newborn grace** — vectors < `RAG_GRACE_HOURS` old are excluded from eviction scroll (score=0 otherwise → immediate deletion)
+2. **All-pinned freeze** — if scroll returns 0 evictable vectors, log warning and break loop
+3. **Race** — `asyncio.Lock()` guards `maybe_evict()`; concurrent callers wait their turn
+4. **Zero config** — `RAG_MAX_VECTORS <= 0` → eviction disabled; `RAG_EVICTION_BATCH <= 0` → clamped to 1
+5. **Legacy payloads** — vectors without `retrieval_count` or `last_accessed` get defaults (0, `ingest_date`)
+
+### Wire eviction:
+
+Call `maybe_evict()` after each upsert batch completes in:
+- `routers/upload.py` — after Qdrant upsert
+- `routers/ingest.py` — after Qdrant upsert
+
+### Admin endpoints — new `routers/rag_admin.py`:
+
+| Method | Endpoint | Description |
+|--------|----------|-------------|
+| GET | `/api/rag/stats` | Operational stats (see `get_rag_operational_stats()`) — admin required |
+| POST | `/api/rag/flush` | Delete ALL points from the Qdrant `caic` collection. Returns `{deleted_count, collection: "caic", status: "flushed"}`. Admin required. |
+
+### In-memory eviction log:
+
+```python
+EVICTION_LOG: list[dict] = []  # managed by rag.py, max 1000 entries
+# Each entry: {timestamp: iso, count: N, remaining: N}
+# Tied to RATE_EVENTS pattern from security.py for rolling window calculations
+```
+
+### Tests — `tests/test_rag_management.py`:
+
+- `get_collection_count()` — mock Qdrant GET, assert correct count
+- `get_collection_stats()` — assert shape matches config
+- `evict_batch()` — mock Qdrant scroll + delete, assert pinned sources excluded, grace period enforced, batch size respected
+- `maybe_evict()` — below high water: 0 evicted; at high water: eviction fires; stops at low water; all-pinned scroll returns 0 → breaks
+- `GET /api/rag/stats` — assert full shape
+- `POST /api/rag/flush` — assert points deleted, admin required, guest 403
+- `POST /api/rag/flush` by guest — assert 403
+- Race lock — concurrent calls to `maybe_evict()` queue up, only one evicts
+
+Mock all Qdrant calls via monkeypatch. Do not require live services.
+
+Run full test suite. All existing tests must continue to pass.
+
+---
+
+## ~~TASK 9 — Roadmap N1: RabbitMQ Install and Service on Coordinator (Infrastructure) [DONE]~~
+
+This task runs on coordinator (this machine). Install RabbitMQ and verify it is operational.
+
+Run the following steps:
+1. `apt-get update && apt-get install -y rabbitmq-server`
+2. `systemctl enable rabbitmq-server && systemctl start rabbitmq-server`
+3. `systemctl status rabbitmq-server` — verify active/running
+4. Enable the management plugin: `rabbitmq-plugins enable rabbitmq_management`
+5. Create a dedicated jC vhost: `rabbitmqctl add_vhost caic`
+6. Create a dedicated user: `rabbitmqctl add_user caic CHANGEME_PASSWORD` — generate a random 24-char alphanumeric password and record it
+7. Grant permissions: `rabbitmqctl set_permissions -p caic caic ".*" ".*" ".*"`
+8. Verify management UI is reachable: `curl -s -u guest:guest http://localhost:15672/api/overview | python3 -m json.tool`
+9. Delete default guest user: `rabbitmqctl delete_user guest`
+
+Declare the two topic exchanges needed by jC:
+- Exchange name: `jc.admin`, type: `topic`, durable: true
+- Exchange name: `jc.system`, type: `topic`, durable: true
+
+Use `rabbitmqadmin` or `curl` against the management API to declare exchanges. Verify both exchanges appear in: `curl -s -u caic:{password} http://localhost:15672/api/exchanges/caic`
+
+Write the generated RabbitMQ password to `/home/gramps/.caic_amqp_secret` with mode 600. This will be read by cAIc as an env var source in subsequent tasks.
+
+No pytest tests required for this infrastructure task.
+
+---
+
+## ~~TASK 10 — Roadmap N2: AMQP Connection Layer in jC [DONE]~~
+
+This task adds the core AMQP connection manager to jC. It must connect to RabbitMQ on coordinator (localhost from jC's perspective since jC runs on coordinator), handle reconnection, and provide a shared channel for all AMQP operations.
+
+**Add to `requirements.txt`:** `aio-pika>=9.0.0`
+
+**Add to `config.py`:**
+- `AMQP_URL` — read from env `CAIC_AMQP_URL`, default `amqp://caic:password@localhost:5672/caic`. The actual password comes from `/home/gramps/.caic_amqp_secret` — read it at startup if the env var is not set.
+- `AMQP_RECONNECT_DELAY` — seconds between reconnect attempts, default 5
+- `AMQP_EXCHANGE_ADMIN` — `jc.admin`
+- `AMQP_EXCHANGE_SYSTEM` — `jc.system`
+
+**Create `amqp.py`** in the project root:
+
+```python
+# Manages a single persistent aio-pika connection and channel.
+# Provides:
+#   connect() -> None          # establish connection, declare exchanges
+#   disconnect() -> None       # graceful close
+#   get_channel()              # returns current channel, reconnects if needed
+#   publish(exchange, routing_key, payload: dict) -> None
+#                              # publishes JSON-serialized payload as persistent message
+```
+
+Connection must:
+- Reconnect automatically on disconnect with `AMQP_RECONNECT_DELAY` backoff
+- Log connection events at INFO level
+- Not raise on publish if disconnected — log error and return (fire-and-forget, jC must not crash if RabbitMQ is down)
+
+**Start AMQP connection in `app.py` lifespan** after `assess_hardware()`. Disconnect in lifespan cleanup.
+
+**Write `tests/test_amqp.py`** covering:
+- `publish()` with mocked aio-pika connection — assert message published with correct exchange and routing key
+- `publish()` when disconnected — assert no exception raised, error logged
+- `get_channel()` when connection is None — assert reconnect attempted
+
+Mock all aio-pika calls via monkeypatch. Do not require a live RabbitMQ instance in tests.
+
+Run full test suite. All existing tests must continue to pass.
+
+---
+
+## ~~TASK 11 — Roadmap N3: Cluster Protocol & Registration Handler (Coordinator Side) [DONE]~~
+
+**Status: Implemented and pushed (899988c).** `amqp.py` subscribe/rebind, `cluster.py` with CLUSTER_NODES/CLUSTER_EVENTS/CLUSTER_COORDINATOR and 6 handlers, `routers/cluster.py` (`GET /api/cluster`), 13 tests. No passive heartbeats — ping/pong on-demand before work routing. 148 tests pass.
+
+jC on the coordinator must listen for nine message types across `jc.admin` and `jc.system`, maintain the cluster registry, and expose an application-level event log.
+
+### 11.1 AMQP Protocol — Message Catalog
+
+All payloads are JSON, published as persistent messages.
+
+| Direction | Exchange | Routing Key | Message Type | Description |
+|-----------|----------|-------------|-------------|-------------|
+| Worker → Coordinator | `jc.admin` | `node.{name}.register` | register | Worker requests admission |
+| Worker → Coordinator | `jc.admin` | `node.{name}.deregister` | deregister | Worker signals graceful departure |
+| Coordinator → Worker | `jc.admin` | `node.{name}.admitted` | admitted | Coordinator grants admission |
+| Coordinator → Worker | `jc.admin` | `node.{name}.rejected` | rejected | Coordinator denies admission (with reason) |
+| Coordinator → Worker | `jc.admin` | `node.{name}.ping` | ping | Coordinator checks if worker is alive (sent before routing work) |
+| Worker → Coordinator | `jc.admin` | `node.{name}.pong` | pong | Worker confirms aliveness |
+| Worker → Coordinator | `jc.system` | `node.{name}.event` | event | Application-level syslog event |
+| Any → All | `jc.system` | `cluster.coordinator.query` | coord_query | Anyone asks "who is coordinator?" |
+| Coordinator → All | `jc.system` | `cluster.coordinator.response` | coord_response | Coordinator announces itself |
+
+Worker presence is assumed from registration onward. No periodic heartbeats — a worker can sit idle for days without chatter. When the coordinator needs to route work to a worker, it pings first; if the worker doesn't pong within timeout, the coordinator deregisters it and moves to the next node.
+
+### 11.2 Payload Schemas
+
+**register** (worker → coordinator):
+```json
+{
+  "node_name": "worker01",
+  "node_type": "worker",
+  "ip": "192.168.50.210",
+  "capabilities": {
+    "gpu": true, "gpu_type": "amd", "vram_mb": 8192,
+    "cpu_cores": 8, "ram_gb": 16
+  },
+  "active_model": {
+    "name": "llama3.1", "version": "latest", "quant": "Q4_K_M",
+    "path": "/var/lib/caic/models/llama3.1-latest-Q4_K_M.gguf",
+    "port": 8081
+  },
+  "inventory": [
+    {"name": "llama3.1", "version": "latest", "quant": "Q4_K_M",
+     "path": "/var/lib/caic/models/llama3.1-latest-Q4_K_M.gguf", "port": 8081}
+  ],
+  "status": "active"
+}
+```
+
+**deregister** (worker → coordinator):
+```json
+{
+  "node_name": "worker01",
+  "reason": "shutdown",
+  "timestamp": "2026-07-06T12:00:00Z"
+}
+```
+
+**ping** (coordinator → worker):
+```json
+{
+  "from": "coordinator",
+  "node_name": "worker01",
+  "type": "ping",
+  "correlation_id": "550e8400-e29b-41d4-a716-446655440000",
+  "timestamp": "2026-07-06T12:00:00Z"
+}
+```
+Worker must respond within 5 seconds or the coordinator considers it absent.
+
+**pong** (worker → coordinator):
+```json
+{
+  "node_name": "worker01",
+  "type": "pong",
+  "correlation_id": "550e8400-e29b-41d4-a716-446655440000",
+  "status": "active",
+  "active_model": {"name": "llama3.1", "port": 8081},
+  "load": {"cpu_pct": 45, "ram_pct": 62, "vram_pct": 38},
+  "timestamp": "2026-07-06T12:00:00Z"
+}
+```
+Correlation ID matches the ping so the coordinator can pair request and response.
+
+**coord_query** (any → `cluster.coordinator.query`):
+```json
+{"type": "coord_query", "timestamp": "2026-07-06T12:00:00Z"}
+```
+Coordinator responds on `cluster.coordinator.response`:
+```json
+{
+  "coordinator_node": "coordinator",
+  "cluster_nodes": ["worker01"],
+  "timestamp": "2026-07-06T12:00:00Z"
+}
+```
+
+**event** (worker → coordinator):
+```json
+{
+  "node_name": "worker01",
+  "severity": "info",
+  "message": "llama-server started with model llama3.1:latest",
+  "details": {"model": "llama3.1:latest", "port": 8081, "pid": 1234},
+  "timestamp": "2026-07-06T12:00:00Z"
+}
+```
+Severity levels: `info`, `warn`, `error`, `critical`. The coordinator assigns `category: "application"` based on the exchange (jc.system). No `event_type` field — the category is determined by the channel, not the payload.
+
+### 11.3 Design — Status Transitions Drive the Event Log
+
+All admin-level events are *derived* from `register()` and `deregister()` as side effects. There are no separate message types for coordinator election, node staleness, quarantine, or release — those are status transitions that `register()`/`deregister()` emit into `CLUSTER_EVENTS` locally.
+
+**Node status lifecycle:**
+
+```
+UNKNOWN ──register()──▶ active ──deregister()──▶ (removed)
+                           │
+               ping timeout│(coordinator publishes
+                           │ deregister on its behalf)
+                           ▼
+                      (removed)
+```
+
+**Coordinator status lifecycle:**
+
+```
+NONE ──register(node_type=coordinator)──▶ CLUSTER_COORDINATOR set
+                                              │
+                                   deregister()│or timeout
+                                              ▼
+                                         CLUSTER_COORDINATOR cleared
+```
+
+**Event categories — two buckets, no granular types:**
+
+| Category | When | severity |
+|----------|------|----------|
+| `cluster` | Node lifecycle, coordinator changes, model swaps, node offline — everything on `jc.admin` | `info` / `warn` / `error` |
+| `application` | Worker syslog events (incoming on `jc.system` `node.*.event`) | `info` / `warn` / `error` / `critical` |
+
+Every `_push_event()` call uses one of these two categories. The `message` field carries the human-readable detail — no need for event type strings. The reporting tool filters by category + severity.
+
+**Channel split — security rationale:**
+
+The two exchanges are not an organizational convenience. They enforce a **data isolation boundary**:
+
+| Exchange | Contains | Exposed to |
+|----------|----------|------------|
+| `jc.admin` | Node lifecycle, heartbeats, model swaps, coordinator changes | Operations / machine-room staff |
+| `jc.system` | Application events — inference queries, RAG context, user-facing data | Application-layer audit only |
+
+`jc.system` events can leak information about what users are doing and asking. The split ensures a sysadmin monitoring cluster health never accidentally consumes user-data-bearing events. The channels can be locked down independently — different AMQP credentials, separate queue permissions, different in-transit encryption policies if needed later.
+
+### 11.4 Implementation
+
+**Add to `amqp.py`:**
+
+```python
+_SUBSCRIPTIONS: list[tuple[str, str, Callable]]  # (exchange, routing_key, callback)
+
+async def subscribe(exchange, routing_key, callback) -> None
+    # Append to _SUBSCRIPTIONS list
+    # Declare a unique queue per subscription (name: f"jc.{exchange}.{sanitized_routing_key}")
+    # Bind queue to exchange/routing_key, consume with callback
+```
+
+Each subscription gets its own queue so multiple subscribers on different routing keys all receive messages. On reconnect: drain old consumers, iterate `_SUBSCRIPTIONS`, re-declare and re-bind each one. The `connect()` function must call `_rebind_subscriptions()` after exchanges are declared.
+
+**Create `cluster.py`** in the project root:
+
+```python
+# In-memory cluster registry + event log
+# Survives only while jC is running (not persisted)
+
+CLUSTER_NODES: dict[str, NodeRecord]
+CLUSTER_EVENTS: deque[EventRecord]   # bounded at 1000 entries
+CLUSTER_COORDINATOR: str | None      # node_name of active coordinator
+
+# NodeRecord fields:
+#   node_name, node_type, ip, status, active_model, inventory,
+#   capabilities: {gpu, gpu_type, vram_mb, cpu_cores, ram_gb}
+#   registered_at, last_seen
+
+# EventRecord:
+#   category: str     ("cluster" | "application")
+#   severity: str     ("info" | "warn" | "error" | "critical")
+#   node_name: str
+#   message: str
+#   details: dict | None
+#   timestamp: str
+
+def _push_event(category, severity, node_name, message, details=None) -> None
+    # Append EventRecord to CLUSTER_EVENTS, pop left if > 1000
+
+async def handle_registration(message) -> None
+    # Parse payload, validate required fields (node_name, node_type, ip, active_model, inventory)
+    # Reject if node_name duplicate and CLUSTER_NODES[node_name].status == "active"
+    # If CLUSTER_COORDINATOR is None AND node_type == "coordinator":
+    #   set CLUSTER_COORDINATOR = node_name
+    #   _push_event("cluster", "info", node_name, "elected coordinator")
+    #   publish cluster.coordinator.response on jc.system {coordinator_node, cluster_nodes, timestamp}
+    # Add node to CLUSTER_NODES with status="active"
+    # _push_event("cluster", "info", node_name, f"admitted as {node_type}")
+    # publish admitted on jc.admin node.{name}.admitted {node_name, timestamp, amqp_url}
+
+async def handle_deregistration(message) -> None
+    # Parse payload (node_name, reason, timestamp)
+    # If node_name == CLUSTER_COORDINATOR:
+    #   clear CLUSTER_COORDINATOR
+    #   _push_event("cluster", "warn", node_name, f"coordinator lost — {reason}")
+    # _push_event("cluster", "info", node_name, f"departed — {reason}")
+    # Remove node from CLUSTER_NODES, log it
+
+async def handle_pong(message) -> None
+    # Parse: node_name, correlation_id, status, active_model, load, timestamp
+    # Match correlation_id to outstanding ping
+    # If node in CLUSTER_NODES: update last_seen, status, active_model
+    # Signal the waiting caller that the node is alive
+    # If node unknown: log warning, do NOT auto-admit
+
+async def handle_event(message) -> None
+    # Parse: node_name, severity, message, details, timestamp
+    # Assigns category="application" (incoming on jc.system)
+    # Append EventRecord to CLUSTER_EVENTS (pop left if > 1000)
+
+async def handle_coordinator_query(message) -> None
+    # Respond on jc.system cluster.coordinator.response
+    # Payload: {coordinator_node, cluster_nodes: list(CLUSTER_NODES.keys()), timestamp}
+
+def get_cluster_state() -> dict
+    # Return: {nodes: CLUSTER_NODES, coordinator: CLUSTER_COORDINATOR,
+    #          events: last 50 CLUSTER_EVENTS}
+```
+
+**Subscribe in `app.py` lifespan** after AMQP connects:
+
+| Exchange | Routing Key | Handler |
+|----------|-------------|---------|
+| `jc.admin` | `node.*.register` | `handle_registration` |
+| `jc.admin` | `node.*.deregister` | `handle_deregistration` |
+| `jc.admin` | `node.*.pong` | `handle_pong` |
+| `jc.system` | `node.*.event` | `handle_event` |
+| `jc.system` | `cluster.coordinator.query` | `handle_coordinator_query` |
+
+### 11.5 API — `GET /api/cluster`
+
+New router `routers/cluster.py`:
+- `GET /api/cluster` — returns full cluster state: `{nodes, coordinator, events}` (last 50 events). No auth required.
+
+Wire `cluster.router` into `app.py`.
+
+### 11.6 Tests — `tests/test_cluster.py`
+
+Mock all aio-pika calls. Do not require live RabbitMQ.
+
+| # | Test | What it asserts |
+|---|------|-----------------|
+| 1 | Valid worker registration | Node admitted, CLUSTER_NODES updated, `cluster` event logged, `admitted` message published |
+| 2 | First coordinator auto-promotion | CLUSTER_COORDINATOR set, `cluster` event with "elected" message, `coord_response` published |
+| 3 | Duplicate node name rejected | `rejected` message with reason=`duplicate_node_name`, `cluster` event logged |
+| 4 | Malformed payload rejected | `rejected` message with reason=`malformed_payload` |
+| 5 | Graceful deregistration | Node removed, `cluster` event logged. If coordinator: CLUSTER_COORDINATOR cleared |
+| 6 | Pong from known node | last_seen updated, load/status refreshed |
+| 7 | Pong from unknown node | Warning logged, node NOT added |
+| 8 | Event stored in log | Event appended to CLUSTER_EVENTS; at 1001 entries the oldest is popped |
+| 9 | Coordinator query produces response | Response published with coordinator name and node list |
+| 10 | GET /api/cluster shape | Response contains `nodes`, `coordinator`, `events` keys |
+
+Run full test suite. All existing tests must continue to pass.
+
+---
+
+## ~~TASK 12 — Roadmap N4: Worker Node Registration Publisher (Worker Side) [DONE]~~
+
+This task creates the worker node AMQP client that runs on worker (192.168.50.210). It is a standalone Python script — not part of the jC FastAPI app — that runs as a systemd service on worker.
+
+**Create `node_agent/agent.py`** in the repo (new directory).
+
+### 12.1 Config & Inventory Discovery
+
+On start, reads `/etc/caic-node-agent.conf` (INI format):
+- `node_name` — hostname, default from `socket.gethostname()`
+- `node_ip` — LAN IP, default from socket
+- `node_type` — `"worker"` (fixed)
+- `capabilities` — comma-separated list, e.g. `llm,rag`
+- `amqp_url` — RabbitMQ URL on coordinator, e.g. `amqp://caic:password@192.168.50.108:5672/caic`
+- `llama_port` — port llama-server/llama-rpc is listening on, default 8081
+- `models_dir` — path to GGUF model files, default `/var/lib/caic/models`
+- `active_model` — filename of currently active model (without path)
+
+Discovers inventory by globbing `models_dir` for `*.gguf` files and parsing name/version/quant from filename using regex pattern: `{name}-{version}-{quant}.gguf` where quant matches `Q[0-9]+_K_[A-Z]+` or similar standard suffixes.
+
+### 12.2 Registration
+
+Publishes registration to `jc.admin`, routing key `node.{node_name}.register`:
+```json
+{
+  "node_name": "worker01",
+  "node_type": "worker",
+  "ip": "192.168.50.210",
+  "capabilities": ["llm"],
+  "active_model": {"name": "...", "version": "...", "quant": "...", "path": "...", "port": 8081}
+}
+```
+
+### 12.3 Admission Response
+
+Listens on `node.{node_name}.admitted` and `node.{node_name}.rejected` (both `jc.admin`). Logs result. If rejected, exits with error.
+
+### 12.4 Ping Listener
+
+After admission: listens on `jc.admin`, routing key `node.{node_name}.ping`. On receipt, responds immediately (within 1 second) with a pong on `jc.admin`, routing key `node.{node_name}.pong`:
+
+```json
+{
+  "node_name": "worker01",
+  "type": "pong",
+  "correlation_id": "<echoed from ping>",
+  "status": "active",
+  "active_model": {"name": "...", "version": "...", "quant": "...", "path": "...", "port": 8081},
+  "load": {"cpu_pct": 45, "ram_pct": 62, "vram_pct": 38},
+  "timestamp": "<utc>"
+}
+```
+
+No periodic heartbeats. Worker sits idle between pings — coordinator only pings when it needs to route work.
+
+### 12.5 Model Swap Command Handler
+
+Listens on `jc.admin`, routing key `node.{node_name}.cmd.swap_model`:
+- Payload: `{model_filename: str}`
+- Stops current llama-server: `systemctl stop llama-server`
+- Updates `/etc/caic-node-agent.conf` active_model field
+- Starts llama-server: `systemctl start llama-server` (assumes service reads active_model from conf or ExecStart is updated)
+- Waits for llama-server to be healthy: poll `http://localhost:{llama_port}/v1/models` every 2s, timeout 120s
+- Publishes to `jc.system`, routing key `node.{node_name}.model_ready`:
+  ```json
+  {"node_name": "...", "active_model": "...", "port": ..., "timestamp": "..."}
+  ```
+- If startup fails within timeout: publishes `node.{node_name}.model_failed` with error detail
+
+### 12.6 Files & Tests
+
+**Create `node_agent/requirements.txt`:** `aio-pika>=9.0.0`
+
+**Document `/etc/caic-node-agent.conf` format** in a comment block at the top of `agent.py`.
+
+**Write `tests/test_node_agent.py`** covering:
+- Registration payload construction from config + model discovery — assert correct JSON shape
+- Model swap command handler: success path — assert systemctl calls made, model_ready published
+- Model swap command handler: timeout path — assert model_failed published
+- Ping handler: on ping, publishes pong with correct correlation_id
+- Agent starts idle after admission, no heartbeat timer
+
+Mock all aio-pika, subprocess, and httpx calls.
+
+**Do not create a systemd service file in this task** — that is a manual deployment step. Document the required service configuration in a comment at the bottom of `agent.py`.
+
+Run full test suite. All existing tests must continue to pass.
+
+---
+
+## ~~TASK 13 — Roadmap N5: Query Routing via AMQP + Phi-4-mini Triage [DONE]~~
+
+This task wires the cluster into jC's chat flow. When a query arrives at `/api/chat`, instead of always routing to the hardcoded `LLAMA_SERVER_BASE`, jC now routes to the best available cluster node based on query context.
+
+**Prerequisites:** Tasks 9–12 complete. At least one worker node admitted to cluster.
+
+**Install Phi-4-mini on coordinator (infrastructure step):**
+- Download `Phi-4-mini-Instruct-Q4_K_M.gguf` from HuggingFace using `hf download microsoft/Phi-4-mini-instruct --include "*.Q4_K_M.gguf" --local-dir /var/lib/caic/models`
+- Create `/etc/systemd/system/llama-server-triage.service` — same pattern as existing llama-server service but: port 8083, model path points to Phi-4-mini GGUF, no `--rpc` flag (runs entirely on coordinator CPU/iGPU), description `Llama.cpp Server (Phi-4-mini — triage/routing)`
+- `systemctl daemon-reload && systemctl enable llama-server-triage && systemctl start llama-server-triage`
+- Verify: `curl -s http://localhost:8083/v1/models`
+
+**Add to `config.py`:**
+- `TRIAGE_BASE` — `http://127.0.0.1:8083/v1` (Phi-4-mini)
+- `TRIAGE_TIMEOUT` — 10 seconds
+- `FALLBACK_TO_DEFAULT` — True (if triage fails or no nodes available, fall back to `LLAMA_SERVER_BASE`)
+
+**Create `triage.py`** in the project root:
+
+```python
+async def classify_query(query: str) -> str
+    # Sends query to Phi-4-mini at TRIAGE_BASE with a classification system prompt.
+    # System prompt instructs model to respond with ONLY one of:
+    #   "general", "code", "search", "rag"
+    # Returns the classification string.
+    # Timeout: TRIAGE_TIMEOUT seconds.
+    # On any error: returns "general" (fail-safe).
+
+async def select_node(classification: str) -> dict | None
+    # Consults CLUSTER_NODES from cluster.py
+    # For "code": prefer nodes where active_model name contains "coder" or "qwen"
+    # For "general": prefer nodes where active_model name contains "mistral" or "llama"
+    # For "search" or "rag": return None (handled locally by jC)
+    # If no matching node found: return None (triggers FALLBACK_TO_DEFAULT)
+    # Returns NodeRecord dict for selected node, or None
+
+async def get_inference_url(query: str) -> str
+    # Combines classify_query + select_node
+    # Returns full base URL: f"http://{node.ip}:{node.active_model.port}/v1"
+    # Falls back to LLAMA_SERVER_BASE if classification=search/rag, no nodes, or triage error
+```
+
+**Update `routers/chat.py`:**
+- Replace the hardcoded `LLAMA_SERVER_BASE` reference with a call to `get_inference_url(user_message)`
+- The rest of the chat flow (RAG, memory, streaming) is unchanged — only the inference target URL changes
+
+**Write `tests/test_triage.py`** covering:
+- `classify_query()` returns valid classification — mock Phi-4-mini response
+- `classify_query()` on timeout — assert returns "general", no exception
+- `select_node("code")` with coder node in cluster — assert correct node returned
+- `select_node("general")` with no matching node — assert None returned
+- `get_inference_url()` with code query and coder node available — assert returns node URL
+- `get_inference_url()` with no nodes in cluster — assert returns LLAMA_SERVER_BASE fallback
+
+**Update `tests/test_chat_streaming_and_memory_paths.py`:**
+- Mock `triage.get_inference_url` to return a fixed URL in all existing tests so they continue to pass without a live cluster
+
+Run full test suite. All existing tests must continue to pass.
+
+---
+
+## ~~TASK 14 — Roadmap N6: Model Swap Command Flow [DONE]~~
+
+**Status: Implemented and pushed (`9d1fd44`).** `request_model_swap()`, `handle_model_ready()`, `handle_model_failed()` in `cluster.py`, async `select_node()` with swap triggering in `triage.py`, `tests/test_model_swap.py` (9 tests). 177 tests pass.
+
+This task implements the coordinator-side logic for requesting a model swap on a worker node when the ideal model is not currently active.
+
+**Add to `cluster.py`:**
+
+```python
+async def request_model_swap(node_name: str, model_filename: str) -> bool
+    # Publishes to jc.admin exchange, routing key node.{node_name}.cmd.swap_model
+    # Payload: {model_filename, requested_at: iso_timestamp}
+    # Sets node status to "swapping" in CLUSTER_NODES
+    # Returns True if message published successfully
+
+async def handle_model_ready(message) -> None
+    # Handles node.{node_name}.model_ready from jc.system
+    # Updates CLUSTER_NODES[node_name].active_model to the new model
+    # Sets node status back to "active"
+    # Logs swap completion with timing
+
+async def handle_model_failed(message) -> None
+    # Handles node.{node_name}.model_failed from jc.system
+    # Sets node status to "error" in CLUSTER_NODES
+    # Logs failure with detail from message payload
+```
+
+**Subscribe in `app.py` lifespan:**
+- `jc.system` exchange, routing key `node.*.model_ready` → `handle_model_ready`
+- `jc.system` exchange, routing key `node.*.model_failed` → `handle_model_failed`
+
+**Update `triage.py` `select_node()`:**
+- If the best-matching node exists but its active_model does not match the ideal model for the classification, AND the node status is "active" (not already swapping):
+  - Call `request_model_swap(node_name, ideal_model_filename)`
+  - Return None (triggers fallback) — the swap happens async, next query will find the right model active
+- If node status is "swapping": return None (fallback, swap in progress)
+
+**Update `GET /api/cluster`** to include node status in response.
+
+**Write `tests/test_model_swap.py`** covering:
+- `request_model_swap()` — assert swap command published, node status set to "swapping"
+- `handle_model_ready()` — assert active_model updated, status set to "active"
+- `handle_model_failed()` — assert status set to "error"
+- `select_node()` with mismatched active model — assert swap requested, None returned
+- `select_node()` with node status "swapping" — assert None returned without publishing another swap
+
+Run full test suite. All existing tests must continue to pass.
+
+---
+
+## ~~TASK 15 — Roadmap N7: Cluster Status UI [DONE]~~
+
+Surface cluster awareness in the jC frontend (`templates/index.html`).
+
+**Add a cluster status panel** to the UI. Requirements:
+- Small status bar or collapsible panel, visible but unobtrusive
+- Polls `GET /api/cluster` every 15 seconds
+- For each admitted node: show node name, active model name, and a colored status dot:
+  - Green: active
+  - Yellow: swapping
+  - Red: error or offline (not seen in last 60 seconds based on last_seen timestamp)
+- If no nodes in cluster (empty): show "No worker nodes connected"
+- Panel must not interfere with chat input or conversation list
+
+**Update `GET /api/cluster` response** to include `last_seen` per node and a `status` field (`active`, `swapping`, `error`).
+
+**Update heartbeat handling in `cluster.py`:** add a handler for `node.*.heartbeat` on `jc.system` that updates `last_seen` timestamp for the node.
+
+**Subscribe in `app.py` lifespan:**
+- `jc.system` exchange, routing key `node.*.heartbeat` → `handle_heartbeat`
+
+**Add `handle_heartbeat()` to `cluster.py`:**
+- Updates `CLUSTER_NODES[node_name].last_seen` to current timestamp
+- If node was previously marked offline (not in CLUSTER_NODES), log re-registration warning but do not auto-admit — full registration required
+
+**Write `tests/test_cluster_heartbeat.py`** covering:
+- `handle_heartbeat()` for known node — assert last_seen updated
+- `handle_heartbeat()` for unknown node — assert no crash, warning logged, node not added
+
+Run full test suite. All 26+ existing tests must continue to pass.
+
+~~Commit all changes introduced across Tasks 9–15 with message: `feat: Roadmap N — AMQP cluster nervous system complete`~~
+
+---
+
+## Backlog (Post-Roadmap N) ⏳
+
+### B1 — Context loss in follow-up questions
+
+**Symptom:** After asking "in {context}, explain {b}", a follow-up "what is {b}'s {x}?" gets a non-sequitur response that ignores the original context.
+
+**Diagnosis:** `build_system_prompt()` is called fresh per-request with new RAG/memory results keyed to the current message text. These can change between turns and may dilute or override the conversation history. The original system prompt used for turn 1 (including its RAG context) is not stored in the DB — only user/assistant messages are. The inference server receives a different system prompt each turn.
+
+**Possible fixes:**
+- Store the assembled system prompt with each assistant message in the DB
+- When replaying history, re-send the original system prompts from DB rather than rebuilding
+- Or: cap RAG/memory injection to only fire on the first message of a conversation, then rely solely on conversation history for follow-ups
+- Check that llama-server isn't truncating history due to context window overflow (Mistral-Nemo 12B = 128K context, unlikely)
+
+### B2 — Bang-prefixed search routing
+
+**Spec:** If a query begins with `!`, route to SearXNG search instead of local inference.
+
+**Where:** In `routers/chat.py` `chat()` handler, after `user_message` is extracted. Strip the `!`, set a flag to always trigger auto-search regardless of perplexity/refusal.
+
+**Change:** Add a `force_search` flag when `user_message.startswith("!")`, strip the prefix from the message saved to DB, and route directly to the search+summarize path.
+
+### B3 — Docker distribution (v1.0 gate)
+
+**Goal:** Ship cAIc as a `docker compose` stack so a single command stands up everything.
+
+**Services to containerize:**
+- cAIc (FastAPI app + SQLite)
+- SearXNG
+- Qdrant
+- RabbitMQ
+- llama-server (with optional RPC sidecar for GPU offload)
+- Ollama (embeddings)
+
+**Also needed:**
+- `Dockerfile` for the cAIc app itself
+- `docker-compose.yml` with all services, volumes, networks, env vars
+- Setup wizard script (run on first boot) that:
+  - Probes CPU vs GPU (reuses `hardware.py`)
+  - Queries user for admin PIN, node name, IP
+  - Generates `.env` file with correct `LLAMA_SERVER_BASE`, `EMBED_URL`, etc.
+  - Auto-calculates `RAG_MAX_VECTORS` from available RAM: `max(1000, int(available_ram_gb * 100_000))`
+  - Optionally detects and configures RPC GPU offload
+- Manual install docs remain alongside for bare-metal deployment
+
+**This task is only actionable after Tasks 8–15 (RAG eviction + AMQP cluster) are complete.**
+
+---
+
+### B4 — RAG Corpus Management UI (Display, Edit, CRUD)
+
+**Goal:** Provide a management interface in the UI to browse, search, edit, and delete individual entries in the Qdrant-backed RAG corpus.
+
+**Backend — add to `routers/rag_admin.py`:**
+
+| Method | Endpoint | Description | Auth |
+|--------|----------|-------------|------|
+| GET | `/api/rag/points` | Return paginated list of RAG points with payload (text, source, date). Supports `?offset=0&limit=50&search=` query params | Admin |
+| GET | `/api/rag/point/{point_id}` | Return a single point with full payload | Admin |
+| DELETE | `/api/rag/point/{point_id}` | Delete a single point from Qdrant | Admin |
+| PATCH | `/api/rag/point/{point_id}` | Update a point's text payload (re-embed the new text) | Admin |
+
+Helper functions for Qdrant scroll/delete/update go in `rag.py` or `eviction.py`.
+
+**Frontend — add to `templates/index.html`:**
+
+A "RAG" button in the admin UI (drawer or settings modal) that opens a management panel:
+- **Stats bar**: vector count, max vectors, percent full, pinned sources
+- **Search bar**: text input to search the RAG corpus by semantic similarity
+- **Results table**: paginated list showing each vector's text snippet, source label, ingest date, retrieval count
+  - Click to expand full text
+  - Delete button per row (with confirmation)
+  - Edit button per row (inline text edit → re-embed on save)
+- **Bulk actions**: flush all (existing `/api/rag/flush`) with confirmation
+
+**Tests:**
+
+- `tests/test_rag_admin.py` — cover new endpoints: list, get, delete, update, admin-enforcement
+- Mock all Qdrant calls via monkeypatch
+
+Run full test suite. All existing tests must continue to pass.**

amqp.py

@@ -0,0 +1,141 @@
+"""
+cAIc — AMQP connection manager.
+Single persistent aio-pika connection with auto-reconnect.
+"""
+import asyncio
+import json
+import logging
+
+try:
+    import aio_pika
+    from aio_pika import DeliveryMode, ExchangeType
+    from aio_pika import RobustConnection, RobustChannel
+    HAS_AIO_PIKA = True
+except ImportError:
+    HAS_AIO_PIKA = False
+
+from config import (
+    AMQP_RECONNECT_DELAY,
+    AMQP_EXCHANGE_ADMIN,
+    AMQP_EXCHANGE_SYSTEM,
+    get_amqp_url,
+)
+
+log = logging.getLogger("caic")
+
+_connection = None
+_channel = None
+_lock = asyncio.Lock()
+_subscriptions = []  # list of (exchange, routing_keys, handler_fn)
+
+
+async def subscribe(exchange: str, routing_keys: list[str], handler) -> None:
+    """Subscribe to routing keys on an exchange.
+
+    Creates an exclusive anonymous queue bound to the given routing keys.
+    Handler receives (exchange, routing_key, payload_dict).
+    On reconnect: _rebind_subscriptions recreates all subscriptions.
+    """
+    if not HAS_AIO_PIKA:
+        log.warning("aio-pika not installed — cannot subscribe")
+        return
+    ch = await get_channel()
+    if ch is None:
+        log.error("cannot subscribe — no AMQP channel")
+        return
+    try:
+        queue = await ch.declare_queue("", exclusive=True)
+        ex = await ch.get_exchange(exchange)
+        for rk in routing_keys:
+            await queue.bind(ex, rk)
+
+        async def _dispatch(msg: aio_pika.IncomingMessage):
+            async with msg.process():
+                try:
+                    payload = json.loads(msg.body.decode())
+                    await handler(exchange, msg.routing_key, payload)
+                except Exception:
+                    log.exception("AMQP handler error for %s %s", exchange, msg.routing_key)
+
+        await queue.consume(_dispatch)
+        _subscriptions.append((exchange, routing_keys, handler))
+    except Exception:
+        log.exception("AMQP subscribe failed for %s %s", exchange, routing_keys)
+
+
+async def _rebind_subscriptions() -> None:
+    """Re-create all subscriptions after reconnect."""
+    subs = list(_subscriptions)
+    _subscriptions.clear()
+    for exchange, routing_keys, handler in subs:
+        await subscribe(exchange, routing_keys, handler)
+    if subs:
+        log.info("AMQP subscriptions rebound")
+
+
+async def connect() -> None:
+    if not HAS_AIO_PIKA:
+        log.warning("aio-pika not installed — AMQP disabled")
+        return
+    async with _lock:
+        global _connection, _channel
+        if _connection and not _connection.is_closed:
+            return
+        url = get_amqp_url()
+        log.info("connecting to AMQP broker")
+        try:
+            conn = await aio_pika.connect_robust(url)
+        except Exception as exc:
+            log.warning("AMQP connection failed — %s", exc)
+            return
+        ch = await conn.channel()
+        for ex in (AMQP_EXCHANGE_ADMIN, AMQP_EXCHANGE_SYSTEM):
+            await ch.declare_exchange(ex, ExchangeType.TOPIC, durable=True)
+        _connection = conn
+        _channel = ch
+        await _rebind_subscriptions()
+        log.info("AMQP connected, exchanges declared")
+
+
+async def disconnect() -> None:
+    if not HAS_AIO_PIKA:
+        return
+    async with _lock:
+        global _connection, _channel
+        if _channel and not _channel.is_closed:
+            await _channel.close()
+        if _connection and not _connection.is_closed:
+            await _connection.close()
+        _channel = None
+        _connection = None
+        log.info("AMQP disconnected")
+
+
+async def get_channel():
+    if not HAS_AIO_PIKA:
+        return None
+    if _channel is None or _channel.is_closed:
+        log.warning("AMQP channel missing, attempting reconnect")
+        try:
+            await connect()
+        except Exception:
+            log.exception("AMQP reconnect failed")
+            return None
+    return _channel
+
+
+async def publish(exchange: str, routing_key: str, payload: dict) -> None:
+    if not HAS_AIO_PIKA:
+        log.error("cannot publish — aio-pika not installed")
+        return
+    ch = await get_channel()
+    if ch is None:
+        log.error("cannot publish — no AMQP channel available")
+        return
+    try:
+        body = json.dumps(payload).encode()
+        msg = aio_pika.Message(body, delivery_mode=DeliveryMode.PERSISTENT)
+        ex = await ch.get_exchange(exchange)
+        await ex.publish(msg, routing_key)
+    except Exception:
+        log.exception("AMQP publish failed")

app.py

@@ -1,10 +1,11 @@
 #!/usr/bin/env python3
 """
-JarvisChat - Entry point.
+cAIc - Entry point.
 Creates the FastAPI app, registers middleware, mounts all routers.
 """
 import logging
 import logging.handlers
+import os
 from contextlib import asynccontextmanager
 from pathlib import Path
 
@@ -13,8 +14,11 @@ from fastapi.responses import HTMLResponse, JSONResponse
 from fastapi.staticfiles import StaticFiles
 from fastapi.templating import Jinja2Templates
 
-from config import VERSION, RATE_WINDOW_SECONDS
+from amqp import connect as amqp_connect, disconnect as amqp_disconnect
+from cluster import start_cluster_subscriptions
+from config import VERSION, RATE_WINDOW_SECONDS, UPLOAD_DIR, RAG_MAX_VECTORS, RAG_EVICTION_HIGH_WATER, RAG_EVICTION_LOW_WATER, RAG_EVICTION_BATCH
 from db import init_db
+from hardware import assess_hardware
 from memory import get_memory_count
 from security import (
     get_client_ip, is_ip_allowed, check_rate_limit, rate_policy,
@@ -32,12 +36,17 @@ import routers.skills as skills
 import routers.chat as chat
 import routers.search_route as search_route
 import routers.completions as completions
+import routers.upload as upload
+import routers.ingest as ingest
+import routers.hardware as hardware
+import routers.rag_admin as rag_admin
+import routers.cluster as cluster_router
 
 # --- Logging ---
-log = logging.getLogger("jarvischat")
+log = logging.getLogger("caic")
 log.setLevel(logging.DEBUG)
 syslog_handler = logging.handlers.SysLogHandler(address="/dev/log")
-syslog_handler.setFormatter(logging.Formatter("jarvischat[%(process)d]: %(levelname)s %(message)s"))
+syslog_handler.setFormatter(logging.Formatter("caic[%(process)d]: %(levelname)s %(message)s"))
 log.addHandler(syslog_handler)
 
 BASE_DIR = Path(__file__).parent
@@ -46,14 +55,31 @@ templates = Jinja2Templates(directory=str(BASE_DIR / "templates"))
 
 @asynccontextmanager
 async def lifespan(app: FastAPI):
-    log.info(f"JarvisChat {VERSION} starting up")
+    log.info(f"cAIc {VERSION} starting up")
+    os.makedirs(UPLOAD_DIR, exist_ok=True)
     init_db()
     log.info(f"Memory system: {get_memory_count()} memories loaded")
+    await assess_hardware()
+    await amqp_connect()
+    await start_cluster_subscriptions()
+
+    if RAG_MAX_VECTORS > 0:
+        if RAG_EVICTION_HIGH_WATER <= RAG_EVICTION_LOW_WATER:
+            log.warning(
+                f"RAG_EVICTION_HIGH_WATER={RAG_EVICTION_HIGH_WATER} <= "
+                f"RAG_EVICTION_LOW_WATER={RAG_EVICTION_LOW_WATER} — eviction will never fire"
+            )
+        if RAG_EVICTION_BATCH <= 0:
+            log.warning(f"RAG_EVICTION_BATCH={RAG_EVICTION_BATCH} clamped to 1")
+    else:
+        log.warning("RAG_MAX_VECTORS <= 0 — RAG eviction disabled")
+
     yield
-    log.info("JarvisChat shutting down")
+    log.info("cAIc shutting down")
+    await amqp_disconnect()
 
 
-app = FastAPI(title="JarvisChat", lifespan=lifespan)
+app = FastAPI(title="cAIc", lifespan=lifespan)
 
 
 @app.exception_handler(Exception)
@@ -99,10 +125,10 @@ async def session_auth_middleware(request: Request, call_next):
 
     unauth_paths = {
         "/api/auth/login", "/api/auth/logout", "/api/auth/session",
-        "/api/auth/heartbeat", "/api/auth/guest",
+        "/api/auth/heartbeat", "/api/auth/guest", "/api/ingest", "/api/hardware",
     }
 
-    if path.startswith("/api/") and is_state_changing(request.method):
+    if path.startswith("/api/"):
         if not origin_allowed(request):
             audit_event("origin_check", "denied", ip=ip, role="none",
                         details=f"{request.method} {path}", warning=True)
@@ -138,7 +164,8 @@ async def index(request: Request):
 for router_module in [
     auth_router, conversations.router, memories.router, models.router,
     presets.router, profile.router, settings.router, skills.router,
-    chat.router, search_route.router, completions.router,
+    chat.router, search_route.router, completions.router, upload.router, ingest.router, hardware.router,
+    rag_admin.router, cluster_router.router,
 ]:
     app.include_router(router_module)
 

auth.py

@@ -1,5 +1,5 @@
 """
-JarvisChat - Auth: session management, PIN verification, middleware, auth routes.
+cAIc - Auth: session management, PIN verification, middleware, auth routes.
 """
 import hashlib
 import hmac
@@ -15,13 +15,13 @@ from fastapi.responses import JSONResponse
 from config import SESSION_TIMEOUT_SECONDS, MAX_PIN_ATTEMPTS, PIN_LOCKOUT_SECONDS, RATE_WINDOW_SECONDS
 from db import get_db, get_setting
 from security import (
-    SESSIONS, PIN_ATTEMPTS, SESSION_LOCK, audit_event, get_client_ip,
+    SESSIONS, PIN_ATTEMPTS, SESSION_LOCK, BODY_LIMIT_DEFAULT_BYTES,
-    is_ip_allowed, check_rate_limit, rate_policy, origin_allowed,
+    audit_event, get_client_ip, is_ip_allowed, check_rate_limit,
-    is_state_changing, request_body_limit, read_json_body, hash_pin,
+    rate_policy, origin_allowed, is_state_changing, request_body_limit,
-    customer_error_envelope, log_incident,
+    read_json_body, hash_pin, customer_error_envelope, log_incident,
 )
 
-log = logging.getLogger("jarvischat")
+log = logging.getLogger("caic")
 router = APIRouter()
 
 
@@ -146,7 +146,6 @@ async def auth_guest(request: Request):
 
 @router.post("/api/auth/login")
 async def auth_login(request: Request):
-    from security import BODY_LIMIT_DEFAULT_BYTES
     body = await read_json_body(request, BODY_LIMIT_DEFAULT_BYTES)
     pin = str(body.get("pin", ""))
     ip = get_client_ip(request)
@@ -183,7 +182,6 @@ async def auth_heartbeat(request: Request):
 
 @router.post("/api/auth/logout")
 async def auth_logout(request: Request):
-    from security import BODY_LIMIT_DEFAULT_BYTES
     ip = get_client_ip(request)
     sid = request.headers.get("x-session-id", "").strip()
     role = "none"

cluster.py

@@ -0,0 +1,257 @@
+"""
+cAIc — Cluster protocol implementation.
+Maintains node registry, event log, coordinator state, and ping-based health checks.
+"""
+import asyncio
+import logging
+import uuid
+from collections import deque
+from datetime import datetime, timezone
+
+from amqp import publish, subscribe
+from config import AMQP_EXCHANGE_ADMIN, AMQP_EXCHANGE_SYSTEM
+
+log = logging.getLogger("caic")
+
+CLUSTER_NODES: dict[str, dict] = {}
+CLUSTER_EVENTS: deque = deque(maxlen=1000)
+CLUSTER_COORDINATOR: str | None = None
+_pending_pings: dict[str, tuple[str, asyncio.Event]] = {}
+NODE_NAME: str = "ultron"
+PING_TIMEOUT: float = 5.0
+
+
+def _push_event(category: str, severity: str, node_name: str | None, message: str, details: dict | None = None) -> dict:
+    record = {
+        "category": category,
+        "severity": severity,
+        "node": node_name,
+        "message": message,
+        "details": details or {},
+        "timestamp": datetime.now(timezone.utc).isoformat() + "Z",
+    }
+    CLUSTER_EVENTS.append(record)
+    level = {"info": logging.INFO, "warn": logging.WARNING, "error": logging.ERROR, "critical": logging.CRITICAL}.get(severity, logging.INFO)
+    log.log(level, "[cluster] %s: %s", node_name or "system", message)
+    return record
+
+
+async def handle_registration(exchange: str, routing_key: str, payload: dict) -> None:
+    global CLUSTER_COORDINATOR
+    node_name = payload.get("node_name", routing_key.split(".")[1] if "." in routing_key else "unknown")
+    node_type = payload.get("node_type", "worker")
+
+    if node_name in CLUSTER_NODES:
+        _push_event("cluster", "warn", node_name, "Duplicate registration rejected")
+        await publish(AMQP_EXCHANGE_ADMIN, f"node.{node_name}.rejected", {
+            "from": NODE_NAME, "node_name": node_name, "type": "rejected",
+            "reason": "duplicate_node_name",
+            "timestamp": datetime.now(timezone.utc).isoformat() + "Z",
+        })
+        return
+
+    if "node_type" not in payload or "capabilities" not in payload:
+        _push_event("cluster", "warn", node_name, "Malformed registration rejected")
+        await publish(AMQP_EXCHANGE_ADMIN, f"node.{node_name}.rejected", {
+            "from": NODE_NAME, "node_name": node_name, "type": "rejected",
+            "reason": "malformed_payload",
+            "timestamp": datetime.now(timezone.utc).isoformat() + "Z",
+        })
+        return
+
+    now = datetime.now(timezone.utc).isoformat() + "Z"
+    node = {
+        "name": node_name,
+        "type": node_type,
+        "status": "active",
+        "ip": payload.get("ip"),
+        "capabilities": payload.get("capabilities", []),
+        "active_model": payload.get("active_model"),
+        "inventory": payload.get("inventory", []),
+        "load": payload.get("load"),
+        "registered_at": now,
+        "last_seen": now,
+    }
+    CLUSTER_NODES[node_name] = node
+    _push_event("cluster", "info", node_name, f"Node registered (type={node_type})")
+
+    await publish(AMQP_EXCHANGE_ADMIN, f"node.{node_name}.admitted", {
+        "from": NODE_NAME, "node_name": node_name, "type": "admitted",
+        "coordinator": CLUSTER_COORDINATOR,
+        "timestamp": now,
+    })
+
+    if node_type == "coordinator" and CLUSTER_COORDINATOR is None:
+        CLUSTER_COORDINATOR = node_name
+        _push_event("cluster", "info", node_name, "Elected as coordinator")
+        await publish(AMQP_EXCHANGE_SYSTEM, "cluster.coordinator.response", {
+            "from": NODE_NAME, "type": "coord_response",
+            "coordinator": node_name, "nodes": list(CLUSTER_NODES.keys()),
+            "timestamp": now,
+        })
+
+
+async def handle_deregistration(exchange: str, routing_key: str, payload: dict) -> None:
+    global CLUSTER_COORDINATOR
+    node_name = payload.get("node_name", routing_key.split(".")[1] if "." in routing_key else "unknown")
+
+    removed = CLUSTER_NODES.pop(node_name, None)
+    if not removed:
+        log.warning("[cluster] deregistration for unknown node %s", node_name)
+        return
+
+    _push_event("cluster", "info", node_name, "Node deregistered")
+
+    if CLUSTER_COORDINATOR == node_name:
+        CLUSTER_COORDINATOR = None
+        _push_event("cluster", "warn", node_name, "Coordinator deregistered — no coordinator active")
+
+
+async def handle_pong(exchange: str, routing_key: str, payload: dict) -> None:
+    node_name = payload.get("node_name", routing_key.split(".")[1] if "." in routing_key else "unknown")
+    correlation_id = payload.get("correlation_id")
+
+    if correlation_id and correlation_id in _pending_pings:
+        _, event = _pending_pings.pop(correlation_id)
+        event.set()
+
+    if node_name in CLUSTER_NODES:
+        now = datetime.now(timezone.utc).isoformat() + "Z"
+        CLUSTER_NODES[node_name]["last_seen"] = now
+        if "status" in payload:
+            CLUSTER_NODES[node_name]["status"] = payload["status"]
+        if "active_model" in payload:
+            CLUSTER_NODES[node_name]["active_model"] = payload["active_model"]
+        if "load" in payload:
+            CLUSTER_NODES[node_name]["load"] = payload["load"]
+    else:
+        log.warning("[cluster] pong from unknown node %s", node_name)
+
+
+async def handle_event(exchange: str, routing_key: str, payload: dict) -> None:
+    node_name = payload.get("node_name", routing_key.split(".")[1] if "." in routing_key else "unknown")
+    severity = payload.get("severity", "info")
+    message = payload.get("message", "")
+    details = payload.get("details")
+    _push_event("application", severity, node_name, message, details)
+
+
+async def handle_coordinator_query(exchange: str, routing_key: str, payload: dict) -> None:
+    if CLUSTER_COORDINATOR is None:
+        return
+    now = datetime.now(timezone.utc).isoformat() + "Z"
+    await publish(AMQP_EXCHANGE_SYSTEM, "cluster.coordinator.response", {
+        "from": NODE_NAME, "type": "coord_response",
+        "coordinator": CLUSTER_COORDINATOR,
+        "nodes": list(CLUSTER_NODES.keys()),
+        "timestamp": now,
+    })
+
+
+async def ping_node(node_name: str) -> bool:
+    global CLUSTER_COORDINATOR
+    if node_name not in CLUSTER_NODES:
+        return False
+
+    correlation_id = str(uuid.uuid4())
+    event = asyncio.Event()
+    _pending_pings[correlation_id] = (node_name, event)
+
+    now = datetime.now(timezone.utc).isoformat() + "Z"
+    await publish(AMQP_EXCHANGE_ADMIN, f"node.{node_name}.ping", {
+        "from": NODE_NAME, "node_name": node_name, "type": "ping",
+        "correlation_id": correlation_id, "timestamp": now,
+    })
+
+    try:
+        await asyncio.wait_for(event.wait(), timeout=PING_TIMEOUT)
+        return True
+    except asyncio.TimeoutError:
+        _pending_pings.pop(correlation_id, None)
+        CLUSTER_NODES.pop(node_name, None)
+        _push_event("cluster", "warn", node_name, "Node unresponsive — deregistered after ping timeout")
+        if CLUSTER_COORDINATOR == node_name:
+            CLUSTER_COORDINATOR = None
+            _push_event("cluster", "warn", node_name, "Coordinator unresponsive — no coordinator active")
+        return False
+
+
+async def request_model_swap(node_name: str, model_filename: str) -> bool:
+    """Request a worker node to swap its active model."""
+    if node_name not in CLUSTER_NODES:
+        log.warning("request_model_swap: unknown node %s", node_name)
+        return False
+
+    CLUSTER_NODES[node_name]["status"] = "swapping"
+    _push_event("cluster", "info", node_name, f"Model swap requested: {model_filename}")
+
+    now = datetime.now(timezone.utc).isoformat() + "Z"
+    await publish(AMQP_EXCHANGE_ADMIN, f"node.{node_name}.cmd.swap_model", {
+        "model_filename": model_filename,
+        "requested_at": now,
+    })
+    return True
+
+
+async def handle_model_ready(exchange: str, routing_key: str, payload: dict) -> None:
+    node_name = payload.get("node_name", routing_key.split(".")[1] if "." in routing_key else "unknown")
+    model_filename = payload.get("active_model")
+    port = payload.get("port", 8081)
+
+    if node_name not in CLUSTER_NODES:
+        log.warning("handle_model_ready: unknown node %s", node_name)
+        return
+
+    inventory = CLUSTER_NODES[node_name].get("inventory") or []
+    model_info = None
+    for inv in inventory:
+        if inv.get("filename") == model_filename:
+            model_info = {**inv, "port": port}
+            break
+    if model_info is None:
+        model_info = {"filename": model_filename, "port": port}
+
+    now = datetime.now(timezone.utc).isoformat() + "Z"
+    CLUSTER_NODES[node_name]["active_model"] = model_info
+    CLUSTER_NODES[node_name]["status"] = "active"
+    CLUSTER_NODES[node_name]["last_seen"] = now
+    _push_event("cluster", "info", node_name, f"Model swap complete: {model_filename}")
+
+
+async def handle_heartbeat(exchange: str, routing_key: str, payload: dict) -> None:
+    node_name = payload.get("node_name", routing_key.split(".")[1] if "." in routing_key else "unknown")
+    if node_name not in CLUSTER_NODES:
+        log.warning("heartbeat from unknown node %s — ignore, full registration required", node_name)
+        return
+    now = datetime.now(timezone.utc).isoformat() + "Z"
+    CLUSTER_NODES[node_name]["last_seen"] = now
+
+
+async def handle_model_failed(exchange: str, routing_key: str, payload: dict) -> None:
+    node_name = payload.get("node_name", routing_key.split(".")[1] if "." in routing_key else "unknown")
+    error = payload.get("error", "unknown error")
+
+    if node_name not in CLUSTER_NODES:
+        log.warning("handle_model_failed: unknown node %s", node_name)
+        return
+
+    CLUSTER_NODES[node_name]["status"] = "error"
+    _push_event("cluster", "error", node_name, f"Model swap failed: {error}")
+
+
+SUBSCRIBE_TABLE = [
+    (AMQP_EXCHANGE_ADMIN, ["node.*.register"], handle_registration),
+    (AMQP_EXCHANGE_ADMIN, ["node.*.deregister"], handle_deregistration),
+    (AMQP_EXCHANGE_ADMIN, ["node.*.pong"], handle_pong),
+    (AMQP_EXCHANGE_SYSTEM, ["node.*.event"], handle_event),
+    (AMQP_EXCHANGE_SYSTEM, ["cluster.coordinator.query"], handle_coordinator_query),
+    (AMQP_EXCHANGE_SYSTEM, ["node.*.heartbeat"], handle_heartbeat),
+    (AMQP_EXCHANGE_SYSTEM, ["node.*.model_ready"], handle_model_ready),
+    (AMQP_EXCHANGE_SYSTEM, ["node.*.model_failed"], handle_model_failed),
+]
+
+
+async def start_cluster_subscriptions() -> None:
+    for exchange, routing_keys, handler in SUBSCRIBE_TABLE:
+        await subscribe(exchange, routing_keys, handler)
+    log.info("cluster subscriptions started")

config.py

@@ -1,5 +1,5 @@
 """
-JarvisChat - Central configuration.
+cAIc - Central configuration.
 All constants, environment variables, limits, and skill registry live here.
 """
 import os
@@ -7,28 +7,47 @@ import re
 import ipaddress
 import logging
 
-log = logging.getLogger("jarvischat")
+log = logging.getLogger("caic")
 
-VERSION = "v1.8.0"
+VERSION = "v0.18.0"
 OLLAMA_BASE = os.environ.get("OLLAMA_BASE", "http://localhost:11434")
 LLAMA_SERVER_BASE = os.environ.get("LLAMA_SERVER_BASE", "http://192.168.50.108:8081")
 SEARXNG_BASE = "http://localhost:8888"
-DEFAULT_MODEL = "llama3.1:latest"
+DEFAULT_MODEL = "qwen2.5-7b-instruct"
+COMPLETIONS_API_KEY = os.environ.get("CAIC_COMPLETIONS_API_KEY", "caic-sk-" + os.urandom(24).hex())
+MODEL_CONTEXT_LENGTH = 4096
+
+# --- AMQP ---
+AMQP_RECONNECT_DELAY = 5
+AMQP_EXCHANGE_ADMIN = "jc.admin"
+AMQP_EXCHANGE_SYSTEM = "jc.system"
+AMQP_SECRET_PATH = "/home/gramps/.caic_amqp_secret"
+
+def get_amqp_url() -> str:
+    url = os.environ.get("CAIC_AMQP_URL")
+    if url:
+        return url
+    try:
+        with open(AMQP_SECRET_PATH) as f:
+            pw = f.read().strip()
+    except (FileNotFoundError, OSError):
+        pw = "password"
+    return f"amqp://caic:{pw}@localhost:5672/caic"
 
 # --- Auth ---
-SESSION_TIMEOUT_SECONDS = 90
+SESSION_TIMEOUT_SECONDS = 3600
 MAX_PIN_ATTEMPTS = 5
 PIN_LOCKOUT_SECONDS = 300
-ALLOW_DEFAULT_PIN = os.getenv("JARVISCHAT_ALLOW_DEFAULT_PIN", "false").lower() == "true"
+ALLOW_DEFAULT_PIN = os.getenv("CAIC_ALLOW_DEFAULT_PIN", "false").lower() == "true"
 TRUSTED_ORIGINS = {
     origin.strip().rstrip("/")
-    for origin in os.getenv("JARVISCHAT_TRUSTED_ORIGINS", "").split(",")
+    for origin in os.getenv("CAIC_TRUSTED_ORIGINS", "").split(",")
     if origin.strip()
 }
 DEFAULT_ALLOWED_CIDRS = "127.0.0.0/8,::1/128,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16"
-ALLOWED_CIDRS_RAW = os.getenv("JARVISCHAT_ALLOWED_CIDRS", DEFAULT_ALLOWED_CIDRS)
+ALLOWED_CIDRS_RAW = os.getenv("CAIC_ALLOWED_CIDRS", DEFAULT_ALLOWED_CIDRS)
 TRUST_X_FORWARDED_FOR = (
-    os.getenv("JARVISCHAT_TRUST_X_FORWARDED_FOR", "false").lower() == "true"
+    os.getenv("CAIC_TRUST_X_FORWARDED_FOR", "false").lower() == "true"
 )
 
 # --- Rate limits ---
@@ -45,6 +64,25 @@ BODY_LIMIT_DEFAULT_BYTES = 64 * 1024
 BODY_LIMIT_CHAT_BYTES = 128 * 1024
 BODY_LIMIT_PROFILE_BYTES = 256 * 1024
 
+# --- Upload ---
+UPLOAD_DIR = "/tmp/caic_uploads"
+MAX_UPLOAD_BYTES = 20 * 1024 * 1024
+SUPPORTED_UPLOAD_TYPES = {"text/plain", "text/markdown", "application/pdf", "application/json", "text/x-python", "text/html", "image/png", "image/jpeg", "image/gif", "image/svg+xml", "image/webp"}
+QDRANT_URL = "http://192.168.50.108:6333"
+RAG_COLLECTION = "caic_rag"
+UPLOAD_CONTEXT_EXPIRY_HOURS = 1
+BODY_LIMIT_UPLOAD_BYTES = MAX_UPLOAD_BYTES
+
+# --- RAG eviction ---
+RAG_MAX_VECTORS = 50000
+RAG_EVICTION_HIGH_WATER = 0.80
+RAG_EVICTION_LOW_WATER = 0.20
+RAG_EVICTION_BATCH = 1000
+RAG_PINNED_SOURCES = ["upload", "profile"]
+RAG_GRACE_HOURS = 1
+RAG_ACCESS_WEIGHT = 1.0
+RAG_AGE_WEIGHT = 0.1
+
 MAX_CHAT_MESSAGE_CHARS = 8000
 MAX_SEARCH_QUERY_CHARS = 500
 MAX_PROFILE_CHARS = 32000
@@ -65,6 +103,11 @@ ALLOWED_SETTINGS_KEYS = {
     "skills_enabled",
 }
 
+# --- Triage / query routing ---
+TRIAGE_BASE = os.environ.get("CAIC_TRIAGE_BASE", "http://127.0.0.1:8083/v1")
+TRIAGE_TIMEOUT = 10
+FALLBACK_TO_DEFAULT = True
+
 # --- Perplexity ---
 PERPLEXITY_THRESHOLD = 15.0
 

db.py

@@ -1,5 +1,5 @@
 """
-JarvisChat - Database layer.
+cAIc - Database layer.
 Schema init, connection factory, settings helpers, skill state management.
 """
 import logging
@@ -16,10 +16,10 @@ from config import (
     MAX_SKILL_PROMPT_CHARS, ALLOWED_NETWORKS,
 )
 
-log = logging.getLogger("jarvischat")
+log = logging.getLogger("caic")
 
 BASE_DIR = Path(__file__).parent
-DB_PATH = BASE_DIR / "jarvischat.db"
+DB_PATH = BASE_DIR / "caic.db"
 
 
 def get_db():
@@ -68,6 +68,43 @@ def format_active_skills_prompt(skills: list) -> str:
     return text
 
 
+def insert_upload_context(db, conversation_id: str, filename: str, content: str, expires_at: str, content_type: str = "text/plain") -> int:
+    now = datetime.now(timezone.utc).isoformat()
+    cur = db.execute(
+        "INSERT INTO upload_context (conversation_id, filename, content, content_type, created_at, expires_at) VALUES (?, ?, ?, ?, ?, ?)",
+        (conversation_id, filename, content, content_type, now, expires_at),
+    )
+    return cur.lastrowid
+
+
+def list_upload_context_by_conversation(db, conversation_id: str):
+    rows = db.execute(
+        "SELECT id, conversation_id, filename, content_type, created_at, expires_at FROM upload_context WHERE conversation_id = ? ORDER BY id ASC",
+        (conversation_id,),
+    ).fetchall()
+    return [dict(r) for r in rows]
+
+
+def delete_upload_context_by_id(db, context_id: int) -> bool:
+    cur = db.execute("DELETE FROM upload_context WHERE id = ?", (context_id,))
+    return cur.rowcount > 0
+
+
+def get_upload_context(db, context_id: int):
+    row = db.execute(
+        "SELECT id, conversation_id, filename, content, content_type, expires_at FROM upload_context WHERE id = ?",
+        (context_id,),
+    ).fetchone()
+    if not row:
+        return None
+    expires = datetime.fromisoformat(row["expires_at"])
+    if expires < datetime.now(timezone.utc):
+        db.execute("DELETE FROM upload_context WHERE id = ?", (context_id,))
+        db.commit()
+        return None
+    return dict(row)
+
+
 def init_db():
     from security import hash_pin
     conn = sqlite3.connect(DB_PATH)
@@ -108,6 +145,21 @@ def init_db():
             fact, topic, source, created_at UNINDEXED
         )
     """)
+    conn.execute("""
+        CREATE TABLE IF NOT EXISTS upload_context (
+            id INTEGER PRIMARY KEY AUTOINCREMENT,
+            conversation_id TEXT,
+            filename TEXT NOT NULL,
+            content TEXT NOT NULL,
+            content_type TEXT DEFAULT 'text/plain',
+            created_at TEXT NOT NULL,
+            expires_at TEXT NOT NULL
+        )
+    """)
+    try:
+        conn.execute("ALTER TABLE upload_context ADD COLUMN content_type TEXT DEFAULT 'text/plain'")
+    except Exception:
+        pass
 
     if not conn.execute("SELECT id FROM profile WHERE id = 1").fetchone():
         now = datetime.now(timezone.utc).isoformat()
@@ -138,15 +190,15 @@ def init_db():
     existing_pin_salt = conn.execute("SELECT value FROM settings WHERE key = 'admin_pin_salt'").fetchone()
     if not existing_pin_hash or not existing_pin_salt:
         from config import ALLOW_DEFAULT_PIN
-        configured_pin = os.getenv("JARVISCHAT_ADMIN_PIN", "").strip()
+        configured_pin = os.getenv("CAIC_ADMIN_PIN", "").strip()
         if re.fullmatch(r"\d{4}", configured_pin):
             seed_pin, pin_source = configured_pin, "env"
         elif ALLOW_DEFAULT_PIN:
             seed_pin, pin_source = "1234", "default"
         else:
             raise RuntimeError(
-                "Admin PIN bootstrap blocked: set JARVISCHAT_ADMIN_PIN to a 4-digit PIN "
+                "Admin PIN bootstrap blocked: set CAIC_ADMIN_PIN to a 4-digit PIN "
-                "or set JARVISCHAT_ALLOW_DEFAULT_PIN=true."
+                "or set CAIC_ALLOW_DEFAULT_PIN=true."
             )
         salt_hex, pin_hash_hex = hash_pin(seed_pin)
         conn.execute("INSERT OR REPLACE INTO settings (key, value) VALUES (?, ?)", ("admin_pin_hash", pin_hash_hex))

docker.md

@@ -0,0 +1,750 @@
+# Docker Distribution — Architecture & Planning
+
+> **Part of B3 (v1.0 gate).** This document catalogs every service, volume, port, configuration, and decision needed to ship cAIc as a `docker compose` stack. It also defines extraction (setup) and back-out (uninstall) procedures so nothing is lost when reality disagrees with the plan.
+
+## 1. Stack Overview
+
+```
+┌─────────────────────────────────────────────────────────┐
+│                    docker compose stack                  │
+│                                                         │
+│  ┌────────────┐  ┌──────────┐  ┌────────────────────┐  │
+│  │  SearXNG    │  │  Qdrant  │  │  RabbitMQ          │  │
+│  │  :8888      │  │  :6333   │  │  :5672 / :15672    │  │
+│  └──────┬──────┘  └────┬─────┘  └────────┬───────────┘  │
+│         │              │                  │              │
+│         ▼              ▼                  ▼              │
+│  ┌──────────────────────────────────────────────────┐   │
+│  │              cAIc (FastAPI)                     │   │
+│  │              :8080 (HTTP)                        │   │
+│  │                                                  │   │
+│  │  SQLite  ◄── caic.db                  (volume)    │   │
+│  │  Uploads ◄── /app/uploads            (volume)    │   │
+│  └──────────┬──────────────┬───────────────────────┘   │
+│             │              │                            │
+│             ▼              ▼                            │
+│  ┌──────────────┐  ┌──────────────┐                    │
+│  │ llama-server │  │   Ollama     │                    │
+│  │ :8081        │  │ :11434       │                    │
+│  │ (GPU/RPC)    │  │ (embeddings) │                    │
+│  └──────────────┘  └──────────────┘                    │
+└─────────────────────────────────────────────────────────┘
+```
+
+> **This compose stack defines the coordinator.** A coordinator runs cAIc, the broker, and optional infrastructure services. Workers (headless inference nodes) do not use Docker — they install just llama-server + a Python node agent. See §9 for the worker deployment model.
+
+### Service roles
+
+| Service | Image | Role |
+|---------|-------|------|
+| **cAIc** | Custom `Dockerfile` | FastAPI app serving UI + API |
+| **SearXNG** | `searxng/searxng:latest` | Privacy-respecting web search |
+| **Qdrant** | `qdrant/qdrant:latest` | Vector database for RAG |
+| **RabbitMQ** | `rabbitmq:4-management` | Message broker for AMQP cluster |
+| **llama-server** | `ghcr.io/ggml-org/llama.cpp:server` | LLM inference (OpenAI-compat API) |
+| **Ollama** | `ollama/ollama:latest` | Embeddings for RAG chunk vectors |
+
+### Non-containerized (host-level)
+
+| Component | Reason |
+|-----------|--------|
+| AMD GPU driver + ROCm | Kernel access required for GPU compute |
+| llama.cpp RPC workers | Runs on *other* hosts — not on the Docker host |
+| `rocm-smi` | Hardware stats — not needed for core function |
+| `psutil` | Already inside the container via pip |
+
+---
+
+## 2. Service Catalog
+
+### 2.1 cAIc (FastAPI app)
+
+**Image:** `caic:latest` (built from `Dockerfile`)
+
+**Ports:**
+| Container | Host | Purpose |
+|-----------|------|---------|
+| 8080 | 8080 | HTTP API + UI |
+
+**Volumes:**
+| Container path | Type | Purpose |
+|----------------|------|---------|
+| `/app/caic.db` | named volume `caic_data` | SQLite database |
+| `/app/uploads` | named volume `caic_uploads` | Uploaded files |
+| `/app/hardware_state.json` | (inside volume) | Cached hardware probe |
+
+**Dependencies:** Wait for SearXNG, Qdrant, RabbitMQ, llama-server, Ollama before serving.
+
+**Restart:** `unless-stopped`
+
+**Healthcheck:** `curl -f http://localhost:8080/`
+
+### 2.2 SearXNG
+
+**Image:** `searxng/searxng:latest`
+
+**Ports:**
+| Container | Host | Purpose |
+|-----------|------|---------|
+| 8080 | 8888 | Search API |
+
+**Volumes:**
+| Container path | Type | Purpose |
+|----------------|------|---------|
+| `/etc/searxng` | named volume `searxng_config` | `settings.yml` |
+
+**Environment:**
+```env
+SEARXNG_BASE_URL=https://localhost:8888
+```
+
+**Config override (`/etc/searxng/settings.yml`):**
+```yaml
+search:
+  safe_search: 0
+  autocomplete: ""
+server:
+  secret_key: ${SEARXNG_SECRET_KEY}
+  limiter: false
+  image_proxy: false
+  method: GET
+  port: 8080
+  bind_address: "0.0.0.0"
+```
+
+**Restart:** `unless-stopped`
+
+### 2.3 Qdrant
+
+**Image:** `qdrant/qdrant:latest`
+
+**Ports:**
+| Container | Host | Purpose |
+|-----------|------|---------|
+| 6333 | 6333 | HTTP API |
+| 6334 | — | gRPC (internal only) |
+
+**Volumes:**
+| Container path | Type | Purpose |
+|----------------|------|---------|
+| `/qdrant/storage` | named volume `qdrant_storage` | Vector index data |
+
+**Environment:**
+```env
+QDRANT__SERVICE__GRPC_PORT=6334
+```
+
+**Restart:** `unless-stopped`
+
+### 2.4 RabbitMQ
+
+**Image:** `rabbitmq:4-management`
+
+**Ports:**
+| Container | Host | Purpose |
+|-----------|------|---------|
+| 5672 | 5672 | AMQP messaging |
+| 15672 | — | Management UI (internal only) |
+
+**Volumes:**
+| Container path | Type | Purpose |
+|----------------|------|---------|
+| `/var/lib/rabbitmq` | named volume `rabbitmq_data` | Message store |
+
+**Environment:**
+```env
+RABBITMQ_DEFAULT_USER=caic
+RABBITMQ_DEFAULT_PASS_FILE=/run/secrets/rabbitmq_password
+RABBITMQ_DEFAULT_VHOST=/
+```
+
+**Restart:** `unless-stopped`
+
+### 2.5 llama-server
+
+**Image:** `ghcr.io/ggml-org/llama.cpp:server`
+
+**Ports:**
+| Container | Host | Purpose |
+|-----------|------|---------|
+| 8081 | 8081 | OpenAI-compat API |
+
+**Volumes:**
+| Container path | Type | Purpose |
+|----------------|------|---------|
+| `/models` | bind mount `./models` | Model GGUF files |
+
+**Environment:**
+```env
+LLAMA_ARG_MODEL=/models/<model-file>
+LLAMA_ARG_N_GPU_LAYERS=0              # set >0 for GPU offload
+LLAMA_ARG_MAIN_GPU=0
+LLAMA_ARG_CTX_SIZE=4096
+LLAMA_ARG_HOST=0.0.0.0
+LLAMA_ARG_PORT=8081
+LLAMA_ARG_EMBEDDINGS=1
+LLAMA_ARG_LOGPROBS=1
+LLAMA_ARG_RPC=                         # optional: comma-separated RPC endpoints
+```
+
+**Restart:** `unless-stopped`
+
+**Healthcheck:** `curl -f http://localhost:8081/health`
+
+**Notes:**
+- Models directory bind mount — user places `.gguf` files in `./models/` on the host
+- RPC offload to other machines (e.g., `10.0.0.50:50052,10.0.0.51:50052`)
+- If no GPU, set `LLAMA_ARG_N_GPU_LAYERS=0` for CPU-only
+- `LLAMA_ARG_EMBEDDINGS=1` required for perplexity scoring
+- `LLAMA_ARG_LOGPROBS=1` required for auto-search trigger
+
+### 2.6 Ollama
+
+**Image:** `ollama/ollama:latest`
+
+**Ports:**
+| Container | Host | Purpose |
+|-----------|------|---------|
+| 11434 | 11434 | Embeddings API |
+
+**Volumes:**
+| Container path | Type | Purpose |
+|----------------|------|---------|
+| `/root/.ollama` | named volume `ollama_models` | Pulled model blobs |
+
+**Restart:** `unless-stopped`
+
+**Notes:**
+- Used exclusively for embeddings (`/api/embeddings`), not inference
+- Typically needs a small model like `all-minilm:latest` or `nomic-embed-text:latest`
+- Consider replacing Ollama with llama-server's built-in embedding if it supports the same model — would remove one container
+
+---
+
+## 3. Configuration Management
+
+### 3.1 `.env` file (generated by setup wizard)
+
+```env
+# --- Secrets (auto-generated, change before production) ---
+CAIC_ADMIN_PIN=
+CAIC_COMPLETIONS_API_KEY=
+CAIC_ALLOW_DEFAULT_PIN=false
+RABBITMQ_PASSWORD=
+SEARXNG_SECRET_KEY=
+
+# --- Host discovery (auto-detected by setup wizard) ---
+LLAMA_SERVER_BASE=http://llama-server:8081
+OLLAMA_BASE=http://ollama:11434
+SEARXNG_BASE=http://searxng:8888
+QDRANT_URL=http://qdrant:6333
+RABBITMQ_HOST=rabbitmq
+RABBITMQ_PORT=5672
+
+# --- Performance tuning (calculated by setup wizard) ---
+RAG_MAX_VECTORS=50000
+RAG_EVICTION_HIGH_WATER=0.80
+RAG_EVICTION_LOW_WATER=0.20
+RAG_EVICTION_BATCH=1000
+
+# --- llama-server options ---
+LLAMA_MODEL=llama3.1-8b-instruct.Q4_K_M.gguf
+LLAMA_N_GPU_LAYERS=0
+LLAMA_RPC_ENDPOINTS=
+LLAMA_CTX_SIZE=4096
+
+# --- Ollama ---
+OLLAMA_EMBED_MODEL=all-minilm:latest
+
+# --- Network ---
+CAIC_ALLOWED_CIDRS=127.0.0.0/8,::1/128,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16
+CAIC_TRUSTED_ORIGINS=
+CAIC_TRUST_X_FORWARDED_FOR=false
+```
+
+### 3.2 Mapping of config.py → .env variable
+
+Every config.py default that references an external service must accept a matching env var at runtime:
+
+| config.py constant | .env variable | Service |
+|-------------------|---------------|---------|
+| `LLAMA_SERVER_BASE` | `LLAMA_SERVER_BASE` | llama-server |
+| `OLLAMA_BASE` | `OLLAMA_BASE` | Ollama |
+| `SEARXNG_BASE` | `SEARXNG_BASE` | SearXNG |
+| `QDRANT_URL` | `QDRANT_URL` | Qdrant |
+| `COMPLETIONS_API_KEY` | `CAIC_COMPLETIONS_API_KEY` | — |
+| `ALLOWED_CIDRS_RAW` | `CAIC_ALLOWED_CIDRS` | — |
+| `TRUST_X_FORWARDED_FOR` | `CAIC_TRUST_X_FORWARDED_FOR` | — |
+| `TRUSTED_ORIGINS` | `CAIC_TRUSTED_ORIGINS` | — |
+| `RAG_MAX_VECTORS` | `RAG_MAX_VECTORS` | — (calc'd from RAM) |
+
+### 3.3 Secrets management
+
+| Secret | Generated by | Stored in | Mounted to |
+|--------|-------------|-----------|------------|
+| `CAIC_ADMIN_PIN` | User prompt | `.env` | cAIc container |
+| `CAIC_COMPLETIONS_API_KEY` | Auto-generated, shown to user | `.env` | cAIc container |
+| `RABBITMQ_PASSWORD` | Auto-generated | `.env` + Docker secret | RabbitMQ container |
+| `SEARXNG_SECRET_KEY` | Auto-generated | `.env` | SearXNG container |
+
+**Docker secrets approach:** Use `secrets:` in compose file for RabbitMQ password (mounted as file) rather than passing via env var, since `settings.yml` in SearXNG and RabbitMQ config can reference file-based secrets without env-var leakage.
+
+### 3.4 Dockerfile for cAIc
+
+```dockerfile
+FROM python:3.13-slim-bookworm AS builder
+WORKDIR /app
+COPY requirements.txt .
+RUN pip install --no-cache-dir -r requirements.txt
+
+FROM python:3.13-slim-bookworm
+WORKDIR /app
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    curl && \
+    rm -rf /var/lib/apt/lists/*
+COPY --from=builder /usr/local/lib/python3.13/site-packages /usr/local/lib/python3.13/site-packages
+COPY --from=builder /usr/local/bin /usr/local/bin
+COPY . .
+
+EXPOSE 8080
+HEALTHCHECK --interval=30s --timeout=10s --start-period=15s --retries=3 \
+    CMD curl -f http://localhost:8080/ || exit 1
+
+CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8080"]
+```
+
+**Multi-stage rationale:** First stage compiles/bundles packages (wheels), final stage is minimal. Devs can skip builder with `--target builder` for live-reload with volume mount.
+
+---
+
+## 4. docker-compose.yml structure
+
+```yaml
+services:
+  caic:
+    build: .
+    ports: ["8080:8080"]
+    volumes:
+      - caic_data:/app/caic.db
+      - caic_uploads:/app/uploads
+    env_file: .env
+    depends_on:
+      searxng: { condition: service_started }
+      qdrant: { condition: service_started }
+      rabbitmq: { condition: service_healthy }
+      llama-server: { condition: service_healthy }
+      ollama: { condition: service_started }
+    restart: unless-stopped
+
+  searxng:
+    image: searxng/searxng:latest
+    ports: ["8888:8080"]
+    volumes:
+      - ./searxng/settings.yml:/etc/searxng/settings.yml:ro
+      - searxng_config:/etc/searxng
+    env_file: .env
+    restart: unless-stopped
+
+  qdrant:
+    image: qdrant/qdrant:latest
+    ports: ["6333:6333"]
+    volumes:
+      - qdrant_storage:/qdrant/storage
+    restart: unless-stopped
+
+  rabbitmq:
+    image: rabbitmq:4-management
+    ports: ["5672:5672"]
+    volumes:
+      - rabbitmq_data:/var/lib/rabbitmq
+    env_file: .env
+    secrets:
+      - rabbitmq_password
+    healthcheck:
+      test: ["CMD", "rabbitmq-diagnostics", "check_port_connectivity"]
+      interval: 15s
+      timeout: 5s
+      retries: 3
+    restart: unless-stopped
+
+  llama-server:
+    image: ghcr.io/ggml-org/llama.cpp:server
+    ports: ["8081:8081"]
+    volumes:
+      - ./models:/models:ro
+    env_file: .env
+    command: >
+      --model /models/${LLAMA_MODEL}
+      --host 0.0.0.0 --port 8081
+      --ctx-size ${LLAMA_CTX_SIZE:-4096}
+      --n-gpu-layers ${LLAMA_N_GPU_LAYERS:-0}
+      --embeddings
+      --logprobs
+      ${LLAMA_RPC_ENDPOINTS:+--rpc ${LLAMA_RPC_ENDPOINTS}}
+    healthcheck:
+      test: ["CMD", "curl", "-f", "http://localhost:8081/health"]
+      interval: 30s
+      timeout: 10s
+      retries: 5
+      start_period: 60s
+    deploy:
+      resources:
+        reservations:
+          devices:
+            - driver: nvidia
+              count: all
+              capabilities: [gpu]
+    restart: unless-stopped
+
+  ollama:
+    image: ollama/ollama:latest
+    ports: ["11434:11434"]
+    volumes:
+      - ollama_models:/root/.ollama
+    healthcheck:
+      test: ["CMD", "ollama", "list"]
+      interval: 30s
+      timeout: 10s
+      retries: 3
+    deploy:
+      resources:
+        reservations:
+          devices:
+            - driver: nvidia
+              count: all
+              capabilities: [gpu]
+    restart: unless-stopped
+
+volumes:
+  caic_data:
+  caic_uploads:
+  searxng_config:
+  qdrant_storage:
+  rabbitmq_data:
+  ollama_models:
+
+secrets:
+  rabbitmq_password:
+    file: ./secrets/rabbitmq_password.txt
+```
+
+**Notes:**
+- GPU reservations use `resources.reservations.devices` — this is compose v3.8+. For AMD GPUs, replace `driver: nvidia` with `driver: amd` (experimental Docker support). For hosts without GPU, omit the `deploy` block entirely.
+- The `deploy` block only applies when deployed as a swarm stack. For `docker compose`, GPU access may need `--gpus all` or `device_requests` in config. Verify compatibility.
+- SearXNG config file (`settings.yml`) is bind-mounted read-only from the host repo clone — the setup wizard should generate this file.
+
+---
+
+## 5. Networking
+
+### 5.1 Internal communication (compose network)
+
+| From | To | Port | Protocol |
+|------|----|------|----------|
+| cAIc | llama-server | 8081 | HTTP |
+| cAIc | Ollama | 11434 | HTTP |
+| cAIc | SearXNG | 8080 | HTTP |
+| cAIc | Qdrant | 6333 | HTTP |
+| cAIc | RabbitMQ | 5672 | AMQP |
+| RabbitMQ | (cluster peers) | 4369 | EPMD |
+| RabbitMQ | (cluster peers) | 25672 | Inter-node |
+
+### 5.2 Exposed ports (host-facing)
+
+| Port | Service | Should expose? | Notes |
+|------|---------|---------------|-------|
+| 8080 | cAIc | ✅ Required | UI + API |
+| 8888 | SearXNG | Optional | Only if user wants standalone search |
+| 6333 | Qdrant | Optional | Only for external tooling |
+| 5672 | RabbitMQ | Optional | Only for remote AMQP clients |
+| 15672 | RabbitMQ mgmt | ❌ Internal | Healthcheck only |
+| 8081 | llama-server | Optional | Only for external tooling |
+| 11434 | Ollama | Optional | Only for external tooling |
+
+**Design decision:** By default, only port 8080 (cAIc) is published. All other services remain on the internal compose network. Advanced users can opt-in by uncommenting `ports:` blocks.
+
+### 5.3 Reverse proxy consideration
+
+For production, a reverse proxy (Caddy, nginx, Traefik) should sit in front:
+
+```yaml
+# Optional — compose profile: "proxy"
+caddy:
+  image: caddy:latest
+  ports: ["80:80", "443:443"]
+  volumes:
+    - ./Caddyfile:/etc/caddy/Caddyfile:ro
+    - caddy_data:/data
+```
+
+This is out of scope for v1.0 but documented for future.
+
+---
+
+## 6. Setup Wizard (Extraction)
+
+`setup.sh` — idempotent, interactive, runs on first boot.
+
+### Flow
+
+```
+1. CHECK: Is .env present?
+   ├── YES → skip to step 7 (or ask to regenerate)
+   └── NO  → continue
+
+2. INTRO: Print banner, explain what's about to happen
+
+3. PROBE: Run hardware assessment
+   ├── psutil → RAM total, CPU count
+   ├── rocm-smi → VRAM (optional, best-effort)
+   └── nvidia-smi → VRAM (optional, best-effort)
+
+4. NETWORK: Ask for
+   ├── Hostname / LAN IP for this machine
+   ├── Admin PIN (4 digits, or accept auto-generated)
+   └── (Optional) RPC endpoints for GPU offload
+
+5. CALCULATE:
+   ├── RAG_MAX_VECTORS = max(1000, int(available_ram_gb * 100_000))
+   ├── LLAMA_N_GPU_LAYERS = 0 (CPU default; offer GPU detection)
+   ├── LLAMA_MODEL = default gguf filename
+   └── RABBITMQ_PASSWORD = openssl rand -hex 20
+
+6. GENERATE:
+   ├── .env file from template
+   ├── ./secrets/rabbitmq_password.txt
+   ├── ./searxng/settings.yml (with generated secret_key)
+   └── ./models/README.txt (instructions for placing .gguf)
+
+7. VERIFY:
+   ├── docker and docker compose plugin installed
+   ├── docker compose version >= 2.x
+   ├── SUCCESS → "Run: docker compose up -d"
+   └── FAILURE  → show diagnostics and links
+
+8. EXTRACT model:
+   ├── Prompt for download URL or local path
+   ├── Offer to pull from HuggingFace if huggingface-cli available
+   └── Guides user to place file in ./models/
+```
+
+### What setup.sh creates on disk
+
+```
+./docker-deploy/
+├── .env                        # All env vars (SECRET — add to .gitignore)
+├── docker-compose.yml          # Compose stack definition
+├── Dockerfile                  # cAIc image build
+├── secrets/
+│   └── rabbitmq_password.txt   # RabbitMQ password file
+├── searxng/
+│   └── settings.yml            # SearXNG config with generated secret_key
+├── models/
+│   ├── README.txt              # Instructions for model placement
+│   └── <model>.gguf            # (user-provided)
+└── setup.log                   # Wizard run log
+```
+
+### Idempotency
+
+Re-running `setup.sh`:
+- With `.env` present: ask "Regenerate? This will overwrite existing config."
+- Without `.env`: fresh run
+- Never overwrites `./models/*.gguf` files
+- Never touches running containers — only modifies files on disk
+
+---
+
+## 7. Back-out Procedure (Uninstall)
+
+`teardown.sh` — returns the host system to its pre-install state.
+
+### What gets removed
+
+| Item | Removal method |
+|------|---------------|
+| Docker containers | `docker compose down -v` |
+| Docker images | `docker rmi caic:latest` (ask about other images) |
+| Docker volumes | `docker volume rm caic_data ...` (prompt first) |
+| Network `caic_default` | Removed with compose |
+| `.env` file | `rm .env` |
+| `secrets/` directory | `rm -rf secrets/` |
+| `searxng/` directory | `rm -rf searxng/` |
+| `setup.log` | `rm setup.log` |
+| `hardware_state.json` | `rm hardware_state.json` |
+
+### What is preserved (by default)
+
+| Item | Reason |
+|------|--------|
+| `./models/*.gguf` | User data — prompt for deletion |
+| `caic.db` (in volume) | Prompt: "Keep database snapshot?" |
+| `./uploads/` (in volume) | Prompt: "Keep uploaded files?" |
+| Docker Engine itself | Not installed by this project — leave it |
+
+### Script flow
+
+```
+1. CHECK: docker compose file exists?
+   ├── NO  → warn, continue
+   └── YES → docker compose down -v
+
+2. CHECK: .env exists?
+   ├── NO  → skip
+   └── YES → ask: "Remove .env?" (default no)
+
+3. ASK: "Remove secrets/ and searxng/ directories?" (default no)
+
+4. ASK: "Remove Docker images? (y/N)" (default no)
+   ├── Y → docker rmi caic:latest
+   ├── Y → docker image ls | grep searxng/qdrant/rabbitmq → prompt per image
+   └── N → skip
+
+5. ASK: "Keep database volume snapshot? (Y/n)" (default yes)
+   ├── N → docker volume rm caic_data
+   └── Y → leave volume (can be reattached later)
+
+6. ASK: "Remove model files from ./models/? (y/N)" (default no)
+
+7. CLEANUP generated artifacts:
+   ├── rm -f setup.log
+   ├── rm -f hardware_state.json
+   └── rm -f docker-compose.yml
+
+8. SUMMARY:
+   ├── "Docker stack removed"
+   ├── "Persistent data preserved at: <paths>"
+   └── "Models kept at: ./models/"
+```
+
+### Partial rollback
+
+If the setup wizard fails mid-way, a partial rollback is better than leaving detritus:
+
+| Failure point | Clean up |
+|--------------|----------|
+| After .env, before compose | `rm .env; rm -rf secrets/ searxng/` |
+| After compose, before first `up` | `rm docker-compose.yml; rm -rf *` |
+| After `up` but before healthcheck | `docker compose down -v; rm -rf ./*` |
+
+`setup.sh` should trap EXIT on failure and prompt: "Clean up partial install? [y/N]"
+
+---
+
+## 8. Open Decisions
+
+| Decision | Options | Priority |
+|----------|---------|----------|
+| **Ollama vs llama-server embeddings** | Both work. Keep both for now — remove Ollama if llama-server handles embeddings. Reduce containers = simpler. | Medium |
+| **GPU support in compose** | NVIDIA: well-supported. AMD: requires `--device=/dev/kfd --device=/dev/dri` and ROCm image. Document both. | High |
+| **RabbitMQ clustering vs single node** | Single node in v1.0. Clustering docs for multi-host later. | Low |
+| **SearXNG config management** | Bind-mount a generated `settings.yml`, or let container create default and post-process. Bind-mount is cleaner. | Medium |
+| **Reverse proxy** | Caddy is simplest for auto-HTTPS. Out of scope for v1.0 but design for it. | Low |
+| **Healthcheck strategy** | `depends_on` with `condition: service_healthy` is the safest approach but increases startup time. Acceptable. | Medium |
+| **Database migration** | SQLite file in volume — no migration needed for v1.0 format. If schema changes post-v1.0, need a migration container. | Low |
+| **Linux vs macOS vs Windows** | Linux-primary. macOS may work with changes (no rocm-smi). Windows via WSL2 only. | Low |
+| **LLM model download** | HuggingFace CLI integration in setup.sh, or manual download. Manual is simpler. | Low |
+| **Dockerfile optimization** | Pin pip hashes, use `--no-cache-dir`, consider `slim` vs `alpine`. Alpine has musl compatibility issues with psutil. Stay with slim. | Medium |
+
+## 9. Worker Node Deployment Model
+
+The Docker stack above defines the **coordinator** only. Workers (headless inference nodes) have a radically lighter footprint.
+
+### 9.1 What a worker runs
+
+```
+Worker machine (e.g. worker01, worker02)
+┌────────────────────────────────────┐
+│  llama-server                      │
+│  (single binary, no build needed)  │
+│                                    │
+│  node_agent.py                     │
+│  (Python script, aio-pika client)  │
+│    ─ connects to coordinator's RMQ │
+│    ─ publishes heartbeat + reg     │
+│    ─ consumes model_swap commands  │
+│                                    │
+│  ROCm or CUDA runtime (if GPU)     │
+└────────────────────────────────────┘
+```
+
+### 9.2 What a worker does NOT run
+
+| Service | Reason |
+|---------|--------|
+| RabbitMQ server | Connects as AMQP *client* only (aio-pika) |
+| FastAPI / uvicorn / jC | No HTTP API, no UI, no database |
+| SQLite | No persistent state of its own |
+| SearXNG | No web search needs |
+| Qdrant | No local vector store |
+| Ollama | Uses coordinator's embedding endpoint |
+| Docker | Everything runs as bare binaries |
+| Python venv with full jC deps | Only needs `aio-pika` + `httpx` |
+
+### 9.3 Worker setup
+
+```bash
+# Install llama-server binary
+wget https://github.com/ggml-org/llama.cpp/releases/.../llama-server
+chmod +x llama-server
+
+# Install node agent deps
+pip install aio-pika httpx
+
+# Create node agent script (from repo: node_agent/agent.py)
+# Configure COORDINATOR_AMQP_URL in environment
+```
+
+### 9.4 Multiple workers
+
+Each worker registers independently with the coordinator's RabbitMQ. The coordinator tracks all registered workers via `CLUSTER_NODES` and routes inference requests to the best-matching node based on classification and availability.
+
+### 9.5 RabbitMQ and workers — architecture note
+
+Workers connect to RabbitMQ as **standard AMQP TCP clients** — no broker software required. The AMQP-0-9-1 protocol has always been client-server (since 2006), and libraries like `aio-pika`, `pika`, `amqplib`, `php-amqplib`, etc. connect over a single persistent socket. This is distinct from a service-mesh design where every node runs the same software stack and role is determined by config.
+
+```
+Broker-mediated model (this project):
+  Coordinator runs  RabbitMQ broker  ←── Workers connect as AMQP clients
+
+Service-mesh model (alternative):
+  Every node runs    RabbitMQ broker  ←── Nodes cluster together, all autonomous
+```
+
+The broker-mediated model is the preferred architecture for this project because workers are intentionally heterogeneous (different GPUs, different models, ARM vs x86) and should not be burdened with infrastructure services.
+
+## 10. Checklist (pre-v1.0 gate)
+
+- [ ] `Dockerfile` written and builds clean
+- [ ] `docker-compose.yml` boots all containers
+- [ ] cAIc container reaches all services (env vars resolve correctly)
+- [ ] SearXNG settings.yml generated correctly by setup.sh
+- [ ] RabbitMQ password secret mounted correctly
+- [ ] GPU (NVIDIA) passes through to llama-server container
+- [ ] GPU (AMD) passes through to llama-server container (or documented limitation)
+- [ ] `.env.example` checked in (no real secrets)
+- [ ] `setup.sh` written, idempotent, tested on clean Debian
+- [ ] `teardown.sh` written, tested, doesn't delete models without confirmation
+- [ ] `docker compose up -d` works without any manual steps beyond setup.sh
+- [ ] `docker compose down -v` followed by `setup.sh && docker compose up -d` = fresh stack
+- [ ] Healthchecks prevent serving before dependencies are ready
+- [ ] v1.0 release tag created
+
+---
+
+## 11. Files to create for B3
+
+```
+docker.md             ← this file (planning doc)
+Dockerfile            ← cAIc image
+docker-compose.yml    ← full stack
+.env.example          ← template without secrets
+setup.sh              ← extraction wizard
+teardown.sh           ← back-out utility
+searxng/
+  settings.yml        ← SearXNG config (generated by setup.sh)
+secrets/
+  rabbitmq_password.txt  ← generated by setup.sh
+models/
+  README.txt          ← instructions for placing .gguf
+```

docs/jc-ingest.sh

@@ -0,0 +1,31 @@
+#!/bin/bash
+# jc-ingest.sh — pipe terminal commands into jarvisChat RAG
+# Deploy to /home/gramps/bin/jc-ingest.sh on jarvis (192.168.50.210)
+#
+# Usage:
+#   1. chmod +x /home/gramps/bin/jc-ingest.sh
+#   2. Add to ~/.bashrc:
+#        export JARVISCHAT_COMPLETIONS_API_KEY="$(cat /opt/jarvischat/.completions_key)"
+#        export PROMPT_COMMAND="jc_capture"
+#        source /home/gramps/bin/jc-ingest.sh
+#
+#   The PROMPT_COMMAND hook runs jc_capture() after each command.
+#   Only commands matching the filter pattern are ingested.
+#
+# Filter: currently captures git, pip, systemctl, sudo, vi/vim, curl,
+# wget, apt, python, pytest commands. Edit the grep pattern to adjust.
+
+JC_URL="http://192.168.50.210:8080/api/ingest"
+JC_TOKEN="${JARVISCHAT_COMPLETIONS_API_KEY}"
+
+jc_capture() {
+    local cmd
+    cmd=$(history 1 | sed 's/^[ ]*[0-9]*[ ]*//')
+    if echo "$cmd" | grep -qE '^(git|pip|systemctl|sudo|vi|vim|curl|wget|apt|python|pytest)'; then
+        curl -s -X POST "$JC_URL" \
+            -H "Authorization: Bearer $JC_TOKEN" \
+            -H "Content-Type: application/json" \
+            -d "{\"content\": $(echo "$cmd" | python3 -c 'import json,sys; print(json.dumps(sys.stdin.read().strip()))'), \"source\": \"terminal\"}" \
+            > /dev/null 2>&1 &
+    fi
+}

docs/wiki/Developer-Architecture.md

@@ -1,165 +1,313 @@
 # Developer Architecture Guide
 
-This document explains how JarvisChat is structured, why key guardrails exist, and what the test suite validates.
+This document explains how cAIc is structured, the external services it integrates with, and the key architectural changes made during development.
 
 ## 1. System Overview
 
-JarvisChat is a single-process FastAPI service with a Jinja2 frontend and SQLite persistence.
+cAIc is a single-process FastAPI service with a Jinja2 frontend and SQLite persistence. It connects to an external llama-server for inference and optionally to SearXNG (web search), Qdrant (vector search), and RabbitMQ (AMQP cluster messaging).
 
-Primary files:
+### 1.1 Module Layout
 
-- `app.py`: API, middleware, streaming/chat logic, auth, memory, skills, and DB bootstrap
+Refactored from single-file (`app.py`) into modules under project root:
-- `templates/index.html`: main WebUX, settings panels, auth flow, streaming UI handlers
-- `jarvischat.db`: runtime SQLite database created and migrated at startup
 
-Core runtime integrations:
+| File | Role |
+|------|------|
+| `app.py` | FastAPI app, middleware, router registration, lifespan |
+| `config.py` | Constants, env vars, rate/payload limits, built-in skills registry, upload limits, RAG eviction config |
+| `db.py` | SQLite schema, connection factory, settings helpers, upload_context CRUD |
+| `auth.py` | PIN-based guest/admin sessions, auth routes |
+| `security.py` | Rate limiting, origin checks, IP allowlist, audit/incident logging |
+| `memory.py` | FTS5 memory CRUD, remember/forget command parsing |
+| `search.py` | SearXNG integration, perplexity scoring, refusal detection |
+| `rag.py` | Qdrant vector search, system prompt assembly, chunk_text() helper, collection stats |
+| `eviction.py` | Score-based RAG eviction engine (extracted from rag.py) |
+| `gpu.py` | AMD GPU stats via rocm-smi |
+| `hardware.py` | Hardware self-assessment — CPU, RAM, VRAM, service health probes |
+| `amqp.py` | aio-pika connection manager for RabbitMQ (connect, disconnect, publish, subscribe, auto-reconnect) |
+| `cluster.py` | Cluster node registry, event log, coordinator election, ping/pong, model swap handlers |
+| `triage.py` | Phi-4-mini query classification + `select_node()` for cluster routing |
+| `routers/` | One module per endpoint group |
 
-- Ollama for chat/model interaction
+### 1.2 External Services
-- SearXNG for web search (optional)
+
-- wttr.in for weather shortcut queries
+| Service | Required | Port | Purpose |
-- rocm-smi for GPU stats when available
+|---------|----------|------|---------|
+| llama-server (coordinator) | Yes | 8081 | LLM inference (OpenAI-compat), RPC offload to worker:50052 |
+| SearXNG | No | 8888 | Privacy-respecting web search |
+| Qdrant (coordinator) | No | 6333 | Vector database for RAG |
+| Ollama (worker) | No | 11434 | Embeddings for RAG chunk vectors |
+| RabbitMQ (coordinator) | No | 5672 | AMQP broker for cluster messaging |
+| rocm-smi | No | — | AMD GPU stats (host-level) |
+
+### 1.3 Config Discovery
+
+Key base URLs are configured via environment variables with sensible defaults:
+
+| Variable | Default | Service |
+|----------|---------|---------|
+| `LLAMA_SERVER_BASE` | `http://192.168.50.108:8081` | llama-server on coordinator |
+| `OLLAMA_BASE` | `http://localhost:11434` | Legacy — all inference goes through LLAMA_SERVER_BASE |
+| `SEARXNG_BASE` | `http://localhost:8888` | SearXNG |
+| `QDRANT_URL` | `http://192.168.50.108:6333` | Qdrant on coordinator |
+| `CAIC_AMQP_URL` | `amqp://caic:password@localhost:5672/caic` | RabbitMQ |
 
 ## 2. Request/Response Architecture
 
 ### 2.1 Chat Pipeline (`/api/chat`)
 
 1. Validate session, role, origin, rate, and payload limits in middleware
-2. Persist user message and conversation metadata
+2. Intercept "remember that..." / "forget about..." commands → process_remember_command()
-3. Build system prompt from enabled profile, memory context, and active skills metadata
+3. Persist user message and conversation metadata
-4. Stream model response over SSE token-by-token
+4. Build system prompt: profile + FTS5 memory + Qdrant RAG results + preset + active skills + uploaded document (if upload_context_id)
-5. Evaluate uncertainty/refusal; if needed, trigger search augmentation and stream augmented result
+5. Stream from llama-server with `logprobs: true` for perplexity scoring
-6. Persist final assistant message and emit terminal SSE event
+6. If perplexity > 15.0 OR refusal patterns match → re-query with SearXNG results
+7. Persist final assistant message and emit terminal SSE event
 
 ### 2.2 Explicit Search Pipeline (`/api/search`)
 
-1. Persist search-as-message into the target/new conversation
+1. Persist search-as-message into conversation
 2. Emit `searching` SSE event
 3. Pull web results from SearXNG
-4. Summarize with Ollama via SSE stream
+4. Summarize via llama-server SSE stream
-5. Persist summary and emit `done` event (plus raw results payload)
+5. Persist summary and emit `done` event
 
-### 2.3 Settings/Control Surface
+### 2.3 RAG Ingest Pipeline (`/api/ingest`)
 
-- Profile, presets, memory, conversation management, and settings APIs
+1. Bearer token auth (same key as completions API)
-- Skills APIs for phase-1 registry and enable/disable controls
+2. Chunk text via shared `chunk_text()` helper (512-token chunks, 128-token overlap)
-- Auth/session APIs for guest/admin role handling and keepalive
+3. Embed via Ollama `/api/embeddings`
+4. Upsert to Qdrant collection `caic_rag`
+5. Trigger `maybe_evict()` if collection exceeds high-water mark
+
+### 2.4 Upload Pipeline (`/api/upload`)
+
+1. Admin required, multipart file upload
+2. Validate MIME type + size against config limits
+3. PDF text extraction via pypdf; plain text for all other types
+4. Three modes: `context` (SQLite with 1hr expiry), `ingest` (RAG/Qdrant), `both`
+5. Trigger `maybe_evict()` if ingest mode
 
 ## 3. Data Model (SQLite)
 
 Key tables:
 
-- `conversations`: conversation headers and timestamps
+- `conversations` — headers, timestamps, attachment_count
-- `messages`: ordered chat history entries
+- `messages` — ordered chat history per conversation
-- `profile`: singleton row for injected profile prompt
+- `profile` — singleton row for injected profile prompt
-- `settings`: runtime toggles and selected defaults
+- `settings` — runtime toggles and selected defaults
-- `system_presets`: named reusable system prompts
+- `system_presets` — named reusable system prompts
-- `skills`: per-skill enabled state and timestamp
+- `skills` — per-skill enabled state and timestamp
-- `memories` (FTS5 virtual table): searchable user memory facts
+- `memories` (FTS5 virtual table) — full-text searchable user memory facts
+- `upload_context` — auto-expiring document storage for context injection
 
 Design notes:
-
+- Startup is idempotent: tables created if missing, defaults seeded only when absent
-- Startup is idempotent: tables are created if missing and defaults seeded only when absent
+- No connection pool: each request opens and closes a short-lived SQLite connection
-- No connection pool: each request opens a short-lived SQLite connection
+- `init_db()` called in FastAPI lifespan
 
 ## 4. Security Implementations
 
-This section documents explicit controls currently in code.
-
 ### 4.1 Auth Model
 
-- Guest session is default for conversational access
+- Guest session by default (POST /api/auth/guest)
-- Admin unlock uses 4-digit PIN and creates admin-capable session
+- Admin unlock via 4-digit PIN (POST /api/auth/login)
-- Admin required for write/destructive routes
+- Admin required for PUT/DELETE/PATCH + all POST except allowlist (/api/chat, /api/search, /api/auth/*)
-- Session heartbeat/timeout and explicit logout/revoke flow
+- /api/ingest is exempt from session auth — self-authenticates via Bearer token
+- Session heartbeat/timeout (90s default) and explicit logout
 
-### 4.2 PIN and Session Hardening
+### 4.2 PIN Hardening
 
 - Admin PIN hashed with PBKDF2-HMAC-SHA256 + salt
-- Failed PIN attempts tracked per client IP
+- Failed PIN attempts tracked per client IP (max 5, 300s lockout)
-- Lockout window enforced after max failed attempts
+- Default PIN allowed only if CAIC_ALLOW_DEFAULT_PIN=true
 
 ### 4.3 Browser and API Abuse Controls
 
-- Origin checks on state-changing requests
+- Origin checks on all /api/ requests (rejects absent Origin AND Referer)
-- Rate limiting by endpoint category and identity (IP/session)
+- Rate limiting per endpoint category and identity (IP/session)
-- Payload size limits per route class
+- Payload size limits per route class (64KB default, 128KB chat, 20MB upload)
-- Settings key allowlist to block arbitrary configuration injection
+- Settings key allowlist (5 keys: profile_enabled, default_model, etc.)
-- IP allowlist/CIDR gate with optional trusted proxy forwarding mode
+- IP allowlist/CIDR gate with trusted proxy forwarding mode
 
 ### 4.4 Output and Error Safety
 
-- Search result URLs sanitized to `http`/`https` only
+- Search result URLs sanitized to http/https only
 - Client-safe error envelopes with incident key correlation
-- Full stack traces and diagnostic metadata logged server-side only
+- Full stack traces logged server-side only
 
 ### 4.5 Operational Auditability
 
-- Structured audit events for auth actions, admin operations, and guardrail denials
+- Structured audit events for auth actions, admin ops, guardrail denials
-- Incident logs include event type, key, path/method context, and runtime metadata
+- Incident logs with event type, key, path/method, and runtime metadata
 
-## 5. Skills Framework (Phase 1)
+## 5. RAG Architecture
 
-Goal: introduce a governed skills control plane inside the local JarvisChat sandbox.
+### 5.1 Vector Search
 
-Current behavior:
+- Qdrant collection `caic_rag` on coordinator:6333
+- Embeddings via Ollama on worker:11434 (`/api/embeddings`)
+- Shared `chunk_text(text, chunk_size=512, overlap=128)` helper in rag.py
+- Upload and ingest endpoints share the same chunk+embed+upsert pipeline
 
-- Built-in skill registry defined server-side
+### 5.2 Score-Based Eviction
-- Per-skill enable/disable persisted in DB
-- Global `skills_enabled` master toggle in settings
-- Active skills injected into system prompt with bounded text budget
-- API endpoints to list skills, list active skills, and toggle skill state
-- WebUX settings panel to control master/per-skill toggles
 
-Non-goals in phase 1:
+When `RAG_MAX_VECTORS` is exceeded, eviction fires with hysteresis:
 
-- No unrestricted shell/tool execution
+- High-water mark: 80% of max → trigger eviction
-- No external connector execution (filesystem, Gmail, etc.)
+- Low-water mark: 20% of max → stop eviction
+- Batch size: 1000 vectors per cycle
+- Score formula: `score = (access_weight * retrieval_count) + (age_weight * hours_since_ingested)`
+- Lower score evicted first (least useful)
+- Tiebreaker: oldest last_accessed ASC
+- Excluded sources: `upload`, `profile` (pinned)
+- Grace period: 1 hour before any vector is eligible
+- Thread-safe via `asyncio.Lock`
 
-## 6. Testing Strategy and Validation Intent
+Eviction module at `eviction.py` (re-exported through `rag.py` for backward compat).
 
-The test suite validates both behavior and guardrail assumptions.
+### 5.3 Operational Stats
 
-### 6.1 What We Test
+`GET /api/rag/stats` (admin required) returns:
+- vector_count, max_vectors, high_water_pct, low_water_pct, percent_full
+- pinned_sources list, grace_hours
+- at_risk_count, pinned_count, avg_retrieval_count
+- eviction_counts_last_{1,5,30}m
 
-- Auth capability separation (guest vs admin)
+### 5.4 Flush
-- URL sanitization safety for outbound links
-- Rate and payload guardrails
-- IP allowlist behavior
-- Safe error envelope behavior and SSE error leakage prevention
-- Streaming chat/search and memory command paths
-- Skills framework toggles and prompt-injection behavior
 
-### 6.2 Why These Tests Matter
+`POST /api/rag/flush` (admin required) — deletes all non-pinned vectors. Returns `{deleted_count, collection, status}`.
 
-- Confirms security controls are active and regression-resistant
+## 6. Cluster Architecture
-- Ensures streaming UX protocol remains stable (`token`, `searching`, `done`, `error`)
-- Verifies policy intent: dangerous actions require admin capability
-- Validates new features preserve prior guarantees
 
-### 6.3 Internal Process Validation
+### 6.1 Design Model: Broker-Mediated
 
-For substantive changes, Definition of Done includes:
+cAIc uses a **broker-mediated** cluster design. This is the preferred architecture and is reflected in all implementation decisions below.
 
+**How it works:**
+- A single RabbitMQ broker (or clustered set of brokers) acts as the central nervous system
+- **Coordinator nodes** run the FastAPI app, host the HTTP API/UI, and publish commands to the broker
+- **Worker nodes** connect as AMQP *clients only* — they consume commands and publish status events, but run no broker software themselves
+- Communication is asynchronous and persistent: each node opens a TCP connection on startup and keeps it alive. The coordinator probes worker health via on-demand AMQP ping/pong messages (5s timeout) rather than relying on the AMQP-0-9-1 transport-level heartbeat.
+
+**Why broker-mediated:**
+- Workers are heterogeneous (different GPUs, different models, ARM vs x86) — no assumption of uniform software
+- Workers are lightweight — a Raspberry Pi with a USB AI accelerator can participate without running a broker
+- The coordinator delegates work via messages, not by SSH'ing into workers or requiring shared filesystems
+- Failure is isolated: a crashed worker stops responding to ping; the coordinator auto-deregisters it and reassigns its work
+
+**What it is NOT:**
+- Not a service mesh — workers do not run identical software stacks
+- Not autonomous failover — if the coordinator dies, a replacement must be manually promoted (or pre-configured as a secondary coordinator). Workers cannot self-promote to coordinator because they lack the required services (FastAPI, SQLite, DB schema, SearXNG, Qdrant, etc.)
+- Not a peer-to-peer cluster — all orchestration flows through the coordinator
+
+### 6.2 Node Types
+
+Every physical machine in the cluster is classified by which services it runs. Two node types are defined:
+
+| Aspect | Coordinator | Worker |
+|--------|------------|--------|
+| **Role** | Serves HTTP API/UI, orchestrates inference, owns cluster state | Runs inference models on behalf of the coordinator |
+| **Python** | Required — runs FastAPI app | Required — runs node agent (aio-pika consumer) |
+| **RabbitMQ server** | Required — hosts the broker | Not required — connects as AMQP client only |
+| **RabbitMQ client (aio-pika)** | Required — publishes commands, consumes events | Required — consumes commands, publishes events |
+| **FastAPI / uvicorn** | Required | Not needed |
+| **SQLite** | Required — owns caic.db | Not needed |
+| **Qdrant** | Optional (recommended) — vector DB for RAG | Not needed |
+| **SearXNG** | Optional — web search | Not needed |
+| **llama-server** | Optional — can share its own GPU for inference | Required — this is why the worker exists |
+| **Ollama** | Optional — embeddings for RAG | Not needed |
+| **rocm-smi / nvidia-smi** | Optional — hardware stats | Optional — node agent reports this at registration |
+
+### 6.3 Service Distribution Summary
+
+```
+Coordinator                          Worker(s)
+┌────────────────────┐               ┌──────────────────────────┐
+│  cAIc              │               │  llama-server            │
+│  (FastAPI + SQLite)│               │  (inference)             │
+│  RabbitMQ server   │◄──AMQP───────│  aio-pika (agent)        │
+│  SearXNG (opt)     │    persistent │  ROCm / CUDA (if GPU)    │
+│  Qdrant (opt)      │    TCP        │  Ollama (embeddings,opt) │
+│  llama-server(opt) │    conn       │                          │
+└────────────────────┘               │  No broker               │
+                                     │  No cAIc                 │
+                                     │  No DB                   │
+                                     │  No search/vector        │
+                                     └──────────────────────────┘
+```
+
+### 6.4 RabbitMQ Topology
+
+Every RabbitMQ server belongs to a cluster. Currently only the coordinator runs one; if high availability is needed, additional nodes can join the RMQ cluster without changing the architecture.
+
+| Exchange | Type | Purpose |
+|----------|------|---------|
+| `jc.admin` | topic | Lifecycle commands: register, deregister, ping, pong, admitted, rejected; model commands: cmd.swap_model |
+| `jc.system` | topic | Events: model_ready, model_failed, node.*.heartbeat, event; coordinator queries: coord_query, coord_response |
+
+All exchanges, queues, and bindings are declared by `amqp.py` at startup. Worker runs `node_agent/agent.py` which connects as an AMQP client, registers, responds to ping, and handles model swap commands.
+
+## 7. SSE Protocol
+
+All streaming endpoints yield `data: {json}\n\n`:
+
+- `{token, conversation_id}` — streaming token
+- `{searching: true}` — web search triggered
+- `{search_results: N}` — N results found (no raw payload)
+- `{done: true, perplexity, tokens_per_sec, searched?}` — terminal
+- `{error: "...", error_key: "..."}` — error with incident key
+
+## 8. Testing Strategy
+
+### 8.1 Test Framework
+
+- pytest with `tmp_path` + monkeypatched httpx.AsyncClient
+- No live external services required
+- Test factories reset `SESSIONS`, `PIN_ATTEMPTS`, `RATE_EVENTS` globals per test
+
+### 8.2 Test Coverage Areas (179 tests)
+
+| Test file | Coverage |
+|-----------|----------|
+| test_auth_capabilities.py | Guest/admin sessions, origin blocking, logout |
+| test_chat_streaming_and_memory_paths.py | Streaming, auto-search, remember/forget, upload context injection |
+| test_cluster.py | Registration, deregistration, pong, events, coordinator query |
+| test_cluster_heartbeat.py | Heartbeat handler, known/unknown node |
+| test_completions.py | API key auth, FIM, streaming, blocking, errors |
+| test_conversations.py | Full CRUD, guest admin, attachment_count |
+| test_ingest.py | Bearer auth, chunk/embed/upsert, validation |
+| test_ip_allowlist.py | IP allowlist helper + middleware |
+| test_memories.py | Edit, search, stats |
+| test_model_swap.py | request_model_swap, handle_model_ready/failed, select_node swap triggering |
+| test_models_router.py | Models list, ps, show, stats, search/status |
+| test_node_agent.py | Node agent registration, ping/pong, model swap |
+| test_presets.py | Full CRUD, default preset protection |
+| test_profile.py | Get, update, default, length validation |
+| test_rag_management.py | Collection stats, eviction algorithm (pinned/grace/scoring/batch), maybe_evict hysteresis, operational stats, flush, concurrency lock |
+| test_rate_and_payload_guardrails.py | Rate limits + payload size |
+| test_search_route.py | Explicit search flow, no results, errors |
+| test_search_url_sanitization.py | URL sanitizer |
+| test_settings_allowlist.py | Allowlisted key enforcement |
+| test_skills_framework.py | List, toggle, unknown skill, prompt injection |
+| test_triage.py | classify_query, select_node, get_inference_url |
+| test_upload.py | Upload, delete, link, by-conversation, attachment_count |
+| test_error_envelopes.py | Global exception handler + stream errors |
+
+### 8.3 DoD Process
+
+For substantive changes:
 1. Implement code change
 2. Add/adjust tests proving behavior and guardrail intent
-3. Update README release notes for user-facing impact
+3. Update this wiki and README in the same change set
-4. Update wiki architecture/security/testing docs for maintainers
+4. Validate with full test run before commit
-5. Validate with targeted test runs before merge/deploy
 
-This process is intentionally explicit so design decisions remain auditable over time.
+## 9. Hardware Self-Assessment
 
-## 7. Deployment and Operations Notes
+On startup, `assess_hardware()` probes:
+- RAM total/available (psutil)
+- VRAM total/free (rocm-smi, best-effort)
+- llama-server reachability + model list
+- Qdrant reachability + collection list
+- SearXNG reachability
 
-- Primary deployment target: local/homelab systemd service
+Writes `hardware_state.json` to working directory.
-- Required dependency: Ollama
-- Optional dependency: SearXNG
-- Recommended log review path: system journal for startup, guardrail denials, and incidents
-
-## 8. Contribution Guidance
-
-When adding a feature:
-
-1. Define security posture first (who can execute, what can fail, and failure mode)
-2. Implement smallest safe slice with clear limits
-3. Add tests that prove both happy path and guardrail path
-4. Update this wiki and README in the same change

docs/wiki/Home.md

@@ -1,6 +1,6 @@
-# JarvisChat Developer Wiki
+# cAIc Developer Wiki
 
-This wiki is the developer-facing architecture and process reference for JarvisChat.
+This wiki is the developer-facing architecture and process reference for cAIc.
 
 ## Audience
 
@@ -14,7 +14,7 @@ This wiki is the developer-facing architecture and process reference for JarvisC
 
 ## Scope and Support Model
 
-JarvisChat is designed for local and trusted-LAN operation.
+cAIc is designed for local and trusted-LAN operation.
 
 The code may technically function against external or commercial endpoints, but this deployment mode is not a supported target in this project.
 

docs/wiki/current-wip.md

@@ -1,84 +1,23 @@
-# JarvisChat Current WiP Backlog
+# cAIc Current WiP Backlog
 
-Last updated: 2026-04-27
+Last updated: 2026-07-06
-Owner: Gramps + Copilot
+Owner: Gramps
-Scope: issues, bugs, security exposures, and feature enhancements.
+Scope: Active roadmap items and backlog.
 
-Total identified items: 27
+## ~~Roadmap N: AMQP Cluster Nervous System [COMPLETE]~~
 
-## Priority Definitions
+All 15 tasks are done. Final commit: `f0689ac feat: Roadmap N — AMQP cluster nervous system complete`.
-- P0: Critical risk or data-loss/security exposure; do first.
-- P1: High impact reliability/correctness work.
-- P2: Important feature/UX improvements.
-- P3: Nice-to-have polish.
 
-## Top 10 (Urgency Order)
+## Backlog
-1. [P0][DONE] Add authentication/authorization for all write and admin endpoints.
-2. [P0][DONE] Add CSRF/origin protection for browser-initiated state-changing requests.
-3. [P0][DONE] Block unsafe URL schemes in rendered search-result links (e.g., javascript:).
-4. [P0][DONE] Add rate limiting and request body size limits for chat/search/profile APIs.
-5. [P1][DONE] Restrict settings updates to an allowlist of valid keys.
-6. [P1] Add pagination + hard caps on list endpoints (memories, conversations, message history).
-7. [P1][DONE] Stop returning raw exception text to clients; use safe error envelopes.
-8. [P1][DONE] Add automated tests for chat streaming, auto-search trigger, and memory command paths.
-9. [P2][DONE] Implement skills/tool-call framework (MCP-style) with per-skill enable controls.
-10. [P2] Implement heartbeat/check-in pipeline with scheduler + summary endpoint.
 
-## Item 1 Executive Summary (Scope + Security)
+- B1 — Context loss in follow-up questions (investigation)
-
+- B2 — Bang-prefixed (`!`) search routing
-- Status: Complete. Guest/admin capability split implemented with admin-only write enforcement, origin checks on state-changing requests, audit logging, and endpoint capability tests.
+- B3 — Docker distribution (planning doc at `docker.md`)
-
+- **B4 — RAG Corpus Management UI (NEXT)** — browse, search, edit, delete individual RAG entries. Backend endpoints at `routers/rag_admin.py`, frontend panel in `templates/index.html`
-- Decision: JarvisChat is local-first by design. Primary mode is same-host Ollama; optional mode allows RFC1918 LAN endpoints only.
+- HTTPS / reverse proxy (Caddy)
-- Constraint: Public Internet AI endpoints are out of scope unless explicitly enabled in a future advanced mode.
+- Conversation search/filter and export tooling
-- Risk: Even on LAN, unauthenticated write/admin endpoints permit unauthorized data tampering and deletion.
+- Keyboard shortcuts, retry button, source-link polish
-- Requirement: Add mandatory admin authentication for all POST/PUT/DELETE routes and destructive actions.
-- Authentication shape (scope-locked): two capability tiers only: guest (chat-only) and admin (4-digit PIN unlock).
-- Scope guardrail: Avoid full RBAC. Keep capability split minimal: conversational chat for guest, advanced/destructive actions for admin.
-- Definition of done:
-	1. Auth required on all state-changing endpoints.
-	2. Destructive actions require admin authorization.
-	3. Endpoint configuration rejects non-local/non-RFC1918 AI backends by default.
-	4. Strong rate limiting + lockout controls in place for PIN attempts.
-	5. Security events logged for failed and successful admin actions.
-
-## Full Backlog (Sorted by Priority)
-
-### P0 Critical
-1. Add auth for write/admin endpoints (`POST/PUT/DELETE` routes, mass delete, profile/settings changes).
-2. Add CSRF or strict origin checks for browser session protection.
-3. Validate/sanitize outbound href URLs before rendering in HTML (allow http/https only).
-4. Add per-IP rate limiting on `/api/chat`, `/api/search`, `/api/profile`, `/api/settings`.
-5. Enforce request size limits (message/profile text and JSON body) to prevent memory abuse.
-
-### P1 High
-6. Add settings key allowlist in `/api/settings` to prevent arbitrary key injection.
-7. Add pagination (`limit`, `offset`) with enforced maximums for list APIs.
-8. Add DB indexes and query hygiene for scalability (`messages.conversation_id`, timestamps).
-9. Replace raw exception leakage to clients with generic safe error messages + server-side logs.
-10. Add request/response timeout and retry policy consistency across external calls.
-11. Add endpoint-level audit logging for destructive operations.
-12. Add unit/integration tests for: remember/forget parsing, refusal detection, search fallback, SSE done/error shape.
-13. Add conversation title sanitization and length constraints.
-14. Ensure default preset semantics are correct (currently all seeded presets are marked default).
-15. Add preflight validation for required model/preset selection and block send with clear user guidance instead of timing out.
-
-### P2 Important Features
-16. Skills system: load markdown skill files with YAML frontmatter from skills directory.
-17. Skills registry API: list/enable/disable skills and expose active skills to UI.
-18. Inject active skill instructions into system prompt with bounded token budget.
-19. Tool execution guardrails: allowlist, confirmation mode, and execution logs.
-20. Heartbeat scheduler (cron/systemd timer) for daily check-ins.
-21. Heartbeat endpoint for generated briefings and anomaly summaries.
-22. Model info UI panel (description, updated date, best-use purpose).
-23. Default model selection improvements and persistence validation.
-24. Hidden model list support (exclude models from dropdown).
-25. Model update action from UI (trigger controlled model pull).
-
-### P3 Nice to Have
-26. Conversation search/filter and export tooling.
-27. Keyboard shortcuts, retry button, and source-link polish.
 
 ## Maintenance Rules
-- Keep this file as the single source of truth.
+- Keep this file as the single source of truth for roadmap tracking.
-- Update item priority/status whenever work starts or completes.
+- Update as work starts or completes.
-- Mirror the Top 10 summary in README and keep counts aligned.

eviction.py

@@ -0,0 +1,268 @@
+"""
+cAIc — Score-based RAG vector eviction with hysteresis.
+"""
+import asyncio
+import logging
+from datetime import datetime, timezone, timedelta
+
+import httpx
+
+from config import (
+    QDRANT_URL, RAG_COLLECTION,
+    RAG_MAX_VECTORS, RAG_EVICTION_HIGH_WATER, RAG_EVICTION_LOW_WATER,
+    RAG_EVICTION_BATCH, RAG_PINNED_SOURCES, RAG_GRACE_HOURS,
+    RAG_ACCESS_WEIGHT, RAG_AGE_WEIGHT,
+)
+
+log = logging.getLogger("caic")
+
+eviction_lock = asyncio.Lock()
+EVICTION_LOG: list[dict] = []
+
+
+async def _update_retrieval_count(point_id: str, current_count: int = 0):
+    try:
+        async with httpx.AsyncClient() as client:
+            payload = {
+                "retrieval_count": current_count + 1,
+                "last_accessed": datetime.now(timezone.utc).isoformat(),
+            }
+            resp = await client.put(
+                f"{QDRANT_URL}/collections/{RAG_COLLECTION}/points/payload",
+                json={"points": [point_id], "payload": payload},
+                timeout=5.0,
+            )
+            if resp.status_code not in (200, 201):
+                log.warning(f"Failed to increment retrieval count for {point_id}: {resp.status_code}")
+    except Exception as e:
+        log.warning(f"Error incrementing retrieval count for {point_id}: {e}")
+
+
+async def get_collection_count() -> int:
+    try:
+        async with httpx.AsyncClient() as client:
+            resp = await client.get(
+                f"{QDRANT_URL}/collections/{RAG_COLLECTION}",
+                timeout=10.0,
+            )
+            if resp.status_code == 200:
+                return resp.json().get("result", {}).get("vectors_count", 0)
+    except Exception as e:
+        log.warning(f"get_collection_count error: {e}")
+    return 0
+
+
+async def get_collection_stats() -> dict:
+    count = await get_collection_count()
+    high_water_pct = int(RAG_EVICTION_HIGH_WATER * 100)
+    low_water_pct = int(RAG_EVICTION_LOW_WATER * 100)
+    percent_full = round((count / RAG_MAX_VECTORS) * 100, 1) if RAG_MAX_VECTORS > 0 else 0
+    return {
+        "vector_count": count,
+        "max_vectors": RAG_MAX_VECTORS,
+        "high_water_mark": int(RAG_MAX_VECTORS * RAG_EVICTION_HIGH_WATER),
+        "low_water_mark": int(RAG_MAX_VECTORS * RAG_EVICTION_LOW_WATER),
+        "high_water_pct": high_water_pct,
+        "low_water_pct": low_water_pct,
+        "percent_full": percent_full,
+        "pinned_sources": list(RAG_PINNED_SOURCES),
+    }
+
+
+async def evict_batch(batch_size: int) -> int:
+    filter_conditions = {
+        "must_not": [
+            {"match": {"key": "source", "value": src}}
+            for src in RAG_PINNED_SOURCES
+        ]
+    }
+    try:
+        async with httpx.AsyncClient() as client:
+            scroll_resp = await client.post(
+                f"{QDRANT_URL}/collections/{RAG_COLLECTION}/points/scroll",
+                json={
+                    "filter": filter_conditions,
+                    "limit": min(batch_size * 10, 10000),
+                    "with_payload": True,
+                    "with_vector": False,
+                },
+                timeout=30.0,
+            )
+            if scroll_resp.status_code != 200:
+                log.warning(f"Eviction scroll failed: {scroll_resp.status_code}")
+                return 0
+
+            points = scroll_resp.json().get("result", {}).get("points", [])
+            if not points:
+                return 0
+
+            now = datetime.now(timezone.utc)
+            scored = []
+            for p in points:
+                payload = p.get("payload", {})
+                date_str = payload.get("ingest_date") or payload.get("upload_date", "")
+                if date_str:
+                    age_hours = (now - datetime.fromisoformat(date_str)).total_seconds() / 3600
+                else:
+                    age_hours = 999999
+
+                if age_hours < RAG_GRACE_HOURS:
+                    continue
+
+                retrieval_count = payload.get("retrieval_count", 0) or 0
+                score = retrieval_count * RAG_ACCESS_WEIGHT + age_hours * RAG_AGE_WEIGHT
+                last_accessed = payload.get("last_accessed", date_str)
+                scored.append((score, last_accessed, p["id"]))
+
+            if not scored:
+                log.warning("No evictable vectors found (all pinned or newborn)")
+                return 0
+
+            scored.sort(key=lambda x: (x[0], x[1]))
+            to_delete = [p[2] for p in scored[:batch_size]]
+            if not to_delete:
+                return 0
+
+            delete_resp = await client.post(
+                f"{QDRANT_URL}/collections/{RAG_COLLECTION}/points/delete",
+                json={"points": to_delete},
+                timeout=30.0,
+            )
+            if delete_resp.status_code not in (200, 201):
+                log.warning(f"Eviction delete failed: {delete_resp.status_code}")
+                return 0
+
+            return len(to_delete)
+    except Exception as e:
+        log.warning(f"evict_batch error: {e}")
+        return 0
+
+
+async def maybe_evict() -> int:
+    if RAG_MAX_VECTORS <= 0:
+        return 0
+    effective_batch = max(RAG_EVICTION_BATCH, 1)
+
+    async with eviction_lock:
+        count = await get_collection_count()
+        threshold_high = int(RAG_MAX_VECTORS * RAG_EVICTION_HIGH_WATER)
+        threshold_low = int(RAG_MAX_VECTORS * RAG_EVICTION_LOW_WATER)
+
+        if count < threshold_high:
+            return 0
+
+        total_evicted = 0
+        while count >= threshold_low:
+            if total_evicted > 0 and count < threshold_low:
+                break
+            deleted = await evict_batch(effective_batch)
+            if deleted == 0:
+                break
+            total_evicted += deleted
+            count -= deleted
+            if count < threshold_high and total_evicted > 0:
+                break
+            if count < threshold_low:
+                break
+
+        if total_evicted > 0:
+            entry = {
+                "timestamp": datetime.now(timezone.utc).isoformat(),
+                "count": total_evicted,
+                "remaining": count,
+            }
+            EVICTION_LOG.append(entry)
+            if len(EVICTION_LOG) > 1000:
+                EVICTION_LOG.pop(0)
+            log.info(f"Evicted {total_evicted} vectors ({count} remaining)")
+
+        return total_evicted
+
+
+async def get_rag_operational_stats() -> dict:
+    stats = await get_collection_stats()
+    now = datetime.now(timezone.utc)
+    cutoff_1m = now - timedelta(minutes=1)
+    cutoff_5m = now - timedelta(minutes=5)
+    cutoff_30m = now - timedelta(minutes=30)
+
+    eviction_1m = sum(
+        e["count"] for e in EVICTION_LOG
+        if datetime.fromisoformat(e["timestamp"]) > cutoff_1m
+    )
+    eviction_5m = sum(
+        e["count"] for e in EVICTION_LOG
+        if datetime.fromisoformat(e["timestamp"]) > cutoff_5m
+    )
+    eviction_30m = sum(
+        e["count"] for e in EVICTION_LOG
+        if datetime.fromisoformat(e["timestamp"]) > cutoff_30m
+    )
+
+    pinned_count = 0
+    avg_retrieval_count = 0.0
+    at_risk_count = 0
+
+    try:
+        async with httpx.AsyncClient() as client:
+            pinned_filter = {
+                "should": [
+                    {"match": {"key": "source", "value": src}}
+                    for src in RAG_PINNED_SOURCES
+                ]
+            }
+            pinned_resp = await client.post(
+                f"{QDRANT_URL}/collections/{RAG_COLLECTION}/points/scroll",
+                json={"filter": pinned_filter, "limit": 10000, "with_payload": True, "with_vector": False},
+                timeout=10.0,
+            )
+            if pinned_resp.status_code == 200:
+                pinned_count = len(pinned_resp.json().get("result", {}).get("points", []))
+
+            nonpinned_filter = {
+                "must_not": [
+                    {"match": {"key": "source", "value": src}}
+                    for src in RAG_PINNED_SOURCES
+                ]
+            }
+            np_resp = await client.post(
+                f"{QDRANT_URL}/collections/{RAG_COLLECTION}/points/scroll",
+                json={"filter": nonpinned_filter, "limit": 10000, "with_payload": True, "with_vector": False},
+                timeout=10.0,
+            )
+            if np_resp.status_code == 200:
+                points = np_resp.json().get("result", {}).get("points", [])
+                if points:
+                    retrievals = []
+                    scored = []
+                    for p in points:
+                        payload = p.get("payload", {})
+                        rc = payload.get("retrieval_count", 0) or 0
+                        retrievals.append(rc)
+                        date_str = payload.get("ingest_date") or payload.get("upload_date", "")
+                        if date_str:
+                            age_hours = (now - datetime.fromisoformat(date_str)).total_seconds() / 3600
+                        else:
+                            age_hours = 999999
+                        score = rc * RAG_ACCESS_WEIGHT + age_hours * RAG_AGE_WEIGHT
+                        last_accessed = payload.get("last_accessed", date_str)
+                        scored.append((score, last_accessed))
+
+                    avg_retrieval_count = round(sum(retrievals) / len(retrievals), 2)
+
+                    scored.sort(key=lambda x: (x[0], x[1]))
+                    at_risk_threshold = max(1, len(scored) // 10)
+                    at_risk_count = at_risk_threshold
+    except Exception as e:
+        log.warning(f"RAG operational stats scroll error: {e}")
+
+    stats.update({
+        "grace_hours": RAG_GRACE_HOURS,
+        "eviction_counts_last_1m": eviction_1m,
+        "eviction_counts_last_5m": eviction_5m,
+        "eviction_counts_last_30m": eviction_30m,
+        "pinned_count": pinned_count,
+        "avg_retrieval_count": avg_retrieval_count,
+        "at_risk_count": at_risk_count,
+    })
+    return stats

gpu.py

@@ -1,11 +1,11 @@
 """
-JarvisChat - AMD GPU stats via rocm-smi.
+cAIc - AMD GPU stats via rocm-smi.
 """
 import json
 import logging
 import subprocess
 
-log = logging.getLogger("jarvischat")
+log = logging.getLogger("caic")
 
 
 def get_gpu_stats() -> dict:

hardware.py

@@ -0,0 +1,102 @@
+"""
+cAIc — Startup hardware self-assessment.
+"""
+import asyncio
+import json
+import logging
+import subprocess
+from pathlib import Path
+
+import httpx
+import psutil
+
+from config import LLAMA_SERVER_BASE, SEARXNG_BASE
+
+log = logging.getLogger("caic")
+
+HARDWARE_STATE_PATH = Path("hardware_state.json")
+_TIMEOUT_EXPIRED = subprocess.TimeoutExpired
+
+
+async def assess_hardware() -> dict:
+    mem = psutil.virtual_memory()
+    ram_total_gb = round(mem.total / (1024 ** 3), 1)
+    ram_available_gb = round(mem.available / (1024 ** 3), 1)
+    cpu_count = psutil.cpu_count()
+
+    vram_total_mb = 0
+    vram_free_mb = 0
+    try:
+        result = subprocess.run(
+            ["rocm-smi", "--showmeminfo", "vram", "--json"],
+            capture_output=True, text=True, timeout=5,
+        )
+        if result.returncode == 0:
+            data = json.loads(result.stdout)
+            for card, info in data.items():
+                tot = info.get("VRAM Total (MB)", 0)
+                free = info.get("VRAM Free (MB)", None)
+                used = info.get("VRAM Used (MB)", 0)
+                if tot:
+                    vram_total_mb += int(tot)
+                if free is not None:
+                    vram_free_mb += int(free)
+                else:
+                    vram_free_mb += int(tot) - int(used)
+    except (FileNotFoundError, _TIMEOUT_EXPIRED, json.JSONDecodeError):
+        log.warning("rocm-smi not available or failed — VRAM stats set to 0")
+    except Exception as e:
+        log.warning(f"rocm-smi error: {e}")
+
+    llama_reachable = False
+    llama_models = []
+    try:
+        async with httpx.AsyncClient(timeout=3) as client:
+            resp = await client.get(f"{LLAMA_SERVER_BASE}/v1/models")
+            if resp.status_code == 200:
+                llama_reachable = True
+                data = resp.json()
+                llama_models = [m.get("id", "") for m in data.get("data", [])]
+    except Exception:
+        log.warning("llama-server not reachable")
+
+    qdrant_reachable = False
+    qdrant_collections = []
+    try:
+        async with httpx.AsyncClient(timeout=3) as client:
+            resp = await client.get("http://192.168.50.108:6333/collections")
+            if resp.status_code == 200:
+                qdrant_reachable = True
+                data = resp.json()
+                raw = data.get("result", {}).get("collections", [])
+                qdrant_collections = [c.get("name", "") for c in raw]
+    except Exception:
+        log.warning("Qdrant not reachable")
+
+    searxng_reachable = False
+    try:
+        async with httpx.AsyncClient(timeout=3) as client:
+            resp = await client.get(SEARXNG_BASE)
+            if resp.status_code == 200:
+                searxng_reachable = True
+    except Exception:
+        log.warning("SearXNG not reachable")
+
+    state = {
+        "ram_total_gb": ram_total_gb,
+        "ram_available_gb": ram_available_gb,
+        "cpu_count": cpu_count,
+        "vram_total_mb": vram_total_mb,
+        "vram_free_mb": vram_free_mb,
+        "llama_reachable": llama_reachable,
+        "llama_models": llama_models,
+        "qdrant_reachable": qdrant_reachable,
+        "qdrant_collections": qdrant_collections,
+        "searxng_reachable": searxng_reachable,
+    }
+    HARDWARE_STATE_PATH.write_text(json.dumps(state, indent=2))
+    log.info(
+        f"HW: {ram_total_gb}GB RAM, {vram_total_mb}MB VRAM, "
+        f"llama={llama_reachable}, qdrant={qdrant_reachable}, searxng={searxng_reachable}"
+    )
+    return state

memory.py

@@ -1,5 +1,5 @@
 """
-JarvisChat - FTS5 memory system.
+cAIc - FTS5 memory system.
 CRUD, search, remember/forget command processing, topic detection.
 """
 import logging
@@ -10,7 +10,7 @@ from typing import Optional
 from db import get_db
 from config import MAX_MEMORY_FACT_CHARS
 
-log = logging.getLogger("jarvischat")
+log = logging.getLogger("caic")
 
 REMEMBER_PATTERNS = [
     (r"remember that (.+)", "explicit"),
@@ -27,6 +27,80 @@ FORGET_PATTERNS = [
 ]
 
 
+AUTO_FACT_PATTERNS = [
+    re.compile(r"\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b"),
+    re.compile(r"\b(?:systemd|nginx|docker|ssh|ufw|iptables|postgres(?:ql)?|redis|mosquitto|node_exporter|prometheus|grafana|qdrant|rabbitmq|searxng|llama-server)\b", re.IGNORECASE),
+    re.compile(r"/(?:etc|home|usr|var|opt|tmp|mnt)/\S+"),
+    re.compile(r"\b(?:Ryzen|RX\s*\d{4}|RTX\s*\d{4}|Radeon|AMD|NVIDIA|Core\s*i[579]|Threadripper)\b", re.IGNORECASE),
+    re.compile(r"\b(?:Qwen|Llama|Gemma|Phi|Mistral|DeepSeek)\S*\b", re.IGNORECASE),
+    re.compile(r"\b(?:systemd\.service|docker\s+(?:compose|container|service)|systemctl|journalctl)\b", re.IGNORECASE),
+]
+SOCIAL_TRIGGERS = {"hi", "hello", "hey", "yo", "sup", "howdy", "good morning", "good evening"}
+
+
+def _is_social(text: str) -> bool:
+    t = text.strip().lower()
+    if t in SOCIAL_TRIGGERS or any(t.startswith(w) for w in ("thanks", "thank you", "ty")):
+        return True
+    return False
+
+
+def auto_detect_facts(user_message: str, assistant_message: str) -> list[str]:
+    """Extract environmental/factual content from a chat turn.
+
+    Returns a list of fact strings ready for storage. Empty list means
+    nothing worth persisting.
+    """
+    if _is_social(user_message):
+        return []
+    if len(assistant_message) < 40:
+        return []
+    if process_remember_command(user_message) is not None:
+        return []
+
+    found = []
+    for pat in AUTO_FACT_PATTERNS:
+        if pat.search(user_message):
+            found.append(user_message)
+            break
+
+    # Also capture when the user is reporting a change they made
+    change_match = re.search(
+        r"(?:I\s+)?(?:set|changed?|updated|installed|configured|enabled|disabled|added|removed|created|deleted|restarted|reloaded|switched|moved|copied|renamed|symlinked|mounted|unmounted)\s+(?:the\s+)?(.+)",
+        user_message, re.IGNORECASE,
+    )
+    if change_match and user_message not in found:
+        found.append(user_message)
+
+    seen = set()
+    deduped = []
+    for f in found:
+        key = f.strip().lower()
+        if key not in seen:
+            seen.add(key)
+            deduped.append(f.strip()[:MAX_MEMORY_FACT_CHARS])
+    return deduped
+
+
+def check_fact_conflicts(facts: list[str]) -> list[dict]:
+    """Search for existing memories that conflict with detected facts.
+
+    Returns list of {memory_id, old_fact, new_fact} for each conflict.
+    """
+    conflicts = []
+    for new_fact in facts:
+        related = search_memories(new_fact, limit=1)
+        if related:
+            old = related[0]["fact"]
+            if old.rstrip(".") != new_fact.rstrip("."):
+                conflicts.append({
+                    "memory_id": related[0]["rowid"],
+                    "old_fact": old,
+                    "new_fact": new_fact,
+                })
+    return conflicts
+
+
 def detect_topic(fact: str) -> str:
     fact_lower = fact.lower()
     if any(w in fact_lower for w in ["prefer", "like", "hate", "always", "never", "favorite"]):
@@ -62,7 +136,13 @@ def search_memories(query: str, limit: int = 5) -> list:
     if not words:
         db.close()
         return []
-    safe_query = " OR ".join(word + "*" for word in words[:10])
+    escaped = []
+    for word in words[:10]:
+        if word.upper() in {"AND", "OR", "NOT", "NEAR"}:
+            escaped.append(f'"{word}"*')
+        else:
+            escaped.append(word + "*")
+    safe_query = " OR ".join(escaped)
     try:
         rows = db.execute(
             "SELECT rowid, fact, topic, source, created_at, bm25(memories) AS rank "

node_agent/agent.py

@@ -0,0 +1,412 @@
+"""
+cAIc — Worker node agent.
+
+Standalone AMQP client that registers with the cAIc coordinator,
+responds to pings, and handles model swap commands.
+
+## Config file: /etc/caic-node-agent.conf
+
+```ini
+[agent]
+# hostname — defaults to socket.gethostname()
+node_name = jarvis
+# LAN IP — defaults from socket
+node_ip = 192.168.50.210
+# "worker" (fixed)
+node_type = worker
+# comma-separated capability list
+capabilities = llm
+# RabbitMQ URL on coordinator
+amqp_url = amqp://caic:password@192.168.50.108:5672/caic
+# port llama-server listens on
+llama_port = 8081
+# path to GGUF model files
+models_dir = /var/lib/caic/models
+# currently active model filename
+active_model = llama3.1-latest-Q4_K_M.gguf
+```
+
+## systemd unit: /etc/systemd/system/caic-node-agent.service
+
+```ini
+[Unit]
+Description=cAIc Worker Node Agent
+After=network.target rabbitmq.service
+Wants=rabbitmq.service
+
+[Service]
+Type=simple
+User=root
+Group=root
+WorkingDirectory=/opt/caic
+ExecStart=/usr/bin/python3 /opt/caic/node_agent/agent.py
+Restart=always
+RestartSec=5
+
+[Install]
+WantedBy=multi-user.target
+```
+"""
+import asyncio
+import json
+import logging
+import os
+import socket
+import subprocess
+import sys
+import time
+from configparser import ConfigParser
+from datetime import datetime, timezone
+from pathlib import Path
+from typing import Optional
+
+try:
+    import psutil
+    HAS_PSUTIL = True
+except ImportError:
+    HAS_PSUTIL = False
+
+try:
+    import aio_pika
+    from aio_pika import DeliveryMode, ExchangeType
+    HAS_AIO_PIKA = True
+except ImportError:
+    HAS_AIO_PIKA = False
+
+try:
+    import httpx
+    HAS_HTTPX = True
+except ImportError:
+    HAS_HTTPX = False
+
+log = logging.getLogger("caic")
+CONFIG_PATH = "/etc/caic-node-agent.conf"
+
+# ── data types ──────────────────────────────────────────────────────────
+
+class AgentConfig:
+    def __init__(self):
+        self.node_name: str = socket.gethostname()
+        self.node_ip: str = "127.0.0.1"
+        self.node_type: str = "worker"
+        self.capabilities: list[str] = ["llm"]
+        self.amqp_url: str = "amqp://caic:password@localhost:5672/caic"
+        self.llama_port: int = 8081
+        self.models_dir: str = "/var/lib/caic/models"
+        self.active_model: str = ""
+
+    @classmethod
+    def from_ini(cls, path: str = CONFIG_PATH) -> "AgentConfig":
+        cfg = cls()
+        parser = ConfigParser()
+        if not os.path.exists(path):
+            log.warning("config %s not found, using defaults", path)
+            return cfg
+        parser.read(path)
+        sec = "agent"
+        if parser.has_section(sec):
+            cfg.node_name = parser.get(sec, "node_name", fallback=cfg.node_name)
+            cfg.node_ip = parser.get(sec, "node_ip", fallback=cfg.node_ip)
+            cfg.node_type = parser.get(sec, "node_type", fallback=cfg.node_type)
+            raw_caps = parser.get(sec, "capabilities", fallback="llm")
+            cfg.capabilities = [c.strip() for c in raw_caps.split(",") if c.strip()]
+            cfg.amqp_url = parser.get(sec, "amqp_url", fallback=cfg.amqp_url)
+            cfg.llama_port = parser.getint(sec, "llama_port", fallback=cfg.llama_port)
+            cfg.models_dir = parser.get(sec, "models_dir", fallback=cfg.models_dir)
+            cfg.active_model = parser.get(sec, "active_model", fallback=cfg.active_model)
+        return cfg
+
+
+class ModelInfo:
+    def __init__(self, filename: str, name: str = "", version: str = "", quant: str = ""):
+        self.filename = filename
+        self.name = name
+        self.version = version
+        self.quant = quant
+        self.path = ""
+
+    def to_dict(self) -> dict:
+        return {
+            "name": self.name,
+            "version": self.version,
+            "quant": self.quant,
+            "filename": self.filename,
+        }
+
+
+# ── model discovery ─────────────────────────────────────────────────────
+
+_MODEL_PATTERN = None  # lazy compile
+
+
+def discover_models(models_dir: str) -> list[dict]:
+    import re
+    global _MODEL_PATTERN
+    if _MODEL_PATTERN is None:
+        _MODEL_PATTERN = re.compile(
+            r"^(?P<name>.+?)-(?P<version>[^-]+)-(?P<quant>Q\d+_K_[A-Z]+|IQ\d_[A-Z]+|fp\d+)\.gguf$"
+        )
+    root = Path(models_dir)
+    if not root.is_dir():
+        log.warning("models_dir %s not found", models_dir)
+        return []
+
+    results = []
+    for fpath in sorted(root.glob("*.gguf")):
+        m = _MODEL_PATTERN.match(fpath.name)
+        if m:
+            info = ModelInfo(
+                filename=fpath.name,
+                name=m.group("name"),
+                version=m.group("version"),
+                quant=m.group("quant"),
+            )
+            info.path = str(fpath)
+            results.append(info.to_dict())
+        else:
+            log.debug("skipping unrecognized model filename: %s", fpath.name)
+    return results
+
+
+# ── load reporting ──────────────────────────────────────────────────────
+
+def get_load() -> dict:
+    load = {}
+    if HAS_PSUTIL:
+        load["cpu_pct"] = round(psutil.cpu_percent(interval=0.5))
+        load["ram_pct"] = round(psutil.virtual_memory().percent)
+    try:
+        result = subprocess.run(
+            ["rocm-smi", "--showmeminfo", "vram"],
+            capture_output=True, text=True, timeout=3,
+        )
+        if result.returncode == 0:
+            for line in result.stdout.splitlines():
+                if "VRAM Total" in line:
+                    parts = line.split()
+                    if len(parts) >= 3:
+                        total = int(parts[-1])
+                elif "VRAM Used" in line:
+                    parts = line.split()
+                    if len(parts) >= 3:
+                        used = int(parts[-1])
+            if total and total > 0:
+                load["vram_pct"] = round(used / total * 100)
+    except (FileNotFoundError, subprocess.TimeoutExpired):
+        pass
+    return load
+
+
+# ── AMQP helpers ────────────────────────────────────────────────────────
+
+async def declare_exchanges(channel) -> tuple:
+    admin = await channel.declare_exchange("jc.admin", ExchangeType.TOPIC, durable=True)
+    system = await channel.declare_exchange("jc.system", ExchangeType.TOPIC, durable=True)
+    return admin, system
+
+
+async def publish(channel, exchange, routing_key, payload):
+    body = json.dumps(payload).encode()
+    msg = aio_pika.Message(body, delivery_mode=DeliveryMode.PERSISTENT)
+    await exchange.publish(msg, routing_key)
+
+
+# ── registration ────────────────────────────────────────────────────────
+
+def build_registration_payload(cfg: AgentConfig, inventory: list[dict]) -> dict:
+    model_dict = None
+    if cfg.active_model:
+        for inv in inventory:
+            if inv["filename"] == cfg.active_model:
+                model_dict = {**inv, "port": cfg.llama_port}
+                break
+    return {
+        "node_name": cfg.node_name,
+        "node_type": cfg.node_type,
+        "ip": cfg.node_ip,
+        "capabilities": cfg.capabilities,
+        "active_model": model_dict,
+        "inventory": inventory,
+    }
+
+
+# ── ping / pong ─────────────────────────────────────────────────────────
+
+async def handle_ping(cfg: AgentConfig, channel, exchange, msg: aio_pika.IncomingMessage):
+    async with msg.process():
+        try:
+            payload = json.loads(msg.body.decode())
+        except json.JSONDecodeError:
+            return
+        correlation_id = payload.get("correlation_id")
+        if not correlation_id:
+            return
+
+        load = get_load()
+        now = datetime.now(timezone.utc).isoformat() + "Z"
+        pong = {
+            "node_name": cfg.node_name,
+            "type": "pong",
+            "correlation_id": correlation_id,
+            "status": "active",
+            "active_model": None,  # simplified; could read current
+            "load": load,
+            "timestamp": now,
+        }
+        await publish(channel, exchange, f"node.{cfg.node_name}.pong", pong)
+
+
+# ── model swap ──────────────────────────────────────────────────────────
+
+async def handle_swap_model(cfg: AgentConfig, channel, exchanges, msg: aio_pika.IncomingMessage):
+    admin_ex, system_ex = exchanges
+    async with msg.process():
+        try:
+            payload = json.loads(msg.body.decode())
+        except json.JSONDecodeError:
+            return
+
+        model_filename = payload.get("model_filename")
+        if not model_filename:
+            log.error("swap_model missing model_filename")
+            return
+
+        log.info("swapping model to %s", model_filename)
+
+        # 1. Stop current llama-server
+        log.info("stopping llama-server")
+        subprocess.run(["systemctl", "stop", "llama-server"], check=False)
+
+        # 2. Update config
+        _update_config_active_model(cfg, model_filename)
+
+        # 3. Start llama-server
+        log.info("starting llama-server")
+        subprocess.run(["systemctl", "start", "llama-server"], check=False)
+
+        # 4. Poll health endpoint
+        healthy = await _wait_for_llama(cfg.llama_port, timeout=120, interval=2)
+
+        now = datetime.now(timezone.utc).isoformat() + "Z"
+        if healthy:
+            result_payload = {
+                "node_name": cfg.node_name,
+                "type": "model_ready",
+                "active_model": model_filename,
+                "port": cfg.llama_port,
+                "timestamp": now,
+            }
+            log.info("model swap successful: %s", model_filename)
+        else:
+            result_payload = {
+                "node_name": cfg.node_name,
+                "type": "model_failed",
+                "active_model": model_filename,
+                "port": cfg.llama_port,
+                "error": "llama-server did not become healthy within 120s",
+                "timestamp": now,
+            }
+            log.error("model swap failed: %s", model_filename)
+
+        await publish(channel, system_ex, f"node.{cfg.node_name}.{result_payload['type']}", result_payload)
+
+
+def _update_config_active_model(cfg: AgentConfig, model_filename: str):
+    parser = ConfigParser()
+    if os.path.exists(CONFIG_PATH):
+        parser.read(CONFIG_PATH)
+    if not parser.has_section("agent"):
+        parser.add_section("agent")
+    parser.set("agent", "active_model", model_filename)
+    with open(CONFIG_PATH, "w") as f:
+        parser.write(f)
+    cfg.active_model = model_filename
+
+
+async def _wait_for_llama(port: int, timeout: int = 120, interval: int = 2) -> bool:
+    if not HAS_HTTPX:
+        log.warning("httpx not installed, skipping health check")
+        return True
+    deadline = time.time() + timeout
+    url = f"http://localhost:{port}/v1/models"
+    async with httpx.AsyncClient() as client:
+        while time.time() < deadline:
+            try:
+                resp = await client.get(url, timeout=5)
+                if resp.status_code == 200:
+                    return True
+            except (httpx.ConnectError, httpx.TimeoutException):
+                pass
+            await asyncio.sleep(interval)
+    return False
+
+
+# ── main ────────────────────────────────────────────────────────────────
+
+async def amain():
+    logging.basicConfig(
+        level=logging.INFO,
+        format="%(asctime)s caic[%(process)d]: %(levelname)s %(message)s",
+    )
+    log.info("cAIc node agent starting")
+
+    if not HAS_AIO_PIKA:
+        log.error("aio-pika not installed")
+        sys.exit(1)
+
+    cfg = AgentConfig.from_ini()
+    log.info("node_name=%s node_ip=%s", cfg.node_name, cfg.node_ip)
+
+    inventory = discover_models(cfg.models_dir)
+    log.info("discovered %d models", len(inventory))
+
+    # Connect to AMQP
+    conn = await aio_pika.connect_robust(cfg.amqp_url)
+    channel = await conn.channel()
+    admin_ex, system_ex = await declare_exchanges(channel)
+    log.info("connected to AMQP broker")
+
+    # Publish registration
+    reg_payload = build_registration_payload(cfg, inventory)
+    await publish(channel, admin_ex, f"node.{cfg.node_name}.register", reg_payload)
+    log.info("registration published")
+
+    # Wait for admission
+    response_queue = await channel.declare_queue("", exclusive=True)
+    await response_queue.bind(admin_ex, f"node.{cfg.node_name}.admitted")
+    await response_queue.bind(admin_ex, f"node.{cfg.node_name}.rejected")
+
+    admitted = False
+    async with response_queue.iterator() as iterator:
+        async for message in iterator:
+            async with message.process():
+                payload = json.loads(message.body.decode())
+                if payload.get("type") == "admitted":
+                    log.info("admitted to cluster")
+                    admitted = True
+                    break
+                else:
+                    log.error("rejected: %s", payload.get("reason"))
+                    sys.exit(1)
+
+    if not admitted:
+        log.error("no admission response received")
+        sys.exit(1)
+
+    # Set up ping consumer
+    ping_queue = await channel.declare_queue("", exclusive=True)
+    await ping_queue.bind(admin_ex, f"node.{cfg.node_name}.ping")
+    await ping_queue.consume(lambda msg: handle_ping(cfg, channel, admin_ex, msg))
+
+    # Set up swap consumer
+    swap_queue = await channel.declare_queue("", exclusive=True)
+    await swap_queue.bind(admin_ex, f"node.{cfg.node_name}.cmd.swap_model")
+    await swap_queue.consume(lambda msg: handle_swap_model(cfg, channel, (admin_ex, system_ex), msg))
+
+    log.info("listening for pings and commands")
+    # Run forever
+    await asyncio.Event().wait()
+
+
+if __name__ == "__main__":
+    asyncio.run(amain())

node_agent/requirements.txt

@@ -0,0 +1,3 @@
+aio-pika>=9.0.0
+psutil>=5.9.0
+httpx>=0.27.0

rag.py

@@ -1,22 +1,143 @@
 """
-JarvisChat - RAG pipeline: Qdrant vector search + system prompt assembly.
+cAIc - RAG pipeline: Qdrant vector search + system prompt assembly.
 """
+import asyncio
 import logging
+from datetime import datetime, timezone
 
 import httpx
 
+from eviction import _update_retrieval_count
 from db import get_db, get_setting, list_skills_with_state, format_active_skills_prompt
 from memory import search_memories
-from config import MAX_SKILL_PROMPT_CHARS
+from config import MAX_SKILL_PROMPT_CHARS, QDRANT_URL, RAG_COLLECTION
 
-log = logging.getLogger("jarvischat")
+log = logging.getLogger("caic")
 
-QDRANT_URL = "http://192.168.50.108:6333"
+EMBED_URL = "http://192.168.50.210:11434"
-EMBED_URL = "http://192.168.50.108:11434"
 EMBED_MODEL = "mxbai-embed-large"
-RAG_COLLECTION = "jarvis_rag"
 RAG_SCORE_THRESHOLD = 0.25
 
+# Re-export eviction symbols for backward compatibility
+from eviction import (  # noqa: E402
+    maybe_evict, get_rag_operational_stats, EVICTION_LOG,
+    get_collection_count, get_collection_stats, evict_batch,
+)
+
+
+async def _upsert_fact(fact: str, text: str, topic: str,
+                       client: httpx.AsyncClient) -> bool:
+    """Embed text and upsert a fact to Qdrant."""
+    chunks = chunk_text(text)
+    if not chunks:
+        return False
+    ts = datetime.now(timezone.utc).timestamp()
+    ok = False
+    for i, chunk in enumerate(chunks):
+        try:
+            er = await client.post(
+                f"{EMBED_URL}/api/embeddings",
+                json={"model": EMBED_MODEL, "prompt": chunk},
+                timeout=10.0,
+            )
+            if er.status_code != 200:
+                continue
+            vector = er.json()["embedding"]
+            pid = f"auto-{ts}-{i}"
+            payload = {
+                "text": chunk, "source": "auto_fact", "fact": fact,
+                "ingest_date": datetime.now(timezone.utc).isoformat(),
+                "type": "auto_fact", "topic": topic,
+            }
+            r = await client.put(
+                f"{QDRANT_URL}/collections/{RAG_COLLECTION}/points?wait=true",
+                json={"points": [{"id": pid, "vector": vector, "payload": payload}]},
+                timeout=10.0,
+            )
+            if r.status_code in (200, 201):
+                ok = True
+        except Exception as e:
+            log.warning(f"Qdrant upsert error: {e}")
+    return ok
+
+
+async def ingest_auto_fact(facts: list[str], user_message: str,
+                           assistant_message: str) -> int:
+    """Persist pre-detected facts to memories + Qdrant.
+
+    Call this when no conflicts exist — silent ingest.
+    Returns the number of facts stored.
+    """
+    from memory import add_memory, detect_topic
+
+    ingested = 0
+    async with httpx.AsyncClient() as client:
+        for fact in facts:
+            topic = detect_topic(fact)
+            add_memory(fact, topic=topic, source="auto")
+            ingested += 1
+            text = f"Q: {user_message}\nA: {assistant_message}"
+            await _upsert_fact(fact, text, topic, client)
+    if ingested:
+        log.info(f"Auto-ingested {ingested} fact(s) from conversation")
+    return ingested
+
+
+async def confirm_fact_update(memory_id: int, old_fact: str, new_fact: str,
+                              user_message: str, assistant_message: str) -> bool:
+    """Confirm a user-accepted fact update: replace memory + Qdrant entry."""
+    from memory import update_memory, detect_topic
+
+    if not update_memory(memory_id, new_fact):
+        return False
+
+    topic = detect_topic(new_fact)
+    try:
+        async with httpx.AsyncClient() as client:
+            # scroll old points with matching fact and delete them
+            scroll_r = await client.post(
+                f"{QDRANT_URL}/collections/{RAG_COLLECTION}/points/scroll",
+                json={
+                    "filter": {"must": [{"key": "fact", "match": {"value": old_fact}}]},
+                    "limit": 100,
+                    "with_payload": False,
+                },
+                timeout=10.0,
+            )
+            if scroll_r.status_code == 200:
+                ids = [p["id"] for p in scroll_r.json().get("result", [])]
+                if ids:
+                    await client.post(
+                        f"{QDRANT_URL}/collections/{RAG_COLLECTION}/points/delete",
+                        json={"points": ids},
+                        timeout=10.0,
+                    )
+
+            text = f"Q: {user_message}\nA: {assistant_message}"
+            await _upsert_fact(new_fact, text, topic, client)
+    except Exception as e:
+        log.warning(f"Fact update RAG error: {e}")
+
+    log.info(f"Fact updated [memory_id={memory_id}]: {new_fact}")
+    return True
+
+
+def chunk_text(text: str, chunk_size: int = 512, overlap: int = 128) -> list:
+    words = text.split()
+    target_words = int(chunk_size / 1.3)
+    overlap_words = int(overlap / 1.3)
+    if not words:
+        return []
+    chunks = []
+    start = 0
+    while start < len(words):
+        end = min(start + target_words, len(words))
+        chunks.append(" ".join(words[start:end]))
+        if end == len(words):
+            break
+        start += target_words - overlap_words
+    return chunks
+
 
 async def query_rag(query: str, limit: int = 3) -> list:
     try:
@@ -36,7 +157,13 @@ async def query_rag(query: str, limit: int = 3) -> list:
             )
             if search_resp.status_code != 200:
                 return []
-            return search_resp.json().get("result", [])
+            results = search_resp.json().get("result", [])
+            for r in results:
+                pid = r.get("id")
+                if pid:
+                    current = r.get("payload", {}).get("retrieval_count", 0) or 0
+                    asyncio.ensure_future(_update_retrieval_count(pid, current))
+            return results
     except Exception as e:
         log.warning(f"RAG query error: {e}")
         return []
@@ -65,7 +192,7 @@ async def build_system_prompt(db, extra_prompt: str = "", user_message: str = ""
                 rag_lines = [r["payload"]["text"] for r in rag_results if r["score"] > RAG_SCORE_THRESHOLD]
                 if rag_lines:
                     parts.append("## Retrieved Context\n" + "\n\n---\n\n".join(rag_lines))
-                    log.warning(f"RAG injected {len(rag_lines)} chunks into context")
+                    log.info(f"RAG injected {len(rag_lines)} chunks into context")
         except Exception as e:
             log.warning(f"RAG injection error: {e}")
 

readme.md

@@ -1,355 +0,0 @@
-# ⚡ JarvisChat v1.7.8
-
-![screenshot](docs/images/screenshot.png)
-
-**A lightweight Ollama coding companion with persistent memory, web search, and real-time system monitoring.**
-
-Built with FastAPI + SQLite + Jinja2. Runs on Python 3.13. No Docker required.
-
-Developer wiki: [docs/wiki/Home.md](docs/wiki/Home.md)
-
-Core architecture deep-dive: [docs/wiki/Developer-Architecture.md](docs/wiki/Developer-Architecture.md)
-
-## Security Scope Disclaimer
-
-JarvisChat is designed for local and home-lab use (same host or trusted LAN).
-
-JarvisChat may technically work with frontier or commercial AI endpoints, but the author does not recommend or support that usage.
-
-Supported deployments are contained local/home-lab environments.
-
-By default, API access is limited to loopback + private LAN CIDRs. You can override with `JARVISCHAT_ALLOWED_CIDRS` (comma-separated CIDRs) and optionally trust reverse-proxy forwarding with `JARVISCHAT_TRUST_X_FORWARDED_FOR=true`.
-
-If you deploy outside a trusted local subnet, your risk profile changes significantly and the default protections here may be insufficient.
-
-Use at your own risk. No warranty is provided for Internet-exposed deployments.
-
-## What's New in v1.7.x
-
-- **Security hardening suite completed** - request rate limits, payload caps, settings allowlist, safe error envelopes, and LAN CIDR gate controls
-- **Customer-safe incident handling** - client-facing errors include support-friendly incident keys while full traces remain in server logs
-- **Streaming and regression test expansion** - automated coverage for SSE chat/search paths, memory remember/forget command handling, and auth/guardrail behavior
-- **Skills framework (Phase 1)** - built-in local skill registry with per-skill enable controls, API endpoints, and bounded prompt injection
-- **Skills WebUX controls** - Settings modal now includes a master skills toggle and per-skill toggles for admin users
-
-## What's New in v1.6.x
-
-- **Guest/admin capability split** - guest chat by default with 4-digit admin PIN for advanced or destructive operations
-- **Session + lockout controls** - session lifecycle endpoints, heartbeat, logout/revoke behavior, failed PIN lockout protections, and auth audit events
-- **Browser request protections** - strict origin checks for state-changing requests and admin-only write enforcement
-- **Unsafe link protection** - outbound search links sanitized to allow only http/https absolute URLs
-- **Operational stability fixes** - safer first-boot PIN policy handling and memory-search tokenization fix for punctuation/FTS edge cases
-
-## What's New in v1.5.0
-
-- **Explicit Web Search Button** — 🔍 button next to SEND forces a web search, bypassing model uncertainty detection
-- **Orange Search Styling** — Search results, WEB badge, and search button share consistent orange color scheme
-- **Expanded Refusal Patterns** — Added "As an AI model", "based on my training data", "I don't have the capability"
-- **Code cleanup** — Removed unused `JSONResponse` import and dead `raw_results_md` variable
-- **Bug fixes** — Replaced bare `except` clauses with `except Exception`; corrected `add_memory()` return type to `int | None`; updated `TemplateResponse` call to Starlette's current API signature
-
-## What's New in v1.4.0
-
-- **FTS5 Memory System**: Say "remember that..." to store facts — they're automatically retrieved by relevance and injected into context
-- **Forget Command**: Say "forget about..." to remove memories
-- **Memory Toggle**: Enable/disable memory injection from topbar or settings
-- **Multi-file Structure**: Backend and frontend separated for easier maintenance
-
-## Features
-
-- **Persistent Memory** — SQLite FTS5 full-text search for fast, relevant memory retrieval
-- **Web Search** — SearXNG integration for automatic web lookups when the model is uncertain
-- **Explicit Search** — 🔍 button to force web search without waiting for model uncertainty
-- **Profile Injection** — Custom system prompt injected into every conversation
-- **System Presets** — Save and switch between different system prompts
-- **Real-time Stats** — CPU, RAM, GPU, VRAM monitoring in sidebar
-- **Token Thermometer** — Visual context window usage indicator
-- **Streaming Responses** — Server-sent events for real-time token display
-- **Conversation History** — SQLite-backed chat persistence with mass-delete option
-- **Model Switching** — Change Ollama models on the fly
-
-## Current WiP (Prioritized)
-
-Canonical backlog: [docs/wiki/current-wip.md](docs/wiki/current-wip.md)
-
-Scope boundary: local-first (same-host Ollama), optional RFC1918 LAN endpoints, no public Internet AI endpoints by default.
-
-Total identified items: 27
-
-Top 10 (brief):
-
-1. P0 [DONE]: Add auth for write/admin endpoints
-2. P0 [DONE]: Add CSRF/origin protection for state-changing requests
-3. P0 [DONE]: Block unsafe URL schemes in rendered links
-4. P0 [DONE]: Add rate limiting and request size limits
-5. P1 [DONE]: Restrict `/api/settings` updates to allowlisted keys
-6. P1: Add pagination + hard caps for list APIs
-7. P1 [DONE]: Replace raw exception leakage with safe client errors
-8. P1 [DONE]: Add automated tests for streaming/search/memory paths
-9. P2 [DONE]: Implement MCP-style skills/tool-call framework
-10. P2: Implement heartbeat/check-in scheduler + summary endpoint
-
-Item 1 executive summary: keep guest mode for conversational chat, require 4-digit admin PIN for advanced/destructive actions, and enforce local/LAN-only backend policy by default.
-
-Implementation status: complete (guest session by default + admin unlock + admin-only write enforcement + origin checks + safe-link sanitization + audit logging + rate/payload guardrails + capability tests).
-
-## TODO
-
-1. ~~Verify SearXNG and Docker services persist across reboots~~
-2. Conversation search/filter by keyword
-3. Export conversation to markdown/text
-4. Keyboard shortcuts (Ctrl+N new chat, Ctrl+Enter send)
-5. Retry button on assistant messages
-6. Source links — clickable links when search used
-7. Allow conversation renaming
-8. Multiple profiles — coding/sysadmin/general
-9. Auto-generate conversation tags (client-side KWIC, top 5, filterable badges)
-10. Image input support — pull vision model, file input/drag-drop, base64 encode, pass `images` array to Ollama `/api/chat`
-11. Split-screen option for btop display
-12. Skills as markdown files — `/opt/jarvischat/skills/`, YAML frontmatter + instructions, injected into context for tool calls
-13. Heartbeats / proactive check-ins — cron + endpoint for daily briefings, HA anomaly alerts
-14. Model info button — (i) icon next to Model dropdown, shows div with model description, last updated date, best-use purpose
-15. Set default model — toggle any model as the default selection
-16. Hide/remove model from list — exclude models from dropdown
-17. Update model function — trigger `ollama pull` for selected model from UI
-18. Add mouseover tooltip to SEND button
-19. Add preflight validation for required model/preset selection and show a clear warning before send to prevent avoidable timeout loops
-
-## File Structure
-
-```
-/opt/jarvischat/
-├── app.py              # FastAPI backend
-├── jarvischat.db       # SQLite database (auto-created)
-├── static/
-│   └── logo.png        # Logo image (optional)
-└── templates/
-    └── index.html      # Frontend
-```
-
-## Requirements
-
-- Python 3.11+ (tested on 3.13)
-- Ollama running locally or on network
-- SearXNG (optional, for web search)
-
-## Installation
-
-### Fresh Install
-
-```bash
-# Create directory and venv
-sudo mkdir -p /opt/jarvischat
-sudo chown $USER:$USER /opt/jarvischat
-cd /opt/jarvischat
-python3 -m venv venv
-
-# Install dependencies
-./venv/bin/pip install fastapi uvicorn httpx psutil jinja2 python-multipart
-
-# Set admin PIN before first startup (4 digits)
-export JARVISCHAT_ADMIN_PIN=4827
-
-# Create subdirectories
-mkdir -p templates static
-
-# Copy files
-# (copy app.py to /opt/jarvischat/)
-# (copy index.html to /opt/jarvischat/templates/)
-# (copy logo.png to /opt/jarvischat/static/ — optional)
-```
-
-WARNING: Do not use `1234` as your admin PIN unless you accept weak local security.
-
-NOTE: First boot now requires `JARVISCHAT_ADMIN_PIN` unless you explicitly opt into insecure fallback with `JARVISCHAT_ALLOW_DEFAULT_PIN=true`.
-
-### Upgrading from v1.4.x
-
-```bash
-cd /opt/jarvischat
-
-# Backup
-cp app.py app.py.bak
-cp templates/index.html templates/index.html.bak
-
-# Copy new files
-# (copy app.py, replacing old version)
-# (copy index.html to templates/)
-
-# Restart
-sudo systemctl restart jarvischat
-```
-
-## Systemd Service
-
-Create `/etc/systemd/system/jarvischat.service`:
-
-```ini
-[Unit]
-Description=JarvisChat - Local Ollama Web Interface
-After=network.target
-
-[Service]
-Type=simple
-User=jarvischat
-Group=jarvischat
-WorkingDirectory=/opt/jarvischat
-ExecStart=/opt/jarvischat/venv/bin/uvicorn app:app --host 0.0.0.0 --port 8080
-Restart=always
-RestartSec=5
-
-[Install]
-WantedBy=multi-user.target
-```
-
-```bash
-sudo systemctl daemon-reload
-sudo systemctl enable jarvischat
-sudo systemctl start jarvischat
-```
-
-## Memory Commands
-
-In chat, natural language triggers memory operations:
-
-| You say | What happens |
-|---------|--------------|
-| "remember that I prefer Rust over Go" | Stores as `preference` |
-| "remember that JarvisChat runs on port 8080" | Stores as `infrastructure` |
-| "note that the deadline is Friday" | Stores as `general` |
-| "forget about the deadline" | Removes matching memories |
-
-Memories are automatically searched based on your message content and injected into the system prompt when relevant.
-
-### Memory Topics
-
-Memories are auto-categorized:
-- `preference` — likes, dislikes, choices
-- `project` — active work, repos, tasks
-- `infrastructure` — servers, services, configs
-- `personal` — name, location, background
-- `general` — everything else
-
-## API Endpoints
-
-### Memory
-
-| Method | Endpoint | Description |
-|--------|----------|-------------|
-| GET | `/api/memories` | List all memories |
-| POST | `/api/memories` | Add memory `{"fact": "...", "topic": "general"}` |
-| DELETE | `/api/memories/{rowid}` | Delete memory by ID |
-| GET | `/api/memories/search?q=term` | Search memories |
-| GET | `/api/memories/stats` | Get counts by topic |
-
-### Chat & Models
-
-| Method | Endpoint | Description |
-|--------|----------|-------------|
-| GET | `/api/models` | List available Ollama models |
-| POST | `/api/chat` | Send message (streaming SSE) |
-| POST | `/api/search` | Explicit web search (streaming SSE) |
-| POST | `/api/show` | Get model info (context size) |
-| GET | `/api/ps` | Get running models |
-
-### Settings & Profile
-
-| Method | Endpoint | Description |
-|--------|----------|-------------|
-| GET | `/api/profile` | Get profile content |
-| PUT | `/api/profile` | Update profile |
-| GET | `/api/profile/default` | Get default profile |
-| GET | `/api/settings` | Get settings |
-| PUT | `/api/settings` | Update settings |
-
-### Conversations
-
-| Method | Endpoint | Description |
-|--------|----------|-------------|
-| GET | `/api/conversations` | List conversations |
-| GET | `/api/conversations/{id}` | Get conversation with messages |
-| DELETE | `/api/conversations/{id}` | Delete conversation |
-| DELETE | `/api/conversations` | Delete ALL conversations |
-
-### Presets
-
-| Method | Endpoint | Description |
-|--------|----------|-------------|
-| GET | `/api/presets` | List presets |
-| POST | `/api/presets` | Create preset |
-| PUT | `/api/presets/{id}` | Update preset |
-| DELETE | `/api/presets/{id}` | Delete preset |
-
-### System
-
-| Method | Endpoint | Description |
-|--------|----------|-------------|
-| GET | `/api/stats` | CPU, RAM, GPU, VRAM stats |
-| GET | `/api/search/status` | SearXNG availability |
-
-## Configuration
-
-Settings are stored in the `settings` table and include:
-
-- `profile_enabled` — Inject profile into chats (true/false)
-- `search_enabled` — Auto web search (true/false)
-- `memory_enabled` — Memory injection (true/false)
-- `default_model` — Default Ollama model
-- `searxng_url` — SearXNG instance URL (default: `http://localhost:8888`)
-
-## Testing Memory
-
-```bash
-# Add a memory via API
-curl -X POST http://jarvis:8080/api/memories \
-  -H "Content-Type: application/json" \
-  -d '{"fact": "User prefers native installs over Docker", "topic": "preference"}'
-
-# Search memories
-curl "http://jarvis:8080/api/memories/search?q=docker"
-
-# List all memories
-curl http://jarvis:8080/api/memories
-
-# Get stats
-curl http://jarvis:8080/api/memories/stats
-```
-
-Or in chat:
-1. Say "remember that I hate YAML"
-2. Later ask "what markup languages should I avoid?"
-3. JarvisChat will inject the YAML preference into context
-
-## Troubleshooting
-
-### Service won't start
-
-Check logs:
-```bash
-journalctl -u jarvischat -n 50 --no-pager
-```
-
-Common issues:
-- Missing `jinja2`: `./venv/bin/pip install jinja2`
-- Missing `templates/` directory
-- Wrong permissions on `/opt/jarvischat`
-
-### Memory not working
-
-1. Check memory is enabled (🧠 MEM ON in topbar)
-2. Verify memories exist: `curl http://jarvis:8080/api/memories`
-3. Check FTS5 table: `sqlite3 jarvischat.db "SELECT * FROM memories_fts;"`
-
-### Web search not working
-
-1. Verify SearXNG is running: `curl http://localhost:8888/search?q=test&format=json`
-2. Check search status: `curl http://jarvis:8080/api/search/status`
-3. Ensure JSON format is enabled in SearXNG settings
-
-## License
-
-MIT
-
-## Repository
-
-Gitea: `ssh://gitea@llgit.llamachile.tube:1319/gramps/jarvisChat.git`

requirements.txt

@@ -1,3 +1,6 @@
 fastapi>=0.115.0
 uvicorn[standard]>=0.32.0
 httpx>=0.27.0
+pypdf>=5.0.0
+python-multipart>=0.0.9
+aio-pika>=9.0.0

routers/chat.py

@@ -1,4 +1,5 @@
 """JarvisChat routers - /api/chat streaming endpoint."""
+import asyncio
 import json
 import logging
 import uuid
@@ -9,16 +10,16 @@ from fastapi import APIRouter, HTTPException, Request
 from fastapi.responses import StreamingResponse
 
 from config import DEFAULT_MODEL, LLAMA_SERVER_BASE
-from db import get_db
+from db import get_db, get_upload_context
-from memory import process_remember_command
+from memory import process_remember_command, auto_detect_facts, check_fact_conflicts
-from rag import build_system_prompt
+from rag import build_system_prompt, ingest_auto_fact
 from search import (calculate_perplexity, is_uncertain, is_refusal,
                     clean_hedging, format_search_results, format_direct_answer,
                     extract_search_query, query_searxng)
 from security import read_json_body, log_incident, BODY_LIMIT_CHAT_BYTES
-from config import MAX_CHAT_MESSAGE_CHARS
+from config import MAX_CHAT_MESSAGE_CHARS, MODEL_CONTEXT_LENGTH
 
-log = logging.getLogger("jarvischat")
+log = logging.getLogger("caic")
 router = APIRouter()
 
 
@@ -26,7 +27,7 @@ def parse_llama_stream_chunk(line: str) -> tuple:
     if line.startswith("data: "):
         line = line[6:]
     if line.strip() == "[DONE]":
-        return None, True, {}
+        return None, True, {}, []
     try:
         chunk = json.loads(line)
         choices = chunk.get("choices", [])
@@ -35,10 +36,19 @@ def parse_llama_stream_chunk(line: str) -> tuple:
             token = delta.get("content")
             finish = choices[0].get("finish_reason")
             stats = {}
+            logprobs_list = []
+            logprobs_info = choices[0].get("logprobs")
+            if logprobs_info:
+                content_logprobs = logprobs_info.get("content", [])
+                for entry in content_logprobs:
+                    if "logprob" in entry:
+                        logprobs_list.append({"logprob": entry["logprob"]})
             if finish == "stop":
                 usage = chunk.get("usage", {})
                 stats["tokens_per_sec"] = usage.get("tokens_per_second", 0.0)
-            return token, finish == "stop", stats
+                stats["completion_tokens"] = usage.get("completion_tokens", 0)
+                stats["prompt_tokens"] = usage.get("prompt_tokens", 0)
+            return token, finish == "stop", stats, logprobs_list
         if "message" in chunk and "content" in chunk["message"]:
             token = chunk["message"]["content"]
             done = chunk.get("done", False)
@@ -47,10 +57,11 @@ def parse_llama_stream_chunk(line: str) -> tuple:
                 eval_count = chunk.get("eval_count", 0)
                 eval_duration = chunk.get("eval_duration", 0)
                 stats["tokens_per_sec"] = (eval_count / (eval_duration / 1e9)) if eval_duration > 0 else 0
-            return token, done, stats
+                stats["completion_tokens"] = eval_count
+            return token, done, stats, []
     except json.JSONDecodeError:
         pass
-    return None, False, {}
+    return None, False, {}, []
 
 
 @router.post("/api/chat")
@@ -62,6 +73,7 @@ async def chat(request: Request):
         raise HTTPException(status_code=413, detail="Chat message is too long")
     model = body.get("model", DEFAULT_MODEL)
     preset_prompt = body.get("system_prompt", "")
+    upload_context_id = body.get("upload_context_id")
 
     if not user_message:
         raise HTTPException(status_code=400, detail="Empty message")
@@ -71,6 +83,14 @@ async def chat(request: Request):
     settings = {row["key"]: row["value"] for row in db.execute("SELECT key, value FROM settings").fetchall()}
     search_enabled = settings.get("search_enabled", "true") == "true"
 
+    upload_doc = None
+    if upload_context_id:
+        ctx = get_upload_context(db, upload_context_id)
+        if ctx:
+            upload_doc = f"[ATTACHED DOCUMENT: {ctx['filename']}]\n{ctx['content']}\n[END DOCUMENT]"
+        else:
+            log.warning(f"upload_context_id {upload_context_id} not found or expired, continuing without it")
+
     remember_response = process_remember_command(user_message)
 
     if not conv_id:
@@ -88,7 +108,10 @@ async def chat(request: Request):
     history_rows = db.execute(
         "SELECT role, content FROM messages WHERE conversation_id = ? ORDER BY id ASC", (conv_id,)
     ).fetchall()
-    system_prompt = await build_system_prompt(db, preset_prompt, user_message)
+    extra_prompt = preset_prompt
+    if upload_doc:
+        extra_prompt = (extra_prompt + "\n\n" + upload_doc) if extra_prompt else upload_doc
+    system_prompt = await build_system_prompt(db, extra_prompt, user_message)
     db.close()
 
     messages = []
@@ -97,12 +120,15 @@ async def chat(request: Request):
     for row in history_rows:
         messages.append({"role": row["role"], "content": row["content"]})
 
-    ollama_payload = {"model": model, "messages": messages, "stream": True}
+    upstream_payload = {"model": model, "messages": messages, "stream": True, "logprobs": True}
 
     async def stream_response():
         full_response = []
         all_logprobs = []
         tokens_per_sec = 0.0
+        completion_tokens = 0
+        prompt_tokens = 0
+        rag_update = None
 
         if remember_response:
             yield f"data: {json.dumps({'token': remember_response + chr(10) + chr(10), 'conversation_id': conv_id})}\n\n"
@@ -111,17 +137,21 @@ async def chat(request: Request):
             try:
                 async with client.stream(
                     "POST", f"{LLAMA_SERVER_BASE}/v1/chat/completions",
-                    json=ollama_payload,
+                    json=upstream_payload,
                     timeout=httpx.Timeout(300.0, connect=10.0),
                 ) as resp:
                     async for line in resp.aiter_lines():
                         if line.strip():
-                            token, done, stats = parse_llama_stream_chunk(line)
+                            token, done, stats, chunk_logprobs = parse_llama_stream_chunk(line)
+                            if chunk_logprobs:
+                                all_logprobs.extend(chunk_logprobs)
                             if token:
                                 full_response.append(token)
                                 yield f"data: {json.dumps({'token': token, 'conversation_id': conv_id})}\n\n"
                             if done:
                                 tokens_per_sec = stats.get("tokens_per_sec", 0.0)
+                                completion_tokens = stats.get("completion_tokens", 0)
+                                prompt_tokens = stats.get("prompt_tokens", 0)
 
                 assistant_msg = "".join(full_response)
                 perplexity = calculate_perplexity(all_logprobs) if all_logprobs else 0.0
@@ -153,7 +183,7 @@ async def chat(request: Request):
                         ) as resp2:
                             async for line in resp2.aiter_lines():
                                 if line.strip():
-                                    token2, done2, _ = parse_llama_stream_chunk(line)
+                                    token2, done2, _, _ = parse_llama_stream_chunk(line)
                                     if token2:
                                         augmented_response.append(token2)
                                     if done2:
@@ -176,7 +206,15 @@ async def chat(request: Request):
                         db2.commit()
                         db2.close()
 
-                        yield f"data: {json.dumps({'done': True, 'conversation_id': conv_id, 'searched': True, 'perplexity': round(perplexity, 2), 'tokens_per_sec': round(tokens_per_sec, 1)})}\n\n"
+                        facts = auto_detect_facts(user_message, cleaned_response)
+                        if facts:
+                            conflicts = check_fact_conflicts(facts)
+                            if conflicts:
+                                rag_update = {"conflicts": conflicts}
+                            else:
+                                asyncio.ensure_future(ingest_auto_fact(facts, user_message, cleaned_response))
+
+                        yield f"data: {json.dumps({'done': True, 'conversation_id': conv_id, 'searched': True, 'perplexity': round(perplexity, 2), 'tokens_per_sec': round(tokens_per_sec, 1), 'prompt_tokens': prompt_tokens, 'completion_tokens': completion_tokens, 'context_length': MODEL_CONTEXT_LENGTH, **(rag_update and {'rag_update_suggestion': rag_update} or {})})}\n\n"
                         return
 
                 saved_msg = assistant_msg
@@ -189,14 +227,22 @@ async def chat(request: Request):
                 db2.commit()
                 db2.close()
 
-                yield f"data: {json.dumps({'done': True, 'conversation_id': conv_id, 'perplexity': round(perplexity, 2), 'tokens_per_sec': round(tokens_per_sec, 1)})}\n\n"
+                facts = auto_detect_facts(user_message, assistant_msg)
+                if facts:
+                    conflicts = check_fact_conflicts(facts)
+                    if conflicts:
+                        rag_update = {"conflicts": conflicts}
+                    else:
+                        asyncio.ensure_future(ingest_auto_fact(facts, user_message, assistant_msg))
+
+                yield f"data: {json.dumps({'done': True, 'conversation_id': conv_id, 'perplexity': round(perplexity, 2), 'tokens_per_sec': round(tokens_per_sec, 1), 'prompt_tokens': prompt_tokens, 'completion_tokens': completion_tokens, 'context_length': MODEL_CONTEXT_LENGTH, **(rag_update and {'rag_update_suggestion': rag_update} or {})})}\n\n"
 
             except httpx.RemoteProtocolError:
                 pass
             except httpx.ConnectError:
-                yield f"data: {json.dumps({'error': 'Cannot connect to Ollama. Is it running?'})}\n\n"
+                yield f"data: {json.dumps({'error': 'Cannot connect to inference server. Is it running?'})}\n\n"
             except Exception as e:
-                incident_key = log_incident("chat_stream", message="Ollama stream failure during chat response",
+                incident_key = log_incident("chat_stream", message="Inference stream failure during chat response",
                                             request=request, exc=e)
                 yield f"data: {json.dumps({'error': 'Chat response generation failed before completion. Use the incident key for support lookup.', 'error_key': incident_key})}\n\n"
 

routers/cluster.py

@@ -0,0 +1,23 @@
+"""JarvisChat routers - Cluster status API."""
+from fastapi import APIRouter
+
+import cluster
+
+router = APIRouter()
+
+
+@router.get("/api/cluster")
+async def cluster_status():
+    return {
+        "nodes": {name: _strip_internal(node) for name, node in cluster.CLUSTER_NODES.items()},
+        "node_count": len(cluster.CLUSTER_NODES),
+        "coordinator": cluster.CLUSTER_COORDINATOR,
+        "events": list(cluster.CLUSTER_EVENTS),
+    }
+
+
+def _strip_internal(node: dict) -> dict:
+    return {k: v for k, v in node.items() if k in {
+        "name", "type", "status", "capabilities", "active_model", "load",
+        "registered_at", "last_seen",
+    }}

routers/completions.py

@@ -20,7 +20,7 @@ from db import get_db
 from rag import build_system_prompt
 from routers.chat import parse_llama_stream_chunk
 
-log = logging.getLogger("jarvischat")
+log = logging.getLogger("caic")
 router = APIRouter()
 
 
@@ -86,6 +86,7 @@ def _build_openai_response(content: str, model: str, conv_id: str) -> dict:
 
 
 @router.post("/v1/chat/completions")
+@router.post("/v1/completions")
 async def chat_completions(request: Request):
     _check_api_key(request)
 
@@ -178,7 +179,7 @@ async def _stream_chat(payload: dict, model: str, conv_id: str, request: Request
                 async for line in resp.aiter_lines():
                     if not line.strip():
                         continue
-                    token, done, _ = parse_llama_stream_chunk(line)
+                    token, done, _, _ = parse_llama_stream_chunk(line)
                     if token:
                         full_response.append(token)
                         yield _build_openai_chunk(token, model, conv_id)
@@ -222,7 +223,7 @@ async def _blocking_chat(payload: dict, model: str, conv_id: str, request: Reque
                 async for line in resp.aiter_lines():
                     if not line.strip():
                         continue
-                    token, done, _ = parse_llama_stream_chunk(line)
+                    token, done, _, _ = parse_llama_stream_chunk(line)
                     if token:
                         full_response.append(token)
                     if done:
@@ -247,6 +248,79 @@ async def _blocking_chat(payload: dict, model: str, conv_id: str, request: Reque
     return JSONResponse(content=_build_openai_response(assistant_msg, model, conv_id))
 
 
+@router.post("/v1/fim/completions")
+@router.post("/fim/completions")
+async def fim_completions(request: Request):
+    _check_api_key(request)
+    try:
+        body = await request.json()
+    except Exception:
+        raise HTTPException(status_code=400, detail="Invalid JSON body")
+
+    prefix = body.get("prompt", "")
+    suffix = body.get("suffix", "")
+    stream = body.get("stream", True)
+    max_tokens = body.get("max_tokens", 128)
+
+    fim_prompt = f"<|fim_prefix|>{prefix}<|fim_suffix|>{suffix}<|fim_middle|>"
+
+    upstream = {
+        "prompt": fim_prompt,
+        "n_predict": max_tokens,
+        "temperature": body.get("temperature", 0),
+        "stop": body.get("stop", []),
+        "stream": True,
+        "cache_prompt": False,
+    }
+
+    async def _stream_fim():
+        async with httpx.AsyncClient() as client:
+            try:
+                async with client.stream(
+                    "POST", f"{LLAMA_SERVER_BASE}/completion",
+                    json=upstream,
+                    timeout=httpx.Timeout(30.0, connect=5.0),
+                ) as resp:
+                    async for line in resp.aiter_lines():
+                        if not line or not line.startswith("data: "):
+                            continue
+                        try:
+                            d = json.loads(line[6:])
+                            content = d.get("content", "")
+                            if content:
+                                chunk = {
+                                    "choices": [{"delta": {"content": content}, "index": 0}]
+                                }
+                                yield f"data: {json.dumps(chunk)}\n\n"
+                            if d.get("stop"):
+                                break
+                        except json.JSONDecodeError:
+                            continue
+                yield "data: [DONE]\n\n"
+            except httpx.ConnectError:
+                yield f"data: {json.dumps({'error': 'Cannot connect to inference server'})}\n\n"
+
+    if stream:
+        return StreamingResponse(_stream_fim(), media_type="text/event-stream")
+
+    # Non-streaming: accumulate and return
+    full = []
+    async for chunk in _stream_fim():
+        if chunk.startswith("data: [DONE]"):
+            break
+        try:
+            d = json.loads(chunk[6:])
+            content = d.get("choices", [{}])[0].get("delta", {}).get("content", "")
+            if content:
+                full.append(content)
+        except (json.JSONDecodeError, IndexError):
+            pass
+    return JSONResponse(content={
+        "choices": [{"text": "".join(full), "index": 0, "finish_reason": "stop"}],
+        "usage": {},
+    })
+
+
 async def _fim_passthrough(body: dict) -> JSONResponse:
     """
     Proxy FIM requests directly to llama-server without pipeline injection.
@@ -255,11 +329,20 @@ async def _fim_passthrough(body: dict) -> JSONResponse:
     async with httpx.AsyncClient() as client:
         try:
             resp = await client.post(
-                f"{LLAMA_SERVER_BASE}/v1/completions",
+                f"{LLAMA_SERVER_BASE}/completion",
                 json=body,
                 timeout=httpx.Timeout(30.0, connect=5.0),
             )
-            return JSONResponse(content=resp.json(), status_code=resp.status_code)
+            data = resp.json()
+            wrapped = {
+                "choices": [{
+                    "text": data.get("content", ""),
+                    "index": 0,
+                    "finish_reason": data.get("stop", False),
+                }],
+                "usage": {},
+            }
+            return JSONResponse(content=wrapped, status_code=resp.status_code)
         except httpx.ConnectError:
             raise HTTPException(status_code=503, detail="Cannot connect to inference server")
         except Exception as e:

routers/conversations.py

@@ -7,7 +7,7 @@ from db import get_db
 from security import read_json_body, BODY_LIMIT_DEFAULT_BYTES
 from config import DEFAULT_MODEL, MAX_CONVERSATION_TITLE_CHARS
 
-log = logging.getLogger("jarvischat")
+log = logging.getLogger("caic")
 router = APIRouter()
 
 
@@ -15,8 +15,16 @@ router = APIRouter()
 async def list_conversations():
     db = get_db()
     rows = db.execute("SELECT * FROM conversations ORDER BY updated_at DESC").fetchall()
+    result = []
+    for r in rows:
+        c = dict(r)
+        attach_count = db.execute(
+            "SELECT COUNT(*) FROM upload_context WHERE conversation_id = ?", (c["id"],)
+        ).fetchone()[0]
+        c["attachment_count"] = attach_count
+        result.append(c)
     db.close()
-    return [dict(r) for r in rows]
+    return result
 
 
 @router.post("/api/conversations")

routers/hardware.py

@@ -0,0 +1,15 @@
+"""JarvisChat routers — Hardware self-assessment endpoint."""
+import json
+
+from fastapi import APIRouter
+
+import hardware
+
+router = APIRouter()
+
+
+@router.get("/api/hardware")
+async def get_hardware_state():
+    if hardware.HARDWARE_STATE_PATH.exists():
+        return json.loads(hardware.HARDWARE_STATE_PATH.read_text())
+    return {"status": "not_ready", "message": "Hardware assessment not yet complete"}

routers/ingest.py

@@ -0,0 +1,70 @@
+"""JarvisChat routers - /api/ingest terminal command RAG hook."""
+import logging
+from datetime import datetime, timezone
+
+import httpx
+from fastapi import APIRouter, HTTPException, Request
+from fastapi.responses import JSONResponse
+
+from config import COMPLETIONS_API_KEY
+from eviction import maybe_evict
+from rag import chunk_text, QDRANT_URL, EMBED_URL, EMBED_MODEL, RAG_COLLECTION
+
+log = logging.getLogger("caic")
+router = APIRouter()
+
+
+def _check_api_key(request: Request):
+    auth = request.headers.get("Authorization", "")
+    if not auth.startswith("Bearer "):
+        raise HTTPException(status_code=401, detail="Missing Bearer token")
+    token = auth[7:].strip()
+    if token != COMPLETIONS_API_KEY:
+        raise HTTPException(status_code=401, detail="Invalid API key")
+
+
+@router.post("/api/ingest")
+async def ingest_content(request: Request):
+    _check_api_key(request)
+    body = await request.json()
+    content = (body.get("content") or "").strip()
+    if not content:
+        raise HTTPException(status_code=422, detail="content is required")
+    source = str(body.get("source", "external")).strip() or "external"
+    metadata = body.get("metadata") or {}
+
+    chunks = chunk_text(content)
+    if not chunks:
+        raise HTTPException(status_code=422, detail="content produced no chunks")
+
+    ingested = 0
+    async with httpx.AsyncClient() as client:
+        for i, chunk in enumerate(chunks):
+            embed_resp = await client.post(
+                f"{EMBED_URL}/api/embeddings",
+                json={"model": EMBED_MODEL, "prompt": chunk},
+                timeout=30.0,
+            )
+            if embed_resp.status_code != 200:
+                log.warning(f"Ingest embedding failed for chunk {i}: {embed_resp.status_code}")
+                continue
+            vector = embed_resp.json()["embedding"]
+            point_id = f"ingest-{source}-{datetime.now(timezone.utc).timestamp()}-{i}"
+            payload = {"text": chunk, "source": source, "ingest_date": datetime.now(timezone.utc).isoformat(), "type": "ingest"}
+            payload.update(metadata)
+            upsert_resp = await client.put(
+                f"{QDRANT_URL}/collections/{RAG_COLLECTION}/points?wait=true",
+                json={"points": [{"id": point_id, "vector": vector, "payload": payload}]},
+                timeout=30.0,
+            )
+            if upsert_resp.status_code in (200, 201):
+                ingested += 1
+            else:
+                log.warning(f"Ingest Qdrant upsert failed for chunk {i}: {upsert_resp.status_code}")
+
+    if ingested > 0:
+        evicted = await maybe_evict()
+        if evicted:
+            log.info(f"Evicted {evicted} vectors after ingest")
+
+    return {"chunks_ingested": ingested, "source": source, "message": f"Ingested {ingested} chunks from {source}"}

routers/memories.py

@@ -4,6 +4,7 @@ from typing import Optional
 
 from db import get_db
 from memory import add_memory, delete_memory, update_memory, get_all_memories, search_memories
+from rag import confirm_fact_update
 from security import read_json_body, BODY_LIMIT_DEFAULT_BYTES
 from config import MAX_MEMORY_FACT_CHARS
 
@@ -54,6 +55,22 @@ async def search_memories_api(q: str, limit: int = 10):
     return {"results": results, "count": len(results)}
 
 
+@router.post("/api/memories/confirm-update")
+async def confirm_memory_update(request: Request):
+    body = await read_json_body(request, BODY_LIMIT_DEFAULT_BYTES)
+    memory_id = body.get("memory_id")
+    new_fact = str(body.get("new_fact", "")).strip()
+    old_fact = str(body.get("old_fact", "")).strip()
+    user_message = str(body.get("user_message", "")).strip()
+    assistant_message = str(body.get("assistant_message", "")).strip()
+    if not memory_id or not new_fact:
+        raise HTTPException(status_code=400, detail="memory_id and new_fact are required")
+    ok = await confirm_fact_update(memory_id, old_fact, new_fact, user_message, assistant_message)
+    if not ok:
+        raise HTTPException(status_code=404, detail="Memory not found")
+    return {"status": "ok"}
+
+
 @router.get("/api/memories/stats")
 async def memory_stats():
     db = get_db()

routers/models.py

@@ -8,11 +8,11 @@ import httpx
 import psutil
 from fastapi import APIRouter, HTTPException, Request
 
-from config import OLLAMA_BASE
+from config import LLAMA_SERVER_BASE
 from gpu import get_gpu_stats
 from security import read_json_body, BODY_LIMIT_DEFAULT_BYTES
 
-log = logging.getLogger("jarvischat")
+log = logging.getLogger("caic")
 router = APIRouter()
 
 
@@ -20,34 +20,33 @@ router = APIRouter()
 async def list_models():
     async with httpx.AsyncClient() as client:
         try:
-            resp = await client.get(f"{OLLAMA_BASE}/v1/models", timeout=10)
+            resp = await client.get(f"{LLAMA_SERVER_BASE}/v1/models", timeout=10)
             data = resp.json()
             models = [{"name": m["id"], "model": m["id"]} for m in data.get("data", [])]
             return {"models": models}
         except httpx.ConnectError:
-            raise HTTPException(status_code=502, detail="Cannot connect to llama-server.")
+            raise HTTPException(status_code=502, detail="Cannot connect to inference server.")
 
 
 @router.get("/api/ps")
 async def running_models():
     async with httpx.AsyncClient() as client:
         try:
-            resp = await client.get(f"{OLLAMA_BASE}/api/ps", timeout=10)
+            resp = await client.get(f"{LLAMA_SERVER_BASE}/v1/models", timeout=10)
             return resp.json()
         except httpx.ConnectError:
-            raise HTTPException(status_code=502, detail="Cannot connect to Ollama.")
+            raise HTTPException(status_code=502, detail="Cannot connect to inference server.")
 
 
 @router.post("/api/show")
 async def show_model(request: Request):
-    from security import BODY_LIMIT_DEFAULT_BYTES
     body = await read_json_body(request, BODY_LIMIT_DEFAULT_BYTES)
     async with httpx.AsyncClient() as client:
         try:
-            resp = await client.post(f"{OLLAMA_BASE}/api/show", json=body, timeout=10)
+            resp = await client.post(f"{LLAMA_SERVER_BASE}/api/show", json=body, timeout=10)
             return resp.json()
         except httpx.ConnectError:
-            raise HTTPException(status_code=502, detail="Cannot connect to Ollama.")
+            raise HTTPException(status_code=502, detail="Cannot connect to inference server.")
 
 
 @router.get("/api/stats")

routers/rag_admin.py

@@ -0,0 +1,60 @@
+"""JarvisChat routers — RAG corpus management admin endpoints."""
+import logging
+
+import httpx
+from fastapi import APIRouter, HTTPException, Request
+from fastapi.responses import JSONResponse
+
+from eviction import get_rag_operational_stats, EVICTION_LOG
+from rag import QDRANT_URL, RAG_COLLECTION
+
+log = logging.getLogger("caic")
+router = APIRouter()
+
+
+@router.get("/api/rag/stats")
+async def rag_stats(request: Request):
+    if getattr(request.state, "session_role", "none") != "admin":
+        raise HTTPException(status_code=403, detail="Admin PIN required for this action")
+    stats = await get_rag_operational_stats()
+    stats["eviction_log_size"] = len(EVICTION_LOG)
+    return stats
+
+
+@router.post("/api/rag/flush")
+async def rag_flush(request: Request):
+    try:
+        async with httpx.AsyncClient() as client:
+            scroll_resp = await client.post(
+                f"{QDRANT_URL}/collections/{RAG_COLLECTION}/points/scroll",
+                json={"limit": 10000, "with_payload": False, "with_vector": False},
+                timeout=30.0,
+            )
+            if scroll_resp.status_code != 200:
+                return JSONResponse(status_code=502, content={"detail": f"Qdrant scroll failed: {scroll_resp.status_code}"})
+
+            all_points = scroll_resp.json().get("result", {}).get("points", [])
+            point_ids = [p["id"] for p in all_points]
+
+            if not point_ids:
+                return {"deleted_count": 0, "collection": RAG_COLLECTION, "status": "flushed"}
+
+            delete_resp = await client.post(
+                f"{QDRANT_URL}/collections/{RAG_COLLECTION}/points/delete",
+                json={"points": point_ids},
+                timeout=30.0,
+            )
+            if delete_resp.status_code not in (200, 201):
+                return JSONResponse(status_code=502, content={"detail": f"Qdrant delete failed: {delete_resp.status_code}"})
+
+            EVICTION_LOG.clear()
+            log.warning(f"RAG collection '{RAG_COLLECTION}' flushed ({len(point_ids)} points deleted)")
+
+            return {
+                "deleted_count": len(point_ids),
+                "collection": RAG_COLLECTION,
+                "status": "flushed",
+            }
+    except Exception as e:
+        log.warning(f"RAG flush error: {e}")
+        return JSONResponse(status_code=502, content={"detail": f"RAG flush error: {e}"})

routers/search_route.py

@@ -14,7 +14,7 @@ from search import query_searxng, format_search_results
 from routers.chat import parse_llama_stream_chunk
 from security import read_json_body, log_incident, BODY_LIMIT_CHAT_BYTES
 
-log = logging.getLogger("jarvischat")
+log = logging.getLogger("caic")
 router = APIRouter()
 
 
@@ -35,14 +35,14 @@ async def explicit_search(request: Request):
 
     if not conv_id:
         conv_id = str(uuid.uuid4())
-        title = f"🔍 {query[:70]}..." if len(query) > 70 else f"🔍 {query}"
+        title = query[:70] + "..." if len(query) > 70 else query
         db.execute("INSERT INTO conversations (id, title, model, created_at, updated_at) VALUES (?, ?, ?, ?, ?)",
                    (conv_id, title, model, now, now))
     else:
         db.execute("UPDATE conversations SET updated_at = ? WHERE id = ?", (now, conv_id))
 
     db.execute("INSERT INTO messages (conversation_id, role, content, created_at) VALUES (?, ?, ?, ?)",
-               (conv_id, "user", f"🔍 {query}", now))
+               (conv_id, "user", query, now))
     db.commit()
     db.close()
 
@@ -80,7 +80,7 @@ async def explicit_search(request: Request):
                 ) as resp:
                     async for line in resp.aiter_lines():
                         if line.strip():
-                            token, done, _ = parse_llama_stream_chunk(line)
+                            token, done, _, _ = parse_llama_stream_chunk(line)
                             if token:
                                 full_response.append(token)
                                 yield f"data: {json.dumps({'token': token, 'conversation_id': conv_id})}\n\n"
@@ -102,7 +102,6 @@ async def explicit_search(request: Request):
         db2.commit()
         db2.close()
 
-        yield f"data: {json.dumps({'raw_results': results, 'conversation_id': conv_id})}\n\n"
         yield f"data: {json.dumps({'done': True, 'conversation_id': conv_id, 'searched': True})}\n\n"
 
     return StreamingResponse(stream_search(), media_type="text/event-stream")

routers/upload.py

@@ -0,0 +1,173 @@
+"""JarvisChat routers - /api/upload file/document attachment endpoint."""
+import json
+import logging
+import os
+from datetime import datetime, timezone, timedelta
+
+import httpx
+from fastapi import APIRouter, HTTPException, Request, UploadFile, File, Form
+from fastapi.responses import JSONResponse
+
+from config import UPLOAD_DIR, MAX_UPLOAD_BYTES, SUPPORTED_UPLOAD_TYPES, UPLOAD_CONTEXT_EXPIRY_HOURS
+from db import get_db, insert_upload_context, list_upload_context_by_conversation, delete_upload_context_by_id
+from eviction import maybe_evict
+from rag import chunk_text, QDRANT_URL, EMBED_URL, EMBED_MODEL, RAG_COLLECTION
+
+log = logging.getLogger("caic")
+router = APIRouter()
+
+
+def _point_id(filename: str, chunk_idx: int) -> str:
+    return f"upload-{filename}-{chunk_idx}"
+
+
+@router.post("/api/upload")
+async def upload_file(
+    request: Request,
+    file: UploadFile = File(...),
+    mode: str = Form("both"),
+    conversation_id: str = Form(""),
+):
+    if mode not in ("context", "ingest", "both"):
+        raise HTTPException(status_code=422, detail="mode must be context, ingest, or both")
+
+    if file.size and file.size > MAX_UPLOAD_BYTES:
+        return JSONResponse(status_code=413, content={"detail": f"File exceeds {MAX_UPLOAD_BYTES} byte limit"})
+
+    content_type = file.content_type or "application/octet-stream"
+    if content_type not in SUPPORTED_UPLOAD_TYPES:
+        return JSONResponse(status_code=415, content={"detail": f"Unsupported file type: {content_type}"})
+
+    raw_bytes = await file.read()
+    if not raw_bytes:
+        raise HTTPException(status_code=422, detail="Empty file")
+
+    if content_type == "application/pdf":
+        try:
+            from pypdf import PdfReader
+            import io
+            reader = PdfReader(io.BytesIO(raw_bytes))
+            extracted = "\n".join(page.extract_text() or "" for page in reader.pages)
+        except Exception as e:
+            log.warning(f"PDF extraction error: {e}")
+            raise HTTPException(status_code=422, detail="Failed to extract text from PDF")
+    else:
+        extracted = raw_bytes.decode("utf-8", errors="replace")
+
+    result = {"filename": file.filename, "size_bytes": len(raw_bytes), "mode": mode}
+
+    if mode in ("ingest", "both"):
+        os.makedirs(UPLOAD_DIR, exist_ok=True)
+        chunks = chunk_text(extracted)
+        ingested = 0
+        async with httpx.AsyncClient() as client:
+            for i, chunk in enumerate(chunks):
+                embed_resp = await client.post(
+                    f"{EMBED_URL}/api/embeddings",
+                    json={"model": EMBED_MODEL, "prompt": chunk},
+                    timeout=30.0,
+                )
+                if embed_resp.status_code != 200:
+                    log.warning(f"Embedding failed for chunk {i}: {embed_resp.status_code}")
+                    continue
+                vector = embed_resp.json()["embedding"]
+                pid = _point_id(file.filename or "unnamed", i)
+                upsert_resp = await client.put(
+                    f"{QDRANT_URL}/collections/{RAG_COLLECTION}/points?wait=true",
+                    json={
+                        "points": [{
+                            "id": pid,
+                            "vector": vector,
+                            "payload": {"text": chunk, "source": file.filename, "upload_date": datetime.now(timezone.utc).isoformat(), "type": "upload"},
+                        }]
+                    },
+                    timeout=30.0,
+                )
+                if upsert_resp.status_code in (200, 201):
+                    ingested += 1
+                else:
+                    log.warning(f"Qdrant upsert failed for chunk {i}: {upsert_resp.status_code}")
+        result["chunks_ingested"] = ingested
+        if ingested > 0:
+            evicted = await maybe_evict()
+            if evicted:
+                log.info(f"Evicted {evicted} vectors after upload")
+
+    if mode in ("context", "both"):
+        expires = (datetime.now(timezone.utc) + timedelta(hours=UPLOAD_CONTEXT_EXPIRY_HOURS)).isoformat()
+        db = get_db()
+        try:
+            cid = insert_upload_context(db, conversation_id or "", file.filename or "unnamed", extracted, expires, content_type)
+            db.commit()
+            result["context_id"] = cid
+        finally:
+            db.close()
+
+    result["message"] = f"Uploaded {file.filename}"
+    return result
+
+
+@router.patch("/api/upload/{context_id}/link")
+async def link_upload_to_conversation(context_id: int, request: Request):
+    body = await request.json()
+    conv_id = body.get("conversation_id", "").strip()
+    if not conv_id:
+        raise HTTPException(status_code=422, detail="conversation_id required")
+    db = get_db()
+    try:
+        row = db.execute("SELECT id FROM upload_context WHERE id = ?", (context_id,)).fetchone()
+        if not row:
+            raise HTTPException(status_code=404, detail="Upload context not found")
+        db.execute("UPDATE upload_context SET conversation_id = ? WHERE id = ?", (conv_id, context_id))
+        db.commit()
+        return {"status": "ok"}
+    finally:
+        db.close()
+
+
+@router.get("/api/upload/by-conversation/{conv_id}")
+async def get_upload_by_conversation(conv_id: str):
+    db = get_db()
+    try:
+        items = list_upload_context_by_conversation(db, conv_id)
+        return items
+    finally:
+        db.close()
+
+
+@router.delete("/api/upload/{context_id}")
+async def delete_upload(context_id: int):
+    db = get_db()
+    try:
+        row = db.execute("SELECT filename FROM upload_context WHERE id = ?", (context_id,)).fetchone()
+        if not row:
+            raise HTTPException(status_code=404, detail="Attachment not found")
+        filename = row["filename"]
+        if not filename:
+            filename = "unnamed"
+        delete_upload_context_by_id(db, context_id)
+        db.commit()
+    finally:
+        db.close()
+
+    try:
+        async with httpx.AsyncClient() as client:
+            scroll_resp = await client.post(
+                f"{QDRANT_URL}/collections/{RAG_COLLECTION}/points/scroll",
+                json={"filter": {"must": [{"key": "source", "match": {"value": filename}}]}, "limit": 100, "with_payload": False},
+                timeout=10.0,
+            )
+            if scroll_resp.status_code == 200:
+                points = scroll_resp.json().get("result", {}).get("points", [])
+                point_ids = [p["id"] for p in points]
+                if point_ids:
+                    await client.post(
+                        f"{QDRANT_URL}/collections/{RAG_COLLECTION}/points/delete",
+                        json={"points": point_ids},
+                        timeout=10.0,
+                    )
+                    log.info(f"Deleted {len(point_ids)} Qdrant points for source '{filename}'")
+    except Exception as e:
+        log.warning(f"Qdrant cleanup for '{filename}' failed: {e}")
+
+    return {"status": "ok", "filename": filename}

search.py

@@ -1,5 +1,5 @@
 """
-JarvisChat - SearXNG integration, perplexity scoring, refusal/hedge detection.
+cAIc - SearXNG integration, perplexity scoring, refusal/hedge detection.
 """
 import logging
 import math
@@ -10,7 +10,7 @@ import httpx
 
 from config import SEARXNG_BASE, PERPLEXITY_THRESHOLD, REFUSAL_PATTERNS, HEDGE_PATTERNS
 
-log = logging.getLogger("jarvischat")
+log = logging.getLogger("caic")
 
 
 def sanitize_outbound_url(url: str) -> str:
@@ -80,16 +80,13 @@ def format_direct_answer(question: str, results: list) -> str:
 
 def extract_search_query(user_message: str) -> str:
     query = user_message.strip()
-    if re.search(r"temperature|weather", query, re.IGNORECASE):
+    weather_lead = re.match(r"^(?:what('?s| is) the\s+)?(?:weather|temperature|forecast)\s+(?:in\s+|for\s+)?(.+)", query, re.IGNORECASE)
-        query = re.sub(r"^what('?s| is) the ", "", query, flags=re.IGNORECASE) + " right now degrees"
+    if weather_lead:
-    if re.search(r"price|spot price", query, re.IGNORECASE):
+        return (weather_lead.group(2) + " weather").strip()[:100]
-        query = re.sub(r"^(what('?s| is)|can you tell me) the ", "", query, flags=re.IGNORECASE) + " today USD"
+    price_lead = re.match(r"^(?:what('?s| is| are)\s+)?(?:the\s+)?(?:price|spot price)\s+(?:of\s+|for\s+)?(.+)", query, re.IGNORECASE)
-    query = re.sub(
+    if price_lead:
-        r"^(what|who|where|when|why|how|is|are|can|could|would|should|do|does|did)\s+",
+        return (price_lead.group(2) + " price today USD").strip()[:100]
-        "", query, flags=re.IGNORECASE,
+    return query[:100]
-    )
-    query = re.sub(r"[?!.]+$", "", query)
-    return query[:100].strip() or user_message[:100]
 
 
 async def query_searxng(query: str, max_results: int = 5) -> list:
@@ -120,6 +117,7 @@ async def query_searxng(query: str, max_results: int = 5) -> list:
                 f"{SEARXNG_BASE}/search",
                 params={"q": query, "format": "json", "categories": "general"},
                 timeout=10.0,
+                follow_redirects=True,
             )
             if resp.status_code == 200:
                 data = resp.json()

security.py

@@ -1,5 +1,5 @@
 """
-JarvisChat - Security utilities.
+cAIc - Security utilities.
 PIN hashing, audit logging, incident tracking, CSRF/origin checks,
 rate limiting, request helpers.
 """
@@ -22,7 +22,7 @@ from fastapi import HTTPException, Request
 
 from config import (
     ALLOWED_NETWORKS, TRUST_X_FORWARDED_FOR, TRUSTED_ORIGINS,
-    BODY_LIMIT_DEFAULT_BYTES, BODY_LIMIT_CHAT_BYTES, BODY_LIMIT_PROFILE_BYTES,
+    BODY_LIMIT_DEFAULT_BYTES, BODY_LIMIT_CHAT_BYTES, BODY_LIMIT_PROFILE_BYTES, BODY_LIMIT_UPLOAD_BYTES,
     RATE_WINDOW_SECONDS, RL_LOGIN_PER_WINDOW, RL_CHAT_PER_WINDOW,
     RL_SEARCH_PER_WINDOW, RL_STATS_PER_WINDOW, RL_WRITE_PER_WINDOW,
     RL_DEFAULT_PER_WINDOW, VERSION,
@@ -30,7 +30,7 @@ from config import (
 
 import ipaddress
 
-log = logging.getLogger("jarvischat")
+log = logging.getLogger("caic")
 
 SESSIONS: dict = {}
 PIN_ATTEMPTS: dict = {}
@@ -114,6 +114,8 @@ def request_body_limit(path: str) -> int:
         return BODY_LIMIT_CHAT_BYTES
     if path == "/api/profile":
         return BODY_LIMIT_PROFILE_BYTES
+    if path == "/api/upload":
+        return BODY_LIMIT_UPLOAD_BYTES
     return BODY_LIMIT_DEFAULT_BYTES
 
 
@@ -156,7 +158,7 @@ def origin_allowed(request: Request) -> bool:
         parsed = urlparse(referer)
         ref_origin = f"{parsed.scheme}://{parsed.netloc}".rstrip("/")
         return ref_origin == expected_origin or ref_origin in TRUSTED_ORIGINS
-    return True
+    return False
 
 
 def is_state_changing(method: str) -> bool:

tests/test_amqp.py

@@ -0,0 +1,90 @@
+import asyncio
+import logging
+
+import amqp
+from config import AMQP_EXCHANGE_ADMIN
+
+
+def _reset():
+    amqp._connection = None
+    amqp._channel = None
+    amqp._lock = asyncio.Lock()
+
+
+class FakeExchange:
+    def __init__(self):
+        self.messages = []
+
+    async def publish(self, msg, routing_key):
+        self.messages.append((msg, routing_key))
+
+
+class FakeChannel:
+    def __init__(self):
+        self.is_closed = False
+        self.exchanges = {}
+
+    async def close(self):
+        self.is_closed = True
+
+    async def declare_exchange(self, name, typ, durable=True):
+        self.exchanges[name] = typ
+
+    async def get_exchange(self, name):
+        return self.exchanges.setdefault(name, FakeExchange())
+
+
+class FakeConnection:
+    def __init__(self):
+        self.is_closed = False
+        self._channel = FakeChannel()
+
+    async def channel(self):
+        return self._channel
+
+    async def close(self):
+        self.is_closed = True
+
+
+async def fake_connect_robust(url, **_):
+    return FakeConnection()
+
+
+# ---------- tests ----------
+
+
+def test_publish_success(monkeypatch):
+    _reset()
+    monkeypatch.setattr(amqp, "HAS_AIO_PIKA", True)
+    monkeypatch.setattr("aio_pika.connect_robust", fake_connect_robust)
+
+    asyncio.run(amqp.connect())
+    ch = asyncio.run(amqp.get_channel())
+    ex = FakeExchange()
+    ch.exchanges[AMQP_EXCHANGE_ADMIN] = ex
+
+    asyncio.run(amqp.publish(AMQP_EXCHANGE_ADMIN, "test.key", {"foo": "bar"}))
+
+    assert len(ex.messages) == 1
+    msg, rk = ex.messages[0]
+    assert rk == "test.key"
+    assert msg.body == b'{"foo": "bar"}'
+
+
+def test_publish_disconnected_no_raise(caplog):
+    _reset()
+    caplog.set_level(logging.ERROR)
+    asyncio.run(amqp.publish(AMQP_EXCHANGE_ADMIN, "test.key", {"x": 1}))
+
+    assert len(caplog.records) > 0
+    assert any("cannot publish" in r.message for r in caplog.records)
+
+
+def test_get_channel_reconnects_when_none(monkeypatch):
+    _reset()
+    monkeypatch.setattr(amqp, "HAS_AIO_PIKA", True)
+    monkeypatch.setattr("aio_pika.connect_robust", fake_connect_robust)
+
+    ch = asyncio.run(amqp.get_channel())
+    assert ch is not None
+    assert not ch.is_closed

tests/test_auth_capabilities.py

@@ -3,16 +3,18 @@ from pathlib import Path
 
 from fastapi.testclient import TestClient
 
-import app as app_module
+import app
+import db
+from security import SESSIONS, PIN_ATTEMPTS
 
 
 def make_client(tmp_path: Path) -> TestClient:
-    os.environ["JARVISCHAT_ADMIN_PIN"] = "1234"
+    os.environ["CAIC_ADMIN_PIN"] = "1234"
-    app_module.DB_PATH = tmp_path / "jarvischat-test.db"
+    db.DB_PATH = tmp_path / "caic-test.db"
-    app_module.SESSIONS.clear()
+    SESSIONS.clear()
-    app_module.PIN_ATTEMPTS.clear()
+    PIN_ATTEMPTS.clear()
-    app_module.init_db()
+    db.init_db()
-    return TestClient(app_module.app)
+    return TestClient(app.app)
 
 
 def test_guest_read_only_admin_write_blocked(tmp_path: Path):
@@ -20,7 +22,7 @@ def test_guest_read_only_admin_write_blocked(tmp_path: Path):
         guest = client.post("/api/auth/guest", headers={"Origin": "http://testserver"})
         assert guest.status_code == 200
         sid = guest.json()["session_id"]
-        headers = {"X-Session-ID": sid}
+        headers = {"X-Session-ID": sid, "Origin": "http://testserver"}
 
         read_resp = client.get("/api/memories", headers=headers)
         assert read_resp.status_code == 200
@@ -74,5 +76,5 @@ def test_logout_revokes_session(tmp_path: Path):
         logout = client.post("/api/auth/logout", headers=headers)
         assert logout.status_code == 200
 
-        after = client.get("/api/memories", headers={"X-Session-ID": sid})
+        after = client.get("/api/memories", headers={"X-Session-ID": sid, "Origin": "http://testserver"})
         assert after.status_code == 401

tests/test_chat_streaming_and_memory_paths.py

@@ -2,19 +2,29 @@ import json
 import os
 from pathlib import Path
 
+import httpx
 from fastapi.testclient import TestClient
 
-import app as app_module
+import app
+import config
+import db
+import routers.chat
+import triage
+from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS
+
+
+def _mock_triage_url(monkeypatch, url: str = config.LLAMA_SERVER_BASE):
+    monkeypatch.setattr(config, "LLAMA_SERVER_BASE", url)
 
 
 def make_client(tmp_path: Path) -> TestClient:
-    os.environ["JARVISCHAT_ADMIN_PIN"] = "1234"
+    os.environ["CAIC_ADMIN_PIN"] = "1234"
-    app_module.DB_PATH = tmp_path / "jarvischat-streaming.db"
+    db.DB_PATH = tmp_path / "caic-streaming.db"
-    app_module.SESSIONS.clear()
+    SESSIONS.clear()
-    app_module.PIN_ATTEMPTS.clear()
+    PIN_ATTEMPTS.clear()
-    app_module.RATE_EVENTS.clear()
+    RATE_EVENTS.clear()
-    app_module.init_db()
+    db.init_db()
-    return TestClient(app_module.app, raise_server_exceptions=False)
+    return TestClient(app.app, raise_server_exceptions=False)
 
 
 def parse_sse_payloads(body: str) -> list[dict]:
@@ -48,6 +58,7 @@ def _stream_json_lines(events: list[dict]) -> list[str]:
 
 
 def test_chat_stream_emits_tokens_and_done(tmp_path: Path, monkeypatch):
+    _mock_triage_url(monkeypatch)
     with make_client(tmp_path) as client:
         sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()[
             "session_id"
@@ -65,11 +76,11 @@ def test_chat_stream_emits_tokens_and_done(tmp_path: Path, monkeypatch):
         def stream_stub(self, method, url, json=None, timeout=None):
             return _MockStreamResponse(events)
 
-        monkeypatch.setattr(app_module.httpx.AsyncClient, "stream", stream_stub)
+        monkeypatch.setattr(httpx.AsyncClient, "stream", stream_stub)
 
         resp = client.post(
             "/api/chat",
-            json={"message": "hello", "model": app_module.DEFAULT_MODEL},
+            json={"message": "hello", "model": config.DEFAULT_MODEL},
             headers=headers,
         )
         assert resp.status_code == 200
@@ -83,6 +94,7 @@ def test_chat_stream_emits_tokens_and_done(tmp_path: Path, monkeypatch):
 
 
 def test_chat_auto_search_trigger_emits_search_events(tmp_path: Path, monkeypatch):
+    _mock_triage_url(monkeypatch)
     with make_client(tmp_path) as client:
         sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()[
             "session_id"
@@ -92,7 +104,7 @@ def test_chat_auto_search_trigger_emits_search_events(tmp_path: Path, monkeypatc
         first_stream = _stream_json_lines(
             [
                 {
-                    "message": {"content": "I am uncertain."},
+                    "message": {"content": "I don't have current data on that question."},
                     "logprobs": [{"logprob": -5.0}],
                 },
                 {"done": True, "eval_count": 2, "eval_duration": 1000000000},
@@ -118,12 +130,12 @@ def test_chat_auto_search_trigger_emits_search_events(tmp_path: Path, monkeypatc
                 }
             ]
 
-        monkeypatch.setattr(app_module.httpx.AsyncClient, "stream", stream_stub)
+        monkeypatch.setattr(httpx.AsyncClient, "stream", stream_stub)
-        monkeypatch.setattr(app_module, "query_searxng", search_stub)
+        monkeypatch.setattr(routers.chat, "query_searxng", search_stub)
 
         resp = client.post(
             "/api/chat",
-            json={"message": "what is the latest value", "model": app_module.DEFAULT_MODEL},
+            json={"message": "what is the latest value", "model": config.DEFAULT_MODEL},
             headers=headers,
         )
         assert resp.status_code == 200
@@ -136,7 +148,73 @@ def test_chat_auto_search_trigger_emits_search_events(tmp_path: Path, monkeypatc
         assert done_events and done_events[-1].get("searched") is True
 
 
+def test_chat_with_upload_context_id_injects_document(tmp_path: Path, monkeypatch):
+    _mock_triage_url(monkeypatch)
+    captured_payload = {}
+
+    def stream_stub(self, method, url, json=None, timeout=None):
+        nonlocal captured_payload
+        captured_payload = json
+        events = [{"message": {"content": "ok"}, "logprobs": [{"logprob": -0.01}]}, {"done": True, "eval_count": 1, "eval_duration": 1000000000}]
+        return _MockStreamResponse([__import__('json').dumps(e) for e in events])
+
+    monkeypatch.setattr(httpx.AsyncClient, "stream", stream_stub)
+
+    with make_client(tmp_path) as client:
+        sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()["session_id"]
+        headers = {"X-Session-ID": sid, "Origin": "http://testserver"}
+
+        db_local = db.get_db()
+        expires = "2099-12-31T23:59:59+00:00"
+        cid = db.insert_upload_context(db_local, "conv-up", "report.txt", "Confidential document content here", expires, "text/plain")
+        db_local.commit()
+        db_local.close()
+
+        resp = client.post(
+            "/api/chat",
+            json={"message": "summarize this", "upload_context_id": cid, "model": config.DEFAULT_MODEL},
+            headers=headers,
+        )
+        assert resp.status_code == 200
+        system_content = next((m["content"] for m in captured_payload.get("messages", []) if m["role"] == "system"), "")
+        assert "Confidential document content here" in system_content
+
+
+def test_chat_with_expired_upload_context_id_silent(tmp_path: Path, monkeypatch):
+    _mock_triage_url(monkeypatch)
+    captured_payload = {}
+
+    def stream_stub(self, method, url, json=None, timeout=None):
+        nonlocal captured_payload
+        captured_payload = json
+        events = [{"message": {"content": "ok"}, "logprobs": [{"logprob": -0.01}]}, {"done": True, "eval_count": 1, "eval_duration": 1000000000}]
+        return _MockStreamResponse([__import__('json').dumps(e) for e in events])
+
+    monkeypatch.setattr(httpx.AsyncClient, "stream", stream_stub)
+
+    with make_client(tmp_path) as client:
+        sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()["session_id"]
+        headers = {"X-Session-ID": sid, "Origin": "http://testserver"}
+
+        import datetime
+        db_local = db.get_db()
+        expires = (datetime.datetime.now(datetime.timezone.utc) - datetime.timedelta(hours=2)).isoformat()
+        cid = db.insert_upload_context(db_local, "conv-exp", "old.txt", "Stale data", expires, "text/plain")
+        db_local.commit()
+        db_local.close()
+
+        resp = client.post(
+            "/api/chat",
+            json={"message": "hi", "upload_context_id": cid, "model": config.DEFAULT_MODEL},
+            headers=headers,
+        )
+        assert resp.status_code == 200
+        system_content = next((m["content"] for m in captured_payload.get("messages", []) if m["role"] == "system"), "")
+        assert "Stale data" not in system_content
+
+
 def test_memory_command_paths_remember_and_forget(tmp_path: Path, monkeypatch):
+    _mock_triage_url(monkeypatch)
     with make_client(tmp_path) as client:
         sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()[
             "session_id"
@@ -153,13 +231,13 @@ def test_memory_command_paths_remember_and_forget(tmp_path: Path, monkeypatch):
         def stream_stub(self, method, url, json=None, timeout=None):
             return _MockStreamResponse(base_stream)
 
-        monkeypatch.setattr(app_module.httpx.AsyncClient, "stream", stream_stub)
+        monkeypatch.setattr(httpx.AsyncClient, "stream", stream_stub)
 
         remember_resp = client.post(
             "/api/chat",
             json={
                 "message": "remember that my favorite language is rust",
-                "model": app_module.DEFAULT_MODEL,
+                "model": config.DEFAULT_MODEL,
             },
             headers=headers,
         )
@@ -167,7 +245,7 @@ def test_memory_command_paths_remember_and_forget(tmp_path: Path, monkeypatch):
         remember_events = parse_sse_payloads(remember_resp.text)
         assert any("Remembered" in p.get("token", "") for p in remember_events)
 
-        memories_after_add = client.get("/api/memories", headers={"X-Session-ID": sid})
+        memories_after_add = client.get("/api/memories", headers={"X-Session-ID": sid, "Origin": "http://testserver"})
         assert memories_after_add.status_code == 200
         assert memories_after_add.json().get("count", 0) >= 1
 
@@ -175,7 +253,7 @@ def test_memory_command_paths_remember_and_forget(tmp_path: Path, monkeypatch):
             "/api/chat",
             json={
                 "message": "forget about my favorite language",
-                "model": app_module.DEFAULT_MODEL,
+                "model": config.DEFAULT_MODEL,
             },
             headers=headers,
         )
@@ -183,6 +261,6 @@ def test_memory_command_paths_remember_and_forget(tmp_path: Path, monkeypatch):
         forget_events = parse_sse_payloads(forget_resp.text)
         assert any("Forgot" in p.get("token", "") for p in forget_events)
 
-        memories_after_forget = client.get("/api/memories", headers={"X-Session-ID": sid})
+        memories_after_forget = client.get("/api/memories", headers={"X-Session-ID": sid, "Origin": "http://testserver"})
         assert memories_after_forget.status_code == 200
         assert memories_after_forget.json().get("count", 0) == 0

tests/test_cluster.py

@@ -0,0 +1,313 @@
+"""Tests for cluster.py — no live AMQP, all handlers called directly."""
+import asyncio
+from collections import deque
+
+import cluster
+from config import AMQP_EXCHANGE_ADMIN, AMQP_EXCHANGE_SYSTEM
+
+
+def _reset():
+    cluster.CLUSTER_NODES.clear()
+    cluster.CLUSTER_EVENTS.clear()
+    cluster.CLUSTER_COORDINATOR = None
+    cluster._pending_pings.clear()
+
+
+_published = []
+
+
+async def _fake_publish(exchange, routing_key, payload):
+    _published.append((exchange, routing_key, payload))
+
+
+# ---------- 1. Valid worker registration ----------
+
+
+def test_valid_worker_registration(monkeypatch):
+    _reset()
+    _published.clear()
+    monkeypatch.setattr(cluster, "publish", _fake_publish)
+
+    asyncio.run(cluster.handle_registration(
+        AMQP_EXCHANGE_ADMIN, "node.jarvis.register",
+        {"node_name": "jarvis", "node_type": "worker", "capabilities": ["llm"]},
+    ))
+
+    assert "jarvis" in cluster.CLUSTER_NODES
+    assert cluster.CLUSTER_NODES["jarvis"]["type"] == "worker"
+    assert cluster.CLUSTER_NODES["jarvis"]["status"] == "active"
+    assert len(cluster.CLUSTER_EVENTS) == 1
+    assert cluster.CLUSTER_EVENTS[0]["category"] == "cluster"
+    assert cluster.CLUSTER_EVENTS[0]["message"] == "Node registered (type=worker)"
+
+    assert len(_published) == 1
+    exchange, rk, payload = _published[0]
+    assert exchange == AMQP_EXCHANGE_ADMIN
+    assert rk == "node.jarvis.admitted"
+    assert payload["type"] == "admitted"
+    assert payload["node_name"] == "jarvis"
+
+
+# ---------- 2. First coordinator auto-promotion ----------
+
+
+def test_first_coordinator_auto_promotion(monkeypatch):
+    _reset()
+    _published.clear()
+    monkeypatch.setattr(cluster, "publish", _fake_publish)
+
+    asyncio.run(cluster.handle_registration(
+        AMQP_EXCHANGE_ADMIN, "node.ultron.register",
+        {"node_name": "ultron", "node_type": "coordinator", "capabilities": ["llm", "rag"]},
+    ))
+
+    assert cluster.CLUSTER_COORDINATOR == "ultron"
+    assert len(cluster.CLUSTER_EVENTS) == 2
+    assert cluster.CLUSTER_EVENTS[1]["message"] == "Elected as coordinator"
+
+    assert len(_published) == 2
+    # First publish: admitted
+    assert _published[0][1] == "node.ultron.admitted"
+    # Second publish: coord_response
+    exchange, rk, payload = _published[1]
+    assert exchange == AMQP_EXCHANGE_SYSTEM
+    assert rk == "cluster.coordinator.response"
+    assert payload["type"] == "coord_response"
+    assert payload["coordinator"] == "ultron"
+
+
+# ---------- 3. Duplicate node name rejected ----------
+
+
+def test_duplicate_node_rejected(monkeypatch):
+    _reset()
+    _published.clear()
+    monkeypatch.setattr(cluster, "publish", _fake_publish)
+
+    asyncio.run(cluster.handle_registration(
+        AMQP_EXCHANGE_ADMIN, "node.jarvis.register",
+        {"node_name": "jarvis", "node_type": "worker", "capabilities": ["llm"]},
+    ))
+    _published.clear()
+
+    asyncio.run(cluster.handle_registration(
+        AMQP_EXCHANGE_ADMIN, "node.jarvis.register",
+        {"node_name": "jarvis", "node_type": "worker", "capabilities": ["llm"]},
+    ))
+
+    assert len(cluster.CLUSTER_NODES) == 1
+    assert len(_published) == 1
+    exchange, rk, payload = _published[0]
+    assert exchange == AMQP_EXCHANGE_ADMIN
+    assert rk == "node.jarvis.rejected"
+    assert payload["reason"] == "duplicate_node_name"
+
+
+# ---------- 4. Malformed payload rejected ----------
+
+
+def test_malformed_payload_rejected(monkeypatch):
+    _reset()
+    _published.clear()
+    monkeypatch.setattr(cluster, "publish", _fake_publish)
+
+    asyncio.run(cluster.handle_registration(
+        AMQP_EXCHANGE_ADMIN, "node.jarvis.register",
+        {"node_name": "jarvis"},  # missing node_type and capabilities
+    ))
+
+    assert len(cluster.CLUSTER_NODES) == 0
+    assert len(_published) == 1
+    exchange, rk, payload = _published[0]
+    assert rk == "node.jarvis.rejected"
+    assert payload["reason"] == "malformed_payload"
+
+
+# ---------- 5. Graceful deregistration ----------
+
+
+def test_graceful_deregistration(monkeypatch):
+    _reset()
+    _published.clear()
+    monkeypatch.setattr(cluster, "publish", _fake_publish)
+
+    asyncio.run(cluster.handle_registration(
+        AMQP_EXCHANGE_ADMIN, "node.jarvis.register",
+        {"node_name": "jarvis", "node_type": "worker", "capabilities": ["llm"]},
+    ))
+    _published.clear()
+
+    asyncio.run(cluster.handle_deregistration(
+        AMQP_EXCHANGE_ADMIN, "node.jarvis.deregister",
+        {"node_name": "jarvis"},
+    ))
+
+    assert "jarvis" not in cluster.CLUSTER_NODES
+    assert len(cluster.CLUSTER_EVENTS) == 2
+    assert cluster.CLUSTER_EVENTS[1]["message"] == "Node deregistered"
+
+
+def test_deregister_coordinator_clears(monkeypatch):
+    _reset()
+    _published.clear()
+    monkeypatch.setattr(cluster, "publish", _fake_publish)
+
+    asyncio.run(cluster.handle_registration(
+        AMQP_EXCHANGE_ADMIN, "node.ultron.register",
+        {"node_name": "ultron", "node_type": "coordinator", "capabilities": ["llm"]},
+    ))
+    assert cluster.CLUSTER_COORDINATOR == "ultron"
+    _published.clear()
+
+    asyncio.run(cluster.handle_deregistration(
+        AMQP_EXCHANGE_ADMIN, "node.ultron.deregister",
+        {"node_name": "ultron"},
+    ))
+
+    assert "ultron" not in cluster.CLUSTER_NODES
+    assert cluster.CLUSTER_COORDINATOR is None
+
+
+# ---------- 6. Pong from known node ----------
+
+
+def test_pong_updates_known_node(monkeypatch):
+    _reset()
+    monkeypatch.setattr(cluster, "publish", _fake_publish)
+
+    asyncio.run(cluster.handle_registration(
+        AMQP_EXCHANGE_ADMIN, "node.jarvis.register",
+        {"node_name": "jarvis", "node_type": "worker", "capabilities": ["llm"]},
+    ))
+    original_seen = cluster.CLUSTER_NODES["jarvis"]["last_seen"]
+
+    asyncio.run(cluster.handle_pong(
+        AMQP_EXCHANGE_ADMIN, "node.jarvis.pong",
+        {"node_name": "jarvis", "status": "busy", "load": {"cpu_pct": 80}},
+    ))
+
+    assert cluster.CLUSTER_NODES["jarvis"]["status"] == "busy"
+    assert cluster.CLUSTER_NODES["jarvis"]["load"] == {"cpu_pct": 80}
+    assert cluster.CLUSTER_NODES["jarvis"]["last_seen"] != original_seen
+
+
+# ---------- 7. Pong from unknown node ----------
+
+
+def test_pong_from_unknown_node(caplog, monkeypatch):
+    _reset()
+    caplog.set_level("WARNING")
+    monkeypatch.setattr(cluster, "publish", _fake_publish)
+
+    asyncio.run(cluster.handle_pong(
+        AMQP_EXCHANGE_ADMIN, "node.stranger.pong",
+        {"node_name": "stranger", "status": "active"},
+    ))
+
+    assert "stranger" not in cluster.CLUSTER_NODES
+    assert any("pong from unknown node" in rec.message for rec in caplog.records)
+
+
+# ---------- 8. Event stored in log ----------
+
+
+def test_event_appended_to_log(monkeypatch):
+    _reset()
+    monkeypatch.setattr(cluster, "publish", _fake_publish)
+
+    asyncio.run(cluster.handle_event(
+        AMQP_EXCHANGE_SYSTEM, "node.jarvis.event",
+        {"node_name": "jarvis", "severity": "error", "message": "OOM on GPU"},
+    ))
+
+    assert len(cluster.CLUSTER_EVENTS) == 1
+    ev = cluster.CLUSTER_EVENTS[0]
+    assert ev["category"] == "application"
+    assert ev["severity"] == "error"
+    assert ev["node"] == "jarvis"
+
+
+def test_event_log_bounded(monkeypatch):
+    _reset()
+    monkeypatch.setattr(cluster, "publish", _fake_publish)
+
+    for i in range(1001):
+        cluster._push_event("application", "info", "jarvis", f"event {i}")
+
+    assert len(cluster.CLUSTER_EVENTS) == 1000
+    assert cluster.CLUSTER_EVENTS[0]["message"] == "event 1"
+    assert cluster.CLUSTER_EVENTS[-1]["message"] == "event 1000"
+
+
+# ---------- 9. Coordinator query produces response ----------
+
+
+def test_coordinator_query_response(monkeypatch):
+    _reset()
+    _published.clear()
+    monkeypatch.setattr(cluster, "publish", _fake_publish)
+
+    asyncio.run(cluster.handle_registration(
+        AMQP_EXCHANGE_ADMIN, "node.ultron.register",
+        {"node_name": "ultron", "node_type": "coordinator", "capabilities": ["llm"]},
+    ))
+    _published.clear()
+
+    asyncio.run(cluster.handle_coordinator_query(
+        AMQP_EXCHANGE_SYSTEM, "cluster.coordinator.query",
+        {"from": "jarvis", "type": "coord_query"},
+    ))
+
+    assert len(_published) == 1
+    exchange, rk, payload = _published[0]
+    assert exchange == AMQP_EXCHANGE_SYSTEM
+    assert rk == "cluster.coordinator.response"
+    assert payload["coordinator"] == "ultron"
+    assert "nodes" in payload
+
+
+def test_coordinator_query_no_coordinator(monkeypatch):
+    _reset()
+    _published.clear()
+    monkeypatch.setattr(cluster, "publish", _fake_publish)
+
+    asyncio.run(cluster.handle_coordinator_query(
+        AMQP_EXCHANGE_SYSTEM, "cluster.coordinator.query",
+        {"from": "jarvis", "type": "coord_query"},
+    ))
+
+    assert len(_published) == 0
+
+
+# ---------- 10. GET /api/cluster shape ----------
+
+
+def test_cluster_api_shape():
+    _reset()
+    cluster.CLUSTER_NODES["jarvis"] = {
+        "name": "jarvis", "type": "worker", "status": "active",
+        "capabilities": ["llm"], "active_model": None, "load": None,
+        "registered_at": "2026-01-01T00:00:00Z", "last_seen": "2026-01-01T00:00:00Z",
+    }
+    cluster.CLUSTER_COORDINATOR = "ultron"
+    cluster._push_event("cluster", "info", "ultron", "Elected")
+
+    from routers.cluster import cluster_status
+
+    resp = asyncio.run(cluster_status())
+
+    assert "nodes" in resp
+    assert "node_count" in resp
+    assert "coordinator" in resp
+    assert "events" in resp
+    assert resp["node_count"] == 1
+    assert resp["coordinator"] == "ultron"
+    assert "jarvis" in resp["nodes"]
+    assert len(resp["events"]) == 1
+    # No internal keys leaked
+    for node in resp["nodes"].values():
+        for key in node:
+            assert key in {
+                "name", "type", "status", "capabilities", "active_model",
+                "load", "registered_at", "last_seen",
+            }

tests/test_cluster_heartbeat.py

@@ -0,0 +1,49 @@
+"""Tests for cluster.py heartbeat handler."""
+import asyncio
+
+import cluster
+from config import AMQP_EXCHANGE_SYSTEM
+
+
+def _reset():
+    cluster.CLUSTER_NODES.clear()
+    cluster.CLUSTER_EVENTS.clear()
+    cluster.CLUSTER_COORDINATOR = None
+    cluster._pending_pings.clear()
+
+
+# ---------- 1. handle_heartbeat() for known node updates last_seen ----------
+
+
+def test_heartbeat_updates_last_seen(monkeypatch):
+    _reset()
+
+    cluster.CLUSTER_NODES["jarvis"] = {
+        "name": "jarvis", "type": "worker", "status": "active",
+        "last_seen": "2020-01-01T00:00:00Z",
+    }
+    original = cluster.CLUSTER_NODES["jarvis"]["last_seen"]
+
+    asyncio.run(cluster.handle_heartbeat(
+        AMQP_EXCHANGE_SYSTEM, "node.jarvis.heartbeat",
+        {"node_name": "jarvis"},
+    ))
+
+    assert cluster.CLUSTER_NODES["jarvis"]["last_seen"] != original
+    assert cluster.CLUSTER_NODES["jarvis"]["status"] == "active"  # unchanged
+
+
+# ---------- 2. handle_heartbeat() for unknown node logs warning, no add ----------
+
+
+def test_heartbeat_unknown_node_logs_warning(caplog, monkeypatch):
+    _reset()
+    caplog.set_level("WARNING")
+
+    asyncio.run(cluster.handle_heartbeat(
+        AMQP_EXCHANGE_SYSTEM, "node.stranger.heartbeat",
+        {"node_name": "stranger"},
+    ))
+
+    assert "stranger" not in cluster.CLUSTER_NODES
+    assert any("unknown node" in rec.message and "stranger" in rec.message for rec in caplog.records)

tests/test_completions.py

@@ -0,0 +1,222 @@
+import json
+import os
+from pathlib import Path
+
+import httpx
+from fastapi.testclient import TestClient
+
+import app
+import config
+import db
+import routers.completions
+from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS
+
+
+def make_client(tmp_path: Path) -> TestClient:
+    os.environ["CAIC_ADMIN_PIN"] = "1234"
+    db.DB_PATH = tmp_path / "caic-completions.db"
+    SESSIONS.clear()
+    PIN_ATTEMPTS.clear()
+    RATE_EVENTS.clear()
+    db.init_db()
+    return TestClient(app.app, raise_server_exceptions=False)
+
+
+TEST_API_KEY = "test-sk-caic-completions"
+
+
+def _auth_headers(extra: dict = None) -> dict:
+    h = {"Authorization": f"Bearer {TEST_API_KEY}", "Content-Type": "application/json", "Origin": "http://testserver"}
+    if extra:
+        h.update(extra)
+    return h
+
+
+class _MockStreamResponse:
+    def __init__(self, lines: list[str]):
+        self._lines = lines
+
+    async def __aenter__(self):
+        return self
+
+    async def __aexit__(self, exc_type, exc, tb):
+        return False
+
+    async def aiter_lines(self):
+        for line in self._lines:
+            yield line
+
+
+class _MockAsyncPostResponse:
+    def __init__(self, status_code=200, json_data=None):
+        self.status_code = status_code
+        self._json_data = json_data or {}
+
+    def json(self):
+        return self._json_data
+
+
+def _stream_json_lines(events: list[dict]) -> list[str]:
+    return [json.dumps(event) for event in events]
+
+
+def test_completions_missing_api_key(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.post(
+            "/v1/chat/completions",
+            json={"messages": [{"role": "user", "content": "hi"}]},
+        )
+        assert resp.status_code == 401
+
+
+def test_completions_invalid_api_key(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.post(
+            "/v1/chat/completions",
+            json={"messages": [{"role": "user", "content": "hi"}]},
+            headers={"Authorization": "Bearer wrong-key", "Origin": "http://testserver"},
+        )
+        assert resp.status_code == 401
+
+
+def test_completions_no_messages(tmp_path: Path, monkeypatch):
+    monkeypatch.setattr(routers.completions, "COMPLETIONS_API_KEY", TEST_API_KEY)
+    with make_client(tmp_path) as client:
+        resp = client.post("/v1/chat/completions", json={}, headers=_auth_headers())
+        assert resp.status_code == 400
+
+
+def test_completions_empty_messages(tmp_path: Path, monkeypatch):
+    monkeypatch.setattr(routers.completions, "COMPLETIONS_API_KEY", TEST_API_KEY)
+    with make_client(tmp_path) as client:
+        resp = client.post("/v1/chat/completions", json={"messages": []}, headers=_auth_headers())
+        assert resp.status_code == 400
+
+
+def test_completions_no_user_message(tmp_path: Path, monkeypatch):
+    monkeypatch.setattr(routers.completions, "COMPLETIONS_API_KEY", TEST_API_KEY)
+    with make_client(tmp_path) as client:
+        resp = client.post(
+            "/v1/chat/completions",
+            json={"messages": [{"role": "assistant", "content": "hello"}]},
+            headers=_auth_headers(),
+        )
+        assert resp.status_code == 400
+
+
+def test_completions_streaming(tmp_path: Path, monkeypatch):
+    monkeypatch.setattr(routers.completions, "COMPLETIONS_API_KEY", TEST_API_KEY)
+    events = _stream_json_lines([
+        {"choices": [{"delta": {"content": "Hello"}, "logprobs": None}]},
+        {"choices": [{"delta": {"content": " world"}, "logprobs": None}]},
+        {"choices": [{"delta": {}, "finish_reason": "stop"}], "usage": {"tokens_per_second": 15.0}},
+    ])
+
+    call_count = 0
+
+    def stream_stub(self, method, url, json=None, timeout=None):
+        nonlocal call_count
+        call_count += 1
+        return _MockStreamResponse(events)
+
+    monkeypatch.setattr(httpx.AsyncClient, "stream", stream_stub)
+
+    with make_client(tmp_path) as client:
+        resp = client.post(
+            "/v1/chat/completions",
+            json={"messages": [{"role": "user", "content": "hi"}], "stream": True},
+            headers=_auth_headers(),
+        )
+        assert resp.status_code == 200
+        body = resp.text
+        assert "data: [DONE]" in body
+        assert "Hello" in body or "world" in body
+        assert "chatcmpl-" in body
+
+
+def test_completions_blocking(tmp_path: Path, monkeypatch):
+    monkeypatch.setattr(routers.completions, "COMPLETIONS_API_KEY", TEST_API_KEY)
+    events = _stream_json_lines([
+        {"choices": [{"delta": {"content": "Hello world"}, "logprobs": None}]},
+        {"choices": [{"delta": {}, "finish_reason": "stop"}], "usage": {}},
+    ])
+
+    def stream_stub(self, method, url, json=None, timeout=None):
+        return _MockStreamResponse(events)
+
+    monkeypatch.setattr(httpx.AsyncClient, "stream", stream_stub)
+
+    with make_client(tmp_path) as client:
+        resp = client.post(
+            "/v1/chat/completions",
+            json={"messages": [{"role": "user", "content": "hi"}], "stream": False},
+            headers=_auth_headers(),
+        )
+        assert resp.status_code == 200
+        data = resp.json()
+        assert data["object"] == "chat.completion"
+        assert data["choices"][0]["message"]["content"] == "Hello world"
+
+
+def test_completions_fim_passthrough(tmp_path: Path, monkeypatch):
+    monkeypatch.setattr(routers.completions, "COMPLETIONS_API_KEY", TEST_API_KEY)
+    fim_data = {"prompt": "def foo():\n    ", "suffix": "\n    return x", "model": "llama3.1:latest"}
+
+    async def mock_post(self, url, json=None, timeout=None):
+        return _MockAsyncPostResponse(json_data={"choices": [{"text": "pass"}], "usage": {}})
+
+    monkeypatch.setattr(httpx.AsyncClient, "post", mock_post)
+
+    with make_client(tmp_path) as client:
+        resp = client.post("/v1/chat/completions", json=fim_data, headers=_auth_headers())
+        assert resp.status_code == 200
+        assert "choices" in resp.json()
+
+
+def test_completions_connect_error_stream(tmp_path: Path, monkeypatch):
+    monkeypatch.setattr(routers.completions, "COMPLETIONS_API_KEY", TEST_API_KEY)
+
+    def broken_stream(self, method, url, json=None, timeout=None):
+        raise httpx.ConnectError("Connection refused")
+
+    monkeypatch.setattr(httpx.AsyncClient, "stream", broken_stream)
+
+    with make_client(tmp_path) as client:
+        resp = client.post(
+            "/v1/chat/completions",
+            json={"messages": [{"role": "user", "content": "hi"}], "stream": True},
+            headers=_auth_headers(),
+        )
+        assert resp.status_code == 200
+        assert "connection_error" in resp.text
+
+
+def test_completions_connect_error_blocking(tmp_path: Path, monkeypatch):
+    monkeypatch.setattr(routers.completions, "COMPLETIONS_API_KEY", TEST_API_KEY)
+
+    def broken_stream(self, method, url, json=None, timeout=None):
+        raise httpx.ConnectError("Connection refused")
+
+    monkeypatch.setattr(httpx.AsyncClient, "stream", broken_stream)
+
+    with make_client(tmp_path) as client:
+        resp = client.post(
+            "/v1/chat/completions",
+            json={"messages": [{"role": "user", "content": "hi"}], "stream": False},
+            headers=_auth_headers(),
+        )
+        assert resp.status_code == 503
+
+
+def test_completions_fim_connect_error(tmp_path: Path, monkeypatch):
+    monkeypatch.setattr(routers.completions, "COMPLETIONS_API_KEY", TEST_API_KEY)
+    fim_data = {"prompt": "def foo():", "model": "llama3.1:latest"}
+
+    def broken_post(self, url, json=None, timeout=None):
+        raise httpx.ConnectError("Connection refused")
+
+    monkeypatch.setattr(httpx.AsyncClient, "post", broken_post)
+
+    with make_client(tmp_path) as client:
+        resp = client.post("/v1/chat/completions", json=fim_data, headers=_auth_headers())
+        assert resp.status_code == 503

tests/test_conversations.py

@@ -0,0 +1,153 @@
+import os
+from pathlib import Path
+
+from fastapi.testclient import TestClient
+
+import app
+import db
+from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS
+
+
+def make_client(tmp_path: Path) -> TestClient:
+    os.environ["CAIC_ADMIN_PIN"] = "1234"
+    db.DB_PATH = tmp_path / "caic-conversations.db"
+    SESSIONS.clear()
+    PIN_ATTEMPTS.clear()
+    RATE_EVENTS.clear()
+    db.init_db()
+    return TestClient(app.app, raise_server_exceptions=False)
+
+
+def _admin_headers(client: TestClient) -> dict:
+    login = client.post("/api/auth/login", json={"pin": "1234"}, headers={"Origin": "http://testserver"})
+    sid = login.json()["session_id"]
+    return {"X-Session-ID": sid, "Origin": "http://testserver"}
+
+
+def _guest_headers(client: TestClient) -> dict:
+    sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()["session_id"]
+    return {"X-Session-ID": sid, "Origin": "http://testserver"}
+
+
+def test_list_conversations_empty(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.get("/api/conversations", headers=_guest_headers(client))
+        assert resp.status_code == 200
+        assert resp.json() == []
+
+
+def test_create_and_list_conversation(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        headers = _admin_headers(client)
+
+        create = client.post("/api/conversations", json={"title": "Test Chat", "model": "llama3.1:latest"}, headers=headers)
+        assert create.status_code == 200
+        data = create.json()
+        assert data["title"] == "Test Chat"
+        assert data["model"] == "llama3.1:latest"
+
+        list_resp = client.get("/api/conversations", headers=headers)
+        assert list_resp.status_code == 200
+        convs = list_resp.json()
+        assert len(convs) == 1
+        assert convs[0]["title"] == "Test Chat"
+
+
+def test_get_conversation_returns_messages(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        headers = _admin_headers(client)
+        create = client.post("/api/conversations", json={"title": "My Chat"}, headers=headers)
+        conv_id = create.json()["id"]
+
+        resp = client.get(f"/api/conversations/{conv_id}", headers=headers)
+        assert resp.status_code == 200
+        data = resp.json()
+        assert data["conversation"]["id"] == conv_id
+        assert data["messages"] == []
+
+
+def test_get_conversation_not_found(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.get("/api/conversations/nope", headers=_guest_headers(client))
+        assert resp.status_code == 404
+
+
+def test_update_conversation_title(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        headers = _admin_headers(client)
+        create = client.post("/api/conversations", json={"title": "Old"}, headers=headers)
+        conv_id = create.json()["id"]
+
+        update = client.put(f"/api/conversations/{conv_id}", json={"title": "New Title"}, headers=headers)
+        assert update.status_code == 200
+
+        get = client.get(f"/api/conversations/{conv_id}", headers=headers)
+        assert get.json()["conversation"]["title"] == "New Title"
+
+
+def test_update_conversation_model(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        headers = _admin_headers(client)
+        create = client.post("/api/conversations", json={"title": "Test"}, headers=headers)
+        conv_id = create.json()["id"]
+
+        update = client.put(f"/api/conversations/{conv_id}", json={"model": "qwen2:latest"}, headers=headers)
+        assert update.status_code == 200
+
+        get = client.get(f"/api/conversations/{conv_id}", headers=headers)
+        assert get.json()["conversation"]["model"] == "qwen2:latest"
+
+
+def test_delete_conversation(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        headers = _admin_headers(client)
+        create = client.post("/api/conversations", json={"title": "Delete Me"}, headers=headers)
+        conv_id = create.json()["id"]
+
+        delete = client.delete(f"/api/conversations/{conv_id}", headers=headers)
+        assert delete.status_code == 200
+
+        get = client.get(f"/api/conversations/{conv_id}", headers=_guest_headers(client))
+        assert get.status_code == 404
+
+
+def test_delete_all_conversations(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        headers = _admin_headers(client)
+        client.post("/api/conversations", json={"title": "One"}, headers=headers)
+        client.post("/api/conversations", json={"title": "Two"}, headers=headers)
+
+        delete_all = client.delete("/api/conversations", headers=headers)
+        assert delete_all.status_code == 200
+
+        list_resp = client.get("/api/conversations", headers=_guest_headers(client))
+        assert list_resp.json() == []
+
+
+def test_guest_cannot_create_conversation(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.post("/api/conversations", json={"title": "test"}, headers=_guest_headers(client))
+        assert resp.status_code == 403
+
+
+def test_guest_cannot_update_conversation(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        headers = _admin_headers(client)
+        create = client.post("/api/conversations", json={"title": "Test"}, headers=headers)
+        conv_id = create.json()["id"]
+
+        guest_headers = _guest_headers(client)
+        resp = client.put(f"/api/conversations/{conv_id}", json={"title": "hack"}, headers=guest_headers)
+        assert resp.status_code == 403
+
+
+def test_guest_cannot_delete_conversation(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.delete("/api/conversations/some-id", headers=_guest_headers(client))
+        assert resp.status_code == 403
+
+
+def test_guest_cannot_delete_all(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.delete("/api/conversations", headers=_guest_headers(client))
+        assert resp.status_code == 403

tests/test_error_envelopes.py

@@ -1,19 +1,24 @@
 import os
 from pathlib import Path
 
+import httpx
 from fastapi.testclient import TestClient
 
-import app as app_module
+import app
+import config
+import db
+import routers.memories
+from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS
 
 
 def make_client(tmp_path: Path) -> TestClient:
-    os.environ["JARVISCHAT_ADMIN_PIN"] = "1234"
+    os.environ["CAIC_ADMIN_PIN"] = "1234"
-    app_module.DB_PATH = tmp_path / "jarvischat-errors.db"
+    db.DB_PATH = tmp_path / "caic-errors.db"
-    app_module.SESSIONS.clear()
+    SESSIONS.clear()
-    app_module.PIN_ATTEMPTS.clear()
+    PIN_ATTEMPTS.clear()
-    app_module.RATE_EVENTS.clear()
+    RATE_EVENTS.clear()
-    app_module.init_db()
+    db.init_db()
-    return TestClient(app_module.app, raise_server_exceptions=False)
+    return TestClient(app.app, raise_server_exceptions=False)
 
 
 def test_unhandled_api_exception_returns_friendly_error_with_incident_key(
@@ -23,12 +28,12 @@ def test_unhandled_api_exception_returns_friendly_error_with_incident_key(
         sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()[
             "session_id"
         ]
-        headers = {"X-Session-ID": sid}
+        headers = {"X-Session-ID": sid, "Origin": "http://testserver"}
 
         def boom(_topic=None):
             raise RuntimeError("super secret db internals")
 
-        monkeypatch.setattr(app_module, "get_all_memories", boom)
+        monkeypatch.setattr(routers.memories, "get_all_memories", boom)
 
         resp = client.get("/api/memories", headers=headers)
         assert resp.status_code == 500
@@ -57,11 +62,11 @@ def test_chat_stream_error_hides_internal_exception_and_emits_incident_key(
         def broken_stream(*args, **kwargs):
             return BrokenStreamContext()
 
-        monkeypatch.setattr(app_module.httpx.AsyncClient, "stream", broken_stream)
+        monkeypatch.setattr(httpx.AsyncClient, "stream", broken_stream)
 
         resp = client.post(
             "/api/chat",
-            json={"message": "hello", "model": app_module.DEFAULT_MODEL},
+            json={"message": "hello", "model": config.DEFAULT_MODEL},
             headers=headers,
         )
 

tests/test_hardware.py

@@ -0,0 +1,176 @@
+import asyncio
+import json
+import os
+import subprocess
+from pathlib import Path
+
+import httpx
+import psutil
+from fastapi.testclient import TestClient
+
+import app as app_module
+import config
+import db
+import hardware
+from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS
+
+
+def make_client(tmp_path: Path) -> TestClient:
+    os.environ["CAIC_ADMIN_PIN"] = "1234"
+    db.DB_PATH = tmp_path / "caic-hardware.db"
+    hardware.HARDWARE_STATE_PATH = tmp_path / "hardware_state.json"
+    SESSIONS.clear()
+    PIN_ATTEMPTS.clear()
+    RATE_EVENTS.clear()
+    db.init_db()
+    return TestClient(app_module.app, raise_server_exceptions=False)
+
+
+def _guest_headers(client: TestClient) -> dict:
+    sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()["session_id"]
+    return {"X-Session-ID": sid, "Origin": "http://testserver"}
+
+
+class _MockGet:
+    def __init__(self, status_code: int = 200, json_data: dict | None = None):
+        self.status_code = status_code
+        self._json_data = json_data or {}
+
+    def json(self):
+        return self._json_data
+
+
+def _mock_subprocess(rocm_stdout: str = "") -> object:
+    class MockProc:
+        returncode = 0
+        stdout = rocm_stdout
+
+    class MockSP:
+        TimeoutExpired = subprocess.TimeoutExpired
+        run = staticmethod(lambda cmd, **kw: MockProc())
+
+    return MockSP()
+
+
+def _broken_subprocess(exception: Exception) -> object:
+    class MockSP:
+        TimeoutExpired = subprocess.TimeoutExpired
+        run = staticmethod(lambda cmd, **kw: (_ for _ in ()).throw(exception))
+
+    return MockSP()
+
+
+def test_assess_hardware_all_services_reachable(tmp_path: Path, monkeypatch):
+    hardware.HARDWARE_STATE_PATH = tmp_path / "hardware_state.json"
+    monkeypatch.setattr(psutil, "virtual_memory", lambda: type("M", (), {"total": 16 * 1024 ** 3, "available": 8 * 1024 ** 3})())
+    monkeypatch.setattr(psutil, "cpu_count", lambda: 8)
+    monkeypatch.setattr(hardware, "subprocess", _mock_subprocess(
+        json.dumps({"card0": {"VRAM Total (MB)": 8192, "VRAM Free (MB)": 4096}})
+    ))
+
+    async def mock_get(self, url, *args, **kwargs):
+        if "v1/models" in url:
+            return _MockGet(200, {"data": [{"id": "mistral-nemo:latest"}]})
+        if "6333" in url:
+            return _MockGet(200, {"result": {"collections": [{"name": "caic"}]}})
+        if "8888" in url:
+            return _MockGet(200, {})
+        return _MockGet(200, {})
+    monkeypatch.setattr(httpx.AsyncClient, "get", mock_get)
+
+    state = asyncio.run(hardware.assess_hardware())
+
+    assert state["ram_total_gb"] == 16.0
+    assert state["ram_available_gb"] == 8.0
+    assert state["cpu_count"] == 8
+    assert state["vram_total_mb"] == 8192
+    assert state["vram_free_mb"] == 4096
+    assert state["llama_reachable"] is True
+    assert state["llama_models"] == ["mistral-nemo:latest"]
+    assert state["qdrant_reachable"] is True
+    assert state["qdrant_collections"] == ["caic"]
+    assert state["searxng_reachable"] is True
+    assert tmp_path.joinpath("hardware_state.json").exists()
+
+
+def test_assess_hardware_rocm_smi_absent(tmp_path: Path, monkeypatch):
+    hardware.HARDWARE_STATE_PATH = tmp_path / "hardware_state.json"
+    monkeypatch.setattr(psutil, "virtual_memory", lambda: type("M", (), {"total": 16 * 1024 ** 3, "available": 8 * 1024 ** 3})())
+    monkeypatch.setattr(psutil, "cpu_count", lambda: 8)
+    monkeypatch.setattr(hardware, "subprocess", _broken_subprocess(FileNotFoundError("no rocm-smi")))
+
+    async def mock_get(self, url, *args, **kwargs):
+        if "v1/models" in url:
+            return _MockGet(200, {"data": []})
+        if "6333" in url:
+            return _MockGet(200, {"result": {"collections": []}})
+        if "8888" in url:
+            return _MockGet(200, {})
+        return _MockGet(200, {})
+    monkeypatch.setattr(httpx.AsyncClient, "get", mock_get)
+
+    state = asyncio.run(hardware.assess_hardware())
+
+    assert state["vram_total_mb"] == 0
+    assert state["vram_free_mb"] == 0
+    assert state["llama_reachable"] is True
+    assert state["qdrant_reachable"] is True
+    assert state["searxng_reachable"] is True
+
+
+def test_assess_hardware_llama_unreachable(tmp_path: Path, monkeypatch):
+    hardware.HARDWARE_STATE_PATH = tmp_path / "hardware_state.json"
+    monkeypatch.setattr(psutil, "virtual_memory", lambda: type("M", (), {"total": 16 * 1024 ** 3, "available": 8 * 1024 ** 3})())
+    monkeypatch.setattr(psutil, "cpu_count", lambda: 8)
+    monkeypatch.setattr(hardware, "subprocess", _mock_subprocess(
+        json.dumps({"card0": {"VRAM Total (MB)": 8192, "VRAM Free (MB)": 4096}})
+    ))
+
+    async def mock_get(self, url, *args, **kwargs):
+        if "v1/models" in url:
+            raise httpx.ConnectError("refused")
+        if "6333" in url:
+            return _MockGet(200, {"result": {"collections": [{"name": "caic"}]}})
+        if "8888" in url:
+            return _MockGet(200, {})
+        return _MockGet(200, {})
+    monkeypatch.setattr(httpx.AsyncClient, "get", mock_get)
+
+    state = asyncio.run(hardware.assess_hardware())
+
+    assert state["llama_reachable"] is False
+    assert state["llama_models"] == []
+    assert state["qdrant_reachable"] is True
+    assert state["searxng_reachable"] is True
+
+
+def test_get_hardware_endpoint(tmp_path: Path, monkeypatch):
+    hardware.HARDWARE_STATE_PATH = tmp_path / "hardware_state.json"
+    monkeypatch.setattr(psutil, "virtual_memory", lambda: type("M", (), {"total": 16 * 1024 ** 3, "available": 8 * 1024 ** 3})())
+    monkeypatch.setattr(psutil, "cpu_count", lambda: 8)
+    monkeypatch.setattr(hardware, "subprocess", _mock_subprocess(
+        json.dumps({"card0": {"VRAM Total (MB)": 8192, "VRAM Free (MB)": 4096}})
+    ))
+
+    async def mock_get(self, url, *args, **kwargs):
+        if "v1/models" in url:
+            return _MockGet(200, {"data": [{"id": "mistral-nemo:latest"}]})
+        if "6333" in url:
+            return _MockGet(200, {"result": {"collections": [{"name": "caic"}]}})
+        if "8888" in url:
+            return _MockGet(200, {})
+        return _MockGet(200, {})
+    monkeypatch.setattr(httpx.AsyncClient, "get", mock_get)
+
+    with make_client(tmp_path) as client:
+        resp = client.get("/api/hardware", headers=_guest_headers(client))
+        assert resp.status_code == 200
+        data = resp.json()
+        assert data["ram_total_gb"] == 16.0
+        assert data["cpu_count"] == 8
+        assert data["vram_total_mb"] == 8192
+        assert data["llama_reachable"] is True
+        assert data["qdrant_reachable"] is True
+        assert data["searxng_reachable"] is True
+        assert "llama_models" in data
+        assert "qdrant_collections" in data

tests/test_ingest.py

@@ -0,0 +1,104 @@
+import json
+import os
+from pathlib import Path
+
+import httpx
+from fastapi.testclient import TestClient
+
+import app
+import db
+import routers.ingest as ingest_route
+from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS
+
+
+def make_client(tmp_path: Path) -> TestClient:
+    os.environ["CAIC_ADMIN_PIN"] = "1234"
+    db.DB_PATH = tmp_path / "caic-ingest.db"
+    SESSIONS.clear()
+    PIN_ATTEMPTS.clear()
+    RATE_EVENTS.clear()
+    db.init_db()
+    return TestClient(app.app, raise_server_exceptions=False)
+
+
+TEST_API_KEY = "test-sk-caic-ingest"
+
+
+def _auth_headers() -> dict:
+    return {"Authorization": f"Bearer {TEST_API_KEY}", "Content-Type": "application/json", "Origin": "http://testserver"}
+
+
+def test_ingest_missing_api_key(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.post("/api/ingest", json={"content": "test"}, headers={"Origin": "http://testserver"})
+        assert resp.status_code == 401
+
+
+def test_ingest_wrong_api_key(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.post("/api/ingest", json={"content": "test"},
+                           headers={"Authorization": "Bearer wrong", "Origin": "http://testserver"})
+        assert resp.status_code == 401
+
+
+def test_ingest_empty_content(tmp_path: Path):
+    monkeypatch = __import__('pytest').MonkeyPatch()
+    monkeypatch.setattr(ingest_route, "COMPLETIONS_API_KEY", TEST_API_KEY)
+    with make_client(tmp_path) as client:
+        resp = client.post("/api/ingest", json={"content": ""}, headers=_auth_headers())
+        assert resp.status_code == 422
+    monkeypatch.undo()
+
+
+def test_ingest_missing_content(tmp_path: Path):
+    monkeypatch = __import__('pytest').MonkeyPatch()
+    monkeypatch.setattr(ingest_route, "COMPLETIONS_API_KEY", TEST_API_KEY)
+    with make_client(tmp_path) as client:
+        resp = client.post("/api/ingest", json={}, headers=_auth_headers())
+        assert resp.status_code == 422
+    monkeypatch.undo()
+
+
+def test_ingest_success(tmp_path: Path, monkeypatch):
+    monkeypatch.setattr(ingest_route, "COMPLETIONS_API_KEY", TEST_API_KEY)
+
+    embed_count = 0
+
+    class FakeAsyncClient:
+        class FakeResponse:
+            def __init__(self, status, json_data=None):
+                self.status_code = status
+                self._json = json_data or {}
+
+            def json(self):
+                return self._json
+
+        def __init__(self, *a, **kw):
+            pass
+
+        async def __aenter__(self):
+            return self
+
+        async def __aexit__(self, *a):
+            pass
+
+        async def post(self, url, **kw):
+            nonlocal embed_count
+            if "/api/embeddings" in url:
+                embed_count += 1
+                return self.FakeResponse(200, {"embedding": [0.1] * 768})
+            return self.FakeResponse(200)
+
+        async def put(self, url, **kw):
+            return self.FakeResponse(200)
+
+    monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: FakeAsyncClient())
+
+    with make_client(tmp_path) as client:
+        resp = client.post("/api/ingest", json={"content": "test " * 1000, "source": "terminal"}, headers=_auth_headers())
+        assert resp.status_code == 200
+        data = resp.json()
+        assert data["source"] == "terminal"
+        assert data["chunks_ingested"] > 0
+        assert embed_count == data["chunks_ingested"]
+        assert "message" in data

tests/test_ip_allowlist.py

@@ -3,48 +3,42 @@ from pathlib import Path
 
 from fastapi.testclient import TestClient
 
-import app as app_module
+import app
+import db
+from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS, is_ip_allowed
 
 
 def make_client(tmp_path: Path) -> TestClient:
-    os.environ["JARVISCHAT_ADMIN_PIN"] = "1234"
+    os.environ["CAIC_ADMIN_PIN"] = "1234"
-    app_module.DB_PATH = tmp_path / "jarvischat-ip.db"
+    db.DB_PATH = tmp_path / "caic-ip.db"
-    app_module.SESSIONS.clear()
+    SESSIONS.clear()
-    app_module.PIN_ATTEMPTS.clear()
+    PIN_ATTEMPTS.clear()
-    app_module.RATE_EVENTS.clear()
+    RATE_EVENTS.clear()
-    app_module.init_db()
+    db.init_db()
-    return TestClient(app_module.app)
+    return TestClient(app.app)
 
 
 def test_ip_helper_allows_local_defaults():
-    assert app_module.is_ip_allowed("127.0.0.1")
+    assert is_ip_allowed("127.0.0.1")
-    assert app_module.is_ip_allowed("192.168.1.10")
+    assert is_ip_allowed("192.168.1.10")
-    assert app_module.is_ip_allowed("10.0.0.42")
+    assert is_ip_allowed("10.0.0.42")
-    assert app_module.is_ip_allowed("172.16.1.2")
+    assert is_ip_allowed("172.16.1.2")
-    assert app_module.is_ip_allowed("testclient")
+    assert is_ip_allowed("testclient")
 
 
 def test_ip_helper_blocks_public_ip():
-    assert not app_module.is_ip_allowed("8.8.8.8")
+    assert not is_ip_allowed("8.8.8.8")
 
 
-def test_middleware_blocks_disallowed_ip(tmp_path: Path):
+def test_middleware_blocks_disallowed_ip(tmp_path: Path, monkeypatch):
+    monkeypatch.setattr(app, "get_client_ip", lambda _req: "8.8.8.8")
     with make_client(tmp_path) as client:
-        original_get_client_ip = app_module.get_client_ip
+        resp = client.post("/api/auth/guest")
-        try:
+        assert resp.status_code == 403
-            app_module.get_client_ip = lambda _req: "8.8.8.8"
-            resp = client.post("/api/auth/guest")
-            assert resp.status_code == 403
-        finally:
-            app_module.get_client_ip = original_get_client_ip
 
 
-def test_middleware_allows_local_ip(tmp_path: Path):
+def test_middleware_allows_local_ip(tmp_path: Path, monkeypatch):
+    monkeypatch.setattr(app, "get_client_ip", lambda _req: "192.168.50.109")
     with make_client(tmp_path) as client:
-        original_get_client_ip = app_module.get_client_ip
+        resp = client.post("/api/auth/guest", headers={"Origin": "http://testserver"})
-        try:
+        assert resp.status_code == 200
-            app_module.get_client_ip = lambda _req: "192.168.50.109"
-            resp = client.post("/api/auth/guest")
-            assert resp.status_code == 200
-        finally:
-            app_module.get_client_ip = original_get_client_ip

tests/test_memories.py

@@ -0,0 +1,161 @@
+import os
+from pathlib import Path
+
+from fastapi.testclient import TestClient
+
+import app
+import config
+import db
+from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS
+
+
+def make_client(tmp_path: Path) -> TestClient:
+    os.environ["CAIC_ADMIN_PIN"] = "1234"
+    db.DB_PATH = tmp_path / "caic-memories.db"
+    SESSIONS.clear()
+    PIN_ATTEMPTS.clear()
+    RATE_EVENTS.clear()
+    db.init_db()
+    return TestClient(app.app, raise_server_exceptions=False)
+
+
+def _admin_headers(client: TestClient) -> dict:
+    login = client.post("/api/auth/login", json={"pin": "1234"}, headers={"Origin": "http://testserver"})
+    sid = login.json()["session_id"]
+    return {"X-Session-ID": sid, "Origin": "http://testserver"}
+
+
+def _guest_headers(client: TestClient) -> dict:
+    sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()["session_id"]
+    return {"X-Session-ID": sid, "Origin": "http://testserver"}
+
+
+def _create_memory(client: TestClient, headers: dict, fact: str = "test fact", topic: str = "general") -> int:
+    resp = client.post("/api/memories", json={"fact": fact, "topic": topic}, headers=headers)
+    assert resp.status_code == 200
+    return resp.json()["rowid"]
+
+
+def test_list_memories_empty(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.get("/api/memories", headers=_guest_headers(client))
+        assert resp.status_code == 200
+        assert resp.json()["count"] == 0
+
+
+def test_list_memories_by_topic(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        headers = _admin_headers(client)
+        _create_memory(client, headers, "I like Python", "preference")
+        _create_memory(client, headers, "Building a game", "project")
+
+        general = client.get("/api/memories?topic=preference", headers=_guest_headers(client))
+        assert general.json()["count"] == 1
+        assert general.json()["memories"][0]["topic"] == "preference"
+
+
+def test_create_memory_requires_fact(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.post("/api/memories", json={"fact": ""}, headers=_admin_headers(client))
+        assert resp.status_code == 400
+
+
+def test_create_memory_too_long(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        long_fact = "x" * (config.MAX_MEMORY_FACT_CHARS + 1)
+        resp = client.post("/api/memories", json={"fact": long_fact}, headers=_admin_headers(client))
+        assert resp.status_code == 413
+
+
+def test_edit_memory(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        headers = _admin_headers(client)
+        rowid = _create_memory(client, headers, "original fact")
+
+        edit = client.put(f"/api/memories/{rowid}", json={"fact": "updated fact"}, headers=headers)
+        assert edit.status_code == 200
+
+        memories = client.get("/api/memories", headers=_guest_headers(client)).json()
+        assert any(m["fact"] == "updated fact" for m in memories["memories"])
+
+
+def test_edit_memory_not_found(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.put("/api/memories/99999", json={"fact": "nope"}, headers=_admin_headers(client))
+        assert resp.status_code == 404
+
+
+def test_edit_memory_empty_fact(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        headers = _admin_headers(client)
+        rowid = _create_memory(client, headers, "some fact")
+        resp = client.put(f"/api/memories/{rowid}", json={"fact": ""}, headers=headers)
+        assert resp.status_code == 400
+
+
+def test_edit_memory_too_long(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        headers = _admin_headers(client)
+        rowid = _create_memory(client, headers, "some fact")
+        long_fact = "x" * (config.MAX_MEMORY_FACT_CHARS + 1)
+        resp = client.put(f"/api/memories/{rowid}", json={"fact": long_fact}, headers=headers)
+        assert resp.status_code == 413
+
+
+def test_delete_memory_not_found(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.delete("/api/memories/99999", headers=_admin_headers(client))
+        assert resp.status_code == 404
+
+
+def test_search_memories(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        headers = _admin_headers(client)
+        _create_memory(client, headers, "my favorite color is blue", "preference")
+        _create_memory(client, headers, "running nginx on port 443", "infrastructure")
+
+        resp = client.get("/api/memories/search?q=nginx&limit=5", headers=_guest_headers(client))
+        assert resp.status_code == 200
+        data = resp.json()
+        assert data["count"] >= 1
+        assert any("nginx" in r["fact"] for r in data["results"])
+
+
+def test_search_memories_no_results(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.get("/api/memories/search?q=xyznonexistent&limit=5", headers=_guest_headers(client))
+        assert resp.status_code == 200
+        assert resp.json()["count"] == 0
+
+
+def test_memory_stats(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        headers = _admin_headers(client)
+        _create_memory(client, headers, "like rust", "preference")
+        _create_memory(client, headers, "like python", "preference")
+        _create_memory(client, headers, "project game", "project")
+
+        resp = client.get("/api/memories/stats", headers=_guest_headers(client))
+        assert resp.status_code == 200
+        data = resp.json()
+        assert data["total"] == 3
+        assert data["by_topic"]["preference"] == 2
+        assert data["by_topic"]["project"] == 1
+
+
+def test_guest_cannot_create_memory(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.post("/api/memories", json={"fact": "hack"}, headers=_guest_headers(client))
+        assert resp.status_code == 403
+
+
+def test_guest_cannot_edit_memory(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.put("/api/memories/1", json={"fact": "hack"}, headers=_guest_headers(client))
+        assert resp.status_code == 403
+
+
+def test_guest_cannot_delete_memory(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.delete("/api/memories/1", headers=_guest_headers(client))
+        assert resp.status_code == 403

tests/test_model_swap.py

@@ -0,0 +1,202 @@
+"""Tests for cluster.py model swap flow."""
+import asyncio
+
+import cluster
+import triage
+from config import AMQP_EXCHANGE_ADMIN, AMQP_EXCHANGE_SYSTEM
+
+
+def _reset():
+    cluster.CLUSTER_NODES.clear()
+    cluster.CLUSTER_EVENTS.clear()
+    cluster.CLUSTER_COORDINATOR = None
+    cluster._pending_pings.clear()
+
+
+_published = []
+
+
+async def _fake_publish(exchange, routing_key, payload):
+    _published.append((exchange, routing_key, payload))
+
+
+# ---------- 1. request_model_swap() publishes swap command ----------
+
+
+def test_request_model_swap_publishes_command(monkeypatch):
+    _reset()
+    _published.clear()
+    monkeypatch.setattr(cluster, "publish", _fake_publish)
+
+    cluster.CLUSTER_NODES["jarvis"] = {
+        "name": "jarvis", "type": "worker", "status": "active",
+    }
+
+    asyncio.run(cluster.request_model_swap("jarvis", "qwen2.5-coder-Q4_K_M.gguf"))
+
+    assert cluster.CLUSTER_NODES["jarvis"]["status"] == "swapping"
+    assert len(_published) == 1
+    exchange, rk, payload = _published[0]
+    assert exchange == AMQP_EXCHANGE_ADMIN
+    assert rk == "node.jarvis.cmd.swap_model"
+    assert payload["model_filename"] == "qwen2.5-coder-Q4_K_M.gguf"
+    assert "requested_at" in payload
+
+
+def test_request_model_swap_unknown_node(monkeypatch):
+    _reset()
+    _published.clear()
+    monkeypatch.setattr(cluster, "publish", _fake_publish)
+
+    result = asyncio.run(cluster.request_model_swap("ghost", "any.gguf"))
+    assert result is False
+    assert len(_published) == 0
+
+
+# ---------- 2. handle_model_ready() updates node ----------
+
+
+def test_handle_model_ready_updates_active_model(monkeypatch):
+    _reset()
+    _published.clear()
+    monkeypatch.setattr(cluster, "publish", _fake_publish)
+
+    cluster.CLUSTER_NODES["jarvis"] = {
+        "name": "jarvis", "type": "worker", "status": "swapping",
+        "active_model": {"name": "llama3.1", "port": 8081},
+        "inventory": [
+            {"filename": "qwen2.5-coder-Q4_K_M.gguf", "name": "qwen2.5-coder", "version": "14b", "quant": "Q4_K_M"},
+        ],
+    }
+
+    asyncio.run(cluster.handle_model_ready(
+        AMQP_EXCHANGE_SYSTEM, "node.jarvis.model_ready",
+        {"node_name": "jarvis", "active_model": "qwen2.5-coder-Q4_K_M.gguf", "port": 8082},
+    ))
+
+    assert cluster.CLUSTER_NODES["jarvis"]["status"] == "active"
+    am = cluster.CLUSTER_NODES["jarvis"]["active_model"]
+    assert am["name"] == "qwen2.5-coder"
+    assert am["port"] == 8082
+    assert am["filename"] == "qwen2.5-coder-Q4_K_M.gguf"
+
+
+def test_handle_model_ready_inventory_lookup_fallback(monkeypatch):
+    _reset()
+    _published.clear()
+    monkeypatch.setattr(cluster, "publish", _fake_publish)
+
+    cluster.CLUSTER_NODES["jarvis"] = {
+        "name": "jarvis", "type": "worker", "status": "swapping",
+        "active_model": {"name": "llama3.1", "port": 8081},
+        "inventory": [],
+    }
+
+    asyncio.run(cluster.handle_model_ready(
+        AMQP_EXCHANGE_SYSTEM, "node.jarvis.model_ready",
+        {"node_name": "jarvis", "active_model": "unknown.gguf", "port": 9999},
+    ))
+
+    assert cluster.CLUSTER_NODES["jarvis"]["status"] == "active"
+    am = cluster.CLUSTER_NODES["jarvis"]["active_model"]
+    assert am["filename"] == "unknown.gguf"
+    assert am["port"] == 9999
+
+
+def test_handle_model_ready_unknown_node(caplog, monkeypatch):
+    _reset()
+    caplog.set_level("WARNING")
+    monkeypatch.setattr(cluster, "publish", _fake_publish)
+
+    asyncio.run(cluster.handle_model_ready(
+        AMQP_EXCHANGE_SYSTEM, "node.ghost.model_ready",
+        {"node_name": "ghost", "active_model": "any.gguf"},
+    ))
+
+    assert any("unknown node" in rec.message for rec in caplog.records)
+
+
+# ---------- 3. handle_model_failed() sets error status ----------
+
+
+def test_handle_model_failed_sets_error(monkeypatch):
+    _reset()
+    _published.clear()
+    monkeypatch.setattr(cluster, "publish", _fake_publish)
+
+    cluster.CLUSTER_NODES["jarvis"] = {
+        "name": "jarvis", "type": "worker", "status": "swapping",
+    }
+
+    asyncio.run(cluster.handle_model_failed(
+        AMQP_EXCHANGE_SYSTEM, "node.jarvis.model_failed",
+        {"node_name": "jarvis", "error": "llama-server unhealthy after 120s"},
+    ))
+
+    assert cluster.CLUSTER_NODES["jarvis"]["status"] == "error"
+    assert len(cluster.CLUSTER_EVENTS) == 1
+    assert "swap failed" in cluster.CLUSTER_EVENTS[0]["message"].lower()
+
+
+def test_handle_model_failed_unknown_node(caplog, monkeypatch):
+    _reset()
+    caplog.set_level("WARNING")
+    monkeypatch.setattr(cluster, "publish", _fake_publish)
+
+    asyncio.run(cluster.handle_model_failed(
+        AMQP_EXCHANGE_SYSTEM, "node.ghost.model_failed",
+        {"node_name": "ghost", "error": "OOM"},
+    ))
+
+    assert any("unknown node" in rec.message for rec in caplog.records)
+
+
+# ---------- 4. select_node() triggers swap when model mismatched ----------
+
+
+def test_select_node_code_triggers_swap(monkeypatch):
+    _reset()
+    _published.clear()
+    monkeypatch.setattr(cluster, "publish", _fake_publish)
+
+    cluster.CLUSTER_NODES["jarvis"] = {
+        "name": "jarvis", "type": "worker", "status": "active",
+        "ip": "192.168.50.210",
+        "active_model": {"name": "llama3.1", "port": 8081},
+        "inventory": [
+            {"filename": "qwen2.5-coder-14b-Q4_K_M.gguf", "name": "qwen2.5-coder", "version": "14b", "quant": "Q4_K_M"},
+        ],
+    }
+
+    result = asyncio.run(triage.select_node("code"))
+
+    assert result is None
+    # Swap should have been published
+    assert any("cmd.swap_model" in rk for _, rk, _ in _published)
+    # Node should now be swapping
+    assert cluster.CLUSTER_NODES["jarvis"]["status"] == "swapping"
+
+
+# ---------- 5. select_node() returns None when node is already swapping ----------
+
+
+def test_select_node_swapping_returns_none(monkeypatch):
+    _reset()
+    _published.clear()
+    monkeypatch.setattr(cluster, "publish", _fake_publish)
+
+    cluster.CLUSTER_NODES["jarvis"] = {
+        "name": "jarvis", "type": "worker", "status": "swapping",
+        "ip": "192.168.50.210",
+        "active_model": {"name": "llama3.1", "port": 8081},
+        "inventory": [
+            {"filename": "qwen2.5-coder-14b-Q4_K_M.gguf", "name": "qwen2.5-coder"},
+        ],
+    }
+
+    result = asyncio.run(triage.select_node("code"))
+
+    assert result is None
+    # No swap command should be published while already swapping
+    swap_published = any("cmd.swap_model" in rk for _, rk, _ in _published)
+    assert not swap_published

tests/test_models_router.py

@@ -0,0 +1,138 @@
+import os
+from pathlib import Path
+
+import httpx
+from fastapi.testclient import TestClient
+
+import app
+import db
+import routers.models
+from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS
+
+
+def make_client(tmp_path: Path) -> TestClient:
+    os.environ["CAIC_ADMIN_PIN"] = "1234"
+    db.DB_PATH = tmp_path / "caic-models.db"
+    SESSIONS.clear()
+    PIN_ATTEMPTS.clear()
+    RATE_EVENTS.clear()
+    db.init_db()
+    return TestClient(app.app, raise_server_exceptions=False)
+
+
+def _guest_headers(client: TestClient) -> dict:
+    sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()["session_id"]
+    return {"X-Session-ID": sid, "Origin": "http://testserver"}
+
+
+class _MockAsyncResponse:
+    """Mock for httpx.AsyncClient.get/post that returns a JSON response."""
+    def __init__(self, status_code=200, json_data=None):
+        self.status_code = status_code
+        self._json_data = json_data or {}
+
+    def json(self):
+        return self._json_data
+
+
+async def _mock_get_models(*args, **kwargs):
+    return _MockAsyncResponse(json_data={
+        "data": [{"id": "llama3.1:latest"}, {"id": "qwen2:latest"}]
+    })
+
+
+async def _mock_get_empty_models(*args, **kwargs):
+    return _MockAsyncResponse(json_data={"data": []})
+
+
+async def _mock_connect_error(*args, **kwargs):
+    raise httpx.ConnectError("Connection refused")
+
+
+async def _mock_show_model(*args, **kwargs):
+    return _MockAsyncResponse(json_data={
+        "modelfile": "FROM llama3.1", "parameters": {}
+    })
+
+
+async def _mock_search_available(*args, **kwargs):
+    return _MockAsyncResponse(status_code=200)
+
+
+async def _mock_search_unavailable(*args, **kwargs):
+    raise httpx.ConnectError("refused")
+
+
+def test_list_models(tmp_path: Path, monkeypatch):
+    monkeypatch.setattr(httpx.AsyncClient, "get", _mock_get_models)
+    with make_client(tmp_path) as client:
+        resp = client.get("/api/models", headers=_guest_headers(client))
+        assert resp.status_code == 200
+        models = resp.json()["models"]
+        assert len(models) == 2
+        assert models[0]["name"] == "llama3.1:latest"
+
+
+def test_list_models_connect_error(tmp_path: Path, monkeypatch):
+    monkeypatch.setattr(httpx.AsyncClient, "get", _mock_connect_error)
+    with make_client(tmp_path) as client:
+        resp = client.get("/api/models", headers=_guest_headers(client))
+        assert resp.status_code == 502
+
+
+def test_running_models(tmp_path: Path, monkeypatch):
+    monkeypatch.setattr(httpx.AsyncClient, "get", _mock_get_models)
+    with make_client(tmp_path) as client:
+        resp = client.get("/api/ps", headers=_guest_headers(client))
+        assert resp.status_code == 200
+        assert "data" in resp.json()
+
+
+def test_running_models_connect_error(tmp_path: Path, monkeypatch):
+    monkeypatch.setattr(httpx.AsyncClient, "get", _mock_connect_error)
+    with make_client(tmp_path) as client:
+        resp = client.get("/api/ps", headers=_guest_headers(client))
+        assert resp.status_code == 502
+
+
+def test_show_model(tmp_path: Path, monkeypatch):
+    monkeypatch.setattr(httpx.AsyncClient, "post", _mock_show_model)
+    with make_client(tmp_path) as client:
+        resp = client.post("/api/show", json={"model": "llama3.1:latest"}, headers=_guest_headers(client))
+        assert resp.status_code == 200
+        assert resp.json()["modelfile"] == "FROM llama3.1"
+
+
+def test_show_model_connect_error(tmp_path: Path, monkeypatch):
+    monkeypatch.setattr(httpx.AsyncClient, "post", _mock_connect_error)
+    with make_client(tmp_path) as client:
+        resp = client.post("/api/show", json={"model": "llama3.1:latest"}, headers=_guest_headers(client))
+        assert resp.status_code == 502
+
+
+def test_system_stats(tmp_path: Path, monkeypatch):
+    monkeypatch.setattr(routers.models, "get_gpu_stats", lambda: {"gpu_percent": 15, "vram_percent": 30, "available": True})
+    with make_client(tmp_path) as client:
+        resp = client.get("/api/stats", headers=_guest_headers(client))
+        assert resp.status_code == 200
+        data = resp.json()
+        assert "cpu_percent" in data
+        assert "memory_percent" in data
+        assert data["gpu_percent"] == 15
+        assert data["gpu_available"] is True
+
+
+def test_search_status_available(tmp_path: Path, monkeypatch):
+    monkeypatch.setattr(httpx.AsyncClient, "get", _mock_search_available)
+    with make_client(tmp_path) as client:
+        resp = client.get("/api/search/status", headers=_guest_headers(client))
+        assert resp.status_code == 200
+        assert resp.json()["available"] is True
+
+
+def test_search_status_unavailable(tmp_path: Path, monkeypatch):
+    monkeypatch.setattr(httpx.AsyncClient, "get", _mock_search_unavailable)
+    with make_client(tmp_path) as client:
+        resp = client.get("/api/search/status", headers=_guest_headers(client))
+        assert resp.status_code == 200
+        assert resp.json()["available"] is False

tests/test_node_agent.py

@@ -0,0 +1,340 @@
+"""Tests for node_agent/agent.py — standalone worker agent."""
+import asyncio
+import json
+import os
+import subprocess
+from contextlib import asynccontextmanager
+from pathlib import Path
+
+import node_agent.agent as agent
+from node_agent.agent import (
+    AgentConfig,
+    ModelInfo,
+    build_registration_payload,
+    discover_models,
+    get_load,
+    handle_ping,
+    handle_swap_model,
+)
+
+
+class FakeMsg:
+    def __init__(self, body_dict: dict):
+        self.body = json.dumps(body_dict).encode()
+
+    @asynccontextmanager
+    async def process(self):
+        yield
+
+
+class FakeExchange:
+    def __init__(self, name=""):
+        self.name = name
+        self.published = []
+
+    async def publish(self, msg, routing_key):
+        self.published.append((msg, routing_key))
+
+
+class FakeChannel:
+    def __init__(self):
+        self.exchanges = {}
+        self.is_closed = False
+
+    async def get_exchange(self, name):
+        if name not in self.exchanges:
+            self.exchanges[name] = FakeExchange(name)
+        return self.exchanges[name]
+
+    async def declare_exchange(self, name, typ, durable=True):
+        self.exchanges[name] = FakeExchange(name)
+        return self.exchanges[name]
+
+    async def declare_queue(self, name="", exclusive=True):
+        return self
+
+    async def bind(self, exchange, routing_key):
+        pass
+
+    async def close(self):
+        self.is_closed = True
+
+
+# ── 1. Registration payload shape ────────────────────────────────────
+
+
+def test_registration_payload_shape():
+    cfg = AgentConfig()
+    cfg.node_name = "worker01"
+    cfg.node_ip = "192.168.50.210"
+    cfg.capabilities = ["llm"]
+    cfg.active_model = "llama3.1-latest-Q4_K_M.gguf"
+    cfg.llama_port = 8081
+
+    inventory = [{"filename": "llama3.1-latest-Q4_K_M.gguf", "name": "llama3.1", "quant": "Q4_K_M"}]
+    payload = build_registration_payload(cfg, inventory)
+
+    assert payload["node_name"] == "worker01"
+    assert payload["node_type"] == "worker"
+    assert payload["ip"] == "192.168.50.210"
+    assert payload["capabilities"] == ["llm"]
+    assert "active_model" in payload
+    assert payload["active_model"]["filename"] == "llama3.1-latest-Q4_K_M.gguf"
+    assert payload["active_model"]["port"] == 8081
+    assert len(payload["inventory"]) == 1
+
+
+def test_registration_payload_active_model_none():
+    cfg = AgentConfig()
+    cfg.active_model = ""
+    payload = build_registration_payload(cfg, [])
+    assert payload["active_model"] is None
+
+
+# ── 2. Model discovery ───────────────────────────────────────────────
+
+
+def test_discover_models(tmp_path):
+    models_dir = tmp_path / "models"
+    models_dir.mkdir()
+    # Create valid model files
+    (models_dir / "llama3.1-latest-Q4_K_M.gguf").write_text("")
+    (models_dir / "mistral-nemo-7b-Q6_K_L.gguf").write_text("")
+    # Create file that doesn't match pattern
+    (models_dir / "readme.txt").write_text("")
+    # Create file with unrecognized naming
+    (models_dir / "my_custom_model.gguf").write_text("")
+
+    result = discover_models(str(models_dir))
+    assert len(result) == 2
+    names = {m["name"] for m in result}
+    assert "llama3.1" in names
+    assert "mistral-nemo" in names
+
+
+def test_discover_models_no_directory(tmp_path):
+    result = discover_models(str(tmp_path / "nonexistent"))
+    assert result == []
+
+
+# ── 3. Config reading ────────────────────────────────────────────────
+
+
+def test_config_from_ini(tmp_path):
+    ini = tmp_path / "caic-node-agent.conf"
+    ini.write_text(
+        "[agent]\n"
+        "node_name = testnode\n"
+        "node_ip = 10.0.0.5\n"
+        "capabilities = llm,rag\n"
+        "amqp_url = amqp://user:pass@host/vhost\n"
+        "llama_port = 9090\n"
+        "models_dir = /tmp/models\n"
+        "active_model = test.gguf\n"
+    )
+    cfg = AgentConfig.from_ini(str(ini))
+    assert cfg.node_name == "testnode"
+    assert cfg.node_ip == "10.0.0.5"
+    assert cfg.capabilities == ["llm", "rag"]
+    assert cfg.amqp_url == "amqp://user:pass@host/vhost"
+    assert cfg.llama_port == 9090
+    assert cfg.models_dir == "/tmp/models"
+    assert cfg.active_model == "test.gguf"
+
+
+def test_config_from_ini_missing_uses_defaults(tmp_path):
+    cfg = AgentConfig.from_ini(str(tmp_path / "nonexistent.conf"))
+    assert cfg.node_name != ""  # socket.gethostname() returns something
+    assert cfg.node_type == "worker"
+    assert cfg.capabilities == ["llm"]
+
+
+# ── 4. Ping handler publishes pong ──────────────────────────────────
+
+
+def test_ping_handler_publishes_pong(monkeypatch):
+    monkeypatch.setattr(agent, "HAS_PSUTIL", False)
+    monkeypatch.setattr(agent, "HAS_AIO_PIKA", True)
+
+    cfg = AgentConfig()
+    cfg.node_name = "testworker"
+
+    channel = FakeChannel()
+    admin_ex = FakeExchange("jc.admin")
+    channel.exchanges["jc.admin"] = admin_ex
+    exchange = admin_ex
+
+    async def run():
+        await handle_ping(cfg, channel, exchange, FakeMsg({
+            "from": "coordinator",
+            "node_name": "testworker",
+            "type": "ping",
+            "correlation_id": "abc-123",
+            "timestamp": "2026-01-01T00:00:00Z",
+        }))
+
+    asyncio.run(run())
+
+    assert len(exchange.published) == 1
+    msg, rk = exchange.published[0]
+    assert rk == "node.testworker.pong"
+    payload = json.loads(msg.body)
+    assert payload["type"] == "pong"
+    assert payload["correlation_id"] == "abc-123"
+    assert payload["node_name"] == "testworker"
+
+
+# ── 5. Model swap success path ──────────────────────────────────────
+
+
+def test_model_swap_success(monkeypatch):
+    monkeypatch.setattr(agent, "HAS_AIO_PIKA", True)
+    monkeypatch.setattr(agent, "HAS_HTTPX", True)
+
+    cfg = AgentConfig()
+    cfg.node_name = "testworker"
+    cfg.llama_port = 9999
+
+    captured_cmds = []
+
+    def fake_run(cmd, **kwargs):
+        captured_cmds.append(cmd)
+        return subprocess.CompletedProcess(cmd, 0, b"", b"")
+
+    monkeypatch.setattr(subprocess, "run", fake_run)
+
+    # Mock _wait_for_llama to succeed
+    async def fake_wait(*a, **kw):
+        return True
+    monkeypatch.setattr(agent, "_wait_for_llama", fake_wait)
+
+    # Mock _update_config_active_model to no-op
+    monkeypatch.setattr(agent, "_update_config_active_model", lambda c, m: None)
+
+    channel = FakeChannel()
+    admin_ex = FakeExchange("jc.admin")
+    system_ex = FakeExchange("jc.system")
+    channel.exchanges["jc.admin"] = admin_ex
+    channel.exchanges["jc.system"] = system_ex
+
+    asyncio.run(handle_swap_model(cfg, channel, (admin_ex, system_ex), FakeMsg({"model_filename": "new-model-Q4_K_M.gguf"})))
+
+    # Check systemctl calls
+    assert len(captured_cmds) == 2
+    assert captured_cmds[0] == ["systemctl", "stop", "llama-server"]
+    assert captured_cmds[1] == ["systemctl", "start", "llama-server"]
+
+    # Check model_ready published on jc.system
+    assert len(system_ex.published) == 1
+    msg, rk = system_ex.published[0]
+    assert rk == "node.testworker.model_ready"
+    payload = json.loads(msg.body)
+    assert payload["type"] == "model_ready"
+    assert payload["active_model"] == "new-model-Q4_K_M.gguf"
+
+
+# ── 6. Model swap timeout path ──────────────────────────────────────
+
+
+def test_model_swap_timeout(monkeypatch):
+    monkeypatch.setattr(agent, "HAS_AIO_PIKA", True)
+    monkeypatch.setattr(agent, "HAS_HTTPX", True)
+
+    cfg = AgentConfig()
+    cfg.node_name = "testworker"
+
+    monkeypatch.setattr(subprocess, "run", lambda cmd, **kw: subprocess.CompletedProcess(cmd, 0, b"", b""))
+
+    # Mock _wait_for_llama to fail
+    async def fake_wait_fail(*a, **kw):
+        return False
+    monkeypatch.setattr(agent, "_wait_for_llama", fake_wait_fail)
+    monkeypatch.setattr(agent, "_update_config_active_model", lambda c, m: None)
+
+    channel = FakeChannel()
+    admin_ex = FakeExchange("jc.admin")
+    system_ex = FakeExchange("jc.system")
+    channel.exchanges["jc.admin"] = admin_ex
+    channel.exchanges["jc.system"] = system_ex
+
+    asyncio.run(handle_swap_model(cfg, channel, (admin_ex, system_ex), FakeMsg({"model_filename": "broken-model-Q4_K_M.gguf"})))
+
+    assert len(system_ex.published) == 1
+    msg, rk = system_ex.published[0]
+    assert rk == "node.testworker.model_failed"
+    payload = json.loads(msg.body)
+    assert payload["type"] == "model_failed"
+    assert "error" in payload
+
+
+# ── 7. Load reporting ───────────────────────────────────────────────
+
+
+def test_get_load_with_psutil(monkeypatch):
+    class FakePsutil:
+        @staticmethod
+        def cpu_percent(interval=0.5):
+            return 42.0
+
+        @staticmethod
+        def virtual_memory():
+            class VM:
+                percent = 65.0
+            return VM()
+
+    monkeypatch.setattr(agent, "HAS_PSUTIL", True)
+    monkeypatch.setattr(agent, "psutil", FakePsutil)
+
+    # Mock subprocess to fail (no rocm-smi)
+    def fake_run(*a, **kw):
+        raise FileNotFoundError("rocm-smi not found")
+    monkeypatch.setattr(subprocess, "run", fake_run)
+
+    load = get_load()
+    assert load["cpu_pct"] == 42
+    assert load["ram_pct"] == 65
+    assert "vram_pct" not in load
+
+
+def test_get_load_without_psutil(monkeypatch):
+    monkeypatch.setattr(agent, "HAS_PSUTIL", False)
+    monkeypatch.setattr(subprocess, "run", lambda *a, **kw: (_ for _ in ()).throw(FileNotFoundError("")))
+    load = get_load()
+    assert "cpu_pct" not in load
+
+
+# ── 8. Agent idle after admission (no heartbeat) ────────────────────
+
+
+def test_no_heartbeat_timer():
+    """Agent has no background heartbeat mechanism. This test asserts that
+    the codebase contains no heartbeat-related logic in the agent itself."""
+    import inspect
+    source = inspect.getsource(agent)
+    # There should be no heartbeat timer in the agent
+    assert "heartbeat" not in source.lower(), \
+        "agent.py should contain no heartbeat logic"
+
+
+# ── 9. Config writer for model swap ─────────────────────────────────
+
+
+def test_update_config_active_model(tmp_path, monkeypatch):
+    monkeypatch.setattr(agent, "CONFIG_PATH", str(tmp_path / "caic-node-agent.conf"))
+    cfg = AgentConfig()
+    cfg.active_model = "old.gguf"
+    agent._update_config_active_model(cfg, "new.gguf")
+    assert cfg.active_model == "new.gguf"
+    content = (tmp_path / "caic-node-agent.conf").read_text()
+    assert "new.gguf" in content
+
+
+# ── 10. ModelInfo to_dict ────────────────────────────────────────────
+
+
+def test_model_info_to_dict():
+    info = ModelInfo(filename="test-Q4_K_M.gguf", name="test", version="latest", quant="Q4_K_M")
+    d = info.to_dict()
+    assert d["name"] == "test"
+    assert d["quant"] == "Q4_K_M"
+    assert d["filename"] == "test-Q4_K_M.gguf"

tests/test_presets.py

@@ -0,0 +1,128 @@
+import os
+from pathlib import Path
+
+from fastapi.testclient import TestClient
+
+import app
+import db
+from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS
+
+
+def make_client(tmp_path: Path) -> TestClient:
+    os.environ["CAIC_ADMIN_PIN"] = "1234"
+    db.DB_PATH = tmp_path / "caic-presets.db"
+    SESSIONS.clear()
+    PIN_ATTEMPTS.clear()
+    RATE_EVENTS.clear()
+    db.init_db()
+    return TestClient(app.app, raise_server_exceptions=False)
+
+
+def _admin_headers(client: TestClient) -> dict:
+    login = client.post("/api/auth/login", json={"pin": "1234"}, headers={"Origin": "http://testserver"})
+    sid = login.json()["session_id"]
+    return {"X-Session-ID": sid, "Origin": "http://testserver"}
+
+
+def _guest_headers(client: TestClient) -> dict:
+    sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()["session_id"]
+    return {"X-Session-ID": sid, "Origin": "http://testserver"}
+
+
+def test_list_presets_returns_defaults(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.get("/api/presets", headers=_guest_headers(client))
+        assert resp.status_code == 200
+        presets = resp.json()
+        assert len(presets) >= 3
+        names = [p["name"] for p in presets]
+        assert "Coding Companion" in names
+
+
+def test_create_preset(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        headers = _admin_headers(client)
+        resp = client.post("/api/presets", json={"name": "My Preset", "prompt": "You are helpful."}, headers=headers)
+        assert resp.status_code == 200
+        data = resp.json()
+        assert data["name"] == "My Preset"
+        assert data["prompt"] == "You are helpful."
+
+        presets = client.get("/api/presets", headers=_guest_headers(client)).json()
+        assert any(p["name"] == "My Preset" for p in presets)
+
+
+def test_create_preset_requires_name_and_prompt(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        headers = _admin_headers(client)
+        resp = client.post("/api/presets", json={"name": "", "prompt": ""}, headers=headers)
+        assert resp.status_code == 400
+
+        resp = client.post("/api/presets", json={"name": "Only Name"}, headers=headers)
+        assert resp.status_code == 400
+
+
+def test_update_preset(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        headers = _admin_headers(client)
+        create = client.post("/api/presets", json={"name": "Old", "prompt": "Old prompt."}, headers=headers)
+        preset_id = create.json()["id"]
+
+        update = client.put(f"/api/presets/{preset_id}", json={"name": "New", "prompt": "New prompt."}, headers=headers)
+        assert update.status_code == 200
+
+        presets = client.get("/api/presets", headers=_guest_headers(client)).json()
+        updated = next(p for p in presets if p["id"] == preset_id)
+        assert updated["name"] == "New"
+        assert updated["prompt"] == "New prompt."
+
+
+def test_update_preset_requires_fields(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        headers = _admin_headers(client)
+        resp = client.put("/api/presets/nope", json={"name": "", "prompt": ""}, headers=headers)
+        assert resp.status_code == 400
+
+
+def test_delete_preset(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        headers = _admin_headers(client)
+        create = client.post("/api/presets", json={"name": "Temp", "prompt": "Temp."}, headers=headers)
+        preset_id = create.json()["id"]
+
+        delete = client.delete(f"/api/presets/{preset_id}", headers=headers)
+        assert delete.status_code == 200
+
+        presets = client.get("/api/presets", headers=_guest_headers(client)).json()
+        assert not any(p["id"] == preset_id for p in presets)
+
+
+def test_delete_default_preset_is_noop(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        headers = _admin_headers(client)
+        presets_before = client.get("/api/presets", headers=_guest_headers(client)).json()
+        default = next(p for p in presets_before if p["is_default"])
+
+        delete = client.delete(f"/api/presets/{default['id']}", headers=headers)
+        assert delete.status_code == 200
+
+        presets_after = client.get("/api/presets", headers=_guest_headers(client)).json()
+        assert any(p["id"] == default["id"] for p in presets_after)
+
+
+def test_guest_cannot_create_preset(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.post("/api/presets", json={"name": "Hack", "prompt": "Hack"}, headers=_guest_headers(client))
+        assert resp.status_code == 403
+
+
+def test_guest_cannot_update_preset(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.put("/api/presets/some-id", json={"name": "Hack", "prompt": "Hack"}, headers=_guest_headers(client))
+        assert resp.status_code == 403
+
+
+def test_guest_cannot_delete_preset(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.delete("/api/presets/some-id", headers=_guest_headers(client))
+        assert resp.status_code == 403

tests/test_profile.py

@@ -0,0 +1,72 @@
+import os
+from pathlib import Path
+
+from fastapi.testclient import TestClient
+
+import app
+import config
+import db
+from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS
+
+
+def make_client(tmp_path: Path) -> TestClient:
+    os.environ["CAIC_ADMIN_PIN"] = "1234"
+    db.DB_PATH = tmp_path / "caic-profile.db"
+    SESSIONS.clear()
+    PIN_ATTEMPTS.clear()
+    RATE_EVENTS.clear()
+    db.init_db()
+    return TestClient(app.app, raise_server_exceptions=False)
+
+
+def _admin_headers(client: TestClient) -> dict:
+    login = client.post("/api/auth/login", json={"pin": "1234"}, headers={"Origin": "http://testserver"})
+    sid = login.json()["session_id"]
+    return {"X-Session-ID": sid, "Origin": "http://testserver"}
+
+
+def _guest_headers(client: TestClient) -> dict:
+    sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()["session_id"]
+    return {"X-Session-ID": sid, "Origin": "http://testserver"}
+
+
+def test_get_profile_returns_content(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.get("/api/profile", headers=_guest_headers(client))
+        assert resp.status_code == 200
+        data = resp.json()
+        assert "content" in data
+        assert "updated_at" in data
+        assert len(data["content"]) > 0
+
+
+def test_get_default_profile(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.get("/api/profile/default", headers=_guest_headers(client))
+        assert resp.status_code == 200
+        assert resp.json()["content"] == config.DEFAULT_PROFILE
+
+
+def test_update_profile(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        headers = _admin_headers(client)
+        resp = client.put("/api/profile", json={"content": "Custom profile text."}, headers=headers)
+        assert resp.status_code == 200
+        assert "updated_at" in resp.json()
+
+        get_resp = client.get("/api/profile", headers=_guest_headers(client))
+        assert get_resp.json()["content"] == "Custom profile text."
+
+
+def test_update_profile_too_long(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        headers = _admin_headers(client)
+        long_content = "x" * (config.MAX_PROFILE_CHARS + 1)
+        resp = client.put("/api/profile", json={"content": long_content}, headers=headers)
+        assert resp.status_code == 413
+
+
+def test_guest_cannot_update_profile(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.put("/api/profile", json={"content": "hack"}, headers=_guest_headers(client))
+        assert resp.status_code == 403

tests/test_rag_management.py

@@ -0,0 +1,409 @@
+import asyncio
+import os
+from datetime import datetime, timezone, timedelta
+from pathlib import Path
+
+import httpx
+from fastapi.testclient import TestClient
+
+import app
+import config
+import db
+import rag
+from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS
+
+
+def make_client(tmp_path: Path) -> TestClient:
+    os.environ["CAIC_ADMIN_PIN"] = "1234"
+    db.DB_PATH = tmp_path / "caic-rag-mgmt.db"
+    SESSIONS.clear()
+    PIN_ATTEMPTS.clear()
+    RATE_EVENTS.clear()
+    db.init_db()
+    return TestClient(app.app, raise_server_exceptions=False)
+
+
+def _admin_headers(client: TestClient) -> dict:
+    login = client.post("/api/auth/login", json={"pin": "1234"}, headers={"Origin": "http://testserver"})
+    sid = login.json()["session_id"]
+    return {"X-Session-ID": sid, "Origin": "http://testserver"}
+
+
+def _guest_headers(client: TestClient) -> dict:
+    sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()["session_id"]
+    return {"X-Session-ID": sid, "Origin": "http://testserver"}
+
+
+class FakeResponse:
+    def __init__(self, status, json_data=None):
+        self.status_code = status
+        self._json = json_data or {}
+
+    def json(self):
+        return self._json
+
+
+class FakeAsyncClient:
+    def __init__(self, *a, **kw):
+        pass
+
+    async def __aenter__(self):
+        return self
+
+    async def __aexit__(self, *a):
+        pass
+
+    async def get(self, url, **kw):
+        if "/collections/caic_rag" in url:
+            return FakeResponse(200, {"result": {"vectors_count": 123}})
+        return FakeResponse(200)
+
+    async def post(self, url, **kw):
+        if "/points/scroll" in url:
+            return FakeResponse(200, {"result": {"points": []}})
+        if "/points/delete" in url:
+            return FakeResponse(200)
+        return FakeResponse(200)
+
+    async def put(self, url, **kw):
+        return FakeResponse(200)
+
+
+def _old_ts(hours_ago: float = 24) -> str:
+    return (datetime.now(timezone.utc) - timedelta(hours=hours_ago)).isoformat()
+
+
+def _young_ts(hours_ago: float = 0.1) -> str:
+    return (datetime.now(timezone.utc) - timedelta(hours=hours_ago)).isoformat()
+
+
+# ---------- get_collection_count ----------
+
+def test_get_collection_count(monkeypatch):
+    monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: FakeAsyncClient())
+    count = asyncio.run(rag.get_collection_count())
+    assert count == 123
+
+
+# ---------- get_collection_stats ----------
+
+def test_get_collection_stats_shape(monkeypatch):
+    monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: FakeAsyncClient())
+    stats = asyncio.run(rag.get_collection_stats())
+    assert stats["vector_count"] == 123
+    assert stats["max_vectors"] == 50000
+    assert stats["high_water_mark"] == 40000
+    assert stats["low_water_mark"] == 10000
+    assert stats["high_water_pct"] == 80
+    assert stats["low_water_pct"] == 20
+    assert 0 < stats["percent_full"] < 1
+    assert "upload" in stats["pinned_sources"]
+    assert "profile" in stats["pinned_sources"]
+
+
+# ---------- evict_batch ----------
+
+def test_evict_batch_excludes_pinned_sources(monkeypatch):
+    """Pinned sources ('upload', 'profile') should be in the must_not scroll filter."""
+    old = _old_ts(48)
+    # Only non-pinned points are returned (real Qdrant would honour must_not filter)
+    scroll_points = [
+        {"id": "old-data", "payload": {"source": "terminal", "ingest_date": old, "retrieval_count": 0}},
+    ]
+
+    class ScrollClient(FakeAsyncClient):
+        async def post(self, url, **kw):
+            if "/points/scroll" in url:
+                must_not = kw.get("json", {}).get("filter", {}).get("must_not", [])
+                pinned_values = [m["match"]["value"] for m in must_not]
+                assert "upload" in pinned_values
+                assert "profile" in pinned_values
+                return FakeResponse(200, {"result": {"points": scroll_points}})
+            if "/points/delete" in url:
+                return FakeResponse(200)
+            return FakeResponse(200)
+
+    monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: ScrollClient())
+    deleted = asyncio.run(rag.evict_batch(10))
+    assert deleted == 1
+
+
+def test_evict_batch_respects_grace_period(monkeypatch):
+    """Vectors younger than RAG_GRACE_HOURS should be skipped."""
+    old = _old_ts(48)
+    young = _young_ts(0.1)
+    scroll_points = [
+        {"id": "mature", "payload": {"source": "terminal", "ingest_date": old, "retrieval_count": 0}},
+        {"id": "newborn", "payload": {"source": "terminal", "ingest_date": young, "retrieval_count": 0}},
+    ]
+
+    class GraceClient(FakeAsyncClient):
+        async def post(self, url, **kw):
+            if "/points/scroll" in url:
+                return FakeResponse(200, {"result": {"points": scroll_points}})
+            if "/points/delete" in url:
+                deleted = kw.get("json", {}).get("points", [])
+                assert "newborn" not in deleted
+                return FakeResponse(200)
+            return FakeResponse(200)
+
+    monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: GraceClient())
+    deleted = asyncio.run(rag.evict_batch(10))
+    assert deleted == 1
+
+
+def test_evict_batch_respects_batch_size(monkeypatch):
+    """Only up to batch_size vectors should be deleted per call."""
+    old = _old_ts(48)
+    points = [{"id": f"p{i}", "payload": {"source": "terminal", "ingest_date": old, "retrieval_count": 0}} for i in range(50)]
+
+    class BatchClient(FakeAsyncClient):
+        async def post(self, url, **kw):
+            if "/points/delete" in url:
+                deleted = kw.get("json", {}).get("points", [])
+                assert len(deleted) == 10
+                return FakeResponse(200)
+            if "/points/scroll" in url:
+                return FakeResponse(200, {"result": {"points": points}})
+            return FakeResponse(200)
+
+    monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: BatchClient())
+    deleted = asyncio.run(rag.evict_batch(10))
+    assert deleted == 10
+
+
+def test_evict_batch_all_pinned_returns_zero(monkeypatch):
+    """If scroll returns nothing (all points filtered by must_not), evict_batch returns 0."""
+    class EmptyClient(FakeAsyncClient):
+        async def post(self, url, **kw):
+            if "/points/scroll" in url:
+                return FakeResponse(200, {"result": {"points": []}})
+            return FakeResponse(200)
+
+    monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: EmptyClient())
+    deleted = asyncio.run(rag.evict_batch(10))
+    assert deleted == 0
+
+
+def test_evict_batch_scores_lowest_first(monkeypatch):
+    """Vectors with lower scores should be evicted first."""
+    old = _old_ts(48)
+    points = [
+        {"id": "high-score", "payload": {"source": "terminal", "ingest_date": old, "retrieval_count": 100}},
+        {"id": "low-score", "payload": {"source": "terminal", "ingest_date": old, "retrieval_count": 0}},
+        {"id": "mid-score", "payload": {"source": "terminal", "ingest_date": old, "retrieval_count": 50}},
+    ]
+
+    class ScoreClient(FakeAsyncClient):
+        async def post(self, url, **kw):
+            if "/points/delete" in url:
+                deleted = kw.get("json", {}).get("points", [])
+                assert "low-score" in deleted
+                assert "high-score" not in deleted
+                assert "mid-score" not in deleted
+                return FakeResponse(200)
+            if "/points/scroll" in url:
+                return FakeResponse(200, {"result": {"points": points}})
+            return FakeResponse(200)
+
+    monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: ScoreClient())
+    deleted = asyncio.run(rag.evict_batch(1))
+    assert deleted == 1
+
+
+# ---------- maybe_evict ----------
+
+def test_maybe_evict_below_high_water(monkeypatch):
+    """When count is below high-water mark, eviction should not fire."""
+    class LowCountClient(FakeAsyncClient):
+        async def get(self, url, **kw):
+            # 30000 < 40000 high water
+            return FakeResponse(200, {"result": {"vectors_count": 30000}})
+
+    monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: LowCountClient())
+    rag.EVICTION_LOG.clear()
+    evicted = asyncio.run(rag.maybe_evict())
+    assert evicted == 0
+    assert len(rag.EVICTION_LOG) == 0
+
+
+def test_maybe_evict_at_high_water(monkeypatch):
+    """When count reaches high-water mark, eviction should fire."""
+    class HighCountClient(FakeAsyncClient):
+        def __init__(self, *a, **kw):
+            super().__init__()
+            self.call_count = 0
+
+        async def get(self, url, **kw):
+            return FakeResponse(200, {"result": {"vectors_count": 45000}})
+
+        async def post(self, url, **kw):
+            if "/points/scroll" in url:
+                old = _old_ts(48)
+                points = [{"id": f"evict-me-{i}", "payload": {"source": "terminal", "ingest_date": old, "retrieval_count": 0}} for i in range(100)]
+                return FakeResponse(200, {"result": {"points": points}})
+            if "/points/delete" in url:
+                return FakeResponse(200)
+            return FakeResponse(200)
+
+    monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: HighCountClient())
+    rag.EVICTION_LOG.clear()
+    evicted = asyncio.run(rag.maybe_evict())
+    assert evicted > 0
+    assert len(rag.EVICTION_LOG) == 1
+    entry = rag.EVICTION_LOG[0]
+    assert "timestamp" in entry
+    assert entry["count"] > 0
+
+
+def test_maybe_evict_zero_config_disabled(monkeypatch):
+    """RAG_MAX_VECTORS <= 0 should disable eviction."""
+    orig = config.RAG_MAX_VECTORS
+    try:
+        config.RAG_MAX_VECTORS = 0
+        evicted = asyncio.run(rag.maybe_evict())
+        assert evicted == 0
+    finally:
+        config.RAG_MAX_VECTORS = orig
+
+
+def test_maybe_evict_all_pinned_breaks(monkeypatch):
+    """Above high water but only pinned points exist → eviction breaks with 0 deleted."""
+    class AllPinnedClient(FakeAsyncClient):
+        async def get(self, url, **kw):
+            return FakeResponse(200, {"result": {"vectors_count": 45000}})
+        async def post(self, url, **kw):
+            if "/points/scroll" in url:
+                return FakeResponse(200, {"result": {"points": []}})
+            return FakeResponse(200)
+
+    monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: AllPinnedClient())
+    rag.EVICTION_LOG.clear()
+    evicted = asyncio.run(rag.maybe_evict())
+    assert evicted == 0
+    assert len(rag.EVICTION_LOG) == 0
+
+
+# ---------- get_rag_operational_stats ----------
+
+def test_rag_operational_stats_shape(monkeypatch):
+    monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: FakeAsyncClient())
+    rag.EVICTION_LOG.clear()
+    rag.EVICTION_LOG.append({
+        "timestamp": datetime.now(timezone.utc).isoformat(),
+        "count": 500,
+        "remaining": 40000,
+    })
+    stats = asyncio.run(rag.get_rag_operational_stats())
+    assert stats["vector_count"] == 123
+    assert stats["grace_hours"] == 1
+    assert "eviction_counts_last_1m" in stats
+    assert "eviction_counts_last_5m" in stats
+    assert "eviction_counts_last_30m" in stats
+    assert stats["eviction_counts_last_1m"] == 500
+
+
+# ---------- GET /api/rag/stats ----------
+
+def test_rag_stats_endpoint(tmp_path, monkeypatch):
+    monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: FakeAsyncClient())
+    with make_client(tmp_path) as client:
+        resp = client.get("/api/rag/stats", headers=_admin_headers(client))
+        assert resp.status_code == 200
+        data = resp.json()
+        assert data["vector_count"] == 123
+        assert data["max_vectors"] == 50000
+        assert "high_water_mark" in data
+        assert "low_water_mark" in data
+        assert data["high_water_pct"] == 80
+        assert data["low_water_pct"] == 20
+        assert "percent_full" in data
+        assert data["pinned_sources"] == ["upload", "profile"]
+        assert data["grace_hours"] == 1
+        assert "eviction_counts_last_1m" in data
+        assert "eviction_counts_last_5m" in data
+        assert "eviction_counts_last_30m" in data
+        assert "pinned_count" in data
+        assert "avg_retrieval_count" in data
+        assert "at_risk_count" in data
+        assert "eviction_log_size" in data
+
+
+def test_rag_stats_requires_admin(tmp_path, monkeypatch):
+    monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: FakeAsyncClient())
+    with make_client(tmp_path) as client:
+        resp = client.get("/api/rag/stats", headers=_guest_headers(client))
+        assert resp.status_code == 403
+
+
+# ---------- POST /api/rag/flush ----------
+
+def test_rag_flush_endpoint(tmp_path, monkeypatch):
+    class FlushClient(FakeAsyncClient):
+        async def post(self, url, **kw):
+            if "/points/scroll" in url:
+                return FakeResponse(200, {"result": {"points": [{"id": "a"}, {"id": "b"}]}})
+            if "/points/delete" in url:
+                return FakeResponse(200)
+            return FakeResponse(200)
+
+    monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: FlushClient())
+    with make_client(tmp_path) as client:
+        resp = client.post("/api/rag/flush", headers=_admin_headers(client))
+        assert resp.status_code == 200
+        data = resp.json()
+        assert data["status"] == "flushed"
+        assert data["deleted_count"] == 2
+        assert data["collection"] == rag.RAG_COLLECTION
+
+
+def test_rag_flush_requires_admin(tmp_path, monkeypatch):
+    monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: FakeAsyncClient())
+    with make_client(tmp_path) as client:
+        resp = client.post("/api/rag/flush", headers=_guest_headers(client))
+        assert resp.status_code == 403
+
+
+def test_rag_flush_empty_collection(tmp_path, monkeypatch):
+    class EmptyClient(FakeAsyncClient):
+        async def post(self, url, **kw):
+            if "/points/scroll" in url:
+                return FakeResponse(200, {"result": {"points": []}})
+            return FakeResponse(200)
+
+    monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: EmptyClient())
+    with make_client(tmp_path) as client:
+        resp = client.post("/api/rag/flush", headers=_admin_headers(client))
+        assert resp.status_code == 200
+        data = resp.json()
+        assert data["deleted_count"] == 0
+        assert data["status"] == "flushed"
+
+
+# ---------- Race lock ----------
+
+def test_eviction_lock_prevents_concurrent_eviction(monkeypatch):
+    """Concurrent calls to maybe_evict should queue; only one evicts."""
+    call_order = []
+
+    async def slow_get_collection_count():
+        call_order.append("count")
+        return 45000
+
+    async def slow_evict_batch(bs):
+        call_order.append("evict")
+        await asyncio.sleep(0.05)
+        return 500
+
+    monkeypatch.setattr(rag, "get_collection_count", slow_get_collection_count)
+    monkeypatch.setattr(rag, "evict_batch", slow_evict_batch)
+    rag.EVICTION_LOG.clear()
+
+    async def run_concurrent():
+        r1, r2 = await asyncio.gather(rag.maybe_evict(), rag.maybe_evict())
+        return r1, r2
+
+    r1, r2 = asyncio.run(run_concurrent())
+    # First call evicted, second found count already below high water or lock serialized
+    assert r1 >= 0
+    assert r2 >= 0

tests/test_rate_and_payload_guardrails.py

@@ -4,28 +4,32 @@ from pathlib import Path
 
 from fastapi.testclient import TestClient
 
-import app as app_module
+import app
+import config
+import db
+import security
+from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS
 
 
 def make_client(tmp_path: Path) -> TestClient:
-    os.environ["JARVISCHAT_ADMIN_PIN"] = "1234"
+    os.environ["CAIC_ADMIN_PIN"] = "1234"
-    app_module.DB_PATH = tmp_path / "jarvischat-rate.db"
+    db.DB_PATH = tmp_path / "caic-rate.db"
-    app_module.SESSIONS.clear()
+    SESSIONS.clear()
-    app_module.PIN_ATTEMPTS.clear()
+    PIN_ATTEMPTS.clear()
-    app_module.RATE_EVENTS.clear()
+    RATE_EVENTS.clear()
-    app_module.init_db()
+    db.init_db()
-    return TestClient(app_module.app)
+    return TestClient(app.app)
 
 
 def test_stats_rate_limit_hits_429(tmp_path: Path):
-    old_limit = app_module.RL_STATS_PER_WINDOW
+    old_limit = security.RL_STATS_PER_WINDOW
-    old_window = app_module.RATE_WINDOW_SECONDS
+    old_window = app.RATE_WINDOW_SECONDS
-    app_module.RL_STATS_PER_WINDOW = 2
+    security.RL_STATS_PER_WINDOW = 2
-    app_module.RATE_WINDOW_SECONDS = 60
+    app.RATE_WINDOW_SECONDS = 60
     try:
         with make_client(tmp_path) as client:
-            sid = client.post("/api/auth/guest").json()["session_id"]
+            sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()["session_id"]
-            headers = {"X-Session-ID": sid}
+            headers = {"X-Session-ID": sid, "Origin": "http://testserver"}
 
             r1 = client.get("/api/stats", headers=headers)
             r2 = client.get("/api/stats", headers=headers)
@@ -35,13 +39,13 @@ def test_stats_rate_limit_hits_429(tmp_path: Path):
             assert r2.status_code == 200
             assert r3.status_code == 429
     finally:
-        app_module.RL_STATS_PER_WINDOW = old_limit
+        security.RL_STATS_PER_WINDOW = old_limit
-        app_module.RATE_WINDOW_SECONDS = old_window
+        app.RATE_WINDOW_SECONDS = old_window
 
 
 def test_large_login_payload_rejected_413(tmp_path: Path):
     with make_client(tmp_path) as client:
-        huge_pin = "1" * (app_module.BODY_LIMIT_DEFAULT_BYTES + 100)
+        huge_pin = "1" * (config.BODY_LIMIT_DEFAULT_BYTES + 100)
         resp = client.post(
             "/api/auth/login",
             data=json.dumps({"pin": huge_pin}),
@@ -52,12 +56,12 @@ def test_large_login_payload_rejected_413(tmp_path: Path):
 
 def test_chat_message_length_rejected_413(tmp_path: Path):
     with make_client(tmp_path) as client:
-        sid = client.post("/api/auth/guest").json()["session_id"]
+        sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()["session_id"]
         headers = {"X-Session-ID": sid, "Origin": "http://testserver"}
-        message = "x" * (app_module.MAX_CHAT_MESSAGE_CHARS + 1)
+        message = "x" * (config.MAX_CHAT_MESSAGE_CHARS + 1)
         resp = client.post(
             "/api/chat",
-            json={"message": message, "model": app_module.DEFAULT_MODEL},
+            json={"message": message, "model": config.DEFAULT_MODEL},
             headers=headers,
         )
         assert resp.status_code == 413
@@ -65,12 +69,12 @@ def test_chat_message_length_rejected_413(tmp_path: Path):
 
 def test_search_query_length_rejected_413(tmp_path: Path):
     with make_client(tmp_path) as client:
-        sid = client.post("/api/auth/guest").json()["session_id"]
+        sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()["session_id"]
         headers = {"X-Session-ID": sid, "Origin": "http://testserver"}
-        query = "q" * (app_module.MAX_SEARCH_QUERY_CHARS + 1)
+        query = "q" * (config.MAX_SEARCH_QUERY_CHARS + 1)
         resp = client.post(
             "/api/search",
-            json={"query": query, "model": app_module.DEFAULT_MODEL},
+            json={"query": query, "model": config.DEFAULT_MODEL},
             headers=headers,
         )
         assert resp.status_code == 413

tests/test_search_route.py

@@ -0,0 +1,186 @@
+import json
+import os
+from pathlib import Path
+
+import httpx
+from fastapi.testclient import TestClient
+
+import app
+import config
+import db
+import routers.search_route
+from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS
+
+
+def make_client(tmp_path: Path) -> TestClient:
+    os.environ["CAIC_ADMIN_PIN"] = "1234"
+    db.DB_PATH = tmp_path / "caic-search-route.db"
+    SESSIONS.clear()
+    PIN_ATTEMPTS.clear()
+    RATE_EVENTS.clear()
+    db.init_db()
+    return TestClient(app.app, raise_server_exceptions=False)
+
+
+def _guest_headers(client: TestClient) -> dict:
+    sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()["session_id"]
+    return {"X-Session-ID": sid, "Origin": "http://testserver"}
+
+
+def parse_sse_payloads(body: str) -> list[dict]:
+    payloads: list[dict] = []
+    for chunk in body.split("\n\n"):
+        chunk = chunk.strip()
+        if not chunk.startswith("data: "):
+            continue
+        raw = chunk[len("data: ") :]
+        payloads.append(json.loads(raw))
+    return payloads
+
+
+class _MockStreamResponse:
+    def __init__(self, lines: list[str]):
+        self._lines = lines
+
+    async def __aenter__(self):
+        return self
+
+    async def __aexit__(self, exc_type, exc, tb):
+        return False
+
+    async def aiter_lines(self):
+        for line in self._lines:
+            yield line
+
+
+def _stream_json_lines(events: list[dict]) -> list[str]:
+    return [json.dumps(event) for event in events]
+
+
+def test_explicit_search_with_results(tmp_path: Path, monkeypatch):
+    with make_client(tmp_path) as client:
+        headers = _guest_headers(client)
+
+        async def search_stub(query: str, max_results: int = 5):
+            return [
+                {"title": "Result One", "url": "https://example.com/1", "content": "First result content."},
+                {"title": "Result Two", "url": "https://example.com/2", "content": "Second result content."},
+            ]
+
+        monkeypatch.setattr(routers.search_route, "query_searxng", search_stub)
+
+        events = _stream_json_lines([
+            {"choices": [{"delta": {"content": "Here's what I found"}, "logprobs": None}]},
+            {"choices": [{"delta": {"content": " about your query."}, "logprobs": None}]},
+            {"choices": [{"delta": {}, "finish_reason": "stop"}], "usage": {}},
+        ])
+
+        def stream_stub(self, method, url, json=None, timeout=None):
+            return _MockStreamResponse(events)
+
+        monkeypatch.setattr(httpx.AsyncClient, "stream", stream_stub)
+
+        resp = client.post(
+            "/api/search",
+            json={"query": "current events", "model": config.DEFAULT_MODEL},
+            headers=headers,
+        )
+        assert resp.status_code == 200
+        payloads = parse_sse_payloads(resp.text)
+
+        assert any(p.get("searching") is True for p in payloads)
+        assert any("search_results" in p for p in payloads)
+        token_text = "".join(p.get("token", "") for p in payloads if "token" in p)
+        assert "found" in token_text.lower()
+        assert any(p.get("done") and p.get("searched") for p in payloads)
+
+
+def test_explicit_search_no_results(tmp_path: Path, monkeypatch):
+    with make_client(tmp_path) as client:
+        headers = _guest_headers(client)
+
+        async def empty_search(query: str, max_results: int = 5):
+            return []
+
+        monkeypatch.setattr(routers.search_route, "query_searxng", empty_search)
+
+        resp = client.post(
+            "/api/search",
+            json={"query": "nothingness", "model": config.DEFAULT_MODEL},
+            headers=headers,
+        )
+        assert resp.status_code == 200
+        payloads = parse_sse_payloads(resp.text)
+
+        assert any("No search results found" in p.get("token", "") for p in payloads)
+        assert any(p.get("done") for p in payloads)
+        assert not any("search_results" in p for p in payloads)
+
+
+def test_explicit_search_new_conversation_created(tmp_path: Path, monkeypatch):
+    with make_client(tmp_path) as client:
+        headers = _guest_headers(client)
+
+        async def search_stub(query: str, max_results: int = 5):
+            return [{"title": "T", "url": "https://ex.com", "content": "Content."}]
+
+        monkeypatch.setattr(routers.search_route, "query_searxng", search_stub)
+
+        events = _stream_json_lines([
+            {"choices": [{"delta": {"content": "Answer."}, "logprobs": None}]},
+            {"choices": [{"delta": {}, "finish_reason": "stop"}], "usage": {}},
+        ])
+
+        def stream_stub(self, method, url, json=None, timeout=None):
+            return _MockStreamResponse(events)
+
+        monkeypatch.setattr(httpx.AsyncClient, "stream", stream_stub)
+
+        resp = client.post(
+            "/api/search",
+            json={"query": "tell me something", "model": config.DEFAULT_MODEL},
+            headers=headers,
+        )
+        assert resp.status_code == 200
+        payloads = parse_sse_payloads(resp.text)
+
+        conv_id = None
+        for p in payloads:
+            if "conversation_id" in p:
+                conv_id = p["conversation_id"]
+                break
+        assert conv_id is not None
+
+        conv_resp = client.get(f"/api/conversations/{conv_id}", headers=_guest_headers(client))
+        assert conv_resp.status_code == 200
+        data = conv_resp.json()
+        assert len(data["messages"]) >= 2
+
+
+def test_explicit_search_stream_error(tmp_path: Path, monkeypatch):
+    with make_client(tmp_path) as client:
+        headers = _guest_headers(client)
+
+        async def search_stub(query: str, max_results: int = 5):
+            return [{"title": "T", "url": "https://ex.com", "content": "Content."}]
+
+        monkeypatch.setattr(routers.search_route, "query_searxng", search_stub)
+
+        def broken_stream(self, method, url, json=None, timeout=None):
+            class BrokenCtx:
+                async def __aenter__(self):
+                    raise RuntimeError("summarization failed")
+                async def __aexit__(self, exc_type, exc, tb):
+                    return False
+            return BrokenCtx()
+
+        monkeypatch.setattr(httpx.AsyncClient, "stream", broken_stream)
+
+        resp = client.post(
+            "/api/search",
+            json={"query": "breaking news", "model": config.DEFAULT_MODEL},
+            headers=headers,
+        )
+        assert resp.status_code == 200
+        assert "error_key" in resp.text
+        assert "INC-" in resp.text

tests/test_search_url_sanitization.py

@@ -1,17 +1,17 @@
-import app as app_module
+from search import sanitize_outbound_url
 
 
 def test_sanitize_outbound_url_allows_http_https():
-    assert app_module.sanitize_outbound_url("https://example.com/path") == "https://example.com/path"
+    assert sanitize_outbound_url("https://example.com/path") == "https://example.com/path"
-    assert app_module.sanitize_outbound_url("http://example.com") == "http://example.com"
+    assert sanitize_outbound_url("http://example.com") == "http://example.com"
 
 
 def test_sanitize_outbound_url_blocks_unsafe_schemes():
-    assert app_module.sanitize_outbound_url("javascript:alert(1)") == ""
+    assert sanitize_outbound_url("javascript:alert(1)") == ""
-    assert app_module.sanitize_outbound_url("data:text/html,evil") == ""
+    assert sanitize_outbound_url("data:text/html,evil") == ""
-    assert app_module.sanitize_outbound_url("file:///etc/passwd") == ""
+    assert sanitize_outbound_url("file:///etc/passwd") == ""
 
 
 def test_sanitize_outbound_url_blocks_relative_and_empty():
-    assert app_module.sanitize_outbound_url("/relative/path") == ""
+    assert sanitize_outbound_url("/relative/path") == ""
-    assert app_module.sanitize_outbound_url("") == ""
+    assert sanitize_outbound_url("") == ""

tests/test_settings_allowlist.py

@@ -3,17 +3,19 @@ from pathlib import Path
 
 from fastapi.testclient import TestClient
 
-import app as app_module
+import app
+import db
+from security import SESSIONS, PIN_ATTEMPTS
 
 
 def make_admin_client(tmp_path: Path) -> tuple[TestClient, dict[str, str]]:
-    os.environ["JARVISCHAT_ADMIN_PIN"] = "1234"
+    os.environ["CAIC_ADMIN_PIN"] = "1234"
-    app_module.DB_PATH = tmp_path / "jarvischat-settings.db"
+    db.DB_PATH = tmp_path / "caic-settings.db"
-    app_module.SESSIONS.clear()
+    SESSIONS.clear()
-    app_module.PIN_ATTEMPTS.clear()
+    PIN_ATTEMPTS.clear()
-    app_module.init_db()
+    db.init_db()
 
-    client = TestClient(app_module.app)
+    client = TestClient(app.app)
     login = client.post(
         "/api/auth/login",
         json={"pin": "1234"},

tests/test_skills_framework.py

@@ -1,19 +1,23 @@
+import asyncio
 import os
 from pathlib import Path
 
 from fastapi.testclient import TestClient
 
-import app as app_module
+import app
+import db
+from rag import build_system_prompt
+from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS
 
 
 def make_client(tmp_path: Path) -> TestClient:
-    os.environ["JARVISCHAT_ADMIN_PIN"] = "1234"
+    os.environ["CAIC_ADMIN_PIN"] = "1234"
-    app_module.DB_PATH = tmp_path / "jarvischat-skills.db"
+    db.DB_PATH = tmp_path / "caic-skills.db"
-    app_module.SESSIONS.clear()
+    SESSIONS.clear()
-    app_module.PIN_ATTEMPTS.clear()
+    PIN_ATTEMPTS.clear()
-    app_module.RATE_EVENTS.clear()
+    RATE_EVENTS.clear()
-    app_module.init_db()
+    db.init_db()
-    return TestClient(app_module.app, raise_server_exceptions=False)
+    return TestClient(app.app, raise_server_exceptions=False)
 
 
 def test_guest_can_list_skills(tmp_path: Path):
@@ -21,7 +25,7 @@ def test_guest_can_list_skills(tmp_path: Path):
         sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()[
             "session_id"
         ]
-        resp = client.get("/api/skills", headers={"X-Session-ID": sid})
+        resp = client.get("/api/skills", headers={"X-Session-ID": sid, "Origin": "http://testserver"})
         assert resp.status_code == 200
         payload = resp.json()
         assert payload["count"] >= 1
@@ -46,7 +50,7 @@ def test_admin_can_toggle_skill_enabled_state(tmp_path: Path):
         assert disable.status_code == 200
         assert disable.json()["skill"]["enabled"] is False
 
-        active = client.get("/api/skills/active", headers={"X-Session-ID": sid})
+        active = client.get("/api/skills/active", headers={"X-Session-ID": sid, "Origin": "http://testserver"})
         assert active.status_code == 200
         assert all(skill["key"] != "search.web" for skill in active.json()["skills"])
 
@@ -71,23 +75,23 @@ def test_unknown_skill_update_is_rejected(tmp_path: Path):
 
 def test_prompt_injection_respects_skills_enabled_setting(tmp_path: Path):
     with make_client(tmp_path):
-        db = app_module.get_db()
+        conn = db.get_db()
         try:
-            db.execute(
+            conn.execute(
                 "INSERT OR REPLACE INTO settings (key, value) VALUES (?, ?)",
                 ("skills_enabled", "false"),
             )
-            db.commit()
+            conn.commit()
-            without_skills = app_module.build_system_prompt(db, "", "hello")
+            without_skills = asyncio.run(build_system_prompt(conn, "", "hello"))
             assert "## Active Skills" not in without_skills
 
-            db.execute(
+            conn.execute(
                 "INSERT OR REPLACE INTO settings (key, value) VALUES (?, ?)",
                 ("skills_enabled", "true"),
             )
-            db.commit()
+            conn.commit()
-            with_skills = app_module.build_system_prompt(db, "", "hello")
+            with_skills = asyncio.run(build_system_prompt(conn, "", "hello"))
             assert "## Active Skills" in with_skills
             assert "memory.search" in with_skills
         finally:
-            db.close()
+            conn.close()

tests/test_triage.py

@@ -0,0 +1,141 @@
+"""Tests for triage.py — query classification and node selection."""
+import asyncio
+import json
+
+import httpx
+
+import cluster
+import config
+import triage
+
+
+def _reset():
+    cluster.CLUSTER_NODES.clear()
+    cluster.CLUSTER_COORDINATOR = None
+
+
+_published = []
+
+
+async def _fake_publish(exchange, routing_key, payload):
+    _published.append((exchange, routing_key, payload))
+
+
+class _MockPostResponse:
+    def __init__(self, json_data: dict, status_code: int = 200):
+        self._json_data = json_data
+        self.status_code = status_code
+
+    def json(self):
+        return self._json_data
+
+    async def __aenter__(self):
+        return self
+
+    async def __aexit__(self, exc_type, exc, tb):
+        return False
+
+
+class _MockPostContext:
+    def __init__(self, response: _MockPostResponse):
+        self._response = response
+
+    async def __aenter__(self):
+        return self._response
+
+    async def __aexit__(self, exc_type, exc, tb):
+        return False
+
+
+# ---------- 1. classify_query returns valid classification ----------
+
+
+def test_classify_returns_valid(monkeypatch):
+    async def post_stub(self, url, json=None, timeout=None):
+        return _MockPostResponse({
+            "choices": [{"message": {"content": "code"}}]
+        })
+
+    monkeypatch.setattr(httpx.AsyncClient, "post", post_stub)
+
+    result = __import__("asyncio").run(triage.classify_query("write a python function"))
+    assert result == "code"
+
+
+# ---------- 2. classify_query on error returns "general" ----------
+
+
+def test_classify_error_returns_general(monkeypatch):
+    async def post_stub(self, url, json=None, timeout=None):
+        raise httpx.ConnectError("connection refused")
+
+    monkeypatch.setattr(httpx.AsyncClient, "post", post_stub)
+
+    result = __import__("asyncio").run(triage.classify_query("any question"))
+    assert result == "general"
+
+
+# ---------- 3. select_node("code") returns coder node ----------
+
+
+def test_select_node_code_returns_coder():
+    _reset()
+    cluster.CLUSTER_NODES["coder01"] = {
+        "name": "coder01", "type": "worker", "status": "active",
+        "ip": "192.168.50.210",
+        "active_model": {"name": "qwen2.5-coder-14b", "port": 8082},
+    }
+    cluster.CLUSTER_NODES["general01"] = {
+        "name": "general01", "type": "worker", "status": "active",
+        "ip": "192.168.50.211",
+        "active_model": {"name": "llama3.1", "port": 8081},
+    }
+
+    node = asyncio.run(triage.select_node("code"))
+    assert node is not None
+    assert node["name"] == "coder01"
+
+
+# ---------- 4. select_node("general") with no matching node returns None ----------
+
+
+def test_select_node_general_no_match_returns_none():
+    _reset()
+    cluster.CLUSTER_NODES["coder01"] = {
+        "name": "coder01", "type": "worker", "status": "active",
+        "active_model": {"name": "qwen2.5-coder-14b", "port": 8082},
+    }
+    node = asyncio.run(triage.select_node("general"))
+    assert node is None
+
+
+# ---------- 5. get_inference_url with coder node ----------
+
+
+def test_get_inference_url_with_coder_node(monkeypatch):
+    _reset()
+    async def fake_classify(query: str) -> str:
+        return "code"
+    monkeypatch.setattr(triage, "classify_query", fake_classify)
+
+    cluster.CLUSTER_NODES["coder01"] = {
+        "name": "coder01", "type": "worker", "status": "active",
+        "ip": "192.168.50.210",
+        "active_model": {"name": "qwen2.5-coder-14b", "port": 8082},
+    }
+
+    url = __import__("asyncio").run(triage.get_inference_url("write a loop in rust"))
+    assert url == "http://192.168.50.210:8082/v1"
+
+
+# ---------- 6. get_inference_url with no nodes returns LLAMA_SERVER_BASE ----------
+
+
+def test_get_inference_url_no_nodes(monkeypatch):
+    _reset()
+    async def fake_classify(query: str) -> str:
+        return "code"
+    monkeypatch.setattr(triage, "classify_query", fake_classify)
+
+    url = __import__("asyncio").run(triage.get_inference_url("any question"))
+    assert url == config.LLAMA_SERVER_BASE

tests/test_upload.py

@@ -0,0 +1,274 @@
+import os
+from pathlib import Path
+
+import httpx
+import pytest
+from fastapi.testclient import TestClient
+
+import app
+import db
+import routers.upload as upload_route
+from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS
+
+
+def make_client(tmp_path: Path) -> TestClient:
+    os.environ["CAIC_ADMIN_PIN"] = "1234"
+    db.DB_PATH = tmp_path / "caic-upload.db"
+    SESSIONS.clear()
+    PIN_ATTEMPTS.clear()
+    RATE_EVENTS.clear()
+    db.init_db()
+    return TestClient(app.app, raise_server_exceptions=False)
+
+
+def _admin_headers(client: TestClient) -> dict:
+    login = client.post("/api/auth/login", json={"pin": "1234"}, headers={"Origin": "http://testserver"})
+    sid = login.json()["session_id"]
+    return {"X-Session-ID": sid, "Origin": "http://testserver"}
+
+
+def _guest_headers(client: TestClient) -> dict:
+    sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()["session_id"]
+    return {"X-Session-ID": sid, "Origin": "http://testserver"}
+
+
+def test_upload_requires_admin(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.post("/api/upload", headers=_guest_headers(client), files={"file": ("test.txt", b"hello")})
+        assert resp.status_code == 403
+
+
+def test_upload_unsupported_mime(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.post(
+            "/api/upload", headers=_admin_headers(client),
+            files={"file": ("test.exe", b"fake", "application/x-msdownload")},
+        )
+        assert resp.status_code == 415
+
+
+def test_upload_context_mode(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.post(
+            "/api/upload", headers=_admin_headers(client),
+            data={"mode": "context", "conversation_id": "conv-1"},
+            files={"file": ("notes.txt", b"Hello world notes")},
+        )
+        assert resp.status_code == 200
+        data = resp.json()
+        assert data["filename"] == "notes.txt"
+        assert data["mode"] == "context"
+        assert "context_id" in data
+        assert "chunks_ingested" not in data
+
+        row = db.get_db().execute("SELECT content FROM upload_context WHERE id = ?", (data["context_id"],)).fetchone()
+        assert row["content"] == "Hello world notes"
+
+
+def test_upload_ingest_mode(tmp_path: Path, monkeypatch):
+    embed_count = 0
+
+    class FakeAsyncClient:
+        class FakeResponse:
+            def __init__(self, status, json_data=None):
+                self.status_code = status
+                self._json = json_data or {}
+
+            def json(self):
+                return self._json
+
+        def __init__(self, *a, **kw):
+            pass
+
+        async def __aenter__(self):
+            return self
+
+        async def __aexit__(self, *a):
+            pass
+
+        async def post(self, url, **kw):
+            nonlocal embed_count
+            if "/api/embeddings" in url:
+                embed_count += 1
+                return self.FakeResponse(200, {"embedding": [0.1] * 768})
+            return self.FakeResponse(200)
+
+        async def put(self, url, **kw):
+            return self.FakeResponse(200)
+
+    monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: FakeAsyncClient())
+
+    with make_client(tmp_path) as client:
+        resp = client.post(
+            "/api/upload", headers=_admin_headers(client),
+            data={"mode": "ingest"},
+            files={"file": ("data.txt", b"word " * 1000)},
+        )
+        assert resp.status_code == 200
+        data = resp.json()
+        assert data["mode"] == "ingest"
+        assert data["chunks_ingested"] > 0
+        assert "context_id" not in data
+        assert embed_count == data["chunks_ingested"]
+
+
+def test_upload_both_mode(tmp_path: Path, monkeypatch):
+    class FakeAsyncClient:
+        class FakeResponse:
+            def __init__(self, status, json_data=None):
+                self.status_code = status
+                self._json = json_data or {}
+
+            def json(self):
+                return self._json
+
+        def __init__(self, *a, **kw):
+            pass
+
+        async def __aenter__(self):
+            return self
+
+        async def __aexit__(self, *a):
+            pass
+
+        async def post(self, url, **kw):
+            if "/api/embeddings" in url:
+                return self.FakeResponse(200, {"embedding": [0.1] * 768})
+            return self.FakeResponse(200)
+
+        async def put(self, url, **kw):
+            return self.FakeResponse(200)
+
+    monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: FakeAsyncClient())
+
+    with make_client(tmp_path) as client:
+        resp = client.post(
+            "/api/upload", headers=_admin_headers(client),
+            data={"mode": "both", "conversation_id": "conv-2"},
+            files={"file": ("both.txt", b"test " * 500)},
+        )
+        assert resp.status_code == 200
+        data = resp.json()
+        assert data["mode"] == "both"
+        assert "context_id" in data
+        assert data["chunks_ingested"] > 0
+
+
+def test_upload_image_type(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.post(
+            "/api/upload", headers=_admin_headers(client),
+            data={"mode": "context"},
+            files={"file": ("photo.png", b"fake-png", "image/png")},
+        )
+        assert resp.status_code == 200
+        data = resp.json()
+        assert data["filename"] == "photo.png"
+        assert "context_id" in data
+
+
+def test_get_upload_by_conversation(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        headers = _admin_headers(client)
+        resp1 = client.post("/api/upload", headers=headers,
+                            data={"mode": "context", "conversation_id": "conv-gal"},
+                            files={"file": ("a.txt", b"alpha")})
+        cid1 = resp1.json()["context_id"]
+        resp2 = client.post("/api/upload", headers=headers,
+                            data={"mode": "context", "conversation_id": "conv-gal"},
+                            files={"file": ("b.txt", b"beta")})
+        cid2 = resp2.json()["context_id"]
+
+        gal = client.get("/api/upload/by-conversation/conv-gal", headers=headers)
+        assert gal.status_code == 200
+        items = gal.json()
+        assert len(items) == 2
+        assert items[0]["filename"] == "a.txt"
+        assert items[1]["filename"] == "b.txt"
+
+
+def test_link_upload_to_conversation(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        headers = _admin_headers(client)
+        resp = client.post("/api/upload", headers=headers,
+                           data={"mode": "context"},
+                           files={"file": ("orphan.txt", b"lonely")})
+        cid = resp.json()["context_id"]
+
+        link = client.patch(f"/api/upload/{cid}/link", headers=headers,
+                            json={"conversation_id": "new-conv"})
+        assert link.status_code == 200
+
+        gal = client.get("/api/upload/by-conversation/new-conv", headers=headers)
+        assert len(gal.json()) == 1
+
+
+def test_delete_upload_removes_context(tmp_path: Path, monkeypatch):
+    class FakeAsyncClient:
+        class FakeResponse:
+            def __init__(self, status, json_data=None):
+                self.status_code = status
+                self._json = json_data or {}
+
+            def json(self):
+                return self._json
+
+        def __init__(self, *a, **kw):
+            pass
+
+        async def __aenter__(self):
+            return self
+
+        async def __aexit__(self, *a):
+            pass
+
+        async def post(self, url, **kw):
+            if "/points/scroll" in url:
+                return self.FakeResponse(200, {"result": {"points": [{"id": "upload-test.txt-0"}, {"id": "upload-test.txt-1"}]}})
+            if "/points/delete" in url:
+                return self.FakeResponse(200)
+            return self.FakeResponse(200)
+
+        async def put(self, url, **kw):
+            return self.FakeResponse(200)
+
+    monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: FakeAsyncClient())
+
+    with make_client(tmp_path) as client:
+        headers = _admin_headers(client)
+        resp = client.post("/api/upload", headers=headers,
+                           data={"mode": "context", "conversation_id": "del-test"},
+                           files={"file": ("test.txt", b"delete me")})
+        cid = resp.json()["context_id"]
+
+        del_resp = client.delete(f"/api/upload/{cid}", headers=headers)
+        assert del_resp.status_code == 200
+        assert del_resp.json()["status"] == "ok"
+
+        row = db.get_db().execute("SELECT id FROM upload_context WHERE id = ?", (cid,)).fetchone()
+        assert row is None
+
+
+def test_delete_upload_not_found(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.delete("/api/upload/999", headers=_admin_headers(client))
+        assert resp.status_code == 404
+
+
+def test_conversation_list_includes_attachment_count(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        headers = _admin_headers(client)
+        client.post("/api/conversations", headers=headers, json={"title": "NoAttach"})
+        with_attach = client.post("/api/conversations", headers=headers, json={"title": "WithAttach"}).json()
+        conv_id = with_attach["id"]
+        client.post("/api/upload", headers=headers,
+                    data={"mode": "context", "conversation_id": conv_id},
+                    files={"file": ("f.txt", b"data")})
+
+        list_resp = client.get("/api/conversations", headers=headers)
+        convs = list_resp.json()
+        for c in convs:
+            if c["title"] == "NoAttach":
+                assert c["attachment_count"] == 0
+            if c["title"] == "WithAttach":
+                assert c["attachment_count"] == 1

triage.py

@@ -0,0 +1,98 @@
+"""cAIc — Query triage and cluster node selection."""
+import logging
+
+import httpx
+
+from config import TRIAGE_BASE, TRIAGE_TIMEOUT, LLAMA_SERVER_BASE
+
+log = logging.getLogger("caic")
+
+_IDEAL_MODEL_MAP = {
+    "code": {"name_contains": ["coder", "qwen"]},
+    "general": {"name_contains": ["mistral", "llama"]},
+}
+
+_CLASSIFICATION_PROMPT = """Classify the following user query into exactly one category. Respond with only the category name.
+
+Categories:
+- general: everyday questions, chitchat, creative writing, advice, explanations
+- code: programming, debugging, code generation, technical questions about software
+- search: questions about current events, real-time information, weather, news, specific things that may have changed since training
+- rag: questions about specific documents, personal data, notes, memory, uploaded content
+
+Query: {query}
+Category:"""
+
+
+async def classify_query(query: str) -> str:
+    try:
+        async with httpx.AsyncClient() as client:
+            resp = await client.post(
+                f"{TRIAGE_BASE}/chat/completions",
+                json={
+                    "model": "phi-4-mini",
+                    "messages": [
+                        {"role": "system", "content": "You are a query classifier. Respond with exactly one word."},
+                        {"role": "user", "content": _CLASSIFICATION_PROMPT.format(query=query)},
+                    ],
+                    "temperature": 0.0,
+                    "max_tokens": 10,
+                },
+                timeout=TRIAGE_TIMEOUT,
+            )
+            text = resp.json()["choices"][0]["message"]["content"].strip().lower()
+            valid = {"general", "code", "search", "rag"}
+            for v in valid:
+                if v in text:
+                    return v
+    except Exception:
+        log.warning("triage classify_query failed, falling back to general", exc_info=True)
+    return "general"
+
+
+async def select_node(classification: str) -> dict | None:
+    from cluster import CLUSTER_NODES
+
+    if classification in ("search", "rag"):
+        return None
+
+    ideal = _IDEAL_MODEL_MAP.get(classification, {})
+    ideal_contains = ideal.get("name_contains", [])
+
+    # First pass: find an active node with the right model already loaded
+    for node in CLUSTER_NODES.values():
+        if node.get("status") != "active":
+            continue
+        am = node.get("active_model") or {}
+        name = (am.get("name") or "").lower()
+        if any(ideal in name for ideal in ideal_contains):
+            return node
+
+    # Second pass: find an active node that can swap to the right model
+    for node in CLUSTER_NODES.values():
+        if node.get("status") != "active":
+            continue
+        inventory = node.get("inventory") or []
+        for inv in inventory:
+            inv_name = (inv.get("name") or "").lower()
+            if any(ideal in inv_name for ideal in ideal_contains):
+                from cluster import request_model_swap
+                await request_model_swap(node["name"], inv["filename"])
+                return None
+
+    return None
+
+
+async def get_inference_url(query: str) -> str:
+    if not query:
+        return LLAMA_SERVER_BASE
+    classification = await classify_query(query)
+    if classification in ("search", "rag"):
+        return LLAMA_SERVER_BASE
+    node = await select_node(classification)
+    if node:
+        am = node.get("active_model") or {}
+        port = am.get("port", 8081)
+        ip = node.get("ip") or "127.0.0.1"
+        return f"http://{ip}:{port}/v1"
+    return LLAMA_SERVER_BASE