docs: add docker.md — full Docker distribution architecture plan

Covers all six services (jarvisChat, SearXNG, Qdrant, RabbitMQ,
llama-server, Ollama) with images, ports, volumes, healthchecks,
env mapping, secrets management, setup wizard spec, and back-out
procedure. Includes pre-v1.0 gate checklist and open decisions.

Also add .env, secrets/, searxng/ to .gitignore (these are
generated by setup.sh and contain secrets).

.gitignore

@@ -5,3 +5,7 @@ __pycache__/
 venv/
 readme.md-
 *.bak
+hardware_state.json
+.env
+secrets/
+searxng/

AGENTS.md

@@ -0,0 +1,120 @@
+# JarvisChat — Agents Guide
+
+## Run
+
+```bash
+./venv/bin/uvicorn app:app --host 0.0.0.0 --port 8080 --reload
+```
+
+## Tests
+
+```bash
+./venv/bin/python -m pytest tests/ -v
+```
+
+All tests use `tmp_path` fixtures + monkeypatched `httpx.AsyncClient.stream/get/post/put`. No external services needed. Test factories reset `SESSIONS`, `PIN_ATTEMPTS`, `RATE_EVENTS` globals — be careful not to let test state leak. Tests import directly from the correct modules (`db`, `security`, `config`, `search`, `rag`, `memory`, `routers.*`).
+
+Every router has a dedicated test file:
+| File | Covers |
+|------|--------|
+| `test_auth_capabilities.py` | `auth.py` — guest/admin sessions, origin blocking, logout |
+| `test_chat_streaming_and_memory_paths.py` | `routers/chat.py` — streaming, auto-search, remember/forget, upload context injection |
+| `test_completions.py` | `routers/completions.py` — API key auth, FIM, streaming, blocking, errors |
+| `test_conversations.py` | `routers/conversations.py` — full CRUD, guest admin enforcement, attachment_count |
+| `test_ingest.py` | `routers/ingest.py` — Bearer auth, chunk/embed/upsert, validation |
+| `test_memories.py` | `routers/memories.py` — edit, search, stats endpoints |
+| `test_models_router.py` | `routers/models.py` — models list, ps, show, stats, search/status |
+| `test_presets.py` | `routers/presets.py` — full CRUD, default preset protection |
+| `test_profile.py` | `routers/profile.py` — get, update, default, length validation |
+| `test_search_route.py` | `routers/search_route.py` — explicit search flow, no results, errors |
+| `test_search_url_sanitization.py` | `search.py` URL sanitizer |
+| `test_settings_allowlist.py` | `routers/settings.py` — allowlisted key enforcement |
+| `test_skills_framework.py` | `routers/skills.py` — list, toggle, unknown skill, prompt injection |
+| `test_ip_allowlist.py` | IP allowlist helper + middleware |
+| `test_rate_and_payload_guardrails.py` | Rate limits + payload size enforcement |
+| `test_error_envelopes.py` | Global exception handler + stream error incidents |
+| `test_upload.py` | `routers/upload.py` — upload, delete, link, by-conversation, attachment_count integration |
+
+Modules that call `httpx.AsyncClient` (chat, completions, models, search_route, upload, ingest)
+are mocked via `monkeypatch.setattr` on `AsyncClient.stream`, `.get`, or `.post`.
+CPU stats in `models.py` (`api/stats`) use real `psutil`; GPU stats are
+monkeypatched via `routers.models.get_gpu_stats`.
+
+## Architecture
+
+Refactored from single-file (`app.py`) into modules under project root:
+
+| File | Role |
+|------|------|
+| `app.py` | FastAPI app, middleware, router registration |
+| `config.py` | Constants, env vars, rate/payload limits, built-in skills registry, upload limits |
+| `db.py` | SQLite schema, connection factory, settings helpers, upload_context CRUD |
+| `auth.py` | PIN-based guest/admin sessions, auth routes |
+| `security.py` | Rate limiting, origin checks, IP allowlist, audit/incident logging |
+| `memory.py` | FTS5 memory CRUD, remember/forget command parsing |
+| `search.py` | SearXNG integration, perplexity scoring, refusal detection |
+| `rag.py` | Qdrant vector search + system prompt assembly + chunk_text() helper |
+| `gpu.py` | AMD GPU stats via `rocm-smi` |
+| `routers/` | One module per endpoint group (chat, search, skills, completions, upload, ingest) |
+
+### Entrypoint / API keys
+
+- `app.py` line 148: `uvicorn.run(app, ...)` when called directly
+- `config.py` line 14: `LLAMA_SERVER_BASE` defaults to `http://192.168.50.108:8081` — llama-server on ultron, RPC-offloads GPU layers to jarvis :50052
+- `config.py` line 17: `COMPLETIONS_API_KEY` read from `JARVISCHAT_COMPLETIONS_API_KEY` env var or auto-generates
+- `config.py` line 13: `OLLAMA_BASE` is legacy/unused — all endpoints use `LLAMA_SERVER_BASE`
+
+### Key flows
+
+1. **`/api/chat`** → `process_remember_command()` intercepts "remember that..." / "forget about..." first → optional `upload_context_id` fetches document text from SQLite → `build_system_prompt()` (profile + FTS5 memory + Qdrant RAG + preset + skills + uploaded doc) → stream from llama-server with `logprobs: true` → if perplexity > 15.0 OR `REFUSAL_PATTERNS` match, re-query with SearXNG results
+2. **`/api/search`** → bypasses perplexity/refusal, queries SearXNG directly → summarizes via llama-server
+3. **`/v1/chat/completions`** → OpenAI-compatible for Continue.dev/IDE integration; FIM requests proxied without persistence
+4. **`/api/upload`** → multipart file upload, PDF/text extraction, `mode=(context|ingest|both)`, stores SQLite context (1hr expiry) + Qdrant upsert
+5. **`/api/ingest`** → Bearer token auth, programmatic RAG ingest (terminal hook, external tools)
+
+### Perplexity / auto-search
+
+The upstream request includes `"logprobs": true`. `parse_llama_stream_chunk()` extracts per-token logprobs from each chunk's `choices[0].logprobs.content[].logprob`. The `all_logprobs` list is populated during streaming, so `calculate_perplexity()` and `is_uncertain()` work correctly.
+
+### Auth / lockdown
+
+- Guest session by default (`POST /api/auth/guest`), admin unlock via 4-digit PIN (`POST /api/auth/login`)
+- Admin required for PUT/DELETE/PATCH + all POST except allowlist (`/api/chat`, `/api/search`, `/api/auth/*`)
+- `/api/ingest` is exempt from session auth — self-authenticates via Bearer token
+- IP allowlist, rate limiting, origin checking, payload size limits — all enforced in `app.py` middleware
+- Origin check applies to **all** `/api/` requests; returns `False` when both `Origin` and `Referer` are absent
+- `JARVISCHAT_ADMIN_PIN` env var required on first boot (or `JARVISCHAT_ALLOW_DEFAULT_PIN=true`)
+
+### Database
+
+- SQLite at `jarvischat.db`, auto-created by `init_db()` on startup via FastAPI `lifespan`
+- `get_db()` opens new connection per request (no pool). Close after use.
+- FTS5 virtual table `memories` for full-text search with BM25 ranking.
+- `upload_context` table: auto-expiring document storage for chat context injection.
+
+### External services
+
+| Service | Required | Port |
+|---------|----------|------|
+| llama-server (ultron) | Yes | 8081 + RPC :50052 (jarvis GPU) |
+| SearXNG | No | 8888 |
+| wttr.in | No | weather shortcut |
+| rocm-smi | No | AMD GPU stats |
+| Qdrant | No | 6333 (ultron) — RAG vector search |
+
+### Config quirks
+
+- `BODY_LIMIT_UPLOAD_BYTES` = 20MB for `/api/upload`; other paths use smaller limits
+- `SUPPORTED_UPLOAD_TYPES` includes images (png/jpeg/gif/svg/webp) + text + PDF + JSON
+- `UPLOAD_CONTEXT_EXPIRY_HOURS` = 1 hour
+- Rate limits and payload caps in `config.py` — patch `security.RL_*` not `config.RL_*` for tests
+- RAG embedding requests go to `EMBED_URL` at `/api/embeddings` (Ollama on jarvis :11434)
+
+### SSE Protocol
+
+All streaming endpoints yield `data: {json}\n\n`. Key shapes:
+- `{token, conversation_id}` — streaming token
+- `{searching: true}` — web search triggered
+- `{search_results: N}` — N results (no raw_results payload)
+- `{done: true, perplexity, tokens_per_sec, searched?}` — terminal
+- `{error: "...", error_key: "..."}` — error with incident key

CLAUDE.md

@@ -19,46 +19,61 @@ sudo systemctl restart jarvischat
 
 ```bash
 ./venv/bin/pip install -r requirements.txt
-# Also requires: psutil jinja2 python-multipart (not in requirements.txt)
+# Also requires: psutil jinja2 python-multipart pypdf (not in requirements.txt)
 ```
 
 ## Architecture
 
-Single-file FastAPI backend (`app.py`) + single-template frontend (`templates/index.html`). No build step. SQLite database auto-created at `jarvischat.db` on first run.
+Modular FastAPI app — `app.py` wires routers, middleware, and lifespan. SQLite database auto-created at `jarvischat.db` on first run. No build step, single `templates/index.html`.
 
 ### Request Flow: `/api/chat`
 
 1. User message saved to DB → conversation created if new
-2. `build_system_prompt()` assembles: profile + FTS5 memory search results + preset prompt
-3. Streamed to Ollama (`/api/chat`, `stream: true`, `logprobs: true`) via SSE
-4. **Auto web search trigger**: if perplexity > 15.0 OR response matches `REFUSAL_PATTERNS`, re-queries Ollama with SearXNG results prepended to system prompt
-5. Final response saved to DB; SSE `done` event sent with perplexity + tokens/sec
+2. `process_remember_command()` intercepts "remember that..." / "forget about..." first
+3. Optional `upload_context_id` → fetches document text from `upload_context` table, injects `[ATTACHED DOCUMENT]` into system prompt
+4. `build_system_prompt()` assembles: profile + FTS5 memory search + Qdrant RAG + preset + skills + uploaded doc
+5. Streamed to llama-server (`/v1/chat/completions`, `stream: true`, `logprobs: true`) via SSE
+6. **Auto web search trigger**: if perplexity > 15.0 OR response matches `REFUSAL_PATTERNS`, re-queries with SearXNG results
+7. Final response saved to DB; SSE `done` event sent with perplexity + tokens/sec
 
 ### Request Flow: `/api/search` (explicit search)
 
-Bypasses perplexity/refusal detection entirely — queries SearXNG directly then asks Ollama to summarize with results as system context.
+Bypasses perplexity/refusal — queries SearXNG directly then asks llama-server to summarize results.
+
+### Request Flow: `/api/upload`
+
+Multipart file upload → PDF/text extraction + chunking → optional Qdrant upsert + SQLite context storage (1hr expiry). Supports `mode=(context|ingest|both)`. Images upload as storage only — model cannot process image content.
+
+### Request Flow: `/api/ingest`
+
+Bearer-token-authenticated terminal RAG hook. Accepts raw text, chunks via `chunk_text()`, embeds via Ollama `/api/embeddings`, upserts to Qdrant.
 
 ### Memory System
 
-FTS5 virtual table (`memories`) in SQLite. `search_memories()` uses BM25 ranking. `process_remember_command()` intercepts "remember that..." / "forget about..." before the message reaches Ollama and returns a confirmation string. Topic auto-detection via keyword matching in `detect_topic()`.
+FTS5 virtual table (`memories`) in SQLite. `search_memories()` uses BM25 ranking. `process_remember_command()` intercepts "remember that..." / "forget about..." before the message reaches the model and returns a confirmation string.
 
-### Key Constants (top of `app.py`)
+### Key Constants (`config.py`)
 
-- `OLLAMA_BASE` — `http://localhost:11434`
+- `LLAMA_SERVER_BASE` — `http://192.168.50.108:8081` (ultron llama-server, RPC offloads to jarvis GPU)
 - `SEARXNG_BASE` — `http://localhost:8888`
-- `PERPLEXITY_THRESHOLD` — `15.0` (controls auto-search sensitivity)
-- `DEFAULT_MODEL` — `llama3.1:latest`
+- `PERPLEXITY_THRESHOLD` — `15.0`
+- `EMBED_URL` — `http://192.168.50.210:11434/api/embeddings` (Ollama on jarvis)
+- `VERSION` — current version string
 
 ### External Services
 
-- **Ollama** — required, must be running on port 11434
-- **SearXNG** — optional, port 8888; `GET /api/search/status` probes availability
-- **wttr.in** — weather shortcut in `query_searxng()`, bypasses SearXNG for weather queries
-- **rocm-smi** — AMD GPU stats via subprocess; gracefully degrades if not available
+| Service | Required | Port |
+|---------|----------|------|
+| **llama-server** (ultron) | Yes | 8081 + RPC :50052 (jarvis GPU) |
+| **SearXNG** | No | 8888 |
+| **wttr.in** | No | weather shortcut |
+| **rocm-smi** | No | AMD GPU stats |
+| **Qdrant** (ultron) | No | 6333 — RAG vector search |
+| **Ollama** (jarvis) | No | 11434 — embeddings only |
 
 ### Database
 
-`get_db()` opens a new connection per request (no connection pool). `init_db()` runs at startup via the FastAPI `lifespan` handler. The `profile` table uses a singleton row (`id = 1`). Default settings are seeded but never overwritten by `init_db()`.
+`get_db()` opens a new connection per request (no pool). `init_db()` runs at startup via FastAPI `lifespan`. Tables: `conversations`, `messages`, `settings`, `profile` (singleton id=1), `memories` (FTS5), `upload_context`. Default settings seeded but never overwritten.
 
 ### SSE Protocol
 
@@ -67,8 +82,8 @@ All streaming endpoints yield `data: {json}\n\n`. Key event shapes:
 - `{searching: true}` — web search triggered
 - `{search_results: N}` — N results retrieved
 - `{done: true, perplexity, tokens_per_sec, searched?}` — terminal event
-- `{error: "..."}` — error event
+- `{error: "...", error_key: "..."}` — error with incident key
 
 ### Deployment
 
-Runs as systemd service under user `jarvischat`, working directory `/opt/jarvischat`. Logs via syslog (`journalctl -u jarvischat`).
+Runs as systemd service under user `jarvischat`, working directory `/opt/jarvischat`. Logs via syslog (`journalctl -u jarvischat`). Version bumps via git tag + commit, deployed via `git pull && systemctl restart jarvischat`.

README.md

@@ -1,453 +1,287 @@
-![jarvisChat logo](static/jcscreenie.png)
-# ⚡ JarvisChat v1.9.0
+# jarvisChat v0.11.0
 
-**A privacy-first, homelab-native developer knowledge platform.**
+You have a garage full of retired office PCs, a GPU that was mid-range when Obama was president, and a burning desire to chat with a language model without renting some billionaire's server farm. Congratulations — you've found your people.
 
-> JarvisChat turns a heterogeneous LAN of budget hardware into a distributed local AI inference cluster — accumulating institutional knowledge over time, keeping all data off the cloud, and squeezing real performance out of modest consumer hardware through architecture rather than dollars.
+jarvisChat is a chat UI that grew limbs. It started as a single-file Python script because OpenWebUI wouldn't install on Debian 13, and somewhere along the way it learned to file paperwork (file attachments), write things down (RAG ingest), boss around other computers (AMQP clustering), and check its own pulse (hardware self-assessment). It now does all the things you didn't ask for, plus a few you might actually use.
 
-This is not another AI chat wrapper. jC is the UX and knowledge-management layer for a local AI brain — analogous to what Windows was to DOS, or what the web is to the internet. The intelligence lives in the model and the RAG corpus. jC makes it accessible and keeps feeding it.
+Under the hood: FastAPI + SQLite + Jinja2 on Python 3.13. Works with or without Docker — no container required for basic operation. Stitches together mismatched hardware via llama.cpp RPC — your gaming PC's dusty RX 580, the NUC in the closet, that old workstation from 2017 — and spreads inference across them like peanut butter on stale bread. It shouldn't work, but somehow it does.
 
----
+At v1.0, this ships as a Docker-based distribution with a setup wizard that detects CPU vs GPU, probes your hardware, and stands up SearXNG, Qdrant, RabbitMQ, and everything else with a single `docker compose up`. Manual install docs are maintained alongside for the bare-metal crowd.
 
-## The Four Pillars
+Developer wiki: [docs/wiki/Home.md](docs/wiki/Home.md)
 
-### 1. Privacy
-Everything runs on your LAN. No API keys, no cloud endpoints, no data leaving your network, no subscription, no terms-of-service surprises. Your conversations, your codebase, your decisions — stay yours.
+## What's New in v0.11.0
 
-### 2. Knowledge Retention
-Unlike stateless chat tools that forget everything when you close the tab, jC accumulates institutional memory. Every solved problem, every architectural decision, every working command gets absorbed into the RAG corpus via Qdrant. The system gets smarter the longer you use it.
+### File & Document Attachments (v1.9.0–v1.10.0)
+- **`POST /api/upload`** — multipart file upload with PDF/text extraction; modes: `context` (chat injection), `ingest` (RAG corpus), `both`
+- **`DELETE /api/upload/{id}`** — removes upload from SQLite + Qdrant
+- **`PATCH /api/upload/{id}/link`** — associates upload with a conversation
+- **`GET /api/upload/by-conversation/{id}`** — list attachments per conversation
+- **Paperclip UI** — file picker, preview pill, image thumbnails, gallery overlay
+- **Attachment indicators** — 📎 badge on conversations with attachments
+- **Chat context injection** — `upload_context_id` prepends document text to system prompt
 
-### 3. Budget Hardware Maximization
-You don't need a $10,000 workstation. jC is designed for the developer who has a drawer full of machines and the skills to wire them together. RPC clustering, model splitting across CPU and GPU nodes, dynamic resource negotiation, and smart RAG eviction squeeze real performance out of modest consumer hardware.
+### Terminal RAG Hook — `POST /api/ingest` (v0.11.0)
+- Bearer token auth (same key as `/v1/chat/completions`)
+- Chunking via shared `chunk_text()` helper, embed via Ollama, upsert to Qdrant
+- `jc-ingest.sh` — PROMPT_COMMAND shell script for autonomous terminal history ingestion
 
-### 4. Homelab-Native Architecture
-Built specifically for the heterogeneous homelab: mixed hardware, mixed OS, consumer GPUs, ARM boards, NAS storage — all working together as a coherent AI platform. A designated master node hosts jC, llama-server, and SearXNG. GPU nodes self-register as RPC inference workers. The architecture scales horizontally across whatever you've got.
+### v1.8.0 Foundation (refactor & fixes)
+- **Modular refactor** — single-file `app.py` split into `config.py`, `db.py`, `auth.py`, `security.py`, `memory.py`, `search.py`, `rag.py`, `gpu.py`, and `routers/` package
+- **Perplexity auto-search fixed** — `logprobs: true` now properly extracted from stream chunks
+- **All `/api/models` endpoints** target `LLAMA_SERVER_BASE` (llama-server) not Ollama
+- **RAG embedding** via Ollama at `http://192.168.50.210:11434`
+- **Origin check** applies to all API methods, rejects absent Origin/Referer
 
----
+## Features
 
-## Target Audience
+- **Persistent Memory** — SQLite FTS5 full-text search for fast, relevant memory retrieval
+- **Web Search** — SearXNG integration for automatic web lookups when the model is uncertain
+- **Explicit Search** — Search button to force web search without waiting for model uncertainty
+- **Profile Injection** — Custom system prompt injected into every conversation
+- **System Presets** — Save and switch between different system prompts
+- **Real-time Stats** — CPU, RAM, GPU, VRAM monitoring in sidebar
+- **Token Thermometer** — Visual context window usage indicator
+- **Streaming Responses** — Server-sent events for real-time token display
+- **Conversation History** — SQLite-backed chat persistence with mass-delete option
+- **Model Switching** — Change inference models on the fly
+- **Skills Framework** — Built-in skill registry with per-skill enable/disable controls
 
-Solo developers and homelab enthusiasts who are:
-- Budget-constrained but hardware-rich (multiple machines, NAS, spare GPUs)
-- Privacy-conscious (no cloud AI subscriptions)
-- Technically capable (if you can install jC, you can designate the master node)
-- Building something over time and want their AI to remember it
-
----
-
-## Architecture
+## File Structure
 
 ```
-┌─────────────────────────────────────────────────────────────┐
-│                        YOUR LAN                             │
-│                                                             │
-│  ┌─────────────────┐         ┌──────────────────────────┐  │
-│  │   jarvis        │◄──RPC───│   ultron                 │  │
-│  │   192.168.50.212│  50052  │   192.168.50.108         │  │
-│  │                 │         │                          │  │
-│  │  jC :8080       │         │  llama-server :8081      │  │
-│  │  SearXNG :8888  │         │  llama-server :8082 (*)  │  │
-│  │  RX 6600 XT 8GB │         │  Qdrant :6333            │  │
-│  │  GPU RPC worker │         │  mxbai-embed :11434      │  │
-│  │  Vulkan backend │         │  AMD Ryzen 7 7840HS      │  │
-│  └─────────────────┘         │  Radeon 780M iGPU        │  │
-│                              └──────────────────────────┘  │
-│                                                             │
-│  ┌─────────────────┐         ┌──────────────────────────┐  │
-│  │   pivault       │         │   corsair                │  │
-│  │   192.168.50.158│         │   192.168.50.132         │  │
-│  │                 │         │                          │  │
-│  │  10.83TB RAID5  │         │  RTX 5070 Ti 16GB        │  │
-│  │  RPi 5 8GB      │         │  Ryzen 7 7800X3D         │  │
-│  │  NAS / Kopia    │         │  Gaming / Streaming      │  │
-│  └─────────────────┘         └──────────────────────────┘  │
-│                                                             │
-│  (*) Planned: Qwen2.5-Coder-14B on :8082                   │
-└─────────────────────────────────────────────────────────────┘
+/opt/jarvischat/
+├── app.py              # FastAPI app entry point
+├── config.py           # Constants, env vars, limits, skill registry
+├── db.py               # SQLite schema, connection factory
+├── auth.py             # PIN-based guest/admin sessions, auth routes
+├── security.py         # Rate limiting, origin checks, IP allowlist, audit
+├── memory.py           # FTS5 memory CRUD, remember/forget commands
+├── search.py           # SearXNG integration, perplexity, refusal detection
+├── rag.py              # Qdrant vector search + system prompt assembly
+├── gpu.py              # AMD GPU stats via rocm-smi
+├── routers/
+│   ├── chat.py         # /api/chat streaming endpoint
+│   ├── search_route.py # /api/search explicit search endpoint
+│   ├── completions.py  # /v1/chat/completions OpenAI-compat endpoint
+│   ├── conversations.py# Conversation CRUD
+│   ├── memories.py     # Memory CRUD API
+│   ├── models.py       # Model listing, system stats
+│   ├── presets.py      # System prompt presets
+│   ├── profile.py      # User profile
+│   ├── settings.py     # Runtime settings
+│   ├── skills.py       # Skills management
+│   ├── upload.py       # File attachment endpoints
+│   └── ingest.py       # Terminal RAG ingest
+├── static/
+│   └── logo.png        # Logo image (optional)
+├── templates/
+│   └── index.html      # Frontend
+└── tests/              # 110 pytest tests
 ```
 
-**Data flow:**
-```
-Browser / IDE (Continue.dev)
-    → jC :8080 (FastAPI — auth, RAG, memory, conversation history)
-        → Qdrant :6333 (vector search, mxbai-embed-large for embeddings)
-        → llama-server :8081 (inference)
-            → jarvis RPC :50052 (GPU layer offload — RX 6600 XT)
-```
+## Requirements
 
----
-
-## The AMD + NVIDIA Cross-Cluster Reality
-
-This cluster intentionally mixes GPU architectures — **AMD RX 6600 XT on jarvis** and **NVIDIA RTX 5070 Ti on corsair**. This is deliberate and it works.
-
-The RPC layer in llama.cpp is GPU-vendor-agnostic. jarvis runs llama-rpc with a **Vulkan backend** (not ROCm, not CUDA) which provides hardware-neutral GPU acceleration. ultron's llama-server connects to it over TCP and offloads tensor layers without caring what GPU is on the other end.
-
-This means any machine on your LAN with any GPU (AMD, NVIDIA, Intel Arc) can participate as an RPC worker — as long as it can run llama-rpc with Vulkan support.
-
----
-
-## Cluster Performance Tuning
-
-### The Layer Offloading Trick
-
-The key to squeezing performance out of a CPU+GPU split cluster is `--n-gpu-layers`. This controls how many transformer layers get offloaded to the RPC GPU backend versus staying on the CPU.
-
-**Starting point (before tuning):** ~7 t/s  
-**After initial layer optimization:** ~17 t/s  
-**After full cluster tuning:** 30–35 t/s
-
-The progression that got us there:
-
-1. **Start with `--n-gpu-layers 99`** — tells llama-server to offload as many layers as possible. With Mistral-Nemo-12B Q4_K_M this results in all 41/41 layers offloading to jarvis GPU via RPC.
-
-2. **Verify GPU is actually working** — watch the llama-server startup log for:
-   ```
-   load_tensors: offloaded 41/41 layers to GPU
-   load_tensors: RPC[192.168.50.210:50052] model buffer size = 6763.30 MiB
-   load_tensors: CPU_Mapped model buffer size = 360.00 MiB
-   ```
-   If layers aren't offloading, the RPC connection isn't established.
-
-3. **Check actual throughput** — the timings block in llama-server responses shows real t/s. Tune from there.
-
-**Current llama-server service on ultron (`/etc/systemd/system/llama-server.service`):**
-```ini
-[Unit]
-Description=Llama.cpp Server (RPC frontend — Mistral-Nemo general)
-After=network.target
-
-[Service]
-Type=simple
-User=root
-ExecStart=/root/llama.cpp/build/bin/llama-server \
-  --model /home/gramps/models/Mistral-Nemo-Instruct-2407-Q4_K_M.gguf \
-  --rpc 192.168.50.212:50052 \
-  --host 0.0.0.0 \
-  --port 8081 \
-  --n-gpu-layers 99
-Restart=on-failure
-RestartSec=5
-
-[Install]
-WantedBy=multi-user.target
-```
-
-**llama-rpc service on jarvis (`/etc/systemd/system/llama-rpc.service`):**
-```ini
-[Unit]
-Description=Llama.cpp RPC Server (GPU backend — RX 6600 XT Vulkan)
-After=network.target
-
-[Service]
-Type=simple
-User=root
-ExecStart=/root/llama.cpp/build/bin/llama-rpc-server \
-  --host 0.0.0.0 \
-  --port 50052
-Restart=on-failure
-RestartSec=5
-
-[Install]
-WantedBy=multi-user.target
-```
-
----
-
-## Models
-
-### Current
-| Model | Location | Port | Purpose |
-|-------|----------|------|---------|
-| Mistral-Nemo-Instruct-2407-Q4_K_M | `/home/gramps/models/` on jarvis | ultron:8081 | General assistant, chat |
-| mxbai-embed-large | ultron (Docker/Ollama) | ultron:11434 | RAG embeddings |
-
-### Planned
-| Model | Size | Port | Purpose |
-|-------|------|------|---------|
-| Qwen2.5-Coder-14B-Q5_K_M | ~10GB | ultron:8082 | Code completion, pair programming |
-
-> **Note:** ultron has 16GB RAM. Only one primary inference model can be hot at a time. llama-server instances are swapped via systemd when switching between general and code models.
-
----
-
-## RAG System
-
-jC uses **Qdrant** for vector storage and **mxbai-embed-large** (1024-dim) for embeddings.
-
-### Qdrant Collection
-- **Collection:** `jarvis_rag`
-- **Vector size:** 1024 (mxbai-embed-large output)
-- **Distance:** Cosine
-- **Score threshold:** 0.25 (filters low-relevance chunks)
-- **Chunks retrieved per query:** 3 (configurable)
-
-### RAM Ceiling
-Each vector = 4KB (1024 dims × float32). With ultron's ~4-6GB available to Qdrant after llama-server:
-- Practical ceiling: ~1–1.5M chunks before RAM becomes the bottleneck
-- Current corpus: 219 points (early stage)
-- Storage on disk: negligible against pivault's 10.83TB
-
-### What Gets Ingested
-- Code repositories (your actual codebase)
-- Pair-programming conversation history
-- Architecture decisions and working commands
-- Documentation and URLs (fetched and stripped via beautifulsoup4/httpx)
-
----
-
-## JarvisChat Service (`/etc/systemd/system/jarvischat.service`)
-
-```ini
-[Unit]
-Description=JarvisChat - Local LLM Developer Platform
-After=network.target
-
-[Service]
-Type=simple
-User=root
-WorkingDirectory=/opt/jarvischat
-ExecStart=/opt/jarvischat/venv/bin/uvicorn app:app --host 0.0.0.0 --port 8080
-Restart=always
-RestartSec=5
-Environment=PYTHONUNBUFFERED=1
-Environment=OLLAMA_BASE=http://192.168.50.108:8081
-Environment=LLAMA_SERVER_BASE=http://192.168.50.108:8081
-
-[Install]
-WantedBy=multi-user.target
-```
-
----
+- Python 3.11+ (tested on 3.13)
+- llama-server running locally or on network (OpenAI-compatible API on port 8081)
+- SearXNG (optional, for web search)
 
 ## Installation
 
-### Prerequisites
-- Python 3.11+ (tested on 3.13)
-- llama.cpp built from source on both jarvis (RPC server) and ultron (llama-server)
-- Qdrant running on ultron
-- Ollama on ultron (for mxbai-embed-large embeddings)
-- SearXNG on jarvis:8888 (optional, for web search)
-
 ### Fresh Install
 
 ```bash
+# Create directory and venv
 sudo mkdir -p /opt/jarvischat
 sudo chown $USER:$USER /opt/jarvischat
 cd /opt/jarvischat
 python3 -m venv venv
-./venv/bin/pip install fastapi uvicorn httpx psutil jinja2 python-multipart qdrant-client
+
+# Install dependencies
+./venv/bin/pip install fastapi uvicorn httpx psutil jinja2 python-multipart pypdf
+
+# Set admin PIN before first startup (4 digits)
+export JARVISCHAT_ADMIN_PIN=4827
+
+# Create subdirectories
 mkdir -p templates static
+
+# Copy files
+# (copy all .py files to /opt/jarvischat/)
+# (copy routers/ directory to /opt/jarvischat/)
+# (copy templates/index.html to /opt/jarvischat/templates/)
 ```
 
-Copy `app.py` to `/opt/jarvischat/` and `index.html` to `/opt/jarvischat/templates/`.
+WARNING: Do not use `1234` as your admin PIN unless you accept weak local security.
 
-### Bootstrap the PIN
+NOTE: First boot requires `JARVISCHAT_ADMIN_PIN` unless you explicitly opt into insecure fallback with `JARVISCHAT_ALLOW_DEFAULT_PIN=true`.
+
+## Systemd Service
+
+Create `/etc/systemd/system/jarvischat.service`:
+
+```ini
+[Unit]
+Description=jarvisChat - Local Inference Web Interface
+After=network.target
+
+[Service]
+Type=simple
+User=jarvischat
+Group=jarvischat
+WorkingDirectory=/opt/jarvischat
+ExecStart=/opt/jarvischat/venv/bin/uvicorn app:app --host 0.0.0.0 --port 8080
+Restart=always
+RestartSec=5
+
+[Install]
+WantedBy=multi-user.target
+```
 
 ```bash
-export JARVISCHAT_ADMIN_PIN=XXXX  # your 4-digit PIN
+sudo systemctl daemon-reload
+sudo systemctl enable jarvischat
+sudo systemctl start jarvischat
 ```
 
-Or allow the insecure default for testing:
-```bash
-export JARVISCHAT_ALLOW_DEFAULT_PIN=true
-```
+## Memory Commands
 
-### Environment Variables
+In chat, natural language triggers memory operations:
 
-| Variable | Default | Description |
-|----------|---------|-------------|
-| `OLLAMA_BASE` | `http://localhost:11434` | Ollama-compatible endpoint (legacy) |
-| `LLAMA_SERVER_BASE` | `http://192.168.50.108:8081` | llama-server OpenAI-compat inference endpoint |
-| `JARVISCHAT_ADMIN_PIN` | (none) | 4-digit admin PIN (required on first boot) |
-| `JARVISCHAT_ALLOW_DEFAULT_PIN` | `false` | Allow insecure default PIN 1234 |
-| `JARVISCHAT_TRUSTED_ORIGINS` | (none) | Comma-separated trusted origins for CSRF |
-| `JARVISCHAT_ALLOWED_CIDRS` | RFC1918 + loopback | Allowed client IP CIDRs |
+| You say | What happens |
+|---------|--------------|
+| "remember that I prefer Rust over Go" | Stores as `preference` |
+| "remember that JarvisChat runs on port 8080" | Stores as `infrastructure` |
+| "note that the deadline is Friday" | Stores as `general` |
+| "forget about the deadline" | Removes matching memories |
 
----
+Memories are automatically searched based on your message content and injected into the system prompt when relevant.
+
+### Memory Topics
+
+Memories are auto-categorized:
+- `preference` — likes, dislikes, choices
+- `project` — active work, repos, tasks
+- `infrastructure` — servers, services, configs
+- `personal` — name, location, background
+- `general` — everything else
 
 ## API Endpoints
 
-### Auth
-| Method | Path | Description |
-|--------|------|-------------|
-| POST | `/api/auth/guest` | Create guest session |
-| POST | `/api/auth/login` | Admin PIN login |
-| POST | `/api/auth/logout` | Revoke session |
-| GET | `/api/auth/session` | Check session status |
-| POST | `/api/auth/heartbeat` | Keep session alive |
+### Completions (OpenAI-compatible)
+
+| Method | Endpoint | Description |
+|--------|----------|-------------|
+| POST | `/v1/chat/completions` | OpenAI-compatible chat (requires Bearer API key) |
 
 ### Chat & Search
-| Method | Path | Description |
-|--------|------|-------------|
-| POST | `/api/chat` | Streaming chat (SSE) |
-| POST | `/api/search` | Explicit web search via SearXNG |
-| GET | `/api/search/status` | SearXNG health check |
 
-### Models
-| Method | Path | Description |
-|--------|------|-------------|
-| GET | `/api/models` | List available models from llama-server |
-| GET | `/api/ps` | Running models |
-| POST | `/api/show` | Model info |
+| Method | Endpoint | Description |
+|--------|----------|-------------|
+| POST | `/api/chat` | Send message (streaming SSE) |
+| POST | `/api/search` | Explicit web search (streaming SSE) |
+
+### File Upload & Ingest
+
+| Method | Endpoint | Description |
+|--------|----------|-------------|
+| POST | `/api/upload` | Upload file (multipart, admin) |
+| DELETE | `/api/upload/{id}` | Delete upload (admin) |
+| PATCH | `/api/upload/{id}/link` | Link upload to conversation (admin) |
+| GET | `/api/upload/by-conversation/{id}` | List uploads for conversation |
+| POST | `/api/ingest` | Ingest text into RAG (Bearer token auth) |
 
 ### Memory
-| Method | Path | Description |
-|--------|------|-------------|
+
+| Method | Endpoint | Description |
+|--------|----------|-------------|
 | GET | `/api/memories` | List all memories |
 | POST | `/api/memories` | Add memory |
 | PUT | `/api/memories/{rowid}` | Update memory |
 | DELETE | `/api/memories/{rowid}` | Delete memory |
-| GET | `/api/memories/search?q=` | FTS5 search memories |
-| GET | `/api/memories/stats` | Memory statistics |
+| GET | `/api/memories/search?q=term` | Search memories |
+| GET | `/api/memories/stats` | Get counts by topic |
+
+### Models & System
+
+| Method | Endpoint | Description |
+|--------|----------|-------------|
+| GET | `/api/models` | List available models |
+| GET | `/api/ps` | List loaded models |
+| POST | `/api/show` | Get model info |
+| GET | `/api/stats` | CPU, RAM, GPU, VRAM stats |
+| GET | `/api/search/status` | SearXNG availability |
+
+### Settings & Profile
+
+| Method | Endpoint | Description |
+|--------|----------|-------------|
+| GET | `/api/profile` | Get profile content |
+| PUT | `/api/profile` | Update profile (admin) |
+| GET | `/api/profile/default` | Get default profile |
+| GET | `/api/settings` | Get settings |
+| PUT | `/api/settings` | Update settings (admin) |
 
 ### Conversations
-| Method | Path | Description |
-|--------|------|-------------|
+
+| Method | Endpoint | Description |
+|--------|----------|-------------|
 | GET | `/api/conversations` | List conversations |
 | POST | `/api/conversations` | Create conversation |
-| GET | `/api/conversations/{id}` | Get conversation + messages |
-| PUT | `/api/conversations/{id}` | Update title/model |
+| GET | `/api/conversations/{id}` | Get conversation with messages |
+| PUT | `/api/conversations/{id}` | Update conversation title/model |
 | DELETE | `/api/conversations/{id}` | Delete conversation |
-| DELETE | `/api/conversations` | Delete all conversations |
+| DELETE | `/api/conversations` | Delete ALL conversations |
 
-### Profile & Settings
-| Method | Path | Description |
-|--------|------|-------------|
-| GET | `/api/profile` | Get profile |
-| PUT | `/api/profile` | Update profile |
-| GET | `/api/settings` | Get settings |
-| PUT | `/api/settings` | Update settings |
-| GET | `/api/stats` | CPU/RAM/GPU stats |
+### Presets
+
+| Method | Endpoint | Description |
+|--------|----------|-------------|
+| GET | `/api/presets` | List presets |
+| POST | `/api/presets` | Create preset |
+| PUT | `/api/presets/{id}` | Update preset |
+| DELETE | `/api/presets/{id}` | Delete preset |
 
 ### Skills
-| Method | Path | Description |
-|--------|------|-------------|
-| GET | `/api/skills` | List all skills |
-| GET | `/api/skills/active` | List enabled skills |
-| PUT | `/api/skills/{key}` | Enable/disable skill |
 
----
+| Method | Endpoint | Description |
+|--------|----------|-------------|
+| GET | `/api/skills` | List all skills with state |
+| GET | `/api/skills/active` | List active skills |
+| PUT | `/api/skills/{key}` | Toggle skill enabled (admin) |
 
-## Memory Commands
+### Auth
 
-Say these in chat to interact with the memory system:
+| Method | Endpoint | Description |
+|--------|----------|-------------|
+| POST | `/api/auth/guest` | Create guest session |
+| POST | `/api/auth/login` | Admin PIN login |
+| POST | `/api/auth/logout` | Revoke session |
+| GET | `/api/auth/session` | Check session validity |
+| POST | `/api/auth/heartbeat` | Extend session TTL |
 
-| Command | Effect |
-|---------|--------|
-| `remember that [fact]` | Stores fact in FTS5 memory |
-| `please remember [fact]` | Same |
-| `don't forget [fact]` | Same |
-| `forget about [topic]` | Deletes matching memories |
+## Configuration
 
----
+Settings are stored in the `settings` table and include:
 
-## Troubleshooting
+- `profile_enabled` — Inject profile into chats (true/false)
+- `search_enabled` — Auto web search (true/false)
+- `memory_enabled` — Memory injection (true/false)
+- `skills_enabled` — Skills framework (true/false)
+- `default_model` — Default inference model
+
+## Testing
 
-### jC starts but inference is slow or failing
-Check that llama-rpc is running on jarvis and llama-server is connected:
 ```bash
-# On jarvis
-systemctl status llama-rpc
-
-# On ultron — look for "offloaded N/N layers to GPU" in logs
-journalctl -u llama-server -n 50 --no-pager
+./venv/bin/python -m pytest tests/ -v
 ```
 
-### ultron shows no CPU activity during inference
-Inference is being handled entirely by jarvis GPU via RPC — this is correct and expected. ultron's CPU is only involved for non-offloaded tensors (a small fraction of the model).
-
-### RAG not returning results
-Check Qdrant is up and the collection exists:
-```bash
-curl http://192.168.50.108:6333/collections/jarvis_rag
-```
-Verify `points_count` > 0. If zero, the corpus hasn't been seeded yet.
-
-### jC won't start — PIN bootstrap error
-Set the PIN via environment before first boot:
-```bash
-export JARVISCHAT_ADMIN_PIN=XXXX
-systemctl restart jarvischat
-```
-
-### sqlite3 not found
-Use Python instead:
-```bash
-python3 -c "import sqlite3; print(sqlite3.connect('/opt/jarvischat/jarvischat.db').execute('SELECT * FROM settings').fetchall())"
-```
-
----
-
-## Roadmap
-
-### TODO (Priority Order)
-1. **Tool calling** — read_file/write_file with /opt/jarvischat whitelist, tool_calls dispatch loop
-2. **git_tool** — Gitea integration for commit/push from jC
-3. **Audit logging** — structured audit trail to syslog
-4. SearXNG persistence (DONE ✅)
-5. search+ prefix for explicit search
-6. profile.example.md
-7. Conversation search/filter
-8. Export to markdown
-9. Keyboard shortcuts
-10. Retry button
-11. Source links in responses
-12. Rename conversations
-13. Multiple profiles
-14. KWIC auto-tags
-15. Image input (vision)
-16. btop split-screen integration
-17. Containerize
-18. SearXNG health indicator in UI
-19. check_patch_notes tool
-20. GitLab mirror of llgit repo
-
-### ROADMAP (Longer Horizon)
-
-**(A) Modular refactor** — Split monolithic app.py into routers/, services/, config.py, db.py, auth.py. Prerequisite for everything below.
-
-**(B) RAG ingest/manage UI** — File upload, URL ingest (fetch + strip HTML via beautifulsoup4/httpx, store URL as source metadata for citation), delete chunks/collections.
-
-**(C) Backend config panel** — Switch between Ollama/llama-server, endpoint URLs, model switching, restart — all from the UI without touching config files.
-
-**(D) Response metrics display** — tokens/sec, TTFT, context size, RAG chunks retrieved + scores — visible in the UI per response.
-
-**(E) Response quality feedback** — thumbs/stars/tags per response → feedback corpus → future RLHF dataset.
-
-**(F) IDE integration** — Continue.dev + VS Code, pointed at jC:8080 (not direct to inference endpoint). All IDE traffic — including pair-programming conversations — goes through jC so sessions are persisted and become RAG-worthy content. jC needs FIM request format handling to support inline autocomplete.
-
-**(G) Conversation history export → RAG ingest** — Bulk ingest existing conversation history into Qdrant.
-
-**(H) Fine-tuning pipeline** — LoRA on Mistral-Nemo from feedback corpus (item E).
-
-**(I) Autonomous RAG** — At conversation end, jC self-evaluates the transcript, extracts significant chunks (solved problems, working commands, architectural decisions), and ingests them into Qdrant automatically with metadata (date, conversation_id, reason). jC decides what it needs to remember. Closes the loop.
-
-**(J) Startup hardware/resource self-assessment** — On boot, jC queries ultron for available RAM, Qdrant consumption, and llama-server footprint. Derives dynamic high-water marks for RAG chunk limits, context window sizing, retrieval limits, and eviction thresholds. Writes a living config file. Replaces magic numbers with runtime-negotiated values.
-
-**(K) RAG corpus management** — Weighted LRU eviction with composite score (recency + frequency + content age) + manual pin flag for load-bearing knowledge. Prevents corpus bloat from degrading retrieval quality. Analogous to memcache eviction policy.
-
-**(L) Dual inference model architecture** — Mistral-Nemo-12B on ultron:8081 (general assistant), Qwen2.5-Coder-14B-Q5_K_M on ultron:8082 (code/pair programming). jC selects endpoint based on active model. Only one model hot at a time given ultron's 16GB RAM constraint.
-
----
-
-## Primary Cluster Objectives
-
-1. **Generative AI inference** — Local, private, fast enough to be useful
-2. **Agentic functionality** — Autonomous RAG self-management is the canonical first example. The system acts, not just responds.
-
----
-
-## Repository
-
-```
-ssh://gitea@llgit.llamachile.tube:1319/gramps/jarvisChat.git
-```
-
-> SSH username is `gitea`, not `git`. Port 1319.
-
----
+All 110 tests use `tmp_path` fixtures + monkeypatched `httpx.AsyncClient`. No external services needed.
 
 ## License
 
 MIT
+
+## Repository
+
+Gitea: `ssh://gitea@llgit.llamachile.tube:1319/gramps/jarvisChat.git`

TASKS.md

@@ -0,0 +1,727 @@
+# jarvisChat — OpenCode Prompt Sequence
+# Generated: 2026-07-01
+# Execute sequentially. Run full test suite after each task before proceeding.
+# Test command: ./venv/bin/python -m pytest tests/ -v
+
+---
+
+## ~~TASK 1 — README Cleanup [DONE]~~
+
+Review README.md in the current repo. Remove any node references other than `ultron` (192.168.50.108) and `jarvis` (192.168.50.210). Ensure all references to the project use the exact casing `jarvisChat` — not `Jarvischat`, `JarvisChat`, or `jarvischat`. Do not change any functional content, endpoint documentation, or architecture descriptions — this is a text cleanup only. After editing, verify the file renders cleanly as markdown. Commit with message: `docs: clean up node references and branding consistency`.
+
+No new tests required for this task.
+
+---
+
+## ~~TASK 2 — Qwen2.5-Coder llama-server Service on Ultron (Infrastructure) [DONE]~~
+
+**Status: Systemd unit created, verified, and restored.**
+
+This task originally defined creation of `/etc/systemd/system/llama-server-coder.service` (port 8082, Qwen2.5-Coder-14B Q5_K_M) as a prerequisite for dynamic model swapping. That sysadmin work is done.
+
+**The real Task 2 deliverable — the ability to dynamically swap models based on query classification — is delivered by Roadmap N (Tasks 9–15).** The flow:
+
+1. **Task 13** — Phi-4-mini triage (`triage.py`) classifies the query as `general`, `code`, `search`, or `rag`
+2. **Task 13** — `select_node()` picks the best worker node; if the ideal model isn't active, it triggers a swap
+3. **Task 14** — `request_model_swap()` publishes `cmd.swap_model` via AMQP `jc.admin` exchange
+4. **Task 12** — The node agent on jarvis receives the command, stops the current llama-server, starts the correct one, waits for health, and publishes `model_ready`
+5. **Task 14** — ultron receives `model_ready`, updates the cluster registry, and routes the query to the node
+
+The swap is async and transparent — the user sees only latency. The UI (Task 15) shows a yellow "swapping" status dot during the transition.
+
+The service unit at `/etc/systemd/system/llama-server-coder.service` is the **target** the node agent starts when swapping to code inference. It is not enabled at boot — the AMQP cluster manages activation.
+
+See Tasks 9–15 for the actual model swap implementation.
+
+No pytest tests required for this infrastructure task.
+
+---
+
+## ~~TASK 3 — Update OpenCode Config to Use Qwen on :8082 [DONE]~~
+
+Update `/home/gramps/.config/opencode/opencode.jsonc` (on this machine, ultron) to point the configured provider at `http://127.0.0.1:8082/v1` instead of `http://127.0.0.1:8081/v1`. The model name in the config should be updated to reflect `qwen2.5-coder-14b` or whatever model ID the llama-server instance at :8082 reports via `/v1/models`. Verify the endpoint is reachable before writing the config change. Do not restart OpenCode — the config change takes effect on next session start.
+
+No pytest tests required for this task.
+
+---
+
+## ~~TASK 4 — File/Document Attachment: Backend Ingest Endpoint [DONE]~~
+
+**Status: `POST /api/upload` with mode=(context|ingest|both), PDF/text extraction, Qdrant upsert, SQLite context (1hr expiry). Committed `4a891c8` (v1.9.0).**
+
+This task implements the backend half of file/document attachment (TODO #21). The goal is dual-aspect upload: a file can be used as immediate chat context, ingested into the RAG corpus (Qdrant), or both.
+
+**Add to `config.py`:**
+- `UPLOAD_DIR` — path for temporary upload storage, default `/tmp/jarvischat_uploads`
+- `MAX_UPLOAD_BYTES` — max file size, default 20MB
+- `SUPPORTED_UPLOAD_TYPES` — set of MIME types: `text/plain`, `text/markdown`, `application/pdf`, `application/json`, `text/x-python`, `text/html`
+
+**Create `routers/upload.py`:**
+
+Implement `POST /api/upload` (admin required). Accept `multipart/form-data` with:
+- `file` — the uploaded file (required)
+- `mode` — string enum: `context` (inject into next chat only), `ingest` (add to RAG corpus), `both` (default: `both`)
+- `conversation_id` — optional, associates context-mode content with a specific conversation
+
+Behavior:
+- Validate file size against `MAX_UPLOAD_BYTES` — return 413 if exceeded
+- Validate MIME type against `SUPPORTED_UPLOAD_TYPES` — return 415 if unsupported
+- For PDF files, extract text using `pypdf` (add to requirements.txt)
+- For all other types, read as UTF-8 text
+- If mode includes `ingest`: chunk the extracted text into 512-token overlapping chunks (128-token overlap), generate embeddings via `EMBED_URL` (http://192.168.50.108:11434/api/embeddings, model mxbai-embed-large), upsert into Qdrant collection `jarvischat` with metadata `{source: filename, upload_date: iso_timestamp, type: "upload"}`
+- If mode includes `context`: store the full extracted text in a new SQLite table `upload_context` with columns `(id INTEGER PRIMARY KEY, conversation_id TEXT, filename TEXT, content TEXT, created_at TEXT, expires_at TEXT)`. Context entries expire after 1 hour.
+- Return JSON: `{filename, size_bytes, mode, chunks_ingested (if ingest), context_id (if context), message}`
+
+**Add `upload_context` table to `db.py`** `init_db()`.
+
+**Wire `upload.router` into `app.py`** in the router registration block.
+
+**Write `tests/test_upload.py`** covering:
+- Valid text file upload, mode=ingest — assert chunks_ingested > 0, Qdrant upsert called
+- Valid text file upload, mode=context — assert context_id returned, row exists in upload_context
+- Valid text file upload, mode=both — assert both behaviors
+- File exceeds MAX_UPLOAD_BYTES — assert 413
+- Unsupported MIME type — assert 415
+- Guest session attempt — assert 403
+- PDF extraction path — mock pypdf, assert text extracted and processed
+
+Mock Qdrant and EMBED_URL calls via monkeypatch. Do not require live external services in tests.
+
+Run full test suite after implementation. All 26 existing tests must continue to pass.
+
+---
+
+## ~~TASK 5 — File/Document Attachment: UI Integration [DONE]~~
+
+**Status: Paperclip icon, file preview pill, gallery overlay, attachment indicators, DELETE/PATCH link/by-conversation endpoints, chat context injection. Committed `81238c0` (v1.10.0).**
+
+This task implements the frontend half of TODO #21. The UI is a single file at `templates/index.html`.
+
+Add a file attachment button to the chat input area. Requirements:
+- Paperclip icon button adjacent to the send button
+- Clicking opens a file picker filtered to supported types (`.txt`, `.md`, `.pdf`, `.json`, `.py`, `.html`)
+- On file selection, show a pill/badge above the input showing the filename with an X to remove it
+- On send, if a file is attached: POST to `/api/upload` with `mode=both` and the current `conversation_id`, then include the returned `context_id` in the subsequent `/api/chat` POST body as `upload_context_id`
+- If the upload fails, show an inline error and do not send the chat message
+- File attachment state clears after send
+
+**Update `/api/chat` in `routers/chat.py`:**
+- Accept optional `upload_context_id` in the request body
+- If present, look up the content in `upload_context` table and prepend it to the system prompt as: `\n\n[ATTACHED DOCUMENT: {filename}]\n{content}\n[END DOCUMENT]`
+- If the context_id is expired or missing, log a warning and continue without it (do not error)
+
+**Add to `tests/test_chat_streaming_and_memory_paths.py`:**
+- Test that a valid `upload_context_id` results in document content being prepended to the system prompt
+- Test that an expired/missing `upload_context_id` is silently ignored
+
+Run full test suite. All existing tests must continue to pass.
+
+---
+
+## ~~TASK 6 — Roadmap I: Terminal Command RAG Hook [DONE]~~
+
+**Status: `POST /api/ingest` with Bearer token auth, `chunk_text()` shared helper, `jc-ingest.sh` script. Committed `1ac21ad` (v0.11.0).**
+
+This task implements autonomous RAG ingestion of significant terminal activity (TODO #23).
+
+**Create `routers/ingest.py`:**
+
+Implement `POST /api/ingest` (requires Bearer token auth — use same `COMPLETIONS_API_KEY` mechanism as `routers/completions.py`). Accept JSON body:
+- `content` — string, the text to ingest (required)
+- `source` — string, origin label e.g. `terminal`, `file`, `external` (default: `external`)
+- `metadata` — optional dict of additional key/value pairs
+
+Behavior:
+- Chunk `content` into 512-token overlapping chunks (128-token overlap) — extract this logic into a shared helper `chunk_text(text, chunk_size=512, overlap=128)` in `rag.py` if not already present
+- Generate embeddings via `EMBED_URL`
+- Upsert into Qdrant collection `jarvischat` with metadata `{source, ingest_date: iso_timestamp, ...metadata}`
+- Return JSON: `{chunks_ingested, source, message}`
+
+**Wire `ingest.router` into `app.py`.**
+
+**Create `/home/gramps/bin/jc-ingest.sh` on jarvis (192.168.50.210)** — this is a shell script, not a Python file, and lives outside the repo. Write it to stdout/document it clearly so gramps can deploy it manually:
+
+```bash
+#!/bin/bash
+# jc-ingest.sh — pipe terminal commands into jarvisChat RAG
+# Add to ~/.bashrc: export PROMPT_COMMAND="jc_capture"
+# Function to call after significant commands
+
+JC_URL="http://192.168.50.210:8080/api/ingest"
+JC_TOKEN="${JARVISCHAT_COMPLETIONS_API_KEY}"
+
+jc_capture() {
+    local cmd
+    cmd=$(history 1 | sed 's/^[ ]*[0-9]*[ ]*//')
+    # Only ingest significant commands
+    if echo "$cmd" | grep -qE '^(git|pip|systemctl|sudo|vi|vim|curl|wget|apt|python|pytest)'; then
+        curl -s -X POST "$JC_URL" \
+            -H "Authorization: Bearer $JC_TOKEN" \
+            -H "Content-Type: application/json" \
+            -d "{\"content\": $(echo "$cmd" | python3 -c 'import json,sys; print(json.dumps(sys.stdin.read().strip()))'), \"source\": \"terminal\"}" \
+            > /dev/null 2>&1 &
+    fi
+}
+```
+
+**Write `tests/test_ingest.py`** covering:
+- Valid ingest with content — assert chunks_ingested > 0
+- Missing Bearer token — assert 401
+- Wrong Bearer token — assert 403
+- Empty content — assert 422
+- Qdrant and embed calls mocked via monkeypatch
+
+Run full test suite. All existing tests must continue to pass.
+
+---
+
+## ~~TASK 7 — Roadmap J: Startup Hardware Self-Assessment [DONE]~~
+
+**Status: `hardware.py` + `routers/hardware.py` + 4 tests. Committed `7291b8f` (v0.12.0).**
+
+On jC startup, probe available hardware and write a living config snapshot. This replaces hardcoded assumptions about VRAM and RAM.
+
+**Create `hardware.py`** in the project root:
+
+```
+async def assess_hardware() -> dict
+```
+
+Probes:
+- System RAM: `psutil.virtual_memory().total` and `.available`
+- CPU count: `psutil.cpu_count()`
+- GPU VRAM total and free: call `rocm-smi --showmeminfo vram --json` via subprocess, parse output. If rocm-smi absent or fails, set VRAM values to 0 and log a warning.
+- llama-server reachable: GET `LLAMA_SERVER_BASE/v1/models`, timeout 3s. Record True/False and list of available model IDs.
+- Qdrant reachable: GET `http://192.168.50.108:6333/collections`, timeout 3s. Record True/False and collection list.
+- SearXNG reachable: GET `http://localhost:8888`, timeout 3s. Record True/False.
+
+Returns a dict with all of the above. Writes result as JSON to `hardware_state.json` in the working directory.
+
+**Call `assess_hardware()` from the FastAPI `lifespan` context** in `app.py` on startup, after `init_db()`. Log a summary line: `HW: {ram_gb}GB RAM, {vram_mb}MB VRAM, llama={reachable}, qdrant={reachable}, searxng={reachable}`.
+
+**Expose `GET /api/hardware`** in a new `routers/hardware.py` — returns the current `hardware_state.json` content as JSON. No auth required (read-only, non-sensitive aggregate stats).
+
+**Wire `hardware.router` into `app.py`.**
+
+**Write `tests/test_hardware.py`** covering:
+- `assess_hardware()` with all services reachable (mock subprocess and httpx calls) — assert all fields present
+- `assess_hardware()` with rocm-smi absent — assert VRAM=0, no exception raised
+- `assess_hardware()` with llama-server unreachable — assert `llama_reachable=False`, no exception
+- `GET /api/hardware` — assert returns JSON with expected keys
+
+Run full test suite. All existing tests must continue to pass.
+
+---
+
+## TASK 8 — Roadmap K: RAG Corpus Management
+
+Qdrant collection `jarvischat` currently grows without bound. Implement score-based eviction with hysteresis, pinned sources, operational stats, and a flush command.
+
+### Config — add to `config.py`:
+
+```python
+RAG_MAX_VECTORS = 50000               # absolute ceiling; eviction targets thresholds below it
+RAG_EVICTION_HIGH_WATER = 0.80        # fraction of RAG_MAX_VECTORS that triggers eviction
+RAG_EVICTION_LOW_WATER = 0.20         # fraction where eviction stops
+RAG_EVICTION_BATCH = 1000             # max points to delete per Qdrant scroll/delete cycle
+RAG_PINNED_SOURCES = ["upload", "profile"]  # never evicted
+RAG_GRACE_HOURS = 1                   # new vectors ineligible for eviction until this old
+RAG_ACCESS_WEIGHT = 1.0               # score factor: retrieval_count * ACCESS_WEIGHT
+RAG_AGE_WEIGHT = 0.1                  # score factor: ingest_age_hours * AGE_WEIGHT
+```
+
+Validations on boot: `high_water > low_water`, `batch > 0`, `max_vectors > 0`.
+
+### Eviction algorithm — add to `rag.py`:
+
+```
+score = (retrieval_count * ACCESS_WEIGHT) + (age_hours * AGE_WEIGHT)
+```
+
+Lower score = evicted first. Tiebreak: `last_accessed` ASC (older wins).
+
+```python
+async def get_collection_count() -> int
+    # GET /collections/jarvischat → return vectors_count
+
+async def get_collection_stats() -> dict
+    # Return {vector_count, max_vectors, high_water, low_water, percent_full, pinned_sources}
+
+async def evict_batch(batch_size: int) -> int
+    # Scroll Qdrant for vectors NOT in RAG_PINNED_SOURCES, WHERE ingest_age > RAG_GRACE_HOURS,
+    # ordered by score ASC, last_accessed ASC.
+    # Delete up to batch_size. Return count deleted.
+    # If 0 evictable vectors found: log warning, return 0 (break loop).
+
+async def maybe_evict() -> int
+    # Acquire eviction_lock (asyncio.Lock).
+    # count = get_collection_count()
+    # threshold_high = RAG_MAX_VECTORS * RAG_EVICTION_HIGH_WATER
+    # threshold_low  = RAG_MAX_VECTORS * RAG_EVICTION_LOW_WATER
+    # total_evicted = 0
+    # while count >= threshold_low:
+    #     if total_evicted > 0 and count < threshold_low: break
+    #     deleted = evict_batch(RAG_EVICTION_BATCH)
+    #     if deleted == 0: break  # no more unpinned targets
+    #     total_evicted += deleted
+    #     count -= deleted
+    #     if count < threshold_high and total_evicted > 0: break
+    #     # only one pass if batch spans the full gap
+    #     if count < threshold_low: break
+    # Record total_evicted + timestamp in EVICTION_LOG (list of dicts, kept in memory, max 1000 entries)
+    # Release lock. Return total_evicted.
+
+async def get_rag_operational_stats() -> dict
+    # Returns: vector_count, max_vectors, high_water_pct, low_water_pct,
+    # percent_full, pinned_sources, grace_hours,
+    # eviction_counts_last_1m, eviction_counts_last_5m, eviction_counts_last_30m,
+    # at_risk_count (vectors in bottom 10% by score),
+    # pinned_count, avg_retrieval_count
+```
+
+### Edge cases & guards:
+
+1. **Newborn grace** — vectors < `RAG_GRACE_HOURS` old are excluded from eviction scroll (score=0 otherwise → immediate deletion)
+2. **All-pinned freeze** — if scroll returns 0 evictable vectors, log warning and break loop
+3. **Race** — `asyncio.Lock()` guards `maybe_evict()`; concurrent callers wait their turn
+4. **Zero config** — `RAG_MAX_VECTORS <= 0` → eviction disabled; `RAG_EVICTION_BATCH <= 0` → clamped to 1
+5. **Legacy payloads** — vectors without `retrieval_count` or `last_accessed` get defaults (0, `ingest_date`)
+
+### Wire eviction:
+
+Call `maybe_evict()` after each upsert batch completes in:
+- `routers/upload.py` — after Qdrant upsert
+- `routers/ingest.py` — after Qdrant upsert
+
+### Admin endpoints — new `routers/rag_admin.py`:
+
+| Method | Endpoint | Description |
+|--------|----------|-------------|
+| GET | `/api/rag/stats` | Operational stats (see `get_rag_operational_stats()`) — admin required |
+| POST | `/api/rag/flush` | Delete ALL points from the Qdrant `jarvischat` collection. Returns `{deleted_count, collection: "jarvischat", status: "flushed"}`. Admin required. |
+
+### In-memory eviction log:
+
+```python
+EVICTION_LOG: list[dict] = []  # managed by rag.py, max 1000 entries
+# Each entry: {timestamp: iso, count: N, remaining: N}
+# Tied to RATE_EVENTS pattern from security.py for rolling window calculations
+```
+
+### Tests — `tests/test_rag_management.py`:
+
+- `get_collection_count()` — mock Qdrant GET, assert correct count
+- `get_collection_stats()` — assert shape matches config
+- `evict_batch()` — mock Qdrant scroll + delete, assert pinned sources excluded, grace period enforced, batch size respected
+- `maybe_evict()` — below high water: 0 evicted; at high water: eviction fires; stops at low water; all-pinned scroll returns 0 → breaks
+- `GET /api/rag/stats` — assert full shape
+- `POST /api/rag/flush` — assert points deleted, admin required, guest 403
+- `POST /api/rag/flush` by guest — assert 403
+- Race lock — concurrent calls to `maybe_evict()` queue up, only one evicts
+
+Mock all Qdrant calls via monkeypatch. Do not require live services.
+
+Run full test suite. All existing tests must continue to pass.
+
+---
+
+## TASK 9 — Roadmap N1: RabbitMQ Install and Service on Ultron (Infrastructure)
+
+This task runs on ultron (this machine). Install RabbitMQ and verify it is operational.
+
+Run the following steps:
+1. `apt-get update && apt-get install -y rabbitmq-server`
+2. `systemctl enable rabbitmq-server && systemctl start rabbitmq-server`
+3. `systemctl status rabbitmq-server` — verify active/running
+4. Enable the management plugin: `rabbitmq-plugins enable rabbitmq_management`
+5. Create a dedicated jC vhost: `rabbitmqctl add_vhost jarvischat`
+6. Create a dedicated user: `rabbitmqctl add_user jarvischat CHANGEME_PASSWORD` — generate a random 24-char alphanumeric password and record it
+7. Grant permissions: `rabbitmqctl set_permissions -p jarvischat jarvischat ".*" ".*" ".*"`
+8. Verify management UI is reachable: `curl -s -u guest:guest http://localhost:15672/api/overview | python3 -m json.tool`
+9. Delete default guest user: `rabbitmqctl delete_user guest`
+
+Declare the two topic exchanges needed by jC:
+- Exchange name: `jc.admin`, type: `topic`, durable: true
+- Exchange name: `jc.system`, type: `topic`, durable: true
+
+Use `rabbitmqadmin` or `curl` against the management API to declare exchanges. Verify both exchanges appear in: `curl -s -u jarvischat:{password} http://localhost:15672/api/exchanges/jarvischat`
+
+Write the generated RabbitMQ password to `/home/gramps/.jc_amqp_secret` with mode 600. This will be read by jC as an env var source in subsequent tasks.
+
+No pytest tests required for this infrastructure task.
+
+---
+
+## TASK 10 — Roadmap N2: AMQP Connection Layer in jC
+
+This task adds the core AMQP connection manager to jC. It must connect to RabbitMQ on ultron (localhost from jC's perspective since jC runs on ultron), handle reconnection, and provide a shared channel for all AMQP operations.
+
+**Add to `requirements.txt`:** `aio-pika>=9.0.0`
+
+**Add to `config.py`:**
+- `AMQP_URL` — read from env `JARVISCHAT_AMQP_URL`, default `amqp://jarvischat:password@localhost:5672/jarvischat`. The actual password comes from `/home/gramps/.jc_amqp_secret` — read it at startup if the env var is not set.
+- `AMQP_RECONNECT_DELAY` — seconds between reconnect attempts, default 5
+- `AMQP_EXCHANGE_ADMIN` — `jc.admin`
+- `AMQP_EXCHANGE_SYSTEM` — `jc.system`
+
+**Create `amqp.py`** in the project root:
+
+```python
+# Manages a single persistent aio-pika connection and channel.
+# Provides:
+#   connect() -> None          # establish connection, declare exchanges
+#   disconnect() -> None       # graceful close
+#   get_channel()              # returns current channel, reconnects if needed
+#   publish(exchange, routing_key, payload: dict) -> None
+#                              # publishes JSON-serialized payload as persistent message
+```
+
+Connection must:
+- Reconnect automatically on disconnect with `AMQP_RECONNECT_DELAY` backoff
+- Log connection events at INFO level
+- Not raise on publish if disconnected — log error and return (fire-and-forget, jC must not crash if RabbitMQ is down)
+
+**Start AMQP connection in `app.py` lifespan** after `assess_hardware()`. Disconnect in lifespan cleanup.
+
+**Write `tests/test_amqp.py`** covering:
+- `publish()` with mocked aio-pika connection — assert message published with correct exchange and routing key
+- `publish()` when disconnected — assert no exception raised, error logged
+- `get_channel()` when connection is None — assert reconnect attempted
+
+Mock all aio-pika calls via monkeypatch. Do not require a live RabbitMQ instance in tests.
+
+Run full test suite. All existing tests must continue to pass.
+
+---
+
+## TASK 11 — Roadmap N3: Worker Node Registration Handler (Ultron/jC Side)
+
+jC on ultron must listen on the `jc.admin` exchange for worker node registration requests and respond with admission or rejection.
+
+**Add to `amqp.py`:**
+
+```python
+async def subscribe(exchange, routing_key, callback) -> None
+    # Declare a queue, bind to exchange/routing_key, consume with callback
+```
+
+**Create `cluster.py`** in the project root:
+
+```python
+# In-memory cluster registry (survives only while jC is running)
+# Structure:
+# CLUSTER_NODES: dict[str, NodeRecord]
+#
+# NodeRecord fields:
+#   node_name: str
+#   ip: str
+#   active_model: ModelRecord
+#   inventory: list[ModelRecord]
+#   registered_at: str  (ISO timestamp)
+#   last_seen: str      (ISO timestamp)
+#
+# ModelRecord fields:
+#   name: str
+#   version: str
+#   quant: str
+#   path: str
+#   port: int           (llama-server port this model is served on)
+
+async def handle_registration(message: aio_pika.IncomingMessage) -> None
+    # Parse JSON payload from message body
+    # Validate required fields: node_name, ip, active_model, inventory
+    # Reject if node_name already in CLUSTER_NODES with status="active":
+    #   publish to jc.admin routing_key=f"node.{node_name}.rejected"
+    #   payload: {node_name, reason: "duplicate_node_name", timestamp}
+    # Reject if payload malformed:
+    #   publish to jc.admin routing_key=f"node.{node_name}.rejected"
+    #   payload: {node_name, reason: "malformed_payload", timestamp}
+    # Otherwise admit:
+    #   add to CLUSTER_NODES
+    #   publish to jc.admin routing_key=f"node.{node_name}.admitted"
+    #   payload: {node_name, timestamp, amqp_url: AMQP_URL}
+
+async def handle_deregistration(message) -> None
+    # Remove node from CLUSTER_NODES, log it
+
+def get_cluster_state() -> dict
+    # Return serializable snapshot of CLUSTER_NODES
+```
+
+**Subscribe to registration messages in `app.py` lifespan** after AMQP connects:
+- `jc.admin` exchange, routing key `node.*.register` → `handle_registration`
+- `jc.admin` exchange, routing key `node.*.deregister` → `handle_deregistration`
+
+**Add `GET /api/cluster`** to a new `routers/cluster.py`:
+- Returns `get_cluster_state()` as JSON
+- No auth required (read-only status endpoint)
+
+**Wire `cluster.router` into `app.py`.**
+
+**Write `tests/test_cluster.py`** covering:
+- Valid registration payload — assert node admitted, added to CLUSTER_NODES, admitted message published
+- Duplicate node name — assert rejected, reason=`duplicate_node_name`
+- Malformed payload (missing required field) — assert rejected, reason=`malformed_payload`
+- Deregistration — assert node removed from CLUSTER_NODES
+- `GET /api/cluster` — assert returns current node list
+
+Mock all aio-pika calls. Do not require live RabbitMQ.
+
+Run full test suite. All existing tests must continue to pass.
+
+---
+
+## TASK 12 — Roadmap N4: Worker Node Registration Publisher (Jarvis Side)
+
+This task creates the worker node AMQP client that runs on jarvis (192.168.50.210). It is a standalone Python script — not part of the jC FastAPI app — that runs as a systemd service on jarvis.
+
+**Create `node_agent/agent.py`** in the repo (new directory):
+
+The agent:
+1. On start: reads local config from `/etc/jc-node-agent.conf` (INI format):
+   - `node_name` — hostname, default from `socket.gethostname()`
+   - `node_ip` — LAN IP, default from socket
+   - `amqp_url` — RabbitMQ URL on ultron, e.g. `amqp://jarvischat:password@192.168.50.108:5672/jarvischat`
+   - `llama_port` — port llama-server/llama-rpc is listening on, default 8081
+   - `models_dir` — path to GGUF model files, default `/home/gramps/models`
+   - `active_model` — filename of currently active model (without path)
+
+2. Discovers inventory by globbing `models_dir` for `*.gguf` files and parsing name/version/quant from filename using regex pattern: `{name}-{version}-{quant}.gguf` where quant matches `Q[0-9]+_K_[A-Z]+` or similar standard suffixes.
+
+3. Publishes registration request to `jc.admin` exchange, routing key `node.{node_name}.register`:
+   ```json
+   {
+     "node_name": "jarvis",
+     "ip": "192.168.50.210",
+     "active_model": {"name": "...", "version": "...", "quant": "...", "path": "...", "port": 8081},
+     "inventory": [...]
+   }
+   ```
+
+4. Listens for response on `jc.admin`, routing key `node.{node_name}.admitted` or `node.{node_name}.rejected`. Logs result. If rejected, exits with error.
+
+5. After admission: publishes heartbeat every 30 seconds to `jc.system`, routing key `node.{node_name}.heartbeat`:
+   ```json
+   {"node_name": "...", "ip": "...", "active_model": "...", "timestamp": "..."}
+   ```
+
+6. Listens on `jc.admin`, routing key `node.{node_name}.cmd.swap_model`:
+   - Payload: `{model_filename: str}`
+   - Stops current llama-server: `systemctl stop llama-server`
+   - Updates `/etc/jc-node-agent.conf` active_model field
+   - Starts llama-server: `systemctl start llama-server` (assumes service reads active_model from conf or ExecStart is updated)
+   - Waits for llama-server to be healthy: poll `http://localhost:{llama_port}/v1/models` every 2s, timeout 120s
+   - Publishes to `jc.system`, routing key `node.{node_name}.model_ready`:
+     ```json
+     {"node_name": "...", "active_model": "...", "port": ..., "timestamp": "..."}
+     ```
+   - If startup fails within timeout: publishes `node.{node_name}.model_failed` with error detail
+
+**Create `node_agent/requirements.txt`:** `aio-pika>=9.0.0`
+
+**Document `/etc/jc-node-agent.conf` format** in a comment block at the top of `agent.py`.
+
+**Write `tests/test_node_agent.py`** covering:
+- Registration payload construction from config + model discovery — assert correct JSON shape
+- Model swap command handler: success path — assert systemctl calls made, model_ready published
+- Model swap command handler: timeout path — assert model_failed published
+- Heartbeat: assert published every interval (mock asyncio.sleep)
+
+Mock all aio-pika, subprocess, and httpx calls.
+
+**Do not create a systemd service file in this task** — that is a manual deployment step. Document the required service configuration in a comment at the bottom of `agent.py`.
+
+Run full test suite. All existing tests must continue to pass.
+
+---
+
+## TASK 13 — Roadmap N5: Query Routing via AMQP + Phi-4-mini Triage
+
+This task wires the cluster into jC's chat flow. When a query arrives at `/api/chat`, instead of always routing to the hardcoded `LLAMA_SERVER_BASE`, jC now routes to the best available cluster node based on query context.
+
+**Prerequisites:** Tasks 9–12 complete. At least one worker node admitted to cluster.
+
+**Install Phi-4-mini on ultron (infrastructure step):**
+- Download `Phi-4-mini-Instruct-Q4_K_M.gguf` from HuggingFace using `hf download microsoft/Phi-4-mini-instruct --include "*.Q4_K_M.gguf" --local-dir /home/gramps/models`
+- Create `/etc/systemd/system/llama-server-triage.service` — same pattern as existing llama-server service but: port 8083, model path points to Phi-4-mini GGUF, no `--rpc` flag (runs entirely on ultron CPU/iGPU), description `Llama.cpp Server (Phi-4-mini — triage/routing)`
+- `systemctl daemon-reload && systemctl enable llama-server-triage && systemctl start llama-server-triage`
+- Verify: `curl -s http://localhost:8083/v1/models`
+
+**Add to `config.py`:**
+- `TRIAGE_BASE` — `http://127.0.0.1:8083/v1` (Phi-4-mini)
+- `TRIAGE_TIMEOUT` — 10 seconds
+- `FALLBACK_TO_DEFAULT` — True (if triage fails or no nodes available, fall back to `LLAMA_SERVER_BASE`)
+
+**Create `triage.py`** in the project root:
+
+```python
+async def classify_query(query: str) -> str
+    # Sends query to Phi-4-mini at TRIAGE_BASE with a classification system prompt.
+    # System prompt instructs model to respond with ONLY one of:
+    #   "general", "code", "search", "rag"
+    # Returns the classification string.
+    # Timeout: TRIAGE_TIMEOUT seconds.
+    # On any error: returns "general" (fail-safe).
+
+async def select_node(classification: str) -> dict | None
+    # Consults CLUSTER_NODES from cluster.py
+    # For "code": prefer nodes where active_model name contains "coder" or "qwen"
+    # For "general": prefer nodes where active_model name contains "mistral" or "llama"
+    # For "search" or "rag": return None (handled locally by jC)
+    # If no matching node found: return None (triggers FALLBACK_TO_DEFAULT)
+    # Returns NodeRecord dict for selected node, or None
+
+async def get_inference_url(query: str) -> str
+    # Combines classify_query + select_node
+    # Returns full base URL: f"http://{node.ip}:{node.active_model.port}/v1"
+    # Falls back to LLAMA_SERVER_BASE if classification=search/rag, no nodes, or triage error
+```
+
+**Update `routers/chat.py`:**
+- Replace the hardcoded `LLAMA_SERVER_BASE` reference with a call to `get_inference_url(user_message)`
+- The rest of the chat flow (RAG, memory, streaming) is unchanged — only the inference target URL changes
+
+**Write `tests/test_triage.py`** covering:
+- `classify_query()` returns valid classification — mock Phi-4-mini response
+- `classify_query()` on timeout — assert returns "general", no exception
+- `select_node("code")` with coder node in cluster — assert correct node returned
+- `select_node("general")` with no matching node — assert None returned
+- `get_inference_url()` with code query and coder node available — assert returns node URL
+- `get_inference_url()` with no nodes in cluster — assert returns LLAMA_SERVER_BASE fallback
+
+**Update `tests/test_chat_streaming_and_memory_paths.py`:**
+- Mock `triage.get_inference_url` to return a fixed URL in all existing tests so they continue to pass without a live cluster
+
+Run full test suite. All existing tests must continue to pass.
+
+---
+
+## TASK 14 — Roadmap N6: Model Swap Command Flow
+
+This task implements the ultron-side logic for requesting a model swap on a worker node when the ideal model is not currently active.
+
+**Add to `cluster.py`:**
+
+```python
+async def request_model_swap(node_name: str, model_filename: str) -> bool
+    # Publishes to jc.admin exchange, routing key node.{node_name}.cmd.swap_model
+    # Payload: {model_filename, requested_at: iso_timestamp}
+    # Sets node status to "swapping" in CLUSTER_NODES
+    # Returns True if message published successfully
+
+async def handle_model_ready(message) -> None
+    # Handles node.{node_name}.model_ready from jc.system
+    # Updates CLUSTER_NODES[node_name].active_model to the new model
+    # Sets node status back to "active"
+    # Logs swap completion with timing
+
+async def handle_model_failed(message) -> None
+    # Handles node.{node_name}.model_failed from jc.system
+    # Sets node status to "error" in CLUSTER_NODES
+    # Logs failure with detail from message payload
+```
+
+**Subscribe in `app.py` lifespan:**
+- `jc.system` exchange, routing key `node.*.model_ready` → `handle_model_ready`
+- `jc.system` exchange, routing key `node.*.model_failed` → `handle_model_failed`
+
+**Update `triage.py` `select_node()`:**
+- If the best-matching node exists but its active_model does not match the ideal model for the classification, AND the node status is "active" (not already swapping):
+  - Call `request_model_swap(node_name, ideal_model_filename)`
+  - Return None (triggers fallback) — the swap happens async, next query will find the right model active
+- If node status is "swapping": return None (fallback, swap in progress)
+
+**Update `GET /api/cluster`** to include node status in response.
+
+**Write `tests/test_model_swap.py`** covering:
+- `request_model_swap()` — assert swap command published, node status set to "swapping"
+- `handle_model_ready()` — assert active_model updated, status set to "active"
+- `handle_model_failed()` — assert status set to "error"
+- `select_node()` with mismatched active model — assert swap requested, None returned
+- `select_node()` with node status "swapping" — assert None returned without publishing another swap
+
+Run full test suite. All existing tests must continue to pass.
+
+---
+
+## TASK 15 — Roadmap N7: Cluster Status UI
+
+Surface cluster awareness in the jC frontend (`templates/index.html`).
+
+**Add a cluster status panel** to the UI. Requirements:
+- Small status bar or collapsible panel, visible but unobtrusive
+- Polls `GET /api/cluster` every 15 seconds
+- For each admitted node: show node name, active model name, and a colored status dot:
+  - Green: active
+  - Yellow: swapping
+  - Red: error or offline (not seen in last 60 seconds based on last_seen timestamp)
+- If no nodes in cluster (empty): show "No worker nodes connected"
+- Panel must not interfere with chat input or conversation list
+
+**Update `GET /api/cluster` response** to include `last_seen` per node and a `status` field (`active`, `swapping`, `error`).
+
+**Update heartbeat handling in `cluster.py`:** add a handler for `node.*.heartbeat` on `jc.system` that updates `last_seen` timestamp for the node.
+
+**Subscribe in `app.py` lifespan:**
+- `jc.system` exchange, routing key `node.*.heartbeat` → `handle_heartbeat`
+
+**Add `handle_heartbeat()` to `cluster.py`:**
+- Updates `CLUSTER_NODES[node_name].last_seen` to current timestamp
+- If node was previously marked offline (not in CLUSTER_NODES), log re-registration warning but do not auto-admit — full registration required
+
+**Write `tests/test_cluster_heartbeat.py`** covering:
+- `handle_heartbeat()` for known node — assert last_seen updated
+- `handle_heartbeat()` for unknown node — assert no crash, warning logged, node not added
+
+Run full test suite. All 26+ existing tests must continue to pass.
+
+Commit all changes introduced across Tasks 9–15 with message: `feat: Roadmap N — AMQP cluster nervous system complete`
+
+---
+
+## Backlog (Post-Roadmap N) ⏳
+
+### B1 — Context loss in follow-up questions
+
+**Symptom:** After asking "in {context}, explain {b}", a follow-up "what is {b}'s {x}?" gets a non-sequitur response that ignores the original context.
+
+**Diagnosis:** `build_system_prompt()` is called fresh per-request with new RAG/memory results keyed to the current message text. These can change between turns and may dilute or override the conversation history. The original system prompt used for turn 1 (including its RAG context) is not stored in the DB — only user/assistant messages are. The inference server receives a different system prompt each turn.
+
+**Possible fixes:**
+- Store the assembled system prompt with each assistant message in the DB
+- When replaying history, re-send the original system prompts from DB rather than rebuilding
+- Or: cap RAG/memory injection to only fire on the first message of a conversation, then rely solely on conversation history for follow-ups
+- Check that llama-server isn't truncating history due to context window overflow (Mistral-Nemo 12B = 128K context, unlikely)
+
+### B2 — Bang-prefixed search routing
+
+**Spec:** If a query begins with `!`, route to SearXNG search instead of local inference.
+
+**Where:** In `routers/chat.py` `chat()` handler, after `user_message` is extracted. Strip the `!`, set a flag to always trigger auto-search regardless of perplexity/refusal.
+
+**Change:** Add a `force_search` flag when `user_message.startswith("!")`, strip the prefix from the message saved to DB, and route directly to the search+summarize path.
+
+### B3 — Docker distribution (v1.0 gate)
+
+**Goal:** Ship jarvisChat as a `docker compose` stack so a single command stands up everything.
+
+**Services to containerize:**
+- jarvisChat (FastAPI app + SQLite)
+- SearXNG
+- Qdrant
+- RabbitMQ
+- llama-server (with optional RPC sidecar for GPU offload)
+- Ollama (embeddings)
+
+**Also needed:**
+- `Dockerfile` for the jarvisChat app itself
+- `docker-compose.yml` with all services, volumes, networks, env vars
+- Setup wizard script (run on first boot) that:
+  - Probes CPU vs GPU (reuses `hardware.py`)
+  - Queries user for admin PIN, node name, IP
+  - Generates `.env` file with correct `LLAMA_SERVER_BASE`, `EMBED_URL`, etc.
+  - Auto-calculates `RAG_MAX_VECTORS` from available RAM: `max(1000, int(available_ram_gb * 100_000))`
+  - Optionally detects and configures RPC GPU offload
+- Manual install docs remain alongside for bare-metal deployment
+
+**This task is only actionable after Tasks 8–15 (RAG eviction + AMQP cluster) are complete.**

app.py

@@ -5,6 +5,7 @@ Creates the FastAPI app, registers middleware, mounts all routers.
 """
 import logging
 import logging.handlers
+import os
 from contextlib import asynccontextmanager
 from pathlib import Path
 
@@ -13,8 +14,9 @@ from fastapi.responses import HTMLResponse, JSONResponse
 from fastapi.staticfiles import StaticFiles
 from fastapi.templating import Jinja2Templates
 
-from config import VERSION, RATE_WINDOW_SECONDS
+from config import VERSION, RATE_WINDOW_SECONDS, UPLOAD_DIR, RAG_MAX_VECTORS, RAG_EVICTION_HIGH_WATER, RAG_EVICTION_LOW_WATER, RAG_EVICTION_BATCH
 from db import init_db
+from hardware import assess_hardware
 from memory import get_memory_count
 from security import (
     get_client_ip, is_ip_allowed, check_rate_limit, rate_policy,
@@ -32,6 +34,10 @@ import routers.skills as skills
 import routers.chat as chat
 import routers.search_route as search_route
 import routers.completions as completions
+import routers.upload as upload
+import routers.ingest as ingest
+import routers.hardware as hardware
+import routers.rag_admin as rag_admin
 
 # --- Logging ---
 log = logging.getLogger("jarvischat")
@@ -47,8 +53,22 @@ templates = Jinja2Templates(directory=str(BASE_DIR / "templates"))
 @asynccontextmanager
 async def lifespan(app: FastAPI):
     log.info(f"JarvisChat {VERSION} starting up")
+    os.makedirs(UPLOAD_DIR, exist_ok=True)
     init_db()
     log.info(f"Memory system: {get_memory_count()} memories loaded")
+    await assess_hardware()
+
+    if RAG_MAX_VECTORS > 0:
+        if RAG_EVICTION_HIGH_WATER <= RAG_EVICTION_LOW_WATER:
+            log.warning(
+                f"RAG_EVICTION_HIGH_WATER={RAG_EVICTION_HIGH_WATER} <= "
+                f"RAG_EVICTION_LOW_WATER={RAG_EVICTION_LOW_WATER} — eviction will never fire"
+            )
+        if RAG_EVICTION_BATCH <= 0:
+            log.warning(f"RAG_EVICTION_BATCH={RAG_EVICTION_BATCH} clamped to 1")
+    else:
+        log.warning("RAG_MAX_VECTORS <= 0 — RAG eviction disabled")
+
     yield
     log.info("JarvisChat shutting down")
 
@@ -99,10 +119,10 @@ async def session_auth_middleware(request: Request, call_next):
 
     unauth_paths = {
         "/api/auth/login", "/api/auth/logout", "/api/auth/session",
-        "/api/auth/heartbeat", "/api/auth/guest",
+        "/api/auth/heartbeat", "/api/auth/guest", "/api/ingest", "/api/hardware",
     }
 
-    if path.startswith("/api/") and is_state_changing(request.method):
+    if path.startswith("/api/"):
         if not origin_allowed(request):
             audit_event("origin_check", "denied", ip=ip, role="none",
                         details=f"{request.method} {path}", warning=True)
@@ -138,7 +158,8 @@ async def index(request: Request):
 for router_module in [
     auth_router, conversations.router, memories.router, models.router,
     presets.router, profile.router, settings.router, skills.router,
-    chat.router, search_route.router, completions.router,
+    chat.router, search_route.router, completions.router, upload.router, ingest.router, hardware.router,
+    rag_admin.router,
 ]:
     app.include_router(router_module)
 

auth.py

@@ -15,10 +15,10 @@ from fastapi.responses import JSONResponse
 from config import SESSION_TIMEOUT_SECONDS, MAX_PIN_ATTEMPTS, PIN_LOCKOUT_SECONDS, RATE_WINDOW_SECONDS
 from db import get_db, get_setting
 from security import (
-    SESSIONS, PIN_ATTEMPTS, SESSION_LOCK, audit_event, get_client_ip,
-    is_ip_allowed, check_rate_limit, rate_policy, origin_allowed,
-    is_state_changing, request_body_limit, read_json_body, hash_pin,
-    customer_error_envelope, log_incident,
+    SESSIONS, PIN_ATTEMPTS, SESSION_LOCK, BODY_LIMIT_DEFAULT_BYTES,
+    audit_event, get_client_ip, is_ip_allowed, check_rate_limit,
+    rate_policy, origin_allowed, is_state_changing, request_body_limit,
+    read_json_body, hash_pin, customer_error_envelope, log_incident,
 )
 
 log = logging.getLogger("jarvischat")
@@ -146,7 +146,6 @@ async def auth_guest(request: Request):
 
 @router.post("/api/auth/login")
 async def auth_login(request: Request):
-    from security import BODY_LIMIT_DEFAULT_BYTES
     body = await read_json_body(request, BODY_LIMIT_DEFAULT_BYTES)
     pin = str(body.get("pin", ""))
     ip = get_client_ip(request)
@@ -183,7 +182,6 @@ async def auth_heartbeat(request: Request):
 
 @router.post("/api/auth/logout")
 async def auth_logout(request: Request):
-    from security import BODY_LIMIT_DEFAULT_BYTES
     ip = get_client_ip(request)
     sid = request.headers.get("x-session-id", "").strip()
     role = "none"

config.py

@@ -9,11 +9,12 @@ import logging
 
 log = logging.getLogger("jarvischat")
 
-VERSION = "v1.8.0"
+VERSION = "v0.13.0"
 OLLAMA_BASE = os.environ.get("OLLAMA_BASE", "http://localhost:11434")
 LLAMA_SERVER_BASE = os.environ.get("LLAMA_SERVER_BASE", "http://192.168.50.108:8081")
 SEARXNG_BASE = "http://localhost:8888"
 DEFAULT_MODEL = "llama3.1:latest"
+COMPLETIONS_API_KEY = os.environ.get("JARVISCHAT_COMPLETIONS_API_KEY", "jc-sk-" + os.urandom(24).hex())
 
 # --- Auth ---
 SESSION_TIMEOUT_SECONDS = 90
@@ -45,6 +46,25 @@ BODY_LIMIT_DEFAULT_BYTES = 64 * 1024
 BODY_LIMIT_CHAT_BYTES = 128 * 1024
 BODY_LIMIT_PROFILE_BYTES = 256 * 1024
 
+# --- Upload ---
+UPLOAD_DIR = "/tmp/jarvischat_uploads"
+MAX_UPLOAD_BYTES = 20 * 1024 * 1024
+SUPPORTED_UPLOAD_TYPES = {"text/plain", "text/markdown", "application/pdf", "application/json", "text/x-python", "text/html", "image/png", "image/jpeg", "image/gif", "image/svg+xml", "image/webp"}
+QDRANT_URL = "http://192.168.50.108:6333"
+RAG_COLLECTION = "jarvis_rag"
+UPLOAD_CONTEXT_EXPIRY_HOURS = 1
+BODY_LIMIT_UPLOAD_BYTES = MAX_UPLOAD_BYTES
+
+# --- RAG eviction ---
+RAG_MAX_VECTORS = 50000
+RAG_EVICTION_HIGH_WATER = 0.80
+RAG_EVICTION_LOW_WATER = 0.20
+RAG_EVICTION_BATCH = 1000
+RAG_PINNED_SOURCES = ["upload", "profile"]
+RAG_GRACE_HOURS = 1
+RAG_ACCESS_WEIGHT = 1.0
+RAG_AGE_WEIGHT = 0.1
+
 MAX_CHAT_MESSAGE_CHARS = 8000
 MAX_SEARCH_QUERY_CHARS = 500
 MAX_PROFILE_CHARS = 32000

db.py

@@ -68,6 +68,43 @@ def format_active_skills_prompt(skills: list) -> str:
     return text
 
 
+def insert_upload_context(db, conversation_id: str, filename: str, content: str, expires_at: str, content_type: str = "text/plain") -> int:
+    now = datetime.now(timezone.utc).isoformat()
+    cur = db.execute(
+        "INSERT INTO upload_context (conversation_id, filename, content, content_type, created_at, expires_at) VALUES (?, ?, ?, ?, ?, ?)",
+        (conversation_id, filename, content, content_type, now, expires_at),
+    )
+    return cur.lastrowid
+
+
+def list_upload_context_by_conversation(db, conversation_id: str):
+    rows = db.execute(
+        "SELECT id, conversation_id, filename, content_type, created_at, expires_at FROM upload_context WHERE conversation_id = ? ORDER BY id ASC",
+        (conversation_id,),
+    ).fetchall()
+    return [dict(r) for r in rows]
+
+
+def delete_upload_context_by_id(db, context_id: int) -> bool:
+    cur = db.execute("DELETE FROM upload_context WHERE id = ?", (context_id,))
+    return cur.rowcount > 0
+
+
+def get_upload_context(db, context_id: int):
+    row = db.execute(
+        "SELECT id, conversation_id, filename, content, content_type, expires_at FROM upload_context WHERE id = ?",
+        (context_id,),
+    ).fetchone()
+    if not row:
+        return None
+    expires = datetime.fromisoformat(row["expires_at"])
+    if expires < datetime.now(timezone.utc):
+        db.execute("DELETE FROM upload_context WHERE id = ?", (context_id,))
+        db.commit()
+        return None
+    return dict(row)
+
+
 def init_db():
     from security import hash_pin
     conn = sqlite3.connect(DB_PATH)
@@ -108,6 +145,21 @@ def init_db():
             fact, topic, source, created_at UNINDEXED
         )
     """)
+    conn.execute("""
+        CREATE TABLE IF NOT EXISTS upload_context (
+            id INTEGER PRIMARY KEY AUTOINCREMENT,
+            conversation_id TEXT,
+            filename TEXT NOT NULL,
+            content TEXT NOT NULL,
+            content_type TEXT DEFAULT 'text/plain',
+            created_at TEXT NOT NULL,
+            expires_at TEXT NOT NULL
+        )
+    """)
+    try:
+        conn.execute("ALTER TABLE upload_context ADD COLUMN content_type TEXT DEFAULT 'text/plain'")
+    except Exception:
+        pass
 
     if not conn.execute("SELECT id FROM profile WHERE id = 1").fetchone():
         now = datetime.now(timezone.utc).isoformat()

docker.md

@@ -0,0 +1,681 @@
+# Docker Distribution — Architecture & Planning
+
+> **Part of B3 (v1.0 gate).** This document catalogs every service, volume, port, configuration, and decision needed to ship jarvisChat as a `docker compose` stack. It also defines extraction (setup) and back-out (uninstall) procedures so nothing is lost when reality disagrees with the plan.
+
+## 1. Stack Overview
+
+```
+┌─────────────────────────────────────────────────────────┐
+│                    docker compose stack                  │
+│                                                         │
+│  ┌────────────┐  ┌──────────┐  ┌────────────────────┐  │
+│  │  SearXNG    │  │  Qdrant  │  │  RabbitMQ          │  │
+│  │  :8888      │  │  :6333   │  │  :5672 / :15672    │  │
+│  └──────┬──────┘  └────┬─────┘  └────────┬───────────┘  │
+│         │              │                  │              │
+│         ▼              ▼                  ▼              │
+│  ┌──────────────────────────────────────────────────┐   │
+│  │              jarvisChat (FastAPI)                │   │
+│  │              :8080 (HTTP)                        │   │
+│  │                                                  │   │
+│  │  SQLite  ◄── jarvischat.db           (volume)    │   │
+│  │  Uploads ◄── /app/uploads            (volume)    │   │
+│  └──────────┬──────────────┬───────────────────────┘   │
+│             │              │                            │
+│             ▼              ▼                            │
+│  ┌──────────────┐  ┌──────────────┐                    │
+│  │ llama-server │  │   Ollama     │                    │
+│  │ :8081        │  │ :11434       │                    │
+│  │ (GPU/RPC)    │  │ (embeddings) │                    │
+│  └──────────────┘  └──────────────┘                    │
+└─────────────────────────────────────────────────────────┘
+```
+
+### Service roles
+
+| Service | Image | Role |
+|---------|-------|------|
+| **jarvisChat** | Custom `Dockerfile` | FastAPI app serving UI + API |
+| **SearXNG** | `searxng/searxng:latest` | Privacy-respecting web search |
+| **Qdrant** | `qdrant/qdrant:latest` | Vector database for RAG |
+| **RabbitMQ** | `rabbitmq:4-management` | Message broker for AMQP cluster |
+| **llama-server** | `ghcr.io/ggml-org/llama.cpp:server` | LLM inference (OpenAI-compat API) |
+| **Ollama** | `ollama/ollama:latest` | Embeddings for RAG chunk vectors |
+
+### Non-containerized (host-level)
+
+| Component | Reason |
+|-----------|--------|
+| AMD GPU driver + ROCm | Kernel access required for GPU compute |
+| llama.cpp RPC workers | Runs on *other* hosts — not on the Docker host |
+| `rocm-smi` | Hardware stats — not needed for core function |
+| `psutil` | Already inside the container via pip |
+
+---
+
+## 2. Service Catalog
+
+### 2.1 jarvisChat (FastAPI app)
+
+**Image:** `jarvischat:latest` (built from `Dockerfile`)
+
+**Ports:**
+| Container | Host | Purpose |
+|-----------|------|---------|
+| 8080 | 8080 | HTTP API + UI |
+
+**Volumes:**
+| Container path | Type | Purpose |
+|----------------|------|---------|
+| `/app/jarvischat.db` | named volume `jarvischat_data` | SQLite database |
+| `/app/uploads` | named volume `jarvischat_uploads` | Uploaded files |
+| `/app/hardware_state.json` | (inside volume) | Cached hardware probe |
+
+**Dependencies:** Wait for SearXNG, Qdrant, RabbitMQ, llama-server, Ollama before serving.
+
+**Restart:** `unless-stopped`
+
+**Healthcheck:** `curl -f http://localhost:8080/`
+
+### 2.2 SearXNG
+
+**Image:** `searxng/searxng:latest`
+
+**Ports:**
+| Container | Host | Purpose |
+|-----------|------|---------|
+| 8080 | 8888 | Search API |
+
+**Volumes:**
+| Container path | Type | Purpose |
+|----------------|------|---------|
+| `/etc/searxng` | named volume `searxng_config` | `settings.yml` |
+
+**Environment:**
+```env
+SEARXNG_BASE_URL=https://localhost:8888
+```
+
+**Config override (`/etc/searxng/settings.yml`):**
+```yaml
+search:
+  safe_search: 0
+  autocomplete: ""
+server:
+  secret_key: ${SEARXNG_SECRET_KEY}
+  limiter: false
+  image_proxy: false
+  method: GET
+  port: 8080
+  bind_address: "0.0.0.0"
+```
+
+**Restart:** `unless-stopped`
+
+### 2.3 Qdrant
+
+**Image:** `qdrant/qdrant:latest`
+
+**Ports:**
+| Container | Host | Purpose |
+|-----------|------|---------|
+| 6333 | 6333 | HTTP API |
+| 6334 | — | gRPC (internal only) |
+
+**Volumes:**
+| Container path | Type | Purpose |
+|----------------|------|---------|
+| `/qdrant/storage` | named volume `qdrant_storage` | Vector index data |
+
+**Environment:**
+```env
+QDRANT__SERVICE__GRPC_PORT=6334
+```
+
+**Restart:** `unless-stopped`
+
+### 2.4 RabbitMQ
+
+**Image:** `rabbitmq:4-management`
+
+**Ports:**
+| Container | Host | Purpose |
+|-----------|------|---------|
+| 5672 | 5672 | AMQP messaging |
+| 15672 | — | Management UI (internal only) |
+
+**Volumes:**
+| Container path | Type | Purpose |
+|----------------|------|---------|
+| `/var/lib/rabbitmq` | named volume `rabbitmq_data` | Message store |
+
+**Environment:**
+```env
+RABBITMQ_DEFAULT_USER=jarvischat
+RABBITMQ_DEFAULT_PASS_FILE=/run/secrets/rabbitmq_password
+RABBITMQ_DEFAULT_VHOST=/
+```
+
+**Restart:** `unless-stopped`
+
+### 2.5 llama-server
+
+**Image:** `ghcr.io/ggml-org/llama.cpp:server`
+
+**Ports:**
+| Container | Host | Purpose |
+|-----------|------|---------|
+| 8081 | 8081 | OpenAI-compat API |
+
+**Volumes:**
+| Container path | Type | Purpose |
+|----------------|------|---------|
+| `/models` | bind mount `./models` | Model GGUF files |
+
+**Environment:**
+```env
+LLAMA_ARG_MODEL=/models/<model-file>
+LLAMA_ARG_N_GPU_LAYERS=0              # set >0 for GPU offload
+LLAMA_ARG_MAIN_GPU=0
+LLAMA_ARG_CTX_SIZE=4096
+LLAMA_ARG_HOST=0.0.0.0
+LLAMA_ARG_PORT=8081
+LLAMA_ARG_EMBEDDINGS=1
+LLAMA_ARG_LOGPROBS=1
+LLAMA_ARG_RPC=                         # optional: comma-separated RPC endpoints
+```
+
+**Restart:** `unless-stopped`
+
+**Healthcheck:** `curl -f http://localhost:8081/health`
+
+**Notes:**
+- Models directory bind mount — user places `.gguf` files in `./models/` on the host
+- RPC offload to other machines (e.g., `10.0.0.50:50052,10.0.0.51:50052`)
+- If no GPU, set `LLAMA_ARG_N_GPU_LAYERS=0` for CPU-only
+- `LLAMA_ARG_EMBEDDINGS=1` required for perplexity scoring
+- `LLAMA_ARG_LOGPROBS=1` required for auto-search trigger
+
+### 2.6 Ollama
+
+**Image:** `ollama/ollama:latest`
+
+**Ports:**
+| Container | Host | Purpose |
+|-----------|------|---------|
+| 11434 | 11434 | Embeddings API |
+
+**Volumes:**
+| Container path | Type | Purpose |
+|----------------|------|---------|
+| `/root/.ollama` | named volume `ollama_models` | Pulled model blobs |
+
+**Restart:** `unless-stopped`
+
+**Notes:**
+- Used exclusively for embeddings (`/api/embeddings`), not inference
+- Typically needs a small model like `all-minilm:latest` or `nomic-embed-text:latest`
+- Consider replacing Ollama with llama-server's built-in embedding if it supports the same model — would remove one container
+
+---
+
+## 3. Configuration Management
+
+### 3.1 `.env` file (generated by setup wizard)
+
+```env
+# --- Secrets (auto-generated, change before production) ---
+JARVISCHAT_ADMIN_PIN=
+JARVISCHAT_COMPLETIONS_API_KEY=
+JARVISCHAT_ALLOW_DEFAULT_PIN=false
+RABBITMQ_PASSWORD=
+SEARXNG_SECRET_KEY=
+
+# --- Host discovery (auto-detected by setup wizard) ---
+LLAMA_SERVER_BASE=http://llama-server:8081
+OLLAMA_BASE=http://ollama:11434
+SEARXNG_BASE=http://searxng:8888
+QDRANT_URL=http://qdrant:6333
+RABBITMQ_HOST=rabbitmq
+RABBITMQ_PORT=5672
+
+# --- Performance tuning (calculated by setup wizard) ---
+RAG_MAX_VECTORS=50000
+RAG_EVICTION_HIGH_WATER=0.80
+RAG_EVICTION_LOW_WATER=0.20
+RAG_EVICTION_BATCH=1000
+
+# --- llama-server options ---
+LLAMA_MODEL=llama3.1-8b-instruct.Q4_K_M.gguf
+LLAMA_N_GPU_LAYERS=0
+LLAMA_RPC_ENDPOINTS=
+LLAMA_CTX_SIZE=4096
+
+# --- Ollama ---
+OLLAMA_EMBED_MODEL=all-minilm:latest
+
+# --- Network ---
+JARVISCHAT_ALLOWED_CIDRS=127.0.0.0/8,::1/128,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16
+JARVISCHAT_TRUSTED_ORIGINS=
+JARVISCHAT_TRUST_X_FORWARDED_FOR=false
+```
+
+### 3.2 Mapping of config.py → .env variable
+
+Every config.py default that references an external service must accept a matching env var at runtime:
+
+| config.py constant | .env variable | Service |
+|-------------------|---------------|---------|
+| `LLAMA_SERVER_BASE` | `LLAMA_SERVER_BASE` | llama-server |
+| `OLLAMA_BASE` | `OLLAMA_BASE` | Ollama |
+| `SEARXNG_BASE` | `SEARXNG_BASE` | SearXNG |
+| `QDRANT_URL` | `QDRANT_URL` | Qdrant |
+| `COMPLETIONS_API_KEY` | `JARVISCHAT_COMPLETIONS_API_KEY` | — |
+| `ALLOWED_CIDRS_RAW` | `JARVISCHAT_ALLOWED_CIDRS` | — |
+| `TRUST_X_FORWARDED_FOR` | `JARVISCHAT_TRUST_X_FORWARDED_FOR` | — |
+| `TRUSTED_ORIGINS` | `JARVISCHAT_TRUSTED_ORIGINS` | — |
+| `RAG_MAX_VECTORS` | `RAG_MAX_VECTORS` | — (calc'd from RAM) |
+
+### 3.3 Secrets management
+
+| Secret | Generated by | Stored in | Mounted to |
+|--------|-------------|-----------|------------|
+| `JARVISCHAT_ADMIN_PIN` | User prompt | `.env` | jarvisChat container |
+| `JARVISCHAT_COMPLETIONS_API_KEY` | Auto-generated, shown to user | `.env` | jarvisChat container |
+| `RABBITMQ_PASSWORD` | Auto-generated | `.env` + Docker secret | RabbitMQ container |
+| `SEARXNG_SECRET_KEY` | Auto-generated | `.env` | SearXNG container |
+
+**Docker secrets approach:** Use `secrets:` in compose file for RabbitMQ password (mounted as file) rather than passing via env var, since `settings.yml` in SearXNG and RabbitMQ config can reference file-based secrets without env-var leakage.
+
+### 3.4 Dockerfile for jarvisChat
+
+```dockerfile
+FROM python:3.13-slim-bookworm AS builder
+WORKDIR /app
+COPY requirements.txt .
+RUN pip install --no-cache-dir -r requirements.txt
+
+FROM python:3.13-slim-bookworm
+WORKDIR /app
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    curl && \
+    rm -rf /var/lib/apt/lists/*
+COPY --from=builder /usr/local/lib/python3.13/site-packages /usr/local/lib/python3.13/site-packages
+COPY --from=builder /usr/local/bin /usr/local/bin
+COPY . .
+
+EXPOSE 8080
+HEALTHCHECK --interval=30s --timeout=10s --start-period=15s --retries=3 \
+    CMD curl -f http://localhost:8080/ || exit 1
+
+CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8080"]
+```
+
+**Multi-stage rationale:** First stage compiles/bundles packages (wheels), final stage is minimal. Devs can skip builder with `--target builder` for live-reload with volume mount.
+
+---
+
+## 4. docker-compose.yml structure
+
+```yaml
+services:
+  jarvischat:
+    build: .
+    ports: ["8080:8080"]
+    volumes:
+      - jarvischat_data:/app/jarvischat.db
+      - jarvischat_uploads:/app/uploads
+    env_file: .env
+    depends_on:
+      searxng: { condition: service_started }
+      qdrant: { condition: service_started }
+      rabbitmq: { condition: service_healthy }
+      llama-server: { condition: service_healthy }
+      ollama: { condition: service_started }
+    restart: unless-stopped
+
+  searxng:
+    image: searxng/searxng:latest
+    ports: ["8888:8080"]
+    volumes:
+      - ./searxng/settings.yml:/etc/searxng/settings.yml:ro
+      - searxng_config:/etc/searxng
+    env_file: .env
+    restart: unless-stopped
+
+  qdrant:
+    image: qdrant/qdrant:latest
+    ports: ["6333:6333"]
+    volumes:
+      - qdrant_storage:/qdrant/storage
+    restart: unless-stopped
+
+  rabbitmq:
+    image: rabbitmq:4-management
+    ports: ["5672:5672"]
+    volumes:
+      - rabbitmq_data:/var/lib/rabbitmq
+    env_file: .env
+    secrets:
+      - rabbitmq_password
+    healthcheck:
+      test: ["CMD", "rabbitmq-diagnostics", "check_port_connectivity"]
+      interval: 15s
+      timeout: 5s
+      retries: 3
+    restart: unless-stopped
+
+  llama-server:
+    image: ghcr.io/ggml-org/llama.cpp:server
+    ports: ["8081:8081"]
+    volumes:
+      - ./models:/models:ro
+    env_file: .env
+    command: >
+      --model /models/${LLAMA_MODEL}
+      --host 0.0.0.0 --port 8081
+      --ctx-size ${LLAMA_CTX_SIZE:-4096}
+      --n-gpu-layers ${LLAMA_N_GPU_LAYERS:-0}
+      --embeddings
+      --logprobs
+      ${LLAMA_RPC_ENDPOINTS:+--rpc ${LLAMA_RPC_ENDPOINTS}}
+    healthcheck:
+      test: ["CMD", "curl", "-f", "http://localhost:8081/health"]
+      interval: 30s
+      timeout: 10s
+      retries: 5
+      start_period: 60s
+    deploy:
+      resources:
+        reservations:
+          devices:
+            - driver: nvidia
+              count: all
+              capabilities: [gpu]
+    restart: unless-stopped
+
+  ollama:
+    image: ollama/ollama:latest
+    ports: ["11434:11434"]
+    volumes:
+      - ollama_models:/root/.ollama
+    healthcheck:
+      test: ["CMD", "ollama", "list"]
+      interval: 30s
+      timeout: 10s
+      retries: 3
+    deploy:
+      resources:
+        reservations:
+          devices:
+            - driver: nvidia
+              count: all
+              capabilities: [gpu]
+    restart: unless-stopped
+
+volumes:
+  jarvischat_data:
+  jarvischat_uploads:
+  searxng_config:
+  qdrant_storage:
+  rabbitmq_data:
+  ollama_models:
+
+secrets:
+  rabbitmq_password:
+    file: ./secrets/rabbitmq_password.txt
+```
+
+**Notes:**
+- GPU reservations use `resources.reservations.devices` — this is compose v3.8+. For AMD GPUs, replace `driver: nvidia` with `driver: amd` (experimental Docker support). For hosts without GPU, omit the `deploy` block entirely.
+- The `deploy` block only applies when deployed as a swarm stack. For `docker compose`, GPU access may need `--gpus all` or `device_requests` in config. Verify compatibility.
+- SearXNG config file (`settings.yml`) is bind-mounted read-only from the host repo clone — the setup wizard should generate this file.
+
+---
+
+## 5. Networking
+
+### 5.1 Internal communication (compose network)
+
+| From | To | Port | Protocol |
+|------|----|------|----------|
+| jarvisChat | llama-server | 8081 | HTTP |
+| jarvisChat | Ollama | 11434 | HTTP |
+| jarvisChat | SearXNG | 8080 | HTTP |
+| jarvisChat | Qdrant | 6333 | HTTP |
+| jarvisChat | RabbitMQ | 5672 | AMQP |
+| RabbitMQ | (cluster peers) | 4369 | EPMD |
+| RabbitMQ | (cluster peers) | 25672 | Inter-node |
+
+### 5.2 Exposed ports (host-facing)
+
+| Port | Service | Should expose? | Notes |
+|------|---------|---------------|-------|
+| 8080 | jarvisChat | ✅ Required | UI + API |
+| 8888 | SearXNG | Optional | Only if user wants standalone search |
+| 6333 | Qdrant | Optional | Only for external tooling |
+| 5672 | RabbitMQ | Optional | Only for remote AMQP clients |
+| 15672 | RabbitMQ mgmt | ❌ Internal | Healthcheck only |
+| 8081 | llama-server | Optional | Only for external tooling |
+| 11434 | Ollama | Optional | Only for external tooling |
+
+**Design decision:** By default, only port 8080 (jarvisChat) is published. All other services remain on the internal compose network. Advanced users can opt-in by uncommenting `ports:` blocks.
+
+### 5.3 Reverse proxy consideration
+
+For production, a reverse proxy (Caddy, nginx, Traefik) should sit in front:
+
+```yaml
+# Optional — compose profile: "proxy"
+caddy:
+  image: caddy:latest
+  ports: ["80:80", "443:443"]
+  volumes:
+    - ./Caddyfile:/etc/caddy/Caddyfile:ro
+    - caddy_data:/data
+```
+
+This is out of scope for v1.0 but documented for future.
+
+---
+
+## 6. Setup Wizard (Extraction)
+
+`setup.sh` — idempotent, interactive, runs on first boot.
+
+### Flow
+
+```
+1. CHECK: Is .env present?
+   ├── YES → skip to step 7 (or ask to regenerate)
+   └── NO  → continue
+
+2. INTRO: Print banner, explain what's about to happen
+
+3. PROBE: Run hardware assessment
+   ├── psutil → RAM total, CPU count
+   ├── rocm-smi → VRAM (optional, best-effort)
+   └── nvidia-smi → VRAM (optional, best-effort)
+
+4. NETWORK: Ask for
+   ├── Hostname / LAN IP for this machine
+   ├── Admin PIN (4 digits, or accept auto-generated)
+   └── (Optional) RPC endpoints for GPU offload
+
+5. CALCULATE:
+   ├── RAG_MAX_VECTORS = max(1000, int(available_ram_gb * 100_000))
+   ├── LLAMA_N_GPU_LAYERS = 0 (CPU default; offer GPU detection)
+   ├── LLAMA_MODEL = default gguf filename
+   └── RABBITMQ_PASSWORD = openssl rand -hex 20
+
+6. GENERATE:
+   ├── .env file from template
+   ├── ./secrets/rabbitmq_password.txt
+   ├── ./searxng/settings.yml (with generated secret_key)
+   └── ./models/README.txt (instructions for placing .gguf)
+
+7. VERIFY:
+   ├── docker and docker compose plugin installed
+   ├── docker compose version >= 2.x
+   ├── SUCCESS → "Run: docker compose up -d"
+   └── FAILURE  → show diagnostics and links
+
+8. EXTRACT model:
+   ├── Prompt for download URL or local path
+   ├── Offer to pull from HuggingFace if huggingface-cli available
+   └── Guides user to place file in ./models/
+```
+
+### What setup.sh creates on disk
+
+```
+./docker-deploy/
+├── .env                        # All env vars (SECRET — add to .gitignore)
+├── docker-compose.yml          # Compose stack definition
+├── Dockerfile                  # jarvisChat image build
+├── secrets/
+│   └── rabbitmq_password.txt   # RabbitMQ password file
+├── searxng/
+│   └── settings.yml            # SearXNG config with generated secret_key
+├── models/
+│   ├── README.txt              # Instructions for model placement
+│   └── <model>.gguf            # (user-provided)
+└── setup.log                   # Wizard run log
+```
+
+### Idempotency
+
+Re-running `setup.sh`:
+- With `.env` present: ask "Regenerate? This will overwrite existing config."
+- Without `.env`: fresh run
+- Never overwrites `./models/*.gguf` files
+- Never touches running containers — only modifies files on disk
+
+---
+
+## 7. Back-out Procedure (Uninstall)
+
+`teardown.sh` — returns the host system to its pre-install state.
+
+### What gets removed
+
+| Item | Removal method |
+|------|---------------|
+| Docker containers | `docker compose down -v` |
+| Docker images | `docker rmi jarvischat:latest` (ask about other images) |
+| Docker volumes | `docker volume rm jarvischat_data ...` (prompt first) |
+| Network `jarvischat_default` | Removed with compose |
+| `.env` file | `rm .env` |
+| `secrets/` directory | `rm -rf secrets/` |
+| `searxng/` directory | `rm -rf searxng/` |
+| `setup.log` | `rm setup.log` |
+| `hardware_state.json` | `rm hardware_state.json` |
+
+### What is preserved (by default)
+
+| Item | Reason |
+|------|--------|
+| `./models/*.gguf` | User data — prompt for deletion |
+| `jarvischat.db` (in volume) | Prompt: "Keep database snapshot?" |
+| `./uploads/` (in volume) | Prompt: "Keep uploaded files?" |
+| Docker Engine itself | Not installed by this project — leave it |
+
+### Script flow
+
+```
+1. CHECK: docker compose file exists?
+   ├── NO  → warn, continue
+   └── YES → docker compose down -v
+
+2. CHECK: .env exists?
+   ├── NO  → skip
+   └── YES → ask: "Remove .env?" (default no)
+
+3. ASK: "Remove secrets/ and searxng/ directories?" (default no)
+
+4. ASK: "Remove Docker images? (y/N)" (default no)
+   ├── Y → docker rmi jarvischat:latest
+   ├── Y → docker image ls | grep searxng/qdrant/rabbitmq → prompt per image
+   └── N → skip
+
+5. ASK: "Keep database volume snapshot? (Y/n)" (default yes)
+   ├── N → docker volume rm jarvischat_data
+   └── Y → leave volume (can be reattached later)
+
+6. ASK: "Remove model files from ./models/? (y/N)" (default no)
+
+7. CLEANUP generated artifacts:
+   ├── rm -f setup.log
+   ├── rm -f hardware_state.json
+   └── rm -f docker-compose.yml
+
+8. SUMMARY:
+   ├── "Docker stack removed"
+   ├── "Persistent data preserved at: <paths>"
+   └── "Models kept at: ./models/"
+```
+
+### Partial rollback
+
+If the setup wizard fails mid-way, a partial rollback is better than leaving detritus:
+
+| Failure point | Clean up |
+|--------------|----------|
+| After .env, before compose | `rm .env; rm -rf secrets/ searxng/` |
+| After compose, before first `up` | `rm docker-compose.yml; rm -rf *` |
+| After `up` but before healthcheck | `docker compose down -v; rm -rf ./*` |
+
+`setup.sh` should trap EXIT on failure and prompt: "Clean up partial install? [y/N]"
+
+---
+
+## 8. Open Decisions
+
+| Decision | Options | Priority |
+|----------|---------|----------|
+| **Ollama vs llama-server embeddings** | Both work. Keep both for now — remove Ollama if llama-server handles embeddings. Reduce containers = simpler. | Medium |
+| **GPU support in compose** | NVIDIA: well-supported. AMD: requires `--device=/dev/kfd --device=/dev/dri` and ROCm image. Document both. | High |
+| **RabbitMQ clustering vs single node** | Single node in v1.0. Clustering docs for multi-host later. | Low |
+| **SearXNG config management** | Bind-mount a generated `settings.yml`, or let container create default and post-process. Bind-mount is cleaner. | Medium |
+| **Reverse proxy** | Caddy is simplest for auto-HTTPS. Out of scope for v1.0 but design for it. | Low |
+| **Healthcheck strategy** | `depends_on` with `condition: service_healthy` is the safest approach but increases startup time. Acceptable. | Medium |
+| **Database migration** | SQLite file in volume — no migration needed for v1.0 format. If schema changes post-v1.0, need a migration container. | Low |
+| **Linux vs macOS vs Windows** | Linux-primary. macOS may work with changes (no rocm-smi). Windows via WSL2 only. | Low |
+| **LLM model download** | HuggingFace CLI integration in setup.sh, or manual download. Manual is simpler. | Low |
+| **Dockerfile optimization** | Pin pip hashes, use `--no-cache-dir`, consider `slim` vs `alpine`. Alpine has musl compatibility issues with psutil. Stay with slim. | Medium |
+
+## 9. Checklist (pre-v1.0 gate)
+
+- [ ] `Dockerfile` written and builds clean
+- [ ] `docker-compose.yml` boots all containers
+- [ ] jarvisChat container reaches all services (env vars resolve correctly)
+- [ ] SearXNG settings.yml generated correctly by setup.sh
+- [ ] RabbitMQ password secret mounted correctly
+- [ ] GPU (NVIDIA) passes through to llama-server container
+- [ ] GPU (AMD) passes through to llama-server container (or documented limitation)
+- [ ] `.env.example` checked in (no real secrets)
+- [ ] `setup.sh` written, idempotent, tested on clean Debian
+- [ ] `teardown.sh` written, tested, doesn't delete models without confirmation
+- [ ] `docker compose up -d` works without any manual steps beyond setup.sh
+- [ ] `docker compose down -v` followed by `setup.sh && docker compose up -d` = fresh stack
+- [ ] Healthchecks prevent serving before dependencies are ready
+- [ ] v1.0 release tag created
+
+---
+
+## 10. Files to create for B3
+
+```
+docker.md             ← this file (planning doc)
+Dockerfile            ← jarvisChat image
+docker-compose.yml    ← full stack
+.env.example          ← template without secrets
+setup.sh              ← extraction wizard
+teardown.sh           ← back-out utility
+searxng/
+  settings.yml        ← SearXNG config (generated by setup.sh)
+secrets/
+  rabbitmq_password.txt  ← generated by setup.sh
+models/
+  README.txt          ← instructions for placing .gguf
+```

docs/jc-ingest.sh

@@ -0,0 +1,31 @@
+#!/bin/bash
+# jc-ingest.sh — pipe terminal commands into jarvisChat RAG
+# Deploy to /home/gramps/bin/jc-ingest.sh on jarvis (192.168.50.210)
+#
+# Usage:
+#   1. chmod +x /home/gramps/bin/jc-ingest.sh
+#   2. Add to ~/.bashrc:
+#        export JARVISCHAT_COMPLETIONS_API_KEY="$(cat /opt/jarvischat/.completions_key)"
+#        export PROMPT_COMMAND="jc_capture"
+#        source /home/gramps/bin/jc-ingest.sh
+#
+#   The PROMPT_COMMAND hook runs jc_capture() after each command.
+#   Only commands matching the filter pattern are ingested.
+#
+# Filter: currently captures git, pip, systemctl, sudo, vi/vim, curl,
+# wget, apt, python, pytest commands. Edit the grep pattern to adjust.
+
+JC_URL="http://192.168.50.210:8080/api/ingest"
+JC_TOKEN="${JARVISCHAT_COMPLETIONS_API_KEY}"
+
+jc_capture() {
+    local cmd
+    cmd=$(history 1 | sed 's/^[ ]*[0-9]*[ ]*//')
+    if echo "$cmd" | grep -qE '^(git|pip|systemctl|sudo|vi|vim|curl|wget|apt|python|pytest)'; then
+        curl -s -X POST "$JC_URL" \
+            -H "Authorization: Bearer $JC_TOKEN" \
+            -H "Content-Type: application/json" \
+            -d "{\"content\": $(echo "$cmd" | python3 -c 'import json,sys; print(json.dumps(sys.stdin.read().strip()))'), \"source\": \"terminal\"}" \
+            > /dev/null 2>&1 &
+    fi
+}

docs/wiki/current-wip.md

@@ -4,7 +4,7 @@ Last updated: 2026-04-27
 Owner: Gramps + Copilot
 Scope: issues, bugs, security exposures, and feature enhancements.
 
-Total identified items: 27
+Total identified items: 29
 
 ## Priority Definitions
 - P0: Critical risk or data-loss/security exposure; do first.
@@ -63,20 +63,22 @@ Total identified items: 27
 15. Add preflight validation for required model/preset selection and block send with clear user guidance instead of timing out.
 
 ### P2 Important Features
-16. Skills system: load markdown skill files with YAML frontmatter from skills directory.
-17. Skills registry API: list/enable/disable skills and expose active skills to UI.
-18. Inject active skill instructions into system prompt with bounded token budget.
-19. Tool execution guardrails: allowlist, confirmation mode, and execution logs.
-20. Heartbeat scheduler (cron/systemd timer) for daily check-ins.
-21. Heartbeat endpoint for generated briefings and anomaly summaries.
-22. Model info UI panel (description, updated date, best-use purpose).
-23. Default model selection improvements and persistence validation.
-24. Hidden model list support (exclude models from dropdown).
-25. Model update action from UI (trigger controlled model pull).
+16. HTTPS via Let's Encrypt: set up reverse proxy (Caddy or Nginx) with auto-cert to fix clipboard API and enable secure context features.
+17. Bidirectional image I/O: load multimodal models for both vision understanding (image inputs, describe/answer) and image generation (diffusion model or multimodal output). Enable image upload/paste and AI-generated image display in chat.
+18. Skills system: load markdown skill files with YAML frontmatter from skills directory.
+19. Skills registry API: list/enable/disable skills and expose active skills to UI.
+20. Inject active skill instructions into system prompt with bounded token budget.
+21. Tool execution guardrails: allowlist, confirmation mode, and execution logs.
+22. Heartbeat scheduler (cron/systemd timer) for daily check-ins.
+23. Heartbeat endpoint for generated briefings and anomaly summaries.
+24. Model info UI panel (description, updated date, best-use purpose).
+25. Default model selection improvements and persistence validation.
+26. Hidden model list support (exclude models from dropdown).
+27. Model update action from UI (trigger controlled model pull).
 
 ### P3 Nice to Have
-26. Conversation search/filter and export tooling.
-27. Keyboard shortcuts, retry button, and source-link polish.
+28. Conversation search/filter and export tooling.
+29. Keyboard shortcuts, retry button, and source-link polish.
 
 ## Maintenance Rules
 - Keep this file as the single source of truth.

eviction.py

@@ -0,0 +1,268 @@
+"""
+JarvisChat — Score-based RAG vector eviction with hysteresis.
+"""
+import asyncio
+import logging
+from datetime import datetime, timezone, timedelta
+
+import httpx
+
+from config import (
+    QDRANT_URL, RAG_COLLECTION,
+    RAG_MAX_VECTORS, RAG_EVICTION_HIGH_WATER, RAG_EVICTION_LOW_WATER,
+    RAG_EVICTION_BATCH, RAG_PINNED_SOURCES, RAG_GRACE_HOURS,
+    RAG_ACCESS_WEIGHT, RAG_AGE_WEIGHT,
+)
+
+log = logging.getLogger("jarvischat")
+
+eviction_lock = asyncio.Lock()
+EVICTION_LOG: list[dict] = []
+
+
+async def _update_retrieval_count(point_id: str, current_count: int = 0):
+    try:
+        async with httpx.AsyncClient() as client:
+            payload = {
+                "retrieval_count": current_count + 1,
+                "last_accessed": datetime.now(timezone.utc).isoformat(),
+            }
+            resp = await client.put(
+                f"{QDRANT_URL}/collections/{RAG_COLLECTION}/points/payload",
+                json={"points": [point_id], "payload": payload},
+                timeout=5.0,
+            )
+            if resp.status_code not in (200, 201):
+                log.warning(f"Failed to increment retrieval count for {point_id}: {resp.status_code}")
+    except Exception as e:
+        log.warning(f"Error incrementing retrieval count for {point_id}: {e}")
+
+
+async def get_collection_count() -> int:
+    try:
+        async with httpx.AsyncClient() as client:
+            resp = await client.get(
+                f"{QDRANT_URL}/collections/{RAG_COLLECTION}",
+                timeout=10.0,
+            )
+            if resp.status_code == 200:
+                return resp.json().get("result", {}).get("vectors_count", 0)
+    except Exception as e:
+        log.warning(f"get_collection_count error: {e}")
+    return 0
+
+
+async def get_collection_stats() -> dict:
+    count = await get_collection_count()
+    high_water_pct = int(RAG_EVICTION_HIGH_WATER * 100)
+    low_water_pct = int(RAG_EVICTION_LOW_WATER * 100)
+    percent_full = round((count / RAG_MAX_VECTORS) * 100, 1) if RAG_MAX_VECTORS > 0 else 0
+    return {
+        "vector_count": count,
+        "max_vectors": RAG_MAX_VECTORS,
+        "high_water_mark": int(RAG_MAX_VECTORS * RAG_EVICTION_HIGH_WATER),
+        "low_water_mark": int(RAG_MAX_VECTORS * RAG_EVICTION_LOW_WATER),
+        "high_water_pct": high_water_pct,
+        "low_water_pct": low_water_pct,
+        "percent_full": percent_full,
+        "pinned_sources": list(RAG_PINNED_SOURCES),
+    }
+
+
+async def evict_batch(batch_size: int) -> int:
+    filter_conditions = {
+        "must_not": [
+            {"match": {"key": "source", "value": src}}
+            for src in RAG_PINNED_SOURCES
+        ]
+    }
+    try:
+        async with httpx.AsyncClient() as client:
+            scroll_resp = await client.post(
+                f"{QDRANT_URL}/collections/{RAG_COLLECTION}/points/scroll",
+                json={
+                    "filter": filter_conditions,
+                    "limit": min(batch_size * 10, 10000),
+                    "with_payload": True,
+                    "with_vector": False,
+                },
+                timeout=30.0,
+            )
+            if scroll_resp.status_code != 200:
+                log.warning(f"Eviction scroll failed: {scroll_resp.status_code}")
+                return 0
+
+            points = scroll_resp.json().get("result", {}).get("points", [])
+            if not points:
+                return 0
+
+            now = datetime.now(timezone.utc)
+            scored = []
+            for p in points:
+                payload = p.get("payload", {})
+                date_str = payload.get("ingest_date") or payload.get("upload_date", "")
+                if date_str:
+                    age_hours = (now - datetime.fromisoformat(date_str)).total_seconds() / 3600
+                else:
+                    age_hours = 999999
+
+                if age_hours < RAG_GRACE_HOURS:
+                    continue
+
+                retrieval_count = payload.get("retrieval_count", 0) or 0
+                score = retrieval_count * RAG_ACCESS_WEIGHT + age_hours * RAG_AGE_WEIGHT
+                last_accessed = payload.get("last_accessed", date_str)
+                scored.append((score, last_accessed, p["id"]))
+
+            if not scored:
+                log.warning("No evictable vectors found (all pinned or newborn)")
+                return 0
+
+            scored.sort(key=lambda x: (x[0], x[1]))
+            to_delete = [p[2] for p in scored[:batch_size]]
+            if not to_delete:
+                return 0
+
+            delete_resp = await client.post(
+                f"{QDRANT_URL}/collections/{RAG_COLLECTION}/points/delete",
+                json={"points": to_delete},
+                timeout=30.0,
+            )
+            if delete_resp.status_code not in (200, 201):
+                log.warning(f"Eviction delete failed: {delete_resp.status_code}")
+                return 0
+
+            return len(to_delete)
+    except Exception as e:
+        log.warning(f"evict_batch error: {e}")
+        return 0
+
+
+async def maybe_evict() -> int:
+    if RAG_MAX_VECTORS <= 0:
+        return 0
+    effective_batch = max(RAG_EVICTION_BATCH, 1)
+
+    async with eviction_lock:
+        count = await get_collection_count()
+        threshold_high = int(RAG_MAX_VECTORS * RAG_EVICTION_HIGH_WATER)
+        threshold_low = int(RAG_MAX_VECTORS * RAG_EVICTION_LOW_WATER)
+
+        if count < threshold_high:
+            return 0
+
+        total_evicted = 0
+        while count >= threshold_low:
+            if total_evicted > 0 and count < threshold_low:
+                break
+            deleted = await evict_batch(effective_batch)
+            if deleted == 0:
+                break
+            total_evicted += deleted
+            count -= deleted
+            if count < threshold_high and total_evicted > 0:
+                break
+            if count < threshold_low:
+                break
+
+        if total_evicted > 0:
+            entry = {
+                "timestamp": datetime.now(timezone.utc).isoformat(),
+                "count": total_evicted,
+                "remaining": count,
+            }
+            EVICTION_LOG.append(entry)
+            if len(EVICTION_LOG) > 1000:
+                EVICTION_LOG.pop(0)
+            log.info(f"Evicted {total_evicted} vectors ({count} remaining)")
+
+        return total_evicted
+
+
+async def get_rag_operational_stats() -> dict:
+    stats = await get_collection_stats()
+    now = datetime.now(timezone.utc)
+    cutoff_1m = now - timedelta(minutes=1)
+    cutoff_5m = now - timedelta(minutes=5)
+    cutoff_30m = now - timedelta(minutes=30)
+
+    eviction_1m = sum(
+        e["count"] for e in EVICTION_LOG
+        if datetime.fromisoformat(e["timestamp"]) > cutoff_1m
+    )
+    eviction_5m = sum(
+        e["count"] for e in EVICTION_LOG
+        if datetime.fromisoformat(e["timestamp"]) > cutoff_5m
+    )
+    eviction_30m = sum(
+        e["count"] for e in EVICTION_LOG
+        if datetime.fromisoformat(e["timestamp"]) > cutoff_30m
+    )
+
+    pinned_count = 0
+    avg_retrieval_count = 0.0
+    at_risk_count = 0
+
+    try:
+        async with httpx.AsyncClient() as client:
+            pinned_filter = {
+                "should": [
+                    {"match": {"key": "source", "value": src}}
+                    for src in RAG_PINNED_SOURCES
+                ]
+            }
+            pinned_resp = await client.post(
+                f"{QDRANT_URL}/collections/{RAG_COLLECTION}/points/scroll",
+                json={"filter": pinned_filter, "limit": 10000, "with_payload": True, "with_vector": False},
+                timeout=10.0,
+            )
+            if pinned_resp.status_code == 200:
+                pinned_count = len(pinned_resp.json().get("result", {}).get("points", []))
+
+            nonpinned_filter = {
+                "must_not": [
+                    {"match": {"key": "source", "value": src}}
+                    for src in RAG_PINNED_SOURCES
+                ]
+            }
+            np_resp = await client.post(
+                f"{QDRANT_URL}/collections/{RAG_COLLECTION}/points/scroll",
+                json={"filter": nonpinned_filter, "limit": 10000, "with_payload": True, "with_vector": False},
+                timeout=10.0,
+            )
+            if np_resp.status_code == 200:
+                points = np_resp.json().get("result", {}).get("points", [])
+                if points:
+                    retrievals = []
+                    scored = []
+                    for p in points:
+                        payload = p.get("payload", {})
+                        rc = payload.get("retrieval_count", 0) or 0
+                        retrievals.append(rc)
+                        date_str = payload.get("ingest_date") or payload.get("upload_date", "")
+                        if date_str:
+                            age_hours = (now - datetime.fromisoformat(date_str)).total_seconds() / 3600
+                        else:
+                            age_hours = 999999
+                        score = rc * RAG_ACCESS_WEIGHT + age_hours * RAG_AGE_WEIGHT
+                        last_accessed = payload.get("last_accessed", date_str)
+                        scored.append((score, last_accessed))
+
+                    avg_retrieval_count = round(sum(retrievals) / len(retrievals), 2)
+
+                    scored.sort(key=lambda x: (x[0], x[1]))
+                    at_risk_threshold = max(1, len(scored) // 10)
+                    at_risk_count = at_risk_threshold
+    except Exception as e:
+        log.warning(f"RAG operational stats scroll error: {e}")
+
+    stats.update({
+        "grace_hours": RAG_GRACE_HOURS,
+        "eviction_counts_last_1m": eviction_1m,
+        "eviction_counts_last_5m": eviction_5m,
+        "eviction_counts_last_30m": eviction_30m,
+        "pinned_count": pinned_count,
+        "avg_retrieval_count": avg_retrieval_count,
+        "at_risk_count": at_risk_count,
+    })
+    return stats

hardware.py

@@ -0,0 +1,102 @@
+"""
+JarvisChat — Startup hardware self-assessment.
+"""
+import asyncio
+import json
+import logging
+import subprocess
+from pathlib import Path
+
+import httpx
+import psutil
+
+from config import LLAMA_SERVER_BASE, SEARXNG_BASE
+
+log = logging.getLogger("jarvischat")
+
+HARDWARE_STATE_PATH = Path("hardware_state.json")
+_TIMEOUT_EXPIRED = subprocess.TimeoutExpired
+
+
+async def assess_hardware() -> dict:
+    mem = psutil.virtual_memory()
+    ram_total_gb = round(mem.total / (1024 ** 3), 1)
+    ram_available_gb = round(mem.available / (1024 ** 3), 1)
+    cpu_count = psutil.cpu_count()
+
+    vram_total_mb = 0
+    vram_free_mb = 0
+    try:
+        result = subprocess.run(
+            ["rocm-smi", "--showmeminfo", "vram", "--json"],
+            capture_output=True, text=True, timeout=5,
+        )
+        if result.returncode == 0:
+            data = json.loads(result.stdout)
+            for card, info in data.items():
+                tot = info.get("VRAM Total (MB)", 0)
+                free = info.get("VRAM Free (MB)", None)
+                used = info.get("VRAM Used (MB)", 0)
+                if tot:
+                    vram_total_mb += int(tot)
+                if free is not None:
+                    vram_free_mb += int(free)
+                else:
+                    vram_free_mb += int(tot) - int(used)
+    except (FileNotFoundError, _TIMEOUT_EXPIRED, json.JSONDecodeError):
+        log.warning("rocm-smi not available or failed — VRAM stats set to 0")
+    except Exception as e:
+        log.warning(f"rocm-smi error: {e}")
+
+    llama_reachable = False
+    llama_models = []
+    try:
+        async with httpx.AsyncClient(timeout=3) as client:
+            resp = await client.get(f"{LLAMA_SERVER_BASE}/v1/models")
+            if resp.status_code == 200:
+                llama_reachable = True
+                data = resp.json()
+                llama_models = [m.get("id", "") for m in data.get("data", [])]
+    except Exception:
+        log.warning("llama-server not reachable")
+
+    qdrant_reachable = False
+    qdrant_collections = []
+    try:
+        async with httpx.AsyncClient(timeout=3) as client:
+            resp = await client.get("http://192.168.50.108:6333/collections")
+            if resp.status_code == 200:
+                qdrant_reachable = True
+                data = resp.json()
+                raw = data.get("result", {}).get("collections", [])
+                qdrant_collections = [c.get("name", "") for c in raw]
+    except Exception:
+        log.warning("Qdrant not reachable")
+
+    searxng_reachable = False
+    try:
+        async with httpx.AsyncClient(timeout=3) as client:
+            resp = await client.get(SEARXNG_BASE)
+            if resp.status_code == 200:
+                searxng_reachable = True
+    except Exception:
+        log.warning("SearXNG not reachable")
+
+    state = {
+        "ram_total_gb": ram_total_gb,
+        "ram_available_gb": ram_available_gb,
+        "cpu_count": cpu_count,
+        "vram_total_mb": vram_total_mb,
+        "vram_free_mb": vram_free_mb,
+        "llama_reachable": llama_reachable,
+        "llama_models": llama_models,
+        "qdrant_reachable": qdrant_reachable,
+        "qdrant_collections": qdrant_collections,
+        "searxng_reachable": searxng_reachable,
+    }
+    HARDWARE_STATE_PATH.write_text(json.dumps(state, indent=2))
+    log.info(
+        f"HW: {ram_total_gb}GB RAM, {vram_total_mb}MB VRAM, "
+        f"llama={llama_reachable}, qdrant={qdrant_reachable}, searxng={searxng_reachable}"
+    )
+    return state

memory.py

@@ -62,7 +62,13 @@ def search_memories(query: str, limit: int = 5) -> list:
     if not words:
         db.close()
         return []
-    safe_query = " OR ".join(word + "*" for word in words[:10])
+    escaped = []
+    for word in words[:10]:
+        if word.upper() in {"AND", "OR", "NOT", "NEAR"}:
+            escaped.append(f'"{word}"*')
+        else:
+            escaped.append(word + "*")
+    safe_query = " OR ".join(escaped)
     try:
         rows = db.execute(
             "SELECT rowid, fact, topic, source, created_at, bm25(memories) AS rank "

rag.py

@@ -1,22 +1,45 @@
 """
 JarvisChat - RAG pipeline: Qdrant vector search + system prompt assembly.
 """
+import asyncio
 import logging
 
 import httpx
 
+from eviction import _update_retrieval_count
 from db import get_db, get_setting, list_skills_with_state, format_active_skills_prompt
 from memory import search_memories
-from config import MAX_SKILL_PROMPT_CHARS
+from config import MAX_SKILL_PROMPT_CHARS, QDRANT_URL, RAG_COLLECTION
 
 log = logging.getLogger("jarvischat")
 
-QDRANT_URL = "http://192.168.50.108:6333"
-EMBED_URL = "http://192.168.50.108:11434"
+EMBED_URL = "http://192.168.50.210:11434"
 EMBED_MODEL = "mxbai-embed-large"
-RAG_COLLECTION = "jarvis_rag"
 RAG_SCORE_THRESHOLD = 0.25
 
+# Re-export eviction symbols for backward compatibility
+from eviction import (  # noqa: E402
+    maybe_evict, get_rag_operational_stats, EVICTION_LOG,
+    get_collection_count, get_collection_stats, evict_batch,
+)
+
+
+def chunk_text(text: str, chunk_size: int = 512, overlap: int = 128) -> list:
+    words = text.split()
+    target_words = int(chunk_size / 1.3)
+    overlap_words = int(overlap / 1.3)
+    if not words:
+        return []
+    chunks = []
+    start = 0
+    while start < len(words):
+        end = min(start + target_words, len(words))
+        chunks.append(" ".join(words[start:end]))
+        if end == len(words):
+            break
+        start += target_words - overlap_words
+    return chunks
+
 
 async def query_rag(query: str, limit: int = 3) -> list:
     try:
@@ -36,7 +59,13 @@ async def query_rag(query: str, limit: int = 3) -> list:
             )
             if search_resp.status_code != 200:
                 return []
-            return search_resp.json().get("result", [])
+            results = search_resp.json().get("result", [])
+            for r in results:
+                pid = r.get("id")
+                if pid:
+                    current = r.get("payload", {}).get("retrieval_count", 0) or 0
+                    asyncio.ensure_future(_update_retrieval_count(pid, current))
+            return results
     except Exception as e:
         log.warning(f"RAG query error: {e}")
         return []
@@ -65,7 +94,7 @@ async def build_system_prompt(db, extra_prompt: str = "", user_message: str = ""
                 rag_lines = [r["payload"]["text"] for r in rag_results if r["score"] > RAG_SCORE_THRESHOLD]
                 if rag_lines:
                     parts.append("## Retrieved Context\n" + "\n\n---\n\n".join(rag_lines))
-                    log.warning(f"RAG injected {len(rag_lines)} chunks into context")
+                    log.info(f"RAG injected {len(rag_lines)} chunks into context")
         except Exception as e:
             log.warning(f"RAG injection error: {e}")
 

readme.md

@@ -1,355 +0,0 @@
-# ⚡ JarvisChat v1.7.8
-
-![screenshot](docs/images/screenshot.png)
-
-**A lightweight Ollama coding companion with persistent memory, web search, and real-time system monitoring.**
-
-Built with FastAPI + SQLite + Jinja2. Runs on Python 3.13. No Docker required.
-
-Developer wiki: [docs/wiki/Home.md](docs/wiki/Home.md)
-
-Core architecture deep-dive: [docs/wiki/Developer-Architecture.md](docs/wiki/Developer-Architecture.md)
-
-## Security Scope Disclaimer
-
-JarvisChat is designed for local and home-lab use (same host or trusted LAN).
-
-JarvisChat may technically work with frontier or commercial AI endpoints, but the author does not recommend or support that usage.
-
-Supported deployments are contained local/home-lab environments.
-
-By default, API access is limited to loopback + private LAN CIDRs. You can override with `JARVISCHAT_ALLOWED_CIDRS` (comma-separated CIDRs) and optionally trust reverse-proxy forwarding with `JARVISCHAT_TRUST_X_FORWARDED_FOR=true`.
-
-If you deploy outside a trusted local subnet, your risk profile changes significantly and the default protections here may be insufficient.
-
-Use at your own risk. No warranty is provided for Internet-exposed deployments.
-
-## What's New in v1.7.x
-
-- **Security hardening suite completed** - request rate limits, payload caps, settings allowlist, safe error envelopes, and LAN CIDR gate controls
-- **Customer-safe incident handling** - client-facing errors include support-friendly incident keys while full traces remain in server logs
-- **Streaming and regression test expansion** - automated coverage for SSE chat/search paths, memory remember/forget command handling, and auth/guardrail behavior
-- **Skills framework (Phase 1)** - built-in local skill registry with per-skill enable controls, API endpoints, and bounded prompt injection
-- **Skills WebUX controls** - Settings modal now includes a master skills toggle and per-skill toggles for admin users
-
-## What's New in v1.6.x
-
-- **Guest/admin capability split** - guest chat by default with 4-digit admin PIN for advanced or destructive operations
-- **Session + lockout controls** - session lifecycle endpoints, heartbeat, logout/revoke behavior, failed PIN lockout protections, and auth audit events
-- **Browser request protections** - strict origin checks for state-changing requests and admin-only write enforcement
-- **Unsafe link protection** - outbound search links sanitized to allow only http/https absolute URLs
-- **Operational stability fixes** - safer first-boot PIN policy handling and memory-search tokenization fix for punctuation/FTS edge cases
-
-## What's New in v1.5.0
-
-- **Explicit Web Search Button** — 🔍 button next to SEND forces a web search, bypassing model uncertainty detection
-- **Orange Search Styling** — Search results, WEB badge, and search button share consistent orange color scheme
-- **Expanded Refusal Patterns** — Added "As an AI model", "based on my training data", "I don't have the capability"
-- **Code cleanup** — Removed unused `JSONResponse` import and dead `raw_results_md` variable
-- **Bug fixes** — Replaced bare `except` clauses with `except Exception`; corrected `add_memory()` return type to `int | None`; updated `TemplateResponse` call to Starlette's current API signature
-
-## What's New in v1.4.0
-
-- **FTS5 Memory System**: Say "remember that..." to store facts — they're automatically retrieved by relevance and injected into context
-- **Forget Command**: Say "forget about..." to remove memories
-- **Memory Toggle**: Enable/disable memory injection from topbar or settings
-- **Multi-file Structure**: Backend and frontend separated for easier maintenance
-
-## Features
-
-- **Persistent Memory** — SQLite FTS5 full-text search for fast, relevant memory retrieval
-- **Web Search** — SearXNG integration for automatic web lookups when the model is uncertain
-- **Explicit Search** — 🔍 button to force web search without waiting for model uncertainty
-- **Profile Injection** — Custom system prompt injected into every conversation
-- **System Presets** — Save and switch between different system prompts
-- **Real-time Stats** — CPU, RAM, GPU, VRAM monitoring in sidebar
-- **Token Thermometer** — Visual context window usage indicator
-- **Streaming Responses** — Server-sent events for real-time token display
-- **Conversation History** — SQLite-backed chat persistence with mass-delete option
-- **Model Switching** — Change Ollama models on the fly
-
-## Current WiP (Prioritized)
-
-Canonical backlog: [docs/wiki/current-wip.md](docs/wiki/current-wip.md)
-
-Scope boundary: local-first (same-host Ollama), optional RFC1918 LAN endpoints, no public Internet AI endpoints by default.
-
-Total identified items: 27
-
-Top 10 (brief):
-
-1. P0 [DONE]: Add auth for write/admin endpoints
-2. P0 [DONE]: Add CSRF/origin protection for state-changing requests
-3. P0 [DONE]: Block unsafe URL schemes in rendered links
-4. P0 [DONE]: Add rate limiting and request size limits
-5. P1 [DONE]: Restrict `/api/settings` updates to allowlisted keys
-6. P1: Add pagination + hard caps for list APIs
-7. P1 [DONE]: Replace raw exception leakage with safe client errors
-8. P1 [DONE]: Add automated tests for streaming/search/memory paths
-9. P2 [DONE]: Implement MCP-style skills/tool-call framework
-10. P2: Implement heartbeat/check-in scheduler + summary endpoint
-
-Item 1 executive summary: keep guest mode for conversational chat, require 4-digit admin PIN for advanced/destructive actions, and enforce local/LAN-only backend policy by default.
-
-Implementation status: complete (guest session by default + admin unlock + admin-only write enforcement + origin checks + safe-link sanitization + audit logging + rate/payload guardrails + capability tests).
-
-## TODO
-
-1. ~~Verify SearXNG and Docker services persist across reboots~~
-2. Conversation search/filter by keyword
-3. Export conversation to markdown/text
-4. Keyboard shortcuts (Ctrl+N new chat, Ctrl+Enter send)
-5. Retry button on assistant messages
-6. Source links — clickable links when search used
-7. Allow conversation renaming
-8. Multiple profiles — coding/sysadmin/general
-9. Auto-generate conversation tags (client-side KWIC, top 5, filterable badges)
-10. Image input support — pull vision model, file input/drag-drop, base64 encode, pass `images` array to Ollama `/api/chat`
-11. Split-screen option for btop display
-12. Skills as markdown files — `/opt/jarvischat/skills/`, YAML frontmatter + instructions, injected into context for tool calls
-13. Heartbeats / proactive check-ins — cron + endpoint for daily briefings, HA anomaly alerts
-14. Model info button — (i) icon next to Model dropdown, shows div with model description, last updated date, best-use purpose
-15. Set default model — toggle any model as the default selection
-16. Hide/remove model from list — exclude models from dropdown
-17. Update model function — trigger `ollama pull` for selected model from UI
-18. Add mouseover tooltip to SEND button
-19. Add preflight validation for required model/preset selection and show a clear warning before send to prevent avoidable timeout loops
-
-## File Structure
-
-```
-/opt/jarvischat/
-├── app.py              # FastAPI backend
-├── jarvischat.db       # SQLite database (auto-created)
-├── static/
-│   └── logo.png        # Logo image (optional)
-└── templates/
-    └── index.html      # Frontend
-```
-
-## Requirements
-
-- Python 3.11+ (tested on 3.13)
-- Ollama running locally or on network
-- SearXNG (optional, for web search)
-
-## Installation
-
-### Fresh Install
-
-```bash
-# Create directory and venv
-sudo mkdir -p /opt/jarvischat
-sudo chown $USER:$USER /opt/jarvischat
-cd /opt/jarvischat
-python3 -m venv venv
-
-# Install dependencies
-./venv/bin/pip install fastapi uvicorn httpx psutil jinja2 python-multipart
-
-# Set admin PIN before first startup (4 digits)
-export JARVISCHAT_ADMIN_PIN=4827
-
-# Create subdirectories
-mkdir -p templates static
-
-# Copy files
-# (copy app.py to /opt/jarvischat/)
-# (copy index.html to /opt/jarvischat/templates/)
-# (copy logo.png to /opt/jarvischat/static/ — optional)
-```
-
-WARNING: Do not use `1234` as your admin PIN unless you accept weak local security.
-
-NOTE: First boot now requires `JARVISCHAT_ADMIN_PIN` unless you explicitly opt into insecure fallback with `JARVISCHAT_ALLOW_DEFAULT_PIN=true`.
-
-### Upgrading from v1.4.x
-
-```bash
-cd /opt/jarvischat
-
-# Backup
-cp app.py app.py.bak
-cp templates/index.html templates/index.html.bak
-
-# Copy new files
-# (copy app.py, replacing old version)
-# (copy index.html to templates/)
-
-# Restart
-sudo systemctl restart jarvischat
-```
-
-## Systemd Service
-
-Create `/etc/systemd/system/jarvischat.service`:
-
-```ini
-[Unit]
-Description=JarvisChat - Local Ollama Web Interface
-After=network.target
-
-[Service]
-Type=simple
-User=jarvischat
-Group=jarvischat
-WorkingDirectory=/opt/jarvischat
-ExecStart=/opt/jarvischat/venv/bin/uvicorn app:app --host 0.0.0.0 --port 8080
-Restart=always
-RestartSec=5
-
-[Install]
-WantedBy=multi-user.target
-```
-
-```bash
-sudo systemctl daemon-reload
-sudo systemctl enable jarvischat
-sudo systemctl start jarvischat
-```
-
-## Memory Commands
-
-In chat, natural language triggers memory operations:
-
-| You say | What happens |
-|---------|--------------|
-| "remember that I prefer Rust over Go" | Stores as `preference` |
-| "remember that JarvisChat runs on port 8080" | Stores as `infrastructure` |
-| "note that the deadline is Friday" | Stores as `general` |
-| "forget about the deadline" | Removes matching memories |
-
-Memories are automatically searched based on your message content and injected into the system prompt when relevant.
-
-### Memory Topics
-
-Memories are auto-categorized:
-- `preference` — likes, dislikes, choices
-- `project` — active work, repos, tasks
-- `infrastructure` — servers, services, configs
-- `personal` — name, location, background
-- `general` — everything else
-
-## API Endpoints
-
-### Memory
-
-| Method | Endpoint | Description |
-|--------|----------|-------------|
-| GET | `/api/memories` | List all memories |
-| POST | `/api/memories` | Add memory `{"fact": "...", "topic": "general"}` |
-| DELETE | `/api/memories/{rowid}` | Delete memory by ID |
-| GET | `/api/memories/search?q=term` | Search memories |
-| GET | `/api/memories/stats` | Get counts by topic |
-
-### Chat & Models
-
-| Method | Endpoint | Description |
-|--------|----------|-------------|
-| GET | `/api/models` | List available Ollama models |
-| POST | `/api/chat` | Send message (streaming SSE) |
-| POST | `/api/search` | Explicit web search (streaming SSE) |
-| POST | `/api/show` | Get model info (context size) |
-| GET | `/api/ps` | Get running models |
-
-### Settings & Profile
-
-| Method | Endpoint | Description |
-|--------|----------|-------------|
-| GET | `/api/profile` | Get profile content |
-| PUT | `/api/profile` | Update profile |
-| GET | `/api/profile/default` | Get default profile |
-| GET | `/api/settings` | Get settings |
-| PUT | `/api/settings` | Update settings |
-
-### Conversations
-
-| Method | Endpoint | Description |
-|--------|----------|-------------|
-| GET | `/api/conversations` | List conversations |
-| GET | `/api/conversations/{id}` | Get conversation with messages |
-| DELETE | `/api/conversations/{id}` | Delete conversation |
-| DELETE | `/api/conversations` | Delete ALL conversations |
-
-### Presets
-
-| Method | Endpoint | Description |
-|--------|----------|-------------|
-| GET | `/api/presets` | List presets |
-| POST | `/api/presets` | Create preset |
-| PUT | `/api/presets/{id}` | Update preset |
-| DELETE | `/api/presets/{id}` | Delete preset |
-
-### System
-
-| Method | Endpoint | Description |
-|--------|----------|-------------|
-| GET | `/api/stats` | CPU, RAM, GPU, VRAM stats |
-| GET | `/api/search/status` | SearXNG availability |
-
-## Configuration
-
-Settings are stored in the `settings` table and include:
-
-- `profile_enabled` — Inject profile into chats (true/false)
-- `search_enabled` — Auto web search (true/false)
-- `memory_enabled` — Memory injection (true/false)
-- `default_model` — Default Ollama model
-- `searxng_url` — SearXNG instance URL (default: `http://localhost:8888`)
-
-## Testing Memory
-
-```bash
-# Add a memory via API
-curl -X POST http://jarvis:8080/api/memories \
-  -H "Content-Type: application/json" \
-  -d '{"fact": "User prefers native installs over Docker", "topic": "preference"}'
-
-# Search memories
-curl "http://jarvis:8080/api/memories/search?q=docker"
-
-# List all memories
-curl http://jarvis:8080/api/memories
-
-# Get stats
-curl http://jarvis:8080/api/memories/stats
-```
-
-Or in chat:
-1. Say "remember that I hate YAML"
-2. Later ask "what markup languages should I avoid?"
-3. JarvisChat will inject the YAML preference into context
-
-## Troubleshooting
-
-### Service won't start
-
-Check logs:
-```bash
-journalctl -u jarvischat -n 50 --no-pager
-```
-
-Common issues:
-- Missing `jinja2`: `./venv/bin/pip install jinja2`
-- Missing `templates/` directory
-- Wrong permissions on `/opt/jarvischat`
-
-### Memory not working
-
-1. Check memory is enabled (🧠 MEM ON in topbar)
-2. Verify memories exist: `curl http://jarvis:8080/api/memories`
-3. Check FTS5 table: `sqlite3 jarvischat.db "SELECT * FROM memories_fts;"`
-
-### Web search not working
-
-1. Verify SearXNG is running: `curl http://localhost:8888/search?q=test&format=json`
-2. Check search status: `curl http://jarvis:8080/api/search/status`
-3. Ensure JSON format is enabled in SearXNG settings
-
-## License
-
-MIT
-
-## Repository
-
-Gitea: `ssh://gitea@llgit.llamachile.tube:1319/gramps/jarvisChat.git`

requirements.txt

@@ -1,3 +1,5 @@
 fastapi>=0.115.0
 uvicorn[standard]>=0.32.0
 httpx>=0.27.0
+pypdf>=5.0.0
+python-multipart>=0.0.9

routers/chat.py

@@ -9,7 +9,7 @@ from fastapi import APIRouter, HTTPException, Request
 from fastapi.responses import StreamingResponse
 
 from config import DEFAULT_MODEL, LLAMA_SERVER_BASE
-from db import get_db
+from db import get_db, get_upload_context
 from memory import process_remember_command
 from rag import build_system_prompt
 from search import (calculate_perplexity, is_uncertain, is_refusal,
@@ -26,7 +26,7 @@ def parse_llama_stream_chunk(line: str) -> tuple:
     if line.startswith("data: "):
         line = line[6:]
     if line.strip() == "[DONE]":
-        return None, True, {}
+        return None, True, {}, []
     try:
         chunk = json.loads(line)
         choices = chunk.get("choices", [])
@@ -35,10 +35,17 @@ def parse_llama_stream_chunk(line: str) -> tuple:
             token = delta.get("content")
             finish = choices[0].get("finish_reason")
             stats = {}
+            logprobs_list = []
+            logprobs_info = choices[0].get("logprobs")
+            if logprobs_info:
+                content_logprobs = logprobs_info.get("content", [])
+                for entry in content_logprobs:
+                    if "logprob" in entry:
+                        logprobs_list.append({"logprob": entry["logprob"]})
             if finish == "stop":
                 usage = chunk.get("usage", {})
                 stats["tokens_per_sec"] = usage.get("tokens_per_second", 0.0)
-            return token, finish == "stop", stats
+            return token, finish == "stop", stats, logprobs_list
         if "message" in chunk and "content" in chunk["message"]:
             token = chunk["message"]["content"]
             done = chunk.get("done", False)
@@ -47,10 +54,10 @@ def parse_llama_stream_chunk(line: str) -> tuple:
                 eval_count = chunk.get("eval_count", 0)
                 eval_duration = chunk.get("eval_duration", 0)
                 stats["tokens_per_sec"] = (eval_count / (eval_duration / 1e9)) if eval_duration > 0 else 0
-            return token, done, stats
+            return token, done, stats, []
     except json.JSONDecodeError:
         pass
-    return None, False, {}
+    return None, False, {}, []
 
 
 @router.post("/api/chat")
@@ -62,6 +69,7 @@ async def chat(request: Request):
         raise HTTPException(status_code=413, detail="Chat message is too long")
     model = body.get("model", DEFAULT_MODEL)
     preset_prompt = body.get("system_prompt", "")
+    upload_context_id = body.get("upload_context_id")
 
     if not user_message:
         raise HTTPException(status_code=400, detail="Empty message")
@@ -71,6 +79,14 @@ async def chat(request: Request):
     settings = {row["key"]: row["value"] for row in db.execute("SELECT key, value FROM settings").fetchall()}
     search_enabled = settings.get("search_enabled", "true") == "true"
 
+    upload_doc = None
+    if upload_context_id:
+        ctx = get_upload_context(db, upload_context_id)
+        if ctx:
+            upload_doc = f"[ATTACHED DOCUMENT: {ctx['filename']}]\n{ctx['content']}\n[END DOCUMENT]"
+        else:
+            log.warning(f"upload_context_id {upload_context_id} not found or expired, continuing without it")
+
     remember_response = process_remember_command(user_message)
 
     if not conv_id:
@@ -88,7 +104,10 @@ async def chat(request: Request):
     history_rows = db.execute(
         "SELECT role, content FROM messages WHERE conversation_id = ? ORDER BY id ASC", (conv_id,)
     ).fetchall()
-    system_prompt = await build_system_prompt(db, preset_prompt, user_message)
+    extra_prompt = preset_prompt
+    if upload_doc:
+        extra_prompt = (extra_prompt + "\n\n" + upload_doc) if extra_prompt else upload_doc
+    system_prompt = await build_system_prompt(db, extra_prompt, user_message)
     db.close()
 
     messages = []
@@ -97,7 +116,7 @@ async def chat(request: Request):
     for row in history_rows:
         messages.append({"role": row["role"], "content": row["content"]})
 
-    ollama_payload = {"model": model, "messages": messages, "stream": True}
+    upstream_payload = {"model": model, "messages": messages, "stream": True, "logprobs": True}
 
     async def stream_response():
         full_response = []
@@ -111,12 +130,14 @@ async def chat(request: Request):
             try:
                 async with client.stream(
                     "POST", f"{LLAMA_SERVER_BASE}/v1/chat/completions",
-                    json=ollama_payload,
+                    json=upstream_payload,
                     timeout=httpx.Timeout(300.0, connect=10.0),
                 ) as resp:
                     async for line in resp.aiter_lines():
                         if line.strip():
-                            token, done, stats = parse_llama_stream_chunk(line)
+                            token, done, stats, chunk_logprobs = parse_llama_stream_chunk(line)
+                            if chunk_logprobs:
+                                all_logprobs.extend(chunk_logprobs)
                             if token:
                                 full_response.append(token)
                                 yield f"data: {json.dumps({'token': token, 'conversation_id': conv_id})}\n\n"
@@ -153,7 +174,7 @@ async def chat(request: Request):
                         ) as resp2:
                             async for line in resp2.aiter_lines():
                                 if line.strip():
-                                    token2, done2, _ = parse_llama_stream_chunk(line)
+                                    token2, done2, _, _ = parse_llama_stream_chunk(line)
                                     if token2:
                                         augmented_response.append(token2)
                                     if done2:
@@ -194,9 +215,9 @@ async def chat(request: Request):
             except httpx.RemoteProtocolError:
                 pass
             except httpx.ConnectError:
-                yield f"data: {json.dumps({'error': 'Cannot connect to Ollama. Is it running?'})}\n\n"
+                yield f"data: {json.dumps({'error': 'Cannot connect to inference server. Is it running?'})}\n\n"
             except Exception as e:
-                incident_key = log_incident("chat_stream", message="Ollama stream failure during chat response",
+                incident_key = log_incident("chat_stream", message="Inference stream failure during chat response",
                                             request=request, exc=e)
                 yield f"data: {json.dumps({'error': 'Chat response generation failed before completion. Use the incident key for support lookup.', 'error_key': incident_key})}\n\n"
 

routers/completions.py

@@ -178,7 +178,7 @@ async def _stream_chat(payload: dict, model: str, conv_id: str, request: Request
                 async for line in resp.aiter_lines():
                     if not line.strip():
                         continue
-                    token, done, _ = parse_llama_stream_chunk(line)
+                    token, done, _, _ = parse_llama_stream_chunk(line)
                     if token:
                         full_response.append(token)
                         yield _build_openai_chunk(token, model, conv_id)
@@ -222,7 +222,7 @@ async def _blocking_chat(payload: dict, model: str, conv_id: str, request: Reque
                 async for line in resp.aiter_lines():
                     if not line.strip():
                         continue
-                    token, done, _ = parse_llama_stream_chunk(line)
+                    token, done, _, _ = parse_llama_stream_chunk(line)
                     if token:
                         full_response.append(token)
                     if done:

routers/conversations.py

@@ -15,8 +15,16 @@ router = APIRouter()
 async def list_conversations():
     db = get_db()
     rows = db.execute("SELECT * FROM conversations ORDER BY updated_at DESC").fetchall()
+    result = []
+    for r in rows:
+        c = dict(r)
+        attach_count = db.execute(
+            "SELECT COUNT(*) FROM upload_context WHERE conversation_id = ?", (c["id"],)
+        ).fetchone()[0]
+        c["attachment_count"] = attach_count
+        result.append(c)
     db.close()
-    return [dict(r) for r in rows]
+    return result
 
 
 @router.post("/api/conversations")

routers/hardware.py

@@ -0,0 +1,15 @@
+"""JarvisChat routers — Hardware self-assessment endpoint."""
+import json
+
+from fastapi import APIRouter
+
+import hardware
+
+router = APIRouter()
+
+
+@router.get("/api/hardware")
+async def get_hardware_state():
+    if hardware.HARDWARE_STATE_PATH.exists():
+        return json.loads(hardware.HARDWARE_STATE_PATH.read_text())
+    return {"status": "not_ready", "message": "Hardware assessment not yet complete"}

routers/ingest.py

@@ -0,0 +1,70 @@
+"""JarvisChat routers - /api/ingest terminal command RAG hook."""
+import logging
+from datetime import datetime, timezone
+
+import httpx
+from fastapi import APIRouter, HTTPException, Request
+from fastapi.responses import JSONResponse
+
+from config import COMPLETIONS_API_KEY
+from eviction import maybe_evict
+from rag import chunk_text, QDRANT_URL, EMBED_URL, EMBED_MODEL, RAG_COLLECTION
+
+log = logging.getLogger("jarvischat")
+router = APIRouter()
+
+
+def _check_api_key(request: Request):
+    auth = request.headers.get("Authorization", "")
+    if not auth.startswith("Bearer "):
+        raise HTTPException(status_code=401, detail="Missing Bearer token")
+    token = auth[7:].strip()
+    if token != COMPLETIONS_API_KEY:
+        raise HTTPException(status_code=401, detail="Invalid API key")
+
+
+@router.post("/api/ingest")
+async def ingest_content(request: Request):
+    _check_api_key(request)
+    body = await request.json()
+    content = (body.get("content") or "").strip()
+    if not content:
+        raise HTTPException(status_code=422, detail="content is required")
+    source = str(body.get("source", "external")).strip() or "external"
+    metadata = body.get("metadata") or {}
+
+    chunks = chunk_text(content)
+    if not chunks:
+        raise HTTPException(status_code=422, detail="content produced no chunks")
+
+    ingested = 0
+    async with httpx.AsyncClient() as client:
+        for i, chunk in enumerate(chunks):
+            embed_resp = await client.post(
+                f"{EMBED_URL}/api/embeddings",
+                json={"model": EMBED_MODEL, "prompt": chunk},
+                timeout=30.0,
+            )
+            if embed_resp.status_code != 200:
+                log.warning(f"Ingest embedding failed for chunk {i}: {embed_resp.status_code}")
+                continue
+            vector = embed_resp.json()["embedding"]
+            point_id = f"ingest-{source}-{datetime.now(timezone.utc).timestamp()}-{i}"
+            payload = {"text": chunk, "source": source, "ingest_date": datetime.now(timezone.utc).isoformat(), "type": "ingest"}
+            payload.update(metadata)
+            upsert_resp = await client.put(
+                f"{QDRANT_URL}/collections/{RAG_COLLECTION}/points?wait=true",
+                json={"points": [{"id": point_id, "vector": vector, "payload": payload}]},
+                timeout=30.0,
+            )
+            if upsert_resp.status_code in (200, 201):
+                ingested += 1
+            else:
+                log.warning(f"Ingest Qdrant upsert failed for chunk {i}: {upsert_resp.status_code}")
+
+    if ingested > 0:
+        evicted = await maybe_evict()
+        if evicted:
+            log.info(f"Evicted {evicted} vectors after ingest")
+
+    return {"chunks_ingested": ingested, "source": source, "message": f"Ingested {ingested} chunks from {source}"}

routers/models.py

@@ -8,7 +8,7 @@ import httpx
 import psutil
 from fastapi import APIRouter, HTTPException, Request
 
-from config import OLLAMA_BASE
+from config import LLAMA_SERVER_BASE
 from gpu import get_gpu_stats
 from security import read_json_body, BODY_LIMIT_DEFAULT_BYTES
 
@@ -20,34 +20,33 @@ router = APIRouter()
 async def list_models():
     async with httpx.AsyncClient() as client:
         try:
-            resp = await client.get(f"{OLLAMA_BASE}/v1/models", timeout=10)
+            resp = await client.get(f"{LLAMA_SERVER_BASE}/v1/models", timeout=10)
             data = resp.json()
             models = [{"name": m["id"], "model": m["id"]} for m in data.get("data", [])]
             return {"models": models}
         except httpx.ConnectError:
-            raise HTTPException(status_code=502, detail="Cannot connect to llama-server.")
+            raise HTTPException(status_code=502, detail="Cannot connect to inference server.")
 
 
 @router.get("/api/ps")
 async def running_models():
     async with httpx.AsyncClient() as client:
         try:
-            resp = await client.get(f"{OLLAMA_BASE}/api/ps", timeout=10)
+            resp = await client.get(f"{LLAMA_SERVER_BASE}/v1/models", timeout=10)
             return resp.json()
         except httpx.ConnectError:
-            raise HTTPException(status_code=502, detail="Cannot connect to Ollama.")
+            raise HTTPException(status_code=502, detail="Cannot connect to inference server.")
 
 
 @router.post("/api/show")
 async def show_model(request: Request):
-    from security import BODY_LIMIT_DEFAULT_BYTES
     body = await read_json_body(request, BODY_LIMIT_DEFAULT_BYTES)
     async with httpx.AsyncClient() as client:
         try:
-            resp = await client.post(f"{OLLAMA_BASE}/api/show", json=body, timeout=10)
+            resp = await client.post(f"{LLAMA_SERVER_BASE}/api/show", json=body, timeout=10)
             return resp.json()
         except httpx.ConnectError:
-            raise HTTPException(status_code=502, detail="Cannot connect to Ollama.")
+            raise HTTPException(status_code=502, detail="Cannot connect to inference server.")
 
 
 @router.get("/api/stats")

routers/rag_admin.py

@@ -0,0 +1,60 @@
+"""JarvisChat routers — RAG corpus management admin endpoints."""
+import logging
+
+import httpx
+from fastapi import APIRouter, HTTPException, Request
+from fastapi.responses import JSONResponse
+
+from eviction import get_rag_operational_stats, EVICTION_LOG
+from rag import QDRANT_URL, RAG_COLLECTION
+
+log = logging.getLogger("jarvischat")
+router = APIRouter()
+
+
+@router.get("/api/rag/stats")
+async def rag_stats(request: Request):
+    if getattr(request.state, "session_role", "none") != "admin":
+        raise HTTPException(status_code=403, detail="Admin PIN required for this action")
+    stats = await get_rag_operational_stats()
+    stats["eviction_log_size"] = len(EVICTION_LOG)
+    return stats
+
+
+@router.post("/api/rag/flush")
+async def rag_flush(request: Request):
+    try:
+        async with httpx.AsyncClient() as client:
+            scroll_resp = await client.post(
+                f"{QDRANT_URL}/collections/{RAG_COLLECTION}/points/scroll",
+                json={"limit": 10000, "with_payload": False, "with_vector": False},
+                timeout=30.0,
+            )
+            if scroll_resp.status_code != 200:
+                return JSONResponse(status_code=502, content={"detail": f"Qdrant scroll failed: {scroll_resp.status_code}"})
+
+            all_points = scroll_resp.json().get("result", {}).get("points", [])
+            point_ids = [p["id"] for p in all_points]
+
+            if not point_ids:
+                return {"deleted_count": 0, "collection": RAG_COLLECTION, "status": "flushed"}
+
+            delete_resp = await client.post(
+                f"{QDRANT_URL}/collections/{RAG_COLLECTION}/points/delete",
+                json={"points": point_ids},
+                timeout=30.0,
+            )
+            if delete_resp.status_code not in (200, 201):
+                return JSONResponse(status_code=502, content={"detail": f"Qdrant delete failed: {delete_resp.status_code}"})
+
+            EVICTION_LOG.clear()
+            log.warning(f"RAG collection '{RAG_COLLECTION}' flushed ({len(point_ids)} points deleted)")
+
+            return {
+                "deleted_count": len(point_ids),
+                "collection": RAG_COLLECTION,
+                "status": "flushed",
+            }
+    except Exception as e:
+        log.warning(f"RAG flush error: {e}")
+        return JSONResponse(status_code=502, content={"detail": f"RAG flush error: {e}"})

routers/search_route.py

@@ -35,14 +35,14 @@ async def explicit_search(request: Request):
 
     if not conv_id:
         conv_id = str(uuid.uuid4())
-        title = f"🔍 {query[:70]}..." if len(query) > 70 else f"🔍 {query}"
+        title = query[:70] + "..." if len(query) > 70 else query
         db.execute("INSERT INTO conversations (id, title, model, created_at, updated_at) VALUES (?, ?, ?, ?, ?)",
                    (conv_id, title, model, now, now))
     else:
         db.execute("UPDATE conversations SET updated_at = ? WHERE id = ?", (now, conv_id))
 
     db.execute("INSERT INTO messages (conversation_id, role, content, created_at) VALUES (?, ?, ?, ?)",
-               (conv_id, "user", f"🔍 {query}", now))
+               (conv_id, "user", query, now))
     db.commit()
     db.close()
 
@@ -80,7 +80,7 @@ async def explicit_search(request: Request):
                 ) as resp:
                     async for line in resp.aiter_lines():
                         if line.strip():
-                            token, done, _ = parse_llama_stream_chunk(line)
+                            token, done, _, _ = parse_llama_stream_chunk(line)
                             if token:
                                 full_response.append(token)
                                 yield f"data: {json.dumps({'token': token, 'conversation_id': conv_id})}\n\n"
@@ -102,7 +102,6 @@ async def explicit_search(request: Request):
         db2.commit()
         db2.close()
 
-        yield f"data: {json.dumps({'raw_results': results, 'conversation_id': conv_id})}\n\n"
         yield f"data: {json.dumps({'done': True, 'conversation_id': conv_id, 'searched': True})}\n\n"
 
     return StreamingResponse(stream_search(), media_type="text/event-stream")

routers/upload.py

@@ -0,0 +1,173 @@
+"""JarvisChat routers - /api/upload file/document attachment endpoint."""
+import json
+import logging
+import os
+from datetime import datetime, timezone, timedelta
+
+import httpx
+from fastapi import APIRouter, HTTPException, Request, UploadFile, File, Form
+from fastapi.responses import JSONResponse
+
+from config import UPLOAD_DIR, MAX_UPLOAD_BYTES, SUPPORTED_UPLOAD_TYPES, UPLOAD_CONTEXT_EXPIRY_HOURS
+from db import get_db, insert_upload_context, list_upload_context_by_conversation, delete_upload_context_by_id
+from eviction import maybe_evict
+from rag import chunk_text, QDRANT_URL, EMBED_URL, EMBED_MODEL, RAG_COLLECTION
+
+log = logging.getLogger("jarvischat")
+router = APIRouter()
+
+
+def _point_id(filename: str, chunk_idx: int) -> str:
+    return f"upload-{filename}-{chunk_idx}"
+
+
+@router.post("/api/upload")
+async def upload_file(
+    request: Request,
+    file: UploadFile = File(...),
+    mode: str = Form("both"),
+    conversation_id: str = Form(""),
+):
+    if mode not in ("context", "ingest", "both"):
+        raise HTTPException(status_code=422, detail="mode must be context, ingest, or both")
+
+    if file.size and file.size > MAX_UPLOAD_BYTES:
+        return JSONResponse(status_code=413, content={"detail": f"File exceeds {MAX_UPLOAD_BYTES} byte limit"})
+
+    content_type = file.content_type or "application/octet-stream"
+    if content_type not in SUPPORTED_UPLOAD_TYPES:
+        return JSONResponse(status_code=415, content={"detail": f"Unsupported file type: {content_type}"})
+
+    raw_bytes = await file.read()
+    if not raw_bytes:
+        raise HTTPException(status_code=422, detail="Empty file")
+
+    if content_type == "application/pdf":
+        try:
+            from pypdf import PdfReader
+            import io
+            reader = PdfReader(io.BytesIO(raw_bytes))
+            extracted = "\n".join(page.extract_text() or "" for page in reader.pages)
+        except Exception as e:
+            log.warning(f"PDF extraction error: {e}")
+            raise HTTPException(status_code=422, detail="Failed to extract text from PDF")
+    else:
+        extracted = raw_bytes.decode("utf-8", errors="replace")
+
+    result = {"filename": file.filename, "size_bytes": len(raw_bytes), "mode": mode}
+
+    if mode in ("ingest", "both"):
+        os.makedirs(UPLOAD_DIR, exist_ok=True)
+        chunks = chunk_text(extracted)
+        ingested = 0
+        async with httpx.AsyncClient() as client:
+            for i, chunk in enumerate(chunks):
+                embed_resp = await client.post(
+                    f"{EMBED_URL}/api/embeddings",
+                    json={"model": EMBED_MODEL, "prompt": chunk},
+                    timeout=30.0,
+                )
+                if embed_resp.status_code != 200:
+                    log.warning(f"Embedding failed for chunk {i}: {embed_resp.status_code}")
+                    continue
+                vector = embed_resp.json()["embedding"]
+                pid = _point_id(file.filename or "unnamed", i)
+                upsert_resp = await client.put(
+                    f"{QDRANT_URL}/collections/{RAG_COLLECTION}/points?wait=true",
+                    json={
+                        "points": [{
+                            "id": pid,
+                            "vector": vector,
+                            "payload": {"text": chunk, "source": file.filename, "upload_date": datetime.now(timezone.utc).isoformat(), "type": "upload"},
+                        }]
+                    },
+                    timeout=30.0,
+                )
+                if upsert_resp.status_code in (200, 201):
+                    ingested += 1
+                else:
+                    log.warning(f"Qdrant upsert failed for chunk {i}: {upsert_resp.status_code}")
+        result["chunks_ingested"] = ingested
+        if ingested > 0:
+            evicted = await maybe_evict()
+            if evicted:
+                log.info(f"Evicted {evicted} vectors after upload")
+
+    if mode in ("context", "both"):
+        expires = (datetime.now(timezone.utc) + timedelta(hours=UPLOAD_CONTEXT_EXPIRY_HOURS)).isoformat()
+        db = get_db()
+        try:
+            cid = insert_upload_context(db, conversation_id or "", file.filename or "unnamed", extracted, expires, content_type)
+            db.commit()
+            result["context_id"] = cid
+        finally:
+            db.close()
+
+    result["message"] = f"Uploaded {file.filename}"
+    return result
+
+
+@router.patch("/api/upload/{context_id}/link")
+async def link_upload_to_conversation(context_id: int, request: Request):
+    body = await request.json()
+    conv_id = body.get("conversation_id", "").strip()
+    if not conv_id:
+        raise HTTPException(status_code=422, detail="conversation_id required")
+    db = get_db()
+    try:
+        row = db.execute("SELECT id FROM upload_context WHERE id = ?", (context_id,)).fetchone()
+        if not row:
+            raise HTTPException(status_code=404, detail="Upload context not found")
+        db.execute("UPDATE upload_context SET conversation_id = ? WHERE id = ?", (conv_id, context_id))
+        db.commit()
+        return {"status": "ok"}
+    finally:
+        db.close()
+
+
+@router.get("/api/upload/by-conversation/{conv_id}")
+async def get_upload_by_conversation(conv_id: str):
+    db = get_db()
+    try:
+        items = list_upload_context_by_conversation(db, conv_id)
+        return items
+    finally:
+        db.close()
+
+
+@router.delete("/api/upload/{context_id}")
+async def delete_upload(context_id: int):
+    db = get_db()
+    try:
+        row = db.execute("SELECT filename FROM upload_context WHERE id = ?", (context_id,)).fetchone()
+        if not row:
+            raise HTTPException(status_code=404, detail="Attachment not found")
+        filename = row["filename"]
+        if not filename:
+            filename = "unnamed"
+        delete_upload_context_by_id(db, context_id)
+        db.commit()
+    finally:
+        db.close()
+
+    try:
+        async with httpx.AsyncClient() as client:
+            scroll_resp = await client.post(
+                f"{QDRANT_URL}/collections/{RAG_COLLECTION}/points/scroll",
+                json={"filter": {"must": [{"key": "source", "match": {"value": filename}}]}, "limit": 100, "with_payload": False},
+                timeout=10.0,
+            )
+            if scroll_resp.status_code == 200:
+                points = scroll_resp.json().get("result", {}).get("points", [])
+                point_ids = [p["id"] for p in points]
+                if point_ids:
+                    await client.post(
+                        f"{QDRANT_URL}/collections/{RAG_COLLECTION}/points/delete",
+                        json={"points": point_ids},
+                        timeout=10.0,
+                    )
+                    log.info(f"Deleted {len(point_ids)} Qdrant points for source '{filename}'")
+    except Exception as e:
+        log.warning(f"Qdrant cleanup for '{filename}' failed: {e}")
+
+    return {"status": "ok", "filename": filename}

search.py

@@ -80,16 +80,13 @@ def format_direct_answer(question: str, results: list) -> str:
 
 def extract_search_query(user_message: str) -> str:
     query = user_message.strip()
-    if re.search(r"temperature|weather", query, re.IGNORECASE):
-        query = re.sub(r"^what('?s| is) the ", "", query, flags=re.IGNORECASE) + " right now degrees"
-    if re.search(r"price|spot price", query, re.IGNORECASE):
-        query = re.sub(r"^(what('?s| is)|can you tell me) the ", "", query, flags=re.IGNORECASE) + " today USD"
-    query = re.sub(
-        r"^(what|who|where|when|why|how|is|are|can|could|would|should|do|does|did)\s+",
-        "", query, flags=re.IGNORECASE,
-    )
-    query = re.sub(r"[?!.]+$", "", query)
-    return query[:100].strip() or user_message[:100]
+    weather_lead = re.match(r"^(?:what('?s| is) the\s+)?(?:weather|temperature|forecast)\s+(?:in\s+|for\s+)?(.+)", query, re.IGNORECASE)
+    if weather_lead:
+        return (weather_lead.group(2) + " weather").strip()[:100]
+    price_lead = re.match(r"^(?:what('?s| is| are)\s+)?(?:the\s+)?(?:price|spot price)\s+(?:of\s+|for\s+)?(.+)", query, re.IGNORECASE)
+    if price_lead:
+        return (price_lead.group(2) + " price today USD").strip()[:100]
+    return query[:100]
 
 
 async def query_searxng(query: str, max_results: int = 5) -> list:

security.py

@@ -22,7 +22,7 @@ from fastapi import HTTPException, Request
 
 from config import (
     ALLOWED_NETWORKS, TRUST_X_FORWARDED_FOR, TRUSTED_ORIGINS,
-    BODY_LIMIT_DEFAULT_BYTES, BODY_LIMIT_CHAT_BYTES, BODY_LIMIT_PROFILE_BYTES,
+    BODY_LIMIT_DEFAULT_BYTES, BODY_LIMIT_CHAT_BYTES, BODY_LIMIT_PROFILE_BYTES, BODY_LIMIT_UPLOAD_BYTES,
     RATE_WINDOW_SECONDS, RL_LOGIN_PER_WINDOW, RL_CHAT_PER_WINDOW,
     RL_SEARCH_PER_WINDOW, RL_STATS_PER_WINDOW, RL_WRITE_PER_WINDOW,
     RL_DEFAULT_PER_WINDOW, VERSION,
@@ -114,6 +114,8 @@ def request_body_limit(path: str) -> int:
         return BODY_LIMIT_CHAT_BYTES
     if path == "/api/profile":
         return BODY_LIMIT_PROFILE_BYTES
+    if path == "/api/upload":
+        return BODY_LIMIT_UPLOAD_BYTES
     return BODY_LIMIT_DEFAULT_BYTES
 
 
@@ -156,7 +158,7 @@ def origin_allowed(request: Request) -> bool:
         parsed = urlparse(referer)
         ref_origin = f"{parsed.scheme}://{parsed.netloc}".rstrip("/")
         return ref_origin == expected_origin or ref_origin in TRUSTED_ORIGINS
-    return True
+    return False
 
 
 def is_state_changing(method: str) -> bool:

templates/index.html

@@ -4,6 +4,9 @@
 <meta charset="UTF-8">
 <meta name="viewport" content="width=device-width, initial-scale=1.0">
 <title>JarvisChat</title>
+<meta http-equiv="Cache-Control" content="no-cache, no-store, must-revalidate">
+<meta http-equiv="Pragma" content="no-cache">
+<meta http-equiv="Expires" content="0">
 <link rel="preconnect" href="https://fonts.googleapis.com">
 <link href="https://fonts.googleapis.com/css2?family=JetBrains+Mono:wght@400;600&family=IBM+Plex+Sans:wght@300;400;500;600&display=swap" rel="stylesheet">
 <style>
@@ -47,13 +50,12 @@ body { font-family: var(--font-body); background: var(--bg-primary); color: var(
 .delete-all-btn { padding: 10px 12px; background: transparent; border: 1px solid var(--danger); border-radius: var(--radius); color: var(--danger); font-size: 14px; cursor: pointer; transition: all 0.2s; }
 .delete-all-btn:hover { background: var(--danger); color: #fff; }
 .conversation-list { flex: 1; overflow-y: auto; padding: 8px; }
-.conv-item { padding: 10px 12px; border-radius: var(--radius); cursor: pointer; margin-bottom: 2px; display: flex; justify-content: space-between; align-items: center; transition: background 0.15s; font-size: 13px; color: var(--text-secondary); }
+.conv-item { padding: 10px 12px; border-radius: var(--radius); cursor: pointer; margin-bottom: 2px; display: flex; align-items: center; gap: 8px; transition: background 0.15s; font-size: 13px; color: var(--text-secondary); }
 .conv-item:hover { background: var(--bg-hover); color: var(--text-primary); }
 .conv-item.active { background: var(--bg-tertiary); color: var(--text-primary); }
-.conv-item .conv-title { overflow: hidden; text-overflow: ellipsis; white-space: nowrap; flex: 1; }
-.conv-item .conv-delete { opacity: 0; color: var(--danger); cursor: pointer; padding: 2px 6px; font-size: 16px; }
-.conv-item:hover .conv-delete { opacity: 0.7; }
-.conv-item .conv-delete:hover { opacity: 1; }
+.conv-item .conv-trash { color: var(--text-muted); cursor: pointer; padding: 2px 2px; font-size: 15px; flex-shrink: 0; transition: color 0.15s; }
+.conv-item .conv-trash:hover { opacity: 1; color: var(--danger); }
+.conv-item .conv-title { overflow: hidden; text-overflow: ellipsis; white-space: nowrap; flex: 1; min-width: 0; }
 .sidebar-footer { padding: 12px 16px; border-top: 1px solid var(--border); font-size: 11px; color: var(--text-muted); font-family: var(--font-mono); }
 .sidebar-footer .status-row { display: flex; align-items: center; gap: 8px; margin-bottom: 4px; }
 .stats-panel { margin-top: 10px; padding-top: 10px; border-top: 1px solid var(--border); }
@@ -155,8 +157,22 @@ body { font-family: var(--font-body); background: var(--bg-primary); color: var(
 .perplexity-badge.low { background:rgba(46,204,113,0.15); border:1px solid rgba(46,204,113,0.3); color:var(--success); }
 .perplexity-badge.medium { background:rgba(243,156,18,0.15); border:1px solid rgba(243,156,18,0.3); color:var(--warning); }
 .perplexity-badge.high { background:rgba(231,76,60,0.15); border:1px solid rgba(231,76,60,0.3); color:var(--danger); }
+
+.ttr-badge { display:inline-block; padding:2px 8px; border-radius:10px; font-family:var(--font-mono); font-size:10px; margin-left:8px; }
+.ttr-badge.low { background:rgba(46,204,113,0.15); border:1px solid rgba(46,204,113,0.3); color:var(--success); }
+.ttr-badge.medium { background:rgba(243,156,18,0.15); border:1px solid rgba(243,156,18,0.3); color:var(--warning); }
+.ttr-badge.high { background:rgba(231,76,60,0.15); border:1px solid rgba(231,76,60,0.3); color:var(--danger); }
+.ttr-badge.search { background:rgba(230,126,34,0.15); border:1px solid rgba(230,126,34,0.3); color:#e67e22; }
 .tps-badge { display:inline-block; padding:2px 8px; border-radius:10px; font-family:var(--font-mono); font-size:10px; margin-left:8px; background:rgba(72,181,224,0.15); border:1px solid rgba(72,181,224,0.3); color:var(--accent); }
 
+.msg-toolbar { display:flex; gap:2px; margin-top:8px; opacity:0; transition:opacity 0.15s; align-items:center; }
+.message .content:hover .msg-toolbar { opacity:1; }
+.msg-toolbar button { background:none; border:none; color:var(--text-muted); cursor:pointer; padding:4px 6px; border-radius:4px; font-size:13px; line-height:1; display:flex; align-items:center; gap:4px; transition:color 0.15s,background 0.15s; }
+.msg-toolbar button:hover { color:var(--text-primary); background:var(--bg-tertiary); }
+.msg-toolbar button.active { color:var(--accent); }
+.msg-toolbar .sep { width:1px; height:14px; background:var(--border); margin:0 4px; }
+.msg-toolbar .toolbar-label { font-size:10px; font-family:var(--font-mono); color:var(--text-muted); margin-right:4px; }
+
 .input-area { padding:16px 20px; border-top:1px solid var(--border); background:var(--bg-secondary); }
 .input-row-top { max-width:900px; margin:0 auto 8px; display:flex; gap:8px; align-items:center; }
 .input-row-top select { background:var(--bg-tertiary); border:1px solid var(--border); color:var(--text-secondary); font-family:var(--font-mono); font-size:11px; padding:4px 8px; border-radius:var(--radius); cursor:pointer; }
@@ -172,6 +188,39 @@ body { font-family: var(--font-body); background: var(--bg-primary); color: var(
 .search-btn { padding:12px 14px; background:var(--warning); border:none; border-radius:var(--radius); color:#fff; font-size:16px; cursor:pointer; transition:background 0.2s; }
 .search-btn:hover { background:#e67e22; }
 .search-btn:disabled { background:var(--text-muted); cursor:not-allowed; }
+.paperclip-btn { padding:12px 12px; background:var(--bg-tertiary); border:1px solid var(--border); border-radius:var(--radius); color:var(--text-secondary); font-size:18px; cursor:pointer; transition:all 0.2s; line-height:1; }
+.paperclip-btn:hover { background:var(--bg-hover); color:var(--accent); border-color:var(--accent-dim); }
+.file-preview { max-width:900px; margin:0 auto 8px; display:none; align-items:center; gap:10px; background:var(--bg-tertiary); border:1px solid var(--border); border-radius:var(--radius); padding:8px 12px; }
+.file-preview.visible { display:flex; }
+.file-preview-thumb { width:48px; height:48px; object-fit:cover; border-radius:4px; border:1px solid var(--border); }
+.file-preview-icon { font-size:28px; flex-shrink:0; color:var(--text-muted); }
+.file-preview-info { flex:1; min-width:0; }
+.file-preview-name { font-size:13px; color:var(--text-primary); overflow:hidden; text-overflow:ellipsis; white-space:nowrap; }
+.file-preview-size { font-size:11px; color:var(--text-muted); }
+.file-preview-clear { background:none; border:none; color:var(--text-muted); font-size:18px; cursor:pointer; padding:4px; line-height:1; transition:color 0.15s; }
+.file-preview-clear:hover { color:var(--danger); }
+.file-preview-notice { font-size:11px; color:var(--warning); }
+.conv-attach { color:var(--text-muted); cursor:pointer; padding:2px 2px; font-size:14px; flex-shrink:0; transition:color 0.15s; margin-left:2px; }
+.conv-attach:hover { color:var(--accent); }
+.conv-attach.has-attachments { color:var(--accent-dim); }
+.gallery-overlay { position:fixed; top:0; left:0; width:100%; height:100%; background:rgba(0,0,0,0.85); z-index:10000; display:none; justify-content:center; align-items:center; }
+.gallery-overlay.visible { display:flex; }
+.gallery-panel { background:var(--bg-secondary); border:1px solid var(--border); border-radius:var(--radius); width:90%; max-width:700px; max-height:85vh; display:flex; flex-direction:column; }
+.gallery-header { display:flex; align-items:center; justify-content:space-between; padding:16px 20px; border-bottom:1px solid var(--border); }
+.gallery-header h3 { font-size:16px; color:var(--text-primary); font-family:var(--font-mono); }
+.gallery-close { background:none; border:none; color:var(--text-muted); font-size:24px; cursor:pointer; padding:4px; line-height:1; transition:color 0.15s; }
+.gallery-close:hover { color:var(--text-primary); }
+.gallery-body { flex:1; overflow-y:auto; padding:12px 20px; }
+.gallery-empty { text-align:center; padding:40px 20px; color:var(--text-muted); font-size:14px; }
+.gallery-item { display:flex; align-items:center; gap:12px; padding:12px; border-radius:var(--radius); margin-bottom:8px; background:var(--bg-tertiary); transition:background 0.15s; }
+.gallery-item:hover { background:var(--bg-hover); }
+.gallery-item-thumb { width:60px; height:60px; object-fit:cover; border-radius:4px; border:1px solid var(--border); flex-shrink:0; }
+.gallery-item-icon { font-size:32px; flex-shrink:0; color:var(--text-muted); text-align:center; width:60px; }
+.gallery-item-info { flex:1; min-width:0; }
+.gallery-item-name { font-size:14px; color:var(--text-primary); overflow:hidden; text-overflow:ellipsis; white-space:nowrap; }
+.gallery-item-meta { font-size:11px; color:var(--text-muted); margin-top:2px; }
+.gallery-item-delete { background:none; border:1px solid var(--danger); border-radius:var(--radius); color:var(--danger); font-size:12px; cursor:pointer; padding:6px 12px; transition:all 0.15s; flex-shrink:0; }
+.gallery-item-delete:hover { background:var(--danger); color:#fff; }
 
 .token-thermometer { display:flex; flex-direction:column; align-items:center; gap:4px; }
 .thermometer-bar { width:12px; height:80px; background:var(--bg-tertiary); border:1px solid var(--border); border-radius:6px; position:relative; overflow:hidden; }
@@ -319,10 +368,7 @@ body { font-family: var(--font-body); background: var(--bg-primary); color: var(
 
 <main class="main">
     <div class="topbar">
-        <div class="topbar-left">
-            <span class="topbar-label">Model</span>
-            <select id="modelSelect"></select>
-        </div>
+        <div class="topbar-left"></div>
         <div class="topbar-right">
             <button class="memory-badge on" id="memoryBadge" onclick="toggleMemory()" title="Toggle memory injection">🧠 MEM ON</button>
             <button class="search-badge on" id="searchBadge" onclick="toggleSearch()" title="Toggle auto web search">🔍 SEARCH ON</button>
@@ -330,6 +376,15 @@ body { font-family: var(--font-body); background: var(--bg-primary); color: var(
             <button class="logout-btn" id="authActionBtn" onclick="handleAuthAction()" title="Unlock admin">ADMIN</button>
         </div>
     </div>
+    <div class="gallery-overlay" id="galleryOverlay" onclick="closeGallery(event)">
+        <div class="gallery-panel" onclick="event.stopPropagation()">
+            <div class="gallery-header">
+                <h3>📎 Attachments</h3>
+                <button class="gallery-close" onclick="closeGallery()">✕</button>
+            </div>
+            <div class="gallery-body" id="galleryBody"></div>
+        </div>
+    </div>
     <div class="chat-container" id="chatContainer">
         <div class="welcome-screen" id="welcomeScreen">
             <div class="logo">⚡</div>
@@ -341,7 +396,19 @@ body { font-family: var(--font-body); background: var(--bg-primary); color: var(
             <span class="preset-label">PRESET</span>
             <select id="presetSelect"><option value="">None (profile only)</option></select>
         </div>
+        <div class="file-preview" id="filePreview">
+            <img class="file-preview-thumb" id="filePreviewThumb" src="" alt="" style="display:none">
+            <span class="file-preview-icon" id="filePreviewIcon">📄</span>
+            <div class="file-preview-info">
+                <div class="file-preview-name" id="filePreviewName"></div>
+                <div class="file-preview-size" id="filePreviewSize"></div>
+                <div class="file-preview-notice" id="filePreviewNotice"></div>
+            </div>
+            <button class="file-preview-clear" onclick="clearFileSelection()" title="Remove file">✕</button>
+        </div>
         <div class="input-wrapper">
+            <input type="file" id="fileInput" style="display:none" accept=".txt,.md,.pdf,.json,.py,.html,.png,.jpg,.jpeg,.gif,.svg,.webp" onchange="onFileSelected(event)">
+            <button class="paperclip-btn" id="paperclipBtn" onclick="document.getElementById('fileInput').click()" title="Attach file">📎</button>
             <textarea id="userInput" placeholder="Type a message... (Shift+Enter for new line)" rows="1" autofocus></textarea>
             <div class="token-thermometer" title="Context usage">
                 <div class="thermometer-bar"><div class="thermometer-fill" id="thermometerFill" style="height:0%"></div></div>
@@ -359,12 +426,14 @@ body { font-family: var(--font-body); background: var(--bg-primary); color: var(
 let currentConvId = null;
 let isStreaming = false;
 let abortController = null;
+let selectedFile = null;
 let profileEnabled = true;
 let searchEnabled = true;
 let memoryEnabled = true;
 let skillsEnabled = true;
 let presets = [];
 let skillsRegistry = [];
+let currentModel = '';
 let modelContextSize = 8192;
 let cachedProfile = '';
 let conversationHistory = [];
@@ -665,21 +734,20 @@ async function loadModels() {
     try {
         const resp = await authFetch('/api/models');
         const data = await resp.json();
-        const select = document.getElementById('modelSelect');
         const settingSelect = document.getElementById('defaultModelSetting');
-        select.innerHTML = '';
         settingSelect.innerHTML = '';
         (data.models || []).forEach(m => {
-            const gb = (m.size / (1024*1024*1024)).toFixed(1);
-            select.add(new Option(m.name + ' (' + gb + 'GB)', m.name));
             settingSelect.add(new Option(m.name, m.name));
         });
-        select.addEventListener('change', fetchModelContextSize);
+        if (!currentModel && data.models?.length) {
+            currentModel = data.models[0].name;
+            await fetchModelContextSize();
+        }
     } catch(e) {}
 }
 
 async function fetchModelContextSize() {
-    const model = document.getElementById('modelSelect').value;
+    const model = currentModel;
     if (!model) return;
     try {
         const resp = await authFetch('/api/show', { method: 'POST', headers: {'Content-Type': 'application/json'}, body: JSON.stringify({ name: model }) });
@@ -703,7 +771,7 @@ async function loadSettings() {
         updateMemoryUI();
         updateSkillsUI();
         if (s.default_model) {
-            document.getElementById('modelSelect').value = s.default_model;
+            currentModel = s.default_model;
             document.getElementById('defaultModelSetting').value = s.default_model;
         }
     } catch(e) {}
@@ -925,7 +993,12 @@ function renderPresetSelect() {
     const current = select.value;
     select.innerHTML = '<option value="">None (profile only)</option>';
     presets.forEach(p => select.add(new Option(p.name, p.id)));
-    select.value = current;
+    if (current) {
+        select.value = current;
+    } else {
+        const ga = presets.find(p => p.name === 'General Assistant');
+        if (ga) select.value = ga.id;
+    }
 }
 
 async function addPreset() {
@@ -983,8 +1056,8 @@ async function loadConversations() {
         convs.forEach(c => {
             const div = document.createElement('div');
             div.className = 'conv-item' + (c.id === currentConvId ? ' active' : '');
-            const delBtn = currentRole === 'admin' ? `<span class="conv-delete" onclick="event.stopPropagation();deleteConversation('${c.id}')">×</span>` : '';
-            div.innerHTML = `<span class="conv-title" onclick="loadConversation('${c.id}')">${c.title}</span>${delBtn}`;
+            const attachIcon = c.attachment_count > 0 ? `<span class="conv-attach has-attachments" onclick="event.stopPropagation();showGallery('${c.id}')" title="${c.attachment_count} attachment(s)">📎</span>` : '';
+            div.innerHTML = `<span class="conv-trash" onclick="event.stopPropagation();deleteConversation('${c.id}')" title="Delete conversation">🗑</span>${attachIcon}<span class="conv-title" onclick="loadConversation('${c.id}')">${c.title}</span>`;
             list.appendChild(div);
         });
     } catch(e) {}
@@ -995,7 +1068,7 @@ async function loadConversation(convId) {
         const resp = await authFetch(`/api/conversations/${convId}`);
         const data = await resp.json();
         currentConvId = convId;
-        document.getElementById('modelSelect').value = data.conversation.model;
+        currentModel = data.conversation.model;
         fetchModelContextSize();
         const container = document.getElementById('chatContainer');
         container.innerHTML = '';
@@ -1034,6 +1107,7 @@ function newChat() {
     updateTokenThermometer();
 }
 
+function escapeHtml(str) { const div = document.createElement('div'); div.textContent = str; return div.innerHTML; }
 function showWelcome() {
     document.getElementById('chatContainer').innerHTML = '<div class="welcome-screen" id="welcomeScreen"><div class="logo">⚡</div><p>JarvisChat — your local coding companion.<br>Profile + Memory context injected automatically.<br>Web search kicks in when the model is uncertain.<br>Use 🔍 to force a web search.<br>Say "remember that..." to teach me things.</p></div>';
 }
@@ -1042,7 +1116,7 @@ async function sendSearch() {
     const input = document.getElementById('userInput');
     const query = input.value.trim();
     if (!query || isStreaming) return;
-    const model = document.getElementById('modelSelect').value;
+    const model = currentModel;
     const welcome = document.getElementById('welcomeScreen');
     if (welcome) welcome.remove();
     appendMessage('user', '🔍 ' + query, true);
@@ -1054,6 +1128,8 @@ async function sendSearch() {
     const textEl = assistantDiv.querySelector('.text');
     textEl.innerHTML = '<div class="search-indicator"><div class="spinner"></div>Searching the web...</div>';
     setStreamingState(true);
+    const ttrStart = performance.now();
+    let ttr = 0;
     try {
         abortController = new AbortController();
         const resp = await authFetch('/api/search', { method: 'POST', headers: {'Content-Type': 'application/json'}, body: JSON.stringify({ conversation_id: currentConvId, query, model }), signal: abortController.signal });
@@ -1075,7 +1151,7 @@ async function sendSearch() {
                     if (data.error) { textEl.textContent = 'Error: ' + data.error; setStreamingState(false); return; }
                     if (data.conversation_id && !currentConvId) { currentConvId = data.conversation_id; await loadConversations(); }
                     if (data.search_results) { textEl.innerHTML = '<div class="search-indicator">🔍 Found ' + data.search_results + ' results, summarizing...</div>'; }
-                    if (data.token) { if (firstToken) { textEl.innerHTML = ''; firstToken = false; } fullText += data.token; textEl.innerHTML = renderMarkdown(fullText); scrollToBottom(); }
+                    if (data.token) { if (firstToken) { textEl.innerHTML = ''; firstToken = false; ttr = performance.now() - ttrStart; } fullText += data.token; textEl.innerHTML = renderMarkdown(fullText); scrollToBottom(); }
                     if (data.raw_results) {
                         let rawHtml = '<details class="raw-results"><summary>🔍 View raw search results (' + data.raw_results.length + ')</summary><ul>';
                         data.raw_results.forEach(r => { 
@@ -1094,10 +1170,17 @@ async function sendSearch() {
                     }
                     if (data.done) {
                         const roleLabel = assistantDiv.querySelector('.role-label');
-                        if (roleLabel) roleLabel.innerHTML += '<span class="search-badge-inline">🔍 web</span>';
+                        if (roleLabel) {
+                            roleLabel.innerHTML += '<span class="search-badge-inline">🔍 web</span>';
+                            if (ttr > 0) {
+                                const ttrSec = ttr / 1000;
+                                roleLabel.innerHTML += `<span class="ttr-badge search">ttr: ${ttrSec.toFixed(1)}s</span>`;
+                            }
+                        }
                         conversationHistory.push({ role: 'assistant', content: fullText });
                         updateTokenThermometer();
                         addCopyButtons(assistantDiv);
+                        addMessageToolbar(assistantDiv);
                         setStreamingState(false);
                         await loadConversations();
                     }
@@ -1111,11 +1194,89 @@ async function sendSearch() {
     }
 }
 
+function onFileSelected(event) {
+    const file = event.target.files[0];
+    if (!file) return;
+    selectedFile = file;
+    const preview = document.getElementById('filePreview');
+    const thumb = document.getElementById('filePreviewThumb');
+    const icon = document.getElementById('filePreviewIcon');
+    const nameEl = document.getElementById('filePreviewName');
+    const sizeEl = document.getElementById('filePreviewSize');
+    const notice = document.getElementById('filePreviewNotice');
+    nameEl.textContent = file.name;
+    sizeEl.textContent = (file.size / 1024).toFixed(1) + ' KB';
+    if (file.type.startsWith('image/')) {
+        thumb.style.display = 'block';
+        thumb.src = URL.createObjectURL(file);
+        icon.style.display = 'none';
+        notice.textContent = 'Model does not support image input — filename only will be used as context.';
+    } else {
+        thumb.style.display = 'none';
+        icon.style.display = 'inline';
+        icon.textContent = '📄';
+        notice.textContent = '';
+    }
+    preview.classList.add('visible');
+    event.target.value = '';
+}
+function clearFileSelection() {
+    selectedFile = null;
+    const preview = document.getElementById('filePreview');
+    preview.classList.remove('visible');
+    const thumb = document.getElementById('filePreviewThumb');
+    thumb.src = '';
+}
+function showGallery(convId) {
+    document.getElementById('galleryOverlay').classList.add('visible');
+    loadGallery(convId);
+}
+function closeGallery(event) {
+    if (event && event.target !== event.currentTarget) return;
+    document.getElementById('galleryOverlay').classList.remove('visible');
+}
+async function loadGallery(convId) {
+    const body = document.getElementById('galleryBody');
+    body.innerHTML = '<div class="gallery-empty">Loading...</div>';
+    try {
+        const resp = await authFetch('/api/upload/by-conversation/' + encodeURIComponent(convId));
+        const items = await resp.json();
+        if (!items || items.length === 0) {
+            body.innerHTML = '<div class="gallery-empty">No attachments in this conversation.</div>';
+            return;
+        }
+        body.innerHTML = '';
+        for (const item of items) {
+            const div = document.createElement('div');
+            div.className = 'gallery-item';
+            const isImage = item.content_type && item.content_type.startsWith('image/');
+            const thumbHtml = isImage
+                ? '<span class="gallery-item-icon">🖼</span>'
+                : '<span class="gallery-item-icon">📄</span>';
+            const expires = new Date(item.expires_at);
+            const expiresStr = expires.toLocaleString();
+            div.innerHTML = thumbHtml + '<div class="gallery-item-info"><div class="gallery-item-name">' + escapeHtml(item.filename) + '</div><div class="gallery-item-meta">Expires: ' + expiresStr + '</div></div><button class="gallery-item-delete" onclick="deleteGalleryItem(' + item.id + ')">Delete</button>';
+            body.appendChild(div);
+        }
+    } catch(e) {
+        body.innerHTML = '<div class="gallery-empty">Error loading attachments.</div>';
+    }
+}
+async function deleteGalleryItem(contextId) {
+    if (!confirm('Delete this attachment? It will also be removed from the RAG corpus.')) return;
+    try {
+        const resp = await authFetch('/api/upload/' + contextId, { method: 'DELETE' });
+        if (resp.ok) {
+            await loadGallery(currentConvId);
+            await loadConversations();
+        }
+    } catch(e) {}
+}
 async function sendMessage() {
     const input = document.getElementById('userInput');
     const message = input.value.trim();
     if (!message || isStreaming) return;
-    const model = document.getElementById('modelSelect').value;
+    const model = currentModel;
     const presetPrompt = getSelectedPresetPrompt();
     const welcome = document.getElementById('welcomeScreen');
     if (welcome) welcome.remove();
@@ -1129,9 +1290,36 @@ async function sendMessage() {
     textEl.innerHTML = '<div class="typing-indicator"><span></span><span></span><span></span></div>';
     setStreamingState(true);
     let searchTriggered = false;
+    const ttrStart = performance.now();
+    let ttr = 0;
     try {
         abortController = new AbortController();
-        const resp = await authFetch('/api/chat', { method: 'POST', headers: {'Content-Type': 'application/json'}, body: JSON.stringify({ conversation_id: currentConvId, message, model, system_prompt: presetPrompt }), signal: abortController.signal });
+        let uploadContextId = null;
+        if (selectedFile) {
+            const formData = new FormData();
+            formData.append('file', selectedFile, selectedFile.name);
+            formData.append('mode', 'both');
+            if (currentConvId) formData.append('conversation_id', currentConvId);
+            try {
+                const uploadResp = await authFetch('/api/upload', { method: 'POST', body: formData, signal: abortController.signal });
+                const uploadData = await uploadResp.json();
+                if (uploadResp.ok && uploadData.context_id) {
+                    uploadContextId = uploadData.context_id;
+                    if (!currentConvId && uploadData.conversation_id) currentConvId = uploadData.conversation_id;
+                } else {
+                    throw new Error(uploadData.detail || 'Upload failed');
+                }
+            } catch(e) {
+                if (e.name !== 'AbortError') textEl.textContent = 'Error uploading file: ' + e.message;
+                setStreamingState(false);
+                return;
+            }
+            clearFileSelection();
+            await loadConversations();
+        }
+        const chatBody = { conversation_id: currentConvId, message, model, system_prompt: presetPrompt };
+        if (uploadContextId) chatBody.upload_context_id = uploadContextId;
+        const resp = await authFetch('/api/chat', { method: 'POST', headers: {'Content-Type': 'application/json'}, body: JSON.stringify(chatBody), signal: abortController.signal });
         const reader = resp.body.getReader();
         const decoder = new TextDecoder();
         let fullText = '';
@@ -1148,18 +1336,30 @@ async function sendMessage() {
                 try {
                     const data = JSON.parse(line.slice(6));
                     if (data.error) { textEl.textContent = 'Error: ' + data.error; setStreamingState(false); return; }
-                    if (data.conversation_id && !currentConvId) { currentConvId = data.conversation_id; await loadConversations(); }
+                    if (data.conversation_id && !currentConvId) {
+                        currentConvId = data.conversation_id;
+                        if (uploadContextId) {
+                            authFetch('/api/upload/' + uploadContextId + '/link', { method: 'PATCH', headers: {'Content-Type': 'application/json'}, body: JSON.stringify({ conversation_id: currentConvId }) }).catch(()=>{});
+                        }
+                        await loadConversations();
+                    }
                     if (data.searching) { textEl.innerHTML = fullText ? renderMarkdown(fullText) + '<div class="search-indicator"><div class="spinner"></div>Searching...</div>' : '<div class="search-indicator"><div class="spinner"></div>Searching...</div>'; searchTriggered = true; }
                     if (data.search_results) { textEl.innerHTML = '<div class="search-indicator">🔍 Found ' + data.search_results + ' results...</div>'; fullText = ''; firstToken = true; }
-                    if (data.token) { if (firstToken) { textEl.innerHTML = ''; firstToken = false; } fullText += data.token; textEl.innerHTML = renderMarkdown(fullText); scrollToBottom(); }
+                    if (data.token) { if (firstToken) { textEl.innerHTML = ''; firstToken = false; ttr = performance.now() - ttrStart; } fullText += data.token; textEl.innerHTML = renderMarkdown(fullText); scrollToBottom(); }
                     if (data.done) {
                         const roleLabel = assistantDiv.querySelector('.role-label');
                         if (data.searched && roleLabel) roleLabel.innerHTML += '<span class="search-badge-inline">🔍 web</span>';
                         if (typeof data.perplexity === 'number' && roleLabel) { const ppl = data.perplexity; roleLabel.innerHTML += `<span class="perplexity-badge ${ppl >= 15 ? 'high' : ppl >= 8 ? 'medium' : 'low'}">ppl: ${ppl.toFixed(1)}</span>`; }
                         if (typeof data.tokens_per_sec === 'number' && data.tokens_per_sec > 0 && roleLabel) roleLabel.innerHTML += `<span class="tps-badge">${data.tokens_per_sec.toFixed(1)} t/s</span>`;
+                        if (roleLabel && ttr > 0) {
+                            const ttrSec = ttr / 1000;
+                            const ttrClass = data.searched ? 'search' : (ttrSec <= 2 ? 'low' : ttrSec < 11 ? 'medium' : 'high');
+                            roleLabel.innerHTML += `<span class="ttr-badge ${ttrClass}">ttr: ${ttrSec.toFixed(1)}s</span>`;
+                        }
                         conversationHistory.push({ role: 'assistant', content: fullText });
                         updateTokenThermometer();
                         addCopyButtons(assistantDiv);
+                        addMessageToolbar(assistantDiv);
                         setStreamingState(false);
                         await loadConversations();
                         await loadMemoryStats();
@@ -1197,9 +1397,9 @@ function appendMessage(role, content, animate, isSearch = false) {
     const div = document.createElement('div');
     div.className = 'message ' + role + (isSearch && role === 'assistant' ? ' search-result' : '');
     if (!animate) div.style.animation = 'none';
-    div.innerHTML = `<div class="avatar">${role === 'user' ? 'YOU' : 'AI'}</div><div class="content"><div class="role-label">${role}</div><div class="text">${content ? renderMarkdown(content) : ''}</div></div>`;
+    div.innerHTML = `<div class="avatar">${role === 'user' ? 'YOU' : 'AI'}</div><div class="content"><div class="role-label">${role}</div><div class="text">${content ? renderMarkdown(content) : ''}</div>${role === 'assistant' ? '<div class="msg-toolbar"></div>' : ''}</div>`;
     container.appendChild(div);
-    if (content && role === 'assistant') addCopyButtons(div);
+    if (content && role === 'assistant') { addCopyButtons(div); addMessageToolbar(div); }
     scrollToBottom();
     return div;
 }
@@ -1234,18 +1434,111 @@ function addCopyButtons(msgDiv) {
         const btn = document.createElement('button');
         btn.className = 'copy-btn';
         btn.textContent = 'copy';
-        btn.onclick = () => navigator.clipboard.writeText(pre.querySelector('code')?.textContent || pre.textContent).then(() => { btn.textContent = 'copied!'; setTimeout(() => btn.textContent = 'copy', 1500); });
+        btn.onclick = () => {
+            const text = pre.querySelector('code')?.textContent || pre.textContent;
+            if (navigator.clipboard) {
+                navigator.clipboard.writeText(text).then(() => { btn.textContent = 'copied!'; setTimeout(() => btn.textContent = 'copy', 1500); }).catch(() => fallbackCopy(text, btn));
+            } else { fallbackCopy(text, btn); }
+        };
         pre.style.position = 'relative';
         pre.appendChild(btn);
     });
 }
 
+function addMessageToolbar(msgDiv) {
+    const toolbar = msgDiv.querySelector('.msg-toolbar');
+    if (!toolbar || toolbar.hasChildNodes()) return;
+    const textDiv = msgDiv.querySelector('.text');
+    const getText = () => textDiv ? textDiv.textContent || textDiv.innerText : '';
+
+    const copyBtn = document.createElement('button');
+    copyBtn.innerHTML = '📋';
+    copyBtn.title = 'Copy response';
+    copyBtn.onclick = () => {
+        const t = getText();
+        if (navigator.clipboard) {
+            navigator.clipboard.writeText(t).then(() => { copyBtn.textContent = '✓'; setTimeout(() => copyBtn.innerHTML = '📋', 1500); });
+        }
+    };
+
+    const printBtn = document.createElement('button');
+    printBtn.innerHTML = '🖨️';
+    printBtn.title = 'Print response';
+    printBtn.onclick = () => {
+        const w = window.open('', '_blank', 'width=800,height=600');
+        w.document.write(`<!DOCTYPE html><html><head><title>Print</title><style>body{font-family:system-ui;max-width:700px;margin:40px auto;padding:0 20px;line-height:1.6;font-size:14px}pre{background:#f4f4f4;padding:12px;border-radius:6px;overflow-x:auto}code{background:#f4f4f4;padding:2px 5px;border-radius:3px}pre code{background:none;padding:0}img{max-width:100%}@media print{body{margin:0}}</style></head><body>${textDiv ? textDiv.innerHTML : ''}</body></html>`);
+        w.document.close();
+        setTimeout(() => { w.focus(); w.print(); }, 500);
+    };
+
+    const saveBtn = document.createElement('button');
+    saveBtn.innerHTML = '💾';
+    saveBtn.title = 'Save as .md';
+    saveBtn.onclick = () => {
+        const t = getText();
+        const blob = new Blob([t], { type: 'text/markdown' });
+        const a = document.createElement('a');
+        a.href = URL.createObjectURL(blob);
+        a.download = `jarvischat-response-${Date.now()}.md`;
+        a.click();
+        URL.revokeObjectURL(a.href);
+    };
+
+    const rating = { value: 0 };
+    const upBtn = document.createElement('button');
+    upBtn.innerHTML = '👍';
+    upBtn.title = 'Rate up';
+    upBtn.onclick = () => {
+        rating.value = rating.value === 1 ? 0 : 1;
+        upBtn.classList.toggle('active', rating.value === 1);
+        dnBtn.classList.toggle('active', false);
+    };
+
+    const dnBtn = document.createElement('button');
+    dnBtn.innerHTML = '👎';
+    dnBtn.title = 'Rate down';
+    dnBtn.onclick = () => {
+        rating.value = rating.value === -1 ? 0 : -1;
+        dnBtn.classList.toggle('active', rating.value === -1);
+        upBtn.classList.toggle('active', false);
+    };
+
+    toolbar.append(copyBtn, printBtn, saveBtn, sep(), upBtn, dnBtn);
+    function sep() { const s = document.createElement('span'); s.className = 'sep'; return s; }
+}
+
 function escapeHtml(t) { const d = document.createElement('div'); d.textContent = t; return d.innerHTML; }
 function scrollToBottom() { const c = document.getElementById('chatContainer'); c.scrollTop = c.scrollHeight; }
+function fallbackCopy(text, btn) {
+    const ta = document.createElement('textarea');
+    ta.value = text;
+    ta.style.position = 'fixed';
+    ta.style.opacity = '0';
+    document.body.appendChild(ta);
+    ta.select();
+    document.execCommand('copy');
+    document.body.removeChild(ta);
+    btn.textContent = 'copied!';
+    setTimeout(() => btn.textContent = 'copy', 1500);
+}
 
 const userInput = document.getElementById('userInput');
 userInput.addEventListener('input', function() { this.style.height = 'auto'; this.style.height = Math.min(this.scrollHeight, 200) + 'px'; });
 userInput.addEventListener('keydown', e => { if (e.key === 'Enter' && !e.shiftKey) { e.preventDefault(); sendMessage(); } });
+const toastStyle = document.createElement('style');
+toastStyle.textContent = '.toast-error { position:fixed; bottom:80px; left:50%; transform:translateX(-50%); background:var(--danger); color:#fff; padding:12px 24px; border-radius:var(--radius); font-family:var(--font-body); font-size:14px; z-index:9999; animation:fadeIn 0.2s; } @keyframes fadeIn { from{opacity:0;transform:translateX(-50%) translateY(10px)} to{opacity:1;transform:translateX(-50%) translateY(0)} }';
+document.head.appendChild(toastStyle);
+userInput.addEventListener('paste', e => {
+    const hasImage = Array.from(e.clipboardData.items).some(i => i.type.startsWith('image/'));
+    if (hasImage) {
+        e.preventDefault();
+        const toast = document.createElement('div');
+        toast.className = 'toast-error';
+        toast.textContent = 'Use the 📎 button to attach images — paste is text only.';
+        document.body.appendChild(toast);
+        setTimeout(() => toast.remove(), 3000);
+    }
+});
 </script>
 </body>
 </html>

tests/test_auth_capabilities.py

@@ -3,16 +3,18 @@ from pathlib import Path
 
 from fastapi.testclient import TestClient
 
-import app as app_module
+import app
+import db
+from security import SESSIONS, PIN_ATTEMPTS
 
 
 def make_client(tmp_path: Path) -> TestClient:
     os.environ["JARVISCHAT_ADMIN_PIN"] = "1234"
-    app_module.DB_PATH = tmp_path / "jarvischat-test.db"
-    app_module.SESSIONS.clear()
-    app_module.PIN_ATTEMPTS.clear()
-    app_module.init_db()
-    return TestClient(app_module.app)
+    db.DB_PATH = tmp_path / "jarvischat-test.db"
+    SESSIONS.clear()
+    PIN_ATTEMPTS.clear()
+    db.init_db()
+    return TestClient(app.app)
 
 
 def test_guest_read_only_admin_write_blocked(tmp_path: Path):
@@ -20,7 +22,7 @@ def test_guest_read_only_admin_write_blocked(tmp_path: Path):
         guest = client.post("/api/auth/guest", headers={"Origin": "http://testserver"})
         assert guest.status_code == 200
         sid = guest.json()["session_id"]
-        headers = {"X-Session-ID": sid}
+        headers = {"X-Session-ID": sid, "Origin": "http://testserver"}
 
         read_resp = client.get("/api/memories", headers=headers)
         assert read_resp.status_code == 200
@@ -74,5 +76,5 @@ def test_logout_revokes_session(tmp_path: Path):
         logout = client.post("/api/auth/logout", headers=headers)
         assert logout.status_code == 200
 
-        after = client.get("/api/memories", headers={"X-Session-ID": sid})
+        after = client.get("/api/memories", headers={"X-Session-ID": sid, "Origin": "http://testserver"})
         assert after.status_code == 401

tests/test_chat_streaming_and_memory_paths.py

@@ -2,19 +2,24 @@ import json
 import os
 from pathlib import Path
 
+import httpx
 from fastapi.testclient import TestClient
 
-import app as app_module
+import app
+import config
+import db
+import routers.chat
+from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS
 
 
 def make_client(tmp_path: Path) -> TestClient:
     os.environ["JARVISCHAT_ADMIN_PIN"] = "1234"
-    app_module.DB_PATH = tmp_path / "jarvischat-streaming.db"
-    app_module.SESSIONS.clear()
-    app_module.PIN_ATTEMPTS.clear()
-    app_module.RATE_EVENTS.clear()
-    app_module.init_db()
-    return TestClient(app_module.app, raise_server_exceptions=False)
+    db.DB_PATH = tmp_path / "jarvischat-streaming.db"
+    SESSIONS.clear()
+    PIN_ATTEMPTS.clear()
+    RATE_EVENTS.clear()
+    db.init_db()
+    return TestClient(app.app, raise_server_exceptions=False)
 
 
 def parse_sse_payloads(body: str) -> list[dict]:
@@ -65,11 +70,11 @@ def test_chat_stream_emits_tokens_and_done(tmp_path: Path, monkeypatch):
         def stream_stub(self, method, url, json=None, timeout=None):
             return _MockStreamResponse(events)
 
-        monkeypatch.setattr(app_module.httpx.AsyncClient, "stream", stream_stub)
+        monkeypatch.setattr(httpx.AsyncClient, "stream", stream_stub)
 
         resp = client.post(
             "/api/chat",
-            json={"message": "hello", "model": app_module.DEFAULT_MODEL},
+            json={"message": "hello", "model": config.DEFAULT_MODEL},
             headers=headers,
         )
         assert resp.status_code == 200
@@ -92,7 +97,7 @@ def test_chat_auto_search_trigger_emits_search_events(tmp_path: Path, monkeypatc
         first_stream = _stream_json_lines(
             [
                 {
-                    "message": {"content": "I am uncertain."},
+                    "message": {"content": "I don't have current data on that question."},
                     "logprobs": [{"logprob": -5.0}],
                 },
                 {"done": True, "eval_count": 2, "eval_duration": 1000000000},
@@ -118,12 +123,12 @@ def test_chat_auto_search_trigger_emits_search_events(tmp_path: Path, monkeypatc
                 }
             ]
 
-        monkeypatch.setattr(app_module.httpx.AsyncClient, "stream", stream_stub)
-        monkeypatch.setattr(app_module, "query_searxng", search_stub)
+        monkeypatch.setattr(httpx.AsyncClient, "stream", stream_stub)
+        monkeypatch.setattr(routers.chat, "query_searxng", search_stub)
 
         resp = client.post(
             "/api/chat",
-            json={"message": "what is the latest value", "model": app_module.DEFAULT_MODEL},
+            json={"message": "what is the latest value", "model": config.DEFAULT_MODEL},
             headers=headers,
         )
         assert resp.status_code == 200
@@ -136,6 +141,69 @@ def test_chat_auto_search_trigger_emits_search_events(tmp_path: Path, monkeypatc
         assert done_events and done_events[-1].get("searched") is True
 
 
+def test_chat_with_upload_context_id_injects_document(tmp_path: Path, monkeypatch):
+    captured_payload = {}
+
+    def stream_stub(self, method, url, json=None, timeout=None):
+        nonlocal captured_payload
+        captured_payload = json
+        events = [{"message": {"content": "ok"}, "logprobs": [{"logprob": -0.01}]}, {"done": True, "eval_count": 1, "eval_duration": 1000000000}]
+        return _MockStreamResponse([__import__('json').dumps(e) for e in events])
+
+    monkeypatch.setattr(httpx.AsyncClient, "stream", stream_stub)
+
+    with make_client(tmp_path) as client:
+        sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()["session_id"]
+        headers = {"X-Session-ID": sid, "Origin": "http://testserver"}
+
+        db_local = db.get_db()
+        expires = "2099-12-31T23:59:59+00:00"
+        cid = db.insert_upload_context(db_local, "conv-up", "report.txt", "Confidential document content here", expires, "text/plain")
+        db_local.commit()
+        db_local.close()
+
+        resp = client.post(
+            "/api/chat",
+            json={"message": "summarize this", "upload_context_id": cid, "model": config.DEFAULT_MODEL},
+            headers=headers,
+        )
+        assert resp.status_code == 200
+        system_content = next((m["content"] for m in captured_payload.get("messages", []) if m["role"] == "system"), "")
+        assert "Confidential document content here" in system_content
+
+
+def test_chat_with_expired_upload_context_id_silent(tmp_path: Path, monkeypatch):
+    captured_payload = {}
+
+    def stream_stub(self, method, url, json=None, timeout=None):
+        nonlocal captured_payload
+        captured_payload = json
+        events = [{"message": {"content": "ok"}, "logprobs": [{"logprob": -0.01}]}, {"done": True, "eval_count": 1, "eval_duration": 1000000000}]
+        return _MockStreamResponse([__import__('json').dumps(e) for e in events])
+
+    monkeypatch.setattr(httpx.AsyncClient, "stream", stream_stub)
+
+    with make_client(tmp_path) as client:
+        sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()["session_id"]
+        headers = {"X-Session-ID": sid, "Origin": "http://testserver"}
+
+        import datetime
+        db_local = db.get_db()
+        expires = (datetime.datetime.now(datetime.timezone.utc) - datetime.timedelta(hours=2)).isoformat()
+        cid = db.insert_upload_context(db_local, "conv-exp", "old.txt", "Stale data", expires, "text/plain")
+        db_local.commit()
+        db_local.close()
+
+        resp = client.post(
+            "/api/chat",
+            json={"message": "hi", "upload_context_id": cid, "model": config.DEFAULT_MODEL},
+            headers=headers,
+        )
+        assert resp.status_code == 200
+        system_content = next((m["content"] for m in captured_payload.get("messages", []) if m["role"] == "system"), "")
+        assert "Stale data" not in system_content
+
+
 def test_memory_command_paths_remember_and_forget(tmp_path: Path, monkeypatch):
     with make_client(tmp_path) as client:
         sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()[
@@ -153,13 +221,13 @@ def test_memory_command_paths_remember_and_forget(tmp_path: Path, monkeypatch):
         def stream_stub(self, method, url, json=None, timeout=None):
             return _MockStreamResponse(base_stream)
 
-        monkeypatch.setattr(app_module.httpx.AsyncClient, "stream", stream_stub)
+        monkeypatch.setattr(httpx.AsyncClient, "stream", stream_stub)
 
         remember_resp = client.post(
             "/api/chat",
             json={
                 "message": "remember that my favorite language is rust",
-                "model": app_module.DEFAULT_MODEL,
+                "model": config.DEFAULT_MODEL,
             },
             headers=headers,
         )
@@ -167,7 +235,7 @@ def test_memory_command_paths_remember_and_forget(tmp_path: Path, monkeypatch):
         remember_events = parse_sse_payloads(remember_resp.text)
         assert any("Remembered" in p.get("token", "") for p in remember_events)
 
-        memories_after_add = client.get("/api/memories", headers={"X-Session-ID": sid})
+        memories_after_add = client.get("/api/memories", headers={"X-Session-ID": sid, "Origin": "http://testserver"})
         assert memories_after_add.status_code == 200
         assert memories_after_add.json().get("count", 0) >= 1
 
@@ -175,7 +243,7 @@ def test_memory_command_paths_remember_and_forget(tmp_path: Path, monkeypatch):
             "/api/chat",
             json={
                 "message": "forget about my favorite language",
-                "model": app_module.DEFAULT_MODEL,
+                "model": config.DEFAULT_MODEL,
             },
             headers=headers,
         )
@@ -183,6 +251,6 @@ def test_memory_command_paths_remember_and_forget(tmp_path: Path, monkeypatch):
         forget_events = parse_sse_payloads(forget_resp.text)
         assert any("Forgot" in p.get("token", "") for p in forget_events)
 
-        memories_after_forget = client.get("/api/memories", headers={"X-Session-ID": sid})
+        memories_after_forget = client.get("/api/memories", headers={"X-Session-ID": sid, "Origin": "http://testserver"})
         assert memories_after_forget.status_code == 200
         assert memories_after_forget.json().get("count", 0) == 0

tests/test_completions.py

@@ -0,0 +1,222 @@
+import json
+import os
+from pathlib import Path
+
+import httpx
+from fastapi.testclient import TestClient
+
+import app
+import config
+import db
+import routers.completions
+from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS
+
+
+def make_client(tmp_path: Path) -> TestClient:
+    os.environ["JARVISCHAT_ADMIN_PIN"] = "1234"
+    db.DB_PATH = tmp_path / "jarvischat-completions.db"
+    SESSIONS.clear()
+    PIN_ATTEMPTS.clear()
+    RATE_EVENTS.clear()
+    db.init_db()
+    return TestClient(app.app, raise_server_exceptions=False)
+
+
+TEST_API_KEY = "test-sk-jarvischat-completions"
+
+
+def _auth_headers(extra: dict = None) -> dict:
+    h = {"Authorization": f"Bearer {TEST_API_KEY}", "Content-Type": "application/json", "Origin": "http://testserver"}
+    if extra:
+        h.update(extra)
+    return h
+
+
+class _MockStreamResponse:
+    def __init__(self, lines: list[str]):
+        self._lines = lines
+
+    async def __aenter__(self):
+        return self
+
+    async def __aexit__(self, exc_type, exc, tb):
+        return False
+
+    async def aiter_lines(self):
+        for line in self._lines:
+            yield line
+
+
+class _MockAsyncPostResponse:
+    def __init__(self, status_code=200, json_data=None):
+        self.status_code = status_code
+        self._json_data = json_data or {}
+
+    def json(self):
+        return self._json_data
+
+
+def _stream_json_lines(events: list[dict]) -> list[str]:
+    return [json.dumps(event) for event in events]
+
+
+def test_completions_missing_api_key(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.post(
+            "/v1/chat/completions",
+            json={"messages": [{"role": "user", "content": "hi"}]},
+        )
+        assert resp.status_code == 401
+
+
+def test_completions_invalid_api_key(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.post(
+            "/v1/chat/completions",
+            json={"messages": [{"role": "user", "content": "hi"}]},
+            headers={"Authorization": "Bearer wrong-key", "Origin": "http://testserver"},
+        )
+        assert resp.status_code == 401
+
+
+def test_completions_no_messages(tmp_path: Path, monkeypatch):
+    monkeypatch.setattr(routers.completions, "COMPLETIONS_API_KEY", TEST_API_KEY)
+    with make_client(tmp_path) as client:
+        resp = client.post("/v1/chat/completions", json={}, headers=_auth_headers())
+        assert resp.status_code == 400
+
+
+def test_completions_empty_messages(tmp_path: Path, monkeypatch):
+    monkeypatch.setattr(routers.completions, "COMPLETIONS_API_KEY", TEST_API_KEY)
+    with make_client(tmp_path) as client:
+        resp = client.post("/v1/chat/completions", json={"messages": []}, headers=_auth_headers())
+        assert resp.status_code == 400
+
+
+def test_completions_no_user_message(tmp_path: Path, monkeypatch):
+    monkeypatch.setattr(routers.completions, "COMPLETIONS_API_KEY", TEST_API_KEY)
+    with make_client(tmp_path) as client:
+        resp = client.post(
+            "/v1/chat/completions",
+            json={"messages": [{"role": "assistant", "content": "hello"}]},
+            headers=_auth_headers(),
+        )
+        assert resp.status_code == 400
+
+
+def test_completions_streaming(tmp_path: Path, monkeypatch):
+    monkeypatch.setattr(routers.completions, "COMPLETIONS_API_KEY", TEST_API_KEY)
+    events = _stream_json_lines([
+        {"choices": [{"delta": {"content": "Hello"}, "logprobs": None}]},
+        {"choices": [{"delta": {"content": " world"}, "logprobs": None}]},
+        {"choices": [{"delta": {}, "finish_reason": "stop"}], "usage": {"tokens_per_second": 15.0}},
+    ])
+
+    call_count = 0
+
+    def stream_stub(self, method, url, json=None, timeout=None):
+        nonlocal call_count
+        call_count += 1
+        return _MockStreamResponse(events)
+
+    monkeypatch.setattr(httpx.AsyncClient, "stream", stream_stub)
+
+    with make_client(tmp_path) as client:
+        resp = client.post(
+            "/v1/chat/completions",
+            json={"messages": [{"role": "user", "content": "hi"}], "stream": True},
+            headers=_auth_headers(),
+        )
+        assert resp.status_code == 200
+        body = resp.text
+        assert "data: [DONE]" in body
+        assert "Hello" in body or "world" in body
+        assert "chatcmpl-" in body
+
+
+def test_completions_blocking(tmp_path: Path, monkeypatch):
+    monkeypatch.setattr(routers.completions, "COMPLETIONS_API_KEY", TEST_API_KEY)
+    events = _stream_json_lines([
+        {"choices": [{"delta": {"content": "Hello world"}, "logprobs": None}]},
+        {"choices": [{"delta": {}, "finish_reason": "stop"}], "usage": {}},
+    ])
+
+    def stream_stub(self, method, url, json=None, timeout=None):
+        return _MockStreamResponse(events)
+
+    monkeypatch.setattr(httpx.AsyncClient, "stream", stream_stub)
+
+    with make_client(tmp_path) as client:
+        resp = client.post(
+            "/v1/chat/completions",
+            json={"messages": [{"role": "user", "content": "hi"}], "stream": False},
+            headers=_auth_headers(),
+        )
+        assert resp.status_code == 200
+        data = resp.json()
+        assert data["object"] == "chat.completion"
+        assert data["choices"][0]["message"]["content"] == "Hello world"
+
+
+def test_completions_fim_passthrough(tmp_path: Path, monkeypatch):
+    monkeypatch.setattr(routers.completions, "COMPLETIONS_API_KEY", TEST_API_KEY)
+    fim_data = {"prompt": "def foo():\n    ", "suffix": "\n    return x", "model": "llama3.1:latest"}
+
+    async def mock_post(self, url, json=None, timeout=None):
+        return _MockAsyncPostResponse(json_data={"choices": [{"text": "pass"}], "usage": {}})
+
+    monkeypatch.setattr(httpx.AsyncClient, "post", mock_post)
+
+    with make_client(tmp_path) as client:
+        resp = client.post("/v1/chat/completions", json=fim_data, headers=_auth_headers())
+        assert resp.status_code == 200
+        assert "choices" in resp.json()
+
+
+def test_completions_connect_error_stream(tmp_path: Path, monkeypatch):
+    monkeypatch.setattr(routers.completions, "COMPLETIONS_API_KEY", TEST_API_KEY)
+
+    def broken_stream(self, method, url, json=None, timeout=None):
+        raise httpx.ConnectError("Connection refused")
+
+    monkeypatch.setattr(httpx.AsyncClient, "stream", broken_stream)
+
+    with make_client(tmp_path) as client:
+        resp = client.post(
+            "/v1/chat/completions",
+            json={"messages": [{"role": "user", "content": "hi"}], "stream": True},
+            headers=_auth_headers(),
+        )
+        assert resp.status_code == 200
+        assert "connection_error" in resp.text
+
+
+def test_completions_connect_error_blocking(tmp_path: Path, monkeypatch):
+    monkeypatch.setattr(routers.completions, "COMPLETIONS_API_KEY", TEST_API_KEY)
+
+    def broken_stream(self, method, url, json=None, timeout=None):
+        raise httpx.ConnectError("Connection refused")
+
+    monkeypatch.setattr(httpx.AsyncClient, "stream", broken_stream)
+
+    with make_client(tmp_path) as client:
+        resp = client.post(
+            "/v1/chat/completions",
+            json={"messages": [{"role": "user", "content": "hi"}], "stream": False},
+            headers=_auth_headers(),
+        )
+        assert resp.status_code == 503
+
+
+def test_completions_fim_connect_error(tmp_path: Path, monkeypatch):
+    monkeypatch.setattr(routers.completions, "COMPLETIONS_API_KEY", TEST_API_KEY)
+    fim_data = {"prompt": "def foo():", "model": "llama3.1:latest"}
+
+    def broken_post(self, url, json=None, timeout=None):
+        raise httpx.ConnectError("Connection refused")
+
+    monkeypatch.setattr(httpx.AsyncClient, "post", broken_post)
+
+    with make_client(tmp_path) as client:
+        resp = client.post("/v1/chat/completions", json=fim_data, headers=_auth_headers())
+        assert resp.status_code == 503

tests/test_conversations.py

@@ -0,0 +1,153 @@
+import os
+from pathlib import Path
+
+from fastapi.testclient import TestClient
+
+import app
+import db
+from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS
+
+
+def make_client(tmp_path: Path) -> TestClient:
+    os.environ["JARVISCHAT_ADMIN_PIN"] = "1234"
+    db.DB_PATH = tmp_path / "jarvischat-conversations.db"
+    SESSIONS.clear()
+    PIN_ATTEMPTS.clear()
+    RATE_EVENTS.clear()
+    db.init_db()
+    return TestClient(app.app, raise_server_exceptions=False)
+
+
+def _admin_headers(client: TestClient) -> dict:
+    login = client.post("/api/auth/login", json={"pin": "1234"}, headers={"Origin": "http://testserver"})
+    sid = login.json()["session_id"]
+    return {"X-Session-ID": sid, "Origin": "http://testserver"}
+
+
+def _guest_headers(client: TestClient) -> dict:
+    sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()["session_id"]
+    return {"X-Session-ID": sid, "Origin": "http://testserver"}
+
+
+def test_list_conversations_empty(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.get("/api/conversations", headers=_guest_headers(client))
+        assert resp.status_code == 200
+        assert resp.json() == []
+
+
+def test_create_and_list_conversation(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        headers = _admin_headers(client)
+
+        create = client.post("/api/conversations", json={"title": "Test Chat", "model": "llama3.1:latest"}, headers=headers)
+        assert create.status_code == 200
+        data = create.json()
+        assert data["title"] == "Test Chat"
+        assert data["model"] == "llama3.1:latest"
+
+        list_resp = client.get("/api/conversations", headers=headers)
+        assert list_resp.status_code == 200
+        convs = list_resp.json()
+        assert len(convs) == 1
+        assert convs[0]["title"] == "Test Chat"
+
+
+def test_get_conversation_returns_messages(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        headers = _admin_headers(client)
+        create = client.post("/api/conversations", json={"title": "My Chat"}, headers=headers)
+        conv_id = create.json()["id"]
+
+        resp = client.get(f"/api/conversations/{conv_id}", headers=headers)
+        assert resp.status_code == 200
+        data = resp.json()
+        assert data["conversation"]["id"] == conv_id
+        assert data["messages"] == []
+
+
+def test_get_conversation_not_found(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.get("/api/conversations/nope", headers=_guest_headers(client))
+        assert resp.status_code == 404
+
+
+def test_update_conversation_title(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        headers = _admin_headers(client)
+        create = client.post("/api/conversations", json={"title": "Old"}, headers=headers)
+        conv_id = create.json()["id"]
+
+        update = client.put(f"/api/conversations/{conv_id}", json={"title": "New Title"}, headers=headers)
+        assert update.status_code == 200
+
+        get = client.get(f"/api/conversations/{conv_id}", headers=headers)
+        assert get.json()["conversation"]["title"] == "New Title"
+
+
+def test_update_conversation_model(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        headers = _admin_headers(client)
+        create = client.post("/api/conversations", json={"title": "Test"}, headers=headers)
+        conv_id = create.json()["id"]
+
+        update = client.put(f"/api/conversations/{conv_id}", json={"model": "qwen2:latest"}, headers=headers)
+        assert update.status_code == 200
+
+        get = client.get(f"/api/conversations/{conv_id}", headers=headers)
+        assert get.json()["conversation"]["model"] == "qwen2:latest"
+
+
+def test_delete_conversation(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        headers = _admin_headers(client)
+        create = client.post("/api/conversations", json={"title": "Delete Me"}, headers=headers)
+        conv_id = create.json()["id"]
+
+        delete = client.delete(f"/api/conversations/{conv_id}", headers=headers)
+        assert delete.status_code == 200
+
+        get = client.get(f"/api/conversations/{conv_id}", headers=_guest_headers(client))
+        assert get.status_code == 404
+
+
+def test_delete_all_conversations(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        headers = _admin_headers(client)
+        client.post("/api/conversations", json={"title": "One"}, headers=headers)
+        client.post("/api/conversations", json={"title": "Two"}, headers=headers)
+
+        delete_all = client.delete("/api/conversations", headers=headers)
+        assert delete_all.status_code == 200
+
+        list_resp = client.get("/api/conversations", headers=_guest_headers(client))
+        assert list_resp.json() == []
+
+
+def test_guest_cannot_create_conversation(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.post("/api/conversations", json={"title": "test"}, headers=_guest_headers(client))
+        assert resp.status_code == 403
+
+
+def test_guest_cannot_update_conversation(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        headers = _admin_headers(client)
+        create = client.post("/api/conversations", json={"title": "Test"}, headers=headers)
+        conv_id = create.json()["id"]
+
+        guest_headers = _guest_headers(client)
+        resp = client.put(f"/api/conversations/{conv_id}", json={"title": "hack"}, headers=guest_headers)
+        assert resp.status_code == 403
+
+
+def test_guest_cannot_delete_conversation(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.delete("/api/conversations/some-id", headers=_guest_headers(client))
+        assert resp.status_code == 403
+
+
+def test_guest_cannot_delete_all(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.delete("/api/conversations", headers=_guest_headers(client))
+        assert resp.status_code == 403

tests/test_error_envelopes.py

@@ -1,19 +1,24 @@
 import os
 from pathlib import Path
 
+import httpx
 from fastapi.testclient import TestClient
 
-import app as app_module
+import app
+import config
+import db
+import routers.memories
+from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS
 
 
 def make_client(tmp_path: Path) -> TestClient:
     os.environ["JARVISCHAT_ADMIN_PIN"] = "1234"
-    app_module.DB_PATH = tmp_path / "jarvischat-errors.db"
-    app_module.SESSIONS.clear()
-    app_module.PIN_ATTEMPTS.clear()
-    app_module.RATE_EVENTS.clear()
-    app_module.init_db()
-    return TestClient(app_module.app, raise_server_exceptions=False)
+    db.DB_PATH = tmp_path / "jarvischat-errors.db"
+    SESSIONS.clear()
+    PIN_ATTEMPTS.clear()
+    RATE_EVENTS.clear()
+    db.init_db()
+    return TestClient(app.app, raise_server_exceptions=False)
 
 
 def test_unhandled_api_exception_returns_friendly_error_with_incident_key(
@@ -23,12 +28,12 @@ def test_unhandled_api_exception_returns_friendly_error_with_incident_key(
         sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()[
             "session_id"
         ]
-        headers = {"X-Session-ID": sid}
+        headers = {"X-Session-ID": sid, "Origin": "http://testserver"}
 
         def boom(_topic=None):
             raise RuntimeError("super secret db internals")
 
-        monkeypatch.setattr(app_module, "get_all_memories", boom)
+        monkeypatch.setattr(routers.memories, "get_all_memories", boom)
 
         resp = client.get("/api/memories", headers=headers)
         assert resp.status_code == 500
@@ -57,11 +62,11 @@ def test_chat_stream_error_hides_internal_exception_and_emits_incident_key(
         def broken_stream(*args, **kwargs):
             return BrokenStreamContext()
 
-        monkeypatch.setattr(app_module.httpx.AsyncClient, "stream", broken_stream)
+        monkeypatch.setattr(httpx.AsyncClient, "stream", broken_stream)
 
         resp = client.post(
             "/api/chat",
-            json={"message": "hello", "model": app_module.DEFAULT_MODEL},
+            json={"message": "hello", "model": config.DEFAULT_MODEL},
             headers=headers,
         )
 

tests/test_hardware.py

@@ -0,0 +1,176 @@
+import asyncio
+import json
+import os
+import subprocess
+from pathlib import Path
+
+import httpx
+import psutil
+from fastapi.testclient import TestClient
+
+import app as app_module
+import config
+import db
+import hardware
+from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS
+
+
+def make_client(tmp_path: Path) -> TestClient:
+    os.environ["JARVISCHAT_ADMIN_PIN"] = "1234"
+    db.DB_PATH = tmp_path / "jarvischat-hardware.db"
+    hardware.HARDWARE_STATE_PATH = tmp_path / "hardware_state.json"
+    SESSIONS.clear()
+    PIN_ATTEMPTS.clear()
+    RATE_EVENTS.clear()
+    db.init_db()
+    return TestClient(app_module.app, raise_server_exceptions=False)
+
+
+def _guest_headers(client: TestClient) -> dict:
+    sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()["session_id"]
+    return {"X-Session-ID": sid, "Origin": "http://testserver"}
+
+
+class _MockGet:
+    def __init__(self, status_code: int = 200, json_data: dict | None = None):
+        self.status_code = status_code
+        self._json_data = json_data or {}
+
+    def json(self):
+        return self._json_data
+
+
+def _mock_subprocess(rocm_stdout: str = "") -> object:
+    class MockProc:
+        returncode = 0
+        stdout = rocm_stdout
+
+    class MockSP:
+        TimeoutExpired = subprocess.TimeoutExpired
+        run = staticmethod(lambda cmd, **kw: MockProc())
+
+    return MockSP()
+
+
+def _broken_subprocess(exception: Exception) -> object:
+    class MockSP:
+        TimeoutExpired = subprocess.TimeoutExpired
+        run = staticmethod(lambda cmd, **kw: (_ for _ in ()).throw(exception))
+
+    return MockSP()
+
+
+def test_assess_hardware_all_services_reachable(tmp_path: Path, monkeypatch):
+    hardware.HARDWARE_STATE_PATH = tmp_path / "hardware_state.json"
+    monkeypatch.setattr(psutil, "virtual_memory", lambda: type("M", (), {"total": 16 * 1024 ** 3, "available": 8 * 1024 ** 3})())
+    monkeypatch.setattr(psutil, "cpu_count", lambda: 8)
+    monkeypatch.setattr(hardware, "subprocess", _mock_subprocess(
+        json.dumps({"card0": {"VRAM Total (MB)": 8192, "VRAM Free (MB)": 4096}})
+    ))
+
+    async def mock_get(self, url, *args, **kwargs):
+        if "v1/models" in url:
+            return _MockGet(200, {"data": [{"id": "mistral-nemo:latest"}]})
+        if "6333" in url:
+            return _MockGet(200, {"result": {"collections": [{"name": "jarvischat"}]}})
+        if "8888" in url:
+            return _MockGet(200, {})
+        return _MockGet(200, {})
+    monkeypatch.setattr(httpx.AsyncClient, "get", mock_get)
+
+    state = asyncio.run(hardware.assess_hardware())
+
+    assert state["ram_total_gb"] == 16.0
+    assert state["ram_available_gb"] == 8.0
+    assert state["cpu_count"] == 8
+    assert state["vram_total_mb"] == 8192
+    assert state["vram_free_mb"] == 4096
+    assert state["llama_reachable"] is True
+    assert state["llama_models"] == ["mistral-nemo:latest"]
+    assert state["qdrant_reachable"] is True
+    assert state["qdrant_collections"] == ["jarvischat"]
+    assert state["searxng_reachable"] is True
+    assert tmp_path.joinpath("hardware_state.json").exists()
+
+
+def test_assess_hardware_rocm_smi_absent(tmp_path: Path, monkeypatch):
+    hardware.HARDWARE_STATE_PATH = tmp_path / "hardware_state.json"
+    monkeypatch.setattr(psutil, "virtual_memory", lambda: type("M", (), {"total": 16 * 1024 ** 3, "available": 8 * 1024 ** 3})())
+    monkeypatch.setattr(psutil, "cpu_count", lambda: 8)
+    monkeypatch.setattr(hardware, "subprocess", _broken_subprocess(FileNotFoundError("no rocm-smi")))
+
+    async def mock_get(self, url, *args, **kwargs):
+        if "v1/models" in url:
+            return _MockGet(200, {"data": []})
+        if "6333" in url:
+            return _MockGet(200, {"result": {"collections": []}})
+        if "8888" in url:
+            return _MockGet(200, {})
+        return _MockGet(200, {})
+    monkeypatch.setattr(httpx.AsyncClient, "get", mock_get)
+
+    state = asyncio.run(hardware.assess_hardware())
+
+    assert state["vram_total_mb"] == 0
+    assert state["vram_free_mb"] == 0
+    assert state["llama_reachable"] is True
+    assert state["qdrant_reachable"] is True
+    assert state["searxng_reachable"] is True
+
+
+def test_assess_hardware_llama_unreachable(tmp_path: Path, monkeypatch):
+    hardware.HARDWARE_STATE_PATH = tmp_path / "hardware_state.json"
+    monkeypatch.setattr(psutil, "virtual_memory", lambda: type("M", (), {"total": 16 * 1024 ** 3, "available": 8 * 1024 ** 3})())
+    monkeypatch.setattr(psutil, "cpu_count", lambda: 8)
+    monkeypatch.setattr(hardware, "subprocess", _mock_subprocess(
+        json.dumps({"card0": {"VRAM Total (MB)": 8192, "VRAM Free (MB)": 4096}})
+    ))
+
+    async def mock_get(self, url, *args, **kwargs):
+        if "v1/models" in url:
+            raise httpx.ConnectError("refused")
+        if "6333" in url:
+            return _MockGet(200, {"result": {"collections": [{"name": "jarvischat"}]}})
+        if "8888" in url:
+            return _MockGet(200, {})
+        return _MockGet(200, {})
+    monkeypatch.setattr(httpx.AsyncClient, "get", mock_get)
+
+    state = asyncio.run(hardware.assess_hardware())
+
+    assert state["llama_reachable"] is False
+    assert state["llama_models"] == []
+    assert state["qdrant_reachable"] is True
+    assert state["searxng_reachable"] is True
+
+
+def test_get_hardware_endpoint(tmp_path: Path, monkeypatch):
+    hardware.HARDWARE_STATE_PATH = tmp_path / "hardware_state.json"
+    monkeypatch.setattr(psutil, "virtual_memory", lambda: type("M", (), {"total": 16 * 1024 ** 3, "available": 8 * 1024 ** 3})())
+    monkeypatch.setattr(psutil, "cpu_count", lambda: 8)
+    monkeypatch.setattr(hardware, "subprocess", _mock_subprocess(
+        json.dumps({"card0": {"VRAM Total (MB)": 8192, "VRAM Free (MB)": 4096}})
+    ))
+
+    async def mock_get(self, url, *args, **kwargs):
+        if "v1/models" in url:
+            return _MockGet(200, {"data": [{"id": "mistral-nemo:latest"}]})
+        if "6333" in url:
+            return _MockGet(200, {"result": {"collections": [{"name": "jarvischat"}]}})
+        if "8888" in url:
+            return _MockGet(200, {})
+        return _MockGet(200, {})
+    monkeypatch.setattr(httpx.AsyncClient, "get", mock_get)
+
+    with make_client(tmp_path) as client:
+        resp = client.get("/api/hardware", headers=_guest_headers(client))
+        assert resp.status_code == 200
+        data = resp.json()
+        assert data["ram_total_gb"] == 16.0
+        assert data["cpu_count"] == 8
+        assert data["vram_total_mb"] == 8192
+        assert data["llama_reachable"] is True
+        assert data["qdrant_reachable"] is True
+        assert data["searxng_reachable"] is True
+        assert "llama_models" in data
+        assert "qdrant_collections" in data

tests/test_ingest.py

@@ -0,0 +1,104 @@
+import json
+import os
+from pathlib import Path
+
+import httpx
+from fastapi.testclient import TestClient
+
+import app
+import db
+import routers.ingest as ingest_route
+from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS
+
+
+def make_client(tmp_path: Path) -> TestClient:
+    os.environ["JARVISCHAT_ADMIN_PIN"] = "1234"
+    db.DB_PATH = tmp_path / "jarvischat-ingest.db"
+    SESSIONS.clear()
+    PIN_ATTEMPTS.clear()
+    RATE_EVENTS.clear()
+    db.init_db()
+    return TestClient(app.app, raise_server_exceptions=False)
+
+
+TEST_API_KEY = "test-sk-jarvischat-ingest"
+
+
+def _auth_headers() -> dict:
+    return {"Authorization": f"Bearer {TEST_API_KEY}", "Content-Type": "application/json", "Origin": "http://testserver"}
+
+
+def test_ingest_missing_api_key(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.post("/api/ingest", json={"content": "test"}, headers={"Origin": "http://testserver"})
+        assert resp.status_code == 401
+
+
+def test_ingest_wrong_api_key(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.post("/api/ingest", json={"content": "test"},
+                           headers={"Authorization": "Bearer wrong", "Origin": "http://testserver"})
+        assert resp.status_code == 401
+
+
+def test_ingest_empty_content(tmp_path: Path):
+    monkeypatch = __import__('pytest').MonkeyPatch()
+    monkeypatch.setattr(ingest_route, "COMPLETIONS_API_KEY", TEST_API_KEY)
+    with make_client(tmp_path) as client:
+        resp = client.post("/api/ingest", json={"content": ""}, headers=_auth_headers())
+        assert resp.status_code == 422
+    monkeypatch.undo()
+
+
+def test_ingest_missing_content(tmp_path: Path):
+    monkeypatch = __import__('pytest').MonkeyPatch()
+    monkeypatch.setattr(ingest_route, "COMPLETIONS_API_KEY", TEST_API_KEY)
+    with make_client(tmp_path) as client:
+        resp = client.post("/api/ingest", json={}, headers=_auth_headers())
+        assert resp.status_code == 422
+    monkeypatch.undo()
+
+
+def test_ingest_success(tmp_path: Path, monkeypatch):
+    monkeypatch.setattr(ingest_route, "COMPLETIONS_API_KEY", TEST_API_KEY)
+
+    embed_count = 0
+
+    class FakeAsyncClient:
+        class FakeResponse:
+            def __init__(self, status, json_data=None):
+                self.status_code = status
+                self._json = json_data or {}
+
+            def json(self):
+                return self._json
+
+        def __init__(self, *a, **kw):
+            pass
+
+        async def __aenter__(self):
+            return self
+
+        async def __aexit__(self, *a):
+            pass
+
+        async def post(self, url, **kw):
+            nonlocal embed_count
+            if "/api/embeddings" in url:
+                embed_count += 1
+                return self.FakeResponse(200, {"embedding": [0.1] * 768})
+            return self.FakeResponse(200)
+
+        async def put(self, url, **kw):
+            return self.FakeResponse(200)
+
+    monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: FakeAsyncClient())
+
+    with make_client(tmp_path) as client:
+        resp = client.post("/api/ingest", json={"content": "test " * 1000, "source": "terminal"}, headers=_auth_headers())
+        assert resp.status_code == 200
+        data = resp.json()
+        assert data["source"] == "terminal"
+        assert data["chunks_ingested"] > 0
+        assert embed_count == data["chunks_ingested"]
+        assert "message" in data

tests/test_ip_allowlist.py

@@ -3,48 +3,42 @@ from pathlib import Path
 
 from fastapi.testclient import TestClient
 
-import app as app_module
+import app
+import db
+from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS, is_ip_allowed
 
 
 def make_client(tmp_path: Path) -> TestClient:
     os.environ["JARVISCHAT_ADMIN_PIN"] = "1234"
-    app_module.DB_PATH = tmp_path / "jarvischat-ip.db"
-    app_module.SESSIONS.clear()
-    app_module.PIN_ATTEMPTS.clear()
-    app_module.RATE_EVENTS.clear()
-    app_module.init_db()
-    return TestClient(app_module.app)
+    db.DB_PATH = tmp_path / "jarvischat-ip.db"
+    SESSIONS.clear()
+    PIN_ATTEMPTS.clear()
+    RATE_EVENTS.clear()
+    db.init_db()
+    return TestClient(app.app)
 
 
 def test_ip_helper_allows_local_defaults():
-    assert app_module.is_ip_allowed("127.0.0.1")
-    assert app_module.is_ip_allowed("192.168.1.10")
-    assert app_module.is_ip_allowed("10.0.0.42")
-    assert app_module.is_ip_allowed("172.16.1.2")
-    assert app_module.is_ip_allowed("testclient")
+    assert is_ip_allowed("127.0.0.1")
+    assert is_ip_allowed("192.168.1.10")
+    assert is_ip_allowed("10.0.0.42")
+    assert is_ip_allowed("172.16.1.2")
+    assert is_ip_allowed("testclient")
 
 
 def test_ip_helper_blocks_public_ip():
-    assert not app_module.is_ip_allowed("8.8.8.8")
+    assert not is_ip_allowed("8.8.8.8")
 
 
-def test_middleware_blocks_disallowed_ip(tmp_path: Path):
+def test_middleware_blocks_disallowed_ip(tmp_path: Path, monkeypatch):
+    monkeypatch.setattr(app, "get_client_ip", lambda _req: "8.8.8.8")
     with make_client(tmp_path) as client:
-        original_get_client_ip = app_module.get_client_ip
-        try:
-            app_module.get_client_ip = lambda _req: "8.8.8.8"
-            resp = client.post("/api/auth/guest")
-            assert resp.status_code == 403
-        finally:
-            app_module.get_client_ip = original_get_client_ip
+        resp = client.post("/api/auth/guest")
+        assert resp.status_code == 403
 
 
-def test_middleware_allows_local_ip(tmp_path: Path):
+def test_middleware_allows_local_ip(tmp_path: Path, monkeypatch):
+    monkeypatch.setattr(app, "get_client_ip", lambda _req: "192.168.50.109")
     with make_client(tmp_path) as client:
-        original_get_client_ip = app_module.get_client_ip
-        try:
-            app_module.get_client_ip = lambda _req: "192.168.50.109"
-            resp = client.post("/api/auth/guest")
-            assert resp.status_code == 200
-        finally:
-            app_module.get_client_ip = original_get_client_ip
+        resp = client.post("/api/auth/guest", headers={"Origin": "http://testserver"})
+        assert resp.status_code == 200

tests/test_memories.py

@@ -0,0 +1,161 @@
+import os
+from pathlib import Path
+
+from fastapi.testclient import TestClient
+
+import app
+import config
+import db
+from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS
+
+
+def make_client(tmp_path: Path) -> TestClient:
+    os.environ["JARVISCHAT_ADMIN_PIN"] = "1234"
+    db.DB_PATH = tmp_path / "jarvischat-memories.db"
+    SESSIONS.clear()
+    PIN_ATTEMPTS.clear()
+    RATE_EVENTS.clear()
+    db.init_db()
+    return TestClient(app.app, raise_server_exceptions=False)
+
+
+def _admin_headers(client: TestClient) -> dict:
+    login = client.post("/api/auth/login", json={"pin": "1234"}, headers={"Origin": "http://testserver"})
+    sid = login.json()["session_id"]
+    return {"X-Session-ID": sid, "Origin": "http://testserver"}
+
+
+def _guest_headers(client: TestClient) -> dict:
+    sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()["session_id"]
+    return {"X-Session-ID": sid, "Origin": "http://testserver"}
+
+
+def _create_memory(client: TestClient, headers: dict, fact: str = "test fact", topic: str = "general") -> int:
+    resp = client.post("/api/memories", json={"fact": fact, "topic": topic}, headers=headers)
+    assert resp.status_code == 200
+    return resp.json()["rowid"]
+
+
+def test_list_memories_empty(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.get("/api/memories", headers=_guest_headers(client))
+        assert resp.status_code == 200
+        assert resp.json()["count"] == 0
+
+
+def test_list_memories_by_topic(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        headers = _admin_headers(client)
+        _create_memory(client, headers, "I like Python", "preference")
+        _create_memory(client, headers, "Building a game", "project")
+
+        general = client.get("/api/memories?topic=preference", headers=_guest_headers(client))
+        assert general.json()["count"] == 1
+        assert general.json()["memories"][0]["topic"] == "preference"
+
+
+def test_create_memory_requires_fact(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.post("/api/memories", json={"fact": ""}, headers=_admin_headers(client))
+        assert resp.status_code == 400
+
+
+def test_create_memory_too_long(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        long_fact = "x" * (config.MAX_MEMORY_FACT_CHARS + 1)
+        resp = client.post("/api/memories", json={"fact": long_fact}, headers=_admin_headers(client))
+        assert resp.status_code == 413
+
+
+def test_edit_memory(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        headers = _admin_headers(client)
+        rowid = _create_memory(client, headers, "original fact")
+
+        edit = client.put(f"/api/memories/{rowid}", json={"fact": "updated fact"}, headers=headers)
+        assert edit.status_code == 200
+
+        memories = client.get("/api/memories", headers=_guest_headers(client)).json()
+        assert any(m["fact"] == "updated fact" for m in memories["memories"])
+
+
+def test_edit_memory_not_found(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.put("/api/memories/99999", json={"fact": "nope"}, headers=_admin_headers(client))
+        assert resp.status_code == 404
+
+
+def test_edit_memory_empty_fact(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        headers = _admin_headers(client)
+        rowid = _create_memory(client, headers, "some fact")
+        resp = client.put(f"/api/memories/{rowid}", json={"fact": ""}, headers=headers)
+        assert resp.status_code == 400
+
+
+def test_edit_memory_too_long(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        headers = _admin_headers(client)
+        rowid = _create_memory(client, headers, "some fact")
+        long_fact = "x" * (config.MAX_MEMORY_FACT_CHARS + 1)
+        resp = client.put(f"/api/memories/{rowid}", json={"fact": long_fact}, headers=headers)
+        assert resp.status_code == 413
+
+
+def test_delete_memory_not_found(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.delete("/api/memories/99999", headers=_admin_headers(client))
+        assert resp.status_code == 404
+
+
+def test_search_memories(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        headers = _admin_headers(client)
+        _create_memory(client, headers, "my favorite color is blue", "preference")
+        _create_memory(client, headers, "running nginx on port 443", "infrastructure")
+
+        resp = client.get("/api/memories/search?q=nginx&limit=5", headers=_guest_headers(client))
+        assert resp.status_code == 200
+        data = resp.json()
+        assert data["count"] >= 1
+        assert any("nginx" in r["fact"] for r in data["results"])
+
+
+def test_search_memories_no_results(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.get("/api/memories/search?q=xyznonexistent&limit=5", headers=_guest_headers(client))
+        assert resp.status_code == 200
+        assert resp.json()["count"] == 0
+
+
+def test_memory_stats(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        headers = _admin_headers(client)
+        _create_memory(client, headers, "like rust", "preference")
+        _create_memory(client, headers, "like python", "preference")
+        _create_memory(client, headers, "project game", "project")
+
+        resp = client.get("/api/memories/stats", headers=_guest_headers(client))
+        assert resp.status_code == 200
+        data = resp.json()
+        assert data["total"] == 3
+        assert data["by_topic"]["preference"] == 2
+        assert data["by_topic"]["project"] == 1
+
+
+def test_guest_cannot_create_memory(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.post("/api/memories", json={"fact": "hack"}, headers=_guest_headers(client))
+        assert resp.status_code == 403
+
+
+def test_guest_cannot_edit_memory(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.put("/api/memories/1", json={"fact": "hack"}, headers=_guest_headers(client))
+        assert resp.status_code == 403
+
+
+def test_guest_cannot_delete_memory(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.delete("/api/memories/1", headers=_guest_headers(client))
+        assert resp.status_code == 403

tests/test_models_router.py

@@ -0,0 +1,138 @@
+import os
+from pathlib import Path
+
+import httpx
+from fastapi.testclient import TestClient
+
+import app
+import db
+import routers.models
+from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS
+
+
+def make_client(tmp_path: Path) -> TestClient:
+    os.environ["JARVISCHAT_ADMIN_PIN"] = "1234"
+    db.DB_PATH = tmp_path / "jarvischat-models.db"
+    SESSIONS.clear()
+    PIN_ATTEMPTS.clear()
+    RATE_EVENTS.clear()
+    db.init_db()
+    return TestClient(app.app, raise_server_exceptions=False)
+
+
+def _guest_headers(client: TestClient) -> dict:
+    sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()["session_id"]
+    return {"X-Session-ID": sid, "Origin": "http://testserver"}
+
+
+class _MockAsyncResponse:
+    """Mock for httpx.AsyncClient.get/post that returns a JSON response."""
+    def __init__(self, status_code=200, json_data=None):
+        self.status_code = status_code
+        self._json_data = json_data or {}
+
+    def json(self):
+        return self._json_data
+
+
+async def _mock_get_models(*args, **kwargs):
+    return _MockAsyncResponse(json_data={
+        "data": [{"id": "llama3.1:latest"}, {"id": "qwen2:latest"}]
+    })
+
+
+async def _mock_get_empty_models(*args, **kwargs):
+    return _MockAsyncResponse(json_data={"data": []})
+
+
+async def _mock_connect_error(*args, **kwargs):
+    raise httpx.ConnectError("Connection refused")
+
+
+async def _mock_show_model(*args, **kwargs):
+    return _MockAsyncResponse(json_data={
+        "modelfile": "FROM llama3.1", "parameters": {}
+    })
+
+
+async def _mock_search_available(*args, **kwargs):
+    return _MockAsyncResponse(status_code=200)
+
+
+async def _mock_search_unavailable(*args, **kwargs):
+    raise httpx.ConnectError("refused")
+
+
+def test_list_models(tmp_path: Path, monkeypatch):
+    monkeypatch.setattr(httpx.AsyncClient, "get", _mock_get_models)
+    with make_client(tmp_path) as client:
+        resp = client.get("/api/models", headers=_guest_headers(client))
+        assert resp.status_code == 200
+        models = resp.json()["models"]
+        assert len(models) == 2
+        assert models[0]["name"] == "llama3.1:latest"
+
+
+def test_list_models_connect_error(tmp_path: Path, monkeypatch):
+    monkeypatch.setattr(httpx.AsyncClient, "get", _mock_connect_error)
+    with make_client(tmp_path) as client:
+        resp = client.get("/api/models", headers=_guest_headers(client))
+        assert resp.status_code == 502
+
+
+def test_running_models(tmp_path: Path, monkeypatch):
+    monkeypatch.setattr(httpx.AsyncClient, "get", _mock_get_models)
+    with make_client(tmp_path) as client:
+        resp = client.get("/api/ps", headers=_guest_headers(client))
+        assert resp.status_code == 200
+        assert "data" in resp.json()
+
+
+def test_running_models_connect_error(tmp_path: Path, monkeypatch):
+    monkeypatch.setattr(httpx.AsyncClient, "get", _mock_connect_error)
+    with make_client(tmp_path) as client:
+        resp = client.get("/api/ps", headers=_guest_headers(client))
+        assert resp.status_code == 502
+
+
+def test_show_model(tmp_path: Path, monkeypatch):
+    monkeypatch.setattr(httpx.AsyncClient, "post", _mock_show_model)
+    with make_client(tmp_path) as client:
+        resp = client.post("/api/show", json={"model": "llama3.1:latest"}, headers=_guest_headers(client))
+        assert resp.status_code == 200
+        assert resp.json()["modelfile"] == "FROM llama3.1"
+
+
+def test_show_model_connect_error(tmp_path: Path, monkeypatch):
+    monkeypatch.setattr(httpx.AsyncClient, "post", _mock_connect_error)
+    with make_client(tmp_path) as client:
+        resp = client.post("/api/show", json={"model": "llama3.1:latest"}, headers=_guest_headers(client))
+        assert resp.status_code == 502
+
+
+def test_system_stats(tmp_path: Path, monkeypatch):
+    monkeypatch.setattr(routers.models, "get_gpu_stats", lambda: {"gpu_percent": 15, "vram_percent": 30, "available": True})
+    with make_client(tmp_path) as client:
+        resp = client.get("/api/stats", headers=_guest_headers(client))
+        assert resp.status_code == 200
+        data = resp.json()
+        assert "cpu_percent" in data
+        assert "memory_percent" in data
+        assert data["gpu_percent"] == 15
+        assert data["gpu_available"] is True
+
+
+def test_search_status_available(tmp_path: Path, monkeypatch):
+    monkeypatch.setattr(httpx.AsyncClient, "get", _mock_search_available)
+    with make_client(tmp_path) as client:
+        resp = client.get("/api/search/status", headers=_guest_headers(client))
+        assert resp.status_code == 200
+        assert resp.json()["available"] is True
+
+
+def test_search_status_unavailable(tmp_path: Path, monkeypatch):
+    monkeypatch.setattr(httpx.AsyncClient, "get", _mock_search_unavailable)
+    with make_client(tmp_path) as client:
+        resp = client.get("/api/search/status", headers=_guest_headers(client))
+        assert resp.status_code == 200
+        assert resp.json()["available"] is False

tests/test_presets.py

@@ -0,0 +1,128 @@
+import os
+from pathlib import Path
+
+from fastapi.testclient import TestClient
+
+import app
+import db
+from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS
+
+
+def make_client(tmp_path: Path) -> TestClient:
+    os.environ["JARVISCHAT_ADMIN_PIN"] = "1234"
+    db.DB_PATH = tmp_path / "jarvischat-presets.db"
+    SESSIONS.clear()
+    PIN_ATTEMPTS.clear()
+    RATE_EVENTS.clear()
+    db.init_db()
+    return TestClient(app.app, raise_server_exceptions=False)
+
+
+def _admin_headers(client: TestClient) -> dict:
+    login = client.post("/api/auth/login", json={"pin": "1234"}, headers={"Origin": "http://testserver"})
+    sid = login.json()["session_id"]
+    return {"X-Session-ID": sid, "Origin": "http://testserver"}
+
+
+def _guest_headers(client: TestClient) -> dict:
+    sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()["session_id"]
+    return {"X-Session-ID": sid, "Origin": "http://testserver"}
+
+
+def test_list_presets_returns_defaults(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.get("/api/presets", headers=_guest_headers(client))
+        assert resp.status_code == 200
+        presets = resp.json()
+        assert len(presets) >= 3
+        names = [p["name"] for p in presets]
+        assert "Coding Companion" in names
+
+
+def test_create_preset(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        headers = _admin_headers(client)
+        resp = client.post("/api/presets", json={"name": "My Preset", "prompt": "You are helpful."}, headers=headers)
+        assert resp.status_code == 200
+        data = resp.json()
+        assert data["name"] == "My Preset"
+        assert data["prompt"] == "You are helpful."
+
+        presets = client.get("/api/presets", headers=_guest_headers(client)).json()
+        assert any(p["name"] == "My Preset" for p in presets)
+
+
+def test_create_preset_requires_name_and_prompt(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        headers = _admin_headers(client)
+        resp = client.post("/api/presets", json={"name": "", "prompt": ""}, headers=headers)
+        assert resp.status_code == 400
+
+        resp = client.post("/api/presets", json={"name": "Only Name"}, headers=headers)
+        assert resp.status_code == 400
+
+
+def test_update_preset(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        headers = _admin_headers(client)
+        create = client.post("/api/presets", json={"name": "Old", "prompt": "Old prompt."}, headers=headers)
+        preset_id = create.json()["id"]
+
+        update = client.put(f"/api/presets/{preset_id}", json={"name": "New", "prompt": "New prompt."}, headers=headers)
+        assert update.status_code == 200
+
+        presets = client.get("/api/presets", headers=_guest_headers(client)).json()
+        updated = next(p for p in presets if p["id"] == preset_id)
+        assert updated["name"] == "New"
+        assert updated["prompt"] == "New prompt."
+
+
+def test_update_preset_requires_fields(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        headers = _admin_headers(client)
+        resp = client.put("/api/presets/nope", json={"name": "", "prompt": ""}, headers=headers)
+        assert resp.status_code == 400
+
+
+def test_delete_preset(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        headers = _admin_headers(client)
+        create = client.post("/api/presets", json={"name": "Temp", "prompt": "Temp."}, headers=headers)
+        preset_id = create.json()["id"]
+
+        delete = client.delete(f"/api/presets/{preset_id}", headers=headers)
+        assert delete.status_code == 200
+
+        presets = client.get("/api/presets", headers=_guest_headers(client)).json()
+        assert not any(p["id"] == preset_id for p in presets)
+
+
+def test_delete_default_preset_is_noop(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        headers = _admin_headers(client)
+        presets_before = client.get("/api/presets", headers=_guest_headers(client)).json()
+        default = next(p for p in presets_before if p["is_default"])
+
+        delete = client.delete(f"/api/presets/{default['id']}", headers=headers)
+        assert delete.status_code == 200
+
+        presets_after = client.get("/api/presets", headers=_guest_headers(client)).json()
+        assert any(p["id"] == default["id"] for p in presets_after)
+
+
+def test_guest_cannot_create_preset(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.post("/api/presets", json={"name": "Hack", "prompt": "Hack"}, headers=_guest_headers(client))
+        assert resp.status_code == 403
+
+
+def test_guest_cannot_update_preset(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.put("/api/presets/some-id", json={"name": "Hack", "prompt": "Hack"}, headers=_guest_headers(client))
+        assert resp.status_code == 403
+
+
+def test_guest_cannot_delete_preset(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.delete("/api/presets/some-id", headers=_guest_headers(client))
+        assert resp.status_code == 403

tests/test_profile.py

@@ -0,0 +1,72 @@
+import os
+from pathlib import Path
+
+from fastapi.testclient import TestClient
+
+import app
+import config
+import db
+from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS
+
+
+def make_client(tmp_path: Path) -> TestClient:
+    os.environ["JARVISCHAT_ADMIN_PIN"] = "1234"
+    db.DB_PATH = tmp_path / "jarvischat-profile.db"
+    SESSIONS.clear()
+    PIN_ATTEMPTS.clear()
+    RATE_EVENTS.clear()
+    db.init_db()
+    return TestClient(app.app, raise_server_exceptions=False)
+
+
+def _admin_headers(client: TestClient) -> dict:
+    login = client.post("/api/auth/login", json={"pin": "1234"}, headers={"Origin": "http://testserver"})
+    sid = login.json()["session_id"]
+    return {"X-Session-ID": sid, "Origin": "http://testserver"}
+
+
+def _guest_headers(client: TestClient) -> dict:
+    sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()["session_id"]
+    return {"X-Session-ID": sid, "Origin": "http://testserver"}
+
+
+def test_get_profile_returns_content(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.get("/api/profile", headers=_guest_headers(client))
+        assert resp.status_code == 200
+        data = resp.json()
+        assert "content" in data
+        assert "updated_at" in data
+        assert len(data["content"]) > 0
+
+
+def test_get_default_profile(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.get("/api/profile/default", headers=_guest_headers(client))
+        assert resp.status_code == 200
+        assert resp.json()["content"] == config.DEFAULT_PROFILE
+
+
+def test_update_profile(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        headers = _admin_headers(client)
+        resp = client.put("/api/profile", json={"content": "Custom profile text."}, headers=headers)
+        assert resp.status_code == 200
+        assert "updated_at" in resp.json()
+
+        get_resp = client.get("/api/profile", headers=_guest_headers(client))
+        assert get_resp.json()["content"] == "Custom profile text."
+
+
+def test_update_profile_too_long(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        headers = _admin_headers(client)
+        long_content = "x" * (config.MAX_PROFILE_CHARS + 1)
+        resp = client.put("/api/profile", json={"content": long_content}, headers=headers)
+        assert resp.status_code == 413
+
+
+def test_guest_cannot_update_profile(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.put("/api/profile", json={"content": "hack"}, headers=_guest_headers(client))
+        assert resp.status_code == 403

tests/test_rag_management.py

@@ -0,0 +1,409 @@
+import asyncio
+import os
+from datetime import datetime, timezone, timedelta
+from pathlib import Path
+
+import httpx
+from fastapi.testclient import TestClient
+
+import app
+import config
+import db
+import rag
+from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS
+
+
+def make_client(tmp_path: Path) -> TestClient:
+    os.environ["JARVISCHAT_ADMIN_PIN"] = "1234"
+    db.DB_PATH = tmp_path / "jarvischat-rag-mgmt.db"
+    SESSIONS.clear()
+    PIN_ATTEMPTS.clear()
+    RATE_EVENTS.clear()
+    db.init_db()
+    return TestClient(app.app, raise_server_exceptions=False)
+
+
+def _admin_headers(client: TestClient) -> dict:
+    login = client.post("/api/auth/login", json={"pin": "1234"}, headers={"Origin": "http://testserver"})
+    sid = login.json()["session_id"]
+    return {"X-Session-ID": sid, "Origin": "http://testserver"}
+
+
+def _guest_headers(client: TestClient) -> dict:
+    sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()["session_id"]
+    return {"X-Session-ID": sid, "Origin": "http://testserver"}
+
+
+class FakeResponse:
+    def __init__(self, status, json_data=None):
+        self.status_code = status
+        self._json = json_data or {}
+
+    def json(self):
+        return self._json
+
+
+class FakeAsyncClient:
+    def __init__(self, *a, **kw):
+        pass
+
+    async def __aenter__(self):
+        return self
+
+    async def __aexit__(self, *a):
+        pass
+
+    async def get(self, url, **kw):
+        if "/collections/jarvis_rag" in url:
+            return FakeResponse(200, {"result": {"vectors_count": 123}})
+        return FakeResponse(200)
+
+    async def post(self, url, **kw):
+        if "/points/scroll" in url:
+            return FakeResponse(200, {"result": {"points": []}})
+        if "/points/delete" in url:
+            return FakeResponse(200)
+        return FakeResponse(200)
+
+    async def put(self, url, **kw):
+        return FakeResponse(200)
+
+
+def _old_ts(hours_ago: float = 24) -> str:
+    return (datetime.now(timezone.utc) - timedelta(hours=hours_ago)).isoformat()
+
+
+def _young_ts(hours_ago: float = 0.1) -> str:
+    return (datetime.now(timezone.utc) - timedelta(hours=hours_ago)).isoformat()
+
+
+# ---------- get_collection_count ----------
+
+def test_get_collection_count(monkeypatch):
+    monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: FakeAsyncClient())
+    count = asyncio.run(rag.get_collection_count())
+    assert count == 123
+
+
+# ---------- get_collection_stats ----------
+
+def test_get_collection_stats_shape(monkeypatch):
+    monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: FakeAsyncClient())
+    stats = asyncio.run(rag.get_collection_stats())
+    assert stats["vector_count"] == 123
+    assert stats["max_vectors"] == 50000
+    assert stats["high_water_mark"] == 40000
+    assert stats["low_water_mark"] == 10000
+    assert stats["high_water_pct"] == 80
+    assert stats["low_water_pct"] == 20
+    assert 0 < stats["percent_full"] < 1
+    assert "upload" in stats["pinned_sources"]
+    assert "profile" in stats["pinned_sources"]
+
+
+# ---------- evict_batch ----------
+
+def test_evict_batch_excludes_pinned_sources(monkeypatch):
+    """Pinned sources ('upload', 'profile') should be in the must_not scroll filter."""
+    old = _old_ts(48)
+    # Only non-pinned points are returned (real Qdrant would honour must_not filter)
+    scroll_points = [
+        {"id": "old-data", "payload": {"source": "terminal", "ingest_date": old, "retrieval_count": 0}},
+    ]
+
+    class ScrollClient(FakeAsyncClient):
+        async def post(self, url, **kw):
+            if "/points/scroll" in url:
+                must_not = kw.get("json", {}).get("filter", {}).get("must_not", [])
+                pinned_values = [m["match"]["value"] for m in must_not]
+                assert "upload" in pinned_values
+                assert "profile" in pinned_values
+                return FakeResponse(200, {"result": {"points": scroll_points}})
+            if "/points/delete" in url:
+                return FakeResponse(200)
+            return FakeResponse(200)
+
+    monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: ScrollClient())
+    deleted = asyncio.run(rag.evict_batch(10))
+    assert deleted == 1
+
+
+def test_evict_batch_respects_grace_period(monkeypatch):
+    """Vectors younger than RAG_GRACE_HOURS should be skipped."""
+    old = _old_ts(48)
+    young = _young_ts(0.1)
+    scroll_points = [
+        {"id": "mature", "payload": {"source": "terminal", "ingest_date": old, "retrieval_count": 0}},
+        {"id": "newborn", "payload": {"source": "terminal", "ingest_date": young, "retrieval_count": 0}},
+    ]
+
+    class GraceClient(FakeAsyncClient):
+        async def post(self, url, **kw):
+            if "/points/scroll" in url:
+                return FakeResponse(200, {"result": {"points": scroll_points}})
+            if "/points/delete" in url:
+                deleted = kw.get("json", {}).get("points", [])
+                assert "newborn" not in deleted
+                return FakeResponse(200)
+            return FakeResponse(200)
+
+    monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: GraceClient())
+    deleted = asyncio.run(rag.evict_batch(10))
+    assert deleted == 1
+
+
+def test_evict_batch_respects_batch_size(monkeypatch):
+    """Only up to batch_size vectors should be deleted per call."""
+    old = _old_ts(48)
+    points = [{"id": f"p{i}", "payload": {"source": "terminal", "ingest_date": old, "retrieval_count": 0}} for i in range(50)]
+
+    class BatchClient(FakeAsyncClient):
+        async def post(self, url, **kw):
+            if "/points/delete" in url:
+                deleted = kw.get("json", {}).get("points", [])
+                assert len(deleted) == 10
+                return FakeResponse(200)
+            if "/points/scroll" in url:
+                return FakeResponse(200, {"result": {"points": points}})
+            return FakeResponse(200)
+
+    monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: BatchClient())
+    deleted = asyncio.run(rag.evict_batch(10))
+    assert deleted == 10
+
+
+def test_evict_batch_all_pinned_returns_zero(monkeypatch):
+    """If scroll returns nothing (all points filtered by must_not), evict_batch returns 0."""
+    class EmptyClient(FakeAsyncClient):
+        async def post(self, url, **kw):
+            if "/points/scroll" in url:
+                return FakeResponse(200, {"result": {"points": []}})
+            return FakeResponse(200)
+
+    monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: EmptyClient())
+    deleted = asyncio.run(rag.evict_batch(10))
+    assert deleted == 0
+
+
+def test_evict_batch_scores_lowest_first(monkeypatch):
+    """Vectors with lower scores should be evicted first."""
+    old = _old_ts(48)
+    points = [
+        {"id": "high-score", "payload": {"source": "terminal", "ingest_date": old, "retrieval_count": 100}},
+        {"id": "low-score", "payload": {"source": "terminal", "ingest_date": old, "retrieval_count": 0}},
+        {"id": "mid-score", "payload": {"source": "terminal", "ingest_date": old, "retrieval_count": 50}},
+    ]
+
+    class ScoreClient(FakeAsyncClient):
+        async def post(self, url, **kw):
+            if "/points/delete" in url:
+                deleted = kw.get("json", {}).get("points", [])
+                assert "low-score" in deleted
+                assert "high-score" not in deleted
+                assert "mid-score" not in deleted
+                return FakeResponse(200)
+            if "/points/scroll" in url:
+                return FakeResponse(200, {"result": {"points": points}})
+            return FakeResponse(200)
+
+    monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: ScoreClient())
+    deleted = asyncio.run(rag.evict_batch(1))
+    assert deleted == 1
+
+
+# ---------- maybe_evict ----------
+
+def test_maybe_evict_below_high_water(monkeypatch):
+    """When count is below high-water mark, eviction should not fire."""
+    class LowCountClient(FakeAsyncClient):
+        async def get(self, url, **kw):
+            # 30000 < 40000 high water
+            return FakeResponse(200, {"result": {"vectors_count": 30000}})
+
+    monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: LowCountClient())
+    rag.EVICTION_LOG.clear()
+    evicted = asyncio.run(rag.maybe_evict())
+    assert evicted == 0
+    assert len(rag.EVICTION_LOG) == 0
+
+
+def test_maybe_evict_at_high_water(monkeypatch):
+    """When count reaches high-water mark, eviction should fire."""
+    class HighCountClient(FakeAsyncClient):
+        def __init__(self, *a, **kw):
+            super().__init__()
+            self.call_count = 0
+
+        async def get(self, url, **kw):
+            return FakeResponse(200, {"result": {"vectors_count": 45000}})
+
+        async def post(self, url, **kw):
+            if "/points/scroll" in url:
+                old = _old_ts(48)
+                points = [{"id": f"evict-me-{i}", "payload": {"source": "terminal", "ingest_date": old, "retrieval_count": 0}} for i in range(100)]
+                return FakeResponse(200, {"result": {"points": points}})
+            if "/points/delete" in url:
+                return FakeResponse(200)
+            return FakeResponse(200)
+
+    monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: HighCountClient())
+    rag.EVICTION_LOG.clear()
+    evicted = asyncio.run(rag.maybe_evict())
+    assert evicted > 0
+    assert len(rag.EVICTION_LOG) == 1
+    entry = rag.EVICTION_LOG[0]
+    assert "timestamp" in entry
+    assert entry["count"] > 0
+
+
+def test_maybe_evict_zero_config_disabled(monkeypatch):
+    """RAG_MAX_VECTORS <= 0 should disable eviction."""
+    orig = config.RAG_MAX_VECTORS
+    try:
+        config.RAG_MAX_VECTORS = 0
+        evicted = asyncio.run(rag.maybe_evict())
+        assert evicted == 0
+    finally:
+        config.RAG_MAX_VECTORS = orig
+
+
+def test_maybe_evict_all_pinned_breaks(monkeypatch):
+    """Above high water but only pinned points exist → eviction breaks with 0 deleted."""
+    class AllPinnedClient(FakeAsyncClient):
+        async def get(self, url, **kw):
+            return FakeResponse(200, {"result": {"vectors_count": 45000}})
+        async def post(self, url, **kw):
+            if "/points/scroll" in url:
+                return FakeResponse(200, {"result": {"points": []}})
+            return FakeResponse(200)
+
+    monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: AllPinnedClient())
+    rag.EVICTION_LOG.clear()
+    evicted = asyncio.run(rag.maybe_evict())
+    assert evicted == 0
+    assert len(rag.EVICTION_LOG) == 0
+
+
+# ---------- get_rag_operational_stats ----------
+
+def test_rag_operational_stats_shape(monkeypatch):
+    monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: FakeAsyncClient())
+    rag.EVICTION_LOG.clear()
+    rag.EVICTION_LOG.append({
+        "timestamp": datetime.now(timezone.utc).isoformat(),
+        "count": 500,
+        "remaining": 40000,
+    })
+    stats = asyncio.run(rag.get_rag_operational_stats())
+    assert stats["vector_count"] == 123
+    assert stats["grace_hours"] == 1
+    assert "eviction_counts_last_1m" in stats
+    assert "eviction_counts_last_5m" in stats
+    assert "eviction_counts_last_30m" in stats
+    assert stats["eviction_counts_last_1m"] == 500
+
+
+# ---------- GET /api/rag/stats ----------
+
+def test_rag_stats_endpoint(tmp_path, monkeypatch):
+    monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: FakeAsyncClient())
+    with make_client(tmp_path) as client:
+        resp = client.get("/api/rag/stats", headers=_admin_headers(client))
+        assert resp.status_code == 200
+        data = resp.json()
+        assert data["vector_count"] == 123
+        assert data["max_vectors"] == 50000
+        assert "high_water_mark" in data
+        assert "low_water_mark" in data
+        assert data["high_water_pct"] == 80
+        assert data["low_water_pct"] == 20
+        assert "percent_full" in data
+        assert data["pinned_sources"] == ["upload", "profile"]
+        assert data["grace_hours"] == 1
+        assert "eviction_counts_last_1m" in data
+        assert "eviction_counts_last_5m" in data
+        assert "eviction_counts_last_30m" in data
+        assert "pinned_count" in data
+        assert "avg_retrieval_count" in data
+        assert "at_risk_count" in data
+        assert "eviction_log_size" in data
+
+
+def test_rag_stats_requires_admin(tmp_path, monkeypatch):
+    monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: FakeAsyncClient())
+    with make_client(tmp_path) as client:
+        resp = client.get("/api/rag/stats", headers=_guest_headers(client))
+        assert resp.status_code == 403
+
+
+# ---------- POST /api/rag/flush ----------
+
+def test_rag_flush_endpoint(tmp_path, monkeypatch):
+    class FlushClient(FakeAsyncClient):
+        async def post(self, url, **kw):
+            if "/points/scroll" in url:
+                return FakeResponse(200, {"result": {"points": [{"id": "a"}, {"id": "b"}]}})
+            if "/points/delete" in url:
+                return FakeResponse(200)
+            return FakeResponse(200)
+
+    monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: FlushClient())
+    with make_client(tmp_path) as client:
+        resp = client.post("/api/rag/flush", headers=_admin_headers(client))
+        assert resp.status_code == 200
+        data = resp.json()
+        assert data["status"] == "flushed"
+        assert data["deleted_count"] == 2
+        assert data["collection"] == rag.RAG_COLLECTION
+
+
+def test_rag_flush_requires_admin(tmp_path, monkeypatch):
+    monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: FakeAsyncClient())
+    with make_client(tmp_path) as client:
+        resp = client.post("/api/rag/flush", headers=_guest_headers(client))
+        assert resp.status_code == 403
+
+
+def test_rag_flush_empty_collection(tmp_path, monkeypatch):
+    class EmptyClient(FakeAsyncClient):
+        async def post(self, url, **kw):
+            if "/points/scroll" in url:
+                return FakeResponse(200, {"result": {"points": []}})
+            return FakeResponse(200)
+
+    monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: EmptyClient())
+    with make_client(tmp_path) as client:
+        resp = client.post("/api/rag/flush", headers=_admin_headers(client))
+        assert resp.status_code == 200
+        data = resp.json()
+        assert data["deleted_count"] == 0
+        assert data["status"] == "flushed"
+
+
+# ---------- Race lock ----------
+
+def test_eviction_lock_prevents_concurrent_eviction(monkeypatch):
+    """Concurrent calls to maybe_evict should queue; only one evicts."""
+    call_order = []
+
+    async def slow_get_collection_count():
+        call_order.append("count")
+        return 45000
+
+    async def slow_evict_batch(bs):
+        call_order.append("evict")
+        await asyncio.sleep(0.05)
+        return 500
+
+    monkeypatch.setattr(rag, "get_collection_count", slow_get_collection_count)
+    monkeypatch.setattr(rag, "evict_batch", slow_evict_batch)
+    rag.EVICTION_LOG.clear()
+
+    async def run_concurrent():
+        r1, r2 = await asyncio.gather(rag.maybe_evict(), rag.maybe_evict())
+        return r1, r2
+
+    r1, r2 = asyncio.run(run_concurrent())
+    # First call evicted, second found count already below high water or lock serialized
+    assert r1 >= 0
+    assert r2 >= 0

tests/test_rate_and_payload_guardrails.py

@@ -4,28 +4,32 @@ from pathlib import Path
 
 from fastapi.testclient import TestClient
 
-import app as app_module
+import app
+import config
+import db
+import security
+from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS
 
 
 def make_client(tmp_path: Path) -> TestClient:
     os.environ["JARVISCHAT_ADMIN_PIN"] = "1234"
-    app_module.DB_PATH = tmp_path / "jarvischat-rate.db"
-    app_module.SESSIONS.clear()
-    app_module.PIN_ATTEMPTS.clear()
-    app_module.RATE_EVENTS.clear()
-    app_module.init_db()
-    return TestClient(app_module.app)
+    db.DB_PATH = tmp_path / "jarvischat-rate.db"
+    SESSIONS.clear()
+    PIN_ATTEMPTS.clear()
+    RATE_EVENTS.clear()
+    db.init_db()
+    return TestClient(app.app)
 
 
 def test_stats_rate_limit_hits_429(tmp_path: Path):
-    old_limit = app_module.RL_STATS_PER_WINDOW
-    old_window = app_module.RATE_WINDOW_SECONDS
-    app_module.RL_STATS_PER_WINDOW = 2
-    app_module.RATE_WINDOW_SECONDS = 60
+    old_limit = security.RL_STATS_PER_WINDOW
+    old_window = app.RATE_WINDOW_SECONDS
+    security.RL_STATS_PER_WINDOW = 2
+    app.RATE_WINDOW_SECONDS = 60
     try:
         with make_client(tmp_path) as client:
-            sid = client.post("/api/auth/guest").json()["session_id"]
-            headers = {"X-Session-ID": sid}
+            sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()["session_id"]
+            headers = {"X-Session-ID": sid, "Origin": "http://testserver"}
 
             r1 = client.get("/api/stats", headers=headers)
             r2 = client.get("/api/stats", headers=headers)
@@ -35,13 +39,13 @@ def test_stats_rate_limit_hits_429(tmp_path: Path):
             assert r2.status_code == 200
             assert r3.status_code == 429
     finally:
-        app_module.RL_STATS_PER_WINDOW = old_limit
-        app_module.RATE_WINDOW_SECONDS = old_window
+        security.RL_STATS_PER_WINDOW = old_limit
+        app.RATE_WINDOW_SECONDS = old_window
 
 
 def test_large_login_payload_rejected_413(tmp_path: Path):
     with make_client(tmp_path) as client:
-        huge_pin = "1" * (app_module.BODY_LIMIT_DEFAULT_BYTES + 100)
+        huge_pin = "1" * (config.BODY_LIMIT_DEFAULT_BYTES + 100)
         resp = client.post(
             "/api/auth/login",
             data=json.dumps({"pin": huge_pin}),
@@ -52,12 +56,12 @@ def test_large_login_payload_rejected_413(tmp_path: Path):
 
 def test_chat_message_length_rejected_413(tmp_path: Path):
     with make_client(tmp_path) as client:
-        sid = client.post("/api/auth/guest").json()["session_id"]
+        sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()["session_id"]
         headers = {"X-Session-ID": sid, "Origin": "http://testserver"}
-        message = "x" * (app_module.MAX_CHAT_MESSAGE_CHARS + 1)
+        message = "x" * (config.MAX_CHAT_MESSAGE_CHARS + 1)
         resp = client.post(
             "/api/chat",
-            json={"message": message, "model": app_module.DEFAULT_MODEL},
+            json={"message": message, "model": config.DEFAULT_MODEL},
             headers=headers,
         )
         assert resp.status_code == 413
@@ -65,12 +69,12 @@ def test_chat_message_length_rejected_413(tmp_path: Path):
 
 def test_search_query_length_rejected_413(tmp_path: Path):
     with make_client(tmp_path) as client:
-        sid = client.post("/api/auth/guest").json()["session_id"]
+        sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()["session_id"]
         headers = {"X-Session-ID": sid, "Origin": "http://testserver"}
-        query = "q" * (app_module.MAX_SEARCH_QUERY_CHARS + 1)
+        query = "q" * (config.MAX_SEARCH_QUERY_CHARS + 1)
         resp = client.post(
             "/api/search",
-            json={"query": query, "model": app_module.DEFAULT_MODEL},
+            json={"query": query, "model": config.DEFAULT_MODEL},
             headers=headers,
         )
         assert resp.status_code == 413

tests/test_search_route.py

@@ -0,0 +1,186 @@
+import json
+import os
+from pathlib import Path
+
+import httpx
+from fastapi.testclient import TestClient
+
+import app
+import config
+import db
+import routers.search_route
+from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS
+
+
+def make_client(tmp_path: Path) -> TestClient:
+    os.environ["JARVISCHAT_ADMIN_PIN"] = "1234"
+    db.DB_PATH = tmp_path / "jarvischat-search-route.db"
+    SESSIONS.clear()
+    PIN_ATTEMPTS.clear()
+    RATE_EVENTS.clear()
+    db.init_db()
+    return TestClient(app.app, raise_server_exceptions=False)
+
+
+def _guest_headers(client: TestClient) -> dict:
+    sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()["session_id"]
+    return {"X-Session-ID": sid, "Origin": "http://testserver"}
+
+
+def parse_sse_payloads(body: str) -> list[dict]:
+    payloads: list[dict] = []
+    for chunk in body.split("\n\n"):
+        chunk = chunk.strip()
+        if not chunk.startswith("data: "):
+            continue
+        raw = chunk[len("data: ") :]
+        payloads.append(json.loads(raw))
+    return payloads
+
+
+class _MockStreamResponse:
+    def __init__(self, lines: list[str]):
+        self._lines = lines
+
+    async def __aenter__(self):
+        return self
+
+    async def __aexit__(self, exc_type, exc, tb):
+        return False
+
+    async def aiter_lines(self):
+        for line in self._lines:
+            yield line
+
+
+def _stream_json_lines(events: list[dict]) -> list[str]:
+    return [json.dumps(event) for event in events]
+
+
+def test_explicit_search_with_results(tmp_path: Path, monkeypatch):
+    with make_client(tmp_path) as client:
+        headers = _guest_headers(client)
+
+        async def search_stub(query: str, max_results: int = 5):
+            return [
+                {"title": "Result One", "url": "https://example.com/1", "content": "First result content."},
+                {"title": "Result Two", "url": "https://example.com/2", "content": "Second result content."},
+            ]
+
+        monkeypatch.setattr(routers.search_route, "query_searxng", search_stub)
+
+        events = _stream_json_lines([
+            {"choices": [{"delta": {"content": "Here's what I found"}, "logprobs": None}]},
+            {"choices": [{"delta": {"content": " about your query."}, "logprobs": None}]},
+            {"choices": [{"delta": {}, "finish_reason": "stop"}], "usage": {}},
+        ])
+
+        def stream_stub(self, method, url, json=None, timeout=None):
+            return _MockStreamResponse(events)
+
+        monkeypatch.setattr(httpx.AsyncClient, "stream", stream_stub)
+
+        resp = client.post(
+            "/api/search",
+            json={"query": "current events", "model": config.DEFAULT_MODEL},
+            headers=headers,
+        )
+        assert resp.status_code == 200
+        payloads = parse_sse_payloads(resp.text)
+
+        assert any(p.get("searching") is True for p in payloads)
+        assert any("search_results" in p for p in payloads)
+        token_text = "".join(p.get("token", "") for p in payloads if "token" in p)
+        assert "found" in token_text.lower()
+        assert any(p.get("done") and p.get("searched") for p in payloads)
+
+
+def test_explicit_search_no_results(tmp_path: Path, monkeypatch):
+    with make_client(tmp_path) as client:
+        headers = _guest_headers(client)
+
+        async def empty_search(query: str, max_results: int = 5):
+            return []
+
+        monkeypatch.setattr(routers.search_route, "query_searxng", empty_search)
+
+        resp = client.post(
+            "/api/search",
+            json={"query": "nothingness", "model": config.DEFAULT_MODEL},
+            headers=headers,
+        )
+        assert resp.status_code == 200
+        payloads = parse_sse_payloads(resp.text)
+
+        assert any("No search results found" in p.get("token", "") for p in payloads)
+        assert any(p.get("done") for p in payloads)
+        assert not any("search_results" in p for p in payloads)
+
+
+def test_explicit_search_new_conversation_created(tmp_path: Path, monkeypatch):
+    with make_client(tmp_path) as client:
+        headers = _guest_headers(client)
+
+        async def search_stub(query: str, max_results: int = 5):
+            return [{"title": "T", "url": "https://ex.com", "content": "Content."}]
+
+        monkeypatch.setattr(routers.search_route, "query_searxng", search_stub)
+
+        events = _stream_json_lines([
+            {"choices": [{"delta": {"content": "Answer."}, "logprobs": None}]},
+            {"choices": [{"delta": {}, "finish_reason": "stop"}], "usage": {}},
+        ])
+
+        def stream_stub(self, method, url, json=None, timeout=None):
+            return _MockStreamResponse(events)
+
+        monkeypatch.setattr(httpx.AsyncClient, "stream", stream_stub)
+
+        resp = client.post(
+            "/api/search",
+            json={"query": "tell me something", "model": config.DEFAULT_MODEL},
+            headers=headers,
+        )
+        assert resp.status_code == 200
+        payloads = parse_sse_payloads(resp.text)
+
+        conv_id = None
+        for p in payloads:
+            if "conversation_id" in p:
+                conv_id = p["conversation_id"]
+                break
+        assert conv_id is not None
+
+        conv_resp = client.get(f"/api/conversations/{conv_id}", headers=_guest_headers(client))
+        assert conv_resp.status_code == 200
+        data = conv_resp.json()
+        assert len(data["messages"]) >= 2
+
+
+def test_explicit_search_stream_error(tmp_path: Path, monkeypatch):
+    with make_client(tmp_path) as client:
+        headers = _guest_headers(client)
+
+        async def search_stub(query: str, max_results: int = 5):
+            return [{"title": "T", "url": "https://ex.com", "content": "Content."}]
+
+        monkeypatch.setattr(routers.search_route, "query_searxng", search_stub)
+
+        def broken_stream(self, method, url, json=None, timeout=None):
+            class BrokenCtx:
+                async def __aenter__(self):
+                    raise RuntimeError("summarization failed")
+                async def __aexit__(self, exc_type, exc, tb):
+                    return False
+            return BrokenCtx()
+
+        monkeypatch.setattr(httpx.AsyncClient, "stream", broken_stream)
+
+        resp = client.post(
+            "/api/search",
+            json={"query": "breaking news", "model": config.DEFAULT_MODEL},
+            headers=headers,
+        )
+        assert resp.status_code == 200
+        assert "error_key" in resp.text
+        assert "INC-" in resp.text

tests/test_search_url_sanitization.py

@@ -1,17 +1,17 @@
-import app as app_module
+from search import sanitize_outbound_url
 
 
 def test_sanitize_outbound_url_allows_http_https():
-    assert app_module.sanitize_outbound_url("https://example.com/path") == "https://example.com/path"
-    assert app_module.sanitize_outbound_url("http://example.com") == "http://example.com"
+    assert sanitize_outbound_url("https://example.com/path") == "https://example.com/path"
+    assert sanitize_outbound_url("http://example.com") == "http://example.com"
 
 
 def test_sanitize_outbound_url_blocks_unsafe_schemes():
-    assert app_module.sanitize_outbound_url("javascript:alert(1)") == ""
-    assert app_module.sanitize_outbound_url("data:text/html,evil") == ""
-    assert app_module.sanitize_outbound_url("file:///etc/passwd") == ""
+    assert sanitize_outbound_url("javascript:alert(1)") == ""
+    assert sanitize_outbound_url("data:text/html,evil") == ""
+    assert sanitize_outbound_url("file:///etc/passwd") == ""
 
 
 def test_sanitize_outbound_url_blocks_relative_and_empty():
-    assert app_module.sanitize_outbound_url("/relative/path") == ""
-    assert app_module.sanitize_outbound_url("") == ""
+    assert sanitize_outbound_url("/relative/path") == ""
+    assert sanitize_outbound_url("") == ""

tests/test_settings_allowlist.py

@@ -3,17 +3,19 @@ from pathlib import Path
 
 from fastapi.testclient import TestClient
 
-import app as app_module
+import app
+import db
+from security import SESSIONS, PIN_ATTEMPTS
 
 
 def make_admin_client(tmp_path: Path) -> tuple[TestClient, dict[str, str]]:
     os.environ["JARVISCHAT_ADMIN_PIN"] = "1234"
-    app_module.DB_PATH = tmp_path / "jarvischat-settings.db"
-    app_module.SESSIONS.clear()
-    app_module.PIN_ATTEMPTS.clear()
-    app_module.init_db()
+    db.DB_PATH = tmp_path / "jarvischat-settings.db"
+    SESSIONS.clear()
+    PIN_ATTEMPTS.clear()
+    db.init_db()
 
-    client = TestClient(app_module.app)
+    client = TestClient(app.app)
     login = client.post(
         "/api/auth/login",
         json={"pin": "1234"},

tests/test_skills_framework.py

@@ -1,19 +1,23 @@
+import asyncio
 import os
 from pathlib import Path
 
 from fastapi.testclient import TestClient
 
-import app as app_module
+import app
+import db
+from rag import build_system_prompt
+from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS
 
 
 def make_client(tmp_path: Path) -> TestClient:
     os.environ["JARVISCHAT_ADMIN_PIN"] = "1234"
-    app_module.DB_PATH = tmp_path / "jarvischat-skills.db"
-    app_module.SESSIONS.clear()
-    app_module.PIN_ATTEMPTS.clear()
-    app_module.RATE_EVENTS.clear()
-    app_module.init_db()
-    return TestClient(app_module.app, raise_server_exceptions=False)
+    db.DB_PATH = tmp_path / "jarvischat-skills.db"
+    SESSIONS.clear()
+    PIN_ATTEMPTS.clear()
+    RATE_EVENTS.clear()
+    db.init_db()
+    return TestClient(app.app, raise_server_exceptions=False)
 
 
 def test_guest_can_list_skills(tmp_path: Path):
@@ -21,7 +25,7 @@ def test_guest_can_list_skills(tmp_path: Path):
         sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()[
             "session_id"
         ]
-        resp = client.get("/api/skills", headers={"X-Session-ID": sid})
+        resp = client.get("/api/skills", headers={"X-Session-ID": sid, "Origin": "http://testserver"})
         assert resp.status_code == 200
         payload = resp.json()
         assert payload["count"] >= 1
@@ -46,7 +50,7 @@ def test_admin_can_toggle_skill_enabled_state(tmp_path: Path):
         assert disable.status_code == 200
         assert disable.json()["skill"]["enabled"] is False
 
-        active = client.get("/api/skills/active", headers={"X-Session-ID": sid})
+        active = client.get("/api/skills/active", headers={"X-Session-ID": sid, "Origin": "http://testserver"})
         assert active.status_code == 200
         assert all(skill["key"] != "search.web" for skill in active.json()["skills"])
 
@@ -71,23 +75,23 @@ def test_unknown_skill_update_is_rejected(tmp_path: Path):
 
 def test_prompt_injection_respects_skills_enabled_setting(tmp_path: Path):
     with make_client(tmp_path):
-        db = app_module.get_db()
+        conn = db.get_db()
         try:
-            db.execute(
+            conn.execute(
                 "INSERT OR REPLACE INTO settings (key, value) VALUES (?, ?)",
                 ("skills_enabled", "false"),
             )
-            db.commit()
-            without_skills = app_module.build_system_prompt(db, "", "hello")
+            conn.commit()
+            without_skills = asyncio.run(build_system_prompt(conn, "", "hello"))
             assert "## Active Skills" not in without_skills
 
-            db.execute(
+            conn.execute(
                 "INSERT OR REPLACE INTO settings (key, value) VALUES (?, ?)",
                 ("skills_enabled", "true"),
             )
-            db.commit()
-            with_skills = app_module.build_system_prompt(db, "", "hello")
+            conn.commit()
+            with_skills = asyncio.run(build_system_prompt(conn, "", "hello"))
             assert "## Active Skills" in with_skills
             assert "memory.search" in with_skills
         finally:
-            db.close()
+            conn.close()

tests/test_upload.py

@@ -0,0 +1,274 @@
+import os
+from pathlib import Path
+
+import httpx
+import pytest
+from fastapi.testclient import TestClient
+
+import app
+import db
+import routers.upload as upload_route
+from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS
+
+
+def make_client(tmp_path: Path) -> TestClient:
+    os.environ["JARVISCHAT_ADMIN_PIN"] = "1234"
+    db.DB_PATH = tmp_path / "jarvischat-upload.db"
+    SESSIONS.clear()
+    PIN_ATTEMPTS.clear()
+    RATE_EVENTS.clear()
+    db.init_db()
+    return TestClient(app.app, raise_server_exceptions=False)
+
+
+def _admin_headers(client: TestClient) -> dict:
+    login = client.post("/api/auth/login", json={"pin": "1234"}, headers={"Origin": "http://testserver"})
+    sid = login.json()["session_id"]
+    return {"X-Session-ID": sid, "Origin": "http://testserver"}
+
+
+def _guest_headers(client: TestClient) -> dict:
+    sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()["session_id"]
+    return {"X-Session-ID": sid, "Origin": "http://testserver"}
+
+
+def test_upload_requires_admin(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.post("/api/upload", headers=_guest_headers(client), files={"file": ("test.txt", b"hello")})
+        assert resp.status_code == 403
+
+
+def test_upload_unsupported_mime(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.post(
+            "/api/upload", headers=_admin_headers(client),
+            files={"file": ("test.exe", b"fake", "application/x-msdownload")},
+        )
+        assert resp.status_code == 415
+
+
+def test_upload_context_mode(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.post(
+            "/api/upload", headers=_admin_headers(client),
+            data={"mode": "context", "conversation_id": "conv-1"},
+            files={"file": ("notes.txt", b"Hello world notes")},
+        )
+        assert resp.status_code == 200
+        data = resp.json()
+        assert data["filename"] == "notes.txt"
+        assert data["mode"] == "context"
+        assert "context_id" in data
+        assert "chunks_ingested" not in data
+
+        row = db.get_db().execute("SELECT content FROM upload_context WHERE id = ?", (data["context_id"],)).fetchone()
+        assert row["content"] == "Hello world notes"
+
+
+def test_upload_ingest_mode(tmp_path: Path, monkeypatch):
+    embed_count = 0
+
+    class FakeAsyncClient:
+        class FakeResponse:
+            def __init__(self, status, json_data=None):
+                self.status_code = status
+                self._json = json_data or {}
+
+            def json(self):
+                return self._json
+
+        def __init__(self, *a, **kw):
+            pass
+
+        async def __aenter__(self):
+            return self
+
+        async def __aexit__(self, *a):
+            pass
+
+        async def post(self, url, **kw):
+            nonlocal embed_count
+            if "/api/embeddings" in url:
+                embed_count += 1
+                return self.FakeResponse(200, {"embedding": [0.1] * 768})
+            return self.FakeResponse(200)
+
+        async def put(self, url, **kw):
+            return self.FakeResponse(200)
+
+    monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: FakeAsyncClient())
+
+    with make_client(tmp_path) as client:
+        resp = client.post(
+            "/api/upload", headers=_admin_headers(client),
+            data={"mode": "ingest"},
+            files={"file": ("data.txt", b"word " * 1000)},
+        )
+        assert resp.status_code == 200
+        data = resp.json()
+        assert data["mode"] == "ingest"
+        assert data["chunks_ingested"] > 0
+        assert "context_id" not in data
+        assert embed_count == data["chunks_ingested"]
+
+
+def test_upload_both_mode(tmp_path: Path, monkeypatch):
+    class FakeAsyncClient:
+        class FakeResponse:
+            def __init__(self, status, json_data=None):
+                self.status_code = status
+                self._json = json_data or {}
+
+            def json(self):
+                return self._json
+
+        def __init__(self, *a, **kw):
+            pass
+
+        async def __aenter__(self):
+            return self
+
+        async def __aexit__(self, *a):
+            pass
+
+        async def post(self, url, **kw):
+            if "/api/embeddings" in url:
+                return self.FakeResponse(200, {"embedding": [0.1] * 768})
+            return self.FakeResponse(200)
+
+        async def put(self, url, **kw):
+            return self.FakeResponse(200)
+
+    monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: FakeAsyncClient())
+
+    with make_client(tmp_path) as client:
+        resp = client.post(
+            "/api/upload", headers=_admin_headers(client),
+            data={"mode": "both", "conversation_id": "conv-2"},
+            files={"file": ("both.txt", b"test " * 500)},
+        )
+        assert resp.status_code == 200
+        data = resp.json()
+        assert data["mode"] == "both"
+        assert "context_id" in data
+        assert data["chunks_ingested"] > 0
+
+
+def test_upload_image_type(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.post(
+            "/api/upload", headers=_admin_headers(client),
+            data={"mode": "context"},
+            files={"file": ("photo.png", b"fake-png", "image/png")},
+        )
+        assert resp.status_code == 200
+        data = resp.json()
+        assert data["filename"] == "photo.png"
+        assert "context_id" in data
+
+
+def test_get_upload_by_conversation(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        headers = _admin_headers(client)
+        resp1 = client.post("/api/upload", headers=headers,
+                            data={"mode": "context", "conversation_id": "conv-gal"},
+                            files={"file": ("a.txt", b"alpha")})
+        cid1 = resp1.json()["context_id"]
+        resp2 = client.post("/api/upload", headers=headers,
+                            data={"mode": "context", "conversation_id": "conv-gal"},
+                            files={"file": ("b.txt", b"beta")})
+        cid2 = resp2.json()["context_id"]
+
+        gal = client.get("/api/upload/by-conversation/conv-gal", headers=headers)
+        assert gal.status_code == 200
+        items = gal.json()
+        assert len(items) == 2
+        assert items[0]["filename"] == "a.txt"
+        assert items[1]["filename"] == "b.txt"
+
+
+def test_link_upload_to_conversation(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        headers = _admin_headers(client)
+        resp = client.post("/api/upload", headers=headers,
+                           data={"mode": "context"},
+                           files={"file": ("orphan.txt", b"lonely")})
+        cid = resp.json()["context_id"]
+
+        link = client.patch(f"/api/upload/{cid}/link", headers=headers,
+                            json={"conversation_id": "new-conv"})
+        assert link.status_code == 200
+
+        gal = client.get("/api/upload/by-conversation/new-conv", headers=headers)
+        assert len(gal.json()) == 1
+
+
+def test_delete_upload_removes_context(tmp_path: Path, monkeypatch):
+    class FakeAsyncClient:
+        class FakeResponse:
+            def __init__(self, status, json_data=None):
+                self.status_code = status
+                self._json = json_data or {}
+
+            def json(self):
+                return self._json
+
+        def __init__(self, *a, **kw):
+            pass
+
+        async def __aenter__(self):
+            return self
+
+        async def __aexit__(self, *a):
+            pass
+
+        async def post(self, url, **kw):
+            if "/points/scroll" in url:
+                return self.FakeResponse(200, {"result": {"points": [{"id": "upload-test.txt-0"}, {"id": "upload-test.txt-1"}]}})
+            if "/points/delete" in url:
+                return self.FakeResponse(200)
+            return self.FakeResponse(200)
+
+        async def put(self, url, **kw):
+            return self.FakeResponse(200)
+
+    monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: FakeAsyncClient())
+
+    with make_client(tmp_path) as client:
+        headers = _admin_headers(client)
+        resp = client.post("/api/upload", headers=headers,
+                           data={"mode": "context", "conversation_id": "del-test"},
+                           files={"file": ("test.txt", b"delete me")})
+        cid = resp.json()["context_id"]
+
+        del_resp = client.delete(f"/api/upload/{cid}", headers=headers)
+        assert del_resp.status_code == 200
+        assert del_resp.json()["status"] == "ok"
+
+        row = db.get_db().execute("SELECT id FROM upload_context WHERE id = ?", (cid,)).fetchone()
+        assert row is None
+
+
+def test_delete_upload_not_found(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        resp = client.delete("/api/upload/999", headers=_admin_headers(client))
+        assert resp.status_code == 404
+
+
+def test_conversation_list_includes_attachment_count(tmp_path: Path):
+    with make_client(tmp_path) as client:
+        headers = _admin_headers(client)
+        client.post("/api/conversations", headers=headers, json={"title": "NoAttach"})
+        with_attach = client.post("/api/conversations", headers=headers, json={"title": "WithAttach"}).json()
+        conv_id = with_attach["id"]
+        client.post("/api/upload", headers=headers,
+                    data={"mode": "context", "conversation_id": conv_id},
+                    files={"file": ("f.txt", b"data")})
+
+        list_resp = client.get("/api/conversations", headers=headers)
+        convs = list_resp.json()
+        for c in convs:
+            if c["title"] == "NoAttach":
+                assert c["attachment_count"] == 0
+            if c["title"] == "WithAttach":
+                assert c["attachment_count"] == 1