gramps/cAIc
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Compare commits

base: gramps/cAIc:main
Branches Tags
gramps/cAIc:main
gramps/cAIc:v1.9.0 gramps/cAIc:v1.8.0
..
compare: gramps/cAIc:v1.8.0
Branches Tags
gramps/cAIc:main
gramps/cAIc:v1.9.0 gramps/cAIc:v1.8.0

2 Commits
main .. v1.8.0

Author SHA1 Message Date
gramps 31ba4769c8 feat: RAG pipeline + OpenAI SSE streaming, llama-server cluster integration 2026-06-14 13:57:09 -07:00
gramps 347650b507 feat: switch from Ollama to llama-server OpenAI-compat API, fix streaming parser 2026-06-14 12:37:29 -07:00
79 changed files with 3231 additions and 15582 deletions
Expand all files Collapse all files
.gitignore
-5
View File
@@ -4,8 +4,3 @@
__pycache__/ __pycache__/
venv/ venv/
readme.md- readme.md-
*.bak
hardware_state.json
.env
secrets/
searxng/
=9.0.0
-1
View File
@@ -1 +0,0 @@
/bin/bash: line 1: ./venv/bin/pip: No such file or directory
AGENTS.md
-168
View File
@@ -1,168 +0,0 @@
# cAIc — Agents Guide
## Run
```bash
uvicorn app:app --host 0.0.0.0 --port 8080 --reload
```
## Tests
```bash
python3 -m pytest tests/ -v
```
All tests use `tmp_path` fixtures + monkeypatched `httpx.AsyncClient.stream/get/post/put`. No external services needed. Test factories reset `SESSIONS`, `PIN_ATTEMPTS`, `RATE_EVENTS` globals — be careful not to let test state leak. Tests import directly from the correct modules (`db`, `security`, `config`, `search`, `rag`, `memory`, `routers.*`).
Every router has a dedicated test file:
| File | Covers |
|------|--------|
| `test_auth_capabilities.py` | `auth.py` — guest/admin sessions, origin blocking, logout |
| `test_chat_streaming_and_memory_paths.py` | `routers/chat.py` — streaming, auto-search, remember/forget, upload context injection |
| `test_completions.py` | `routers/completions.py` — API key auth, FIM, streaming, blocking, errors |
| `test_conversations.py` | `routers/conversations.py` — full CRUD, guest admin enforcement, attachment_count |
| `test_ingest.py` | `routers/ingest.py` — Bearer auth, chunk/embed/upsert, validation |
| `test_memories.py` | `routers/memories.py` — edit, search, stats endpoints |
| `test_models_router.py` | `routers/models.py` — models list, ps, show, stats, search/status |
| `test_presets.py` | `routers/presets.py` — full CRUD, default preset protection |
| `test_profile.py` | `routers/profile.py` — get, update, default, length validation |
| `test_search_route.py` | `routers/search_route.py` — explicit search flow, no results, errors |
| `test_search_url_sanitization.py` | `search.py` URL sanitizer |
| `test_cluster.py` | `cluster.py` — registration, deregistration, pong, events, coordinator query |
| `test_cluster_heartbeat.py` | `cluster.py` — heartbeat handler, known/unknown node |
| `test_model_swap.py` | `cluster.py` + `triage.py` — request_model_swap, handle_model_ready/failed, select_node swap triggering |
| `test_node_agent.py` | `node_agent/agent.py` — registration, ping/pong, model swap |
| `test_triage.py` | `triage.py` — classify_query, select_node, get_inference_url |
| `test_settings_allowlist.py` | `routers/settings.py` — allowlisted key enforcement |
| `test_skills_framework.py` | `routers/skills.py` — list, toggle, unknown skill, prompt injection |
| `test_ip_allowlist.py` | IP allowlist helper + middleware |
| `test_rate_and_payload_guardrails.py` | Rate limits + payload size enforcement |
| `test_error_envelopes.py` | Global exception handler + stream error incidents |
| `test_upload.py` | `routers/upload.py` — upload, delete, link, by-conversation, attachment_count integration |
Modules that call `httpx.AsyncClient` (chat, completions, models, search_route, upload, ingest)
are mocked via `monkeypatch.setattr` on `AsyncClient.stream`, `.get`, or `.post`.
CPU stats in `models.py` (`api/stats`) use real `psutil`; GPU stats are
monkeypatched via `routers.models.get_gpu_stats`.
## Architecture
Refactored from single-file (`app.py`) into modules under project root:
| File | Role |
|------|------|
| `app.py` | FastAPI app, middleware, router registration |
| `config.py` | Constants, env vars, rate/payload limits, built-in skills registry, upload limits |
| `db.py` | SQLite schema, connection factory, settings helpers, upload_context CRUD |
| `auth.py` | PIN-based guest/admin sessions, auth routes |
| `security.py` | Rate limiting, origin checks, IP allowlist, audit/incident logging |
| `memory.py` | FTS5 memory CRUD, remember/forget command parsing |
| `search.py` | SearXNG integration, perplexity scoring, refusal detection |
| `rag.py` | Qdrant vector search + system prompt assembly + chunk_text() helper |
| `eviction.py` | Score-based RAG eviction engine |
| `gpu.py` | AMD GPU stats via `rocm-smi` |
| `triage.py` | Phi-4-mini-based query classification + cluster node selection |
| `cluster.py` | Cluster node registry, event log, coordinator election, ping/pong, model swap handlers |
| `amqp.py` | AMQP connection manager — connect, disconnect, publish, subscribe, auto-reconnect |
| `node_agent/` | Standalone worker agent — AMQP client for registration, ping/pong, model swap |
| `routers/` | One module per endpoint group (chat, search, skills, completions, upload, ingest) |
### Entrypoint / API keys
- `app.py` line 148: `uvicorn.run(app, ...)` when called directly
- `config.py` line 14: `LLAMA_SERVER_BASE` defaults to `http://192.168.50.108:8081` — llama-server on coordinator, RPC-offloads GPU layers to worker :50052
- `config.py` line 17: `COMPLETIONS_API_KEY` read from `CAIC_COMPLETIONS_API_KEY` env var or auto-generates
- `config.py` line 13: `OLLAMA_BASE` is legacy/unused — all endpoints use `LLAMA_SERVER_BASE`
### Key flows
1. **`/api/chat`** → `process_remember_command()` intercepts "remember that..." / "forget about..." first → optional `upload_context_id` fetches document text from SQLite → `build_system_prompt()` (profile + FTS5 memory + Qdrant RAG + preset + skills + uploaded doc) → triage classifies query (general/code/search/rag) → `select_node()` picks best worker → stream from chosen node with `logprobs: true` → if perplexity > 15.0 OR `REFUSAL_PATTERNS` match, re-query with SearXNG results
2. **`/api/search`** → bypasses perplexity/refusal, queries SearXNG directly → summarizes via llama-server
3. **`/v1/chat/completions`** → OpenAI-compatible for Continue.dev/IDE integration; FIM requests proxied without persistence
4. **`/api/upload`** → multipart file upload, PDF/text extraction, `mode=(context|ingest|both)`, stores SQLite context (1hr expiry) + Qdrant upsert
5. **`/api/ingest`** → Bearer token auth, programmatic RAG ingest (terminal hook, external tools)
### Perplexity / auto-search
The upstream request includes `"logprobs": true`. `parse_llama_stream_chunk()` extracts per-token logprobs from each chunk's `choices[0].logprobs.content[].logprob`. The `all_logprobs` list is populated during streaming, so `calculate_perplexity()` and `is_uncertain()` work correctly.
### Auth / lockdown
- Guest session by default (`POST /api/auth/guest`), admin unlock via 4-digit PIN (`POST /api/auth/login`)
- Admin required for PUT/DELETE/PATCH + all POST except allowlist (`/api/chat`, `/api/search`, `/api/auth/*`)
- `/api/ingest` is exempt from session auth — self-authenticates via Bearer token
- IP allowlist, rate limiting, origin checking, payload size limits — all enforced in `app.py` middleware
- Origin check applies to **all** `/api/` requests; returns `False` when both `Origin` and `Referer` are absent
- `CAIC_ADMIN_PIN` env var required on first boot (or `CAIC_ALLOW_DEFAULT_PIN=true`)
### Database
- SQLite at `caic.db`, auto-created by `init_db()` on startup via FastAPI `lifespan`
- `get_db()` opens new connection per request (no pool). Close after use.
- FTS5 virtual table `memories` for full-text search with BM25 ranking.
- `upload_context` table: auto-expiring document storage for chat context injection.
### External services
| Service | Required | Port |
|---------|----------|------|
| llama-server (coordinator) | Yes | 8081 + RPC :50052 (worker GPU) |
| Phi-4-mini (triage) | No | 8083 |
| SearXNG | No | 8888 |
| RabbitMQ (coordinator) | No | 5672 — AMQP broker |
| wttr.in | No | weather shortcut |
| rocm-smi | No | AMD GPU stats |
| Qdrant | No | 6333 (coordinator) — RAG vector search |
| Ollama (worker) | No | 11434 — embeddings only |
### Config quirks
- `BODY_LIMIT_UPLOAD_BYTES` = 20MB for `/api/upload`; other paths use smaller limits
- `SUPPORTED_UPLOAD_TYPES` includes images (png/jpeg/gif/svg/webp) + text + PDF + JSON
- `UPLOAD_CONTEXT_EXPIRY_HOURS` = 1 hour
- Rate limits and payload caps in `config.py` — patch `security.RL_*` not `config.RL_*` for tests
- RAG embedding requests go to `EMBED_URL` at `/api/embeddings` (Ollama on worker :11434)
### SSE Protocol
All streaming endpoints yield `data: {json}\n\n`. Key shapes:
- `{token, conversation_id}` — streaming token
- `{searching: true}` — web search triggered
- `{search_results: N}` — N results (no raw_results payload)
- `{done: true, perplexity, tokens_per_sec, searched?}` — terminal
- `{error: "...", error_key: "..."}` — error with incident key
## Work State
### Completed this session
- **Scroll-position fighting** — `scrollToTop()` now respects `_userScrolledAway` flag (100px threshold), skips auto-scroll when user is reading older content. `resetScrollLock()` called on new messages.
- **401 error cascade** — `SESSION_TIMEOUT_SECONDS` bumped 90→3600 (1 hour). All 10 unprotected `authFetch` calls wrapped in try/catch.
- **Token counter** — removed localStorage persistence; resets to 0 on page refresh.
- **Ctrl+Enter for web search** — Shift+Enter now inserts newline (universal convention), Ctrl+Enter triggers search.
- **Search button styling** — WEB button matches SEND: same font/size/weight, orange bg with dark navy text (`var(--bg-tertiary)`). Input placeholder updated.
- **Toast notifications** — `showToast()` helper, every action icon now fires a slide-out notification (copy, save, delete, rate, etc.). Print is the exception (dialog handles it).
- **Clipboard reliability** — `execCopy()` helper for HTTP fallback (navigator.clipboard fails on plain HTTP). All three copy paths (user inline, toolbar, code blocks) now work on jarvis:8080.
- **User copy icon** — single 📋 inline at end of user query text (not a full toolbar). Uses `data-content` attr to avoid HTML injection in onclick.
- **Rating thumbs removed** — 👍👎 cut (no backend, no persistence, privacy concern).
- **Keybinding fix** — Shift+Enter = newline, Ctrl+Enter = search (universal conventions).
### Active
- (none)
### Blocked
- (none)
### Upcoming (backlog)
- B4 — RAG Corpus Management UI
- B5 — default model auto-pull on first start
- B6 — waterfall direction toggle
- B7 — Apple Silicon worker support (gpu.py Metal, hardware.py Darwin)
- B8 — **Encryption & PHI readiness** — spec out encryption at rest (SQLCipher for caic.db, Qdrant payload encryption) and in-transit (TLS for inference, AMQP, RAG). Per-user auth, audit logging, log sanitizer, data lifecycle. Document the "personal LAN HIPAA gap."
- **Preferred approach: Private Chat mode** — a toggle that skips DB persistence, memory/RAG injection, and content logging entirely. Zero stored data = zero data to protect. Simpler, more robust, less code to audit than full encryption. Design this as the primary PHI path before reaching for crypto.
- **In-transit still needs TLS** — Private Chat eliminates at-rest risk, but AMQP (RabbitMQ) and inference (llama-server) traffic between coordinator and workers is still plaintext on the wire. TLS termination on each node is the lightweight fix (self-signed CA, nginx sidecar or RabbitMQ TLS).
### Key config values (current)
- `VERSION = "v0.18.0"` in `config.py`
- `SESSION_TIMEOUT_SECONDS = 3600`
- `DEFAULT_MODEL = "qwen2.5-7b-instruct"`
- `LLAMA_SERVER_BASE = "http://192.168.50.108:8081"`
CLAUDE.md
+20 -40
View File
@@ -9,7 +9,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
./venv/bin/uvicorn app:app --host 0.0.0.0 --port 8080 --reload ./venv/bin/uvicorn app:app --host 0.0.0.0 --port 8080 --reload
# Production (via systemd) # Production (via systemd)
sudo systemctl restart caic sudo systemctl restart jarvischat
# Direct run # Direct run
./venv/bin/python app.py ./venv/bin/python app.py
@@ -19,66 +19,46 @@ sudo systemctl restart caic
```bash ```bash
./venv/bin/pip install -r requirements.txt ./venv/bin/pip install -r requirements.txt
# Also requires: psutil jinja2 python-multipart pypdf (not in requirements.txt) # Also requires: psutil jinja2 python-multipart (not in requirements.txt)
``` ```
## Architecture ## Architecture
Modular FastAPI app — `app.py` wires routers, middleware, and lifespan. SQLite database auto-created at `caic.db` on first run. No build step, single `templates/index.html`. Single-file FastAPI backend (`app.py`) + single-template frontend (`templates/index.html`). No build step. SQLite database auto-created at `jarvischat.db` on first run.
### Request Flow: `/api/chat` ### Request Flow: `/api/chat`
1. User message saved to DB → conversation created if new 1. User message saved to DB → conversation created if new
2. `process_remember_command()` intercepts "remember that..." / "forget about..." first 2. `build_system_prompt()` assembles: profile + FTS5 memory search results + preset prompt
3. Optional `upload_context_id` → fetches document text from `upload_context` table, injects `[ATTACHED DOCUMENT]` into system prompt 3. Streamed to Ollama (`/api/chat`, `stream: true`, `logprobs: true`) via SSE
4. `build_system_prompt()` assembles: profile + FTS5 memory search + Qdrant RAG + preset + skills + uploaded doc 4. **Auto web search trigger**: if perplexity > 15.0 OR response matches `REFUSAL_PATTERNS`, re-queries Ollama with SearXNG results prepended to system prompt
5. Streamed to llama-server (`/v1/chat/completions`, `stream: true`, `logprobs: true`) via SSE 5. Final response saved to DB; SSE `done` event sent with perplexity + tokens/sec
6. **Auto web search trigger**: if perplexity > 15.0 OR response matches `REFUSAL_PATTERNS`, re-queries with SearXNG results
7. Final response saved to DB; SSE `done` event sent with perplexity + tokens/sec
### Request Flow: `/api/search` (explicit search) ### Request Flow: `/api/search` (explicit search)
Bypasses perplexity/refusal — queries SearXNG directly then asks llama-server to summarize results. Bypasses perplexity/refusal detection entirely — queries SearXNG directly then asks Ollama to summarize with results as system context.
### Request Flow: `/api/upload`
Multipart file upload → PDF/text extraction + chunking → optional Qdrant upsert + SQLite context storage (1hr expiry). Supports `mode=(context|ingest|both)`. Images upload as storage only — model cannot process image content.
### Request Flow: `/api/ingest`
Bearer-token-authenticated terminal RAG hook. Accepts raw text, chunks via `chunk_text()`, embeds via Ollama `/api/embeddings`, upserts to Qdrant.
### Memory System ### Memory System
FTS5 virtual table (`memories`) in SQLite. `search_memories()` uses BM25 ranking. `process_remember_command()` intercepts "remember that..." / "forget about..." before the message reaches the model and returns a confirmation string. FTS5 virtual table (`memories`) in SQLite. `search_memories()` uses BM25 ranking. `process_remember_command()` intercepts "remember that..." / "forget about..." before the message reaches Ollama and returns a confirmation string. Topic auto-detection via keyword matching in `detect_topic()`.
### Key Constants (`config.py`) ### Key Constants (top of `app.py`)
- `LLAMA_SERVER_BASE` — `http://192.168.50.108:8081` (coordinator llama-server, RPC offloads to worker GPU) - `OLLAMA_BASE` — `http://localhost:11434`
- `SEARXNG_BASE` — `http://localhost:8888` - `SEARXNG_BASE` — `http://localhost:8888`
- `QDRANT_URL` — `http://192.168.50.108:6333` (Qdrant on coordinator) - `PERPLEXITY_THRESHOLD` — `15.0` (controls auto-search sensitivity)
- `TRIAGE_BASE` — `http://127.0.0.1:8083/v1` (Phi-4-mini) - `DEFAULT_MODEL` — `llama3.1:latest`
- `AMQP_URL` — `amqp://caic:{pw}@localhost:5672/caic` (RabbitMQ, pw read from `~/.caic_amqp_secret`)
- `PERPLEXITY_THRESHOLD` — `15.0`
- `EMBED_URL` — `http://192.168.50.210:11434/api/embeddings` (Ollama on worker)
- `VERSION` — current version string
### External Services ### External Services
| Service | Required | Port | - **Ollama** — required, must be running on port 11434
|---------|----------|------| - **SearXNG** — optional, port 8888; `GET /api/search/status` probes availability
| **llama-server** (coordinator) | Yes | 8081 + RPC :50052 (worker GPU) | - **wttr.in** — weather shortcut in `query_searxng()`, bypasses SearXNG for weather queries
| **Phi-4-mini** (triage) | No | 8083 | - **rocm-smi** — AMD GPU stats via subprocess; gracefully degrades if not available
| **SearXNG** | No | 8888 |
| **RabbitMQ** (coordinator) | No | 5672 — AMQP broker |
| **wttr.in** | No | weather shortcut |
| **rocm-smi** | No | AMD GPU stats |
| **Qdrant** (coordinator) | No | 6333 — RAG vector search |
| **Ollama** (worker) | No | 11434 — embeddings only |
### Database ### Database
`get_db()` opens a new connection per request (no pool). `init_db()` runs at startup via FastAPI `lifespan`. Tables: `conversations`, `messages`, `settings`, `profile` (singleton id=1), `memories` (FTS5), `upload_context`. Default settings seeded but never overwritten. `get_db()` opens a new connection per request (no connection pool). `init_db()` runs at startup via the FastAPI `lifespan` handler. The `profile` table uses a singleton row (`id = 1`). Default settings are seeded but never overwritten by `init_db()`.
### SSE Protocol ### SSE Protocol
@@ -87,8 +67,8 @@ All streaming endpoints yield `data: {json}\n\n`. Key event shapes:
- `{searching: true}` — web search triggered - `{searching: true}` — web search triggered
- `{search_results: N}` — N results retrieved - `{search_results: N}` — N results retrieved
- `{done: true, perplexity, tokens_per_sec, searched?}` — terminal event - `{done: true, perplexity, tokens_per_sec, searched?}` — terminal event
- `{error: "...", error_key: "..."}` — error with incident key - `{error: "..."}` — error event
### Deployment ### Deployment
Runs as systemd service under user `caic`, working directory `/opt/caic`. Logs via syslog (`journalctl -u caic`). Version bumps via git tag + commit, deployed via `git pull && systemctl restart caic`. Runs as systemd service under user `jarvischat`, working directory `/opt/jarvischat`. Logs via syslog (`journalctl -u jarvischat`).
README.md
-394
View File
@@ -1,394 +0,0 @@
![cAIc banner](static/readme-banner.png)
# cAIc v0.18.0
Consumer AI hardware is a wasteland of incompatibility. NVIDIA speaks CUDA, AMD speaks ROCm. Your RTX 5070 Ti lives in one machine with 16 GB VRAM; your RX 6600 XT lives in another with 12 GB. Alone, neither can run a 14B model at usable speed. Together, they could — if the software stack didn't treat heterogeneous hardware as a bug instead of a feature.
The industry consensus — llama.cpp RPC, vLLM, TensorFlow distributed — all assume a homogeneous cluster: same GPU vendor, same VRAM, same driver stack, reachable over a fast fabric. This assumption works for data centers that buy 64 identical H100s at a time. It does not work for the person who has a gaming PC with an NVIDIA card in the living room, an AMD-powered home server in the closet, and an old MacBook on the desk. That person has more aggregate compute than any single consumer machine, but no software stack can make it cooperate.
cAIc is a cluster orchestration layer that fuses mismatched GPUs, CPUs, and machines into a single inference surface. The advantage isn't just compatibility — it's matching each request to the hardware best suited for it. The coordinator (CPU-only, no discrete GPU) handles all CPU-bound work: RAG embedding, query triage, web search, memory, conversation storage, the message broker, and the web UI itself. Workers (discrete GPU) do nothing but inference — no database, no browser sessions, no orchestration overhead stealing VRAM. Triage classifies each query and routes to the node running the optimal model; if the right model isn't loaded, the coordinator requests a swap and the worker handles it asynchronously. Every machine contributes what it does best.
You might also be doing this with retired office PCs and GPUs from the Obama era. That works too. But the core problem cAIc solves isn't budget reuse — it's making non-homogeneous hardware cooperate.
### Paired Programming
Every line of code in this repository was written by an AI (Claude, via opencode). But AI does not architect, design, test, deploy, or decide what to build — that requires experience, judgement, and the discipline to say "no" to feature creep.
**Gramps** (BS Computer Science, Oklahoma State, coding since 1981) performed that role — designing the architecture, managing the development process, writing and maintaining the test suite, operating the deployment pipeline, and directing every feature decision across dozens of sessions spanning months. Without that human steering, this would be yet another AI-generated repo that compiles but doesn't solve a real problem. With it, cAIc ships as a functional, tested, deployed system that runs 24/7 on real hardware serving real users.
This is paired programming, elevated: the AI handles the mechanical work of code generation; the human brings decades of systems-level experience, architectural judgment, and the maturity to ship something that lasts.
### Architecture: CPU Coordinator + GPU Workers
cAIc splits the workload across two machine roles:
**Coordinator** (ultron — Ryzen 7 7840HS, no discrete GPU) runs the FastAPI app, RAG vector search (Qdrant), text embedding (Ollama on CPU), query triage (Phi-4-mini), web search (SearXNG), message broker (RabbitMQ), and all SQLite-backed services — memory, profiles, conversations, settings. Every CPU-bound task stays here.
**Workers** (jarvis — RX 6600 XT 12 GB / corsair — RTX 5070 Ti 16 GB) run only llama-server for GPU inference. The coordinator never touches a model; workers never touch the database. Workers register via AMQP, receive ping/pong health checks, and accept model-swap commands when triage determines a different model is needed for the current query.
This split keeps the UI responsive during inference (the coordinator isn't blocked by GPU compute) and lets workers focus VRAM entirely on model weights rather than browser sessions or API orchestration.
Under the hood: FastAPI + SQLite + Jinja2 on Python 3.13. AMQP-mediated cluster coordination with an OpenAI-compatible inference endpoint.
#### Query-routing vs. layer-splitting — why it matters
Most distributed inference tools (llama.cpp RPC, vLLM with tensor parallelism, exo) split a *single model* across multiple GPUs. The first GPU runs layers 0–15, the second runs 16–31, and so on. This works well in a homogeneous cluster where every GPU is identical, but in a heterogeneous setup the slowest card sets the pace — every forward pass waits for the straggler. Communication overhead between GPUs (NCCL, RPC) adds latency too.
cAIc takes a different approach: **query-routing**. Each worker runs a complete model on its own GPU. When a query comes in, triage classifies it and routes the *whole request* to the worker best suited for it — code questions go to the worker with a coder model, general chat goes to the instruct model. No layer sharing, no lockstep, no straggler problem. The tradeoff is that no single query can use combined VRAM across multiple GPUs, but the throughput and responsiveness of the cluster as a whole isn't dragged down by the weakest link.
This also means a worker with a slow GPU can still contribute meaningfully — it handles less latency-sensitive queries or batch background work, while the fast GPU handles interactive chat.
At v1.0, this ships with a Docker compose stack and setup wizard that detect CPU vs GPU, probe your hardware, and stand up SearXNG, Qdrant, RabbitMQ, and everything else with a single `docker compose up`. The same install docs work bare-metal for those who prefer to skip containers entirely.
Developer wiki: [Home](https://llgit.llamachile.tube/gramps/cAIc/wiki/Home) — includes [FAQ](https://llgit.llamachile.tube/gramps/cAIc/wiki/FAQ), [Installation Guide](https://llgit.llamachile.tube/gramps/cAIc/wiki/Installation), and [full architecture docs](https://llgit.llamachile.tube/gramps/cAIc/wiki/Developer-Architecture)
## What's New in v0.18.0
### Wiki — Installation Guide, Screenshots Gallery, Full Documentation
- **New Installation & Configuration page** — bare-metal walkthrough, cluster setup, config reference, security checklist, 12 troubleshooting topics. Everything a new user needs to get cAIc running.
- **Screenshots gallery** — clickable image gallery on the wiki Screenshots page
- **Wiki fully populated** — 5 pages linked from Home, renders at root URL
- **B5 added to backlog** — auto-download of default GGUF model on first start
### UX Polish — Waterfall Layout, Barcode Stripes, Confidence Badges
- **Waterfall display** — newest messages at top via `prepend()`, scroll to top
- **Barcode alternating pairs** — each Q&A wrapped in `.msg-pair` with alternating tint + left border accent
- **Confidence % badge** (`1/ppl * 100`) replaces raw perplexity, color-coded green/orange/red
- **Cumulative token counter (TOK)** in topbar center, persisted in `localStorage`
- **TOK reformatted** to `# / %` — `#` is all-time tokens, `%` is last response's context-window percentage, color-coded
- **Dot-matrix sprocket strips** on left/right edges of `.main` (24px strips, punch-hole pattern)
- **Paper grain background** on chat container
- **Timestamps on user messages** (`HH:MM`), later upgraded to `MON dd, YYYY HH:MM:SS.ss` centisecond precision
- **Shift+Enter** triggers web search
- **Typing indicator greys out** on abort
- **Token count badge** on search responses using client-side `tokenCount`
- **Removed status dots** from input area (no functional purpose)
- **Removed thumbs** from toolbar, restored only on non-search AI responses
### Version bumped to v0.18.0
## What's New in v0.17.26
### Dynamic Model Swap — `request_model_swap()`, `select_node()` async (Roadmap N Task 14)
- **`cluster.py`** — `request_model_swap()` publishes `cmd.swap_model` to `jc.admin`; `handle_model_ready()` and `handle_model_failed()` consume `model_ready`/`model_failed` on `jc.system`
- **`select_node()` async** — Queries worker `inventory` for ideal model; triggers swap if model not active, returns `None` for fallback during swap
- **`SUBSCRIBE_TABLE`** — 7 AMQP routing key bindings in cluster.py
### Cluster Status UI — Heartbeat + Live Status Panel (Roadmap N Task 15)
- **`handle_heartbeat()`** — Consumes `node.*.heartbeat` on `jc.system` to update `last_seen` per node
- **UI cluster panel** — sidebar polls `GET /api/cluster` every 15s; green=active, yellow=swapping, red=error/offline
- **Version bumped to v0.17.0** — All 179 tests pass
### What's New in v0.14.0
### Cluster Protocol — `GET /api/cluster`, 9 AMQP Message Types (Roadmap N Task 11)
- **`cluster.py`** — Node registry (`CLUSTER_NODES`), bounded event log (`CLUSTER_EVENTS`, max 1000), coordinator auto-promotion
- **Ping/pong health** — No passive heartbeats; coordinator pings workers on-demand before routing work. 5s timeout → auto-deregister
- **9 message types** — register, deregister, admitted, rejected, ping, pong (on `jc.admin`); event, coord_query, coord_response (on `jc.system`)
- **`amqp.py` subscribe()** — Exclusive anonymous queues bound to routing keys; `_rebind_subscriptions()` recreates them on reconnect
- **`routers/cluster.py`** — `GET /api/cluster` returns nodes, coordinator, event log
### RAG Corpus Management — `POST /api/rag/flush`, `GET /api/rag/stats` (v0.13.0)
- **Score-based eviction** with hysteresis (80% high-water, 20% low-water) and pinned sources
- **Eviction engine** in `eviction.py` — scroll Qdrant, score by retrieval count + age, evict lowest scores first
- **Grace period** — vectors younger than 1 hour are never evicted
- **Flush endpoint** — `POST /api/rag/flush` (admin) deletes all non-pinned vectors
- **Stats endpoint** — `GET /api/rag/stats` (admin) returns vector count, at-risk count, pinned count, eviction rates
### File & Document Attachments (v1.9.0–v1.10.0)
- **`POST /api/upload`** — multipart file upload with PDF/text extraction; modes: `context` (chat injection), `ingest` (RAG corpus), `both`
- **`DELETE /api/upload/{id}`** — removes upload from SQLite + Qdrant
- **`PATCH /api/upload/{id}/link`** — associates upload with a conversation
- **`GET /api/upload/by-conversation/{id}`** — list attachments per conversation
- **Paperclip UI** — file picker, preview pill, image thumbnails, gallery overlay
- **Attachment indicators** — 📎 badge on conversations with attachments
- **Chat context injection** — `upload_context_id` prepends document text to system prompt
### Terminal RAG Hook — `POST /api/ingest` (v0.11.0)
- Bearer token auth (same key as `/v1/chat/completions`)
- Chunking via shared `chunk_text()` helper, embed via Ollama, upsert to Qdrant
- `caic-ingest.sh` — PROMPT_COMMAND shell script for autonomous terminal history ingestion
### v1.8.0 Foundation (refactor & fixes)
- **Modular refactor** — single-file `app.py` split into `config.py`, `db.py`, `auth.py`, `security.py`, `memory.py`, `search.py`, `rag.py`, `gpu.py`, and `routers/` package
- **Perplexity auto-search fixed** — `logprobs: true` now properly extracted from stream chunks
- **All `/api/models` endpoints** target `LLAMA_SERVER_BASE` (llama-server) not Ollama
- **RAG embedding** via Ollama at `http://192.168.50.210:11434`
- **Origin check** applies to all API methods, rejects absent Origin/Referer
## Features
- **Persistent Memory** — SQLite FTS5 full-text search for fast, relevant memory retrieval
- **Web Search** — SearXNG integration for automatic web lookups when the model is uncertain
- **Explicit Search** — Search button to force web search without waiting for model uncertainty
- **Profile Injection** — Custom system prompt injected into every conversation
- **System Presets** — Save and switch between different system prompts
- **Real-time Stats** — CPU, RAM, GPU, VRAM monitoring in sidebar
- **Token Thermometer** — Visual context window usage indicator
- **Streaming Responses** — Server-sent events for real-time token display
- **Conversation History** — SQLite-backed chat persistence with mass-delete option
- **Model Switching** — Change inference models on the fly
- **Skills Framework** — Built-in skill registry with per-skill enable/disable controls
## File Structure
```
/opt/caic/
├── amqp.py # aio-pika AMQP connection manager + subscribe/rebind
├── app.py # FastAPI app entry point
├── auth.py # PIN-based guest/admin sessions, auth routes
├── cluster.py # Cluster protocol: node registry, event log, ping/pong
├── config.py # Constants, env vars, limits, skill registry
├── db.py # SQLite schema, connection factory
├── eviction.py # Score-based RAG eviction engine
├── gpu.py # AMD GPU stats via rocm-smi
├── hardware.py # Hardware self-assessment (CPU, RAM, VRAM)
├── memory.py # FTS5 memory CRUD, remember/forget commands
├── rag.py # Qdrant vector search + system prompt assembly
├── search.py # SearXNG integration, perplexity, refusal detection
├── security.py # Rate limiting, origin checks, IP allowlist, audit
├── triage.py # Query classification + cluster node selection
├── routers/
│ ├── chat.py # /api/chat streaming endpoint
│ ├── cluster.py # Cluster status endpoint
│ ├── completions.py # /v1/chat/completions OpenAI-compat endpoint
│ ├── conversations.py# Conversation CRUD
│ ├── ingest.py # Terminal RAG ingest
│ ├── memories.py # Memory CRUD API
│ ├── models.py # Model listing, system stats
│ ├── presets.py # System prompt presets
│ ├── profile.py # User profile
│ ├── search_route.py # /api/search explicit search endpoint
│ ├── settings.py # Runtime settings
│ ├── skills.py # Skills management
│ └── upload.py # File attachment endpoints
├── static/
│ └── logo.png # Logo image (optional)
├── templates/
│ └── index.html # Frontend
├── node_agent/
│ ├── agent.py # Standalone worker agent (AMQP client)
│ └── requirements.txt
└── tests/ # 179 pytest tests
```
## Requirements
- Python 3.11+ (tested on 3.13)
- llama-server running locally or on network (OpenAI-compatible API on port 8081)
- SearXNG (optional, for web search)
- RabbitMQ (optional, for AMQP cluster — coordinator only)
- Qdrant (optional, for RAG vector search)
## Installation
### Fresh Install
```bash
# Create directory and venv
sudo mkdir -p /opt/caic
sudo chown $USER:$USER /opt/caic
cd /opt/caic
python3 -m venv venv
# Install dependencies
pip install fastapi uvicorn httpx psutil jinja2 python-multipart pypdf aio-pika
# Set admin PIN before first startup (4 digits)
export CAIC_ADMIN_PIN=4827
# Create subdirectories
mkdir -p templates static
# Copy files
# (copy all .py files to /opt/caic/)
# (copy routers/ directory to /opt/caic/)
# (copy templates/index.html to /opt/caic/templates/)
```
WARNING: Do not use `1234` as your admin PIN unless you accept weak local security.
NOTE: First boot requires `CAIC_ADMIN_PIN` unless you explicitly opt into insecure fallback with `CAIC_ALLOW_DEFAULT_PIN=true`.
## Systemd Service
Create `/etc/systemd/system/caic.service`:
```ini
[Unit]
Description=cAIc - Local Inference Web Interface
After=network.target
[Service]
Type=simple
User=caic
Group=caic
WorkingDirectory=/opt/caic
ExecStart=/opt/caic/venv/bin/uvicorn app:app --host 0.0.0.0 --port 8080
Restart=always
RestartSec=5
[Install]
WantedBy=multi-user.target
```
```bash
sudo systemctl daemon-reload
sudo systemctl enable caic
sudo systemctl start caic
```
## Memory Commands
In chat, natural language triggers memory operations:
| You say | What happens |
|---------|--------------|
| "remember that I prefer Rust over Go" | Stores as `preference` |
| "remember that cAIc runs on port 8080" | Stores as `infrastructure` |
| "note that the deadline is Friday" | Stores as `general` |
| "forget about the deadline" | Removes matching memories |
Memories are automatically searched based on your message content and injected into the system prompt when relevant.
### Memory Topics
Memories are auto-categorized:
- `preference` — likes, dislikes, choices
- `project` — active work, repos, tasks
- `infrastructure` — servers, services, configs
- `personal` — name, location, background
- `general` — everything else
## API Endpoints
### Completions (OpenAI-compatible)
| Method | Endpoint | Description |
|--------|----------|-------------|
| POST | `/v1/chat/completions` | OpenAI-compatible chat (requires Bearer API key) |
### Chat & Search
| Method | Endpoint | Description |
|--------|----------|-------------|
| POST | `/api/chat` | Send message (streaming SSE) |
| POST | `/api/search` | Explicit web search (streaming SSE) |
### File Upload & Ingest
| Method | Endpoint | Description |
|--------|----------|-------------|
| POST | `/api/upload` | Upload file (multipart, admin) |
| DELETE | `/api/upload/{id}` | Delete upload (admin) |
| PATCH | `/api/upload/{id}/link` | Link upload to conversation (admin) |
| GET | `/api/upload/by-conversation/{id}` | List uploads for conversation |
| POST | `/api/ingest` | Ingest text into RAG (Bearer token auth) |
### Memory
| Method | Endpoint | Description |
|--------|----------|-------------|
| GET | `/api/memories` | List all memories |
| POST | `/api/memories` | Add memory |
| PUT | `/api/memories/{rowid}` | Update memory |
| DELETE | `/api/memories/{rowid}` | Delete memory |
| GET | `/api/memories/search?q=term` | Search memories |
| GET | `/api/memories/stats` | Get counts by topic |
### Cluster
| Method | Endpoint | Description |
|--------|----------|-------------|
| GET | `/api/cluster` | Cluster status (nodes, coordinator, event log) |
### RAG Management
| Method | Endpoint | Description |
|--------|----------|-------------|
| GET | `/api/rag/stats` | RAG corpus stats (admin) |
| POST | `/api/rag/flush` | Delete non-pinned vectors (admin) |
### Models & System
| Method | Endpoint | Description |
|--------|----------|-------------|
| GET | `/api/models` | List available models |
| GET | `/api/ps` | List loaded models |
| POST | `/api/show` | Get model info |
| GET | `/api/stats` | CPU, RAM, GPU, VRAM stats |
| GET | `/api/search/status` | SearXNG availability |
### Settings & Profile
| Method | Endpoint | Description |
|--------|----------|-------------|
| GET | `/api/profile` | Get profile content |
| PUT | `/api/profile` | Update profile (admin) |
| GET | `/api/profile/default` | Get default profile |
| GET | `/api/settings` | Get settings |
| PUT | `/api/settings` | Update settings (admin) |
### Conversations
| Method | Endpoint | Description |
|--------|----------|-------------|
| GET | `/api/conversations` | List conversations |
| POST | `/api/conversations` | Create conversation |
| GET | `/api/conversations/{id}` | Get conversation with messages |
| PUT | `/api/conversations/{id}` | Update conversation title/model |
| DELETE | `/api/conversations/{id}` | Delete conversation |
| DELETE | `/api/conversations` | Delete ALL conversations |
### Presets
| Method | Endpoint | Description |
|--------|----------|-------------|
| GET | `/api/presets` | List presets |
| POST | `/api/presets` | Create preset |
| PUT | `/api/presets/{id}` | Update preset |
| DELETE | `/api/presets/{id}` | Delete preset |
### Skills
| Method | Endpoint | Description |
|--------|----------|-------------|
| GET | `/api/skills` | List all skills with state |
| GET | `/api/skills/active` | List active skills |
| PUT | `/api/skills/{key}` | Toggle skill enabled (admin) |
### Auth
| Method | Endpoint | Description |
|--------|----------|-------------|
| POST | `/api/auth/guest` | Create guest session |
| POST | `/api/auth/login` | Admin PIN login |
| POST | `/api/auth/logout` | Revoke session |
| GET | `/api/auth/session` | Check session validity |
| POST | `/api/auth/heartbeat` | Extend session TTL |
## Configuration
Settings are stored in the `settings` table and include:
- `profile_enabled` — Inject profile into chats (true/false)
- `search_enabled` — Auto web search (true/false)
- `memory_enabled` — Memory injection (true/false)
- `skills_enabled` — Skills framework (true/false)
- `default_model` — Default inference model
## Testing
```bash
python3 -m pytest tests/ -v
```
All 179 tests use `tmp_path` fixtures + monkeypatched `httpx.AsyncClient`/`aio-pika`. No external services needed.
## License
MIT
## Repository
Gitea: `ssh://gitea@llgit.llamachile.tube:1319/gramps/caic.git`
TASKS.md
-986
View File
@@ -1,986 +0,0 @@
# cAIc — OpenCode Prompt Sequence
# Generated: 2026-07-01
# Execute sequentially. Run full test suite after each task before proceeding.
# Test command: ./venv/bin/python -m pytest tests/ -v
---
## ~~TASK 1 — README Cleanup [DONE]~~
Review README.md in the current repo. Remove any node references other than `coordinator` (192.168.50.108) and `worker` (192.168.50.210). Ensure all references to the project use the exact casing `cAIc` — not `Jarvischat`, `JarvisChat`, or `jarvischat`. Do not change any functional content, endpoint documentation, or architecture descriptions — this is a text cleanup only. After editing, verify the file renders cleanly as markdown. Commit with message: `docs: clean up node references and branding consistency`.
No new tests required for this task.
---
## ~~TASK 2 — Qwen2.5-Coder llama-server Service on Coordinator (Infrastructure) [DONE]~~
**Status: Systemd unit created, verified, and restored.**
This task originally defined creation of `/etc/systemd/system/llama-server-coder.service` (port 8082, Qwen2.5-Coder-14B Q5_K_M) as a prerequisite for dynamic model swapping. That sysadmin work is done.
**The real Task 2 deliverable — the ability to dynamically swap models based on query classification — is delivered by Roadmap N (Tasks 9–15).** The flow:
1. **Task 13** — Phi-4-mini triage (`triage.py`) classifies the query as `general`, `code`, `search`, or `rag`
2. **Task 13** — `select_node()` picks the best worker node; if the ideal model isn't active, it triggers a swap
3. **Task 14** — `request_model_swap()` publishes `cmd.swap_model` via AMQP `jc.admin` exchange
4. **Task 12** — The node agent on worker receives the command, stops the current llama-server, starts the correct one, waits for health, and publishes `model_ready`
5. **Task 14** — coordinator receives `model_ready`, updates the cluster registry, and routes the query to the node
The swap is async and transparent — the user sees only latency. The UI (Task 15) shows a yellow "swapping" status dot during the transition.
The service unit at `/etc/systemd/system/llama-server-coder.service` is the **target** the node agent starts when swapping to code inference. It is not enabled at boot — the AMQP cluster manages activation.
See Tasks 9–15 for the actual model swap implementation.
No pytest tests required for this infrastructure task.
---
## ~~TASK 3 — Update OpenCode Config to Use Qwen on :8082 [DONE]~~
Update `/home/gramps/.config/opencode/opencode.jsonc` (on this machine, coordinator) to point the configured provider at `http://127.0.0.1:8082/v1` instead of `http://127.0.0.1:8081/v1`. The model name in the config should be updated to reflect `qwen2.5-coder-14b` or whatever model ID the llama-server instance at :8082 reports via `/v1/models`. Verify the endpoint is reachable before writing the config change. Do not restart OpenCode — the config change takes effect on next session start.
No pytest tests required for this task.
---
## ~~TASK 4 — File/Document Attachment: Backend Ingest Endpoint [DONE]~~
**Status: `POST /api/upload` with mode=(context|ingest|both), PDF/text extraction, Qdrant upsert, SQLite context (1hr expiry). Committed `4a891c8` (v1.9.0).**
This task implements the backend half of file/document attachment (TODO #21). The goal is dual-aspect upload: a file can be used as immediate chat context, ingested into the RAG corpus (Qdrant), or both.
**Add to `config.py`:**
- `UPLOAD_DIR` — path for temporary upload storage, default `/tmp/caic_uploads`
- `MAX_UPLOAD_BYTES` — max file size, default 20MB
- `SUPPORTED_UPLOAD_TYPES` — set of MIME types: `text/plain`, `text/markdown`, `application/pdf`, `application/json`, `text/x-python`, `text/html`
**Create `routers/upload.py`:**
Implement `POST /api/upload` (admin required). Accept `multipart/form-data` with:
- `file` — the uploaded file (required)
- `mode` — string enum: `context` (inject into next chat only), `ingest` (add to RAG corpus), `both` (default: `both`)
- `conversation_id` — optional, associates context-mode content with a specific conversation
Behavior:
- Validate file size against `MAX_UPLOAD_BYTES` — return 413 if exceeded
- Validate MIME type against `SUPPORTED_UPLOAD_TYPES` — return 415 if unsupported
- For PDF files, extract text using `pypdf` (add to requirements.txt)
- For all other types, read as UTF-8 text
- If mode includes `ingest`: chunk the extracted text into 512-token overlapping chunks (128-token overlap), generate embeddings via `EMBED_URL` (http://192.168.50.108:11434/api/embeddings, model mxbai-embed-large), upsert into Qdrant collection `caic` with metadata `{source: filename, upload_date: iso_timestamp, type: "upload"}`
- If mode includes `context`: store the full extracted text in a new SQLite table `upload_context` with columns `(id INTEGER PRIMARY KEY, conversation_id TEXT, filename TEXT, content TEXT, created_at TEXT, expires_at TEXT)`. Context entries expire after 1 hour.
- Return JSON: `{filename, size_bytes, mode, chunks_ingested (if ingest), context_id (if context), message}`
**Add `upload_context` table to `db.py`** `init_db()`.
**Wire `upload.router` into `app.py`** in the router registration block.
**Write `tests/test_upload.py`** covering:
- Valid text file upload, mode=ingest — assert chunks_ingested > 0, Qdrant upsert called
- Valid text file upload, mode=context — assert context_id returned, row exists in upload_context
- Valid text file upload, mode=both — assert both behaviors
- File exceeds MAX_UPLOAD_BYTES — assert 413
- Unsupported MIME type — assert 415
- Guest session attempt — assert 403
- PDF extraction path — mock pypdf, assert text extracted and processed
Mock Qdrant and EMBED_URL calls via monkeypatch. Do not require live external services in tests.
Run full test suite after implementation. All 26 existing tests must continue to pass.
---
## ~~TASK 5 — File/Document Attachment: UI Integration [DONE]~~
**Status: Paperclip icon, file preview pill, gallery overlay, attachment indicators, DELETE/PATCH link/by-conversation endpoints, chat context injection. Committed `81238c0` (v1.10.0).**
This task implements the frontend half of TODO #21. The UI is a single file at `templates/index.html`.
Add a file attachment button to the chat input area. Requirements:
- Paperclip icon button adjacent to the send button
- Clicking opens a file picker filtered to supported types (`.txt`, `.md`, `.pdf`, `.json`, `.py`, `.html`)
- On file selection, show a pill/badge above the input showing the filename with an X to remove it
- On send, if a file is attached: POST to `/api/upload` with `mode=both` and the current `conversation_id`, then include the returned `context_id` in the subsequent `/api/chat` POST body as `upload_context_id`
- If the upload fails, show an inline error and do not send the chat message
- File attachment state clears after send
**Update `/api/chat` in `routers/chat.py`:**
- Accept optional `upload_context_id` in the request body
- If present, look up the content in `upload_context` table and prepend it to the system prompt as: `\n\n[ATTACHED DOCUMENT: {filename}]\n{content}\n[END DOCUMENT]`
- If the context_id is expired or missing, log a warning and continue without it (do not error)
**Add to `tests/test_chat_streaming_and_memory_paths.py`:**
- Test that a valid `upload_context_id` results in document content being prepended to the system prompt
- Test that an expired/missing `upload_context_id` is silently ignored
Run full test suite. All existing tests must continue to pass.
---
## ~~TASK 6 — Roadmap I: Terminal Command RAG Hook [DONE]~~
**Status: `POST /api/ingest` with Bearer token auth, `chunk_text()` shared helper, `caic-ingest.sh` script. Committed `1ac21ad` (v0.11.0).**
This task implements autonomous RAG ingestion of significant terminal activity (TODO #23).
**Create `routers/ingest.py`:**
Implement `POST /api/ingest` (requires Bearer token auth — use same `COMPLETIONS_API_KEY` mechanism as `routers/completions.py`). Accept JSON body:
- `content` — string, the text to ingest (required)
- `source` — string, origin label e.g. `terminal`, `file`, `external` (default: `external`)
- `metadata` — optional dict of additional key/value pairs
Behavior:
- Chunk `content` into 512-token overlapping chunks (128-token overlap) — extract this logic into a shared helper `chunk_text(text, chunk_size=512, overlap=128)` in `rag.py` if not already present
- Generate embeddings via `EMBED_URL`
- Upsert into Qdrant collection `caic` with metadata `{source, ingest_date: iso_timestamp, ...metadata}`
- Return JSON: `{chunks_ingested, source, message}`
**Wire `ingest.router` into `app.py`.**
**Create `/usr/local/bin/caic-ingest.sh` on worker (192.168.50.210)** — this is a shell script, not a Python file, and lives outside the repo. Write it to stdout/document it clearly so gramps can deploy it manually:
```bash
#!/bin/bash
# caic-ingest.sh — pipe terminal commands into cAIc RAG
# Add to ~/.bashrc: export PROMPT_COMMAND="jc_capture"
# Function to call after significant commands
JC_URL="http://192.168.50.210:8080/api/ingest"
JC_TOKEN="${CAIC_COMPLETIONS_API_KEY}"
jc_capture() {
local cmd
cmd=$(history 1 | sed 's/^[ ]*[0-9]*[ ]*//')
# Only ingest significant commands
if echo "$cmd" | grep -qE '^(git|pip|systemctl|sudo|vi|vim|curl|wget|apt|python|pytest)'; then
curl -s -X POST "$JC_URL" \
-H "Authorization: Bearer $JC_TOKEN" \
-H "Content-Type: application/json" \
-d "{\"content\": $(echo "$cmd" | python3 -c 'import json,sys; print(json.dumps(sys.stdin.read().strip()))'), \"source\": \"terminal\"}" \
> /dev/null 2>&1 &
fi
}
```
**Write `tests/test_ingest.py`** covering:
- Valid ingest with content — assert chunks_ingested > 0
- Missing Bearer token — assert 401
- Wrong Bearer token — assert 403
- Empty content — assert 422
- Qdrant and embed calls mocked via monkeypatch
Run full test suite. All existing tests must continue to pass.
---
## ~~TASK 7 — Roadmap J: Startup Hardware Self-Assessment [DONE]~~
**Status: `hardware.py` + `routers/hardware.py` + 4 tests. Committed `7291b8f` (v0.12.0).**
On jC startup, probe available hardware and write a living config snapshot. This replaces hardcoded assumptions about VRAM and RAM.
**Create `hardware.py`** in the project root:
```
async def assess_hardware() -> dict
```
Probes:
- System RAM: `psutil.virtual_memory().total` and `.available`
- CPU count: `psutil.cpu_count()`
- GPU VRAM total and free: call `rocm-smi --showmeminfo vram --json` via subprocess, parse output. If rocm-smi absent or fails, set VRAM values to 0 and log a warning.
- llama-server reachable: GET `LLAMA_SERVER_BASE/v1/models`, timeout 3s. Record True/False and list of available model IDs.
- Qdrant reachable: GET `http://192.168.50.108:6333/collections`, timeout 3s. Record True/False and collection list.
- SearXNG reachable: GET `http://localhost:8888`, timeout 3s. Record True/False.
Returns a dict with all of the above. Writes result as JSON to `hardware_state.json` in the working directory.
**Call `assess_hardware()` from the FastAPI `lifespan` context** in `app.py` on startup, after `init_db()`. Log a summary line: `HW: {ram_gb}GB RAM, {vram_mb}MB VRAM, llama={reachable}, qdrant={reachable}, searxng={reachable}`.
**Expose `GET /api/hardware`** in a new `routers/hardware.py` — returns the current `hardware_state.json` content as JSON. No auth required (read-only, non-sensitive aggregate stats).
**Wire `hardware.router` into `app.py`.**
**Write `tests/test_hardware.py`** covering:
- `assess_hardware()` with all services reachable (mock subprocess and httpx calls) — assert all fields present
- `assess_hardware()` with rocm-smi absent — assert VRAM=0, no exception raised
- `assess_hardware()` with llama-server unreachable — assert `llama_reachable=False`, no exception
- `GET /api/hardware` — assert returns JSON with expected keys
Run full test suite. All existing tests must continue to pass.
---
## ~~TASK 8 — Roadmap K: RAG Corpus Management [DONE]~~
Qdrant collection `caic` currently grows without bound. Implement score-based eviction with hysteresis, pinned sources, operational stats, and a flush command.
### Config — add to `config.py`:
```python
RAG_MAX_VECTORS = 50000 # absolute ceiling; eviction targets thresholds below it
RAG_EVICTION_HIGH_WATER = 0.80 # fraction of RAG_MAX_VECTORS that triggers eviction
RAG_EVICTION_LOW_WATER = 0.20 # fraction where eviction stops
RAG_EVICTION_BATCH = 1000 # max points to delete per Qdrant scroll/delete cycle
RAG_PINNED_SOURCES = ["upload", "profile"] # never evicted
RAG_GRACE_HOURS = 1 # new vectors ineligible for eviction until this old
RAG_ACCESS_WEIGHT = 1.0 # score factor: retrieval_count * ACCESS_WEIGHT
RAG_AGE_WEIGHT = 0.1 # score factor: ingest_age_hours * AGE_WEIGHT
```
Validations on boot: `high_water > low_water`, `batch > 0`, `max_vectors > 0`.
### Eviction algorithm — add to `rag.py`:
```
score = (retrieval_count * ACCESS_WEIGHT) + (age_hours * AGE_WEIGHT)
```
Lower score = evicted first. Tiebreak: `last_accessed` ASC (older wins).
```python
async def get_collection_count() -> int
# GET /collections/caic → return vectors_count
async def get_collection_stats() -> dict
# Return {vector_count, max_vectors, high_water, low_water, percent_full, pinned_sources}
async def evict_batch(batch_size: int) -> int
# Scroll Qdrant for vectors NOT in RAG_PINNED_SOURCES, WHERE ingest_age > RAG_GRACE_HOURS,
# ordered by score ASC, last_accessed ASC.
# Delete up to batch_size. Return count deleted.
# If 0 evictable vectors found: log warning, return 0 (break loop).
async def maybe_evict() -> int
# Acquire eviction_lock (asyncio.Lock).
# count = get_collection_count()
# threshold_high = RAG_MAX_VECTORS * RAG_EVICTION_HIGH_WATER
# threshold_low = RAG_MAX_VECTORS * RAG_EVICTION_LOW_WATER
# total_evicted = 0
# while count >= threshold_low:
# if total_evicted > 0 and count < threshold_low: break
# deleted = evict_batch(RAG_EVICTION_BATCH)
# if deleted == 0: break # no more unpinned targets
# total_evicted += deleted
# count -= deleted
# if count < threshold_high and total_evicted > 0: break
# # only one pass if batch spans the full gap
# if count < threshold_low: break
# Record total_evicted + timestamp in EVICTION_LOG (list of dicts, kept in memory, max 1000 entries)
# Release lock. Return total_evicted.
async def get_rag_operational_stats() -> dict
# Returns: vector_count, max_vectors, high_water_pct, low_water_pct,
# percent_full, pinned_sources, grace_hours,
# eviction_counts_last_1m, eviction_counts_last_5m, eviction_counts_last_30m,
# at_risk_count (vectors in bottom 10% by score),
# pinned_count, avg_retrieval_count
```
### Edge cases & guards:
1. **Newborn grace** — vectors < `RAG_GRACE_HOURS` old are excluded from eviction scroll (score=0 otherwise → immediate deletion)
2. **All-pinned freeze** — if scroll returns 0 evictable vectors, log warning and break loop
3. **Race** — `asyncio.Lock()` guards `maybe_evict()`; concurrent callers wait their turn
4. **Zero config** — `RAG_MAX_VECTORS <= 0` → eviction disabled; `RAG_EVICTION_BATCH <= 0` → clamped to 1
5. **Legacy payloads** — vectors without `retrieval_count` or `last_accessed` get defaults (0, `ingest_date`)
### Wire eviction:
Call `maybe_evict()` after each upsert batch completes in:
- `routers/upload.py` — after Qdrant upsert
- `routers/ingest.py` — after Qdrant upsert
### Admin endpoints — new `routers/rag_admin.py`:
| Method | Endpoint | Description |
|--------|----------|-------------|
| GET | `/api/rag/stats` | Operational stats (see `get_rag_operational_stats()`) — admin required |
| POST | `/api/rag/flush` | Delete ALL points from the Qdrant `caic` collection. Returns `{deleted_count, collection: "caic", status: "flushed"}`. Admin required. |
### In-memory eviction log:
```python
EVICTION_LOG: list[dict] = [] # managed by rag.py, max 1000 entries
# Each entry: {timestamp: iso, count: N, remaining: N}
# Tied to RATE_EVENTS pattern from security.py for rolling window calculations
```
### Tests — `tests/test_rag_management.py`:
- `get_collection_count()` — mock Qdrant GET, assert correct count
- `get_collection_stats()` — assert shape matches config
- `evict_batch()` — mock Qdrant scroll + delete, assert pinned sources excluded, grace period enforced, batch size respected
- `maybe_evict()` — below high water: 0 evicted; at high water: eviction fires; stops at low water; all-pinned scroll returns 0 → breaks
- `GET /api/rag/stats` — assert full shape
- `POST /api/rag/flush` — assert points deleted, admin required, guest 403
- `POST /api/rag/flush` by guest — assert 403
- Race lock — concurrent calls to `maybe_evict()` queue up, only one evicts
Mock all Qdrant calls via monkeypatch. Do not require live services.
Run full test suite. All existing tests must continue to pass.
---
## ~~TASK 9 — Roadmap N1: RabbitMQ Install and Service on Coordinator (Infrastructure) [DONE]~~
This task runs on coordinator (this machine). Install RabbitMQ and verify it is operational.
Run the following steps:
1. `apt-get update && apt-get install -y rabbitmq-server`
2. `systemctl enable rabbitmq-server && systemctl start rabbitmq-server`
3. `systemctl status rabbitmq-server` — verify active/running
4. Enable the management plugin: `rabbitmq-plugins enable rabbitmq_management`
5. Create a dedicated jC vhost: `rabbitmqctl add_vhost caic`
6. Create a dedicated user: `rabbitmqctl add_user caic CHANGEME_PASSWORD` — generate a random 24-char alphanumeric password and record it
7. Grant permissions: `rabbitmqctl set_permissions -p caic caic ".*" ".*" ".*"`
8. Verify management UI is reachable: `curl -s -u guest:guest http://localhost:15672/api/overview | python3 -m json.tool`
9. Delete default guest user: `rabbitmqctl delete_user guest`
Declare the two topic exchanges needed by jC:
- Exchange name: `jc.admin`, type: `topic`, durable: true
- Exchange name: `jc.system`, type: `topic`, durable: true
Use `rabbitmqadmin` or `curl` against the management API to declare exchanges. Verify both exchanges appear in: `curl -s -u caic:{password} http://localhost:15672/api/exchanges/caic`
Write the generated RabbitMQ password to `/home/gramps/.caic_amqp_secret` with mode 600. This will be read by cAIc as an env var source in subsequent tasks.
No pytest tests required for this infrastructure task.
---
## ~~TASK 10 — Roadmap N2: AMQP Connection Layer in jC [DONE]~~
This task adds the core AMQP connection manager to jC. It must connect to RabbitMQ on coordinator (localhost from jC's perspective since jC runs on coordinator), handle reconnection, and provide a shared channel for all AMQP operations.
**Add to `requirements.txt`:** `aio-pika>=9.0.0`
**Add to `config.py`:**
- `AMQP_URL` — read from env `CAIC_AMQP_URL`, default `amqp://caic:password@localhost:5672/caic`. The actual password comes from `/home/gramps/.caic_amqp_secret` — read it at startup if the env var is not set.
- `AMQP_RECONNECT_DELAY` — seconds between reconnect attempts, default 5
- `AMQP_EXCHANGE_ADMIN` — `jc.admin`
- `AMQP_EXCHANGE_SYSTEM` — `jc.system`
**Create `amqp.py`** in the project root:
```python
# Manages a single persistent aio-pika connection and channel.
# Provides:
# connect() -> None # establish connection, declare exchanges
# disconnect() -> None # graceful close
# get_channel() # returns current channel, reconnects if needed
# publish(exchange, routing_key, payload: dict) -> None
# # publishes JSON-serialized payload as persistent message
```
Connection must:
- Reconnect automatically on disconnect with `AMQP_RECONNECT_DELAY` backoff
- Log connection events at INFO level
- Not raise on publish if disconnected — log error and return (fire-and-forget, jC must not crash if RabbitMQ is down)
**Start AMQP connection in `app.py` lifespan** after `assess_hardware()`. Disconnect in lifespan cleanup.
**Write `tests/test_amqp.py`** covering:
- `publish()` with mocked aio-pika connection — assert message published with correct exchange and routing key
- `publish()` when disconnected — assert no exception raised, error logged
- `get_channel()` when connection is None — assert reconnect attempted
Mock all aio-pika calls via monkeypatch. Do not require a live RabbitMQ instance in tests.
Run full test suite. All existing tests must continue to pass.
---
## ~~TASK 11 — Roadmap N3: Cluster Protocol & Registration Handler (Coordinator Side) [DONE]~~
**Status: Implemented and pushed (899988c).** `amqp.py` subscribe/rebind, `cluster.py` with CLUSTER_NODES/CLUSTER_EVENTS/CLUSTER_COORDINATOR and 6 handlers, `routers/cluster.py` (`GET /api/cluster`), 13 tests. No passive heartbeats — ping/pong on-demand before work routing. 148 tests pass.
jC on the coordinator must listen for nine message types across `jc.admin` and `jc.system`, maintain the cluster registry, and expose an application-level event log.
### 11.1 AMQP Protocol — Message Catalog
All payloads are JSON, published as persistent messages.
| Direction | Exchange | Routing Key | Message Type | Description |
|-----------|----------|-------------|-------------|-------------|
| Worker → Coordinator | `jc.admin` | `node.{name}.register` | register | Worker requests admission |
| Worker → Coordinator | `jc.admin` | `node.{name}.deregister` | deregister | Worker signals graceful departure |
| Coordinator → Worker | `jc.admin` | `node.{name}.admitted` | admitted | Coordinator grants admission |
| Coordinator → Worker | `jc.admin` | `node.{name}.rejected` | rejected | Coordinator denies admission (with reason) |
| Coordinator → Worker | `jc.admin` | `node.{name}.ping` | ping | Coordinator checks if worker is alive (sent before routing work) |
| Worker → Coordinator | `jc.admin` | `node.{name}.pong` | pong | Worker confirms aliveness |
| Worker → Coordinator | `jc.system` | `node.{name}.event` | event | Application-level syslog event |
| Any → All | `jc.system` | `cluster.coordinator.query` | coord_query | Anyone asks "who is coordinator?" |
| Coordinator → All | `jc.system` | `cluster.coordinator.response` | coord_response | Coordinator announces itself |
Worker presence is assumed from registration onward. No periodic heartbeats — a worker can sit idle for days without chatter. When the coordinator needs to route work to a worker, it pings first; if the worker doesn't pong within timeout, the coordinator deregisters it and moves to the next node.
### 11.2 Payload Schemas
**register** (worker → coordinator):
```json
{
"node_name": "worker01",
"node_type": "worker",
"ip": "192.168.50.210",
"capabilities": {
"gpu": true, "gpu_type": "amd", "vram_mb": 8192,
"cpu_cores": 8, "ram_gb": 16
},
"active_model": {
"name": "llama3.1", "version": "latest", "quant": "Q4_K_M",
"path": "/var/lib/caic/models/llama3.1-latest-Q4_K_M.gguf",
"port": 8081
},
"inventory": [
{"name": "llama3.1", "version": "latest", "quant": "Q4_K_M",
"path": "/var/lib/caic/models/llama3.1-latest-Q4_K_M.gguf", "port": 8081}
],
"status": "active"
}
```
**deregister** (worker → coordinator):
```json
{
"node_name": "worker01",
"reason": "shutdown",
"timestamp": "2026-07-06T12:00:00Z"
}
```
**ping** (coordinator → worker):
```json
{
"from": "coordinator",
"node_name": "worker01",
"type": "ping",
"correlation_id": "550e8400-e29b-41d4-a716-446655440000",
"timestamp": "2026-07-06T12:00:00Z"
}
```
Worker must respond within 5 seconds or the coordinator considers it absent.
**pong** (worker → coordinator):
```json
{
"node_name": "worker01",
"type": "pong",
"correlation_id": "550e8400-e29b-41d4-a716-446655440000",
"status": "active",
"active_model": {"name": "llama3.1", "port": 8081},
"load": {"cpu_pct": 45, "ram_pct": 62, "vram_pct": 38},
"timestamp": "2026-07-06T12:00:00Z"
}
```
Correlation ID matches the ping so the coordinator can pair request and response.
**coord_query** (any → `cluster.coordinator.query`):
```json
{"type": "coord_query", "timestamp": "2026-07-06T12:00:00Z"}
```
Coordinator responds on `cluster.coordinator.response`:
```json
{
"coordinator_node": "coordinator",
"cluster_nodes": ["worker01"],
"timestamp": "2026-07-06T12:00:00Z"
}
```
**event** (worker → coordinator):
```json
{
"node_name": "worker01",
"severity": "info",
"message": "llama-server started with model llama3.1:latest",
"details": {"model": "llama3.1:latest", "port": 8081, "pid": 1234},
"timestamp": "2026-07-06T12:00:00Z"
}
```
Severity levels: `info`, `warn`, `error`, `critical`. The coordinator assigns `category: "application"` based on the exchange (jc.system). No `event_type` field — the category is determined by the channel, not the payload.
### 11.3 Design — Status Transitions Drive the Event Log
All admin-level events are *derived* from `register()` and `deregister()` as side effects. There are no separate message types for coordinator election, node staleness, quarantine, or release — those are status transitions that `register()`/`deregister()` emit into `CLUSTER_EVENTS` locally.
**Node status lifecycle:**
```
UNKNOWN ──register()──▶ active ──deregister()──▶ (removed)
│
ping timeout│(coordinator publishes
│ deregister on its behalf)
▼
(removed)
```
**Coordinator status lifecycle:**
```
NONE ──register(node_type=coordinator)──▶ CLUSTER_COORDINATOR set
│
deregister()│or timeout
▼
CLUSTER_COORDINATOR cleared
```
**Event categories — two buckets, no granular types:**
| Category | When | severity |
|----------|------|----------|
| `cluster` | Node lifecycle, coordinator changes, model swaps, node offline — everything on `jc.admin` | `info` / `warn` / `error` |
| `application` | Worker syslog events (incoming on `jc.system` `node.*.event`) | `info` / `warn` / `error` / `critical` |
Every `_push_event()` call uses one of these two categories. The `message` field carries the human-readable detail — no need for event type strings. The reporting tool filters by category + severity.
**Channel split — security rationale:**
The two exchanges are not an organizational convenience. They enforce a **data isolation boundary**:
| Exchange | Contains | Exposed to |
|----------|----------|------------|
| `jc.admin` | Node lifecycle, heartbeats, model swaps, coordinator changes | Operations / machine-room staff |
| `jc.system` | Application events — inference queries, RAG context, user-facing data | Application-layer audit only |
`jc.system` events can leak information about what users are doing and asking. The split ensures a sysadmin monitoring cluster health never accidentally consumes user-data-bearing events. The channels can be locked down independently — different AMQP credentials, separate queue permissions, different in-transit encryption policies if needed later.
### 11.4 Implementation
**Add to `amqp.py`:**
```python
_SUBSCRIPTIONS: list[tuple[str, str, Callable]] # (exchange, routing_key, callback)
async def subscribe(exchange, routing_key, callback) -> None
# Append to _SUBSCRIPTIONS list
# Declare a unique queue per subscription (name: f"jc.{exchange}.{sanitized_routing_key}")
# Bind queue to exchange/routing_key, consume with callback
```
Each subscription gets its own queue so multiple subscribers on different routing keys all receive messages. On reconnect: drain old consumers, iterate `_SUBSCRIPTIONS`, re-declare and re-bind each one. The `connect()` function must call `_rebind_subscriptions()` after exchanges are declared.
**Create `cluster.py`** in the project root:
```python
# In-memory cluster registry + event log
# Survives only while jC is running (not persisted)
CLUSTER_NODES: dict[str, NodeRecord]
CLUSTER_EVENTS: deque[EventRecord] # bounded at 1000 entries
CLUSTER_COORDINATOR: str | None # node_name of active coordinator
# NodeRecord fields:
# node_name, node_type, ip, status, active_model, inventory,
# capabilities: {gpu, gpu_type, vram_mb, cpu_cores, ram_gb}
# registered_at, last_seen
# EventRecord:
# category: str ("cluster" | "application")
# severity: str ("info" | "warn" | "error" | "critical")
# node_name: str
# message: str
# details: dict | None
# timestamp: str
def _push_event(category, severity, node_name, message, details=None) -> None
# Append EventRecord to CLUSTER_EVENTS, pop left if > 1000
async def handle_registration(message) -> None
# Parse payload, validate required fields (node_name, node_type, ip, active_model, inventory)
# Reject if node_name duplicate and CLUSTER_NODES[node_name].status == "active"
# If CLUSTER_COORDINATOR is None AND node_type == "coordinator":
# set CLUSTER_COORDINATOR = node_name
# _push_event("cluster", "info", node_name, "elected coordinator")
# publish cluster.coordinator.response on jc.system {coordinator_node, cluster_nodes, timestamp}
# Add node to CLUSTER_NODES with status="active"
# _push_event("cluster", "info", node_name, f"admitted as {node_type}")
# publish admitted on jc.admin node.{name}.admitted {node_name, timestamp, amqp_url}
async def handle_deregistration(message) -> None
# Parse payload (node_name, reason, timestamp)
# If node_name == CLUSTER_COORDINATOR:
# clear CLUSTER_COORDINATOR
# _push_event("cluster", "warn", node_name, f"coordinator lost — {reason}")
# _push_event("cluster", "info", node_name, f"departed — {reason}")
# Remove node from CLUSTER_NODES, log it
async def handle_pong(message) -> None
# Parse: node_name, correlation_id, status, active_model, load, timestamp
# Match correlation_id to outstanding ping
# If node in CLUSTER_NODES: update last_seen, status, active_model
# Signal the waiting caller that the node is alive
# If node unknown: log warning, do NOT auto-admit
async def handle_event(message) -> None
# Parse: node_name, severity, message, details, timestamp
# Assigns category="application" (incoming on jc.system)
# Append EventRecord to CLUSTER_EVENTS (pop left if > 1000)
async def handle_coordinator_query(message) -> None
# Respond on jc.system cluster.coordinator.response
# Payload: {coordinator_node, cluster_nodes: list(CLUSTER_NODES.keys()), timestamp}
def get_cluster_state() -> dict
# Return: {nodes: CLUSTER_NODES, coordinator: CLUSTER_COORDINATOR,
# events: last 50 CLUSTER_EVENTS}
```
**Subscribe in `app.py` lifespan** after AMQP connects:
| Exchange | Routing Key | Handler |
|----------|-------------|---------|
| `jc.admin` | `node.*.register` | `handle_registration` |
| `jc.admin` | `node.*.deregister` | `handle_deregistration` |
| `jc.admin` | `node.*.pong` | `handle_pong` |
| `jc.system` | `node.*.event` | `handle_event` |
| `jc.system` | `cluster.coordinator.query` | `handle_coordinator_query` |
### 11.5 API — `GET /api/cluster`
New router `routers/cluster.py`:
- `GET /api/cluster` — returns full cluster state: `{nodes, coordinator, events}` (last 50 events). No auth required.
Wire `cluster.router` into `app.py`.
### 11.6 Tests — `tests/test_cluster.py`
Mock all aio-pika calls. Do not require live RabbitMQ.
| # | Test | What it asserts |
|---|------|-----------------|
| 1 | Valid worker registration | Node admitted, CLUSTER_NODES updated, `cluster` event logged, `admitted` message published |
| 2 | First coordinator auto-promotion | CLUSTER_COORDINATOR set, `cluster` event with "elected" message, `coord_response` published |
| 3 | Duplicate node name rejected | `rejected` message with reason=`duplicate_node_name`, `cluster` event logged |
| 4 | Malformed payload rejected | `rejected` message with reason=`malformed_payload` |
| 5 | Graceful deregistration | Node removed, `cluster` event logged. If coordinator: CLUSTER_COORDINATOR cleared |
| 6 | Pong from known node | last_seen updated, load/status refreshed |
| 7 | Pong from unknown node | Warning logged, node NOT added |
| 8 | Event stored in log | Event appended to CLUSTER_EVENTS; at 1001 entries the oldest is popped |
| 9 | Coordinator query produces response | Response published with coordinator name and node list |
| 10 | GET /api/cluster shape | Response contains `nodes`, `coordinator`, `events` keys |
Run full test suite. All existing tests must continue to pass.
---
## ~~TASK 12 — Roadmap N4: Worker Node Registration Publisher (Worker Side) [DONE]~~
This task creates the worker node AMQP client that runs on worker (192.168.50.210). It is a standalone Python script — not part of the jC FastAPI app — that runs as a systemd service on worker.
**Create `node_agent/agent.py`** in the repo (new directory).
### 12.1 Config & Inventory Discovery
On start, reads `/etc/caic-node-agent.conf` (INI format):
- `node_name` — hostname, default from `socket.gethostname()`
- `node_ip` — LAN IP, default from socket
- `node_type` — `"worker"` (fixed)
- `capabilities` — comma-separated list, e.g. `llm,rag`
- `amqp_url` — RabbitMQ URL on coordinator, e.g. `amqp://caic:password@192.168.50.108:5672/caic`
- `llama_port` — port llama-server/llama-rpc is listening on, default 8081
- `models_dir` — path to GGUF model files, default `/var/lib/caic/models`
- `active_model` — filename of currently active model (without path)
Discovers inventory by globbing `models_dir` for `*.gguf` files and parsing name/version/quant from filename using regex pattern: `{name}-{version}-{quant}.gguf` where quant matches `Q[0-9]+_K_[A-Z]+` or similar standard suffixes.
### 12.2 Registration
Publishes registration to `jc.admin`, routing key `node.{node_name}.register`:
```json
{
"node_name": "worker01",
"node_type": "worker",
"ip": "192.168.50.210",
"capabilities": ["llm"],
"active_model": {"name": "...", "version": "...", "quant": "...", "path": "...", "port": 8081}
}
```
### 12.3 Admission Response
Listens on `node.{node_name}.admitted` and `node.{node_name}.rejected` (both `jc.admin`). Logs result. If rejected, exits with error.
### 12.4 Ping Listener
After admission: listens on `jc.admin`, routing key `node.{node_name}.ping`. On receipt, responds immediately (within 1 second) with a pong on `jc.admin`, routing key `node.{node_name}.pong`:
```json
{
"node_name": "worker01",
"type": "pong",
"correlation_id": "<echoed from ping>",
"status": "active",
"active_model": {"name": "...", "version": "...", "quant": "...", "path": "...", "port": 8081},
"load": {"cpu_pct": 45, "ram_pct": 62, "vram_pct": 38},
"timestamp": "<utc>"
}
```
No periodic heartbeats. Worker sits idle between pings — coordinator only pings when it needs to route work.
### 12.5 Model Swap Command Handler
Listens on `jc.admin`, routing key `node.{node_name}.cmd.swap_model`:
- Payload: `{model_filename: str}`
- Stops current llama-server: `systemctl stop llama-server`
- Updates `/etc/caic-node-agent.conf` active_model field
- Starts llama-server: `systemctl start llama-server` (assumes service reads active_model from conf or ExecStart is updated)
- Waits for llama-server to be healthy: poll `http://localhost:{llama_port}/v1/models` every 2s, timeout 120s
- Publishes to `jc.system`, routing key `node.{node_name}.model_ready`:
```json
{"node_name": "...", "active_model": "...", "port": ..., "timestamp": "..."}
```
- If startup fails within timeout: publishes `node.{node_name}.model_failed` with error detail
### 12.6 Files & Tests
**Create `node_agent/requirements.txt`:** `aio-pika>=9.0.0`
**Document `/etc/caic-node-agent.conf` format** in a comment block at the top of `agent.py`.
**Write `tests/test_node_agent.py`** covering:
- Registration payload construction from config + model discovery — assert correct JSON shape
- Model swap command handler: success path — assert systemctl calls made, model_ready published
- Model swap command handler: timeout path — assert model_failed published
- Ping handler: on ping, publishes pong with correct correlation_id
- Agent starts idle after admission, no heartbeat timer
Mock all aio-pika, subprocess, and httpx calls.
**Do not create a systemd service file in this task** — that is a manual deployment step. Document the required service configuration in a comment at the bottom of `agent.py`.
Run full test suite. All existing tests must continue to pass.
---
## ~~TASK 13 — Roadmap N5: Query Routing via AMQP + Phi-4-mini Triage [DONE]~~
This task wires the cluster into jC's chat flow. When a query arrives at `/api/chat`, instead of always routing to the hardcoded `LLAMA_SERVER_BASE`, jC now routes to the best available cluster node based on query context.
**Prerequisites:** Tasks 9–12 complete. At least one worker node admitted to cluster.
**Install Phi-4-mini on coordinator (infrastructure step):**
- Download `Phi-4-mini-Instruct-Q4_K_M.gguf` from HuggingFace using `hf download microsoft/Phi-4-mini-instruct --include "*.Q4_K_M.gguf" --local-dir /var/lib/caic/models`
- Create `/etc/systemd/system/llama-server-triage.service` — same pattern as existing llama-server service but: port 8083, model path points to Phi-4-mini GGUF, no `--rpc` flag (runs entirely on coordinator CPU/iGPU), description `Llama.cpp Server (Phi-4-mini — triage/routing)`
- `systemctl daemon-reload && systemctl enable llama-server-triage && systemctl start llama-server-triage`
- Verify: `curl -s http://localhost:8083/v1/models`
**Add to `config.py`:**
- `TRIAGE_BASE` — `http://127.0.0.1:8083/v1` (Phi-4-mini)
- `TRIAGE_TIMEOUT` — 10 seconds
- `FALLBACK_TO_DEFAULT` — True (if triage fails or no nodes available, fall back to `LLAMA_SERVER_BASE`)
**Create `triage.py`** in the project root:
```python
async def classify_query(query: str) -> str
# Sends query to Phi-4-mini at TRIAGE_BASE with a classification system prompt.
# System prompt instructs model to respond with ONLY one of:
# "general", "code", "search", "rag"
# Returns the classification string.
# Timeout: TRIAGE_TIMEOUT seconds.
# On any error: returns "general" (fail-safe).
async def select_node(classification: str) -> dict | None
# Consults CLUSTER_NODES from cluster.py
# For "code": prefer nodes where active_model name contains "coder" or "qwen"
# For "general": prefer nodes where active_model name contains "mistral" or "llama"
# For "search" or "rag": return None (handled locally by jC)
# If no matching node found: return None (triggers FALLBACK_TO_DEFAULT)
# Returns NodeRecord dict for selected node, or None
async def get_inference_url(query: str) -> str
# Combines classify_query + select_node
# Returns full base URL: f"http://{node.ip}:{node.active_model.port}/v1"
# Falls back to LLAMA_SERVER_BASE if classification=search/rag, no nodes, or triage error
```
**Update `routers/chat.py`:**
- Replace the hardcoded `LLAMA_SERVER_BASE` reference with a call to `get_inference_url(user_message)`
- The rest of the chat flow (RAG, memory, streaming) is unchanged — only the inference target URL changes
**Write `tests/test_triage.py`** covering:
- `classify_query()` returns valid classification — mock Phi-4-mini response
- `classify_query()` on timeout — assert returns "general", no exception
- `select_node("code")` with coder node in cluster — assert correct node returned
- `select_node("general")` with no matching node — assert None returned
- `get_inference_url()` with code query and coder node available — assert returns node URL
- `get_inference_url()` with no nodes in cluster — assert returns LLAMA_SERVER_BASE fallback
**Update `tests/test_chat_streaming_and_memory_paths.py`:**
- Mock `triage.get_inference_url` to return a fixed URL in all existing tests so they continue to pass without a live cluster
Run full test suite. All existing tests must continue to pass.
---
## ~~TASK 14 — Roadmap N6: Model Swap Command Flow [DONE]~~
**Status: Implemented and pushed (`9d1fd44`).** `request_model_swap()`, `handle_model_ready()`, `handle_model_failed()` in `cluster.py`, async `select_node()` with swap triggering in `triage.py`, `tests/test_model_swap.py` (9 tests). 177 tests pass.
This task implements the coordinator-side logic for requesting a model swap on a worker node when the ideal model is not currently active.
**Add to `cluster.py`:**
```python
async def request_model_swap(node_name: str, model_filename: str) -> bool
# Publishes to jc.admin exchange, routing key node.{node_name}.cmd.swap_model
# Payload: {model_filename, requested_at: iso_timestamp}
# Sets node status to "swapping" in CLUSTER_NODES
# Returns True if message published successfully
async def handle_model_ready(message) -> None
# Handles node.{node_name}.model_ready from jc.system
# Updates CLUSTER_NODES[node_name].active_model to the new model
# Sets node status back to "active"
# Logs swap completion with timing
async def handle_model_failed(message) -> None
# Handles node.{node_name}.model_failed from jc.system
# Sets node status to "error" in CLUSTER_NODES
# Logs failure with detail from message payload
```
**Subscribe in `app.py` lifespan:**
- `jc.system` exchange, routing key `node.*.model_ready` → `handle_model_ready`
- `jc.system` exchange, routing key `node.*.model_failed` → `handle_model_failed`
**Update `triage.py` `select_node()`:**
- If the best-matching node exists but its active_model does not match the ideal model for the classification, AND the node status is "active" (not already swapping):
- Call `request_model_swap(node_name, ideal_model_filename)`
- Return None (triggers fallback) — the swap happens async, next query will find the right model active
- If node status is "swapping": return None (fallback, swap in progress)
**Update `GET /api/cluster`** to include node status in response.
**Write `tests/test_model_swap.py`** covering:
- `request_model_swap()` — assert swap command published, node status set to "swapping"
- `handle_model_ready()` — assert active_model updated, status set to "active"
- `handle_model_failed()` — assert status set to "error"
- `select_node()` with mismatched active model — assert swap requested, None returned
- `select_node()` with node status "swapping" — assert None returned without publishing another swap
Run full test suite. All existing tests must continue to pass.
---
## ~~TASK 15 — Roadmap N7: Cluster Status UI [DONE]~~
Surface cluster awareness in the jC frontend (`templates/index.html`).
**Add a cluster status panel** to the UI. Requirements:
- Small status bar or collapsible panel, visible but unobtrusive
- Polls `GET /api/cluster` every 15 seconds
- For each admitted node: show node name, active model name, and a colored status dot:
- Green: active
- Yellow: swapping
- Red: error or offline (not seen in last 60 seconds based on last_seen timestamp)
- If no nodes in cluster (empty): show "No worker nodes connected"
- Panel must not interfere with chat input or conversation list
**Update `GET /api/cluster` response** to include `last_seen` per node and a `status` field (`active`, `swapping`, `error`).
**Update heartbeat handling in `cluster.py`:** add a handler for `node.*.heartbeat` on `jc.system` that updates `last_seen` timestamp for the node.
**Subscribe in `app.py` lifespan:**
- `jc.system` exchange, routing key `node.*.heartbeat` → `handle_heartbeat`
**Add `handle_heartbeat()` to `cluster.py`:**
- Updates `CLUSTER_NODES[node_name].last_seen` to current timestamp
- If node was previously marked offline (not in CLUSTER_NODES), log re-registration warning but do not auto-admit — full registration required
**Write `tests/test_cluster_heartbeat.py`** covering:
- `handle_heartbeat()` for known node — assert last_seen updated
- `handle_heartbeat()` for unknown node — assert no crash, warning logged, node not added
Run full test suite. All 26+ existing tests must continue to pass.
~~Commit all changes introduced across Tasks 9–15 with message: `feat: Roadmap N — AMQP cluster nervous system complete`~~
---
## Backlog (Post-Roadmap N) ⏳
### B1 — Context loss in follow-up questions
**Symptom:** After asking "in {context}, explain {b}", a follow-up "what is {b}'s {x}?" gets a non-sequitur response that ignores the original context.
**Diagnosis:** `build_system_prompt()` is called fresh per-request with new RAG/memory results keyed to the current message text. These can change between turns and may dilute or override the conversation history. The original system prompt used for turn 1 (including its RAG context) is not stored in the DB — only user/assistant messages are. The inference server receives a different system prompt each turn.
**Possible fixes:**
- Store the assembled system prompt with each assistant message in the DB
- When replaying history, re-send the original system prompts from DB rather than rebuilding
- Or: cap RAG/memory injection to only fire on the first message of a conversation, then rely solely on conversation history for follow-ups
- Check that llama-server isn't truncating history due to context window overflow (Mistral-Nemo 12B = 128K context, unlikely)
### B2 — Bang-prefixed search routing
**Spec:** If a query begins with `!`, route to SearXNG search instead of local inference.
**Where:** In `routers/chat.py` `chat()` handler, after `user_message` is extracted. Strip the `!`, set a flag to always trigger auto-search regardless of perplexity/refusal.
**Change:** Add a `force_search` flag when `user_message.startswith("!")`, strip the prefix from the message saved to DB, and route directly to the search+summarize path.
### B3 — Docker distribution (v1.0 gate)
**Goal:** Ship cAIc as a `docker compose` stack so a single command stands up everything.
**Services to containerize:**
- cAIc (FastAPI app + SQLite)
- SearXNG
- Qdrant
- RabbitMQ
- llama-server (with optional RPC sidecar for GPU offload)
- Ollama (embeddings)
**Also needed:**
- `Dockerfile` for the cAIc app itself
- `docker-compose.yml` with all services, volumes, networks, env vars
- Setup wizard script (run on first boot) that:
- Probes CPU vs GPU (reuses `hardware.py`)
- Queries user for admin PIN, node name, IP
- Generates `.env` file with correct `LLAMA_SERVER_BASE`, `EMBED_URL`, etc.
- Auto-calculates `RAG_MAX_VECTORS` from available RAM: `max(1000, int(available_ram_gb * 100_000))`
- Optionally detects and configures RPC GPU offload
- Manual install docs remain alongside for bare-metal deployment
**This task is only actionable after Tasks 8–15 (RAG eviction + AMQP cluster) are complete.**
---
### B4 — RAG Corpus Management UI (Display, Edit, CRUD)
**Goal:** Provide a management interface in the UI to browse, search, edit, and delete individual entries in the Qdrant-backed RAG corpus.
**Backend — add to `routers/rag_admin.py`:**
| Method | Endpoint | Description | Auth |
|--------|----------|-------------|------|
| GET | `/api/rag/points` | Return paginated list of RAG points with payload (text, source, date). Supports `?offset=0&limit=50&search=` query params | Admin |
| GET | `/api/rag/point/{point_id}` | Return a single point with full payload | Admin |
| DELETE | `/api/rag/point/{point_id}` | Delete a single point from Qdrant | Admin |
| PATCH | `/api/rag/point/{point_id}` | Update a point's text payload (re-embed the new text) | Admin |
Helper functions for Qdrant scroll/delete/update go in `rag.py` or `eviction.py`.
**Frontend — add to `templates/index.html`:**
A "RAG" button in the admin UI (drawer or settings modal) that opens a management panel:
- **Stats bar**: vector count, max vectors, percent full, pinned sources
- **Search bar**: text input to search the RAG corpus by semantic similarity
- **Results table**: paginated list showing each vector's text snippet, source label, ingest date, retrieval count
- Click to expand full text
- Delete button per row (with confirmation)
- Edit button per row (inline text edit → re-embed on save)
- **Bulk actions**: flush all (existing `/api/rag/flush`) with confirmation
**Tests:**
- `tests/test_rag_admin.py` — cover new endpoints: list, get, delete, update, admin-enforcement
- Mock all Qdrant calls via monkeypatch
Run full test suite. All existing tests must continue to pass.**
amqp.py
-141
View File
@@ -1,141 +0,0 @@
"""
cAIc — AMQP connection manager.
Single persistent aio-pika connection with auto-reconnect.
"""
import asyncio
import json
import logging
try:
import aio_pika
from aio_pika import DeliveryMode, ExchangeType
from aio_pika import RobustConnection, RobustChannel
HAS_AIO_PIKA = True
except ImportError:
HAS_AIO_PIKA = False
from config import (
AMQP_RECONNECT_DELAY,
AMQP_EXCHANGE_ADMIN,
AMQP_EXCHANGE_SYSTEM,
get_amqp_url,
)
log = logging.getLogger("caic")
_connection = None
_channel = None
_lock = asyncio.Lock()
_subscriptions = [] # list of (exchange, routing_keys, handler_fn)
async def subscribe(exchange: str, routing_keys: list[str], handler) -> None:
"""Subscribe to routing keys on an exchange.
Creates an exclusive anonymous queue bound to the given routing keys.
Handler receives (exchange, routing_key, payload_dict).
On reconnect: _rebind_subscriptions recreates all subscriptions.
"""
if not HAS_AIO_PIKA:
log.warning("aio-pika not installed — cannot subscribe")
return
ch = await get_channel()
if ch is None:
log.error("cannot subscribe — no AMQP channel")
return
try:
queue = await ch.declare_queue("", exclusive=True)
ex = await ch.get_exchange(exchange)
for rk in routing_keys:
await queue.bind(ex, rk)
async def _dispatch(msg: aio_pika.IncomingMessage):
async with msg.process():
try:
payload = json.loads(msg.body.decode())
await handler(exchange, msg.routing_key, payload)
except Exception:
log.exception("AMQP handler error for %s %s", exchange, msg.routing_key)
await queue.consume(_dispatch)
_subscriptions.append((exchange, routing_keys, handler))
except Exception:
log.exception("AMQP subscribe failed for %s %s", exchange, routing_keys)
async def _rebind_subscriptions() -> None:
"""Re-create all subscriptions after reconnect."""
subs = list(_subscriptions)
_subscriptions.clear()
for exchange, routing_keys, handler in subs:
await subscribe(exchange, routing_keys, handler)
if subs:
log.info("AMQP subscriptions rebound")
async def connect() -> None:
if not HAS_AIO_PIKA:
log.warning("aio-pika not installed — AMQP disabled")
return
async with _lock:
global _connection, _channel
if _connection and not _connection.is_closed:
return
url = get_amqp_url()
log.info("connecting to AMQP broker")
try:
conn = await aio_pika.connect_robust(url)
except Exception as exc:
log.warning("AMQP connection failed — %s", exc)
return
ch = await conn.channel()
for ex in (AMQP_EXCHANGE_ADMIN, AMQP_EXCHANGE_SYSTEM):
await ch.declare_exchange(ex, ExchangeType.TOPIC, durable=True)
_connection = conn
_channel = ch
await _rebind_subscriptions()
log.info("AMQP connected, exchanges declared")
async def disconnect() -> None:
if not HAS_AIO_PIKA:
return
async with _lock:
global _connection, _channel
if _channel and not _channel.is_closed:
await _channel.close()
if _connection and not _connection.is_closed:
await _connection.close()
_channel = None
_connection = None
log.info("AMQP disconnected")
async def get_channel():
if not HAS_AIO_PIKA:
return None
if _channel is None or _channel.is_closed:
log.warning("AMQP channel missing, attempting reconnect")
try:
await connect()
except Exception:
log.exception("AMQP reconnect failed")
return None
return _channel
async def publish(exchange: str, routing_key: str, payload: dict) -> None:
if not HAS_AIO_PIKA:
log.error("cannot publish — aio-pika not installed")
return
ch = await get_channel()
if ch is None:
log.error("cannot publish — no AMQP channel available")
return
try:
body = json.dumps(payload).encode()
msg = aio_pika.Message(body, delivery_mode=DeliveryMode.PERSISTENT)
ex = await ch.get_exchange(exchange)
await ex.publish(msg, routing_key)
except Exception:
log.exception("AMQP publish failed")
app.py
+2253 -95
View File
File diff suppressed because it is too large Load Diff
app.py.bak
+45 -94
View File
@@ -1,6 +1,6 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
""" """
JarvisChat - Privacy-First Homelab Developer Platform JarvisChat - Lightweight Ollama Coding Companion
A minimal replacement for Open-WebUI that actually runs on Python 3.13 A minimal replacement for Open-WebUI that actually runs on Python 3.13
Talks to Ollama API on localhost:11434 Talks to Ollama API on localhost:11434
@@ -56,9 +56,8 @@ syslog_handler.setFormatter(
log.addHandler(syslog_handler) log.addHandler(syslog_handler)
# --- Configuration --- # --- Configuration ---
VERSION = "v1.8.0" VERSION = "1.7.6"
OLLAMA_BASE = os.environ.get("OLLAMA_BASE", "http://localhost:11434") OLLAMA_BASE = os.environ.get("OLLAMA_BASE", "http://localhost:11434")
LLAMA_SERVER_BASE = os.environ.get("LLAMA_SERVER_BASE", "http://192.168.50.108:8081")
SEARXNG_BASE = "http://localhost:8888" SEARXNG_BASE = "http://localhost:8888"
BASE_DIR = Path(__file__).parent BASE_DIR = Path(__file__).parent
DB_PATH = BASE_DIR / "jarvischat.db" DB_PATH = BASE_DIR / "jarvischat.db"
@@ -1039,7 +1038,7 @@ def get_gpu_stats() -> dict:
@asynccontextmanager @asynccontextmanager
async def lifespan(app: FastAPI): async def lifespan(app: FastAPI):
log.info(f"JarvisChat v{VERSION} starting up") log.info(f"JarvisChat v{VERSION} starting up")
log.info(f"Ollama: {OLLAMA_BASE}, llama-server: {LLAMA_SERVER_BASE}, SearXNG: {SEARXNG_BASE}") log.info(f"Ollama: {OLLAMA_BASE}, SearXNG: {SEARXNG_BASE}")
init_db() init_db()
log.info(f"Memory system: {get_memory_count()} memories loaded") log.info(f"Memory system: {get_memory_count()} memories loaded")
yield yield
@@ -1967,18 +1966,22 @@ async def explicit_search(request: Request):
try: try:
async with client.stream( async with client.stream(
"POST", "POST",
f"{LLAMA_SERVER_BASE}/v1/chat/completions", f"{OLLAMA_BASE}/api/chat",
json={"model": model, "messages": messages, "stream": True}, json={"model": model, "messages": messages, "stream": True},
timeout=httpx.Timeout(300.0, connect=10.0), timeout=httpx.Timeout(300.0, connect=10.0),
) as resp: ) as resp:
async for line in resp.aiter_lines(): async for line in resp.aiter_lines():
if line.strip(): if line.strip():
token, done, _ = parse_llama_stream_chunk(line) try:
if token: chunk = json.loads(line)
if "message" in chunk and "content" in chunk["message"]:
token = chunk["message"]["content"]
full_response.append(token) full_response.append(token)
yield f"data: {json.dumps({'token': token, 'conversation_id': conv_id})}\n\n" yield f"data: {json.dumps({'token': token, 'conversation_id': conv_id})}\n\n"
if done: if chunk.get("done"):
break break
except json.JSONDecodeError:
pass
except Exception as e: except Exception as e:
incident_key = log_incident( incident_key = log_incident(
"search_summarization_stream", "search_summarization_stream",
@@ -2017,32 +2020,7 @@ async def explicit_search(request: Request):
# ============================================================================= # =============================================================================
def build_system_prompt(db, extra_prompt="", user_message=""):
async def query_rag(query: str, limit: int = 3) -> list[dict]:
"""Query Qdrant for semantically relevant chunks."""
try:
async with httpx.AsyncClient() as client:
embed_resp = await client.post(
"http://192.168.50.108:11434/api/embeddings",
json={"model": "mxbai-embed-large", "prompt": query},
timeout=10.0,
)
if embed_resp.status_code != 200:
return []
vector = embed_resp.json()["embedding"]
search_resp = await client.post(
"http://192.168.50.108:6333/collections/jarvis_rag/points/search",
json={"vector": vector, "limit": limit, "with_payload": True},
timeout=10.0,
)
if search_resp.status_code != 200:
return []
return search_resp.json().get("result", [])
except Exception as e:
log.warning(f"RAG query error: {e}")
return []
async def build_system_prompt(db, extra_prompt="", user_message=""):
"""Build the full system prompt: profile + memories + preset.""" """Build the full system prompt: profile + memories + preset."""
parts = [] parts = []
settings = { settings = {
@@ -2062,17 +2040,6 @@ async def build_system_prompt(db, extra_prompt="", user_message=""):
parts.append("## Relevant Context from Memory\n" + "\n".join(memory_lines)) parts.append("## Relevant Context from Memory\n" + "\n".join(memory_lines))
log.debug(f"Injected {len(memories)} memories into context") log.debug(f"Injected {len(memories)} memories into context")
if user_message:
try:
rag_results = await query_rag(user_message)
if rag_results:
rag_lines = [r["payload"]["text"] for r in rag_results if r["score"] > 0.25]
if rag_lines:
parts.append("## Retrieved Context\n" + "\n\n---\n\n".join(rag_lines))
log.warning(f"RAG injected {len(rag_lines)} chunks into context")
except Exception as e:
log.warning(f"RAG injection error: {e}")
if settings.get("skills_enabled", "true") == "true": if settings.get("skills_enabled", "true") == "true":
active_skills = [s for s in list_skills_with_state(db) if s["enabled"]] active_skills = [s for s in list_skills_with_state(db) if s["enabled"]]
if active_skills: if active_skills:
@@ -2084,42 +2051,6 @@ async def build_system_prompt(db, extra_prompt="", user_message=""):
return "\n\n---\n\n".join(parts) if parts else "" return "\n\n---\n\n".join(parts) if parts else ""
def parse_llama_stream_chunk(line: str) -> tuple[str | None, bool, dict]:
"""Parse OpenAI-compatible SSE chunk. Returns (token, is_done, stats)."""
if line.startswith("data: "):
line = line[6:]
if line.strip() == "[DONE]":
return None, True, {}
try:
chunk = json.loads(line)
# OpenAI format
choices = chunk.get("choices", [])
if choices:
delta = choices[0].get("delta", {})
token = delta.get("content")
finish = choices[0].get("finish_reason")
stats = {}
if finish == "stop":
usage = chunk.get("usage", {})
stats["tokens_per_sec"] = usage.get("tokens_per_second", 0.0)
return token, finish == "stop", stats
# Ollama format fallback
if "message" in chunk and "content" in chunk["message"]:
token = chunk["message"]["content"]
done = chunk.get("done", False)
stats = {}
if done:
eval_count = chunk.get("eval_count", 0)
eval_duration = chunk.get("eval_duration", 0)
stats["tokens_per_sec"] = (
(eval_count / (eval_duration / 1e9)) if eval_duration > 0 else 0
)
return token, done, stats
except json.JSONDecodeError:
pass
return None, False, {}
@app.post("/api/chat") @app.post("/api/chat")
async def chat(request: Request): async def chat(request: Request):
body = await read_json_body(request, BODY_LIMIT_CHAT_BYTES) body = await read_json_body(request, BODY_LIMIT_CHAT_BYTES)
@@ -2165,7 +2096,7 @@ async def chat(request: Request):
"SELECT role, content FROM messages WHERE conversation_id = ? ORDER BY id ASC", "SELECT role, content FROM messages WHERE conversation_id = ? ORDER BY id ASC",
(conv_id,), (conv_id,),
).fetchall() ).fetchall()
system_prompt = await build_system_prompt(db, preset_prompt, user_message) system_prompt = build_system_prompt(db, preset_prompt, user_message)
db.close() db.close()
messages = [] messages = []
@@ -2178,6 +2109,7 @@ async def chat(request: Request):
"model": model, "model": model,
"messages": messages, "messages": messages,
"stream": True, "stream": True,
"logprobs": True,
} }
async def stream_response(): async def stream_response():
@@ -2192,18 +2124,31 @@ async def chat(request: Request):
try: try:
async with client.stream( async with client.stream(
"POST", "POST",
f"{LLAMA_SERVER_BASE}/v1/chat/completions", f"{OLLAMA_BASE}/api/chat",
json=ollama_payload, json=ollama_payload,
timeout=httpx.Timeout(300.0, connect=10.0), timeout=httpx.Timeout(300.0, connect=10.0),
) as resp: ) as resp:
async for line in resp.aiter_lines(): async for line in resp.aiter_lines():
if line.strip(): if line.strip():
token, done, stats = parse_llama_stream_chunk(line) try:
if token: chunk = json.loads(line)
if "message" in chunk and "content" in chunk["message"]:
token = chunk["message"]["content"]
full_response.append(token) full_response.append(token)
yield f"data: {json.dumps({'token': token, 'conversation_id': conv_id})}\n\n" yield f"data: {json.dumps({'token': token, 'conversation_id': conv_id})}\n\n"
if done: if "logprobs" in chunk and chunk["logprobs"]:
tokens_per_sec = stats.get("tokens_per_sec", 0.0) all_logprobs.extend(chunk["logprobs"])
if chunk.get("done"):
eval_count = chunk.get("eval_count", 0)
eval_duration = chunk.get("eval_duration", 0)
tokens_per_sec = (
(eval_count / (eval_duration / 1e9))
if eval_duration > 0
else 0
)
break
except json.JSONDecodeError:
pass
assistant_msg = "".join(full_response) assistant_msg = "".join(full_response)
perplexity = calculate_perplexity(all_logprobs) if all_logprobs else 0.0 perplexity = calculate_perplexity(all_logprobs) if all_logprobs else 0.0
@@ -2241,7 +2186,7 @@ async def chat(request: Request):
augmented_response = [] augmented_response = []
async with client.stream( async with client.stream(
"POST", "POST",
f"{LLAMA_SERVER_BASE}/v1/chat/completions", f"{OLLAMA_BASE}/api/chat",
json={ json={
"model": model, "model": model,
"messages": augmented_messages, "messages": augmented_messages,
@@ -2251,11 +2196,19 @@ async def chat(request: Request):
) as resp2: ) as resp2:
async for line in resp2.aiter_lines(): async for line in resp2.aiter_lines():
if line.strip(): if line.strip():
token2, done2, _ = parse_llama_stream_chunk(line) try:
if token2: chunk = json.loads(line)
augmented_response.append(token2) if (
if done2: "message" in chunk
and "content" in chunk["message"]
):
augmented_response.append(
chunk["message"]["content"]
)
if chunk.get("done"):
break break
except json.JSONDecodeError:
pass
raw_response = "".join(augmented_response) or assistant_msg raw_response = "".join(augmented_response) or assistant_msg
cleaned_response = clean_hedging(raw_response) cleaned_response = clean_hedging(raw_response)
@@ -2308,8 +2261,6 @@ async def chat(request: Request):
yield f"data: {json.dumps({'done': True, 'conversation_id': conv_id, 'perplexity': round(perplexity, 2), 'tokens_per_sec': round(tokens_per_sec, 1)})}\n\n" yield f"data: {json.dumps({'done': True, 'conversation_id': conv_id, 'perplexity': round(perplexity, 2), 'tokens_per_sec': round(tokens_per_sec, 1)})}\n\n"
except httpx.RemoteProtocolError:
pass # llama-server closes connection after [DONE] — normal
except httpx.ConnectError: except httpx.ConnectError:
yield f"data: {json.dumps({'error': 'Cannot connect to Ollama. Is it running?'})}\n\n" yield f"data: {json.dumps({'error': 'Cannot connect to Ollama. Is it running?'})}\n\n"
except Exception as e: except Exception as e:
app.py.pre-refactor-20260616-081744
-2334
View File
File diff suppressed because it is too large Load Diff
auth.py
-202
View File
@@ -1,202 +0,0 @@
"""
cAIc - Auth: session management, PIN verification, middleware, auth routes.
"""
import hashlib
import hmac
import logging
import re
import time
import uuid
from typing import Optional
from fastapi import APIRouter, HTTPException, Request
from fastapi.responses import JSONResponse
from config import SESSION_TIMEOUT_SECONDS, MAX_PIN_ATTEMPTS, PIN_LOCKOUT_SECONDS, RATE_WINDOW_SECONDS
from db import get_db, get_setting
from security import (
SESSIONS, PIN_ATTEMPTS, SESSION_LOCK, BODY_LIMIT_DEFAULT_BYTES,
audit_event, get_client_ip, is_ip_allowed, check_rate_limit,
rate_policy, origin_allowed, is_state_changing, request_body_limit,
read_json_body, hash_pin, customer_error_envelope, log_incident,
)
log = logging.getLogger("caic")
router = APIRouter()
def verify_admin_pin(pin: str) -> bool:
if not re.fullmatch(r"\d{4}", pin or ""):
return False
db = get_db()
pin_hash = get_setting(db, "admin_pin_hash", "")
pin_salt = get_setting(db, "admin_pin_salt", "")
db.close()
if not pin_hash or not pin_salt:
return False
_, candidate_hash = hash_pin(pin, salt_hex=pin_salt)
return hmac.compare_digest(candidate_hash, pin_hash)
def is_ip_locked(ip: str) -> tuple:
now_ts = time.time()
with SESSION_LOCK:
state = PIN_ATTEMPTS.get(ip)
if not state:
return False, 0
locked_until = state.get("locked_until", 0)
if locked_until > now_ts:
return True, int(locked_until - now_ts)
if locked_until:
PIN_ATTEMPTS.pop(ip, None)
return False, 0
def record_pin_failure(ip: str) -> None:
now_ts = time.time()
with SESSION_LOCK:
state = PIN_ATTEMPTS.get(ip, {"fail_count": 0, "locked_until": 0})
state["fail_count"] = int(state.get("fail_count", 0)) + 1
if state["fail_count"] >= MAX_PIN_ATTEMPTS:
state["locked_until"] = now_ts + PIN_LOCKOUT_SECONDS
state["fail_count"] = 0
PIN_ATTEMPTS[ip] = state
def clear_pin_failures(ip: str) -> None:
with SESSION_LOCK:
PIN_ATTEMPTS.pop(ip, None)
def cleanup_sessions(now_ts: Optional[float] = None) -> None:
now_ts = now_ts or time.time()
with SESSION_LOCK:
expired = [
sid for sid, meta in SESSIONS.items()
if (now_ts - meta.get("last_seen", 0)) > SESSION_TIMEOUT_SECONDS
]
for sid in expired:
del SESSIONS[sid]
def create_session(ip: str, role: str) -> str:
now_ts = time.time()
sid = uuid.uuid4().hex
with SESSION_LOCK:
SESSIONS[sid] = {"ip": ip, "role": role, "created_at": now_ts, "last_seen": now_ts}
return sid
def validate_session(sid: str, ip: str, touch: bool = True) -> bool:
if not sid:
return False
now_ts = time.time()
cleanup_sessions(now_ts)
with SESSION_LOCK:
session = SESSIONS.get(sid)
if not session or session.get("ip") != ip:
return False
if touch:
session["last_seen"] = now_ts
return True
def get_session(sid: str, ip: str, touch: bool = True) -> Optional[dict]:
if not sid:
return None
now_ts = time.time()
cleanup_sessions(now_ts)
with SESSION_LOCK:
session = SESSIONS.get(sid)
if not session or session.get("ip") != ip:
return None
if touch:
session["last_seen"] = now_ts
return dict(session)
def revoke_session(sid: str) -> None:
if not sid:
return
with SESSION_LOCK:
SESSIONS.pop(sid, None)
def is_admin_only(path: str, method: str) -> bool:
if method in {"PUT", "DELETE", "PATCH"}:
return True
if method != "POST":
return False
guest_allowed_posts = {
"/api/chat", "/api/search", "/api/show", "/api/auth/login",
"/api/auth/logout", "/api/auth/session", "/api/auth/heartbeat", "/api/auth/guest",
}
return path not in guest_allowed_posts
# --- Auth routes ---
@router.post("/api/auth/guest")
async def auth_guest(request: Request):
ip = get_client_ip(request)
sid = create_session(ip, role="guest")
audit_event("guest_session", "success", ip=ip, role="guest")
return {"status": "ok", "session_id": sid, "role": "guest", "timeout_seconds": SESSION_TIMEOUT_SECONDS}
@router.post("/api/auth/login")
async def auth_login(request: Request):
body = await read_json_body(request, BODY_LIMIT_DEFAULT_BYTES)
pin = str(body.get("pin", ""))
ip = get_client_ip(request)
locked, retry_after = is_ip_locked(ip)
if locked:
audit_event("admin_login", "locked", ip=ip, role="none", details=f"retry_after={retry_after}", warning=True)
raise HTTPException(status_code=429, detail=f"Too many failed PIN attempts. Retry in {retry_after}s.")
if not verify_admin_pin(pin):
record_pin_failure(ip)
audit_event("admin_login", "failed", ip=ip, role="none", warning=True)
raise HTTPException(status_code=401, detail="Invalid PIN")
clear_pin_failures(ip)
sid = create_session(ip, role="admin")
audit_event("admin_login", "success", ip=ip, role="admin")
return {"status": "ok", "session_id": sid, "role": "admin", "timeout_seconds": SESSION_TIMEOUT_SECONDS}
@router.get("/api/auth/session")
async def auth_session(request: Request):
sid = request.headers.get("x-session-id", "").strip()
ip = get_client_ip(request)
session = get_session(sid, ip, touch=True)
return {"authenticated": bool(session), "role": session.get("role") if session else "none"}
@router.post("/api/auth/heartbeat")
async def auth_heartbeat(request: Request):
sid = request.headers.get("x-session-id", "").strip()
ip = get_client_ip(request)
if not sid or not validate_session(sid, ip, touch=True):
raise HTTPException(status_code=401, detail="Authentication required")
return {"status": "ok"}
@router.post("/api/auth/logout")
async def auth_logout(request: Request):
ip = get_client_ip(request)
sid = request.headers.get("x-session-id", "").strip()
role = "none"
if sid:
session = get_session(sid, ip, touch=False)
role = session.get("role", "none") if session else "none"
if not sid:
try:
body = await read_json_body(request, BODY_LIMIT_DEFAULT_BYTES)
sid = str(body.get("session_id", "")).strip()
except Exception:
try:
sid = (await request.body()).decode("utf-8", errors="ignore").strip()
except Exception:
sid = ""
revoke_session(sid)
audit_event("logout", "success", ip=ip, role=role)
return {"status": "ok"}
cluster.py
-257
View File
@@ -1,257 +0,0 @@
"""
cAIc — Cluster protocol implementation.
Maintains node registry, event log, coordinator state, and ping-based health checks.
"""
import asyncio
import logging
import uuid
from collections import deque
from datetime import datetime, timezone
from amqp import publish, subscribe
from config import AMQP_EXCHANGE_ADMIN, AMQP_EXCHANGE_SYSTEM
log = logging.getLogger("caic")
CLUSTER_NODES: dict[str, dict] = {}
CLUSTER_EVENTS: deque = deque(maxlen=1000)
CLUSTER_COORDINATOR: str | None = None
_pending_pings: dict[str, tuple[str, asyncio.Event]] = {}
NODE_NAME: str = "ultron"
PING_TIMEOUT: float = 5.0
def _push_event(category: str, severity: str, node_name: str | None, message: str, details: dict | None = None) -> dict:
record = {
"category": category,
"severity": severity,
"node": node_name,
"message": message,
"details": details or {},
"timestamp": datetime.now(timezone.utc).isoformat() + "Z",
}
CLUSTER_EVENTS.append(record)
level = {"info": logging.INFO, "warn": logging.WARNING, "error": logging.ERROR, "critical": logging.CRITICAL}.get(severity, logging.INFO)
log.log(level, "[cluster] %s: %s", node_name or "system", message)
return record
async def handle_registration(exchange: str, routing_key: str, payload: dict) -> None:
global CLUSTER_COORDINATOR
node_name = payload.get("node_name", routing_key.split(".")[1] if "." in routing_key else "unknown")
node_type = payload.get("node_type", "worker")
if node_name in CLUSTER_NODES:
_push_event("cluster", "warn", node_name, "Duplicate registration rejected")
await publish(AMQP_EXCHANGE_ADMIN, f"node.{node_name}.rejected", {
"from": NODE_NAME, "node_name": node_name, "type": "rejected",
"reason": "duplicate_node_name",
"timestamp": datetime.now(timezone.utc).isoformat() + "Z",
})
return
if "node_type" not in payload or "capabilities" not in payload:
_push_event("cluster", "warn", node_name, "Malformed registration rejected")
await publish(AMQP_EXCHANGE_ADMIN, f"node.{node_name}.rejected", {
"from": NODE_NAME, "node_name": node_name, "type": "rejected",
"reason": "malformed_payload",
"timestamp": datetime.now(timezone.utc).isoformat() + "Z",
})
return
now = datetime.now(timezone.utc).isoformat() + "Z"
node = {
"name": node_name,
"type": node_type,
"status": "active",
"ip": payload.get("ip"),
"capabilities": payload.get("capabilities", []),
"active_model": payload.get("active_model"),
"inventory": payload.get("inventory", []),
"load": payload.get("load"),
"registered_at": now,
"last_seen": now,
}
CLUSTER_NODES[node_name] = node
_push_event("cluster", "info", node_name, f"Node registered (type={node_type})")
await publish(AMQP_EXCHANGE_ADMIN, f"node.{node_name}.admitted", {
"from": NODE_NAME, "node_name": node_name, "type": "admitted",
"coordinator": CLUSTER_COORDINATOR,
"timestamp": now,
})
if node_type == "coordinator" and CLUSTER_COORDINATOR is None:
CLUSTER_COORDINATOR = node_name
_push_event("cluster", "info", node_name, "Elected as coordinator")
await publish(AMQP_EXCHANGE_SYSTEM, "cluster.coordinator.response", {
"from": NODE_NAME, "type": "coord_response",
"coordinator": node_name, "nodes": list(CLUSTER_NODES.keys()),
"timestamp": now,
})
async def handle_deregistration(exchange: str, routing_key: str, payload: dict) -> None:
global CLUSTER_COORDINATOR
node_name = payload.get("node_name", routing_key.split(".")[1] if "." in routing_key else "unknown")
removed = CLUSTER_NODES.pop(node_name, None)
if not removed:
log.warning("[cluster] deregistration for unknown node %s", node_name)
return
_push_event("cluster", "info", node_name, "Node deregistered")
if CLUSTER_COORDINATOR == node_name:
CLUSTER_COORDINATOR = None
_push_event("cluster", "warn", node_name, "Coordinator deregistered — no coordinator active")
async def handle_pong(exchange: str, routing_key: str, payload: dict) -> None:
node_name = payload.get("node_name", routing_key.split(".")[1] if "." in routing_key else "unknown")
correlation_id = payload.get("correlation_id")
if correlation_id and correlation_id in _pending_pings:
_, event = _pending_pings.pop(correlation_id)
event.set()
if node_name in CLUSTER_NODES:
now = datetime.now(timezone.utc).isoformat() + "Z"
CLUSTER_NODES[node_name]["last_seen"] = now
if "status" in payload:
CLUSTER_NODES[node_name]["status"] = payload["status"]
if "active_model" in payload:
CLUSTER_NODES[node_name]["active_model"] = payload["active_model"]
if "load" in payload:
CLUSTER_NODES[node_name]["load"] = payload["load"]
else:
log.warning("[cluster] pong from unknown node %s", node_name)
async def handle_event(exchange: str, routing_key: str, payload: dict) -> None:
node_name = payload.get("node_name", routing_key.split(".")[1] if "." in routing_key else "unknown")
severity = payload.get("severity", "info")
message = payload.get("message", "")
details = payload.get("details")
_push_event("application", severity, node_name, message, details)
async def handle_coordinator_query(exchange: str, routing_key: str, payload: dict) -> None:
if CLUSTER_COORDINATOR is None:
return
now = datetime.now(timezone.utc).isoformat() + "Z"
await publish(AMQP_EXCHANGE_SYSTEM, "cluster.coordinator.response", {
"from": NODE_NAME, "type": "coord_response",
"coordinator": CLUSTER_COORDINATOR,
"nodes": list(CLUSTER_NODES.keys()),
"timestamp": now,
})
async def ping_node(node_name: str) -> bool:
global CLUSTER_COORDINATOR
if node_name not in CLUSTER_NODES:
return False
correlation_id = str(uuid.uuid4())
event = asyncio.Event()
_pending_pings[correlation_id] = (node_name, event)
now = datetime.now(timezone.utc).isoformat() + "Z"
await publish(AMQP_EXCHANGE_ADMIN, f"node.{node_name}.ping", {
"from": NODE_NAME, "node_name": node_name, "type": "ping",
"correlation_id": correlation_id, "timestamp": now,
})
try:
await asyncio.wait_for(event.wait(), timeout=PING_TIMEOUT)
return True
except asyncio.TimeoutError:
_pending_pings.pop(correlation_id, None)
CLUSTER_NODES.pop(node_name, None)
_push_event("cluster", "warn", node_name, "Node unresponsive — deregistered after ping timeout")
if CLUSTER_COORDINATOR == node_name:
CLUSTER_COORDINATOR = None
_push_event("cluster", "warn", node_name, "Coordinator unresponsive — no coordinator active")
return False
async def request_model_swap(node_name: str, model_filename: str) -> bool:
"""Request a worker node to swap its active model."""
if node_name not in CLUSTER_NODES:
log.warning("request_model_swap: unknown node %s", node_name)
return False
CLUSTER_NODES[node_name]["status"] = "swapping"
_push_event("cluster", "info", node_name, f"Model swap requested: {model_filename}")
now = datetime.now(timezone.utc).isoformat() + "Z"
await publish(AMQP_EXCHANGE_ADMIN, f"node.{node_name}.cmd.swap_model", {
"model_filename": model_filename,
"requested_at": now,
})
return True
async def handle_model_ready(exchange: str, routing_key: str, payload: dict) -> None:
node_name = payload.get("node_name", routing_key.split(".")[1] if "." in routing_key else "unknown")
model_filename = payload.get("active_model")
port = payload.get("port", 8081)
if node_name not in CLUSTER_NODES:
log.warning("handle_model_ready: unknown node %s", node_name)
return
inventory = CLUSTER_NODES[node_name].get("inventory") or []
model_info = None
for inv in inventory:
if inv.get("filename") == model_filename:
model_info = {**inv, "port": port}
break
if model_info is None:
model_info = {"filename": model_filename, "port": port}
now = datetime.now(timezone.utc).isoformat() + "Z"
CLUSTER_NODES[node_name]["active_model"] = model_info
CLUSTER_NODES[node_name]["status"] = "active"
CLUSTER_NODES[node_name]["last_seen"] = now
_push_event("cluster", "info", node_name, f"Model swap complete: {model_filename}")
async def handle_heartbeat(exchange: str, routing_key: str, payload: dict) -> None:
node_name = payload.get("node_name", routing_key.split(".")[1] if "." in routing_key else "unknown")
if node_name not in CLUSTER_NODES:
log.warning("heartbeat from unknown node %s — ignore, full registration required", node_name)
return
now = datetime.now(timezone.utc).isoformat() + "Z"
CLUSTER_NODES[node_name]["last_seen"] = now
async def handle_model_failed(exchange: str, routing_key: str, payload: dict) -> None:
node_name = payload.get("node_name", routing_key.split(".")[1] if "." in routing_key else "unknown")
error = payload.get("error", "unknown error")
if node_name not in CLUSTER_NODES:
log.warning("handle_model_failed: unknown node %s", node_name)
return
CLUSTER_NODES[node_name]["status"] = "error"
_push_event("cluster", "error", node_name, f"Model swap failed: {error}")
SUBSCRIBE_TABLE = [
(AMQP_EXCHANGE_ADMIN, ["node.*.register"], handle_registration),
(AMQP_EXCHANGE_ADMIN, ["node.*.deregister"], handle_deregistration),
(AMQP_EXCHANGE_ADMIN, ["node.*.pong"], handle_pong),
(AMQP_EXCHANGE_SYSTEM, ["node.*.event"], handle_event),
(AMQP_EXCHANGE_SYSTEM, ["cluster.coordinator.query"], handle_coordinator_query),
(AMQP_EXCHANGE_SYSTEM, ["node.*.heartbeat"], handle_heartbeat),
(AMQP_EXCHANGE_SYSTEM, ["node.*.model_ready"], handle_model_ready),
(AMQP_EXCHANGE_SYSTEM, ["node.*.model_failed"], handle_model_failed),
]
async def start_cluster_subscriptions() -> None:
for exchange, routing_keys, handler in SUBSCRIBE_TABLE:
await subscribe(exchange, routing_keys, handler)
log.info("cluster subscriptions started")
config.py
-202
View File
@@ -1,202 +0,0 @@
"""
cAIc - Central configuration.
All constants, environment variables, limits, and skill registry live here.
"""
import os
import re
import ipaddress
import logging
log = logging.getLogger("caic")
VERSION = "v0.18.0"
OLLAMA_BASE = os.environ.get("OLLAMA_BASE", "http://localhost:11434")
LLAMA_SERVER_BASE = os.environ.get("LLAMA_SERVER_BASE", "http://192.168.50.108:8081")
SEARXNG_BASE = "http://localhost:8888"
DEFAULT_MODEL = "qwen2.5-7b-instruct"
COMPLETIONS_API_KEY = os.environ.get("CAIC_COMPLETIONS_API_KEY", "caic-sk-" + os.urandom(24).hex())
MODEL_CONTEXT_LENGTH = 4096
# --- AMQP ---
AMQP_RECONNECT_DELAY = 5
AMQP_EXCHANGE_ADMIN = "jc.admin"
AMQP_EXCHANGE_SYSTEM = "jc.system"
AMQP_SECRET_PATH = "/home/gramps/.caic_amqp_secret"
def get_amqp_url() -> str:
url = os.environ.get("CAIC_AMQP_URL")
if url:
return url
try:
with open(AMQP_SECRET_PATH) as f:
pw = f.read().strip()
except (FileNotFoundError, OSError):
pw = "password"
return f"amqp://caic:{pw}@localhost:5672/caic"
# --- Auth ---
SESSION_TIMEOUT_SECONDS = 3600
MAX_PIN_ATTEMPTS = 5
PIN_LOCKOUT_SECONDS = 300
ALLOW_DEFAULT_PIN = os.getenv("CAIC_ALLOW_DEFAULT_PIN", "false").lower() == "true"
TRUSTED_ORIGINS = {
origin.strip().rstrip("/")
for origin in os.getenv("CAIC_TRUSTED_ORIGINS", "").split(",")
if origin.strip()
}
DEFAULT_ALLOWED_CIDRS = "127.0.0.0/8,::1/128,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16"
ALLOWED_CIDRS_RAW = os.getenv("CAIC_ALLOWED_CIDRS", DEFAULT_ALLOWED_CIDRS)
TRUST_X_FORWARDED_FOR = (
os.getenv("CAIC_TRUST_X_FORWARDED_FOR", "false").lower() == "true"
)
# --- Rate limits ---
RATE_WINDOW_SECONDS = 60
RL_LOGIN_PER_WINDOW = 10
RL_CHAT_PER_WINDOW = 24
RL_SEARCH_PER_WINDOW = 16
RL_WRITE_PER_WINDOW = 30
RL_DEFAULT_PER_WINDOW = 240
RL_STATS_PER_WINDOW = 600
# --- Payload limits ---
BODY_LIMIT_DEFAULT_BYTES = 64 * 1024
BODY_LIMIT_CHAT_BYTES = 128 * 1024
BODY_LIMIT_PROFILE_BYTES = 256 * 1024
# --- Upload ---
UPLOAD_DIR = "/tmp/caic_uploads"
MAX_UPLOAD_BYTES = 20 * 1024 * 1024
SUPPORTED_UPLOAD_TYPES = {"text/plain", "text/markdown", "application/pdf", "application/json", "text/x-python", "text/html", "image/png", "image/jpeg", "image/gif", "image/svg+xml", "image/webp"}
QDRANT_URL = "http://192.168.50.108:6333"
RAG_COLLECTION = "caic_rag"
UPLOAD_CONTEXT_EXPIRY_HOURS = 1
BODY_LIMIT_UPLOAD_BYTES = MAX_UPLOAD_BYTES
# --- RAG eviction ---
RAG_MAX_VECTORS = 50000
RAG_EVICTION_HIGH_WATER = 0.80
RAG_EVICTION_LOW_WATER = 0.20
RAG_EVICTION_BATCH = 1000
RAG_PINNED_SOURCES = ["upload", "profile"]
RAG_GRACE_HOURS = 1
RAG_ACCESS_WEIGHT = 1.0
RAG_AGE_WEIGHT = 0.1
MAX_CHAT_MESSAGE_CHARS = 8000
MAX_SEARCH_QUERY_CHARS = 500
MAX_PROFILE_CHARS = 32000
MAX_MEMORY_FACT_CHARS = 2000
MAX_PRESET_NAME_CHARS = 120
MAX_PRESET_PROMPT_CHARS = 12000
MAX_SETTINGS_KEYS = 16
MAX_SETTINGS_VALUE_CHARS = 8000
MAX_CONVERSATION_TITLE_CHARS = 200
MAX_SKILL_KEY_CHARS = 120
MAX_SKILL_PROMPT_CHARS = 1600
ALLOWED_SETTINGS_KEYS = {
"profile_enabled",
"default_model",
"search_enabled",
"memory_enabled",
"skills_enabled",
}
# --- Triage / query routing ---
TRIAGE_BASE = os.environ.get("CAIC_TRIAGE_BASE", "http://127.0.0.1:8083/v1")
TRIAGE_TIMEOUT = 10
FALLBACK_TO_DEFAULT = True
# --- Perplexity ---
PERPLEXITY_THRESHOLD = 15.0
# --- Refusal / hedge patterns ---
REFUSAL_PATTERNS = re.compile(
r"|".join([
r"i don'?t have (?:real-?time|current|live)",
r"i (?:can'?t|cannot) provide (?:current|real-?time|live)",
r"i don'?t have access to (?:current|real-?time|live)",
r"(?:current|live|real-?time) (?:data|information|prices?|weather)",
r"my (?:knowledge|training) (?:cutoff|only goes|ends)",
r"as of my (?:knowledge|training) cutoff",
r"i'?m not able to (?:access|provide|browse)",
r"(?:check|visit|use) a (?:website|financial|news)",
r"as an ai model",
r"based on my training data",
r"i don'?t have the capability",
]),
re.IGNORECASE,
)
HEDGE_PATTERNS = [
r"^I'?m sorry,?\s*but\s*I\s*(?:can'?t|cannot)\s*assist\s*with\s*that[^.]*\.\s*",
r"^I'?m sorry,?\s*but[^.]*(?:previous|incorrect)[^.]*\.\s*",
r"(?:But\s+)?[Pp]lease\s+(?:make\s+sure\s+to\s+)?verify\s+(?:the\s+)?(?:data|information|this)\s+(?:from\s+)?(?:reliable\s+)?sources[^.]*\.\s*",
r"[Pp]lease\s+verify[^.]*(?:accurate|reliability)[^.]*\.\s*",
r"[Bb]ut\s+please\s+(?:make\s+sure|verify|check)[^.]*\.\s*",
]
# --- Built-in skills registry ---
BUILTIN_SKILLS = [
{"key": "memory.search", "name": "Memory Search", "category": "memory", "risk": "low", "description": "Search stored memory facts relevant to the current prompt."},
{"key": "memory.add", "name": "Memory Add", "category": "memory", "risk": "medium", "description": "Store a new memory fact with topic tagging."},
{"key": "memory.forget", "name": "Memory Forget", "category": "memory", "risk": "high", "description": "Delete matching memories when asked to forget information."},
{"key": "conversation.list", "name": "Conversation List", "category": "conversation", "risk": "low", "description": "List existing conversations with metadata."},
{"key": "conversation.get", "name": "Conversation Get", "category": "conversation", "risk": "low", "description": "Read a conversation and its message history."},
{"key": "conversation.delete", "name": "Conversation Delete", "category": "conversation", "risk": "high", "description": "Delete a single conversation thread."},
{"key": "conversation.delete_all", "name": "Conversation Delete All", "category": "conversation", "risk": "high", "description": "Delete all conversations and messages."},
{"key": "search.web", "name": "Web Search", "category": "search", "risk": "low", "description": "Run explicit web search and summarize results."},
{"key": "settings.get", "name": "Settings Get", "category": "settings", "risk": "low", "description": "Read current runtime settings."},
{"key": "settings.update", "name": "Settings Update", "category": "settings", "risk": "high", "description": "Update allowlisted runtime settings keys."},
]
SKILLS_BY_KEY = {s["key"]: s for s in BUILTIN_SKILLS}
def parse_allowed_cidrs(raw: str) -> list:
networks = []
for entry in (raw or "").split(","):
value = entry.strip()
if not value:
continue
try:
networks.append(ipaddress.ip_network(value, strict=False))
except ValueError:
log.warning(f"Invalid CIDR ignored: {value}")
return networks
ALLOWED_NETWORKS = parse_allowed_cidrs(ALLOWED_CIDRS_RAW)
DEFAULT_PROFILE = """You are a coding companion running locally on a machine called "jarvis".
## Environment
- jarvis: Debian 13 (trixie) x86_64, AMD Ryzen 5 5600X, 16GB RAM, AMD RX 6600 XT (8GB VRAM)
- ultron: Debian 13, Ryzen 7 7840HS, 16GB RAM, primary AI inference node, IP 192.168.50.108
- Corsair: Windows 11, gaming/streaming rig, RTX 5070 Ti
- pivault: RPi 5, 8GB RAM, Debian 13, 11TB RAID5 NAS at /mnt/pivault, IP 192.168.50.158
- Router: ASUS ROG Rapture GT-BE98 Pro "BigBlinkyRouter" at 192.168.50.1
- llama-server on ultron:8081 (OpenAI-compat API), Qdrant on ultron:6333
## About the User
- Experienced developer, BS in Computer Science (Oklahoma State), coding since 1981 (TRS-80)
- Deep Unix/Linux background — wrote device drivers at SCO during Xenix era (1990s)
- Currently learning Rust, transitioning from decades of PHP
- Building a WW2 mobile game in Godot Engine for Android
- Veteran on fixed income — prefers free/open-source solutions
- Home lab enthusiast with Zigbee, Z-Wave and Tapo smart home devices
## How to Respond
- Be direct and concise — no hand-holding, this user knows what they're doing
- When showing code, prefer complete working examples over snippets
- Default to command-line solutions over GUI when possible
- Consider resource constraints (fixed income, specific hardware limits)
- Use Rust, Python, or bash unless another language is specifically needed
- Explain trade-offs when multiple approaches exist"""
DEFAULT_PRESETS = [
{"name": "Coding Companion", "prompt": "You are a senior software engineer and coding companion. Focus on writing clean, efficient, well-documented code. Provide complete working examples. Explain architectural decisions and trade-offs. Prefer Rust, Python, and bash."},
{"name": "Linux Sysadmin", "prompt": "You are an experienced Linux systems administrator. Focus on command-line solutions, systemd services, networking, storage, and security. Prefer Debian/Ubuntu conventions. Be concise and direct."},
{"name": "General Assistant","prompt": "You are a helpful general-purpose assistant. Be clear and concise."},
]
db.py
-212
View File
@@ -1,212 +0,0 @@
"""
cAIc - Database layer.
Schema init, connection factory, settings helpers, skill state management.
"""
import logging
import os
import re
import sqlite3
import uuid
from datetime import datetime, timezone
from pathlib import Path
from typing import Optional
from config import (
BUILTIN_SKILLS, DEFAULT_MODEL, DEFAULT_PRESETS, DEFAULT_PROFILE,
MAX_SKILL_PROMPT_CHARS, ALLOWED_NETWORKS,
)
log = logging.getLogger("caic")
BASE_DIR = Path(__file__).parent
DB_PATH = BASE_DIR / "caic.db"
def get_db():
conn = sqlite3.connect(DB_PATH)
conn.row_factory = sqlite3.Row
conn.execute("PRAGMA foreign_keys = ON")
return conn
def get_setting(db, key: str, default: str = "") -> str:
row = db.execute("SELECT value FROM settings WHERE key = ?", (key,)).fetchone()
return row["value"] if row else default
def list_skills_with_state(db) -> list:
rows = db.execute("SELECT skill_key, enabled, updated_at FROM skills").fetchall()
state_by_key = {
row["skill_key"]: {"enabled": bool(row["enabled"]), "updated_at": row["updated_at"]}
for row in rows
}
merged = []
for skill in BUILTIN_SKILLS:
state = state_by_key.get(skill["key"], {"enabled": True, "updated_at": ""})
merged.append({**skill, "enabled": state["enabled"], "updated_at": state["updated_at"]})
return sorted(merged, key=lambda s: (s["category"], s["name"]))
def set_skill_enabled(db, skill_key: str, enabled: bool) -> None:
now = datetime.now(timezone.utc).isoformat()
db.execute(
"INSERT OR REPLACE INTO skills (skill_key, enabled, updated_at) VALUES (?, ?, ?)",
(skill_key, 1 if enabled else 0, now),
)
def format_active_skills_prompt(skills: list) -> str:
lines = [
"## Active Skills",
"Use these skills only when needed. Prefer concise answers over unnecessary tool usage.",
]
for skill in skills:
lines.append(f"- {skill['key']} ({skill['risk']} risk): {skill['description']}")
text = "\n".join(lines)
if len(text) > MAX_SKILL_PROMPT_CHARS:
return text[:MAX_SKILL_PROMPT_CHARS - 3] + "..."
return text
def insert_upload_context(db, conversation_id: str, filename: str, content: str, expires_at: str, content_type: str = "text/plain") -> int:
now = datetime.now(timezone.utc).isoformat()
cur = db.execute(
"INSERT INTO upload_context (conversation_id, filename, content, content_type, created_at, expires_at) VALUES (?, ?, ?, ?, ?, ?)",
(conversation_id, filename, content, content_type, now, expires_at),
)
return cur.lastrowid
def list_upload_context_by_conversation(db, conversation_id: str):
rows = db.execute(
"SELECT id, conversation_id, filename, content_type, created_at, expires_at FROM upload_context WHERE conversation_id = ? ORDER BY id ASC",
(conversation_id,),
).fetchall()
return [dict(r) for r in rows]
def delete_upload_context_by_id(db, context_id: int) -> bool:
cur = db.execute("DELETE FROM upload_context WHERE id = ?", (context_id,))
return cur.rowcount > 0
def get_upload_context(db, context_id: int):
row = db.execute(
"SELECT id, conversation_id, filename, content, content_type, expires_at FROM upload_context WHERE id = ?",
(context_id,),
).fetchone()
if not row:
return None
expires = datetime.fromisoformat(row["expires_at"])
if expires < datetime.now(timezone.utc):
db.execute("DELETE FROM upload_context WHERE id = ?", (context_id,))
db.commit()
return None
return dict(row)
def init_db():
from security import hash_pin
conn = sqlite3.connect(DB_PATH)
conn.row_factory = sqlite3.Row
conn.execute("""
CREATE TABLE IF NOT EXISTS conversations (
id TEXT PRIMARY KEY, title TEXT NOT NULL DEFAULT 'New Chat',
model TEXT NOT NULL, created_at TEXT NOT NULL, updated_at TEXT NOT NULL
)
""")
conn.execute("""
CREATE TABLE IF NOT EXISTS messages (
id INTEGER PRIMARY KEY AUTOINCREMENT, conversation_id TEXT NOT NULL,
role TEXT NOT NULL, content TEXT NOT NULL, created_at TEXT NOT NULL,
FOREIGN KEY (conversation_id) REFERENCES conversations(id) ON DELETE CASCADE
)
""")
conn.execute("""
CREATE TABLE IF NOT EXISTS system_presets (
id TEXT PRIMARY KEY, name TEXT NOT NULL, prompt TEXT NOT NULL,
is_default INTEGER NOT NULL DEFAULT 0, created_at TEXT NOT NULL
)
""")
conn.execute("""
CREATE TABLE IF NOT EXISTS profile (
id INTEGER PRIMARY KEY CHECK (id = 1), content TEXT NOT NULL, updated_at TEXT NOT NULL
)
""")
conn.execute("CREATE TABLE IF NOT EXISTS settings (key TEXT PRIMARY KEY, value TEXT NOT NULL)")
conn.execute("""
CREATE TABLE IF NOT EXISTS skills (
skill_key TEXT PRIMARY KEY, enabled INTEGER NOT NULL DEFAULT 1, updated_at TEXT NOT NULL
)
""")
conn.execute("""
CREATE VIRTUAL TABLE IF NOT EXISTS memories USING fts5(
fact, topic, source, created_at UNINDEXED
)
""")
conn.execute("""
CREATE TABLE IF NOT EXISTS upload_context (
id INTEGER PRIMARY KEY AUTOINCREMENT,
conversation_id TEXT,
filename TEXT NOT NULL,
content TEXT NOT NULL,
content_type TEXT DEFAULT 'text/plain',
created_at TEXT NOT NULL,
expires_at TEXT NOT NULL
)
""")
try:
conn.execute("ALTER TABLE upload_context ADD COLUMN content_type TEXT DEFAULT 'text/plain'")
except Exception:
pass
if not conn.execute("SELECT id FROM profile WHERE id = 1").fetchone():
now = datetime.now(timezone.utc).isoformat()
conn.execute("INSERT INTO profile (id, content, updated_at) VALUES (1, ?, ?)", (DEFAULT_PROFILE, now))
if conn.execute("SELECT COUNT(*) as c FROM system_presets").fetchone()["c"] == 0:
now = datetime.now(timezone.utc).isoformat()
for preset in DEFAULT_PRESETS:
conn.execute(
"INSERT INTO system_presets (id, name, prompt, is_default, created_at) VALUES (?, ?, ?, 1, ?)",
(str(uuid.uuid4()), preset["name"], preset["prompt"], now),
)
defaults = {
"profile_enabled": "true", "default_model": DEFAULT_MODEL,
"search_enabled": "true", "memory_enabled": "true", "skills_enabled": "true",
}
for key, value in defaults.items():
if not conn.execute("SELECT key FROM settings WHERE key = ?", (key,)).fetchone():
conn.execute("INSERT INTO settings (key, value) VALUES (?, ?)", (key, value))
now = datetime.now(timezone.utc).isoformat()
for skill in BUILTIN_SKILLS:
if not conn.execute("SELECT skill_key FROM skills WHERE skill_key = ?", (skill["key"],)).fetchone():
conn.execute("INSERT INTO skills (skill_key, enabled, updated_at) VALUES (?, 1, ?)", (skill["key"], now))
existing_pin_hash = conn.execute("SELECT value FROM settings WHERE key = 'admin_pin_hash'").fetchone()
existing_pin_salt = conn.execute("SELECT value FROM settings WHERE key = 'admin_pin_salt'").fetchone()
if not existing_pin_hash or not existing_pin_salt:
from config import ALLOW_DEFAULT_PIN
configured_pin = os.getenv("CAIC_ADMIN_PIN", "").strip()
if re.fullmatch(r"\d{4}", configured_pin):
seed_pin, pin_source = configured_pin, "env"
elif ALLOW_DEFAULT_PIN:
seed_pin, pin_source = "1234", "default"
else:
raise RuntimeError(
"Admin PIN bootstrap blocked: set CAIC_ADMIN_PIN to a 4-digit PIN "
"or set CAIC_ALLOW_DEFAULT_PIN=true."
)
salt_hex, pin_hash_hex = hash_pin(seed_pin)
conn.execute("INSERT OR REPLACE INTO settings (key, value) VALUES (?, ?)", ("admin_pin_hash", pin_hash_hex))
conn.execute("INSERT OR REPLACE INTO settings (key, value) VALUES (?, ?)", ("admin_pin_salt", salt_hex))
if pin_source == "default":
log.warning("Admin PIN seeded from insecure default 1234 (override enabled).")
else:
log.info("Admin PIN hash seeded from configured environment PIN.")
conn.commit()
conn.close()
docker.md
-750
View File
@@ -1,750 +0,0 @@
# Docker Distribution — Architecture & Planning
> **Part of B3 (v1.0 gate).** This document catalogs every service, volume, port, configuration, and decision needed to ship cAIc as a `docker compose` stack. It also defines extraction (setup) and back-out (uninstall) procedures so nothing is lost when reality disagrees with the plan.
## 1. Stack Overview
```
┌─────────────────────────────────────────────────────────┐
│ docker compose stack │
│ │
│ ┌────────────┐ ┌──────────┐ ┌────────────────────┐ │
│ │ SearXNG │ │ Qdrant │ │ RabbitMQ │ │
│ │ :8888 │ │ :6333 │ │ :5672 / :15672 │ │
│ └──────┬──────┘ └────┬─────┘ └────────┬───────────┘ │
│ │ │ │ │
│ ▼ ▼ ▼ │
│ ┌──────────────────────────────────────────────────┐ │
│ │ cAIc (FastAPI) │ │
│ │ :8080 (HTTP) │ │
│ │ │ │
│ │ SQLite ◄── caic.db (volume) │ │
│ │ Uploads ◄── /app/uploads (volume) │ │
│ └──────────┬──────────────┬───────────────────────┘ │
│ │ │ │
│ ▼ ▼ │
│ ┌──────────────┐ ┌──────────────┐ │
│ │ llama-server │ │ Ollama │ │
│ │ :8081 │ │ :11434 │ │
│ │ (GPU/RPC) │ │ (embeddings) │ │
│ └──────────────┘ └──────────────┘ │
└─────────────────────────────────────────────────────────┘
```
> **This compose stack defines the coordinator.** A coordinator runs cAIc, the broker, and optional infrastructure services. Workers (headless inference nodes) do not use Docker — they install just llama-server + a Python node agent. See §9 for the worker deployment model.
### Service roles
| Service | Image | Role |
|---------|-------|------|
| **cAIc** | Custom `Dockerfile` | FastAPI app serving UI + API |
| **SearXNG** | `searxng/searxng:latest` | Privacy-respecting web search |
| **Qdrant** | `qdrant/qdrant:latest` | Vector database for RAG |
| **RabbitMQ** | `rabbitmq:4-management` | Message broker for AMQP cluster |
| **llama-server** | `ghcr.io/ggml-org/llama.cpp:server` | LLM inference (OpenAI-compat API) |
| **Ollama** | `ollama/ollama:latest` | Embeddings for RAG chunk vectors |
### Non-containerized (host-level)
| Component | Reason |
|-----------|--------|
| AMD GPU driver + ROCm | Kernel access required for GPU compute |
| llama.cpp RPC workers | Runs on *other* hosts — not on the Docker host |
| `rocm-smi` | Hardware stats — not needed for core function |
| `psutil` | Already inside the container via pip |
---
## 2. Service Catalog
### 2.1 cAIc (FastAPI app)
**Image:** `caic:latest` (built from `Dockerfile`)
**Ports:**
| Container | Host | Purpose |
|-----------|------|---------|
| 8080 | 8080 | HTTP API + UI |
**Volumes:**
| Container path | Type | Purpose |
|----------------|------|---------|
| `/app/caic.db` | named volume `caic_data` | SQLite database |
| `/app/uploads` | named volume `caic_uploads` | Uploaded files |
| `/app/hardware_state.json` | (inside volume) | Cached hardware probe |
**Dependencies:** Wait for SearXNG, Qdrant, RabbitMQ, llama-server, Ollama before serving.
**Restart:** `unless-stopped`
**Healthcheck:** `curl -f http://localhost:8080/`
### 2.2 SearXNG
**Image:** `searxng/searxng:latest`
**Ports:**
| Container | Host | Purpose |
|-----------|------|---------|
| 8080 | 8888 | Search API |
**Volumes:**
| Container path | Type | Purpose |
|----------------|------|---------|
| `/etc/searxng` | named volume `searxng_config` | `settings.yml` |
**Environment:**
```env
SEARXNG_BASE_URL=https://localhost:8888
```
**Config override (`/etc/searxng/settings.yml`):**
```yaml
search:
safe_search: 0
autocomplete: ""
server:
secret_key: ${SEARXNG_SECRET_KEY}
limiter: false
image_proxy: false
method: GET
port: 8080
bind_address: "0.0.0.0"
```
**Restart:** `unless-stopped`
### 2.3 Qdrant
**Image:** `qdrant/qdrant:latest`
**Ports:**
| Container | Host | Purpose |
|-----------|------|---------|
| 6333 | 6333 | HTTP API |
| 6334 | — | gRPC (internal only) |
**Volumes:**
| Container path | Type | Purpose |
|----------------|------|---------|
| `/qdrant/storage` | named volume `qdrant_storage` | Vector index data |
**Environment:**
```env
QDRANT__SERVICE__GRPC_PORT=6334
```
**Restart:** `unless-stopped`
### 2.4 RabbitMQ
**Image:** `rabbitmq:4-management`
**Ports:**
| Container | Host | Purpose |
|-----------|------|---------|
| 5672 | 5672 | AMQP messaging |
| 15672 | — | Management UI (internal only) |
**Volumes:**
| Container path | Type | Purpose |
|----------------|------|---------|
| `/var/lib/rabbitmq` | named volume `rabbitmq_data` | Message store |
**Environment:**
```env
RABBITMQ_DEFAULT_USER=caic
RABBITMQ_DEFAULT_PASS_FILE=/run/secrets/rabbitmq_password
RABBITMQ_DEFAULT_VHOST=/
```
**Restart:** `unless-stopped`
### 2.5 llama-server
**Image:** `ghcr.io/ggml-org/llama.cpp:server`
**Ports:**
| Container | Host | Purpose |
|-----------|------|---------|
| 8081 | 8081 | OpenAI-compat API |
**Volumes:**
| Container path | Type | Purpose |
|----------------|------|---------|
| `/models` | bind mount `./models` | Model GGUF files |
**Environment:**
```env
LLAMA_ARG_MODEL=/models/<model-file>
LLAMA_ARG_N_GPU_LAYERS=0 # set >0 for GPU offload
LLAMA_ARG_MAIN_GPU=0
LLAMA_ARG_CTX_SIZE=4096
LLAMA_ARG_HOST=0.0.0.0
LLAMA_ARG_PORT=8081
LLAMA_ARG_EMBEDDINGS=1
LLAMA_ARG_LOGPROBS=1
LLAMA_ARG_RPC= # optional: comma-separated RPC endpoints
```
**Restart:** `unless-stopped`
**Healthcheck:** `curl -f http://localhost:8081/health`
**Notes:**
- Models directory bind mount — user places `.gguf` files in `./models/` on the host
- RPC offload to other machines (e.g., `10.0.0.50:50052,10.0.0.51:50052`)
- If no GPU, set `LLAMA_ARG_N_GPU_LAYERS=0` for CPU-only
- `LLAMA_ARG_EMBEDDINGS=1` required for perplexity scoring
- `LLAMA_ARG_LOGPROBS=1` required for auto-search trigger
### 2.6 Ollama
**Image:** `ollama/ollama:latest`
**Ports:**
| Container | Host | Purpose |
|-----------|------|---------|
| 11434 | 11434 | Embeddings API |
**Volumes:**
| Container path | Type | Purpose |
|----------------|------|---------|
| `/root/.ollama` | named volume `ollama_models` | Pulled model blobs |
**Restart:** `unless-stopped`
**Notes:**
- Used exclusively for embeddings (`/api/embeddings`), not inference
- Typically needs a small model like `all-minilm:latest` or `nomic-embed-text:latest`
- Consider replacing Ollama with llama-server's built-in embedding if it supports the same model — would remove one container
---
## 3. Configuration Management
### 3.1 `.env` file (generated by setup wizard)
```env
# --- Secrets (auto-generated, change before production) ---
CAIC_ADMIN_PIN=
CAIC_COMPLETIONS_API_KEY=
CAIC_ALLOW_DEFAULT_PIN=false
RABBITMQ_PASSWORD=
SEARXNG_SECRET_KEY=
# --- Host discovery (auto-detected by setup wizard) ---
LLAMA_SERVER_BASE=http://llama-server:8081
OLLAMA_BASE=http://ollama:11434
SEARXNG_BASE=http://searxng:8888
QDRANT_URL=http://qdrant:6333
RABBITMQ_HOST=rabbitmq
RABBITMQ_PORT=5672
# --- Performance tuning (calculated by setup wizard) ---
RAG_MAX_VECTORS=50000
RAG_EVICTION_HIGH_WATER=0.80
RAG_EVICTION_LOW_WATER=0.20
RAG_EVICTION_BATCH=1000
# --- llama-server options ---
LLAMA_MODEL=llama3.1-8b-instruct.Q4_K_M.gguf
LLAMA_N_GPU_LAYERS=0
LLAMA_RPC_ENDPOINTS=
LLAMA_CTX_SIZE=4096
# --- Ollama ---
OLLAMA_EMBED_MODEL=all-minilm:latest
# --- Network ---
CAIC_ALLOWED_CIDRS=127.0.0.0/8,::1/128,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16
CAIC_TRUSTED_ORIGINS=
CAIC_TRUST_X_FORWARDED_FOR=false
```
### 3.2 Mapping of config.py → .env variable
Every config.py default that references an external service must accept a matching env var at runtime:
| config.py constant | .env variable | Service |
|-------------------|---------------|---------|
| `LLAMA_SERVER_BASE` | `LLAMA_SERVER_BASE` | llama-server |
| `OLLAMA_BASE` | `OLLAMA_BASE` | Ollama |
| `SEARXNG_BASE` | `SEARXNG_BASE` | SearXNG |
| `QDRANT_URL` | `QDRANT_URL` | Qdrant |
| `COMPLETIONS_API_KEY` | `CAIC_COMPLETIONS_API_KEY` | — |
| `ALLOWED_CIDRS_RAW` | `CAIC_ALLOWED_CIDRS` | — |
| `TRUST_X_FORWARDED_FOR` | `CAIC_TRUST_X_FORWARDED_FOR` | — |
| `TRUSTED_ORIGINS` | `CAIC_TRUSTED_ORIGINS` | — |
| `RAG_MAX_VECTORS` | `RAG_MAX_VECTORS` | — (calc'd from RAM) |
### 3.3 Secrets management
| Secret | Generated by | Stored in | Mounted to |
|--------|-------------|-----------|------------|
| `CAIC_ADMIN_PIN` | User prompt | `.env` | cAIc container |
| `CAIC_COMPLETIONS_API_KEY` | Auto-generated, shown to user | `.env` | cAIc container |
| `RABBITMQ_PASSWORD` | Auto-generated | `.env` + Docker secret | RabbitMQ container |
| `SEARXNG_SECRET_KEY` | Auto-generated | `.env` | SearXNG container |
**Docker secrets approach:** Use `secrets:` in compose file for RabbitMQ password (mounted as file) rather than passing via env var, since `settings.yml` in SearXNG and RabbitMQ config can reference file-based secrets without env-var leakage.
### 3.4 Dockerfile for cAIc
```dockerfile
FROM python:3.13-slim-bookworm AS builder
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
FROM python:3.13-slim-bookworm
WORKDIR /app
RUN apt-get update && apt-get install -y --no-install-recommends \
curl && \
rm -rf /var/lib/apt/lists/*
COPY --from=builder /usr/local/lib/python3.13/site-packages /usr/local/lib/python3.13/site-packages
COPY --from=builder /usr/local/bin /usr/local/bin
COPY . .
EXPOSE 8080
HEALTHCHECK --interval=30s --timeout=10s --start-period=15s --retries=3 \
CMD curl -f http://localhost:8080/ || exit 1
CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8080"]
```
**Multi-stage rationale:** First stage compiles/bundles packages (wheels), final stage is minimal. Devs can skip builder with `--target builder` for live-reload with volume mount.
---
## 4. docker-compose.yml structure
```yaml
services:
caic:
build: .
ports: ["8080:8080"]
volumes:
- caic_data:/app/caic.db
- caic_uploads:/app/uploads
env_file: .env
depends_on:
searxng: { condition: service_started }
qdrant: { condition: service_started }
rabbitmq: { condition: service_healthy }
llama-server: { condition: service_healthy }
ollama: { condition: service_started }
restart: unless-stopped
searxng:
image: searxng/searxng:latest
ports: ["8888:8080"]
volumes:
- ./searxng/settings.yml:/etc/searxng/settings.yml:ro
- searxng_config:/etc/searxng
env_file: .env
restart: unless-stopped
qdrant:
image: qdrant/qdrant:latest
ports: ["6333:6333"]
volumes:
- qdrant_storage:/qdrant/storage
restart: unless-stopped
rabbitmq:
image: rabbitmq:4-management
ports: ["5672:5672"]
volumes:
- rabbitmq_data:/var/lib/rabbitmq
env_file: .env
secrets:
- rabbitmq_password
healthcheck:
test: ["CMD", "rabbitmq-diagnostics", "check_port_connectivity"]
interval: 15s
timeout: 5s
retries: 3
restart: unless-stopped
llama-server:
image: ghcr.io/ggml-org/llama.cpp:server
ports: ["8081:8081"]
volumes:
- ./models:/models:ro
env_file: .env
command: >
--model /models/${LLAMA_MODEL}
--host 0.0.0.0 --port 8081
--ctx-size ${LLAMA_CTX_SIZE:-4096}
--n-gpu-layers ${LLAMA_N_GPU_LAYERS:-0}
--embeddings
--logprobs
${LLAMA_RPC_ENDPOINTS:+--rpc ${LLAMA_RPC_ENDPOINTS}}
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8081/health"]
interval: 30s
timeout: 10s
retries: 5
start_period: 60s
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities: [gpu]
restart: unless-stopped
ollama:
image: ollama/ollama:latest
ports: ["11434:11434"]
volumes:
- ollama_models:/root/.ollama
healthcheck:
test: ["CMD", "ollama", "list"]
interval: 30s
timeout: 10s
retries: 3
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities: [gpu]
restart: unless-stopped
volumes:
caic_data:
caic_uploads:
searxng_config:
qdrant_storage:
rabbitmq_data:
ollama_models:
secrets:
rabbitmq_password:
file: ./secrets/rabbitmq_password.txt
```
**Notes:**
- GPU reservations use `resources.reservations.devices` — this is compose v3.8+. For AMD GPUs, replace `driver: nvidia` with `driver: amd` (experimental Docker support). For hosts without GPU, omit the `deploy` block entirely.
- The `deploy` block only applies when deployed as a swarm stack. For `docker compose`, GPU access may need `--gpus all` or `device_requests` in config. Verify compatibility.
- SearXNG config file (`settings.yml`) is bind-mounted read-only from the host repo clone — the setup wizard should generate this file.
---
## 5. Networking
### 5.1 Internal communication (compose network)
| From | To | Port | Protocol |
|------|----|------|----------|
| cAIc | llama-server | 8081 | HTTP |
| cAIc | Ollama | 11434 | HTTP |
| cAIc | SearXNG | 8080 | HTTP |
| cAIc | Qdrant | 6333 | HTTP |
| cAIc | RabbitMQ | 5672 | AMQP |
| RabbitMQ | (cluster peers) | 4369 | EPMD |
| RabbitMQ | (cluster peers) | 25672 | Inter-node |
### 5.2 Exposed ports (host-facing)
| Port | Service | Should expose? | Notes |
|------|---------|---------------|-------|
| 8080 | cAIc | ✅ Required | UI + API |
| 8888 | SearXNG | Optional | Only if user wants standalone search |
| 6333 | Qdrant | Optional | Only for external tooling |
| 5672 | RabbitMQ | Optional | Only for remote AMQP clients |
| 15672 | RabbitMQ mgmt | ❌ Internal | Healthcheck only |
| 8081 | llama-server | Optional | Only for external tooling |
| 11434 | Ollama | Optional | Only for external tooling |
**Design decision:** By default, only port 8080 (cAIc) is published. All other services remain on the internal compose network. Advanced users can opt-in by uncommenting `ports:` blocks.
### 5.3 Reverse proxy consideration
For production, a reverse proxy (Caddy, nginx, Traefik) should sit in front:
```yaml
# Optional — compose profile: "proxy"
caddy:
image: caddy:latest
ports: ["80:80", "443:443"]
volumes:
- ./Caddyfile:/etc/caddy/Caddyfile:ro
- caddy_data:/data
```
This is out of scope for v1.0 but documented for future.
---
## 6. Setup Wizard (Extraction)
`setup.sh` — idempotent, interactive, runs on first boot.
### Flow
```
1. CHECK: Is .env present?
├── YES → skip to step 7 (or ask to regenerate)
└── NO → continue
2. INTRO: Print banner, explain what's about to happen
3. PROBE: Run hardware assessment
├── psutil → RAM total, CPU count
├── rocm-smi → VRAM (optional, best-effort)
└── nvidia-smi → VRAM (optional, best-effort)
4. NETWORK: Ask for
├── Hostname / LAN IP for this machine
├── Admin PIN (4 digits, or accept auto-generated)
└── (Optional) RPC endpoints for GPU offload
5. CALCULATE:
├── RAG_MAX_VECTORS = max(1000, int(available_ram_gb * 100_000))
├── LLAMA_N_GPU_LAYERS = 0 (CPU default; offer GPU detection)
├── LLAMA_MODEL = default gguf filename
└── RABBITMQ_PASSWORD = openssl rand -hex 20
6. GENERATE:
├── .env file from template
├── ./secrets/rabbitmq_password.txt
├── ./searxng/settings.yml (with generated secret_key)
└── ./models/README.txt (instructions for placing .gguf)
7. VERIFY:
├── docker and docker compose plugin installed
├── docker compose version >= 2.x
├── SUCCESS → "Run: docker compose up -d"
└── FAILURE → show diagnostics and links
8. EXTRACT model:
├── Prompt for download URL or local path
├── Offer to pull from HuggingFace if huggingface-cli available
└── Guides user to place file in ./models/
```
### What setup.sh creates on disk
```
./docker-deploy/
├── .env # All env vars (SECRET — add to .gitignore)
├── docker-compose.yml # Compose stack definition
├── Dockerfile # cAIc image build
├── secrets/
│ └── rabbitmq_password.txt # RabbitMQ password file
├── searxng/
│ └── settings.yml # SearXNG config with generated secret_key
├── models/
│ ├── README.txt # Instructions for model placement
│ └── <model>.gguf # (user-provided)
└── setup.log # Wizard run log
```
### Idempotency
Re-running `setup.sh`:
- With `.env` present: ask "Regenerate? This will overwrite existing config."
- Without `.env`: fresh run
- Never overwrites `./models/*.gguf` files
- Never touches running containers — only modifies files on disk
---
## 7. Back-out Procedure (Uninstall)
`teardown.sh` — returns the host system to its pre-install state.
### What gets removed
| Item | Removal method |
|------|---------------|
| Docker containers | `docker compose down -v` |
| Docker images | `docker rmi caic:latest` (ask about other images) |
| Docker volumes | `docker volume rm caic_data ...` (prompt first) |
| Network `caic_default` | Removed with compose |
| `.env` file | `rm .env` |
| `secrets/` directory | `rm -rf secrets/` |
| `searxng/` directory | `rm -rf searxng/` |
| `setup.log` | `rm setup.log` |
| `hardware_state.json` | `rm hardware_state.json` |
### What is preserved (by default)
| Item | Reason |
|------|--------|
| `./models/*.gguf` | User data — prompt for deletion |
| `caic.db` (in volume) | Prompt: "Keep database snapshot?" |
| `./uploads/` (in volume) | Prompt: "Keep uploaded files?" |
| Docker Engine itself | Not installed by this project — leave it |
### Script flow
```
1. CHECK: docker compose file exists?
├── NO → warn, continue
└── YES → docker compose down -v
2. CHECK: .env exists?
├── NO → skip
└── YES → ask: "Remove .env?" (default no)
3. ASK: "Remove secrets/ and searxng/ directories?" (default no)
4. ASK: "Remove Docker images? (y/N)" (default no)
├── Y → docker rmi caic:latest
├── Y → docker image ls | grep searxng/qdrant/rabbitmq → prompt per image
└── N → skip
5. ASK: "Keep database volume snapshot? (Y/n)" (default yes)
├── N → docker volume rm caic_data
└── Y → leave volume (can be reattached later)
6. ASK: "Remove model files from ./models/? (y/N)" (default no)
7. CLEANUP generated artifacts:
├── rm -f setup.log
├── rm -f hardware_state.json
└── rm -f docker-compose.yml
8. SUMMARY:
├── "Docker stack removed"
├── "Persistent data preserved at: <paths>"
└── "Models kept at: ./models/"
```
### Partial rollback
If the setup wizard fails mid-way, a partial rollback is better than leaving detritus:
| Failure point | Clean up |
|--------------|----------|
| After .env, before compose | `rm .env; rm -rf secrets/ searxng/` |
| After compose, before first `up` | `rm docker-compose.yml; rm -rf *` |
| After `up` but before healthcheck | `docker compose down -v; rm -rf ./*` |
`setup.sh` should trap EXIT on failure and prompt: "Clean up partial install? [y/N]"
---
## 8. Open Decisions
| Decision | Options | Priority |
|----------|---------|----------|
| **Ollama vs llama-server embeddings** | Both work. Keep both for now — remove Ollama if llama-server handles embeddings. Reduce containers = simpler. | Medium |
| **GPU support in compose** | NVIDIA: well-supported. AMD: requires `--device=/dev/kfd --device=/dev/dri` and ROCm image. Document both. | High |
| **RabbitMQ clustering vs single node** | Single node in v1.0. Clustering docs for multi-host later. | Low |
| **SearXNG config management** | Bind-mount a generated `settings.yml`, or let container create default and post-process. Bind-mount is cleaner. | Medium |
| **Reverse proxy** | Caddy is simplest for auto-HTTPS. Out of scope for v1.0 but design for it. | Low |
| **Healthcheck strategy** | `depends_on` with `condition: service_healthy` is the safest approach but increases startup time. Acceptable. | Medium |
| **Database migration** | SQLite file in volume — no migration needed for v1.0 format. If schema changes post-v1.0, need a migration container. | Low |
| **Linux vs macOS vs Windows** | Linux-primary. macOS may work with changes (no rocm-smi). Windows via WSL2 only. | Low |
| **LLM model download** | HuggingFace CLI integration in setup.sh, or manual download. Manual is simpler. | Low |
| **Dockerfile optimization** | Pin pip hashes, use `--no-cache-dir`, consider `slim` vs `alpine`. Alpine has musl compatibility issues with psutil. Stay with slim. | Medium |
## 9. Worker Node Deployment Model
The Docker stack above defines the **coordinator** only. Workers (headless inference nodes) have a radically lighter footprint.
### 9.1 What a worker runs
```
Worker machine (e.g. worker01, worker02)
┌────────────────────────────────────┐
│ llama-server │
│ (single binary, no build needed) │
│ │
│ node_agent.py │
│ (Python script, aio-pika client) │
│ ─ connects to coordinator's RMQ │
│ ─ publishes heartbeat + reg │
│ ─ consumes model_swap commands │
│ │
│ ROCm or CUDA runtime (if GPU) │
└────────────────────────────────────┘
```
### 9.2 What a worker does NOT run
| Service | Reason |
|---------|--------|
| RabbitMQ server | Connects as AMQP *client* only (aio-pika) |
| FastAPI / uvicorn / jC | No HTTP API, no UI, no database |
| SQLite | No persistent state of its own |
| SearXNG | No web search needs |
| Qdrant | No local vector store |
| Ollama | Uses coordinator's embedding endpoint |
| Docker | Everything runs as bare binaries |
| Python venv with full jC deps | Only needs `aio-pika` + `httpx` |
### 9.3 Worker setup
```bash
# Install llama-server binary
wget https://github.com/ggml-org/llama.cpp/releases/.../llama-server
chmod +x llama-server
# Install node agent deps
pip install aio-pika httpx
# Create node agent script (from repo: node_agent/agent.py)
# Configure COORDINATOR_AMQP_URL in environment
```
### 9.4 Multiple workers
Each worker registers independently with the coordinator's RabbitMQ. The coordinator tracks all registered workers via `CLUSTER_NODES` and routes inference requests to the best-matching node based on classification and availability.
### 9.5 RabbitMQ and workers — architecture note
Workers connect to RabbitMQ as **standard AMQP TCP clients** — no broker software required. The AMQP-0-9-1 protocol has always been client-server (since 2006), and libraries like `aio-pika`, `pika`, `amqplib`, `php-amqplib`, etc. connect over a single persistent socket. This is distinct from a service-mesh design where every node runs the same software stack and role is determined by config.
```
Broker-mediated model (this project):
Coordinator runs RabbitMQ broker ←── Workers connect as AMQP clients
Service-mesh model (alternative):
Every node runs RabbitMQ broker ←── Nodes cluster together, all autonomous
```
The broker-mediated model is the preferred architecture for this project because workers are intentionally heterogeneous (different GPUs, different models, ARM vs x86) and should not be burdened with infrastructure services.
## 10. Checklist (pre-v1.0 gate)
- [ ] `Dockerfile` written and builds clean
- [ ] `docker-compose.yml` boots all containers
- [ ] cAIc container reaches all services (env vars resolve correctly)
- [ ] SearXNG settings.yml generated correctly by setup.sh
- [ ] RabbitMQ password secret mounted correctly
- [ ] GPU (NVIDIA) passes through to llama-server container
- [ ] GPU (AMD) passes through to llama-server container (or documented limitation)
- [ ] `.env.example` checked in (no real secrets)
- [ ] `setup.sh` written, idempotent, tested on clean Debian
- [ ] `teardown.sh` written, tested, doesn't delete models without confirmation
- [ ] `docker compose up -d` works without any manual steps beyond setup.sh
- [ ] `docker compose down -v` followed by `setup.sh && docker compose up -d` = fresh stack
- [ ] Healthchecks prevent serving before dependencies are ready
- [ ] v1.0 release tag created
---
## 11. Files to create for B3
```
docker.md ← this file (planning doc)
Dockerfile ← cAIc image
docker-compose.yml ← full stack
.env.example ← template without secrets
setup.sh ← extraction wizard
teardown.sh ← back-out utility
searxng/
settings.yml ← SearXNG config (generated by setup.sh)
secrets/
rabbitmq_password.txt ← generated by setup.sh
models/
README.txt ← instructions for placing .gguf
```
docs/images/screenshot.png
BIN
View File
Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 219 KiB

After

Width:  |  Height:  |  Size: 322 KiB

docs/jc-ingest.sh
-31
View File
@@ -1,31 +0,0 @@
#!/bin/bash
# jc-ingest.sh — pipe terminal commands into jarvisChat RAG
# Deploy to /home/gramps/bin/jc-ingest.sh on jarvis (192.168.50.210)
#
# Usage:
# 1. chmod +x /home/gramps/bin/jc-ingest.sh
# 2. Add to ~/.bashrc:
# export JARVISCHAT_COMPLETIONS_API_KEY="$(cat /opt/jarvischat/.completions_key)"
# export PROMPT_COMMAND="jc_capture"
# source /home/gramps/bin/jc-ingest.sh
#
# The PROMPT_COMMAND hook runs jc_capture() after each command.
# Only commands matching the filter pattern are ingested.
#
# Filter: currently captures git, pip, systemctl, sudo, vi/vim, curl,
# wget, apt, python, pytest commands. Edit the grep pattern to adjust.
JC_URL="http://192.168.50.210:8080/api/ingest"
JC_TOKEN="${JARVISCHAT_COMPLETIONS_API_KEY}"
jc_capture() {
local cmd
cmd=$(history 1 | sed 's/^[ ]*[0-9]*[ ]*//')
if echo "$cmd" | grep -qE '^(git|pip|systemctl|sudo|vi|vim|curl|wget|apt|python|pytest)'; then
curl -s -X POST "$JC_URL" \
-H "Authorization: Bearer $JC_TOKEN" \
-H "Content-Type: application/json" \
-d "{\"content\": $(echo "$cmd" | python3 -c 'import json,sys; print(json.dumps(sys.stdin.read().strip()))'), \"source\": \"terminal\"}" \
> /dev/null 2>&1 &
fi
}
docs/wiki/Developer-Architecture.md
+98 -246
View File
@@ -1,313 +1,165 @@
# Developer Architecture Guide # Developer Architecture Guide
This document explains how cAIc is structured, the external services it integrates with, and the key architectural changes made during development. This document explains how JarvisChat is structured, why key guardrails exist, and what the test suite validates.
## 1. System Overview ## 1. System Overview
cAIc is a single-process FastAPI service with a Jinja2 frontend and SQLite persistence. It connects to an external llama-server for inference and optionally to SearXNG (web search), Qdrant (vector search), and RabbitMQ (AMQP cluster messaging). JarvisChat is a single-process FastAPI service with a Jinja2 frontend and SQLite persistence.
### 1.1 Module Layout Primary files:
Refactored from single-file (`app.py`) into modules under project root: - `app.py`: API, middleware, streaming/chat logic, auth, memory, skills, and DB bootstrap
- `templates/index.html`: main WebUX, settings panels, auth flow, streaming UI handlers
- `jarvischat.db`: runtime SQLite database created and migrated at startup
| File | Role | Core runtime integrations:
|------|------|
| `app.py` | FastAPI app, middleware, router registration, lifespan |
| `config.py` | Constants, env vars, rate/payload limits, built-in skills registry, upload limits, RAG eviction config |
| `db.py` | SQLite schema, connection factory, settings helpers, upload_context CRUD |
| `auth.py` | PIN-based guest/admin sessions, auth routes |
| `security.py` | Rate limiting, origin checks, IP allowlist, audit/incident logging |
| `memory.py` | FTS5 memory CRUD, remember/forget command parsing |
| `search.py` | SearXNG integration, perplexity scoring, refusal detection |
| `rag.py` | Qdrant vector search, system prompt assembly, chunk_text() helper, collection stats |
| `eviction.py` | Score-based RAG eviction engine (extracted from rag.py) |
| `gpu.py` | AMD GPU stats via rocm-smi |
| `hardware.py` | Hardware self-assessment — CPU, RAM, VRAM, service health probes |
| `amqp.py` | aio-pika connection manager for RabbitMQ (connect, disconnect, publish, subscribe, auto-reconnect) |
| `cluster.py` | Cluster node registry, event log, coordinator election, ping/pong, model swap handlers |
| `triage.py` | Phi-4-mini query classification + `select_node()` for cluster routing |
| `routers/` | One module per endpoint group |
### 1.2 External Services - Ollama for chat/model interaction
- SearXNG for web search (optional)
| Service | Required | Port | Purpose | - wttr.in for weather shortcut queries
|---------|----------|------|---------| - rocm-smi for GPU stats when available
| llama-server (coordinator) | Yes | 8081 | LLM inference (OpenAI-compat), RPC offload to worker:50052 |
| SearXNG | No | 8888 | Privacy-respecting web search |
| Qdrant (coordinator) | No | 6333 | Vector database for RAG |
| Ollama (worker) | No | 11434 | Embeddings for RAG chunk vectors |
| RabbitMQ (coordinator) | No | 5672 | AMQP broker for cluster messaging |
| rocm-smi | No | — | AMD GPU stats (host-level) |
### 1.3 Config Discovery
Key base URLs are configured via environment variables with sensible defaults:
| Variable | Default | Service |
|----------|---------|---------|
| `LLAMA_SERVER_BASE` | `http://192.168.50.108:8081` | llama-server on coordinator |
| `OLLAMA_BASE` | `http://localhost:11434` | Legacy — all inference goes through LLAMA_SERVER_BASE |
| `SEARXNG_BASE` | `http://localhost:8888` | SearXNG |
| `QDRANT_URL` | `http://192.168.50.108:6333` | Qdrant on coordinator |
| `CAIC_AMQP_URL` | `amqp://caic:password@localhost:5672/caic` | RabbitMQ |
## 2. Request/Response Architecture ## 2. Request/Response Architecture
### 2.1 Chat Pipeline (`/api/chat`) ### 2.1 Chat Pipeline (`/api/chat`)
1. Validate session, role, origin, rate, and payload limits in middleware 1. Validate session, role, origin, rate, and payload limits in middleware
2. Intercept "remember that..." / "forget about..." commands → process_remember_command() 2. Persist user message and conversation metadata
3. Persist user message and conversation metadata 3. Build system prompt from enabled profile, memory context, and active skills metadata
4. Build system prompt: profile + FTS5 memory + Qdrant RAG results + preset + active skills + uploaded document (if upload_context_id) 4. Stream model response over SSE token-by-token
5. Stream from llama-server with `logprobs: true` for perplexity scoring 5. Evaluate uncertainty/refusal; if needed, trigger search augmentation and stream augmented result
6. If perplexity > 15.0 OR refusal patterns match → re-query with SearXNG results 6. Persist final assistant message and emit terminal SSE event
7. Persist final assistant message and emit terminal SSE event
### 2.2 Explicit Search Pipeline (`/api/search`) ### 2.2 Explicit Search Pipeline (`/api/search`)
1. Persist search-as-message into conversation 1. Persist search-as-message into the target/new conversation
2. Emit `searching` SSE event 2. Emit `searching` SSE event
3. Pull web results from SearXNG 3. Pull web results from SearXNG
4. Summarize via llama-server SSE stream 4. Summarize with Ollama via SSE stream
5. Persist summary and emit `done` event 5. Persist summary and emit `done` event (plus raw results payload)
### 2.3 RAG Ingest Pipeline (`/api/ingest`) ### 2.3 Settings/Control Surface
1. Bearer token auth (same key as completions API) - Profile, presets, memory, conversation management, and settings APIs
2. Chunk text via shared `chunk_text()` helper (512-token chunks, 128-token overlap) - Skills APIs for phase-1 registry and enable/disable controls
3. Embed via Ollama `/api/embeddings` - Auth/session APIs for guest/admin role handling and keepalive
4. Upsert to Qdrant collection `caic_rag`
5. Trigger `maybe_evict()` if collection exceeds high-water mark
### 2.4 Upload Pipeline (`/api/upload`)
1. Admin required, multipart file upload
2. Validate MIME type + size against config limits
3. PDF text extraction via pypdf; plain text for all other types
4. Three modes: `context` (SQLite with 1hr expiry), `ingest` (RAG/Qdrant), `both`
5. Trigger `maybe_evict()` if ingest mode
## 3. Data Model (SQLite) ## 3. Data Model (SQLite)
Key tables: Key tables:
- `conversations` — headers, timestamps, attachment_count - `conversations`: conversation headers and timestamps
- `messages` — ordered chat history per conversation - `messages`: ordered chat history entries
- `profile` — singleton row for injected profile prompt - `profile`: singleton row for injected profile prompt
- `settings` — runtime toggles and selected defaults - `settings`: runtime toggles and selected defaults
- `system_presets` — named reusable system prompts - `system_presets`: named reusable system prompts
- `skills` — per-skill enabled state and timestamp - `skills`: per-skill enabled state and timestamp
- `memories` (FTS5 virtual table) — full-text searchable user memory facts - `memories` (FTS5 virtual table): searchable user memory facts
- `upload_context` — auto-expiring document storage for context injection
Design notes: Design notes:
- Startup is idempotent: tables created if missing, defaults seeded only when absent
- No connection pool: each request opens and closes a short-lived SQLite connection - Startup is idempotent: tables are created if missing and defaults seeded only when absent
- `init_db()` called in FastAPI lifespan - No connection pool: each request opens a short-lived SQLite connection
## 4. Security Implementations ## 4. Security Implementations
This section documents explicit controls currently in code.
### 4.1 Auth Model ### 4.1 Auth Model
- Guest session by default (POST /api/auth/guest) - Guest session is default for conversational access
- Admin unlock via 4-digit PIN (POST /api/auth/login) - Admin unlock uses 4-digit PIN and creates admin-capable session
- Admin required for PUT/DELETE/PATCH + all POST except allowlist (/api/chat, /api/search, /api/auth/*) - Admin required for write/destructive routes
- /api/ingest is exempt from session auth — self-authenticates via Bearer token - Session heartbeat/timeout and explicit logout/revoke flow
- Session heartbeat/timeout (90s default) and explicit logout
### 4.2 PIN Hardening ### 4.2 PIN and Session Hardening
- Admin PIN hashed with PBKDF2-HMAC-SHA256 + salt - Admin PIN hashed with PBKDF2-HMAC-SHA256 + salt
- Failed PIN attempts tracked per client IP (max 5, 300s lockout) - Failed PIN attempts tracked per client IP
- Default PIN allowed only if CAIC_ALLOW_DEFAULT_PIN=true - Lockout window enforced after max failed attempts
### 4.3 Browser and API Abuse Controls ### 4.3 Browser and API Abuse Controls
- Origin checks on all /api/ requests (rejects absent Origin AND Referer) - Origin checks on state-changing requests
- Rate limiting per endpoint category and identity (IP/session) - Rate limiting by endpoint category and identity (IP/session)
- Payload size limits per route class (64KB default, 128KB chat, 20MB upload) - Payload size limits per route class
- Settings key allowlist (5 keys: profile_enabled, default_model, etc.) - Settings key allowlist to block arbitrary configuration injection
- IP allowlist/CIDR gate with trusted proxy forwarding mode - IP allowlist/CIDR gate with optional trusted proxy forwarding mode
### 4.4 Output and Error Safety ### 4.4 Output and Error Safety
- Search result URLs sanitized to http/https only - Search result URLs sanitized to `http`/`https` only
- Client-safe error envelopes with incident key correlation - Client-safe error envelopes with incident key correlation
- Full stack traces logged server-side only - Full stack traces and diagnostic metadata logged server-side only
### 4.5 Operational Auditability ### 4.5 Operational Auditability
- Structured audit events for auth actions, admin ops, guardrail denials - Structured audit events for auth actions, admin operations, and guardrail denials
- Incident logs with event type, key, path/method, and runtime metadata - Incident logs include event type, key, path/method context, and runtime metadata
## 5. RAG Architecture ## 5. Skills Framework (Phase 1)
### 5.1 Vector Search Goal: introduce a governed skills control plane inside the local JarvisChat sandbox.
- Qdrant collection `caic_rag` on coordinator:6333 Current behavior:
- Embeddings via Ollama on worker:11434 (`/api/embeddings`)
- Shared `chunk_text(text, chunk_size=512, overlap=128)` helper in rag.py
- Upload and ingest endpoints share the same chunk+embed+upsert pipeline
### 5.2 Score-Based Eviction - Built-in skill registry defined server-side
- Per-skill enable/disable persisted in DB
- Global `skills_enabled` master toggle in settings
- Active skills injected into system prompt with bounded text budget
- API endpoints to list skills, list active skills, and toggle skill state
- WebUX settings panel to control master/per-skill toggles
When `RAG_MAX_VECTORS` is exceeded, eviction fires with hysteresis: Non-goals in phase 1:
- High-water mark: 80% of max → trigger eviction - No unrestricted shell/tool execution
- Low-water mark: 20% of max → stop eviction - No external connector execution (filesystem, Gmail, etc.)
- Batch size: 1000 vectors per cycle
- Score formula: `score = (access_weight * retrieval_count) + (age_weight * hours_since_ingested)`
- Lower score evicted first (least useful)
- Tiebreaker: oldest last_accessed ASC
- Excluded sources: `upload`, `profile` (pinned)
- Grace period: 1 hour before any vector is eligible
- Thread-safe via `asyncio.Lock`
Eviction module at `eviction.py` (re-exported through `rag.py` for backward compat). ## 6. Testing Strategy and Validation Intent
### 5.3 Operational Stats The test suite validates both behavior and guardrail assumptions.
`GET /api/rag/stats` (admin required) returns: ### 6.1 What We Test
- vector_count, max_vectors, high_water_pct, low_water_pct, percent_full
- pinned_sources list, grace_hours
- at_risk_count, pinned_count, avg_retrieval_count
- eviction_counts_last_{1,5,30}m
### 5.4 Flush - Auth capability separation (guest vs admin)
- URL sanitization safety for outbound links
- Rate and payload guardrails
- IP allowlist behavior
- Safe error envelope behavior and SSE error leakage prevention
- Streaming chat/search and memory command paths
- Skills framework toggles and prompt-injection behavior
`POST /api/rag/flush` (admin required) — deletes all non-pinned vectors. Returns `{deleted_count, collection, status}`. ### 6.2 Why These Tests Matter
## 6. Cluster Architecture - Confirms security controls are active and regression-resistant
- Ensures streaming UX protocol remains stable (`token`, `searching`, `done`, `error`)
- Verifies policy intent: dangerous actions require admin capability
- Validates new features preserve prior guarantees
### 6.1 Design Model: Broker-Mediated ### 6.3 Internal Process Validation
cAIc uses a **broker-mediated** cluster design. This is the preferred architecture and is reflected in all implementation decisions below. For substantive changes, Definition of Done includes:
**How it works:**
- A single RabbitMQ broker (or clustered set of brokers) acts as the central nervous system
- **Coordinator nodes** run the FastAPI app, host the HTTP API/UI, and publish commands to the broker
- **Worker nodes** connect as AMQP *clients only* — they consume commands and publish status events, but run no broker software themselves
- Communication is asynchronous and persistent: each node opens a TCP connection on startup and keeps it alive. The coordinator probes worker health via on-demand AMQP ping/pong messages (5s timeout) rather than relying on the AMQP-0-9-1 transport-level heartbeat.
**Why broker-mediated:**
- Workers are heterogeneous (different GPUs, different models, ARM vs x86) — no assumption of uniform software
- Workers are lightweight — a Raspberry Pi with a USB AI accelerator can participate without running a broker
- The coordinator delegates work via messages, not by SSH'ing into workers or requiring shared filesystems
- Failure is isolated: a crashed worker stops responding to ping; the coordinator auto-deregisters it and reassigns its work
**What it is NOT:**
- Not a service mesh — workers do not run identical software stacks
- Not autonomous failover — if the coordinator dies, a replacement must be manually promoted (or pre-configured as a secondary coordinator). Workers cannot self-promote to coordinator because they lack the required services (FastAPI, SQLite, DB schema, SearXNG, Qdrant, etc.)
- Not a peer-to-peer cluster — all orchestration flows through the coordinator
### 6.2 Node Types
Every physical machine in the cluster is classified by which services it runs. Two node types are defined:
| Aspect | Coordinator | Worker |
|--------|------------|--------|
| **Role** | Serves HTTP API/UI, orchestrates inference, owns cluster state | Runs inference models on behalf of the coordinator |
| **Python** | Required — runs FastAPI app | Required — runs node agent (aio-pika consumer) |
| **RabbitMQ server** | Required — hosts the broker | Not required — connects as AMQP client only |
| **RabbitMQ client (aio-pika)** | Required — publishes commands, consumes events | Required — consumes commands, publishes events |
| **FastAPI / uvicorn** | Required | Not needed |
| **SQLite** | Required — owns caic.db | Not needed |
| **Qdrant** | Optional (recommended) — vector DB for RAG | Not needed |
| **SearXNG** | Optional — web search | Not needed |
| **llama-server** | Optional — can share its own GPU for inference | Required — this is why the worker exists |
| **Ollama** | Optional — embeddings for RAG | Not needed |
| **rocm-smi / nvidia-smi** | Optional — hardware stats | Optional — node agent reports this at registration |
### 6.3 Service Distribution Summary
```
Coordinator Worker(s)
┌────────────────────┐ ┌──────────────────────────┐
│ cAIc │ │ llama-server │
│ (FastAPI + SQLite)│ │ (inference) │
│ RabbitMQ server │◄──AMQP───────│ aio-pika (agent) │
│ SearXNG (opt) │ persistent │ ROCm / CUDA (if GPU) │
│ Qdrant (opt) │ TCP │ Ollama (embeddings,opt) │
│ llama-server(opt) │ conn │ │
└────────────────────┘ │ No broker │
│ No cAIc │
│ No DB │
│ No search/vector │
└──────────────────────────┘
```
### 6.4 RabbitMQ Topology
Every RabbitMQ server belongs to a cluster. Currently only the coordinator runs one; if high availability is needed, additional nodes can join the RMQ cluster without changing the architecture.
| Exchange | Type | Purpose |
|----------|------|---------|
| `jc.admin` | topic | Lifecycle commands: register, deregister, ping, pong, admitted, rejected; model commands: cmd.swap_model |
| `jc.system` | topic | Events: model_ready, model_failed, node.*.heartbeat, event; coordinator queries: coord_query, coord_response |
All exchanges, queues, and bindings are declared by `amqp.py` at startup. Worker runs `node_agent/agent.py` which connects as an AMQP client, registers, responds to ping, and handles model swap commands.
## 7. SSE Protocol
All streaming endpoints yield `data: {json}\n\n`:
- `{token, conversation_id}` — streaming token
- `{searching: true}` — web search triggered
- `{search_results: N}` — N results found (no raw payload)
- `{done: true, perplexity, tokens_per_sec, searched?}` — terminal
- `{error: "...", error_key: "..."}` — error with incident key
## 8. Testing Strategy
### 8.1 Test Framework
- pytest with `tmp_path` + monkeypatched httpx.AsyncClient
- No live external services required
- Test factories reset `SESSIONS`, `PIN_ATTEMPTS`, `RATE_EVENTS` globals per test
### 8.2 Test Coverage Areas (179 tests)
| Test file | Coverage |
|-----------|----------|
| test_auth_capabilities.py | Guest/admin sessions, origin blocking, logout |
| test_chat_streaming_and_memory_paths.py | Streaming, auto-search, remember/forget, upload context injection |
| test_cluster.py | Registration, deregistration, pong, events, coordinator query |
| test_cluster_heartbeat.py | Heartbeat handler, known/unknown node |
| test_completions.py | API key auth, FIM, streaming, blocking, errors |
| test_conversations.py | Full CRUD, guest admin, attachment_count |
| test_ingest.py | Bearer auth, chunk/embed/upsert, validation |
| test_ip_allowlist.py | IP allowlist helper + middleware |
| test_memories.py | Edit, search, stats |
| test_model_swap.py | request_model_swap, handle_model_ready/failed, select_node swap triggering |
| test_models_router.py | Models list, ps, show, stats, search/status |
| test_node_agent.py | Node agent registration, ping/pong, model swap |
| test_presets.py | Full CRUD, default preset protection |
| test_profile.py | Get, update, default, length validation |
| test_rag_management.py | Collection stats, eviction algorithm (pinned/grace/scoring/batch), maybe_evict hysteresis, operational stats, flush, concurrency lock |
| test_rate_and_payload_guardrails.py | Rate limits + payload size |
| test_search_route.py | Explicit search flow, no results, errors |
| test_search_url_sanitization.py | URL sanitizer |
| test_settings_allowlist.py | Allowlisted key enforcement |
| test_skills_framework.py | List, toggle, unknown skill, prompt injection |
| test_triage.py | classify_query, select_node, get_inference_url |
| test_upload.py | Upload, delete, link, by-conversation, attachment_count |
| test_error_envelopes.py | Global exception handler + stream errors |
### 8.3 DoD Process
For substantive changes:
1. Implement code change 1. Implement code change
2. Add/adjust tests proving behavior and guardrail intent 2. Add/adjust tests proving behavior and guardrail intent
3. Update this wiki and README in the same change set 3. Update README release notes for user-facing impact
4. Validate with full test run before commit 4. Update wiki architecture/security/testing docs for maintainers
5. Validate with targeted test runs before merge/deploy
## 9. Hardware Self-Assessment This process is intentionally explicit so design decisions remain auditable over time.
On startup, `assess_hardware()` probes: ## 7. Deployment and Operations Notes
- RAM total/available (psutil)
- VRAM total/free (rocm-smi, best-effort)
- llama-server reachability + model list
- Qdrant reachability + collection list
- SearXNG reachability
Writes `hardware_state.json` to working directory. - Primary deployment target: local/homelab systemd service
- Required dependency: Ollama
- Optional dependency: SearXNG
- Recommended log review path: system journal for startup, guardrail denials, and incidents
## 8. Contribution Guidance
When adding a feature:
1. Define security posture first (who can execute, what can fail, and failure mode)
2. Implement smallest safe slice with clear limits
3. Add tests that prove both happy path and guardrail path
4. Update this wiki and README in the same change
docs/wiki/Home.md
+3 -3
View File
@@ -1,6 +1,6 @@
# cAIc Developer Wiki # JarvisChat Developer Wiki
This wiki is the developer-facing architecture and process reference for cAIc. This wiki is the developer-facing architecture and process reference for JarvisChat.
## Audience ## Audience
@@ -14,7 +14,7 @@ This wiki is the developer-facing architecture and process reference for cAIc.
## Scope and Support Model ## Scope and Support Model
cAIc is designed for local and trusted-LAN operation. JarvisChat is designed for local and trusted-LAN operation.
The code may technically function against external or commercial endpoints, but this deployment mode is not a supported target in this project. The code may technically function against external or commercial endpoints, but this deployment mode is not a supported target in this project.
docs/wiki/current-wip.md
+76 -16
View File
@@ -1,23 +1,83 @@
# cAIc Current WiP Backlog # JarvisChat Current WiP Backlog
Last updated: 2026-07-06 Last updated: 2026-04-27
Owner: Gramps Owner: Gramps + Copilot
Scope: Active roadmap items and backlog. Scope: issues, bugs, security exposures, and feature enhancements.
## ~~Roadmap N: AMQP Cluster Nervous System [COMPLETE]~~ Total identified items: 26
All 15 tasks are done. Final commit: `f0689ac feat: Roadmap N — AMQP cluster nervous system complete`. ## Priority Definitions
- P0: Critical risk or data-loss/security exposure; do first.
- P1: High impact reliability/correctness work.
- P2: Important feature/UX improvements.
- P3: Nice-to-have polish.
## Backlog ## Top 10 (Urgency Order)
1. [P0][DONE] Add authentication/authorization for all write and admin endpoints.
2. [P0][DONE] Add CSRF/origin protection for browser-initiated state-changing requests.
3. [P0][DONE] Block unsafe URL schemes in rendered search-result links (e.g., javascript:).
4. [P0][DONE] Add rate limiting and request body size limits for chat/search/profile APIs.
5. [P1][DONE] Restrict settings updates to an allowlist of valid keys.
6. [P1] Add pagination + hard caps on list endpoints (memories, conversations, message history).
7. [P1][DONE] Stop returning raw exception text to clients; use safe error envelopes.
8. [P1][DONE] Add automated tests for chat streaming, auto-search trigger, and memory command paths.
9. [P2][DONE] Implement skills/tool-call framework (MCP-style) with per-skill enable controls.
10. [P2] Implement heartbeat/check-in pipeline with scheduler + summary endpoint.
- B1 — Context loss in follow-up questions (investigation) ## Item 1 Executive Summary (Scope + Security)
- B2 — Bang-prefixed (`!`) search routing
- B3 — Docker distribution (planning doc at `docker.md`) - Status: Complete. Guest/admin capability split implemented with admin-only write enforcement, origin checks on state-changing requests, audit logging, and endpoint capability tests.
- **B4 — RAG Corpus Management UI (NEXT)** — browse, search, edit, delete individual RAG entries. Backend endpoints at `routers/rag_admin.py`, frontend panel in `templates/index.html`
- HTTPS / reverse proxy (Caddy) - Decision: JarvisChat is local-first by design. Primary mode is same-host Ollama; optional mode allows RFC1918 LAN endpoints only.
- Conversation search/filter and export tooling - Constraint: Public Internet AI endpoints are out of scope unless explicitly enabled in a future advanced mode.
- Keyboard shortcuts, retry button, source-link polish - Risk: Even on LAN, unauthenticated write/admin endpoints permit unauthorized data tampering and deletion.
- Requirement: Add mandatory admin authentication for all POST/PUT/DELETE routes and destructive actions.
- Authentication shape (scope-locked): two capability tiers only: guest (chat-only) and admin (4-digit PIN unlock).
- Scope guardrail: Avoid full RBAC. Keep capability split minimal: conversational chat for guest, advanced/destructive actions for admin.
- Definition of done:
1. Auth required on all state-changing endpoints.
2. Destructive actions require admin authorization.
3. Endpoint configuration rejects non-local/non-RFC1918 AI backends by default.
4. Strong rate limiting + lockout controls in place for PIN attempts.
5. Security events logged for failed and successful admin actions.
## Full Backlog (Sorted by Priority)
### P0 Critical
1. Add auth for write/admin endpoints (`POST/PUT/DELETE` routes, mass delete, profile/settings changes).
2. Add CSRF or strict origin checks for browser session protection.
3. Validate/sanitize outbound href URLs before rendering in HTML (allow http/https only).
4. Add per-IP rate limiting on `/api/chat`, `/api/search`, `/api/profile`, `/api/settings`.
5. Enforce request size limits (message/profile text and JSON body) to prevent memory abuse.
### P1 High
6. Add settings key allowlist in `/api/settings` to prevent arbitrary key injection.
7. Add pagination (`limit`, `offset`) with enforced maximums for list APIs.
8. Add DB indexes and query hygiene for scalability (`messages.conversation_id`, timestamps).
9. Replace raw exception leakage to clients with generic safe error messages + server-side logs.
10. Add request/response timeout and retry policy consistency across external calls.
11. Add endpoint-level audit logging for destructive operations.
12. Add unit/integration tests for: remember/forget parsing, refusal detection, search fallback, SSE done/error shape.
13. Add conversation title sanitization and length constraints.
14. Ensure default preset semantics are correct (currently all seeded presets are marked default).
### P2 Important Features
15. Skills system: load markdown skill files with YAML frontmatter from skills directory.
16. Skills registry API: list/enable/disable skills and expose active skills to UI.
17. Inject active skill instructions into system prompt with bounded token budget.
18. Tool execution guardrails: allowlist, confirmation mode, and execution logs.
19. Heartbeat scheduler (cron/systemd timer) for daily check-ins.
20. Heartbeat endpoint for generated briefings and anomaly summaries.
21. Model info UI panel (description, updated date, best-use purpose).
22. Default model selection improvements and persistence validation.
23. Hidden model list support (exclude models from dropdown).
24. Model update action from UI (trigger controlled model pull).
### P3 Nice to Have
25. Conversation search/filter and export tooling.
26. Keyboard shortcuts, retry button, and source-link polish.
## Maintenance Rules ## Maintenance Rules
- Keep this file as the single source of truth for roadmap tracking. - Keep this file as the single source of truth.
- Update as work starts or completes. - Update item priority/status whenever work starts or completes.
- Mirror the Top 10 summary in README and keep counts aligned.
eviction.py
-268
View File
@@ -1,268 +0,0 @@
"""
cAIc — Score-based RAG vector eviction with hysteresis.
"""
import asyncio
import logging
from datetime import datetime, timezone, timedelta
import httpx
from config import (
QDRANT_URL, RAG_COLLECTION,
RAG_MAX_VECTORS, RAG_EVICTION_HIGH_WATER, RAG_EVICTION_LOW_WATER,
RAG_EVICTION_BATCH, RAG_PINNED_SOURCES, RAG_GRACE_HOURS,
RAG_ACCESS_WEIGHT, RAG_AGE_WEIGHT,
)
log = logging.getLogger("caic")
eviction_lock = asyncio.Lock()
EVICTION_LOG: list[dict] = []
async def _update_retrieval_count(point_id: str, current_count: int = 0):
try:
async with httpx.AsyncClient() as client:
payload = {
"retrieval_count": current_count + 1,
"last_accessed": datetime.now(timezone.utc).isoformat(),
}
resp = await client.put(
f"{QDRANT_URL}/collections/{RAG_COLLECTION}/points/payload",
json={"points": [point_id], "payload": payload},
timeout=5.0,
)
if resp.status_code not in (200, 201):
log.warning(f"Failed to increment retrieval count for {point_id}: {resp.status_code}")
except Exception as e:
log.warning(f"Error incrementing retrieval count for {point_id}: {e}")
async def get_collection_count() -> int:
try:
async with httpx.AsyncClient() as client:
resp = await client.get(
f"{QDRANT_URL}/collections/{RAG_COLLECTION}",
timeout=10.0,
)
if resp.status_code == 200:
return resp.json().get("result", {}).get("vectors_count", 0)
except Exception as e:
log.warning(f"get_collection_count error: {e}")
return 0
async def get_collection_stats() -> dict:
count = await get_collection_count()
high_water_pct = int(RAG_EVICTION_HIGH_WATER * 100)
low_water_pct = int(RAG_EVICTION_LOW_WATER * 100)
percent_full = round((count / RAG_MAX_VECTORS) * 100, 1) if RAG_MAX_VECTORS > 0 else 0
return {
"vector_count": count,
"max_vectors": RAG_MAX_VECTORS,
"high_water_mark": int(RAG_MAX_VECTORS * RAG_EVICTION_HIGH_WATER),
"low_water_mark": int(RAG_MAX_VECTORS * RAG_EVICTION_LOW_WATER),
"high_water_pct": high_water_pct,
"low_water_pct": low_water_pct,
"percent_full": percent_full,
"pinned_sources": list(RAG_PINNED_SOURCES),
}
async def evict_batch(batch_size: int) -> int:
filter_conditions = {
"must_not": [
{"match": {"key": "source", "value": src}}
for src in RAG_PINNED_SOURCES
]
}
try:
async with httpx.AsyncClient() as client:
scroll_resp = await client.post(
f"{QDRANT_URL}/collections/{RAG_COLLECTION}/points/scroll",
json={
"filter": filter_conditions,
"limit": min(batch_size * 10, 10000),
"with_payload": True,
"with_vector": False,
},
timeout=30.0,
)
if scroll_resp.status_code != 200:
log.warning(f"Eviction scroll failed: {scroll_resp.status_code}")
return 0
points = scroll_resp.json().get("result", {}).get("points", [])
if not points:
return 0
now = datetime.now(timezone.utc)
scored = []
for p in points:
payload = p.get("payload", {})
date_str = payload.get("ingest_date") or payload.get("upload_date", "")
if date_str:
age_hours = (now - datetime.fromisoformat(date_str)).total_seconds() / 3600
else:
age_hours = 999999
if age_hours < RAG_GRACE_HOURS:
continue
retrieval_count = payload.get("retrieval_count", 0) or 0
score = retrieval_count * RAG_ACCESS_WEIGHT + age_hours * RAG_AGE_WEIGHT
last_accessed = payload.get("last_accessed", date_str)
scored.append((score, last_accessed, p["id"]))
if not scored:
log.warning("No evictable vectors found (all pinned or newborn)")
return 0
scored.sort(key=lambda x: (x[0], x[1]))
to_delete = [p[2] for p in scored[:batch_size]]
if not to_delete:
return 0
delete_resp = await client.post(
f"{QDRANT_URL}/collections/{RAG_COLLECTION}/points/delete",
json={"points": to_delete},
timeout=30.0,
)
if delete_resp.status_code not in (200, 201):
log.warning(f"Eviction delete failed: {delete_resp.status_code}")
return 0
return len(to_delete)
except Exception as e:
log.warning(f"evict_batch error: {e}")
return 0
async def maybe_evict() -> int:
if RAG_MAX_VECTORS <= 0:
return 0
effective_batch = max(RAG_EVICTION_BATCH, 1)
async with eviction_lock:
count = await get_collection_count()
threshold_high = int(RAG_MAX_VECTORS * RAG_EVICTION_HIGH_WATER)
threshold_low = int(RAG_MAX_VECTORS * RAG_EVICTION_LOW_WATER)
if count < threshold_high:
return 0
total_evicted = 0
while count >= threshold_low:
if total_evicted > 0 and count < threshold_low:
break
deleted = await evict_batch(effective_batch)
if deleted == 0:
break
total_evicted += deleted
count -= deleted
if count < threshold_high and total_evicted > 0:
break
if count < threshold_low:
break
if total_evicted > 0:
entry = {
"timestamp": datetime.now(timezone.utc).isoformat(),
"count": total_evicted,
"remaining": count,
}
EVICTION_LOG.append(entry)
if len(EVICTION_LOG) > 1000:
EVICTION_LOG.pop(0)
log.info(f"Evicted {total_evicted} vectors ({count} remaining)")
return total_evicted
async def get_rag_operational_stats() -> dict:
stats = await get_collection_stats()
now = datetime.now(timezone.utc)
cutoff_1m = now - timedelta(minutes=1)
cutoff_5m = now - timedelta(minutes=5)
cutoff_30m = now - timedelta(minutes=30)
eviction_1m = sum(
e["count"] for e in EVICTION_LOG
if datetime.fromisoformat(e["timestamp"]) > cutoff_1m
)
eviction_5m = sum(
e["count"] for e in EVICTION_LOG
if datetime.fromisoformat(e["timestamp"]) > cutoff_5m
)
eviction_30m = sum(
e["count"] for e in EVICTION_LOG
if datetime.fromisoformat(e["timestamp"]) > cutoff_30m
)
pinned_count = 0
avg_retrieval_count = 0.0
at_risk_count = 0
try:
async with httpx.AsyncClient() as client:
pinned_filter = {
"should": [
{"match": {"key": "source", "value": src}}
for src in RAG_PINNED_SOURCES
]
}
pinned_resp = await client.post(
f"{QDRANT_URL}/collections/{RAG_COLLECTION}/points/scroll",
json={"filter": pinned_filter, "limit": 10000, "with_payload": True, "with_vector": False},
timeout=10.0,
)
if pinned_resp.status_code == 200:
pinned_count = len(pinned_resp.json().get("result", {}).get("points", []))
nonpinned_filter = {
"must_not": [
{"match": {"key": "source", "value": src}}
for src in RAG_PINNED_SOURCES
]
}
np_resp = await client.post(
f"{QDRANT_URL}/collections/{RAG_COLLECTION}/points/scroll",
json={"filter": nonpinned_filter, "limit": 10000, "with_payload": True, "with_vector": False},
timeout=10.0,
)
if np_resp.status_code == 200:
points = np_resp.json().get("result", {}).get("points", [])
if points:
retrievals = []
scored = []
for p in points:
payload = p.get("payload", {})
rc = payload.get("retrieval_count", 0) or 0
retrievals.append(rc)
date_str = payload.get("ingest_date") or payload.get("upload_date", "")
if date_str:
age_hours = (now - datetime.fromisoformat(date_str)).total_seconds() / 3600
else:
age_hours = 999999
score = rc * RAG_ACCESS_WEIGHT + age_hours * RAG_AGE_WEIGHT
last_accessed = payload.get("last_accessed", date_str)
scored.append((score, last_accessed))
avg_retrieval_count = round(sum(retrievals) / len(retrievals), 2)
scored.sort(key=lambda x: (x[0], x[1]))
at_risk_threshold = max(1, len(scored) // 10)
at_risk_count = at_risk_threshold
except Exception as e:
log.warning(f"RAG operational stats scroll error: {e}")
stats.update({
"grace_hours": RAG_GRACE_HOURS,
"eviction_counts_last_1m": eviction_1m,
"eviction_counts_last_5m": eviction_5m,
"eviction_counts_last_30m": eviction_30m,
"pinned_count": pinned_count,
"avg_retrieval_count": avg_retrieval_count,
"at_risk_count": at_risk_count,
})
return stats
gpu.py
-31
View File
@@ -1,31 +0,0 @@
"""
cAIc - AMD GPU stats via rocm-smi.
"""
import json
import logging
import subprocess
log = logging.getLogger("caic")
def get_gpu_stats() -> dict:
try:
result = subprocess.run(
["rocm-smi", "--showuse", "--showmemuse", "--json"],
capture_output=True, text=True, timeout=5,
)
if result.returncode == 0:
data = json.loads(result.stdout)
gpu_info = data.get("card0", {})
gpu_use = gpu_info.get("GPU use (%)", 0)
vram_use = gpu_info.get("GPU Memory Allocated (VRAM%)", 0)
if isinstance(gpu_use, str):
gpu_use = int(gpu_use.replace("%", "").strip() or 0)
if isinstance(vram_use, str):
vram_use = int(vram_use.replace("%", "").strip() or 0)
return {"gpu_percent": gpu_use, "vram_percent": vram_use, "available": True}
except (subprocess.TimeoutExpired, FileNotFoundError, json.JSONDecodeError):
pass
except Exception as e:
log.warning(f"GPU stats error: {e}")
return {"gpu_percent": 0, "vram_percent": 0, "available": False}
hardware.py
-102
View File
@@ -1,102 +0,0 @@
"""
cAIc — Startup hardware self-assessment.
"""
import asyncio
import json
import logging
import subprocess
from pathlib import Path
import httpx
import psutil
from config import LLAMA_SERVER_BASE, SEARXNG_BASE
log = logging.getLogger("caic")
HARDWARE_STATE_PATH = Path("hardware_state.json")
_TIMEOUT_EXPIRED = subprocess.TimeoutExpired
async def assess_hardware() -> dict:
mem = psutil.virtual_memory()
ram_total_gb = round(mem.total / (1024 ** 3), 1)
ram_available_gb = round(mem.available / (1024 ** 3), 1)
cpu_count = psutil.cpu_count()
vram_total_mb = 0
vram_free_mb = 0
try:
result = subprocess.run(
["rocm-smi", "--showmeminfo", "vram", "--json"],
capture_output=True, text=True, timeout=5,
)
if result.returncode == 0:
data = json.loads(result.stdout)
for card, info in data.items():
tot = info.get("VRAM Total (MB)", 0)
free = info.get("VRAM Free (MB)", None)
used = info.get("VRAM Used (MB)", 0)
if tot:
vram_total_mb += int(tot)
if free is not None:
vram_free_mb += int(free)
else:
vram_free_mb += int(tot) - int(used)
except (FileNotFoundError, _TIMEOUT_EXPIRED, json.JSONDecodeError):
log.warning("rocm-smi not available or failed — VRAM stats set to 0")
except Exception as e:
log.warning(f"rocm-smi error: {e}")
llama_reachable = False
llama_models = []
try:
async with httpx.AsyncClient(timeout=3) as client:
resp = await client.get(f"{LLAMA_SERVER_BASE}/v1/models")
if resp.status_code == 200:
llama_reachable = True
data = resp.json()
llama_models = [m.get("id", "") for m in data.get("data", [])]
except Exception:
log.warning("llama-server not reachable")
qdrant_reachable = False
qdrant_collections = []
try:
async with httpx.AsyncClient(timeout=3) as client:
resp = await client.get("http://192.168.50.108:6333/collections")
if resp.status_code == 200:
qdrant_reachable = True
data = resp.json()
raw = data.get("result", {}).get("collections", [])
qdrant_collections = [c.get("name", "") for c in raw]
except Exception:
log.warning("Qdrant not reachable")
searxng_reachable = False
try:
async with httpx.AsyncClient(timeout=3) as client:
resp = await client.get(SEARXNG_BASE)
if resp.status_code == 200:
searxng_reachable = True
except Exception:
log.warning("SearXNG not reachable")
state = {
"ram_total_gb": ram_total_gb,
"ram_available_gb": ram_available_gb,
"cpu_count": cpu_count,
"vram_total_mb": vram_total_mb,
"vram_free_mb": vram_free_mb,
"llama_reachable": llama_reachable,
"llama_models": llama_models,
"qdrant_reachable": qdrant_reachable,
"qdrant_collections": qdrant_collections,
"searxng_reachable": searxng_reachable,
}
HARDWARE_STATE_PATH.write_text(json.dumps(state, indent=2))
log.info(
f"HW: {ram_total_gb}GB RAM, {vram_total_mb}MB VRAM, "
f"llama={llama_reachable}, qdrant={qdrant_reachable}, searxng={searxng_reachable}"
)
return state
jarvischat_refactor.sh
-2174
View File
File diff suppressed because it is too large Load Diff
memory.py
-219
View File
@@ -1,219 +0,0 @@
"""
cAIc - FTS5 memory system.
CRUD, search, remember/forget command processing, topic detection.
"""
import logging
import re
from datetime import datetime, timezone
from typing import Optional
from db import get_db
from config import MAX_MEMORY_FACT_CHARS
log = logging.getLogger("caic")
REMEMBER_PATTERNS = [
(r"remember that (.+)", "explicit"),
(r"please remember (.+)", "explicit"),
(r"don'?t forget (.+)", "explicit"),
(r"note that (.+)", "explicit"),
(r"keep in mind (?:that )?(.+)", "explicit"),
]
FORGET_PATTERNS = [
r"forget (?:that )?(.+)",
r"don'?t remember (.+)",
r"remove (?:the )?memory (?:about |that )?(.+)",
]
AUTO_FACT_PATTERNS = [
re.compile(r"\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b"),
re.compile(r"\b(?:systemd|nginx|docker|ssh|ufw|iptables|postgres(?:ql)?|redis|mosquitto|node_exporter|prometheus|grafana|qdrant|rabbitmq|searxng|llama-server)\b", re.IGNORECASE),
re.compile(r"/(?:etc|home|usr|var|opt|tmp|mnt)/\S+"),
re.compile(r"\b(?:Ryzen|RX\s*\d{4}|RTX\s*\d{4}|Radeon|AMD|NVIDIA|Core\s*i[579]|Threadripper)\b", re.IGNORECASE),
re.compile(r"\b(?:Qwen|Llama|Gemma|Phi|Mistral|DeepSeek)\S*\b", re.IGNORECASE),
re.compile(r"\b(?:systemd\.service|docker\s+(?:compose|container|service)|systemctl|journalctl)\b", re.IGNORECASE),
]
SOCIAL_TRIGGERS = {"hi", "hello", "hey", "yo", "sup", "howdy", "good morning", "good evening"}
def _is_social(text: str) -> bool:
t = text.strip().lower()
if t in SOCIAL_TRIGGERS or any(t.startswith(w) for w in ("thanks", "thank you", "ty")):
return True
return False
def auto_detect_facts(user_message: str, assistant_message: str) -> list[str]:
"""Extract environmental/factual content from a chat turn.
Returns a list of fact strings ready for storage. Empty list means
nothing worth persisting.
"""
if _is_social(user_message):
return []
if len(assistant_message) < 40:
return []
if process_remember_command(user_message) is not None:
return []
found = []
for pat in AUTO_FACT_PATTERNS:
if pat.search(user_message):
found.append(user_message)
break
# Also capture when the user is reporting a change they made
change_match = re.search(
r"(?:I\s+)?(?:set|changed?|updated|installed|configured|enabled|disabled|added|removed|created|deleted|restarted|reloaded|switched|moved|copied|renamed|symlinked|mounted|unmounted)\s+(?:the\s+)?(.+)",
user_message, re.IGNORECASE,
)
if change_match and user_message not in found:
found.append(user_message)
seen = set()
deduped = []
for f in found:
key = f.strip().lower()
if key not in seen:
seen.add(key)
deduped.append(f.strip()[:MAX_MEMORY_FACT_CHARS])
return deduped
def check_fact_conflicts(facts: list[str]) -> list[dict]:
"""Search for existing memories that conflict with detected facts.
Returns list of {memory_id, old_fact, new_fact} for each conflict.
"""
conflicts = []
for new_fact in facts:
related = search_memories(new_fact, limit=1)
if related:
old = related[0]["fact"]
if old.rstrip(".") != new_fact.rstrip("."):
conflicts.append({
"memory_id": related[0]["rowid"],
"old_fact": old,
"new_fact": new_fact,
})
return conflicts
def detect_topic(fact: str) -> str:
fact_lower = fact.lower()
if any(w in fact_lower for w in ["prefer", "like", "hate", "always", "never", "favorite"]):
return "preference"
elif any(w in fact_lower for w in ["working on", "building", "project", "developing"]):
return "project"
elif any(w in fact_lower for w in ["run", "install", "server", "ip", "port", "service", "docker", "systemd"]):
return "infrastructure"
elif any(w in fact_lower for w in ["my name", "i am", "i'm a", "i live", "my wife", "my partner"]):
return "personal"
return "general"
def add_memory(fact: str, topic: str = "general", source: str = "explicit") -> Optional[int]:
db = get_db()
now = datetime.now(timezone.utc).isoformat()
cur = db.execute(
"INSERT INTO memories (fact, topic, source, created_at) VALUES (?, ?, ?, ?)",
(fact, topic, source, now),
)
db.commit()
rowid = cur.lastrowid
db.close()
log.info(f"Memory added [{topic}]: {fact[:50]}...")
return rowid
def search_memories(query: str, limit: int = 5) -> list:
if not query.strip():
return []
db = get_db()
words = re.findall(r"[A-Za-z0-9_]+", query)
if not words:
db.close()
return []
escaped = []
for word in words[:10]:
if word.upper() in {"AND", "OR", "NOT", "NEAR"}:
escaped.append(f'"{word}"*')
else:
escaped.append(word + "*")
safe_query = " OR ".join(escaped)
try:
rows = db.execute(
"SELECT rowid, fact, topic, source, created_at, bm25(memories) AS rank "
"FROM memories WHERE memories MATCH ? ORDER BY rank LIMIT ?",
(safe_query, limit),
).fetchall()
results = [dict(row) for row in rows]
log.debug(f"Memory search '{query}' returned {len(results)} results")
except Exception as e:
log.warning(f"Memory search error: {e}")
results = []
db.close()
return results
def get_all_memories(topic: Optional[str] = None) -> list:
db = get_db()
if topic:
rows = db.execute(
"SELECT rowid, * FROM memories WHERE topic = ? ORDER BY created_at DESC", (topic,)
).fetchall()
else:
rows = db.execute("SELECT rowid, * FROM memories ORDER BY created_at DESC").fetchall()
db.close()
return [dict(row) for row in rows]
def delete_memory(rowid: int) -> bool:
db = get_db()
cur = db.execute("DELETE FROM memories WHERE rowid = ?", (rowid,))
db.commit()
deleted = cur.rowcount > 0
db.close()
if deleted:
log.info(f"Memory deleted: rowid={rowid}")
return deleted
def update_memory(rowid: int, fact: str) -> bool:
db = get_db()
cur = db.execute("UPDATE memories SET fact = ? WHERE rowid = ?", (fact, rowid))
db.commit()
updated = cur.rowcount > 0
db.close()
return updated
def get_memory_count() -> int:
db = get_db()
count = db.execute("SELECT COUNT(*) as c FROM memories").fetchone()["c"]
db.close()
return count
def process_remember_command(user_message: str) -> Optional[str]:
for pattern, source in REMEMBER_PATTERNS:
match = re.search(pattern, user_message, re.IGNORECASE)
if match:
fact = match.group(1).strip().rstrip(".")
topic = detect_topic(fact)
add_memory(fact, topic=topic, source=source)
return f"✓ Remembered [{topic}]: {fact}"
for pattern in FORGET_PATTERNS:
match = re.search(pattern, user_message, re.IGNORECASE)
if match:
search_term = match.group(1).strip().rstrip(".")
memories = search_memories(search_term, limit=3)
if memories:
for m in memories:
delete_memory(m["rowid"])
return f"✓ Forgot {len(memories)} memory/memories about: {search_term}"
else:
return f"✗ No memories found about: {search_term}"
return None
node_agent/agent.py
-412
View File
@@ -1,412 +0,0 @@
"""
cAIc — Worker node agent.
Standalone AMQP client that registers with the cAIc coordinator,
responds to pings, and handles model swap commands.
## Config file: /etc/caic-node-agent.conf
```ini
[agent]
# hostname — defaults to socket.gethostname()
node_name = jarvis
# LAN IP — defaults from socket
node_ip = 192.168.50.210
# "worker" (fixed)
node_type = worker
# comma-separated capability list
capabilities = llm
# RabbitMQ URL on coordinator
amqp_url = amqp://caic:password@192.168.50.108:5672/caic
# port llama-server listens on
llama_port = 8081
# path to GGUF model files
models_dir = /var/lib/caic/models
# currently active model filename
active_model = llama3.1-latest-Q4_K_M.gguf
```
## systemd unit: /etc/systemd/system/caic-node-agent.service
```ini
[Unit]
Description=cAIc Worker Node Agent
After=network.target rabbitmq.service
Wants=rabbitmq.service
[Service]
Type=simple
User=root
Group=root
WorkingDirectory=/opt/caic
ExecStart=/usr/bin/python3 /opt/caic/node_agent/agent.py
Restart=always
RestartSec=5
[Install]
WantedBy=multi-user.target
```
"""
import asyncio
import json
import logging
import os
import socket
import subprocess
import sys
import time
from configparser import ConfigParser
from datetime import datetime, timezone
from pathlib import Path
from typing import Optional
try:
import psutil
HAS_PSUTIL = True
except ImportError:
HAS_PSUTIL = False
try:
import aio_pika
from aio_pika import DeliveryMode, ExchangeType
HAS_AIO_PIKA = True
except ImportError:
HAS_AIO_PIKA = False
try:
import httpx
HAS_HTTPX = True
except ImportError:
HAS_HTTPX = False
log = logging.getLogger("caic")
CONFIG_PATH = "/etc/caic-node-agent.conf"
# ── data types ──────────────────────────────────────────────────────────
class AgentConfig:
def __init__(self):
self.node_name: str = socket.gethostname()
self.node_ip: str = "127.0.0.1"
self.node_type: str = "worker"
self.capabilities: list[str] = ["llm"]
self.amqp_url: str = "amqp://caic:password@localhost:5672/caic"
self.llama_port: int = 8081
self.models_dir: str = "/var/lib/caic/models"
self.active_model: str = ""
@classmethod
def from_ini(cls, path: str = CONFIG_PATH) -> "AgentConfig":
cfg = cls()
parser = ConfigParser()
if not os.path.exists(path):
log.warning("config %s not found, using defaults", path)
return cfg
parser.read(path)
sec = "agent"
if parser.has_section(sec):
cfg.node_name = parser.get(sec, "node_name", fallback=cfg.node_name)
cfg.node_ip = parser.get(sec, "node_ip", fallback=cfg.node_ip)
cfg.node_type = parser.get(sec, "node_type", fallback=cfg.node_type)
raw_caps = parser.get(sec, "capabilities", fallback="llm")
cfg.capabilities = [c.strip() for c in raw_caps.split(",") if c.strip()]
cfg.amqp_url = parser.get(sec, "amqp_url", fallback=cfg.amqp_url)
cfg.llama_port = parser.getint(sec, "llama_port", fallback=cfg.llama_port)
cfg.models_dir = parser.get(sec, "models_dir", fallback=cfg.models_dir)
cfg.active_model = parser.get(sec, "active_model", fallback=cfg.active_model)
return cfg
class ModelInfo:
def __init__(self, filename: str, name: str = "", version: str = "", quant: str = ""):
self.filename = filename
self.name = name
self.version = version
self.quant = quant
self.path = ""
def to_dict(self) -> dict:
return {
"name": self.name,
"version": self.version,
"quant": self.quant,
"filename": self.filename,
}
# ── model discovery ─────────────────────────────────────────────────────
_MODEL_PATTERN = None # lazy compile
def discover_models(models_dir: str) -> list[dict]:
import re
global _MODEL_PATTERN
if _MODEL_PATTERN is None:
_MODEL_PATTERN = re.compile(
r"^(?P<name>.+?)-(?P<version>[^-]+)-(?P<quant>Q\d+_K_[A-Z]+|IQ\d_[A-Z]+|fp\d+)\.gguf$"
)
root = Path(models_dir)
if not root.is_dir():
log.warning("models_dir %s not found", models_dir)
return []
results = []
for fpath in sorted(root.glob("*.gguf")):
m = _MODEL_PATTERN.match(fpath.name)
if m:
info = ModelInfo(
filename=fpath.name,
name=m.group("name"),
version=m.group("version"),
quant=m.group("quant"),
)
info.path = str(fpath)
results.append(info.to_dict())
else:
log.debug("skipping unrecognized model filename: %s", fpath.name)
return results
# ── load reporting ──────────────────────────────────────────────────────
def get_load() -> dict:
load = {}
if HAS_PSUTIL:
load["cpu_pct"] = round(psutil.cpu_percent(interval=0.5))
load["ram_pct"] = round(psutil.virtual_memory().percent)
try:
result = subprocess.run(
["rocm-smi", "--showmeminfo", "vram"],
capture_output=True, text=True, timeout=3,
)
if result.returncode == 0:
for line in result.stdout.splitlines():
if "VRAM Total" in line:
parts = line.split()
if len(parts) >= 3:
total = int(parts[-1])
elif "VRAM Used" in line:
parts = line.split()
if len(parts) >= 3:
used = int(parts[-1])
if total and total > 0:
load["vram_pct"] = round(used / total * 100)
except (FileNotFoundError, subprocess.TimeoutExpired):
pass
return load
# ── AMQP helpers ────────────────────────────────────────────────────────
async def declare_exchanges(channel) -> tuple:
admin = await channel.declare_exchange("jc.admin", ExchangeType.TOPIC, durable=True)
system = await channel.declare_exchange("jc.system", ExchangeType.TOPIC, durable=True)
return admin, system
async def publish(channel, exchange, routing_key, payload):
body = json.dumps(payload).encode()
msg = aio_pika.Message(body, delivery_mode=DeliveryMode.PERSISTENT)
await exchange.publish(msg, routing_key)
# ── registration ────────────────────────────────────────────────────────
def build_registration_payload(cfg: AgentConfig, inventory: list[dict]) -> dict:
model_dict = None
if cfg.active_model:
for inv in inventory:
if inv["filename"] == cfg.active_model:
model_dict = {**inv, "port": cfg.llama_port}
break
return {
"node_name": cfg.node_name,
"node_type": cfg.node_type,
"ip": cfg.node_ip,
"capabilities": cfg.capabilities,
"active_model": model_dict,
"inventory": inventory,
}
# ── ping / pong ─────────────────────────────────────────────────────────
async def handle_ping(cfg: AgentConfig, channel, exchange, msg: aio_pika.IncomingMessage):
async with msg.process():
try:
payload = json.loads(msg.body.decode())
except json.JSONDecodeError:
return
correlation_id = payload.get("correlation_id")
if not correlation_id:
return
load = get_load()
now = datetime.now(timezone.utc).isoformat() + "Z"
pong = {
"node_name": cfg.node_name,
"type": "pong",
"correlation_id": correlation_id,
"status": "active",
"active_model": None, # simplified; could read current
"load": load,
"timestamp": now,
}
await publish(channel, exchange, f"node.{cfg.node_name}.pong", pong)
# ── model swap ──────────────────────────────────────────────────────────
async def handle_swap_model(cfg: AgentConfig, channel, exchanges, msg: aio_pika.IncomingMessage):
admin_ex, system_ex = exchanges
async with msg.process():
try:
payload = json.loads(msg.body.decode())
except json.JSONDecodeError:
return
model_filename = payload.get("model_filename")
if not model_filename:
log.error("swap_model missing model_filename")
return
log.info("swapping model to %s", model_filename)
# 1. Stop current llama-server
log.info("stopping llama-server")
subprocess.run(["systemctl", "stop", "llama-server"], check=False)
# 2. Update config
_update_config_active_model(cfg, model_filename)
# 3. Start llama-server
log.info("starting llama-server")
subprocess.run(["systemctl", "start", "llama-server"], check=False)
# 4. Poll health endpoint
healthy = await _wait_for_llama(cfg.llama_port, timeout=120, interval=2)
now = datetime.now(timezone.utc).isoformat() + "Z"
if healthy:
result_payload = {
"node_name": cfg.node_name,
"type": "model_ready",
"active_model": model_filename,
"port": cfg.llama_port,
"timestamp": now,
}
log.info("model swap successful: %s", model_filename)
else:
result_payload = {
"node_name": cfg.node_name,
"type": "model_failed",
"active_model": model_filename,
"port": cfg.llama_port,
"error": "llama-server did not become healthy within 120s",
"timestamp": now,
}
log.error("model swap failed: %s", model_filename)
await publish(channel, system_ex, f"node.{cfg.node_name}.{result_payload['type']}", result_payload)
def _update_config_active_model(cfg: AgentConfig, model_filename: str):
parser = ConfigParser()
if os.path.exists(CONFIG_PATH):
parser.read(CONFIG_PATH)
if not parser.has_section("agent"):
parser.add_section("agent")
parser.set("agent", "active_model", model_filename)
with open(CONFIG_PATH, "w") as f:
parser.write(f)
cfg.active_model = model_filename
async def _wait_for_llama(port: int, timeout: int = 120, interval: int = 2) -> bool:
if not HAS_HTTPX:
log.warning("httpx not installed, skipping health check")
return True
deadline = time.time() + timeout
url = f"http://localhost:{port}/v1/models"
async with httpx.AsyncClient() as client:
while time.time() < deadline:
try:
resp = await client.get(url, timeout=5)
if resp.status_code == 200:
return True
except (httpx.ConnectError, httpx.TimeoutException):
pass
await asyncio.sleep(interval)
return False
# ── main ────────────────────────────────────────────────────────────────
async def amain():
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s caic[%(process)d]: %(levelname)s %(message)s",
)
log.info("cAIc node agent starting")
if not HAS_AIO_PIKA:
log.error("aio-pika not installed")
sys.exit(1)
cfg = AgentConfig.from_ini()
log.info("node_name=%s node_ip=%s", cfg.node_name, cfg.node_ip)
inventory = discover_models(cfg.models_dir)
log.info("discovered %d models", len(inventory))
# Connect to AMQP
conn = await aio_pika.connect_robust(cfg.amqp_url)
channel = await conn.channel()
admin_ex, system_ex = await declare_exchanges(channel)
log.info("connected to AMQP broker")
# Publish registration
reg_payload = build_registration_payload(cfg, inventory)
await publish(channel, admin_ex, f"node.{cfg.node_name}.register", reg_payload)
log.info("registration published")
# Wait for admission
response_queue = await channel.declare_queue("", exclusive=True)
await response_queue.bind(admin_ex, f"node.{cfg.node_name}.admitted")
await response_queue.bind(admin_ex, f"node.{cfg.node_name}.rejected")
admitted = False
async with response_queue.iterator() as iterator:
async for message in iterator:
async with message.process():
payload = json.loads(message.body.decode())
if payload.get("type") == "admitted":
log.info("admitted to cluster")
admitted = True
break
else:
log.error("rejected: %s", payload.get("reason"))
sys.exit(1)
if not admitted:
log.error("no admission response received")
sys.exit(1)
# Set up ping consumer
ping_queue = await channel.declare_queue("", exclusive=True)
await ping_queue.bind(admin_ex, f"node.{cfg.node_name}.ping")
await ping_queue.consume(lambda msg: handle_ping(cfg, channel, admin_ex, msg))
# Set up swap consumer
swap_queue = await channel.declare_queue("", exclusive=True)
await swap_queue.bind(admin_ex, f"node.{cfg.node_name}.cmd.swap_model")
await swap_queue.consume(lambda msg: handle_swap_model(cfg, channel, (admin_ex, system_ex), msg))
log.info("listening for pings and commands")
# Run forever
await asyncio.Event().wait()
if __name__ == "__main__":
asyncio.run(amain())
node_agent/requirements.txt
-3
View File
@@ -1,3 +0,0 @@
aio-pika>=9.0.0
psutil>=5.9.0
httpx>=0.27.0
rag.py
-207
View File
@@ -1,207 +0,0 @@
"""
cAIc - RAG pipeline: Qdrant vector search + system prompt assembly.
"""
import asyncio
import logging
from datetime import datetime, timezone
import httpx
from eviction import _update_retrieval_count
from db import get_db, get_setting, list_skills_with_state, format_active_skills_prompt
from memory import search_memories
from config import MAX_SKILL_PROMPT_CHARS, QDRANT_URL, RAG_COLLECTION
log = logging.getLogger("caic")
EMBED_URL = "http://192.168.50.210:11434"
EMBED_MODEL = "mxbai-embed-large"
RAG_SCORE_THRESHOLD = 0.25
# Re-export eviction symbols for backward compatibility
from eviction import ( # noqa: E402
maybe_evict, get_rag_operational_stats, EVICTION_LOG,
get_collection_count, get_collection_stats, evict_batch,
)
async def _upsert_fact(fact: str, text: str, topic: str,
client: httpx.AsyncClient) -> bool:
"""Embed text and upsert a fact to Qdrant."""
chunks = chunk_text(text)
if not chunks:
return False
ts = datetime.now(timezone.utc).timestamp()
ok = False
for i, chunk in enumerate(chunks):
try:
er = await client.post(
f"{EMBED_URL}/api/embeddings",
json={"model": EMBED_MODEL, "prompt": chunk},
timeout=10.0,
)
if er.status_code != 200:
continue
vector = er.json()["embedding"]
pid = f"auto-{ts}-{i}"
payload = {
"text": chunk, "source": "auto_fact", "fact": fact,
"ingest_date": datetime.now(timezone.utc).isoformat(),
"type": "auto_fact", "topic": topic,
}
r = await client.put(
f"{QDRANT_URL}/collections/{RAG_COLLECTION}/points?wait=true",
json={"points": [{"id": pid, "vector": vector, "payload": payload}]},
timeout=10.0,
)
if r.status_code in (200, 201):
ok = True
except Exception as e:
log.warning(f"Qdrant upsert error: {e}")
return ok
async def ingest_auto_fact(facts: list[str], user_message: str,
assistant_message: str) -> int:
"""Persist pre-detected facts to memories + Qdrant.
Call this when no conflicts exist — silent ingest.
Returns the number of facts stored.
"""
from memory import add_memory, detect_topic
ingested = 0
async with httpx.AsyncClient() as client:
for fact in facts:
topic = detect_topic(fact)
add_memory(fact, topic=topic, source="auto")
ingested += 1
text = f"Q: {user_message}\nA: {assistant_message}"
await _upsert_fact(fact, text, topic, client)
if ingested:
log.info(f"Auto-ingested {ingested} fact(s) from conversation")
return ingested
async def confirm_fact_update(memory_id: int, old_fact: str, new_fact: str,
user_message: str, assistant_message: str) -> bool:
"""Confirm a user-accepted fact update: replace memory + Qdrant entry."""
from memory import update_memory, detect_topic
if not update_memory(memory_id, new_fact):
return False
topic = detect_topic(new_fact)
try:
async with httpx.AsyncClient() as client:
# scroll old points with matching fact and delete them
scroll_r = await client.post(
f"{QDRANT_URL}/collections/{RAG_COLLECTION}/points/scroll",
json={
"filter": {"must": [{"key": "fact", "match": {"value": old_fact}}]},
"limit": 100,
"with_payload": False,
},
timeout=10.0,
)
if scroll_r.status_code == 200:
ids = [p["id"] for p in scroll_r.json().get("result", [])]
if ids:
await client.post(
f"{QDRANT_URL}/collections/{RAG_COLLECTION}/points/delete",
json={"points": ids},
timeout=10.0,
)
text = f"Q: {user_message}\nA: {assistant_message}"
await _upsert_fact(new_fact, text, topic, client)
except Exception as e:
log.warning(f"Fact update RAG error: {e}")
log.info(f"Fact updated [memory_id={memory_id}]: {new_fact}")
return True
def chunk_text(text: str, chunk_size: int = 512, overlap: int = 128) -> list:
words = text.split()
target_words = int(chunk_size / 1.3)
overlap_words = int(overlap / 1.3)
if not words:
return []
chunks = []
start = 0
while start < len(words):
end = min(start + target_words, len(words))
chunks.append(" ".join(words[start:end]))
if end == len(words):
break
start += target_words - overlap_words
return chunks
async def query_rag(query: str, limit: int = 3) -> list:
try:
async with httpx.AsyncClient() as client:
embed_resp = await client.post(
f"{EMBED_URL}/api/embeddings",
json={"model": EMBED_MODEL, "prompt": query},
timeout=10.0,
)
if embed_resp.status_code != 200:
return []
vector = embed_resp.json()["embedding"]
search_resp = await client.post(
f"{QDRANT_URL}/collections/{RAG_COLLECTION}/points/search",
json={"vector": vector, "limit": limit, "with_payload": True},
timeout=10.0,
)
if search_resp.status_code != 200:
return []
results = search_resp.json().get("result", [])
for r in results:
pid = r.get("id")
if pid:
current = r.get("payload", {}).get("retrieval_count", 0) or 0
asyncio.ensure_future(_update_retrieval_count(pid, current))
return results
except Exception as e:
log.warning(f"RAG query error: {e}")
return []
async def build_system_prompt(db, extra_prompt: str = "", user_message: str = "") -> str:
parts = []
settings = {row["key"]: row["value"] for row in db.execute("SELECT key, value FROM settings").fetchall()}
if settings.get("profile_enabled", "true") == "true":
profile = db.execute("SELECT content FROM profile WHERE id = 1").fetchone()
if profile and profile["content"].strip():
parts.append(profile["content"].strip())
if settings.get("memory_enabled", "true") == "true" and user_message:
memories = search_memories(user_message, limit=5)
if memories:
memory_lines = [f"- {m['fact']}" for m in memories]
parts.append("## Relevant Context from Memory\n" + "\n".join(memory_lines))
log.debug(f"Injected {len(memories)} memories into context")
if user_message:
try:
rag_results = await query_rag(user_message)
if rag_results:
rag_lines = [r["payload"]["text"] for r in rag_results if r["score"] > RAG_SCORE_THRESHOLD]
if rag_lines:
parts.append("## Retrieved Context\n" + "\n\n---\n\n".join(rag_lines))
log.info(f"RAG injected {len(rag_lines)} chunks into context")
except Exception as e:
log.warning(f"RAG injection error: {e}")
if settings.get("skills_enabled", "true") == "true":
active_skills = [s for s in list_skills_with_state(db) if s["enabled"]]
if active_skills:
parts.append(format_active_skills_prompt(active_skills))
if extra_prompt and extra_prompt.strip():
parts.append(extra_prompt.strip())
return "\n\n---\n\n".join(parts) if parts else ""
readme.md
+354
View File
@@ -0,0 +1,354 @@
# ⚡ JarvisChat v1.7.6
![screenshot](docs/images/screenshot.png)
**A lightweight Ollama coding companion with persistent memory, web search, and real-time system monitoring.**
Built with FastAPI + SQLite + Jinja2. Runs on Python 3.13. No Docker required.
Developer wiki: [docs/wiki/Home.md](docs/wiki/Home.md)
Core architecture deep-dive: [docs/wiki/Developer-Architecture.md](docs/wiki/Developer-Architecture.md)
## Security Scope Disclaimer
JarvisChat is designed for local and home-lab use (same host or trusted LAN).
JarvisChat may technically work with frontier or commercial AI endpoints, but the author does not recommend or support that usage.
Supported deployments are contained local/home-lab environments.
By default, API access is limited to loopback + private LAN CIDRs. You can override with `JARVISCHAT_ALLOWED_CIDRS` (comma-separated CIDRs) and optionally trust reverse-proxy forwarding with `JARVISCHAT_TRUST_X_FORWARDED_FOR=true`.
If you deploy outside a trusted local subnet, your risk profile changes significantly and the default protections here may be insufficient.
Use at your own risk. No warranty is provided for Internet-exposed deployments.
## What's New in v1.7.x
- **Security hardening suite completed** - request rate limits, payload caps, settings allowlist, safe error envelopes, and LAN CIDR gate controls
- **Customer-safe incident handling** - client-facing errors include support-friendly incident keys while full traces remain in server logs
- **Streaming and regression test expansion** - automated coverage for SSE chat/search paths, memory remember/forget command handling, and auth/guardrail behavior
- **Skills framework (Phase 1)** - built-in local skill registry with per-skill enable controls, API endpoints, and bounded prompt injection
- **Skills WebUX controls** - Settings modal now includes a master skills toggle and per-skill toggles for admin users
## What's New in v1.6.x
- **Guest/admin capability split** - guest chat by default with 4-digit admin PIN for advanced or destructive operations
- **Session + lockout controls** - session lifecycle endpoints, heartbeat, logout/revoke behavior, failed PIN lockout protections, and auth audit events
- **Browser request protections** - strict origin checks for state-changing requests and admin-only write enforcement
- **Unsafe link protection** - outbound search links sanitized to allow only http/https absolute URLs
- **Operational stability fixes** - safer first-boot PIN policy handling and memory-search tokenization fix for punctuation/FTS edge cases
## What's New in v1.5.0
- **Explicit Web Search Button** — 🔍 button next to SEND forces a web search, bypassing model uncertainty detection
- **Orange Search Styling** — Search results, WEB badge, and search button share consistent orange color scheme
- **Expanded Refusal Patterns** — Added "As an AI model", "based on my training data", "I don't have the capability"
- **Code cleanup** — Removed unused `JSONResponse` import and dead `raw_results_md` variable
- **Bug fixes** — Replaced bare `except` clauses with `except Exception`; corrected `add_memory()` return type to `int | None`; updated `TemplateResponse` call to Starlette's current API signature
## What's New in v1.4.0
- **FTS5 Memory System**: Say "remember that..." to store facts — they're automatically retrieved by relevance and injected into context
- **Forget Command**: Say "forget about..." to remove memories
- **Memory Toggle**: Enable/disable memory injection from topbar or settings
- **Multi-file Structure**: Backend and frontend separated for easier maintenance
## Features
- **Persistent Memory** — SQLite FTS5 full-text search for fast, relevant memory retrieval
- **Web Search** — SearXNG integration for automatic web lookups when the model is uncertain
- **Explicit Search** — 🔍 button to force web search without waiting for model uncertainty
- **Profile Injection** — Custom system prompt injected into every conversation
- **System Presets** — Save and switch between different system prompts
- **Real-time Stats** — CPU, RAM, GPU, VRAM monitoring in sidebar
- **Token Thermometer** — Visual context window usage indicator
- **Streaming Responses** — Server-sent events for real-time token display
- **Conversation History** — SQLite-backed chat persistence with mass-delete option
- **Model Switching** — Change Ollama models on the fly
## Current WiP (Prioritized)
Canonical backlog: [docs/wiki/current-wip.md](docs/wiki/current-wip.md)
Scope boundary: local-first (same-host Ollama), optional RFC1918 LAN endpoints, no public Internet AI endpoints by default.
Total identified items: 26
Top 10 (brief):
1. P0 [DONE]: Add auth for write/admin endpoints
2. P0 [DONE]: Add CSRF/origin protection for state-changing requests
3. P0 [DONE]: Block unsafe URL schemes in rendered links
4. P0 [DONE]: Add rate limiting and request size limits
5. P1 [DONE]: Restrict `/api/settings` updates to allowlisted keys
6. P1: Add pagination + hard caps for list APIs
7. P1 [DONE]: Replace raw exception leakage with safe client errors
8. P1 [DONE]: Add automated tests for streaming/search/memory paths
9. P2 [DONE]: Implement MCP-style skills/tool-call framework
10. P2: Implement heartbeat/check-in scheduler + summary endpoint
Item 1 executive summary: keep guest mode for conversational chat, require 4-digit admin PIN for advanced/destructive actions, and enforce local/LAN-only backend policy by default.
Implementation status: complete (guest session by default + admin unlock + admin-only write enforcement + origin checks + safe-link sanitization + audit logging + rate/payload guardrails + capability tests).
## TODO
1. ~~Verify SearXNG and Docker services persist across reboots~~
2. Conversation search/filter by keyword
3. Export conversation to markdown/text
4. Keyboard shortcuts (Ctrl+N new chat, Ctrl+Enter send)
5. Retry button on assistant messages
6. Source links — clickable links when search used
7. Allow conversation renaming
8. Multiple profiles — coding/sysadmin/general
9. Auto-generate conversation tags (client-side KWIC, top 5, filterable badges)
10. Image input support — pull vision model, file input/drag-drop, base64 encode, pass `images` array to Ollama `/api/chat`
11. Split-screen option for btop display
12. Skills as markdown files — `/opt/jarvischat/skills/`, YAML frontmatter + instructions, injected into context for tool calls
13. Heartbeats / proactive check-ins — cron + endpoint for daily briefings, HA anomaly alerts
14. Model info button — (i) icon next to Model dropdown, shows div with model description, last updated date, best-use purpose
15. Set default model — toggle any model as the default selection
16. Hide/remove model from list — exclude models from dropdown
17. Update model function — trigger `ollama pull` for selected model from UI
18. Add mouseover tooltip to SEND button
## File Structure
```
/opt/jarvischat/
├── app.py # FastAPI backend
├── jarvischat.db # SQLite database (auto-created)
├── static/
│ └── logo.png # Logo image (optional)
└── templates/
└── index.html # Frontend
```
## Requirements
- Python 3.11+ (tested on 3.13)
- Ollama running locally or on network
- SearXNG (optional, for web search)
## Installation
### Fresh Install
```bash
# Create directory and venv
sudo mkdir -p /opt/jarvischat
sudo chown $USER:$USER /opt/jarvischat
cd /opt/jarvischat
python3 -m venv venv
# Install dependencies
./venv/bin/pip install fastapi uvicorn httpx psutil jinja2 python-multipart
# Set admin PIN before first startup (4 digits)
export JARVISCHAT_ADMIN_PIN=4827
# Create subdirectories
mkdir -p templates static
# Copy files
# (copy app.py to /opt/jarvischat/)
# (copy index.html to /opt/jarvischat/templates/)
# (copy logo.png to /opt/jarvischat/static/ — optional)
```
WARNING: Do not use `1234` as your admin PIN unless you accept weak local security.
NOTE: First boot now requires `JARVISCHAT_ADMIN_PIN` unless you explicitly opt into insecure fallback with `JARVISCHAT_ALLOW_DEFAULT_PIN=true`.
### Upgrading from v1.4.x
```bash
cd /opt/jarvischat
# Backup
cp app.py app.py.bak
cp templates/index.html templates/index.html.bak
# Copy new files
# (copy app.py, replacing old version)
# (copy index.html to templates/)
# Restart
sudo systemctl restart jarvischat
```
## Systemd Service
Create `/etc/systemd/system/jarvischat.service`:
```ini
[Unit]
Description=JarvisChat - Local Ollama Web Interface
After=network.target
[Service]
Type=simple
User=jarvischat
Group=jarvischat
WorkingDirectory=/opt/jarvischat
ExecStart=/opt/jarvischat/venv/bin/uvicorn app:app --host 0.0.0.0 --port 8080
Restart=always
RestartSec=5
[Install]
WantedBy=multi-user.target
```
```bash
sudo systemctl daemon-reload
sudo systemctl enable jarvischat
sudo systemctl start jarvischat
```
## Memory Commands
In chat, natural language triggers memory operations:
| You say | What happens |
|---------|--------------|
| "remember that I prefer Rust over Go" | Stores as `preference` |
| "remember that JarvisChat runs on port 8080" | Stores as `infrastructure` |
| "note that the deadline is Friday" | Stores as `general` |
| "forget about the deadline" | Removes matching memories |
Memories are automatically searched based on your message content and injected into the system prompt when relevant.
### Memory Topics
Memories are auto-categorized:
- `preference` — likes, dislikes, choices
- `project` — active work, repos, tasks
- `infrastructure` — servers, services, configs
- `personal` — name, location, background
- `general` — everything else
## API Endpoints
### Memory
| Method | Endpoint | Description |
|--------|----------|-------------|
| GET | `/api/memories` | List all memories |
| POST | `/api/memories` | Add memory `{"fact": "...", "topic": "general"}` |
| DELETE | `/api/memories/{rowid}` | Delete memory by ID |
| GET | `/api/memories/search?q=term` | Search memories |
| GET | `/api/memories/stats` | Get counts by topic |
### Chat & Models
| Method | Endpoint | Description |
|--------|----------|-------------|
| GET | `/api/models` | List available Ollama models |
| POST | `/api/chat` | Send message (streaming SSE) |
| POST | `/api/search` | Explicit web search (streaming SSE) |
| POST | `/api/show` | Get model info (context size) |
| GET | `/api/ps` | Get running models |
### Settings & Profile
| Method | Endpoint | Description |
|--------|----------|-------------|
| GET | `/api/profile` | Get profile content |
| PUT | `/api/profile` | Update profile |
| GET | `/api/profile/default` | Get default profile |
| GET | `/api/settings` | Get settings |
| PUT | `/api/settings` | Update settings |
### Conversations
| Method | Endpoint | Description |
|--------|----------|-------------|
| GET | `/api/conversations` | List conversations |
| GET | `/api/conversations/{id}` | Get conversation with messages |
| DELETE | `/api/conversations/{id}` | Delete conversation |
| DELETE | `/api/conversations` | Delete ALL conversations |
### Presets
| Method | Endpoint | Description |
|--------|----------|-------------|
| GET | `/api/presets` | List presets |
| POST | `/api/presets` | Create preset |
| PUT | `/api/presets/{id}` | Update preset |
| DELETE | `/api/presets/{id}` | Delete preset |
### System
| Method | Endpoint | Description |
|--------|----------|-------------|
| GET | `/api/stats` | CPU, RAM, GPU, VRAM stats |
| GET | `/api/search/status` | SearXNG availability |
## Configuration
Settings are stored in the `settings` table and include:
- `profile_enabled` — Inject profile into chats (true/false)
- `search_enabled` — Auto web search (true/false)
- `memory_enabled` — Memory injection (true/false)
- `default_model` — Default Ollama model
- `searxng_url` — SearXNG instance URL (default: `http://localhost:8888`)
## Testing Memory
```bash
# Add a memory via API
curl -X POST http://jarvis:8080/api/memories \
-H "Content-Type: application/json" \
-d '{"fact": "User prefers native installs over Docker", "topic": "preference"}'
# Search memories
curl "http://jarvis:8080/api/memories/search?q=docker"
# List all memories
curl http://jarvis:8080/api/memories
# Get stats
curl http://jarvis:8080/api/memories/stats
```
Or in chat:
1. Say "remember that I hate YAML"
2. Later ask "what markup languages should I avoid?"
3. JarvisChat will inject the YAML preference into context
## Troubleshooting
### Service won't start
Check logs:
```bash
journalctl -u jarvischat -n 50 --no-pager
```
Common issues:
- Missing `jinja2`: `./venv/bin/pip install jinja2`
- Missing `templates/` directory
- Wrong permissions on `/opt/jarvischat`
### Memory not working
1. Check memory is enabled (🧠 MEM ON in topbar)
2. Verify memories exist: `curl http://jarvis:8080/api/memories`
3. Check FTS5 table: `sqlite3 jarvischat.db "SELECT * FROM memories_fts;"`
### Web search not working
1. Verify SearXNG is running: `curl http://localhost:8888/search?q=test&format=json`
2. Check search status: `curl http://jarvis:8080/api/search/status`
3. Ensure JSON format is enabled in SearXNG settings
## License
MIT
## Repository
Gitea: `ssh://gitea@llgit.llamachile.tube:1319/gramps/jarvisChat.git`
requirements.txt
-3
View File
@@ -1,6 +1,3 @@
fastapi>=0.115.0 fastapi>=0.115.0
uvicorn[standard]>=0.32.0 uvicorn[standard]>=0.32.0
httpx>=0.27.0 httpx>=0.27.0
pypdf>=5.0.0
python-multipart>=0.0.9
aio-pika>=9.0.0
routers/__init__.py
View File
routers/chat.py
-249
View File
@@ -1,249 +0,0 @@
"""JarvisChat routers - /api/chat streaming endpoint."""
import asyncio
import json
import logging
import uuid
from datetime import datetime, timezone
import httpx
from fastapi import APIRouter, HTTPException, Request
from fastapi.responses import StreamingResponse
from config import DEFAULT_MODEL, LLAMA_SERVER_BASE
from db import get_db, get_upload_context
from memory import process_remember_command, auto_detect_facts, check_fact_conflicts
from rag import build_system_prompt, ingest_auto_fact
from search import (calculate_perplexity, is_uncertain, is_refusal,
clean_hedging, format_search_results, format_direct_answer,
extract_search_query, query_searxng)
from security import read_json_body, log_incident, BODY_LIMIT_CHAT_BYTES
from config import MAX_CHAT_MESSAGE_CHARS, MODEL_CONTEXT_LENGTH
log = logging.getLogger("caic")
router = APIRouter()
def parse_llama_stream_chunk(line: str) -> tuple:
if line.startswith("data: "):
line = line[6:]
if line.strip() == "[DONE]":
return None, True, {}, []
try:
chunk = json.loads(line)
choices = chunk.get("choices", [])
if choices:
delta = choices[0].get("delta", {})
token = delta.get("content")
finish = choices[0].get("finish_reason")
stats = {}
logprobs_list = []
logprobs_info = choices[0].get("logprobs")
if logprobs_info:
content_logprobs = logprobs_info.get("content", [])
for entry in content_logprobs:
if "logprob" in entry:
logprobs_list.append({"logprob": entry["logprob"]})
if finish == "stop":
usage = chunk.get("usage", {})
stats["tokens_per_sec"] = usage.get("tokens_per_second", 0.0)
stats["completion_tokens"] = usage.get("completion_tokens", 0)
stats["prompt_tokens"] = usage.get("prompt_tokens", 0)
return token, finish == "stop", stats, logprobs_list
if "message" in chunk and "content" in chunk["message"]:
token = chunk["message"]["content"]
done = chunk.get("done", False)
stats = {}
if done:
eval_count = chunk.get("eval_count", 0)
eval_duration = chunk.get("eval_duration", 0)
stats["tokens_per_sec"] = (eval_count / (eval_duration / 1e9)) if eval_duration > 0 else 0
stats["completion_tokens"] = eval_count
return token, done, stats, []
except json.JSONDecodeError:
pass
return None, False, {}, []
@router.post("/api/chat")
async def chat(request: Request):
body = await read_json_body(request, BODY_LIMIT_CHAT_BYTES)
conv_id = body.get("conversation_id")
user_message = body.get("message", "").strip()
if len(user_message) > MAX_CHAT_MESSAGE_CHARS:
raise HTTPException(status_code=413, detail="Chat message is too long")
model = body.get("model", DEFAULT_MODEL)
preset_prompt = body.get("system_prompt", "")
upload_context_id = body.get("upload_context_id")
if not user_message:
raise HTTPException(status_code=400, detail="Empty message")
db = get_db()
now = datetime.now(timezone.utc).isoformat()
settings = {row["key"]: row["value"] for row in db.execute("SELECT key, value FROM settings").fetchall()}
search_enabled = settings.get("search_enabled", "true") == "true"
upload_doc = None
if upload_context_id:
ctx = get_upload_context(db, upload_context_id)
if ctx:
upload_doc = f"[ATTACHED DOCUMENT: {ctx['filename']}]\n{ctx['content']}\n[END DOCUMENT]"
else:
log.warning(f"upload_context_id {upload_context_id} not found or expired, continuing without it")
remember_response = process_remember_command(user_message)
if not conv_id:
conv_id = str(uuid.uuid4())
title = user_message[:80] + ("..." if len(user_message) > 80 else "")
db.execute("INSERT INTO conversations (id, title, model, created_at, updated_at) VALUES (?, ?, ?, ?, ?)",
(conv_id, title, model, now, now))
else:
db.execute("UPDATE conversations SET updated_at = ? WHERE id = ?", (now, conv_id))
db.execute("INSERT INTO messages (conversation_id, role, content, created_at) VALUES (?, ?, ?, ?)",
(conv_id, "user", user_message, now))
db.commit()
history_rows = db.execute(
"SELECT role, content FROM messages WHERE conversation_id = ? ORDER BY id ASC", (conv_id,)
).fetchall()
extra_prompt = preset_prompt
if upload_doc:
extra_prompt = (extra_prompt + "\n\n" + upload_doc) if extra_prompt else upload_doc
system_prompt = await build_system_prompt(db, extra_prompt, user_message)
db.close()
messages = []
if system_prompt:
messages.append({"role": "system", "content": system_prompt})
for row in history_rows:
messages.append({"role": row["role"], "content": row["content"]})
upstream_payload = {"model": model, "messages": messages, "stream": True, "logprobs": True}
async def stream_response():
full_response = []
all_logprobs = []
tokens_per_sec = 0.0
completion_tokens = 0
prompt_tokens = 0
rag_update = None
if remember_response:
yield f"data: {json.dumps({'token': remember_response + chr(10) + chr(10), 'conversation_id': conv_id})}\n\n"
async with httpx.AsyncClient() as client:
try:
async with client.stream(
"POST", f"{LLAMA_SERVER_BASE}/v1/chat/completions",
json=upstream_payload,
timeout=httpx.Timeout(300.0, connect=10.0),
) as resp:
async for line in resp.aiter_lines():
if line.strip():
token, done, stats, chunk_logprobs = parse_llama_stream_chunk(line)
if chunk_logprobs:
all_logprobs.extend(chunk_logprobs)
if token:
full_response.append(token)
yield f"data: {json.dumps({'token': token, 'conversation_id': conv_id})}\n\n"
if done:
tokens_per_sec = stats.get("tokens_per_sec", 0.0)
completion_tokens = stats.get("completion_tokens", 0)
prompt_tokens = stats.get("prompt_tokens", 0)
assistant_msg = "".join(full_response)
perplexity = calculate_perplexity(all_logprobs) if all_logprobs else 0.0
should_search = is_uncertain(all_logprobs) or is_refusal(assistant_msg)
if search_enabled and should_search:
yield f"data: {json.dumps({'searching': True, 'conversation_id': conv_id})}\n\n"
search_query = extract_search_query(user_message)
search_results = await query_searxng(search_query)
if search_results:
search_context = format_search_results(search_results)
augmented_messages = []
if system_prompt:
augmented_messages.append({"role": "system", "content": system_prompt + "\n\n" + search_context})
else:
augmented_messages.append({"role": "system", "content": search_context})
for row in history_rows[:-1]:
augmented_messages.append({"role": row["role"], "content": row["content"]})
augmented_messages.append({"role": "user", "content": user_message})
yield f"data: {json.dumps({'search_results': len(search_results), 'conversation_id': conv_id})}\n\n"
augmented_response = []
async with client.stream(
"POST", f"{LLAMA_SERVER_BASE}/v1/chat/completions",
json={"model": model, "messages": augmented_messages, "stream": True},
timeout=httpx.Timeout(300.0, connect=10.0),
) as resp2:
async for line in resp2.aiter_lines():
if line.strip():
token2, done2, _, _ = parse_llama_stream_chunk(line)
if token2:
augmented_response.append(token2)
if done2:
break
raw_response = "".join(augmented_response) or assistant_msg
cleaned_response = clean_hedging(raw_response)
if is_refusal(cleaned_response) or len(cleaned_response) < 20:
cleaned_response = format_direct_answer(user_message, search_results)
yield f"data: {json.dumps({'token': cleaned_response, 'conversation_id': conv_id, 'augmented': True})}\n\n"
saved_msg = cleaned_response + "\n\n---\n*🔍 Enhanced with web search results*"
if remember_response:
saved_msg = remember_response + "\n\n" + saved_msg
db2 = get_db()
db2.execute("INSERT INTO messages (conversation_id, role, content, created_at) VALUES (?, ?, ?, ?)",
(conv_id, "assistant", saved_msg, datetime.now(timezone.utc).isoformat()))
db2.commit()
db2.close()
facts = auto_detect_facts(user_message, cleaned_response)
if facts:
conflicts = check_fact_conflicts(facts)
if conflicts:
rag_update = {"conflicts": conflicts}
else:
asyncio.ensure_future(ingest_auto_fact(facts, user_message, cleaned_response))
yield f"data: {json.dumps({'done': True, 'conversation_id': conv_id, 'searched': True, 'perplexity': round(perplexity, 2), 'tokens_per_sec': round(tokens_per_sec, 1), 'prompt_tokens': prompt_tokens, 'completion_tokens': completion_tokens, 'context_length': MODEL_CONTEXT_LENGTH, **(rag_update and {'rag_update_suggestion': rag_update} or {})})}\n\n"
return
saved_msg = assistant_msg
if remember_response:
saved_msg = remember_response + "\n\n" + saved_msg
db2 = get_db()
db2.execute("INSERT INTO messages (conversation_id, role, content, created_at) VALUES (?, ?, ?, ?)",
(conv_id, "assistant", saved_msg, datetime.now(timezone.utc).isoformat()))
db2.commit()
db2.close()
facts = auto_detect_facts(user_message, assistant_msg)
if facts:
conflicts = check_fact_conflicts(facts)
if conflicts:
rag_update = {"conflicts": conflicts}
else:
asyncio.ensure_future(ingest_auto_fact(facts, user_message, assistant_msg))
yield f"data: {json.dumps({'done': True, 'conversation_id': conv_id, 'perplexity': round(perplexity, 2), 'tokens_per_sec': round(tokens_per_sec, 1), 'prompt_tokens': prompt_tokens, 'completion_tokens': completion_tokens, 'context_length': MODEL_CONTEXT_LENGTH, **(rag_update and {'rag_update_suggestion': rag_update} or {})})}\n\n"
except httpx.RemoteProtocolError:
pass
except httpx.ConnectError:
yield f"data: {json.dumps({'error': 'Cannot connect to inference server. Is it running?'})}\n\n"
except Exception as e:
incident_key = log_incident("chat_stream", message="Inference stream failure during chat response",
request=request, exc=e)
yield f"data: {json.dumps({'error': 'Chat response generation failed before completion. Use the incident key for support lookup.', 'error_key': incident_key})}\n\n"
return StreamingResponse(stream_response(), media_type="text/event-stream")
routers/cluster.py
-23
View File
@@ -1,23 +0,0 @@
"""JarvisChat routers - Cluster status API."""
from fastapi import APIRouter
import cluster
router = APIRouter()
@router.get("/api/cluster")
async def cluster_status():
return {
"nodes": {name: _strip_internal(node) for name, node in cluster.CLUSTER_NODES.items()},
"node_count": len(cluster.CLUSTER_NODES),
"coordinator": cluster.CLUSTER_COORDINATOR,
"events": list(cluster.CLUSTER_EVENTS),
}
def _strip_internal(node: dict) -> dict:
return {k: v for k, v in node.items() if k in {
"name", "type", "status", "capabilities", "active_model", "load",
"registered_at", "last_seen",
}}
routers/completions.py
-350
View File
@@ -1,350 +0,0 @@
"""
JarvisChat - /v1/chat/completions router.
OpenAI-compatible endpoint for IDE integration (Continue.dev, etc.).
Runs all requests through the full jC pipeline: profile + RAG + memory injection.
FIM (fill-in-the-middle) requests are proxied directly — not persisted.
Chat-style requests are persisted to conversation history.
Auth: static Bearer token via COMPLETIONS_API_KEY in config.
"""
import json
import logging
import uuid
from datetime import datetime, timezone
import httpx
from fastapi import APIRouter, HTTPException, Request
from fastapi.responses import StreamingResponse, JSONResponse
from config import DEFAULT_MODEL, LLAMA_SERVER_BASE, COMPLETIONS_API_KEY
from db import get_db
from rag import build_system_prompt
from routers.chat import parse_llama_stream_chunk
log = logging.getLogger("caic")
router = APIRouter()
def _check_api_key(request: Request):
auth = request.headers.get("Authorization", "")
if not auth.startswith("Bearer "):
raise HTTPException(status_code=401, detail="Missing Bearer token")
token = auth[7:].strip()
if token != COMPLETIONS_API_KEY:
raise HTTPException(status_code=401, detail="Invalid API key")
def _is_fim_request(body: dict) -> bool:
"""
FIM (fill-in-the-middle) requests use a 'prompt' + optional 'suffix' structure
rather than a 'messages' array. Continue.dev sends these for inline autocomplete.
We proxy them directly without pipeline injection or persistence.
"""
return "prompt" in body and "messages" not in body
def _build_openai_chunk(token: str, model: str, conv_id: str) -> str:
chunk = {
"id": f"chatcmpl-{conv_id}",
"object": "chat.completion.chunk",
"model": model,
"choices": [{
"index": 0,
"delta": {"content": token},
"finish_reason": None,
}],
}
return f"data: {json.dumps(chunk)}\n\n"
def _build_openai_stop_chunk(model: str, conv_id: str) -> str:
chunk = {
"id": f"chatcmpl-{conv_id}",
"object": "chat.completion.chunk",
"model": model,
"choices": [{
"index": 0,
"delta": {},
"finish_reason": "stop",
}],
}
return f"data: {json.dumps(chunk)}\n\n"
def _build_openai_response(content: str, model: str, conv_id: str) -> dict:
"""Non-streaming response envelope."""
return {
"id": f"chatcmpl-{conv_id}",
"object": "chat.completion",
"model": model,
"choices": [{
"index": 0,
"message": {"role": "assistant", "content": content},
"finish_reason": "stop",
}],
"usage": {"prompt_tokens": 0, "completion_tokens": 0, "total_tokens": 0},
}
@router.post("/v1/chat/completions")
@router.post("/v1/completions")
async def chat_completions(request: Request):
_check_api_key(request)
try:
body = await request.json()
except Exception:
raise HTTPException(status_code=400, detail="Invalid JSON body")
# --- FIM passthrough ---
if _is_fim_request(body):
return await _fim_passthrough(body)
# --- Chat completion ---
messages = body.get("messages", [])
if not messages:
raise HTTPException(status_code=400, detail="No messages provided")
model = body.get("model", DEFAULT_MODEL)
stream = body.get("stream", True)
# Extract the latest user message for RAG + conversation title
user_message = ""
for msg in reversed(messages):
if msg.get("role") == "user":
user_message = msg.get("content", "").strip()
break
if not user_message:
raise HTTPException(status_code=400, detail="No user message found")
# --- Persist conversation ---
db = get_db()
now = datetime.now(timezone.utc).isoformat()
conv_id = str(uuid.uuid4())
title = f"[IDE] {user_message[:72]}{'...' if len(user_message) > 72 else ''}"
db.execute(
"INSERT INTO conversations (id, title, model, created_at, updated_at) VALUES (?, ?, ?, ?, ?)",
(conv_id, title, model, now, now),
)
for msg in messages:
role = msg.get("role")
content = msg.get("content", "")
if role in ("user", "assistant"):
db.execute(
"INSERT INTO messages (conversation_id, role, content, created_at) VALUES (?, ?, ?, ?)",
(conv_id, role, content, now),
)
db.commit()
# --- Build system prompt through full jC pipeline ---
system_prompt = await build_system_prompt(db, "", user_message)
db.close()
# Assemble messages for upstream: inject jC system prompt, preserve history
upstream_messages = []
if system_prompt:
upstream_messages.append({"role": "system", "content": system_prompt})
# Strip any system messages from the incoming payload — jC owns the system prompt
for msg in messages:
if msg.get("role") != "system":
upstream_messages.append(msg)
upstream_payload = {
"model": model,
"messages": upstream_messages,
"stream": True, # always stream from upstream; we buffer if client wants non-stream
}
if stream:
return StreamingResponse(
_stream_chat(upstream_payload, model, conv_id, request),
media_type="text/event-stream",
)
else:
return await _blocking_chat(upstream_payload, model, conv_id, request)
async def _stream_chat(payload: dict, model: str, conv_id: str, request: Request):
"""Stream tokens to client in OpenAI SSE format, persist assistant response."""
full_response = []
async with httpx.AsyncClient() as client:
try:
async with client.stream(
"POST", f"{LLAMA_SERVER_BASE}/v1/chat/completions",
json=payload,
timeout=httpx.Timeout(300.0, connect=10.0),
) as resp:
async for line in resp.aiter_lines():
if not line.strip():
continue
token, done, _, _ = parse_llama_stream_chunk(line)
if token:
full_response.append(token)
yield _build_openai_chunk(token, model, conv_id)
if done:
break
yield _build_openai_stop_chunk(model, conv_id)
yield "data: [DONE]\n\n"
# Persist assistant response
assistant_msg = "".join(full_response)
if assistant_msg:
db = get_db()
db.execute(
"INSERT INTO messages (conversation_id, role, content, created_at) VALUES (?, ?, ?, ?)",
(conv_id, "assistant", assistant_msg, datetime.now(timezone.utc).isoformat()),
)
db.commit()
db.close()
except httpx.ConnectError:
err = {"error": {"message": "Cannot connect to inference server", "type": "connection_error"}}
yield f"data: {json.dumps(err)}\n\n"
except Exception as e:
log.error(f"completions stream error: {e}")
err = {"error": {"message": "Stream failed", "type": "server_error"}}
yield f"data: {json.dumps(err)}\n\n"
async def _blocking_chat(payload: dict, model: str, conv_id: str, request: Request) -> JSONResponse:
"""Accumulate full response, return as standard OpenAI JSON object."""
full_response = []
async with httpx.AsyncClient() as client:
try:
async with client.stream(
"POST", f"{LLAMA_SERVER_BASE}/v1/chat/completions",
json=payload,
timeout=httpx.Timeout(300.0, connect=10.0),
) as resp:
async for line in resp.aiter_lines():
if not line.strip():
continue
token, done, _, _ = parse_llama_stream_chunk(line)
if token:
full_response.append(token)
if done:
break
except httpx.ConnectError:
raise HTTPException(status_code=503, detail="Cannot connect to inference server")
except Exception as e:
log.error(f"completions blocking error: {e}")
raise HTTPException(status_code=500, detail="Inference request failed")
assistant_msg = "".join(full_response)
if assistant_msg:
db = get_db()
db.execute(
"INSERT INTO messages (conversation_id, role, content, created_at) VALUES (?, ?, ?, ?)",
(conv_id, "assistant", assistant_msg, datetime.now(timezone.utc).isoformat()),
)
db.commit()
db.close()
return JSONResponse(content=_build_openai_response(assistant_msg, model, conv_id))
@router.post("/v1/fim/completions")
@router.post("/fim/completions")
async def fim_completions(request: Request):
_check_api_key(request)
try:
body = await request.json()
except Exception:
raise HTTPException(status_code=400, detail="Invalid JSON body")
prefix = body.get("prompt", "")
suffix = body.get("suffix", "")
stream = body.get("stream", True)
max_tokens = body.get("max_tokens", 128)
fim_prompt = f"<|fim_prefix|>{prefix}<|fim_suffix|>{suffix}<|fim_middle|>"
upstream = {
"prompt": fim_prompt,
"n_predict": max_tokens,
"temperature": body.get("temperature", 0),
"stop": body.get("stop", []),
"stream": True,
"cache_prompt": False,
}
async def _stream_fim():
async with httpx.AsyncClient() as client:
try:
async with client.stream(
"POST", f"{LLAMA_SERVER_BASE}/completion",
json=upstream,
timeout=httpx.Timeout(30.0, connect=5.0),
) as resp:
async for line in resp.aiter_lines():
if not line or not line.startswith("data: "):
continue
try:
d = json.loads(line[6:])
content = d.get("content", "")
if content:
chunk = {
"choices": [{"delta": {"content": content}, "index": 0}]
}
yield f"data: {json.dumps(chunk)}\n\n"
if d.get("stop"):
break
except json.JSONDecodeError:
continue
yield "data: [DONE]\n\n"
except httpx.ConnectError:
yield f"data: {json.dumps({'error': 'Cannot connect to inference server'})}\n\n"
if stream:
return StreamingResponse(_stream_fim(), media_type="text/event-stream")
# Non-streaming: accumulate and return
full = []
async for chunk in _stream_fim():
if chunk.startswith("data: [DONE]"):
break
try:
d = json.loads(chunk[6:])
content = d.get("choices", [{}])[0].get("delta", {}).get("content", "")
if content:
full.append(content)
except (json.JSONDecodeError, IndexError):
pass
return JSONResponse(content={
"choices": [{"text": "".join(full), "index": 0, "finish_reason": "stop"}],
"usage": {},
})
async def _fim_passthrough(body: dict) -> JSONResponse:
"""
Proxy FIM requests directly to llama-server without pipeline injection.
Not persisted — autocomplete noise has no RAG value.
"""
async with httpx.AsyncClient() as client:
try:
resp = await client.post(
f"{LLAMA_SERVER_BASE}/completion",
json=body,
timeout=httpx.Timeout(30.0, connect=5.0),
)
data = resp.json()
wrapped = {
"choices": [{
"text": data.get("content", ""),
"index": 0,
"finish_reason": data.get("stop", False),
}],
"usage": {},
}
return JSONResponse(content=wrapped, status_code=resp.status_code)
except httpx.ConnectError:
raise HTTPException(status_code=503, detail="Cannot connect to inference server")
except Exception as e:
log.error(f"FIM passthrough error: {e}")
raise HTTPException(status_code=500, detail="FIM request failed")
routers/conversations.py
-91
View File
@@ -1,91 +0,0 @@
"""JarvisChat routers - Conversation CRUD."""
import logging
import uuid
from datetime import datetime, timezone
from fastapi import APIRouter, HTTPException, Request
from db import get_db
from security import read_json_body, BODY_LIMIT_DEFAULT_BYTES
from config import DEFAULT_MODEL, MAX_CONVERSATION_TITLE_CHARS
log = logging.getLogger("caic")
router = APIRouter()
@router.get("/api/conversations")
async def list_conversations():
db = get_db()
rows = db.execute("SELECT * FROM conversations ORDER BY updated_at DESC").fetchall()
result = []
for r in rows:
c = dict(r)
attach_count = db.execute(
"SELECT COUNT(*) FROM upload_context WHERE conversation_id = ?", (c["id"],)
).fetchone()[0]
c["attachment_count"] = attach_count
result.append(c)
db.close()
return result
@router.post("/api/conversations")
async def create_conversation(request: Request):
body = await read_json_body(request, BODY_LIMIT_DEFAULT_BYTES)
conv_id = str(uuid.uuid4())
now = datetime.now(timezone.utc).isoformat()
model = body.get("model", DEFAULT_MODEL)
title = str(body.get("title", "New Chat"))[:MAX_CONVERSATION_TITLE_CHARS]
db = get_db()
db.execute("INSERT INTO conversations (id, title, model, created_at, updated_at) VALUES (?, ?, ?, ?, ?)",
(conv_id, title, model, now, now))
db.commit()
db.close()
return {"id": conv_id, "title": title, "model": model, "created_at": now, "updated_at": now}
@router.get("/api/conversations/{conv_id}")
async def get_conversation(conv_id: str):
db = get_db()
conv = db.execute("SELECT * FROM conversations WHERE id = ?", (conv_id,)).fetchone()
if not conv:
db.close()
raise HTTPException(status_code=404, detail="Conversation not found")
messages = db.execute("SELECT * FROM messages WHERE conversation_id = ? ORDER BY id ASC", (conv_id,)).fetchall()
db.close()
return {"conversation": dict(conv), "messages": [dict(m) for m in messages]}
@router.put("/api/conversations/{conv_id}")
async def update_conversation(conv_id: str, request: Request):
body = await read_json_body(request, BODY_LIMIT_DEFAULT_BYTES)
db = get_db()
now = datetime.now(timezone.utc).isoformat()
if "title" in body:
db.execute("UPDATE conversations SET title = ?, updated_at = ? WHERE id = ?",
(str(body["title"])[:MAX_CONVERSATION_TITLE_CHARS], now, conv_id))
if "model" in body:
db.execute("UPDATE conversations SET model = ?, updated_at = ? WHERE id = ?",
(body["model"], now, conv_id))
db.commit()
db.close()
return {"status": "ok"}
@router.delete("/api/conversations/{conv_id}")
async def delete_conversation(conv_id: str):
db = get_db()
db.execute("DELETE FROM messages WHERE conversation_id = ?", (conv_id,))
db.execute("DELETE FROM conversations WHERE id = ?", (conv_id,))
db.commit()
db.close()
return {"status": "ok"}
@router.delete("/api/conversations")
async def delete_all_conversations():
db = get_db()
db.execute("DELETE FROM messages")
db.execute("DELETE FROM conversations")
db.commit()
db.close()
log.info("Deleted all conversations")
return {"status": "ok"}
routers/hardware.py
-15
View File
@@ -1,15 +0,0 @@
"""JarvisChat routers — Hardware self-assessment endpoint."""
import json
from fastapi import APIRouter
import hardware
router = APIRouter()
@router.get("/api/hardware")
async def get_hardware_state():
if hardware.HARDWARE_STATE_PATH.exists():
return json.loads(hardware.HARDWARE_STATE_PATH.read_text())
return {"status": "not_ready", "message": "Hardware assessment not yet complete"}
routers/ingest.py
-70
View File
@@ -1,70 +0,0 @@
"""JarvisChat routers - /api/ingest terminal command RAG hook."""
import logging
from datetime import datetime, timezone
import httpx
from fastapi import APIRouter, HTTPException, Request
from fastapi.responses import JSONResponse
from config import COMPLETIONS_API_KEY
from eviction import maybe_evict
from rag import chunk_text, QDRANT_URL, EMBED_URL, EMBED_MODEL, RAG_COLLECTION
log = logging.getLogger("caic")
router = APIRouter()
def _check_api_key(request: Request):
auth = request.headers.get("Authorization", "")
if not auth.startswith("Bearer "):
raise HTTPException(status_code=401, detail="Missing Bearer token")
token = auth[7:].strip()
if token != COMPLETIONS_API_KEY:
raise HTTPException(status_code=401, detail="Invalid API key")
@router.post("/api/ingest")
async def ingest_content(request: Request):
_check_api_key(request)
body = await request.json()
content = (body.get("content") or "").strip()
if not content:
raise HTTPException(status_code=422, detail="content is required")
source = str(body.get("source", "external")).strip() or "external"
metadata = body.get("metadata") or {}
chunks = chunk_text(content)
if not chunks:
raise HTTPException(status_code=422, detail="content produced no chunks")
ingested = 0
async with httpx.AsyncClient() as client:
for i, chunk in enumerate(chunks):
embed_resp = await client.post(
f"{EMBED_URL}/api/embeddings",
json={"model": EMBED_MODEL, "prompt": chunk},
timeout=30.0,
)
if embed_resp.status_code != 200:
log.warning(f"Ingest embedding failed for chunk {i}: {embed_resp.status_code}")
continue
vector = embed_resp.json()["embedding"]
point_id = f"ingest-{source}-{datetime.now(timezone.utc).timestamp()}-{i}"
payload = {"text": chunk, "source": source, "ingest_date": datetime.now(timezone.utc).isoformat(), "type": "ingest"}
payload.update(metadata)
upsert_resp = await client.put(
f"{QDRANT_URL}/collections/{RAG_COLLECTION}/points?wait=true",
json={"points": [{"id": point_id, "vector": vector, "payload": payload}]},
timeout=30.0,
)
if upsert_resp.status_code in (200, 201):
ingested += 1
else:
log.warning(f"Ingest Qdrant upsert failed for chunk {i}: {upsert_resp.status_code}")
if ingested > 0:
evicted = await maybe_evict()
if evicted:
log.info(f"Evicted {evicted} vectors after ingest")
return {"chunks_ingested": ingested, "source": source, "message": f"Ingested {ingested} chunks from {source}"}
routers/memories.py
-80
View File
@@ -1,80 +0,0 @@
"""JarvisChat routers - Memory CRUD API."""
from fastapi import APIRouter, HTTPException, Request
from typing import Optional
from db import get_db
from memory import add_memory, delete_memory, update_memory, get_all_memories, search_memories
from rag import confirm_fact_update
from security import read_json_body, BODY_LIMIT_DEFAULT_BYTES
from config import MAX_MEMORY_FACT_CHARS
router = APIRouter()
@router.get("/api/memories")
async def list_memories(topic: Optional[str] = None):
memories = get_all_memories(topic)
return {"memories": memories, "count": len(memories)}
@router.post("/api/memories")
async def create_memory(request: Request):
body = await read_json_body(request, BODY_LIMIT_DEFAULT_BYTES)
fact = str(body.get("fact", "")).strip()
if not fact:
raise HTTPException(status_code=400, detail="Memory fact is required")
if len(fact) > MAX_MEMORY_FACT_CHARS:
raise HTTPException(status_code=413, detail="Memory fact is too long")
rowid = add_memory(fact=fact, topic=body.get("topic", "general"), source=body.get("source", "manual"))
return {"rowid": rowid, "status": "ok"}
@router.delete("/api/memories/{rowid}")
async def remove_memory(rowid: int):
if not delete_memory(rowid):
raise HTTPException(status_code=404, detail="Memory not found")
return {"status": "ok"}
@router.put("/api/memories/{rowid}")
async def edit_memory(rowid: int, request: Request):
body = await read_json_body(request, BODY_LIMIT_DEFAULT_BYTES)
fact = str(body.get("fact", "")).strip()
if not fact:
raise HTTPException(status_code=400, detail="Memory fact is required")
if len(fact) > MAX_MEMORY_FACT_CHARS:
raise HTTPException(status_code=413, detail="Memory fact is too long")
if not update_memory(rowid, fact):
raise HTTPException(status_code=404, detail="Memory not found")
return {"status": "ok"}
@router.get("/api/memories/search")
async def search_memories_api(q: str, limit: int = 10):
results = search_memories(q, limit=limit)
return {"results": results, "count": len(results)}
@router.post("/api/memories/confirm-update")
async def confirm_memory_update(request: Request):
body = await read_json_body(request, BODY_LIMIT_DEFAULT_BYTES)
memory_id = body.get("memory_id")
new_fact = str(body.get("new_fact", "")).strip()
old_fact = str(body.get("old_fact", "")).strip()
user_message = str(body.get("user_message", "")).strip()
assistant_message = str(body.get("assistant_message", "")).strip()
if not memory_id or not new_fact:
raise HTTPException(status_code=400, detail="memory_id and new_fact are required")
ok = await confirm_fact_update(memory_id, old_fact, new_fact, user_message, assistant_message)
if not ok:
raise HTTPException(status_code=404, detail="Memory not found")
return {"status": "ok"}
@router.get("/api/memories/stats")
async def memory_stats():
db = get_db()
total = db.execute("SELECT COUNT(*) as c FROM memories").fetchone()["c"]
topics = db.execute("SELECT topic, COUNT(*) as c FROM memories GROUP BY topic ORDER BY c DESC").fetchall()
db.close()
return {"total": total, "by_topic": {row["topic"]: row["c"] for row in topics}}
routers/models.py
-77
View File
@@ -1,77 +0,0 @@
"""
JarvisChat routers - Model listing, system stats.
"""
import logging
from typing import Optional
import httpx
import psutil
from fastapi import APIRouter, HTTPException, Request
from config import LLAMA_SERVER_BASE
from gpu import get_gpu_stats
from security import read_json_body, BODY_LIMIT_DEFAULT_BYTES
log = logging.getLogger("caic")
router = APIRouter()
@router.get("/api/models")
async def list_models():
async with httpx.AsyncClient() as client:
try:
resp = await client.get(f"{LLAMA_SERVER_BASE}/v1/models", timeout=10)
data = resp.json()
models = [{"name": m["id"], "model": m["id"]} for m in data.get("data", [])]
return {"models": models}
except httpx.ConnectError:
raise HTTPException(status_code=502, detail="Cannot connect to inference server.")
@router.get("/api/ps")
async def running_models():
async with httpx.AsyncClient() as client:
try:
resp = await client.get(f"{LLAMA_SERVER_BASE}/v1/models", timeout=10)
return resp.json()
except httpx.ConnectError:
raise HTTPException(status_code=502, detail="Cannot connect to inference server.")
@router.post("/api/show")
async def show_model(request: Request):
body = await read_json_body(request, BODY_LIMIT_DEFAULT_BYTES)
async with httpx.AsyncClient() as client:
try:
resp = await client.post(f"{LLAMA_SERVER_BASE}/api/show", json=body, timeout=10)
return resp.json()
except httpx.ConnectError:
raise HTTPException(status_code=502, detail="Cannot connect to inference server.")
@router.get("/api/stats")
async def system_stats():
cpu_percent = psutil.cpu_percent(interval=0.1)
memory = psutil.virtual_memory()
gpu = get_gpu_stats()
return {
"cpu_percent": round(cpu_percent, 1),
"memory_percent": round(memory.percent, 1),
"memory_used_gb": round(memory.used / (1024**3), 1),
"memory_total_gb": round(memory.total / (1024**3), 1),
"gpu_percent": gpu["gpu_percent"],
"vram_percent": gpu["vram_percent"],
"gpu_available": gpu["available"],
}
@router.get("/api/search/status")
async def search_status():
from config import SEARXNG_BASE
async with httpx.AsyncClient() as client:
try:
resp = await client.get(f"{SEARXNG_BASE}/search",
params={"q": "test", "format": "json"}, timeout=5)
return {"available": resp.status_code == 200}
except Exception:
return {"available": False}
routers/presets.py
-61
View File
@@ -1,61 +0,0 @@
"""JarvisChat routers - System prompt presets."""
import uuid
from datetime import datetime, timezone
from fastapi import APIRouter, HTTPException, Request
from db import get_db
from security import read_json_body, BODY_LIMIT_DEFAULT_BYTES
from config import MAX_PRESET_NAME_CHARS, MAX_PRESET_PROMPT_CHARS
router = APIRouter()
@router.get("/api/presets")
async def list_presets():
db = get_db()
rows = db.execute("SELECT * FROM system_presets ORDER BY is_default DESC, name ASC").fetchall()
db.close()
return [dict(r) for r in rows]
@router.post("/api/presets")
async def create_preset(request: Request):
body = await read_json_body(request, BODY_LIMIT_DEFAULT_BYTES)
name = str(body.get("name", "")).strip()
prompt = str(body.get("prompt", "")).strip()
if not name or not prompt:
raise HTTPException(status_code=400, detail="Preset name and prompt are required")
if len(name) > MAX_PRESET_NAME_CHARS or len(prompt) > MAX_PRESET_PROMPT_CHARS:
raise HTTPException(status_code=413, detail="Preset fields are too long")
preset_id = str(uuid.uuid4())
now = datetime.now(timezone.utc).isoformat()
db = get_db()
db.execute("INSERT INTO system_presets (id, name, prompt, is_default, created_at) VALUES (?, ?, ?, 0, ?)",
(preset_id, name, prompt, now))
db.commit()
db.close()
return {"id": preset_id, "name": name, "prompt": prompt}
@router.put("/api/presets/{preset_id}")
async def update_preset(preset_id: str, request: Request):
body = await read_json_body(request, BODY_LIMIT_DEFAULT_BYTES)
name = str(body.get("name", "")).strip()
prompt = str(body.get("prompt", "")).strip()
if not name or not prompt:
raise HTTPException(status_code=400, detail="Preset name and prompt are required")
if len(name) > MAX_PRESET_NAME_CHARS or len(prompt) > MAX_PRESET_PROMPT_CHARS:
raise HTTPException(status_code=413, detail="Preset fields are too long")
db = get_db()
db.execute("UPDATE system_presets SET name = ?, prompt = ? WHERE id = ?", (name, prompt, preset_id))
db.commit()
db.close()
return {"status": "ok"}
@router.delete("/api/presets/{preset_id}")
async def delete_preset(preset_id: str):
db = get_db()
db.execute("DELETE FROM system_presets WHERE id = ? AND is_default = 0", (preset_id,))
db.commit()
db.close()
return {"status": "ok"}
routers/profile.py
-36
View File
@@ -1,36 +0,0 @@
"""JarvisChat routers - Profile."""
from datetime import datetime, timezone
from fastapi import APIRouter, HTTPException, Request
from db import get_db
from security import read_json_body, BODY_LIMIT_PROFILE_BYTES
from config import MAX_PROFILE_CHARS, DEFAULT_PROFILE
router = APIRouter()
@router.get("/api/profile")
async def get_profile():
db = get_db()
row = db.execute("SELECT content, updated_at FROM profile WHERE id = 1").fetchone()
db.close()
return ({"content": row["content"], "updated_at": row["updated_at"]} if row
else {"content": "", "updated_at": ""})
@router.put("/api/profile")
async def update_profile(request: Request):
body = await read_json_body(request, BODY_LIMIT_PROFILE_BYTES)
content = str(body.get("content", ""))
if len(content) > MAX_PROFILE_CHARS:
raise HTTPException(status_code=413, detail="Profile content is too long")
now = datetime.now(timezone.utc).isoformat()
db = get_db()
db.execute("UPDATE profile SET content = ?, updated_at = ? WHERE id = 1", (content, now))
db.commit()
db.close()
return {"status": "ok", "updated_at": now}
@router.get("/api/profile/default")
async def get_default_profile():
return {"content": DEFAULT_PROFILE}
routers/rag_admin.py
-60
View File
@@ -1,60 +0,0 @@
"""JarvisChat routers — RAG corpus management admin endpoints."""
import logging
import httpx
from fastapi import APIRouter, HTTPException, Request
from fastapi.responses import JSONResponse
from eviction import get_rag_operational_stats, EVICTION_LOG
from rag import QDRANT_URL, RAG_COLLECTION
log = logging.getLogger("caic")
router = APIRouter()
@router.get("/api/rag/stats")
async def rag_stats(request: Request):
if getattr(request.state, "session_role", "none") != "admin":
raise HTTPException(status_code=403, detail="Admin PIN required for this action")
stats = await get_rag_operational_stats()
stats["eviction_log_size"] = len(EVICTION_LOG)
return stats
@router.post("/api/rag/flush")
async def rag_flush(request: Request):
try:
async with httpx.AsyncClient() as client:
scroll_resp = await client.post(
f"{QDRANT_URL}/collections/{RAG_COLLECTION}/points/scroll",
json={"limit": 10000, "with_payload": False, "with_vector": False},
timeout=30.0,
)
if scroll_resp.status_code != 200:
return JSONResponse(status_code=502, content={"detail": f"Qdrant scroll failed: {scroll_resp.status_code}"})
all_points = scroll_resp.json().get("result", {}).get("points", [])
point_ids = [p["id"] for p in all_points]
if not point_ids:
return {"deleted_count": 0, "collection": RAG_COLLECTION, "status": "flushed"}
delete_resp = await client.post(
f"{QDRANT_URL}/collections/{RAG_COLLECTION}/points/delete",
json={"points": point_ids},
timeout=30.0,
)
if delete_resp.status_code not in (200, 201):
return JSONResponse(status_code=502, content={"detail": f"Qdrant delete failed: {delete_resp.status_code}"})
EVICTION_LOG.clear()
log.warning(f"RAG collection '{RAG_COLLECTION}' flushed ({len(point_ids)} points deleted)")
return {
"deleted_count": len(point_ids),
"collection": RAG_COLLECTION,
"status": "flushed",
}
except Exception as e:
log.warning(f"RAG flush error: {e}")
return JSONResponse(status_code=502, content={"detail": f"RAG flush error: {e}"})
routers/search_route.py
-107
View File
@@ -1,107 +0,0 @@
"""JarvisChat routers - /api/search explicit search endpoint."""
import json
import logging
import uuid
from datetime import datetime, timezone
import httpx
from fastapi import APIRouter, HTTPException, Request
from fastapi.responses import StreamingResponse
from config import DEFAULT_MODEL, LLAMA_SERVER_BASE, MAX_SEARCH_QUERY_CHARS
from db import get_db
from search import query_searxng, format_search_results
from routers.chat import parse_llama_stream_chunk
from security import read_json_body, log_incident, BODY_LIMIT_CHAT_BYTES
log = logging.getLogger("caic")
router = APIRouter()
@router.post("/api/search")
async def explicit_search(request: Request):
body = await read_json_body(request, BODY_LIMIT_CHAT_BYTES)
query = body.get("query", "").strip()
if len(query) > MAX_SEARCH_QUERY_CHARS:
raise HTTPException(status_code=413, detail="Search query is too long")
conv_id = body.get("conversation_id")
model = body.get("model", DEFAULT_MODEL)
if not query:
raise HTTPException(status_code=400, detail="Empty query")
db = get_db()
now = datetime.now(timezone.utc).isoformat()
if not conv_id:
conv_id = str(uuid.uuid4())
title = query[:70] + "..." if len(query) > 70 else query
db.execute("INSERT INTO conversations (id, title, model, created_at, updated_at) VALUES (?, ?, ?, ?, ?)",
(conv_id, title, model, now, now))
else:
db.execute("UPDATE conversations SET updated_at = ? WHERE id = ?", (now, conv_id))
db.execute("INSERT INTO messages (conversation_id, role, content, created_at) VALUES (?, ?, ?, ?)",
(conv_id, "user", query, now))
db.commit()
db.close()
async def stream_search():
yield f"data: {json.dumps({'conversation_id': conv_id, 'searching': True})}\n\n"
results = await query_searxng(query, max_results=5)
if not results:
error_msg = "No search results found."
yield f"data: {json.dumps({'token': error_msg, 'conversation_id': conv_id})}\n\n"
db2 = get_db()
db2.execute("INSERT INTO messages (conversation_id, role, content, created_at) VALUES (?, ?, ?, ?)",
(conv_id, "assistant", error_msg, datetime.now(timezone.utc).isoformat()))
db2.commit()
db2.close()
yield f"data: {json.dumps({'done': True, 'conversation_id': conv_id})}\n\n"
return
yield f"data: {json.dumps({'search_results': len(results), 'conversation_id': conv_id})}\n\n"
search_context = format_search_results(results)
messages = [
{"role": "system", "content": f"You have access to current web data. Answer directly using ONLY the data below. Be concise. No apologies. No disclaimers.\n\n{search_context}"},
{"role": "user", "content": query},
]
full_response = []
async with httpx.AsyncClient() as client:
try:
async with client.stream(
"POST", f"{LLAMA_SERVER_BASE}/v1/chat/completions",
json={"model": model, "messages": messages, "stream": True},
timeout=httpx.Timeout(300.0, connect=10.0),
) as resp:
async for line in resp.aiter_lines():
if line.strip():
token, done, _, _ = parse_llama_stream_chunk(line)
if token:
full_response.append(token)
yield f"data: {json.dumps({'token': token, 'conversation_id': conv_id})}\n\n"
if done:
break
except Exception as e:
incident_key = log_incident("search_summarization_stream",
message="Stream failure during explicit search summarization",
request=request, exc=e)
yield f"data: {json.dumps({'error': 'Search summarization could not complete right now.', 'error_key': incident_key})}\n\n"
return
summary = "".join(full_response)
saved_msg = f"{summary}\n\n---\n*🔍 Web search results*"
db2 = get_db()
db2.execute("INSERT INTO messages (conversation_id, role, content, created_at) VALUES (?, ?, ?, ?)",
(conv_id, "assistant", saved_msg, datetime.now(timezone.utc).isoformat()))
db2.commit()
db2.close()
yield f"data: {json.dumps({'done': True, 'conversation_id': conv_id, 'searched': True})}\n\n"
return StreamingResponse(stream_search(), media_type="text/event-stream")
routers/settings.py
-36
View File
@@ -1,36 +0,0 @@
"""JarvisChat routers - Settings."""
from fastapi import APIRouter, HTTPException, Request
from db import get_db
from security import read_json_body, BODY_LIMIT_DEFAULT_BYTES
from config import MAX_SETTINGS_KEYS, MAX_SETTINGS_VALUE_CHARS, ALLOWED_SETTINGS_KEYS
router = APIRouter()
@router.get("/api/settings")
async def get_settings():
db = get_db()
rows = db.execute("SELECT key, value FROM settings").fetchall()
db.close()
return {row["key"]: row["value"] for row in rows}
@router.put("/api/settings")
async def update_settings(request: Request):
body = await read_json_body(request, BODY_LIMIT_DEFAULT_BYTES)
if not isinstance(body, dict):
raise HTTPException(status_code=400, detail="Settings payload must be an object")
if len(body) > MAX_SETTINGS_KEYS:
raise HTTPException(status_code=413, detail="Too many settings in one request")
unknown_keys = sorted(key for key in body.keys() if str(key) not in ALLOWED_SETTINGS_KEYS)
if unknown_keys:
raise HTTPException(status_code=400, detail=f"Unknown setting key(s): {', '.join(unknown_keys)}")
db = get_db()
for key, value in body.items():
if len(str(key)) > 80 or len(str(value)) > MAX_SETTINGS_VALUE_CHARS:
db.close()
raise HTTPException(status_code=413, detail="Setting key/value too long")
db.execute("INSERT OR REPLACE INTO settings (key, value) VALUES (?, ?)", (key, str(value)))
db.commit()
db.close()
return {"status": "ok"}
routers/skills.py
-42
View File
@@ -1,42 +0,0 @@
"""JarvisChat routers - Skills."""
from fastapi import APIRouter, HTTPException, Request
from db import get_db, get_setting, list_skills_with_state, set_skill_enabled
from security import read_json_body, BODY_LIMIT_DEFAULT_BYTES
from config import MAX_SKILL_KEY_CHARS, SKILLS_BY_KEY
router = APIRouter()
@router.get("/api/skills")
async def list_skills():
db = get_db()
skills = list_skills_with_state(db)
db.close()
return {"skills": skills, "count": len(skills)}
@router.get("/api/skills/active")
async def list_active_skills():
db = get_db()
skills_enabled = get_setting(db, "skills_enabled", "true") == "true"
skills = list_skills_with_state(db)
db.close()
active = [s for s in skills if s["enabled"]] if skills_enabled else []
return {"skills": active, "count": len(active), "skills_enabled": skills_enabled}
@router.put("/api/skills/{skill_key}")
async def update_skill(skill_key: str, request: Request):
skill_key = skill_key.strip()
if len(skill_key) > MAX_SKILL_KEY_CHARS or skill_key not in SKILLS_BY_KEY:
raise HTTPException(status_code=404, detail="Unknown skill")
body = await read_json_body(request, BODY_LIMIT_DEFAULT_BYTES)
if "enabled" not in body or not isinstance(body.get("enabled"), bool):
raise HTTPException(status_code=400, detail="Field 'enabled' (boolean) is required")
db = get_db()
set_skill_enabled(db, skill_key, bool(body["enabled"]))
db.commit()
skills = list_skills_with_state(db)
db.close()
updated = next((s for s in skills if s["key"] == skill_key), None)
return {"status": "ok", "skill": updated}
routers/upload.py
-173
View File
@@ -1,173 +0,0 @@
"""JarvisChat routers - /api/upload file/document attachment endpoint."""
import json
import logging
import os
from datetime import datetime, timezone, timedelta
import httpx
from fastapi import APIRouter, HTTPException, Request, UploadFile, File, Form
from fastapi.responses import JSONResponse
from config import UPLOAD_DIR, MAX_UPLOAD_BYTES, SUPPORTED_UPLOAD_TYPES, UPLOAD_CONTEXT_EXPIRY_HOURS
from db import get_db, insert_upload_context, list_upload_context_by_conversation, delete_upload_context_by_id
from eviction import maybe_evict
from rag import chunk_text, QDRANT_URL, EMBED_URL, EMBED_MODEL, RAG_COLLECTION
log = logging.getLogger("caic")
router = APIRouter()
def _point_id(filename: str, chunk_idx: int) -> str:
return f"upload-{filename}-{chunk_idx}"
@router.post("/api/upload")
async def upload_file(
request: Request,
file: UploadFile = File(...),
mode: str = Form("both"),
conversation_id: str = Form(""),
):
if mode not in ("context", "ingest", "both"):
raise HTTPException(status_code=422, detail="mode must be context, ingest, or both")
if file.size and file.size > MAX_UPLOAD_BYTES:
return JSONResponse(status_code=413, content={"detail": f"File exceeds {MAX_UPLOAD_BYTES} byte limit"})
content_type = file.content_type or "application/octet-stream"
if content_type not in SUPPORTED_UPLOAD_TYPES:
return JSONResponse(status_code=415, content={"detail": f"Unsupported file type: {content_type}"})
raw_bytes = await file.read()
if not raw_bytes:
raise HTTPException(status_code=422, detail="Empty file")
if content_type == "application/pdf":
try:
from pypdf import PdfReader
import io
reader = PdfReader(io.BytesIO(raw_bytes))
extracted = "\n".join(page.extract_text() or "" for page in reader.pages)
except Exception as e:
log.warning(f"PDF extraction error: {e}")
raise HTTPException(status_code=422, detail="Failed to extract text from PDF")
else:
extracted = raw_bytes.decode("utf-8", errors="replace")
result = {"filename": file.filename, "size_bytes": len(raw_bytes), "mode": mode}
if mode in ("ingest", "both"):
os.makedirs(UPLOAD_DIR, exist_ok=True)
chunks = chunk_text(extracted)
ingested = 0
async with httpx.AsyncClient() as client:
for i, chunk in enumerate(chunks):
embed_resp = await client.post(
f"{EMBED_URL}/api/embeddings",
json={"model": EMBED_MODEL, "prompt": chunk},
timeout=30.0,
)
if embed_resp.status_code != 200:
log.warning(f"Embedding failed for chunk {i}: {embed_resp.status_code}")
continue
vector = embed_resp.json()["embedding"]
pid = _point_id(file.filename or "unnamed", i)
upsert_resp = await client.put(
f"{QDRANT_URL}/collections/{RAG_COLLECTION}/points?wait=true",
json={
"points": [{
"id": pid,
"vector": vector,
"payload": {"text": chunk, "source": file.filename, "upload_date": datetime.now(timezone.utc).isoformat(), "type": "upload"},
}]
},
timeout=30.0,
)
if upsert_resp.status_code in (200, 201):
ingested += 1
else:
log.warning(f"Qdrant upsert failed for chunk {i}: {upsert_resp.status_code}")
result["chunks_ingested"] = ingested
if ingested > 0:
evicted = await maybe_evict()
if evicted:
log.info(f"Evicted {evicted} vectors after upload")
if mode in ("context", "both"):
expires = (datetime.now(timezone.utc) + timedelta(hours=UPLOAD_CONTEXT_EXPIRY_HOURS)).isoformat()
db = get_db()
try:
cid = insert_upload_context(db, conversation_id or "", file.filename or "unnamed", extracted, expires, content_type)
db.commit()
result["context_id"] = cid
finally:
db.close()
result["message"] = f"Uploaded {file.filename}"
return result
@router.patch("/api/upload/{context_id}/link")
async def link_upload_to_conversation(context_id: int, request: Request):
body = await request.json()
conv_id = body.get("conversation_id", "").strip()
if not conv_id:
raise HTTPException(status_code=422, detail="conversation_id required")
db = get_db()
try:
row = db.execute("SELECT id FROM upload_context WHERE id = ?", (context_id,)).fetchone()
if not row:
raise HTTPException(status_code=404, detail="Upload context not found")
db.execute("UPDATE upload_context SET conversation_id = ? WHERE id = ?", (conv_id, context_id))
db.commit()
return {"status": "ok"}
finally:
db.close()
@router.get("/api/upload/by-conversation/{conv_id}")
async def get_upload_by_conversation(conv_id: str):
db = get_db()
try:
items = list_upload_context_by_conversation(db, conv_id)
return items
finally:
db.close()
@router.delete("/api/upload/{context_id}")
async def delete_upload(context_id: int):
db = get_db()
try:
row = db.execute("SELECT filename FROM upload_context WHERE id = ?", (context_id,)).fetchone()
if not row:
raise HTTPException(status_code=404, detail="Attachment not found")
filename = row["filename"]
if not filename:
filename = "unnamed"
delete_upload_context_by_id(db, context_id)
db.commit()
finally:
db.close()
try:
async with httpx.AsyncClient() as client:
scroll_resp = await client.post(
f"{QDRANT_URL}/collections/{RAG_COLLECTION}/points/scroll",
json={"filter": {"must": [{"key": "source", "match": {"value": filename}}]}, "limit": 100, "with_payload": False},
timeout=10.0,
)
if scroll_resp.status_code == 200:
points = scroll_resp.json().get("result", {}).get("points", [])
point_ids = [p["id"] for p in points]
if point_ids:
await client.post(
f"{QDRANT_URL}/collections/{RAG_COLLECTION}/points/delete",
json={"points": point_ids},
timeout=10.0,
)
log.info(f"Deleted {len(point_ids)} Qdrant points for source '{filename}'")
except Exception as e:
log.warning(f"Qdrant cleanup for '{filename}' failed: {e}")
return {"status": "ok", "filename": filename}
search.py
-142
View File
@@ -1,142 +0,0 @@
"""
cAIc - SearXNG integration, perplexity scoring, refusal/hedge detection.
"""
import logging
import math
import re
from urllib.parse import urlparse
import httpx
from config import SEARXNG_BASE, PERPLEXITY_THRESHOLD, REFUSAL_PATTERNS, HEDGE_PATTERNS
log = logging.getLogger("caic")
def sanitize_outbound_url(url: str) -> str:
if not url:
return ""
candidate = url.strip()
parsed = urlparse(candidate)
if parsed.scheme.lower() in {"http", "https"} and parsed.netloc:
return candidate
return ""
def calculate_perplexity(logprobs: list) -> float:
if not logprobs:
return 0.0
avg_logprob = sum(lp["logprob"] for lp in logprobs) / len(logprobs)
return math.exp(-avg_logprob)
def is_uncertain(logprobs: list, threshold: float = PERPLEXITY_THRESHOLD) -> bool:
if not logprobs:
return False
perplexity = calculate_perplexity(logprobs)
log.info(f"Perplexity: {perplexity:.2f} (threshold: {threshold})")
return perplexity > threshold
def is_refusal(text: str) -> bool:
match = REFUSAL_PATTERNS.search(text)
if match:
log.info(f"Refusal detected: '{match.group()}'")
return True
return False
def clean_hedging(text: str) -> str:
cleaned = text
for pattern in HEDGE_PATTERNS:
cleaned = re.sub(pattern, "", cleaned, flags=re.IGNORECASE)
return cleaned.strip()
def format_search_results(results: list) -> str:
if not results:
return ""
lines = ["[LIVE WEB DATA]\n"]
for i, r in enumerate(results, 1):
lines.append(f"{i}. {r['title']}")
if r["content"]:
lines.append(f" {r['content']}")
lines.append("")
lines.append("\nAnswer directly using the data above. No apologies. No disclaimers. Just answer.")
return "\n".join(lines)
def format_direct_answer(question: str, results: list) -> str:
if not results:
return "No search results found."
lines = ["Here's what I found:\n"]
for r in results[:3]:
lines.append(f"**{r['title']}**")
if r["content"]:
lines.append(f"{r['content']}")
lines.append("")
return "\n".join(lines).strip()
def extract_search_query(user_message: str) -> str:
query = user_message.strip()
weather_lead = re.match(r"^(?:what('?s| is) the\s+)?(?:weather|temperature|forecast)\s+(?:in\s+|for\s+)?(.+)", query, re.IGNORECASE)
if weather_lead:
return (weather_lead.group(2) + " weather").strip()[:100]
price_lead = re.match(r"^(?:what('?s| is| are)\s+)?(?:the\s+)?(?:price|spot price)\s+(?:of\s+|for\s+)?(.+)", query, re.IGNORECASE)
if price_lead:
return (price_lead.group(2) + " price today USD").strip()[:100]
return query[:100]
async def query_searxng(query: str, max_results: int = 5) -> list:
log.info(f"Querying SearXNG: '{query}'")
async with httpx.AsyncClient() as client:
weather_match = re.search(
r"(?:weather|temperature|forecast)\s+(?:in\s+)?(.+?)(?:\s+right now|\s+today|\s+degrees)?$",
query, re.IGNORECASE,
)
if weather_match or "weather" in query.lower() or "temperature" in query.lower():
location = (
weather_match.group(1) if weather_match
else re.sub(r"(weather|temperature|forecast|right now|today|degrees)", "", query, flags=re.IGNORECASE).strip()
)
if location:
try:
resp = await client.get(f"https://wttr.in/{location}?format=3", timeout=10.0,
headers={"User-Agent": "curl/7.68.0"})
if resp.status_code == 200:
return [{"title": "Current Weather",
"url": sanitize_outbound_url(f"https://wttr.in/{location}"),
"content": resp.text.strip()}]
except Exception as e:
log.warning(f"wttr.in error: {e}")
try:
resp = await client.get(
f"{SEARXNG_BASE}/search",
params={"q": query, "format": "json", "categories": "general"},
timeout=10.0,
follow_redirects=True,
)
if resp.status_code == 200:
data = resp.json()
results = []
for answer in data.get("answers", []):
results.append({"title": "Direct Answer", "url": "", "content": answer})
for box in data.get("infoboxes", []):
content = box.get("content", "")
if not content and box.get("attributes"):
content = " | ".join([f"{a.get('label','')}: {a.get('value','')}" for a in box["attributes"]])
results.append({
"title": box.get("infobox", "Info"),
"url": sanitize_outbound_url(box.get("urls", [{}])[0].get("url", "") if box.get("urls") else ""),
"content": content,
})
for r in data.get("results", [])[:max_results]:
results.append({"title": r.get("title", ""), "url": sanitize_outbound_url(r.get("url", "")), "content": r.get("content", "")})
log.info(f"SearXNG returned {len(results)} results")
return results
except Exception as e:
log.error(f"SearXNG error: {e}")
return []
security.py
-177
View File
@@ -1,177 +0,0 @@
"""
cAIc - Security utilities.
PIN hashing, audit logging, incident tracking, CSRF/origin checks,
rate limiting, request helpers.
"""
import hashlib
import hmac
import json
import logging
import math
import os
import platform
import time
import uuid
from collections import defaultdict, deque
from datetime import datetime, timezone
from threading import Lock
from typing import Optional
from urllib.parse import urlparse
from fastapi import HTTPException, Request
from config import (
ALLOWED_NETWORKS, TRUST_X_FORWARDED_FOR, TRUSTED_ORIGINS,
BODY_LIMIT_DEFAULT_BYTES, BODY_LIMIT_CHAT_BYTES, BODY_LIMIT_PROFILE_BYTES, BODY_LIMIT_UPLOAD_BYTES,
RATE_WINDOW_SECONDS, RL_LOGIN_PER_WINDOW, RL_CHAT_PER_WINDOW,
RL_SEARCH_PER_WINDOW, RL_STATS_PER_WINDOW, RL_WRITE_PER_WINDOW,
RL_DEFAULT_PER_WINDOW, VERSION,
)
import ipaddress
log = logging.getLogger("caic")
SESSIONS: dict = {}
PIN_ATTEMPTS: dict = {}
RATE_EVENTS: dict = defaultdict(deque)
SESSION_LOCK = Lock()
RATE_LOCK = Lock()
def hash_pin(pin: str, salt_hex: Optional[str] = None) -> tuple:
salt = bytes.fromhex(salt_hex) if salt_hex else os.urandom(16)
digest = hashlib.pbkdf2_hmac("sha256", pin.encode("utf-8"), salt, 200_000)
return salt.hex(), digest.hex()
def audit_event(event: str, outcome: str, *, ip: str = "unknown", role: str = "none",
details: str = "", warning: bool = False) -> None:
payload = {"event": event, "outcome": outcome, "ip": ip, "role": role, "details": details[:300]}
msg = "AUDIT " + json.dumps(payload, separators=(",", ":"))
if warning:
log.warning(msg)
else:
log.info(msg)
def create_incident_key() -> str:
ts = datetime.now(timezone.utc).strftime("%Y%m%d-%H%M%S")
return f"INC-{ts}-{uuid.uuid4().hex[:8].upper()}"
def customer_error_envelope(message: str, incident_key: str) -> dict:
return {
"detail": message, "error_key": incident_key,
"error": {"message": message, "incident_key": incident_key,
"support_hint": "Share this incident key for exact diagnostics."},
}
def log_incident(event: str, *, message: str, request: Optional[Request] = None,
exc: Optional[Exception] = None) -> str:
incident_key = create_incident_key()
payload = {
"event": event, "incident_key": incident_key, "message": message,
"app_version": VERSION, "pid": os.getpid(), "python": platform.python_version(),
"platform": platform.platform(),
"method": request.method if request else "",
"path": request.url.path if request else "",
"client_ip": get_client_ip(request) if request else "",
}
if exc:
log.exception("INCIDENT " + json.dumps(payload, separators=(",", ":")))
else:
log.error("INCIDENT " + json.dumps(payload, separators=(",", ":")))
return incident_key
def get_client_ip(request: Request) -> str:
forwarded = request.headers.get("x-forwarded-for", "").strip()
if TRUST_X_FORWARDED_FOR and forwarded:
return forwarded.split(",")[0].strip()
if request.client and request.client.host:
return request.client.host
return "unknown"
def is_ip_allowed(ip: str) -> bool:
normalized = ip.strip().lower()
if normalized in {"localhost", "testclient"}:
normalized = "127.0.0.1"
try:
ip_obj = ipaddress.ip_address(normalized)
except ValueError:
return False
for network in ALLOWED_NETWORKS:
if ip_obj in network:
return True
return False
def request_body_limit(path: str) -> int:
if path in {"/api/chat", "/api/search"}:
return BODY_LIMIT_CHAT_BYTES
if path == "/api/profile":
return BODY_LIMIT_PROFILE_BYTES
if path == "/api/upload":
return BODY_LIMIT_UPLOAD_BYTES
return BODY_LIMIT_DEFAULT_BYTES
def rate_policy(path: str, method: str, ip: str, sid: str) -> tuple:
identity = sid or ip
if path == "/api/auth/login":
return f"login:{ip}", RL_LOGIN_PER_WINDOW
if path == "/api/chat":
return f"chat:{identity}", RL_CHAT_PER_WINDOW
if path == "/api/search":
return f"search:{identity}", RL_SEARCH_PER_WINDOW
if path == "/api/stats":
return f"stats:{ip}", RL_STATS_PER_WINDOW
if method in {"POST", "PUT", "DELETE", "PATCH"}:
return f"write:{identity}", RL_WRITE_PER_WINDOW
return f"api:{identity}", RL_DEFAULT_PER_WINDOW
def check_rate_limit(key: str, limit: int, window_seconds: int) -> tuple:
now_ts = time.time()
with RATE_LOCK:
bucket = RATE_EVENTS[key]
while bucket and bucket[0] <= (now_ts - window_seconds):
bucket.popleft()
if len(bucket) >= limit:
retry_after = max(1, int(math.ceil(window_seconds - (now_ts - bucket[0]))))
return False, retry_after
bucket.append(now_ts)
return True, 0
def origin_allowed(request: Request) -> bool:
host = request.headers.get("host", "").strip()
expected_origin = f"{request.url.scheme}://{host}".rstrip("/") if host else ""
origin = request.headers.get("origin", "").strip().rstrip("/")
referer = request.headers.get("referer", "").strip()
if origin:
return origin == expected_origin or origin in TRUSTED_ORIGINS
if referer:
parsed = urlparse(referer)
ref_origin = f"{parsed.scheme}://{parsed.netloc}".rstrip("/")
return ref_origin == expected_origin or ref_origin in TRUSTED_ORIGINS
return False
def is_state_changing(method: str) -> bool:
return method in {"POST", "PUT", "DELETE", "PATCH"}
async def read_json_body(request: Request, max_bytes: int) -> dict:
raw = await request.body()
if len(raw) > max_bytes:
raise HTTPException(status_code=413, detail="Request payload too large")
if not raw:
return {}
try:
return json.loads(raw.decode("utf-8"))
except Exception:
raise HTTPException(status_code=400, detail="Invalid JSON payload")
static/caic-bg.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 2.9 MiB

static/jcscreenie.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 270 KiB

static/readme-banner.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 2.9 MiB

templates/index.html
+250 -720
View File
File diff suppressed because it is too large Load Diff
tests/conftest.py
View File
tests/test_amqp.py
-90
View File
@@ -1,90 +0,0 @@
import asyncio
import logging
import amqp
from config import AMQP_EXCHANGE_ADMIN
def _reset():
amqp._connection = None
amqp._channel = None
amqp._lock = asyncio.Lock()
class FakeExchange:
def __init__(self):
self.messages = []
async def publish(self, msg, routing_key):
self.messages.append((msg, routing_key))
class FakeChannel:
def __init__(self):
self.is_closed = False
self.exchanges = {}
async def close(self):
self.is_closed = True
async def declare_exchange(self, name, typ, durable=True):
self.exchanges[name] = typ
async def get_exchange(self, name):
return self.exchanges.setdefault(name, FakeExchange())
class FakeConnection:
def __init__(self):
self.is_closed = False
self._channel = FakeChannel()
async def channel(self):
return self._channel
async def close(self):
self.is_closed = True
async def fake_connect_robust(url, **_):
return FakeConnection()
# ---------- tests ----------
def test_publish_success(monkeypatch):
_reset()
monkeypatch.setattr(amqp, "HAS_AIO_PIKA", True)
monkeypatch.setattr("aio_pika.connect_robust", fake_connect_robust)
asyncio.run(amqp.connect())
ch = asyncio.run(amqp.get_channel())
ex = FakeExchange()
ch.exchanges[AMQP_EXCHANGE_ADMIN] = ex
asyncio.run(amqp.publish(AMQP_EXCHANGE_ADMIN, "test.key", {"foo": "bar"}))
assert len(ex.messages) == 1
msg, rk = ex.messages[0]
assert rk == "test.key"
assert msg.body == b'{"foo": "bar"}'
def test_publish_disconnected_no_raise(caplog):
_reset()
caplog.set_level(logging.ERROR)
asyncio.run(amqp.publish(AMQP_EXCHANGE_ADMIN, "test.key", {"x": 1}))
assert len(caplog.records) > 0
assert any("cannot publish" in r.message for r in caplog.records)
def test_get_channel_reconnects_when_none(monkeypatch):
_reset()
monkeypatch.setattr(amqp, "HAS_AIO_PIKA", True)
monkeypatch.setattr("aio_pika.connect_robust", fake_connect_robust)
ch = asyncio.run(amqp.get_channel())
assert ch is not None
assert not ch.is_closed
tests/test_auth_capabilities.py
+9 -11
View File
@@ -3,18 +3,16 @@ from pathlib import Path
from fastapi.testclient import TestClient from fastapi.testclient import TestClient
import app import app as app_module
import db
from security import SESSIONS, PIN_ATTEMPTS
def make_client(tmp_path: Path) -> TestClient: def make_client(tmp_path: Path) -> TestClient:
os.environ["CAIC_ADMIN_PIN"] = "1234" os.environ["JARVISCHAT_ADMIN_PIN"] = "1234"
db.DB_PATH = tmp_path / "caic-test.db" app_module.DB_PATH = tmp_path / "jarvischat-test.db"
SESSIONS.clear() app_module.SESSIONS.clear()
PIN_ATTEMPTS.clear() app_module.PIN_ATTEMPTS.clear()
db.init_db() app_module.init_db()
return TestClient(app.app) return TestClient(app_module.app)
def test_guest_read_only_admin_write_blocked(tmp_path: Path): def test_guest_read_only_admin_write_blocked(tmp_path: Path):
@@ -22,7 +20,7 @@ def test_guest_read_only_admin_write_blocked(tmp_path: Path):
guest = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}) guest = client.post("/api/auth/guest", headers={"Origin": "http://testserver"})
assert guest.status_code == 200 assert guest.status_code == 200
sid = guest.json()["session_id"] sid = guest.json()["session_id"]
headers = {"X-Session-ID": sid, "Origin": "http://testserver"} headers = {"X-Session-ID": sid}
read_resp = client.get("/api/memories", headers=headers) read_resp = client.get("/api/memories", headers=headers)
assert read_resp.status_code == 200 assert read_resp.status_code == 200
@@ -76,5 +74,5 @@ def test_logout_revokes_session(tmp_path: Path):
logout = client.post("/api/auth/logout", headers=headers) logout = client.post("/api/auth/logout", headers=headers)
assert logout.status_code == 200 assert logout.status_code == 200
after = client.get("/api/memories", headers={"X-Session-ID": sid, "Origin": "http://testserver"}) after = client.get("/api/memories", headers={"X-Session-ID": sid})
assert after.status_code == 401 assert after.status_code == 401
tests/test_chat_streaming_and_memory_paths.py
+19 -97
View File
@@ -2,29 +2,19 @@ import json
import os import os
from pathlib import Path from pathlib import Path
import httpx
from fastapi.testclient import TestClient from fastapi.testclient import TestClient
import app import app as app_module
import config
import db
import routers.chat
import triage
from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS
def _mock_triage_url(monkeypatch, url: str = config.LLAMA_SERVER_BASE):
monkeypatch.setattr(config, "LLAMA_SERVER_BASE", url)
def make_client(tmp_path: Path) -> TestClient: def make_client(tmp_path: Path) -> TestClient:
os.environ["CAIC_ADMIN_PIN"] = "1234" os.environ["JARVISCHAT_ADMIN_PIN"] = "1234"
db.DB_PATH = tmp_path / "caic-streaming.db" app_module.DB_PATH = tmp_path / "jarvischat-streaming.db"
SESSIONS.clear() app_module.SESSIONS.clear()
PIN_ATTEMPTS.clear() app_module.PIN_ATTEMPTS.clear()
RATE_EVENTS.clear() app_module.RATE_EVENTS.clear()
db.init_db() app_module.init_db()
return TestClient(app.app, raise_server_exceptions=False) return TestClient(app_module.app, raise_server_exceptions=False)
def parse_sse_payloads(body: str) -> list[dict]: def parse_sse_payloads(body: str) -> list[dict]:
@@ -58,7 +48,6 @@ def _stream_json_lines(events: list[dict]) -> list[str]:
def test_chat_stream_emits_tokens_and_done(tmp_path: Path, monkeypatch): def test_chat_stream_emits_tokens_and_done(tmp_path: Path, monkeypatch):
_mock_triage_url(monkeypatch)
with make_client(tmp_path) as client: with make_client(tmp_path) as client:
sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()[ sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()[
"session_id" "session_id"
@@ -76,11 +65,11 @@ def test_chat_stream_emits_tokens_and_done(tmp_path: Path, monkeypatch):
def stream_stub(self, method, url, json=None, timeout=None): def stream_stub(self, method, url, json=None, timeout=None):
return _MockStreamResponse(events) return _MockStreamResponse(events)
monkeypatch.setattr(httpx.AsyncClient, "stream", stream_stub) monkeypatch.setattr(app_module.httpx.AsyncClient, "stream", stream_stub)
resp = client.post( resp = client.post(
"/api/chat", "/api/chat",
json={"message": "hello", "model": config.DEFAULT_MODEL}, json={"message": "hello", "model": app_module.DEFAULT_MODEL},
headers=headers, headers=headers,
) )
assert resp.status_code == 200 assert resp.status_code == 200
@@ -94,7 +83,6 @@ def test_chat_stream_emits_tokens_and_done(tmp_path: Path, monkeypatch):
def test_chat_auto_search_trigger_emits_search_events(tmp_path: Path, monkeypatch): def test_chat_auto_search_trigger_emits_search_events(tmp_path: Path, monkeypatch):
_mock_triage_url(monkeypatch)
with make_client(tmp_path) as client: with make_client(tmp_path) as client:
sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()[ sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()[
"session_id" "session_id"
@@ -104,7 +92,7 @@ def test_chat_auto_search_trigger_emits_search_events(tmp_path: Path, monkeypatc
first_stream = _stream_json_lines( first_stream = _stream_json_lines(
[ [
{ {
"message": {"content": "I don't have current data on that question."}, "message": {"content": "I am uncertain."},
"logprobs": [{"logprob": -5.0}], "logprobs": [{"logprob": -5.0}],
}, },
{"done": True, "eval_count": 2, "eval_duration": 1000000000}, {"done": True, "eval_count": 2, "eval_duration": 1000000000},
@@ -130,12 +118,12 @@ def test_chat_auto_search_trigger_emits_search_events(tmp_path: Path, monkeypatc
} }
] ]
monkeypatch.setattr(httpx.AsyncClient, "stream", stream_stub) monkeypatch.setattr(app_module.httpx.AsyncClient, "stream", stream_stub)
monkeypatch.setattr(routers.chat, "query_searxng", search_stub) monkeypatch.setattr(app_module, "query_searxng", search_stub)
resp = client.post( resp = client.post(
"/api/chat", "/api/chat",
json={"message": "what is the latest value", "model": config.DEFAULT_MODEL}, json={"message": "what is the latest value", "model": app_module.DEFAULT_MODEL},
headers=headers, headers=headers,
) )
assert resp.status_code == 200 assert resp.status_code == 200
@@ -148,73 +136,7 @@ def test_chat_auto_search_trigger_emits_search_events(tmp_path: Path, monkeypatc
assert done_events and done_events[-1].get("searched") is True assert done_events and done_events[-1].get("searched") is True
def test_chat_with_upload_context_id_injects_document(tmp_path: Path, monkeypatch):
_mock_triage_url(monkeypatch)
captured_payload = {}
def stream_stub(self, method, url, json=None, timeout=None):
nonlocal captured_payload
captured_payload = json
events = [{"message": {"content": "ok"}, "logprobs": [{"logprob": -0.01}]}, {"done": True, "eval_count": 1, "eval_duration": 1000000000}]
return _MockStreamResponse([__import__('json').dumps(e) for e in events])
monkeypatch.setattr(httpx.AsyncClient, "stream", stream_stub)
with make_client(tmp_path) as client:
sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()["session_id"]
headers = {"X-Session-ID": sid, "Origin": "http://testserver"}
db_local = db.get_db()
expires = "2099-12-31T23:59:59+00:00"
cid = db.insert_upload_context(db_local, "conv-up", "report.txt", "Confidential document content here", expires, "text/plain")
db_local.commit()
db_local.close()
resp = client.post(
"/api/chat",
json={"message": "summarize this", "upload_context_id": cid, "model": config.DEFAULT_MODEL},
headers=headers,
)
assert resp.status_code == 200
system_content = next((m["content"] for m in captured_payload.get("messages", []) if m["role"] == "system"), "")
assert "Confidential document content here" in system_content
def test_chat_with_expired_upload_context_id_silent(tmp_path: Path, monkeypatch):
_mock_triage_url(monkeypatch)
captured_payload = {}
def stream_stub(self, method, url, json=None, timeout=None):
nonlocal captured_payload
captured_payload = json
events = [{"message": {"content": "ok"}, "logprobs": [{"logprob": -0.01}]}, {"done": True, "eval_count": 1, "eval_duration": 1000000000}]
return _MockStreamResponse([__import__('json').dumps(e) for e in events])
monkeypatch.setattr(httpx.AsyncClient, "stream", stream_stub)
with make_client(tmp_path) as client:
sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()["session_id"]
headers = {"X-Session-ID": sid, "Origin": "http://testserver"}
import datetime
db_local = db.get_db()
expires = (datetime.datetime.now(datetime.timezone.utc) - datetime.timedelta(hours=2)).isoformat()
cid = db.insert_upload_context(db_local, "conv-exp", "old.txt", "Stale data", expires, "text/plain")
db_local.commit()
db_local.close()
resp = client.post(
"/api/chat",
json={"message": "hi", "upload_context_id": cid, "model": config.DEFAULT_MODEL},
headers=headers,
)
assert resp.status_code == 200
system_content = next((m["content"] for m in captured_payload.get("messages", []) if m["role"] == "system"), "")
assert "Stale data" not in system_content
def test_memory_command_paths_remember_and_forget(tmp_path: Path, monkeypatch): def test_memory_command_paths_remember_and_forget(tmp_path: Path, monkeypatch):
_mock_triage_url(monkeypatch)
with make_client(tmp_path) as client: with make_client(tmp_path) as client:
sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()[ sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()[
"session_id" "session_id"
@@ -231,13 +153,13 @@ def test_memory_command_paths_remember_and_forget(tmp_path: Path, monkeypatch):
def stream_stub(self, method, url, json=None, timeout=None): def stream_stub(self, method, url, json=None, timeout=None):
return _MockStreamResponse(base_stream) return _MockStreamResponse(base_stream)
monkeypatch.setattr(httpx.AsyncClient, "stream", stream_stub) monkeypatch.setattr(app_module.httpx.AsyncClient, "stream", stream_stub)
remember_resp = client.post( remember_resp = client.post(
"/api/chat", "/api/chat",
json={ json={
"message": "remember that my favorite language is rust", "message": "remember that my favorite language is rust",
"model": config.DEFAULT_MODEL, "model": app_module.DEFAULT_MODEL,
}, },
headers=headers, headers=headers,
) )
@@ -245,7 +167,7 @@ def test_memory_command_paths_remember_and_forget(tmp_path: Path, monkeypatch):
remember_events = parse_sse_payloads(remember_resp.text) remember_events = parse_sse_payloads(remember_resp.text)
assert any("Remembered" in p.get("token", "") for p in remember_events) assert any("Remembered" in p.get("token", "") for p in remember_events)
memories_after_add = client.get("/api/memories", headers={"X-Session-ID": sid, "Origin": "http://testserver"}) memories_after_add = client.get("/api/memories", headers={"X-Session-ID": sid})
assert memories_after_add.status_code == 200 assert memories_after_add.status_code == 200
assert memories_after_add.json().get("count", 0) >= 1 assert memories_after_add.json().get("count", 0) >= 1
@@ -253,7 +175,7 @@ def test_memory_command_paths_remember_and_forget(tmp_path: Path, monkeypatch):
"/api/chat", "/api/chat",
json={ json={
"message": "forget about my favorite language", "message": "forget about my favorite language",
"model": config.DEFAULT_MODEL, "model": app_module.DEFAULT_MODEL,
}, },
headers=headers, headers=headers,
) )
@@ -261,6 +183,6 @@ def test_memory_command_paths_remember_and_forget(tmp_path: Path, monkeypatch):
forget_events = parse_sse_payloads(forget_resp.text) forget_events = parse_sse_payloads(forget_resp.text)
assert any("Forgot" in p.get("token", "") for p in forget_events) assert any("Forgot" in p.get("token", "") for p in forget_events)
memories_after_forget = client.get("/api/memories", headers={"X-Session-ID": sid, "Origin": "http://testserver"}) memories_after_forget = client.get("/api/memories", headers={"X-Session-ID": sid})
assert memories_after_forget.status_code == 200 assert memories_after_forget.status_code == 200
assert memories_after_forget.json().get("count", 0) == 0 assert memories_after_forget.json().get("count", 0) == 0
tests/test_cluster.py
-313
View File
@@ -1,313 +0,0 @@
"""Tests for cluster.py — no live AMQP, all handlers called directly."""
import asyncio
from collections import deque
import cluster
from config import AMQP_EXCHANGE_ADMIN, AMQP_EXCHANGE_SYSTEM
def _reset():
cluster.CLUSTER_NODES.clear()
cluster.CLUSTER_EVENTS.clear()
cluster.CLUSTER_COORDINATOR = None
cluster._pending_pings.clear()
_published = []
async def _fake_publish(exchange, routing_key, payload):
_published.append((exchange, routing_key, payload))
# ---------- 1. Valid worker registration ----------
def test_valid_worker_registration(monkeypatch):
_reset()
_published.clear()
monkeypatch.setattr(cluster, "publish", _fake_publish)
asyncio.run(cluster.handle_registration(
AMQP_EXCHANGE_ADMIN, "node.jarvis.register",
{"node_name": "jarvis", "node_type": "worker", "capabilities": ["llm"]},
))
assert "jarvis" in cluster.CLUSTER_NODES
assert cluster.CLUSTER_NODES["jarvis"]["type"] == "worker"
assert cluster.CLUSTER_NODES["jarvis"]["status"] == "active"
assert len(cluster.CLUSTER_EVENTS) == 1
assert cluster.CLUSTER_EVENTS[0]["category"] == "cluster"
assert cluster.CLUSTER_EVENTS[0]["message"] == "Node registered (type=worker)"
assert len(_published) == 1
exchange, rk, payload = _published[0]
assert exchange == AMQP_EXCHANGE_ADMIN
assert rk == "node.jarvis.admitted"
assert payload["type"] == "admitted"
assert payload["node_name"] == "jarvis"
# ---------- 2. First coordinator auto-promotion ----------
def test_first_coordinator_auto_promotion(monkeypatch):
_reset()
_published.clear()
monkeypatch.setattr(cluster, "publish", _fake_publish)
asyncio.run(cluster.handle_registration(
AMQP_EXCHANGE_ADMIN, "node.ultron.register",
{"node_name": "ultron", "node_type": "coordinator", "capabilities": ["llm", "rag"]},
))
assert cluster.CLUSTER_COORDINATOR == "ultron"
assert len(cluster.CLUSTER_EVENTS) == 2
assert cluster.CLUSTER_EVENTS[1]["message"] == "Elected as coordinator"
assert len(_published) == 2
# First publish: admitted
assert _published[0][1] == "node.ultron.admitted"
# Second publish: coord_response
exchange, rk, payload = _published[1]
assert exchange == AMQP_EXCHANGE_SYSTEM
assert rk == "cluster.coordinator.response"
assert payload["type"] == "coord_response"
assert payload["coordinator"] == "ultron"
# ---------- 3. Duplicate node name rejected ----------
def test_duplicate_node_rejected(monkeypatch):
_reset()
_published.clear()
monkeypatch.setattr(cluster, "publish", _fake_publish)
asyncio.run(cluster.handle_registration(
AMQP_EXCHANGE_ADMIN, "node.jarvis.register",
{"node_name": "jarvis", "node_type": "worker", "capabilities": ["llm"]},
))
_published.clear()
asyncio.run(cluster.handle_registration(
AMQP_EXCHANGE_ADMIN, "node.jarvis.register",
{"node_name": "jarvis", "node_type": "worker", "capabilities": ["llm"]},
))
assert len(cluster.CLUSTER_NODES) == 1
assert len(_published) == 1
exchange, rk, payload = _published[0]
assert exchange == AMQP_EXCHANGE_ADMIN
assert rk == "node.jarvis.rejected"
assert payload["reason"] == "duplicate_node_name"
# ---------- 4. Malformed payload rejected ----------
def test_malformed_payload_rejected(monkeypatch):
_reset()
_published.clear()
monkeypatch.setattr(cluster, "publish", _fake_publish)
asyncio.run(cluster.handle_registration(
AMQP_EXCHANGE_ADMIN, "node.jarvis.register",
{"node_name": "jarvis"}, # missing node_type and capabilities
))
assert len(cluster.CLUSTER_NODES) == 0
assert len(_published) == 1
exchange, rk, payload = _published[0]
assert rk == "node.jarvis.rejected"
assert payload["reason"] == "malformed_payload"
# ---------- 5. Graceful deregistration ----------
def test_graceful_deregistration(monkeypatch):
_reset()
_published.clear()
monkeypatch.setattr(cluster, "publish", _fake_publish)
asyncio.run(cluster.handle_registration(
AMQP_EXCHANGE_ADMIN, "node.jarvis.register",
{"node_name": "jarvis", "node_type": "worker", "capabilities": ["llm"]},
))
_published.clear()
asyncio.run(cluster.handle_deregistration(
AMQP_EXCHANGE_ADMIN, "node.jarvis.deregister",
{"node_name": "jarvis"},
))
assert "jarvis" not in cluster.CLUSTER_NODES
assert len(cluster.CLUSTER_EVENTS) == 2
assert cluster.CLUSTER_EVENTS[1]["message"] == "Node deregistered"
def test_deregister_coordinator_clears(monkeypatch):
_reset()
_published.clear()
monkeypatch.setattr(cluster, "publish", _fake_publish)
asyncio.run(cluster.handle_registration(
AMQP_EXCHANGE_ADMIN, "node.ultron.register",
{"node_name": "ultron", "node_type": "coordinator", "capabilities": ["llm"]},
))
assert cluster.CLUSTER_COORDINATOR == "ultron"
_published.clear()
asyncio.run(cluster.handle_deregistration(
AMQP_EXCHANGE_ADMIN, "node.ultron.deregister",
{"node_name": "ultron"},
))
assert "ultron" not in cluster.CLUSTER_NODES
assert cluster.CLUSTER_COORDINATOR is None
# ---------- 6. Pong from known node ----------
def test_pong_updates_known_node(monkeypatch):
_reset()
monkeypatch.setattr(cluster, "publish", _fake_publish)
asyncio.run(cluster.handle_registration(
AMQP_EXCHANGE_ADMIN, "node.jarvis.register",
{"node_name": "jarvis", "node_type": "worker", "capabilities": ["llm"]},
))
original_seen = cluster.CLUSTER_NODES["jarvis"]["last_seen"]
asyncio.run(cluster.handle_pong(
AMQP_EXCHANGE_ADMIN, "node.jarvis.pong",
{"node_name": "jarvis", "status": "busy", "load": {"cpu_pct": 80}},
))
assert cluster.CLUSTER_NODES["jarvis"]["status"] == "busy"
assert cluster.CLUSTER_NODES["jarvis"]["load"] == {"cpu_pct": 80}
assert cluster.CLUSTER_NODES["jarvis"]["last_seen"] != original_seen
# ---------- 7. Pong from unknown node ----------
def test_pong_from_unknown_node(caplog, monkeypatch):
_reset()
caplog.set_level("WARNING")
monkeypatch.setattr(cluster, "publish", _fake_publish)
asyncio.run(cluster.handle_pong(
AMQP_EXCHANGE_ADMIN, "node.stranger.pong",
{"node_name": "stranger", "status": "active"},
))
assert "stranger" not in cluster.CLUSTER_NODES
assert any("pong from unknown node" in rec.message for rec in caplog.records)
# ---------- 8. Event stored in log ----------
def test_event_appended_to_log(monkeypatch):
_reset()
monkeypatch.setattr(cluster, "publish", _fake_publish)
asyncio.run(cluster.handle_event(
AMQP_EXCHANGE_SYSTEM, "node.jarvis.event",
{"node_name": "jarvis", "severity": "error", "message": "OOM on GPU"},
))
assert len(cluster.CLUSTER_EVENTS) == 1
ev = cluster.CLUSTER_EVENTS[0]
assert ev["category"] == "application"
assert ev["severity"] == "error"
assert ev["node"] == "jarvis"
def test_event_log_bounded(monkeypatch):
_reset()
monkeypatch.setattr(cluster, "publish", _fake_publish)
for i in range(1001):
cluster._push_event("application", "info", "jarvis", f"event {i}")
assert len(cluster.CLUSTER_EVENTS) == 1000
assert cluster.CLUSTER_EVENTS[0]["message"] == "event 1"
assert cluster.CLUSTER_EVENTS[-1]["message"] == "event 1000"
# ---------- 9. Coordinator query produces response ----------
def test_coordinator_query_response(monkeypatch):
_reset()
_published.clear()
monkeypatch.setattr(cluster, "publish", _fake_publish)
asyncio.run(cluster.handle_registration(
AMQP_EXCHANGE_ADMIN, "node.ultron.register",
{"node_name": "ultron", "node_type": "coordinator", "capabilities": ["llm"]},
))
_published.clear()
asyncio.run(cluster.handle_coordinator_query(
AMQP_EXCHANGE_SYSTEM, "cluster.coordinator.query",
{"from": "jarvis", "type": "coord_query"},
))
assert len(_published) == 1
exchange, rk, payload = _published[0]
assert exchange == AMQP_EXCHANGE_SYSTEM
assert rk == "cluster.coordinator.response"
assert payload["coordinator"] == "ultron"
assert "nodes" in payload
def test_coordinator_query_no_coordinator(monkeypatch):
_reset()
_published.clear()
monkeypatch.setattr(cluster, "publish", _fake_publish)
asyncio.run(cluster.handle_coordinator_query(
AMQP_EXCHANGE_SYSTEM, "cluster.coordinator.query",
{"from": "jarvis", "type": "coord_query"},
))
assert len(_published) == 0
# ---------- 10. GET /api/cluster shape ----------
def test_cluster_api_shape():
_reset()
cluster.CLUSTER_NODES["jarvis"] = {
"name": "jarvis", "type": "worker", "status": "active",
"capabilities": ["llm"], "active_model": None, "load": None,
"registered_at": "2026-01-01T00:00:00Z", "last_seen": "2026-01-01T00:00:00Z",
}
cluster.CLUSTER_COORDINATOR = "ultron"
cluster._push_event("cluster", "info", "ultron", "Elected")
from routers.cluster import cluster_status
resp = asyncio.run(cluster_status())
assert "nodes" in resp
assert "node_count" in resp
assert "coordinator" in resp
assert "events" in resp
assert resp["node_count"] == 1
assert resp["coordinator"] == "ultron"
assert "jarvis" in resp["nodes"]
assert len(resp["events"]) == 1
# No internal keys leaked
for node in resp["nodes"].values():
for key in node:
assert key in {
"name", "type", "status", "capabilities", "active_model",
"load", "registered_at", "last_seen",
}
tests/test_cluster_heartbeat.py
-49
View File
@@ -1,49 +0,0 @@
"""Tests for cluster.py heartbeat handler."""
import asyncio
import cluster
from config import AMQP_EXCHANGE_SYSTEM
def _reset():
cluster.CLUSTER_NODES.clear()
cluster.CLUSTER_EVENTS.clear()
cluster.CLUSTER_COORDINATOR = None
cluster._pending_pings.clear()
# ---------- 1. handle_heartbeat() for known node updates last_seen ----------
def test_heartbeat_updates_last_seen(monkeypatch):
_reset()
cluster.CLUSTER_NODES["jarvis"] = {
"name": "jarvis", "type": "worker", "status": "active",
"last_seen": "2020-01-01T00:00:00Z",
}
original = cluster.CLUSTER_NODES["jarvis"]["last_seen"]
asyncio.run(cluster.handle_heartbeat(
AMQP_EXCHANGE_SYSTEM, "node.jarvis.heartbeat",
{"node_name": "jarvis"},
))
assert cluster.CLUSTER_NODES["jarvis"]["last_seen"] != original
assert cluster.CLUSTER_NODES["jarvis"]["status"] == "active" # unchanged
# ---------- 2. handle_heartbeat() for unknown node logs warning, no add ----------
def test_heartbeat_unknown_node_logs_warning(caplog, monkeypatch):
_reset()
caplog.set_level("WARNING")
asyncio.run(cluster.handle_heartbeat(
AMQP_EXCHANGE_SYSTEM, "node.stranger.heartbeat",
{"node_name": "stranger"},
))
assert "stranger" not in cluster.CLUSTER_NODES
assert any("unknown node" in rec.message and "stranger" in rec.message for rec in caplog.records)
tests/test_completions.py
-222
View File
@@ -1,222 +0,0 @@
import json
import os
from pathlib import Path
import httpx
from fastapi.testclient import TestClient
import app
import config
import db
import routers.completions
from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS
def make_client(tmp_path: Path) -> TestClient:
os.environ["CAIC_ADMIN_PIN"] = "1234"
db.DB_PATH = tmp_path / "caic-completions.db"
SESSIONS.clear()
PIN_ATTEMPTS.clear()
RATE_EVENTS.clear()
db.init_db()
return TestClient(app.app, raise_server_exceptions=False)
TEST_API_KEY = "test-sk-caic-completions"
def _auth_headers(extra: dict = None) -> dict:
h = {"Authorization": f"Bearer {TEST_API_KEY}", "Content-Type": "application/json", "Origin": "http://testserver"}
if extra:
h.update(extra)
return h
class _MockStreamResponse:
def __init__(self, lines: list[str]):
self._lines = lines
async def __aenter__(self):
return self
async def __aexit__(self, exc_type, exc, tb):
return False
async def aiter_lines(self):
for line in self._lines:
yield line
class _MockAsyncPostResponse:
def __init__(self, status_code=200, json_data=None):
self.status_code = status_code
self._json_data = json_data or {}
def json(self):
return self._json_data
def _stream_json_lines(events: list[dict]) -> list[str]:
return [json.dumps(event) for event in events]
def test_completions_missing_api_key(tmp_path: Path):
with make_client(tmp_path) as client:
resp = client.post(
"/v1/chat/completions",
json={"messages": [{"role": "user", "content": "hi"}]},
)
assert resp.status_code == 401
def test_completions_invalid_api_key(tmp_path: Path):
with make_client(tmp_path) as client:
resp = client.post(
"/v1/chat/completions",
json={"messages": [{"role": "user", "content": "hi"}]},
headers={"Authorization": "Bearer wrong-key", "Origin": "http://testserver"},
)
assert resp.status_code == 401
def test_completions_no_messages(tmp_path: Path, monkeypatch):
monkeypatch.setattr(routers.completions, "COMPLETIONS_API_KEY", TEST_API_KEY)
with make_client(tmp_path) as client:
resp = client.post("/v1/chat/completions", json={}, headers=_auth_headers())
assert resp.status_code == 400
def test_completions_empty_messages(tmp_path: Path, monkeypatch):
monkeypatch.setattr(routers.completions, "COMPLETIONS_API_KEY", TEST_API_KEY)
with make_client(tmp_path) as client:
resp = client.post("/v1/chat/completions", json={"messages": []}, headers=_auth_headers())
assert resp.status_code == 400
def test_completions_no_user_message(tmp_path: Path, monkeypatch):
monkeypatch.setattr(routers.completions, "COMPLETIONS_API_KEY", TEST_API_KEY)
with make_client(tmp_path) as client:
resp = client.post(
"/v1/chat/completions",
json={"messages": [{"role": "assistant", "content": "hello"}]},
headers=_auth_headers(),
)
assert resp.status_code == 400
def test_completions_streaming(tmp_path: Path, monkeypatch):
monkeypatch.setattr(routers.completions, "COMPLETIONS_API_KEY", TEST_API_KEY)
events = _stream_json_lines([
{"choices": [{"delta": {"content": "Hello"}, "logprobs": None}]},
{"choices": [{"delta": {"content": " world"}, "logprobs": None}]},
{"choices": [{"delta": {}, "finish_reason": "stop"}], "usage": {"tokens_per_second": 15.0}},
])
call_count = 0
def stream_stub(self, method, url, json=None, timeout=None):
nonlocal call_count
call_count += 1
return _MockStreamResponse(events)
monkeypatch.setattr(httpx.AsyncClient, "stream", stream_stub)
with make_client(tmp_path) as client:
resp = client.post(
"/v1/chat/completions",
json={"messages": [{"role": "user", "content": "hi"}], "stream": True},
headers=_auth_headers(),
)
assert resp.status_code == 200
body = resp.text
assert "data: [DONE]" in body
assert "Hello" in body or "world" in body
assert "chatcmpl-" in body
def test_completions_blocking(tmp_path: Path, monkeypatch):
monkeypatch.setattr(routers.completions, "COMPLETIONS_API_KEY", TEST_API_KEY)
events = _stream_json_lines([
{"choices": [{"delta": {"content": "Hello world"}, "logprobs": None}]},
{"choices": [{"delta": {}, "finish_reason": "stop"}], "usage": {}},
])
def stream_stub(self, method, url, json=None, timeout=None):
return _MockStreamResponse(events)
monkeypatch.setattr(httpx.AsyncClient, "stream", stream_stub)
with make_client(tmp_path) as client:
resp = client.post(
"/v1/chat/completions",
json={"messages": [{"role": "user", "content": "hi"}], "stream": False},
headers=_auth_headers(),
)
assert resp.status_code == 200
data = resp.json()
assert data["object"] == "chat.completion"
assert data["choices"][0]["message"]["content"] == "Hello world"
def test_completions_fim_passthrough(tmp_path: Path, monkeypatch):
monkeypatch.setattr(routers.completions, "COMPLETIONS_API_KEY", TEST_API_KEY)
fim_data = {"prompt": "def foo():\n ", "suffix": "\n return x", "model": "llama3.1:latest"}
async def mock_post(self, url, json=None, timeout=None):
return _MockAsyncPostResponse(json_data={"choices": [{"text": "pass"}], "usage": {}})
monkeypatch.setattr(httpx.AsyncClient, "post", mock_post)
with make_client(tmp_path) as client:
resp = client.post("/v1/chat/completions", json=fim_data, headers=_auth_headers())
assert resp.status_code == 200
assert "choices" in resp.json()
def test_completions_connect_error_stream(tmp_path: Path, monkeypatch):
monkeypatch.setattr(routers.completions, "COMPLETIONS_API_KEY", TEST_API_KEY)
def broken_stream(self, method, url, json=None, timeout=None):
raise httpx.ConnectError("Connection refused")
monkeypatch.setattr(httpx.AsyncClient, "stream", broken_stream)
with make_client(tmp_path) as client:
resp = client.post(
"/v1/chat/completions",
json={"messages": [{"role": "user", "content": "hi"}], "stream": True},
headers=_auth_headers(),
)
assert resp.status_code == 200
assert "connection_error" in resp.text
def test_completions_connect_error_blocking(tmp_path: Path, monkeypatch):
monkeypatch.setattr(routers.completions, "COMPLETIONS_API_KEY", TEST_API_KEY)
def broken_stream(self, method, url, json=None, timeout=None):
raise httpx.ConnectError("Connection refused")
monkeypatch.setattr(httpx.AsyncClient, "stream", broken_stream)
with make_client(tmp_path) as client:
resp = client.post(
"/v1/chat/completions",
json={"messages": [{"role": "user", "content": "hi"}], "stream": False},
headers=_auth_headers(),
)
assert resp.status_code == 503
def test_completions_fim_connect_error(tmp_path: Path, monkeypatch):
monkeypatch.setattr(routers.completions, "COMPLETIONS_API_KEY", TEST_API_KEY)
fim_data = {"prompt": "def foo():", "model": "llama3.1:latest"}
def broken_post(self, url, json=None, timeout=None):
raise httpx.ConnectError("Connection refused")
monkeypatch.setattr(httpx.AsyncClient, "post", broken_post)
with make_client(tmp_path) as client:
resp = client.post("/v1/chat/completions", json=fim_data, headers=_auth_headers())
assert resp.status_code == 503
tests/test_conversations.py
-153
View File
@@ -1,153 +0,0 @@
import os
from pathlib import Path
from fastapi.testclient import TestClient
import app
import db
from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS
def make_client(tmp_path: Path) -> TestClient:
os.environ["CAIC_ADMIN_PIN"] = "1234"
db.DB_PATH = tmp_path / "caic-conversations.db"
SESSIONS.clear()
PIN_ATTEMPTS.clear()
RATE_EVENTS.clear()
db.init_db()
return TestClient(app.app, raise_server_exceptions=False)
def _admin_headers(client: TestClient) -> dict:
login = client.post("/api/auth/login", json={"pin": "1234"}, headers={"Origin": "http://testserver"})
sid = login.json()["session_id"]
return {"X-Session-ID": sid, "Origin": "http://testserver"}
def _guest_headers(client: TestClient) -> dict:
sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()["session_id"]
return {"X-Session-ID": sid, "Origin": "http://testserver"}
def test_list_conversations_empty(tmp_path: Path):
with make_client(tmp_path) as client:
resp = client.get("/api/conversations", headers=_guest_headers(client))
assert resp.status_code == 200
assert resp.json() == []
def test_create_and_list_conversation(tmp_path: Path):
with make_client(tmp_path) as client:
headers = _admin_headers(client)
create = client.post("/api/conversations", json={"title": "Test Chat", "model": "llama3.1:latest"}, headers=headers)
assert create.status_code == 200
data = create.json()
assert data["title"] == "Test Chat"
assert data["model"] == "llama3.1:latest"
list_resp = client.get("/api/conversations", headers=headers)
assert list_resp.status_code == 200
convs = list_resp.json()
assert len(convs) == 1
assert convs[0]["title"] == "Test Chat"
def test_get_conversation_returns_messages(tmp_path: Path):
with make_client(tmp_path) as client:
headers = _admin_headers(client)
create = client.post("/api/conversations", json={"title": "My Chat"}, headers=headers)
conv_id = create.json()["id"]
resp = client.get(f"/api/conversations/{conv_id}", headers=headers)
assert resp.status_code == 200
data = resp.json()
assert data["conversation"]["id"] == conv_id
assert data["messages"] == []
def test_get_conversation_not_found(tmp_path: Path):
with make_client(tmp_path) as client:
resp = client.get("/api/conversations/nope", headers=_guest_headers(client))
assert resp.status_code == 404
def test_update_conversation_title(tmp_path: Path):
with make_client(tmp_path) as client:
headers = _admin_headers(client)
create = client.post("/api/conversations", json={"title": "Old"}, headers=headers)
conv_id = create.json()["id"]
update = client.put(f"/api/conversations/{conv_id}", json={"title": "New Title"}, headers=headers)
assert update.status_code == 200
get = client.get(f"/api/conversations/{conv_id}", headers=headers)
assert get.json()["conversation"]["title"] == "New Title"
def test_update_conversation_model(tmp_path: Path):
with make_client(tmp_path) as client:
headers = _admin_headers(client)
create = client.post("/api/conversations", json={"title": "Test"}, headers=headers)
conv_id = create.json()["id"]
update = client.put(f"/api/conversations/{conv_id}", json={"model": "qwen2:latest"}, headers=headers)
assert update.status_code == 200
get = client.get(f"/api/conversations/{conv_id}", headers=headers)
assert get.json()["conversation"]["model"] == "qwen2:latest"
def test_delete_conversation(tmp_path: Path):
with make_client(tmp_path) as client:
headers = _admin_headers(client)
create = client.post("/api/conversations", json={"title": "Delete Me"}, headers=headers)
conv_id = create.json()["id"]
delete = client.delete(f"/api/conversations/{conv_id}", headers=headers)
assert delete.status_code == 200
get = client.get(f"/api/conversations/{conv_id}", headers=_guest_headers(client))
assert get.status_code == 404
def test_delete_all_conversations(tmp_path: Path):
with make_client(tmp_path) as client:
headers = _admin_headers(client)
client.post("/api/conversations", json={"title": "One"}, headers=headers)
client.post("/api/conversations", json={"title": "Two"}, headers=headers)
delete_all = client.delete("/api/conversations", headers=headers)
assert delete_all.status_code == 200
list_resp = client.get("/api/conversations", headers=_guest_headers(client))
assert list_resp.json() == []
def test_guest_cannot_create_conversation(tmp_path: Path):
with make_client(tmp_path) as client:
resp = client.post("/api/conversations", json={"title": "test"}, headers=_guest_headers(client))
assert resp.status_code == 403
def test_guest_cannot_update_conversation(tmp_path: Path):
with make_client(tmp_path) as client:
headers = _admin_headers(client)
create = client.post("/api/conversations", json={"title": "Test"}, headers=headers)
conv_id = create.json()["id"]
guest_headers = _guest_headers(client)
resp = client.put(f"/api/conversations/{conv_id}", json={"title": "hack"}, headers=guest_headers)
assert resp.status_code == 403
def test_guest_cannot_delete_conversation(tmp_path: Path):
with make_client(tmp_path) as client:
resp = client.delete("/api/conversations/some-id", headers=_guest_headers(client))
assert resp.status_code == 403
def test_guest_cannot_delete_all(tmp_path: Path):
with make_client(tmp_path) as client:
resp = client.delete("/api/conversations", headers=_guest_headers(client))
assert resp.status_code == 403
tests/test_error_envelopes.py
+12 -17
View File
@@ -1,24 +1,19 @@
import os import os
from pathlib import Path from pathlib import Path
import httpx
from fastapi.testclient import TestClient from fastapi.testclient import TestClient
import app import app as app_module
import config
import db
import routers.memories
from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS
def make_client(tmp_path: Path) -> TestClient: def make_client(tmp_path: Path) -> TestClient:
os.environ["CAIC_ADMIN_PIN"] = "1234" os.environ["JARVISCHAT_ADMIN_PIN"] = "1234"
db.DB_PATH = tmp_path / "caic-errors.db" app_module.DB_PATH = tmp_path / "jarvischat-errors.db"
SESSIONS.clear() app_module.SESSIONS.clear()
PIN_ATTEMPTS.clear() app_module.PIN_ATTEMPTS.clear()
RATE_EVENTS.clear() app_module.RATE_EVENTS.clear()
db.init_db() app_module.init_db()
return TestClient(app.app, raise_server_exceptions=False) return TestClient(app_module.app, raise_server_exceptions=False)
def test_unhandled_api_exception_returns_friendly_error_with_incident_key( def test_unhandled_api_exception_returns_friendly_error_with_incident_key(
@@ -28,12 +23,12 @@ def test_unhandled_api_exception_returns_friendly_error_with_incident_key(
sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()[ sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()[
"session_id" "session_id"
] ]
headers = {"X-Session-ID": sid, "Origin": "http://testserver"} headers = {"X-Session-ID": sid}
def boom(_topic=None): def boom(_topic=None):
raise RuntimeError("super secret db internals") raise RuntimeError("super secret db internals")
monkeypatch.setattr(routers.memories, "get_all_memories", boom) monkeypatch.setattr(app_module, "get_all_memories", boom)
resp = client.get("/api/memories", headers=headers) resp = client.get("/api/memories", headers=headers)
assert resp.status_code == 500 assert resp.status_code == 500
@@ -62,11 +57,11 @@ def test_chat_stream_error_hides_internal_exception_and_emits_incident_key(
def broken_stream(*args, **kwargs): def broken_stream(*args, **kwargs):
return BrokenStreamContext() return BrokenStreamContext()
monkeypatch.setattr(httpx.AsyncClient, "stream", broken_stream) monkeypatch.setattr(app_module.httpx.AsyncClient, "stream", broken_stream)
resp = client.post( resp = client.post(
"/api/chat", "/api/chat",
json={"message": "hello", "model": config.DEFAULT_MODEL}, json={"message": "hello", "model": app_module.DEFAULT_MODEL},
headers=headers, headers=headers,
) )
tests/test_hardware.py
-176
View File
@@ -1,176 +0,0 @@
import asyncio
import json
import os
import subprocess
from pathlib import Path
import httpx
import psutil
from fastapi.testclient import TestClient
import app as app_module
import config
import db
import hardware
from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS
def make_client(tmp_path: Path) -> TestClient:
os.environ["CAIC_ADMIN_PIN"] = "1234"
db.DB_PATH = tmp_path / "caic-hardware.db"
hardware.HARDWARE_STATE_PATH = tmp_path / "hardware_state.json"
SESSIONS.clear()
PIN_ATTEMPTS.clear()
RATE_EVENTS.clear()
db.init_db()
return TestClient(app_module.app, raise_server_exceptions=False)
def _guest_headers(client: TestClient) -> dict:
sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()["session_id"]
return {"X-Session-ID": sid, "Origin": "http://testserver"}
class _MockGet:
def __init__(self, status_code: int = 200, json_data: dict | None = None):
self.status_code = status_code
self._json_data = json_data or {}
def json(self):
return self._json_data
def _mock_subprocess(rocm_stdout: str = "") -> object:
class MockProc:
returncode = 0
stdout = rocm_stdout
class MockSP:
TimeoutExpired = subprocess.TimeoutExpired
run = staticmethod(lambda cmd, **kw: MockProc())
return MockSP()
def _broken_subprocess(exception: Exception) -> object:
class MockSP:
TimeoutExpired = subprocess.TimeoutExpired
run = staticmethod(lambda cmd, **kw: (_ for _ in ()).throw(exception))
return MockSP()
def test_assess_hardware_all_services_reachable(tmp_path: Path, monkeypatch):
hardware.HARDWARE_STATE_PATH = tmp_path / "hardware_state.json"
monkeypatch.setattr(psutil, "virtual_memory", lambda: type("M", (), {"total": 16 * 1024 ** 3, "available": 8 * 1024 ** 3})())
monkeypatch.setattr(psutil, "cpu_count", lambda: 8)
monkeypatch.setattr(hardware, "subprocess", _mock_subprocess(
json.dumps({"card0": {"VRAM Total (MB)": 8192, "VRAM Free (MB)": 4096}})
))
async def mock_get(self, url, *args, **kwargs):
if "v1/models" in url:
return _MockGet(200, {"data": [{"id": "mistral-nemo:latest"}]})
if "6333" in url:
return _MockGet(200, {"result": {"collections": [{"name": "caic"}]}})
if "8888" in url:
return _MockGet(200, {})
return _MockGet(200, {})
monkeypatch.setattr(httpx.AsyncClient, "get", mock_get)
state = asyncio.run(hardware.assess_hardware())
assert state["ram_total_gb"] == 16.0
assert state["ram_available_gb"] == 8.0
assert state["cpu_count"] == 8
assert state["vram_total_mb"] == 8192
assert state["vram_free_mb"] == 4096
assert state["llama_reachable"] is True
assert state["llama_models"] == ["mistral-nemo:latest"]
assert state["qdrant_reachable"] is True
assert state["qdrant_collections"] == ["caic"]
assert state["searxng_reachable"] is True
assert tmp_path.joinpath("hardware_state.json").exists()
def test_assess_hardware_rocm_smi_absent(tmp_path: Path, monkeypatch):
hardware.HARDWARE_STATE_PATH = tmp_path / "hardware_state.json"
monkeypatch.setattr(psutil, "virtual_memory", lambda: type("M", (), {"total": 16 * 1024 ** 3, "available": 8 * 1024 ** 3})())
monkeypatch.setattr(psutil, "cpu_count", lambda: 8)
monkeypatch.setattr(hardware, "subprocess", _broken_subprocess(FileNotFoundError("no rocm-smi")))
async def mock_get(self, url, *args, **kwargs):
if "v1/models" in url:
return _MockGet(200, {"data": []})
if "6333" in url:
return _MockGet(200, {"result": {"collections": []}})
if "8888" in url:
return _MockGet(200, {})
return _MockGet(200, {})
monkeypatch.setattr(httpx.AsyncClient, "get", mock_get)
state = asyncio.run(hardware.assess_hardware())
assert state["vram_total_mb"] == 0
assert state["vram_free_mb"] == 0
assert state["llama_reachable"] is True
assert state["qdrant_reachable"] is True
assert state["searxng_reachable"] is True
def test_assess_hardware_llama_unreachable(tmp_path: Path, monkeypatch):
hardware.HARDWARE_STATE_PATH = tmp_path / "hardware_state.json"
monkeypatch.setattr(psutil, "virtual_memory", lambda: type("M", (), {"total": 16 * 1024 ** 3, "available": 8 * 1024 ** 3})())
monkeypatch.setattr(psutil, "cpu_count", lambda: 8)
monkeypatch.setattr(hardware, "subprocess", _mock_subprocess(
json.dumps({"card0": {"VRAM Total (MB)": 8192, "VRAM Free (MB)": 4096}})
))
async def mock_get(self, url, *args, **kwargs):
if "v1/models" in url:
raise httpx.ConnectError("refused")
if "6333" in url:
return _MockGet(200, {"result": {"collections": [{"name": "caic"}]}})
if "8888" in url:
return _MockGet(200, {})
return _MockGet(200, {})
monkeypatch.setattr(httpx.AsyncClient, "get", mock_get)
state = asyncio.run(hardware.assess_hardware())
assert state["llama_reachable"] is False
assert state["llama_models"] == []
assert state["qdrant_reachable"] is True
assert state["searxng_reachable"] is True
def test_get_hardware_endpoint(tmp_path: Path, monkeypatch):
hardware.HARDWARE_STATE_PATH = tmp_path / "hardware_state.json"
monkeypatch.setattr(psutil, "virtual_memory", lambda: type("M", (), {"total": 16 * 1024 ** 3, "available": 8 * 1024 ** 3})())
monkeypatch.setattr(psutil, "cpu_count", lambda: 8)
monkeypatch.setattr(hardware, "subprocess", _mock_subprocess(
json.dumps({"card0": {"VRAM Total (MB)": 8192, "VRAM Free (MB)": 4096}})
))
async def mock_get(self, url, *args, **kwargs):
if "v1/models" in url:
return _MockGet(200, {"data": [{"id": "mistral-nemo:latest"}]})
if "6333" in url:
return _MockGet(200, {"result": {"collections": [{"name": "caic"}]}})
if "8888" in url:
return _MockGet(200, {})
return _MockGet(200, {})
monkeypatch.setattr(httpx.AsyncClient, "get", mock_get)
with make_client(tmp_path) as client:
resp = client.get("/api/hardware", headers=_guest_headers(client))
assert resp.status_code == 200
data = resp.json()
assert data["ram_total_gb"] == 16.0
assert data["cpu_count"] == 8
assert data["vram_total_mb"] == 8192
assert data["llama_reachable"] is True
assert data["qdrant_reachable"] is True
assert data["searxng_reachable"] is True
assert "llama_models" in data
assert "qdrant_collections" in data
tests/test_ingest.py
-104
View File
@@ -1,104 +0,0 @@
import json
import os
from pathlib import Path
import httpx
from fastapi.testclient import TestClient
import app
import db
import routers.ingest as ingest_route
from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS
def make_client(tmp_path: Path) -> TestClient:
os.environ["CAIC_ADMIN_PIN"] = "1234"
db.DB_PATH = tmp_path / "caic-ingest.db"
SESSIONS.clear()
PIN_ATTEMPTS.clear()
RATE_EVENTS.clear()
db.init_db()
return TestClient(app.app, raise_server_exceptions=False)
TEST_API_KEY = "test-sk-caic-ingest"
def _auth_headers() -> dict:
return {"Authorization": f"Bearer {TEST_API_KEY}", "Content-Type": "application/json", "Origin": "http://testserver"}
def test_ingest_missing_api_key(tmp_path: Path):
with make_client(tmp_path) as client:
resp = client.post("/api/ingest", json={"content": "test"}, headers={"Origin": "http://testserver"})
assert resp.status_code == 401
def test_ingest_wrong_api_key(tmp_path: Path):
with make_client(tmp_path) as client:
resp = client.post("/api/ingest", json={"content": "test"},
headers={"Authorization": "Bearer wrong", "Origin": "http://testserver"})
assert resp.status_code == 401
def test_ingest_empty_content(tmp_path: Path):
monkeypatch = __import__('pytest').MonkeyPatch()
monkeypatch.setattr(ingest_route, "COMPLETIONS_API_KEY", TEST_API_KEY)
with make_client(tmp_path) as client:
resp = client.post("/api/ingest", json={"content": ""}, headers=_auth_headers())
assert resp.status_code == 422
monkeypatch.undo()
def test_ingest_missing_content(tmp_path: Path):
monkeypatch = __import__('pytest').MonkeyPatch()
monkeypatch.setattr(ingest_route, "COMPLETIONS_API_KEY", TEST_API_KEY)
with make_client(tmp_path) as client:
resp = client.post("/api/ingest", json={}, headers=_auth_headers())
assert resp.status_code == 422
monkeypatch.undo()
def test_ingest_success(tmp_path: Path, monkeypatch):
monkeypatch.setattr(ingest_route, "COMPLETIONS_API_KEY", TEST_API_KEY)
embed_count = 0
class FakeAsyncClient:
class FakeResponse:
def __init__(self, status, json_data=None):
self.status_code = status
self._json = json_data or {}
def json(self):
return self._json
def __init__(self, *a, **kw):
pass
async def __aenter__(self):
return self
async def __aexit__(self, *a):
pass
async def post(self, url, **kw):
nonlocal embed_count
if "/api/embeddings" in url:
embed_count += 1
return self.FakeResponse(200, {"embedding": [0.1] * 768})
return self.FakeResponse(200)
async def put(self, url, **kw):
return self.FakeResponse(200)
monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: FakeAsyncClient())
with make_client(tmp_path) as client:
resp = client.post("/api/ingest", json={"content": "test " * 1000, "source": "terminal"}, headers=_auth_headers())
assert resp.status_code == 200
data = resp.json()
assert data["source"] == "terminal"
assert data["chunks_ingested"] > 0
assert embed_count == data["chunks_ingested"]
assert "message" in data
tests/test_ip_allowlist.py
+27 -21
View File
@@ -3,42 +3,48 @@ from pathlib import Path
from fastapi.testclient import TestClient from fastapi.testclient import TestClient
import app import app as app_module
import db
from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS, is_ip_allowed
def make_client(tmp_path: Path) -> TestClient: def make_client(tmp_path: Path) -> TestClient:
os.environ["CAIC_ADMIN_PIN"] = "1234" os.environ["JARVISCHAT_ADMIN_PIN"] = "1234"
db.DB_PATH = tmp_path / "caic-ip.db" app_module.DB_PATH = tmp_path / "jarvischat-ip.db"
SESSIONS.clear() app_module.SESSIONS.clear()
PIN_ATTEMPTS.clear() app_module.PIN_ATTEMPTS.clear()
RATE_EVENTS.clear() app_module.RATE_EVENTS.clear()
db.init_db() app_module.init_db()
return TestClient(app.app) return TestClient(app_module.app)
def test_ip_helper_allows_local_defaults(): def test_ip_helper_allows_local_defaults():
assert is_ip_allowed("127.0.0.1") assert app_module.is_ip_allowed("127.0.0.1")
assert is_ip_allowed("192.168.1.10") assert app_module.is_ip_allowed("192.168.1.10")
assert is_ip_allowed("10.0.0.42") assert app_module.is_ip_allowed("10.0.0.42")
assert is_ip_allowed("172.16.1.2") assert app_module.is_ip_allowed("172.16.1.2")
assert is_ip_allowed("testclient") assert app_module.is_ip_allowed("testclient")
def test_ip_helper_blocks_public_ip(): def test_ip_helper_blocks_public_ip():
assert not is_ip_allowed("8.8.8.8") assert not app_module.is_ip_allowed("8.8.8.8")
def test_middleware_blocks_disallowed_ip(tmp_path: Path, monkeypatch): def test_middleware_blocks_disallowed_ip(tmp_path: Path):
monkeypatch.setattr(app, "get_client_ip", lambda _req: "8.8.8.8")
with make_client(tmp_path) as client: with make_client(tmp_path) as client:
original_get_client_ip = app_module.get_client_ip
try:
app_module.get_client_ip = lambda _req: "8.8.8.8"
resp = client.post("/api/auth/guest") resp = client.post("/api/auth/guest")
assert resp.status_code == 403 assert resp.status_code == 403
finally:
app_module.get_client_ip = original_get_client_ip
def test_middleware_allows_local_ip(tmp_path: Path, monkeypatch): def test_middleware_allows_local_ip(tmp_path: Path):
monkeypatch.setattr(app, "get_client_ip", lambda _req: "192.168.50.109")
with make_client(tmp_path) as client: with make_client(tmp_path) as client:
resp = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}) original_get_client_ip = app_module.get_client_ip
try:
app_module.get_client_ip = lambda _req: "192.168.50.109"
resp = client.post("/api/auth/guest")
assert resp.status_code == 200 assert resp.status_code == 200
finally:
app_module.get_client_ip = original_get_client_ip
tests/test_memories.py
-161
View File
@@ -1,161 +0,0 @@
import os
from pathlib import Path
from fastapi.testclient import TestClient
import app
import config
import db
from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS
def make_client(tmp_path: Path) -> TestClient:
os.environ["CAIC_ADMIN_PIN"] = "1234"
db.DB_PATH = tmp_path / "caic-memories.db"
SESSIONS.clear()
PIN_ATTEMPTS.clear()
RATE_EVENTS.clear()
db.init_db()
return TestClient(app.app, raise_server_exceptions=False)
def _admin_headers(client: TestClient) -> dict:
login = client.post("/api/auth/login", json={"pin": "1234"}, headers={"Origin": "http://testserver"})
sid = login.json()["session_id"]
return {"X-Session-ID": sid, "Origin": "http://testserver"}
def _guest_headers(client: TestClient) -> dict:
sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()["session_id"]
return {"X-Session-ID": sid, "Origin": "http://testserver"}
def _create_memory(client: TestClient, headers: dict, fact: str = "test fact", topic: str = "general") -> int:
resp = client.post("/api/memories", json={"fact": fact, "topic": topic}, headers=headers)
assert resp.status_code == 200
return resp.json()["rowid"]
def test_list_memories_empty(tmp_path: Path):
with make_client(tmp_path) as client:
resp = client.get("/api/memories", headers=_guest_headers(client))
assert resp.status_code == 200
assert resp.json()["count"] == 0
def test_list_memories_by_topic(tmp_path: Path):
with make_client(tmp_path) as client:
headers = _admin_headers(client)
_create_memory(client, headers, "I like Python", "preference")
_create_memory(client, headers, "Building a game", "project")
general = client.get("/api/memories?topic=preference", headers=_guest_headers(client))
assert general.json()["count"] == 1
assert general.json()["memories"][0]["topic"] == "preference"
def test_create_memory_requires_fact(tmp_path: Path):
with make_client(tmp_path) as client:
resp = client.post("/api/memories", json={"fact": ""}, headers=_admin_headers(client))
assert resp.status_code == 400
def test_create_memory_too_long(tmp_path: Path):
with make_client(tmp_path) as client:
long_fact = "x" * (config.MAX_MEMORY_FACT_CHARS + 1)
resp = client.post("/api/memories", json={"fact": long_fact}, headers=_admin_headers(client))
assert resp.status_code == 413
def test_edit_memory(tmp_path: Path):
with make_client(tmp_path) as client:
headers = _admin_headers(client)
rowid = _create_memory(client, headers, "original fact")
edit = client.put(f"/api/memories/{rowid}", json={"fact": "updated fact"}, headers=headers)
assert edit.status_code == 200
memories = client.get("/api/memories", headers=_guest_headers(client)).json()
assert any(m["fact"] == "updated fact" for m in memories["memories"])
def test_edit_memory_not_found(tmp_path: Path):
with make_client(tmp_path) as client:
resp = client.put("/api/memories/99999", json={"fact": "nope"}, headers=_admin_headers(client))
assert resp.status_code == 404
def test_edit_memory_empty_fact(tmp_path: Path):
with make_client(tmp_path) as client:
headers = _admin_headers(client)
rowid = _create_memory(client, headers, "some fact")
resp = client.put(f"/api/memories/{rowid}", json={"fact": ""}, headers=headers)
assert resp.status_code == 400
def test_edit_memory_too_long(tmp_path: Path):
with make_client(tmp_path) as client:
headers = _admin_headers(client)
rowid = _create_memory(client, headers, "some fact")
long_fact = "x" * (config.MAX_MEMORY_FACT_CHARS + 1)
resp = client.put(f"/api/memories/{rowid}", json={"fact": long_fact}, headers=headers)
assert resp.status_code == 413
def test_delete_memory_not_found(tmp_path: Path):
with make_client(tmp_path) as client:
resp = client.delete("/api/memories/99999", headers=_admin_headers(client))
assert resp.status_code == 404
def test_search_memories(tmp_path: Path):
with make_client(tmp_path) as client:
headers = _admin_headers(client)
_create_memory(client, headers, "my favorite color is blue", "preference")
_create_memory(client, headers, "running nginx on port 443", "infrastructure")
resp = client.get("/api/memories/search?q=nginx&limit=5", headers=_guest_headers(client))
assert resp.status_code == 200
data = resp.json()
assert data["count"] >= 1
assert any("nginx" in r["fact"] for r in data["results"])
def test_search_memories_no_results(tmp_path: Path):
with make_client(tmp_path) as client:
resp = client.get("/api/memories/search?q=xyznonexistent&limit=5", headers=_guest_headers(client))
assert resp.status_code == 200
assert resp.json()["count"] == 0
def test_memory_stats(tmp_path: Path):
with make_client(tmp_path) as client:
headers = _admin_headers(client)
_create_memory(client, headers, "like rust", "preference")
_create_memory(client, headers, "like python", "preference")
_create_memory(client, headers, "project game", "project")
resp = client.get("/api/memories/stats", headers=_guest_headers(client))
assert resp.status_code == 200
data = resp.json()
assert data["total"] == 3
assert data["by_topic"]["preference"] == 2
assert data["by_topic"]["project"] == 1
def test_guest_cannot_create_memory(tmp_path: Path):
with make_client(tmp_path) as client:
resp = client.post("/api/memories", json={"fact": "hack"}, headers=_guest_headers(client))
assert resp.status_code == 403
def test_guest_cannot_edit_memory(tmp_path: Path):
with make_client(tmp_path) as client:
resp = client.put("/api/memories/1", json={"fact": "hack"}, headers=_guest_headers(client))
assert resp.status_code == 403
def test_guest_cannot_delete_memory(tmp_path: Path):
with make_client(tmp_path) as client:
resp = client.delete("/api/memories/1", headers=_guest_headers(client))
assert resp.status_code == 403
tests/test_model_swap.py
-202
View File
@@ -1,202 +0,0 @@
"""Tests for cluster.py model swap flow."""
import asyncio
import cluster
import triage
from config import AMQP_EXCHANGE_ADMIN, AMQP_EXCHANGE_SYSTEM
def _reset():
cluster.CLUSTER_NODES.clear()
cluster.CLUSTER_EVENTS.clear()
cluster.CLUSTER_COORDINATOR = None
cluster._pending_pings.clear()
_published = []
async def _fake_publish(exchange, routing_key, payload):
_published.append((exchange, routing_key, payload))
# ---------- 1. request_model_swap() publishes swap command ----------
def test_request_model_swap_publishes_command(monkeypatch):
_reset()
_published.clear()
monkeypatch.setattr(cluster, "publish", _fake_publish)
cluster.CLUSTER_NODES["jarvis"] = {
"name": "jarvis", "type": "worker", "status": "active",
}
asyncio.run(cluster.request_model_swap("jarvis", "qwen2.5-coder-Q4_K_M.gguf"))
assert cluster.CLUSTER_NODES["jarvis"]["status"] == "swapping"
assert len(_published) == 1
exchange, rk, payload = _published[0]
assert exchange == AMQP_EXCHANGE_ADMIN
assert rk == "node.jarvis.cmd.swap_model"
assert payload["model_filename"] == "qwen2.5-coder-Q4_K_M.gguf"
assert "requested_at" in payload
def test_request_model_swap_unknown_node(monkeypatch):
_reset()
_published.clear()
monkeypatch.setattr(cluster, "publish", _fake_publish)
result = asyncio.run(cluster.request_model_swap("ghost", "any.gguf"))
assert result is False
assert len(_published) == 0
# ---------- 2. handle_model_ready() updates node ----------
def test_handle_model_ready_updates_active_model(monkeypatch):
_reset()
_published.clear()
monkeypatch.setattr(cluster, "publish", _fake_publish)
cluster.CLUSTER_NODES["jarvis"] = {
"name": "jarvis", "type": "worker", "status": "swapping",
"active_model": {"name": "llama3.1", "port": 8081},
"inventory": [
{"filename": "qwen2.5-coder-Q4_K_M.gguf", "name": "qwen2.5-coder", "version": "14b", "quant": "Q4_K_M"},
],
}
asyncio.run(cluster.handle_model_ready(
AMQP_EXCHANGE_SYSTEM, "node.jarvis.model_ready",
{"node_name": "jarvis", "active_model": "qwen2.5-coder-Q4_K_M.gguf", "port": 8082},
))
assert cluster.CLUSTER_NODES["jarvis"]["status"] == "active"
am = cluster.CLUSTER_NODES["jarvis"]["active_model"]
assert am["name"] == "qwen2.5-coder"
assert am["port"] == 8082
assert am["filename"] == "qwen2.5-coder-Q4_K_M.gguf"
def test_handle_model_ready_inventory_lookup_fallback(monkeypatch):
_reset()
_published.clear()
monkeypatch.setattr(cluster, "publish", _fake_publish)
cluster.CLUSTER_NODES["jarvis"] = {
"name": "jarvis", "type": "worker", "status": "swapping",
"active_model": {"name": "llama3.1", "port": 8081},
"inventory": [],
}
asyncio.run(cluster.handle_model_ready(
AMQP_EXCHANGE_SYSTEM, "node.jarvis.model_ready",
{"node_name": "jarvis", "active_model": "unknown.gguf", "port": 9999},
))
assert cluster.CLUSTER_NODES["jarvis"]["status"] == "active"
am = cluster.CLUSTER_NODES["jarvis"]["active_model"]
assert am["filename"] == "unknown.gguf"
assert am["port"] == 9999
def test_handle_model_ready_unknown_node(caplog, monkeypatch):
_reset()
caplog.set_level("WARNING")
monkeypatch.setattr(cluster, "publish", _fake_publish)
asyncio.run(cluster.handle_model_ready(
AMQP_EXCHANGE_SYSTEM, "node.ghost.model_ready",
{"node_name": "ghost", "active_model": "any.gguf"},
))
assert any("unknown node" in rec.message for rec in caplog.records)
# ---------- 3. handle_model_failed() sets error status ----------
def test_handle_model_failed_sets_error(monkeypatch):
_reset()
_published.clear()
monkeypatch.setattr(cluster, "publish", _fake_publish)
cluster.CLUSTER_NODES["jarvis"] = {
"name": "jarvis", "type": "worker", "status": "swapping",
}
asyncio.run(cluster.handle_model_failed(
AMQP_EXCHANGE_SYSTEM, "node.jarvis.model_failed",
{"node_name": "jarvis", "error": "llama-server unhealthy after 120s"},
))
assert cluster.CLUSTER_NODES["jarvis"]["status"] == "error"
assert len(cluster.CLUSTER_EVENTS) == 1
assert "swap failed" in cluster.CLUSTER_EVENTS[0]["message"].lower()
def test_handle_model_failed_unknown_node(caplog, monkeypatch):
_reset()
caplog.set_level("WARNING")
monkeypatch.setattr(cluster, "publish", _fake_publish)
asyncio.run(cluster.handle_model_failed(
AMQP_EXCHANGE_SYSTEM, "node.ghost.model_failed",
{"node_name": "ghost", "error": "OOM"},
))
assert any("unknown node" in rec.message for rec in caplog.records)
# ---------- 4. select_node() triggers swap when model mismatched ----------
def test_select_node_code_triggers_swap(monkeypatch):
_reset()
_published.clear()
monkeypatch.setattr(cluster, "publish", _fake_publish)
cluster.CLUSTER_NODES["jarvis"] = {
"name": "jarvis", "type": "worker", "status": "active",
"ip": "192.168.50.210",
"active_model": {"name": "llama3.1", "port": 8081},
"inventory": [
{"filename": "qwen2.5-coder-14b-Q4_K_M.gguf", "name": "qwen2.5-coder", "version": "14b", "quant": "Q4_K_M"},
],
}
result = asyncio.run(triage.select_node("code"))
assert result is None
# Swap should have been published
assert any("cmd.swap_model" in rk for _, rk, _ in _published)
# Node should now be swapping
assert cluster.CLUSTER_NODES["jarvis"]["status"] == "swapping"
# ---------- 5. select_node() returns None when node is already swapping ----------
def test_select_node_swapping_returns_none(monkeypatch):
_reset()
_published.clear()
monkeypatch.setattr(cluster, "publish", _fake_publish)
cluster.CLUSTER_NODES["jarvis"] = {
"name": "jarvis", "type": "worker", "status": "swapping",
"ip": "192.168.50.210",
"active_model": {"name": "llama3.1", "port": 8081},
"inventory": [
{"filename": "qwen2.5-coder-14b-Q4_K_M.gguf", "name": "qwen2.5-coder"},
],
}
result = asyncio.run(triage.select_node("code"))
assert result is None
# No swap command should be published while already swapping
swap_published = any("cmd.swap_model" in rk for _, rk, _ in _published)
assert not swap_published
tests/test_models_router.py
-138
View File
@@ -1,138 +0,0 @@
import os
from pathlib import Path
import httpx
from fastapi.testclient import TestClient
import app
import db
import routers.models
from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS
def make_client(tmp_path: Path) -> TestClient:
os.environ["CAIC_ADMIN_PIN"] = "1234"
db.DB_PATH = tmp_path / "caic-models.db"
SESSIONS.clear()
PIN_ATTEMPTS.clear()
RATE_EVENTS.clear()
db.init_db()
return TestClient(app.app, raise_server_exceptions=False)
def _guest_headers(client: TestClient) -> dict:
sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()["session_id"]
return {"X-Session-ID": sid, "Origin": "http://testserver"}
class _MockAsyncResponse:
"""Mock for httpx.AsyncClient.get/post that returns a JSON response."""
def __init__(self, status_code=200, json_data=None):
self.status_code = status_code
self._json_data = json_data or {}
def json(self):
return self._json_data
async def _mock_get_models(*args, **kwargs):
return _MockAsyncResponse(json_data={
"data": [{"id": "llama3.1:latest"}, {"id": "qwen2:latest"}]
})
async def _mock_get_empty_models(*args, **kwargs):
return _MockAsyncResponse(json_data={"data": []})
async def _mock_connect_error(*args, **kwargs):
raise httpx.ConnectError("Connection refused")
async def _mock_show_model(*args, **kwargs):
return _MockAsyncResponse(json_data={
"modelfile": "FROM llama3.1", "parameters": {}
})
async def _mock_search_available(*args, **kwargs):
return _MockAsyncResponse(status_code=200)
async def _mock_search_unavailable(*args, **kwargs):
raise httpx.ConnectError("refused")
def test_list_models(tmp_path: Path, monkeypatch):
monkeypatch.setattr(httpx.AsyncClient, "get", _mock_get_models)
with make_client(tmp_path) as client:
resp = client.get("/api/models", headers=_guest_headers(client))
assert resp.status_code == 200
models = resp.json()["models"]
assert len(models) == 2
assert models[0]["name"] == "llama3.1:latest"
def test_list_models_connect_error(tmp_path: Path, monkeypatch):
monkeypatch.setattr(httpx.AsyncClient, "get", _mock_connect_error)
with make_client(tmp_path) as client:
resp = client.get("/api/models", headers=_guest_headers(client))
assert resp.status_code == 502
def test_running_models(tmp_path: Path, monkeypatch):
monkeypatch.setattr(httpx.AsyncClient, "get", _mock_get_models)
with make_client(tmp_path) as client:
resp = client.get("/api/ps", headers=_guest_headers(client))
assert resp.status_code == 200
assert "data" in resp.json()
def test_running_models_connect_error(tmp_path: Path, monkeypatch):
monkeypatch.setattr(httpx.AsyncClient, "get", _mock_connect_error)
with make_client(tmp_path) as client:
resp = client.get("/api/ps", headers=_guest_headers(client))
assert resp.status_code == 502
def test_show_model(tmp_path: Path, monkeypatch):
monkeypatch.setattr(httpx.AsyncClient, "post", _mock_show_model)
with make_client(tmp_path) as client:
resp = client.post("/api/show", json={"model": "llama3.1:latest"}, headers=_guest_headers(client))
assert resp.status_code == 200
assert resp.json()["modelfile"] == "FROM llama3.1"
def test_show_model_connect_error(tmp_path: Path, monkeypatch):
monkeypatch.setattr(httpx.AsyncClient, "post", _mock_connect_error)
with make_client(tmp_path) as client:
resp = client.post("/api/show", json={"model": "llama3.1:latest"}, headers=_guest_headers(client))
assert resp.status_code == 502
def test_system_stats(tmp_path: Path, monkeypatch):
monkeypatch.setattr(routers.models, "get_gpu_stats", lambda: {"gpu_percent": 15, "vram_percent": 30, "available": True})
with make_client(tmp_path) as client:
resp = client.get("/api/stats", headers=_guest_headers(client))
assert resp.status_code == 200
data = resp.json()
assert "cpu_percent" in data
assert "memory_percent" in data
assert data["gpu_percent"] == 15
assert data["gpu_available"] is True
def test_search_status_available(tmp_path: Path, monkeypatch):
monkeypatch.setattr(httpx.AsyncClient, "get", _mock_search_available)
with make_client(tmp_path) as client:
resp = client.get("/api/search/status", headers=_guest_headers(client))
assert resp.status_code == 200
assert resp.json()["available"] is True
def test_search_status_unavailable(tmp_path: Path, monkeypatch):
monkeypatch.setattr(httpx.AsyncClient, "get", _mock_search_unavailable)
with make_client(tmp_path) as client:
resp = client.get("/api/search/status", headers=_guest_headers(client))
assert resp.status_code == 200
assert resp.json()["available"] is False
tests/test_node_agent.py
-340
View File
@@ -1,340 +0,0 @@
"""Tests for node_agent/agent.py — standalone worker agent."""
import asyncio
import json
import os
import subprocess
from contextlib import asynccontextmanager
from pathlib import Path
import node_agent.agent as agent
from node_agent.agent import (
AgentConfig,
ModelInfo,
build_registration_payload,
discover_models,
get_load,
handle_ping,
handle_swap_model,
)
class FakeMsg:
def __init__(self, body_dict: dict):
self.body = json.dumps(body_dict).encode()
@asynccontextmanager
async def process(self):
yield
class FakeExchange:
def __init__(self, name=""):
self.name = name
self.published = []
async def publish(self, msg, routing_key):
self.published.append((msg, routing_key))
class FakeChannel:
def __init__(self):
self.exchanges = {}
self.is_closed = False
async def get_exchange(self, name):
if name not in self.exchanges:
self.exchanges[name] = FakeExchange(name)
return self.exchanges[name]
async def declare_exchange(self, name, typ, durable=True):
self.exchanges[name] = FakeExchange(name)
return self.exchanges[name]
async def declare_queue(self, name="", exclusive=True):
return self
async def bind(self, exchange, routing_key):
pass
async def close(self):
self.is_closed = True
# ── 1. Registration payload shape ────────────────────────────────────
def test_registration_payload_shape():
cfg = AgentConfig()
cfg.node_name = "worker01"
cfg.node_ip = "192.168.50.210"
cfg.capabilities = ["llm"]
cfg.active_model = "llama3.1-latest-Q4_K_M.gguf"
cfg.llama_port = 8081
inventory = [{"filename": "llama3.1-latest-Q4_K_M.gguf", "name": "llama3.1", "quant": "Q4_K_M"}]
payload = build_registration_payload(cfg, inventory)
assert payload["node_name"] == "worker01"
assert payload["node_type"] == "worker"
assert payload["ip"] == "192.168.50.210"
assert payload["capabilities"] == ["llm"]
assert "active_model" in payload
assert payload["active_model"]["filename"] == "llama3.1-latest-Q4_K_M.gguf"
assert payload["active_model"]["port"] == 8081
assert len(payload["inventory"]) == 1
def test_registration_payload_active_model_none():
cfg = AgentConfig()
cfg.active_model = ""
payload = build_registration_payload(cfg, [])
assert payload["active_model"] is None
# ── 2. Model discovery ───────────────────────────────────────────────
def test_discover_models(tmp_path):
models_dir = tmp_path / "models"
models_dir.mkdir()
# Create valid model files
(models_dir / "llama3.1-latest-Q4_K_M.gguf").write_text("")
(models_dir / "mistral-nemo-7b-Q6_K_L.gguf").write_text("")
# Create file that doesn't match pattern
(models_dir / "readme.txt").write_text("")
# Create file with unrecognized naming
(models_dir / "my_custom_model.gguf").write_text("")
result = discover_models(str(models_dir))
assert len(result) == 2
names = {m["name"] for m in result}
assert "llama3.1" in names
assert "mistral-nemo" in names
def test_discover_models_no_directory(tmp_path):
result = discover_models(str(tmp_path / "nonexistent"))
assert result == []
# ── 3. Config reading ────────────────────────────────────────────────
def test_config_from_ini(tmp_path):
ini = tmp_path / "caic-node-agent.conf"
ini.write_text(
"[agent]\n"
"node_name = testnode\n"
"node_ip = 10.0.0.5\n"
"capabilities = llm,rag\n"
"amqp_url = amqp://user:pass@host/vhost\n"
"llama_port = 9090\n"
"models_dir = /tmp/models\n"
"active_model = test.gguf\n"
)
cfg = AgentConfig.from_ini(str(ini))
assert cfg.node_name == "testnode"
assert cfg.node_ip == "10.0.0.5"
assert cfg.capabilities == ["llm", "rag"]
assert cfg.amqp_url == "amqp://user:pass@host/vhost"
assert cfg.llama_port == 9090
assert cfg.models_dir == "/tmp/models"
assert cfg.active_model == "test.gguf"
def test_config_from_ini_missing_uses_defaults(tmp_path):
cfg = AgentConfig.from_ini(str(tmp_path / "nonexistent.conf"))
assert cfg.node_name != "" # socket.gethostname() returns something
assert cfg.node_type == "worker"
assert cfg.capabilities == ["llm"]
# ── 4. Ping handler publishes pong ──────────────────────────────────
def test_ping_handler_publishes_pong(monkeypatch):
monkeypatch.setattr(agent, "HAS_PSUTIL", False)
monkeypatch.setattr(agent, "HAS_AIO_PIKA", True)
cfg = AgentConfig()
cfg.node_name = "testworker"
channel = FakeChannel()
admin_ex = FakeExchange("jc.admin")
channel.exchanges["jc.admin"] = admin_ex
exchange = admin_ex
async def run():
await handle_ping(cfg, channel, exchange, FakeMsg({
"from": "coordinator",
"node_name": "testworker",
"type": "ping",
"correlation_id": "abc-123",
"timestamp": "2026-01-01T00:00:00Z",
}))
asyncio.run(run())
assert len(exchange.published) == 1
msg, rk = exchange.published[0]
assert rk == "node.testworker.pong"
payload = json.loads(msg.body)
assert payload["type"] == "pong"
assert payload["correlation_id"] == "abc-123"
assert payload["node_name"] == "testworker"
# ── 5. Model swap success path ──────────────────────────────────────
def test_model_swap_success(monkeypatch):
monkeypatch.setattr(agent, "HAS_AIO_PIKA", True)
monkeypatch.setattr(agent, "HAS_HTTPX", True)
cfg = AgentConfig()
cfg.node_name = "testworker"
cfg.llama_port = 9999
captured_cmds = []
def fake_run(cmd, **kwargs):
captured_cmds.append(cmd)
return subprocess.CompletedProcess(cmd, 0, b"", b"")
monkeypatch.setattr(subprocess, "run", fake_run)
# Mock _wait_for_llama to succeed
async def fake_wait(*a, **kw):
return True
monkeypatch.setattr(agent, "_wait_for_llama", fake_wait)
# Mock _update_config_active_model to no-op
monkeypatch.setattr(agent, "_update_config_active_model", lambda c, m: None)
channel = FakeChannel()
admin_ex = FakeExchange("jc.admin")
system_ex = FakeExchange("jc.system")
channel.exchanges["jc.admin"] = admin_ex
channel.exchanges["jc.system"] = system_ex
asyncio.run(handle_swap_model(cfg, channel, (admin_ex, system_ex), FakeMsg({"model_filename": "new-model-Q4_K_M.gguf"})))
# Check systemctl calls
assert len(captured_cmds) == 2
assert captured_cmds[0] == ["systemctl", "stop", "llama-server"]
assert captured_cmds[1] == ["systemctl", "start", "llama-server"]
# Check model_ready published on jc.system
assert len(system_ex.published) == 1
msg, rk = system_ex.published[0]
assert rk == "node.testworker.model_ready"
payload = json.loads(msg.body)
assert payload["type"] == "model_ready"
assert payload["active_model"] == "new-model-Q4_K_M.gguf"
# ── 6. Model swap timeout path ──────────────────────────────────────
def test_model_swap_timeout(monkeypatch):
monkeypatch.setattr(agent, "HAS_AIO_PIKA", True)
monkeypatch.setattr(agent, "HAS_HTTPX", True)
cfg = AgentConfig()
cfg.node_name = "testworker"
monkeypatch.setattr(subprocess, "run", lambda cmd, **kw: subprocess.CompletedProcess(cmd, 0, b"", b""))
# Mock _wait_for_llama to fail
async def fake_wait_fail(*a, **kw):
return False
monkeypatch.setattr(agent, "_wait_for_llama", fake_wait_fail)
monkeypatch.setattr(agent, "_update_config_active_model", lambda c, m: None)
channel = FakeChannel()
admin_ex = FakeExchange("jc.admin")
system_ex = FakeExchange("jc.system")
channel.exchanges["jc.admin"] = admin_ex
channel.exchanges["jc.system"] = system_ex
asyncio.run(handle_swap_model(cfg, channel, (admin_ex, system_ex), FakeMsg({"model_filename": "broken-model-Q4_K_M.gguf"})))
assert len(system_ex.published) == 1
msg, rk = system_ex.published[0]
assert rk == "node.testworker.model_failed"
payload = json.loads(msg.body)
assert payload["type"] == "model_failed"
assert "error" in payload
# ── 7. Load reporting ───────────────────────────────────────────────
def test_get_load_with_psutil(monkeypatch):
class FakePsutil:
@staticmethod
def cpu_percent(interval=0.5):
return 42.0
@staticmethod
def virtual_memory():
class VM:
percent = 65.0
return VM()
monkeypatch.setattr(agent, "HAS_PSUTIL", True)
monkeypatch.setattr(agent, "psutil", FakePsutil)
# Mock subprocess to fail (no rocm-smi)
def fake_run(*a, **kw):
raise FileNotFoundError("rocm-smi not found")
monkeypatch.setattr(subprocess, "run", fake_run)
load = get_load()
assert load["cpu_pct"] == 42
assert load["ram_pct"] == 65
assert "vram_pct" not in load
def test_get_load_without_psutil(monkeypatch):
monkeypatch.setattr(agent, "HAS_PSUTIL", False)
monkeypatch.setattr(subprocess, "run", lambda *a, **kw: (_ for _ in ()).throw(FileNotFoundError("")))
load = get_load()
assert "cpu_pct" not in load
# ── 8. Agent idle after admission (no heartbeat) ────────────────────
def test_no_heartbeat_timer():
"""Agent has no background heartbeat mechanism. This test asserts that
the codebase contains no heartbeat-related logic in the agent itself."""
import inspect
source = inspect.getsource(agent)
# There should be no heartbeat timer in the agent
assert "heartbeat" not in source.lower(), \
"agent.py should contain no heartbeat logic"
# ── 9. Config writer for model swap ─────────────────────────────────
def test_update_config_active_model(tmp_path, monkeypatch):
monkeypatch.setattr(agent, "CONFIG_PATH", str(tmp_path / "caic-node-agent.conf"))
cfg = AgentConfig()
cfg.active_model = "old.gguf"
agent._update_config_active_model(cfg, "new.gguf")
assert cfg.active_model == "new.gguf"
content = (tmp_path / "caic-node-agent.conf").read_text()
assert "new.gguf" in content
# ── 10. ModelInfo to_dict ────────────────────────────────────────────
def test_model_info_to_dict():
info = ModelInfo(filename="test-Q4_K_M.gguf", name="test", version="latest", quant="Q4_K_M")
d = info.to_dict()
assert d["name"] == "test"
assert d["quant"] == "Q4_K_M"
assert d["filename"] == "test-Q4_K_M.gguf"
tests/test_presets.py
-128
View File
@@ -1,128 +0,0 @@
import os
from pathlib import Path
from fastapi.testclient import TestClient
import app
import db
from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS
def make_client(tmp_path: Path) -> TestClient:
os.environ["CAIC_ADMIN_PIN"] = "1234"
db.DB_PATH = tmp_path / "caic-presets.db"
SESSIONS.clear()
PIN_ATTEMPTS.clear()
RATE_EVENTS.clear()
db.init_db()
return TestClient(app.app, raise_server_exceptions=False)
def _admin_headers(client: TestClient) -> dict:
login = client.post("/api/auth/login", json={"pin": "1234"}, headers={"Origin": "http://testserver"})
sid = login.json()["session_id"]
return {"X-Session-ID": sid, "Origin": "http://testserver"}
def _guest_headers(client: TestClient) -> dict:
sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()["session_id"]
return {"X-Session-ID": sid, "Origin": "http://testserver"}
def test_list_presets_returns_defaults(tmp_path: Path):
with make_client(tmp_path) as client:
resp = client.get("/api/presets", headers=_guest_headers(client))
assert resp.status_code == 200
presets = resp.json()
assert len(presets) >= 3
names = [p["name"] for p in presets]
assert "Coding Companion" in names
def test_create_preset(tmp_path: Path):
with make_client(tmp_path) as client:
headers = _admin_headers(client)
resp = client.post("/api/presets", json={"name": "My Preset", "prompt": "You are helpful."}, headers=headers)
assert resp.status_code == 200
data = resp.json()
assert data["name"] == "My Preset"
assert data["prompt"] == "You are helpful."
presets = client.get("/api/presets", headers=_guest_headers(client)).json()
assert any(p["name"] == "My Preset" for p in presets)
def test_create_preset_requires_name_and_prompt(tmp_path: Path):
with make_client(tmp_path) as client:
headers = _admin_headers(client)
resp = client.post("/api/presets", json={"name": "", "prompt": ""}, headers=headers)
assert resp.status_code == 400
resp = client.post("/api/presets", json={"name": "Only Name"}, headers=headers)
assert resp.status_code == 400
def test_update_preset(tmp_path: Path):
with make_client(tmp_path) as client:
headers = _admin_headers(client)
create = client.post("/api/presets", json={"name": "Old", "prompt": "Old prompt."}, headers=headers)
preset_id = create.json()["id"]
update = client.put(f"/api/presets/{preset_id}", json={"name": "New", "prompt": "New prompt."}, headers=headers)
assert update.status_code == 200
presets = client.get("/api/presets", headers=_guest_headers(client)).json()
updated = next(p for p in presets if p["id"] == preset_id)
assert updated["name"] == "New"
assert updated["prompt"] == "New prompt."
def test_update_preset_requires_fields(tmp_path: Path):
with make_client(tmp_path) as client:
headers = _admin_headers(client)
resp = client.put("/api/presets/nope", json={"name": "", "prompt": ""}, headers=headers)
assert resp.status_code == 400
def test_delete_preset(tmp_path: Path):
with make_client(tmp_path) as client:
headers = _admin_headers(client)
create = client.post("/api/presets", json={"name": "Temp", "prompt": "Temp."}, headers=headers)
preset_id = create.json()["id"]
delete = client.delete(f"/api/presets/{preset_id}", headers=headers)
assert delete.status_code == 200
presets = client.get("/api/presets", headers=_guest_headers(client)).json()
assert not any(p["id"] == preset_id for p in presets)
def test_delete_default_preset_is_noop(tmp_path: Path):
with make_client(tmp_path) as client:
headers = _admin_headers(client)
presets_before = client.get("/api/presets", headers=_guest_headers(client)).json()
default = next(p for p in presets_before if p["is_default"])
delete = client.delete(f"/api/presets/{default['id']}", headers=headers)
assert delete.status_code == 200
presets_after = client.get("/api/presets", headers=_guest_headers(client)).json()
assert any(p["id"] == default["id"] for p in presets_after)
def test_guest_cannot_create_preset(tmp_path: Path):
with make_client(tmp_path) as client:
resp = client.post("/api/presets", json={"name": "Hack", "prompt": "Hack"}, headers=_guest_headers(client))
assert resp.status_code == 403
def test_guest_cannot_update_preset(tmp_path: Path):
with make_client(tmp_path) as client:
resp = client.put("/api/presets/some-id", json={"name": "Hack", "prompt": "Hack"}, headers=_guest_headers(client))
assert resp.status_code == 403
def test_guest_cannot_delete_preset(tmp_path: Path):
with make_client(tmp_path) as client:
resp = client.delete("/api/presets/some-id", headers=_guest_headers(client))
assert resp.status_code == 403
tests/test_profile.py
-72
View File
@@ -1,72 +0,0 @@
import os
from pathlib import Path
from fastapi.testclient import TestClient
import app
import config
import db
from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS
def make_client(tmp_path: Path) -> TestClient:
os.environ["CAIC_ADMIN_PIN"] = "1234"
db.DB_PATH = tmp_path / "caic-profile.db"
SESSIONS.clear()
PIN_ATTEMPTS.clear()
RATE_EVENTS.clear()
db.init_db()
return TestClient(app.app, raise_server_exceptions=False)
def _admin_headers(client: TestClient) -> dict:
login = client.post("/api/auth/login", json={"pin": "1234"}, headers={"Origin": "http://testserver"})
sid = login.json()["session_id"]
return {"X-Session-ID": sid, "Origin": "http://testserver"}
def _guest_headers(client: TestClient) -> dict:
sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()["session_id"]
return {"X-Session-ID": sid, "Origin": "http://testserver"}
def test_get_profile_returns_content(tmp_path: Path):
with make_client(tmp_path) as client:
resp = client.get("/api/profile", headers=_guest_headers(client))
assert resp.status_code == 200
data = resp.json()
assert "content" in data
assert "updated_at" in data
assert len(data["content"]) > 0
def test_get_default_profile(tmp_path: Path):
with make_client(tmp_path) as client:
resp = client.get("/api/profile/default", headers=_guest_headers(client))
assert resp.status_code == 200
assert resp.json()["content"] == config.DEFAULT_PROFILE
def test_update_profile(tmp_path: Path):
with make_client(tmp_path) as client:
headers = _admin_headers(client)
resp = client.put("/api/profile", json={"content": "Custom profile text."}, headers=headers)
assert resp.status_code == 200
assert "updated_at" in resp.json()
get_resp = client.get("/api/profile", headers=_guest_headers(client))
assert get_resp.json()["content"] == "Custom profile text."
def test_update_profile_too_long(tmp_path: Path):
with make_client(tmp_path) as client:
headers = _admin_headers(client)
long_content = "x" * (config.MAX_PROFILE_CHARS + 1)
resp = client.put("/api/profile", json={"content": long_content}, headers=headers)
assert resp.status_code == 413
def test_guest_cannot_update_profile(tmp_path: Path):
with make_client(tmp_path) as client:
resp = client.put("/api/profile", json={"content": "hack"}, headers=_guest_headers(client))
assert resp.status_code == 403
tests/test_rag_management.py
-409
View File
@@ -1,409 +0,0 @@
import asyncio
import os
from datetime import datetime, timezone, timedelta
from pathlib import Path
import httpx
from fastapi.testclient import TestClient
import app
import config
import db
import rag
from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS
def make_client(tmp_path: Path) -> TestClient:
os.environ["CAIC_ADMIN_PIN"] = "1234"
db.DB_PATH = tmp_path / "caic-rag-mgmt.db"
SESSIONS.clear()
PIN_ATTEMPTS.clear()
RATE_EVENTS.clear()
db.init_db()
return TestClient(app.app, raise_server_exceptions=False)
def _admin_headers(client: TestClient) -> dict:
login = client.post("/api/auth/login", json={"pin": "1234"}, headers={"Origin": "http://testserver"})
sid = login.json()["session_id"]
return {"X-Session-ID": sid, "Origin": "http://testserver"}
def _guest_headers(client: TestClient) -> dict:
sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()["session_id"]
return {"X-Session-ID": sid, "Origin": "http://testserver"}
class FakeResponse:
def __init__(self, status, json_data=None):
self.status_code = status
self._json = json_data or {}
def json(self):
return self._json
class FakeAsyncClient:
def __init__(self, *a, **kw):
pass
async def __aenter__(self):
return self
async def __aexit__(self, *a):
pass
async def get(self, url, **kw):
if "/collections/caic_rag" in url:
return FakeResponse(200, {"result": {"vectors_count": 123}})
return FakeResponse(200)
async def post(self, url, **kw):
if "/points/scroll" in url:
return FakeResponse(200, {"result": {"points": []}})
if "/points/delete" in url:
return FakeResponse(200)
return FakeResponse(200)
async def put(self, url, **kw):
return FakeResponse(200)
def _old_ts(hours_ago: float = 24) -> str:
return (datetime.now(timezone.utc) - timedelta(hours=hours_ago)).isoformat()
def _young_ts(hours_ago: float = 0.1) -> str:
return (datetime.now(timezone.utc) - timedelta(hours=hours_ago)).isoformat()
# ---------- get_collection_count ----------
def test_get_collection_count(monkeypatch):
monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: FakeAsyncClient())
count = asyncio.run(rag.get_collection_count())
assert count == 123
# ---------- get_collection_stats ----------
def test_get_collection_stats_shape(monkeypatch):
monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: FakeAsyncClient())
stats = asyncio.run(rag.get_collection_stats())
assert stats["vector_count"] == 123
assert stats["max_vectors"] == 50000
assert stats["high_water_mark"] == 40000
assert stats["low_water_mark"] == 10000
assert stats["high_water_pct"] == 80
assert stats["low_water_pct"] == 20
assert 0 < stats["percent_full"] < 1
assert "upload" in stats["pinned_sources"]
assert "profile" in stats["pinned_sources"]
# ---------- evict_batch ----------
def test_evict_batch_excludes_pinned_sources(monkeypatch):
"""Pinned sources ('upload', 'profile') should be in the must_not scroll filter."""
old = _old_ts(48)
# Only non-pinned points are returned (real Qdrant would honour must_not filter)
scroll_points = [
{"id": "old-data", "payload": {"source": "terminal", "ingest_date": old, "retrieval_count": 0}},
]
class ScrollClient(FakeAsyncClient):
async def post(self, url, **kw):
if "/points/scroll" in url:
must_not = kw.get("json", {}).get("filter", {}).get("must_not", [])
pinned_values = [m["match"]["value"] for m in must_not]
assert "upload" in pinned_values
assert "profile" in pinned_values
return FakeResponse(200, {"result": {"points": scroll_points}})
if "/points/delete" in url:
return FakeResponse(200)
return FakeResponse(200)
monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: ScrollClient())
deleted = asyncio.run(rag.evict_batch(10))
assert deleted == 1
def test_evict_batch_respects_grace_period(monkeypatch):
"""Vectors younger than RAG_GRACE_HOURS should be skipped."""
old = _old_ts(48)
young = _young_ts(0.1)
scroll_points = [
{"id": "mature", "payload": {"source": "terminal", "ingest_date": old, "retrieval_count": 0}},
{"id": "newborn", "payload": {"source": "terminal", "ingest_date": young, "retrieval_count": 0}},
]
class GraceClient(FakeAsyncClient):
async def post(self, url, **kw):
if "/points/scroll" in url:
return FakeResponse(200, {"result": {"points": scroll_points}})
if "/points/delete" in url:
deleted = kw.get("json", {}).get("points", [])
assert "newborn" not in deleted
return FakeResponse(200)
return FakeResponse(200)
monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: GraceClient())
deleted = asyncio.run(rag.evict_batch(10))
assert deleted == 1
def test_evict_batch_respects_batch_size(monkeypatch):
"""Only up to batch_size vectors should be deleted per call."""
old = _old_ts(48)
points = [{"id": f"p{i}", "payload": {"source": "terminal", "ingest_date": old, "retrieval_count": 0}} for i in range(50)]
class BatchClient(FakeAsyncClient):
async def post(self, url, **kw):
if "/points/delete" in url:
deleted = kw.get("json", {}).get("points", [])
assert len(deleted) == 10
return FakeResponse(200)
if "/points/scroll" in url:
return FakeResponse(200, {"result": {"points": points}})
return FakeResponse(200)
monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: BatchClient())
deleted = asyncio.run(rag.evict_batch(10))
assert deleted == 10
def test_evict_batch_all_pinned_returns_zero(monkeypatch):
"""If scroll returns nothing (all points filtered by must_not), evict_batch returns 0."""
class EmptyClient(FakeAsyncClient):
async def post(self, url, **kw):
if "/points/scroll" in url:
return FakeResponse(200, {"result": {"points": []}})
return FakeResponse(200)
monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: EmptyClient())
deleted = asyncio.run(rag.evict_batch(10))
assert deleted == 0
def test_evict_batch_scores_lowest_first(monkeypatch):
"""Vectors with lower scores should be evicted first."""
old = _old_ts(48)
points = [
{"id": "high-score", "payload": {"source": "terminal", "ingest_date": old, "retrieval_count": 100}},
{"id": "low-score", "payload": {"source": "terminal", "ingest_date": old, "retrieval_count": 0}},
{"id": "mid-score", "payload": {"source": "terminal", "ingest_date": old, "retrieval_count": 50}},
]
class ScoreClient(FakeAsyncClient):
async def post(self, url, **kw):
if "/points/delete" in url:
deleted = kw.get("json", {}).get("points", [])
assert "low-score" in deleted
assert "high-score" not in deleted
assert "mid-score" not in deleted
return FakeResponse(200)
if "/points/scroll" in url:
return FakeResponse(200, {"result": {"points": points}})
return FakeResponse(200)
monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: ScoreClient())
deleted = asyncio.run(rag.evict_batch(1))
assert deleted == 1
# ---------- maybe_evict ----------
def test_maybe_evict_below_high_water(monkeypatch):
"""When count is below high-water mark, eviction should not fire."""
class LowCountClient(FakeAsyncClient):
async def get(self, url, **kw):
# 30000 < 40000 high water
return FakeResponse(200, {"result": {"vectors_count": 30000}})
monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: LowCountClient())
rag.EVICTION_LOG.clear()
evicted = asyncio.run(rag.maybe_evict())
assert evicted == 0
assert len(rag.EVICTION_LOG) == 0
def test_maybe_evict_at_high_water(monkeypatch):
"""When count reaches high-water mark, eviction should fire."""
class HighCountClient(FakeAsyncClient):
def __init__(self, *a, **kw):
super().__init__()
self.call_count = 0
async def get(self, url, **kw):
return FakeResponse(200, {"result": {"vectors_count": 45000}})
async def post(self, url, **kw):
if "/points/scroll" in url:
old = _old_ts(48)
points = [{"id": f"evict-me-{i}", "payload": {"source": "terminal", "ingest_date": old, "retrieval_count": 0}} for i in range(100)]
return FakeResponse(200, {"result": {"points": points}})
if "/points/delete" in url:
return FakeResponse(200)
return FakeResponse(200)
monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: HighCountClient())
rag.EVICTION_LOG.clear()
evicted = asyncio.run(rag.maybe_evict())
assert evicted > 0
assert len(rag.EVICTION_LOG) == 1
entry = rag.EVICTION_LOG[0]
assert "timestamp" in entry
assert entry["count"] > 0
def test_maybe_evict_zero_config_disabled(monkeypatch):
"""RAG_MAX_VECTORS <= 0 should disable eviction."""
orig = config.RAG_MAX_VECTORS
try:
config.RAG_MAX_VECTORS = 0
evicted = asyncio.run(rag.maybe_evict())
assert evicted == 0
finally:
config.RAG_MAX_VECTORS = orig
def test_maybe_evict_all_pinned_breaks(monkeypatch):
"""Above high water but only pinned points exist → eviction breaks with 0 deleted."""
class AllPinnedClient(FakeAsyncClient):
async def get(self, url, **kw):
return FakeResponse(200, {"result": {"vectors_count": 45000}})
async def post(self, url, **kw):
if "/points/scroll" in url:
return FakeResponse(200, {"result": {"points": []}})
return FakeResponse(200)
monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: AllPinnedClient())
rag.EVICTION_LOG.clear()
evicted = asyncio.run(rag.maybe_evict())
assert evicted == 0
assert len(rag.EVICTION_LOG) == 0
# ---------- get_rag_operational_stats ----------
def test_rag_operational_stats_shape(monkeypatch):
monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: FakeAsyncClient())
rag.EVICTION_LOG.clear()
rag.EVICTION_LOG.append({
"timestamp": datetime.now(timezone.utc).isoformat(),
"count": 500,
"remaining": 40000,
})
stats = asyncio.run(rag.get_rag_operational_stats())
assert stats["vector_count"] == 123
assert stats["grace_hours"] == 1
assert "eviction_counts_last_1m" in stats
assert "eviction_counts_last_5m" in stats
assert "eviction_counts_last_30m" in stats
assert stats["eviction_counts_last_1m"] == 500
# ---------- GET /api/rag/stats ----------
def test_rag_stats_endpoint(tmp_path, monkeypatch):
monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: FakeAsyncClient())
with make_client(tmp_path) as client:
resp = client.get("/api/rag/stats", headers=_admin_headers(client))
assert resp.status_code == 200
data = resp.json()
assert data["vector_count"] == 123
assert data["max_vectors"] == 50000
assert "high_water_mark" in data
assert "low_water_mark" in data
assert data["high_water_pct"] == 80
assert data["low_water_pct"] == 20
assert "percent_full" in data
assert data["pinned_sources"] == ["upload", "profile"]
assert data["grace_hours"] == 1
assert "eviction_counts_last_1m" in data
assert "eviction_counts_last_5m" in data
assert "eviction_counts_last_30m" in data
assert "pinned_count" in data
assert "avg_retrieval_count" in data
assert "at_risk_count" in data
assert "eviction_log_size" in data
def test_rag_stats_requires_admin(tmp_path, monkeypatch):
monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: FakeAsyncClient())
with make_client(tmp_path) as client:
resp = client.get("/api/rag/stats", headers=_guest_headers(client))
assert resp.status_code == 403
# ---------- POST /api/rag/flush ----------
def test_rag_flush_endpoint(tmp_path, monkeypatch):
class FlushClient(FakeAsyncClient):
async def post(self, url, **kw):
if "/points/scroll" in url:
return FakeResponse(200, {"result": {"points": [{"id": "a"}, {"id": "b"}]}})
if "/points/delete" in url:
return FakeResponse(200)
return FakeResponse(200)
monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: FlushClient())
with make_client(tmp_path) as client:
resp = client.post("/api/rag/flush", headers=_admin_headers(client))
assert resp.status_code == 200
data = resp.json()
assert data["status"] == "flushed"
assert data["deleted_count"] == 2
assert data["collection"] == rag.RAG_COLLECTION
def test_rag_flush_requires_admin(tmp_path, monkeypatch):
monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: FakeAsyncClient())
with make_client(tmp_path) as client:
resp = client.post("/api/rag/flush", headers=_guest_headers(client))
assert resp.status_code == 403
def test_rag_flush_empty_collection(tmp_path, monkeypatch):
class EmptyClient(FakeAsyncClient):
async def post(self, url, **kw):
if "/points/scroll" in url:
return FakeResponse(200, {"result": {"points": []}})
return FakeResponse(200)
monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: EmptyClient())
with make_client(tmp_path) as client:
resp = client.post("/api/rag/flush", headers=_admin_headers(client))
assert resp.status_code == 200
data = resp.json()
assert data["deleted_count"] == 0
assert data["status"] == "flushed"
# ---------- Race lock ----------
def test_eviction_lock_prevents_concurrent_eviction(monkeypatch):
"""Concurrent calls to maybe_evict should queue; only one evicts."""
call_order = []
async def slow_get_collection_count():
call_order.append("count")
return 45000
async def slow_evict_batch(bs):
call_order.append("evict")
await asyncio.sleep(0.05)
return 500
monkeypatch.setattr(rag, "get_collection_count", slow_get_collection_count)
monkeypatch.setattr(rag, "evict_batch", slow_evict_batch)
rag.EVICTION_LOG.clear()
async def run_concurrent():
r1, r2 = await asyncio.gather(rag.maybe_evict(), rag.maybe_evict())
return r1, r2
r1, r2 = asyncio.run(run_concurrent())
# First call evicted, second found count already below high water or lock serialized
assert r1 >= 0
assert r2 >= 0
tests/test_rate_and_payload_guardrails.py
+23 -27
View File
@@ -4,32 +4,28 @@ from pathlib import Path
from fastapi.testclient import TestClient from fastapi.testclient import TestClient
import app import app as app_module
import config
import db
import security
from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS
def make_client(tmp_path: Path) -> TestClient: def make_client(tmp_path: Path) -> TestClient:
os.environ["CAIC_ADMIN_PIN"] = "1234" os.environ["JARVISCHAT_ADMIN_PIN"] = "1234"
db.DB_PATH = tmp_path / "caic-rate.db" app_module.DB_PATH = tmp_path / "jarvischat-rate.db"
SESSIONS.clear() app_module.SESSIONS.clear()
PIN_ATTEMPTS.clear() app_module.PIN_ATTEMPTS.clear()
RATE_EVENTS.clear() app_module.RATE_EVENTS.clear()
db.init_db() app_module.init_db()
return TestClient(app.app) return TestClient(app_module.app)
def test_stats_rate_limit_hits_429(tmp_path: Path): def test_stats_rate_limit_hits_429(tmp_path: Path):
old_limit = security.RL_STATS_PER_WINDOW old_limit = app_module.RL_STATS_PER_WINDOW
old_window = app.RATE_WINDOW_SECONDS old_window = app_module.RATE_WINDOW_SECONDS
security.RL_STATS_PER_WINDOW = 2 app_module.RL_STATS_PER_WINDOW = 2
app.RATE_WINDOW_SECONDS = 60 app_module.RATE_WINDOW_SECONDS = 60
try: try:
with make_client(tmp_path) as client: with make_client(tmp_path) as client:
sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()["session_id"] sid = client.post("/api/auth/guest").json()["session_id"]
headers = {"X-Session-ID": sid, "Origin": "http://testserver"} headers = {"X-Session-ID": sid}
r1 = client.get("/api/stats", headers=headers) r1 = client.get("/api/stats", headers=headers)
r2 = client.get("/api/stats", headers=headers) r2 = client.get("/api/stats", headers=headers)
@@ -39,13 +35,13 @@ def test_stats_rate_limit_hits_429(tmp_path: Path):
assert r2.status_code == 200 assert r2.status_code == 200
assert r3.status_code == 429 assert r3.status_code == 429
finally: finally:
security.RL_STATS_PER_WINDOW = old_limit app_module.RL_STATS_PER_WINDOW = old_limit
app.RATE_WINDOW_SECONDS = old_window app_module.RATE_WINDOW_SECONDS = old_window
def test_large_login_payload_rejected_413(tmp_path: Path): def test_large_login_payload_rejected_413(tmp_path: Path):
with make_client(tmp_path) as client: with make_client(tmp_path) as client:
huge_pin = "1" * (config.BODY_LIMIT_DEFAULT_BYTES + 100) huge_pin = "1" * (app_module.BODY_LIMIT_DEFAULT_BYTES + 100)
resp = client.post( resp = client.post(
"/api/auth/login", "/api/auth/login",
data=json.dumps({"pin": huge_pin}), data=json.dumps({"pin": huge_pin}),
@@ -56,12 +52,12 @@ def test_large_login_payload_rejected_413(tmp_path: Path):
def test_chat_message_length_rejected_413(tmp_path: Path): def test_chat_message_length_rejected_413(tmp_path: Path):
with make_client(tmp_path) as client: with make_client(tmp_path) as client:
sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()["session_id"] sid = client.post("/api/auth/guest").json()["session_id"]
headers = {"X-Session-ID": sid, "Origin": "http://testserver"} headers = {"X-Session-ID": sid, "Origin": "http://testserver"}
message = "x" * (config.MAX_CHAT_MESSAGE_CHARS + 1) message = "x" * (app_module.MAX_CHAT_MESSAGE_CHARS + 1)
resp = client.post( resp = client.post(
"/api/chat", "/api/chat",
json={"message": message, "model": config.DEFAULT_MODEL}, json={"message": message, "model": app_module.DEFAULT_MODEL},
headers=headers, headers=headers,
) )
assert resp.status_code == 413 assert resp.status_code == 413
@@ -69,12 +65,12 @@ def test_chat_message_length_rejected_413(tmp_path: Path):
def test_search_query_length_rejected_413(tmp_path: Path): def test_search_query_length_rejected_413(tmp_path: Path):
with make_client(tmp_path) as client: with make_client(tmp_path) as client:
sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()["session_id"] sid = client.post("/api/auth/guest").json()["session_id"]
headers = {"X-Session-ID": sid, "Origin": "http://testserver"} headers = {"X-Session-ID": sid, "Origin": "http://testserver"}
query = "q" * (config.MAX_SEARCH_QUERY_CHARS + 1) query = "q" * (app_module.MAX_SEARCH_QUERY_CHARS + 1)
resp = client.post( resp = client.post(
"/api/search", "/api/search",
json={"query": query, "model": config.DEFAULT_MODEL}, json={"query": query, "model": app_module.DEFAULT_MODEL},
headers=headers, headers=headers,
) )
assert resp.status_code == 413 assert resp.status_code == 413
tests/test_search_route.py
-186
View File
@@ -1,186 +0,0 @@
import json
import os
from pathlib import Path
import httpx
from fastapi.testclient import TestClient
import app
import config
import db
import routers.search_route
from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS
def make_client(tmp_path: Path) -> TestClient:
os.environ["CAIC_ADMIN_PIN"] = "1234"
db.DB_PATH = tmp_path / "caic-search-route.db"
SESSIONS.clear()
PIN_ATTEMPTS.clear()
RATE_EVENTS.clear()
db.init_db()
return TestClient(app.app, raise_server_exceptions=False)
def _guest_headers(client: TestClient) -> dict:
sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()["session_id"]
return {"X-Session-ID": sid, "Origin": "http://testserver"}
def parse_sse_payloads(body: str) -> list[dict]:
payloads: list[dict] = []
for chunk in body.split("\n\n"):
chunk = chunk.strip()
if not chunk.startswith("data: "):
continue
raw = chunk[len("data: ") :]
payloads.append(json.loads(raw))
return payloads
class _MockStreamResponse:
def __init__(self, lines: list[str]):
self._lines = lines
async def __aenter__(self):
return self
async def __aexit__(self, exc_type, exc, tb):
return False
async def aiter_lines(self):
for line in self._lines:
yield line
def _stream_json_lines(events: list[dict]) -> list[str]:
return [json.dumps(event) for event in events]
def test_explicit_search_with_results(tmp_path: Path, monkeypatch):
with make_client(tmp_path) as client:
headers = _guest_headers(client)
async def search_stub(query: str, max_results: int = 5):
return [
{"title": "Result One", "url": "https://example.com/1", "content": "First result content."},
{"title": "Result Two", "url": "https://example.com/2", "content": "Second result content."},
]
monkeypatch.setattr(routers.search_route, "query_searxng", search_stub)
events = _stream_json_lines([
{"choices": [{"delta": {"content": "Here's what I found"}, "logprobs": None}]},
{"choices": [{"delta": {"content": " about your query."}, "logprobs": None}]},
{"choices": [{"delta": {}, "finish_reason": "stop"}], "usage": {}},
])
def stream_stub(self, method, url, json=None, timeout=None):
return _MockStreamResponse(events)
monkeypatch.setattr(httpx.AsyncClient, "stream", stream_stub)
resp = client.post(
"/api/search",
json={"query": "current events", "model": config.DEFAULT_MODEL},
headers=headers,
)
assert resp.status_code == 200
payloads = parse_sse_payloads(resp.text)
assert any(p.get("searching") is True for p in payloads)
assert any("search_results" in p for p in payloads)
token_text = "".join(p.get("token", "") for p in payloads if "token" in p)
assert "found" in token_text.lower()
assert any(p.get("done") and p.get("searched") for p in payloads)
def test_explicit_search_no_results(tmp_path: Path, monkeypatch):
with make_client(tmp_path) as client:
headers = _guest_headers(client)
async def empty_search(query: str, max_results: int = 5):
return []
monkeypatch.setattr(routers.search_route, "query_searxng", empty_search)
resp = client.post(
"/api/search",
json={"query": "nothingness", "model": config.DEFAULT_MODEL},
headers=headers,
)
assert resp.status_code == 200
payloads = parse_sse_payloads(resp.text)
assert any("No search results found" in p.get("token", "") for p in payloads)
assert any(p.get("done") for p in payloads)
assert not any("search_results" in p for p in payloads)
def test_explicit_search_new_conversation_created(tmp_path: Path, monkeypatch):
with make_client(tmp_path) as client:
headers = _guest_headers(client)
async def search_stub(query: str, max_results: int = 5):
return [{"title": "T", "url": "https://ex.com", "content": "Content."}]
monkeypatch.setattr(routers.search_route, "query_searxng", search_stub)
events = _stream_json_lines([
{"choices": [{"delta": {"content": "Answer."}, "logprobs": None}]},
{"choices": [{"delta": {}, "finish_reason": "stop"}], "usage": {}},
])
def stream_stub(self, method, url, json=None, timeout=None):
return _MockStreamResponse(events)
monkeypatch.setattr(httpx.AsyncClient, "stream", stream_stub)
resp = client.post(
"/api/search",
json={"query": "tell me something", "model": config.DEFAULT_MODEL},
headers=headers,
)
assert resp.status_code == 200
payloads = parse_sse_payloads(resp.text)
conv_id = None
for p in payloads:
if "conversation_id" in p:
conv_id = p["conversation_id"]
break
assert conv_id is not None
conv_resp = client.get(f"/api/conversations/{conv_id}", headers=_guest_headers(client))
assert conv_resp.status_code == 200
data = conv_resp.json()
assert len(data["messages"]) >= 2
def test_explicit_search_stream_error(tmp_path: Path, monkeypatch):
with make_client(tmp_path) as client:
headers = _guest_headers(client)
async def search_stub(query: str, max_results: int = 5):
return [{"title": "T", "url": "https://ex.com", "content": "Content."}]
monkeypatch.setattr(routers.search_route, "query_searxng", search_stub)
def broken_stream(self, method, url, json=None, timeout=None):
class BrokenCtx:
async def __aenter__(self):
raise RuntimeError("summarization failed")
async def __aexit__(self, exc_type, exc, tb):
return False
return BrokenCtx()
monkeypatch.setattr(httpx.AsyncClient, "stream", broken_stream)
resp = client.post(
"/api/search",
json={"query": "breaking news", "model": config.DEFAULT_MODEL},
headers=headers,
)
assert resp.status_code == 200
assert "error_key" in resp.text
assert "INC-" in resp.text
tests/test_search_url_sanitization.py
+8 -8
View File
@@ -1,17 +1,17 @@
from search import sanitize_outbound_url import app as app_module
def test_sanitize_outbound_url_allows_http_https(): def test_sanitize_outbound_url_allows_http_https():
assert sanitize_outbound_url("https://example.com/path") == "https://example.com/path" assert app_module.sanitize_outbound_url("https://example.com/path") == "https://example.com/path"
assert sanitize_outbound_url("http://example.com") == "http://example.com" assert app_module.sanitize_outbound_url("http://example.com") == "http://example.com"
def test_sanitize_outbound_url_blocks_unsafe_schemes(): def test_sanitize_outbound_url_blocks_unsafe_schemes():
assert sanitize_outbound_url("javascript:alert(1)") == "" assert app_module.sanitize_outbound_url("javascript:alert(1)") == ""
assert sanitize_outbound_url("data:text/html,evil") == "" assert app_module.sanitize_outbound_url("data:text/html,evil") == ""
assert sanitize_outbound_url("file:///etc/passwd") == "" assert app_module.sanitize_outbound_url("file:///etc/passwd") == ""
def test_sanitize_outbound_url_blocks_relative_and_empty(): def test_sanitize_outbound_url_blocks_relative_and_empty():
assert sanitize_outbound_url("/relative/path") == "" assert app_module.sanitize_outbound_url("/relative/path") == ""
assert sanitize_outbound_url("") == "" assert app_module.sanitize_outbound_url("") == ""
tests/test_settings_allowlist.py
+7 -9
View File
@@ -3,19 +3,17 @@ from pathlib import Path
from fastapi.testclient import TestClient from fastapi.testclient import TestClient
import app import app as app_module
import db
from security import SESSIONS, PIN_ATTEMPTS
def make_admin_client(tmp_path: Path) -> tuple[TestClient, dict[str, str]]: def make_admin_client(tmp_path: Path) -> tuple[TestClient, dict[str, str]]:
os.environ["CAIC_ADMIN_PIN"] = "1234" os.environ["JARVISCHAT_ADMIN_PIN"] = "1234"
db.DB_PATH = tmp_path / "caic-settings.db" app_module.DB_PATH = tmp_path / "jarvischat-settings.db"
SESSIONS.clear() app_module.SESSIONS.clear()
PIN_ATTEMPTS.clear() app_module.PIN_ATTEMPTS.clear()
db.init_db() app_module.init_db()
client = TestClient(app.app) client = TestClient(app_module.app)
login = client.post( login = client.post(
"/api/auth/login", "/api/auth/login",
json={"pin": "1234"}, json={"pin": "1234"},
tests/test_skills_framework.py
+18 -22
View File
@@ -1,23 +1,19 @@
import asyncio
import os import os
from pathlib import Path from pathlib import Path
from fastapi.testclient import TestClient from fastapi.testclient import TestClient
import app import app as app_module
import db
from rag import build_system_prompt
from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS
def make_client(tmp_path: Path) -> TestClient: def make_client(tmp_path: Path) -> TestClient:
os.environ["CAIC_ADMIN_PIN"] = "1234" os.environ["JARVISCHAT_ADMIN_PIN"] = "1234"
db.DB_PATH = tmp_path / "caic-skills.db" app_module.DB_PATH = tmp_path / "jarvischat-skills.db"
SESSIONS.clear() app_module.SESSIONS.clear()
PIN_ATTEMPTS.clear() app_module.PIN_ATTEMPTS.clear()
RATE_EVENTS.clear() app_module.RATE_EVENTS.clear()
db.init_db() app_module.init_db()
return TestClient(app.app, raise_server_exceptions=False) return TestClient(app_module.app, raise_server_exceptions=False)
def test_guest_can_list_skills(tmp_path: Path): def test_guest_can_list_skills(tmp_path: Path):
@@ -25,7 +21,7 @@ def test_guest_can_list_skills(tmp_path: Path):
sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()[ sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()[
"session_id" "session_id"
] ]
resp = client.get("/api/skills", headers={"X-Session-ID": sid, "Origin": "http://testserver"}) resp = client.get("/api/skills", headers={"X-Session-ID": sid})
assert resp.status_code == 200 assert resp.status_code == 200
payload = resp.json() payload = resp.json()
assert payload["count"] >= 1 assert payload["count"] >= 1
@@ -50,7 +46,7 @@ def test_admin_can_toggle_skill_enabled_state(tmp_path: Path):
assert disable.status_code == 200 assert disable.status_code == 200
assert disable.json()["skill"]["enabled"] is False assert disable.json()["skill"]["enabled"] is False
active = client.get("/api/skills/active", headers={"X-Session-ID": sid, "Origin": "http://testserver"}) active = client.get("/api/skills/active", headers={"X-Session-ID": sid})
assert active.status_code == 200 assert active.status_code == 200
assert all(skill["key"] != "search.web" for skill in active.json()["skills"]) assert all(skill["key"] != "search.web" for skill in active.json()["skills"])
@@ -75,23 +71,23 @@ def test_unknown_skill_update_is_rejected(tmp_path: Path):
def test_prompt_injection_respects_skills_enabled_setting(tmp_path: Path): def test_prompt_injection_respects_skills_enabled_setting(tmp_path: Path):
with make_client(tmp_path): with make_client(tmp_path):
conn = db.get_db() db = app_module.get_db()
try: try:
conn.execute( db.execute(
"INSERT OR REPLACE INTO settings (key, value) VALUES (?, ?)", "INSERT OR REPLACE INTO settings (key, value) VALUES (?, ?)",
("skills_enabled", "false"), ("skills_enabled", "false"),
) )
conn.commit() db.commit()
without_skills = asyncio.run(build_system_prompt(conn, "", "hello")) without_skills = app_module.build_system_prompt(db, "", "hello")
assert "## Active Skills" not in without_skills assert "## Active Skills" not in without_skills
conn.execute( db.execute(
"INSERT OR REPLACE INTO settings (key, value) VALUES (?, ?)", "INSERT OR REPLACE INTO settings (key, value) VALUES (?, ?)",
("skills_enabled", "true"), ("skills_enabled", "true"),
) )
conn.commit() db.commit()
with_skills = asyncio.run(build_system_prompt(conn, "", "hello")) with_skills = app_module.build_system_prompt(db, "", "hello")
assert "## Active Skills" in with_skills assert "## Active Skills" in with_skills
assert "memory.search" in with_skills assert "memory.search" in with_skills
finally: finally:
conn.close() db.close()
tests/test_triage.py
-141
View File
@@ -1,141 +0,0 @@
"""Tests for triage.py — query classification and node selection."""
import asyncio
import json
import httpx
import cluster
import config
import triage
def _reset():
cluster.CLUSTER_NODES.clear()
cluster.CLUSTER_COORDINATOR = None
_published = []
async def _fake_publish(exchange, routing_key, payload):
_published.append((exchange, routing_key, payload))
class _MockPostResponse:
def __init__(self, json_data: dict, status_code: int = 200):
self._json_data = json_data
self.status_code = status_code
def json(self):
return self._json_data
async def __aenter__(self):
return self
async def __aexit__(self, exc_type, exc, tb):
return False
class _MockPostContext:
def __init__(self, response: _MockPostResponse):
self._response = response
async def __aenter__(self):
return self._response
async def __aexit__(self, exc_type, exc, tb):
return False
# ---------- 1. classify_query returns valid classification ----------
def test_classify_returns_valid(monkeypatch):
async def post_stub(self, url, json=None, timeout=None):
return _MockPostResponse({
"choices": [{"message": {"content": "code"}}]
})
monkeypatch.setattr(httpx.AsyncClient, "post", post_stub)
result = __import__("asyncio").run(triage.classify_query("write a python function"))
assert result == "code"
# ---------- 2. classify_query on error returns "general" ----------
def test_classify_error_returns_general(monkeypatch):
async def post_stub(self, url, json=None, timeout=None):
raise httpx.ConnectError("connection refused")
monkeypatch.setattr(httpx.AsyncClient, "post", post_stub)
result = __import__("asyncio").run(triage.classify_query("any question"))
assert result == "general"
# ---------- 3. select_node("code") returns coder node ----------
def test_select_node_code_returns_coder():
_reset()
cluster.CLUSTER_NODES["coder01"] = {
"name": "coder01", "type": "worker", "status": "active",
"ip": "192.168.50.210",
"active_model": {"name": "qwen2.5-coder-14b", "port": 8082},
}
cluster.CLUSTER_NODES["general01"] = {
"name": "general01", "type": "worker", "status": "active",
"ip": "192.168.50.211",
"active_model": {"name": "llama3.1", "port": 8081},
}
node = asyncio.run(triage.select_node("code"))
assert node is not None
assert node["name"] == "coder01"
# ---------- 4. select_node("general") with no matching node returns None ----------
def test_select_node_general_no_match_returns_none():
_reset()
cluster.CLUSTER_NODES["coder01"] = {
"name": "coder01", "type": "worker", "status": "active",
"active_model": {"name": "qwen2.5-coder-14b", "port": 8082},
}
node = asyncio.run(triage.select_node("general"))
assert node is None
# ---------- 5. get_inference_url with coder node ----------
def test_get_inference_url_with_coder_node(monkeypatch):
_reset()
async def fake_classify(query: str) -> str:
return "code"
monkeypatch.setattr(triage, "classify_query", fake_classify)
cluster.CLUSTER_NODES["coder01"] = {
"name": "coder01", "type": "worker", "status": "active",
"ip": "192.168.50.210",
"active_model": {"name": "qwen2.5-coder-14b", "port": 8082},
}
url = __import__("asyncio").run(triage.get_inference_url("write a loop in rust"))
assert url == "http://192.168.50.210:8082/v1"
# ---------- 6. get_inference_url with no nodes returns LLAMA_SERVER_BASE ----------
def test_get_inference_url_no_nodes(monkeypatch):
_reset()
async def fake_classify(query: str) -> str:
return "code"
monkeypatch.setattr(triage, "classify_query", fake_classify)
url = __import__("asyncio").run(triage.get_inference_url("any question"))
assert url == config.LLAMA_SERVER_BASE
tests/test_upload.py
-274
View File
@@ -1,274 +0,0 @@
import os
from pathlib import Path
import httpx
import pytest
from fastapi.testclient import TestClient
import app
import db
import routers.upload as upload_route
from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS
def make_client(tmp_path: Path) -> TestClient:
os.environ["CAIC_ADMIN_PIN"] = "1234"
db.DB_PATH = tmp_path / "caic-upload.db"
SESSIONS.clear()
PIN_ATTEMPTS.clear()
RATE_EVENTS.clear()
db.init_db()
return TestClient(app.app, raise_server_exceptions=False)
def _admin_headers(client: TestClient) -> dict:
login = client.post("/api/auth/login", json={"pin": "1234"}, headers={"Origin": "http://testserver"})
sid = login.json()["session_id"]
return {"X-Session-ID": sid, "Origin": "http://testserver"}
def _guest_headers(client: TestClient) -> dict:
sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()["session_id"]
return {"X-Session-ID": sid, "Origin": "http://testserver"}
def test_upload_requires_admin(tmp_path: Path):
with make_client(tmp_path) as client:
resp = client.post("/api/upload", headers=_guest_headers(client), files={"file": ("test.txt", b"hello")})
assert resp.status_code == 403
def test_upload_unsupported_mime(tmp_path: Path):
with make_client(tmp_path) as client:
resp = client.post(
"/api/upload", headers=_admin_headers(client),
files={"file": ("test.exe", b"fake", "application/x-msdownload")},
)
assert resp.status_code == 415
def test_upload_context_mode(tmp_path: Path):
with make_client(tmp_path) as client:
resp = client.post(
"/api/upload", headers=_admin_headers(client),
data={"mode": "context", "conversation_id": "conv-1"},
files={"file": ("notes.txt", b"Hello world notes")},
)
assert resp.status_code == 200
data = resp.json()
assert data["filename"] == "notes.txt"
assert data["mode"] == "context"
assert "context_id" in data
assert "chunks_ingested" not in data
row = db.get_db().execute("SELECT content FROM upload_context WHERE id = ?", (data["context_id"],)).fetchone()
assert row["content"] == "Hello world notes"
def test_upload_ingest_mode(tmp_path: Path, monkeypatch):
embed_count = 0
class FakeAsyncClient:
class FakeResponse:
def __init__(self, status, json_data=None):
self.status_code = status
self._json = json_data or {}
def json(self):
return self._json
def __init__(self, *a, **kw):
pass
async def __aenter__(self):
return self
async def __aexit__(self, *a):
pass
async def post(self, url, **kw):
nonlocal embed_count
if "/api/embeddings" in url:
embed_count += 1
return self.FakeResponse(200, {"embedding": [0.1] * 768})
return self.FakeResponse(200)
async def put(self, url, **kw):
return self.FakeResponse(200)
monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: FakeAsyncClient())
with make_client(tmp_path) as client:
resp = client.post(
"/api/upload", headers=_admin_headers(client),
data={"mode": "ingest"},
files={"file": ("data.txt", b"word " * 1000)},
)
assert resp.status_code == 200
data = resp.json()
assert data["mode"] == "ingest"
assert data["chunks_ingested"] > 0
assert "context_id" not in data
assert embed_count == data["chunks_ingested"]
def test_upload_both_mode(tmp_path: Path, monkeypatch):
class FakeAsyncClient:
class FakeResponse:
def __init__(self, status, json_data=None):
self.status_code = status
self._json = json_data or {}
def json(self):
return self._json
def __init__(self, *a, **kw):
pass
async def __aenter__(self):
return self
async def __aexit__(self, *a):
pass
async def post(self, url, **kw):
if "/api/embeddings" in url:
return self.FakeResponse(200, {"embedding": [0.1] * 768})
return self.FakeResponse(200)
async def put(self, url, **kw):
return self.FakeResponse(200)
monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: FakeAsyncClient())
with make_client(tmp_path) as client:
resp = client.post(
"/api/upload", headers=_admin_headers(client),
data={"mode": "both", "conversation_id": "conv-2"},
files={"file": ("both.txt", b"test " * 500)},
)
assert resp.status_code == 200
data = resp.json()
assert data["mode"] == "both"
assert "context_id" in data
assert data["chunks_ingested"] > 0
def test_upload_image_type(tmp_path: Path):
with make_client(tmp_path) as client:
resp = client.post(
"/api/upload", headers=_admin_headers(client),
data={"mode": "context"},
files={"file": ("photo.png", b"fake-png", "image/png")},
)
assert resp.status_code == 200
data = resp.json()
assert data["filename"] == "photo.png"
assert "context_id" in data
def test_get_upload_by_conversation(tmp_path: Path):
with make_client(tmp_path) as client:
headers = _admin_headers(client)
resp1 = client.post("/api/upload", headers=headers,
data={"mode": "context", "conversation_id": "conv-gal"},
files={"file": ("a.txt", b"alpha")})
cid1 = resp1.json()["context_id"]
resp2 = client.post("/api/upload", headers=headers,
data={"mode": "context", "conversation_id": "conv-gal"},
files={"file": ("b.txt", b"beta")})
cid2 = resp2.json()["context_id"]
gal = client.get("/api/upload/by-conversation/conv-gal", headers=headers)
assert gal.status_code == 200
items = gal.json()
assert len(items) == 2
assert items[0]["filename"] == "a.txt"
assert items[1]["filename"] == "b.txt"
def test_link_upload_to_conversation(tmp_path: Path):
with make_client(tmp_path) as client:
headers = _admin_headers(client)
resp = client.post("/api/upload", headers=headers,
data={"mode": "context"},
files={"file": ("orphan.txt", b"lonely")})
cid = resp.json()["context_id"]
link = client.patch(f"/api/upload/{cid}/link", headers=headers,
json={"conversation_id": "new-conv"})
assert link.status_code == 200
gal = client.get("/api/upload/by-conversation/new-conv", headers=headers)
assert len(gal.json()) == 1
def test_delete_upload_removes_context(tmp_path: Path, monkeypatch):
class FakeAsyncClient:
class FakeResponse:
def __init__(self, status, json_data=None):
self.status_code = status
self._json = json_data or {}
def json(self):
return self._json
def __init__(self, *a, **kw):
pass
async def __aenter__(self):
return self
async def __aexit__(self, *a):
pass
async def post(self, url, **kw):
if "/points/scroll" in url:
return self.FakeResponse(200, {"result": {"points": [{"id": "upload-test.txt-0"}, {"id": "upload-test.txt-1"}]}})
if "/points/delete" in url:
return self.FakeResponse(200)
return self.FakeResponse(200)
async def put(self, url, **kw):
return self.FakeResponse(200)
monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: FakeAsyncClient())
with make_client(tmp_path) as client:
headers = _admin_headers(client)
resp = client.post("/api/upload", headers=headers,
data={"mode": "context", "conversation_id": "del-test"},
files={"file": ("test.txt", b"delete me")})
cid = resp.json()["context_id"]
del_resp = client.delete(f"/api/upload/{cid}", headers=headers)
assert del_resp.status_code == 200
assert del_resp.json()["status"] == "ok"
row = db.get_db().execute("SELECT id FROM upload_context WHERE id = ?", (cid,)).fetchone()
assert row is None
def test_delete_upload_not_found(tmp_path: Path):
with make_client(tmp_path) as client:
resp = client.delete("/api/upload/999", headers=_admin_headers(client))
assert resp.status_code == 404
def test_conversation_list_includes_attachment_count(tmp_path: Path):
with make_client(tmp_path) as client:
headers = _admin_headers(client)
client.post("/api/conversations", headers=headers, json={"title": "NoAttach"})
with_attach = client.post("/api/conversations", headers=headers, json={"title": "WithAttach"}).json()
conv_id = with_attach["id"]
client.post("/api/upload", headers=headers,
data={"mode": "context", "conversation_id": conv_id},
files={"file": ("f.txt", b"data")})
list_resp = client.get("/api/conversations", headers=headers)
convs = list_resp.json()
for c in convs:
if c["title"] == "NoAttach":
assert c["attachment_count"] == 0
if c["title"] == "WithAttach":
assert c["attachment_count"] == 1
triage.py
-98
View File
@@ -1,98 +0,0 @@
"""cAIc — Query triage and cluster node selection."""
import logging
import httpx
from config import TRIAGE_BASE, TRIAGE_TIMEOUT, LLAMA_SERVER_BASE
log = logging.getLogger("caic")
_IDEAL_MODEL_MAP = {
"code": {"name_contains": ["coder", "qwen"]},
"general": {"name_contains": ["mistral", "llama"]},
}
_CLASSIFICATION_PROMPT = """Classify the following user query into exactly one category. Respond with only the category name.
Categories:
- general: everyday questions, chitchat, creative writing, advice, explanations
- code: programming, debugging, code generation, technical questions about software
- search: questions about current events, real-time information, weather, news, specific things that may have changed since training
- rag: questions about specific documents, personal data, notes, memory, uploaded content
Query: {query}
Category:"""
async def classify_query(query: str) -> str:
try:
async with httpx.AsyncClient() as client:
resp = await client.post(
f"{TRIAGE_BASE}/chat/completions",
json={
"model": "phi-4-mini",
"messages": [
{"role": "system", "content": "You are a query classifier. Respond with exactly one word."},
{"role": "user", "content": _CLASSIFICATION_PROMPT.format(query=query)},
],
"temperature": 0.0,
"max_tokens": 10,
},
timeout=TRIAGE_TIMEOUT,
)
text = resp.json()["choices"][0]["message"]["content"].strip().lower()
valid = {"general", "code", "search", "rag"}
for v in valid:
if v in text:
return v
except Exception:
log.warning("triage classify_query failed, falling back to general", exc_info=True)
return "general"
async def select_node(classification: str) -> dict | None:
from cluster import CLUSTER_NODES
if classification in ("search", "rag"):
return None
ideal = _IDEAL_MODEL_MAP.get(classification, {})
ideal_contains = ideal.get("name_contains", [])
# First pass: find an active node with the right model already loaded
for node in CLUSTER_NODES.values():
if node.get("status") != "active":
continue
am = node.get("active_model") or {}
name = (am.get("name") or "").lower()
if any(ideal in name for ideal in ideal_contains):
return node
# Second pass: find an active node that can swap to the right model
for node in CLUSTER_NODES.values():
if node.get("status") != "active":
continue
inventory = node.get("inventory") or []
for inv in inventory:
inv_name = (inv.get("name") or "").lower()
if any(ideal in inv_name for ideal in ideal_contains):
from cluster import request_model_swap
await request_model_swap(node["name"], inv["filename"])
return None
return None
async def get_inference_url(query: str) -> str:
if not query:
return LLAMA_SERVER_BASE
classification = await classify_query(query)
if classification in ("search", "rag"):
return LLAMA_SERVER_BASE
node = await select_node(classification)
if node:
am = node.get("active_model") or {}
port = am.get("port", 8081)
ip = node.get("ip") or "127.0.0.1"
return f"http://{ip}:{port}/v1"
return LLAMA_SERVER_BASE