Compare commits
2 Commits
df405a156e
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
| e14ae2bd19 | |||
| 44387919a8 |
+1
-1
@@ -55,5 +55,5 @@ LLAMA_EXPOSE_PORT=8081
|
||||
OLLAMA_EXPOSE_PORT=11434
|
||||
|
||||
# ── Image generation (ComfyUI on worker) ─────────────────────
|
||||
CAIC_COMFYUI_BASE=http://192.168.50.115:8188
|
||||
CAIC_COMFYUI_BASE=http://localhost:8188
|
||||
CAIC_COMFYUI_TIMEOUT=120
|
||||
|
||||
@@ -6,20 +6,20 @@ Previous task history archived at `docs/archive/TASKS-pre-1.0.md`.
|
||||
|
||||
## TASK 1 — Image Generation Service (corsair)
|
||||
|
||||
**Goal:** Add image generation as a cluster capability. corsair (RTX 5070 Ti, 16 GB) registers as an image gen worker in the cAIc cluster.
|
||||
**Goal:** Add image generation as a cluster capability. The image-gen node (currently jarvis — single-node deployment) registers as an image gen worker in the cAIc cluster.
|
||||
|
||||
### Requirements:
|
||||
|
||||
1. **Add `"image_gen"` capability** to the cluster protocol in `cluster.py` — valid capability values should include `image_gen`
|
||||
2. **Image gen API wrapper on corsair** — run ComfyUI, Automatic1111, or a lightweight API (e.g., `sd-api` or `comfyui-api`) that exposes a simple `POST /generate` endpoint accepting a prompt and returning a PNG
|
||||
3. **Proxy endpoint in cAIc** — `POST /api/image/generate` on the coordinator, routes the request to corsair's image gen service via AMQP or direct HTTP
|
||||
2. **Image gen API wrapper** — run ComfyUI, Automatic1111, or a lightweight API (e.g., `sd-api` or `comfyui-api`) that exposes a simple `POST /generate` endpoint accepting a prompt and returning a PNG
|
||||
3. **Proxy endpoint in cAIc** — `POST /api/image/generate` on the coordinator, routes the request to the image gen service via AMQP or direct HTTP
|
||||
4. **Update `hardware.py`** to probe the image gen service for reachability and status
|
||||
5. **Update node_agent** to report image gen capability and service status on registration
|
||||
|
||||
### Architecture:
|
||||
|
||||
```
|
||||
User prompt → cAIc coordinator → AMQP/HTTP → corsair (ComfyUI/API) → PNG → coordinator → user
|
||||
User prompt → cAIc coordinator → AMQP/HTTP → image-gen node (ComfyUI/API) → PNG → coordinator → user
|
||||
```
|
||||
|
||||
### Considerations:
|
||||
@@ -37,7 +37,7 @@ User prompt → cAIc coordinator → AMQP/HTTP → corsair (ComfyUI/API) → PNG
|
||||
- Verify node_agent registers with `image_gen` capability
|
||||
- Verify 429/503 handling when service is busy or down
|
||||
|
||||
### Status: ✅ Backend Complete (ComfyUI install pending on corsair)
|
||||
### Status: ✅ Backend Complete (ComfyUI install pending on jarvis — single-node deployment)
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -35,15 +35,15 @@ Every router has a dedicated test file:
|
||||
| `test_search_url_sanitization.py` | `search.py` URL sanitizer |
|
||||
| `test_cluster.py` | `cluster.py` — registration, deregistration, pong, events, coordinator query |
|
||||
| `test_cluster_heartbeat.py` | `cluster.py` — heartbeat handler, known/unknown node |
|
||||
| `test_model_swap.py` | `cluster.py` + `triage.py` — request_model_swap, handle_model_ready/failed, select_node swap triggering |
|
||||
| `test_model_swap.py` | `cluster.py` — request_model_swap, handle_model_ready/failed |
|
||||
| `test_node_agent.py` | `node_agent/agent.py` — registration, ping/pong, model swap |
|
||||
| `test_image.py` | Image generation — cluster handlers, router proxy, node agent ComfyUI integration, hardware probe, capability detection |
|
||||
| `test_triage.py` | `triage.py` — classify_query, select_node, get_inference_url |
|
||||
| `test_settings_allowlist.py` | `routers/settings.py` — allowlisted key enforcement |
|
||||
| `test_skills_framework.py` | `routers/skills.py` — list, toggle, unknown skill, prompt injection |
|
||||
| `test_ip_allowlist.py` | IP allowlist helper + middleware |
|
||||
| `test_rate_and_payload_guardrails.py` | Rate limits + payload size enforcement |
|
||||
| `test_error_envelopes.py` | Global exception handler + stream error incidents |
|
||||
| `test_fixes_regression.py` | Origin-exempt ingest, bogus conversation_id FK, auto-search reset, image uploads, conflict false-positives, deterministic ingest ids, get_load VRAM parsing, version pin |
|
||||
| `test_upload.py` | `routers/upload.py` — upload, delete, link, by-conversation, attachment_count integration |
|
||||
|
||||
Modules that call `httpx.AsyncClient` (chat, completions, models, search_route, upload, ingest, model_pull)
|
||||
@@ -69,7 +69,6 @@ Refactored from single-file (`app.py`) into modules under project root:
|
||||
| `gpu.py` | GPU stats — `rocm-smi` (AMD/Linux) or `system_profiler` (Apple Silicon/macOS) |
|
||||
| `crypto.py` | AES-256-GCM encrypt/decrypt + key management (stored as `heartbeat_interval_ms` in settings) |
|
||||
| `model_pull.py` | Startup model availability check + Ollama pull API |
|
||||
| `triage.py` | Phi-4-mini-based query classification + cluster node selection |
|
||||
| `cluster.py` | Cluster node registry, event log, coordinator election, ping/pong, model swap handlers, image generation request/response |
|
||||
| `amqp.py` | AMQP connection manager — connect, disconnect, publish, subscribe, auto-reconnect |
|
||||
| `node_agent/` | Standalone worker agent — AMQP client for registration, ping/pong, model swap, image generation |
|
||||
@@ -84,12 +83,12 @@ Refactored from single-file (`app.py`) into modules under project root:
|
||||
|
||||
### Key flows
|
||||
|
||||
1. **`/api/chat`** → `process_remember_command()` intercepts "remember that..." / "forget about..." first → optional `upload_context_id` fetches document text from SQLite → `build_system_prompt()` (profile + FTS5 memory + Qdrant RAG + preset + skills + uploaded doc) → triage classifies query (general/code/search/rag) → `select_node()` picks best worker → stream from chosen node with `logprobs: true` → if perplexity > 15.0 OR `REFUSAL_PATTERNS` match, re-query with SearXNG results
|
||||
1. **`/api/chat`** → `process_remember_command()` intercepts "remember that..." / "forget about..." first → optional `upload_context_id` fetches document text from SQLite → `build_system_prompt()` (profile + FTS5 memory + Qdrant RAG + preset + skills + uploaded doc) → stream from `LLAMA_SERVER_BASE` with `logprobs: true` → if perplexity > 15.0 OR `REFUSAL_PATTERNS` match, re-query with SearXNG results
|
||||
2. **`/api/search`** → bypasses perplexity/refusal, queries SearXNG directly → summarizes via llama-server
|
||||
3. **`/v1/chat/completions`** → OpenAI-compatible for Continue.dev/IDE integration; FIM requests proxied without persistence
|
||||
4. **`/api/upload`** → multipart file upload, PDF/text extraction, `mode=(context|ingest|both)`, stores SQLite context (1hr expiry) + Qdrant upsert
|
||||
5. **`/api/ingest`** → Bearer token auth, programmatic RAG ingest (terminal hook, external tools)
|
||||
6. **`POST /api/image/generate`** → admin required, routes to corsair node agent via AMQP → ComfyUI workflow → returns PNG; `GET /api/image/status` lists available image gen nodes
|
||||
6. **`POST /api/image/generate`** → admin required, routes to an image-gen node via AMQP → ComfyUI workflow → returns PNG; `GET /api/image/status` lists available image gen nodes
|
||||
|
||||
### Perplexity / auto-search
|
||||
|
||||
@@ -118,7 +117,6 @@ All services are available bare-metal or as containers in `docker compose up`.
|
||||
| Service | Required | Port | Docker service name |
|
||||
|---------|----------|------|---------------------|
|
||||
| llama-server (coordinator) | Yes | 8081 + RPC :50052 (worker GPU) | `llama-server` |
|
||||
| Phi-4-mini (triage) | No | 8083 | — |
|
||||
| SearXNG | No | 8888 | `searxng` |
|
||||
| RabbitMQ (coordinator) | No | 5672 — AMQP broker | `rabbitmq` |
|
||||
| wttr.in | No | weather shortcut | — |
|
||||
@@ -133,7 +131,7 @@ All services are available bare-metal or as containers in `docker compose up`.
|
||||
- `SUPPORTED_UPLOAD_TYPES` includes images (png/jpeg/gif/svg/webp) + text + PDF + JSON
|
||||
- `UPLOAD_CONTEXT_EXPIRY_HOURS` = 1 hour
|
||||
- Rate limits and payload caps in `config.py` — patch `security.RL_*` not `config.RL_*` for tests
|
||||
- `COMFYUI_BASE` defaults to `http://192.168.50.115:8188` (overridable via `CAIC_COMFYUI_BASE`)
|
||||
- `COMFYUI_BASE` defaults to `http://localhost:8188` (overridable via `CAIC_COMFYUI_BASE`)
|
||||
- `COMFYUI_TIMEOUT` defaults to `120` seconds (overridable via `CAIC_COMFYUI_TIMEOUT`)
|
||||
- RAG embedding requests go to `EMBED_URL` at `/api/embeddings` (Ollama on worker :11434)
|
||||
|
||||
@@ -150,6 +148,8 @@ All streaming endpoints yield `data: {json}\n\n`. Key shapes:
|
||||
|
||||
### Completed this session
|
||||
- **Pre-Docker review**: Full findings report delivered -- 30+ issues across 7 categories (hardcoded hosts/paths, config/secrets, AMQP gaps, resource cleanup, SQLite container safety, completions concurrency, TASKS.md accuracy).
|
||||
- **Project rename**: `jarvisChat` → **cAIc** ("cake") — swept remaining branding (router docstrings, jc-ingest.sh env var), deleted stale `AGENTS.md.local`.
|
||||
- **Single-node consolidation**: all services moved to jarvis (192.168.50.212) — `COMFYUI_BASE` default → `localhost:8188`, AMQP URL default → `localhost:5672`, `NODE_NAME` default → `jarvis`, `DEFAULT_PROFILE` topology rewritten, cluster/AMQP/node_agent left in place (degrades gracefully).
|
||||
- **Deprecation fix**: Replaced `asyncio.ensure_future` with `asyncio.create_task` in `rag.py` and `routers/chat.py`.
|
||||
- **Documentation**: Added inline comments and docstrings to all functions in `db.py`.
|
||||
- **Uninstall scripts**: Created and committed `scripts/uninstall.sh`, `teardown-docker.sh`, `nuclear-clean.sh`.
|
||||
@@ -157,7 +157,22 @@ All streaming endpoints yield `data: {json}\n\n`. Key shapes:
|
||||
- **Docker containerization (B3)**: Created `Dockerfile`, `docker-compose.yml`, `.env.example`, `scripts/setup.sh`, `.dockerignore`, `searxng-settings.yml.dist`, `models/README.txt`. Fixed hardcoded defaults in `config.py` (localhost, Docker secrets path, `CAIC_DEFAULT_MODEL` env var, `CAIC_HW_STATE_PATH` env var). Added missing `psutil` + `jinja2` to `requirements.txt`. Fixed test discovery via `tests/conftest.py` sys.path insertion. 214 tests pass.
|
||||
|
||||
### Active
|
||||
- Image generation service backend complete — cluster handlers, router proxy, node agent ComfyUI integration, hardware probe. 27 tests pass. ComfyUI install pending on corsair.
|
||||
- Image generation service backend complete — cluster handlers, router proxy, node agent ComfyUI integration, hardware probe. 27 tests pass. ComfyUI install pending on jarvis (single-node).
|
||||
|
||||
### Deployed (2026-08-07) — v1.1.0 to production
|
||||
- **Fixed crash-loop**: ultron `llama-server.service` had 911 restarts — its `--rpc 192.168.50.210:50052` pointed at a dead IP. Corrected to `192.168.50.212:50052` (jarvis GPU rpc-server). Model now loads, `/health` = ok.
|
||||
- **Deployed v1.1.0**: workspace repo synced to `/opt/jarvischat` (jarvischat.service cwd). caic.db + venv preserved; `aio-pika` installed into prod venv (was missing → AMQP disabled).
|
||||
- **Env fixes** (`/etc/systemd/system/jarvischat.service.d/override.conf`): added `LLAMA_SERVER_BASE=http://192.168.50.108:8081`, `CAIC_QDRANT_URL=http://192.168.50.108:6333`, `CAIC_COMPLETIONS_API_KEY` (was set as legacy `JARVISCHAT_` name), kept `CAIC_EMBED_URL=http://192.168.50.108:11434` + `CAIC_ADMIN_PIN=1319`. Wrote `/opt/jarvischat/.completions_key` (jc-ingest.sh).
|
||||
- **Deploy-blocking bug fixes** (uncommitted, workspace + deploy):
|
||||
- `rag.py` `chunk_text`: chunk_size 512→200 (chunks exceeded mxbai-embed-large's 512-token context → ollama 500).
|
||||
- Qdrant 1.18.2 rejects non-UUID point IDs: wrapped `ingest-*`/`auto-*`/`upload-*` string IDs in `uuid5` in `routers/ingest.py`, `rag.py`, `routers/upload.py`.
|
||||
- `docs/jc-ingest.sh`: `JC_URL` updated `.210`→`.212`.
|
||||
- **Docs rebuilt**: 159 chunks (source `docs`) re-ingested via `/api/ingest` (README, ai.md, docker.md, CLAUDE.md, wiki/*). RAG now 378 vectors; chat verified injecting "Retrieved Context".
|
||||
- **Tests**: all 244 pass (run per-file in a throwaway venv; the full-suite run deadlocks on TestClient/AMQP ordering, not a code failure).
|
||||
|
||||
### Follow-ups
|
||||
- AMQP wiring: cluster subs degrade gracefully — **moot in the single-node (jarvis) deployment** until a multi-node cluster is stood back up. Needs `CAIC_AMQP_URL` + credentials if that happens.
|
||||
- `CAIC_TRIAGE_BASE` set but triage not yet invoked by chat (config-only until TASK 2 wiring).
|
||||
|
||||
### Blocked
|
||||
- Ball Gunner assets — waiting on Canva designs
|
||||
|
||||
@@ -22,7 +22,7 @@ from hardware import assess_hardware
|
||||
from memory import get_memory_count
|
||||
from security import (
|
||||
get_client_ip, is_ip_allowed, check_rate_limit, rate_policy,
|
||||
origin_allowed, is_state_changing, request_body_limit,
|
||||
origin_allowed, request_body_limit,
|
||||
audit_event, customer_error_envelope, log_incident,
|
||||
)
|
||||
from auth import get_session, is_admin_only, router as auth_router
|
||||
@@ -141,7 +141,12 @@ async def session_auth_middleware(request: Request, call_next):
|
||||
"/api/auth/heartbeat", "/api/auth/guest", "/api/ingest", "/api/hardware",
|
||||
}
|
||||
|
||||
if path.startswith("/api/"):
|
||||
# Bearer-token-authenticated endpoints are reached by CLI/terminal tooling
|
||||
# (curl, caic-ingest.sh) that sends no Origin/Referer header — exempt them
|
||||
# from the browser origin check.
|
||||
origin_exempt_paths = {"/api/ingest"}
|
||||
|
||||
if path.startswith("/api/") and path not in origin_exempt_paths:
|
||||
if not origin_allowed(request):
|
||||
audit_event("origin_check", "denied", ip=ip, role="none",
|
||||
details=f"{request.method} {path}", warning=True)
|
||||
|
||||
-2334
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -17,7 +17,7 @@ from db import get_db, get_setting
|
||||
from security import (
|
||||
SESSIONS, PIN_ATTEMPTS, SESSION_LOCK, BODY_LIMIT_DEFAULT_BYTES,
|
||||
audit_event, get_client_ip, is_ip_allowed, check_rate_limit,
|
||||
rate_policy, origin_allowed, is_state_changing, request_body_limit,
|
||||
rate_policy, origin_allowed, request_body_limit,
|
||||
read_json_body, hash_pin, customer_error_envelope, log_incident,
|
||||
)
|
||||
|
||||
|
||||
+1
-1
@@ -19,7 +19,7 @@ CLUSTER_EVENTS: deque = deque(maxlen=1000)
|
||||
CLUSTER_COORDINATOR: str | None = None
|
||||
_pending_pings: dict[str, tuple[str, asyncio.Event]] = {}
|
||||
_pending_image: dict[str, tuple[str, asyncio.Event]] = {}
|
||||
NODE_NAME: str = os.environ.get("CAIC_NODE_NAME", "ultron")
|
||||
NODE_NAME: str = os.environ.get("CAIC_NODE_NAME", "jarvis")
|
||||
PING_TIMEOUT: float = 5.0
|
||||
|
||||
|
||||
|
||||
@@ -34,7 +34,7 @@ def get_amqp_url() -> str:
|
||||
except (FileNotFoundError, OSError):
|
||||
pw = "password"
|
||||
log.warning("AMQP secret file not found at %s — using default password", AMQP_SECRET_PATH)
|
||||
return f"amqp://caic:{pw}@rabbitmq:5672/caic"
|
||||
return f"amqp://caic:{pw}@localhost:5672/caic"
|
||||
|
||||
# --- Auth ---
|
||||
SESSION_TIMEOUT_SECONDS = 3600
|
||||
@@ -53,7 +53,7 @@ TRUST_X_FORWARDED_FOR = (
|
||||
)
|
||||
|
||||
# --- Image generation (ComfyUI) ---
|
||||
COMFYUI_BASE = os.environ.get("CAIC_COMFYUI_BASE", "http://192.168.50.115:8188")
|
||||
COMFYUI_BASE = os.environ.get("CAIC_COMFYUI_BASE", "http://localhost:8188")
|
||||
COMFYUI_TIMEOUT = int(os.environ.get("CAIC_COMFYUI_TIMEOUT", "120"))
|
||||
|
||||
# --- Rate limits ---
|
||||
@@ -179,12 +179,10 @@ ALLOWED_NETWORKS = parse_allowed_cidrs(ALLOWED_CIDRS_RAW)
|
||||
DEFAULT_PROFILE = """You are a coding companion running locally on a machine called "jarvis".
|
||||
|
||||
## Environment
|
||||
- jarvis: Debian 13 (trixie) x86_64, AMD Ryzen 5 5600X, 16GB RAM, AMD RX 6600 XT (8GB VRAM)
|
||||
- ultron: Debian 13, Ryzen 7 7840HS, 16GB RAM, primary AI inference node, IP 192.168.50.108
|
||||
- Corsair: Windows 11, gaming/streaming rig, RTX 5070 Ti
|
||||
- jarvis: Debian 13 (trixie) x86_64, AMD Ryzen 5 5600X, 16GB RAM, AMD RX 6600 XT (8GB VRAM), IP 192.168.50.212
|
||||
- Single-node deployment — all cAIc services run on jarvis: llama-server :8081 (OpenAI-compat API), Qdrant :6333, Ollama :11434, SearXNG :8888, RabbitMQ :5672, ComfyUI :8188
|
||||
- pivault: RPi 5, 8GB RAM, Debian 13, 11TB RAID5 NAS at /mnt/pivault, IP 192.168.50.158
|
||||
- Router: ASUS ROG Rapture GT-BE98 Pro "BigBlinkyRouter" at 192.168.50.1
|
||||
- llama-server on ultron:8081 (OpenAI-compat API), Qdrant on ultron:6333
|
||||
|
||||
## About the User
|
||||
- Experienced developer, BS in Computer Science (Oklahoma State), coding since 1981 (TRS-80)
|
||||
|
||||
@@ -30,6 +30,7 @@ def get_db():
|
||||
conn.row_factory = sqlite3.Row
|
||||
conn.execute("PRAGMA foreign_keys = ON")
|
||||
conn.execute("PRAGMA journal_mode = WAL")
|
||||
conn.execute("PRAGMA busy_timeout = 5000")
|
||||
return conn
|
||||
|
||||
|
||||
@@ -132,6 +133,8 @@ def init_db():
|
||||
from security import hash_pin
|
||||
conn = sqlite3.connect(DB_PATH)
|
||||
conn.row_factory = sqlite3.Row
|
||||
conn.execute("PRAGMA journal_mode = WAL")
|
||||
conn.execute("PRAGMA busy_timeout = 5000")
|
||||
|
||||
# --- Core tables ---
|
||||
conn.execute("""
|
||||
|
||||
+1
-1
@@ -19,7 +19,7 @@ services:
|
||||
- rabbitmq_password
|
||||
environment:
|
||||
- CAIC_AMQP_SECRET_PATH=/run/secrets/rabbitmq_password
|
||||
- CAIC_COMFYUI_BASE=${CAIC_COMFYUI_BASE:-http://192.168.50.115:8188}
|
||||
- CAIC_COMFYUI_BASE=${CAIC_COMFYUI_BASE:-http://localhost:8188}
|
||||
- CAIC_COMFYUI_TIMEOUT=${CAIC_COMFYUI_TIMEOUT:-120}
|
||||
env_file: .env
|
||||
depends_on:
|
||||
|
||||
+5
-5
@@ -1,11 +1,11 @@
|
||||
#!/bin/bash
|
||||
# jc-ingest.sh — pipe terminal commands into jarvisChat RAG
|
||||
# Deploy to /home/gramps/bin/jc-ingest.sh on jarvis (192.168.50.210)
|
||||
# jc-ingest.sh — pipe terminal commands into cAIc RAG
|
||||
# Deploy to /home/gramps/bin/jc-ingest.sh on jarvis (192.168.50.212)
|
||||
#
|
||||
# Usage:
|
||||
# 1. chmod +x /home/gramps/bin/jc-ingest.sh
|
||||
# 2. Add to ~/.bashrc:
|
||||
# export JARVISCHAT_COMPLETIONS_API_KEY="$(cat /opt/jarvischat/.completions_key)"
|
||||
# export CAIC_COMPLETIONS_API_KEY="$(cat /opt/jarvischat/.completions_key)"
|
||||
# export PROMPT_COMMAND="jc_capture"
|
||||
# source /home/gramps/bin/jc-ingest.sh
|
||||
#
|
||||
@@ -15,8 +15,8 @@
|
||||
# Filter: currently captures git, pip, systemctl, sudo, vi/vim, curl,
|
||||
# wget, apt, python, pytest commands. Edit the grep pattern to adjust.
|
||||
|
||||
JC_URL="http://192.168.50.210:8080/api/ingest"
|
||||
JC_TOKEN="${JARVISCHAT_COMPLETIONS_API_KEY}"
|
||||
JC_URL="http://192.168.50.212:8080/api/ingest"
|
||||
JC_TOKEN="${CAIC_COMPLETIONS_API_KEY}"
|
||||
|
||||
jc_capture() {
|
||||
local cmd
|
||||
|
||||
@@ -25,7 +25,6 @@ Refactored from single-file (`app.py`) into modules under project root:
|
||||
| `hardware.py` | Hardware self-assessment — CPU, RAM, VRAM, service health probes (llama-server, Qdrant, SearXNG, ComfyUI) |
|
||||
| `amqp.py` | aio-pika connection manager for RabbitMQ (connect, disconnect, publish, subscribe, auto-reconnect) |
|
||||
| `cluster.py` | Cluster node registry, event log, coordinator election, ping/pong, model swap handlers, image generation request/response |
|
||||
| `triage.py` | Phi-4-mini query classification + `select_node()` for cluster routing |
|
||||
| `routers/` | One module per endpoint group |
|
||||
|
||||
### 1.2 External Services
|
||||
@@ -46,14 +45,16 @@ Key base URLs are configured via environment variables with sensible defaults:
|
||||
|
||||
| Variable | Default | Service |
|
||||
|----------|---------|---------|
|
||||
| `LLAMA_SERVER_BASE` | `http://192.168.50.108:8081` | llama-server on coordinator |
|
||||
| `LLAMA_SERVER_BASE` | `http://localhost:8081` | llama-server on the same node |
|
||||
| `OLLAMA_BASE` | `http://localhost:11434` | Legacy — all inference goes through LLAMA_SERVER_BASE |
|
||||
| `SEARXNG_BASE` | `http://localhost:8888` | SearXNG |
|
||||
| `QDRANT_URL` | `http://192.168.50.108:6333` | Qdrant on coordinator |
|
||||
| `QDRANT_URL` | `http://localhost:6333` | Qdrant on the same node |
|
||||
| `CAIC_AMQP_URL` | `amqp://caic:password@localhost:5672/caic` | RabbitMQ |
|
||||
| `CAIC_COMFYUI_BASE` | `http://192.168.50.115:8188` | ComfyUI on worker |
|
||||
| `CAIC_COMFYUI_BASE` | `http://localhost:8188` | ComfyUI (image gen) |
|
||||
| `CAIC_COMFYUI_TIMEOUT` | `120` | ComfyUI generation timeout (seconds) |
|
||||
|
||||
> **Current deployment (single-node):** all services run on jarvis (192.168.50.212). The cluster/AMQP/node-agent layer is dormant — it degrades gracefully and can be re-enabled for a multi-node cluster later.
|
||||
|
||||
## 2. Request/Response Architecture
|
||||
|
||||
### 2.1 Chat Pipeline (`/api/chat`)
|
||||
@@ -308,7 +309,6 @@ All streaming endpoints yield `data: {json}\n\n`:
|
||||
| test_search_url_sanitization.py | URL sanitizer |
|
||||
| test_settings_allowlist.py | Allowlisted key enforcement |
|
||||
| test_skills_framework.py | List, toggle, unknown skill, prompt injection |
|
||||
| test_triage.py | classify_query, select_node, get_inference_url |
|
||||
| test_upload.py | Upload, delete, link, by-conversation, attachment_count |
|
||||
|
||||
### 8.3 DoD Process
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# WireGuard Tunnel — Encrypted Node Transit
|
||||
|
||||
> **Status: dormant (single-node deployment).** All cAIc services currently run on one node (jarvis, 192.168.50.212), so there is no inter-node traffic to encrypt. This document is kept as a reference for when a multi-node cluster is stood back up.
|
||||
|
||||
## Why
|
||||
|
||||
cAIc cluster traffic is plaintext today:
|
||||
|
||||
@@ -6,10 +6,12 @@ Scope: Active roadmap items and backlog.
|
||||
|
||||
## In Progress
|
||||
|
||||
- **Image Generation Service** — Backend wired: cluster handlers, `POST /api/image/generate` proxy, node agent ComfyUI integration, hardware probe, 27 tests. ComfyUI install pending on corsair (RTX 5070 Ti).
|
||||
- **Image Generation Service** — Backend wired: cluster handlers, `POST /api/image/generate` proxy, node agent ComfyUI integration, hardware probe, 27 tests. ComfyUI install pending on jarvis (single-node).
|
||||
|
||||
## Completed
|
||||
|
||||
- **Single-node consolidation (2026-08-08)** — all cAIc services moved onto jarvis (192.168.50.212): llama-server, Qdrant, SearXNG, RabbitMQ, Ollama, ComfyUI. Config defaults (`COMFYUI_BASE`, AMQP URL, `NODE_NAME`) updated; cluster/AMQP layer left dormant (degrades gracefully). Project renamed `jarvisChat` → **cAIc**.
|
||||
|
||||
- **B8 (v0.19.3)** — Private Chat mode. Backend skip-DB/skip-RAG/skip-search flag, frontend PRIVATE badge, info popup.
|
||||
- **WireGuard TLS (v0.19.4)** — Self-signed WireGuard mesh encrypts all inter-node traffic (AMQP, inference, RPC). No code changes to cAIc. Documented in wiki/WireGuard-Setup.md + docker.md §5.4.
|
||||
- **At-Rest Encryption (v0.20.0)** — AES-256-GCM encrypts all query-derived text at rest. crypto.py with auto-keygen, key stored as `heartbeat_interval_ms` in settings. All 12 storage paths wired (SQLite: messages, conversations, memories, upload_context; Qdrant: RAG chunks, ingest, upload). 200 tests pass.
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -38,6 +38,21 @@ AUTO_FACT_PATTERNS = [
|
||||
]
|
||||
SOCIAL_TRIGGERS = {"hi", "hello", "hey", "yo", "sup", "howdy", "good morning", "good evening"}
|
||||
|
||||
# Short filler words that shouldn't count as subject overlap between facts.
|
||||
_STOPWORDS = {
|
||||
"with", "that", "have", "this", "from", "they", "what", "when", "where",
|
||||
"which", "there", "your", "will", "would", "about", "these", "their",
|
||||
"been", "into", "than", "then", "them", "were", "being", "more", "most",
|
||||
"some", "other", "only", "still", "also", "after", "before", "during",
|
||||
"because", "through", "without",
|
||||
}
|
||||
|
||||
|
||||
def _subject_words(text: str) -> set:
|
||||
"""Meaningful subject tokens for overlap comparison."""
|
||||
words = re.findall(r"[A-Za-z0-9_]{4,}", text.lower())
|
||||
return {w for w in words if w not in _STOPWORDS}
|
||||
|
||||
|
||||
def _is_social(text: str) -> bool:
|
||||
t = text.strip().lower()
|
||||
@@ -86,6 +101,10 @@ def auto_detect_facts(user_message: str, assistant_message: str) -> list[str]:
|
||||
def check_fact_conflicts(facts: list[str]) -> list[dict]:
|
||||
"""Search for existing memories that conflict with detected facts.
|
||||
|
||||
A conflict is reported only when the existing memory is about the same
|
||||
subject (meaningful keyword overlap) but states something different —
|
||||
unrelated hits that merely share an FTS keyword are not conflicts.
|
||||
|
||||
Returns list of {memory_id, old_fact, new_fact} for each conflict.
|
||||
"""
|
||||
conflicts = []
|
||||
@@ -93,7 +112,7 @@ def check_fact_conflicts(facts: list[str]) -> list[dict]:
|
||||
related = search_memories(new_fact, limit=1)
|
||||
if related:
|
||||
old = related[0]["fact"]
|
||||
if old.rstrip(".") != new_fact.rstrip("."):
|
||||
if old.rstrip(".") != new_fact.rstrip(".") and (_subject_words(new_fact) & _subject_words(old)):
|
||||
conflicts.append({
|
||||
"memory_id": related[0]["rowid"],
|
||||
"old_fact": old,
|
||||
|
||||
+13
-11
@@ -11,13 +11,13 @@ responds to pings, and handles model swap commands.
|
||||
# hostname — defaults to socket.gethostname()
|
||||
node_name = jarvis
|
||||
# LAN IP — defaults from socket
|
||||
node_ip = 192.168.50.210
|
||||
node_ip = 192.168.50.212
|
||||
# "worker" (fixed)
|
||||
node_type = worker
|
||||
# comma-separated capability list
|
||||
capabilities = llm
|
||||
# RabbitMQ URL on coordinator
|
||||
amqp_url = amqp://caic:password@192.168.50.108:5672/caic
|
||||
amqp_url = amqp://caic:password@localhost:5672/caic
|
||||
# port llama-server listens on
|
||||
llama_port = 8081
|
||||
# path to GGUF model files
|
||||
@@ -184,16 +184,18 @@ def get_load() -> dict:
|
||||
capture_output=True, text=True, timeout=3,
|
||||
)
|
||||
if result.returncode == 0:
|
||||
total = 0
|
||||
used = 0
|
||||
for line in result.stdout.splitlines():
|
||||
if "VRAM Total" in line:
|
||||
parts = line.split()
|
||||
if len(parts) >= 3:
|
||||
total = int(parts[-1])
|
||||
elif "VRAM Used" in line:
|
||||
parts = line.split()
|
||||
if len(parts) >= 3:
|
||||
used = int(parts[-1])
|
||||
if total and total > 0:
|
||||
if "VRAM Total Used Memory (B)" in line:
|
||||
parts = line.split(":")
|
||||
if len(parts) >= 2:
|
||||
used = int(parts[-1].strip())
|
||||
elif "VRAM Total Memory (B)" in line:
|
||||
parts = line.split(":")
|
||||
if len(parts) >= 2:
|
||||
total = int(parts[-1].strip())
|
||||
if total > 0:
|
||||
load["vram_pct"] = round(used / total * 100)
|
||||
except (FileNotFoundError, subprocess.TimeoutExpired):
|
||||
pass
|
||||
|
||||
@@ -4,6 +4,7 @@ cAIc - RAG pipeline: Qdrant vector search + system prompt assembly.
|
||||
import asyncio
|
||||
import logging
|
||||
import os
|
||||
import uuid
|
||||
from datetime import datetime, timezone
|
||||
|
||||
import httpx
|
||||
@@ -45,7 +46,7 @@ async def _upsert_fact(fact: str, text: str, topic: str,
|
||||
if er.status_code != 200:
|
||||
continue
|
||||
vector = er.json()["embedding"]
|
||||
pid = f"auto-{ts}-{i}"
|
||||
pid = str(uuid.uuid5(uuid.NAMESPACE_DNS, f"auto-{ts}-{i}"))
|
||||
payload = {
|
||||
"text": encrypt_text(chunk), "source": "auto_fact", "fact": fact,
|
||||
"ingest_date": datetime.now(timezone.utc).isoformat(),
|
||||
@@ -124,7 +125,7 @@ async def confirm_fact_update(memory_id: int, old_fact: str, new_fact: str,
|
||||
return True
|
||||
|
||||
|
||||
def chunk_text(text: str, chunk_size: int = 512, overlap: int = 128) -> list:
|
||||
def chunk_text(text: str, chunk_size: int = 200, overlap: int = 64) -> list:
|
||||
words = text.split()
|
||||
target_words = int(chunk_size / 1.3)
|
||||
overlap_words = int(overlap / 1.3)
|
||||
|
||||
+27
-6
@@ -1,4 +1,4 @@
|
||||
"""JarvisChat routers - /api/chat streaming endpoint."""
|
||||
"""cAIc routers - /api/chat streaming endpoint."""
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
@@ -23,6 +23,22 @@ from config import MAX_CHAT_MESSAGE_CHARS, MODEL_CONTEXT_LENGTH
|
||||
log = logging.getLogger("caic")
|
||||
router = APIRouter()
|
||||
|
||||
# References to background auto-ingest tasks so they are never garbage-collected.
|
||||
_ingest_tasks: set = set()
|
||||
|
||||
|
||||
async def _safe_ingest(coro):
|
||||
try:
|
||||
await coro
|
||||
except Exception as e:
|
||||
log.warning("auto-ingest task failed: %s", e)
|
||||
|
||||
|
||||
def _spawn_ingest(coro):
|
||||
task = asyncio.create_task(_safe_ingest(coro))
|
||||
_ingest_tasks.add(task)
|
||||
task.add_done_callback(_ingest_tasks.discard)
|
||||
|
||||
|
||||
def parse_llama_stream_chunk(line: str) -> tuple:
|
||||
if line.startswith("data: "):
|
||||
@@ -112,6 +128,11 @@ async def chat(request: Request):
|
||||
db.execute("INSERT INTO conversations (id, title, model, created_at, updated_at) VALUES (?, ?, ?, ?, ?)",
|
||||
(conv_id, encrypt_text(title), model, now, now))
|
||||
else:
|
||||
# A client-supplied id may reference a conversation that no longer exists;
|
||||
# recreate the row so the message insert satisfies the FK instead of 500ing.
|
||||
title = user_message[:80] + ("..." if len(user_message) > 80 else "")
|
||||
db.execute("INSERT OR IGNORE INTO conversations (id, title, model, created_at, updated_at) VALUES (?, ?, ?, ?, ?)",
|
||||
(conv_id, encrypt_text(title), model, now, now))
|
||||
db.execute("UPDATE conversations SET updated_at = ? WHERE id = ?", (now, conv_id))
|
||||
|
||||
db.execute("INSERT INTO messages (conversation_id, role, content, created_at, perplexity) VALUES (?, ?, ?, ?, ?)",
|
||||
@@ -171,6 +192,8 @@ async def chat(request: Request):
|
||||
|
||||
assistant_msg = "".join(full_response)
|
||||
perplexity = calculate_perplexity(all_logprobs) if all_logprobs else 0.0
|
||||
if not all_logprobs:
|
||||
log.warning("No logprobs received from inference server — perplexity auto-search unavailable")
|
||||
should_search = is_uncertain(all_logprobs) or is_refusal(assistant_msg)
|
||||
|
||||
if search_enabled and should_search:
|
||||
@@ -210,7 +233,7 @@ async def chat(request: Request):
|
||||
if is_refusal(cleaned_response) or len(cleaned_response) < 20:
|
||||
cleaned_response = format_direct_answer(user_message, search_results)
|
||||
|
||||
yield f"data: {json.dumps({'token': cleaned_response, 'conversation_id': conv_id, 'augmented': True})}\n\n"
|
||||
yield f"data: {json.dumps({'token': cleaned_response, 'conversation_id': conv_id, 'augmented': True, 'reset': True})}\n\n"
|
||||
|
||||
if not private_chat:
|
||||
saved_msg = cleaned_response + "\n\n---\n*🔍 Enhanced with web search results*"
|
||||
@@ -229,8 +252,7 @@ async def chat(request: Request):
|
||||
if conflicts:
|
||||
rag_update = {"conflicts": conflicts}
|
||||
else:
|
||||
# Fire-and-forget: persist facts silently, don't block the response
|
||||
asyncio.create_task(ingest_auto_fact(facts, user_message, cleaned_response))
|
||||
_spawn_ingest(ingest_auto_fact(facts, user_message, cleaned_response))
|
||||
|
||||
yield f"data: {json.dumps({'done': True, 'conversation_id': conv_id, 'searched': True, 'perplexity': round(perplexity, 2), 'tokens_per_sec': round(tokens_per_sec, 1), 'prompt_tokens': prompt_tokens, 'completion_tokens': completion_tokens, 'context_length': MODEL_CONTEXT_LENGTH, **(rag_update and {'rag_update_suggestion': rag_update} or {})})}\n\n"
|
||||
return
|
||||
@@ -252,8 +274,7 @@ async def chat(request: Request):
|
||||
if conflicts:
|
||||
rag_update = {"conflicts": conflicts}
|
||||
else:
|
||||
# Fire-and-forget: persist facts silently, don't block the response
|
||||
asyncio.create_task(ingest_auto_fact(facts, user_message, assistant_msg))
|
||||
_spawn_ingest(ingest_auto_fact(facts, user_message, assistant_msg))
|
||||
|
||||
yield f"data: {json.dumps({'done': True, 'conversation_id': conv_id, 'perplexity': round(perplexity, 2), 'tokens_per_sec': round(tokens_per_sec, 1), 'prompt_tokens': prompt_tokens, 'completion_tokens': completion_tokens, 'context_length': MODEL_CONTEXT_LENGTH, **(rag_update and {'rag_update_suggestion': rag_update} or {})})}\n\n"
|
||||
|
||||
|
||||
+1
-1
@@ -1,4 +1,4 @@
|
||||
"""JarvisChat routers - Cluster status API."""
|
||||
"""cAIc routers - Cluster status API."""
|
||||
from fastapi import APIRouter
|
||||
|
||||
import cluster
|
||||
|
||||
@@ -1,11 +1,12 @@
|
||||
"""
|
||||
JarvisChat - /v1/chat/completions router.
|
||||
cAIc - /v1/chat/completions router.
|
||||
OpenAI-compatible endpoint for IDE integration (Continue.dev, etc.).
|
||||
Runs all requests through the full jC pipeline: profile + RAG + memory injection.
|
||||
FIM (fill-in-the-middle) requests are proxied directly — not persisted.
|
||||
Chat-style requests are persisted to conversation history.
|
||||
Auth: static Bearer token via COMPLETIONS_API_KEY in config.
|
||||
"""
|
||||
import hmac
|
||||
import json
|
||||
import logging
|
||||
import uuid
|
||||
@@ -30,7 +31,7 @@ def _check_api_key(request: Request):
|
||||
if not auth.startswith("Bearer "):
|
||||
raise HTTPException(status_code=401, detail="Missing Bearer token")
|
||||
token = auth[7:].strip()
|
||||
if token != COMPLETIONS_API_KEY:
|
||||
if not hmac.compare_digest(token.encode(), COMPLETIONS_API_KEY.encode()):
|
||||
raise HTTPException(status_code=401, detail="Invalid API key")
|
||||
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
"""JarvisChat routers - Conversation CRUD."""
|
||||
"""cAIc routers - Conversation CRUD."""
|
||||
import logging
|
||||
import uuid
|
||||
from datetime import datetime, timezone
|
||||
|
||||
+1
-1
@@ -1,4 +1,4 @@
|
||||
"""JarvisChat routers — Hardware self-assessment endpoint."""
|
||||
"""cAIc routers — Hardware self-assessment endpoint."""
|
||||
import json
|
||||
|
||||
from fastapi import APIRouter
|
||||
|
||||
+1
-1
@@ -1,4 +1,4 @@
|
||||
"""JarvisChat routers — Image generation proxy endpoint."""
|
||||
"""cAIc routers — Image generation proxy endpoint."""
|
||||
import base64
|
||||
import logging
|
||||
|
||||
|
||||
+7
-3
@@ -1,5 +1,8 @@
|
||||
"""JarvisChat routers - /api/ingest terminal command RAG hook."""
|
||||
"""cAIc routers - /api/ingest terminal command RAG hook."""
|
||||
import hashlib
|
||||
import hmac
|
||||
import logging
|
||||
import uuid
|
||||
from datetime import datetime, timezone
|
||||
|
||||
import httpx
|
||||
@@ -20,7 +23,7 @@ def _check_api_key(request: Request):
|
||||
if not auth.startswith("Bearer "):
|
||||
raise HTTPException(status_code=401, detail="Missing Bearer token")
|
||||
token = auth[7:].strip()
|
||||
if token != COMPLETIONS_API_KEY:
|
||||
if not hmac.compare_digest(token.encode(), COMPLETIONS_API_KEY.encode()):
|
||||
raise HTTPException(status_code=401, detail="Invalid API key")
|
||||
|
||||
|
||||
@@ -50,7 +53,8 @@ async def ingest_content(request: Request):
|
||||
log.warning(f"Ingest embedding failed for chunk {i}: {embed_resp.status_code}")
|
||||
continue
|
||||
vector = embed_resp.json()["embedding"]
|
||||
point_id = f"ingest-{source}-{datetime.now(timezone.utc).timestamp()}-{i}"
|
||||
chunk_hash = hashlib.md5(chunk.encode("utf-8")).hexdigest()[:12]
|
||||
point_id = str(uuid.uuid5(uuid.NAMESPACE_DNS, f"ingest-{source}-{chunk_hash}-{i}"))
|
||||
payload = {"text": encrypt_text(chunk), "source": source, "ingest_date": datetime.now(timezone.utc).isoformat(), "type": "ingest"}
|
||||
payload.update(metadata)
|
||||
upsert_resp = await client.put(
|
||||
|
||||
+1
-1
@@ -1,4 +1,4 @@
|
||||
"""JarvisChat routers - Memory CRUD API."""
|
||||
"""cAIc routers - Memory CRUD API."""
|
||||
from fastapi import APIRouter, HTTPException, Request
|
||||
from typing import Optional
|
||||
|
||||
|
||||
+1
-1
@@ -1,5 +1,5 @@
|
||||
"""
|
||||
JarvisChat routers - Model listing, system stats.
|
||||
cAIc routers - Model listing, system stats.
|
||||
"""
|
||||
import logging
|
||||
from typing import Optional
|
||||
|
||||
+1
-1
@@ -1,4 +1,4 @@
|
||||
"""JarvisChat routers - System prompt presets."""
|
||||
"""cAIc routers - System prompt presets."""
|
||||
import uuid
|
||||
from datetime import datetime, timezone
|
||||
from fastapi import APIRouter, HTTPException, Request
|
||||
|
||||
+1
-1
@@ -1,4 +1,4 @@
|
||||
"""JarvisChat routers - Profile."""
|
||||
"""cAIc routers - Profile."""
|
||||
from datetime import datetime, timezone
|
||||
from fastapi import APIRouter, HTTPException, Request
|
||||
from db import get_db
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
"""JarvisChat routers — RAG corpus management admin endpoints."""
|
||||
"""cAIc routers — RAG corpus management admin endpoints."""
|
||||
import logging
|
||||
from datetime import datetime, timezone
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
"""JarvisChat routers - /api/search explicit search endpoint."""
|
||||
"""cAIc routers - /api/search explicit search endpoint."""
|
||||
import json
|
||||
import logging
|
||||
import uuid
|
||||
@@ -44,6 +44,9 @@ async def explicit_search(request: Request):
|
||||
db.execute("INSERT INTO conversations (id, title, model, created_at, updated_at) VALUES (?, ?, ?, ?, ?)",
|
||||
(conv_id, encrypt_text(title), model, now, now))
|
||||
else:
|
||||
title = query[:70] + "..." if len(query) > 70 else query
|
||||
db.execute("INSERT OR IGNORE INTO conversations (id, title, model, created_at, updated_at) VALUES (?, ?, ?, ?, ?)",
|
||||
(conv_id, title, model, now, now))
|
||||
db.execute("UPDATE conversations SET updated_at = ? WHERE id = ?", (now, conv_id))
|
||||
|
||||
db.execute("INSERT INTO messages (conversation_id, role, content, created_at, perplexity) VALUES (?, ?, ?, ?, ?)",
|
||||
|
||||
+1
-1
@@ -1,4 +1,4 @@
|
||||
"""JarvisChat routers - Settings."""
|
||||
"""cAIc routers - Settings."""
|
||||
from fastapi import APIRouter, HTTPException, Request
|
||||
from db import get_db
|
||||
from security import read_json_body, BODY_LIMIT_DEFAULT_BYTES
|
||||
|
||||
+1
-1
@@ -1,4 +1,4 @@
|
||||
"""JarvisChat routers - Skills."""
|
||||
"""cAIc routers - Skills."""
|
||||
from fastapi import APIRouter, HTTPException, Request
|
||||
from db import get_db, get_setting, list_skills_with_state, set_skill_enabled
|
||||
from security import read_json_body, BODY_LIMIT_DEFAULT_BYTES
|
||||
|
||||
+13
-3
@@ -1,7 +1,8 @@
|
||||
"""JarvisChat routers - /api/upload file/document attachment endpoint."""
|
||||
"""cAIc routers - /api/upload file/document attachment endpoint."""
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import uuid
|
||||
from datetime import datetime, timezone, timedelta
|
||||
|
||||
import httpx
|
||||
@@ -19,7 +20,7 @@ router = APIRouter()
|
||||
|
||||
|
||||
def _point_id(filename: str, chunk_idx: int) -> str:
|
||||
return f"upload-{filename}-{chunk_idx}"
|
||||
return str(uuid.uuid5(uuid.NAMESPACE_DNS, f"upload-{filename}-{chunk_idx}"))
|
||||
|
||||
|
||||
@router.post("/api/upload")
|
||||
@@ -52,12 +53,21 @@ async def upload_file(
|
||||
except Exception as e:
|
||||
log.warning(f"PDF extraction error: {e}")
|
||||
raise HTTPException(status_code=422, detail="Failed to extract text from PDF")
|
||||
elif content_type.startswith("image/"):
|
||||
# No OCR pipeline exists — store a descriptive placeholder so images
|
||||
# remain usable in the gallery/context but never pollute the RAG corpus.
|
||||
extracted = f"[Image: {file.filename}]"
|
||||
else:
|
||||
extracted = raw_bytes.decode("utf-8", errors="replace")
|
||||
|
||||
result = {"filename": file.filename, "size_bytes": len(raw_bytes), "mode": mode}
|
||||
|
||||
if mode in ("ingest", "both"):
|
||||
is_image = content_type.startswith("image/")
|
||||
if is_image and mode in ("ingest", "both"):
|
||||
result["chunks_ingested"] = 0
|
||||
result["note"] = "Image files cannot be text-ingested; stored for gallery/context only"
|
||||
|
||||
if mode in ("ingest", "both") and not is_image:
|
||||
os.makedirs(UPLOAD_DIR, exist_ok=True)
|
||||
chunks = chunk_text(extracted)
|
||||
ingested = 0
|
||||
|
||||
@@ -161,10 +161,6 @@ def origin_allowed(request: Request) -> bool:
|
||||
return False
|
||||
|
||||
|
||||
def is_state_changing(method: str) -> bool:
|
||||
return method in {"POST", "PUT", "DELETE", "PATCH"}
|
||||
|
||||
|
||||
async def read_json_body(request: Request, max_bytes: int) -> dict:
|
||||
raw = await request.body()
|
||||
if len(raw) > max_bytes:
|
||||
|
||||
@@ -1635,6 +1635,7 @@ async function sendSearch() {
|
||||
if (data.error) { textEl.textContent = 'Error: ' + data.error; setStreamingState(false); return; }
|
||||
if (data.conversation_id && !currentConvId) { currentConvId = data.conversation_id; await loadConversations(); }
|
||||
if (data.search_results) { textEl.innerHTML = '<div class="search-indicator">🔍 Found ' + data.search_results + ' results, summarizing...</div>'; }
|
||||
if (data.reset) { fullText = ''; textEl.innerHTML = ''; firstToken = false; }
|
||||
if (data.token) { if (firstToken) { textEl.innerHTML = ''; firstToken = false; ttr = performance.now() - ttrStart; tokenCount = 0; } fullText += data.token; tokenCount++; textEl.innerHTML = renderMarkdown(fullText); scrollToLatest(); }
|
||||
if (data.raw_results) {
|
||||
let rawHtml = '<details class="raw-results"><summary>🔍 View raw search results (' + data.raw_results.length + ')</summary><ul>';
|
||||
@@ -1843,7 +1844,7 @@ async function sendMessage() {
|
||||
}
|
||||
if (data.searching) { textEl.innerHTML = fullText ? renderMarkdown(fullText) + '<div class="search-indicator"><div class="spinner"></div>Searching...</div>' : '<div class="search-indicator"><div class="spinner"></div>Searching...</div>'; searchTriggered = true; }
|
||||
if (data.search_results) { textEl.innerHTML = '<div class="search-indicator">🔍 Found ' + data.search_results + ' results...</div>'; fullText = ''; firstToken = true; }
|
||||
if (data.token) { if (firstToken) { textEl.innerHTML = ''; firstToken = false; ttr = performance.now() - ttrStart; tokenCount = 0; } fullText += data.token; tokenCount++; textEl.innerHTML = renderMarkdown(fullText); scrollToLatest(); }
|
||||
if (data.token) { if (data.reset) { fullText = ''; firstToken = true; } if (firstToken) { textEl.innerHTML = ''; firstToken = false; ttr = performance.now() - ttrStart; tokenCount = 0; } fullText += data.token; tokenCount++; textEl.innerHTML = renderMarkdown(fullText); scrollToLatest(); }
|
||||
if (data.done) {
|
||||
const roleLabel = assistantDiv.querySelector('.role-label');
|
||||
if (data.searched && roleLabel) roleLabel.textContent = 'web search';
|
||||
|
||||
@@ -4,3 +4,31 @@ import sys
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
|
||||
|
||||
"""Shared pytest fixtures.
|
||||
|
||||
All test modules manipulate in-process globals (sessions, rate buckets,
|
||||
cluster registry, eviction log). An autouse fixture resets every global
|
||||
before each test so no state leaks between tests, regardless of whether an
|
||||
individual test file remembers to clear it.
|
||||
"""
|
||||
import pytest
|
||||
|
||||
import cluster
|
||||
import routers.chat
|
||||
from eviction import EVICTION_LOG
|
||||
from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _reset_global_state():
|
||||
SESSIONS.clear()
|
||||
PIN_ATTEMPTS.clear()
|
||||
RATE_EVENTS.clear()
|
||||
cluster.CLUSTER_NODES.clear()
|
||||
cluster.CLUSTER_EVENTS.clear()
|
||||
cluster.CLUSTER_COORDINATOR = None
|
||||
cluster._pending_pings.clear()
|
||||
EVICTION_LOG.clear()
|
||||
routers.chat._ingest_tasks.clear()
|
||||
yield
|
||||
|
||||
@@ -10,7 +10,6 @@ import app
|
||||
import config
|
||||
import db
|
||||
import routers.chat
|
||||
import triage
|
||||
from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS
|
||||
|
||||
|
||||
@@ -269,7 +268,6 @@ def test_private_chat_does_not_persist(tmp_path: Path, monkeypatch):
|
||||
'data: {"choices":[{"delta":{"content":""},"finish_reason":"stop","logprobs":{"content":[]}}],"usage":{"completion_tokens":2,"prompt_tokens":10,"tokens_per_second":5.0}}',
|
||||
"data: [DONE]",
|
||||
]))
|
||||
monkeypatch.setattr(triage, "classify_query", lambda q: "general")
|
||||
|
||||
async def _mock_ensure(m): return True
|
||||
monkeypatch.setattr("model_pull.ensure_model", _mock_ensure)
|
||||
@@ -301,7 +299,6 @@ def test_private_chat_does_not_auto_search(tmp_path: Path, monkeypatch):
|
||||
'data: {"choices":[{"delta":{"content":""},"finish_reason":"stop","logprobs":{"content":[{"logprob":-2.5}]}}],"usage":{"completion_tokens":1,"prompt_tokens":10,"tokens_per_second":5.0}}',
|
||||
"data: [DONE]",
|
||||
]))
|
||||
monkeypatch.setattr(triage, "classify_query", lambda q: "general")
|
||||
monkeypatch.setattr(routers.chat, "query_searxng", lambda q: [{"title": "result"}])
|
||||
|
||||
with make_client(tmp_path) as client:
|
||||
|
||||
@@ -0,0 +1,357 @@
|
||||
"""Regression tests for bug fixes.
|
||||
|
||||
Covers: /api/ingest origin exemption for CLI/Bearer clients, bogus
|
||||
conversation_id FK handling in chat + search, the auto-search reset flag,
|
||||
image uploads being stored as placeholders instead of text-ingested,
|
||||
false-positive conflict detection, deterministic ingest point IDs,
|
||||
get_load() VRAM parsing, and version pinning.
|
||||
"""
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
import httpx
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
import app
|
||||
import config
|
||||
import db
|
||||
import memory
|
||||
from crypto import decrypt_text
|
||||
import node_agent.agent as agent
|
||||
import routers.chat
|
||||
import routers.ingest as ingest_route
|
||||
import routers.search_route
|
||||
from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS
|
||||
|
||||
|
||||
def make_client(tmp_path: Path) -> TestClient:
|
||||
os.environ["CAIC_ADMIN_PIN"] = "1234"
|
||||
db.DB_PATH = tmp_path / "caic-regression.db"
|
||||
SESSIONS.clear()
|
||||
PIN_ATTEMPTS.clear()
|
||||
RATE_EVENTS.clear()
|
||||
db.init_db()
|
||||
return TestClient(app.app, raise_server_exceptions=False)
|
||||
|
||||
|
||||
def _guest_headers(client: TestClient) -> dict:
|
||||
sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()["session_id"]
|
||||
return {"X-Session-ID": sid, "Origin": "http://testserver"}
|
||||
|
||||
|
||||
def _admin_headers(client: TestClient) -> dict:
|
||||
login = client.post("/api/auth/login", json={"pin": "1234"}, headers={"Origin": "http://testserver"})
|
||||
sid = login.json()["session_id"]
|
||||
return {"X-Session-ID": sid, "Origin": "http://testserver"}
|
||||
|
||||
|
||||
def parse_sse_payloads(body: str) -> list[dict]:
|
||||
payloads = []
|
||||
for chunk in body.split("\n\n"):
|
||||
chunk = chunk.strip()
|
||||
if not chunk.startswith("data: "):
|
||||
continue
|
||||
payloads.append(json.loads(chunk[len("data: "):]))
|
||||
return payloads
|
||||
|
||||
|
||||
class _MockStreamResponse:
|
||||
def __init__(self, lines: list[str]):
|
||||
self._lines = lines
|
||||
|
||||
async def __aenter__(self):
|
||||
return self
|
||||
|
||||
async def __aexit__(self, exc_type, exc, tb):
|
||||
return False
|
||||
|
||||
async def aiter_lines(self):
|
||||
for line in self._lines:
|
||||
yield line
|
||||
|
||||
|
||||
def _stream_json_lines(events: list[dict]) -> list[str]:
|
||||
return [json.dumps(event) for event in events]
|
||||
|
||||
|
||||
class _FakeAsyncClient:
|
||||
class FakeResponse:
|
||||
def __init__(self, status, json_data=None):
|
||||
self.status_code = status
|
||||
self._json = json_data or {}
|
||||
|
||||
def json(self):
|
||||
return self._json
|
||||
|
||||
def __init__(self, *a, **kw):
|
||||
self.put_payloads = []
|
||||
|
||||
async def __aenter__(self):
|
||||
return self
|
||||
|
||||
async def __aexit__(self, *a):
|
||||
pass
|
||||
|
||||
async def post(self, url, **kw):
|
||||
if "/api/embeddings" in url:
|
||||
return self.FakeResponse(200, {"embedding": [0.1] * 768})
|
||||
return self.FakeResponse(200)
|
||||
|
||||
async def put(self, url, **kw):
|
||||
self.put_payloads.append(kw.get("json", {}))
|
||||
return self.FakeResponse(200)
|
||||
|
||||
|
||||
# ── /api/ingest is reached by CLI tools with no Origin header ──────────
|
||||
|
||||
|
||||
def test_ingest_origin_exemption(tmp_path: Path, monkeypatch):
|
||||
monkeypatch.setattr(ingest_route, "COMPLETIONS_API_KEY", "sk-regression")
|
||||
monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: _FakeAsyncClient())
|
||||
with make_client(tmp_path) as client:
|
||||
resp = client.post(
|
||||
"/api/ingest",
|
||||
json={"content": "regression test content " * 20, "source": "cli"},
|
||||
headers={"Authorization": "Bearer sk-regression", "Content-Type": "application/json"},
|
||||
)
|
||||
assert resp.status_code == 200, resp.text
|
||||
assert resp.json()["source"] == "cli"
|
||||
|
||||
|
||||
def test_ingest_bad_key_still_blocked_without_origin(tmp_path: Path, monkeypatch):
|
||||
monkeypatch.setattr(ingest_route, "COMPLETIONS_API_KEY", "sk-regression")
|
||||
with make_client(tmp_path) as client:
|
||||
resp = client.post(
|
||||
"/api/ingest",
|
||||
json={"content": "x " * 50},
|
||||
headers={"Authorization": "Bearer wrong-key", "Content-Type": "application/json"},
|
||||
)
|
||||
assert resp.status_code == 401
|
||||
|
||||
|
||||
# ── a client-supplied conversation_id that no longer exists ────────────
|
||||
|
||||
|
||||
def test_chat_bogus_conversation_id_creates_row(tmp_path: Path, monkeypatch):
|
||||
events = _stream_json_lines([
|
||||
{"message": {"content": "hi"}, "logprobs": [{"logprob": -0.01}]},
|
||||
{"done": True, "eval_count": 1, "eval_duration": 1000000000},
|
||||
])
|
||||
|
||||
def stream_stub(self, method, url, json=None, timeout=None):
|
||||
return _MockStreamResponse(events)
|
||||
|
||||
monkeypatch.setattr(httpx.AsyncClient, "stream", stream_stub)
|
||||
|
||||
with make_client(tmp_path) as client:
|
||||
resp = client.post(
|
||||
"/api/chat",
|
||||
json={"message": "hello", "conversation_id": "ghost-conv", "model": config.DEFAULT_MODEL},
|
||||
headers=_guest_headers(client),
|
||||
)
|
||||
assert resp.status_code == 200, resp.text
|
||||
conv_resp = client.get("/api/conversations/ghost-conv", headers=_guest_headers(client))
|
||||
assert conv_resp.status_code == 200
|
||||
assert len(conv_resp.json()["messages"]) >= 2
|
||||
|
||||
|
||||
def test_search_bogus_conversation_id_creates_row(tmp_path: Path, monkeypatch):
|
||||
async def empty_search(query: str, max_results: int = 5):
|
||||
return []
|
||||
|
||||
monkeypatch.setattr(routers.search_route, "query_searxng", empty_search)
|
||||
|
||||
with make_client(tmp_path) as client:
|
||||
resp = client.post(
|
||||
"/api/search",
|
||||
json={"query": "nothing here", "conversation_id": "ghost-search", "model": config.DEFAULT_MODEL},
|
||||
headers=_guest_headers(client),
|
||||
)
|
||||
assert resp.status_code == 200, resp.text
|
||||
conv_resp = client.get("/api/conversations/ghost-search", headers=_guest_headers(client))
|
||||
assert conv_resp.status_code == 200
|
||||
assert len(conv_resp.json()["messages"]) >= 1
|
||||
|
||||
|
||||
# ── auto-search augmentation must reset the streamed text ──────────────
|
||||
|
||||
|
||||
def test_auto_search_augmented_event_has_reset_flag(tmp_path: Path, monkeypatch):
|
||||
first_stream = _stream_json_lines([
|
||||
{"message": {"content": "I don't have current data on that."}, "logprobs": [{"logprob": -5.0}]},
|
||||
{"done": True, "eval_count": 2, "eval_duration": 1000000000},
|
||||
])
|
||||
second_stream = _stream_json_lines([
|
||||
{"message": {"content": "According to the search results, the value is forty-two."}},
|
||||
{"done": True},
|
||||
])
|
||||
batches = [first_stream, second_stream]
|
||||
|
||||
def stream_stub(self, method, url, json=None, timeout=None):
|
||||
return _MockStreamResponse(batches.pop(0))
|
||||
|
||||
async def search_stub(query: str, max_results: int = 5):
|
||||
return [{"title": "Answer", "url": "https://example.com", "content": "The value is 42."}]
|
||||
|
||||
with make_client(tmp_path) as client:
|
||||
monkeypatch.setattr(httpx.AsyncClient, "stream", stream_stub)
|
||||
monkeypatch.setattr(routers.chat, "query_searxng", search_stub)
|
||||
resp = client.post(
|
||||
"/api/chat",
|
||||
json={"message": "what is the latest value", "model": config.DEFAULT_MODEL},
|
||||
headers=_guest_headers(client),
|
||||
)
|
||||
assert resp.status_code == 200, resp.text
|
||||
payloads = parse_sse_payloads(resp.text)
|
||||
|
||||
augmented = [p for p in payloads if p.get("augmented")]
|
||||
assert augmented, "expected an augmented token event"
|
||||
assert augmented[0].get("reset") is True
|
||||
# The augmented token must carry the fresh answer, not the discarded
|
||||
# first-pass "I don't have current data" text.
|
||||
assert "According to the search results" in augmented[0]["token"]
|
||||
assert "I don't have current data" not in augmented[0]["token"]
|
||||
|
||||
|
||||
# ── image uploads are placeholders, never text-ingested ────────────────
|
||||
|
||||
|
||||
def test_upload_image_skips_ingest(tmp_path: Path, monkeypatch):
|
||||
fake = _FakeAsyncClient()
|
||||
monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: fake)
|
||||
with make_client(tmp_path) as client:
|
||||
resp = client.post(
|
||||
"/api/upload",
|
||||
headers=_admin_headers(client),
|
||||
data={"mode": "both"},
|
||||
files={"file": ("photo.png", b"\x89PNG\r\n\x1a\nfake", "image/png")},
|
||||
)
|
||||
assert resp.status_code == 200, resp.text
|
||||
data = resp.json()
|
||||
context_id = data["context_id"]
|
||||
|
||||
assert data["chunks_ingested"] == 0
|
||||
assert data["note"]
|
||||
assert fake.put_payloads == []
|
||||
assert data["filename"] == "photo.png"
|
||||
|
||||
row = db.get_db().execute(
|
||||
"SELECT content FROM upload_context WHERE id = ?", (context_id,)
|
||||
).fetchone()
|
||||
assert row and decrypt_text(row["content"]) == "[Image: photo.png]"
|
||||
|
||||
|
||||
# ── conflict detection needs a shared subject, not just an FTS hit ─────
|
||||
|
||||
|
||||
def test_conflict_detection_requires_shared_subject(tmp_path: Path):
|
||||
os.environ["CAIC_ADMIN_PIN"] = "1234"
|
||||
db.DB_PATH = tmp_path / "caic-mem-regression.db"
|
||||
SESSIONS.clear()
|
||||
PIN_ATTEMPTS.clear()
|
||||
RATE_EVENTS.clear()
|
||||
db.init_db()
|
||||
|
||||
memory.add_memory("the cat sat on the mat", "general")
|
||||
conflicts = memory.check_fact_conflicts(["the dog is brown"])
|
||||
assert conflicts == []
|
||||
|
||||
memory.add_memory("I prefer Rust over Go", "preference")
|
||||
conflicts = memory.check_fact_conflicts(["I prefer Go over Rust"])
|
||||
assert len(conflicts) == 1
|
||||
assert conflicts[0]["new_fact"] == "I prefer Go over Rust"
|
||||
assert conflicts[0]["old_fact"] == "I prefer Rust over Go"
|
||||
assert "memory_id" in conflicts[0]
|
||||
|
||||
|
||||
# ── ingest point IDs are deterministic (no duplicate vectors) ──────────
|
||||
|
||||
|
||||
def test_ingest_deterministic_point_ids(tmp_path: Path, monkeypatch):
|
||||
monkeypatch.setattr(ingest_route, "COMPLETIONS_API_KEY", "sk-regression")
|
||||
|
||||
captured_first = []
|
||||
captured_second = []
|
||||
|
||||
class CaptureClient:
|
||||
FakeResponse = _FakeAsyncClient.FakeResponse
|
||||
|
||||
def __init__(self, *a, **kw):
|
||||
self.capture = None
|
||||
|
||||
async def __aenter__(self):
|
||||
return self
|
||||
|
||||
async def __aexit__(self, *a):
|
||||
pass
|
||||
|
||||
async def post(self, url, **kw):
|
||||
if "/api/embeddings" in url:
|
||||
return self.FakeResponse(200, {"embedding": [0.2] * 768})
|
||||
return self.FakeResponse(200)
|
||||
|
||||
async def put(self, url, **kw):
|
||||
payload = kw.get("json", {})
|
||||
if self.capture is not None:
|
||||
self.capture.append(payload["points"][0]["id"])
|
||||
return self.FakeResponse(200)
|
||||
|
||||
body = {"content": "alpha beta gamma delta epsilon " * 8, "source": "hook"}
|
||||
headers = {"Authorization": "Bearer sk-regression", "Content-Type": "application/json"}
|
||||
|
||||
fake1 = CaptureClient()
|
||||
fake1.capture = captured_first
|
||||
monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: fake1)
|
||||
with make_client(tmp_path) as client:
|
||||
r1 = client.post("/api/ingest", json=body, headers=headers)
|
||||
assert r1.status_code == 200, r1.text
|
||||
|
||||
fake2 = CaptureClient()
|
||||
fake2.capture = captured_second
|
||||
monkeypatch.setattr(httpx, "AsyncClient", lambda *a, **kw: fake2)
|
||||
with make_client(tmp_path) as client:
|
||||
r2 = client.post("/api/ingest", json=body, headers=headers)
|
||||
assert r2.status_code == 200, r2.text
|
||||
|
||||
assert captured_first and captured_second
|
||||
assert len(captured_first) == len(captured_second)
|
||||
assert captured_first == captured_second, "re-ingesting identical content changed point ids"
|
||||
assert len(set(captured_first)) == len(captured_first)
|
||||
|
||||
|
||||
# ── get_load() VRAM parsing ────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_get_load_vram_parses_rocm_output(monkeypatch):
|
||||
monkeypatch.setattr(agent, "HAS_PSUTIL", False)
|
||||
output = (
|
||||
"======================= ROCm System Management Interface =======================\n"
|
||||
"GPU[0] : gfx1030\n"
|
||||
"VRAM Total Used Memory (B): 3221225472\n"
|
||||
"VRAM Total Memory (B): 17179869184\n"
|
||||
)
|
||||
fake = subprocess.CompletedProcess(["rocm-smi", "--showmeminfo", "vram"], 0, output, "")
|
||||
monkeypatch.setattr(subprocess, "run", lambda *a, **kw: fake)
|
||||
load = agent.get_load()
|
||||
assert load["vram_pct"] == 19 # 3 GiB / 16 GiB
|
||||
|
||||
|
||||
def test_get_load_vram_absent_does_not_crash(monkeypatch):
|
||||
# Regression: rocm-smi returned no parseable VRAM lines, so the old code
|
||||
# left total/used unbound and raised.
|
||||
monkeypatch.setattr(agent, "HAS_PSUTIL", False)
|
||||
output = "======================= ROCm System Management Interface =======================\nNo GPU detected\n"
|
||||
fake = subprocess.CompletedProcess(["rocm-smi", "--showmeminfo", "vram"], 0, output, "")
|
||||
monkeypatch.setattr(subprocess, "run", lambda *a, **kw: fake)
|
||||
load = agent.get_load()
|
||||
assert "vram_pct" not in load
|
||||
|
||||
|
||||
# ── version pinning ────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_version_is_bumped():
|
||||
assert re.fullmatch(r"v\d+\.\d+\.\d+", config.VERSION)
|
||||
@@ -2,7 +2,6 @@
|
||||
import asyncio
|
||||
|
||||
import cluster
|
||||
import triage
|
||||
from config import AMQP_EXCHANGE_ADMIN, AMQP_EXCHANGE_SYSTEM
|
||||
|
||||
|
||||
@@ -149,54 +148,3 @@ def test_handle_model_failed_unknown_node(caplog, monkeypatch):
|
||||
))
|
||||
|
||||
assert any("unknown node" in rec.message for rec in caplog.records)
|
||||
|
||||
|
||||
# ---------- 4. select_node() triggers swap when model mismatched ----------
|
||||
|
||||
|
||||
def test_select_node_code_triggers_swap(monkeypatch):
|
||||
_reset()
|
||||
_published.clear()
|
||||
monkeypatch.setattr(cluster, "publish", _fake_publish)
|
||||
|
||||
cluster.CLUSTER_NODES["jarvis"] = {
|
||||
"name": "jarvis", "type": "worker", "status": "active",
|
||||
"ip": "192.168.50.210",
|
||||
"active_model": {"name": "llama3.1", "port": 8081},
|
||||
"inventory": [
|
||||
{"filename": "qwen2.5-coder-14b-Q4_K_M.gguf", "name": "qwen2.5-coder", "version": "14b", "quant": "Q4_K_M"},
|
||||
],
|
||||
}
|
||||
|
||||
result = asyncio.run(triage.select_node("code"))
|
||||
|
||||
assert result is None
|
||||
# Swap should have been published
|
||||
assert any("cmd.swap_model" in rk for _, rk, _ in _published)
|
||||
# Node should now be swapping
|
||||
assert cluster.CLUSTER_NODES["jarvis"]["status"] == "swapping"
|
||||
|
||||
|
||||
# ---------- 5. select_node() returns None when node is already swapping ----------
|
||||
|
||||
|
||||
def test_select_node_swapping_returns_none(monkeypatch):
|
||||
_reset()
|
||||
_published.clear()
|
||||
monkeypatch.setattr(cluster, "publish", _fake_publish)
|
||||
|
||||
cluster.CLUSTER_NODES["jarvis"] = {
|
||||
"name": "jarvis", "type": "worker", "status": "swapping",
|
||||
"ip": "192.168.50.210",
|
||||
"active_model": {"name": "llama3.1", "port": 8081},
|
||||
"inventory": [
|
||||
{"filename": "qwen2.5-coder-14b-Q4_K_M.gguf", "name": "qwen2.5-coder"},
|
||||
],
|
||||
}
|
||||
|
||||
result = asyncio.run(triage.select_node("code"))
|
||||
|
||||
assert result is None
|
||||
# No swap command should be published while already swapping
|
||||
swap_published = any("cmd.swap_model" in rk for _, rk, _ in _published)
|
||||
assert not swap_published
|
||||
|
||||
@@ -1,141 +0,0 @@
|
||||
"""Tests for triage.py — query classification and node selection."""
|
||||
import asyncio
|
||||
import json
|
||||
|
||||
import httpx
|
||||
|
||||
import cluster
|
||||
import config
|
||||
import triage
|
||||
|
||||
|
||||
def _reset():
|
||||
cluster.CLUSTER_NODES.clear()
|
||||
cluster.CLUSTER_COORDINATOR = None
|
||||
|
||||
|
||||
_published = []
|
||||
|
||||
|
||||
async def _fake_publish(exchange, routing_key, payload):
|
||||
_published.append((exchange, routing_key, payload))
|
||||
|
||||
|
||||
class _MockPostResponse:
|
||||
def __init__(self, json_data: dict, status_code: int = 200):
|
||||
self._json_data = json_data
|
||||
self.status_code = status_code
|
||||
|
||||
def json(self):
|
||||
return self._json_data
|
||||
|
||||
async def __aenter__(self):
|
||||
return self
|
||||
|
||||
async def __aexit__(self, exc_type, exc, tb):
|
||||
return False
|
||||
|
||||
|
||||
class _MockPostContext:
|
||||
def __init__(self, response: _MockPostResponse):
|
||||
self._response = response
|
||||
|
||||
async def __aenter__(self):
|
||||
return self._response
|
||||
|
||||
async def __aexit__(self, exc_type, exc, tb):
|
||||
return False
|
||||
|
||||
|
||||
# ---------- 1. classify_query returns valid classification ----------
|
||||
|
||||
|
||||
def test_classify_returns_valid(monkeypatch):
|
||||
async def post_stub(self, url, json=None, timeout=None):
|
||||
return _MockPostResponse({
|
||||
"choices": [{"message": {"content": "code"}}]
|
||||
})
|
||||
|
||||
monkeypatch.setattr(httpx.AsyncClient, "post", post_stub)
|
||||
|
||||
result = __import__("asyncio").run(triage.classify_query("write a python function"))
|
||||
assert result == "code"
|
||||
|
||||
|
||||
# ---------- 2. classify_query on error returns "general" ----------
|
||||
|
||||
|
||||
def test_classify_error_returns_general(monkeypatch):
|
||||
async def post_stub(self, url, json=None, timeout=None):
|
||||
raise httpx.ConnectError("connection refused")
|
||||
|
||||
monkeypatch.setattr(httpx.AsyncClient, "post", post_stub)
|
||||
|
||||
result = __import__("asyncio").run(triage.classify_query("any question"))
|
||||
assert result == "general"
|
||||
|
||||
|
||||
# ---------- 3. select_node("code") returns coder node ----------
|
||||
|
||||
|
||||
def test_select_node_code_returns_coder():
|
||||
_reset()
|
||||
cluster.CLUSTER_NODES["coder01"] = {
|
||||
"name": "coder01", "type": "worker", "status": "active",
|
||||
"ip": "192.168.50.210",
|
||||
"active_model": {"name": "qwen2.5-coder-14b", "port": 8082},
|
||||
}
|
||||
cluster.CLUSTER_NODES["general01"] = {
|
||||
"name": "general01", "type": "worker", "status": "active",
|
||||
"ip": "192.168.50.211",
|
||||
"active_model": {"name": "llama3.1", "port": 8081},
|
||||
}
|
||||
|
||||
node = asyncio.run(triage.select_node("code"))
|
||||
assert node is not None
|
||||
assert node["name"] == "coder01"
|
||||
|
||||
|
||||
# ---------- 4. select_node("general") with no matching node returns None ----------
|
||||
|
||||
|
||||
def test_select_node_general_no_match_returns_none():
|
||||
_reset()
|
||||
cluster.CLUSTER_NODES["coder01"] = {
|
||||
"name": "coder01", "type": "worker", "status": "active",
|
||||
"active_model": {"name": "qwen2.5-coder-14b", "port": 8082},
|
||||
}
|
||||
node = asyncio.run(triage.select_node("general"))
|
||||
assert node is None
|
||||
|
||||
|
||||
# ---------- 5. get_inference_url with coder node ----------
|
||||
|
||||
|
||||
def test_get_inference_url_with_coder_node(monkeypatch):
|
||||
_reset()
|
||||
async def fake_classify(query: str) -> str:
|
||||
return "code"
|
||||
monkeypatch.setattr(triage, "classify_query", fake_classify)
|
||||
|
||||
cluster.CLUSTER_NODES["coder01"] = {
|
||||
"name": "coder01", "type": "worker", "status": "active",
|
||||
"ip": "192.168.50.210",
|
||||
"active_model": {"name": "qwen2.5-coder-14b", "port": 8082},
|
||||
}
|
||||
|
||||
url = __import__("asyncio").run(triage.get_inference_url("write a loop in rust"))
|
||||
assert url == "http://192.168.50.210:8082/v1"
|
||||
|
||||
|
||||
# ---------- 6. get_inference_url with no nodes returns LLAMA_SERVER_BASE ----------
|
||||
|
||||
|
||||
def test_get_inference_url_no_nodes(monkeypatch):
|
||||
_reset()
|
||||
async def fake_classify(query: str) -> str:
|
||||
return "code"
|
||||
monkeypatch.setattr(triage, "classify_query", fake_classify)
|
||||
|
||||
url = __import__("asyncio").run(triage.get_inference_url("any question"))
|
||||
assert url == config.LLAMA_SERVER_BASE
|
||||
@@ -1,98 +0,0 @@
|
||||
"""cAIc — Query triage and cluster node selection."""
|
||||
import logging
|
||||
|
||||
import httpx
|
||||
|
||||
from config import TRIAGE_BASE, TRIAGE_TIMEOUT, LLAMA_SERVER_BASE
|
||||
|
||||
log = logging.getLogger("caic")
|
||||
|
||||
_IDEAL_MODEL_MAP = {
|
||||
"code": {"name_contains": ["coder", "qwen"]},
|
||||
"general": {"name_contains": ["mistral", "llama"]},
|
||||
}
|
||||
|
||||
_CLASSIFICATION_PROMPT = """Classify the following user query into exactly one category. Respond with only the category name.
|
||||
|
||||
Categories:
|
||||
- general: everyday questions, chitchat, creative writing, advice, explanations
|
||||
- code: programming, debugging, code generation, technical questions about software
|
||||
- search: questions about current events, real-time information, weather, news, specific things that may have changed since training
|
||||
- rag: questions about specific documents, personal data, notes, memory, uploaded content
|
||||
|
||||
Query: {query}
|
||||
Category:"""
|
||||
|
||||
|
||||
async def classify_query(query: str) -> str:
|
||||
try:
|
||||
async with httpx.AsyncClient() as client:
|
||||
resp = await client.post(
|
||||
f"{TRIAGE_BASE}/chat/completions",
|
||||
json={
|
||||
"model": "phi-4-mini",
|
||||
"messages": [
|
||||
{"role": "system", "content": "You are a query classifier. Respond with exactly one word."},
|
||||
{"role": "user", "content": _CLASSIFICATION_PROMPT.format(query=query)},
|
||||
],
|
||||
"temperature": 0.0,
|
||||
"max_tokens": 10,
|
||||
},
|
||||
timeout=TRIAGE_TIMEOUT,
|
||||
)
|
||||
text = resp.json()["choices"][0]["message"]["content"].strip().lower()
|
||||
valid = {"general", "code", "search", "rag"}
|
||||
for v in valid:
|
||||
if v in text:
|
||||
return v
|
||||
except Exception:
|
||||
log.warning("triage classify_query failed, falling back to general", exc_info=True)
|
||||
return "general"
|
||||
|
||||
|
||||
async def select_node(classification: str) -> dict | None:
|
||||
from cluster import CLUSTER_NODES
|
||||
|
||||
if classification in ("search", "rag"):
|
||||
return None
|
||||
|
||||
ideal = _IDEAL_MODEL_MAP.get(classification, {})
|
||||
ideal_contains = ideal.get("name_contains", [])
|
||||
|
||||
# First pass: find an active node with the right model already loaded
|
||||
for node in CLUSTER_NODES.values():
|
||||
if node.get("status") != "active":
|
||||
continue
|
||||
am = node.get("active_model") or {}
|
||||
name = (am.get("name") or "").lower()
|
||||
if any(ideal in name for ideal in ideal_contains):
|
||||
return node
|
||||
|
||||
# Second pass: find an active node that can swap to the right model
|
||||
for node in CLUSTER_NODES.values():
|
||||
if node.get("status") != "active":
|
||||
continue
|
||||
inventory = node.get("inventory") or []
|
||||
for inv in inventory:
|
||||
inv_name = (inv.get("name") or "").lower()
|
||||
if any(ideal in inv_name for ideal in ideal_contains):
|
||||
from cluster import request_model_swap
|
||||
await request_model_swap(node["name"], inv["filename"])
|
||||
return None
|
||||
|
||||
return None
|
||||
|
||||
|
||||
async def get_inference_url(query: str) -> str:
|
||||
if not query:
|
||||
return LLAMA_SERVER_BASE
|
||||
classification = await classify_query(query)
|
||||
if classification in ("search", "rag"):
|
||||
return LLAMA_SERVER_BASE
|
||||
node = await select_node(classification)
|
||||
if node:
|
||||
am = node.get("active_model") or {}
|
||||
port = am.get("port", 8081)
|
||||
ip = node.get("ip") or "127.0.0.1"
|
||||
return f"http://{ip}:{port}/v1"
|
||||
return LLAMA_SERVER_BASE
|
||||
Reference in New Issue
Block a user