diff --git a/AGENTS.md b/AGENTS.md index a252dce..8b832c1 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -59,6 +59,7 @@ Refactored from single-file (`app.py`) into modules under project root: | `memory.py` | FTS5 memory CRUD, remember/forget command parsing | | `search.py` | SearXNG integration, perplexity scoring, refusal detection | | `rag.py` | Qdrant vector search + system prompt assembly + chunk_text() helper | +| `eviction.py` | Score-based RAG eviction engine | | `gpu.py` | AMD GPU stats via `rocm-smi` | | `triage.py` | Phi-4-mini-based query classification + cluster node selection | | `cluster.py` | Cluster node registry, event log, coordinator election, ping/pong, model swap handlers | @@ -108,9 +109,11 @@ The upstream request includes `"logprobs": true`. `parse_llama_stream_chunk()` e | llama-server (coordinator) | Yes | 8081 + RPC :50052 (worker GPU) | | Phi-4-mini (triage) | No | 8083 | | SearXNG | No | 8888 | +| RabbitMQ (coordinator) | No | 5672 — AMQP broker | | wttr.in | No | weather shortcut | | rocm-smi | No | AMD GPU stats | | Qdrant | No | 6333 (coordinator) — RAG vector search | +| Ollama (worker) | No | 11434 — embeddings only | ### Config quirks diff --git a/CLAUDE.md b/CLAUDE.md index 6e5ac9a..9638f08 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -56,6 +56,9 @@ FTS5 virtual table (`memories`) in SQLite. `search_memories()` uses BM25 ranking - `LLAMA_SERVER_BASE` — `http://192.168.50.108:8081` (coordinator llama-server, RPC offloads to worker GPU) - `SEARXNG_BASE` — `http://localhost:8888` +- `QDRANT_URL` — `http://192.168.50.108:6333` (Qdrant on coordinator) +- `TRIAGE_BASE` — `http://127.0.0.1:8083/v1` (Phi-4-mini) +- `AMQP_URL` — `amqp://caic:{pw}@localhost:5672/caic` (RabbitMQ, pw read from `~/.caic_amqp_secret`) - `PERPLEXITY_THRESHOLD` — `15.0` - `EMBED_URL` — `http://192.168.50.210:11434/api/embeddings` (Ollama on worker) - `VERSION` — current version string @@ -65,7 +68,9 @@ FTS5 virtual table (`memories`) in SQLite. `search_memories()` uses BM25 ranking | Service | Required | Port | |---------|----------|------| | **llama-server** (coordinator) | Yes | 8081 + RPC :50052 (worker GPU) | +| **Phi-4-mini** (triage) | No | 8083 | | **SearXNG** | No | 8888 | +| **RabbitMQ** (coordinator) | No | 5672 — AMQP broker | | **wttr.in** | No | weather shortcut | | **rocm-smi** | No | AMD GPU stats | | **Qdrant** (coordinator) | No | 6333 — RAG vector search | diff --git a/README.md b/README.md index 2c7edc7..2a7fcd3 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ -# cAIc v0.14.0 +# cAIc v0.17.0 You have a garage full of retired office PCs, a GPU that was mid-range when Obama was president, and a burning desire to chat with a language model without renting some billionaire's server farm. Congratulations — you've found your people. @@ -10,7 +10,19 @@ At v1.0, this ships with a Docker compose stack and setup wizard that detect CPU Developer wiki: [docs/wiki/Home.md](docs/wiki/Home.md) -## What's New in v0.14.0 +## What's New in v0.17.0 + +### Dynamic Model Swap — `request_model_swap()`, `select_node()` async (Roadmap N Task 14) +- **`cluster.py`** — `request_model_swap()` publishes `cmd.swap_model` to `jc.admin`; `handle_model_ready()` and `handle_model_failed()` consume `model_ready`/`model_failed` on `jc.system` +- **`select_node()` async** — Queries worker `inventory` for ideal model; triggers swap if model not active, returns `None` for fallback during swap +- **`SUBSCRIBE_TABLE`** — 7 AMQP routing key bindings in cluster.py + +### Cluster Status UI — Heartbeat + Live Status Panel (Roadmap N Task 15) +- **`handle_heartbeat()`** — Consumes `node.*.heartbeat` on `jc.system` to update `last_seen` per node +- **UI cluster panel** — sidebar polls `GET /api/cluster` every 15s; green=active, yellow=swapping, red=error/offline +- **Version bumped to v0.17.0** — All 179 tests pass + +### What's New in v0.14.0 ### Cluster Protocol — `GET /api/cluster`, 9 AMQP Message Types (Roadmap N Task 11) - **`cluster.py`** — Node registry (`CLUSTER_NODES`), bounded event log (`CLUSTER_EVENTS`, max 1000), coordinator auto-promotion @@ -67,30 +79,32 @@ Developer wiki: [docs/wiki/Home.md](docs/wiki/Home.md) /opt/caic/ ├── amqp.py # aio-pika AMQP connection manager + subscribe/rebind ├── app.py # FastAPI app entry point +├── auth.py # PIN-based guest/admin sessions, auth routes ├── cluster.py # Cluster protocol: node registry, event log, ping/pong ├── config.py # Constants, env vars, limits, skill registry ├── db.py # SQLite schema, connection factory -├── auth.py # PIN-based guest/admin sessions, auth routes ├── eviction.py # Score-based RAG eviction engine -├── security.py # Rate limiting, origin checks, IP allowlist, audit -├── memory.py # FTS5 memory CRUD, remember/forget commands -├── search.py # SearXNG integration, perplexity, refusal detection -├── rag.py # Qdrant vector search + system prompt assembly ├── gpu.py # AMD GPU stats via rocm-smi +├── hardware.py # Hardware self-assessment (CPU, RAM, VRAM) +├── memory.py # FTS5 memory CRUD, remember/forget commands +├── rag.py # Qdrant vector search + system prompt assembly +├── search.py # SearXNG integration, perplexity, refusal detection +├── security.py # Rate limiting, origin checks, IP allowlist, audit +├── triage.py # Query classification + cluster node selection ├── routers/ │ ├── chat.py # /api/chat streaming endpoint -│ ├── search_route.py # /api/search explicit search endpoint +│ ├── cluster.py # Cluster status endpoint │ ├── completions.py # /v1/chat/completions OpenAI-compat endpoint │ ├── conversations.py# Conversation CRUD +│ ├── ingest.py # Terminal RAG ingest │ ├── memories.py # Memory CRUD API │ ├── models.py # Model listing, system stats │ ├── presets.py # System prompt presets │ ├── profile.py # User profile +│ ├── search_route.py # /api/search explicit search endpoint │ ├── settings.py # Runtime settings │ ├── skills.py # Skills management -│ ├── upload.py # File attachment endpoints -│ ├── ingest.py # Terminal RAG ingest -│ └── cluster.py # Cluster status endpoint +│ └── upload.py # File attachment endpoints ├── static/ │ └── logo.png # Logo image (optional) ├── templates/ @@ -98,8 +112,7 @@ Developer wiki: [docs/wiki/Home.md](docs/wiki/Home.md) ├── node_agent/ │ ├── agent.py # Standalone worker agent (AMQP client) │ └── requirements.txt -├── triage.py # Query classification + cluster node selection -└── tests/ # 168 pytest tests +└── tests/ # 179 pytest tests ``` ## Requirements @@ -313,7 +326,7 @@ Settings are stored in the `settings` table and include: python3 -m pytest tests/ -v ``` -All 168 tests use `tmp_path` fixtures + monkeypatched `httpx.AsyncClient`/`aio-pika`. No external services needed. +All 179 tests use `tmp_path` fixtures + monkeypatched `httpx.AsyncClient`/`aio-pika`. No external services needed. ## License diff --git a/TASKS.md b/TASKS.md index c2bbee4..0998644 100644 --- a/TASKS.md +++ b/TASKS.md @@ -346,7 +346,7 @@ Declare the two topic exchanges needed by jC: Use `rabbitmqadmin` or `curl` against the management API to declare exchanges. Verify both exchanges appear in: `curl -s -u caic:{password} http://localhost:15672/api/exchanges/caic` -Write the generated RabbitMQ password to `/home/gramps/.jc_amqp_secret` with mode 600. This will be read by jC as an env var source in subsequent tasks. +Write the generated RabbitMQ password to `/home/gramps/.caic_amqp_secret` with mode 600. This will be read by cAIc as an env var source in subsequent tasks. No pytest tests required for this infrastructure task. @@ -359,7 +359,7 @@ This task adds the core AMQP connection manager to jC. It must connect to Rabbit **Add to `requirements.txt`:** `aio-pika>=9.0.0` **Add to `config.py`:** -- `AMQP_URL` — read from env `CAIC_AMQP_URL`, default `amqp://caic:password@localhost:5672/caic`. The actual password comes from `/home/gramps/.jc_amqp_secret` — read it at startup if the env var is not set. +- `AMQP_URL` — read from env `CAIC_AMQP_URL`, default `amqp://caic:password@localhost:5672/caic`. The actual password comes from `/home/gramps/.caic_amqp_secret` — read it at startup if the env var is not set. - `AMQP_RECONNECT_DELAY` — seconds between reconnect attempts, default 5 - `AMQP_EXCHANGE_ADMIN` — `jc.admin` - `AMQP_EXCHANGE_SYSTEM` — `jc.system` @@ -764,7 +764,7 @@ This task wires the cluster into jC's chat flow. When a query arrives at `/api/c **Prerequisites:** Tasks 9–12 complete. At least one worker node admitted to cluster. **Install Phi-4-mini on coordinator (infrastructure step):** -- Download `Phi-4-mini-Instruct-Q4_K_M.gguf` from HuggingFace using `hf download microsoft/Phi-4-mini-instruct --include "*.Q4_K_M.gguf" --local-dir /var/lib/jc/models` +- Download `Phi-4-mini-Instruct-Q4_K_M.gguf` from HuggingFace using `hf download microsoft/Phi-4-mini-instruct --include "*.Q4_K_M.gguf" --local-dir /var/lib/caic/models` - Create `/etc/systemd/system/llama-server-triage.service` — same pattern as existing llama-server service but: port 8083, model path points to Phi-4-mini GGUF, no `--rpc` flag (runs entirely on coordinator CPU/iGPU), description `Llama.cpp Server (Phi-4-mini — triage/routing)` - `systemctl daemon-reload && systemctl enable llama-server-triage && systemctl start llama-server-triage` - Verify: `curl -s http://localhost:8083/v1/models` diff --git a/docker.md b/docker.md index 229f43b..2d3e699 100644 --- a/docker.md +++ b/docker.md @@ -31,7 +31,7 @@ └─────────────────────────────────────────────────────────┘ ``` -> **This compose stack defines the coordinator.** A coordinator runs jC, the broker, and optional infrastructure services. Workers (headless inference nodes) do not use Docker — they install just llama-server + a Python node agent. See §12 for the worker deployment model. +> **This compose stack defines the coordinator.** A coordinator runs cAIc, the broker, and optional infrastructure services. Workers (headless inference nodes) do not use Docker — they install just llama-server + a Python node agent. See §9 for the worker deployment model. ### Service roles @@ -691,7 +691,7 @@ chmod +x llama-server # Install node agent deps pip install aio-pika httpx -# Create node agent script (from repo: tools/node_agent.py) +# Create node agent script (from repo: node_agent/agent.py) # Configure COORDINATOR_AMQP_URL in environment ``` diff --git a/docs/wiki/Developer-Architecture.md b/docs/wiki/Developer-Architecture.md index 81da708..d1c1401 100644 --- a/docs/wiki/Developer-Architecture.md +++ b/docs/wiki/Developer-Architecture.md @@ -22,7 +22,10 @@ Refactored from single-file (`app.py`) into modules under project root: | `rag.py` | Qdrant vector search, system prompt assembly, chunk_text() helper, collection stats | | `eviction.py` | Score-based RAG eviction engine (extracted from rag.py) | | `gpu.py` | AMD GPU stats via rocm-smi | -| `amqp.py` | (WIP) aio-pika connection manager for RabbitMQ | +| `hardware.py` | Hardware self-assessment — CPU, RAM, VRAM, service health probes | +| `amqp.py` | aio-pika connection manager for RabbitMQ (connect, disconnect, publish, subscribe, auto-reconnect) | +| `cluster.py` | Cluster node registry, event log, coordinator election, ping/pong, model swap handlers | +| `triage.py` | Phi-4-mini query classification + `select_node()` for cluster routing | | `routers/` | One module per endpoint group | ### 1.2 External Services @@ -184,13 +187,13 @@ cAIc uses a **broker-mediated** cluster design. This is the preferred architectu - A single RabbitMQ broker (or clustered set of brokers) acts as the central nervous system - **Coordinator nodes** run the FastAPI app, host the HTTP API/UI, and publish commands to the broker - **Worker nodes** connect as AMQP *clients only* — they consume commands and publish status events, but run no broker software themselves -- Communication is asynchronous and persistent: each node opens a TCP connection on startup and keeps it alive. The AMQP-0-9-1 heartbeat detects silent failures within ~60s. +- Communication is asynchronous and persistent: each node opens a TCP connection on startup and keeps it alive. The coordinator probes worker health via on-demand AMQP ping/pong messages (5s timeout) rather than relying on the AMQP-0-9-1 transport-level heartbeat. **Why broker-mediated:** - Workers are heterogeneous (different GPUs, different models, ARM vs x86) — no assumption of uniform software - Workers are lightweight — a Raspberry Pi with a USB AI accelerator can participate without running a broker - The coordinator delegates work via messages, not by SSH'ing into workers or requiring shared filesystems -- Failure is isolated: a crashed worker drops off the heartbeat list; the coordinator reassigns its work +- Failure is isolated: a crashed worker stops responding to ping; the coordinator auto-deregisters it and reassigns its work **What it is NOT:** - Not a service mesh — workers do not run identical software stacks @@ -219,17 +222,18 @@ Every physical machine in the cluster is classified by which services it runs. T ``` Coordinator Worker(s) -┌────────────────────┐ ┌─────────────────────────┐ -│ cAIc │ │ llama-server │ -│ (FastAPI + SQLite)│ │ (inference) │ -│ RabbitMQ server │◄──AMQP───────│ aio-pika (agent) │ -│ SearXNG (opt) │ persistent │ ROCm / CUDA (if GPU) │ -│ Qdrant (opt) │ TCP │ │ -│ Ollama (opt) │ conn │ No broker │ -│ llama-server(opt) │ │ No jC │ -└────────────────────┘ │ No DB │ - │ No search/vector │ - └─────────────────────────┘ +┌────────────────────┐ ┌──────────────────────────┐ +│ cAIc │ │ llama-server │ +│ (FastAPI + SQLite)│ │ (inference) │ +│ RabbitMQ server │◄──AMQP───────│ aio-pika (agent) │ +│ SearXNG (opt) │ persistent │ ROCm / CUDA (if GPU) │ +│ Qdrant (opt) │ TCP │ Ollama (embeddings,opt) │ +│ llama-server(opt) │ conn │ │ +└────────────────────┘ │ No broker │ + │ No cAIc │ + │ No DB │ + │ No search/vector │ + └──────────────────────────┘ ``` ### 6.4 RabbitMQ Topology @@ -238,14 +242,10 @@ Every RabbitMQ server belongs to a cluster. Currently only the coordinator runs | Exchange | Type | Purpose | |----------|------|---------| -| `jc.admin` | topic | Commands: swap model, shutdown, heartbeat request | -| `jc.system` | topic | Events: model_ready, model_failed, heartbeat, registration | +| `jc.admin` | topic | Lifecycle commands: register, deregister, ping, pong, admitted, rejected; model commands: cmd.swap_model | +| `jc.system` | topic | Events: model_ready, model_failed, node.*.heartbeat, event; coordinator queries: coord_query, coord_response | -Pending implementation (Tasks 10–15): -- `amqp.py` — aio-pika connection manager with reconnect -- Node agent on worker — registration, heartbeat, command consumer -- `triage.py` — Phi-4-mini query classification (general/code/search/rag) -- Dynamic model swap via llama-server RPC +All exchanges, queues, and bindings are declared by `amqp.py` at startup. Worker runs `node_agent/agent.py` which connects as an AMQP client, registers, responds to ping, and handles model swap commands. ## 7. SSE Protocol @@ -265,28 +265,33 @@ All streaming endpoints yield `data: {json}\n\n`: - No live external services required - Test factories reset `SESSIONS`, `PIN_ATTEMPTS`, `RATE_EVENTS` globals per test -### 8.2 Test Coverage Areas (132 tests) +### 8.2 Test Coverage Areas (179 tests) | Test file | Coverage | |-----------|----------| | test_auth_capabilities.py | Guest/admin sessions, origin blocking, logout | | test_chat_streaming_and_memory_paths.py | Streaming, auto-search, remember/forget, upload context injection | +| test_cluster.py | Registration, deregistration, pong, events, coordinator query | +| test_cluster_heartbeat.py | Heartbeat handler, known/unknown node | | test_completions.py | API key auth, FIM, streaming, blocking, errors | | test_conversations.py | Full CRUD, guest admin, attachment_count | | test_ingest.py | Bearer auth, chunk/embed/upsert, validation | +| test_ip_allowlist.py | IP allowlist helper + middleware | | test_memories.py | Edit, search, stats | +| test_model_swap.py | request_model_swap, handle_model_ready/failed, select_node swap triggering | | test_models_router.py | Models list, ps, show, stats, search/status | +| test_node_agent.py | Node agent registration, ping/pong, model swap | | test_presets.py | Full CRUD, default preset protection | | test_profile.py | Get, update, default, length validation | | test_rag_management.py | Collection stats, eviction algorithm (pinned/grace/scoring/batch), maybe_evict hysteresis, operational stats, flush, concurrency lock | +| test_rate_and_payload_guardrails.py | Rate limits + payload size | | test_search_route.py | Explicit search flow, no results, errors | | test_search_url_sanitization.py | URL sanitizer | | test_settings_allowlist.py | Allowlisted key enforcement | | test_skills_framework.py | List, toggle, unknown skill, prompt injection | -| test_ip_allowlist.py | IP allowlist helper + middleware | -| test_rate_and_payload_guardrails.py | Rate limits + payload size | -| test_error_envelopes.py | Global exception handler + stream errors | +| test_triage.py | classify_query, select_node, get_inference_url | | test_upload.py | Upload, delete, link, by-conversation, attachment_count | +| test_error_envelopes.py | Global exception handler + stream errors | ### 8.3 DoD Process diff --git a/docs/wiki/Home.md b/docs/wiki/Home.md index 2777ae3..c536f8b 100644 --- a/docs/wiki/Home.md +++ b/docs/wiki/Home.md @@ -1,6 +1,6 @@ -# JarvisChat Developer Wiki +# cAIc Developer Wiki -This wiki is the developer-facing architecture and process reference for JarvisChat. +This wiki is the developer-facing architecture and process reference for cAIc. ## Audience @@ -14,7 +14,7 @@ This wiki is the developer-facing architecture and process reference for JarvisC ## Scope and Support Model -JarvisChat is designed for local and trusted-LAN operation. +cAIc is designed for local and trusted-LAN operation. The code may technically function against external or commercial endpoints, but this deployment mode is not a supported target in this project. diff --git a/docs/wiki/current-wip.md b/docs/wiki/current-wip.md index a940ab3..937f99c 100644 --- a/docs/wiki/current-wip.md +++ b/docs/wiki/current-wip.md @@ -4,20 +4,11 @@ Last updated: 2026-07-06 Owner: Gramps Scope: Active roadmap items and backlog. -## Active Roadmap — Roadmap N: AMQP Cluster Nervous System +## ~~Roadmap N: AMQP Cluster Nervous System [COMPLETE]~~ -| Task | Description | Status | -|------|-------------|--------| -| Task 8 | RAG Corpus Management (score-based eviction, pinned sources, operational stats) | **DONE** | -| Task 9 | RabbitMQ install + exchange setup on coordinator | **DONE** | -| Task 10 | AMQP connection layer in jC (`amqp.py`, aio-pika) | **NEXT** | -| Task 11 | Node agent on worker — registration, heartbeat, command listeners | Pending | -| Task 12 | AMQP wiring: inject context into chat/completions pipeline | Pending | -| Task 13 | Query triage via Phi-4-mini (`triage.py`) | Pending | -| Task 14 | Dynamic model swap — publish cmd, handle ready/failed | Pending | -| Task 15 | Cluster status UI panel in templates/index.html | Pending | +All 15 tasks are done. Final commit: `f0689ac feat: Roadmap N — AMQP cluster nervous system complete`. -## Backlog (Post-Roadmap N) +## Backlog - B1 — Context loss in follow-up questions (investigation) - B2 — Bang-prefixed (`!`) search routing