gramps/cAIc
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

docs: sync all docs through v0.17.0 — Roadmap N complete, cluster/AMQP fully documented

Browse Source
This commit is contained in:
gramps
2026-07-09 09:15:08 -07:00
parent f0689ac12b
commit fbacb1861d
8 changed files with 76 additions and 59 deletions
Download Patch File Download Diff File Expand all files Collapse all files
AGENTS.md
+3
View File
@@ -59,6 +59,7 @@ Refactored from single-file (`app.py`) into modules under project root:
| `memory.py` | FTS5 memory CRUD, remember/forget command parsing |
| `search.py` | SearXNG integration, perplexity scoring, refusal detection |
| `rag.py` | Qdrant vector search + system prompt assembly + chunk_text() helper |
| `eviction.py` | Score-based RAG eviction engine |
| `gpu.py` | AMD GPU stats via `rocm-smi` |
| `triage.py` | Phi-4-mini-based query classification + cluster node selection |
| `cluster.py` | Cluster node registry, event log, coordinator election, ping/pong, model swap handlers |
@@ -108,9 +109,11 @@ The upstream request includes `"logprobs": true`. `parse_llama_stream_chunk()` e
| llama-server (coordinator) | Yes | 8081 + RPC :50052 (worker GPU) |
| Phi-4-mini (triage) | No | 8083 |
| SearXNG | No | 8888 |
| RabbitMQ (coordinator) | No | 5672 — AMQP broker |
| wttr.in | No | weather shortcut |
| rocm-smi | No | AMD GPU stats |
| Qdrant | No | 6333 (coordinator) — RAG vector search |
| Ollama (worker) | No | 11434 — embeddings only |
### Config quirks
CLAUDE.md
+5
View File
@@ -56,6 +56,9 @@ FTS5 virtual table (`memories`) in SQLite. `search_memories()` uses BM25 ranking
- `LLAMA_SERVER_BASE` — `http://192.168.50.108:8081` (coordinator llama-server, RPC offloads to worker GPU)
- `SEARXNG_BASE` — `http://localhost:8888`
- `QDRANT_URL` — `http://192.168.50.108:6333` (Qdrant on coordinator)
- `TRIAGE_BASE` — `http://127.0.0.1:8083/v1` (Phi-4-mini)
- `AMQP_URL` — `amqp://caic:{pw}@localhost:5672/caic` (RabbitMQ, pw read from `~/.caic_amqp_secret`)
- `PERPLEXITY_THRESHOLD` — `15.0`
- `EMBED_URL` — `http://192.168.50.210:11434/api/embeddings` (Ollama on worker)
- `VERSION` — current version string
@@ -65,7 +68,9 @@ FTS5 virtual table (`memories`) in SQLite. `search_memories()` uses BM25 ranking
| Service | Required | Port |
|---------|----------|------|
| **llama-server** (coordinator) | Yes | 8081 + RPC :50052 (worker GPU) |
| **Phi-4-mini** (triage) | No | 8083 |
| **SearXNG** | No | 8888 |
| **RabbitMQ** (coordinator) | No | 5672 — AMQP broker |
| **wttr.in** | No | weather shortcut |
| **rocm-smi** | No | AMD GPU stats |
| **Qdrant** (coordinator) | No | 6333 — RAG vector search |
README.md
+27 -14
View File
@@ -1,4 +1,4 @@
# cAIc v0.14.0
# cAIc v0.17.0
You have a garage full of retired office PCs, a GPU that was mid-range when Obama was president, and a burning desire to chat with a language model without renting some billionaire's server farm. Congratulations — you've found your people.
@@ -10,7 +10,19 @@ At v1.0, this ships with a Docker compose stack and setup wizard that detect CPU
Developer wiki: [docs/wiki/Home.md](docs/wiki/Home.md)
## What's New in v0.14.0
## What's New in v0.17.0
### Dynamic Model Swap — `request_model_swap()`, `select_node()` async (Roadmap N Task 14)
- **`cluster.py`** — `request_model_swap()` publishes `cmd.swap_model` to `jc.admin`; `handle_model_ready()` and `handle_model_failed()` consume `model_ready`/`model_failed` on `jc.system`
- **`select_node()` async** — Queries worker `inventory` for ideal model; triggers swap if model not active, returns `None` for fallback during swap
- **`SUBSCRIBE_TABLE`** — 7 AMQP routing key bindings in cluster.py
### Cluster Status UI — Heartbeat + Live Status Panel (Roadmap N Task 15)
- **`handle_heartbeat()`** — Consumes `node.*.heartbeat` on `jc.system` to update `last_seen` per node
- **UI cluster panel** — sidebar polls `GET /api/cluster` every 15s; green=active, yellow=swapping, red=error/offline
- **Version bumped to v0.17.0** — All 179 tests pass
### What's New in v0.14.0
### Cluster Protocol — `GET /api/cluster`, 9 AMQP Message Types (Roadmap N Task 11)
- **`cluster.py`** — Node registry (`CLUSTER_NODES`), bounded event log (`CLUSTER_EVENTS`, max 1000), coordinator auto-promotion
@@ -67,30 +79,32 @@ Developer wiki: [docs/wiki/Home.md](docs/wiki/Home.md)
/opt/caic/
├── amqp.py # aio-pika AMQP connection manager + subscribe/rebind
├── app.py # FastAPI app entry point
├── auth.py # PIN-based guest/admin sessions, auth routes
├── cluster.py # Cluster protocol: node registry, event log, ping/pong
├── config.py # Constants, env vars, limits, skill registry
├── db.py # SQLite schema, connection factory
├── auth.py # PIN-based guest/admin sessions, auth routes
├── eviction.py # Score-based RAG eviction engine
├── security.py # Rate limiting, origin checks, IP allowlist, audit
├── memory.py # FTS5 memory CRUD, remember/forget commands
├── search.py # SearXNG integration, perplexity, refusal detection
├── rag.py # Qdrant vector search + system prompt assembly
├── gpu.py # AMD GPU stats via rocm-smi
├── hardware.py # Hardware self-assessment (CPU, RAM, VRAM)
├── memory.py # FTS5 memory CRUD, remember/forget commands
├── rag.py # Qdrant vector search + system prompt assembly
├── search.py # SearXNG integration, perplexity, refusal detection
├── security.py # Rate limiting, origin checks, IP allowlist, audit
├── triage.py # Query classification + cluster node selection
├── routers/
│ ├── chat.py # /api/chat streaming endpoint
│ ├── search_route.py # /api/search explicit search endpoint
│ ├── cluster.py # Cluster status endpoint
│ ├── completions.py # /v1/chat/completions OpenAI-compat endpoint
│ ├── conversations.py# Conversation CRUD
│ ├── ingest.py # Terminal RAG ingest
│ ├── memories.py # Memory CRUD API
│ ├── models.py # Model listing, system stats
│ ├── presets.py # System prompt presets
│ ├── profile.py # User profile
│ ├── search_route.py # /api/search explicit search endpoint
│ ├── settings.py # Runtime settings
│ ├── skills.py # Skills management
│ ├── upload.py # File attachment endpoints
│ ├── ingest.py # Terminal RAG ingest
│ └── cluster.py # Cluster status endpoint
│ └── upload.py # File attachment endpoints
├── static/
│ └── logo.png # Logo image (optional)
├── templates/
@@ -98,8 +112,7 @@ Developer wiki: [docs/wiki/Home.md](docs/wiki/Home.md)
├── node_agent/
│ ├── agent.py # Standalone worker agent (AMQP client)
│ └── requirements.txt
├── triage.py # Query classification + cluster node selection
└── tests/ # 168 pytest tests
└── tests/ # 179 pytest tests
```
## Requirements
@@ -313,7 +326,7 @@ Settings are stored in the `settings` table and include:
python3 -m pytest tests/ -v
```
All 168 tests use `tmp_path` fixtures + monkeypatched `httpx.AsyncClient`/`aio-pika`. No external services needed.
All 179 tests use `tmp_path` fixtures + monkeypatched `httpx.AsyncClient`/`aio-pika`. No external services needed.
## License
TASKS.md
+3 -3
View File
@@ -346,7 +346,7 @@ Declare the two topic exchanges needed by jC:
Use `rabbitmqadmin` or `curl` against the management API to declare exchanges. Verify both exchanges appear in: `curl -s -u caic:{password} http://localhost:15672/api/exchanges/caic`
Write the generated RabbitMQ password to `/home/gramps/.jc_amqp_secret` with mode 600. This will be read by jC as an env var source in subsequent tasks.
Write the generated RabbitMQ password to `/home/gramps/.caic_amqp_secret` with mode 600. This will be read by cAIc as an env var source in subsequent tasks.
No pytest tests required for this infrastructure task.
@@ -359,7 +359,7 @@ This task adds the core AMQP connection manager to jC. It must connect to Rabbit
**Add to `requirements.txt`:** `aio-pika>=9.0.0`
**Add to `config.py`:**
- `AMQP_URL` — read from env `CAIC_AMQP_URL`, default `amqp://caic:password@localhost:5672/caic`. The actual password comes from `/home/gramps/.jc_amqp_secret` — read it at startup if the env var is not set.
- `AMQP_URL` — read from env `CAIC_AMQP_URL`, default `amqp://caic:password@localhost:5672/caic`. The actual password comes from `/home/gramps/.caic_amqp_secret` — read it at startup if the env var is not set.
- `AMQP_RECONNECT_DELAY` — seconds between reconnect attempts, default 5
- `AMQP_EXCHANGE_ADMIN` — `jc.admin`
- `AMQP_EXCHANGE_SYSTEM` — `jc.system`
@@ -764,7 +764,7 @@ This task wires the cluster into jC's chat flow. When a query arrives at `/api/c
**Prerequisites:** Tasks 9–12 complete. At least one worker node admitted to cluster.
**Install Phi-4-mini on coordinator (infrastructure step):**
- Download `Phi-4-mini-Instruct-Q4_K_M.gguf` from HuggingFace using `hf download microsoft/Phi-4-mini-instruct --include "*.Q4_K_M.gguf" --local-dir /var/lib/jc/models`
- Download `Phi-4-mini-Instruct-Q4_K_M.gguf` from HuggingFace using `hf download microsoft/Phi-4-mini-instruct --include "*.Q4_K_M.gguf" --local-dir /var/lib/caic/models`
- Create `/etc/systemd/system/llama-server-triage.service` — same pattern as existing llama-server service but: port 8083, model path points to Phi-4-mini GGUF, no `--rpc` flag (runs entirely on coordinator CPU/iGPU), description `Llama.cpp Server (Phi-4-mini — triage/routing)`
- `systemctl daemon-reload && systemctl enable llama-server-triage && systemctl start llama-server-triage`
- Verify: `curl -s http://localhost:8083/v1/models`
docker.md
+2 -2
View File
@@ -31,7 +31,7 @@
└─────────────────────────────────────────────────────────┘
```
> **This compose stack defines the coordinator.** A coordinator runs jC, the broker, and optional infrastructure services. Workers (headless inference nodes) do not use Docker — they install just llama-server + a Python node agent. See §12 for the worker deployment model.
> **This compose stack defines the coordinator.** A coordinator runs cAIc, the broker, and optional infrastructure services. Workers (headless inference nodes) do not use Docker — they install just llama-server + a Python node agent. See §9 for the worker deployment model.
### Service roles
@@ -691,7 +691,7 @@ chmod +x llama-server
# Install node agent deps
pip install aio-pika httpx
# Create node agent script (from repo: tools/node_agent.py)
# Create node agent script (from repo: node_agent/agent.py)
# Configure COORDINATOR_AMQP_URL in environment
```
docs/wiki/Developer-Architecture.md
+25 -20
View File
@@ -22,7 +22,10 @@ Refactored from single-file (`app.py`) into modules under project root:
| `rag.py` | Qdrant vector search, system prompt assembly, chunk_text() helper, collection stats |
| `eviction.py` | Score-based RAG eviction engine (extracted from rag.py) |
| `gpu.py` | AMD GPU stats via rocm-smi |
| `amqp.py` | (WIP) aio-pika connection manager for RabbitMQ |
| `hardware.py` | Hardware self-assessment — CPU, RAM, VRAM, service health probes |
| `amqp.py` | aio-pika connection manager for RabbitMQ (connect, disconnect, publish, subscribe, auto-reconnect) |
| `cluster.py` | Cluster node registry, event log, coordinator election, ping/pong, model swap handlers |
| `triage.py` | Phi-4-mini query classification + `select_node()` for cluster routing |
| `routers/` | One module per endpoint group |
### 1.2 External Services
@@ -184,13 +187,13 @@ cAIc uses a **broker-mediated** cluster design. This is the preferred architectu
- A single RabbitMQ broker (or clustered set of brokers) acts as the central nervous system
- **Coordinator nodes** run the FastAPI app, host the HTTP API/UI, and publish commands to the broker
- **Worker nodes** connect as AMQP *clients only* — they consume commands and publish status events, but run no broker software themselves
- Communication is asynchronous and persistent: each node opens a TCP connection on startup and keeps it alive. The AMQP-0-9-1 heartbeat detects silent failures within ~60s.
- Communication is asynchronous and persistent: each node opens a TCP connection on startup and keeps it alive. The coordinator probes worker health via on-demand AMQP ping/pong messages (5s timeout) rather than relying on the AMQP-0-9-1 transport-level heartbeat.
**Why broker-mediated:**
- Workers are heterogeneous (different GPUs, different models, ARM vs x86) — no assumption of uniform software
- Workers are lightweight — a Raspberry Pi with a USB AI accelerator can participate without running a broker
- The coordinator delegates work via messages, not by SSH'ing into workers or requiring shared filesystems
- Failure is isolated: a crashed worker drops off the heartbeat list; the coordinator reassigns its work
- Failure is isolated: a crashed worker stops responding to ping; the coordinator auto-deregisters it and reassigns its work
**What it is NOT:**
- Not a service mesh — workers do not run identical software stacks
@@ -219,17 +222,18 @@ Every physical machine in the cluster is classified by which services it runs. T
```
Coordinator Worker(s)
┌────────────────────┐ ┌─────────────────────────┐
┌────────────────────┐ ┌──────────────────────────┐
│ cAIc │ │ llama-server │
│ (FastAPI + SQLite)│ │ (inference) │
│ RabbitMQ server │◄──AMQP───────│ aio-pika (agent) │
│ SearXNG (opt) │ persistent │ ROCm / CUDA (if GPU) │
│ Qdrant (opt) │ TCP │ │
│ Ollama (opt) │ conn │ No broker │
│ llama-server(opt) │ │ No jC │
└────────────────────┘ │ No DB │
│ Qdrant (opt) │ TCP │ Ollama (embeddings,opt) │
│ llama-server(opt) │ conn │ │
└────────────────────┘ │ No broker │
│ No cAIc │
│ No DB │
│ No search/vector │
└─────────────────────────┘
└──────────────────────────┘
```
### 6.4 RabbitMQ Topology
@@ -238,14 +242,10 @@ Every RabbitMQ server belongs to a cluster. Currently only the coordinator runs
| Exchange | Type | Purpose |
|----------|------|---------|
| `jc.admin` | topic | Commands: swap model, shutdown, heartbeat request |
| `jc.system` | topic | Events: model_ready, model_failed, heartbeat, registration |
| `jc.admin` | topic | Lifecycle commands: register, deregister, ping, pong, admitted, rejected; model commands: cmd.swap_model |
| `jc.system` | topic | Events: model_ready, model_failed, node.*.heartbeat, event; coordinator queries: coord_query, coord_response |
Pending implementation (Tasks 10–15):
- `amqp.py` — aio-pika connection manager with reconnect
- Node agent on worker — registration, heartbeat, command consumer
- `triage.py` — Phi-4-mini query classification (general/code/search/rag)
- Dynamic model swap via llama-server RPC
All exchanges, queues, and bindings are declared by `amqp.py` at startup. Worker runs `node_agent/agent.py` which connects as an AMQP client, registers, responds to ping, and handles model swap commands.
## 7. SSE Protocol
@@ -265,28 +265,33 @@ All streaming endpoints yield `data: {json}\n\n`:
- No live external services required
- Test factories reset `SESSIONS`, `PIN_ATTEMPTS`, `RATE_EVENTS` globals per test
### 8.2 Test Coverage Areas (132 tests)
### 8.2 Test Coverage Areas (179 tests)
| Test file | Coverage |
|-----------|----------|
| test_auth_capabilities.py | Guest/admin sessions, origin blocking, logout |
| test_chat_streaming_and_memory_paths.py | Streaming, auto-search, remember/forget, upload context injection |
| test_cluster.py | Registration, deregistration, pong, events, coordinator query |
| test_cluster_heartbeat.py | Heartbeat handler, known/unknown node |
| test_completions.py | API key auth, FIM, streaming, blocking, errors |
| test_conversations.py | Full CRUD, guest admin, attachment_count |
| test_ingest.py | Bearer auth, chunk/embed/upsert, validation |
| test_ip_allowlist.py | IP allowlist helper + middleware |
| test_memories.py | Edit, search, stats |
| test_model_swap.py | request_model_swap, handle_model_ready/failed, select_node swap triggering |
| test_models_router.py | Models list, ps, show, stats, search/status |
| test_node_agent.py | Node agent registration, ping/pong, model swap |
| test_presets.py | Full CRUD, default preset protection |
| test_profile.py | Get, update, default, length validation |
| test_rag_management.py | Collection stats, eviction algorithm (pinned/grace/scoring/batch), maybe_evict hysteresis, operational stats, flush, concurrency lock |
| test_rate_and_payload_guardrails.py | Rate limits + payload size |
| test_search_route.py | Explicit search flow, no results, errors |
| test_search_url_sanitization.py | URL sanitizer |
| test_settings_allowlist.py | Allowlisted key enforcement |
| test_skills_framework.py | List, toggle, unknown skill, prompt injection |
| test_ip_allowlist.py | IP allowlist helper + middleware |
| test_rate_and_payload_guardrails.py | Rate limits + payload size |
| test_error_envelopes.py | Global exception handler + stream errors |
| test_triage.py | classify_query, select_node, get_inference_url |
| test_upload.py | Upload, delete, link, by-conversation, attachment_count |
| test_error_envelopes.py | Global exception handler + stream errors |
### 8.3 DoD Process
docs/wiki/Home.md
+3 -3
View File
@@ -1,6 +1,6 @@
# JarvisChat Developer Wiki
# cAIc Developer Wiki
This wiki is the developer-facing architecture and process reference for JarvisChat.
This wiki is the developer-facing architecture and process reference for cAIc.
## Audience
@@ -14,7 +14,7 @@ This wiki is the developer-facing architecture and process reference for JarvisC
## Scope and Support Model
JarvisChat is designed for local and trusted-LAN operation.
cAIc is designed for local and trusted-LAN operation.
The code may technically function against external or commercial endpoints, but this deployment mode is not a supported target in this project.
docs/wiki/current-wip.md
+3 -12
View File
@@ -4,20 +4,11 @@ Last updated: 2026-07-06
Owner: Gramps
Scope: Active roadmap items and backlog.
## Active Roadmap — Roadmap N: AMQP Cluster Nervous System
## ~~Roadmap N: AMQP Cluster Nervous System [COMPLETE]~~
| Task | Description | Status |
|------|-------------|--------|
| Task 8 | RAG Corpus Management (score-based eviction, pinned sources, operational stats) | **DONE** |
| Task 9 | RabbitMQ install + exchange setup on coordinator | **DONE** |
| Task 10 | AMQP connection layer in jC (`amqp.py`, aio-pika) | **NEXT** |
| Task 11 | Node agent on worker — registration, heartbeat, command listeners | Pending |
| Task 12 | AMQP wiring: inject context into chat/completions pipeline | Pending |
| Task 13 | Query triage via Phi-4-mini (`triage.py`) | Pending |
| Task 14 | Dynamic model swap — publish cmd, handle ready/failed | Pending |
| Task 15 | Cluster status UI panel in templates/index.html | Pending |
All 15 tasks are done. Final commit: `f0689ac feat: Roadmap N — AMQP cluster nervous system complete`.
## Backlog (Post-Roadmap N)
## Backlog
- B1 — Context loss in follow-up questions (investigation)
- B2 — Bang-prefixed (`!`) search routing