cAIc/ai.md

# cAIc — Agents Guide

## Run

```bash
# Docker (recommended)
scripts/setup.sh && docker compose up -d

# Bare-metal
uvicorn app:app --host 0.0.0.0 --port 8080 --reload
```

## Tests

```bash
python3 -m pytest tests/ -v
```

All tests use `tmp_path` fixtures + monkeypatched `httpx.AsyncClient.stream/get/post/put`. No external services needed. Test factories reset `SESSIONS`, `PIN_ATTEMPTS`, `RATE_EVENTS` globals — be careful not to let test state leak. Tests import directly from the correct modules (`db`, `security`, `config`, `search`, `rag`, `memory`, `routers.*`).

Every router has a dedicated test file:
| File | Covers |
|------|--------|
| `test_auth_capabilities.py` | `auth.py` — guest/admin sessions, origin blocking, logout |
| `test_chat_streaming_and_memory_paths.py` | `routers/chat.py` — streaming, auto-search, remember/forget, upload context injection |
| `test_completions.py` | `routers/completions.py` — API key auth, FIM, streaming, blocking, errors |
| `test_conversations.py` | `routers/conversations.py` — full CRUD, guest admin enforcement, attachment_count |
| `test_ingest.py` | `routers/ingest.py` — Bearer auth, chunk/embed/upsert, validation |
| `test_memories.py` | `routers/memories.py` — edit, search, stats endpoints |
| `test_models_router.py` | `routers/models.py` — models list, ps, show, stats, search/status |
| `test_presets.py` | `routers/presets.py` — full CRUD, default preset protection |
| `test_profile.py` | `routers/profile.py` — get, update, default, length validation |
| `test_rag_management.py` | `eviction.py` + `routers/rag_admin.py` — eviction engine, stats, flush, browse, search, edit, delete individual points |
| `test_search_route.py` | `routers/search_route.py` — explicit search flow, no results, errors |
| `test_search_url_sanitization.py` | `search.py` URL sanitizer |
| `test_cluster.py` | `cluster.py` — registration, deregistration, pong, events, coordinator query |
| `test_cluster_heartbeat.py` | `cluster.py` — heartbeat handler, known/unknown node |
| `test_model_swap.py` | `cluster.py` + `triage.py` — request_model_swap, handle_model_ready/failed, select_node swap triggering |
| `test_node_agent.py` | `node_agent/agent.py` — registration, ping/pong, model swap |
| `test_triage.py` | `triage.py` — classify_query, select_node, get_inference_url |
| `test_settings_allowlist.py` | `routers/settings.py` — allowlisted key enforcement |
| `test_skills_framework.py` | `routers/skills.py` — list, toggle, unknown skill, prompt injection |
| `test_ip_allowlist.py` | IP allowlist helper + middleware |
| `test_rate_and_payload_guardrails.py` | Rate limits + payload size enforcement |
| `test_error_envelopes.py` | Global exception handler + stream error incidents |
| `test_upload.py` | `routers/upload.py` — upload, delete, link, by-conversation, attachment_count integration |

Modules that call `httpx.AsyncClient` (chat, completions, models, search_route, upload, ingest, model_pull)
are mocked via `monkeypatch.setattr` on `AsyncClient.stream`, `.get`, or `.post`.
CPU stats in `models.py` (`api/stats`) use real `psutil`; GPU stats are
monkeypatched via `routers.models.get_gpu_stats`.

## Architecture

Refactored from single-file (`app.py`) into modules under project root:

| File | Role |
|------|------|
| `app.py` | FastAPI app, middleware, router registration |
| `config.py` | Constants, env vars, rate/payload limits, built-in skills registry, upload limits |
| `db.py` | SQLite schema, connection factory, settings helpers, upload_context CRUD |
| `auth.py` | PIN-based guest/admin sessions, auth routes |
| `security.py` | Rate limiting, origin checks, IP allowlist, audit/incident logging |
| `memory.py` | FTS5 memory CRUD (encrypted facts, Python-side matching), remember/forget command parsing |
| `search.py` | SearXNG integration, perplexity scoring, refusal detection |
| `rag.py` | Qdrant vector search (encrypted payload text) + system prompt assembly + chunk_text() helper |
| `eviction.py` | Score-based RAG eviction engine |
| `gpu.py` | GPU stats — `rocm-smi` (AMD/Linux) or `system_profiler` (Apple Silicon/macOS) |
| `crypto.py` | AES-256-GCM encrypt/decrypt + key management (stored as `heartbeat_interval_ms` in settings) |
| `model_pull.py` | Startup model availability check + Ollama pull API |
| `triage.py` | Phi-4-mini-based query classification + cluster node selection |
| `cluster.py` | Cluster node registry, event log, coordinator election, ping/pong, model swap handlers |
| `amqp.py` | AMQP connection manager — connect, disconnect, publish, subscribe, auto-reconnect |
| `node_agent/` | Standalone worker agent — AMQP client for registration, ping/pong, model swap |
| `routers/` | One module per endpoint group (chat, search, skills, completions, upload, ingest) |

### Entrypoint / API keys

- `app.py` line 148: `uvicorn.run(app, ...)` when called directly
- `config.py` line 14: `LLAMA_SERVER_BASE` defaults to `http://localhost:8081` — configurable via env var; Docker uses `http://llama-server:8081`
- `config.py` line 17: `DEFAULT_MODEL` read from `CAIC_DEFAULT_MODEL` env var or defaults to `qwen2.5-7b-instruct`
- `config.py` line 18: `COMPLETIONS_API_KEY` read from `CAIC_COMPLETIONS_API_KEY` env var or auto-generates

### Key flows

1. **`/api/chat`** → `process_remember_command()` intercepts "remember that..." / "forget about..." first → optional `upload_context_id` fetches document text from SQLite → `build_system_prompt()` (profile + FTS5 memory + Qdrant RAG + preset + skills + uploaded doc) → triage classifies query (general/code/search/rag) → `select_node()` picks best worker → stream from chosen node with `logprobs: true` → if perplexity > 15.0 OR `REFUSAL_PATTERNS` match, re-query with SearXNG results
2. **`/api/search`** → bypasses perplexity/refusal, queries SearXNG directly → summarizes via llama-server
3. **`/v1/chat/completions`** → OpenAI-compatible for Continue.dev/IDE integration; FIM requests proxied without persistence
4. **`/api/upload`** → multipart file upload, PDF/text extraction, `mode=(context|ingest|both)`, stores SQLite context (1hr expiry) + Qdrant upsert
5. **`/api/ingest`** → Bearer token auth, programmatic RAG ingest (terminal hook, external tools)

### Perplexity / auto-search

The upstream request includes `"logprobs": true`. `parse_llama_stream_chunk()` extracts per-token logprobs from each chunk's `choices[0].logprobs.content[].logprob`. The `all_logprobs` list is populated during streaming, so `calculate_perplexity()` and `is_uncertain()` work correctly.

### Auth / lockdown

- Guest session by default (`POST /api/auth/guest`), admin unlock via 4-digit PIN (`POST /api/auth/login`)
- Admin required for PUT/DELETE/PATCH + all POST except allowlist (`/api/chat`, `/api/search`, `/api/auth/*`)
- `/api/ingest` is exempt from session auth — self-authenticates via Bearer token
- IP allowlist, rate limiting, origin checking, payload size limits — all enforced in `app.py` middleware
- Origin check applies to **all** `/api/` requests; returns `False` when both `Origin` and `Referer` are absent
- `CAIC_ADMIN_PIN` env var required on first boot (or `CAIC_ALLOW_DEFAULT_PIN=true`)

### Database

- SQLite at `caic.db`, auto-created by `init_db()` on startup via FastAPI `lifespan`
- `get_db()` opens new connection per request (no pool). Close after use.
- FTS5 virtual table `memories` for full-text search with BM25 ranking.
- `upload_context` table: auto-expiring document storage for chat context injection.

### External services

All services are available bare-metal or as containers in `docker compose up`.

| Service | Required | Port | Docker service name |
|---------|----------|------|---------------------|
| llama-server (coordinator) | Yes | 8081 + RPC :50052 (worker GPU) | `llama-server` |
| Phi-4-mini (triage) | No | 8083 | — |
| SearXNG | No | 8888 | `searxng` |
| RabbitMQ (coordinator) | No | 5672 — AMQP broker | `rabbitmq` |
| wttr.in | No | weather shortcut | — |
| rocm-smi | No | AMD GPU stats | — |
| Qdrant | No | 6333 (coordinator) — RAG vector search | `qdrant` |
| Ollama (worker) | No | 11434 — embeddings + model pull | `ollama` |

### Config quirks

- `BODY_LIMIT_UPLOAD_BYTES` = 20MB for `/api/upload`; other paths use smaller limits
- `SUPPORTED_UPLOAD_TYPES` includes images (png/jpeg/gif/svg/webp) + text + PDF + JSON
- `UPLOAD_CONTEXT_EXPIRY_HOURS` = 1 hour
- Rate limits and payload caps in `config.py` — patch `security.RL_*` not `config.RL_*` for tests
- RAG embedding requests go to `EMBED_URL` at `/api/embeddings` (Ollama on worker :11434)

### SSE Protocol

All streaming endpoints yield `data: {json}\n\n`. Key shapes:
- `{token, conversation_id}` — streaming token
- `{searching: true}` — web search triggered
- `{search_results: N}` — N results (no raw_results payload)
- `{done: true, perplexity, tokens_per_sec, searched?}` — terminal
- `{error: "...", error_key: "..."}` — error with incident key

## Work State

### Completed this session
- **Pre-Docker review**: Full findings report delivered -- 30+ issues across 7 categories (hardcoded hosts/paths, config/secrets, AMQP gaps, resource cleanup, SQLite container safety, completions concurrency, TASKS.md accuracy).
- **Deprecation fix**: Replaced `asyncio.ensure_future` with `asyncio.create_task` in `rag.py` and `routers/chat.py`.
- **Documentation**: Added inline comments and docstrings to all functions in `db.py`.
- **Uninstall scripts**: Created and committed `scripts/uninstall.sh`, `teardown-docker.sh`, `nuclear-clean.sh`.
- **README**: Added "Uninstalling cAIc" section.
- **Docker containerization (B3)**: Created `Dockerfile`, `docker-compose.yml`, `.env.example`, `scripts/setup.sh`, `.dockerignore`, `searxng-settings.yml.dist`, `models/README.txt`. Fixed hardcoded defaults in `config.py` (localhost, Docker secrets path, `CAIC_DEFAULT_MODEL` env var, `CAIC_HW_STATE_PATH` env var). Added missing `psutil` + `jinja2` to `requirements.txt`. Fixed test discovery via `tests/conftest.py` sys.path insertion. 214 tests pass.

### Active
- (none)

### Blocked
- (none)

### Upcoming (backlog)
- ~~B3 — Docker distribution~~ [DONE]

### Key config values (current)
- **Current VERSION**: `v1.0.0` in `config.py`.
- `SESSION_TIMEOUT_SECONDS = 3600`
- `DEFAULT_MODEL = "qwen2.5-7b-instruct"` (overridable via `CAIC_DEFAULT_MODEL`)
- `LLAMA_SERVER_BASE = "http://localhost:8081"` (overridable via env var)