cAIc/ai.md

cAIc — Agents Guide

Run

# Docker (recommended)
scripts/setup.sh && docker compose up -d

# Bare-metal
uvicorn app:app --host 0.0.0.0 --port 8080 --reload

Tests

python3 -m pytest tests/ -v

All tests use tmp_path fixtures + monkeypatched httpx.AsyncClient.stream/get/post/put. No external services needed. Test factories reset SESSIONS, PIN_ATTEMPTS, RATE_EVENTS globals — be careful not to let test state leak. Tests import directly from the correct modules (db, security, config, search, rag, memory, routers.*).

Every router has a dedicated test file:

File	Covers
test_auth_capabilities.py	auth.py — guest/admin sessions, origin blocking, logout
test_chat_streaming_and_memory_paths.py	routers/chat.py — streaming, auto-search, remember/forget, upload context injection
test_completions.py	routers/completions.py — API key auth, FIM, streaming, blocking, errors
test_conversations.py	routers/conversations.py — full CRUD, guest admin enforcement, attachment_count
test_ingest.py	routers/ingest.py — Bearer auth, chunk/embed/upsert, validation
test_memories.py	routers/memories.py — edit, search, stats endpoints
test_models_router.py	routers/models.py — models list, ps, show, stats, search/status
test_presets.py	routers/presets.py — full CRUD, default preset protection
test_profile.py	routers/profile.py — get, update, default, length validation
test_rag_management.py	eviction.py + routers/rag_admin.py — eviction engine, stats, flush, browse, search, edit, delete individual points
test_search_route.py	routers/search_route.py — explicit search flow, no results, errors
test_search_url_sanitization.py	search.py URL sanitizer
test_cluster.py	cluster.py — registration, deregistration, pong, events, coordinator query
test_cluster_heartbeat.py	cluster.py — heartbeat handler, known/unknown node
test_model_swap.py	cluster.py — request_model_swap, handle_model_ready/failed
test_node_agent.py	node_agent/agent.py — registration, ping/pong, model swap
test_image.py	Image generation — cluster handlers, router proxy, node agent ComfyUI integration, hardware probe, capability detection
test_settings_allowlist.py	routers/settings.py — allowlisted key enforcement
test_skills_framework.py	routers/skills.py — list, toggle, unknown skill, prompt injection
test_ip_allowlist.py	IP allowlist helper + middleware
test_rate_and_payload_guardrails.py	Rate limits + payload size enforcement
test_error_envelopes.py	Global exception handler + stream error incidents
test_fixes_regression.py	Origin-exempt ingest, bogus conversation_id FK, auto-search reset, image uploads, conflict false-positives, deterministic ingest ids, get_load VRAM parsing, version pin
test_upload.py	routers/upload.py — upload, delete, link, by-conversation, attachment_count integration

Modules that call httpx.AsyncClient (chat, completions, models, search_route, upload, ingest, model_pull) are mocked via monkeypatch.setattr on AsyncClient.stream, .get, or .post. CPU stats in models.py (api/stats) use real psutil; GPU stats are monkeypatched via routers.models.get_gpu_stats.

Architecture

Refactored from single-file (app.py) into modules under project root:

File	Role
app.py	FastAPI app, middleware, router registration
config.py	Constants, env vars, rate/payload limits, built-in skills registry, upload limits
db.py	SQLite schema, connection factory, settings helpers, upload_context CRUD
auth.py	PIN-based guest/admin sessions, auth routes
security.py	Rate limiting, origin checks, IP allowlist, audit/incident logging
memory.py	FTS5 memory CRUD (encrypted facts, Python-side matching), remember/forget command parsing
search.py	SearXNG integration, perplexity scoring, refusal detection
rag.py	Qdrant vector search (encrypted payload text) + system prompt assembly + chunk_text() helper
eviction.py	Score-based RAG eviction engine
gpu.py	GPU stats — rocm-smi (AMD/Linux) or system_profiler (Apple Silicon/macOS)
crypto.py	AES-256-GCM encrypt/decrypt + key management (stored as heartbeat_interval_ms in settings)
model_pull.py	Startup model availability check + Ollama pull API
cluster.py	Cluster node registry, event log, coordinator election, ping/pong, model swap handlers, image generation request/response
amqp.py	AMQP connection manager — connect, disconnect, publish, subscribe, auto-reconnect
node_agent/	Standalone worker agent — AMQP client for registration, ping/pong, model swap, image generation
routers/	One module per endpoint group (chat, search, skills, completions, upload, ingest, image)

Entrypoint / API keys

• app.py line 148: uvicorn.run(app, ...) when called directly
• config.py line 14: LLAMA_SERVER_BASE defaults to http://localhost:8081 — configurable via env var; Docker uses http://llama-server:8081
• config.py line 17: DEFAULT_MODEL read from CAIC_DEFAULT_MODEL env var or defaults to qwen2.5-7b-instruct
• config.py line 18: COMPLETIONS_API_KEY read from CAIC_COMPLETIONS_API_KEY env var or auto-generates

Key flows

1. /api/chat → process_remember_command() intercepts "remember that..." / "forget about..." first → optional upload_context_id fetches document text from SQLite → build_system_prompt() (profile + FTS5 memory + Qdrant RAG + preset + skills + uploaded doc) → stream from LLAMA_SERVER_BASE with logprobs: true → if perplexity > 15.0 OR REFUSAL_PATTERNS match, re-query with SearXNG results
2. /api/search → bypasses perplexity/refusal, queries SearXNG directly → summarizes via llama-server
3. /v1/chat/completions → OpenAI-compatible for Continue.dev/IDE integration; FIM requests proxied without persistence
4. /api/upload → multipart file upload, PDF/text extraction, mode=(context|ingest|both), stores SQLite context (1hr expiry) + Qdrant upsert
5. /api/ingest → Bearer token auth, programmatic RAG ingest (terminal hook, external tools)
6. POST /api/image/generate → admin required, routes to an image-gen node via AMQP → ComfyUI workflow → returns PNG; GET /api/image/status lists available image gen nodes

Perplexity / auto-search

The upstream request includes "logprobs": true. parse_llama_stream_chunk() extracts per-token logprobs from each chunk's choices[0].logprobs.content[].logprob. The all_logprobs list is populated during streaming, so calculate_perplexity() and is_uncertain() work correctly.

Auth / lockdown

• Guest session by default (POST /api/auth/guest), admin unlock via 4-digit PIN (POST /api/auth/login)
• Admin required for PUT/DELETE/PATCH + all POST except allowlist (/api/chat, /api/search, /api/auth/*)
• /api/ingest is exempt from session auth — self-authenticates via Bearer token
• IP allowlist, rate limiting, origin checking, payload size limits — all enforced in app.py middleware
• Origin check applies to all /api/ requests; returns False when both Origin and Referer are absent
• CAIC_ADMIN_PIN env var required on first boot (or CAIC_ALLOW_DEFAULT_PIN=true)

Database

• SQLite at caic.db, auto-created by init_db() on startup via FastAPI lifespan
• get_db() opens new connection per request (no pool). Close after use.
• FTS5 virtual table memories for full-text search with BM25 ranking.
• upload_context table: auto-expiring document storage for chat context injection.

External services

All services are available bare-metal or as containers in docker compose up.

Service	Required	Port	Docker service name
llama-server (coordinator)	Yes	8081 + RPC :50052 (worker GPU)	llama-server
SearXNG	No	8888	searxng
RabbitMQ (coordinator)	No	5672 — AMQP broker	rabbitmq
wttr.in	No	weather shortcut	—
rocm-smi	No	AMD GPU stats	—
Qdrant	No	6333 (coordinator) — RAG vector search	qdrant
Ollama (worker)	No	11434 — embeddings + model pull	ollama
ComfyUI (worker)	No	8188 — image generation API	—

Config quirks

• BODY_LIMIT_UPLOAD_BYTES = 20MB for /api/upload; other paths use smaller limits
• SUPPORTED_UPLOAD_TYPES includes images (png/jpeg/gif/svg/webp) + text + PDF + JSON
• UPLOAD_CONTEXT_EXPIRY_HOURS = 1 hour
• Rate limits and payload caps in config.py — patch security.RL_* not config.RL_* for tests
• COMFYUI_BASE defaults to http://localhost:8188 (overridable via CAIC_COMFYUI_BASE)
• COMFYUI_TIMEOUT defaults to 120 seconds (overridable via CAIC_COMFYUI_TIMEOUT)
• RAG embedding requests go to EMBED_URL at /api/embeddings (Ollama on worker :11434)

SSE Protocol

All streaming endpoints yield data: {json}\n\n. Key shapes:

• {token, conversation_id} — streaming token
• {searching: true} — web search triggered
• {search_results: N} — N results (no raw_results payload)
• {done: true, perplexity, tokens_per_sec, searched?} — terminal
• {error: "...", error_key: "..."} — error with incident key

Work State

Completed this session

• Pre-Docker review: Full findings report delivered -- 30+ issues across 7 categories (hardcoded hosts/paths, config/secrets, AMQP gaps, resource cleanup, SQLite container safety, completions concurrency, TASKS.md accuracy).
• Project rename: jarvisChat → cAIc ("cake") — swept remaining branding (router docstrings, jc-ingest.sh env var), deleted stale AGENTS.md.local.
• Single-node consolidation: all services moved to jarvis (192.168.50.212) — COMFYUI_BASE default → localhost:8188, AMQP URL default → localhost:5672, NODE_NAME default → jarvis, DEFAULT_PROFILE topology rewritten, cluster/AMQP/node_agent left in place (degrades gracefully).
• Deprecation fix: Replaced asyncio.ensure_future with asyncio.create_task in rag.py and routers/chat.py.
• Documentation: Added inline comments and docstrings to all functions in db.py.
• Uninstall scripts: Created and committed scripts/uninstall.sh, teardown-docker.sh, nuclear-clean.sh.
• README: Added "Uninstalling cAIc" section.
• Docker containerization (B3): Created Dockerfile, docker-compose.yml, .env.example, scripts/setup.sh, .dockerignore, searxng-settings.yml.dist, models/README.txt. Fixed hardcoded defaults in config.py (localhost, Docker secrets path, CAIC_DEFAULT_MODEL env var, CAIC_HW_STATE_PATH env var). Added missing psutil + jinja2 to requirements.txt. Fixed test discovery via tests/conftest.py sys.path insertion. 214 tests pass.

Active

• Image generation service backend complete — cluster handlers, router proxy, node agent ComfyUI integration, hardware probe. 27 tests pass. ComfyUI install pending on jarvis (single-node).

Deployed (2026-08-07) — v1.1.0 to production

• Fixed crash-loop: ultron llama-server.service had 911 restarts — its --rpc 192.168.50.210:50052 pointed at a dead IP. Corrected to 192.168.50.212:50052 (jarvis GPU rpc-server). Model now loads, /health = ok.
• Deployed v1.1.0: workspace repo synced to /opt/jarvischat (jarvischat.service cwd). caic.db + venv preserved; aio-pika installed into prod venv (was missing → AMQP disabled).
• Env fixes (/etc/systemd/system/jarvischat.service.d/override.conf): added LLAMA_SERVER_BASE=http://192.168.50.108:8081, CAIC_QDRANT_URL=http://192.168.50.108:6333, CAIC_COMPLETIONS_API_KEY (was set as legacy JARVISCHAT_ name), kept CAIC_EMBED_URL=http://192.168.50.108:11434 + CAIC_ADMIN_PIN=1319. Wrote /opt/jarvischat/.completions_key (jc-ingest.sh).
• Deploy-blocking bug fixes (uncommitted, workspace + deploy):
  • rag.py chunk_text: chunk_size 512→200 (chunks exceeded mxbai-embed-large's 512-token context → ollama 500).
  • Qdrant 1.18.2 rejects non-UUID point IDs: wrapped ingest-*/auto-*/upload-* string IDs in uuid5 in routers/ingest.py, rag.py, routers/upload.py.
  • docs/jc-ingest.sh: JC_URL updated .210→.212.
• Docs rebuilt: 159 chunks (source docs) re-ingested via /api/ingest (README, ai.md, docker.md, CLAUDE.md, wiki/*). RAG now 378 vectors; chat verified injecting "Retrieved Context".
• Tests: all 244 pass (run per-file in a throwaway venv; the full-suite run deadlocks on TestClient/AMQP ordering, not a code failure).

Follow-ups

• AMQP wiring: cluster subs degrade gracefully — moot in the single-node (jarvis) deployment until a multi-node cluster is stood back up. Needs CAIC_AMQP_URL + credentials if that happens.
• CAIC_TRIAGE_BASE set but triage not yet invoked by chat (config-only until TASK 2 wiring).

Blocked

• Ball Gunner assets — waiting on Canva designs

Upcoming (backlog)

• B3 — Docker distribution [DONE]

Key config values (current)

• Current VERSION: v1.1.0 in config.py.
• SESSION_TIMEOUT_SECONDS = 3600
• DEFAULT_MODEL = "qwen2.5-7b-instruct" (overridable via CAIC_DEFAULT_MODEL)
• LLAMA_SERVER_BASE = "http://localhost:8081" (overridable via env var)