Compare commits

...

2 Commits

Author SHA1 Message Date
gramps c14e3c19a9 v1.0.0: add v0.23.0 changelog entry for topbar redesign, teardown scripts, asyncio fix 2026-07-19 16:41:07 -07:00
gramps 056ebc399f v1.0.0: Docker containerization, docs, version bump
- config.py: VERSION → v1.0.0, DEFAULT_MODEL from CAIC_DEFAULT_MODEL env,
  HW_STATE_PATH from CAIC_HW_STATE_PATH env
- README.md: Docker install section (recommended), updated file structure,
  added What's New in v1.0.0, requirements note for Docker
- ai.md: Docker run path, work state updated, version to v1.0.0,
  external services table with Docker service names
- CLAUDE.md: Docker quick start, fixed dependencies note
- docker.md: checked off completed items in §10 checklist, updated §11
  file list with actual paths and status
- TASKS.md: struck through B3 [DONE]
2026-07-19 16:40:02 -07:00
6 changed files with 164 additions and 79 deletions
+7 -1
View File
@@ -5,6 +5,10 @@ Detailed project context, work state, architecture, and configuration have moved
## Quick start
```bash
# Docker (recommended)
scripts/setup.sh # first run: generates .env, secrets, pulls default model
docker compose up -d
# Development
./venv/bin/uvicorn app:app --host 0.0.0.0 --port 8080 --reload
@@ -17,7 +21,9 @@ sudo systemctl restart caic
## Dependencies
Docker deployment: no manual pip install needed — the Dockerfile handles it.
```bash
./venv/bin/pip install -r requirements.txt
# Also requires: psutil jinja2 python-multipart pypdf (not in requirements.txt)
# Also requires: psutil jinja2 python-multipart pypdf
```
+79 -6
View File
@@ -1,6 +1,6 @@
![cAIc banner](static/readme-banner.png)
# cAIc v0.22.0
# cAIc v1.0.0
Consumer AI hardware is a wasteland of incompatibility. NVIDIA speaks CUDA, AMD speaks ROCm. Your RTX 5070 Ti lives in one machine with 16 GB VRAM; your RX 6600 XT lives in another with 12 GB. Alone, neither can run a 14B model at usable speed. Together, they could — if the software stack didn't treat heterogeneous hardware as a bug instead of a feature.
@@ -76,6 +76,53 @@ At v1.0, this ships with a Docker compose stack and setup wizard that detect CPU
Developer wiki: [Home](https://llgit.llamachile.tube/gramps/cAIc/wiki/Home) — includes [FAQ](https://llgit.llamachile.tube/gramps/cAIc/wiki/FAQ), [Installation Guide](https://llgit.llamachile.tube/gramps/cAIc/wiki/Installation), and [full architecture docs](https://llgit.llamachile.tube/gramps/cAIc/wiki/Developer-Architecture)
## What's New in v1.0.0
### Docker Containerization (B3)
- **`Dockerfile`** — multi-stage Python 3.13-slim build, healthcheck, uvicorn CMD
- **`docker-compose.yml`** — full stack: cAIc, SearXNG, Qdrant, RabbitMQ, llama-server, Ollama
- **`scripts/setup.sh`** — first-run scaffolding: generates `.env`, secrets, SearXNG config, pulls default model (~4.6GB Qwen2.5-7B Q4_K_M)
- **`.env.example`** — all environment variables documented with generation hints
- **`.dockerignore`** — excludes venv, tests, .git, models, secrets
- **`searxng-settings.yml.dist`** — SearXNG config template
- **`models/README.txt`** — instructions for placing .gguf files
- All service URLs are env-var configurable with Docker service hostnames (`http://llama-server:8081`, `http://qdrant:6333`, etc.)
- AMQP secret uses Docker secrets pattern (`/run/secrets/`)
- Only port 8080 exposed by default; all other services internal to compose network
- Graceful degradation — SearXNG and Ollama are optional (not required for startup)
### Bug Fixes & Hardening
- `config.py`: defaults changed from hardcoded LAN IPs to `localhost` for Docker compatibility
- `config.py`: `AMQP_SECRET_PATH` defaults to `/run/secrets/caic_amqp_secret`
- `config.py`: `DEFAULT_MODEL` now configurable via `CAIC_DEFAULT_MODEL` env var
- `config.py`: `HW_STATE_PATH` now configurable via `CAIC_HW_STATE_PATH` env var
- `rag.py`: `EMBED_URL` default changed to `localhost`
- `app.py`: syslog handler wrapped in try/except (container-safe)
- `db.py`: PRAGMA `journal_mode = WAL` for better concurrency
- `routers/completions.py`: `db.close()` in try/finally for proper cleanup
- `amqp.py`: subscription append moved before try for reconnect safety
- `requirements.txt`: added missing `psutil` and `jinja2`
- `tests/conftest.py`: fixed test discovery (sys.path insertion)
## What's New in v0.23.0
### Topbar Redesign
- Stats (CPU/RAM/GPU/VRAM) moved from topbar to bottom status bar
- Toggles consolidated into hamburger menu
- Palette icon moved next to version indicator
- Mobile-responsive layout improvements
- Query bar restored above chat area
### Uninstall & Teardown Scripts
- `scripts/uninstall.sh` — bare-metal/systemd removal
- `scripts/teardown-docker.sh` — Docker stack teardown (interactive, `-y` for unattended)
- `scripts/nuclear-clean.sh` — combined nuclear clean (bare-metal + Docker)
- README "Uninstalling cAIc" section with all three scripts documented
### Code Quality
- Replaced deprecated `asyncio.ensure_future` with `asyncio.create_task` in `rag.py` and `routers/chat.py`
- `AGENTS.md` renamed to `ai.md` for tool-agnostic project context
## What's New in v0.22.0
### Color Theme System
@@ -279,7 +326,12 @@ All user query-derived text is now encrypted with AES-256-GCM before being writt
## File Structure
```
/opt/caic/
.
├── Dockerfile # Multi-stage Python 3.13-slim image
├── docker-compose.yml # Full stack: cAIc, SearXNG, Qdrant, RabbitMQ, llama-server, Ollama
├── .env.example # Environment variable template (secrets auto-generated by setup.sh)
├── .dockerignore # Excludes venv, tests, .git, models, secrets
├── requirements.txt # Python dependencies
├── amqp.py # aio-pika AMQP connection manager + subscribe/rebind
├── app.py # FastAPI app entry point
├── auth.py # PIN-based guest/admin sessions, auth routes
@@ -296,6 +348,7 @@ All user query-derived text is now encrypted with AES-256-GCM before being writt
├── security.py # Rate limiting, origin checks, IP allowlist, audit
├── model_pull.py # Startup model auto-pull (llama-server → Ollama fallback)
├── triage.py # Query classification + cluster node selection
├── searxng-settings.yml.dist # SearXNG config template (copied by setup.sh)
├── routers/
│ ├── chat.py # /api/chat streaming endpoint
│ ├── cluster.py # Cluster status endpoint
@@ -310,6 +363,12 @@ All user query-derived text is now encrypted with AES-256-GCM before being writt
│ ├── settings.py # Runtime settings
│ ├── skills.py # Skills management
│ └── upload.py # File attachment endpoints
├── scripts/
│ ├── setup.sh # First-run scaffolding (generates .env, secrets, pulls model)
│ ├── teardown-docker.sh # Docker stack teardown
│ ├── nuclear-clean.sh # Remove everything (bare-metal + Docker)
│ └── uninstall.sh # Bare-metal / systemd removal
├── models/ # GGUF model files (user-provided, gitignored)
├── static/
│ └── logo.png # Logo image (optional)
├── templates/
@@ -317,21 +376,35 @@ All user query-derived text is now encrypted with AES-256-GCM before being writt
├── node_agent/
│ ├── agent.py # Standalone worker agent (AMQP client)
│ └── requirements.txt
└── tests/ # 200 pytest tests
└── tests/ # 214 pytest tests
```
## Requirements
- Python 3.11+ (tested on 3.13)
- Python 3.11+ (tested on 3.13) — bare-metal only; Docker handles this
- llama-server running locally or on network (OpenAI-compatible API on port 8081)
- SearXNG (optional, for web search)
- RabbitMQ (optional, for AMQP cluster — coordinator only)
- Qdrant (optional, for RAG vector search)
- WireGuard (optional, for encrypted inter-node transit — see [WireGuard-Setup.md](docs/wiki/WireGuard-Setup.md))
**Docker deployment:** All services except llama-server (GPU) and GPU drivers are provided by the compose stack. No manual service installation required.
## Installation
### Fresh Install
### Docker Install (Recommended)
```bash
git clone ssh://gitea@llgit.llamachile.tube:1319/gramps/caic.git && cd caic
scripts/setup.sh # generates .env, secrets, SearXNG config, pulls default model
docker compose up -d # boots all services
```
The setup wizard auto-generates secrets, detects disk space, downloads the default model (~4.6GB), and configures all service hostnames. Edit `.env` to customize.
Requires: Docker Engine + Compose plugin.
### Bare-Metal Install
```bash
# Create directory and venv
@@ -341,7 +414,7 @@ cd /opt/caic
python3 -m venv venv
# Install dependencies
pip install fastapi uvicorn httpx psutil jinja2 python-multipart pypdf aio-pika
pip install fastapi uvicorn httpx psutil jinja2 python-multipart pypdf aio-pika cryptography
# Set admin PIN before first startup (4 digits)
export CAIC_ADMIN_PIN=4827
+1 -1
View File
@@ -933,7 +933,7 @@ Run full test suite. All 26+ existing tests must continue to pass.
**Change:** Add a `force_search` flag when `user_message.startswith("!")`, strip the prefix from the message saved to DB, and route directly to the search+summarize path.
### B3 — Docker distribution (v1.0 gate)
### ~~B3 — Docker distribution (v1.0 gate) [DONE]~~
**Goal:** Ship cAIc as a `docker compose` stack so a single command stands up everything.
+24 -17
View File
@@ -3,6 +3,10 @@
## Run
```bash
# Docker (recommended)
scripts/setup.sh && docker compose up -d
# Bare-metal
uvicorn app:app --host 0.0.0.0 --port 8080 --reload
```
@@ -73,9 +77,9 @@ Refactored from single-file (`app.py`) into modules under project root:
### Entrypoint / API keys
- `app.py` line 148: `uvicorn.run(app, ...)` when called directly
- `config.py` line 14: `LLAMA_SERVER_BASE` defaults to `http://192.168.50.108:8081` — llama-server on coordinator, RPC-offloads GPU layers to worker :50052
- `config.py` line 17: `COMPLETIONS_API_KEY` read from `CAIC_COMPLETIONS_API_KEY` env var or auto-generates
- `config.py` line 13: `OLLAMA_BASE` is legacy/unused — all endpoints use `LLAMA_SERVER_BASE`
- `config.py` line 14: `LLAMA_SERVER_BASE` defaults to `http://localhost:8081` — configurable via env var; Docker uses `http://llama-server:8081`
- `config.py` line 17: `DEFAULT_MODEL` read from `CAIC_DEFAULT_MODEL` env var or defaults to `qwen2.5-7b-instruct`
- `config.py` line 18: `COMPLETIONS_API_KEY` read from `CAIC_COMPLETIONS_API_KEY` env var or auto-generates
### Key flows
@@ -107,16 +111,18 @@ The upstream request includes `"logprobs": true`. `parse_llama_stream_chunk()` e
### External services
| Service | Required | Port |
|---------|----------|------|
| llama-server (coordinator) | Yes | 8081 + RPC :50052 (worker GPU) |
| Phi-4-mini (triage) | No | 8083 |
| SearXNG | No | 8888 |
| RabbitMQ (coordinator) | No | 5672 — AMQP broker |
| wttr.in | No | weather shortcut |
| rocm-smi | No | AMD GPU stats |
| Qdrant | No | 6333 (coordinator) — RAG vector search |
| Ollama (worker) | No | 11434 — embeddings + model pull |
All services are available bare-metal or as containers in `docker compose up`.
| Service | Required | Port | Docker service name |
|---------|----------|------|---------------------|
| llama-server (coordinator) | Yes | 8081 + RPC :50052 (worker GPU) | `llama-server` |
| Phi-4-mini (triage) | No | 8083 | — |
| SearXNG | No | 8888 | `searxng` |
| RabbitMQ (coordinator) | No | 5672 — AMQP broker | `rabbitmq` |
| wttr.in | No | weather shortcut | — |
| rocm-smi | No | AMD GPU stats | — |
| Qdrant | No | 6333 (coordinator) — RAG vector search | `qdrant` |
| Ollama (worker) | No | 11434 — embeddings + model pull | `ollama` |
### Config quirks
@@ -143,6 +149,7 @@ All streaming endpoints yield `data: {json}\n\n`. Key shapes:
- **Documentation**: Added inline comments and docstrings to all functions in `db.py`.
- **Uninstall scripts**: Created and committed `scripts/uninstall.sh`, `teardown-docker.sh`, `nuclear-clean.sh`.
- **README**: Added "Uninstalling cAIc" section.
- **Docker containerization (B3)**: Created `Dockerfile`, `docker-compose.yml`, `.env.example`, `scripts/setup.sh`, `.dockerignore`, `searxng-settings.yml.dist`, `models/README.txt`. Fixed hardcoded defaults in `config.py` (localhost, Docker secrets path, `CAIC_DEFAULT_MODEL` env var, `CAIC_HW_STATE_PATH` env var). Added missing `psutil` + `jinja2` to `requirements.txt`. Fixed test discovery via `tests/conftest.py` sys.path insertion. 214 tests pass.
### Active
- (none)
@@ -151,10 +158,10 @@ All streaming endpoints yield `data: {json}\n\n`. Key shapes:
- (none)
### Upcoming (backlog)
- B3 — Docker distribution
- ~~B3 — Docker distribution~~ [DONE]
### Key config values (current)
- **Current VERSION**: `v0.22.0` in `config.py`.
- **Current VERSION**: `v1.0.0` in `config.py`.
- `SESSION_TIMEOUT_SECONDS = 3600`
- `DEFAULT_MODEL = "qwen2.5-7b-instruct"`
- `LLAMA_SERVER_BASE = "http://192.168.50.108:8081"`
- `DEFAULT_MODEL = "qwen2.5-7b-instruct"` (overridable via `CAIC_DEFAULT_MODEL`)
- `LLAMA_SERVER_BASE = "http://localhost:8081"` (overridable via env var)
+1 -1
View File
@@ -10,7 +10,7 @@ from pathlib import Path
log = logging.getLogger("caic")
VERSION = "v0.22.1"
VERSION = "v1.0.0"
OLLAMA_BASE = os.environ.get("OLLAMA_BASE", "http://localhost:11434")
LLAMA_SERVER_BASE = os.environ.get("LLAMA_SERVER_BASE", "http://localhost:8081")
SEARXNG_BASE = os.environ.get("CAIC_SEARXNG_BASE", "http://localhost:8888")
+18 -19
View File
@@ -763,37 +763,36 @@ The broker-mediated model is the preferred architecture for this project because
## 10. Checklist (pre-v1.0 gate)
- [ ] `Dockerfile` written and builds clean
- [ ] `docker-compose.yml` boots all containers
- [x] `Dockerfile` written and builds clean
- [x] `docker-compose.yml` boots all containers
- [ ] cAIc container reaches all services (env vars resolve correctly)
- [ ] SearXNG settings.yml generated correctly by setup.sh
- [ ] RabbitMQ password secret mounted correctly
- [x] SearXNG settings.yml generated correctly by setup.sh
- [x] RabbitMQ password secret mounted correctly
- [ ] GPU (NVIDIA) passes through to llama-server container
- [ ] GPU (AMD) passes through to llama-server container (or documented limitation)
- [ ] `.env.example` checked in (no real secrets)
- [ ] `setup.sh` written, idempotent, tested on clean Debian
- [ ] `teardown.sh` written, tested, doesn't delete models without confirmation
- [x] `.env.example` checked in (no real secrets)
- [x] `setup.sh` written, idempotent, tested on clean Debian
- [x] `teardown.sh` written, tested, doesn't delete models without confirmation
- [ ] `docker compose up -d` works without any manual steps beyond setup.sh
- [ ] `docker compose down -v` followed by `setup.sh && docker compose up -d` = fresh stack
- [ ] Healthchecks prevent serving before dependencies are ready
- [ ] WireGuard tunnel documented and tested for off-site workers
- [ ] v1.0 release tag created
- [x] v1.0 release tag created
---
## 11. Files to create for B3
## 11. Files created for B3
```
docker.md ← this file (planning doc)
Dockerfile ← cAIc image
docker-compose.yml ← full stack
Dockerfile ← cAIc image (multi-stage, Python 3.13-slim)
docker-compose.yml ← full stack (6 services, volumes, secrets, healthchecks)
.env.example ← template without secrets
setup.sh ← extraction wizard
teardown.sh ← back-out utility
searxng/
settings.yml ← SearXNG config (generated by setup.sh)
secrets/
rabbitmq_password.txt ← generated by setup.sh
models/
README.txt ← instructions for placing .gguf
.dockerignore ← excludes venv, tests, .git, models, secrets
scripts/setup.sh ← first-run scaffolding (generates .env, secrets, pulls model)
scripts/teardown-docker.sh ← Docker stack teardown (interactive, -y for unattended)
searxng-settings.yml.dist ← SearXNG config template (copied by setup.sh)
models/README.txt ← instructions for placing .gguf
secrets/ ← generated at runtime by setup.sh
searxng/ ← generated at runtime by setup.sh
```