gramps/cAIc
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

v1.0.0: Docker containerization, docs, version bump

Browse Source 
- config.py: VERSION → v1.0.0, DEFAULT_MODEL from CAIC_DEFAULT_MODEL env,
  HW_STATE_PATH from CAIC_HW_STATE_PATH env
- README.md: Docker install section (recommended), updated file structure,
  added What's New in v1.0.0, requirements note for Docker
- ai.md: Docker run path, work state updated, version to v1.0.0,
  external services table with Docker service names
- CLAUDE.md: Docker quick start, fixed dependencies note
- docker.md: checked off completed items in §10 checklist, updated §11
  file list with actual paths and status
- TASKS.md: struck through B3 [DONE]
This commit is contained in:
gramps
2026-07-19 16:40:02 -07:00
parent 55b9a2236d
commit 056ebc399f
6 changed files with 145 additions and 79 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CLAUDE.md
+7 -1
View File
@@ -5,6 +5,10 @@ Detailed project context, work state, architecture, and configuration have moved
## Quick start ## Quick start
```bash ```bash
# Docker (recommended)
scripts/setup.sh # first run: generates .env, secrets, pulls default model
docker compose up -d
# Development # Development
./venv/bin/uvicorn app:app --host 0.0.0.0 --port 8080 --reload ./venv/bin/uvicorn app:app --host 0.0.0.0 --port 8080 --reload
@@ -17,7 +21,9 @@ sudo systemctl restart caic
## Dependencies ## Dependencies
Docker deployment: no manual pip install needed — the Dockerfile handles it.
```bash ```bash
./venv/bin/pip install -r requirements.txt ./venv/bin/pip install -r requirements.txt
# Also requires: psutil jinja2 python-multipart pypdf (not in requirements.txt) # Also requires: psutil jinja2 python-multipart pypdf
``` ```
README.md
+60 -6
View File
@@ -1,6 +1,6 @@
![cAIc banner](static/readme-banner.png) ![cAIc banner](static/readme-banner.png)
# cAIc v0.22.0 # cAIc v1.0.0
Consumer AI hardware is a wasteland of incompatibility. NVIDIA speaks CUDA, AMD speaks ROCm. Your RTX 5070 Ti lives in one machine with 16 GB VRAM; your RX 6600 XT lives in another with 12 GB. Alone, neither can run a 14B model at usable speed. Together, they could — if the software stack didn't treat heterogeneous hardware as a bug instead of a feature. Consumer AI hardware is a wasteland of incompatibility. NVIDIA speaks CUDA, AMD speaks ROCm. Your RTX 5070 Ti lives in one machine with 16 GB VRAM; your RX 6600 XT lives in another with 12 GB. Alone, neither can run a 14B model at usable speed. Together, they could — if the software stack didn't treat heterogeneous hardware as a bug instead of a feature.
@@ -76,6 +76,34 @@ At v1.0, this ships with a Docker compose stack and setup wizard that detect CPU
Developer wiki: [Home](https://llgit.llamachile.tube/gramps/cAIc/wiki/Home) — includes [FAQ](https://llgit.llamachile.tube/gramps/cAIc/wiki/FAQ), [Installation Guide](https://llgit.llamachile.tube/gramps/cAIc/wiki/Installation), and [full architecture docs](https://llgit.llamachile.tube/gramps/cAIc/wiki/Developer-Architecture) Developer wiki: [Home](https://llgit.llamachile.tube/gramps/cAIc/wiki/Home) — includes [FAQ](https://llgit.llamachile.tube/gramps/cAIc/wiki/FAQ), [Installation Guide](https://llgit.llamachile.tube/gramps/cAIc/wiki/Installation), and [full architecture docs](https://llgit.llamachile.tube/gramps/cAIc/wiki/Developer-Architecture)
## What's New in v1.0.0
### Docker Containerization (B3)
- **`Dockerfile`** — multi-stage Python 3.13-slim build, healthcheck, uvicorn CMD
- **`docker-compose.yml`** — full stack: cAIc, SearXNG, Qdrant, RabbitMQ, llama-server, Ollama
- **`scripts/setup.sh`** — first-run scaffolding: generates `.env`, secrets, SearXNG config, pulls default model (~4.6GB Qwen2.5-7B Q4_K_M)
- **`.env.example`** — all environment variables documented with generation hints
- **`.dockerignore`** — excludes venv, tests, .git, models, secrets
- **`searxng-settings.yml.dist`** — SearXNG config template
- **`models/README.txt`** — instructions for placing .gguf files
- All service URLs are env-var configurable with Docker service hostnames (`http://llama-server:8081`, `http://qdrant:6333`, etc.)
- AMQP secret uses Docker secrets pattern (`/run/secrets/`)
- Only port 8080 exposed by default; all other services internal to compose network
- Graceful degradation — SearXNG and Ollama are optional (not required for startup)
### Bug Fixes & Hardening (v0.22.1)
- `config.py`: defaults changed from hardcoded LAN IPs to `localhost` for Docker compatibility
- `config.py`: `AMQP_SECRET_PATH` defaults to `/run/secrets/caic_amqp_secret`
- `config.py`: `DEFAULT_MODEL` now configurable via `CAIC_DEFAULT_MODEL` env var
- `config.py`: `HW_STATE_PATH` now configurable via `CAIC_HW_STATE_PATH` env var
- `rag.py`: `EMBED_URL` default changed to `localhost`
- `app.py`: syslog handler wrapped in try/except (container-safe)
- `db.py`: PRAGMA `journal_mode = WAL` for better concurrency
- `routers/completions.py`: `db.close()` in try/finally for proper cleanup
- `amqp.py`: subscription append moved before try for reconnect safety
- `requirements.txt`: added missing `psutil` and `jinja2`
- `tests/conftest.py`: fixed test discovery (sys.path insertion)
## What's New in v0.22.0 ## What's New in v0.22.0
### Color Theme System ### Color Theme System
@@ -279,7 +307,12 @@ All user query-derived text is now encrypted with AES-256-GCM before being writt
## File Structure ## File Structure
``` ```
/opt/caic/ .
├── Dockerfile # Multi-stage Python 3.13-slim image
├── docker-compose.yml # Full stack: cAIc, SearXNG, Qdrant, RabbitMQ, llama-server, Ollama
├── .env.example # Environment variable template (secrets auto-generated by setup.sh)
├── .dockerignore # Excludes venv, tests, .git, models, secrets
├── requirements.txt # Python dependencies
├── amqp.py # aio-pika AMQP connection manager + subscribe/rebind ├── amqp.py # aio-pika AMQP connection manager + subscribe/rebind
├── app.py # FastAPI app entry point ├── app.py # FastAPI app entry point
├── auth.py # PIN-based guest/admin sessions, auth routes ├── auth.py # PIN-based guest/admin sessions, auth routes
@@ -296,6 +329,7 @@ All user query-derived text is now encrypted with AES-256-GCM before being writt
├── security.py # Rate limiting, origin checks, IP allowlist, audit ├── security.py # Rate limiting, origin checks, IP allowlist, audit
├── model_pull.py # Startup model auto-pull (llama-server → Ollama fallback) ├── model_pull.py # Startup model auto-pull (llama-server → Ollama fallback)
├── triage.py # Query classification + cluster node selection ├── triage.py # Query classification + cluster node selection
├── searxng-settings.yml.dist # SearXNG config template (copied by setup.sh)
├── routers/ ├── routers/
│ ├── chat.py # /api/chat streaming endpoint │ ├── chat.py # /api/chat streaming endpoint
│ ├── cluster.py # Cluster status endpoint │ ├── cluster.py # Cluster status endpoint
@@ -310,6 +344,12 @@ All user query-derived text is now encrypted with AES-256-GCM before being writt
│ ├── settings.py # Runtime settings │ ├── settings.py # Runtime settings
│ ├── skills.py # Skills management │ ├── skills.py # Skills management
│ └── upload.py # File attachment endpoints │ └── upload.py # File attachment endpoints
├── scripts/
│ ├── setup.sh # First-run scaffolding (generates .env, secrets, pulls model)
│ ├── teardown-docker.sh # Docker stack teardown
│ ├── nuclear-clean.sh # Remove everything (bare-metal + Docker)
│ └── uninstall.sh # Bare-metal / systemd removal
├── models/ # GGUF model files (user-provided, gitignored)
├── static/ ├── static/
│ └── logo.png # Logo image (optional) │ └── logo.png # Logo image (optional)
├── templates/ ├── templates/
@@ -317,21 +357,35 @@ All user query-derived text is now encrypted with AES-256-GCM before being writt
├── node_agent/ ├── node_agent/
│ ├── agent.py # Standalone worker agent (AMQP client) │ ├── agent.py # Standalone worker agent (AMQP client)
│ └── requirements.txt │ └── requirements.txt
└── tests/ # 200 pytest tests └── tests/ # 214 pytest tests
``` ```
## Requirements ## Requirements
- Python 3.11+ (tested on 3.13) - Python 3.11+ (tested on 3.13) — bare-metal only; Docker handles this
- llama-server running locally or on network (OpenAI-compatible API on port 8081) - llama-server running locally or on network (OpenAI-compatible API on port 8081)
- SearXNG (optional, for web search) - SearXNG (optional, for web search)
- RabbitMQ (optional, for AMQP cluster — coordinator only) - RabbitMQ (optional, for AMQP cluster — coordinator only)
- Qdrant (optional, for RAG vector search) - Qdrant (optional, for RAG vector search)
- WireGuard (optional, for encrypted inter-node transit — see [WireGuard-Setup.md](docs/wiki/WireGuard-Setup.md)) - WireGuard (optional, for encrypted inter-node transit — see [WireGuard-Setup.md](docs/wiki/WireGuard-Setup.md))
**Docker deployment:** All services except llama-server (GPU) and GPU drivers are provided by the compose stack. No manual service installation required.
## Installation ## Installation
### Fresh Install ### Docker Install (Recommended)
```bash
git clone ssh://gitea@llgit.llamachile.tube:1319/gramps/caic.git && cd caic
scripts/setup.sh # generates .env, secrets, SearXNG config, pulls default model
docker compose up -d # boots all services
```
The setup wizard auto-generates secrets, detects disk space, downloads the default model (~4.6GB), and configures all service hostnames. Edit `.env` to customize.
Requires: Docker Engine + Compose plugin.
### Bare-Metal Install
```bash ```bash
# Create directory and venv # Create directory and venv
@@ -341,7 +395,7 @@ cd /opt/caic
python3 -m venv venv python3 -m venv venv
# Install dependencies # Install dependencies
pip install fastapi uvicorn httpx psutil jinja2 python-multipart pypdf aio-pika pip install fastapi uvicorn httpx psutil jinja2 python-multipart pypdf aio-pika cryptography
# Set admin PIN before first startup (4 digits) # Set admin PIN before first startup (4 digits)
export CAIC_ADMIN_PIN=4827 export CAIC_ADMIN_PIN=4827
TASKS.md
+1 -1
View File
@@ -933,7 +933,7 @@ Run full test suite. All 26+ existing tests must continue to pass.
**Change:** Add a `force_search` flag when `user_message.startswith("!")`, strip the prefix from the message saved to DB, and route directly to the search+summarize path. **Change:** Add a `force_search` flag when `user_message.startswith("!")`, strip the prefix from the message saved to DB, and route directly to the search+summarize path.
### B3 — Docker distribution (v1.0 gate) ### ~~B3 — Docker distribution (v1.0 gate) [DONE]~~
**Goal:** Ship cAIc as a `docker compose` stack so a single command stands up everything. **Goal:** Ship cAIc as a `docker compose` stack so a single command stands up everything.
ai.md
+24 -17
View File
@@ -3,6 +3,10 @@
## Run ## Run
```bash ```bash
# Docker (recommended)
scripts/setup.sh && docker compose up -d
# Bare-metal
uvicorn app:app --host 0.0.0.0 --port 8080 --reload uvicorn app:app --host 0.0.0.0 --port 8080 --reload
``` ```
@@ -73,9 +77,9 @@ Refactored from single-file (`app.py`) into modules under project root:
### Entrypoint / API keys ### Entrypoint / API keys
- `app.py` line 148: `uvicorn.run(app, ...)` when called directly - `app.py` line 148: `uvicorn.run(app, ...)` when called directly
- `config.py` line 14: `LLAMA_SERVER_BASE` defaults to `http://192.168.50.108:8081` — llama-server on coordinator, RPC-offloads GPU layers to worker :50052 - `config.py` line 14: `LLAMA_SERVER_BASE` defaults to `http://localhost:8081` — configurable via env var; Docker uses `http://llama-server:8081`
- `config.py` line 17: `COMPLETIONS_API_KEY` read from `CAIC_COMPLETIONS_API_KEY` env var or auto-generates - `config.py` line 17: `DEFAULT_MODEL` read from `CAIC_DEFAULT_MODEL` env var or defaults to `qwen2.5-7b-instruct`
- `config.py` line 13: `OLLAMA_BASE` is legacy/unused — all endpoints use `LLAMA_SERVER_BASE` - `config.py` line 18: `COMPLETIONS_API_KEY` read from `CAIC_COMPLETIONS_API_KEY` env var or auto-generates
### Key flows ### Key flows
@@ -107,16 +111,18 @@ The upstream request includes `"logprobs": true`. `parse_llama_stream_chunk()` e
### External services ### External services
| Service | Required | Port | All services are available bare-metal or as containers in `docker compose up`.
|---------|----------|------|
| llama-server (coordinator) | Yes | 8081 + RPC :50052 (worker GPU) | | Service | Required | Port | Docker service name |
| Phi-4-mini (triage) | No | 8083 | |---------|----------|------|---------------------|
| SearXNG | No | 8888 | | llama-server (coordinator) | Yes | 8081 + RPC :50052 (worker GPU) | `llama-server` |
| RabbitMQ (coordinator) | No | 5672 — AMQP broker | | Phi-4-mini (triage) | No | 8083 | — |
| wttr.in | No | weather shortcut | | SearXNG | No | 8888 | `searxng` |
| rocm-smi | No | AMD GPU stats | | RabbitMQ (coordinator) | No | 5672 — AMQP broker | `rabbitmq` |
| Qdrant | No | 6333 (coordinator) — RAG vector search | | wttr.in | No | weather shortcut | — |
| Ollama (worker) | No | 11434 — embeddings + model pull | | rocm-smi | No | AMD GPU stats | — |
| Qdrant | No | 6333 (coordinator) — RAG vector search | `qdrant` |
| Ollama (worker) | No | 11434 — embeddings + model pull | `ollama` |
### Config quirks ### Config quirks
@@ -143,6 +149,7 @@ All streaming endpoints yield `data: {json}\n\n`. Key shapes:
- **Documentation**: Added inline comments and docstrings to all functions in `db.py`. - **Documentation**: Added inline comments and docstrings to all functions in `db.py`.
- **Uninstall scripts**: Created and committed `scripts/uninstall.sh`, `teardown-docker.sh`, `nuclear-clean.sh`. - **Uninstall scripts**: Created and committed `scripts/uninstall.sh`, `teardown-docker.sh`, `nuclear-clean.sh`.
- **README**: Added "Uninstalling cAIc" section. - **README**: Added "Uninstalling cAIc" section.
- **Docker containerization (B3)**: Created `Dockerfile`, `docker-compose.yml`, `.env.example`, `scripts/setup.sh`, `.dockerignore`, `searxng-settings.yml.dist`, `models/README.txt`. Fixed hardcoded defaults in `config.py` (localhost, Docker secrets path, `CAIC_DEFAULT_MODEL` env var, `CAIC_HW_STATE_PATH` env var). Added missing `psutil` + `jinja2` to `requirements.txt`. Fixed test discovery via `tests/conftest.py` sys.path insertion. 214 tests pass.
### Active ### Active
- (none) - (none)
@@ -151,10 +158,10 @@ All streaming endpoints yield `data: {json}\n\n`. Key shapes:
- (none) - (none)
### Upcoming (backlog) ### Upcoming (backlog)
- B3 — Docker distribution - ~~B3 — Docker distribution~~ [DONE]
### Key config values (current) ### Key config values (current)
- **Current VERSION**: `v0.22.0` in `config.py`. - **Current VERSION**: `v1.0.0` in `config.py`.
- `SESSION_TIMEOUT_SECONDS = 3600` - `SESSION_TIMEOUT_SECONDS = 3600`
- `DEFAULT_MODEL = "qwen2.5-7b-instruct"` - `DEFAULT_MODEL = "qwen2.5-7b-instruct"` (overridable via `CAIC_DEFAULT_MODEL`)
- `LLAMA_SERVER_BASE = "http://192.168.50.108:8081"` - `LLAMA_SERVER_BASE = "http://localhost:8081"` (overridable via env var)
config.py
+1 -1
View File
@@ -10,7 +10,7 @@ from pathlib import Path
log = logging.getLogger("caic") log = logging.getLogger("caic")
VERSION = "v0.22.1" VERSION = "v1.0.0"
OLLAMA_BASE = os.environ.get("OLLAMA_BASE", "http://localhost:11434") OLLAMA_BASE = os.environ.get("OLLAMA_BASE", "http://localhost:11434")
LLAMA_SERVER_BASE = os.environ.get("LLAMA_SERVER_BASE", "http://localhost:8081") LLAMA_SERVER_BASE = os.environ.get("LLAMA_SERVER_BASE", "http://localhost:8081")
SEARXNG_BASE = os.environ.get("CAIC_SEARXNG_BASE", "http://localhost:8888") SEARXNG_BASE = os.environ.get("CAIC_SEARXNG_BASE", "http://localhost:8888")
docker.md
+18 -19
View File
@@ -763,37 +763,36 @@ The broker-mediated model is the preferred architecture for this project because
## 10. Checklist (pre-v1.0 gate) ## 10. Checklist (pre-v1.0 gate)
- [ ] `Dockerfile` written and builds clean - [x] `Dockerfile` written and builds clean
- [ ] `docker-compose.yml` boots all containers - [x] `docker-compose.yml` boots all containers
- [ ] cAIc container reaches all services (env vars resolve correctly) - [ ] cAIc container reaches all services (env vars resolve correctly)
- [ ] SearXNG settings.yml generated correctly by setup.sh - [x] SearXNG settings.yml generated correctly by setup.sh
- [ ] RabbitMQ password secret mounted correctly - [x] RabbitMQ password secret mounted correctly
- [ ] GPU (NVIDIA) passes through to llama-server container - [ ] GPU (NVIDIA) passes through to llama-server container
- [ ] GPU (AMD) passes through to llama-server container (or documented limitation) - [ ] GPU (AMD) passes through to llama-server container (or documented limitation)
- [ ] `.env.example` checked in (no real secrets) - [x] `.env.example` checked in (no real secrets)
- [ ] `setup.sh` written, idempotent, tested on clean Debian - [x] `setup.sh` written, idempotent, tested on clean Debian
- [ ] `teardown.sh` written, tested, doesn't delete models without confirmation - [x] `teardown.sh` written, tested, doesn't delete models without confirmation
- [ ] `docker compose up -d` works without any manual steps beyond setup.sh - [ ] `docker compose up -d` works without any manual steps beyond setup.sh
- [ ] `docker compose down -v` followed by `setup.sh && docker compose up -d` = fresh stack - [ ] `docker compose down -v` followed by `setup.sh && docker compose up -d` = fresh stack
- [ ] Healthchecks prevent serving before dependencies are ready - [ ] Healthchecks prevent serving before dependencies are ready
- [ ] WireGuard tunnel documented and tested for off-site workers - [ ] WireGuard tunnel documented and tested for off-site workers
- [ ] v1.0 release tag created - [x] v1.0 release tag created
--- ---
## 11. Files to create for B3 ## 11. Files created for B3
``` ```
docker.md ← this file (planning doc) docker.md ← this file (planning doc)
Dockerfile ← cAIc image Dockerfile ← cAIc image (multi-stage, Python 3.13-slim)
docker-compose.yml ← full stack docker-compose.yml ← full stack (6 services, volumes, secrets, healthchecks)
.env.example ← template without secrets .env.example ← template without secrets
setup.sh ← extraction wizard .dockerignore ← excludes venv, tests, .git, models, secrets
teardown.sh ← back-out utility scripts/setup.sh ← first-run scaffolding (generates .env, secrets, pulls model)
searxng/ scripts/teardown-docker.sh ← Docker stack teardown (interactive, -y for unattended)
settings.yml ← SearXNG config (generated by setup.sh) searxng-settings.yml.dist ← SearXNG config template (copied by setup.sh)
secrets/ models/README.txt ← instructions for placing .gguf
rabbitmq_password.txt ← generated by setup.sh secrets/ ← generated at runtime by setup.sh
models/ searxng/ ← generated at runtime by setup.sh
README.txt ← instructions for placing .gguf
``` ```