make QDRANT_URL, EMBED_URL, AMQP_SECRET_PATH env-overridable; document single-node/WSL deployment

This commit is contained in:
gramps
2026-07-14 14:33:54 -07:00
parent e584e4983c
commit 3f64bed485
3 changed files with 21 additions and 3 deletions
+17
View File
@@ -28,6 +28,23 @@ cAIc splits the workload across two machine roles:
This split keeps the UI responsive during inference (the coordinator isn't blocked by GPU compute) and lets workers focus VRAM entirely on model weights rather than browser sessions or API orchestration. This split keeps the UI responsive during inference (the coordinator isn't blocked by GPU compute) and lets workers focus VRAM entirely on model weights rather than browser sessions or API orchestration.
### Single-Node Deployment (Experimental)
cAIc can also run entirely on one machine with all services colocated — coordinator, llama-server, Qdrant, SearXNG, and RabbitMQ all on localhost. This is useful for testing, laptops, or WSL2 under Windows 11.
To deploy single-node, override the remote service URLs:
```bash
export CAIC_QDRANT_URL=http://localhost:6333
export CAIC_EMBED_URL=http://localhost:11434
export LLAMA_SERVER_BASE=http://localhost:8081
# AMQP URL is already configurable via CAIC_AMQP_URL
```
All services degrade gracefully if unreachable — RAG, search, cluster, and triage log warnings and continue. Only llama-server (inference) is strictly required.
Untested: Windows 11 / WSL2 (Debian). The codebase is pure Python with no platform-specific dependencies beyond `rocm-smi` (AMD GPU stats, gracefully absent) and `system_profiler` (macOS, absent on Linux/WSL). llama.cpp builds and runs on WSL2 with NVIDIA GPU passthrough.
Under the hood: FastAPI + SQLite + Jinja2 on Python 3.13. AMQP-mediated cluster coordination with an OpenAI-compatible inference endpoint. Under the hood: FastAPI + SQLite + Jinja2 on Python 3.13. AMQP-mediated cluster coordination with an OpenAI-compatible inference endpoint.
#### Query-routing vs. layer-splitting — why it matters #### Query-routing vs. layer-splitting — why it matters
+2 -2
View File
@@ -21,7 +21,7 @@ MODEL_CONTEXT_LENGTH = 4096
AMQP_RECONNECT_DELAY = 5 AMQP_RECONNECT_DELAY = 5
AMQP_EXCHANGE_ADMIN = "jc.admin" AMQP_EXCHANGE_ADMIN = "jc.admin"
AMQP_EXCHANGE_SYSTEM = "jc.system" AMQP_EXCHANGE_SYSTEM = "jc.system"
AMQP_SECRET_PATH = "/home/gramps/.caic_amqp_secret" AMQP_SECRET_PATH = os.environ.get("CAIC_AMQP_SECRET_PATH", "/home/gramps/.caic_amqp_secret")
def get_amqp_url() -> str: def get_amqp_url() -> str:
url = os.environ.get("CAIC_AMQP_URL") url = os.environ.get("CAIC_AMQP_URL")
@@ -68,7 +68,7 @@ BODY_LIMIT_PROFILE_BYTES = 256 * 1024
UPLOAD_DIR = "/tmp/caic_uploads" UPLOAD_DIR = "/tmp/caic_uploads"
MAX_UPLOAD_BYTES = 20 * 1024 * 1024 MAX_UPLOAD_BYTES = 20 * 1024 * 1024
SUPPORTED_UPLOAD_TYPES = {"text/plain", "text/markdown", "application/pdf", "application/json", "text/x-python", "text/html", "image/png", "image/jpeg", "image/gif", "image/svg+xml", "image/webp"} SUPPORTED_UPLOAD_TYPES = {"text/plain", "text/markdown", "application/pdf", "application/json", "text/x-python", "text/html", "image/png", "image/jpeg", "image/gif", "image/svg+xml", "image/webp"}
QDRANT_URL = "http://192.168.50.108:6333" QDRANT_URL = os.environ.get("CAIC_QDRANT_URL", "http://192.168.50.108:6333")
RAG_COLLECTION = "caic_rag" RAG_COLLECTION = "caic_rag"
UPLOAD_CONTEXT_EXPIRY_HOURS = 1 UPLOAD_CONTEXT_EXPIRY_HOURS = 1
BODY_LIMIT_UPLOAD_BYTES = MAX_UPLOAD_BYTES BODY_LIMIT_UPLOAD_BYTES = MAX_UPLOAD_BYTES
+2 -1
View File
@@ -3,6 +3,7 @@ cAIc - RAG pipeline: Qdrant vector search + system prompt assembly.
""" """
import asyncio import asyncio
import logging import logging
import os
from datetime import datetime, timezone from datetime import datetime, timezone
import httpx import httpx
@@ -15,7 +16,7 @@ from config import MAX_SKILL_PROMPT_CHARS, QDRANT_URL, RAG_COLLECTION
log = logging.getLogger("caic") log = logging.getLogger("caic")
EMBED_URL = "http://192.168.50.210:11434" EMBED_URL = os.environ.get("CAIC_EMBED_URL", "http://192.168.50.210:11434")
EMBED_MODEL = "mxbai-embed-large" EMBED_MODEL = "mxbai-embed-large"
RAG_SCORE_THRESHOLD = 0.25 RAG_SCORE_THRESHOLD = 0.25