make QDRANT_URL, EMBED_URL, AMQP_SECRET_PATH env-overridable; document single-node/WSL deployment
This commit is contained in:
@@ -28,6 +28,23 @@ cAIc splits the workload across two machine roles:
|
||||
|
||||
This split keeps the UI responsive during inference (the coordinator isn't blocked by GPU compute) and lets workers focus VRAM entirely on model weights rather than browser sessions or API orchestration.
|
||||
|
||||
### Single-Node Deployment (Experimental)
|
||||
|
||||
cAIc can also run entirely on one machine with all services colocated — coordinator, llama-server, Qdrant, SearXNG, and RabbitMQ all on localhost. This is useful for testing, laptops, or WSL2 under Windows 11.
|
||||
|
||||
To deploy single-node, override the remote service URLs:
|
||||
|
||||
```bash
|
||||
export CAIC_QDRANT_URL=http://localhost:6333
|
||||
export CAIC_EMBED_URL=http://localhost:11434
|
||||
export LLAMA_SERVER_BASE=http://localhost:8081
|
||||
# AMQP URL is already configurable via CAIC_AMQP_URL
|
||||
```
|
||||
|
||||
All services degrade gracefully if unreachable — RAG, search, cluster, and triage log warnings and continue. Only llama-server (inference) is strictly required.
|
||||
|
||||
Untested: Windows 11 / WSL2 (Debian). The codebase is pure Python with no platform-specific dependencies beyond `rocm-smi` (AMD GPU stats, gracefully absent) and `system_profiler` (macOS, absent on Linux/WSL). llama.cpp builds and runs on WSL2 with NVIDIA GPU passthrough.
|
||||
|
||||
Under the hood: FastAPI + SQLite + Jinja2 on Python 3.13. AMQP-mediated cluster coordination with an OpenAI-compatible inference endpoint.
|
||||
|
||||
#### Query-routing vs. layer-splitting — why it matters
|
||||
|
||||
@@ -21,7 +21,7 @@ MODEL_CONTEXT_LENGTH = 4096
|
||||
AMQP_RECONNECT_DELAY = 5
|
||||
AMQP_EXCHANGE_ADMIN = "jc.admin"
|
||||
AMQP_EXCHANGE_SYSTEM = "jc.system"
|
||||
AMQP_SECRET_PATH = "/home/gramps/.caic_amqp_secret"
|
||||
AMQP_SECRET_PATH = os.environ.get("CAIC_AMQP_SECRET_PATH", "/home/gramps/.caic_amqp_secret")
|
||||
|
||||
def get_amqp_url() -> str:
|
||||
url = os.environ.get("CAIC_AMQP_URL")
|
||||
@@ -68,7 +68,7 @@ BODY_LIMIT_PROFILE_BYTES = 256 * 1024
|
||||
UPLOAD_DIR = "/tmp/caic_uploads"
|
||||
MAX_UPLOAD_BYTES = 20 * 1024 * 1024
|
||||
SUPPORTED_UPLOAD_TYPES = {"text/plain", "text/markdown", "application/pdf", "application/json", "text/x-python", "text/html", "image/png", "image/jpeg", "image/gif", "image/svg+xml", "image/webp"}
|
||||
QDRANT_URL = "http://192.168.50.108:6333"
|
||||
QDRANT_URL = os.environ.get("CAIC_QDRANT_URL", "http://192.168.50.108:6333")
|
||||
RAG_COLLECTION = "caic_rag"
|
||||
UPLOAD_CONTEXT_EXPIRY_HOURS = 1
|
||||
BODY_LIMIT_UPLOAD_BYTES = MAX_UPLOAD_BYTES
|
||||
|
||||
@@ -3,6 +3,7 @@ cAIc - RAG pipeline: Qdrant vector search + system prompt assembly.
|
||||
"""
|
||||
import asyncio
|
||||
import logging
|
||||
import os
|
||||
from datetime import datetime, timezone
|
||||
|
||||
import httpx
|
||||
@@ -15,7 +16,7 @@ from config import MAX_SKILL_PROMPT_CHARS, QDRANT_URL, RAG_COLLECTION
|
||||
|
||||
log = logging.getLogger("caic")
|
||||
|
||||
EMBED_URL = "http://192.168.50.210:11434"
|
||||
EMBED_URL = os.environ.get("CAIC_EMBED_URL", "http://192.168.50.210:11434")
|
||||
EMBED_MODEL = "mxbai-embed-large"
|
||||
RAG_SCORE_THRESHOLD = 0.25
|
||||
|
||||
|
||||
Reference in New Issue
Block a user