diff --git a/AGENTS.md b/AGENTS.md index a5e0388..efbc316 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -137,7 +137,7 @@ All streaming endpoints yield `data: {json}\n\n`. Key shapes: ## Work State ### Completed this session -- **At-Rest Encryption (v0.20.0)** — `crypto.py` with AES-256-GCM encrypt/decrypt + `ensure_key()`. Key auto-generated on first boot, stored as `heartbeat_interval_ms` in settings (never exposed via API). All 12 storage call sites wired: `routers/chat.py`, `routers/search_route.py`, `routers/conversations.py`, `routers/completions.py`, `memory.py`, `db.py` (upload_context), `rag.py`, `routers/upload.py`, `routers/ingest.py`. `memory.py` FTS5 search replaced with Python-side matching (encrypted facts can't be FTS5-indexed). `app.py` lifespan calls `ensure_key()` after `init_db()`. 200/200 tests pass. Deployed to jarvis with fresh DB. +- **v0.21.0 Release** — Scrollbar/DOM fixes, perplexity persistence, all service URLs env-overridable, single-node deployment docs, hardware.py Qdrant URL bugfix. See README for full changelog. ### Active - (none) @@ -145,12 +145,15 @@ All streaming endpoints yield `data: {json}\n\n`. Key shapes: ### Blocked - (none) +### Blocked +- (none) + ### Upcoming (backlog) - B4 — RAG Corpus Management UI - Private Chat mode was implemented in B8 (v0.19.3). WireGuard in-transit encryption documented in wiki. At-rest encryption implemented in v0.20.0. ### Key config values (current) -- `VERSION = "v0.20.0"` in `config.py` +- **Current VERSION**: `v0.21.0` in `config.py`. - `SESSION_TIMEOUT_SECONDS = 3600` - `DEFAULT_MODEL = "qwen2.5-7b-instruct"` - `LLAMA_SERVER_BASE = "http://192.168.50.108:8081"` diff --git a/README.md b/README.md index 38b4d3d..d75cec9 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ ![cAIc banner](static/readme-banner.png) -# cAIc v0.20.0 +# cAIc v0.21.0 Consumer AI hardware is a wasteland of incompatibility. NVIDIA speaks CUDA, AMD speaks ROCm. Your RTX 5070 Ti lives in one machine with 16 GB VRAM; your RX 6600 XT lives in another with 12 GB. Alone, neither can run a 14B model at usable speed. Together, they could — if the software stack didn't treat heterogeneous hardware as a bug instead of a feature. @@ -37,8 +37,16 @@ To deploy single-node, override the remote service URLs: ```bash export CAIC_QDRANT_URL=http://localhost:6333 export CAIC_EMBED_URL=http://localhost:11434 +export CAIC_EMBED_MODEL=mxbai-embed-large +export CAIC_SEARXNG_BASE=http://localhost:8888 export LLAMA_SERVER_BASE=http://localhost:8081 -# AMQP URL is already configurable via CAIC_AMQP_URL +export CAIC_NODE_NAME=$(hostname) +export CAIC_UPLOAD_DIR=/tmp/caic_uploads +export CAIC_DB_PATH=/opt/caic/caic.db +export CAIC_HOST=0.0.0.0 +export CAIC_PORT=8080 +# AMQP URL is already configurable via CAIC_AMQP_URL or CAIC_AMQP_SECRET_PATH +# Syslog: set CAIC_SYSLOG_ADDRESS to /dev/log (Linux), empty to disable, or a remote address ``` All services degrade gracefully if unreachable — RAG, search, cluster, and triage log warnings and continue. Only llama-server (inference) is strictly required. @@ -47,6 +55,48 @@ Untested: Windows 11 / WSL2 (Debian). The codebase is pure Python with no platfo Under the hood: FastAPI + SQLite + Jinja2 on Python 3.13. AMQP-mediated cluster coordination with an OpenAI-compatible inference endpoint. +## What's New in v0.21.0 + +### Scrollbar + DOM Fixes +- Scrollbar hidden behind `.main::after` barcode strip — fixed by elevating `.chat-container` z-index above the pseudo-element overlay. +- Scrollbar repositioned to the dark-blue channel between content and spool-hole strip via `margin-right: 28px`, widened to 10px. +- `scrollToLatest()` now uses `requestAnimationFrame` so `scrollHeight` reflects rendered content — fixes "responses below viewport" during streaming. +- Direction-aware `_userScrolledAway` guard: in `oldest` mode, detects scroll-away from bottom (not from top, which broke `newest` mode). +- Removed `_userScrolledAway` guard from `oldest` branch to restore always-scroll-to-bottom behavior. + +### Perplexity Persistence +- New `perplexity REAL` column in `messages` table (auto-migration on existing DBs). +- Assistant responses now store `perplexity` alongside content in all storage paths (chat, search, completions). +- Loaded conversations display confidence badges from stored perplexity. + +### DOM Pairing Bugfix +- `appendMessage('assistant', ...)` was finding the **first** `.message.user` via `querySelector`, appending Q2's response to Q1's pair in multi-turn conversations. +- Fixed by capturing `appendMessage('user', ...)` return value and passing the exact user element as `afterEl`. + +### Config Overhaul — All Service URLs Now Env-Overridable +| Env Var | Default | Purpose | +|---------|---------|---------| +| `CAIC_QDRANT_URL` | `http://192.168.50.108:6333` | Qdrant vector search | +| `CAIC_EMBED_URL` | `http://192.168.50.210:11434` | Ollama embeddings | +| `CAIC_EMBED_MODEL` | `mxbai-embed-large` | Embedding model name | +| `CAIC_SEARXNG_BASE` | `http://localhost:8888` | SearXNG web search | +| `CAIC_NODE_NAME` | `ultron` | Coordinator node name | +| `CAIC_UPLOAD_DIR` | `/tmp/caic_uploads` | Upload temp directory | +| `CAIC_DB_PATH` | `/caic.db` | SQLite database path | +| `CAIC_HOST` | `0.0.0.0` | uvicorn listen address | +| `CAIC_PORT` | `8080` | uvicorn listen port | +| `CAIC_SYSLOG_ADDRESS` | `/dev/log` | Syslog socket (empty=disable) | +| `CAIC_AMQP_SECRET_PATH` | `/home/gramps/.caic_amqp_secret` | AMQP password file | +| `CAIC_AMQP_URL` | (from secret file or default) | Full AMQP connection string | + +### Bugfixes +- `hardware.py` Qdrant health check was hardcoded to `192.168.50.108:6333`, bypassing `CAIC_QDRANT_URL` — now uses `QDRANT_URL` from config. +- AMQP fallback password now logs a warning when the secret file is missing. + +### Single-Node Deployment +- Documented at `### Single-Node Deployment (Experimental)` — all services can colocate on localhost with the env vars above. +- Untested: Windows 11 / WSL2 (Debian). No platform-specific code beyond gracefully-absent `rocm-smi` and `system_profiler`. + #### Query-routing vs. layer-splitting — why it matters Most distributed inference tools (llama.cpp RPC, vLLM with tensor parallelism, exo) split a *single model* across multiple GPUs. The first GPU runs layers 0–15, the second runs 16–31, and so on. This works well in a homogeneous cluster where every GPU is identical, but in a heterogeneous setup the slowest card sets the pace — every forward pass waits for the straggler. Communication overhead between GPUs (NCCL, RPC) adds latency too. diff --git a/app.py b/app.py index 143b0c9..bb1dfec 100644 --- a/app.py +++ b/app.py @@ -45,9 +45,11 @@ import routers.cluster as cluster_router # --- Logging --- log = logging.getLogger("caic") log.setLevel(logging.DEBUG) -syslog_handler = logging.handlers.SysLogHandler(address="/dev/log") -syslog_handler.setFormatter(logging.Formatter("caic[%(process)d]: %(levelname)s %(message)s")) -log.addHandler(syslog_handler) +syslog_address = os.environ.get("CAIC_SYSLOG_ADDRESS", "/dev/log") +if syslog_address: + syslog_handler = logging.handlers.SysLogHandler(address=syslog_address) + syslog_handler.setFormatter(logging.Formatter("caic[%(process)d]: %(levelname)s %(message)s")) + log.addHandler(syslog_handler) BASE_DIR = Path(__file__).parent templates = Jinja2Templates(directory=str(BASE_DIR / "templates")) @@ -179,4 +181,4 @@ for router_module in [ if __name__ == "__main__": import uvicorn - uvicorn.run(app, host="0.0.0.0", port=8080) + uvicorn.run(app, host=os.environ.get("CAIC_HOST", "0.0.0.0"), port=int(os.environ.get("CAIC_PORT", "8080"))) diff --git a/cluster.py b/cluster.py index ea8fd9a..a13a3fc 100644 --- a/cluster.py +++ b/cluster.py @@ -4,6 +4,7 @@ Maintains node registry, event log, coordinator state, and ping-based health che """ import asyncio import logging +import os import uuid from collections import deque from datetime import datetime, timezone @@ -17,7 +18,7 @@ CLUSTER_NODES: dict[str, dict] = {} CLUSTER_EVENTS: deque = deque(maxlen=1000) CLUSTER_COORDINATOR: str | None = None _pending_pings: dict[str, tuple[str, asyncio.Event]] = {} -NODE_NAME: str = "ultron" +NODE_NAME: str = os.environ.get("CAIC_NODE_NAME", "ultron") PING_TIMEOUT: float = 5.0 diff --git a/config.py b/config.py index 14a5611..24b2582 100644 --- a/config.py +++ b/config.py @@ -9,10 +9,10 @@ import logging log = logging.getLogger("caic") -VERSION = "v0.20.0" +VERSION = "v0.21.0" OLLAMA_BASE = os.environ.get("OLLAMA_BASE", "http://localhost:11434") LLAMA_SERVER_BASE = os.environ.get("LLAMA_SERVER_BASE", "http://192.168.50.108:8081") -SEARXNG_BASE = "http://localhost:8888" +SEARXNG_BASE = os.environ.get("CAIC_SEARXNG_BASE", "http://localhost:8888") DEFAULT_MODEL = "qwen2.5-7b-instruct" COMPLETIONS_API_KEY = os.environ.get("CAIC_COMPLETIONS_API_KEY", "caic-sk-" + os.urandom(24).hex()) MODEL_CONTEXT_LENGTH = 4096 @@ -32,6 +32,7 @@ def get_amqp_url() -> str: pw = f.read().strip() except (FileNotFoundError, OSError): pw = "password" + log.warning("AMQP secret file not found at %s — using default password", AMQP_SECRET_PATH) return f"amqp://caic:{pw}@localhost:5672/caic" # --- Auth --- @@ -65,7 +66,7 @@ BODY_LIMIT_CHAT_BYTES = 128 * 1024 BODY_LIMIT_PROFILE_BYTES = 256 * 1024 # --- Upload --- -UPLOAD_DIR = "/tmp/caic_uploads" +UPLOAD_DIR = os.environ.get("CAIC_UPLOAD_DIR", "/tmp/caic_uploads") MAX_UPLOAD_BYTES = 20 * 1024 * 1024 SUPPORTED_UPLOAD_TYPES = {"text/plain", "text/markdown", "application/pdf", "application/json", "text/x-python", "text/html", "image/png", "image/jpeg", "image/gif", "image/svg+xml", "image/webp"} QDRANT_URL = os.environ.get("CAIC_QDRANT_URL", "http://192.168.50.108:6333") diff --git a/db.py b/db.py index 38515de..b85c59a 100644 --- a/db.py +++ b/db.py @@ -20,7 +20,7 @@ from crypto import encrypt_text, decrypt_text log = logging.getLogger("caic") BASE_DIR = Path(__file__).parent -DB_PATH = BASE_DIR / "caic.db" +DB_PATH = Path(os.environ.get("CAIC_DB_PATH", str(BASE_DIR / "caic.db"))) def get_db(): diff --git a/docs/wiki/current-wip.md b/docs/wiki/current-wip.md index 9d6fd8b..ea4fcac 100644 --- a/docs/wiki/current-wip.md +++ b/docs/wiki/current-wip.md @@ -9,6 +9,7 @@ Scope: Active roadmap items and backlog. - **B8 (v0.19.3)** — Private Chat mode. Backend skip-DB/skip-RAG/skip-search flag, frontend PRIVATE badge, info popup. - **WireGuard TLS (v0.19.4)** — Self-signed WireGuard mesh encrypts all inter-node traffic (AMQP, inference, RPC). No code changes to cAIc. Documented in wiki/WireGuard-Setup.md + docker.md §5.4. - **At-Rest Encryption (v0.20.0)** — AES-256-GCM encrypts all query-derived text at rest. crypto.py with auto-keygen, key stored as `heartbeat_interval_ms` in settings. All 12 storage paths wired (SQLite: messages, conversations, memories, upload_context; Qdrant: RAG chunks, ingest, upload). 200 tests pass. +- **v0.21.0** — Scrollbar/DOM fixes, perplexity persistence per message, all service URLs env-overridable, single-node deployment docs, DOM pairing bugfix, hardware.py Qdrant URL bugfix. ## Backlog diff --git a/hardware.py b/hardware.py index 7d6f873..9dac24b 100644 --- a/hardware.py +++ b/hardware.py @@ -12,7 +12,7 @@ from pathlib import Path import httpx import psutil -from config import LLAMA_SERVER_BASE, SEARXNG_BASE +from config import LLAMA_SERVER_BASE, SEARXNG_BASE, QDRANT_URL log = logging.getLogger("caic") @@ -95,7 +95,7 @@ async def assess_hardware() -> dict: qdrant_collections = [] try: async with httpx.AsyncClient(timeout=3) as client: - resp = await client.get("http://192.168.50.108:6333/collections") + resp = await client.get(f"{QDRANT_URL}/collections") if resp.status_code == 200: qdrant_reachable = True data = resp.json() diff --git a/rag.py b/rag.py index 6cd6a63..7563951 100644 --- a/rag.py +++ b/rag.py @@ -17,7 +17,7 @@ from config import MAX_SKILL_PROMPT_CHARS, QDRANT_URL, RAG_COLLECTION log = logging.getLogger("caic") EMBED_URL = os.environ.get("CAIC_EMBED_URL", "http://192.168.50.210:11434") -EMBED_MODEL = "mxbai-embed-large" +EMBED_MODEL = os.environ.get("CAIC_EMBED_MODEL", "mxbai-embed-large") RAG_SCORE_THRESHOLD = 0.25 # Re-export eviction symbols for backward compatibility