gramps/cAIc
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

v0.21.0 — perplexity persistence, env-overridable config, scroll/DOM fixes, single-node docs

Browse Source
This commit is contained in:
gramps
2026-07-14 14:39:30 -07:00
parent 3f64bed485
commit 7acea605ac
9 changed files with 74 additions and 16 deletions
Download Patch File Download Diff File Expand all files Collapse all files
AGENTS.md
+5 -2
View File
@@ -137,7 +137,7 @@ All streaming endpoints yield `data: {json}\n\n`. Key shapes:
## Work State
### Completed this session
- **At-Rest Encryption (v0.20.0)** — `crypto.py` with AES-256-GCM encrypt/decrypt + `ensure_key()`. Key auto-generated on first boot, stored as `heartbeat_interval_ms` in settings (never exposed via API). All 12 storage call sites wired: `routers/chat.py`, `routers/search_route.py`, `routers/conversations.py`, `routers/completions.py`, `memory.py`, `db.py` (upload_context), `rag.py`, `routers/upload.py`, `routers/ingest.py`. `memory.py` FTS5 search replaced with Python-side matching (encrypted facts can't be FTS5-indexed). `app.py` lifespan calls `ensure_key()` after `init_db()`. 200/200 tests pass. Deployed to jarvis with fresh DB.
- **v0.21.0 Release** — Scrollbar/DOM fixes, perplexity persistence, all service URLs env-overridable, single-node deployment docs, hardware.py Qdrant URL bugfix. See README for full changelog.
### Active
- (none)
@@ -145,12 +145,15 @@ All streaming endpoints yield `data: {json}\n\n`. Key shapes:
### Blocked
- (none)
### Blocked
- (none)
### Upcoming (backlog)
- B4 — RAG Corpus Management UI
- Private Chat mode was implemented in B8 (v0.19.3). WireGuard in-transit encryption documented in wiki. At-rest encryption implemented in v0.20.0.
### Key config values (current)
- `VERSION = "v0.20.0"` in `config.py`
- **Current VERSION**: `v0.21.0` in `config.py`.
- `SESSION_TIMEOUT_SECONDS = 3600`
- `DEFAULT_MODEL = "qwen2.5-7b-instruct"`
- `LLAMA_SERVER_BASE = "http://192.168.50.108:8081"`
README.md
+52 -2
View File
@@ -1,6 +1,6 @@
![cAIc banner](static/readme-banner.png)
# cAIc v0.20.0
# cAIc v0.21.0
Consumer AI hardware is a wasteland of incompatibility. NVIDIA speaks CUDA, AMD speaks ROCm. Your RTX 5070 Ti lives in one machine with 16 GB VRAM; your RX 6600 XT lives in another with 12 GB. Alone, neither can run a 14B model at usable speed. Together, they could — if the software stack didn't treat heterogeneous hardware as a bug instead of a feature.
@@ -37,8 +37,16 @@ To deploy single-node, override the remote service URLs:
```bash
export CAIC_QDRANT_URL=http://localhost:6333
export CAIC_EMBED_URL=http://localhost:11434
export CAIC_EMBED_MODEL=mxbai-embed-large
export CAIC_SEARXNG_BASE=http://localhost:8888
export LLAMA_SERVER_BASE=http://localhost:8081
# AMQP URL is already configurable via CAIC_AMQP_URL
export CAIC_NODE_NAME=$(hostname)
export CAIC_UPLOAD_DIR=/tmp/caic_uploads
export CAIC_DB_PATH=/opt/caic/caic.db
export CAIC_HOST=0.0.0.0
export CAIC_PORT=8080
# AMQP URL is already configurable via CAIC_AMQP_URL or CAIC_AMQP_SECRET_PATH
# Syslog: set CAIC_SYSLOG_ADDRESS to /dev/log (Linux), empty to disable, or a remote address
```
All services degrade gracefully if unreachable — RAG, search, cluster, and triage log warnings and continue. Only llama-server (inference) is strictly required.
@@ -47,6 +55,48 @@ Untested: Windows 11 / WSL2 (Debian). The codebase is pure Python with no platfo
Under the hood: FastAPI + SQLite + Jinja2 on Python 3.13. AMQP-mediated cluster coordination with an OpenAI-compatible inference endpoint.
## What's New in v0.21.0
### Scrollbar + DOM Fixes
- Scrollbar hidden behind `.main::after` barcode strip — fixed by elevating `.chat-container` z-index above the pseudo-element overlay.
- Scrollbar repositioned to the dark-blue channel between content and spool-hole strip via `margin-right: 28px`, widened to 10px.
- `scrollToLatest()` now uses `requestAnimationFrame` so `scrollHeight` reflects rendered content — fixes "responses below viewport" during streaming.
- Direction-aware `_userScrolledAway` guard: in `oldest` mode, detects scroll-away from bottom (not from top, which broke `newest` mode).
- Removed `_userScrolledAway` guard from `oldest` branch to restore always-scroll-to-bottom behavior.
### Perplexity Persistence
- New `perplexity REAL` column in `messages` table (auto-migration on existing DBs).
- Assistant responses now store `perplexity` alongside content in all storage paths (chat, search, completions).
- Loaded conversations display confidence badges from stored perplexity.
### DOM Pairing Bugfix
- `appendMessage('assistant', ...)` was finding the **first** `.message.user` via `querySelector`, appending Q2's response to Q1's pair in multi-turn conversations.
- Fixed by capturing `appendMessage('user', ...)` return value and passing the exact user element as `afterEl`.
### Config Overhaul — All Service URLs Now Env-Overridable
| Env Var | Default | Purpose |
|---------|---------|---------|
| `CAIC_QDRANT_URL` | `http://192.168.50.108:6333` | Qdrant vector search |
| `CAIC_EMBED_URL` | `http://192.168.50.210:11434` | Ollama embeddings |
| `CAIC_EMBED_MODEL` | `mxbai-embed-large` | Embedding model name |
| `CAIC_SEARXNG_BASE` | `http://localhost:8888` | SearXNG web search |
| `CAIC_NODE_NAME` | `ultron` | Coordinator node name |
| `CAIC_UPLOAD_DIR` | `/tmp/caic_uploads` | Upload temp directory |
| `CAIC_DB_PATH` | `<cwd>/caic.db` | SQLite database path |
| `CAIC_HOST` | `0.0.0.0` | uvicorn listen address |
| `CAIC_PORT` | `8080` | uvicorn listen port |
| `CAIC_SYSLOG_ADDRESS` | `/dev/log` | Syslog socket (empty=disable) |
| `CAIC_AMQP_SECRET_PATH` | `/home/gramps/.caic_amqp_secret` | AMQP password file |
| `CAIC_AMQP_URL` | (from secret file or default) | Full AMQP connection string |
### Bugfixes
- `hardware.py` Qdrant health check was hardcoded to `192.168.50.108:6333`, bypassing `CAIC_QDRANT_URL` — now uses `QDRANT_URL` from config.
- AMQP fallback password now logs a warning when the secret file is missing.
### Single-Node Deployment
- Documented at `### Single-Node Deployment (Experimental)` — all services can colocate on localhost with the env vars above.
- Untested: Windows 11 / WSL2 (Debian). No platform-specific code beyond gracefully-absent `rocm-smi` and `system_profiler`.
#### Query-routing vs. layer-splitting — why it matters
Most distributed inference tools (llama.cpp RPC, vLLM with tensor parallelism, exo) split a *single model* across multiple GPUs. The first GPU runs layers 0–15, the second runs 16–31, and so on. This works well in a homogeneous cluster where every GPU is identical, but in a heterogeneous setup the slowest card sets the pace — every forward pass waits for the straggler. Communication overhead between GPUs (NCCL, RPC) adds latency too.
app.py
+6 -4
View File
@@ -45,9 +45,11 @@ import routers.cluster as cluster_router
# --- Logging ---
log = logging.getLogger("caic")
log.setLevel(logging.DEBUG)
syslog_handler = logging.handlers.SysLogHandler(address="/dev/log")
syslog_handler.setFormatter(logging.Formatter("caic[%(process)d]: %(levelname)s %(message)s"))
log.addHandler(syslog_handler)
syslog_address = os.environ.get("CAIC_SYSLOG_ADDRESS", "/dev/log")
if syslog_address:
syslog_handler = logging.handlers.SysLogHandler(address=syslog_address)
syslog_handler.setFormatter(logging.Formatter("caic[%(process)d]: %(levelname)s %(message)s"))
log.addHandler(syslog_handler)
BASE_DIR = Path(__file__).parent
templates = Jinja2Templates(directory=str(BASE_DIR / "templates"))
@@ -179,4 +181,4 @@ for router_module in [
if __name__ == "__main__":
import uvicorn
uvicorn.run(app, host="0.0.0.0", port=8080)
uvicorn.run(app, host=os.environ.get("CAIC_HOST", "0.0.0.0"), port=int(os.environ.get("CAIC_PORT", "8080")))
cluster.py
+2 -1
View File
@@ -4,6 +4,7 @@ Maintains node registry, event log, coordinator state, and ping-based health che
"""
import asyncio
import logging
import os
import uuid
from collections import deque
from datetime import datetime, timezone
@@ -17,7 +18,7 @@ CLUSTER_NODES: dict[str, dict] = {}
CLUSTER_EVENTS: deque = deque(maxlen=1000)
CLUSTER_COORDINATOR: str | None = None
_pending_pings: dict[str, tuple[str, asyncio.Event]] = {}
NODE_NAME: str = "ultron"
NODE_NAME: str = os.environ.get("CAIC_NODE_NAME", "ultron")
PING_TIMEOUT: float = 5.0
config.py
+4 -3
View File
@@ -9,10 +9,10 @@ import logging
log = logging.getLogger("caic")
VERSION = "v0.20.0"
VERSION = "v0.21.0"
OLLAMA_BASE = os.environ.get("OLLAMA_BASE", "http://localhost:11434")
LLAMA_SERVER_BASE = os.environ.get("LLAMA_SERVER_BASE", "http://192.168.50.108:8081")
SEARXNG_BASE = "http://localhost:8888"
SEARXNG_BASE = os.environ.get("CAIC_SEARXNG_BASE", "http://localhost:8888")
DEFAULT_MODEL = "qwen2.5-7b-instruct"
COMPLETIONS_API_KEY = os.environ.get("CAIC_COMPLETIONS_API_KEY", "caic-sk-" + os.urandom(24).hex())
MODEL_CONTEXT_LENGTH = 4096
@@ -32,6 +32,7 @@ def get_amqp_url() -> str:
pw = f.read().strip()
except (FileNotFoundError, OSError):
pw = "password"
log.warning("AMQP secret file not found at %s — using default password", AMQP_SECRET_PATH)
return f"amqp://caic:{pw}@localhost:5672/caic"
# --- Auth ---
@@ -65,7 +66,7 @@ BODY_LIMIT_CHAT_BYTES = 128 * 1024
BODY_LIMIT_PROFILE_BYTES = 256 * 1024
# --- Upload ---
UPLOAD_DIR = "/tmp/caic_uploads"
UPLOAD_DIR = os.environ.get("CAIC_UPLOAD_DIR", "/tmp/caic_uploads")
MAX_UPLOAD_BYTES = 20 * 1024 * 1024
SUPPORTED_UPLOAD_TYPES = {"text/plain", "text/markdown", "application/pdf", "application/json", "text/x-python", "text/html", "image/png", "image/jpeg", "image/gif", "image/svg+xml", "image/webp"}
QDRANT_URL = os.environ.get("CAIC_QDRANT_URL", "http://192.168.50.108:6333")
db.py
+1 -1
View File
@@ -20,7 +20,7 @@ from crypto import encrypt_text, decrypt_text
log = logging.getLogger("caic")
BASE_DIR = Path(__file__).parent
DB_PATH = BASE_DIR / "caic.db"
DB_PATH = Path(os.environ.get("CAIC_DB_PATH", str(BASE_DIR / "caic.db")))
def get_db():
docs/wiki/current-wip.md
+1
View File
@@ -9,6 +9,7 @@ Scope: Active roadmap items and backlog.
- **B8 (v0.19.3)** — Private Chat mode. Backend skip-DB/skip-RAG/skip-search flag, frontend PRIVATE badge, info popup.
- **WireGuard TLS (v0.19.4)** — Self-signed WireGuard mesh encrypts all inter-node traffic (AMQP, inference, RPC). No code changes to cAIc. Documented in wiki/WireGuard-Setup.md + docker.md §5.4.
- **At-Rest Encryption (v0.20.0)** — AES-256-GCM encrypts all query-derived text at rest. crypto.py with auto-keygen, key stored as `heartbeat_interval_ms` in settings. All 12 storage paths wired (SQLite: messages, conversations, memories, upload_context; Qdrant: RAG chunks, ingest, upload). 200 tests pass.
- **v0.21.0** — Scrollbar/DOM fixes, perplexity persistence per message, all service URLs env-overridable, single-node deployment docs, DOM pairing bugfix, hardware.py Qdrant URL bugfix.
## Backlog
hardware.py
+2 -2
View File
@@ -12,7 +12,7 @@ from pathlib import Path
import httpx
import psutil
from config import LLAMA_SERVER_BASE, SEARXNG_BASE
from config import LLAMA_SERVER_BASE, SEARXNG_BASE, QDRANT_URL
log = logging.getLogger("caic")
@@ -95,7 +95,7 @@ async def assess_hardware() -> dict:
qdrant_collections = []
try:
async with httpx.AsyncClient(timeout=3) as client:
resp = await client.get("http://192.168.50.108:6333/collections")
resp = await client.get(f"{QDRANT_URL}/collections")
if resp.status_code == 200:
qdrant_reachable = True
data = resp.json()
rag.py
+1 -1
View File
@@ -17,7 +17,7 @@ from config import MAX_SKILL_PROMPT_CHARS, QDRANT_URL, RAG_COLLECTION
log = logging.getLogger("caic")
EMBED_URL = os.environ.get("CAIC_EMBED_URL", "http://192.168.50.210:11434")
EMBED_MODEL = "mxbai-embed-large"
EMBED_MODEL = os.environ.get("CAIC_EMBED_MODEL", "mxbai-embed-large")
RAG_SCORE_THRESHOLD = 0.25
# Re-export eviction symbols for backward compatibility