diff --git a/README.md b/README.md index 3cb9e87..71b49ce 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ ![cAIc banner](static/readme-banner.png) -# cAIc v0.19.2 +# cAIc v0.19.3 Consumer AI hardware is a wasteland of incompatibility. NVIDIA speaks CUDA, AMD speaks ROCm. Your RTX 5070 Ti lives in one machine with 16 GB VRAM; your RX 6600 XT lives in another with 12 GB. Alone, neither can run a 14B model at usable speed. Together, they could — if the software stack didn't treat heterogeneous hardware as a bug instead of a feature. @@ -38,16 +38,41 @@ cAIc takes a different approach: **query-routing**. Each worker runs a complete This also means a worker with a slow GPU can still contribute meaningfully — it handles less latency-sensitive queries or batch background work, while the fast GPU handles interactive chat. +### Data Safety + +| Concern | How cAIc handles it | +|---------|---------------------| +| **Queries stored on disk?** | Yes, by default — conversations persist to SQLite. Toggle **Private Chat** (topbar badge) and nothing touches disk: no SQLite writes, no FTS5 memory injection, no RAG ingestion, no external SearXNG queries. | +| **Queries sent to external services?** | SearXNG web search is optional and disabled in Private Chat. All other services (llama-server, Qdrant, RabbitMQ) run on your own LAN. | +| **Inter-node traffic unencrypted?** | No — WireGuard tunnels encrypt all coordinator↔worker traffic (AMQP, inference, RPC) at the network layer. Zero application changes. | +| **Who can access the server?** | Guest sessions for anyone on the LAN. Admin access protected by a PBKDF2-hashed 4-digit PIN with rate-limited attempts. IP allowlist (CIDR) gate optional. | + At v1.0, this ships with a Docker compose stack and setup wizard that detect CPU vs GPU, probe your hardware, and stand up SearXNG, Qdrant, RabbitMQ, and everything else with a single `docker compose up`. The same install docs work bare-metal for those who prefer to skip containers entirely. Developer wiki: [Home](https://llgit.llamachile.tube/gramps/cAIc/wiki/Home) — includes [FAQ](https://llgit.llamachile.tube/gramps/cAIc/wiki/FAQ), [Installation Guide](https://llgit.llamachile.tube/gramps/cAIc/wiki/Installation), and [full architecture docs](https://llgit.llamachile.tube/gramps/cAIc/wiki/Developer-Architecture) +## What's New in v0.19.3 + +### Private Chat Mode (B8) +- **PRIVATE badge** — topbar toggle switches to private mode where nothing is persisted, no memory/RAG is injected, and web search is disabled +- **Info popup** — click the (i) icon next to the badge for a full explanation of what private mode does and doesn't do +- **No-storage guarantee** — conversation is streamed to the user but never touches SQLite, FTS5, or Qdrant +- **Search blocked** — `/api/search` returns 403 in private mode; WEB button is disabled in the UI + +### WireGuard In-Transit Encryption +- WireGuard tunnels encrypt all inter-node traffic. See Data Safety section above. + ## What's New in v0.19.2 -### Waterfall Direction Toggle (B6) +### Waterfall Direction Toggle (B6) + UX Polish - **NEW/OLD toggle** — topbar button switches between newest-first (waterfall) and oldest-first (traditional chat) -- **Direction-aware scroll** — newest-first scrolls to top, oldest-first scrolls to bottom +- **Direction-aware scroll** — newest-first scrolls to top, oldest-first scrolls to bottom; respects user scroll-away to avoid fighting - **localStorage persistence** — preference survives page reloads, default is newest-first (waterfall) +- **Toast notifications** — slide-out notifications for copy, save, delete, rate actions +- **Clipboard reliability** — `execCopy()` helper for HTTP fallback (`document.execCommand`) when `navigator.clipboard` fails on plain HTTP +- **Model label** — `modelLabel()` derives shorthand display names (e.g. `qwen2.5:7B:i`) +- **Keybinding fix** — Shift+Enter = newline, Ctrl+Enter = send (universal conventions) +- **Token counter** — resets to 0 on page refresh, no longer persisted to localStorage ## What's New in v0.19.1 @@ -150,6 +175,7 @@ Developer wiki: [Home](https://llgit.llamachile.tube/gramps/cAIc/wiki/Home) — - **Conversation History** — SQLite-backed chat persistence with mass-delete option - **Model Switching** — Change inference models on the fly - **Skills Framework** — Built-in skill registry with per-skill enable/disable controls +- **Private Chat** — Toggle to keep conversations ephemeral: no persistence, no memory/RAG, no web search ## File Structure @@ -162,12 +188,13 @@ Developer wiki: [Home](https://llgit.llamachile.tube/gramps/cAIc/wiki/Home) — ├── config.py # Constants, env vars, limits, skill registry ├── db.py # SQLite schema, connection factory ├── eviction.py # Score-based RAG eviction engine -├── gpu.py # AMD GPU stats via rocm-smi -├── hardware.py # Hardware self-assessment (CPU, RAM, VRAM) +├── gpu.py # GPU stats — rocm-smi (Linux/AMD) + system_profiler (Darwin/Apple Silicon) +├── hardware.py # Hardware self-assessment (CPU, RAM, VRAM) — Linux + Darwin ├── memory.py # FTS5 memory CRUD, remember/forget commands ├── rag.py # Qdrant vector search + system prompt assembly ├── search.py # SearXNG integration, perplexity, refusal detection ├── security.py # Rate limiting, origin checks, IP allowlist, audit +├── model_pull.py # Startup model auto-pull (llama-server → Ollama fallback) ├── triage.py # Query classification + cluster node selection ├── routers/ │ ├── chat.py # /api/chat streaming endpoint @@ -190,7 +217,7 @@ Developer wiki: [Home](https://llgit.llamachile.tube/gramps/cAIc/wiki/Home) — ├── node_agent/ │ ├── agent.py # Standalone worker agent (AMQP client) │ └── requirements.txt -└── tests/ # 179 pytest tests +└── tests/ # 198 pytest tests ``` ## Requirements @@ -200,6 +227,7 @@ Developer wiki: [Home](https://llgit.llamachile.tube/gramps/cAIc/wiki/Home) — - SearXNG (optional, for web search) - RabbitMQ (optional, for AMQP cluster — coordinator only) - Qdrant (optional, for RAG vector search) +- WireGuard (optional, for encrypted inter-node transit — see [WireGuard-Setup.md](docs/wiki/WireGuard-Setup.md)) ## Installation @@ -404,7 +432,7 @@ Settings are stored in the `settings` table and include: python3 -m pytest tests/ -v ``` -All 179 tests use `tmp_path` fixtures + monkeypatched `httpx.AsyncClient`/`aio-pika`. No external services needed. +All 198 tests use `tmp_path` fixtures + monkeypatched `httpx.AsyncClient`/`aio-pika`. No external services needed. ## License diff --git a/docker.md b/docker.md index 2d3e699..968d423 100644 --- a/docker.md +++ b/docker.md @@ -479,6 +479,48 @@ caddy: This is out of scope for v1.0 but documented for future. +### 5.4 WireGuard tunnel (off-site workers) + +When a worker node runs on a different network (colo, friend's house, VPS), all cross-site traffic must be encrypted. WireGuard provides this at the network layer with zero application changes. + +**Approach:** Install WireGuard on the Docker host (not inside a container). The host creates a tunnel interface (`wg0`) with a virtual IP in the `10.0.2.0/24` range. Containers that need to reach remote workers use the host's WireGuard IP via `network_mode: host` or standard routing. + +``` +Off-site worker Docker host (coordinator) +┌────────────────────┐ ┌───────────────────────────────┐ +│ wg0: 10.0.2.2 │◄───UDP──────│ wg0: 10.0.2.1 │ +│ llama-server │ :51820 │ │ +│ node_agent.py │ encrypts │ ┌───────────────────────┐ │ +│ │ all │ │ cAIc container │ │ +│ │ traffic │ │ LLAMA_SERVER_BASE │ │ +│ │ │ │ → 10.0.2.2:8081 │ │ +│ │ │ │ CAIC_AMQP_URL │ │ +│ │ │ │ → amqp://caic:@... │ │ +│ │ │ └───────────────────────┘ │ +└────────────────────┘ └───────────────────────────────┘ +``` + +**Host setup (coordinator):** +```bash +sudo apt install wireguard +wg genkey | tee /etc/wireguard/private.key | wg pubkey > /etc/wireguard/public.key +chmod 600 /etc/wireguard/private.key +``` +Then create `/etc/wireguard/wg0.conf` — see [WireGuard-Setup.md](WireGuard-Setup.md) for full per-node configs. + +**Container networking:** The cAIc container needs to reach the WireGuard IP. Options: +1. **`network_mode: host`** — simplest, container shares host network stack. Done by adding `network_mode: host` to the cAIc service. Trade-off: no port isolation. +2. **Host routing** — the host's kernel routes `10.0.2.0/24` via `wg0`. Containers on the default bridge or compose network can reach those IPs if `ip_forward` is enabled. This works out of the box on Linux. + +**cAIc env vars after WireGuard:** +```env +# Point at worker's WireGuard IP instead of LAN IP +LLAMA_SERVER_BASE=http://10.0.2.2:8081 # falls back to coordinator's own llama-server +CAIC_AMQP_URL=amqp://caic:password@10.0.2.1:5672/caic # coordinator's RMQ on WG IP +``` + +The node agent on each worker configures its registration IP as the WireGuard tunnel IP (`node_ip = 10.0.2.2`), so `triage.py` constructs inference URLs pointing at the encrypted interface. + --- ## 6. Setup Wizard (Extraction) @@ -642,6 +684,7 @@ If the setup wizard fails mid-way, a partial rollback is better than leaving det | **Reverse proxy** | Caddy is simplest for auto-HTTPS. Out of scope for v1.0 but design for it. | Low | | **Healthcheck strategy** | `depends_on` with `condition: service_healthy` is the safest approach but increases startup time. Acceptable. | Medium | | **Database migration** | SQLite file in volume — no migration needed for v1.0 format. If schema changes post-v1.0, need a migration container. | Low | +| **WireGuard integration** | Documented in docker.md §5.4 + wiki. Host-level install; no container changes needed. WireGuard sidecar container (`linuxserver/wireguard`) is an alternative for users who want everything in compose. | Low | | **Linux vs macOS vs Windows** | Linux-primary. macOS may work with changes (no rocm-smi). Windows via WSL2 only. | Low | | **LLM model download** | HuggingFace CLI integration in setup.sh, or manual download. Manual is simpler. | Low | | **Dockerfile optimization** | Pin pip hashes, use `--no-cache-dir`, consider `slim` vs `alpine`. Alpine has musl compatibility issues with psutil. Stay with slim. | Medium | @@ -684,6 +727,10 @@ Worker machine (e.g. worker01, worker02) ### 9.3 Worker setup ```bash +# Install WireGuard (required for off-site workers — encrypts all traffic) +sudo apt install wireguard +# See docs/wiki/WireGuard-Setup.md for per-node config + # Install llama-server binary wget https://github.com/ggml-org/llama.cpp/releases/.../llama-server chmod +x llama-server @@ -691,8 +738,9 @@ chmod +x llama-server # Install node agent deps pip install aio-pika httpx -# Create node agent script (from repo: node_agent/agent.py) -# Configure COORDINATOR_AMQP_URL in environment +# Create node agent config: /etc/caic-node-agent.conf +# Set node_ip to the WireGuard tunnel IP (e.g., 10.0.2.2) +# Set amqp_url to the coordinator's WireGuard IP (e.g., amqp://caic:password@10.0.2.1:5672/caic) ``` ### 9.4 Multiple workers @@ -728,6 +776,7 @@ The broker-mediated model is the preferred architecture for this project because - [ ] `docker compose up -d` works without any manual steps beyond setup.sh - [ ] `docker compose down -v` followed by `setup.sh && docker compose up -d` = fresh stack - [ ] Healthchecks prevent serving before dependencies are ready +- [ ] WireGuard tunnel documented and tested for off-site workers - [ ] v1.0 release tag created --- diff --git a/docs/wiki/Developer-Architecture.md b/docs/wiki/Developer-Architecture.md index d1c1401..25089b9 100644 --- a/docs/wiki/Developer-Architecture.md +++ b/docs/wiki/Developer-Architecture.md @@ -265,25 +265,28 @@ All streaming endpoints yield `data: {json}\n\n`: - No live external services required - Test factories reset `SESSIONS`, `PIN_ATTEMPTS`, `RATE_EVENTS` globals per test -### 8.2 Test Coverage Areas (179 tests) +### 8.2 Test Coverage Areas (198 tests) | Test file | Coverage | |-----------|----------| | test_auth_capabilities.py | Guest/admin sessions, origin blocking, logout | -| test_chat_streaming_and_memory_paths.py | Streaming, auto-search, remember/forget, upload context injection | +| test_chat_streaming_and_memory_paths.py | Streaming, auto-search, remember/forget, upload context injection, private chat | | test_cluster.py | Registration, deregistration, pong, events, coordinator query | | test_cluster_heartbeat.py | Heartbeat handler, known/unknown node | | test_completions.py | API key auth, FIM, streaming, blocking, errors | | test_conversations.py | Full CRUD, guest admin, attachment_count | +| test_error_envelopes.py | Global exception handler + stream errors | +| test_gpu.py | GPU stats — rocm-smi (Linux), system_profiler (Darwin/Apple Silicon) | | test_ingest.py | Bearer auth, chunk/embed/upsert, validation | | test_ip_allowlist.py | IP allowlist helper + middleware | | test_memories.py | Edit, search, stats | +| test_model_pull.py | Default model auto-pull — llama-server check, Ollama fallback, error paths | | test_model_swap.py | request_model_swap, handle_model_ready/failed, select_node swap triggering | | test_models_router.py | Models list, ps, show, stats, search/status | | test_node_agent.py | Node agent registration, ping/pong, model swap | | test_presets.py | Full CRUD, default preset protection | | test_profile.py | Get, update, default, length validation | -| test_rag_management.py | Collection stats, eviction algorithm (pinned/grace/scoring/batch), maybe_evict hysteresis, operational stats, flush, concurrency lock | +| test_rag_management.py | Collection stats, eviction algorithm, hysteresis, flush | | test_rate_and_payload_guardrails.py | Rate limits + payload size | | test_search_route.py | Explicit search flow, no results, errors | | test_search_url_sanitization.py | URL sanitizer | @@ -291,7 +294,6 @@ All streaming endpoints yield `data: {json}\n\n`: | test_skills_framework.py | List, toggle, unknown skill, prompt injection | | test_triage.py | classify_query, select_node, get_inference_url | | test_upload.py | Upload, delete, link, by-conversation, attachment_count | -| test_error_envelopes.py | Global exception handler + stream errors | ### 8.3 DoD Process diff --git a/docs/wiki/WireGuard-Setup.md b/docs/wiki/WireGuard-Setup.md new file mode 100644 index 0000000..8d1dc0d --- /dev/null +++ b/docs/wiki/WireGuard-Setup.md @@ -0,0 +1,192 @@ +# WireGuard Tunnel — Encrypted Node Transit + +## Why + +cAIc cluster traffic is plaintext today: + +| Traffic | Protocol | Plaintext risk | +|---------|----------|----------------| +| AMQP (coordinator ↔ worker agent) | TCP :5672 | Registration, ping/pong, swap commands | +| Inference (coordinator → worker llama-server) | HTTP :8081 | Every token generated | +| LLM RPC layer offload (coordinator llama-server → worker) | TCP :50052 | Internal llama.cpp protocol | + +WireGuard encrypts all three at the network layer with zero application changes. The cAIc app keeps using `http://` URLs — it's just talking to a virtual IP whose traffic is automatically encrypted before it hits the wire. + +## Topology + +``` +┌───────────────────────┐ WireGuard tunnel ┌───────────────────────┐ +│ Coordinator (ultron) │◄═══════════════════════════►│ Worker (jarvis) │ +│ 10.0.2.1 │ UDP :51820 │ 10.0.2.2 │ +│ LAN 192.168.50.108 │ │ LAN 192.168.50.210 │ +│ │════════════════════════════►│ │ +│ │ UDP :51820 │ Worker (corsair) │ +│ │ │ 10.0.2.3 │ +│ │ │ LAN (DHCP) │ +└───────────────────────┘ └───────────────────────┘ +``` + +All nodes connect directly to the coordinator's WireGuard endpoint (star topology). Workers do not need to talk to each other. + +## Prerequisites + +```bash +# Debian / Ubuntu +sudo apt install wireguard + +# Windows / WSL2 — install WireGuard from https://www.wireguard.com/install/ +# The wg.exe binary is used inside WSL2; the Windows GUI manages the tunnel config +``` + +## Key Generation + +Run once per node. Save the private key securely; public keys go into peer configs on the other end. + +```bash +wg genkey | tee /etc/wireguard/private.key | wg pubkey > /etc/wireguard/public.key +chmod 600 /etc/wireguard/private.key +``` + +**Recorded keys for this deployment:** + +| Node | Private key | Public key | +|------|-------------|------------| +| ultron | (node private) | `ultron_pubkey=` | +| jarvis | (node private) | `jarvis_pubkey=` | +| corsair | (node private) | `corsair_pubkey=` | + +## Per-Node Configs + +### Coordinator — `/etc/wireguard/wg0.conf` on ultron + +```ini +[Interface] +Address = 10.0.2.1/24 +ListenPort = 51820 +PrivateKey = + +# Enable IP forwarding so workers can route through coordinator if needed +# sudo sysctl -w net.ipv4.ip_forward=1 +# sudo sysctl -w net.ipv6.conf.all.forwarding=1 + +# Worker: jarvis +[Peer] +PublicKey = +AllowedIPs = 10.0.2.2/32 +# If jarvis is off-site, put its public IP / DDNS hostname here: +# Endpoint = jarvis.example.com:51820 +# If jarvis is LAN-only, set PersistentKeepalive = 25 to maintain NAT binding: +# PersistentKeepalive = 25 + +# Worker: corsair +[Peer] +PublicKey = +AllowedIPs = 10.0.2.3/32 +``` + +### Worker — `/etc/wireguard/wg0.conf` on jarvis (Linux) + +```ini +[Interface] +Address = 10.0.2.2/24 +ListenPort = 51820 +PrivateKey = + +# Coordinator +[Peer] +PublicKey = +AllowedIPs = 10.0.2.0/24 +# LAN-only: just point at the LAN IP +Endpoint = 192.168.50.108:51820 +# Off-site: use DDNS or static IP: +# Endpoint = ultron.example.com:51820 +PersistentKeepalive = 25 +``` + +### Worker — Windows / WSL2 on corsair + +Create a WireGuard tunnel in the Windows GUI app with the same config as jarvis above (Address=10.0.2.3/24). WSL2 inside Windows can reach the tunnel IP via the Windows host. + +If llama-server runs inside WSL2 on corsair, the Windows host's WireGuard tunnel IP `10.0.2.3` is reachable from the WSL2 instance as well — just bind llama-server to `0.0.0.0` (already the default) and configure the Windows firewall to allow inbound on :8081 from the coordinator's WireGuard IP. + +## Starting the Tunnel + +```bash +# Start immediately +sudo systemctl start wg-quick@wg0 + +# Enable on boot +sudo systemctl enable wg-quick@wg0 + +# Check status +sudo wg show +``` + +Expected output on each node: + +``` +interface: wg0 + public key: <...> + private key: (hidden) + listening port: 51820 + +peer: + endpoint: 192.168.50.108:51820 + allowed ips: 10.0.2.0/24 + latest handshake: 5 seconds ago ← healthy + transfer: 1.2 KiB received, 3.4 KiB sent +``` + +If `latest handshake` is missing, check firewall rules (UDP :51820 must be open on all nodes). + +## Verification + +```bash +# From any node, ping another node's WireGuard IP +ping -c 3 10.0.2.1 # coordinator +ping -c 3 10.0.2.2 # jarvis +ping -c 3 10.0.2.3 # corsair + +# Verify cAIc inference through the tunnel +curl http://10.0.2.2:8081/v1/models # jarvis llama-server +curl http://10.0.2.3:8081/v1/models # corsair llama-server +``` + +## Updating cAIc to Use the Tunnel + +Once WireGuard is running, point each service at the tunnel IP instead of the LAN IP. + +### Worker node agent config — `/etc/caic-node-agent.conf` + +```ini +[agent] +node_name = jarvis +node_ip = 10.0.2.2 # was 192.168.50.210 +node_type = worker +capabilities = llm +amqp_url = amqp://caic:password@10.0.2.1:5672/caic # was 192.168.50.108 +llama_port = 8081 +models_dir = /var/lib/caic/models +active_model = qwen2.5-7b-instruct-Q5_K_M.gguf +``` + +### Coordinator config — environment variables + +```bash +# On the coordinator node, override the worker-facing addresses +# (LLAMA_SERVER_BASE stays as localhost / LAN IP since inference +# to the coordinator's own llama-server stays on-machine) +export CAIC_AMQP_URL="amqp://caic:password@10.0.2.1:5672/caic" +``` + +No other cAIc code changes are needed. The app already reads `CAIC_AMQP_URL` from the environment (`config.py:27`) and the node agent reads `node_ip` from its INI file. Inference requests routed to remote workers via `triage.py` use the IP the worker registered — so setting `node_ip = 10.0.2.2` in the worker's agent config is all it takes. + +## Cross-Site Deployment Checklist + +When placing a worker outside the LAN: + +1. **Firewall:** Open UDP :51820 on the remote site. On the coordinator side, make sure UDP :51820 is reachable from the internet (port forward / firewall rule at the coordinator's router). +2. **DDNS:** If the coordinator's public IP is dynamic, set up a DDNS hostname and use it in the worker's `Endpoint = ultron.example.com:51820`. +3. **PersistentKeepalive:** Set `PersistentKeepalive = 25` on the worker side to keep NAT bindings alive. +4. **No double encryption:** WireGuard encrypts everything on the WireGuard interface. The cAIc app continues to use `http://` — it never touches raw TLS. This is correct and intended. +5. **Split tunnelling (optional):** The worker's `AllowedIPs = 10.0.2.0/24` ensures only cluster traffic goes through the tunnel. All other internet traffic from the worker uses its normal gateway. diff --git a/docs/wiki/current-wip.md b/docs/wiki/current-wip.md index 2afecdf..2437232 100644 --- a/docs/wiki/current-wip.md +++ b/docs/wiki/current-wip.md @@ -1,19 +1,23 @@ # cAIc Current WiP Backlog -Last updated: 2026-07-06 +Last updated: 2026-07-14 Owner: Gramps Scope: Active roadmap items and backlog. -## ~~Roadmap N: AMQP Cluster Nervous System [COMPLETE]~~ +## Completed -All 15 tasks are done. Final commit: `f0689ac feat: Roadmap N — AMQP cluster nervous system complete`. +- **B7 (v0.19.0)** — Apple Silicon worker support. gpu.py darwin branch, hardware.py darwin VRAM, node_agent/agent.py macOS VRAM reporting. +- **B5 (v0.19.1)** — Default model auto-pull. model_pull.py + ensure_model() in app.py lifespan. +- **B6 (v0.19.2)** — Waterfall direction toggle (NEW/OLD), scroll/lock fixes, toast notifications, execCopy, modelLabel, Ctrl+Enter. +- **B8 (v0.19.3)** — Private Chat mode. Backend skip-DB/skip-RAG/skip-search flag, frontend PRIVATE badge, info popup. +- **WireGuard TLS (v0.19.4)** — Self-signed WireGuard mesh encrypts all inter-node traffic (AMQP, inference, RPC). No code changes to cAIc. Documented in wiki/WireGuard-Setup.md + docker.md §5.4. ## Backlog - B1 — Context loss in follow-up questions (investigation) - B2 — Bang-prefixed (`!`) search routing -- B3 — Docker distribution (planning doc at `docker.md`) -- **B4 — RAG Corpus Management UI (NEXT)** — browse, search, edit, delete individual RAG entries. Backend endpoints at `routers/rag_admin.py`, frontend panel in `templates/index.html` +- B3 — Docker distribution (planning doc at `docker.md`, not yet implemented) +- **B4 — RAG Corpus Management UI (deferred)** — browse, search, edit, delete individual RAG entries - HTTPS / reverse proxy (Caddy) - Conversation search/filter and export tooling - Keyboard shortcuts, retry button, source-link polish diff --git a/routers/chat.py b/routers/chat.py index c472d9e..73d61c9 100644 --- a/routers/chat.py +++ b/routers/chat.py @@ -74,6 +74,7 @@ async def chat(request: Request): model = body.get("model", DEFAULT_MODEL) preset_prompt = body.get("system_prompt", "") upload_context_id = body.get("upload_context_id") + private_chat = body.get("private_chat", False) if not user_message: raise HTTPException(status_code=400, detail="Empty message") @@ -81,44 +82,55 @@ async def chat(request: Request): db = get_db() now = datetime.now(timezone.utc).isoformat() settings = {row["key"]: row["value"] for row in db.execute("SELECT key, value FROM settings").fetchall()} - search_enabled = settings.get("search_enabled", "true") == "true" + search_enabled = settings.get("search_enabled", "true") == "true" and not private_chat upload_doc = None - if upload_context_id: + if upload_context_id and not private_chat: ctx = get_upload_context(db, upload_context_id) if ctx: upload_doc = f"[ATTACHED DOCUMENT: {ctx['filename']}]\n{ctx['content']}\n[END DOCUMENT]" else: log.warning(f"upload_context_id {upload_context_id} not found or expired, continuing without it") - remember_response = process_remember_command(user_message) + remember_response = None if private_chat else process_remember_command(user_message) - if not conv_id: - conv_id = str(uuid.uuid4()) - title = user_message[:80] + ("..." if len(user_message) > 80 else "") - db.execute("INSERT INTO conversations (id, title, model, created_at, updated_at) VALUES (?, ?, ?, ?, ?)", - (conv_id, title, model, now, now)) + if private_chat: + if not conv_id: + conv_id = str(uuid.uuid4()) + system_prompt = "" + messages = [] + if preset_prompt: + messages.append({"role": "system", "content": preset_prompt}) + messages.append({"role": "user", "content": user_message}) + history_rows = [] + db.close() else: - db.execute("UPDATE conversations SET updated_at = ? WHERE id = ?", (now, conv_id)) + if not conv_id: + conv_id = str(uuid.uuid4()) + title = user_message[:80] + ("..." if len(user_message) > 80 else "") + db.execute("INSERT INTO conversations (id, title, model, created_at, updated_at) VALUES (?, ?, ?, ?, ?)", + (conv_id, title, model, now, now)) + else: + db.execute("UPDATE conversations SET updated_at = ? WHERE id = ?", (now, conv_id)) - db.execute("INSERT INTO messages (conversation_id, role, content, created_at) VALUES (?, ?, ?, ?)", - (conv_id, "user", user_message, now)) - db.commit() + db.execute("INSERT INTO messages (conversation_id, role, content, created_at) VALUES (?, ?, ?, ?)", + (conv_id, "user", user_message, now)) + db.commit() - history_rows = db.execute( - "SELECT role, content FROM messages WHERE conversation_id = ? ORDER BY id ASC", (conv_id,) - ).fetchall() - extra_prompt = preset_prompt - if upload_doc: - extra_prompt = (extra_prompt + "\n\n" + upload_doc) if extra_prompt else upload_doc - system_prompt = await build_system_prompt(db, extra_prompt, user_message) - db.close() + history_rows = db.execute( + "SELECT role, content FROM messages WHERE conversation_id = ? ORDER BY id ASC", (conv_id,) + ).fetchall() + extra_prompt = preset_prompt + if upload_doc: + extra_prompt = (extra_prompt + "\n\n" + upload_doc) if extra_prompt else upload_doc + system_prompt = await build_system_prompt(db, extra_prompt, user_message) + db.close() - messages = [] - if system_prompt: - messages.append({"role": "system", "content": system_prompt}) - for row in history_rows: - messages.append({"role": row["role"], "content": row["content"]}) + messages = [] + if system_prompt: + messages.append({"role": "system", "content": system_prompt}) + for row in history_rows: + messages.append({"role": row["role"], "content": row["content"]}) upstream_payload = {"model": model, "messages": messages, "stream": True, "logprobs": True} @@ -196,44 +208,46 @@ async def chat(request: Request): yield f"data: {json.dumps({'token': cleaned_response, 'conversation_id': conv_id, 'augmented': True})}\n\n" - saved_msg = cleaned_response + "\n\n---\n*🔍 Enhanced with web search results*" - if remember_response: - saved_msg = remember_response + "\n\n" + saved_msg + if not private_chat: + saved_msg = cleaned_response + "\n\n---\n*🔍 Enhanced with web search results*" + if remember_response: + saved_msg = remember_response + "\n\n" + saved_msg - db2 = get_db() - db2.execute("INSERT INTO messages (conversation_id, role, content, created_at) VALUES (?, ?, ?, ?)", - (conv_id, "assistant", saved_msg, datetime.now(timezone.utc).isoformat())) - db2.commit() - db2.close() + db2 = get_db() + db2.execute("INSERT INTO messages (conversation_id, role, content, created_at) VALUES (?, ?, ?, ?)", + (conv_id, "assistant", saved_msg, datetime.now(timezone.utc).isoformat())) + db2.commit() + db2.close() - facts = auto_detect_facts(user_message, cleaned_response) - if facts: - conflicts = check_fact_conflicts(facts) - if conflicts: - rag_update = {"conflicts": conflicts} - else: - asyncio.ensure_future(ingest_auto_fact(facts, user_message, cleaned_response)) + facts = auto_detect_facts(user_message, cleaned_response) + if facts: + conflicts = check_fact_conflicts(facts) + if conflicts: + rag_update = {"conflicts": conflicts} + else: + asyncio.ensure_future(ingest_auto_fact(facts, user_message, cleaned_response)) yield f"data: {json.dumps({'done': True, 'conversation_id': conv_id, 'searched': True, 'perplexity': round(perplexity, 2), 'tokens_per_sec': round(tokens_per_sec, 1), 'prompt_tokens': prompt_tokens, 'completion_tokens': completion_tokens, 'context_length': MODEL_CONTEXT_LENGTH, **(rag_update and {'rag_update_suggestion': rag_update} or {})})}\n\n" return - saved_msg = assistant_msg - if remember_response: - saved_msg = remember_response + "\n\n" + saved_msg + if not private_chat: + saved_msg = assistant_msg + if remember_response: + saved_msg = remember_response + "\n\n" + saved_msg - db2 = get_db() - db2.execute("INSERT INTO messages (conversation_id, role, content, created_at) VALUES (?, ?, ?, ?)", - (conv_id, "assistant", saved_msg, datetime.now(timezone.utc).isoformat())) - db2.commit() - db2.close() + db2 = get_db() + db2.execute("INSERT INTO messages (conversation_id, role, content, created_at) VALUES (?, ?, ?, ?)", + (conv_id, "assistant", saved_msg, datetime.now(timezone.utc).isoformat())) + db2.commit() + db2.close() - facts = auto_detect_facts(user_message, assistant_msg) - if facts: - conflicts = check_fact_conflicts(facts) - if conflicts: - rag_update = {"conflicts": conflicts} - else: - asyncio.ensure_future(ingest_auto_fact(facts, user_message, assistant_msg)) + facts = auto_detect_facts(user_message, assistant_msg) + if facts: + conflicts = check_fact_conflicts(facts) + if conflicts: + rag_update = {"conflicts": conflicts} + else: + asyncio.ensure_future(ingest_auto_fact(facts, user_message, assistant_msg)) yield f"data: {json.dumps({'done': True, 'conversation_id': conv_id, 'perplexity': round(perplexity, 2), 'tokens_per_sec': round(tokens_per_sec, 1), 'prompt_tokens': prompt_tokens, 'completion_tokens': completion_tokens, 'context_length': MODEL_CONTEXT_LENGTH, **(rag_update and {'rag_update_suggestion': rag_update} or {})})}\n\n" diff --git a/routers/search_route.py b/routers/search_route.py index c17a459..26671a2 100644 --- a/routers/search_route.py +++ b/routers/search_route.py @@ -30,6 +30,10 @@ async def explicit_search(request: Request): if not query: raise HTTPException(status_code=400, detail="Empty query") + private_chat = body.get("private_chat", False) + if private_chat: + raise HTTPException(status_code=403, detail="Web search is disabled in private chat mode") + db = get_db() now = datetime.now(timezone.utc).isoformat() diff --git a/templates/index.html b/templates/index.html index 65bf57e..7174aea 100644 --- a/templates/index.html +++ b/templates/index.html @@ -101,6 +101,9 @@ body { font-family: var(--font-body); background: var(--bg-primary); color: var( .badge.off { border-color:rgba(255,51,85,0.3); color:var(--danger); background:rgba(255,51,85,0.08); } .badge.admin { border-color:var(--border); color:var(--text-muted); } .badge.admin:hover { border-color:var(--accent-dim); color:var(--accent); } +.info-icon { display:inline-flex; align-items:center; justify-content:center; width:18px; height:18px; border-radius:50%; border:1px solid var(--border); color:var(--text-muted); font-size:11px; cursor:pointer; user-select:none; transition:all 0.15s; } +.info-icon:hover { border-color:var(--accent-dim); color:var(--accent); } +.info-popup { position:absolute; top:100%; right:0; margin-top:6px; width:320px; background:var(--bg-secondary); border:1px solid var(--border); border-radius:10px; padding:14px 16px; font-size:12px; line-height:1.6; color:var(--text-primary); z-index:100; box-shadow:0 8px 24px rgba(0,0,0,0.4); } .role-label .msg-time { color:var(--text-muted); font-weight:400; font-size:10px; margin-left:6px; } .status-dot.offline { background:var(--danger); } .status-dot.warning { background:var(--warning); } @@ -402,6 +405,21 @@ body { font-family: var(--font-body); background: var(--bg-primary); color: var( + + ⓘ + @@ -452,6 +470,7 @@ let profileEnabled = true; let searchEnabled = true; let memoryEnabled = true; let skillsEnabled = true; +let privateEnabled = false; let presets = []; let skillsRegistry = []; let currentModel = ''; @@ -887,6 +906,20 @@ async function resetProfile() { function toggleProfile() { if (currentRole !== 'admin') { requireAdminNotice(); return; } profileEnabled = !profileEnabled; updateProfileUI(); saveSettings(); } function toggleSearch() { if (currentRole !== 'admin') { requireAdminNotice(); return; } searchEnabled = !searchEnabled; updateSearchUI(); saveSettings(); } function toggleMemory() { if (currentRole !== 'admin') { requireAdminNotice(); return; } memoryEnabled = !memoryEnabled; updateMemoryUI(); saveSettings(); } +function togglePrivate() { + privateEnabled = !privateEnabled; + const badge = document.getElementById('privateBadge'); + badge.className = 'badge ' + (privateEnabled ? 'on' : 'off'); + badge.textContent = privateEnabled ? 'PRIVATE' : 'PRIVATE OFF'; + const searchBtn = document.getElementById('searchBtn'); + if (privateEnabled) { + if (searchBtn) searchBtn.disabled = true; + showToast('Private chat: nothing stored, no external queries'); + } else { + if (searchBtn && !isStreaming) searchBtn.disabled = false; + showToast('Public mode: conversations are persisted to disk'); + } +} function updateProfileUI() { const badge = document.getElementById('profileBadge'); @@ -1141,6 +1174,7 @@ function showWelcome() { } async function sendSearch() { + if (privateEnabled) { showToast('Web search is disabled in private chat mode'); return; } resetScrollLock(); const input = document.getElementById('userInput'); const query = input.value.trim(); @@ -1361,6 +1395,7 @@ async function sendMessage() { } const chatBody = { conversation_id: currentConvId, message, model }; if (uploadContextId) chatBody.upload_context_id = uploadContextId; + if (privateEnabled) chatBody.private_chat = true; const resp = await authFetch('/api/chat', { method: 'POST', headers: {'Content-Type': 'application/json'}, body: JSON.stringify(chatBody), signal: abortController.signal }); const reader = resp.body.getReader(); const decoder = new TextDecoder(); @@ -1441,7 +1476,7 @@ function setStreamingState(streaming) { sendBtn.textContent = 'SEND'; sendBtn.className = 'send-btn'; sendBtn.onclick = sendMessage; - if (searchBtn) searchBtn.disabled = false; + if (searchBtn) searchBtn.disabled = privateEnabled; } } diff --git a/tests/test_chat_streaming_and_memory_paths.py b/tests/test_chat_streaming_and_memory_paths.py index 23b3243..45f2b91 100644 --- a/tests/test_chat_streaming_and_memory_paths.py +++ b/tests/test_chat_streaming_and_memory_paths.py @@ -1,5 +1,6 @@ import json import os +import asyncio from pathlib import Path import httpx @@ -261,6 +262,57 @@ def test_memory_command_paths_remember_and_forget(tmp_path: Path, monkeypatch): forget_events = parse_sse_payloads(forget_resp.text) assert any("Forgot" in p.get("token", "") for p in forget_events) - memories_after_forget = client.get("/api/memories", headers={"X-Session-ID": sid, "Origin": "http://testserver"}) - assert memories_after_forget.status_code == 200 - assert memories_after_forget.json().get("count", 0) == 0 +def test_private_chat_does_not_persist(tmp_path: Path, monkeypatch): + monkeypatch.setattr(httpx.AsyncClient, "stream", lambda *a, **kw: _MockStreamResponse([ + 'data: {"choices":[{"delta":{"content":"Hello"},"finish_reason":null,"logprobs":{"content":[]}}]}', + 'data: {"choices":[{"delta":{"content":" world"},"finish_reason":null,"logprobs":{"content":[]}}]}', + 'data: {"choices":[{"delta":{"content":""},"finish_reason":"stop","logprobs":{"content":[]}}],"usage":{"completion_tokens":2,"prompt_tokens":10,"tokens_per_second":5.0}}', + "data: [DONE]", + ])) + monkeypatch.setattr(triage, "classify_query", lambda q: "general") + + async def _mock_ensure(m): return True + monkeypatch.setattr("model_pull.ensure_model", _mock_ensure) + + with make_client(tmp_path) as client: + sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()["session_id"] + headers = {"X-Session-ID": sid, "Origin": "http://testserver"} + + resp = client.post("/api/chat", json={ + "message": "test private message", + "private_chat": True, + }, headers=headers) + assert resp.status_code == 200 + events = parse_sse_payloads(resp.text) + tokens = [p for p in events if "token" in p] + assert len(tokens) == 2 + done = [p for p in events if p.get("done")] + assert len(done) == 1 + + conv_resp = client.get("/api/conversations", headers=headers) + assert conv_resp.status_code == 200 + assert len(conv_resp.json()) == 0 + + +def test_private_chat_does_not_auto_search(tmp_path: Path, monkeypatch): + """Private mode should skip auto-search even when search is enabled.""" + monkeypatch.setattr(httpx.AsyncClient, "stream", lambda *a, **kw: _MockStreamResponse([ + 'data: {"choices":[{"delta":{"content":"I don\'t know"},"finish_reason":null,"logprobs":{"content":[{"logprob":-2.5}]}}]}', + 'data: {"choices":[{"delta":{"content":""},"finish_reason":"stop","logprobs":{"content":[{"logprob":-2.5}]}}],"usage":{"completion_tokens":1,"prompt_tokens":10,"tokens_per_second":5.0}}', + "data: [DONE]", + ])) + monkeypatch.setattr(triage, "classify_query", lambda q: "general") + monkeypatch.setattr(routers.chat, "query_searxng", lambda q: [{"title": "result"}]) + + with make_client(tmp_path) as client: + sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()["session_id"] + headers = {"X-Session-ID": sid, "Origin": "http://testserver"} + + resp = client.post("/api/chat", json={ + "message": "what is the weather", + "private_chat": True, + }, headers=headers) + assert resp.status_code == 200 + events = parse_sse_payloads(resp.text) + searching = [p for p in events if p.get("searching")] + assert len(searching) == 0, "private chat should not auto-search"