B8: Private Chat mode + WireGuard docs + README data safety section (v0.19.3)
This commit is contained in:
@@ -1,6 +1,6 @@
|
|||||||

|

|
||||||
|
|
||||||
# cAIc v0.19.2
|
# cAIc v0.19.3
|
||||||
|
|
||||||
Consumer AI hardware is a wasteland of incompatibility. NVIDIA speaks CUDA, AMD speaks ROCm. Your RTX 5070 Ti lives in one machine with 16 GB VRAM; your RX 6600 XT lives in another with 12 GB. Alone, neither can run a 14B model at usable speed. Together, they could — if the software stack didn't treat heterogeneous hardware as a bug instead of a feature.
|
Consumer AI hardware is a wasteland of incompatibility. NVIDIA speaks CUDA, AMD speaks ROCm. Your RTX 5070 Ti lives in one machine with 16 GB VRAM; your RX 6600 XT lives in another with 12 GB. Alone, neither can run a 14B model at usable speed. Together, they could — if the software stack didn't treat heterogeneous hardware as a bug instead of a feature.
|
||||||
|
|
||||||
@@ -38,16 +38,41 @@ cAIc takes a different approach: **query-routing**. Each worker runs a complete
|
|||||||
|
|
||||||
This also means a worker with a slow GPU can still contribute meaningfully — it handles less latency-sensitive queries or batch background work, while the fast GPU handles interactive chat.
|
This also means a worker with a slow GPU can still contribute meaningfully — it handles less latency-sensitive queries or batch background work, while the fast GPU handles interactive chat.
|
||||||
|
|
||||||
|
### Data Safety
|
||||||
|
|
||||||
|
| Concern | How cAIc handles it |
|
||||||
|
|---------|---------------------|
|
||||||
|
| **Queries stored on disk?** | Yes, by default — conversations persist to SQLite. Toggle **Private Chat** (topbar badge) and nothing touches disk: no SQLite writes, no FTS5 memory injection, no RAG ingestion, no external SearXNG queries. |
|
||||||
|
| **Queries sent to external services?** | SearXNG web search is optional and disabled in Private Chat. All other services (llama-server, Qdrant, RabbitMQ) run on your own LAN. |
|
||||||
|
| **Inter-node traffic unencrypted?** | No — WireGuard tunnels encrypt all coordinator↔worker traffic (AMQP, inference, RPC) at the network layer. Zero application changes. |
|
||||||
|
| **Who can access the server?** | Guest sessions for anyone on the LAN. Admin access protected by a PBKDF2-hashed 4-digit PIN with rate-limited attempts. IP allowlist (CIDR) gate optional. |
|
||||||
|
|
||||||
At v1.0, this ships with a Docker compose stack and setup wizard that detect CPU vs GPU, probe your hardware, and stand up SearXNG, Qdrant, RabbitMQ, and everything else with a single `docker compose up`. The same install docs work bare-metal for those who prefer to skip containers entirely.
|
At v1.0, this ships with a Docker compose stack and setup wizard that detect CPU vs GPU, probe your hardware, and stand up SearXNG, Qdrant, RabbitMQ, and everything else with a single `docker compose up`. The same install docs work bare-metal for those who prefer to skip containers entirely.
|
||||||
|
|
||||||
Developer wiki: [Home](https://llgit.llamachile.tube/gramps/cAIc/wiki/Home) — includes [FAQ](https://llgit.llamachile.tube/gramps/cAIc/wiki/FAQ), [Installation Guide](https://llgit.llamachile.tube/gramps/cAIc/wiki/Installation), and [full architecture docs](https://llgit.llamachile.tube/gramps/cAIc/wiki/Developer-Architecture)
|
Developer wiki: [Home](https://llgit.llamachile.tube/gramps/cAIc/wiki/Home) — includes [FAQ](https://llgit.llamachile.tube/gramps/cAIc/wiki/FAQ), [Installation Guide](https://llgit.llamachile.tube/gramps/cAIc/wiki/Installation), and [full architecture docs](https://llgit.llamachile.tube/gramps/cAIc/wiki/Developer-Architecture)
|
||||||
|
|
||||||
|
## What's New in v0.19.3
|
||||||
|
|
||||||
|
### Private Chat Mode (B8)
|
||||||
|
- **PRIVATE badge** — topbar toggle switches to private mode where nothing is persisted, no memory/RAG is injected, and web search is disabled
|
||||||
|
- **Info popup** — click the (i) icon next to the badge for a full explanation of what private mode does and doesn't do
|
||||||
|
- **No-storage guarantee** — conversation is streamed to the user but never touches SQLite, FTS5, or Qdrant
|
||||||
|
- **Search blocked** — `/api/search` returns 403 in private mode; WEB button is disabled in the UI
|
||||||
|
|
||||||
|
### WireGuard In-Transit Encryption
|
||||||
|
- WireGuard tunnels encrypt all inter-node traffic. See Data Safety section above.
|
||||||
|
|
||||||
## What's New in v0.19.2
|
## What's New in v0.19.2
|
||||||
|
|
||||||
### Waterfall Direction Toggle (B6)
|
### Waterfall Direction Toggle (B6) + UX Polish
|
||||||
- **NEW/OLD toggle** — topbar button switches between newest-first (waterfall) and oldest-first (traditional chat)
|
- **NEW/OLD toggle** — topbar button switches between newest-first (waterfall) and oldest-first (traditional chat)
|
||||||
- **Direction-aware scroll** — newest-first scrolls to top, oldest-first scrolls to bottom
|
- **Direction-aware scroll** — newest-first scrolls to top, oldest-first scrolls to bottom; respects user scroll-away to avoid fighting
|
||||||
- **localStorage persistence** — preference survives page reloads, default is newest-first (waterfall)
|
- **localStorage persistence** — preference survives page reloads, default is newest-first (waterfall)
|
||||||
|
- **Toast notifications** — slide-out notifications for copy, save, delete, rate actions
|
||||||
|
- **Clipboard reliability** — `execCopy()` helper for HTTP fallback (`document.execCommand`) when `navigator.clipboard` fails on plain HTTP
|
||||||
|
- **Model label** — `modelLabel()` derives shorthand display names (e.g. `qwen2.5:7B:i`)
|
||||||
|
- **Keybinding fix** — Shift+Enter = newline, Ctrl+Enter = send (universal conventions)
|
||||||
|
- **Token counter** — resets to 0 on page refresh, no longer persisted to localStorage
|
||||||
|
|
||||||
## What's New in v0.19.1
|
## What's New in v0.19.1
|
||||||
|
|
||||||
@@ -150,6 +175,7 @@ Developer wiki: [Home](https://llgit.llamachile.tube/gramps/cAIc/wiki/Home) —
|
|||||||
- **Conversation History** — SQLite-backed chat persistence with mass-delete option
|
- **Conversation History** — SQLite-backed chat persistence with mass-delete option
|
||||||
- **Model Switching** — Change inference models on the fly
|
- **Model Switching** — Change inference models on the fly
|
||||||
- **Skills Framework** — Built-in skill registry with per-skill enable/disable controls
|
- **Skills Framework** — Built-in skill registry with per-skill enable/disable controls
|
||||||
|
- **Private Chat** — Toggle to keep conversations ephemeral: no persistence, no memory/RAG, no web search
|
||||||
|
|
||||||
## File Structure
|
## File Structure
|
||||||
|
|
||||||
@@ -162,12 +188,13 @@ Developer wiki: [Home](https://llgit.llamachile.tube/gramps/cAIc/wiki/Home) —
|
|||||||
├── config.py # Constants, env vars, limits, skill registry
|
├── config.py # Constants, env vars, limits, skill registry
|
||||||
├── db.py # SQLite schema, connection factory
|
├── db.py # SQLite schema, connection factory
|
||||||
├── eviction.py # Score-based RAG eviction engine
|
├── eviction.py # Score-based RAG eviction engine
|
||||||
├── gpu.py # AMD GPU stats via rocm-smi
|
├── gpu.py # GPU stats — rocm-smi (Linux/AMD) + system_profiler (Darwin/Apple Silicon)
|
||||||
├── hardware.py # Hardware self-assessment (CPU, RAM, VRAM)
|
├── hardware.py # Hardware self-assessment (CPU, RAM, VRAM) — Linux + Darwin
|
||||||
├── memory.py # FTS5 memory CRUD, remember/forget commands
|
├── memory.py # FTS5 memory CRUD, remember/forget commands
|
||||||
├── rag.py # Qdrant vector search + system prompt assembly
|
├── rag.py # Qdrant vector search + system prompt assembly
|
||||||
├── search.py # SearXNG integration, perplexity, refusal detection
|
├── search.py # SearXNG integration, perplexity, refusal detection
|
||||||
├── security.py # Rate limiting, origin checks, IP allowlist, audit
|
├── security.py # Rate limiting, origin checks, IP allowlist, audit
|
||||||
|
├── model_pull.py # Startup model auto-pull (llama-server → Ollama fallback)
|
||||||
├── triage.py # Query classification + cluster node selection
|
├── triage.py # Query classification + cluster node selection
|
||||||
├── routers/
|
├── routers/
|
||||||
│ ├── chat.py # /api/chat streaming endpoint
|
│ ├── chat.py # /api/chat streaming endpoint
|
||||||
@@ -190,7 +217,7 @@ Developer wiki: [Home](https://llgit.llamachile.tube/gramps/cAIc/wiki/Home) —
|
|||||||
├── node_agent/
|
├── node_agent/
|
||||||
│ ├── agent.py # Standalone worker agent (AMQP client)
|
│ ├── agent.py # Standalone worker agent (AMQP client)
|
||||||
│ └── requirements.txt
|
│ └── requirements.txt
|
||||||
└── tests/ # 179 pytest tests
|
└── tests/ # 198 pytest tests
|
||||||
```
|
```
|
||||||
|
|
||||||
## Requirements
|
## Requirements
|
||||||
@@ -200,6 +227,7 @@ Developer wiki: [Home](https://llgit.llamachile.tube/gramps/cAIc/wiki/Home) —
|
|||||||
- SearXNG (optional, for web search)
|
- SearXNG (optional, for web search)
|
||||||
- RabbitMQ (optional, for AMQP cluster — coordinator only)
|
- RabbitMQ (optional, for AMQP cluster — coordinator only)
|
||||||
- Qdrant (optional, for RAG vector search)
|
- Qdrant (optional, for RAG vector search)
|
||||||
|
- WireGuard (optional, for encrypted inter-node transit — see [WireGuard-Setup.md](docs/wiki/WireGuard-Setup.md))
|
||||||
|
|
||||||
## Installation
|
## Installation
|
||||||
|
|
||||||
@@ -404,7 +432,7 @@ Settings are stored in the `settings` table and include:
|
|||||||
python3 -m pytest tests/ -v
|
python3 -m pytest tests/ -v
|
||||||
```
|
```
|
||||||
|
|
||||||
All 179 tests use `tmp_path` fixtures + monkeypatched `httpx.AsyncClient`/`aio-pika`. No external services needed.
|
All 198 tests use `tmp_path` fixtures + monkeypatched `httpx.AsyncClient`/`aio-pika`. No external services needed.
|
||||||
|
|
||||||
## License
|
## License
|
||||||
|
|
||||||
|
|||||||
@@ -479,6 +479,48 @@ caddy:
|
|||||||
|
|
||||||
This is out of scope for v1.0 but documented for future.
|
This is out of scope for v1.0 but documented for future.
|
||||||
|
|
||||||
|
### 5.4 WireGuard tunnel (off-site workers)
|
||||||
|
|
||||||
|
When a worker node runs on a different network (colo, friend's house, VPS), all cross-site traffic must be encrypted. WireGuard provides this at the network layer with zero application changes.
|
||||||
|
|
||||||
|
**Approach:** Install WireGuard on the Docker host (not inside a container). The host creates a tunnel interface (`wg0`) with a virtual IP in the `10.0.2.0/24` range. Containers that need to reach remote workers use the host's WireGuard IP via `network_mode: host` or standard routing.
|
||||||
|
|
||||||
|
```
|
||||||
|
Off-site worker Docker host (coordinator)
|
||||||
|
┌────────────────────┐ ┌───────────────────────────────┐
|
||||||
|
│ wg0: 10.0.2.2 │◄───UDP──────│ wg0: 10.0.2.1 │
|
||||||
|
│ llama-server │ :51820 │ │
|
||||||
|
│ node_agent.py │ encrypts │ ┌───────────────────────┐ │
|
||||||
|
│ │ all │ │ cAIc container │ │
|
||||||
|
│ │ traffic │ │ LLAMA_SERVER_BASE │ │
|
||||||
|
│ │ │ │ → 10.0.2.2:8081 │ │
|
||||||
|
│ │ │ │ CAIC_AMQP_URL │ │
|
||||||
|
│ │ │ │ → amqp://caic:@... │ │
|
||||||
|
│ │ │ └───────────────────────┘ │
|
||||||
|
└────────────────────┘ └───────────────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
**Host setup (coordinator):**
|
||||||
|
```bash
|
||||||
|
sudo apt install wireguard
|
||||||
|
wg genkey | tee /etc/wireguard/private.key | wg pubkey > /etc/wireguard/public.key
|
||||||
|
chmod 600 /etc/wireguard/private.key
|
||||||
|
```
|
||||||
|
Then create `/etc/wireguard/wg0.conf` — see [WireGuard-Setup.md](WireGuard-Setup.md) for full per-node configs.
|
||||||
|
|
||||||
|
**Container networking:** The cAIc container needs to reach the WireGuard IP. Options:
|
||||||
|
1. **`network_mode: host`** — simplest, container shares host network stack. Done by adding `network_mode: host` to the cAIc service. Trade-off: no port isolation.
|
||||||
|
2. **Host routing** — the host's kernel routes `10.0.2.0/24` via `wg0`. Containers on the default bridge or compose network can reach those IPs if `ip_forward` is enabled. This works out of the box on Linux.
|
||||||
|
|
||||||
|
**cAIc env vars after WireGuard:**
|
||||||
|
```env
|
||||||
|
# Point at worker's WireGuard IP instead of LAN IP
|
||||||
|
LLAMA_SERVER_BASE=http://10.0.2.2:8081 # falls back to coordinator's own llama-server
|
||||||
|
CAIC_AMQP_URL=amqp://caic:password@10.0.2.1:5672/caic # coordinator's RMQ on WG IP
|
||||||
|
```
|
||||||
|
|
||||||
|
The node agent on each worker configures its registration IP as the WireGuard tunnel IP (`node_ip = 10.0.2.2`), so `triage.py` constructs inference URLs pointing at the encrypted interface.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 6. Setup Wizard (Extraction)
|
## 6. Setup Wizard (Extraction)
|
||||||
@@ -642,6 +684,7 @@ If the setup wizard fails mid-way, a partial rollback is better than leaving det
|
|||||||
| **Reverse proxy** | Caddy is simplest for auto-HTTPS. Out of scope for v1.0 but design for it. | Low |
|
| **Reverse proxy** | Caddy is simplest for auto-HTTPS. Out of scope for v1.0 but design for it. | Low |
|
||||||
| **Healthcheck strategy** | `depends_on` with `condition: service_healthy` is the safest approach but increases startup time. Acceptable. | Medium |
|
| **Healthcheck strategy** | `depends_on` with `condition: service_healthy` is the safest approach but increases startup time. Acceptable. | Medium |
|
||||||
| **Database migration** | SQLite file in volume — no migration needed for v1.0 format. If schema changes post-v1.0, need a migration container. | Low |
|
| **Database migration** | SQLite file in volume — no migration needed for v1.0 format. If schema changes post-v1.0, need a migration container. | Low |
|
||||||
|
| **WireGuard integration** | Documented in docker.md §5.4 + wiki. Host-level install; no container changes needed. WireGuard sidecar container (`linuxserver/wireguard`) is an alternative for users who want everything in compose. | Low |
|
||||||
| **Linux vs macOS vs Windows** | Linux-primary. macOS may work with changes (no rocm-smi). Windows via WSL2 only. | Low |
|
| **Linux vs macOS vs Windows** | Linux-primary. macOS may work with changes (no rocm-smi). Windows via WSL2 only. | Low |
|
||||||
| **LLM model download** | HuggingFace CLI integration in setup.sh, or manual download. Manual is simpler. | Low |
|
| **LLM model download** | HuggingFace CLI integration in setup.sh, or manual download. Manual is simpler. | Low |
|
||||||
| **Dockerfile optimization** | Pin pip hashes, use `--no-cache-dir`, consider `slim` vs `alpine`. Alpine has musl compatibility issues with psutil. Stay with slim. | Medium |
|
| **Dockerfile optimization** | Pin pip hashes, use `--no-cache-dir`, consider `slim` vs `alpine`. Alpine has musl compatibility issues with psutil. Stay with slim. | Medium |
|
||||||
@@ -684,6 +727,10 @@ Worker machine (e.g. worker01, worker02)
|
|||||||
### 9.3 Worker setup
|
### 9.3 Worker setup
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
# Install WireGuard (required for off-site workers — encrypts all traffic)
|
||||||
|
sudo apt install wireguard
|
||||||
|
# See docs/wiki/WireGuard-Setup.md for per-node config
|
||||||
|
|
||||||
# Install llama-server binary
|
# Install llama-server binary
|
||||||
wget https://github.com/ggml-org/llama.cpp/releases/.../llama-server
|
wget https://github.com/ggml-org/llama.cpp/releases/.../llama-server
|
||||||
chmod +x llama-server
|
chmod +x llama-server
|
||||||
@@ -691,8 +738,9 @@ chmod +x llama-server
|
|||||||
# Install node agent deps
|
# Install node agent deps
|
||||||
pip install aio-pika httpx
|
pip install aio-pika httpx
|
||||||
|
|
||||||
# Create node agent script (from repo: node_agent/agent.py)
|
# Create node agent config: /etc/caic-node-agent.conf
|
||||||
# Configure COORDINATOR_AMQP_URL in environment
|
# Set node_ip to the WireGuard tunnel IP (e.g., 10.0.2.2)
|
||||||
|
# Set amqp_url to the coordinator's WireGuard IP (e.g., amqp://caic:password@10.0.2.1:5672/caic)
|
||||||
```
|
```
|
||||||
|
|
||||||
### 9.4 Multiple workers
|
### 9.4 Multiple workers
|
||||||
@@ -728,6 +776,7 @@ The broker-mediated model is the preferred architecture for this project because
|
|||||||
- [ ] `docker compose up -d` works without any manual steps beyond setup.sh
|
- [ ] `docker compose up -d` works without any manual steps beyond setup.sh
|
||||||
- [ ] `docker compose down -v` followed by `setup.sh && docker compose up -d` = fresh stack
|
- [ ] `docker compose down -v` followed by `setup.sh && docker compose up -d` = fresh stack
|
||||||
- [ ] Healthchecks prevent serving before dependencies are ready
|
- [ ] Healthchecks prevent serving before dependencies are ready
|
||||||
|
- [ ] WireGuard tunnel documented and tested for off-site workers
|
||||||
- [ ] v1.0 release tag created
|
- [ ] v1.0 release tag created
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|||||||
@@ -265,25 +265,28 @@ All streaming endpoints yield `data: {json}\n\n`:
|
|||||||
- No live external services required
|
- No live external services required
|
||||||
- Test factories reset `SESSIONS`, `PIN_ATTEMPTS`, `RATE_EVENTS` globals per test
|
- Test factories reset `SESSIONS`, `PIN_ATTEMPTS`, `RATE_EVENTS` globals per test
|
||||||
|
|
||||||
### 8.2 Test Coverage Areas (179 tests)
|
### 8.2 Test Coverage Areas (198 tests)
|
||||||
|
|
||||||
| Test file | Coverage |
|
| Test file | Coverage |
|
||||||
|-----------|----------|
|
|-----------|----------|
|
||||||
| test_auth_capabilities.py | Guest/admin sessions, origin blocking, logout |
|
| test_auth_capabilities.py | Guest/admin sessions, origin blocking, logout |
|
||||||
| test_chat_streaming_and_memory_paths.py | Streaming, auto-search, remember/forget, upload context injection |
|
| test_chat_streaming_and_memory_paths.py | Streaming, auto-search, remember/forget, upload context injection, private chat |
|
||||||
| test_cluster.py | Registration, deregistration, pong, events, coordinator query |
|
| test_cluster.py | Registration, deregistration, pong, events, coordinator query |
|
||||||
| test_cluster_heartbeat.py | Heartbeat handler, known/unknown node |
|
| test_cluster_heartbeat.py | Heartbeat handler, known/unknown node |
|
||||||
| test_completions.py | API key auth, FIM, streaming, blocking, errors |
|
| test_completions.py | API key auth, FIM, streaming, blocking, errors |
|
||||||
| test_conversations.py | Full CRUD, guest admin, attachment_count |
|
| test_conversations.py | Full CRUD, guest admin, attachment_count |
|
||||||
|
| test_error_envelopes.py | Global exception handler + stream errors |
|
||||||
|
| test_gpu.py | GPU stats — rocm-smi (Linux), system_profiler (Darwin/Apple Silicon) |
|
||||||
| test_ingest.py | Bearer auth, chunk/embed/upsert, validation |
|
| test_ingest.py | Bearer auth, chunk/embed/upsert, validation |
|
||||||
| test_ip_allowlist.py | IP allowlist helper + middleware |
|
| test_ip_allowlist.py | IP allowlist helper + middleware |
|
||||||
| test_memories.py | Edit, search, stats |
|
| test_memories.py | Edit, search, stats |
|
||||||
|
| test_model_pull.py | Default model auto-pull — llama-server check, Ollama fallback, error paths |
|
||||||
| test_model_swap.py | request_model_swap, handle_model_ready/failed, select_node swap triggering |
|
| test_model_swap.py | request_model_swap, handle_model_ready/failed, select_node swap triggering |
|
||||||
| test_models_router.py | Models list, ps, show, stats, search/status |
|
| test_models_router.py | Models list, ps, show, stats, search/status |
|
||||||
| test_node_agent.py | Node agent registration, ping/pong, model swap |
|
| test_node_agent.py | Node agent registration, ping/pong, model swap |
|
||||||
| test_presets.py | Full CRUD, default preset protection |
|
| test_presets.py | Full CRUD, default preset protection |
|
||||||
| test_profile.py | Get, update, default, length validation |
|
| test_profile.py | Get, update, default, length validation |
|
||||||
| test_rag_management.py | Collection stats, eviction algorithm (pinned/grace/scoring/batch), maybe_evict hysteresis, operational stats, flush, concurrency lock |
|
| test_rag_management.py | Collection stats, eviction algorithm, hysteresis, flush |
|
||||||
| test_rate_and_payload_guardrails.py | Rate limits + payload size |
|
| test_rate_and_payload_guardrails.py | Rate limits + payload size |
|
||||||
| test_search_route.py | Explicit search flow, no results, errors |
|
| test_search_route.py | Explicit search flow, no results, errors |
|
||||||
| test_search_url_sanitization.py | URL sanitizer |
|
| test_search_url_sanitization.py | URL sanitizer |
|
||||||
@@ -291,7 +294,6 @@ All streaming endpoints yield `data: {json}\n\n`:
|
|||||||
| test_skills_framework.py | List, toggle, unknown skill, prompt injection |
|
| test_skills_framework.py | List, toggle, unknown skill, prompt injection |
|
||||||
| test_triage.py | classify_query, select_node, get_inference_url |
|
| test_triage.py | classify_query, select_node, get_inference_url |
|
||||||
| test_upload.py | Upload, delete, link, by-conversation, attachment_count |
|
| test_upload.py | Upload, delete, link, by-conversation, attachment_count |
|
||||||
| test_error_envelopes.py | Global exception handler + stream errors |
|
|
||||||
|
|
||||||
### 8.3 DoD Process
|
### 8.3 DoD Process
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,192 @@
|
|||||||
|
# WireGuard Tunnel — Encrypted Node Transit
|
||||||
|
|
||||||
|
## Why
|
||||||
|
|
||||||
|
cAIc cluster traffic is plaintext today:
|
||||||
|
|
||||||
|
| Traffic | Protocol | Plaintext risk |
|
||||||
|
|---------|----------|----------------|
|
||||||
|
| AMQP (coordinator ↔ worker agent) | TCP :5672 | Registration, ping/pong, swap commands |
|
||||||
|
| Inference (coordinator → worker llama-server) | HTTP :8081 | Every token generated |
|
||||||
|
| LLM RPC layer offload (coordinator llama-server → worker) | TCP :50052 | Internal llama.cpp protocol |
|
||||||
|
|
||||||
|
WireGuard encrypts all three at the network layer with zero application changes. The cAIc app keeps using `http://` URLs — it's just talking to a virtual IP whose traffic is automatically encrypted before it hits the wire.
|
||||||
|
|
||||||
|
## Topology
|
||||||
|
|
||||||
|
```
|
||||||
|
┌───────────────────────┐ WireGuard tunnel ┌───────────────────────┐
|
||||||
|
│ Coordinator (ultron) │◄═══════════════════════════►│ Worker (jarvis) │
|
||||||
|
│ 10.0.2.1 │ UDP :51820 │ 10.0.2.2 │
|
||||||
|
│ LAN 192.168.50.108 │ │ LAN 192.168.50.210 │
|
||||||
|
│ │════════════════════════════►│ │
|
||||||
|
│ │ UDP :51820 │ Worker (corsair) │
|
||||||
|
│ │ │ 10.0.2.3 │
|
||||||
|
│ │ │ LAN (DHCP) │
|
||||||
|
└───────────────────────┘ └───────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
All nodes connect directly to the coordinator's WireGuard endpoint (star topology). Workers do not need to talk to each other.
|
||||||
|
|
||||||
|
## Prerequisites
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Debian / Ubuntu
|
||||||
|
sudo apt install wireguard
|
||||||
|
|
||||||
|
# Windows / WSL2 — install WireGuard from https://www.wireguard.com/install/
|
||||||
|
# The wg.exe binary is used inside WSL2; the Windows GUI manages the tunnel config
|
||||||
|
```
|
||||||
|
|
||||||
|
## Key Generation
|
||||||
|
|
||||||
|
Run once per node. Save the private key securely; public keys go into peer configs on the other end.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
wg genkey | tee /etc/wireguard/private.key | wg pubkey > /etc/wireguard/public.key
|
||||||
|
chmod 600 /etc/wireguard/private.key
|
||||||
|
```
|
||||||
|
|
||||||
|
**Recorded keys for this deployment:**
|
||||||
|
|
||||||
|
| Node | Private key | Public key |
|
||||||
|
|------|-------------|------------|
|
||||||
|
| ultron | (node private) | `ultron_pubkey=` |
|
||||||
|
| jarvis | (node private) | `jarvis_pubkey=` |
|
||||||
|
| corsair | (node private) | `corsair_pubkey=` |
|
||||||
|
|
||||||
|
## Per-Node Configs
|
||||||
|
|
||||||
|
### Coordinator — `/etc/wireguard/wg0.conf` on ultron
|
||||||
|
|
||||||
|
```ini
|
||||||
|
[Interface]
|
||||||
|
Address = 10.0.2.1/24
|
||||||
|
ListenPort = 51820
|
||||||
|
PrivateKey = <ultron_private_key>
|
||||||
|
|
||||||
|
# Enable IP forwarding so workers can route through coordinator if needed
|
||||||
|
# sudo sysctl -w net.ipv4.ip_forward=1
|
||||||
|
# sudo sysctl -w net.ipv6.conf.all.forwarding=1
|
||||||
|
|
||||||
|
# Worker: jarvis
|
||||||
|
[Peer]
|
||||||
|
PublicKey = <jarvis_pubkey>
|
||||||
|
AllowedIPs = 10.0.2.2/32
|
||||||
|
# If jarvis is off-site, put its public IP / DDNS hostname here:
|
||||||
|
# Endpoint = jarvis.example.com:51820
|
||||||
|
# If jarvis is LAN-only, set PersistentKeepalive = 25 to maintain NAT binding:
|
||||||
|
# PersistentKeepalive = 25
|
||||||
|
|
||||||
|
# Worker: corsair
|
||||||
|
[Peer]
|
||||||
|
PublicKey = <corsair_pubkey>
|
||||||
|
AllowedIPs = 10.0.2.3/32
|
||||||
|
```
|
||||||
|
|
||||||
|
### Worker — `/etc/wireguard/wg0.conf` on jarvis (Linux)
|
||||||
|
|
||||||
|
```ini
|
||||||
|
[Interface]
|
||||||
|
Address = 10.0.2.2/24
|
||||||
|
ListenPort = 51820
|
||||||
|
PrivateKey = <jarvis_private_key>
|
||||||
|
|
||||||
|
# Coordinator
|
||||||
|
[Peer]
|
||||||
|
PublicKey = <ultron_pubkey>
|
||||||
|
AllowedIPs = 10.0.2.0/24
|
||||||
|
# LAN-only: just point at the LAN IP
|
||||||
|
Endpoint = 192.168.50.108:51820
|
||||||
|
# Off-site: use DDNS or static IP:
|
||||||
|
# Endpoint = ultron.example.com:51820
|
||||||
|
PersistentKeepalive = 25
|
||||||
|
```
|
||||||
|
|
||||||
|
### Worker — Windows / WSL2 on corsair
|
||||||
|
|
||||||
|
Create a WireGuard tunnel in the Windows GUI app with the same config as jarvis above (Address=10.0.2.3/24). WSL2 inside Windows can reach the tunnel IP via the Windows host.
|
||||||
|
|
||||||
|
If llama-server runs inside WSL2 on corsair, the Windows host's WireGuard tunnel IP `10.0.2.3` is reachable from the WSL2 instance as well — just bind llama-server to `0.0.0.0` (already the default) and configure the Windows firewall to allow inbound on :8081 from the coordinator's WireGuard IP.
|
||||||
|
|
||||||
|
## Starting the Tunnel
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Start immediately
|
||||||
|
sudo systemctl start wg-quick@wg0
|
||||||
|
|
||||||
|
# Enable on boot
|
||||||
|
sudo systemctl enable wg-quick@wg0
|
||||||
|
|
||||||
|
# Check status
|
||||||
|
sudo wg show
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected output on each node:
|
||||||
|
|
||||||
|
```
|
||||||
|
interface: wg0
|
||||||
|
public key: <...>
|
||||||
|
private key: (hidden)
|
||||||
|
listening port: 51820
|
||||||
|
|
||||||
|
peer: <ultron_pubkey>
|
||||||
|
endpoint: 192.168.50.108:51820
|
||||||
|
allowed ips: 10.0.2.0/24
|
||||||
|
latest handshake: 5 seconds ago ← healthy
|
||||||
|
transfer: 1.2 KiB received, 3.4 KiB sent
|
||||||
|
```
|
||||||
|
|
||||||
|
If `latest handshake` is missing, check firewall rules (UDP :51820 must be open on all nodes).
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# From any node, ping another node's WireGuard IP
|
||||||
|
ping -c 3 10.0.2.1 # coordinator
|
||||||
|
ping -c 3 10.0.2.2 # jarvis
|
||||||
|
ping -c 3 10.0.2.3 # corsair
|
||||||
|
|
||||||
|
# Verify cAIc inference through the tunnel
|
||||||
|
curl http://10.0.2.2:8081/v1/models # jarvis llama-server
|
||||||
|
curl http://10.0.2.3:8081/v1/models # corsair llama-server
|
||||||
|
```
|
||||||
|
|
||||||
|
## Updating cAIc to Use the Tunnel
|
||||||
|
|
||||||
|
Once WireGuard is running, point each service at the tunnel IP instead of the LAN IP.
|
||||||
|
|
||||||
|
### Worker node agent config — `/etc/caic-node-agent.conf`
|
||||||
|
|
||||||
|
```ini
|
||||||
|
[agent]
|
||||||
|
node_name = jarvis
|
||||||
|
node_ip = 10.0.2.2 # was 192.168.50.210
|
||||||
|
node_type = worker
|
||||||
|
capabilities = llm
|
||||||
|
amqp_url = amqp://caic:password@10.0.2.1:5672/caic # was 192.168.50.108
|
||||||
|
llama_port = 8081
|
||||||
|
models_dir = /var/lib/caic/models
|
||||||
|
active_model = qwen2.5-7b-instruct-Q5_K_M.gguf
|
||||||
|
```
|
||||||
|
|
||||||
|
### Coordinator config — environment variables
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# On the coordinator node, override the worker-facing addresses
|
||||||
|
# (LLAMA_SERVER_BASE stays as localhost / LAN IP since inference
|
||||||
|
# to the coordinator's own llama-server stays on-machine)
|
||||||
|
export CAIC_AMQP_URL="amqp://caic:password@10.0.2.1:5672/caic"
|
||||||
|
```
|
||||||
|
|
||||||
|
No other cAIc code changes are needed. The app already reads `CAIC_AMQP_URL` from the environment (`config.py:27`) and the node agent reads `node_ip` from its INI file. Inference requests routed to remote workers via `triage.py` use the IP the worker registered — so setting `node_ip = 10.0.2.2` in the worker's agent config is all it takes.
|
||||||
|
|
||||||
|
## Cross-Site Deployment Checklist
|
||||||
|
|
||||||
|
When placing a worker outside the LAN:
|
||||||
|
|
||||||
|
1. **Firewall:** Open UDP :51820 on the remote site. On the coordinator side, make sure UDP :51820 is reachable from the internet (port forward / firewall rule at the coordinator's router).
|
||||||
|
2. **DDNS:** If the coordinator's public IP is dynamic, set up a DDNS hostname and use it in the worker's `Endpoint = ultron.example.com:51820`.
|
||||||
|
3. **PersistentKeepalive:** Set `PersistentKeepalive = 25` on the worker side to keep NAT bindings alive.
|
||||||
|
4. **No double encryption:** WireGuard encrypts everything on the WireGuard interface. The cAIc app continues to use `http://` — it never touches raw TLS. This is correct and intended.
|
||||||
|
5. **Split tunnelling (optional):** The worker's `AllowedIPs = 10.0.2.0/24` ensures only cluster traffic goes through the tunnel. All other internet traffic from the worker uses its normal gateway.
|
||||||
@@ -1,19 +1,23 @@
|
|||||||
# cAIc Current WiP Backlog
|
# cAIc Current WiP Backlog
|
||||||
|
|
||||||
Last updated: 2026-07-06
|
Last updated: 2026-07-14
|
||||||
Owner: Gramps
|
Owner: Gramps
|
||||||
Scope: Active roadmap items and backlog.
|
Scope: Active roadmap items and backlog.
|
||||||
|
|
||||||
## ~~Roadmap N: AMQP Cluster Nervous System [COMPLETE]~~
|
## Completed
|
||||||
|
|
||||||
All 15 tasks are done. Final commit: `f0689ac feat: Roadmap N — AMQP cluster nervous system complete`.
|
- **B7 (v0.19.0)** — Apple Silicon worker support. gpu.py darwin branch, hardware.py darwin VRAM, node_agent/agent.py macOS VRAM reporting.
|
||||||
|
- **B5 (v0.19.1)** — Default model auto-pull. model_pull.py + ensure_model() in app.py lifespan.
|
||||||
|
- **B6 (v0.19.2)** — Waterfall direction toggle (NEW/OLD), scroll/lock fixes, toast notifications, execCopy, modelLabel, Ctrl+Enter.
|
||||||
|
- **B8 (v0.19.3)** — Private Chat mode. Backend skip-DB/skip-RAG/skip-search flag, frontend PRIVATE badge, info popup.
|
||||||
|
- **WireGuard TLS (v0.19.4)** — Self-signed WireGuard mesh encrypts all inter-node traffic (AMQP, inference, RPC). No code changes to cAIc. Documented in wiki/WireGuard-Setup.md + docker.md §5.4.
|
||||||
|
|
||||||
## Backlog
|
## Backlog
|
||||||
|
|
||||||
- B1 — Context loss in follow-up questions (investigation)
|
- B1 — Context loss in follow-up questions (investigation)
|
||||||
- B2 — Bang-prefixed (`!`) search routing
|
- B2 — Bang-prefixed (`!`) search routing
|
||||||
- B3 — Docker distribution (planning doc at `docker.md`)
|
- B3 — Docker distribution (planning doc at `docker.md`, not yet implemented)
|
||||||
- **B4 — RAG Corpus Management UI (NEXT)** — browse, search, edit, delete individual RAG entries. Backend endpoints at `routers/rag_admin.py`, frontend panel in `templates/index.html`
|
- **B4 — RAG Corpus Management UI (deferred)** — browse, search, edit, delete individual RAG entries
|
||||||
- HTTPS / reverse proxy (Caddy)
|
- HTTPS / reverse proxy (Caddy)
|
||||||
- Conversation search/filter and export tooling
|
- Conversation search/filter and export tooling
|
||||||
- Keyboard shortcuts, retry button, source-link polish
|
- Keyboard shortcuts, retry button, source-link polish
|
||||||
|
|||||||
+17
-3
@@ -74,6 +74,7 @@ async def chat(request: Request):
|
|||||||
model = body.get("model", DEFAULT_MODEL)
|
model = body.get("model", DEFAULT_MODEL)
|
||||||
preset_prompt = body.get("system_prompt", "")
|
preset_prompt = body.get("system_prompt", "")
|
||||||
upload_context_id = body.get("upload_context_id")
|
upload_context_id = body.get("upload_context_id")
|
||||||
|
private_chat = body.get("private_chat", False)
|
||||||
|
|
||||||
if not user_message:
|
if not user_message:
|
||||||
raise HTTPException(status_code=400, detail="Empty message")
|
raise HTTPException(status_code=400, detail="Empty message")
|
||||||
@@ -81,18 +82,29 @@ async def chat(request: Request):
|
|||||||
db = get_db()
|
db = get_db()
|
||||||
now = datetime.now(timezone.utc).isoformat()
|
now = datetime.now(timezone.utc).isoformat()
|
||||||
settings = {row["key"]: row["value"] for row in db.execute("SELECT key, value FROM settings").fetchall()}
|
settings = {row["key"]: row["value"] for row in db.execute("SELECT key, value FROM settings").fetchall()}
|
||||||
search_enabled = settings.get("search_enabled", "true") == "true"
|
search_enabled = settings.get("search_enabled", "true") == "true" and not private_chat
|
||||||
|
|
||||||
upload_doc = None
|
upload_doc = None
|
||||||
if upload_context_id:
|
if upload_context_id and not private_chat:
|
||||||
ctx = get_upload_context(db, upload_context_id)
|
ctx = get_upload_context(db, upload_context_id)
|
||||||
if ctx:
|
if ctx:
|
||||||
upload_doc = f"[ATTACHED DOCUMENT: {ctx['filename']}]\n{ctx['content']}\n[END DOCUMENT]"
|
upload_doc = f"[ATTACHED DOCUMENT: {ctx['filename']}]\n{ctx['content']}\n[END DOCUMENT]"
|
||||||
else:
|
else:
|
||||||
log.warning(f"upload_context_id {upload_context_id} not found or expired, continuing without it")
|
log.warning(f"upload_context_id {upload_context_id} not found or expired, continuing without it")
|
||||||
|
|
||||||
remember_response = process_remember_command(user_message)
|
remember_response = None if private_chat else process_remember_command(user_message)
|
||||||
|
|
||||||
|
if private_chat:
|
||||||
|
if not conv_id:
|
||||||
|
conv_id = str(uuid.uuid4())
|
||||||
|
system_prompt = ""
|
||||||
|
messages = []
|
||||||
|
if preset_prompt:
|
||||||
|
messages.append({"role": "system", "content": preset_prompt})
|
||||||
|
messages.append({"role": "user", "content": user_message})
|
||||||
|
history_rows = []
|
||||||
|
db.close()
|
||||||
|
else:
|
||||||
if not conv_id:
|
if not conv_id:
|
||||||
conv_id = str(uuid.uuid4())
|
conv_id = str(uuid.uuid4())
|
||||||
title = user_message[:80] + ("..." if len(user_message) > 80 else "")
|
title = user_message[:80] + ("..." if len(user_message) > 80 else "")
|
||||||
@@ -196,6 +208,7 @@ async def chat(request: Request):
|
|||||||
|
|
||||||
yield f"data: {json.dumps({'token': cleaned_response, 'conversation_id': conv_id, 'augmented': True})}\n\n"
|
yield f"data: {json.dumps({'token': cleaned_response, 'conversation_id': conv_id, 'augmented': True})}\n\n"
|
||||||
|
|
||||||
|
if not private_chat:
|
||||||
saved_msg = cleaned_response + "\n\n---\n*🔍 Enhanced with web search results*"
|
saved_msg = cleaned_response + "\n\n---\n*🔍 Enhanced with web search results*"
|
||||||
if remember_response:
|
if remember_response:
|
||||||
saved_msg = remember_response + "\n\n" + saved_msg
|
saved_msg = remember_response + "\n\n" + saved_msg
|
||||||
@@ -217,6 +230,7 @@ async def chat(request: Request):
|
|||||||
yield f"data: {json.dumps({'done': True, 'conversation_id': conv_id, 'searched': True, 'perplexity': round(perplexity, 2), 'tokens_per_sec': round(tokens_per_sec, 1), 'prompt_tokens': prompt_tokens, 'completion_tokens': completion_tokens, 'context_length': MODEL_CONTEXT_LENGTH, **(rag_update and {'rag_update_suggestion': rag_update} or {})})}\n\n"
|
yield f"data: {json.dumps({'done': True, 'conversation_id': conv_id, 'searched': True, 'perplexity': round(perplexity, 2), 'tokens_per_sec': round(tokens_per_sec, 1), 'prompt_tokens': prompt_tokens, 'completion_tokens': completion_tokens, 'context_length': MODEL_CONTEXT_LENGTH, **(rag_update and {'rag_update_suggestion': rag_update} or {})})}\n\n"
|
||||||
return
|
return
|
||||||
|
|
||||||
|
if not private_chat:
|
||||||
saved_msg = assistant_msg
|
saved_msg = assistant_msg
|
||||||
if remember_response:
|
if remember_response:
|
||||||
saved_msg = remember_response + "\n\n" + saved_msg
|
saved_msg = remember_response + "\n\n" + saved_msg
|
||||||
|
|||||||
@@ -30,6 +30,10 @@ async def explicit_search(request: Request):
|
|||||||
if not query:
|
if not query:
|
||||||
raise HTTPException(status_code=400, detail="Empty query")
|
raise HTTPException(status_code=400, detail="Empty query")
|
||||||
|
|
||||||
|
private_chat = body.get("private_chat", False)
|
||||||
|
if private_chat:
|
||||||
|
raise HTTPException(status_code=403, detail="Web search is disabled in private chat mode")
|
||||||
|
|
||||||
db = get_db()
|
db = get_db()
|
||||||
now = datetime.now(timezone.utc).isoformat()
|
now = datetime.now(timezone.utc).isoformat()
|
||||||
|
|
||||||
|
|||||||
+36
-1
@@ -101,6 +101,9 @@ body { font-family: var(--font-body); background: var(--bg-primary); color: var(
|
|||||||
.badge.off { border-color:rgba(255,51,85,0.3); color:var(--danger); background:rgba(255,51,85,0.08); }
|
.badge.off { border-color:rgba(255,51,85,0.3); color:var(--danger); background:rgba(255,51,85,0.08); }
|
||||||
.badge.admin { border-color:var(--border); color:var(--text-muted); }
|
.badge.admin { border-color:var(--border); color:var(--text-muted); }
|
||||||
.badge.admin:hover { border-color:var(--accent-dim); color:var(--accent); }
|
.badge.admin:hover { border-color:var(--accent-dim); color:var(--accent); }
|
||||||
|
.info-icon { display:inline-flex; align-items:center; justify-content:center; width:18px; height:18px; border-radius:50%; border:1px solid var(--border); color:var(--text-muted); font-size:11px; cursor:pointer; user-select:none; transition:all 0.15s; }
|
||||||
|
.info-icon:hover { border-color:var(--accent-dim); color:var(--accent); }
|
||||||
|
.info-popup { position:absolute; top:100%; right:0; margin-top:6px; width:320px; background:var(--bg-secondary); border:1px solid var(--border); border-radius:10px; padding:14px 16px; font-size:12px; line-height:1.6; color:var(--text-primary); z-index:100; box-shadow:0 8px 24px rgba(0,0,0,0.4); }
|
||||||
.role-label .msg-time { color:var(--text-muted); font-weight:400; font-size:10px; margin-left:6px; }
|
.role-label .msg-time { color:var(--text-muted); font-weight:400; font-size:10px; margin-left:6px; }
|
||||||
.status-dot.offline { background:var(--danger); }
|
.status-dot.offline { background:var(--danger); }
|
||||||
.status-dot.warning { background:var(--warning); }
|
.status-dot.warning { background:var(--warning); }
|
||||||
@@ -402,6 +405,21 @@ body { font-family: var(--font-body); background: var(--bg-primary); color: var(
|
|||||||
<button class="badge on" id="searchBadge" onclick="toggleSearch()" title="Toggle auto web search">SEARCH</button>
|
<button class="badge on" id="searchBadge" onclick="toggleSearch()" title="Toggle auto web search">SEARCH</button>
|
||||||
<button class="badge on" id="profileBadge" onclick="toggleProfile()" title="Toggle profile injection">PROFILE</button>
|
<button class="badge on" id="profileBadge" onclick="toggleProfile()" title="Toggle profile injection">PROFILE</button>
|
||||||
<button class="badge on" id="directionBadge" onclick="toggleDirection()" title="Toggle message ordering">NEW</button>
|
<button class="badge on" id="directionBadge" onclick="toggleDirection()" title="Toggle message ordering">NEW</button>
|
||||||
|
<button class="badge off" id="privateBadge" onclick="togglePrivate()" title="Toggle private chat mode (no persistence, no external queries)">PRIVATE</button>
|
||||||
|
<span class="info-icon" id="privateInfo" onclick="togglePrivateInfo(event)" title="What is Private Chat?">ⓘ</span>
|
||||||
|
<div class="info-popup" id="privateInfoPopup" style="display:none">
|
||||||
|
<strong>🔒 Private Chat</strong><br><br>
|
||||||
|
When enabled:<br>
|
||||||
|
• Nothing is saved to the database<br>
|
||||||
|
• No memory or RAG context is injected<br>
|
||||||
|
• Web search is disabled (no external queries)<br>
|
||||||
|
• Messages are not auto-ingested as facts<br>
|
||||||
|
• No conversation history is kept after refresh<br><br>
|
||||||
|
When disabled:<br>
|
||||||
|
• All conversations are persisted to disk<br>
|
||||||
|
• Queries may be stored for training purposes<br>
|
||||||
|
• Web search uses external services (SearXNG)
|
||||||
|
</div>
|
||||||
<button class="badge admin" id="authActionBtn" onclick="handleAuthAction()" title="Unlock admin">ADMIN</button>
|
<button class="badge admin" id="authActionBtn" onclick="handleAuthAction()" title="Unlock admin">ADMIN</button>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
@@ -452,6 +470,7 @@ let profileEnabled = true;
|
|||||||
let searchEnabled = true;
|
let searchEnabled = true;
|
||||||
let memoryEnabled = true;
|
let memoryEnabled = true;
|
||||||
let skillsEnabled = true;
|
let skillsEnabled = true;
|
||||||
|
let privateEnabled = false;
|
||||||
let presets = [];
|
let presets = [];
|
||||||
let skillsRegistry = [];
|
let skillsRegistry = [];
|
||||||
let currentModel = '';
|
let currentModel = '';
|
||||||
@@ -887,6 +906,20 @@ async function resetProfile() {
|
|||||||
function toggleProfile() { if (currentRole !== 'admin') { requireAdminNotice(); return; } profileEnabled = !profileEnabled; updateProfileUI(); saveSettings(); }
|
function toggleProfile() { if (currentRole !== 'admin') { requireAdminNotice(); return; } profileEnabled = !profileEnabled; updateProfileUI(); saveSettings(); }
|
||||||
function toggleSearch() { if (currentRole !== 'admin') { requireAdminNotice(); return; } searchEnabled = !searchEnabled; updateSearchUI(); saveSettings(); }
|
function toggleSearch() { if (currentRole !== 'admin') { requireAdminNotice(); return; } searchEnabled = !searchEnabled; updateSearchUI(); saveSettings(); }
|
||||||
function toggleMemory() { if (currentRole !== 'admin') { requireAdminNotice(); return; } memoryEnabled = !memoryEnabled; updateMemoryUI(); saveSettings(); }
|
function toggleMemory() { if (currentRole !== 'admin') { requireAdminNotice(); return; } memoryEnabled = !memoryEnabled; updateMemoryUI(); saveSettings(); }
|
||||||
|
function togglePrivate() {
|
||||||
|
privateEnabled = !privateEnabled;
|
||||||
|
const badge = document.getElementById('privateBadge');
|
||||||
|
badge.className = 'badge ' + (privateEnabled ? 'on' : 'off');
|
||||||
|
badge.textContent = privateEnabled ? 'PRIVATE' : 'PRIVATE OFF';
|
||||||
|
const searchBtn = document.getElementById('searchBtn');
|
||||||
|
if (privateEnabled) {
|
||||||
|
if (searchBtn) searchBtn.disabled = true;
|
||||||
|
showToast('Private chat: nothing stored, no external queries');
|
||||||
|
} else {
|
||||||
|
if (searchBtn && !isStreaming) searchBtn.disabled = false;
|
||||||
|
showToast('Public mode: conversations are persisted to disk');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
function updateProfileUI() {
|
function updateProfileUI() {
|
||||||
const badge = document.getElementById('profileBadge');
|
const badge = document.getElementById('profileBadge');
|
||||||
@@ -1141,6 +1174,7 @@ function showWelcome() {
|
|||||||
}
|
}
|
||||||
|
|
||||||
async function sendSearch() {
|
async function sendSearch() {
|
||||||
|
if (privateEnabled) { showToast('Web search is disabled in private chat mode'); return; }
|
||||||
resetScrollLock();
|
resetScrollLock();
|
||||||
const input = document.getElementById('userInput');
|
const input = document.getElementById('userInput');
|
||||||
const query = input.value.trim();
|
const query = input.value.trim();
|
||||||
@@ -1361,6 +1395,7 @@ async function sendMessage() {
|
|||||||
}
|
}
|
||||||
const chatBody = { conversation_id: currentConvId, message, model };
|
const chatBody = { conversation_id: currentConvId, message, model };
|
||||||
if (uploadContextId) chatBody.upload_context_id = uploadContextId;
|
if (uploadContextId) chatBody.upload_context_id = uploadContextId;
|
||||||
|
if (privateEnabled) chatBody.private_chat = true;
|
||||||
const resp = await authFetch('/api/chat', { method: 'POST', headers: {'Content-Type': 'application/json'}, body: JSON.stringify(chatBody), signal: abortController.signal });
|
const resp = await authFetch('/api/chat', { method: 'POST', headers: {'Content-Type': 'application/json'}, body: JSON.stringify(chatBody), signal: abortController.signal });
|
||||||
const reader = resp.body.getReader();
|
const reader = resp.body.getReader();
|
||||||
const decoder = new TextDecoder();
|
const decoder = new TextDecoder();
|
||||||
@@ -1441,7 +1476,7 @@ function setStreamingState(streaming) {
|
|||||||
sendBtn.textContent = 'SEND';
|
sendBtn.textContent = 'SEND';
|
||||||
sendBtn.className = 'send-btn';
|
sendBtn.className = 'send-btn';
|
||||||
sendBtn.onclick = sendMessage;
|
sendBtn.onclick = sendMessage;
|
||||||
if (searchBtn) searchBtn.disabled = false;
|
if (searchBtn) searchBtn.disabled = privateEnabled;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -1,5 +1,6 @@
|
|||||||
import json
|
import json
|
||||||
import os
|
import os
|
||||||
|
import asyncio
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
import httpx
|
import httpx
|
||||||
@@ -261,6 +262,57 @@ def test_memory_command_paths_remember_and_forget(tmp_path: Path, monkeypatch):
|
|||||||
forget_events = parse_sse_payloads(forget_resp.text)
|
forget_events = parse_sse_payloads(forget_resp.text)
|
||||||
assert any("Forgot" in p.get("token", "") for p in forget_events)
|
assert any("Forgot" in p.get("token", "") for p in forget_events)
|
||||||
|
|
||||||
memories_after_forget = client.get("/api/memories", headers={"X-Session-ID": sid, "Origin": "http://testserver"})
|
def test_private_chat_does_not_persist(tmp_path: Path, monkeypatch):
|
||||||
assert memories_after_forget.status_code == 200
|
monkeypatch.setattr(httpx.AsyncClient, "stream", lambda *a, **kw: _MockStreamResponse([
|
||||||
assert memories_after_forget.json().get("count", 0) == 0
|
'data: {"choices":[{"delta":{"content":"Hello"},"finish_reason":null,"logprobs":{"content":[]}}]}',
|
||||||
|
'data: {"choices":[{"delta":{"content":" world"},"finish_reason":null,"logprobs":{"content":[]}}]}',
|
||||||
|
'data: {"choices":[{"delta":{"content":""},"finish_reason":"stop","logprobs":{"content":[]}}],"usage":{"completion_tokens":2,"prompt_tokens":10,"tokens_per_second":5.0}}',
|
||||||
|
"data: [DONE]",
|
||||||
|
]))
|
||||||
|
monkeypatch.setattr(triage, "classify_query", lambda q: "general")
|
||||||
|
|
||||||
|
async def _mock_ensure(m): return True
|
||||||
|
monkeypatch.setattr("model_pull.ensure_model", _mock_ensure)
|
||||||
|
|
||||||
|
with make_client(tmp_path) as client:
|
||||||
|
sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()["session_id"]
|
||||||
|
headers = {"X-Session-ID": sid, "Origin": "http://testserver"}
|
||||||
|
|
||||||
|
resp = client.post("/api/chat", json={
|
||||||
|
"message": "test private message",
|
||||||
|
"private_chat": True,
|
||||||
|
}, headers=headers)
|
||||||
|
assert resp.status_code == 200
|
||||||
|
events = parse_sse_payloads(resp.text)
|
||||||
|
tokens = [p for p in events if "token" in p]
|
||||||
|
assert len(tokens) == 2
|
||||||
|
done = [p for p in events if p.get("done")]
|
||||||
|
assert len(done) == 1
|
||||||
|
|
||||||
|
conv_resp = client.get("/api/conversations", headers=headers)
|
||||||
|
assert conv_resp.status_code == 200
|
||||||
|
assert len(conv_resp.json()) == 0
|
||||||
|
|
||||||
|
|
||||||
|
def test_private_chat_does_not_auto_search(tmp_path: Path, monkeypatch):
|
||||||
|
"""Private mode should skip auto-search even when search is enabled."""
|
||||||
|
monkeypatch.setattr(httpx.AsyncClient, "stream", lambda *a, **kw: _MockStreamResponse([
|
||||||
|
'data: {"choices":[{"delta":{"content":"I don\'t know"},"finish_reason":null,"logprobs":{"content":[{"logprob":-2.5}]}}]}',
|
||||||
|
'data: {"choices":[{"delta":{"content":""},"finish_reason":"stop","logprobs":{"content":[{"logprob":-2.5}]}}],"usage":{"completion_tokens":1,"prompt_tokens":10,"tokens_per_second":5.0}}',
|
||||||
|
"data: [DONE]",
|
||||||
|
]))
|
||||||
|
monkeypatch.setattr(triage, "classify_query", lambda q: "general")
|
||||||
|
monkeypatch.setattr(routers.chat, "query_searxng", lambda q: [{"title": "result"}])
|
||||||
|
|
||||||
|
with make_client(tmp_path) as client:
|
||||||
|
sid = client.post("/api/auth/guest", headers={"Origin": "http://testserver"}).json()["session_id"]
|
||||||
|
headers = {"X-Session-ID": sid, "Origin": "http://testserver"}
|
||||||
|
|
||||||
|
resp = client.post("/api/chat", json={
|
||||||
|
"message": "what is the weather",
|
||||||
|
"private_chat": True,
|
||||||
|
}, headers=headers)
|
||||||
|
assert resp.status_code == 200
|
||||||
|
events = parse_sse_payloads(resp.text)
|
||||||
|
searching = [p for p in events if p.get("searching")]
|
||||||
|
assert len(searching) == 0, "private chat should not auto-search"
|
||||||
|
|||||||
Reference in New Issue
Block a user