gramps/cAIc
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

B8: Private Chat mode + WireGuard docs + README data safety section (v0.19.3)

Browse Source
This commit is contained in:
gramps
2026-07-14 08:52:32 -07:00
parent 4f16a9c078
commit f6b01ec9ac
9 changed files with 457 additions and 77 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/wiki/Developer-Architecture.md
+6 -4
View File
@@ -265,25 +265,28 @@ All streaming endpoints yield `data: {json}\n\n`:
- No live external services required
- Test factories reset `SESSIONS`, `PIN_ATTEMPTS`, `RATE_EVENTS` globals per test
### 8.2 Test Coverage Areas (179 tests)
### 8.2 Test Coverage Areas (198 tests)
| Test file | Coverage |
|-----------|----------|
| test_auth_capabilities.py | Guest/admin sessions, origin blocking, logout |
| test_chat_streaming_and_memory_paths.py | Streaming, auto-search, remember/forget, upload context injection |
| test_chat_streaming_and_memory_paths.py | Streaming, auto-search, remember/forget, upload context injection, private chat |
| test_cluster.py | Registration, deregistration, pong, events, coordinator query |
| test_cluster_heartbeat.py | Heartbeat handler, known/unknown node |
| test_completions.py | API key auth, FIM, streaming, blocking, errors |
| test_conversations.py | Full CRUD, guest admin, attachment_count |
| test_error_envelopes.py | Global exception handler + stream errors |
| test_gpu.py | GPU stats — rocm-smi (Linux), system_profiler (Darwin/Apple Silicon) |
| test_ingest.py | Bearer auth, chunk/embed/upsert, validation |
| test_ip_allowlist.py | IP allowlist helper + middleware |
| test_memories.py | Edit, search, stats |
| test_model_pull.py | Default model auto-pull — llama-server check, Ollama fallback, error paths |
| test_model_swap.py | request_model_swap, handle_model_ready/failed, select_node swap triggering |
| test_models_router.py | Models list, ps, show, stats, search/status |
| test_node_agent.py | Node agent registration, ping/pong, model swap |
| test_presets.py | Full CRUD, default preset protection |
| test_profile.py | Get, update, default, length validation |
| test_rag_management.py | Collection stats, eviction algorithm (pinned/grace/scoring/batch), maybe_evict hysteresis, operational stats, flush, concurrency lock |
| test_rag_management.py | Collection stats, eviction algorithm, hysteresis, flush |
| test_rate_and_payload_guardrails.py | Rate limits + payload size |
| test_search_route.py | Explicit search flow, no results, errors |
| test_search_url_sanitization.py | URL sanitizer |
@@ -291,7 +294,6 @@ All streaming endpoints yield `data: {json}\n\n`:
| test_skills_framework.py | List, toggle, unknown skill, prompt injection |
| test_triage.py | classify_query, select_node, get_inference_url |
| test_upload.py | Upload, delete, link, by-conversation, attachment_count |
| test_error_envelopes.py | Global exception handler + stream errors |
### 8.3 DoD Process
docs/wiki/WireGuard-Setup.md
+192
View File
@@ -0,0 +1,192 @@
# WireGuard Tunnel — Encrypted Node Transit
## Why
cAIc cluster traffic is plaintext today:
| Traffic | Protocol | Plaintext risk |
|---------|----------|----------------|
| AMQP (coordinator ↔ worker agent) | TCP :5672 | Registration, ping/pong, swap commands |
| Inference (coordinator → worker llama-server) | HTTP :8081 | Every token generated |
| LLM RPC layer offload (coordinator llama-server → worker) | TCP :50052 | Internal llama.cpp protocol |
WireGuard encrypts all three at the network layer with zero application changes. The cAIc app keeps using `http://` URLs — it's just talking to a virtual IP whose traffic is automatically encrypted before it hits the wire.
## Topology
```
┌───────────────────────┐ WireGuard tunnel ┌───────────────────────┐
│ Coordinator (ultron) │◄═══════════════════════════►│ Worker (jarvis) │
│ 10.0.2.1 │ UDP :51820 │ 10.0.2.2 │
│ LAN 192.168.50.108 │ │ LAN 192.168.50.210 │
│ │════════════════════════════►│ │
│ │ UDP :51820 │ Worker (corsair) │
│ │ │ 10.0.2.3 │
│ │ │ LAN (DHCP) │
└───────────────────────┘ └───────────────────────┘
```
All nodes connect directly to the coordinator's WireGuard endpoint (star topology). Workers do not need to talk to each other.
## Prerequisites
```bash
# Debian / Ubuntu
sudo apt install wireguard
# Windows / WSL2 — install WireGuard from https://www.wireguard.com/install/
# The wg.exe binary is used inside WSL2; the Windows GUI manages the tunnel config
```
## Key Generation
Run once per node. Save the private key securely; public keys go into peer configs on the other end.
```bash
wg genkey | tee /etc/wireguard/private.key | wg pubkey > /etc/wireguard/public.key
chmod 600 /etc/wireguard/private.key
```
**Recorded keys for this deployment:**
| Node | Private key | Public key |
|------|-------------|------------|
| ultron | (node private) | `ultron_pubkey=` |
| jarvis | (node private) | `jarvis_pubkey=` |
| corsair | (node private) | `corsair_pubkey=` |
## Per-Node Configs
### Coordinator — `/etc/wireguard/wg0.conf` on ultron
```ini
[Interface]
Address = 10.0.2.1/24
ListenPort = 51820
PrivateKey = <ultron_private_key>
# Enable IP forwarding so workers can route through coordinator if needed
# sudo sysctl -w net.ipv4.ip_forward=1
# sudo sysctl -w net.ipv6.conf.all.forwarding=1
# Worker: jarvis
[Peer]
PublicKey = <jarvis_pubkey>
AllowedIPs = 10.0.2.2/32
# If jarvis is off-site, put its public IP / DDNS hostname here:
# Endpoint = jarvis.example.com:51820
# If jarvis is LAN-only, set PersistentKeepalive = 25 to maintain NAT binding:
# PersistentKeepalive = 25
# Worker: corsair
[Peer]
PublicKey = <corsair_pubkey>
AllowedIPs = 10.0.2.3/32
```
### Worker — `/etc/wireguard/wg0.conf` on jarvis (Linux)
```ini
[Interface]
Address = 10.0.2.2/24
ListenPort = 51820
PrivateKey = <jarvis_private_key>
# Coordinator
[Peer]
PublicKey = <ultron_pubkey>
AllowedIPs = 10.0.2.0/24
# LAN-only: just point at the LAN IP
Endpoint = 192.168.50.108:51820
# Off-site: use DDNS or static IP:
# Endpoint = ultron.example.com:51820
PersistentKeepalive = 25
```
### Worker — Windows / WSL2 on corsair
Create a WireGuard tunnel in the Windows GUI app with the same config as jarvis above (Address=10.0.2.3/24). WSL2 inside Windows can reach the tunnel IP via the Windows host.
If llama-server runs inside WSL2 on corsair, the Windows host's WireGuard tunnel IP `10.0.2.3` is reachable from the WSL2 instance as well — just bind llama-server to `0.0.0.0` (already the default) and configure the Windows firewall to allow inbound on :8081 from the coordinator's WireGuard IP.
## Starting the Tunnel
```bash
# Start immediately
sudo systemctl start wg-quick@wg0
# Enable on boot
sudo systemctl enable wg-quick@wg0
# Check status
sudo wg show
```
Expected output on each node:
```
interface: wg0
public key: <...>
private key: (hidden)
listening port: 51820
peer: <ultron_pubkey>
endpoint: 192.168.50.108:51820
allowed ips: 10.0.2.0/24
latest handshake: 5 seconds ago ← healthy
transfer: 1.2 KiB received, 3.4 KiB sent
```
If `latest handshake` is missing, check firewall rules (UDP :51820 must be open on all nodes).
## Verification
```bash
# From any node, ping another node's WireGuard IP
ping -c 3 10.0.2.1 # coordinator
ping -c 3 10.0.2.2 # jarvis
ping -c 3 10.0.2.3 # corsair
# Verify cAIc inference through the tunnel
curl http://10.0.2.2:8081/v1/models # jarvis llama-server
curl http://10.0.2.3:8081/v1/models # corsair llama-server
```
## Updating cAIc to Use the Tunnel
Once WireGuard is running, point each service at the tunnel IP instead of the LAN IP.
### Worker node agent config — `/etc/caic-node-agent.conf`
```ini
[agent]
node_name = jarvis
node_ip = 10.0.2.2 # was 192.168.50.210
node_type = worker
capabilities = llm
amqp_url = amqp://caic:password@10.0.2.1:5672/caic # was 192.168.50.108
llama_port = 8081
models_dir = /var/lib/caic/models
active_model = qwen2.5-7b-instruct-Q5_K_M.gguf
```
### Coordinator config — environment variables
```bash
# On the coordinator node, override the worker-facing addresses
# (LLAMA_SERVER_BASE stays as localhost / LAN IP since inference
# to the coordinator's own llama-server stays on-machine)
export CAIC_AMQP_URL="amqp://caic:password@10.0.2.1:5672/caic"
```
No other cAIc code changes are needed. The app already reads `CAIC_AMQP_URL` from the environment (`config.py:27`) and the node agent reads `node_ip` from its INI file. Inference requests routed to remote workers via `triage.py` use the IP the worker registered — so setting `node_ip = 10.0.2.2` in the worker's agent config is all it takes.
## Cross-Site Deployment Checklist
When placing a worker outside the LAN:
1. **Firewall:** Open UDP :51820 on the remote site. On the coordinator side, make sure UDP :51820 is reachable from the internet (port forward / firewall rule at the coordinator's router).
2. **DDNS:** If the coordinator's public IP is dynamic, set up a DDNS hostname and use it in the worker's `Endpoint = ultron.example.com:51820`.
3. **PersistentKeepalive:** Set `PersistentKeepalive = 25` on the worker side to keep NAT bindings alive.
4. **No double encryption:** WireGuard encrypts everything on the WireGuard interface. The cAIc app continues to use `http://` — it never touches raw TLS. This is correct and intended.
5. **Split tunnelling (optional):** The worker's `AllowedIPs = 10.0.2.0/24` ensures only cluster traffic goes through the tunnel. All other internet traffic from the worker uses its normal gateway.
docs/wiki/current-wip.md
+9 -5
View File
@@ -1,19 +1,23 @@
# cAIc Current WiP Backlog
Last updated: 2026-07-06
Last updated: 2026-07-14
Owner: Gramps
Scope: Active roadmap items and backlog.
## ~~Roadmap N: AMQP Cluster Nervous System [COMPLETE]~~
## Completed
All 15 tasks are done. Final commit: `f0689ac feat: Roadmap N — AMQP cluster nervous system complete`.
- **B7 (v0.19.0)** — Apple Silicon worker support. gpu.py darwin branch, hardware.py darwin VRAM, node_agent/agent.py macOS VRAM reporting.
- **B5 (v0.19.1)** — Default model auto-pull. model_pull.py + ensure_model() in app.py lifespan.
- **B6 (v0.19.2)** — Waterfall direction toggle (NEW/OLD), scroll/lock fixes, toast notifications, execCopy, modelLabel, Ctrl+Enter.
- **B8 (v0.19.3)** — Private Chat mode. Backend skip-DB/skip-RAG/skip-search flag, frontend PRIVATE badge, info popup.
- **WireGuard TLS (v0.19.4)** — Self-signed WireGuard mesh encrypts all inter-node traffic (AMQP, inference, RPC). No code changes to cAIc. Documented in wiki/WireGuard-Setup.md + docker.md §5.4.
## Backlog
- B1 — Context loss in follow-up questions (investigation)
- B2 — Bang-prefixed (`!`) search routing
- B3 — Docker distribution (planning doc at `docker.md`)
- **B4 — RAG Corpus Management UI (NEXT)** — browse, search, edit, delete individual RAG entries. Backend endpoints at `routers/rag_admin.py`, frontend panel in `templates/index.html`
- B3 — Docker distribution (planning doc at `docker.md`, not yet implemented)
- **B4 — RAG Corpus Management UI (deferred)** — browse, search, edit, delete individual RAG entries
- HTTPS / reverse proxy (Caddy)
- Conversation search/filter and export tooling
- Keyboard shortcuts, retry button, source-link polish