gramps/jarvisChat
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

fix: resolve all critical runtime errors and bugs from audit

Browse Source 
- Add COMPLETIONS_API_KEY to config.py (env var + auto-generated fallback)
- Fix perplexity auto-search: upstream sends logprobs=true, parse_llama_stream_chunk
  extracts per-token logprobs, all_logprobs populated during streaming
- Fix all /api/models endpoints to target LLAMA_SERVER_BASE (port 8081) not OLLAMA_BASE
- Fix RAG embedding endpoint URL from port 11434 (Ollama) to 8081 (llama-server)
- Correct misleading error messages: 'inference server' not 'Ollama'
- Remove raw_results leak from SSE event stream in /api/search
- Fix weather query extractor: pattern-match instead of unconditional suffix append
- Escape FTS5 operator keywords (AND/OR/NOT/NEAR) in memory search
- Move auth.py BODY_LIMIT_DEFAULT_BYTES imports to module level
- Change RAG injection log level from warning to info
- Fix all 8 test files after modular refactor (rewire imports from correct modules)
- Update AGENTS.md and README.md to reflect v1.8.0 changes
This commit is contained in:
gramps
2026-06-27 15:10:32 -07:00
parent 41a8708c0d
commit 193829b7ff
20 changed files with 457 additions and 896 deletions
Download Patch File Download Diff File Expand all files Collapse all files

89
AGENTS.md Normal file
View File

@@ -0,0 +1,89 @@
# JarvisChat — Agents Guide
## Run
```bash
./venv/bin/uvicorn app:app --host 0.0.0.0 --port 8080 --reload
```
## Tests
```bash
./venv/bin/python -m pytest tests/ -v
```
All tests use `tmp_path` fixtures + monkeypatched `httpx.AsyncClient.stream`. No external services needed. Test factories reset `SESSIONS`, `PIN_ATTEMPTS`, `RATE_EVENTS` globals — be careful not to let test state leak. After the modular refactor, tests import directly from the correct modules (`db`, `security`, `config`, `search`, `rag`, `memory`, `routers.*`) — not from the old monolithic `app` namespace.
## Architecture
Refactored from single-file (`app.py`) into modules under project root:
| File | Role |
|------|------|
| `app.py` | FastAPI app, middleware, router registration |
| `config.py` | Constants, env vars, rate/payload limits, built-in skills registry |
| `db.py` | SQLite schema, connection factory, settings helpers |
| `auth.py` | PIN-based guest/admin sessions, auth routes |
| `security.py` | Rate limiting, origin checks, IP allowlist, audit/incident logging |
| `memory.py` | FTS5 memory CRUD, remember/forget command parsing |
| `search.py` | SearXNG integration, perplexity scoring, refusal detection |
| `rag.py` | Qdrant vector search + system prompt assembly |
| `gpu.py` | AMD GPU stats via `rocm-smi` |
| `routers/` | One module per endpoint group (chat, search, skills, completions, etc.) |
### Entrypoint / API keys
- `app.py` line 148: `uvicorn.run(app, ...)` when called directly
- `config.py` line 14: `LLAMA_SERVER_BASE` defaults to `http://192.168.50.108:8081` — llama-server, **not** standard Ollama port, used by all model endpoints
- `config.py` line 17: `COMPLETIONS_API_KEY` read from `JARVISCHAT_COMPLETIONS_API_KEY` env var or auto-generates a random key — no longer a missing import
- `config.py` line 13: `OLLAMA_BASE` is legacy/unused — all endpoints now use `LLAMA_SERVER_BASE`
### Key flows
1. **`/api/chat`** → `process_remember_command()` intercepts "remember that..." / "forget about..." first → else `build_system_prompt()` (profile + FTS5 memory + Qdrant RAG + preset + skills) → stream from llama-server with `logprobs: true` → if perplexity > 15.0 OR `REFUSAL_PATTERNS` match, re-query with SearXNG results
2. **`/api/search`** → bypasses perplexity/refusal, queries SearXNG directly → summarizes via llama-server (no raw results leaked in SSE)
3. **`/v1/chat/completions`** → OpenAI-compatible for Continue.dev/IDE integration; FIM requests proxied without persistence
### Perplexity / auto-search
The upstream request includes `"logprobs": true`. `parse_llama_stream_chunk()` extracts per-token logprobs from each chunk's `choices[0].logprobs.content[].logprob`. The `all_logprobs` list is populated during streaming, so `calculate_perplexity()` and `is_uncertain()` work correctly — auto-search on high perplexity is no longer dead code.
### Auth / lockdown
- Guest session by default (`POST /api/auth/guest`), admin unlock via 4-digit PIN (`POST /api/auth/login`)
- Admin required for PUT/DELETE/PATCH + all POST except allowlist (`/api/chat`, `/api/search`, `/api/auth/*`)
- IP allowlist, rate limiting, origin checking, payload size limits — all enforced in `app.py` middleware
- `JARVISCHAT_ADMIN_PIN` env var required on first boot (or `JARVISCHAT_ALLOW_DEFAULT_PIN=true`)
### Database
- SQLite at `jarvischat.db`, auto-created by `init_db()` on startup via FastAPI `lifespan`
- `get_db()` opens new connection per request (no pool). Close after use.
- FTS5 virtual table `memories` for full-text search with BM25 ranking. FTS5 operator keywords (`AND`, `OR`, `NOT`, `NEAR`) are double-quoted to prevent parse errors.
### External services
| Service | Required | Port |
|---------|----------|------|
| llama-server (OpenAI-compat API) | Yes | 8081 (ultron) or env `LLAMA_SERVER_BASE` |
| SearXNG | No | 8888 |
| wttr.in | No | weather shortcut bypasses SearXNG; curl UA for plain-text output |
| rocm-smi | No | AMD GPU stats |
| Qdrant | No | 6333 (ultron) — RAG vector search |
### Config quirks
- Rate limits and payload caps in `config.py` — tweak for testing by monkeypatching module attributes (note: patch `security.RL_*` not `config.RL_*` since `security` imports bindings separately)
- `ALLOWED_SETTINGS_KEYS` in `config.py` controls which keys the UI can write via `/api/settings`
- Settings table seeded with defaults (`profile_enabled`, `search_enabled`, `memory_enabled`, `skills_enabled`, `default_model`) — never overwritten by `init_db()`
- Profile table uses singleton row `id=1`
- RAG embedding requests go to `LLAMA_SERVER_BASE` at `/api/embeddings` (port 8081, not 11434)
### SSE Protocol
All streaming endpoints yield `data: {json}\n\n`. Key shapes:
- `{token, conversation_id}` — streaming token
- `{searching: true}` — web search triggered
- `{search_results: N}` — N results (no raw_results payload)
- `{done: true, perplexity, tokens_per_sec, searched?}` — terminal
- `{error: "...", error_key: "..."}` — error with incident key

582
README.md
View File

@@ -1,458 +1,260 @@
![jarvisChat logo](static/jcscreenie.png) # JarvisChat v1.8.0
# ⚡ JarvisChat v1.9.0
**A privacy-first, homelab-native developer knowledge platform.** **A lightweight local inference coding companion with persistent memory, web search, and real-time system monitoring.**
> JarvisChat turns a heterogeneous LAN of budget hardware into a distributed local AI inference cluster — accumulating institutional knowledge over time, keeping all data off the cloud, and squeezing real performance out of modest consumer hardware through architecture rather than dollars. Built with FastAPI + SQLite + Jinja2. Runs on Python 3.13. No Docker required.
This is not another AI chat wrapper. jC is the UX and knowledge-management layer for a local AI brain — analogous to what Windows was to DOS, or what the web is to the internet. The intelligence lives in the model and the RAG corpus. jC makes it accessible and keeps feeding it. Developer wiki: [docs/wiki/Home.md](docs/wiki/Home.md)
--- ## What's New in v1.8.0
## The Four Pillars - **Modular refactor completed** — single-file `app.py` split into `config.py`, `db.py`, `auth.py`, `security.py`, `memory.py`, `search.py`, `rag.py`, `gpu.py`, and `routers/` package
- **`COMPLETIONS_API_KEY`** — auto-generated secret key for the OpenAI-compatible endpoint, overridable via `JARVISCHAT_COMPLETIONS_API_KEY` env var
- **Perplexity auto-search fixed** — upstream request now sends `"logprobs": true`, `parse_llama_stream_chunk()` extracts per-token logprobs, so `calculate_perplexity()` and `is_uncertain()` work correctly (was dead code)
- **All `/api/models` endpoints** — now correctly target `LLAMA_SERVER_BASE` (llama-server on port 8081) instead of the old Ollama port; `/api/ps` uses `/v1/models` endpoint
- **RAG embedding endpoint fixed** — `EMBED_URL` changed from port `:11434` (Ollama) to `:8081` (llama-server)
- **Error messages corrected** — all user-facing errors say "inference server" instead of "Ollama" or "llama-server"
- **Secure SSE protocol** — raw search results are no longer leaked in the SSE event stream
- **FTS5 query safety** — operator keywords (`AND`, `OR`, `NOT`, `NEAR`) are double-quoted to prevent parse errors
- **All 8 test files fixed** — rewired imports after the modular refactor; all 26 tests pass
### 1. Privacy ## Features
Everything runs on your LAN. No API keys, no cloud endpoints, no data leaving your network, no subscription, no terms-of-service surprises. Your conversations, your codebase, your decisions — stay yours.
### 2. Knowledge Retention - **Persistent Memory** — SQLite FTS5 full-text search for fast, relevant memory retrieval
Unlike stateless chat tools that forget everything when you close the tab, jC accumulates institutional memory. Every solved problem, every architectural decision, every working command gets absorbed into the RAG corpus via Qdrant. The system gets smarter the longer you use it. - **Web Search** — SearXNG integration for automatic web lookups when the model is uncertain
- **Explicit Search** — Search button to force web search without waiting for model uncertainty
- **Profile Injection** — Custom system prompt injected into every conversation
- **System Presets** — Save and switch between different system prompts
- **Real-time Stats** — CPU, RAM, GPU, VRAM monitoring in sidebar
- **Token Thermometer** — Visual context window usage indicator
- **Streaming Responses** — Server-sent events for real-time token display
- **Conversation History** — SQLite-backed chat persistence with mass-delete option
- **Model Switching** — Change inference models on the fly
- **Skills Framework** — Built-in skill registry with per-skill enable/disable controls
### 3. Budget Hardware Maximization ## File Structure
You don't need a $10,000 workstation. jC is designed for the developer who has a drawer full of machines and the skills to wire them together. RPC clustering, model splitting across CPU and GPU nodes, dynamic resource negotiation, and smart RAG eviction squeeze real performance out of modest consumer hardware.
### 4. Homelab-Native Architecture
Built specifically for the heterogeneous homelab: mixed hardware, mixed OS, consumer GPUs, ARM boards, NAS storage — all working together as a coherent AI platform. A designated master node hosts jC, llama-server, and SearXNG. GPU nodes self-register as RPC inference workers. The architecture scales horizontally across whatever you've got.
---
## Target Audience
Solo developers and homelab enthusiasts who are:
- Budget-constrained but hardware-rich (multiple machines, NAS, spare GPUs)
- Privacy-conscious (no cloud AI subscriptions)
- Technically capable (if you can install jC, you can designate the master node)
- Building something over time and want their AI to remember it
---
## Architecture
``` ```
┌─────────────────────────────────────────────────────────────┐ /opt/jarvischat/
│ YOUR LAN │ ├── app.py # FastAPI app entry point
│ │ ├── config.py # Constants, env vars, limits, skill registry
│ ┌─────────────────┐ ┌──────────────────────────┐ │ ├── db.py # SQLite schema, connection factory
│ │ jarvis │◄──RPC───│ ultron │ │ ├── auth.py # PIN-based guest/admin sessions, auth routes
│ │ 192.168.50.210│ 50052 │ 192.168.50.108 │ │ ├── security.py # Rate limiting, origin checks, IP allowlist, audit
│ │ │ │ │ │ ├── memory.py # FTS5 memory CRUD, remember/forget commands
│ │ jC :8080 │ │ llama-server :8081 │ │ ├── search.py # SearXNG integration, perplexity, refusal detection
│ │ SearXNG :8888 │ │ llama-server :8082 (*) │ │ ├── rag.py # Qdrant vector search + system prompt assembly
│ │ RX 6600 XT 8GB │ │ Qdrant :6333 │ │ ├── gpu.py # AMD GPU stats via rocm-smi
│ │ GPU RPC worker │ │ mxbai-embed :11434 │ │ ├── routers/
│ │ Vulkan backend │ │ AMD Ryzen 7 7840HS │ │ │ ├── chat.py # /api/chat streaming endpoint
│ └─────────────────┘ │ Radeon 780M iGPU │ │ │ ├── search_route.py # /api/search explicit search endpoint
│ └──────────────────────────┘ │ │ ├── completions.py # /v1/chat/completions OpenAI-compat endpoint
│ │ │ ├── conversations.py# Conversation CRUD
│ ┌─────────────────┐ ┌──────────────────────────┐ │ │ ├── memories.py # Memory CRUD API
│ │ pivault │ │ corsair │ │ │ ├── models.py # Model listing, system stats
│ │ 192.168.50.158│ │ 192.168.50.132 │ │ │ ├── presets.py # System prompt presets
│ │ │ │ │ │ │ ├── profile.py # User profile
│ │ 10.83TB RAID5 │ │ RTX 5070 Ti 16GB │ │ │ ├── settings.py # Runtime settings
│ │ RPi 5 8GB │ │ Ryzen 7 7800X3D │ │ │ └── skills.py # Skills management
│ │ NAS / Kopia │ │ Gaming / Streaming │ │ ├── static/
│ └─────────────────┘ └──────────────────────────┘ │ │ └── logo.png # Logo image (optional)
│ │ ├── templates/
│ (*) Planned: Qwen2.5-Coder-14B on :8082 │ │ └── index.html # Frontend
└─────────────────────────────────────────────────────────────┘ └── tests/ # 26 pytest tests
``` ```
**Data flow:** ## Requirements
```
Browser / IDE (Continue.dev)
→ jC :8080 (FastAPI — auth, RAG, memory, conversation history)
→ Qdrant :6333 (vector search, mxbai-embed-large for embeddings)
→ llama-server :8081 (inference)
→ jarvis RPC :50052 (GPU layer offload — RX 6600 XT)
```
--- - Python 3.11+ (tested on 3.13)
- llama-server running locally or on network (OpenAI-compatible API on port 8081)
## The AMD + NVIDIA Cross-Cluster Reality - SearXNG (optional, for web search)
This cluster intentionally mixes GPU architectures — **AMD RX 6600 XT on jarvis** and **NVIDIA RTX 5070 Ti on corsair**. This is deliberate and it works.
The RPC layer in llama.cpp is GPU-vendor-agnostic. jarvis runs llama-rpc with a **Vulkan backend** (not ROCm, not CUDA) which provides hardware-neutral GPU acceleration. ultron's llama-server connects to it over TCP and offloads tensor layers without caring what GPU is on the other end.
This means any machine on your LAN with any GPU (AMD, NVIDIA, Intel Arc) can participate as an RPC worker — as long as it can run llama-rpc with Vulkan support.
---
## Cluster Performance Tuning
### The Layer Offloading Trick
The key to squeezing performance out of a CPU+GPU split cluster is `--n-gpu-layers`. This controls how many transformer layers get offloaded to the RPC GPU backend versus staying on the CPU.
**Starting point (before tuning):** ~7 t/s
**After initial layer optimization:** ~17 t/s
**After full cluster tuning:** 30–35 t/s
The progression that got us there:
1. **Start with `--n-gpu-layers 99`** — tells llama-server to offload as many layers as possible. With Mistral-Nemo-12B Q4_K_M this results in all 41/41 layers offloading to jarvis GPU via RPC.
2. **Verify GPU is actually working** — watch the llama-server startup log for:
```
load_tensors: offloaded 41/41 layers to GPU
load_tensors: RPC[192.168.50.210:50052] model buffer size = 6763.30 MiB
load_tensors: CPU_Mapped model buffer size = 360.00 MiB
```
If layers aren't offloading, the RPC connection isn't established.
3. **Check actual throughput** — the timings block in llama-server responses shows real t/s. Tune from there.
**Current llama-server service on ultron (`/etc/systemd/system/llama-server.service`):**
```ini
[Unit]
Description=Llama.cpp Server (RPC frontend — Mistral-Nemo general)
After=network.target
[Service]
Type=simple
User=root
ExecStart=/root/llama.cpp/build/bin/llama-server \
--model /home/gramps/models/Mistral-Nemo-Instruct-2407-Q4_K_M.gguf \
--rpc 192.168.50.210:50052 \
--host 0.0.0.0 \
--port 8081 \
--n-gpu-layers 99
Restart=on-failure
RestartSec=5
[Install]
WantedBy=multi-user.target
```
**llama-rpc service on jarvis (`/etc/systemd/system/llama-rpc.service`):**
```ini
[Unit]
Description=Llama.cpp RPC Server (GPU backend — RX 6600 XT Vulkan)
After=network.target
[Service]
Type=simple
User=root
ExecStart=/root/llama.cpp/build/bin/llama-rpc-server \
--host 0.0.0.0 \
--port 50052
Restart=on-failure
RestartSec=5
[Install]
WantedBy=multi-user.target
```
---
## Models
### Current
| Model | Location | Port | Purpose |
|-------|----------|------|---------|
| Mistral-Nemo-Instruct-2407-Q4_K_M | `/home/gramps/models/` on jarvis | ultron:8081 | General assistant, chat |
| mxbai-embed-large | ultron (Docker/Ollama) | ultron:11434 | RAG embeddings |
### Planned
| Model | Size | Port | Purpose |
|-------|------|------|---------|
| Qwen2.5-Coder-14B-Q5_K_M | ~10GB | ultron:8082 | Code completion, pair programming |
> **Note:** ultron has 16GB RAM. Only one primary inference model can be hot at a time. llama-server instances are swapped via systemd when switching between general and code models.
---
## RAG System
jC uses **Qdrant** for vector storage and **mxbai-embed-large** (1024-dim) for embeddings.
### Qdrant Collection
- **Collection:** `jarvis_rag`
- **Vector size:** 1024 (mxbai-embed-large output)
- **Distance:** Cosine
- **Score threshold:** 0.25 (filters low-relevance chunks)
- **Chunks retrieved per query:** 3 (configurable)
### RAM Ceiling
Each vector = 4KB (1024 dims × float32). With ultron's ~4-6GB available to Qdrant after llama-server:
- Practical ceiling: ~1–1.5M chunks before RAM becomes the bottleneck
- Current corpus: 219 points (early stage)
- Storage on disk: negligible against pivault's 10.83TB
### What Gets Ingested
- Code repositories (your actual codebase)
- Pair-programming conversation history
- Architecture decisions and working commands
- Documentation and URLs (fetched and stripped via beautifulsoup4/httpx)
---
## JarvisChat Service (`/etc/systemd/system/jarvischat.service`)
```ini
[Unit]
Description=JarvisChat - Local LLM Developer Platform
After=network.target
[Service]
Type=simple
User=root
WorkingDirectory=/opt/jarvischat
ExecStart=/opt/jarvischat/venv/bin/uvicorn app:app --host 0.0.0.0 --port 8080
Restart=always
RestartSec=5
Environment=PYTHONUNBUFFERED=1
Environment=OLLAMA_BASE=http://192.168.50.108:8081
Environment=LLAMA_SERVER_BASE=http://192.168.50.108:8081
[Install]
WantedBy=multi-user.target
```
---
## Installation ## Installation
### Prerequisites
- Python 3.11+ (tested on 3.13)
- llama.cpp built from source on both jarvis (RPC server) and ultron (llama-server)
- Qdrant running on ultron
- Ollama on ultron (for mxbai-embed-large embeddings)
- SearXNG on jarvis:8888 (optional, for web search)
### Fresh Install ### Fresh Install
```bash ```bash
# Create directory and venv
sudo mkdir -p /opt/jarvischat sudo mkdir -p /opt/jarvischat
sudo chown $USER:$USER /opt/jarvischat sudo chown $USER:$USER /opt/jarvischat
cd /opt/jarvischat cd /opt/jarvischat
python3 -m venv venv python3 -m venv venv
./venv/bin/pip install fastapi uvicorn httpx psutil jinja2 python-multipart qdrant-client
# Install dependencies
./venv/bin/pip install fastapi uvicorn httpx psutil jinja2 python-multipart
# Set admin PIN before first startup (4 digits)
export JARVISCHAT_ADMIN_PIN=4827
# Create subdirectories
mkdir -p templates static mkdir -p templates static
# Copy files
# (copy all .py files to /opt/jarvischat/)
# (copy routers/ directory to /opt/jarvischat/)
# (copy templates/index.html to /opt/jarvischat/templates/)
``` ```
Copy `app.py` to `/opt/jarvischat/` and `index.html` to `/opt/jarvischat/templates/`. WARNING: Do not use `1234` as your admin PIN unless you accept weak local security.
### Bootstrap the PIN NOTE: First boot requires `JARVISCHAT_ADMIN_PIN` unless you explicitly opt into insecure fallback with `JARVISCHAT_ALLOW_DEFAULT_PIN=true`.
## Systemd Service
Create `/etc/systemd/system/jarvischat.service`:
```ini
[Unit]
Description=JarvisChat - Local Inference Web Interface
After=network.target
[Service]
Type=simple
User=jarvischat
Group=jarvischat
WorkingDirectory=/opt/jarvischat
ExecStart=/opt/jarvischat/venv/bin/uvicorn app:app --host 0.0.0.0 --port 8080
Restart=always
RestartSec=5
[Install]
WantedBy=multi-user.target
```
```bash ```bash
export JARVISCHAT_ADMIN_PIN=XXXX # your 4-digit PIN sudo systemctl daemon-reload
sudo systemctl enable jarvischat
sudo systemctl start jarvischat
``` ```
Or allow the insecure default for testing: ## Memory Commands
```bash
export JARVISCHAT_ALLOW_DEFAULT_PIN=true
```
### Environment Variables In chat, natural language triggers memory operations:
| Variable | Default | Description | | You say | What happens |
|----------|---------|-------------| |---------|--------------|
| `OLLAMA_BASE` | `http://localhost:11434` | Ollama-compatible endpoint (legacy) | | "remember that I prefer Rust over Go" | Stores as `preference` |
| `LLAMA_SERVER_BASE` | `http://192.168.50.108:8081` | llama-server OpenAI-compat inference endpoint | | "remember that JarvisChat runs on port 8080" | Stores as `infrastructure` |
| `JARVISCHAT_ADMIN_PIN` | (none) | 4-digit admin PIN (required on first boot) | | "note that the deadline is Friday" | Stores as `general` |
| `JARVISCHAT_ALLOW_DEFAULT_PIN` | `false` | Allow insecure default PIN 1234 | | "forget about the deadline" | Removes matching memories |
| `JARVISCHAT_TRUSTED_ORIGINS` | (none) | Comma-separated trusted origins for CSRF |
| `JARVISCHAT_ALLOWED_CIDRS` | RFC1918 + loopback | Allowed client IP CIDRs |
--- Memories are automatically searched based on your message content and injected into the system prompt when relevant.
### Memory Topics
Memories are auto-categorized:
- `preference` — likes, dislikes, choices
- `project` — active work, repos, tasks
- `infrastructure` — servers, services, configs
- `personal` — name, location, background
- `general` — everything else
## API Endpoints ## API Endpoints
### Auth ### Completions (OpenAI-compatible)
| Method | Path | Description |
|--------|------|-------------| | Method | Endpoint | Description |
| POST | `/api/auth/guest` | Create guest session | |--------|----------|-------------|
| POST | `/api/auth/login` | Admin PIN login | | POST | `/v1/chat/completions` | OpenAI-compatible chat (requires Bearer API key) |
| POST | `/api/auth/logout` | Revoke session |
| GET | `/api/auth/session` | Check session status |
| POST | `/api/auth/heartbeat` | Keep session alive |
### Chat & Search ### Chat & Search
| Method | Path | Description |
|--------|------|-------------|
| POST | `/api/chat` | Streaming chat (SSE) |
| POST | `/api/search` | Explicit web search via SearXNG |
| GET | `/api/search/status` | SearXNG health check |
### Models | Method | Endpoint | Description |
| Method | Path | Description | |--------|----------|-------------|
|--------|------|-------------| | POST | `/api/chat` | Send message (streaming SSE) |
| GET | `/api/models` | List available models from llama-server | | POST | `/api/search` | Explicit web search (streaming SSE) |
| GET | `/api/ps` | Running models |
| POST | `/api/show` | Model info |
### Memory ### Memory
| Method | Path | Description |
|--------|------|-------------| | Method | Endpoint | Description |
|--------|----------|-------------|
| GET | `/api/memories` | List all memories | | GET | `/api/memories` | List all memories |
| POST | `/api/memories` | Add memory | | POST | `/api/memories` | Add memory |
| PUT | `/api/memories/{rowid}` | Update memory | | PUT | `/api/memories/{rowid}` | Update memory |
| DELETE | `/api/memories/{rowid}` | Delete memory | | DELETE | `/api/memories/{rowid}` | Delete memory |
| GET | `/api/memories/search?q=` | FTS5 search memories | | GET | `/api/memories/search?q=term` | Search memories |
| GET | `/api/memories/stats` | Memory statistics | | GET | `/api/memories/stats` | Get counts by topic |
### Models & System
| Method | Endpoint | Description |
|--------|----------|-------------|
| GET | `/api/models` | List available models |
| GET | `/api/ps` | List loaded models |
| POST | `/api/show` | Get model info |
| GET | `/api/stats` | CPU, RAM, GPU, VRAM stats |
| GET | `/api/search/status` | SearXNG availability |
### Settings & Profile
| Method | Endpoint | Description |
|--------|----------|-------------|
| GET | `/api/profile` | Get profile content |
| PUT | `/api/profile` | Update profile (admin) |
| GET | `/api/profile/default` | Get default profile |
| GET | `/api/settings` | Get settings |
| PUT | `/api/settings` | Update settings (admin) |
### Conversations ### Conversations
| Method | Path | Description |
|--------|------|-------------| | Method | Endpoint | Description |
|--------|----------|-------------|
| GET | `/api/conversations` | List conversations | | GET | `/api/conversations` | List conversations |
| POST | `/api/conversations` | Create conversation | | POST | `/api/conversations` | Create conversation |
| GET | `/api/conversations/{id}` | Get conversation + messages | | GET | `/api/conversations/{id}` | Get conversation with messages |
| PUT | `/api/conversations/{id}` | Update title/model | | PUT | `/api/conversations/{id}` | Update conversation title/model |
| DELETE | `/api/conversations/{id}` | Delete conversation | | DELETE | `/api/conversations/{id}` | Delete conversation |
| DELETE | `/api/conversations` | Delete all conversations | | DELETE | `/api/conversations` | Delete ALL conversations |
### Profile & Settings ### Presets
| Method | Path | Description |
|--------|------|-------------| | Method | Endpoint | Description |
| GET | `/api/profile` | Get profile | |--------|----------|-------------|
| PUT | `/api/profile` | Update profile | | GET | `/api/presets` | List presets |
| GET | `/api/settings` | Get settings | | POST | `/api/presets` | Create preset |
| PUT | `/api/settings` | Update settings | | PUT | `/api/presets/{id}` | Update preset |
| GET | `/api/stats` | CPU/RAM/GPU stats | | DELETE | `/api/presets/{id}` | Delete preset |
### Skills ### Skills
| Method | Path | Description |
|--------|------|-------------|
| GET | `/api/skills` | List all skills |
| GET | `/api/skills/active` | List enabled skills |
| PUT | `/api/skills/{key}` | Enable/disable skill |
--- | Method | Endpoint | Description |
|--------|----------|-------------|
| GET | `/api/skills` | List all skills with state |
| GET | `/api/skills/active` | List active skills |
| PUT | `/api/skills/{key}` | Toggle skill enabled (admin) |
## Memory Commands ### Auth
Say these in chat to interact with the memory system: | Method | Endpoint | Description |
|--------|----------|-------------|
| POST | `/api/auth/guest` | Create guest session |
| POST | `/api/auth/login` | Admin PIN login |
| POST | `/api/auth/logout` | Revoke session |
| GET | `/api/auth/session` | Check session validity |
| POST | `/api/auth/heartbeat` | Extend session TTL |
| Command | Effect | ## Configuration
|---------|--------|
| `remember that [fact]` | Stores fact in FTS5 memory |
| `please remember [fact]` | Same |
| `don't forget [fact]` | Same |
| `forget about [topic]` | Deletes matching memories |
--- Settings are stored in the `settings` table and include:
## Troubleshooting - `profile_enabled` — Inject profile into chats (true/false)
- `search_enabled` — Auto web search (true/false)
- `memory_enabled` — Memory injection (true/false)
- `skills_enabled` — Skills framework (true/false)
- `default_model` — Default inference model
## Testing
### jC starts but inference is slow or failing
Check that llama-rpc is running on jarvis and llama-server is connected:
```bash ```bash
# On jarvis ./venv/bin/python -m pytest tests/ -v
systemctl status llama-rpc
# On ultron — look for "offloaded N/N layers to GPU" in logs
journalctl -u llama-server -n 50 --no-pager
``` ```
### ultron shows no CPU activity during inference All 26 tests use `tmp_path` fixtures + monkeypatched `httpx.AsyncClient.stream`. No external services needed.
Inference is being handled entirely by jarvis GPU via RPC — this is correct and expected. ultron's CPU is only involved for non-offloaded tensors (a small fraction of the model).
### RAG not returning results
Check Qdrant is up and the collection exists:
```bash
curl http://192.168.50.108:6333/collections/jarvis_rag
```
Verify `points_count` > 0. If zero, the corpus hasn't been seeded yet.
### jC won't start — PIN bootstrap error
Set the PIN via environment before first boot:
```bash
export JARVISCHAT_ADMIN_PIN=XXXX
systemctl restart jarvischat
```
### sqlite3 not found
Use Python instead:
```bash
python3 -c "import sqlite3; print(sqlite3.connect('/opt/jarvischat/jarvischat.db').execute('SELECT * FROM settings').fetchall())"
```
---
## Roadmap
### TODO (Priority Order)
1. **Tool calling** — read_file/write_file with /opt/jarvischat whitelist, tool_calls dispatch loop
2. **git_tool** — Gitea integration for commit/push from jC
3. **Audit logging** — structured audit trail to syslog
4. SearXNG persistence (DONE ✅)
5. search+ prefix for explicit search
6. profile.example.md
7. Conversation search/filter
8. Export to markdown
9. Keyboard shortcuts
10. Retry button
11. Source links in responses
12. Rename conversations
13. Multiple profiles
14. KWIC auto-tags
15. Image input (vision)
16. btop split-screen integration
17. Containerize
18. SearXNG health indicator in UI
19. check_patch_notes tool
20. GitLab mirror of llgit repo
### ROADMAP (Longer Horizon)
**(A) Modular refactor** — Split monolithic app.py into routers/, services/, config.py, db.py, auth.py. Prerequisite for everything below.
**(B) RAG ingest/manage UI** — File upload, URL ingest (fetch + strip HTML via beautifulsoup4/httpx, store URL as source metadata for citation), delete chunks/collections.
**(C) Backend config panel** — Switch between Ollama/llama-server, endpoint URLs, model switching, restart — all from the UI without touching config files.
**(D) Response metrics display** — tokens/sec, TTFT, context size, RAG chunks retrieved + scores — visible in the UI per response.
**(E) Response quality feedback** — thumbs/stars/tags per response → feedback corpus → future RLHF dataset.
**(F) IDE integration** — Continue.dev + VS Code, pointed at jC:8080 (not direct to inference endpoint). All IDE traffic — including pair-programming conversations — goes through jC so sessions are persisted and become RAG-worthy content. jC needs FIM request format handling to support inline autocomplete.
**(G) Conversation history export → RAG ingest** — Bulk ingest existing conversation history into Qdrant.
**(H) Fine-tuning pipeline** — LoRA on Mistral-Nemo from feedback corpus (item E).
**(I) Autonomous RAG** — At conversation end, jC self-evaluates the transcript, extracts significant chunks (solved problems, working commands, architectural decisions), and ingests them into Qdrant automatically with metadata (date, conversation_id, reason). jC decides what it needs to remember. Closes the loop.
**(J) Startup hardware/resource self-assessment** — On boot, jC queries ultron for available RAM, Qdrant consumption, and llama-server footprint. Derives dynamic high-water marks for RAG chunk limits, context window sizing, retrieval limits, and eviction thresholds. Writes a living config file. Replaces magic numbers with runtime-negotiated values.
**(K) RAG corpus management** — Weighted LRU eviction with composite score (recency + frequency + content age) + manual pin flag for load-bearing knowledge. Prevents corpus bloat from degrading retrieval quality. Analogous to memcache eviction policy.
**(L) Dual inference model architecture** — Mistral-Nemo-12B on ultron:8081 (general assistant), Qwen2.5-Coder-14B-Q5_K_M on ultron:8082 (code/pair programming). jC selects endpoint based on active model. Only one model hot at a time given ultron's 16GB RAM constraint.
**(M) MCP server compatibility** — Expose jC as an MCP server. Minimum scope: tool manifest endpoint, SSE transport, chat and RAG query as callable tools. Depends on TODO #22 (OpenAI-compat `/v1/chat/completions` endpoint). Reference: [bubblit](https://github.com/soup-oss/bubblit) for behavior-class lazy loading of tool manifests.
**(N) AMQP Cluster Nervous System** — RabbitMQ on ultron as the cluster master/hub. Topic exchange `jc.cluster`, direct exchange `jc.commands`. Worker nodes (jarvis + future nodes) self-register by connecting to the ultron broker and publishing to `node.<hostname>.health` (GPU/RPC/RAM stats) and `node.<hostname>.models` (available GGUFs). jC subscribes to all `node.*` topics — drives UI status dots, model dropdown, and resource bars. Commands flow ultron→node via `cmd.<hostname>.*` queues (e.g. model load, service restart). **Long-term vision:** a resident AI model on each node acts as the AMQP agent — consuming its command queue, building a prompt, deciding action, publishing result. The message bus becomes the nervous system for a distributed agentic cluster where intelligence lives at the edges. ultron orchestrates; worker nodes are autonomous agents. Scales to arbitrary additional nodes with no topology changes.
---
## Primary Cluster Objectives
1. **Generative AI inference** — Local, private, fast enough to be useful
2. **Agentic functionality** — Autonomous RAG self-management is the canonical first example. The system acts, not just responds.
---
## Repository
```
ssh://gitea@llgit.llamachile.tube:1319/gramps/jarvisChat.git
```
> SSH username is `gitea`, not `git`. Port 1319.
---
## License ## License
MIT MIT
## Repository
Gitea: `ssh://gitea@llgit.llamachile.tube:1319/gramps/jarvisChat.git`

10
auth.py
View File

@@ -15,10 +15,10 @@ from fastapi.responses import JSONResponse
from config import SESSION_TIMEOUT_SECONDS, MAX_PIN_ATTEMPTS, PIN_LOCKOUT_SECONDS, RATE_WINDOW_SECONDS from config import SESSION_TIMEOUT_SECONDS, MAX_PIN_ATTEMPTS, PIN_LOCKOUT_SECONDS, RATE_WINDOW_SECONDS
from db import get_db, get_setting from db import get_db, get_setting
from security import ( from security import (
SESSIONS, PIN_ATTEMPTS, SESSION_LOCK, audit_event, get_client_ip, SESSIONS, PIN_ATTEMPTS, SESSION_LOCK, BODY_LIMIT_DEFAULT_BYTES,
is_ip_allowed, check_rate_limit, rate_policy, origin_allowed, audit_event, get_client_ip, is_ip_allowed, check_rate_limit,
is_state_changing, request_body_limit, read_json_body, hash_pin, rate_policy, origin_allowed, is_state_changing, request_body_limit,
customer_error_envelope, log_incident, read_json_body, hash_pin, customer_error_envelope, log_incident,
) )
log = logging.getLogger("jarvischat") log = logging.getLogger("jarvischat")
@@ -146,7 +146,6 @@ async def auth_guest(request: Request):
@router.post("/api/auth/login") @router.post("/api/auth/login")
async def auth_login(request: Request): async def auth_login(request: Request):
from security import BODY_LIMIT_DEFAULT_BYTES
body = await read_json_body(request, BODY_LIMIT_DEFAULT_BYTES) body = await read_json_body(request, BODY_LIMIT_DEFAULT_BYTES)
pin = str(body.get("pin", "")) pin = str(body.get("pin", ""))
ip = get_client_ip(request) ip = get_client_ip(request)
@@ -183,7 +182,6 @@ async def auth_heartbeat(request: Request):
@router.post("/api/auth/logout") @router.post("/api/auth/logout")
async def auth_logout(request: Request): async def auth_logout(request: Request):
from security import BODY_LIMIT_DEFAULT_BYTES
ip = get_client_ip(request) ip = get_client_ip(request)
sid = request.headers.get("x-session-id", "").strip() sid = request.headers.get("x-session-id", "").strip()
role = "none" role = "none"

1
config.py
View File

@@ -14,6 +14,7 @@ OLLAMA_BASE = os.environ.get("OLLAMA_BASE", "http://localhost:11434")
LLAMA_SERVER_BASE = os.environ.get("LLAMA_SERVER_BASE", "http://192.168.50.108:8081") LLAMA_SERVER_BASE = os.environ.get("LLAMA_SERVER_BASE", "http://192.168.50.108:8081")
SEARXNG_BASE = "http://localhost:8888" SEARXNG_BASE = "http://localhost:8888"
DEFAULT_MODEL = "llama3.1:latest" DEFAULT_MODEL = "llama3.1:latest"
COMPLETIONS_API_KEY = os.environ.get("JARVISCHAT_COMPLETIONS_API_KEY", "jc-sk-" + os.urandom(24).hex())
# --- Auth --- # --- Auth ---
SESSION_TIMEOUT_SECONDS = 90 SESSION_TIMEOUT_SECONDS = 90

8
memory.py
View File

@@ -62,7 +62,13 @@ def search_memories(query: str, limit: int = 5) -> list:
if not words: if not words:
db.close() db.close()
return [] return []
safe_query = " OR ".join(word + "*" for word in words[:10]) escaped = []
for word in words[:10]:
if word.upper() in {"AND", "OR", "NOT", "NEAR"}:
escaped.append(f'"{word}"*')
else:
escaped.append(word + "*")
safe_query = " OR ".join(escaped)
try: try:
rows = db.execute( rows = db.execute(
"SELECT rowid, fact, topic, source, created_at, bm25(memories) AS rank " "SELECT rowid, fact, topic, source, created_at, bm25(memories) AS rank "

4
rag.py
View File

@@ -12,7 +12,7 @@ from config import MAX_SKILL_PROMPT_CHARS
log = logging.getLogger("jarvischat") log = logging.getLogger("jarvischat")
QDRANT_URL = "http://192.168.50.108:6333" QDRANT_URL = "http://192.168.50.108:6333"
EMBED_URL = "http://192.168.50.108:11434" EMBED_URL = "http://192.168.50.108:8081"
EMBED_MODEL = "mxbai-embed-large" EMBED_MODEL = "mxbai-embed-large"
RAG_COLLECTION = "jarvis_rag" RAG_COLLECTION = "jarvis_rag"
RAG_SCORE_THRESHOLD = 0.25 RAG_SCORE_THRESHOLD = 0.25
@@ -65,7 +65,7 @@ async def build_system_prompt(db, extra_prompt: str = "", user_message: str = ""
rag_lines = [r["payload"]["text"] for r in rag_results if r["score"] > RAG_SCORE_THRESHOLD] rag_lines = [r["payload"]["text"] for r in rag_results if r["score"] > RAG_SCORE_THRESHOLD]
if rag_lines: if rag_lines:
parts.append("## Retrieved Context\n" + "\n\n---\n\n".join(rag_lines)) parts.append("## Retrieved Context\n" + "\n\n---\n\n".join(rag_lines))
log.warning(f"RAG injected {len(rag_lines)} chunks into context") log.info(f"RAG injected {len(rag_lines)} chunks into context")
except Exception as e: except Exception as e:
log.warning(f"RAG injection error: {e}") log.warning(f"RAG injection error: {e}")

355
readme.md
View File

@@ -1,355 +0,0 @@
# ⚡ JarvisChat v1.7.8
![screenshot](docs/images/screenshot.png)
**A lightweight Ollama coding companion with persistent memory, web search, and real-time system monitoring.**
Built with FastAPI + SQLite + Jinja2. Runs on Python 3.13. No Docker required.
Developer wiki: [docs/wiki/Home.md](docs/wiki/Home.md)
Core architecture deep-dive: [docs/wiki/Developer-Architecture.md](docs/wiki/Developer-Architecture.md)
## Security Scope Disclaimer
JarvisChat is designed for local and home-lab use (same host or trusted LAN).
JarvisChat may technically work with frontier or commercial AI endpoints, but the author does not recommend or support that usage.
Supported deployments are contained local/home-lab environments.
By default, API access is limited to loopback + private LAN CIDRs. You can override with `JARVISCHAT_ALLOWED_CIDRS` (comma-separated CIDRs) and optionally trust reverse-proxy forwarding with `JARVISCHAT_TRUST_X_FORWARDED_FOR=true`.
If you deploy outside a trusted local subnet, your risk profile changes significantly and the default protections here may be insufficient.
Use at your own risk. No warranty is provided for Internet-exposed deployments.
## What's New in v1.7.x
- **Security hardening suite completed** - request rate limits, payload caps, settings allowlist, safe error envelopes, and LAN CIDR gate controls
- **Customer-safe incident handling** - client-facing errors include support-friendly incident keys while full traces remain in server logs
- **Streaming and regression test expansion** - automated coverage for SSE chat/search paths, memory remember/forget command handling, and auth/guardrail behavior
- **Skills framework (Phase 1)** - built-in local skill registry with per-skill enable controls, API endpoints, and bounded prompt injection
- **Skills WebUX controls** - Settings modal now includes a master skills toggle and per-skill toggles for admin users
## What's New in v1.6.x
- **Guest/admin capability split** - guest chat by default with 4-digit admin PIN for advanced or destructive operations
- **Session + lockout controls** - session lifecycle endpoints, heartbeat, logout/revoke behavior, failed PIN lockout protections, and auth audit events
- **Browser request protections** - strict origin checks for state-changing requests and admin-only write enforcement
- **Unsafe link protection** - outbound search links sanitized to allow only http/https absolute URLs
- **Operational stability fixes** - safer first-boot PIN policy handling and memory-search tokenization fix for punctuation/FTS edge cases
## What's New in v1.5.0
- **Explicit Web Search Button** — 🔍 button next to SEND forces a web search, bypassing model uncertainty detection
- **Orange Search Styling** — Search results, WEB badge, and search button share consistent orange color scheme
- **Expanded Refusal Patterns** — Added "As an AI model", "based on my training data", "I don't have the capability"
- **Code cleanup** — Removed unused `JSONResponse` import and dead `raw_results_md` variable
- **Bug fixes** — Replaced bare `except` clauses with `except Exception`; corrected `add_memory()` return type to `int | None`; updated `TemplateResponse` call to Starlette's current API signature
## What's New in v1.4.0
- **FTS5 Memory System**: Say "remember that..." to store facts — they're automatically retrieved by relevance and injected into context
- **Forget Command**: Say "forget about..." to remove memories
- **Memory Toggle**: Enable/disable memory injection from topbar or settings
- **Multi-file Structure**: Backend and frontend separated for easier maintenance
## Features
- **Persistent Memory** — SQLite FTS5 full-text search for fast, relevant memory retrieval
- **Web Search** — SearXNG integration for automatic web lookups when the model is uncertain
- **Explicit Search** — 🔍 button to force web search without waiting for model uncertainty
- **Profile Injection** — Custom system prompt injected into every conversation
- **System Presets** — Save and switch between different system prompts
- **Real-time Stats** — CPU, RAM, GPU, VRAM monitoring in sidebar
- **Token Thermometer** — Visual context window usage indicator
- **Streaming Responses** — Server-sent events for real-time token display
- **Conversation History** — SQLite-backed chat persistence with mass-delete option
- **Model Switching** — Change Ollama models on the fly
## Current WiP (Prioritized)
Canonical backlog: [docs/wiki/current-wip.md](docs/wiki/current-wip.md)
Scope boundary: local-first (same-host Ollama), optional RFC1918 LAN endpoints, no public Internet AI endpoints by default.
Total identified items: 27
Top 10 (brief):
1. P0 [DONE]: Add auth for write/admin endpoints
2. P0 [DONE]: Add CSRF/origin protection for state-changing requests
3. P0 [DONE]: Block unsafe URL schemes in rendered links
4. P0 [DONE]: Add rate limiting and request size limits
5. P1 [DONE]: Restrict `/api/settings` updates to allowlisted keys
6. P1: Add pagination + hard caps for list APIs
7. P1 [DONE]: Replace raw exception leakage with safe client errors
8. P1 [DONE]: Add automated tests for streaming/search/memory paths
9. P2 [DONE]: Implement MCP-style skills/tool-call framework
10. P2: Implement heartbeat/check-in scheduler + summary endpoint
Item 1 executive summary: keep guest mode for conversational chat, require 4-digit admin PIN for advanced/destructive actions, and enforce local/LAN-only backend policy by default.
Implementation status: complete (guest session by default + admin unlock + admin-only write enforcement + origin checks + safe-link sanitization + audit logging + rate/payload guardrails + capability tests).
## TODO
1. ~~Verify SearXNG and Docker services persist across reboots~~
2. Conversation search/filter by keyword
3. Export conversation to markdown/text
4. Keyboard shortcuts (Ctrl+N new chat, Ctrl+Enter send)
5. Retry button on assistant messages
6. Source links — clickable links when search used
7. Allow conversation renaming
8. Multiple profiles — coding/sysadmin/general
9. Auto-generate conversation tags (client-side KWIC, top 5, filterable badges)
10. Image input support — pull vision model, file input/drag-drop, base64 encode, pass `images` array to Ollama `/api/chat`
11. Split-screen option for btop display
12. Skills as markdown files — `/opt/jarvischat/skills/`, YAML frontmatter + instructions, injected into context for tool calls
13. Heartbeats / proactive check-ins — cron + endpoint for daily briefings, HA anomaly alerts
14. Model info button — (i) icon next to Model dropdown, shows div with model description, last updated date, best-use purpose
15. Set default model — toggle any model as the default selection
16. Hide/remove model from list — exclude models from dropdown
17. Update model function — trigger `ollama pull` for selected model from UI
18. Add mouseover tooltip to SEND button
19. Add preflight validation for required model/preset selection and show a clear warning before send to prevent avoidable timeout loops
## File Structure
```
/opt/jarvischat/
├── app.py # FastAPI backend
├── jarvischat.db # SQLite database (auto-created)
├── static/
│ └── logo.png # Logo image (optional)
└── templates/
└── index.html # Frontend
```
## Requirements
- Python 3.11+ (tested on 3.13)
- Ollama running locally or on network
- SearXNG (optional, for web search)
## Installation
### Fresh Install
```bash
# Create directory and venv
sudo mkdir -p /opt/jarvischat
sudo chown $USER:$USER /opt/jarvischat
cd /opt/jarvischat
python3 -m venv venv
# Install dependencies
./venv/bin/pip install fastapi uvicorn httpx psutil jinja2 python-multipart
# Set admin PIN before first startup (4 digits)
export JARVISCHAT_ADMIN_PIN=4827
# Create subdirectories
mkdir -p templates static
# Copy files
# (copy app.py to /opt/jarvischat/)
# (copy index.html to /opt/jarvischat/templates/)
# (copy logo.png to /opt/jarvischat/static/ — optional)
```
WARNING: Do not use `1234` as your admin PIN unless you accept weak local security.
NOTE: First boot now requires `JARVISCHAT_ADMIN_PIN` unless you explicitly opt into insecure fallback with `JARVISCHAT_ALLOW_DEFAULT_PIN=true`.
### Upgrading from v1.4.x
```bash
cd /opt/jarvischat
# Backup
cp app.py app.py.bak
cp templates/index.html templates/index.html.bak
# Copy new files
# (copy app.py, replacing old version)
# (copy index.html to templates/)
# Restart
sudo systemctl restart jarvischat
```
## Systemd Service
Create `/etc/systemd/system/jarvischat.service`:
```ini
[Unit]
Description=JarvisChat - Local Ollama Web Interface
After=network.target
[Service]
Type=simple
User=jarvischat
Group=jarvischat
WorkingDirectory=/opt/jarvischat
ExecStart=/opt/jarvischat/venv/bin/uvicorn app:app --host 0.0.0.0 --port 8080
Restart=always
RestartSec=5
[Install]
WantedBy=multi-user.target
```
```bash
sudo systemctl daemon-reload
sudo systemctl enable jarvischat
sudo systemctl start jarvischat
```
## Memory Commands
In chat, natural language triggers memory operations:
| You say | What happens |
|---------|--------------|
| "remember that I prefer Rust over Go" | Stores as `preference` |
| "remember that JarvisChat runs on port 8080" | Stores as `infrastructure` |
| "note that the deadline is Friday" | Stores as `general` |
| "forget about the deadline" | Removes matching memories |
Memories are automatically searched based on your message content and injected into the system prompt when relevant.
### Memory Topics
Memories are auto-categorized:
- `preference` — likes, dislikes, choices
- `project` — active work, repos, tasks
- `infrastructure` — servers, services, configs
- `personal` — name, location, background
- `general` — everything else
## API Endpoints
### Memory
| Method | Endpoint | Description |
|--------|----------|-------------|
| GET | `/api/memories` | List all memories |
| POST | `/api/memories` | Add memory `{"fact": "...", "topic": "general"}` |
| DELETE | `/api/memories/{rowid}` | Delete memory by ID |
| GET | `/api/memories/search?q=term` | Search memories |
| GET | `/api/memories/stats` | Get counts by topic |
### Chat & Models
| Method | Endpoint | Description |
|--------|----------|-------------|
| GET | `/api/models` | List available Ollama models |
| POST | `/api/chat` | Send message (streaming SSE) |
| POST | `/api/search` | Explicit web search (streaming SSE) |
| POST | `/api/show` | Get model info (context size) |
| GET | `/api/ps` | Get running models |
### Settings & Profile
| Method | Endpoint | Description |
|--------|----------|-------------|
| GET | `/api/profile` | Get profile content |
| PUT | `/api/profile` | Update profile |
| GET | `/api/profile/default` | Get default profile |
| GET | `/api/settings` | Get settings |
| PUT | `/api/settings` | Update settings |
### Conversations
| Method | Endpoint | Description |
|--------|----------|-------------|
| GET | `/api/conversations` | List conversations |
| GET | `/api/conversations/{id}` | Get conversation with messages |
| DELETE | `/api/conversations/{id}` | Delete conversation |
| DELETE | `/api/conversations` | Delete ALL conversations |
### Presets
| Method | Endpoint | Description |
|--------|----------|-------------|
| GET | `/api/presets` | List presets |
| POST | `/api/presets` | Create preset |
| PUT | `/api/presets/{id}` | Update preset |
| DELETE | `/api/presets/{id}` | Delete preset |
### System
| Method | Endpoint | Description |
|--------|----------|-------------|
| GET | `/api/stats` | CPU, RAM, GPU, VRAM stats |
| GET | `/api/search/status` | SearXNG availability |
## Configuration
Settings are stored in the `settings` table and include:
- `profile_enabled` — Inject profile into chats (true/false)
- `search_enabled` — Auto web search (true/false)
- `memory_enabled` — Memory injection (true/false)
- `default_model` — Default Ollama model
- `searxng_url` — SearXNG instance URL (default: `http://localhost:8888`)
## Testing Memory
```bash
# Add a memory via API
curl -X POST http://jarvis:8080/api/memories \
-H "Content-Type: application/json" \
-d '{"fact": "User prefers native installs over Docker", "topic": "preference"}'
# Search memories
curl "http://jarvis:8080/api/memories/search?q=docker"
# List all memories
curl http://jarvis:8080/api/memories
# Get stats
curl http://jarvis:8080/api/memories/stats
```
Or in chat:
1. Say "remember that I hate YAML"
2. Later ask "what markup languages should I avoid?"
3. JarvisChat will inject the YAML preference into context
## Troubleshooting
### Service won't start
Check logs:
```bash
journalctl -u jarvischat -n 50 --no-pager
```
Common issues:
- Missing `jinja2`: `./venv/bin/pip install jinja2`
- Missing `templates/` directory
- Wrong permissions on `/opt/jarvischat`
### Memory not working
1. Check memory is enabled (🧠 MEM ON in topbar)
2. Verify memories exist: `curl http://jarvis:8080/api/memories`
3. Check FTS5 table: `sqlite3 jarvischat.db "SELECT * FROM memories_fts;"`
### Web search not working
1. Verify SearXNG is running: `curl http://localhost:8888/search?q=test&format=json`
2. Check search status: `curl http://jarvis:8080/api/search/status`
3. Ensure JSON format is enabled in SearXNG settings
## License
MIT
## Repository
Gitea: `ssh://gitea@llgit.llamachile.tube:1319/gramps/jarvisChat.git`

29
routers/chat.py
View File

@@ -26,7 +26,7 @@ def parse_llama_stream_chunk(line: str) -> tuple:
if line.startswith("data: "): if line.startswith("data: "):
line = line[6:] line = line[6:]
if line.strip() == "[DONE]": if line.strip() == "[DONE]":
return None, True, {} return None, True, {}, []
try: try:
chunk = json.loads(line) chunk = json.loads(line)
choices = chunk.get("choices", []) choices = chunk.get("choices", [])
@@ -35,10 +35,17 @@ def parse_llama_stream_chunk(line: str) -> tuple:
token = delta.get("content") token = delta.get("content")
finish = choices[0].get("finish_reason") finish = choices[0].get("finish_reason")
stats = {} stats = {}
logprobs_list = []
logprobs_info = choices[0].get("logprobs")
if logprobs_info:
content_logprobs = logprobs_info.get("content", [])
for entry in content_logprobs:
if "logprob" in entry:
logprobs_list.append({"logprob": entry["logprob"]})
if finish == "stop": if finish == "stop":
usage = chunk.get("usage", {}) usage = chunk.get("usage", {})
stats["tokens_per_sec"] = usage.get("tokens_per_second", 0.0) stats["tokens_per_sec"] = usage.get("tokens_per_second", 0.0)
return token, finish == "stop", stats return token, finish == "stop", stats, logprobs_list
if "message" in chunk and "content" in chunk["message"]: if "message" in chunk and "content" in chunk["message"]:
token = chunk["message"]["content"] token = chunk["message"]["content"]
done = chunk.get("done", False) done = chunk.get("done", False)
@@ -47,10 +54,10 @@ def parse_llama_stream_chunk(line: str) -> tuple:
eval_count = chunk.get("eval_count", 0) eval_count = chunk.get("eval_count", 0)
eval_duration = chunk.get("eval_duration", 0) eval_duration = chunk.get("eval_duration", 0)
stats["tokens_per_sec"] = (eval_count / (eval_duration / 1e9)) if eval_duration > 0 else 0 stats["tokens_per_sec"] = (eval_count / (eval_duration / 1e9)) if eval_duration > 0 else 0
return token, done, stats return token, done, stats, []
except json.JSONDecodeError: except json.JSONDecodeError:
pass pass
return None, False, {} return None, False, {}, []
@router.post("/api/chat") @router.post("/api/chat")
@@ -97,7 +104,7 @@ async def chat(request: Request):
for row in history_rows: for row in history_rows:
messages.append({"role": row["role"], "content": row["content"]}) messages.append({"role": row["role"], "content": row["content"]})
ollama_payload = {"model": model, "messages": messages, "stream": True} upstream_payload = {"model": model, "messages": messages, "stream": True, "logprobs": True}
async def stream_response(): async def stream_response():
full_response = [] full_response = []
@@ -111,12 +118,14 @@ async def chat(request: Request):
try: try:
async with client.stream( async with client.stream(
"POST", f"{LLAMA_SERVER_BASE}/v1/chat/completions", "POST", f"{LLAMA_SERVER_BASE}/v1/chat/completions",
json=ollama_payload, json=upstream_payload,
timeout=httpx.Timeout(300.0, connect=10.0), timeout=httpx.Timeout(300.0, connect=10.0),
) as resp: ) as resp:
async for line in resp.aiter_lines(): async for line in resp.aiter_lines():
if line.strip(): if line.strip():
token, done, stats = parse_llama_stream_chunk(line) token, done, stats, chunk_logprobs = parse_llama_stream_chunk(line)
if chunk_logprobs:
all_logprobs.extend(chunk_logprobs)
if token: if token:
full_response.append(token) full_response.append(token)
yield f"data: {json.dumps({'token': token, 'conversation_id': conv_id})}\n\n" yield f"data: {json.dumps({'token': token, 'conversation_id': conv_id})}\n\n"
@@ -153,7 +162,7 @@ async def chat(request: Request):
) as resp2: ) as resp2:
async for line in resp2.aiter_lines(): async for line in resp2.aiter_lines():
if line.strip(): if line.strip():
token2, done2, _ = parse_llama_stream_chunk(line) token2, done2, _, _ = parse_llama_stream_chunk(line)
if token2: if token2:
augmented_response.append(token2) augmented_response.append(token2)
if done2: if done2:
@@ -194,9 +203,9 @@ async def chat(request: Request):
except httpx.RemoteProtocolError: except httpx.RemoteProtocolError:
pass pass
except httpx.ConnectError: except httpx.ConnectError:
yield f"data: {json.dumps({'error': 'Cannot connect to Ollama. Is it running?'})}\n\n" yield f"data: {json.dumps({'error': 'Cannot connect to inference server. Is it running?'})}\n\n"
except Exception as e: except Exception as e:
incident_key = log_incident("chat_stream", message="Ollama stream failure during chat response", incident_key = log_incident("chat_stream", message="Inference stream failure during chat response",
request=request, exc=e) request=request, exc=e)
yield f"data: {json.dumps({'error': 'Chat response generation failed before completion. Use the incident key for support lookup.', 'error_key': incident_key})}\n\n" yield f"data: {json.dumps({'error': 'Chat response generation failed before completion. Use the incident key for support lookup.', 'error_key': incident_key})}\n\n"

4
routers/completions.py
View File

@@ -178,7 +178,7 @@ async def _stream_chat(payload: dict, model: str, conv_id: str, request: Request
async for line in resp.aiter_lines(): async for line in resp.aiter_lines():
if not line.strip(): if not line.strip():
continue continue
token, done, _ = parse_llama_stream_chunk(line) token, done, _, _ = parse_llama_stream_chunk(line)
if token: if token:
full_response.append(token) full_response.append(token)
yield _build_openai_chunk(token, model, conv_id) yield _build_openai_chunk(token, model, conv_id)
@@ -222,7 +222,7 @@ async def _blocking_chat(payload: dict, model: str, conv_id: str, request: Reque
async for line in resp.aiter_lines(): async for line in resp.aiter_lines():
if not line.strip(): if not line.strip():
continue continue
token, done, _ = parse_llama_stream_chunk(line) token, done, _, _ = parse_llama_stream_chunk(line)
if token: if token:
full_response.append(token) full_response.append(token)
if done: if done:

15
routers/models.py
View File

@@ -8,7 +8,7 @@ import httpx
import psutil import psutil
from fastapi import APIRouter, HTTPException, Request from fastapi import APIRouter, HTTPException, Request
from config import OLLAMA_BASE from config import LLAMA_SERVER_BASE
from gpu import get_gpu_stats from gpu import get_gpu_stats
from security import read_json_body, BODY_LIMIT_DEFAULT_BYTES from security import read_json_body, BODY_LIMIT_DEFAULT_BYTES
@@ -20,34 +20,33 @@ router = APIRouter()
async def list_models(): async def list_models():
async with httpx.AsyncClient() as client: async with httpx.AsyncClient() as client:
try: try:
resp = await client.get(f"{OLLAMA_BASE}/v1/models", timeout=10) resp = await client.get(f"{LLAMA_SERVER_BASE}/v1/models", timeout=10)
data = resp.json() data = resp.json()
models = [{"name": m["id"], "model": m["id"]} for m in data.get("data", [])] models = [{"name": m["id"], "model": m["id"]} for m in data.get("data", [])]
return {"models": models} return {"models": models}
except httpx.ConnectError: except httpx.ConnectError:
raise HTTPException(status_code=502, detail="Cannot connect to llama-server.") raise HTTPException(status_code=502, detail="Cannot connect to inference server.")
@router.get("/api/ps") @router.get("/api/ps")
async def running_models(): async def running_models():
async with httpx.AsyncClient() as client: async with httpx.AsyncClient() as client:
try: try:
resp = await client.get(f"{OLLAMA_BASE}/api/ps", timeout=10) resp = await client.get(f"{LLAMA_SERVER_BASE}/v1/models", timeout=10)
return resp.json() return resp.json()
except httpx.ConnectError: except httpx.ConnectError:
raise HTTPException(status_code=502, detail="Cannot connect to Ollama.") raise HTTPException(status_code=502, detail="Cannot connect to inference server.")
@router.post("/api/show") @router.post("/api/show")
async def show_model(request: Request): async def show_model(request: Request):
from security import BODY_LIMIT_DEFAULT_BYTES
body = await read_json_body(request, BODY_LIMIT_DEFAULT_BYTES) body = await read_json_body(request, BODY_LIMIT_DEFAULT_BYTES)
async with httpx.AsyncClient() as client: async with httpx.AsyncClient() as client:
try: try:
resp = await client.post(f"{OLLAMA_BASE}/api/show", json=body, timeout=10) resp = await client.post(f"{LLAMA_SERVER_BASE}/api/show", json=body, timeout=10)
return resp.json() return resp.json()
except httpx.ConnectError: except httpx.ConnectError:
raise HTTPException(status_code=502, detail="Cannot connect to Ollama.") raise HTTPException(status_code=502, detail="Cannot connect to inference server.")
@router.get("/api/stats") @router.get("/api/stats")

7
routers/search_route.py
View File

@@ -35,14 +35,14 @@ async def explicit_search(request: Request):
if not conv_id: if not conv_id:
conv_id = str(uuid.uuid4()) conv_id = str(uuid.uuid4())
title = f"🔍 {query[:70]}..." if len(query) > 70 else f"🔍 {query}" title = query[:70] + "..." if len(query) > 70 else query
db.execute("INSERT INTO conversations (id, title, model, created_at, updated_at) VALUES (?, ?, ?, ?, ?)", db.execute("INSERT INTO conversations (id, title, model, created_at, updated_at) VALUES (?, ?, ?, ?, ?)",
(conv_id, title, model, now, now)) (conv_id, title, model, now, now))
else: else:
db.execute("UPDATE conversations SET updated_at = ? WHERE id = ?", (now, conv_id)) db.execute("UPDATE conversations SET updated_at = ? WHERE id = ?", (now, conv_id))
db.execute("INSERT INTO messages (conversation_id, role, content, created_at) VALUES (?, ?, ?, ?)", db.execute("INSERT INTO messages (conversation_id, role, content, created_at) VALUES (?, ?, ?, ?)",
(conv_id, "user", f"🔍 {query}", now)) (conv_id, "user", query, now))
db.commit() db.commit()
db.close() db.close()
@@ -80,7 +80,7 @@ async def explicit_search(request: Request):
) as resp: ) as resp:
async for line in resp.aiter_lines(): async for line in resp.aiter_lines():
if line.strip(): if line.strip():
token, done, _ = parse_llama_stream_chunk(line) token, done, _, _ = parse_llama_stream_chunk(line)
if token: if token:
full_response.append(token) full_response.append(token)
yield f"data: {json.dumps({'token': token, 'conversation_id': conv_id})}\n\n" yield f"data: {json.dumps({'token': token, 'conversation_id': conv_id})}\n\n"
@@ -102,7 +102,6 @@ async def explicit_search(request: Request):
db2.commit() db2.commit()
db2.close() db2.close()
yield f"data: {json.dumps({'raw_results': results, 'conversation_id': conv_id})}\n\n"
yield f"data: {json.dumps({'done': True, 'conversation_id': conv_id, 'searched': True})}\n\n" yield f"data: {json.dumps({'done': True, 'conversation_id': conv_id, 'searched': True})}\n\n"
return StreamingResponse(stream_search(), media_type="text/event-stream") return StreamingResponse(stream_search(), media_type="text/event-stream")

17
search.py
View File

@@ -80,16 +80,13 @@ def format_direct_answer(question: str, results: list) -> str:
def extract_search_query(user_message: str) -> str: def extract_search_query(user_message: str) -> str:
query = user_message.strip() query = user_message.strip()
if re.search(r"temperature|weather", query, re.IGNORECASE): weather_lead = re.match(r"^(?:what('?s| is) the\s+)?(?:weather|temperature|forecast)\s+(?:in\s+|for\s+)?(.+)", query, re.IGNORECASE)
query = re.sub(r"^what('?s| is) the ", "", query, flags=re.IGNORECASE) + " right now degrees" if weather_lead:
if re.search(r"price|spot price", query, re.IGNORECASE): return (weather_lead.group(2) + " weather").strip()[:100]
query = re.sub(r"^(what('?s| is)|can you tell me) the ", "", query, flags=re.IGNORECASE) + " today USD" price_lead = re.match(r"^(?:what('?s| is| are)\s+)?(?:the\s+)?(?:price|spot price)\s+(?:of\s+|for\s+)?(.+)", query, re.IGNORECASE)
query = re.sub( if price_lead:
r"^(what|who|where|when|why|how|is|are|can|could|would|should|do|does|did)\s+", return (price_lead.group(2) + " price today USD").strip()[:100]
"", query, flags=re.IGNORECASE, return query[:100]
)
query = re.sub(r"[?!.]+$", "", query)
return query[:100].strip() or user_message[:100]
async def query_searxng(query: str, max_results: int = 5) -> list: async def query_searxng(query: str, max_results: int = 5) -> list:

14
tests/test_auth_capabilities.py
View File

@@ -3,16 +3,18 @@ from pathlib import Path
from fastapi.testclient import TestClient from fastapi.testclient import TestClient
import app as app_module import app
import db
from security import SESSIONS, PIN_ATTEMPTS
def make_client(tmp_path: Path) -> TestClient: def make_client(tmp_path: Path) -> TestClient:
os.environ["JARVISCHAT_ADMIN_PIN"] = "1234" os.environ["JARVISCHAT_ADMIN_PIN"] = "1234"
app_module.DB_PATH = tmp_path / "jarvischat-test.db" db.DB_PATH = tmp_path / "jarvischat-test.db"
app_module.SESSIONS.clear() SESSIONS.clear()
app_module.PIN_ATTEMPTS.clear() PIN_ATTEMPTS.clear()
app_module.init_db() db.init_db()
return TestClient(app_module.app) return TestClient(app.app)
def test_guest_read_only_admin_write_blocked(tmp_path: Path): def test_guest_read_only_admin_write_blocked(tmp_path: Path):

37
tests/test_chat_streaming_and_memory_paths.py
View File

@@ -2,19 +2,24 @@ import json
import os import os
from pathlib import Path from pathlib import Path
import httpx
from fastapi.testclient import TestClient from fastapi.testclient import TestClient
import app as app_module import app
import config
import db
import routers.chat
from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS
def make_client(tmp_path: Path) -> TestClient: def make_client(tmp_path: Path) -> TestClient:
os.environ["JARVISCHAT_ADMIN_PIN"] = "1234" os.environ["JARVISCHAT_ADMIN_PIN"] = "1234"
app_module.DB_PATH = tmp_path / "jarvischat-streaming.db" db.DB_PATH = tmp_path / "jarvischat-streaming.db"
app_module.SESSIONS.clear() SESSIONS.clear()
app_module.PIN_ATTEMPTS.clear() PIN_ATTEMPTS.clear()
app_module.RATE_EVENTS.clear() RATE_EVENTS.clear()
app_module.init_db() db.init_db()
return TestClient(app_module.app, raise_server_exceptions=False) return TestClient(app.app, raise_server_exceptions=False)
def parse_sse_payloads(body: str) -> list[dict]: def parse_sse_payloads(body: str) -> list[dict]:
@@ -65,11 +70,11 @@ def test_chat_stream_emits_tokens_and_done(tmp_path: Path, monkeypatch):
def stream_stub(self, method, url, json=None, timeout=None): def stream_stub(self, method, url, json=None, timeout=None):
return _MockStreamResponse(events) return _MockStreamResponse(events)
monkeypatch.setattr(app_module.httpx.AsyncClient, "stream", stream_stub) monkeypatch.setattr(httpx.AsyncClient, "stream", stream_stub)
resp = client.post( resp = client.post(
"/api/chat", "/api/chat",
json={"message": "hello", "model": app_module.DEFAULT_MODEL}, json={"message": "hello", "model": config.DEFAULT_MODEL},
headers=headers, headers=headers,
) )
assert resp.status_code == 200 assert resp.status_code == 200
@@ -92,7 +97,7 @@ def test_chat_auto_search_trigger_emits_search_events(tmp_path: Path, monkeypatc
first_stream = _stream_json_lines( first_stream = _stream_json_lines(
[ [
{ {
"message": {"content": "I am uncertain."}, "message": {"content": "I don't have current data on that question."},
"logprobs": [{"logprob": -5.0}], "logprobs": [{"logprob": -5.0}],
}, },
{"done": True, "eval_count": 2, "eval_duration": 1000000000}, {"done": True, "eval_count": 2, "eval_duration": 1000000000},
@@ -118,12 +123,12 @@ def test_chat_auto_search_trigger_emits_search_events(tmp_path: Path, monkeypatc
} }
] ]
monkeypatch.setattr(app_module.httpx.AsyncClient, "stream", stream_stub) monkeypatch.setattr(httpx.AsyncClient, "stream", stream_stub)
monkeypatch.setattr(app_module, "query_searxng", search_stub) monkeypatch.setattr(routers.chat, "query_searxng", search_stub)
resp = client.post( resp = client.post(
"/api/chat", "/api/chat",
json={"message": "what is the latest value", "model": app_module.DEFAULT_MODEL}, json={"message": "what is the latest value", "model": config.DEFAULT_MODEL},
headers=headers, headers=headers,
) )
assert resp.status_code == 200 assert resp.status_code == 200
@@ -153,13 +158,13 @@ def test_memory_command_paths_remember_and_forget(tmp_path: Path, monkeypatch):
def stream_stub(self, method, url, json=None, timeout=None): def stream_stub(self, method, url, json=None, timeout=None):
return _MockStreamResponse(base_stream) return _MockStreamResponse(base_stream)
monkeypatch.setattr(app_module.httpx.AsyncClient, "stream", stream_stub) monkeypatch.setattr(httpx.AsyncClient, "stream", stream_stub)
remember_resp = client.post( remember_resp = client.post(
"/api/chat", "/api/chat",
json={ json={
"message": "remember that my favorite language is rust", "message": "remember that my favorite language is rust",
"model": app_module.DEFAULT_MODEL, "model": config.DEFAULT_MODEL,
}, },
headers=headers, headers=headers,
) )
@@ -175,7 +180,7 @@ def test_memory_command_paths_remember_and_forget(tmp_path: Path, monkeypatch):
"/api/chat", "/api/chat",
json={ json={
"message": "forget about my favorite language", "message": "forget about my favorite language",
"model": app_module.DEFAULT_MODEL, "model": config.DEFAULT_MODEL,
}, },
headers=headers, headers=headers,
) )

25
tests/test_error_envelopes.py
View File

@@ -1,19 +1,24 @@
import os import os
from pathlib import Path from pathlib import Path
import httpx
from fastapi.testclient import TestClient from fastapi.testclient import TestClient
import app as app_module import app
import config
import db
import routers.memories
from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS
def make_client(tmp_path: Path) -> TestClient: def make_client(tmp_path: Path) -> TestClient:
os.environ["JARVISCHAT_ADMIN_PIN"] = "1234" os.environ["JARVISCHAT_ADMIN_PIN"] = "1234"
app_module.DB_PATH = tmp_path / "jarvischat-errors.db" db.DB_PATH = tmp_path / "jarvischat-errors.db"
app_module.SESSIONS.clear() SESSIONS.clear()
app_module.PIN_ATTEMPTS.clear() PIN_ATTEMPTS.clear()
app_module.RATE_EVENTS.clear() RATE_EVENTS.clear()
app_module.init_db() db.init_db()
return TestClient(app_module.app, raise_server_exceptions=False) return TestClient(app.app, raise_server_exceptions=False)
def test_unhandled_api_exception_returns_friendly_error_with_incident_key( def test_unhandled_api_exception_returns_friendly_error_with_incident_key(
@@ -28,7 +33,7 @@ def test_unhandled_api_exception_returns_friendly_error_with_incident_key(
def boom(_topic=None): def boom(_topic=None):
raise RuntimeError("super secret db internals") raise RuntimeError("super secret db internals")
monkeypatch.setattr(app_module, "get_all_memories", boom) monkeypatch.setattr(routers.memories, "get_all_memories", boom)
resp = client.get("/api/memories", headers=headers) resp = client.get("/api/memories", headers=headers)
assert resp.status_code == 500 assert resp.status_code == 500
@@ -57,11 +62,11 @@ def test_chat_stream_error_hides_internal_exception_and_emits_incident_key(
def broken_stream(*args, **kwargs): def broken_stream(*args, **kwargs):
return BrokenStreamContext() return BrokenStreamContext()
monkeypatch.setattr(app_module.httpx.AsyncClient, "stream", broken_stream) monkeypatch.setattr(httpx.AsyncClient, "stream", broken_stream)
resp = client.post( resp = client.post(
"/api/chat", "/api/chat",
json={"message": "hello", "model": app_module.DEFAULT_MODEL}, json={"message": "hello", "model": config.DEFAULT_MODEL},
headers=headers, headers=headers,
) )

52
tests/test_ip_allowlist.py
View File

@@ -3,48 +3,42 @@ from pathlib import Path
from fastapi.testclient import TestClient from fastapi.testclient import TestClient
import app as app_module import app
import db
from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS, is_ip_allowed
def make_client(tmp_path: Path) -> TestClient: def make_client(tmp_path: Path) -> TestClient:
os.environ["JARVISCHAT_ADMIN_PIN"] = "1234" os.environ["JARVISCHAT_ADMIN_PIN"] = "1234"
app_module.DB_PATH = tmp_path / "jarvischat-ip.db" db.DB_PATH = tmp_path / "jarvischat-ip.db"
app_module.SESSIONS.clear() SESSIONS.clear()
app_module.PIN_ATTEMPTS.clear() PIN_ATTEMPTS.clear()
app_module.RATE_EVENTS.clear() RATE_EVENTS.clear()
app_module.init_db() db.init_db()
return TestClient(app_module.app) return TestClient(app.app)
def test_ip_helper_allows_local_defaults(): def test_ip_helper_allows_local_defaults():
assert app_module.is_ip_allowed("127.0.0.1") assert is_ip_allowed("127.0.0.1")
assert app_module.is_ip_allowed("192.168.1.10") assert is_ip_allowed("192.168.1.10")
assert app_module.is_ip_allowed("10.0.0.42") assert is_ip_allowed("10.0.0.42")
assert app_module.is_ip_allowed("172.16.1.2") assert is_ip_allowed("172.16.1.2")
assert app_module.is_ip_allowed("testclient") assert is_ip_allowed("testclient")
def test_ip_helper_blocks_public_ip(): def test_ip_helper_blocks_public_ip():
assert not app_module.is_ip_allowed("8.8.8.8") assert not is_ip_allowed("8.8.8.8")
def test_middleware_blocks_disallowed_ip(tmp_path: Path): def test_middleware_blocks_disallowed_ip(tmp_path: Path, monkeypatch):
monkeypatch.setattr(app, "get_client_ip", lambda _req: "8.8.8.8")
with make_client(tmp_path) as client: with make_client(tmp_path) as client:
original_get_client_ip = app_module.get_client_ip resp = client.post("/api/auth/guest")
try: assert resp.status_code == 403
app_module.get_client_ip = lambda _req: "8.8.8.8"
resp = client.post("/api/auth/guest")
assert resp.status_code == 403
finally:
app_module.get_client_ip = original_get_client_ip
def test_middleware_allows_local_ip(tmp_path: Path): def test_middleware_allows_local_ip(tmp_path: Path, monkeypatch):
monkeypatch.setattr(app, "get_client_ip", lambda _req: "192.168.50.109")
with make_client(tmp_path) as client: with make_client(tmp_path) as client:
original_get_client_ip = app_module.get_client_ip resp = client.post("/api/auth/guest")
try: assert resp.status_code == 200
app_module.get_client_ip = lambda _req: "192.168.50.109"
resp = client.post("/api/auth/guest")
assert resp.status_code == 200
finally:
app_module.get_client_ip = original_get_client_ip

40
tests/test_rate_and_payload_guardrails.py
View File

@@ -4,24 +4,28 @@ from pathlib import Path
from fastapi.testclient import TestClient from fastapi.testclient import TestClient
import app as app_module import app
import config
import db
import security
from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS
def make_client(tmp_path: Path) -> TestClient: def make_client(tmp_path: Path) -> TestClient:
os.environ["JARVISCHAT_ADMIN_PIN"] = "1234" os.environ["JARVISCHAT_ADMIN_PIN"] = "1234"
app_module.DB_PATH = tmp_path / "jarvischat-rate.db" db.DB_PATH = tmp_path / "jarvischat-rate.db"
app_module.SESSIONS.clear() SESSIONS.clear()
app_module.PIN_ATTEMPTS.clear() PIN_ATTEMPTS.clear()
app_module.RATE_EVENTS.clear() RATE_EVENTS.clear()
app_module.init_db() db.init_db()
return TestClient(app_module.app) return TestClient(app.app)
def test_stats_rate_limit_hits_429(tmp_path: Path): def test_stats_rate_limit_hits_429(tmp_path: Path):
old_limit = app_module.RL_STATS_PER_WINDOW old_limit = security.RL_STATS_PER_WINDOW
old_window = app_module.RATE_WINDOW_SECONDS old_window = app.RATE_WINDOW_SECONDS
app_module.RL_STATS_PER_WINDOW = 2 security.RL_STATS_PER_WINDOW = 2
app_module.RATE_WINDOW_SECONDS = 60 app.RATE_WINDOW_SECONDS = 60
try: try:
with make_client(tmp_path) as client: with make_client(tmp_path) as client:
sid = client.post("/api/auth/guest").json()["session_id"] sid = client.post("/api/auth/guest").json()["session_id"]
@@ -35,13 +39,13 @@ def test_stats_rate_limit_hits_429(tmp_path: Path):
assert r2.status_code == 200 assert r2.status_code == 200
assert r3.status_code == 429 assert r3.status_code == 429
finally: finally:
app_module.RL_STATS_PER_WINDOW = old_limit security.RL_STATS_PER_WINDOW = old_limit
app_module.RATE_WINDOW_SECONDS = old_window app.RATE_WINDOW_SECONDS = old_window
def test_large_login_payload_rejected_413(tmp_path: Path): def test_large_login_payload_rejected_413(tmp_path: Path):
with make_client(tmp_path) as client: with make_client(tmp_path) as client:
huge_pin = "1" * (app_module.BODY_LIMIT_DEFAULT_BYTES + 100) huge_pin = "1" * (config.BODY_LIMIT_DEFAULT_BYTES + 100)
resp = client.post( resp = client.post(
"/api/auth/login", "/api/auth/login",
data=json.dumps({"pin": huge_pin}), data=json.dumps({"pin": huge_pin}),
@@ -54,10 +58,10 @@ def test_chat_message_length_rejected_413(tmp_path: Path):
with make_client(tmp_path) as client: with make_client(tmp_path) as client:
sid = client.post("/api/auth/guest").json()["session_id"] sid = client.post("/api/auth/guest").json()["session_id"]
headers = {"X-Session-ID": sid, "Origin": "http://testserver"} headers = {"X-Session-ID": sid, "Origin": "http://testserver"}
message = "x" * (app_module.MAX_CHAT_MESSAGE_CHARS + 1) message = "x" * (config.MAX_CHAT_MESSAGE_CHARS + 1)
resp = client.post( resp = client.post(
"/api/chat", "/api/chat",
json={"message": message, "model": app_module.DEFAULT_MODEL}, json={"message": message, "model": config.DEFAULT_MODEL},
headers=headers, headers=headers,
) )
assert resp.status_code == 413 assert resp.status_code == 413
@@ -67,10 +71,10 @@ def test_search_query_length_rejected_413(tmp_path: Path):
with make_client(tmp_path) as client: with make_client(tmp_path) as client:
sid = client.post("/api/auth/guest").json()["session_id"] sid = client.post("/api/auth/guest").json()["session_id"]
headers = {"X-Session-ID": sid, "Origin": "http://testserver"} headers = {"X-Session-ID": sid, "Origin": "http://testserver"}
query = "q" * (app_module.MAX_SEARCH_QUERY_CHARS + 1) query = "q" * (config.MAX_SEARCH_QUERY_CHARS + 1)
resp = client.post( resp = client.post(
"/api/search", "/api/search",
json={"query": query, "model": app_module.DEFAULT_MODEL}, json={"query": query, "model": config.DEFAULT_MODEL},
headers=headers, headers=headers,
) )
assert resp.status_code == 413 assert resp.status_code == 413

16
tests/test_search_url_sanitization.py
View File

@@ -1,17 +1,17 @@
import app as app_module from search import sanitize_outbound_url
def test_sanitize_outbound_url_allows_http_https(): def test_sanitize_outbound_url_allows_http_https():
assert app_module.sanitize_outbound_url("https://example.com/path") == "https://example.com/path" assert sanitize_outbound_url("https://example.com/path") == "https://example.com/path"
assert app_module.sanitize_outbound_url("http://example.com") == "http://example.com" assert sanitize_outbound_url("http://example.com") == "http://example.com"
def test_sanitize_outbound_url_blocks_unsafe_schemes(): def test_sanitize_outbound_url_blocks_unsafe_schemes():
assert app_module.sanitize_outbound_url("javascript:alert(1)") == "" assert sanitize_outbound_url("javascript:alert(1)") == ""
assert app_module.sanitize_outbound_url("data:text/html,evil") == "" assert sanitize_outbound_url("data:text/html,evil") == ""
assert app_module.sanitize_outbound_url("file:///etc/passwd") == "" assert sanitize_outbound_url("file:///etc/passwd") == ""
def test_sanitize_outbound_url_blocks_relative_and_empty(): def test_sanitize_outbound_url_blocks_relative_and_empty():
assert app_module.sanitize_outbound_url("/relative/path") == "" assert sanitize_outbound_url("/relative/path") == ""
assert app_module.sanitize_outbound_url("") == "" assert sanitize_outbound_url("") == ""

14
tests/test_settings_allowlist.py
View File

@@ -3,17 +3,19 @@ from pathlib import Path
from fastapi.testclient import TestClient from fastapi.testclient import TestClient
import app as app_module import app
import db
from security import SESSIONS, PIN_ATTEMPTS
def make_admin_client(tmp_path: Path) -> tuple[TestClient, dict[str, str]]: def make_admin_client(tmp_path: Path) -> tuple[TestClient, dict[str, str]]:
os.environ["JARVISCHAT_ADMIN_PIN"] = "1234" os.environ["JARVISCHAT_ADMIN_PIN"] = "1234"
app_module.DB_PATH = tmp_path / "jarvischat-settings.db" db.DB_PATH = tmp_path / "jarvischat-settings.db"
app_module.SESSIONS.clear() SESSIONS.clear()
app_module.PIN_ATTEMPTS.clear() PIN_ATTEMPTS.clear()
app_module.init_db() db.init_db()
client = TestClient(app_module.app) client = TestClient(app.app)
login = client.post( login = client.post(
"/api/auth/login", "/api/auth/login",
json={"pin": "1234"}, json={"pin": "1234"},

34
tests/test_skills_framework.py
View File

@@ -1,19 +1,23 @@
import asyncio
import os import os
from pathlib import Path from pathlib import Path
from fastapi.testclient import TestClient from fastapi.testclient import TestClient
import app as app_module import app
import db
from rag import build_system_prompt
from security import SESSIONS, PIN_ATTEMPTS, RATE_EVENTS
def make_client(tmp_path: Path) -> TestClient: def make_client(tmp_path: Path) -> TestClient:
os.environ["JARVISCHAT_ADMIN_PIN"] = "1234" os.environ["JARVISCHAT_ADMIN_PIN"] = "1234"
app_module.DB_PATH = tmp_path / "jarvischat-skills.db" db.DB_PATH = tmp_path / "jarvischat-skills.db"
app_module.SESSIONS.clear() SESSIONS.clear()
app_module.PIN_ATTEMPTS.clear() PIN_ATTEMPTS.clear()
app_module.RATE_EVENTS.clear() RATE_EVENTS.clear()
app_module.init_db() db.init_db()
return TestClient(app_module.app, raise_server_exceptions=False) return TestClient(app.app, raise_server_exceptions=False)
def test_guest_can_list_skills(tmp_path: Path): def test_guest_can_list_skills(tmp_path: Path):
@@ -71,23 +75,23 @@ def test_unknown_skill_update_is_rejected(tmp_path: Path):
def test_prompt_injection_respects_skills_enabled_setting(tmp_path: Path): def test_prompt_injection_respects_skills_enabled_setting(tmp_path: Path):
with make_client(tmp_path): with make_client(tmp_path):
db = app_module.get_db() conn = db.get_db()
try: try:
db.execute( conn.execute(
"INSERT OR REPLACE INTO settings (key, value) VALUES (?, ?)", "INSERT OR REPLACE INTO settings (key, value) VALUES (?, ?)",
("skills_enabled", "false"), ("skills_enabled", "false"),
) )
db.commit() conn.commit()
without_skills = app_module.build_system_prompt(db, "", "hello") without_skills = asyncio.run(build_system_prompt(conn, "", "hello"))
assert "## Active Skills" not in without_skills assert "## Active Skills" not in without_skills
db.execute( conn.execute(
"INSERT OR REPLACE INTO settings (key, value) VALUES (?, ?)", "INSERT OR REPLACE INTO settings (key, value) VALUES (?, ?)",
("skills_enabled", "true"), ("skills_enabled", "true"),
) )
db.commit() conn.commit()
with_skills = app_module.build_system_prompt(db, "", "hello") with_skills = asyncio.run(build_system_prompt(conn, "", "hello"))
assert "## Active Skills" in with_skills assert "## Active Skills" in with_skills
assert "memory.search" in with_skills assert "memory.search" in with_skills
finally: finally:
db.close() conn.close()