From 3fd8b01353dcc1d3f547badf7029455524f504eb Mon Sep 17 00:00:00 2001 From: gramps Date: Fri, 3 Jul 2026 12:38:48 -0700 Subject: [PATCH] docs: update AGENTS.md, CLAUDE.md, TASKS.md for Tasks 4-6 completion --- AGENTS.md | 52 +++++++++++++++++++++++++++++----------------------- CLAUDE.md | 51 +++++++++++++++++++++++++++++++++------------------ TASKS.md | 12 +++++++++--- 3 files changed, 71 insertions(+), 44 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 0fee6ca..6e1e539 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -12,15 +12,16 @@ ./venv/bin/python -m pytest tests/ -v ``` -All tests use `tmp_path` fixtures + monkeypatched `httpx.AsyncClient.stream`. No external services needed. Test factories reset `SESSIONS`, `PIN_ATTEMPTS`, `RATE_EVENTS` globals — be careful not to let test state leak. After the modular refactor, tests import directly from the correct modules (`db`, `security`, `config`, `search`, `rag`, `memory`, `routers.*`) — not from the old monolithic `app` namespace. +All tests use `tmp_path` fixtures + monkeypatched `httpx.AsyncClient.stream/get/post/put`. No external services needed. Test factories reset `SESSIONS`, `PIN_ATTEMPTS`, `RATE_EVENTS` globals — be careful not to let test state leak. Tests import directly from the correct modules (`db`, `security`, `config`, `search`, `rag`, `memory`, `routers.*`). Every router has a dedicated test file: | File | Covers | |------|--------| | `test_auth_capabilities.py` | `auth.py` — guest/admin sessions, origin blocking, logout | -| `test_chat_streaming_and_memory_paths.py` | `routers/chat.py` — streaming, auto-search, remember/forget | +| `test_chat_streaming_and_memory_paths.py` | `routers/chat.py` — streaming, auto-search, remember/forget, upload context injection | | `test_completions.py` | `routers/completions.py` — API key auth, FIM, streaming, blocking, errors | -| `test_conversations.py` | `routers/conversations.py` — full CRUD, guest admin enforcement | +| `test_conversations.py` | `routers/conversations.py` — full CRUD, guest admin enforcement, attachment_count | +| `test_ingest.py` | `routers/ingest.py` — Bearer auth, chunk/embed/upsert, validation | | `test_memories.py` | `routers/memories.py` — edit, search, stats endpoints | | `test_models_router.py` | `routers/models.py` — models list, ps, show, stats, search/status | | `test_presets.py` | `routers/presets.py` — full CRUD, default preset protection | @@ -32,8 +33,9 @@ Every router has a dedicated test file: | `test_ip_allowlist.py` | IP allowlist helper + middleware | | `test_rate_and_payload_guardrails.py` | Rate limits + payload size enforcement | | `test_error_envelopes.py` | Global exception handler + stream error incidents | +| `test_upload.py` | `routers/upload.py` — upload, delete, link, by-conversation, attachment_count integration | -Modules that call `httpx.AsyncClient` (chat, completions, models, search_route) +Modules that call `httpx.AsyncClient` (chat, completions, models, search_route, upload, ingest) are mocked via `monkeypatch.setattr` on `AsyncClient.stream`, `.get`, or `.post`. CPU stats in `models.py` (`api/stats`) use real `psutil`; GPU stats are monkeypatched via `routers.models.get_gpu_stats`. @@ -45,64 +47,68 @@ Refactored from single-file (`app.py`) into modules under project root: | File | Role | |------|------| | `app.py` | FastAPI app, middleware, router registration | -| `config.py` | Constants, env vars, rate/payload limits, built-in skills registry | -| `db.py` | SQLite schema, connection factory, settings helpers | +| `config.py` | Constants, env vars, rate/payload limits, built-in skills registry, upload limits | +| `db.py` | SQLite schema, connection factory, settings helpers, upload_context CRUD | | `auth.py` | PIN-based guest/admin sessions, auth routes | | `security.py` | Rate limiting, origin checks, IP allowlist, audit/incident logging | | `memory.py` | FTS5 memory CRUD, remember/forget command parsing | | `search.py` | SearXNG integration, perplexity scoring, refusal detection | -| `rag.py` | Qdrant vector search + system prompt assembly | +| `rag.py` | Qdrant vector search + system prompt assembly + chunk_text() helper | | `gpu.py` | AMD GPU stats via `rocm-smi` | -| `routers/` | One module per endpoint group (chat, search, skills, completions, etc.) | +| `routers/` | One module per endpoint group (chat, search, skills, completions, upload, ingest) | ### Entrypoint / API keys - `app.py` line 148: `uvicorn.run(app, ...)` when called directly -- `config.py` line 14: `LLAMA_SERVER_BASE` defaults to `http://192.168.50.108:8081` — llama-server, **not** standard Ollama port, used by all model endpoints -- `config.py` line 17: `COMPLETIONS_API_KEY` read from `JARVISCHAT_COMPLETIONS_API_KEY` env var or auto-generates a random key — no longer a missing import -- `config.py` line 13: `OLLAMA_BASE` is legacy/unused — all endpoints now use `LLAMA_SERVER_BASE` +- `config.py` line 14: `LLAMA_SERVER_BASE` defaults to `http://192.168.50.108:8081` — llama-server on ultron, RPC-offloads GPU layers to jarvis :50052 +- `config.py` line 17: `COMPLETIONS_API_KEY` read from `JARVISCHAT_COMPLETIONS_API_KEY` env var or auto-generates +- `config.py` line 13: `OLLAMA_BASE` is legacy/unused — all endpoints use `LLAMA_SERVER_BASE` ### Key flows -1. **`/api/chat`** → `process_remember_command()` intercepts "remember that..." / "forget about..." first → else `build_system_prompt()` (profile + FTS5 memory + Qdrant RAG + preset + skills) → stream from llama-server with `logprobs: true` → if perplexity > 15.0 OR `REFUSAL_PATTERNS` match, re-query with SearXNG results -2. **`/api/search`** → bypasses perplexity/refusal, queries SearXNG directly → summarizes via llama-server (no raw results leaked in SSE) +1. **`/api/chat`** → `process_remember_command()` intercepts "remember that..." / "forget about..." first → optional `upload_context_id` fetches document text from SQLite → `build_system_prompt()` (profile + FTS5 memory + Qdrant RAG + preset + skills + uploaded doc) → stream from llama-server with `logprobs: true` → if perplexity > 15.0 OR `REFUSAL_PATTERNS` match, re-query with SearXNG results +2. **`/api/search`** → bypasses perplexity/refusal, queries SearXNG directly → summarizes via llama-server 3. **`/v1/chat/completions`** → OpenAI-compatible for Continue.dev/IDE integration; FIM requests proxied without persistence +4. **`/api/upload`** → multipart file upload, PDF/text extraction, `mode=(context|ingest|both)`, stores SQLite context (1hr expiry) + Qdrant upsert +5. **`/api/ingest`** → Bearer token auth, programmatic RAG ingest (terminal hook, external tools) ### Perplexity / auto-search -The upstream request includes `"logprobs": true`. `parse_llama_stream_chunk()` extracts per-token logprobs from each chunk's `choices[0].logprobs.content[].logprob`. The `all_logprobs` list is populated during streaming, so `calculate_perplexity()` and `is_uncertain()` work correctly — auto-search on high perplexity is no longer dead code. +The upstream request includes `"logprobs": true`. `parse_llama_stream_chunk()` extracts per-token logprobs from each chunk's `choices[0].logprobs.content[].logprob`. The `all_logprobs` list is populated during streaming, so `calculate_perplexity()` and `is_uncertain()` work correctly. ### Auth / lockdown - Guest session by default (`POST /api/auth/guest`), admin unlock via 4-digit PIN (`POST /api/auth/login`) - Admin required for PUT/DELETE/PATCH + all POST except allowlist (`/api/chat`, `/api/search`, `/api/auth/*`) +- `/api/ingest` is exempt from session auth — self-authenticates via Bearer token - IP allowlist, rate limiting, origin checking, payload size limits — all enforced in `app.py` middleware -- Origin check applies to **all** `/api/` requests (not just state-changing methods); `origin_allowed()` returns `False` when both `Origin` and `Referer` headers are absent, closing CSRF read gap +- Origin check applies to **all** `/api/` requests; returns `False` when both `Origin` and `Referer` are absent - `JARVISCHAT_ADMIN_PIN` env var required on first boot (or `JARVISCHAT_ALLOW_DEFAULT_PIN=true`) ### Database - SQLite at `jarvischat.db`, auto-created by `init_db()` on startup via FastAPI `lifespan` - `get_db()` opens new connection per request (no pool). Close after use. -- FTS5 virtual table `memories` for full-text search with BM25 ranking. FTS5 operator keywords (`AND`, `OR`, `NOT`, `NEAR`) are double-quoted to prevent parse errors. +- FTS5 virtual table `memories` for full-text search with BM25 ranking. +- `upload_context` table: auto-expiring document storage for chat context injection. ### External services | Service | Required | Port | |---------|----------|------| -| llama-server (OpenAI-compat API) | Yes | 8081 (ultron) or env `LLAMA_SERVER_BASE` | +| llama-server (ultron) | Yes | 8081 + RPC :50052 (jarvis GPU) | | SearXNG | No | 8888 | -| wttr.in | No | weather shortcut bypasses SearXNG; curl UA for plain-text output | +| wttr.in | No | weather shortcut | | rocm-smi | No | AMD GPU stats | | Qdrant | No | 6333 (ultron) — RAG vector search | ### Config quirks -- Rate limits and payload caps in `config.py` — tweak for testing by monkeypatching module attributes (note: patch `security.RL_*` not `config.RL_*` since `security` imports bindings separately) -- `ALLOWED_SETTINGS_KEYS` in `config.py` controls which keys the UI can write via `/api/settings` -- Settings table seeded with defaults (`profile_enabled`, `search_enabled`, `memory_enabled`, `skills_enabled`, `default_model`) — never overwritten by `init_db()` -- Profile table uses singleton row `id=1` -- RAG embedding requests go to `EMBED_URL` at `/api/embeddings` (separate Ollama instance) +- `BODY_LIMIT_UPLOAD_BYTES` = 20MB for `/api/upload`; other paths use smaller limits +- `SUPPORTED_UPLOAD_TYPES` includes images (png/jpeg/gif/svg/webp) + text + PDF + JSON +- `UPLOAD_CONTEXT_EXPIRY_HOURS` = 1 hour +- Rate limits and payload caps in `config.py` — patch `security.RL_*` not `config.RL_*` for tests +- RAG embedding requests go to `EMBED_URL` at `/api/embeddings` (Ollama on jarvis :11434) ### SSE Protocol diff --git a/CLAUDE.md b/CLAUDE.md index cc8a390..2759a29 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -24,41 +24,56 @@ sudo systemctl restart jarvischat ## Architecture -Single-file FastAPI backend (`app.py`) + single-template frontend (`templates/index.html`). No build step. SQLite database auto-created at `jarvischat.db` on first run. +Modular FastAPI app — `app.py` wires routers, middleware, and lifespan. SQLite database auto-created at `jarvischat.db` on first run. No build step, single `templates/index.html`. ### Request Flow: `/api/chat` 1. User message saved to DB → conversation created if new -2. `build_system_prompt()` assembles: profile + FTS5 memory search results + preset prompt -3. Streamed to Ollama (`/api/chat`, `stream: true`, `logprobs: true`) via SSE -4. **Auto web search trigger**: if perplexity > 15.0 OR response matches `REFUSAL_PATTERNS`, re-queries Ollama with SearXNG results prepended to system prompt -5. Final response saved to DB; SSE `done` event sent with perplexity + tokens/sec +2. `process_remember_command()` intercepts "remember that..." / "forget about..." first +3. Optional `upload_context_id` → fetches document text from `upload_context` table, injects `[ATTACHED DOCUMENT]` into system prompt +4. `build_system_prompt()` assembles: profile + FTS5 memory search + Qdrant RAG + preset + skills + uploaded doc +5. Streamed to llama-server (`/v1/chat/completions`, `stream: true`, `logprobs: true`) via SSE +6. **Auto web search trigger**: if perplexity > 15.0 OR response matches `REFUSAL_PATTERNS`, re-queries with SearXNG results +7. Final response saved to DB; SSE `done` event sent with perplexity + tokens/sec ### Request Flow: `/api/search` (explicit search) -Bypasses perplexity/refusal detection entirely — queries SearXNG directly then asks Ollama to summarize with results as system context. +Bypasses perplexity/refusal — queries SearXNG directly then asks llama-server to summarize results. + +### Request Flow: `/api/upload` + +Multipart file upload → PDF/text extraction + chunking → optional Qdrant upsert + SQLite context storage (1hr expiry). Supports `mode=(context|ingest|both)`. Images upload as storage only — model cannot process image content. + +### Request Flow: `/api/ingest` + +Bearer-token-authenticated terminal RAG hook. Accepts raw text, chunks via `chunk_text()`, embeds via Ollama `/api/embeddings`, upserts to Qdrant. ### Memory System -FTS5 virtual table (`memories`) in SQLite. `search_memories()` uses BM25 ranking. `process_remember_command()` intercepts "remember that..." / "forget about..." before the message reaches Ollama and returns a confirmation string. Topic auto-detection via keyword matching in `detect_topic()`. +FTS5 virtual table (`memories`) in SQLite. `search_memories()` uses BM25 ranking. `process_remember_command()` intercepts "remember that..." / "forget about..." before the message reaches the model and returns a confirmation string. -### Key Constants (top of `app.py`) +### Key Constants (`config.py`) -- `OLLAMA_BASE` — `http://localhost:11434` +- `LLAMA_SERVER_BASE` — `http://192.168.50.108:8081` (ultron llama-server, RPC offloads to jarvis GPU) - `SEARXNG_BASE` — `http://localhost:8888` -- `PERPLEXITY_THRESHOLD` — `15.0` (controls auto-search sensitivity) -- `DEFAULT_MODEL` — `llama3.1:latest` +- `PERPLEXITY_THRESHOLD` — `15.0` +- `EMBED_URL` — `http://192.168.50.210:11434/api/embeddings` (Ollama on jarvis) +- `VERSION` — current version string ### External Services -- **Ollama** — required, must be running on port 11434 -- **SearXNG** — optional, port 8888; `GET /api/search/status` probes availability -- **wttr.in** — weather shortcut in `query_searxng()`, bypasses SearXNG for weather queries -- **rocm-smi** — AMD GPU stats via subprocess; gracefully degrades if not available +| Service | Required | Port | +|---------|----------|------| +| **llama-server** (ultron) | Yes | 8081 + RPC :50052 (jarvis GPU) | +| **SearXNG** | No | 8888 | +| **wttr.in** | No | weather shortcut | +| **rocm-smi** | No | AMD GPU stats | +| **Qdrant** (ultron) | No | 6333 — RAG vector search | +| **Ollama** (jarvis) | No | 11434 — embeddings only | ### Database -`get_db()` opens a new connection per request (no connection pool). `init_db()` runs at startup via the FastAPI `lifespan` handler. The `profile` table uses a singleton row (`id = 1`). Default settings are seeded but never overwritten by `init_db()`. +`get_db()` opens a new connection per request (no pool). `init_db()` runs at startup via FastAPI `lifespan`. Tables: `conversations`, `messages`, `settings`, `profile` (singleton id=1), `memories` (FTS5), `upload_context`. Default settings seeded but never overwritten. ### SSE Protocol @@ -67,8 +82,8 @@ All streaming endpoints yield `data: {json}\n\n`. Key event shapes: - `{searching: true}` — web search triggered - `{search_results: N}` — N results retrieved - `{done: true, perplexity, tokens_per_sec, searched?}` — terminal event -- `{error: "..."}` — error event +- `{error: "...", error_key: "..."}` — error with incident key ### Deployment -Runs as systemd service under user `jarvischat`, working directory `/opt/jarvischat`. Logs via syslog (`journalctl -u jarvischat`). +Runs as systemd service under user `jarvischat`, working directory `/opt/jarvischat`. Logs via syslog (`journalctl -u jarvischat`). Version bumps via git tag + commit, deployed via `git pull && systemctl restart jarvischat`. diff --git a/TASKS.md b/TASKS.md index fbcdc9a..c7dc6d5 100644 --- a/TASKS.md +++ b/TASKS.md @@ -45,7 +45,9 @@ No pytest tests required for this task. --- -## TASK 4 — File/Document Attachment: Backend Ingest Endpoint +## TASK 4 — File/Document Attachment: Backend Ingest Endpoint [DONE] + +**Status: `POST /api/upload` with mode=(context|ingest|both), PDF/text extraction, Qdrant upsert, SQLite context (1hr expiry). Committed `4a891c8` (v1.9.0).** This task implements the backend half of file/document attachment (TODO #21). The goal is dual-aspect upload: a file can be used as immediate chat context, ingested into the RAG corpus (Qdrant), or both. @@ -89,7 +91,9 @@ Run full test suite after implementation. All 26 existing tests must continue to --- -## TASK 5 — File/Document Attachment: UI Integration +## TASK 5 — File/Document Attachment: UI Integration [DONE] + +**Status: Paperclip icon, file preview pill, gallery overlay, attachment indicators, DELETE/PATCH link/by-conversation endpoints, chat context injection. Committed `81238c0` (v1.10.0).** This task implements the frontend half of TODO #21. The UI is a single file at `templates/index.html`. @@ -114,7 +118,9 @@ Run full test suite. All existing tests must continue to pass. --- -## TASK 6 — Roadmap I: Terminal Command RAG Hook +## TASK 6 — Roadmap I: Terminal Command RAG Hook [DONE] + +**Status: `POST /api/ingest` with Bearer token auth, `chunk_text()` shared helper, `jc-ingest.sh` script. Committed `1ac21ad` (v1.11.0).** This task implements autonomous RAG ingestion of significant terminal activity (TODO #23).