gramps/cAIc
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

docs: sync wiki with current architecture and roadmap

Browse Source 
Developer-Architecture.md:
  - Module layout table (eviction.py, amqp.py, routers/)
  - External services table with ports + purposes
  - Config discovery / env var mapping
  - Chat pipeline includes upload_context injection + logprobs/perplexity
  - RAG eviction engine design (score formula, hysteresis, pinned sources)
  - AMQP cluster architecture overview (jc.admin / jc.system exchanges)
  - SSE protocol reference
  - Full test coverage table (132 tests across 18 files)
  - Hardware self-assessment section

current-wip.md:
  - Replace stale backlog with active Roadmap N table (Tasks 8-15)
  - Add post-Roadmap-N backlog (B1-B3)
This commit is contained in:
gramps
2026-07-06 08:21:44 -07:00
parent 775ad5d06e
commit d1676ea73a
2 changed files with 206 additions and 176 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/wiki/Developer-Architecture.md
+183 -98
View File
@@ -1,165 +1,250 @@
# Developer Architecture Guide
This document explains how JarvisChat is structured, why key guardrails exist, and what the test suite validates.
This document explains how JarvisChat is structured, the external services it integrates with, and the key architectural changes made during development.
## 1. System Overview
JarvisChat is a single-process FastAPI service with a Jinja2 frontend and SQLite persistence.
JarvisChat is a single-process FastAPI service with a Jinja2 frontend and SQLite persistence. It connects to an external llama-server for inference and optionally to SearXNG (web search), Qdrant (vector search), and RabbitMQ (AMQP cluster messaging).
Primary files:
### 1.1 Module Layout
- `app.py`: API, middleware, streaming/chat logic, auth, memory, skills, and DB bootstrap
- `templates/index.html`: main WebUX, settings panels, auth flow, streaming UI handlers
- `jarvischat.db`: runtime SQLite database created and migrated at startup
Refactored from single-file (`app.py`) into modules under project root:
Core runtime integrations:
| File | Role |
|------|------|
| `app.py` | FastAPI app, middleware, router registration, lifespan |
| `config.py` | Constants, env vars, rate/payload limits, built-in skills registry, upload limits, RAG eviction config |
| `db.py` | SQLite schema, connection factory, settings helpers, upload_context CRUD |
| `auth.py` | PIN-based guest/admin sessions, auth routes |
| `security.py` | Rate limiting, origin checks, IP allowlist, audit/incident logging |
| `memory.py` | FTS5 memory CRUD, remember/forget command parsing |
| `search.py` | SearXNG integration, perplexity scoring, refusal detection |
| `rag.py` | Qdrant vector search, system prompt assembly, chunk_text() helper, collection stats |
| `eviction.py` | Score-based RAG eviction engine (extracted from rag.py) |
| `gpu.py` | AMD GPU stats via rocm-smi |
| `amqp.py` | (WIP) aio-pika connection manager for RabbitMQ |
| `routers/` | One module per endpoint group |
- Ollama for chat/model interaction
- SearXNG for web search (optional)
- wttr.in for weather shortcut queries
- rocm-smi for GPU stats when available
### 1.2 External Services
| Service | Required | Port | Purpose |
|---------|----------|------|---------|
| llama-server (ultron) | Yes | 8081 | LLM inference (OpenAI-compat), RPC offload to jarvis:50052 |
| SearXNG | No | 8888 | Privacy-respecting web search |
| Qdrant (ultron) | No | 6333 | Vector database for RAG |
| Ollama (jarvis) | No | 11434 | Embeddings for RAG chunk vectors |
| RabbitMQ (ultron) | No | 5672 | AMQP broker for cluster messaging |
| rocm-smi | No | — | AMD GPU stats (host-level) |
### 1.3 Config Discovery
Key base URLs are configured via environment variables with sensible defaults:
| Variable | Default | Service |
|----------|---------|---------|
| `LLAMA_SERVER_BASE` | `http://192.168.50.108:8081` | llama-server on ultron |
| `OLLAMA_BASE` | `http://localhost:11434` | Legacy — all inference goes through LLAMA_SERVER_BASE |
| `SEARXNG_BASE` | `http://localhost:8888` | SearXNG |
| `QDRANT_URL` | `http://192.168.50.108:6333` | Qdrant on ultron |
| `JARVISCHAT_AMQP_URL` | `amqp://jarvischat:password@localhost:5672/jarvischat` | RabbitMQ |
## 2. Request/Response Architecture
### 2.1 Chat Pipeline (`/api/chat`)
1. Validate session, role, origin, rate, and payload limits in middleware
2. Persist user message and conversation metadata
3. Build system prompt from enabled profile, memory context, and active skills metadata
4. Stream model response over SSE token-by-token
5. Evaluate uncertainty/refusal; if needed, trigger search augmentation and stream augmented result
6. Persist final assistant message and emit terminal SSE event
2. Intercept "remember that..." / "forget about..." commands → process_remember_command()
3. Persist user message and conversation metadata
4. Build system prompt: profile + FTS5 memory + Qdrant RAG results + preset + active skills + uploaded document (if upload_context_id)
5. Stream from llama-server with `logprobs: true` for perplexity scoring
6. If perplexity > 15.0 OR refusal patterns match → re-query with SearXNG results
7. Persist final assistant message and emit terminal SSE event
### 2.2 Explicit Search Pipeline (`/api/search`)
1. Persist search-as-message into the target/new conversation
1. Persist search-as-message into conversation
2. Emit `searching` SSE event
3. Pull web results from SearXNG
4. Summarize with Ollama via SSE stream
5. Persist summary and emit `done` event (plus raw results payload)
4. Summarize via llama-server SSE stream
5. Persist summary and emit `done` event
### 2.3 Settings/Control Surface
### 2.3 RAG Ingest Pipeline (`/api/ingest`)
- Profile, presets, memory, conversation management, and settings APIs
- Skills APIs for phase-1 registry and enable/disable controls
- Auth/session APIs for guest/admin role handling and keepalive
1. Bearer token auth (same key as completions API)
2. Chunk text via shared `chunk_text()` helper (512-token chunks, 128-token overlap)
3. Embed via Ollama `/api/embeddings`
4. Upsert to Qdrant collection `jarvis_rag`
5. Trigger `maybe_evict()` if collection exceeds high-water mark
### 2.4 Upload Pipeline (`/api/upload`)
1. Admin required, multipart file upload
2. Validate MIME type + size against config limits
3. PDF text extraction via pypdf; plain text for all other types
4. Three modes: `context` (SQLite with 1hr expiry), `ingest` (RAG/Qdrant), `both`
5. Trigger `maybe_evict()` if ingest mode
## 3. Data Model (SQLite)
Key tables:
- `conversations`: conversation headers and timestamps
- `messages`: ordered chat history entries
- `profile`: singleton row for injected profile prompt
- `settings`: runtime toggles and selected defaults
- `system_presets`: named reusable system prompts
- `skills`: per-skill enabled state and timestamp
- `memories` (FTS5 virtual table): searchable user memory facts
- `conversations` — headers, timestamps, attachment_count
- `messages` — ordered chat history per conversation
- `profile` — singleton row for injected profile prompt
- `settings` — runtime toggles and selected defaults
- `system_presets` — named reusable system prompts
- `skills` — per-skill enabled state and timestamp
- `memories` (FTS5 virtual table) — full-text searchable user memory facts
- `upload_context` — auto-expiring document storage for context injection
Design notes:
- Startup is idempotent: tables are created if missing and defaults seeded only when absent
- No connection pool: each request opens a short-lived SQLite connection
- Startup is idempotent: tables created if missing, defaults seeded only when absent
- No connection pool: each request opens and closes a short-lived SQLite connection
- `init_db()` called in FastAPI lifespan
## 4. Security Implementations
This section documents explicit controls currently in code.
### 4.1 Auth Model
- Guest session is default for conversational access
- Admin unlock uses 4-digit PIN and creates admin-capable session
- Admin required for write/destructive routes
- Session heartbeat/timeout and explicit logout/revoke flow
- Guest session by default (POST /api/auth/guest)
- Admin unlock via 4-digit PIN (POST /api/auth/login)
- Admin required for PUT/DELETE/PATCH + all POST except allowlist (/api/chat, /api/search, /api/auth/*)
- /api/ingest is exempt from session auth — self-authenticates via Bearer token
- Session heartbeat/timeout (90s default) and explicit logout
### 4.2 PIN and Session Hardening
### 4.2 PIN Hardening
- Admin PIN hashed with PBKDF2-HMAC-SHA256 + salt
- Failed PIN attempts tracked per client IP
- Lockout window enforced after max failed attempts
- Failed PIN attempts tracked per client IP (max 5, 300s lockout)
- Default PIN allowed only if JARVISCHAT_ALLOW_DEFAULT_PIN=true
### 4.3 Browser and API Abuse Controls
- Origin checks on state-changing requests
- Rate limiting by endpoint category and identity (IP/session)
- Payload size limits per route class
- Settings key allowlist to block arbitrary configuration injection
- IP allowlist/CIDR gate with optional trusted proxy forwarding mode
- Origin checks on all /api/ requests (rejects absent Origin AND Referer)
- Rate limiting per endpoint category and identity (IP/session)
- Payload size limits per route class (64KB default, 128KB chat, 20MB upload)
- Settings key allowlist (5 keys: profile_enabled, default_model, etc.)
- IP allowlist/CIDR gate with trusted proxy forwarding mode
### 4.4 Output and Error Safety
- Search result URLs sanitized to `http`/`https` only
- Search result URLs sanitized to http/https only
- Client-safe error envelopes with incident key correlation
- Full stack traces and diagnostic metadata logged server-side only
- Full stack traces logged server-side only
### 4.5 Operational Auditability
- Structured audit events for auth actions, admin operations, and guardrail denials
- Incident logs include event type, key, path/method context, and runtime metadata
- Structured audit events for auth actions, admin ops, guardrail denials
- Incident logs with event type, key, path/method, and runtime metadata
## 5. Skills Framework (Phase 1)
## 5. RAG Architecture
Goal: introduce a governed skills control plane inside the local JarvisChat sandbox.
### 5.1 Vector Search
Current behavior:
- Qdrant collection `jarvis_rag` on ultron:6333
- Embeddings via Ollama on jarvis:11434 (`/api/embeddings`)
- Shared `chunk_text(text, chunk_size=512, overlap=128)` helper in rag.py
- Upload and ingest endpoints share the same chunk+embed+upsert pipeline
- Built-in skill registry defined server-side
- Per-skill enable/disable persisted in DB
- Global `skills_enabled` master toggle in settings
- Active skills injected into system prompt with bounded text budget
- API endpoints to list skills, list active skills, and toggle skill state
- WebUX settings panel to control master/per-skill toggles
### 5.2 Score-Based Eviction
Non-goals in phase 1:
When `RAG_MAX_VECTORS` is exceeded, eviction fires with hysteresis:
- No unrestricted shell/tool execution
- No external connector execution (filesystem, Gmail, etc.)
- High-water mark: 80% of max → trigger eviction
- Low-water mark: 20% of max → stop eviction
- Batch size: 1000 vectors per cycle
- Score formula: `score = (access_weight * retrieval_count) + (age_weight * hours_since_ingested)`
- Lower score evicted first (least useful)
- Tiebreaker: oldest last_accessed ASC
- Excluded sources: `upload`, `profile` (pinned)
- Grace period: 1 hour before any vector is eligible
- Thread-safe via `asyncio.Lock`
## 6. Testing Strategy and Validation Intent
Eviction module at `eviction.py` (re-exported through `rag.py` for backward compat).
The test suite validates both behavior and guardrail assumptions.
### 5.3 Operational Stats
### 6.1 What We Test
`GET /api/rag/stats` (admin required) returns:
- vector_count, max_vectors, high_water_pct, low_water_pct, percent_full
- pinned_sources list, grace_hours
- at_risk_count, pinned_count, avg_retrieval_count
- eviction_counts_last_{1,5,30}m
- Auth capability separation (guest vs admin)
- URL sanitization safety for outbound links
- Rate and payload guardrails
- IP allowlist behavior
- Safe error envelope behavior and SSE error leakage prevention
- Streaming chat/search and memory command paths
- Skills framework toggles and prompt-injection behavior
### 5.4 Flush
### 6.2 Why These Tests Matter
`POST /api/rag/flush` (admin required) — deletes all non-pinned vectors. Returns `{deleted_count, collection, status}`.
- Confirms security controls are active and regression-resistant
- Ensures streaming UX protocol remains stable (`token`, `searching`, `done`, `error`)
- Verifies policy intent: dangerous actions require admin capability
- Validates new features preserve prior guarantees
## 6. AMQP Cluster Architecture (WIP)
### 6.3 Internal Process Validation
RabbitMQ on ultron with dedicated `jarvischat` vhost:
For substantive changes, Definition of Done includes:
| Exchange | Type | Purpose |
|----------|------|---------|
| `jc.admin` | topic | Commands: swap model, shutdown, heartbeat request |
| `jc.system` | topic | Events: model_ready, model_failed, heartbeat, registration |
Pending implementation (Tasks 10–15):
- `amqp.py` — aio-pika connection manager with reconnect
- Node agent on jarvis — registration, heartbeat, command consumer
- `triage.py` — Phi-4-mini query classification (general/code/search/rag)
- Dynamic model swap via llama-server RPC
## 7. SSE Protocol
All streaming endpoints yield `data: {json}\n\n`:
- `{token, conversation_id}` — streaming token
- `{searching: true}` — web search triggered
- `{search_results: N}` — N results found (no raw payload)
- `{done: true, perplexity, tokens_per_sec, searched?}` — terminal
- `{error: "...", error_key: "..."}` — error with incident key
## 8. Testing Strategy
### 8.1 Test Framework
- pytest with `tmp_path` + monkeypatched httpx.AsyncClient
- No live external services required
- Test factories reset `SESSIONS`, `PIN_ATTEMPTS`, `RATE_EVENTS` globals per test
### 8.2 Test Coverage Areas (132 tests)
| Test file | Coverage |
|-----------|----------|
| test_auth_capabilities.py | Guest/admin sessions, origin blocking, logout |
| test_chat_streaming_and_memory_paths.py | Streaming, auto-search, remember/forget, upload context injection |
| test_completions.py | API key auth, FIM, streaming, blocking, errors |
| test_conversations.py | Full CRUD, guest admin, attachment_count |
| test_ingest.py | Bearer auth, chunk/embed/upsert, validation |
| test_memories.py | Edit, search, stats |
| test_models_router.py | Models list, ps, show, stats, search/status |
| test_presets.py | Full CRUD, default preset protection |
| test_profile.py | Get, update, default, length validation |
| test_rag_management.py | Collection stats, eviction algorithm (pinned/grace/scoring/batch), maybe_evict hysteresis, operational stats, flush, concurrency lock |
| test_search_route.py | Explicit search flow, no results, errors |
| test_search_url_sanitization.py | URL sanitizer |
| test_settings_allowlist.py | Allowlisted key enforcement |
| test_skills_framework.py | List, toggle, unknown skill, prompt injection |
| test_ip_allowlist.py | IP allowlist helper + middleware |
| test_rate_and_payload_guardrails.py | Rate limits + payload size |
| test_error_envelopes.py | Global exception handler + stream errors |
| test_upload.py | Upload, delete, link, by-conversation, attachment_count |
### 8.3 DoD Process
For substantive changes:
1. Implement code change
2. Add/adjust tests proving behavior and guardrail intent
3. Update README release notes for user-facing impact
4. Update wiki architecture/security/testing docs for maintainers
5. Validate with targeted test runs before merge/deploy
3. Update this wiki and README in the same change set
4. Validate with full test run before commit
This process is intentionally explicit so design decisions remain auditable over time.
## 9. Hardware Self-Assessment
## 7. Deployment and Operations Notes
On startup, `assess_hardware()` probes:
- RAM total/available (psutil)
- VRAM total/free (rocm-smi, best-effort)
- llama-server reachability + model list
- Qdrant reachability + collection list
- SearXNG reachability
- Primary deployment target: local/homelab systemd service
- Required dependency: Ollama
- Optional dependency: SearXNG
- Recommended log review path: system journal for startup, guardrail denials, and incidents
## 8. Contribution Guidance
When adding a feature:
1. Define security posture first (who can execute, what can fail, and failure mode)
2. Implement smallest safe slice with clear limits
3. Add tests that prove both happy path and guardrail path
4. Update this wiki and README in the same change
Writes `hardware_state.json` to working directory.
docs/wiki/current-wip.md
+23 -78
View File
@@ -1,86 +1,31 @@
# JarvisChat Current WiP Backlog
Last updated: 2026-04-27
Owner: Gramps + Copilot
Scope: issues, bugs, security exposures, and feature enhancements.
Last updated: 2026-07-06
Owner: Gramps
Scope: Active roadmap items and backlog.
Total identified items: 29
## Active Roadmap — Roadmap N: AMQP Cluster Nervous System
## Priority Definitions
- P0: Critical risk or data-loss/security exposure; do first.
- P1: High impact reliability/correctness work.
- P2: Important feature/UX improvements.
- P3: Nice-to-have polish.
| Task | Description | Status |
|------|-------------|--------|
| Task 8 | RAG Corpus Management (score-based eviction, pinned sources, operational stats) | **DONE** |
| Task 9 | RabbitMQ install + exchange setup on ultron | **DONE** |
| Task 10 | AMQP connection layer in jC (`amqp.py`, aio-pika) | **NEXT** |
| Task 11 | Node agent on jarvis — registration, heartbeat, command listeners | Pending |
| Task 12 | AMQP wiring: inject context into chat/completions pipeline | Pending |
| Task 13 | Query triage via Phi-4-mini (`triage.py`) | Pending |
| Task 14 | Dynamic model swap — publish cmd, handle ready/failed | Pending |
| Task 15 | Cluster status UI panel in templates/index.html | Pending |
## Top 10 (Urgency Order)
1. [P0][DONE] Add authentication/authorization for all write and admin endpoints.
2. [P0][DONE] Add CSRF/origin protection for browser-initiated state-changing requests.
3. [P0][DONE] Block unsafe URL schemes in rendered search-result links (e.g., javascript:).
4. [P0][DONE] Add rate limiting and request body size limits for chat/search/profile APIs.
5. [P1][DONE] Restrict settings updates to an allowlist of valid keys.
6. [P1] Add pagination + hard caps on list endpoints (memories, conversations, message history).
7. [P1][DONE] Stop returning raw exception text to clients; use safe error envelopes.
8. [P1][DONE] Add automated tests for chat streaming, auto-search trigger, and memory command paths.
9. [P2][DONE] Implement skills/tool-call framework (MCP-style) with per-skill enable controls.
10. [P2] Implement heartbeat/check-in pipeline with scheduler + summary endpoint.
## Backlog (Post-Roadmap N)
## Item 1 Executive Summary (Scope + Security)
- Status: Complete. Guest/admin capability split implemented with admin-only write enforcement, origin checks on state-changing requests, audit logging, and endpoint capability tests.
- Decision: JarvisChat is local-first by design. Primary mode is same-host Ollama; optional mode allows RFC1918 LAN endpoints only.
- Constraint: Public Internet AI endpoints are out of scope unless explicitly enabled in a future advanced mode.
- Risk: Even on LAN, unauthenticated write/admin endpoints permit unauthorized data tampering and deletion.
- Requirement: Add mandatory admin authentication for all POST/PUT/DELETE routes and destructive actions.
- Authentication shape (scope-locked): two capability tiers only: guest (chat-only) and admin (4-digit PIN unlock).
- Scope guardrail: Avoid full RBAC. Keep capability split minimal: conversational chat for guest, advanced/destructive actions for admin.
- Definition of done:
1. Auth required on all state-changing endpoints.
2. Destructive actions require admin authorization.
3. Endpoint configuration rejects non-local/non-RFC1918 AI backends by default.
4. Strong rate limiting + lockout controls in place for PIN attempts.
5. Security events logged for failed and successful admin actions.
## Full Backlog (Sorted by Priority)
### P0 Critical
1. Add auth for write/admin endpoints (`POST/PUT/DELETE` routes, mass delete, profile/settings changes).
2. Add CSRF or strict origin checks for browser session protection.
3. Validate/sanitize outbound href URLs before rendering in HTML (allow http/https only).
4. Add per-IP rate limiting on `/api/chat`, `/api/search`, `/api/profile`, `/api/settings`.
5. Enforce request size limits (message/profile text and JSON body) to prevent memory abuse.
### P1 High
6. Add settings key allowlist in `/api/settings` to prevent arbitrary key injection.
7. Add pagination (`limit`, `offset`) with enforced maximums for list APIs.
8. Add DB indexes and query hygiene for scalability (`messages.conversation_id`, timestamps).
9. Replace raw exception leakage to clients with generic safe error messages + server-side logs.
10. Add request/response timeout and retry policy consistency across external calls.
11. Add endpoint-level audit logging for destructive operations.
12. Add unit/integration tests for: remember/forget parsing, refusal detection, search fallback, SSE done/error shape.
13. Add conversation title sanitization and length constraints.
14. Ensure default preset semantics are correct (currently all seeded presets are marked default).
15. Add preflight validation for required model/preset selection and block send with clear user guidance instead of timing out.
### P2 Important Features
16. HTTPS via Let's Encrypt: set up reverse proxy (Caddy or Nginx) with auto-cert to fix clipboard API and enable secure context features.
17. Bidirectional image I/O: load multimodal models for both vision understanding (image inputs, describe/answer) and image generation (diffusion model or multimodal output). Enable image upload/paste and AI-generated image display in chat.
18. Skills system: load markdown skill files with YAML frontmatter from skills directory.
19. Skills registry API: list/enable/disable skills and expose active skills to UI.
20. Inject active skill instructions into system prompt with bounded token budget.
21. Tool execution guardrails: allowlist, confirmation mode, and execution logs.
22. Heartbeat scheduler (cron/systemd timer) for daily check-ins.
23. Heartbeat endpoint for generated briefings and anomaly summaries.
24. Model info UI panel (description, updated date, best-use purpose).
25. Default model selection improvements and persistence validation.
26. Hidden model list support (exclude models from dropdown).
27. Model update action from UI (trigger controlled model pull).
### P3 Nice to Have
28. Conversation search/filter and export tooling.
29. Keyboard shortcuts, retry button, and source-link polish.
- B1 — Context loss in follow-up questions (investigation)
- B2 — Bang-prefixed (`!`) search routing
- B3 — Docker distribution (planning doc at `docker.md`)
- HTTPS / reverse proxy (Caddy)
- Conversation search/filter and export tooling
- Keyboard shortcuts, retry button, source-link polish
## Maintenance Rules
- Keep this file as the single source of truth.
- Update item priority/status whenever work starts or completes.
- Mirror the Top 10 summary in README and keep counts aligned.
- Keep this file as the single source of truth for roadmap tracking.
- Update as work starts or completes.