feat: image generation service — ComfyUI cluster integration
- POST /api/image/generate proxy endpoint (admin required) - GET /api/image/status lists available image gen nodes - Cluster AMQP protocol: cmd.image_generate, image_generated, image_failed - Node agent auto-detects ComfyUI, registers image_gen capability - ComfyUI workflow builder: CheckpointLoader → KSampler → VAEDecode → SaveImage - Hardware probe checks ComfyUI reachability + checkpoint model list - 27 tests covering cluster handlers, router, node agent, hardware, capabilities - Config: CAIC_COMFYUI_BASE, CAIC_COMFYUI_TIMEOUT, comfyui_port in agent.ini - Version bump to v1.1.0 - Documentation: ai.md, wiki/Developer-Architecture.md, current-wip.md, README.md, .env.example
This commit is contained in:
gramps
@@ -22,9 +22,9 @@ Refactored from single-file (`app.py`) into modules under project root:
|
||||
| `rag.py` | Qdrant vector search, system prompt assembly, chunk_text() helper, collection stats |
|
||||
| `eviction.py` | Score-based RAG eviction engine (extracted from rag.py) |
|
||||
| `gpu.py` | AMD GPU stats via rocm-smi |
|
||||
| `hardware.py` | Hardware self-assessment — CPU, RAM, VRAM, service health probes |
|
||||
| `hardware.py` | Hardware self-assessment — CPU, RAM, VRAM, service health probes (llama-server, Qdrant, SearXNG, ComfyUI) |
|
||||
| `amqp.py` | aio-pika connection manager for RabbitMQ (connect, disconnect, publish, subscribe, auto-reconnect) |
|
||||
| `cluster.py` | Cluster node registry, event log, coordinator election, ping/pong, model swap handlers |
|
||||
| `cluster.py` | Cluster node registry, event log, coordinator election, ping/pong, model swap handlers, image generation request/response |
|
||||
| `triage.py` | Phi-4-mini query classification + `select_node()` for cluster routing |
|
||||
| `routers/` | One module per endpoint group |
|
||||
|
||||
@@ -36,6 +36,7 @@ Refactored from single-file (`app.py`) into modules under project root:
|
||||
| SearXNG | No | 8888 | Privacy-respecting web search |
|
||||
| Qdrant (coordinator) | No | 6333 | Vector database for RAG |
|
||||
| Ollama (worker) | No | 11434 | Embeddings for RAG chunk vectors |
|
||||
| ComfyUI (worker) | No | 8188 | Image generation (Stable Diffusion / Flux) |
|
||||
| RabbitMQ (coordinator) | No | 5672 | AMQP broker for cluster messaging |
|
||||
| rocm-smi | No | — | AMD GPU stats (host-level) |
|
||||
|
||||
@@ -50,6 +51,8 @@ Key base URLs are configured via environment variables with sensible defaults:
|
||||
| `SEARXNG_BASE` | `http://localhost:8888` | SearXNG |
|
||||
| `QDRANT_URL` | `http://192.168.50.108:6333` | Qdrant on coordinator |
|
||||
| `CAIC_AMQP_URL` | `amqp://caic:password@localhost:5672/caic` | RabbitMQ |
|
||||
| `CAIC_COMFYUI_BASE` | `http://192.168.50.115:8188` | ComfyUI on worker |
|
||||
| `CAIC_COMFYUI_TIMEOUT` | `120` | ComfyUI generation timeout (seconds) |
|
||||
|
||||
## 2. Request/Response Architecture
|
||||
|
||||
@@ -87,6 +90,17 @@ Key base URLs are configured via environment variables with sensible defaults:
|
||||
4. Three modes: `context` (SQLite with 1hr expiry), `ingest` (RAG/Qdrant), `both`
|
||||
5. Trigger `maybe_evict()` if ingest mode
|
||||
|
||||
### 2.5 Image Generation Pipeline (`POST /api/image/generate`)
|
||||
|
||||
1. Admin required, JSON body with prompt and optional params (width, height, steps, seed, model)
|
||||
2. Find active node with `image_gen` capability via `_find_image_node()`
|
||||
3. Publish `cmd.image_generate` via AMQP to selected worker node
|
||||
4. Worker node agent builds ComfyUI workflow (CheckpointLoader → KSampler → VAEDecode → SaveImage)
|
||||
5. Worker polls ComfyUI `/history/{prompt_id}` until image is ready
|
||||
6. Worker fetches PNG from ComfyUI `/view` endpoint, base64-encodes, publishes `image_generated` on `jc.system`
|
||||
7. Coordinator receives response, decodes base64, returns `image/png` to client
|
||||
8. `GET /api/image/status` returns available image gen nodes and their status
|
||||
|
||||
## 3. Data Model (SQLite)
|
||||
|
||||
Key tables:
|
||||
@@ -242,8 +256,8 @@ Every RabbitMQ server belongs to a cluster. Currently only the coordinator runs
|
||||
|
||||
| Exchange | Type | Purpose |
|
||||
|----------|------|---------|
|
||||
| `jc.admin` | topic | Lifecycle commands: register, deregister, ping, pong, admitted, rejected; model commands: cmd.swap_model |
|
||||
| `jc.system` | topic | Events: model_ready, model_failed, node.*.heartbeat, event; coordinator queries: coord_query, coord_response |
|
||||
| `jc.admin` | topic | Lifecycle commands: register, deregister, ping, pong, admitted, rejected; model commands: cmd.swap_model; image commands: cmd.image_generate |
|
||||
| `jc.system` | topic | Events: model_ready, model_failed, image_generated, image_failed, node.*.heartbeat, event; coordinator queries: coord_query, coord_response |
|
||||
|
||||
All exchanges, queues, and bindings are declared by `amqp.py` at startup. Worker runs `node_agent/agent.py` which connects as an AMQP client, registers, responds to ping, and handles model swap commands.
|
||||
|
||||
@@ -265,7 +279,7 @@ All streaming endpoints yield `data: {json}\n\n`:
|
||||
- No live external services required
|
||||
- Test factories reset `SESSIONS`, `PIN_ATTEMPTS`, `RATE_EVENTS` globals per test
|
||||
|
||||
### 8.2 Test Coverage Areas (200 tests)
|
||||
### 8.2 Test Coverage Areas (228 tests)
|
||||
|
||||
| Test file | Coverage |
|
||||
|-----------|----------|
|
||||
@@ -277,6 +291,8 @@ All streaming endpoints yield `data: {json}\n\n`:
|
||||
| test_conversations.py | Full CRUD, guest admin, attachment_count |
|
||||
| test_error_envelopes.py | Global exception handler + stream errors |
|
||||
| test_gpu.py | GPU stats — rocm-smi (Linux), system_profiler (Darwin/Apple Silicon) |
|
||||
| test_hardware.py | Hardware assessment, service reachability |
|
||||
| test_image.py | Image generation — cluster handlers, router proxy, node agent ComfyUI integration, hardware probe, capability detection |
|
||||
| test_ingest.py | Bearer auth, chunk/embed/upsert, validation |
|
||||
| test_ip_allowlist.py | IP allowlist helper + middleware |
|
||||
| test_memories.py | Edit, search, stats |
|
||||
@@ -311,5 +327,6 @@ On startup, `assess_hardware()` probes:
|
||||
- llama-server reachability + model list
|
||||
- Qdrant reachability + collection list
|
||||
- SearXNG reachability
|
||||
- ComfyUI reachability + checkpoint model list
|
||||
|
||||
Writes `hardware_state.json` to working directory.
|
||||
|
||||
@@ -1,9 +1,13 @@
|
||||
# cAIc Current WiP Backlog
|
||||
|
||||
Last updated: 2026-07-14
|
||||
Last updated: 2026-07-27
|
||||
Owner: Gramps
|
||||
Scope: Active roadmap items and backlog.
|
||||
|
||||
## In Progress
|
||||
|
||||
- **Image Generation Service** — Backend wired: cluster handlers, `POST /api/image/generate` proxy, node agent ComfyUI integration, hardware probe, 27 tests. ComfyUI install pending on corsair (RTX 5070 Ti).
|
||||
|
||||
## Completed
|
||||
|
||||
- **B8 (v0.19.3)** — Private Chat mode. Backend skip-DB/skip-RAG/skip-search flag, frontend PRIVATE badge, info popup.
|
||||
|
||||
Reference in New Issue
Block a user