feat: image generation service — ComfyUI cluster integration

- POST /api/image/generate proxy endpoint (admin required)
- GET /api/image/status lists available image gen nodes
- Cluster AMQP protocol: cmd.image_generate, image_generated, image_failed
- Node agent auto-detects ComfyUI, registers image_gen capability
- ComfyUI workflow builder: CheckpointLoader → KSampler → VAEDecode → SaveImage
- Hardware probe checks ComfyUI reachability + checkpoint model list
- 27 tests covering cluster handlers, router, node agent, hardware, capabilities
- Config: CAIC_COMFYUI_BASE, CAIC_COMFYUI_TIMEOUT, comfyui_port in agent.ini
- Version bump to v1.1.0
- Documentation: ai.md, wiki/Developer-Architecture.md, current-wip.md, README.md, .env.example
This commit is contained in:
gramps
2026-07-27 08:06:03 -07:00
parent 576d9333b3
commit aecd3330fd
15 changed files with 2115 additions and 1005 deletions
+11 -6
View File
@@ -37,6 +37,7 @@ Every router has a dedicated test file:
| `test_cluster_heartbeat.py` | `cluster.py` — heartbeat handler, known/unknown node |
| `test_model_swap.py` | `cluster.py` + `triage.py` — request_model_swap, handle_model_ready/failed, select_node swap triggering |
| `test_node_agent.py` | `node_agent/agent.py` — registration, ping/pong, model swap |
| `test_image.py` | Image generation — cluster handlers, router proxy, node agent ComfyUI integration, hardware probe, capability detection |
| `test_triage.py` | `triage.py` — classify_query, select_node, get_inference_url |
| `test_settings_allowlist.py` | `routers/settings.py` — allowlisted key enforcement |
| `test_skills_framework.py` | `routers/skills.py` — list, toggle, unknown skill, prompt injection |
@@ -69,10 +70,10 @@ Refactored from single-file (`app.py`) into modules under project root:
| `crypto.py` | AES-256-GCM encrypt/decrypt + key management (stored as `heartbeat_interval_ms` in settings) |
| `model_pull.py` | Startup model availability check + Ollama pull API |
| `triage.py` | Phi-4-mini-based query classification + cluster node selection |
| `cluster.py` | Cluster node registry, event log, coordinator election, ping/pong, model swap handlers |
| `cluster.py` | Cluster node registry, event log, coordinator election, ping/pong, model swap handlers, image generation request/response |
| `amqp.py` | AMQP connection manager — connect, disconnect, publish, subscribe, auto-reconnect |
| `node_agent/` | Standalone worker agent — AMQP client for registration, ping/pong, model swap |
| `routers/` | One module per endpoint group (chat, search, skills, completions, upload, ingest) |
| `node_agent/` | Standalone worker agent — AMQP client for registration, ping/pong, model swap, image generation |
| `routers/` | One module per endpoint group (chat, search, skills, completions, upload, ingest, image) |
### Entrypoint / API keys
@@ -88,6 +89,7 @@ Refactored from single-file (`app.py`) into modules under project root:
3. **`/v1/chat/completions`** → OpenAI-compatible for Continue.dev/IDE integration; FIM requests proxied without persistence
4. **`/api/upload`** → multipart file upload, PDF/text extraction, `mode=(context|ingest|both)`, stores SQLite context (1hr expiry) + Qdrant upsert
5. **`/api/ingest`** → Bearer token auth, programmatic RAG ingest (terminal hook, external tools)
6. **`POST /api/image/generate`** → admin required, routes to corsair node agent via AMQP → ComfyUI workflow → returns PNG; `GET /api/image/status` lists available image gen nodes
### Perplexity / auto-search
@@ -123,6 +125,7 @@ All services are available bare-metal or as containers in `docker compose up`.
| rocm-smi | No | AMD GPU stats | — |
| Qdrant | No | 6333 (coordinator) — RAG vector search | `qdrant` |
| Ollama (worker) | No | 11434 — embeddings + model pull | `ollama` |
| ComfyUI (worker) | No | 8188 — image generation API | — |
### Config quirks
@@ -130,6 +133,8 @@ All services are available bare-metal or as containers in `docker compose up`.
- `SUPPORTED_UPLOAD_TYPES` includes images (png/jpeg/gif/svg/webp) + text + PDF + JSON
- `UPLOAD_CONTEXT_EXPIRY_HOURS` = 1 hour
- Rate limits and payload caps in `config.py` — patch `security.RL_*` not `config.RL_*` for tests
- `COMFYUI_BASE` defaults to `http://192.168.50.115:8188` (overridable via `CAIC_COMFYUI_BASE`)
- `COMFYUI_TIMEOUT` defaults to `120` seconds (overridable via `CAIC_COMFYUI_TIMEOUT`)
- RAG embedding requests go to `EMBED_URL` at `/api/embeddings` (Ollama on worker :11434)
### SSE Protocol
@@ -152,16 +157,16 @@ All streaming endpoints yield `data: {json}\n\n`. Key shapes:
- **Docker containerization (B3)**: Created `Dockerfile`, `docker-compose.yml`, `.env.example`, `scripts/setup.sh`, `.dockerignore`, `searxng-settings.yml.dist`, `models/README.txt`. Fixed hardcoded defaults in `config.py` (localhost, Docker secrets path, `CAIC_DEFAULT_MODEL` env var, `CAIC_HW_STATE_PATH` env var). Added missing `psutil` + `jinja2` to `requirements.txt`. Fixed test discovery via `tests/conftest.py` sys.path insertion. 214 tests pass.
### Active
- (none)
- Image generation service backend complete — cluster handlers, router proxy, node agent ComfyUI integration, hardware probe. 27 tests pass. ComfyUI install pending on corsair.
### Blocked
- (none)
- Ball Gunner assets — waiting on Canva designs
### Upcoming (backlog)
- ~~B3 — Docker distribution~~ [DONE]
### Key config values (current)
- **Current VERSION**: `v1.0.0` in `config.py`.
- **Current VERSION**: `v1.1.0` in `config.py`.
- `SESSION_TIMEOUT_SECONDS = 3600`
- `DEFAULT_MODEL = "qwen2.5-7b-instruct"` (overridable via `CAIC_DEFAULT_MODEL`)
- `LLAMA_SERVER_BASE = "http://localhost:8081"` (overridable via env var)