cAIc/TASKS.md

# cAIc — Task List (v1.0+)

Previous task history archived at `docs/archive/TASKS-pre-1.0.md`.

---

## TASK 1 — Image Generation Service (corsair)

**Goal:** Add image generation as a cluster capability. The image-gen node (currently jarvis — single-node deployment) registers as an image gen worker in the cAIc cluster.

### Requirements:

1. **Add `"image_gen"` capability** to the cluster protocol in `cluster.py` — valid capability values should include `image_gen`
2. **Image gen API wrapper** — run ComfyUI, Automatic1111, or a lightweight API (e.g., `sd-api` or `comfyui-api`) that exposes a simple `POST /generate` endpoint accepting a prompt and returning a PNG
3. **Proxy endpoint in cAIc** — `POST /api/image/generate` on the coordinator, routes the request to the image gen service via AMQP or direct HTTP
4. **Update `hardware.py`** to probe the image gen service for reachability and status
5. **Update node_agent** to report image gen capability and service status on registration

### Architecture:

```
User prompt → cAIc coordinator → AMQP/HTTP → image-gen node (ComfyUI/API) → PNG → coordinator → user
```

### Considerations:

- Response time: expect 5-30 seconds per image depending on model/resolution
- Queue management: what if multiple requests come in at once?
- Model selection: which SD/Flux model to run by default?
- Resolution limits: max image size?
- CORS/headers for serving generated images back to the UI

### Tests:

- Mock image gen service, verify proxy routing
- Verify `hardware.py` probes image gen endpoint
- Verify node_agent registers with `image_gen` capability
- Verify 429/503 handling when service is busy or down

### Status: ✅ Backend Complete (ComfyUI install pending on jarvis — single-node deployment)

---

## TASK 2 — Context-Aware Cluster Routing

**Goal:** Make chat/completions requests route to the best node based on available context capacity, not just model name matching. Solve the core problem: GPU nodes have limited VRAM context, but requests (especially from IDE integrations) can exceed that.

### Requirements:

1. **Node capacity reporting** — node_agent calculates and reports `effective_context_tokens` on registration based on (available RAM/VRAM − model weight size) → KV cache capacity. Updated on model swaps.
2. **Coordinator tracks cluster capacity** — `CLUSTER_NODES` stores `effective_context_tokens` per node, updated via registration and `model_ready` events.
3. **Context budget estimation** — before sending upstream, estimate the token count of the assembled message array (~4 chars/token heuristic, or tiktoken if available). Expose as a helper in `config.py` or `rag.py`.
4. **Resource-aware `select_node()`** — `triage.py` weighs effective context capacity, current load, and model match when picking a node. Nodes that can't fit the estimated context are deprioritized or skipped.
5. **Wire triage into chat router** — `routers/chat.py` calls `get_inference_url(user_message)` instead of hardcoding `LLAMA_SERVER_BASE`.
6. **Wire triage into completions router** — `routers/completions.py` uses the same routing logic for IDE/Continue.dev sessions.
7. **Graceful degradation** — if no node can fit the estimated context, truncate intelligently (drop oldest RAG chunks, summarize history) before falling back to the coordinator.

### Architecture:

```
User request → build messages → estimate tokens → triage.select_node(effective_context)
  → node with enough headroom → stream response
  → no node fits → truncate context → coordinator fallback
```

### Tests:

- Mock CLUSTER_NODES with varying effective_context_tokens, verify select_node picks the right one
- Mock message arrays of different sizes, verify token estimation
- Verify chat router calls get_inference_url instead of hardcoding
- Verify fallback truncation when no node fits
- Verify model swap updates effective_context_tokens

### Status: Not started

---

## TASK 3 — RAM-Based Context Store Node (blue-sky)

**Goal:** Enable a RAM-heavy node (e.g. a workstation with 32GB+ RAM) to join the cAIc cluster as a dedicated context store — holding conversation histories, RAG results, uploaded documents, and memories in RAM for fast retrieval, without running inference.

### Requirements:

1. **New node type: `context_store`** — registers with a `context_store` capability, advertises available RAM and current usage
2. **Lightweight context service** — HTTP API on the context store node exposing:
   - `POST /context/{session_id}` — store conversation context
   - `GET /context/{session_id}` — retrieve full context
   - `PUT /context/{session_id}/chunks` — update RAG/document chunks
   - `GET /context/{session_id}/relevant?q=...` — ranked context retrieval
3. **Coordinator integration** — `build_system_prompt()` pulls from the context store node instead of (or in addition to) SQLite when one is available
4. **Context store discovery** — node_agent supports `context_store` type in config, coordinator queries available RAM on registration
5. **Failover** — if context store is unreachable, fall back to local SQLite/Qdrant as today

### Architecture:

```
Coordinator startup → discover context_store nodes → query available RAM
Build system prompt → pull relevant context from context_store node → assemble → send to inference node
```

### Considerations:

- Context store node needs minimal resources — Python + aiohttp + Redis or in-memory dict
- Latency: LAN round-trip (~1ms) is negligible compared to inference time
- Persistence: optional — RAM-only is fine if the store can repopulate from SQLite on restart
- Security: context store holds unencrypted data locally (encryption stays at the coordinator layer)

### Tests:

- Mock context store node registration, verify coordinator discovers it
- Mock context store HTTP responses, verify build_system_prompt pulls from it
- Verify failover to local SQLite when context store is unreachable

### Status: Not started (blocked on available hardware — Dell Precision Tower 3420 dead, NUC running Home Assistant OS)

---