cAIc/TASKS.md

cAIc — Task List (v1.0+)

Previous task history archived at docs/archive/TASKS-pre-1.0.md.

---

TASK 1 — Image Generation Service (corsair)

Goal: Add image generation as a cluster capability. corsair (RTX 5070 Ti, 16 GB) registers as an image gen worker in the cAIc cluster.

Requirements:

1. Add "image_gen" capability to the cluster protocol in cluster.py — valid capability values should include image_gen
2. Image gen API wrapper on corsair — run ComfyUI, Automatic1111, or a lightweight API (e.g., sd-api or comfyui-api) that exposes a simple POST /generate endpoint accepting a prompt and returning a PNG
3. Proxy endpoint in cAIc — POST /api/image/generate on the coordinator, routes the request to corsair's image gen service via AMQP or direct HTTP
4. Update hardware.py to probe the image gen service for reachability and status
5. Update node_agent to report image gen capability and service status on registration

Architecture:

User prompt → cAIc coordinator → AMQP/HTTP → corsair (ComfyUI/API) → PNG → coordinator → user

Considerations:

• Response time: expect 5-30 seconds per image depending on model/resolution
• Queue management: what if multiple requests come in at once?
• Model selection: which SD/Flux model to run by default?
• Resolution limits: max image size?
• CORS/headers for serving generated images back to the UI

Tests:

• Mock image gen service, verify proxy routing
• Verify hardware.py probes image gen endpoint
• Verify node_agent registers with image_gen capability
• Verify 429/503 handling when service is busy or down

Status: ✅ Backend Complete (ComfyUI install pending on corsair)

---

TASK 2 — Context-Aware Cluster Routing

Goal: Make chat/completions requests route to the best node based on available context capacity, not just model name matching. Solve the core problem: GPU nodes have limited VRAM context, but requests (especially from IDE integrations) can exceed that.

Requirements:

1. Node capacity reporting — node_agent calculates and reports effective_context_tokens on registration based on (available RAM/VRAM − model weight size) → KV cache capacity. Updated on model swaps.
2. Coordinator tracks cluster capacity — CLUSTER_NODES stores effective_context_tokens per node, updated via registration and model_ready events.
3. Context budget estimation — before sending upstream, estimate the token count of the assembled message array (~4 chars/token heuristic, or tiktoken if available). Expose as a helper in config.py or rag.py.
4. Resource-aware select_node() — triage.py weighs effective context capacity, current load, and model match when picking a node. Nodes that can't fit the estimated context are deprioritized or skipped.
5. Wire triage into chat router — routers/chat.py calls get_inference_url(user_message) instead of hardcoding LLAMA_SERVER_BASE.
6. Wire triage into completions router — routers/completions.py uses the same routing logic for IDE/Continue.dev sessions.
7. Graceful degradation — if no node can fit the estimated context, truncate intelligently (drop oldest RAG chunks, summarize history) before falling back to the coordinator.

Architecture:

User request → build messages → estimate tokens → triage.select_node(effective_context)
  → node with enough headroom → stream response
  → no node fits → truncate context → coordinator fallback

Tests:

• Mock CLUSTER_NODES with varying effective_context_tokens, verify select_node picks the right one
• Mock message arrays of different sizes, verify token estimation
• Verify chat router calls get_inference_url instead of hardcoding
• Verify fallback truncation when no node fits
• Verify model swap updates effective_context_tokens

Status: Not started

---

TASK 3 — RAM-Based Context Store Node (blue-sky)

Goal: Enable a RAM-heavy node (e.g. a workstation with 32GB+ RAM) to join the cAIc cluster as a dedicated context store — holding conversation histories, RAG results, uploaded documents, and memories in RAM for fast retrieval, without running inference.

Requirements:

1. New node type: context_store — registers with a context_store capability, advertises available RAM and current usage
2. Lightweight context service — HTTP API on the context store node exposing:
   • POST /context/{session_id} — store conversation context
   • GET /context/{session_id} — retrieve full context
   • PUT /context/{session_id}/chunks — update RAG/document chunks
   • GET /context/{session_id}/relevant?q=... — ranked context retrieval
3. Coordinator integration — build_system_prompt() pulls from the context store node instead of (or in addition to) SQLite when one is available
4. Context store discovery — node_agent supports context_store type in config, coordinator queries available RAM on registration
5. Failover — if context store is unreachable, fall back to local SQLite/Qdrant as today

Architecture:

Coordinator startup → discover context_store nodes → query available RAM
Build system prompt → pull relevant context from context_store node → assemble → send to inference node

Considerations:

• Context store node needs minimal resources — Python + aiohttp + Redis or in-memory dict
• Latency: LAN round-trip (~1ms) is negligible compared to inference time
• Persistence: optional — RAM-only is fine if the store can repopulate from SQLite on restart
• Security: context store holds unencrypted data locally (encryption stays at the coordinator layer)

Tests:

• Mock context store node registration, verify coordinator discovers it
• Mock context store HTTP responses, verify build_system_prompt pulls from it
• Verify failover to local SQLite when context store is unreachable

Status: Not started (blocked on available hardware — Dell Precision Tower 3420 dead, NUC running Home Assistant OS)

---