gramps/cAIc
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

fix: align ASCII architecture diagram box-drawing characters

Browse Source
This commit is contained in:
gramps
2026-07-19 16:56:01 -07:00
parent 5b5fbab206
commit 70014f8e3b
1 changed files with 21 additions and 21 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+21 -21
View File
@@ -19,27 +19,27 @@ You have more aggregate compute than any single consumer machine. The software j
cAIc uses **query-routing** instead of layer-splitting. Each machine runs a complete model on its own GPU. When a query comes in, the coordinator classifies it and routes the *whole request* to the best-suited node — code questions to the coder model, general chat to the instruct model. No layer sharing, no straggler problem, no VRAM negotiation between mismatched GPUs. cAIc uses **query-routing** instead of layer-splitting. Each machine runs a complete model on its own GPU. When a query comes in, the coordinator classifies it and routes the *whole request* to the best-suited node — code questions to the coder model, general chat to the instruct model. No layer sharing, no straggler problem, no VRAM negotiation between mismatched GPUs.
``` ```
┌─────────────────────────────────────────────────┐ ┌──────────────────────────────────────────────────────┐
│ docker compose stack │ │ docker compose stack │
│ │ │ │
│ ┌──────────┐ ┌────────┐ ┌──────────────────┐ │ │ ┌──────────┐ ┌────────┐ ┌────────────────────┐ │
│ │ SearXNG │ │ Qdrant │ │ RabbitMQ │ │ │ │ SearXNG │ │ Qdrant │ │ RabbitMQ │ │
│ │ :8888 │ │ :6333 │ │ :5672/:15672 │ │ │ │ :8888 │ │ :6333 │ │ :5672 / :15672 │ │
│ └────┬─────┘ └───┬────┘ └────────┬─────────┘ │ │ └────┬─────┘ └───┬────┘ └─────────┬──────────┘ │
│ │ │ │ │ │ │ │ │ │
│ ▼ ▼ ▼ │ │ ▼ ▼ ▼ │
│ ┌──────────────────────────────────────────┐ │ │ ┌───────────────────────────────────────────────┐ │
│ │ cAIc (FastAPI) │ │ │ │ cAIc (FastAPI) │ │
│ │ :8080 (HTTP) │ │ │ │ :8080 (HTTP) │ │
│ └──────┬──────────────┬───────────────────┘ │ │ └───────┬──────────────────┬────────────────────┘ │
│ │ │ │ │ │ │ │
│ ▼ ▼ │ │ ▼ ▼ │
│ ┌──────────────┐ ┌──────────────┐ │ │ ┌────────────────┐ ┌────────────────┐ │
│ │ llama-server │ │ Ollama │ │ │ │ llama-server │ │ Ollama │ │
│ │ :8081 │ │ :11434 │ │ │ │ :8081 │ │ :11434 │ │
│ │ (GPU/RPC) │ │ (embeddings) │ │ │ │ (GPU/RPC) │ │ (embeddings) │ │
│ └──────────────┘ └──────────────┘ │ │ └────────────────┘ └────────────────┘ │
└─────────────────────────────────────────────────┘ └──────────────────────────────────────────────────────┘
``` ```
**Coordinator** (CPU-only, no GPU) handles the web UI, RAG embedding, query triage, web search, memory, conversation storage, and the message broker. Every CPU-bound task stays here so it never competes with inference for GPU resources. **Coordinator** (CPU-only, no GPU) handles the web UI, RAG embedding, query triage, web search, memory, conversation storage, and the message broker. Every CPU-bound task stays here so it never competes with inference for GPU resources.