add FAQ page covering common questions

FAQ.md

@@ -0,0 +1,71 @@
+# Frequently Asked Questions
+
+## Why not just use Ollama?
+
+Ollama is a single-machine runner. It does not distribute inference across machines. If your model doesn't fit on one GPU, Ollama can't help — it doesn't support tensor parallelism across hosts, RPC offload, or any form of multi-node clustering. cAIc is for people who have *multiple machines* with different GPUs and want them to cooperate.
+
+Ollama is also a model runner first and a chat UI second. cAIc is a full web application with conversation history, RAG, memory, user profiles, skills, web search, and cluster orchestration. They solve different problems.
+
+## How is this different from llama.cpp RPC?
+
+llama.cpp RPC splits *one model* across multiple GPUs — GPU 0 runs layers 0–15, GPU 1 runs layers 16–31, etc. This is **tensor parallelism** and it has a lockstep problem: every forward pass must wait for the slowest GPU to finish its layers. In a heterogeneous cluster (AMD + NVIDIA, different VRAM sizes, different memory bandwidth), the fast GPU spends most of its time waiting on the slow one.
+
+cAIc uses **query-routing**. Each worker runs a complete model on its own GPU. Queries are classified and routed to the best-matching worker. No layer sharing, no lockstep, no straggler problem. The tradeoff is that no single query can use combined VRAM across GPUs — but your cluster throughput stays high because every GPU runs independently.
+
+| | llama.cpp RPC | cAIc |
+|---|---|---|
+| Splits one model across GPUs? | Yes (tensor parallel) | No (each worker runs full model) |
+| Heterogeneous GPUs? | Slowed by weakest link | Each GPU independent |
+| Requires fast fabric? | Yes (NCCL / RDMA) | No (LAN is fine) |
+| Single query can use combined VRAM? | Yes | No |
+| Cluster throughput capped by slowest GPU? | Yes | No |
+
+## How is this different from vLLM?
+
+vLLM is designed for datacenter deployments with homogeneous GPU clusters and high request volume. It assumes identical GPUs, CUDA-only, and fast interconnects. It does not support AMD ROCm in any meaningful way, nor does it handle a two-machine homelab with one NVIDIA and one AMD card.
+
+cAIc targets the opposite use case: a handful of mismatched machines on a LAN, where ease of setup matters more than maximum throughput.
+
+## How is this different from exo?
+
+exo does zero-config peer discovery across mixed hardware including Apple Silicon, but its throughput is limited — it uses the same layer-splitting approach that cAIc avoids, and it has no persistent state (no conversation history, no RAG, no user profiles). exo is also experimental in ways that cAIc is not: network errors can cascade into full process restarts, and there's no admin model for access control.
+
+cAIc trades exo's zero-config discovery for a more structured AMQP-mediated registration protocol, but gains persistent storage, access control, RAG, web search integration, and a production-style web UI.
+
+## How is this different from a Kubernetes operator (LLMKube)?
+
+LLMKube is a Kubernetes-native operator for heterogeneous GPU fleets. It's the right choice if you already run Kubernetes. But if you're a homelab user with two or three machines, Kubernetes is a massive dependency — you're managing a control plane, etcd, CNI, and all the operational overhead of a container orchestrator just to run inference.
+
+cAIc is a single FastAPI process with a SQLite database and an AMQP connection. No Kubernetes, no Docker required (though Docker compose is planned for v1.0). You can install it on bare Debian in five minutes.
+
+## Does cAIc support Apple Silicon?
+
+Not yet, but there's nothing preventing it. llama.cpp has a Metal backend that exposes the same OpenAI-compatible API that cAIc already uses. A worker on an M-series Mac would look identical to the coordinator: it registers via AMQP, responds to pings, and serves inference requests. The changes needed are just platform-detection branches in `gpu.py`, `hardware.py`, and `node_agent/agent.py` — no architectural changes. This is on the backlog as B7.
+
+## Can I run the coordinator and a worker on the same machine?
+
+Yes. Nothing prevents colocation. The coordinator uses minimal resources (it does no model inference), so there's no meaningful conflict. Just run `llama-server` alongside `uvicorn`.
+
+## Do I need RabbitMQ for a single-machine setup?
+
+No. cAIc runs fine without AMQP — the cluster features simply won't activate. All core features (chat, RAG, memory, search, upload) work with just `llama-server` and optionally Qdrant/SearXNG. RabbitMQ is only needed when you add a second machine as a worker.
+
+## What models work with cAIc?
+
+Any GGUF model that `llama-server` can load. The default is `qwen2.5-7b-instruct`, but you can swap to any model by changing `DEFAULT_MODEL` in `config.py` and pointing `llama-server` at a different GGUF file. Models with "coder" or "qwen" in the name are preferred for code queries; "mistral" or "llama" are preferred for general chat.
+
+## Can I expose cAIc to the internet?
+
+Technically yes, but it's not the target deployment. cAIc is designed for local and trusted-LAN operation. If you must expose it, put a reverse proxy (Caddy, nginx) in front with HTTPS, set strong admin credentials, restrict the IP allowlist, and enable `CAIC_TRUST_X_FORWARDED_FOR`.
+
+## What happens if a worker goes offline?
+
+The coordinator detects failure via ping timeout (5s) and auto-deregisters the node. New queries are routed to remaining workers. If no workers are available, the coordinator returns an error. Workers can rejoin at any time by re-registering — no manual intervention needed.
+
+## Does cAIc support OpenAI-compatible API?
+
+Yes. `/v1/chat/completions` is available for IDE integration (Continue.dev, etc.), authenticated via `CAIC_COMPLETIONS_API_KEY`. Fill-in-the-middle (FIM) requests are proxied directly without persistence. Chat-style requests are persisted to conversation history.
+
+## How many tests does cAIc have?
+
+179 tests across 23 test files, all passing. Every router has a dedicated test file with mocked external services. No live dependencies needed — tests use `tmp_path` fixtures and monkeypatched `httpx.AsyncClient`.

Home.md

@@ -12,6 +12,7 @@ This wiki is the developer-facing architecture and process reference for cAIc.
 - [Installation & Configuration](Installation) — setup guide, bare metal, Docker, cluster, troubleshooting
 - Architecture and components: [Developer Architecture](Developer-Architecture)
 - Active implementation backlog: [Current WiP](Current-WiP)
+- [FAQ](FAQ) — "why not just use Ollama?", query-routing vs layer-splitting, and other common questions
 - [Screenshots](Screenshots) — gallery of the UI in action
 
 ## Scope and Support Model