Qwen3.6-35B-A3B — CPU inference
Measured by AI Stays Local10.762 sCold start7.215 sTime to first token3.68 tok/sGeneration (steady state)31.7 GiBResident set size
4 / 4Instruction following
- Date
- 2026-09
- Hardware
- NVIDIA DGX Spark
- Operating system
- Linux, ARM64
- Execution
- CPU inference, 16 threads
- Model
- Qwen3.6-35B-A3B
- Parameters
- 35B total, 3B active per token
- Expert handling
- Resident expert cache
- Runtime
- AI Stays Local runtime, Colibrì engine
- Endpoint
- Loopback only; no deployment retained after the run
- Result
- A 35B-total Mixture-of-Experts model executed on CPU on a single machine we own, with 3B active per token and the frequently used experts held resident.
- Limitations
- This throughput suits validation and patient work — batch jobs, long documents, questions you are willing to wait for. It is not an instant consumer chat experience, and we are not going to describe it as one. GPU acceleration for this class of model is in validation.
OLMoE 7B-A1B — local inference baseline
Measured by AI Stays LocalOperationalLocal inferenceOperationalOpenAI-compatible APIOperationalStreaming responses
- Date
- 2026-09
- Hardware
- NVIDIA DGX Spark
- Operating system
- Linux, ARM64
- Execution
- CPU inference
- Model
- OLMoE
- Parameters
- 7B total, 1B active per token
- Runtime
- AI Stays Local runtime
- Endpoint
- Loopback only
- Result
- The baseline profile. Local inference, the compatible API and streaming all work against a small MoE model on the same machine.
- Limitations
- Timing figures for this profile are not published here yet; the functional results above are what this record establishes. A small model is good at everyday drafting and document questions and will lose to a large hosted model on hard reasoning.
Local documents — coverage and refusal
Measured by AI Stays LocalPASSCovered question — answered with citationPASSUncovered question — correctly refused
- Date
- 2026-09
- Hardware
- NVIDIA DGX Spark
- Operating system
- Linux, ARM64
- Retrieval
- BM25 lexical, local index
- Model
- Qwen3.6-35B-A3B
- Runtime
- AI Stays Local runtime, Colibrì engine
- Corpus
- Cedar internal document set
- Result
- Asked something the corpus covers, it answered and cited the source. Asked something the corpus does not cover, it declined rather than inventing an answer. The second result is the one that matters.
- Limitations
- Retrieval is lexical: BM25 matches words, not meaning, so a question phrased in vocabulary the document does not use can miss. Semantic retrieval is on the roadmap. The local index is not encrypted at rest today.