Under 8 GB of memory
Below the range we are designing for. A very small model may load, but the experience is not one we would put in front of a professional.
Document search across a large folder is unlikely to be comfortable.
Will it run on your machine?
Honestly: it depends, and anyone who tells you otherwise has not tried. Memory is usually the limiting factor, because the model has to fit in it. Here is how the bands work.
Memory
8 to 16 GB
A small local model for everyday questions, drafting and summarising, plus document search over a moderate set of folders.
Long documents and long conversations will be slower, and the largest local models are out of reach.
16 to 32 GB
The range we are targeting for the private beta. A capable mid-size model with room for a real document index.
Heavy multitasking while a model is loaded will still be felt.
32 GB and above
Larger models, faster answers, and document collections measured in thousands of files rather than hundreds.
More memory helps up to a point; the processor and memory bandwidth then become the limit.
Systems
macOS on Apple Silicon
Where the prototype has been developed and where local inference has been exercised so far. The first private beta build targets this platform.
In developmentWindows
Planned. Recent 64-bit machines with a modern processor, with performance depending heavily on memory and on whether a supported graphics accelerator is present.
PlannedLinux
Planned, and the natural home for the private-server and command-line side of the product.
PlannedmacOS on Intel
Under evaluation. Technically possible, but slower, and we would rather say nothing than promise a result we have not measured.
Under evaluationiOS and Android
Planned as a private connection to the machine at home or in the office, not as a model running on the phone itself.
PlannedTell us what you have
The beta application asks about your machine, and that is the single most useful thing in it. The spread of hardware in the first group decides which platforms and which model sizes we work on first, so an application from an unusual machine is more valuable to us than one from a typical one.