Docs: Docker - AI Stays Local

The container image is planned and has not been published. This page describes how it is designed to work: the same runtime in a container, with explicit volumes, explicit limits and a loopback port. Nothing here can be pulled today.

How it is put together

Application image	Immutable. Updating means running a new tag; rolling back means running the previous one.
Model volume	Weights live in a volume you control, so a container rebuild does not re-download gigabytes.
Document mounts	Mounted read-only. The runtime can read what you grant and cannot write to it.
Index volume	The local index is a separate volume, so you can remove it without touching your documents.
Loopback port	Bound to localhost by default. Exposing it on a network is a decision you make explicitly.
Optional NVIDIA runtime	GPU where it is available and validated; CPU mode otherwise.
Health endpoint	So an orchestrator can tell a slow start from a failed one.
Resource ceilings	Memory and CPU limits, because an inference process without a ceiling will find one.

Running it

Run the runtime with your models and documents

docker

docker run -d --name aisl \
  -p 127.0.0.1:5000:5000 \
  -v aisl-models:/models \
  -v aisl-index:/data \
  -v /path/to/documents:/documents:ro \
  --memory 32g --cpus 16 \
  aistayslocal/runtime:<tag>

With a supported NVIDIA accelerator

docker

docker run -d --gpus all \
  ...same volumes and limits... \
  aistayslocal/runtime:<tag>

Design preview. The image has not been published, so none of these commands can be run yet.

Is it ready?

health

curl -fsS http://127.0.0.1:5000/health

A large model takes time to become ready. /health answers as soon as the process is up and reports whether a model is loaded, so an orchestrator can wait rather than restart. The OpenAI-compatible surface is served under /v1 on the same port and expects an authentication token that is generated on, and stays on, the machine running it.

Where it runs

Linux ARM64	The validated platform. Measurements on this site were taken here.
Linux x86-64	Target for the same image. Under validation.
NVIDIA GPU	Where the accelerator is present and the profile has been validated.
CPU mode	Available wherever the container runs; speed depends on the machine.
macOS via Docker Desktop	Containers on macOS cannot reach Metal acceleration, so a container is CPU-bound there. The desktop app is the right route on a Mac.
Windows via WSL2	Subject to validation.