Skip to content

Mac (Apple Silicon) and MLX ​

Enterprise module

Mac hosts and MLX are part of the GPU Fleet module (gpu-fleet) of your Cognipeer package. If the module is not in your package, the GPU Fleet pages show an upgrade notice instead.

On macOS, Docker containers run inside a Linux virtual machine that cannot reach the Apple GPU. A model that runs in a container on a Mac runs on the CPU, several times slower than the same model on MLX. To use the GPU, deploy the model with the MLX runtime. The agent runs MLX models natively on the Mac (mlx_lm.server, as a launchd job), without Docker.

Prerequisites ​

ComponentRequired?Note
Apple Silicon (M1 to M4)YesMLX is not available on Intel Macs.
Python 3.11+ or uvYes, for MLXbrew install uv is enough. The installer can install uv for you when you pass --auto-install-prereqs.
Docker DesktopNoOnly for Docker runtimes (Ollama, custom images). They run on the CPU.

On the first MLX deployment the agent creates a Python virtual environment and installs mlx-lm once. The Run a model flow shows whether MLX is ready on the machine and what is missing.

Choosing a model ​

When you run a model on an Apple Silicon machine (see Deploying models):

  • MLX runs on the Apple GPU and is the recommended runtime. The catalog lists ready MLX models, and the Hugging Face search also finds mlx-community/... conversions (for example mlx-community/Qwen3-8B-4bit).
  • Docker runtimes are listed with a warning that they run on the CPU.

4-bit conversions fit memory best: roughly 5 to 6 GB for an 8B model, about 18 GB for a 30B-A3B model and about 40 GB for a 70B model.

Memory ​

On a Mac, RAM and GPU memory are shared, but the GPU cannot use all of it. The agent reports two numbers separately:

  • GPU share: the memory Metal allows the GPU to use (the iogpu.wired_limit_mb setting when it is set, otherwise the value MLX reports, otherwise about two thirds to three quarters of RAM). The machine page shows this as its slice, and whether an MLX model fits is worked out against it.
  • Docker VM memory: the memory given to the Docker Desktop virtual machine. Whether a Docker runtime fits is worked out against it.

Model files ​

Downloaded weights are kept under COGNIPEER_MODEL_CACHE_DIR (default <state dir>/models). A restart does not download the model again. When a deployment is deleted, its files are deleted too unless another deployment uses them. The first download shows its progress in the deployment's timeline.

Network ​

MLX models listen on 127.0.0.1 only. The console reaches them through the agent gateway or the relay; for a laptop, Relay is the easiest mode. See Networking and connection modes.

The agent service ​

On a Mac the agent is installed as a launchd service:

bash
launchctl print gui/$(id -u)/com.cognipeer.gpu-agent         # status
launchctl kickstart -k gui/$(id -u)/com.cognipeer.gpu-agent  # restart

You can also use Restart agent or Reinstall agent from the machine's menu in the console.

Planning ​

The Planner can size a Mac before you buy or use it: choose Apple Silicon and the amount of unified memory.

Studio · Pulse · Console · Agent SDK and more — the Cognipeer documentation hub