Apple silicon Macs can now run frontier-class large language models with nearly a trillion parameters locally, according to research released by the company. The MLX framework, designed for Apple's unified memory architecture, enables researchers to experiment with advanced AI techniques on their own hardware without relying on cloud services.

The system leverages the Neural Accelerators in the M5 Ultra chip, which provide dedicated hardware for matrix multiplication operations critical to machine learning workloads. In benchmark tests, a cluster of four Mac Studios equipped with M5 Ultra chips generated over 36 tokens per second when running the Muse Spark Open model, a mixture-of-experts architecture with 816 billion parameters.

Performance varies based on configuration. A single Mac Studio with 512GB of unified memory can run Qwen 3.8 (27B parameters) and Mistral Medium 3.5 (128B parameters) at context sizes up to 65,536 tokens. The M5 Ultra's 1.2 terabytes per second of unified memory bandwidth — 46% higher than the M3 Ultra — directly improves generation speed, achieving 1.6 times faster token output in testing.

Power consumption remains modest, with peak usage around 300 watts during heavy inference workloads — comparable to a household appliance rather than a server rack. This efficiency allows deployment in standard office or home environments.

Very large models like Muse Spark Open require distributed processing across multiple machines. MLX supports two parallelization methods: pipeline parallelism, where layers are split sequentially across devices, and tensor parallelism, where individual layers are distributed across machines. The latter requires low-latency communication, which MLX achieves using RDMA over Thunderbolt 5. In tests, connecting Mac Studios via dual Thunderbolt cables reduced time-to-first-token by 19% compared to single-cable configurations.

The framework also enables practical applications, such as running local AI agents for coding tasks. Developers can launch MLX servers using the mlx_lm.server tool and integrate them with frameworks like OpenCode, which can query the local model via a configurable endpoint.

These capabilities stem from a combination of hardware and software innovations introduced over the past year, including Neural Accelerator support, distributed communication protocols, and optimized handling of mixture-of-experts architectures. The convergence of these features on the M5 Ultra platform allows local execution of models previously requiring dedicated server infrastructure.