Run DeepSeek R1 or V3 with MLX Distributed

from blog Simon Willison's Weblog, | ↗ original
Run DeepSeek R1 or V3 with MLX Distributed Handy detailed instructions from Awni Hannun on running the enormous DeepSeek R1 or v3 models on a cluster of Macs using the distributed communication feature of Apple's MLX library. DeepSeek R1 quantized to 4-bit requires 450GB in aggregate RAM, which can be achieved by a cluster of three 192 GB M2...