Kimi K2 1-trillion parameter AI model runs at usable speed on two 512GB M3 Ultra chips using 4-bit quantization and mlx-lm distributed computing

Awni Hannun on X: "The new Kimi K2 1T model (4-bit quant) runs on 2 512GB M3 Ultras with mlx-lm and mx.distributed. 1 trillion params, at a speed that's actually quite usable: https://t.co/FaP4yFFjtU" / X

PostConversationAwni Hannun@awnihannunThe new Kimi K2 1T model (4-bit quant) runs on 2 512GB M3 Ultras with mlx-lm and mx.distributed. 1 trillion params, at a speed that's actually quite usable:The media could not be played.QuoteKimi.ai@Kimi_MoonshotJul 11 Hello, Kimi K2! Open-Source Agentic Model! 1T total / 32B active MoE model SOTA on SWE Bench Verified, Tau2 & AceBench among open models Strong in coding and agentic tasks Multimodal & thought-mode not supported for now With Kimi K2, advanced...