News Score: Score the News, Sort the News, Rewrite the Headlines

GitHub - carloslfu/slotstream: Run Qwen3.8-Flash-Next (125B MoE, 104 GB at 4-bit) on Macs with a fraction of that RAM by streaming experts from SSD. MLX + Swift, Ollama-compatible API.

Run Qwen3.8-Flash-Next on a Mac that cannot hold it. The model is 104 GB at 4-bit; slotstream streams it from SSD and runs it in whatever memory you give it, down to an 8.1 GB planned floor. One Swift binary with the commonly used Ollama and OpenAI chat/generate endpoints. on a 48 GB Mac Warm decode ~12 tok/s Cold start to first token ~3 s Peak memory 32 GB (auto-sized; you can cap it) Weights on disk 104 GB Will it run on my Mac Disk is the gate that bites first. You need ~110 GB free, so a 512...

Read more at github.com

© News Score  score the news, sort the news, rewrite the headlines