News Score: Score the News, Sort the News, Rewrite the Headlines

[v0.12.2] Deepseek R1 Distill 8B Q40 on 4 x Raspberry Pi 5 8GB ยท b4rtaz/distributed-llama ยท Discussion #162

Model: deepseek_r1_distill_llama_8b_q40 Version: 0.12.2 Evaluation Prediction 2 x Raspberry Pi 5 8GB 7.70 tok/s 3.54 tok/s 4 x Raspberry Pi 5 8GB 11.68 tok/s 6.43 tok/s 2 x Raspberry Pi 5 8GB ... ๐Ÿ”ถ P 278 ms S 288 kB R 522 kB First ๐Ÿ”ถ P 258 ms S 288 kB R 522 kB , ๐Ÿ”ถ P 323 ms S 288 kB R 522 kB I ๐Ÿ”ถ P 275 ms S 288 kB R 522 kB need ๐Ÿ”ถ P 293 ms S 288 kB R 522 kB to ๐Ÿ”ถ P 269 ms S 288 kB R 522 kB understand ๐Ÿ”ถ P 281 ms S 288 kB R 522 kB what Evaluation nBatches:...

Read more at github.com

© News Score  score the news, sort the news, rewrite the headlines