[v0.12.2] Deepseek R1 Distill 8B Q40 on 4 x Raspberry Pi 5 8GB · b4rtaz/distributed-llama · Discussion #162
Model: deepseek_r1_distill_llama_8b_q40
Version: 0.12.2
Evaluation
Prediction
2 x Raspberry Pi 5 8GB
7.70 tok/s
3.54 tok/s
4 x Raspberry Pi 5 8GB
11.68 tok/s
6.43 tok/s
2 x Raspberry Pi 5 8GB
...
🔶 P 278 ms S 288 kB R 522 kB First
🔶 P 258 ms S 288 kB R 522 kB ,
🔶 P 323 ms S 288 kB R 522 kB I
🔶 P 275 ms S 288 kB R 522 kB need
🔶 P 293 ms S 288 kB R 522 kB to
🔶 P 269 ms S 288 kB R 522 kB understand
🔶 P 281 ms S 288 kB R 522 kB what
Evaluation
nBatches:...
Read more at github.com
© News Score score the news, sort the news, rewrite the headlines