How To Run Deepseek R1 671b Fully Locally On a $2000 EPYC Server – Digital Spaceport
Deepseek Ai Rig Build for Local Inference
Let’s start with the good news. I got very solid performance off the same baseline AMD EPYC Rome system that has been at the core of our entire journey 😁 That initial parts selection has remained fantastic! Owners of that system are going to get some great news today also as they can hit between 4.25 to 3.5 TPS (tokens per second) on the Q4 671b full model. This is important as the distilled versions are simply not the same at all. They are vastly infer...
Read more at digitalspaceport.com