Developer builds 7-node ESP32-S3 cluster to run 0.5B language model using 1.58-bit BitNet quantization over SPI daisy-chain architecture

GitHub - Low-Zi-Hong/ESP32s3-LLM-Cluster: 7-node ESP32-S3 cluster running a 0.4B LLM via 1.58-bit (BitNet) ternary quantization over SPI daisy-chain

A distributed pipeline inference engine on multiple ESP32S3 running 1.58-bit (BitNet) Language model. Architecture This project runs a sliced 0.5B LLM across a cluster of 7 ESP32s3. One act as master and others are node. The master node runs the tokenizer and embeding and the other attention layer and MLP ran on the nodes. The master and node communicate through high speed SPI Daisy-Chain. ┌─────────────────────────────────────────────────────────┐ │ MASTER NODE ...