News Score: Score the News, Sort the News, Rewrite the Headlines

A Distributed File System in Go Cut Average Metadata Memory Usage to 100 Bytes

TL;DR: JuiceFS, written in Go, can manage tens of billions of files in a single namespace. Its metadata engine uses an all-in-memory approach and achieves remarkable memory optimization, handling 300 million files with 30 GiB of memory and 100 microseconds response time. Techniques like memory pools, manual memory management, directory compression, and compact file formats reduced metadata memory usage to 100 bytes per file. JuiceFS Enterprise Edition, a cloud-native distributed file system writ...

Read more at juicefs.com

© News Score  score the news, sort the news, rewrite the headlines