ByteDance unveils USDB storage system for recommendation engines, cutting write latency 85.8% and infrastructure costs 42.1% through unified sequence storage and CascadeLSM compaction.

USDB: Scaling Storage for Next-Generation Recommender Systems at ByteDance

Authors:Xiaoran Zhu, Jieming Cai, Weiping Yu, Xi Chen, Shu Cai, Jiahao Li, Yifan Xiao, Yan Deng, Yisha Zhou, Yu Lin, Shiru Ren, Siqiang Luo, Peng Xu, Junjun Chai View PDF HTML (experimental) Abstract:Recommendation models consume increasingly long, attribute-rich user histories, making small continuous updates and repeated large reads a central storage asymmetry. We present USDB, ByteDance's unified sequence storage practice for streaming and batch workloads. CascadeLSM time-based compaction sta...