News Score: Score the News, Sort the News, Rewrite the Headlines

Taking out the Trash: Garbage Collection of Object Storage at Massive Scale

Over the last 10 years, I’ve built several distributed systems on top of object storage, with WarpStream being the most recent. One consistent factor across all of these systems is how much time we spent solving what seems like a relatively straightforward problem: removing files from object storage that had been logically deleted either due to data expiry or compaction.I discussed this in more detail in “The Case for Shared Storage” blog post, but to briefly recap: every shared storage system I...

Read more at warpstream.com

© News Score  score the news, sort the news, rewrite the headlines