Published September 1, 2026 |Updated September 2, 2026 In the beginning, there was mmap. It was convenient: it let us lazily read huge numbers of Arrow IPC files from disk without managing memory ourselves. It fit our file format perfectly — Arrow IPC’s layout is designed for zero-copy random access, and mmap gives you exactly that. Then we deployed to production, ran real concurrent query loads, and mmap became a real problem. Our Workload At Conviva, we analyze trillions of even...