Building a weather data warehouse part I: Loading a trillion rows of weather data into TimescaleDB
Building a weather data warehouse part I: Loading a trillion rows of weather data into TimescaleDB
What are we even doing?
Why build a weather data warehouse?
What’s the data?
The insert statement
Starting with just the single-row insert statement
Multi-valued insert
The copy statement
Upgrading to the copy statement
Sustaining copy insert rates
Parallel copy
Tools
pg_bulkload and timescaledb-parallel-copy
Multiple workers with timescaledb-parallel-copy
Tweaking Postgres settings
So what’s the b...
Read more at aliramadhan.me