Hello, I am interesting in deploying a high availability postgres cluster with the planet OSM database to Fly.io. Disk requirements for the planet wide osm database is currently around 1.5TB. Unfortunately, it seems that the max disk space you can allocate to a postgres instance on fly is 500GB.
Is 500GB a hard limit? Is there any way to build a flyio/postgres instance with 2TB of disk space?
Actually, it looks like the solution is to build an initial postgres cluster, then manually extend the pg_data volume beyond the initial limit. Any risk of doing so? Besides the cost?
Hi @Helmut
If postgres itself isn’t a hard requirement then you could try using a CockroachDB cluster. It’s postgres-like and supports distributing data across servers so the 500gb limit is not as much of an issue.
CockroachDB has a guide for importing osm data that you can find here Migrate from OpenStreetMap
@Helmut, I didn’t get to far to be honest.
Theoretically you should be able to create the initial cluster, then expand the volume size, as I suggested, but be very careful. Fly’s volume pricing is really not suited for big data, especially with clustering and mirroring.
If you need 1000GB of storage for OSM, the cost would be $150/month in storage - for just one instance. If you are running a cluster, multiply by cluster size. If you are mirroring in different locations, multiply by mirror size.
There are cheaper ways of serving maps - planetiler and pmtiles
@Ty1 Thanks for the reply. Guess I’ll have to stock with RDS (we build tiles from business records).