LiteFS & many (tens to hundreds of thousands of) SQLite DBs

Thanks for the info. If you don’t need databases materialized all the time since they’re getting infrequent reads then they could probably be stored compressed. At 500KB/day with a 30 day retention, that would mean the database would be 15MB compressed.

We haven’t determined pricing on the LiteFS cloud yet but using our volume pricing ($0.15/GB), that would put storage at ¢0.225 per database. For bandwidth costs, it’d be ¢0.03 per database so even if you replicated the whole database every day then it’d still cost less than a cent (¢0.9).

I added an issue (#261) to the LiteFS repo for materializing databases on demand. That could help reduce storage costs.

I also added an issue (#262) to introduce batch replication. The idea is that if you don’t need changes immediately replicated, then you could have all changes batched up every minute or hour so that it doesn’t have to send multiple copies of the same page.

1 Like