My PostgreSQL machine went down today with the following logs:
2023-07-31T01:34:28Z app[5683936f609d8e] ord [warn]Virtual machine exited abruptly
2023-07-31T01:44:37Z health[5683936f609d8e] ord [error]Health check for your postgres vm has failed. Your instance has hit resource limits. Upgrading your instance / volume size or reducing your usage might help.
I resized the volume and got the DB back up but I can’t figure out what limits I hit which means I don’t know what monitoring to set up to prevent this from happening again.
I’m seeing the same error this morning. I’m assuming it has to do with an auto update to v2. I suspect the “resource limits” message is a red herring, but haven’t been able to figure it out yet.
I’m posting here as an upvote - hopefully someone has already seen this and solved it.
I’m back up and running now. Just had to restart my db machine: fly machine restart <machine-id> -a <app-name>. Here’s the post that pointed me towards this: PG machines had disruption and never came back
Note that I also updated my rails app to v2 - I’m pretty sure that wasn’t a contributing factor to my db coming back up, but posting here just in case others have issues that a machine restart doesn’t solve.