Postgres database needed to be manually restarted

My Postgres instance started mysteriously failing all connections earlier today, and continued to be unresponsive until I manually restarted it 12 hours later.

I couldn’t find much, just this:

2022-07-31T23:06:43Z app[387d83f3] iad [info]exporter | INFO[1059715] Established new database connection to "fdaa:0:bff:a7b:ab8:0:65c0:2:5432".  source="postgres_exporter.go:970"
2022-07-31T23:06:44Z app[387d83f3] iad [info]exporter | ERRO[1059716] Error opening connection to database (postgresql://flypgadmin:PASSWORD_REMOVED@[fdaa:0:bff:a7b:ab8:0:65c0:2]:5432/postgres?sslmode=disable): dial tcp [fdaa:0:bff:a7b:ab8:0:65c0:2]:5432: connect: connection refused  source="postgres_exporter.go:1658"

Restarting the Postgres instance fixed the issue right away, but I’m a bit puzzled about how to avoid this happening again. Exactly the same thing happened on June 3 2022. The app’s name is mess-with-dns-pg.

Is there a way to add a healthcheck to my Postgres database so that it can automatically restart itself if it gets into a bad state?

It looks like the DB process OOMed several times and then we gave up trying to restart it. We should have cycled the VM when this happened, but I think you may be on an old Fly Postgres build that doesn’t handle this as well. Let me find out if that’s upgradeable.

I think upgrading to 1GB of RAM will prevent this.

thanks so much for looking into it! It looks like it might have automatically upgraded when I restarted it.