Alright I got a few more things for you to try if you don’t mind. I appreciate you trying the previous steps. Hopefully this here will work, I know its been a long one. This is going to be quite a bit of info, but here we go!
If you’re still able to access the running database at all , you can import your data into a new app running PG Flex. If connecting with fly postgres connect and running a pg_dump doesn’t work, you can try using the fly postgres import command to pull the data from your existing db to a new one. This might succeed even if the manual dump failed. More details on fly postgres import are available here.
. If that doesn’t work, then you can set your database into restoration mode. To do so,
-
Scale your app down to a single machine
-
SSH into the instance to remove the consul data and add a FLY_RESTORED_FROM environment variable (this can be set to any value). This triggers the restore logic, which will rebuild your consul configuration data. The consul endpoint can be found within the environment. This isn’t as risk free as the first option, so I’d recommend creating a backup from your most recent snapshot first.