How to recover a fly Postgres that is on host in maintenance?

[viniciusgabriel]

My Postgres Database is on IAD region and seems that is having an emergency maintenance:

  ID               | MESSAGE                                                                                                                                                   | STARTED AT                    | LAST UPDATED
-------------------*-----------------------------------------------------------------------------------------------------------------------------------------------------------*-------------------------------*--------------------------------
  k51vyl4q45om3r2w | We are performing emergency maintenance on a host some of your apps instances are running on. Apps may be unavailable until the maintenance is completed. | 2025-03-11 15:46:20 +0000 UTC | 2025-03-11 15:46:20 +0000 UTC
-------------------*-----------------------------------------------------------------------------------------------------------------------------------------------------------*-------------------------------*--------------------------------

That’s ok, but how can I migrate this machine to other region if I can’t get the snapshots if all my volumes do not appear (running fly volumes list -a my-postgres-app)?

I can’t go through Troubleshoot apps when a host is unavailable · Fly Docs too, because using fly machine clone I received a Error: the machine is on an unreachable host, try again later

am I missing something? Or I really can’t do anything and just waiting fly.io team to solve the maintenance which has already lasted 5 hours?

[halfer]

Is your database running in other regions as well, in a multi-machine arrangement?

[viniciusgabriel]

I have only 2 machines and both on the region if problem (IAD)

My idea was like do a machine clone to other region and so master/slave will be rearranged

[halfer]

Hmm, OK. I assume that if you are on two machines in one region, then you should be OK unless the whole region is down. Is one of your machines operational?

[viniciusgabriel]

Theoretically yes. But the machine that is inaccessible is the master from database and the up one is an replica

So in general database goes down. I tried to create a new machine copying the machine operational, but not success because the new machine try to connect to the master (down)

[halfer]

I’ve not done this, so take this with a pinch of salt. But I would guess you can reconfigure the (working) follower into a leader, and disconnect it from the (dead) leader. From there, you can clone it and turn the clone into a follower.