App down due to Fly emergency maintainence on VM, but cannot move to new VM

[lpil]

Hello! One of my apps has been down for the last 15 hours or so.

On the dashboard it says this:

1. 024-04-25 13:09:01 UTC We are performing emergency maintenance on a host some of your apps instances are running on. Apps may be unavailable until the maintenance is completed.

Service interruptions caused by host issues can take a while to resolve.
You don’t have to wait: Learn how to recover and move away from an unresponsive host.

I tried the various recommendations in that page, but they all fail due to Fly internal errors according to the logs. For example:

This deployment will:

• create 1 “app” machine

No machines in group app, launching a new machine

------
Failed: error creating a new machine: failed to launch VM: server returned a non-200 status code: 500
-------
Checking DNS configuration for puck.fly.dev
Error: error creating a new machine: failed to launch VM: server returned a non-200 status code: 500 (Request ID: 01HWCRY6PJAEB7ATHHF0Q3EXAF-lhr) (Trace ID: f79d1d9e7b25bd226eed39c7fbc93cb4)

Is there something I can do to bring my app back online? Some other forum threads suggested deleting the app but I don’t feel comfortable doing this as I have secrets and a volume that I’m worried about losing.

Thanks,
Louis

[lpil]

I’ve been trying to make new volumes and scale to 1 to move to a different VM but it doesn’t work, possibly because it’s using the VM attached to the bad VM? I tried to delete the volume on the bad VM but that also fails.

It seems that pretty much every command I run results in an internal server error currently.

edit: OK well I tried scaling again and this time it managed to get an instance up. I perhaps it used the other volume by chance? I’m unable to delete the old volume so I’m a bit nervous about it being used in future and the data on each volume getting out of sync. If I try and delete it I get an error.