I attempted to migrate my project (divine-dust-2370) to apps v2, but it is stick part way through the migration because the new machine instances don’t have a volume available. I thought this would be taken care of with the recent enhancement to the migration tool: fly migrate-to-v2: Apps with volumes support 🎉
But my logs for the new machines show that they can’t access the directory where it expected the volume to be mounted and if I run fly list volume I can see that there are no new clones created:
$ fly volumes list
ID STATE NAME SIZE REGION ZONE ENCRYPTED ATTACHED VM CREATED AT
vol_gez1nvx79qj4mxl7 created drawings 1GB ord 4f52 true 89af3e86 1 year ago
vol_ypkl7vzgloerqg60 created drawings 1GB syd 039e true 77ed5ccf 1 year ago
vol_podq4qylj3nvg8w1 created drawings 1GB lhr 81b8 true 4a0ccae2 5 months ago
I tried to get the migration unstuck by running fly machines list and fly machines destroy X for the 3 machines. This stopped the constant crash and re-boot cycle, but the app still shows status: “Suspended”
I also tried changing the app back to the v1 platform by running fly apps set-platform-version nomad after making sure that there were no machines running and only the original volumes. This command worked and I now see the prompts to migrate to v2 again, but the app still shows the status of “Suspended”
For reasons that made sense at the time, Machines apps with zero machines set the “suspended” flag on an app - which used to describe a Nomad app that was paused (“suspended”). You can run the deprecated command fly resume <appname> to restore the Nomad app.
We’ve recently noticed a possible regression in migrate-to-v2 since moving our volumes API to a different service, although we’re still investigating the root cause. I’ll try to keep you up-to-date when we have more info to share about that.
In the meantime, feel free to keep it on V1. Now that your app is on nomad again, it should be working just as it was before (after resuming it).
Thanks. I ran fly resume MyApp and that set the scale back to a single instance. I then had to do fly scale count 3 to get back to 1 instance per region in the 3 regions where I have volume allocated for the app.
My app stores region-specific data so I need to keep 1 instance running in each region so that they can cross-talk to find data from other regions.