Inability to create volumes and deleted ones not going away

Howdy;
My app’s volumes list is littered with deleted and migrated volume states:

❯ fly volumes list
ID                      STATE           NAME            SIZE    REGION  ZONE    ATTACHED VM     CREATED AT    
vol_1g67340gj0prydxw    created         ephemera_cache  25GB    ewr     262d    b038c6a8        2 months ago 
vol_w1q85vgnplp4zdxe    created         ephemera_cache  20GB    ord     4f52    c55be546        9 hours ago  
vol_0nylzre1kno4qmkp    created         ephemera_cache  20GB    ord     d6bf    4968e5d5        9 hours ago  
vol_52en7r1lq994k6yx    created         ephemera_cache  25GB    sea     4df6    33bc3775        2 months ago 
vol_70zy6r7qmj9vdjng    created         ephemera_cache  25GB    sea     bec5    cc0b41f5        2 months ago 
vol_ke628r630onrwmnp    created         ephemera_cache  25GB    dfw     e163    4b2955d7        2 months ago 
vol_0nylzre83n1rqmkp    deleted         ephemera_cache  25GB    ewr     429d                    2 months ago 
vol_jlgz1vp7lkv78m30    created         ephemera_cache  25GB    yyz     c150    23c7f693        10 months ago
vol_2n0l9vl6pjgv635d    created         ephemera_cache  25GB    dfw     ccb3    deab349c        2 months ago 
vol_3xme149mgzrowpln    migrated        ephemera_cache  25GB    ord     3c24                    10 months ago
vol_ke628r6lzqrwmnpy    migrated        ephemera_cache  25GB    ord     3c24                    10 months ago
vol_mjn924ogpqr03lq7    migrated        ephemera_cache  25GB    dfw     b7f2                    10 months ago

I am unable to affect any of the volumes in migrated or deleted states without getting errors, so they hang around like bad pennies when managing volumes.

@kurt has said previously that migrated volumes are not being charged, which is great, but now I’m running into this when trying to create new volumes in some regions:

❯ fly volumes create ephemera_cache --region ewr --size 20
Error failed creating volume: Can't create volume, application is already using 9 of 9 zones in ewr

Related to the volume shenanigans – I was using volume anchor scaling, or I was but I can’t find that in the docs anymore. Is that still the blessed path for picking your regions if you have a volume-backed app? I switched to auto-scaling with a equal min-max to control it at the moment.

Honestly if I could just declare the regions, counts and volume size/name all in my fly.toml it would move from magical to utility, in the best way possible. I realize there would still be decisions or defaults around how permanent a backing volume is during a re-release (e.g. re-use, or drop the old and re-create, etc), but that feels like a hill you’ve already climbed with existing volume behaviors – re-use when it’s there with the same size, drop & create when the size changes, create when there’s no matching volume. Heck, put confirmations in the CLI if you’re feeling saucy.

Wrapping up:

  • Zombie volumes – what do?
  • Anchor scaling – old & busted?