I have a volume which is shown as attached to a destroyed machine. This prevents me from destroying the volume or attaching it to another machine.
Interestingly the error message I get when trying to destroy the volume tells me it is attached to a different machine to the one shown through fly volumes show. Both of the machines have been destroyed.
This all came about when I tried to use the machines API to unmount a volume from one machine and mount it on another.
How can I fix this?
Sounds like a Fly bug (no pun) to me.
If you can post output from relevant flyctl commands, may be someone from Fly will take a look and unblock your app.
Here you go.
> fly volume list
ID STATE NAME SIZE REGION ZONE ENCRYPTED ATTACHED VM CREATED AT
vol_5podq4q1q2nvg8w1 created shared 50GB lhr 81b8 true 1781113a9d40d8 5 days ago
> fly volume destroy vol_5podq4q1q2nvg8w1
Error failed destroying volume: volume is currently bound to machine: 3d8d5d2b276d89
> fly machine list
No machines are available on this app ...
I’m taking a look at this now.
Wonderful. Thank you.
Would you mind trying again? We’ve deployed a fix that checks for the machine state if a volume is still showing as claimed. The underlying issue looks to be where a machine config has a volume and then a machine update occurs without declaring a volume and leaves the claim around.
The volume is still showing as attached to a machine but destroying it worked this time.
The machine update you described was one I intentionally performed through the API in an attempt to unmount the volume. I take it that this is not supported.
Thank you again for your help.
I missed this in the original post and looking at the machine config for 1781113a9d40d8 makes me think there was another issue that occurred at some point which we’ll look into.
It should be supported so we’ll have another follow-up change soon to address this.
Oh, that’s great. I suspect that this isn’t attempted often as the CLI provides no way that I can see to unmount volumes.
@JP_Phillips Hi there, I seem to have encountered the same issue.
It’s with vol vol_jlgz1vpdm92v78m3 and machine 73d8d34df4d989.
The VM seems destroyed, fly vol list says the volume is attached and when I run a new machine with the volume, the CLI errors:
Error No unattached volumes in region 'sin'
I’d appreciate your help. 
@sh4rk I’ve resolved the issue so you should be able to use the volume again with a new machine. There’s an edge case where once a machine is created, you cannot later add a volume because there’s no guarantee (right now) that the volume and machine will be on the same underlying host. We’ll likely get a fix in soon to prevent updating a machine with a volume if the initial version did not define one (or defined a different one).
Hi. I’m running into this as well. Volume vol_2en7r1px960rk6yx can’t be used (or destroyed) because it is attached to machine 91857e3ef9e908, which is already destroyed.
If manual resolution from Fly admins is required: please remove that volume, I’ve created a new one.
Hey @SimonSapin, I looked into your volume and it looks like the app it’s attached to is in a suspended state and has one machine (3d8d9d4db4d5e8) in a stopped state, which might be contributing to the error you’re seeing. Have you tried deleting the machine your app is attached to with the command flyctl machine destroy <id>. You might also want to add the --force flag which overrides the check for state so the app gets destroyed completely.
Thank you for looking into this @shortdiv. However the one attached to 3d8d9d4db4d5e8 is the new volume I recreated from scratch in order to be able to do anything, vol_0zy6r7lg0nn4djng. This one is fine. As mentioned the problem is with vol_2en7r1px960rk6yx, attached to 91857e3ef9e908.
❯ flyctl machine destroy --force 91857e3ef9e908
Error: machine 91857e3ef9e908 has already been destroyed
❯ fly vol destroy vol_2en7r1px960rk6yx
Warning! Individual volumes are pinned to individual hosts. You should create two or more volumes per application. Deleting this volume will leave you with 0 volume(s) for this application, and it is not reversible. Learn more at https://fly.io/docs/reference/volumes/
? Are you sure you want to destroy this volume? Yes
Error: failed destroying volume: This volume is attached to a machine(91857e3ef9e908) and cannot be deleted.
I understand that the Hobby plan does not include official support but this is a billable resource stuck in limbo where I can’t use nor remove it. As far as I can tell because of a Fly bug I have no power over, encountered minutes after trying to create my first volume while following documentation.
Hi @SimonSapin, were you able to destroy this volume? Where I’ve seen mention of this, it’s been due to an occasional short delay in a destroyed Machine’s claim on the volume being released, after which it should be possible to destroy the volume.
If this isn’t the case we’d love to know.
If you did end up incurring charges for an un-destroyable volume, drop us a line at billing@fly.io.
No, the situation is unchanged since my first message on this thread a week ago. I just tried again:
❯ flyctl machine destroy --force 91857e3ef9e908
Error: machine 91857e3ef9e908 has already been destroyed
❯ fly vol destroy vol_2en7r1px960rk6yx
Warning! Individual volumes are pinned to individual hosts. You should create two or more volumes per application. Deleting this volume will leave you with 0 volume(s) for this application, and it is not reversible. Learn more at https://fly.io/docs/reference/volumes/
? Are you sure you want to destroy this volume? Yes
Error: failed destroying volume: This volume is attached to a machine(91857e3ef9e908) and cannot be deleted.
I am encountering this bug as well.
> fly volume list
Update available 0.1.8 -> v0.1.45.
Run "flyctl version upgrade" to upgrade.
ID STATE NAME SIZE REGION ZONE ENCRYPTED ATTACHED VM CREATED AT
vol_915grnwd5j1rn70q created data 3GB hkg 6d56 true 9080e96ec12178 57 minutes ago
> fly volume destroy vol_915grnwd5j1rn70q
Update available 0.1.8 -> v0.1.45.
Run "flyctl version upgrade" to upgrade.
Warning! Individual volumes are pinned to individual hosts. You should create two or more volumes per application. Deleting this volume will leave you with 0 volume(s) for this application, and it is not reversible. Learn more at https://fly.io/docs/reference/volumes/
? Are you sure you want to destroy this volume? Yes
Error: failed destroying volume: This volume is attached to a machine(9080e96ec12178) and cannot be deleted.
> fly machine list
Update available 0.1.8 -> v0.1.45.
Run "flyctl version upgrade" to upgrade.
No machines are available on this app zero2prod
I’m also facing the this problem as well!
PS C:\laragon\www\filament-v3> fly volumes rm vol_lgz1vpz98ye478m3
Warning! Individual volumes are pinned to individual hosts. You should create two or more volumes per application. Deleting this volume will leave you with 1 volume(s) for this application, and it is not reversible. Learn more at https://fly.io/docs/reference/volumes/
? Are you sure you want to destroy this volume? Yes
Error: failed destroying volume: This volume is attached to a machine(e286526c96e986) and cannot be deleted.
Error: machine e286526c96e986 has already been destroyed
Hi, I’m getting this as well after the could not reserve resource for machine error… I think I destroyed the first machine that failed 48ed235f359378 and instead of destroying the volume attached to it I reused it in a second machine d8d9d47a265098 that failed as well. I now replaced this machine and this volume using flyctl machine clone and had to flyctl destroy --force d8d9d47a265098 but the volume vol_jn924oqeq1qv03lq cannot be destroyed and still appears connected to the long destroyed machine 48ed235f359378
Error: failed to update VM d8d9d47a265098: aborted: could not reserve resource for machine: insufficient memory available to fulfill request
...
$ flyctl machine clone d8d9d47a265098
Cloning machine d8d9d47a265098 into region cdg
Volume 'litefs' will start empty
Provisioning a new machine with image registry.fly.io/sqlemail:deployment-01H7B02H1Q8Z6XG7HPYS50DBK5@sha256:3ca9e1114971c7fe280e989c6b15643df3cf36c2a00835382ae1b515b8bf8256...
Machine 568399da42d9d8 has been created...
Waiting for machine 568399da42d9d8 to start...
No health checks found
Machine has been successfully cloned!
$ flyctl machine list
3 machines have been retrieved from app sqlemail.
View them in the UI here (https://fly.io/apps/sqlemail/machines/)
sqlemail
ID NAME STATE REGION IMAGE IP ADDRESS VOLUME CREATED LAST UPDATED APP PLATFORM PROCESS GROUP SIZE
d8d9d47a265098 muddy-bird-7333 failed cdg sqlemail:deployment-01H7B02H1Q8Z6XG7HPYS50DBK5 fdaa:2:b393:a7b:126:ad7a:99fa:2 vol_jn924oqeq1qv03lq 2023-08-08T16:14:24Z 2023-08-12T08:16:27Z v2 app shared-cpu-1x:256MB
568399da42d9d8 ancient-sky-4017 started cdg sqlemail:deployment-01H7B02H1Q8Z6XG7HPYS50DBK5 fdaa:2:b393:a7b:ae02:b748:5db5:2 vol_1q85vg38d8zrzdxe 2023-08-12T08:17:14Z 2023-08-12T08:17:31Z v2 app shared-cpu-1x:256MB
2874de4c016738 wild-feather-8825 started cdg sqlemail:deployment-01H7B02H1Q8Z6XG7HPYS50DBK5 fdaa:2:b393:a7b:16a:3b16:4fda:2 vol_915grnykye14n70q 2023-08-08T14:29:59Z 2023-08-12T08:16:19Z v2 app shared-cpu-1x:256MB
INS mildred@faile:Projets/sqlemail.nim$ flyctl machine destroy d8d9d47a265098
Error: machine d8d9d47a265098 is in a failed state and cannot be destroyed since it is not stopped, either stop first or use --force flag
INS mildred@faile:Projets/sqlemail.nim$ flyctl machine destroy --force d8d9d47a265098
machine d8d9d47a265098 was found and is currently in failed state, attempting to destroy...
d8d9d47a265098 has been destroyed
INS mildred@faile:Projets/sqlemail.nim$ flyctl volume list
ID STATE NAME SIZE REGION ZONE ENCRYPTED ATTACHED VM CREATED AT
vol_1q85vg38d8zrzdxe created litefs 10GB cdg 7509 true 568399da42d9d8 4 minutes ago
vol_jn924oqeq1qv03lq created litefs 10GB cdg 93f3 true 48ed235f359378 3 days ago
vol_915grnykye14n70q created litefs 10GB cdg 9937 true 2874de4c016738 3 days ago
INS mildred@faile:Projets/sqlemail.nim$ flyctl volume destroy vol_jn924oqeq1qv03lq
Deleting a volume is not reversible.
? Are you sure you want to destroy this volume? Yes
Error: failed destroying volume: This volume is attached to a machine(48ed235f359378) and cannot be deleted.
Waited 10min and still cannot destroy this volume.