old VM is hanging / unable to kill old VM instance

Unable to kill old VM instance after deploying a new version. I have ran ‘vm stop 3dd06b9c’ without any success. I think I need help from Fly directly to resolve the issue unless there is some other command to force kill this instance.

Hi @themaxsmith, thanks for reporting, you’re correct it was an internal platform issue.

One of our hosts in the sjc region triggered a bug in Nomad (our VM-orchestration service), which was preventing some instances (including yours) from getting updated correctly. We restarted the service on the affected host which unblocked the stuck instances. Sorry for the inconvenience!

@wjordan I have the same issue right now in the sin region. Can anyone check that out?

1 Like

Hi @mempirate thanks for posting. This issue should be resolved for you now, I hope. Fly.io Status - Slow API response times

@catflydotio, has this issue been auto-resolved for all users, or do folks need to report zombie VMs and they’re being take care of on a case-by-case basis?

I ask because here’s another user complaining of a zombie VM spamming their logs: Interrupt app startup - #3 by alep

1 Like

@ignoramous Resolution for this incident was on a host-by-host basis afaik.

1 Like

I have the same problem in region fra (VM Id 1fb63a19).
I cannot fly vm stop it, or remove it via fly scale count 0.
Also fly scale count 2 has no effect in this app.

I’m also having issues with an instance in hkg region (VM id 1b0d9cce) I also cannot remove it