I get the error that a machine with the same name exists in the app. When I use the dashboard I see some machines with the same name in started state, but command line does not show those machines. When I use the ID to stop those machines using command line it says ‘machine not found’
There is some inconsistency between the machines api, command line and the dashboard. Can someone help me with this?
I may be wrong here, but back when I tried, I found that once a machine name for an app is used, it could never be re-used again, regardless of the status (destroyed / stopped / started etc) of the machine.
Can you list down the IDs of those machines? May be someone from Fly can take a look to see if they’re zombies.
The command line might be filtering out zombie machines (probably doing the right thing). I’ve noticed the Machines-specific dashboard show out-dated information, often.
I have been using the same name for my machines after stopping and destroying them and it was working until today morning. Not sure if anything is changed recently on the fly end.
I used to get an error if destroyed a machine and recreated it with the same name immediately. There was a slight delay. But after that the system used to let me reuse the same name.
Examine the long json output in to see if the machine-name you want used is lingering around as a zombie (ref):
LOG_LEVEL=debug fly m list -a <app-name>
I used
flyctl machines list -a -j --verbose > /temp/fly_machines_list.txt
and it gave me a verbose json file. In that all the machines are destroyed, including the one that is showing as started in the dashboard.
In case it is helpful, here are the IDs of the machines the dashboard still showing as started:
0e286573c01486
73d8d3d0f34789
I think these are the machines that are stopping me from creating new machines using the machines api.
Can anyone at fly help me with this?
I keep running into these situations, wherein the machines that are stopped and destroyed are showing different statuses in the dashboard and flyctl machines list. Dashboard is still showing the machines in the ‘started’ status, whereas flyctl machines list shows them as destroyed.
Are they still running and getting billed?
Looking into this, I’m sorry that this has been a frustrating process for you!
I found both your machines and it looks like they’re current status is destroyed. So you shouldn’t be billed for them. I think you might be experiencing some discrepancy in a migration we did recently from an attribute rename state → status. It looks like the UI is checking state and not status hence the weirdness around stopped vs destroyed. I’ll work on a fix for this.
As for your other concern that machines are stopping you from creating new machines, what is the error you’re seeing?
This isn’t the cmd I shared; I don’t think this would show you zombies in its verbose json output.
Anyways, as mentioned before, I’ve found the UI to show incorrect state for Machines since forever. In my experience, the flyctl output is always the latest.
Dug into this a bit more, the ui state is getting updated but the sync takes a while. If you want the latest state of a machine, check flyctl. The UI catches up eventually but takes a bit, I’ll see if we can add a note in the ui to mention that
It says that a machine with such and such name already exists. If the correct state of the machine is ‘destroyed’ and it is just the dashboard that is out of sync, then the create machine logic should not think that this machine is still in started state. I think the same state vs status attribute name issue is there somewhere in the validation logic of machine creation as well.
I assumed that the UI is out of sync and I know that flyctl as well as the machines api are the real source of truth. My problem was that the create machine api was not letting me create new instances with the name of an already destroyed instance. For now I overcame the issue on my side by changing my logic to not depend on the machine names and let fly pick random machine names.
Sorry for the issue you’re hitting here. The machine API currently still has to communicate back to our central API (which also powers the UI) today when creating a machine to perform certain checks and one of those is whether a machine with the same name already exists. We should have this eventual consistency issue resolved soon.