My Postgres's machine in `created` state and I can't stop/restart/kill?

Questions / Help
1

Hi,

My Postgres’s machine is now in the created state and I can’t do anything about it…

Can you help me?

fly status --all
ID            	STATE  	ROLE 	REGION	HEALTH CHECKS	IMAGE                        	CREATED             	UPDATED
3908031f179787	created	error	ewr   	3 total      	flyio/postgres:14.6 (v0.0.34)	2023-01-23T17:16:39Z	2023-01-23T17:16:39Z	

fly machines list
1 machines have been retrieved from app scanner.
View them in the UI here (​https://fly.io/apps/scanner/machines/)

scanner
ID            	NAME        	STATE  	REGION	IMAGE                        	IP ADDRESS                     	VOLUME              	CREATED             	LAST UPDATED
3908031f179787	shy-sun-5188	created	ewr   	flyio/postgres:14.6 (v0.0.34)	fdaa:0:7313:a7b:ab2:4365:b0e3:2	vol_okgj545z8k94y2wz	2023-01-23T17:16:39Z	2023-01-23T17:16:39Z	

fly machines restart 3908031f179787
Restarting machine 3908031f179787
Error failed to restart machine 3908031f179787: could not stop machine 3908031f179787: failed to restart VM 3908031f179787: unable to restart machine, not currently started or stopped
2

Hi @rickyking ,

Try killing the fly machine and see if that helps.

fly m kill

3

Thanks,

Tried but it does not work:

fly machines kill 3908031f179787
machine 3908031f179787 was found and is currently in a created state, attempting to kill...
Error could not kill machine 3908031f179787: failed to kill VM 3908031f179787: machine not in known state for signaling, created
4

Try restarting the PG cluster members

fly pg restart

5 
fly pg restart --force
No leader found, but continuing with restart
Identifying cluster role(s)
  Machine 3908031f179787: error
Restarting machine 3908031f179787
Error could not stop machine 3908031f179787: failed to restart VM 3908031f179787: unable to restart machine, not currently started or stopped
6

The only option I could think of is to delete that PG cluster and relaunch a new one.

fly m destroy
fly pg create

7

oh …

I need the data from that volume though…

8

Oh no. Then let’s wait. But wondering how you got data in that VM in first place when it is stuck in created state.

Can you post the output for

fly m status

I know sshing to that machine may not work. But can you give a try and see if it works?

fly ssh console

9 
fly m status 3908031f179787
Machine ID: 3908031f179787
Instance ID: 01GQFT3HK838ZRCXQ8D5CWPB4Z
State: created

VM
  ID            = 3908031f179787
  Instance ID   = 01GQFT3HK838ZRCXQ8D5CWPB4Z
  State         = created
  Image         = flyio/postgres:14.6 (v0.0.34)
  Name          = shy-sun-5188
  Private IP    = fdaa:0:7313:a7b:ab2:4365:b0e3:2
  Region        = ewr
  Process Group = app
  Memory        = 2048
  CPUs          = 1
  Created       = 2023-01-23T17:16:39Z
  Updated       = 2023-01-23T17:16:39Z
  Command       =
  Volume        = vol_okgj545z8k94y2wz

Event Logs
STATE  	EVENT 	SOURCE	TIMESTAMP                    	INFO
created	launch	user  	2023-01-23T09:16:39.422-08:00

tried fly ssh console seems not working …

10

Hey there,

If you can’t get the Machine to stop, I would try the following:

Force remove the Machine
fly machines remove <machine-id> --force --app <app-name>

List your volumes
fly volumes list --app <app-name>

Clone the Machine you deleted and attach it to your volume
fly machines clone <machine-id-you-deleted> --attach-volume <volume-id>

1 Like
11

Hi @rickyking ,

As @shaun suggested you can clone a new machine from the existing PG cluster and destroy it once your cloned machine is up and running.

fly m clone 3908031f179787
fly ssh console or fly pg connect (Ensure you are able to connect to your PG cluster)

fly m destroy 3908031f179787

This will only destroy the machine & not the persistent volume. You can destroy the volume if required using

fly vol destroy vol_okgj545z8k94y2wz

But before you destroy this volume ensure you have your data intact in the new running PG cluster.