Image pull failing in AMS

Questions / Help Build debugging
1

After multiple attempts, I had to remove the AMS region to successfully deploy.

https://status.flyio.net/ does not show any blips.

2022-08-01T03:01:50.218 runner[246002b5] ams [info] Unpacking imag
2022-08-01T03:01:50.366 runner[246002b5] ams [info] Pull failed, retrying (attempt #0
2022-08-01T03:01:51.259 runner[246002b5] ams [info] Unpacking image
2022-08-01T03:01:51.400 runner[246002b5] ams [info] Pull failed, retrying (attempt #1)
2022-08-01T03:01:52.175 runner[246002b5] ams [info] Unpacking image
2022-08-01T03:01:52.316 runner[246002b5] ams [info] Pull failed, retrying (attempt #2)
2022-08-01T03:01:52.316 runner[246002b5] ams [info] Pulling image failed
2022-08-01T03:01:56.421 runner[1b089cd6] ams [info] Starting instance
2022-08-01T03:01:56.530 runner[1b089cd6] ams [info] Configuring virtual machine
2022-08-01T03:01:56.552 runner[1b089cd6] ams [info] Pulling container image
2022-08-01T03:01:58.003 runner[1b089cd6] ams [info] Unpacking image
2022-08-01T03:01:58.231 runner[1b089cd6] ams [info] Pull failed, retrying (attempt #0)
2

I have the same problem in ams.

flyctl deploy fails with:

1 desired, 1 placed, 0 healthy, 1 unhealthy
Failed Instances

2022-08-01T04:18:37Z Received Task received by client
2022-08-01T04:18:37Z Task Setup Building Task Directory
2022-08-01T04:18:51Z Driver Failure rpc error: code = Unknown desc = unable to create microvm: error pulling image: unknown

/go/pkg/mod/github.com/hashicorp/nomad@v0.12.0/plugins/drivers/proto/driver.pb.go:4260

                                    google.golang.org/grpc.(*Server).processUnaryRPC                                                                                      
                                            /go/pkg/mod/google.golang.org/grpc@v1.44.0/server.go:1282                                                                             
                                    google.golang.org/grpc.(*Server).handleStream                                                                                         
                                            /go/pkg/mod/google.golang.org/grpc@v1.44.0/server.go:1616                                                                             
                                    google.golang.org/grpc.(*Server).serveStreams.func1.2                                                                                 
                                            /go/pkg/mod/google.golang.org/grpc@v1.44.0/server.go:921                                                                              
                                    runtime.goexit                                                                                                                        
                                            /usr/local/go/src/runtime/asm_amd64.s:1571

2022-08-01T04:18:51Z Not Restarting Error was unrecoverable
2022-08-01T04:18:51Z Alloc Unhealthy Unhealthy because of failed task
2022-08-01T04:18:52Z Killing Sent interrupt. Waiting 10s before force killing

2022-08-01T04:18:39Z [info]Starting instance
2022-08-01T04:18:39Z [info]Configuring virtual machine
2022-08-01T04:18:39Z [info]Pulling container image
2022-08-01T04:18:49Z [info]Unpacking image
2022-08-01T04:18:49Z [info]Pull failed, retrying (attempt #0)
2022-08-01T04:18:50Z [info]Unpacking image
2022-08-01T04:18:50Z [info]Pull failed, retrying (attempt #1)
2022-08-01T04:18:51Z [info]Unpacking image
2022-08-01T04:18:51Z [info]Pull failed, retrying (attempt #2)
2022-08-01T04:18:51Z [info]Pulling image failed
→ v700 failed - Failed due to unhealthy allocations - not rolling back to stable job version 700 as current job has same specification and deploying as v701

→ Troubleshooting guide at Troubleshooting your deployment · Fly Docs
Error abort

3

Thanks for the heads up on this, it seems like one of our hosts is having issues pulling a base image. We’ve pulled it out of the pool for now, will you see if your deploys work better?

2 Likes
4

Thanks for the quick reply. It works now, the latest deployment was successful. Thanks!

1 Like