I’ve got an elixir app in a docker image, and every time I deploy, it seems to get stuck at this last step.
I’ve also been seeing problems/delays while pushing images for the last few weeks. Pushes often hang for a specific layer, and can take two or even three attempts to push that layer.
I’m also seeing some containers fail deployment with the following:
2021-11-07T12:36:26.438 runner[e08dccea] yyz [info] Starting instance
2021-11-07T12:36:26.474 runner[e08dccea] yyz [info] Configuring virtual machine
2021-11-07T12:36:26.484 runner[e08dccea] yyz [info] Pulling container image
2021-11-07T12:36:46.539 runner[e08dccea] yyz [info] Pull failed, retrying (attempt #0)
2021-11-07T12:37:06.549 runner[e08dccea] yyz [info] Pull failed, retrying (attempt #1)
2021-11-07T12:37:17.082 runner[e08dccea] yyz [info] Pull failed, retrying (attempt #2)
2021-11-07T12:37:17.082 runner[e08dccea] yyz [info] Pulling image failed
Will report this… and just to confirm, these are intermittent, right? The pull succeeds eventually, as opposed to the layer actually being missing from the registry altogether?
The deploy getting stuck at image done was happening every time. If I ctrl C’d it to cancel, then the terminal would then print “deployed” and “monitoring”. Strange that I had to cancel the process to get it to continue.
I’ve scaled up the memory in my app and fixed the health check of my (very large) image, and I haven’t seen the issue again.
@user55 That is odd, and sounds like flyctl problem — could you put in a few details on the bug reporter and we can take a look?
Yep, somewhat intermittent. It happens most deploys though.
1 Like