Error building a new image and instance always pending

1

Hi,

I had a app running with more then 50 version.

Now I’m getting a error while building the changes.


2022-10-25T18:04:24.234 app[24740d36] yul [info] Preparing to run: `/cnb/process/rcapibridge` as 1000

2022-10-25T18:04:24.249 app[24740d36] yul [info] 2022/10/25 18:04:24 listening on [fdaa:0:a775:a7b:b729:2474:d36:2]:22 (DNS: [fdaa::3]:53)

2022-10-25T18:29:55.878 runner[1f030eed] yyz [info] Starting instance

2022-10-25T18:29:56.530 runner[1f030eed] yyz [info] Configuring virtual machine

2022-10-25T18:29:56.554 runner[1f030eed] yyz [info] Pulling container image ```

What could cause this ?

Here is my fly status : 
Why the instances are not able to "run" it keep them selfs running in "pending" mode.

App
Name = johndoe
Owner = district-web
Version = 13
Status = running
Hostname = johndoe.fly.dev
Platform = nomad

Deployment Status
ID = UUID
Version = v13
Status = running
Description = Deployment is running pending automatic promotion
Instances = 3 desired, 1 placed, 0 healthy, 0 unhealthy

Instances
ID PROCESS VERSION REGION DESIRED STATUS HEALTH CHECKS RESTARTS CREATED
e0ee88cf app 13 ⇡ yyz run pending 0 2m21s ago



Thanks
2

In another project associated with the same account.

With this basic app (from here Run a Go App · Fly Docs) here is the error I get:

Error error rendering push status stream: EOF

It seems something related to my account, but what ?

3

Hi @Dev_DW,

Here’s a couple ideas.

  • You could try deploying locally instead of using the remote builder. (ie. fly deploy --local-only)
  • You can try deleting your builder and having a new one created.

To delete the builder, you can:

  • fly apps list to see all the apps associated with your account.
  • Apps with prefix of fly-builder- can be deleted too. Get the full app name from here and…
  • fly apps destroy squishy-melon-1234

Then re-deploy your app. A new builder will be created.

4

Hi @Mark ,

Thanks,

I didn’t know for the --local-only, I’ll look into that.

I did try deleting worker but that didn’t change anything.

I’ll try destroying app with the command line.

When I try to deploy now, here is the cli error.

Adding cache layer 'paketo-buildpacks/go-dist:go'
Adding cache layer 'paketo-buildpacks/go-mod-vendor:mod-cache'
Adding cache layer 'paketo-buildpacks/go-build:gocache'
--> Building image done
==> Pushing image to fly
The push refers to repository [registry.fly.io/rcapibridge]
2b0d4ab2912c: Pushing [==================================================>]   2.56kB
1f6e90e10b79: Pushing [==================================================>]  4.096kB
06c379e59ca1: Pushing [==================================================>]  2.112MB
7352fc1e0d52: Pushing  1.536kB
7647a9874f99: Pushing [==================================================>]   14.2MB

It never pushes, it waiting there and retrying.

1 Like
5

With this error after 10 minutes :slight_smile:
Error failed to fetch an image or build from source: error rendering push status stream: EOF

1 Like
6

Getting the exact same error here - trying to deploy to fra region:

Error failed to fetch an image or build from source: error rendering push status stream: EOF
7

Hey there, we’re working through some issues that have shown up on our registry: Fly.io Status - Intermittent image pull failures from registry.fly.io

1 Like
8

We found an issue that was impacting our registry in yyz – a stale DNS entry for an upstream service was causing connections to hang. We fixed the issue and things are looking better on our side. If you’re still seeing issues let us know!

1 Like
9

sea region also seems to be affected. Image pulls end up with timeouts most of the times there.

10

@Hypermind We are not seeing any more errors in the logs from SEA builders. If you get another one, will you post the log snippet here?

11

@kurt, everything is working now. Thank you!

12

Today seems like it is not working again @michael
Can this be permanently fixed? It does affect our deployments often.

13

@kurt still have issues deploying. Is that because the image is too large (750MB) ?

Screenshot at Nov 24 09-29-34
Screenshot at Nov 24 09-29-341018×642 71.8 KB

Please advise on any workarounds.