Error failed to fetch an image or build from source: error rendering push status stream: unauthorized: not allowed

I am getting this error right now:

Error failed to fetch an image or build from source: error rendering push status stream: unauthorized: not allowed

My app region is gru. Can you tell me when it is going to be working again?

It is working now!!!

Same error here now.

Same issue:

Error failed to fetch an image or build from source: error rendering push status stream: unauthorized: not allowed

Two different orgs, both on GRU.


I tried to change the application scale and all the instances stopped working, new instances could not pull the image “Pull failed, retrying (attempt #0)”.

For now I’m using MIA.

It has been happening now for two days for me (yesterday and today). Looking at the status page everything seems to be fine https://status.flyio.net/, so I am not sure if there is something wrong with the registry or on our side.

Although it is curious that for the Registry API there is no data recorded, so if the problem is with this service we can’t know from the status page :sweat_smile:

Not a big deal for me not to be able to deploy, but I wanted to let the team know that it has been an ongoing issue for two consecutive days now.

Regards,

1 Like

Yeah, I’m having this same issue since early this morning (currently in GMT -3 and my instance is in GRU as well)

1 Like

Hi all, we’re looking into this. Please send us the app/machine you were deploying and approximately what time it failed to help us root cause.

Thanks!

We found a periodic connectivity issue from the registry in the gru region to the database used to authorize requests. We believe that connectivity issue has now been fixed, and these won’t recur. Let us know if you see this again.

2 Likes

This is happening to me as well. I’ve double checked our token multiple times but I keep getting this error. Any suggestions?

Hi iragsdale, it looks like the token getting used is not recognized by our registry.

One thing to try is running fly apps list and checking it shows the expected app before calling fly deploy.

If you are pushing docker images directly to registry.fly.io, (re)run fly auth docker to ensure the docker client is using an updated token.

It seems like it would make sense to raise a more specific error in that case. I’ll try again.

Yup, this was my fault! Had missed a character in the token and somehow failed to see the multiple times.

Seeing this error today on ORD when trying to deploy via GitHub actions. It was working yesterday.

As you can see i’ve stuck here half an hour :smiling_face_with_tear:

 => => naming to registry.fly.io/{my-repo}:deployment-01HCHTQRZB7HQADRX0QRT2RS33                             0.0s
--> Building image done
==> Pushing image to fly
The push refers to repository [registry.fly.io/{my-repo}]
33c672f48be5: Pushed 
520e4242401e: Pushing [==================================================>]  16.57MB
520e4242401e: Pushing [==================================================>]  16.57MB
23e6fe9e929c: Pushing [==================================================>]  29.52MB
ca7011e1559e: Pushed 
d2a61f3e24b5: Pushed 
2a141dd3e6b9: Pushing [==================================================>]  7.063MB
2a141dd3e6b9: Pushed 
cc2447e1835a: Pushing [==================================================>]  7.626MB
Error: failed to fetch an image or build from source: error rendering push status stream: unknown: invalid content range

{my-repo} on main via 🐳 desktop-linux took 31m27s ```

We’re also seeing something similar today, via our GitHub actions:

Error: failed to fetch an image or build from source: error rendering push status stream: blob upload unknown

1 Like

Same here

Same here

1 Like

Sorry for the inconvenience. We are actively working on the issue.

We have a “backup” registry that only keeps storage in the IAD region, which we think will work around the issue you are seeing. Try it out with:

% FLY_REGISTRY_HOST=registry-iad.fly.io fly deploy

Use this if you are doing docker push directly to the registry:

% FLY_REGISTRY_HOST=registry-iad.fly.io fly auth docker
% docker push registry-iad.fly.io/your-app:latest

We are working to fix the core issue. We’ll report back once that’s done. In the meantime, we hope the above unblocks deployments.

3 Likes

Was this ever fixed? Seeing this issue now

Sorry. This thread was used for multiple registry issues. Are you getting unauthorized: not allowed?

Hi seeing this issue now too

Error: failed to fetch an image or build from source: error rendering push status stream: unauthorized: not allowed