ERROR: failed to push registry.fly.io/REDACTED-server-stg:deployment-0bbeffba59fd4c0d0b231d476ce7b067: unexpected status from HEAD request to http://_api.internal:5000/v2/REDACTED-server-stg/blobs/sha256:05fb0fff2820b13304a899ef7df1be8133cc7c4eafeadc00911e99dd92f5dad3?ns=registry.fly.io: 401 Unauthorized
Got above error every new deployment was triggered, both stg and production.
Same here! Without making any material changes to the codebase. Deploying from Github actions.
Same here
Same here. Command: flyctl deploy --app <app_name>
Hello everyone!
Thank you for bringing this to our attention–we’re currently investigating intermittent 4XX errors when accessing the Registry: Fly.io Status - Intermittent registry errors . You can subscribe to the incident page to receive immediate updates as we work on resolving it.
Our sincere apologies on the disruption this has caused on deploying your apps; rest assured the team is actively working on resolving this incident as soon as we can. Thank you for your kind patience.
Seems to be working using depot=false:
fly deploy --depot=false
Using depot I get a 499 error every time:
Error: failed to fetch an image or build from source: error building: failed to solve: failed to copy: unexpected status from PUT request to registry.fly.io: 499 status code 499
Building step works but the deployment step fails with the following error:
==> Building image
Searching for image ‘registry.fly.io/:deployment-e35b65653fdeb3d6f68ab9cc166ed801’ remotely…
Error: failed to fetch an image or build from source: failed to run query ($appName: String!, $imageRef: String!) { app(name: $appName) { id image(ref: $imageRef) { id digest ref compressedSize: compressedSizeFull } } }: Could not find image “registry.fly.io/:deployment-e35b65653fdeb3d6f68ab9cc166ed801”
unsuccessful command ‘flyctl deploy -a --image registry.fly.io/:deployment-e35b65653fdeb3d6f68ab9cc166ed801 --depot-scope=app --config fly.toml’
I’m getting this error:
{"errors":[{"code":"NAME_UNKNOWN", "message":"app repository not found"}]}
But only for certain apps (ord region). I’m not sure if this is related to the registry error.
This:
Error: failed to fetch an image or build from source: error building: failed to solve: failed to push registry.fly.io/REDACTED: unexpected status from HEAD request to http://_api.internal:5000/v2/REDACTED/blobs/sha256:bd45561398c2a97780f8655d3b593541a933990f58c0db3a357549841fae6bfc?ns=registry.fly.io: 401 Unauthorized
Hi folks, we’ve shipped a change to the registry flow that should improve things here. We’re seeing a decrease in these errors.
If you do run into a 401 or 499 error please let us know, and include whether you’re deploying via the Dashboard or via flyctl.
If deploying via flyctl, please try updating to the latest version with flyctl version update, as older flyctl versions may see issues.
When I deploy a large image (26GB eg large models) fly deploy --local-only … fails when pushing up the image. There’s no error, just a generic retry.
Did the recent fly cli change cause that?
The same code/image used to deploy fine before.
That would be unrelated to the errors seen in this thread, as they’re occurring when pushing to the registry from Depot builders specifically.
Offhand I don’t think any recent changes should have caused issues for larger images. But if you want to start another thread for what you’re seeing someone might be able to take a look. I’m assuming these are images for GPU machines? (26GB images should fail on a standard machine)
Yes, GPU machines. Was that restriction not enforced before? I thought GPU images size was 50GB to accommodate large model sizes.
GPU image sizes are a 50GB max, yes. Standard (non-gpu) machines are limited to 8GB. Pushing a 26GB image to a GPU machine is fine, but trying to push it to a standard machine will fail. We haven’t changed anything about the enforcement of those sizes 
But that’s out of the scope of this thread. If you’re seeing issues pushing to your GPU machine with --local-only from the latest flyctl version, please start a new thread for it
I am still running into this error when using the command line. I am currently running the latest version I believe too `Already running latest flyctl v0.3.193` - This is when I am trying to deploy a new app created with the Epic Stack (Unrelated to my issue I believe)
When I was running through the steps manually last night I kept hitting an issue when trying to run this command: fly consul attach --app [MY_APP_NAME] so I am unsure if there is still an issue there or not.
Happy to try again and collect more information.
Edit - logging out and back in again fixed the above issues although apps are still not deploying and just seem to be hanging in the Fly.io dashboard.
Updating flyctl solved the problem for me!
Yes, v0.3.193 is the latest version. Could you share the output from flyctl deploy when you see it hanging? As well, does fly deploy --depot=false succeed?
If the deploy succeeds but the apps aren’t starting, I’d recommend checking your app logs to see if it’s actually starting correctly.