Keep getting failed builds after canceling one midstream. Wireguard errors?

Problem: Every deploy attempt fails when the Depot builder tries to push the image to the Fly internal registry. The error is always:

unexpected status from HEAD request to

http://_api.internal:5000/v2/[app_name]/blobs/sha256:<hash>?ns=registry.fly.io

401 Unauthorized

When it started: After a flyctl deploy was cancelled mid-push (Ctrl-C / workflow killed). No deploys have succeeded since.

What we have confirmed:

  • No pending/in-progress releases exist in flyctl releases
  • The external registry endpoint (registry.fly.io) accepts authentication and returns 200 on the same blob HEAD requests, so credentials are valid
  • --recreate-builder forces a brand new Depot builder machine and still gets the same 401
  • --no-cache generates completely new layer SHAs never seen before and still gets the same 401
  • A new API token makes no difference
  • The 401 is consistently on the internal WireGuard proxy (_api.internal:5000), not on the external registry

Workaround in place: --wg=false bypasses the internal proxy and routes over HTTPS, which works. But the underlying WireGuard proxy issue remains. But I dont want to forgo wireguard if I can avoid it. Is this a fly.io cache issue, or other issue fly.io @flyio-support needs to help me with?

If so, Question for Fly Team: Is the internal registry proxy for this app in a corrupted state from the cancelled push? Will it self-heal and if so how long does that take? Or does it need to be manually reset on your side? Thanks!!!

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.