Deploys from Github Actions is failing

HI there,

We are unable to deploy to fly because of the following error -

Error failed to fetch an image or build from source: error connecting to docker: An unknown error occured.

I’m not sure if this is related to Fly.io Status - Connectivity failures in the Seattle region

Please advise.

Having a similar issue but with deploys from anywhere
Have re-tried multiple times, here is a screenshot from my terminal

This will likely take someone to Fly to investigate at their end, however in the meantime have you both previously deployed using Github Actions, or is the first attempt and it’s failed?

From @danalloway it seems it’s not related to the region, so it’s unlikely to be connected to the recent issues in sea

Are you both using the script as shown on Continuous Deployment with Fly.io and GitHub Actions · Fly Docs ? And with a valid token?

Assuming the script and token are ok, it’s theoretically possible the latest version of the fly CLI has caused an issue (total guess) and so this alternative page shows how you can specify the version of the CLI to use. The current version is 0.0.311 so you could experiment trying a prior version number to rule that out. Highly unlikely but you never know:

Other than that … not sure.

here is the error but with LOG_LEVEL=debug turned on

@greg to clarify I’m getting this error locally via the CLI trying to deploy
it also happens on Github actions but I’m re-creating it locally so I don’t think it’s entirely a GitHub Actions issue

Ah, interesting. I’m guessing that @Brewin would also see the same output with that additional flag (given the non-debug error message is exactly the same).

When I see a mention of anything remote-builder-y in an error message, my first attempt to fix it is to destroy it. So the next deploy triggers a new one. As the remote builders do occasionally go wrong, fill up etc (mine did yesterday). Not worth doing multiple times but you could destroy your current builder, try another deploy, and see if that fixes it?

fly apps list
… and look for the builder. And when you are 100% sure you have the right app:
fly destroy NAME

Interestingly the exact same 500 API response is in this report from jamie if you peer at their log output … but (alas) there is no reply to that post to say the fix for it:

good tips, thank you @greg I’ll remember those for next time and pass them on
so, I’ve just tried destroying the builder and and re-deploying and I get the exact same error

also, from the post you linked I tried doing --local-only instead of --remote-only and with my local Docker running and everything works perfectly, so it indeed is isolated to a remote builder issue of some sort (wondering if it is related to the latest release of the CLI)

1 Like

Thanks for the response.

Looks like our issue is resolved. This was probably a transient error. While we have encountered builder errors before, we were able to resolve past issues by deleting the builder like you suggested.

And yes, we have deployed using Github actions before. So I’m not sure what the root cause of today’s issue was.

1 Like

You’re welcome @Brewin .

@danalloway Ah. Well destroying the remote builder is always worth one try but that’s ruled that out. I wonder if there is anything else going on with Fly’s innards. As I saw another post about a possible issue with internal DNS. Especially if only remote builders are affected. You could give another release of the CLI a try (not sure which one you are on locally)) but you could match its number in the deploy script and see if it helps. Seems a long-shot though, especially if it is a remote/API issue (as that API 500 response seems to indicate).

I’m so sorry y’all are debugging this for us. As @greg mentioned, that’s a 500 from our API. Give me a moment to check our logs and see if anything stands out.

2 Likes

Found an issue that was breaking volume creation in some circumstances, fix is going out now.

This should be fixed now, sorry about that!

2 Likes

thanks for the fix @Michael it works great now! :100:

1 Like