[Java] Deploy fails

Questions / Help Build debugging
1

We have a Java REST API that runs in 4 instances.

We do deployment through GitHub action. Most of the time there are no issues but recently it started happening almost with every deploy. This is what we are getting:

 Error failed to fetch an image or build from source: failed to fetch builder image 'index.docker.io/heroku/buildpacks:18': Get "https://registry-1.docker.io/v2/": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

I tried to deploy locally and got this:

Error failed to fetch an image or build from source: failed to fetch builder image 'index.docker.io/heroku/buildpacks:18': resolve auth for ref index.docker.io/heroku/buildpacks:18: error getting credentials - err: exec: "docker-credential-desktop": executable file not found in $PATH, out: `

Not sure what is the issue. It seems it fails to download the image.
Here is the fly.toml file

app = "my-app"

kill_signal = "SIGINT"
kill_timeout = 5
processes = []

[build]
  builder = "heroku/buildpacks:18"

[env]

[experimental]
  allowed_public_ports = []
  auto_rollback = true

[[services]]
  http_checks = []
  internal_port = 7000
  processes = ["app"]
  protocol = "tcp"
  script_checks = []

  [services.concurrency]
    hard_limit = 25
    soft_limit = 20
    type = "connections"

  [[services.ports]]
    force_https = true
    handlers = ["http"]
    port = 80

  [[services.ports]]
    handlers = ["tls", "http"]
    port = 443

  [[services.tcp_checks]]
    grace_period = "4s"
    interval = "15s"
    restart_limit = 2
    timeout = "25s"
2

Hi,

Seems to be an issue with the Docker registry.

I just tried calling https://registry-1.docker.io/v2/ and it responded (with the expected failure, requesting authentication) so I assume they have resolved it. I’d try the deploy again since other people were getting the same issue a while ago.

1 Like
3

Thanks.
It seems the solution is just to wait.
I deployed it now and it worked.

1 Like
4

The issue is still persists. Now every deploy fails. I waited 24 hours and still got the same error.
So the issue seems to be random.

5

Yep, still seems to be an issue with Docker. I just checked their index.docker.io site and some images are loading for me (seemingly the official ones, like nginx, are fine). However I’m now getting 404 errors for others.

If the only image you depend on is the heroku buildpack, you could change to use the other way to build an image: using a Dockerfile. You could look on this forum or their site for how to do that.

Else it will be a case of hoping that image becomes available again to pull from Docker.

6

Thanks for looking into it.

Does deploy via fly.toml is different from using Dockerfile?

I though Dockerfile will try to pull the same image.

7

The answer is: it depends.

What essentially happens when you deploy is the Fly CLI will look at your app and guess what it is in order to make an image of it.

Using a buildpack is one approach, and probably its default.

However if you have a Dockerfile (and sibling .dockerignore file) in that folder, well the Fly CLI will spot that and realise it should instead use that approach. And you would see the output change to mention that, and instead run the content of the Dockerfile.

So I think the only change to the fly.toml file would be the line that references the need for a heroku buildpack (in [build]) would go. When using a Dockerfile, that’s not needed.

And so the builder would no longer need to pull that image from Docker. Yay!. Except … it would need to pull another one. Because generally the first line of a Dockerfile is some image. You may end up moving the problem. From one image to another.

Anyway, if you want to take a look at an example, this is a hello world Nodejs app, with a Dockerfile. Notice there’s no reference to any heroku buildpack anywhere in it. But there is to another image. So like I say, doing it this way may not actually help. You could just wait:

1 Like
8

I see.

Thanks for the insight.

This must be happening to other customers too. I’m just curious how “waiting” could be solution for a production app. Perhaps there is more definite solution for this issue.

I’ll dig more on this to see if i can get it working with a dockerfile. I tried it back then when I migrated form Heroku but I couldn’t get it working.

1 Like
9

I agree, a Docker pull should be working. Especially since they don’t have any issues on their status page as of now (you can see it all went wrong a few days ago Docker system status real-time view)

What complicates things is pulls may work from one region, but not another. Since each will have its own network etc, and probably its own cache. So it’s possible it’s limited to the region your builder is running in and so only affects users deploying from there. Purely out of interest you could try deploying a copy of your app (with a different name, as it has to be globally unique) in another region and see if that one deploys using exactly the same code and buildpack. If it does, points to the region being the issue. If not, something else (image not available, Docker issue, and so on).

1 Like
10

I will try to deploy it in a different region.

Just an update: 24h pasted again and still doesn’t work.

11

I solved the issue by removing the builder image. Once a new builder is created, the deploy went smooth.