Feature request: Images from private Docker registries

Hi! I’m just trying out Fly.io for the first time and it really seems to hit the sweet spot from what I want from a PaaS-provider in terms of functionality and pricing. Good job! The new managed Postgres service is also really nice to see, especially with the entry price point.

A couple of questions / feature-requests:

1. Images from private Docker registries
I run a docker build + push CI on GitHub Actions and I would love to be able to deploy images from my private registry. I have not found a way to do this yet (make flyctl authenticate with the private registry).

If this is not easily do-able it would at least be nice to have the Fly Docker registry adhere to the Docker HTTP API Spec (HTTP API V2 | Docker Documentation). This would let me subscribe to the feed using 3rd party deployment apps such as Octopus Deploy.

2. Setting environment variables as flags in flyctl deploy
In my CI pipeline I want to override some environment variables when deploying. It would be nice to be able to specify these variables when running the deploy command. At the moment I run a replace script that swaps out variables in the fly.toml file before deploying which works ok, but command line flags would be preferable.

Again, awesome product! Keeping my eyes on new developments, going forward :smiley:

1 Like

We’ve been hesitant to deploy directly from private Docker registries because we need to pull images to launch new VMs. We can make sure our registry stays up, but don’t trust third party registries as much. :smiley:

You can pull a private image and then flyctl deploy -i <tag> to push it to us, though. That might help!

Agreed about the environment variables! I frequently need to deploy the same app to different environments. @michael is working on some stuff to make this easier.

1 Like

I see! The pull and push step should work, thanks :smiley:

Awesome! I’ll keep an eye out.

1 Like

I wonder if I can ressurrect this. I’m using Github actions to build my app and pushing the docker image to the Github container registry.

I tried the pull/re-push method but this doesn’t seem to work. I’m using the superfly/flyctl-actions action and I get:

Searching for image 'my-image:my-tag' locally...
Searching for image 'my-image:my-tag' remotely...
Error failed to fetch an image or build from source: Authentication required to access image "docker.io/library/my-image:my-tag"

having previously successfully done:

docker pull ghcr.io/my-image:my-tag

Since this post was created have you added anything to allow authenticating fly.io to the Github container registry (or any other private registry), so I can deploy straight from the Github registry?

I get that you need the registry to be up when you launch a new VM but I’d expect Github’s registry to be pretty reliable.

See if there’s any clues in this gh-action you can find on what’s missing (specifically, note the absence of --remote-only deploy and/or http protocol qualifier… Not sure if both those matter in any materialistic way).


Edit: One could also push to Fly’s registry from gh-action, which is both free and private, like so.

Thanks for the reply. I think the reason why it works in the action you linked is because the container images for that repo are publically accessible, so it works as normal. (The http qualifier was added automatically by docker I think, it’s not actually in my workflow yaml).

I know I can push to the fly registry (I was previously using the GCR registry with public). As mentioned above, I can just switch my registry to public and it works fine. But I’d like it to be possible for my images to be private whilst using the github container registry.

1 Like

Not sure this is related, but in failed to fetch an image or build from source: Authentication required to access image I used to fly deploy an image from a private docker registry repo. Seems this doesn’t work anymore.

1 Like

Any idea whether this is going to be possible in the future?

I’m using fly-terraform, which uses machines, to provision fly related stuff. Machines require an image.

So, right now I would build the image with gitlab ci, upload it to gitlab container register, but cannot use it in the fly-terraform as an image since it’s private.

I know the current work arounds are to upload the image into fly’s register. But, the steps are quite annoying. Terraform to provision app name, gitlab ci to build/upload the image. Modify terraform to include creating a machien with the newly created docker image on fly’s register. it’s more convenient to only use fly in terraform, rather than both in gitlab ci and terraform as that requires more things to keep track of.

3 Likes

This is ironic because I’ve been having registry related issues every day for the last few weeks, unable to deploy.

This is the one I keep running into most often:

2023-10-23T14:20:07Z runner[5683d525c3d398] fra [info]Pulling container image registry.fly.io/foo:deployment-01HDEEFT9FE1279Q810D8Y4GR9

It just gets stuck here and at some point fly deploy times out. I’ve already increased the timeout to like 15 minutes

Support doesn’t seem to be able to fix it and it’s getting reaaaaaaally frustrating.

1 Like

Ah after 12 minutes it successfully pulled container image:

2023-10-23T14:20:07Z runner[foo] fra [info]Pulling container image registry.fly.io/foo-prod:deployment-01HDEEFT9FE1279Q810D8Y4GR9
2023-10-23T14:32:03Z runner[foo] fra [info]Successfully prepared image registry.fly.io/foo:deployment-01HDEEFT9FE1279Q810D8Y4GR9 (11m56.324006658s)
1 Like