We are working on building a POC for deploying individual instances of a single tenant app for our clients as part of a managed hosting offering using Fly.io so we don’t have to directly manage servers or mess with app isolation for security.
The plan would be to build a docker image using Github Actions from the GH Repo with the app code, and host it on GHCR as a private image. Then we would use the machines API to pull that image to build a machine when a new customer signs up automatically.
As we have just discovered the Machines API (annoyingly) only supports public images or the fly registry. We were curious what the best way to handle this is.
We don’t particularly mind using the fly container registry, however our understanding is that the fly registry is ephemeral and so we cannot rely on the image being available there.
We don’t want to make the image public, but maybe there is a way to make it “private” without requiring authentication, where only the machines API can pull it? We have been reading about Flycast and wonder if we can use it to only allow access from the machines API? Or else if the public IPs used for image pulling are published anywhere so we can make it a public image with an IP whitelist?
Or if anyone else has any other suggestions we are open to seeing if we can make Fly work.
Hi… I think the following classic post implies that as long as the image is in use by at least one Machine anywhere in the organization (not necessarily in the same app) then it will remain pinned in the registry:
In that same vein, it’s not obvious at first, but it’s possible to deploy using a different app’s image.
@pb30mentioned that he does this on a regular basis. (The template-app in that case having no Machines whatsoever, if I understand correctly.)
Personally, I’ve never had problems with an image turning out to be missing, and I do a lot of slowpoke experiments with Guix-built images manually uploaded and then—some time later—finally used to create a Machine, .
(It would be nice if some of the limits and expectations around this were officially documented, though.)
Aside: For extra peace of mind, a person could try skopeo copy right before the call to the Machines API. This is a lightweight utility (no background daemon) in the standard Debian packaging system that can transfer between registries. I use a closely related mode of it all the time, and, although there are some caveats about exact knobs to get right for the Fly.io side, it does work well…
Yeah, we assumed “We clean old ones up after a few days” to be quite literal that after a few days there is no guarantee an image remains. It would be helpful there could be some guarantee, even if we had to pay for image storage to guarantee it remains accessible.
skopeo copy to copy from GHCR back to the fly registry could possibly be used as a fallback if the creation fails because it cannot find the image, but obviously we want the quickest possible deployment time. However, if that does end up being rare that the image is removed, it could work.
I interpreted oldness in that quote as referring to more of a garbage-collection process:
the configurations of all the existing Machines (throughout the entire organization) are like the root set. As long as an image is still cited in one, it’s reachable—and hence immune to pruning. I.e., if fly m status -d shows it in the image field for any Machine, in any app, then it definitely is still present in the registry. I think you guys might instead be reading it as the time since it was uploaded,, which may well be too pessimistic.
Yeah it’s hard to tell if they mean its subject to pruning if no machines is running with it, or if it hasn’t been used recently to build a machine - though frankly the ladder seems to make more sense to minimize the images being stored since Fly is essentially providing the registry and the storage for the images for free. Perhaps someone from fly might see this and give more clarity.
Every app will use the same image, any customization would be added onto an attached volume after building the machine. And we don’t necessarily expect to be provisioning a lot, at least not for awhile, thus if it is based on recent usage our concern is that it could be subject to pruning which is why we were looking into if we could run our own registry behind flycast or an IP whitelist because then we could guarantee an image remains as long as it’s needed.
.
, which may well be too pessimistic.