If you’re creating new machines often (perhaps on demand with users’ requests), and you’re using big docker images, you might be interested in this. When creating a new machine, a majority of the time is spent downloading and unpacking the docker image, especially if it’s a couple of GB or greater. By default, docker uses gzip to compress the image layers when uploading, but there’s a newer, better compression algorithm known as zstd . zstd-compressed files are usually smaller than the equivalent gzip-compressed files, and they can be decompressed faster. We now support running zstd-compressed images, so if you’d like to try this out to be able to create machines faster, follow the instructions below.
Make sure you have docker installed locally, version v20.10.23 or newer.
For the initial set up, you’ll need to authenticate your local docker client with our registry and create a new buildkit builder:
$ fly auth docker
$ docker buildx create --name zstd-builder --driver docker-container
$ docker buildx use zstd-builder
docker builder ls should indicate that you are running at least version v0.10.0 of buildkit.
Update your fly.toml to use the new image we will build:
[build]
image = "registry.fly.io/$APP:$VERSION"
To build, compress, and upload a new docker image, run:
$ docker buildx build \
--file Dockerfile \
--output type=image,name=registry.fly.io/$APP:$VERSION,oci-mediatypes=true,compression=zstd,compression-level=3,force-compression=true,push=true .
You can validate that you’ve successfully pushed a zstd-compressed image with:
$ docker manifest inspect -v registry.fly.io/$APP:$VERSION | grep "application/vnd.oci.image.layer.v1.tar+zstd"
If you see "application/vnd.docker.image.rootfs.diff.tar.gzip" instead, it is using gzip for compression. This can happen if you try to push layers that have already been pushed previously.
Finally, deploy your app:
$ fly deploy
Please drop a comment if you try it out to let me know how it goes! We hope to integrate it into flyctl by default in the future.