Extremely slow docker builds

1

For some reason lately the docker builds on the free builder seem extremely slow.

I’m currently deploying and it took 600 seconds to build the image.

 => [internal] load build definition from Dockerfile                                                                                                                              0.3s
 => => transferring dockerfile: 286B                                                                                                                                              0.3s
 => [internal] load .dockerignore                                                                                                                                                 0.2s
 => => transferring context: 2B                                                                                                                                                   0.2s
 => [internal] load metadata for docker.io/library/node:18-alpine3.15                                                                                                             0.6s
 => [1/6] FROM docker.io/library/node:18-alpine3.15@sha256:cd3a7004267e419477bbfc50e0502df8607a0b9b4465092f6e2c2ce4092faa45                                                       0.0s
 => [internal] load build context                                                                                                                                               588.2s
 => => transferring context: 183.43MB                                                                                                                                           588.1s
 => CACHED [2/6] WORKDIR /usr/src/app                                                                                                                                             0.0s
 => [3/6] COPY package.json .                                                                                                                                                     0.0s
 => [4/6] COPY package-lock.json .                                                                                                                                                0.0s
 => [5/6] RUN npm i                                                                                                                                                               8.5s
 => [6/6] COPY . .                                                                                                                                                                2.2s
 => exporting to image                                                                                                                                                            2.2s
 => => exporting layers                                                                                                                                                           2.2s
 => => writing image sha256:88b6aaa7cdab4b879c2cc35464c84e9f18c855b658b23a8e4036a3b14cd97efc                                                                                      0.0s
 => => naming to registry.fly.io/wavekit-dashboard-prod-machines:deployment-01HH54RK22MDQJCRX5PSJBCWK4                                                                            0.0s

Could this be an issue on my end?

All projects for the past days seem to be very slow to deploy.

2

That seems like a large number. I just deployed a tiny bun application, relevant excerpts:

[+] Building 0.7s (16/16) FINISHED                                              

 => => transferring context: 45.58kB

Here’s a larger Rails application:

[+] Building 19.9s (35/35) FINISHED            

 => => transferring context: 1.59MB

The latter application used to take minutes to deploy. So I ran fly console, then from there ran du to see what was so big, found many things I could add to my .dockerignore and now the build step of my deploy routinely takes much less than a minute.

3

It’s hard to help without more information. How long does it take on your local machine to build the dockerfile? Is this a mono-repo (if so, doing ‘COPY . .’ might affect this). How long does doing a fresh npm? That also might help debug it

4

The thing is I’ve been deploying this application to Fly for about two years now. Builds didn’t take more than 20-30 seconds until recently.

It’s definitely not an issue with the .dockerignore either. I’ve already removed all the usual suspects (node_modules, .git, etc).

Basically I’m experiencing the same issue in all my apps where the transferring context is much slower than it used to be.

Edit:

Actually it seems this has resolved itself. Just deployed the same app with a minimal code change and it took about 30 seconds to build.

Building 30.7s (11/11) FINISHED

5

Can you account for183.43MB?

Suggestion: place the following into Dockerfile.min:

FROM debian:bookworm-slim
WORKDIR /build
COPY . .

Then run the following command:

fly console --dockerfile Dockerfile.min -C /bin/bash

Once you get to a prompt, run the following command:

du * .* | sort -n | tail

I’m not sure how long source caches last, but once it expires you will likely require another lengthy upload.

6

@rubys

I just deployed the project again and now the context went down to 3.3MB. I only changed a single line of code.

transferring context: 3.31MB

I will admit I have no idea what this context is… but the fact it went down considerably without me changing anything probably means the magic elves from Fly are doing something :slight_smile:

7

There is a context cache. If you delete the builder, your first build will create a new builder and that builder will start with a clean cache. There may be other reasons (outside of your control) where you start with a clean cache.

1 Like
8

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.