Build images with nixpacks

Nixpacks doesn’t have Elixir support yet, but there’s been some work towards it.

1 Like

How can I pass configuration to nixpacks for things like additional libraries? I don’t see any way to pass the --libs flag and none of the options I can think of for trying to pass the NIXPACKS_LIBS env var (or NIXPACKS_APT_PKGS) have any effect.

The environment variables might work.

It would be a good idea to add a way to specify extra flags for nixpacks w/ something like this: -- --libs.

I tried setting the env vars before invoking flyctl deploy, I tried passing --env, I tried passing --build-arg, I even tried --build-secret, none of them worked. I also tried putting it in fly.toml in the [env] section though I assume that’s identical to the --env flag.

1 Like

Our nixpacks integrations is very basic.

It sounds like we should pass through any NIXPACKS_ prefixed env vars to the nixpacks binary when invoking it.

Essentially modifying this: flyctl/nixpacks_builder.go at 5b2a4e09fb29c2dd89fd39c5755b984d9abefea8 · superfly/flyctl · GitHub

It shouldn’t be incredibly hard, but I can’t guarantee we’ll get to it fast.

I’ve made a pre-release that should pass-through all env vars prefixed with NIXPACKS_ to the nixpacks command.

Depending on how you installed flyctl to begin with, you have a few options:

  • If you installed via homebrew (if you’re on macos), then it might be easier to download the release binary from: Release v0.0.388-pre-1 · superfly/flyctl · GitHub and just running that like ./flyctl.
  • If you’ve installed it from our install script via curl, you can do: curl -L https://fly.io/install.sh | sh -s pre to get a prerelease.

Let me know if it works out for you!

1 Like

My initial build context transfer went from 74MB to 2-300MB before even getting started on the build. This step is quite slow when done on a 20Mbit/sec uplink.

I have a bun/nextjs app and it tried to run npm ci as an added step from the nixpack. This failed, even though my Dockerfile’s first step is to install nodejs and npm. So for me, nixpacks are not currently viable. As a side note, I’m also testing Railway and encounter other problems trying to manually specify my build steps in my Dockerfile, so maybe I’m not using it right in the first place.

This should be fixed on the latest Nixpacks version.
We respect .gitignore now.

1 Like

Running fly deploy --nixpacks --remote-only in 3 ways:

  1. When I don’t have Docker installed on my Apple ARM64 machine, I get the error below.
  2. When I do have Docker installed on the machine, it succeeds.
  3. From a GitHub Action using superfly/flyctl-actions/setup-flyctl@master, it succeeds.
==> Verifying app config
--> Verified app config
==> Building image
Remote builder fly-builder-name ready
Proxying local port /var/folders/random-id/docker.sock to remote [address]:port

╔═══════ Nixpacks v0.5.6 ══════╗
║ setup      │ go_1_18         ║
║──────────────────────────────║
║ install    │ go mod download ║
║──────────────────────────────║
║ build      │ go build -o out ║
║──────────────────────────────║
║ start      │ ./out           ║
╚══════════════════════════════╝

Error: Please install Docker to build the app https://docs.docker.com/engine/install/
Error failed to fetch an image or build from source: exit status 1

I have a suspicion this is Proxying local... not quite working. Can you run it with LOG_LEVEL=debug fly deploy ... and report the output?

With LOG_LEVEL=debug:

DEBUG <-- 200 https://api.fly.io/graphql (64.71ms)

{
  "data": {
    "validateWireGuardPeers": {
      "invalidPeerIps": []
    }
  }
}
Proxying local port /var/folders/ch/3j31bp0j17z55v5kws_szv7w0000gn/T/3178070520/docker.sock to remote [fdaa:0:58da:a7b:21e0:f5df:538:2]:2375
DEBUG calling nixpacks at /Users/croaky/.fly/bin/nixpacks with args: [build --name registry.fly.io/webstack-go:deployment-01GDB46BB5PCH0MG109AQFJ6T1 /Users/croaky/webstack/fly-go] and docker host: unix:///var/folders/ch/3j31bp0j17z55v5kws_szv7w0000gn/T/3178070520/docker.sock
╔═══════ Nixpacks v0.5.7 ══════╗
║ setup      │ go_1_18         ║
║──────────────────────────────║
║ install    │ go mod download ║
║──────────────────────────────║
║ build      │ go build -o out ║
║──────────────────────────────║
║ start      │ ./out           ║
╚══════════════════════════════╝

Error: Please install Docker to build the app https://docs.docker.com/engine/install/
DEBUG result image:<nil> error:exit status 1
Error failed to fetch an image or build from source: exit status 1

This is likely due to docker not being installed. I overlooked that this was necessary.

I’m updating the original post.

1 Like

The past few days I’ve found deploying nixpacks for my go application to be very unreliable. Keep getting grpc connection errors rpc error: code = Unavailable desc = transport is closing or rpc error: code = Canceled desc = grpc: the client connection is closing. Deploying via fly deploy using paketo buildpacks works every single time.

fly deploy --nixpacks --remote-only has the same issue as just normal fly deploy --nixpacks.

I am not connected via wireguard when trying to do this, but I also experienced that failing yesterday for similar grpc issues when I was connected via wireguard.

I do have docker installed and the grpc connection is closing part way through sending the build context.

Still seeing some intermittent issues when deploying nixpacks. Retrying the command seems to solve the issue sometimes; I haven’t been able to find a reliable way to trigger this. I wonder if the grpc connection is more sensitive to wireless connections like WiFi?

Here is what I am sometimes seeing when running fly deploy --nixpacks:

➜  fly deploy --nixpacks
==> Verifying app config
--> Verified app config
==> Building image
Remote builder fly-builder-quiet-forest-1122 ready
Proxying local port /var/folders/3_/qfmfmk2n3s7dx5k19tymhqqr0000gn/T/2464706686/docker.sock to remote [fdaa:0:2e07:a7b:d828:4dc6:2748:2]:2375

╔════════════ Nixpacks v0.2.11 ═══════════╗
║ Packages   │ go_1_18         ║
║─────────────────────────────────────────║
║ Install    │ go get          ║
║─────────────────────────────────────────║
║ Build      │ go build -o out ║
║─────────────────────────────────────────║
║ Start      │ ./out           ║
╚═════════════════════════════════════════╝

[+] Building 16.7s (7/17)                                                                                                                                                                                                                                                       
 => [internal] load build definition from Dockerfile                                                                                                                                                                                                                       0.2s
 => => transferring dockerfile: 723B                                                                                                                                                                                                                                       0.2s
 => [internal] load .dockerignore                                                                                                                                                                                                                                          0.1s
 => => transferring context: 2B                                                                                                                                                                                                                                            0.1s
 => [internal] load metadata for docker.io/library/debian:bullseye-slim                                                                                                                                                                                                    0.4s
 => [internal] load metadata for ghcr.io/railwayapp/nixpacks:debian-1657820962                                                                                                                                                                                             0.5s
 => [stage-0 1/7] FROM ghcr.io/railwayapp/nixpacks:debian-1657820962@sha256:f116f75d26a1ad18c99138f10f3efd6564e88960854a3bbd1f2625396d22fcf9                                                                                                                               0.0s
 => ERROR [internal] load build context                                                                                                                                                                                                                                   16.0s
 => => transferring context: 9.91MB                                                                                                                                                                                                                                       16.0s
 => [stage-1 1/5] FROM docker.io/library/debian:bullseye-slim@sha256:e8ad0bc7d0ee6afd46e904780942033ab83b42b446b58efa88d31ecf3adf4678                                                                                                                                      0.0s
------
 > [internal] load build context:
------
rpc error: code = Canceled desc = grpc: the client connection is closing
Error: Docker build failed
Error failed to fetch an image or build from source: exit status 1

Are you still seeing these issues?

@jsierles yes, I am still seeing the rpc error. Just got the below 4 times in a row before it succeeded the 5th try.

$ fly deploy --nixpacks

=> ERROR [internal] load build context                                                                                           11.4s
 => => transferring context: 8.48MB                                                                                               11.4s
------
 > [internal] load build context:
------
rpc error: code = Unavailable desc = transport is closing
Error: Docker build failed
Error failed to fetch an image or build from source: exit status 1

Getting some odd new behavior when trying to deploy using nixpacks for a Go application

==> Verifying app config
--> Verified app config
==> Building image
Remote builder fly-builder-holy-pine-3450 ready
Proxying local port /var/folders/3_/qfmfmk2n3s7dx5k19tymhqqr0000gn/T/36910806/docker.sock to remote [fdaa:0:2e07:a7b:9ad9:44b7:236e:2]:2375

╔════════════ Nixpacks v0.2.11 ═══════════╗
║ Packages   │ go              ║
║─────────────────────────────────────────║
║ Install    │ go get          ║
║─────────────────────────────────────────║
║ Build      │ go build -o out ║
║─────────────────────────────────────────║
║ Start      │ ./out           ║
╚═════════════════════════════════════════╝

[+] Building 0.5s (3/4)                                                                                                                                                                                                                                                         
 => [internal] load build definition from Dockerfile                                                                                                                                                                                                                       0.1s
 => => transferring dockerfile: 723B                                                                                                                                                                                                                                       0.1s
 => [internal] load .dockerignore                                                                                                                                                                                                                                          0.0s
 => => transferring context: 2B                                                                                                                                                                                                                                            0.0s
 => [internal] load metadata for docker.io/library/debian:bullseye-slim                                                                                                                                                                                                    0.3s
 => [internal] load metadata for ghcr.io/railwayapp/nixpacks:debian-1657820962                                                                                                                                                                                             0.4s
Failed to fire hook: while creating logrus local file hook: user: Current requires cgo or $USER, $HOME set in environment
[2023-02-14T18:03:36.539725000Z][docker-credential-desktop][F] user: Current requires cgo or $USER, $HOME set in environment
[common/pkg/paths.Home()
[	common/pkg/paths/paths.go:105 +0x54
[common/pkg/paths.Container()
[	common/pkg/paths/user_darwin.go:30 +0x1d
[common/pkg/paths.Data()
[+] Building 0.5s (4/4) FINISHED                                                                                                                                                                                                                                                
 => [internal] load build definition from Dockerfile                                                                                                                                                                                                                       0.1s
 => => transferring dockerfile: 723B                                                                                                                                                                                                                                       0.1s
 => [internal] load .dockerignore                                                                                                                                                                                                                                          0.0s
 => => transferring context: 2B                                                                                                                                                                                                                                            0.0s
 => [internal] load metadata for docker.io/library/debian:bullseye-slim                                                                                                                                                                                                    0.3s
 => ERROR [internal] load metadata for ghcr.io/railwayapp/nixpacks:debian-1657820962                                                                                                                                                                                       0.4s
------
 > [internal] load metadata for ghcr.io/railwayapp/nixpacks:debian-1657820962:
------
failed to solve with frontend dockerfile.v0: failed to create LLB definition: rpc error: code = Unknown desc = error getting credentials - err: exit status 1, out: ``
Error: Docker build failed
Error failed to fetch an image or build from source: exit status 1

Seems like either I can’t use the logrus package with nixpacks or that the nixpack itself changed in some way? I tried deploying without adding any custom hooks to logrus and the deploy had the same error.

Seems you are using nixpacks 0.5.8

Is it possible to get nixpacks 1.4.0 which is a lot newer.

Had a hard time debugging whey something worked locally but not when deploying to fly.

Try deleting ~/.fly/bin/nixpacks, it should re-download the latest nixpacks version.

1 Like

Yeah, got there also

But it seemed that the remote agent was borked, since GitHub deploy would still use a really old version.

Then i started getting this also Current requires cgo or $USER, $HOME set in environment and have now dropped using nixpacks for deployments.