Elixir Phoenix - ERRO[0607] Can't close tar writer: io: read/write on closed pipe

Just trying out Fly using the Elixir guides and a freshly generated phoenix project - following to the letter but getting the following error -

Deploying winter-sun-9690

==> Validating app configuration

→ Validating app configuration done

Services

TCP 80/443 ⇢ 4000

Remote builder fly-builder-throbbing-wildflower-986 ready

==> Creating build context

→ Creating build context done

==> Building image with Docker

→ docker host: 20.10.8 linux x86_64

ERRO[0607] Can’t add file /Users/tcg/Source/firefly/.elixir_ls/dialyzer_manifest_24.1.2_elixir-1.12.3_test to tar: io: read/write on closed pipe

ERRO[0607] Can’t close tar writer: io: read/write on closed pipe

Error error building: error building with docker: error during connect: Post “http://[fdaa:0:36eb:a7b:2808:0:4b61:2]:2375/v1.41/build?buildargs=%7B%7D&cachefrom=null&cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=&cpushares=0&dockerfile=Dockerfile&labels=null&memory=0&memswap=0&networkmode=&platform=linux%2Famd64&rm=0&shmsize=0&t=registry.fly.io%2Fwinter-sun-9690%3Adeployment-1634722661&target=&ulimits=null&version=”: write unix ->/Users/tcg/.fly/fly-agent.sock: use of closed network connection

Did this happen multiple times? This seems like a network issue interfering with our upload to the Docker builder. Any idea how long it sat here before it failed?

==> Building image with Docker

→ docker host: 20.10.8 linux x86_64

Yeah I’ve tried quite a few times with various small config tweaks, made sure the docker image builds locally etc. It spits out the message after about 5 minutes. Edit: I’ve just tested it again and it’s almost instantly after the build context is finished.

If it does build locally, a workaround is to run fly deploy --local-only. This will use your local Docker to build and push, bypassing our remote Docker setup (which is substantially more complicated).

I went ahead and removed your remote builder. I don’t think that will help, but feel free to try a remote build again and see how it goes.

I’m now getting some weird qemu: uncaught target signal 11 (Segmentation fault) - core dumped error possibly due to using an arm Mac. I might spin up a remote vm to see if building and deploying from there helps. Does fly have a consistent underlying architecture/distro I should copy?

fly deploy --local-only
Deploying winter-sun-9690
==> Validating app configuration
→ Validating app configuration done
Services
TCP 80/443 ⇢ 4000
==> Creating build context
→ Creating build context done
==> Building image with Docker
→ docker host: 20.10.8 linux aarch64
[+] Building 6.0s (0/1)
[+] Building 252.1s (17/22)
=> [internal] load remote build context 0.0s
=> copy /context / 1.1s
=> [internal] load metadata for docker.io/library/alpine:3.13.3 2.1s
=> [internal] load metadata for docker.io/hexpm/elixir:1.12.1-erlang-24.0.1-alpine-3.13.3 2.1s
=> [build 1/12] FROM docker.io/hexpm/elixir:1.12.1-erlang-24.0.1-alpine-3.13.3@sha256:4eefe11eb82f277cd82ba5d 0.0s
=> [app 1/5] FROM docker.io/library/alpine:3.13.3@sha256:826f70e0ac33e99a72cf20fb0571245a8fee52d68cb26d8bc58e5 0.0s
=> CACHED [app 2/5] RUN apk add --no-cache libstdc++ openssl ncurses-libs 0.0s
=> CACHED [app 3/5] WORKDIR /app 0.0s
=> CACHED [app 4/5] RUN chown nobody:nobody /app 0.0s
=> CACHED [build 2/12] RUN apk add --no-cache build-base npm 0.0s
=> CACHED [build 3/12] WORKDIR /app 0.0s
=> CACHED [build 4/12] RUN mix local.hex --force && mix local.rebar --force 0.0s
=> CACHED copy /context / 0.0s
=> CACHED [build 5/12] COPY mix.exs mix.lock ./ 0.0s
=> CACHED [build 6/12] COPY config config 0.0s
=> CACHED [build 7/12] COPY priv priv 0.0s
=> CACHED [build 8/12] COPY assets assets 0.0s
=> [build 9/12] RUN mix phx.digest 248.7s
=> => # qemu: uncaught target signal 11 (Segmentation fault) - core dumped

Ah that’s a QEMU / Docker on m1 bug. If you haven’t upgraded Docker lately, that might help with local builds.

flyctl builds x86_64 images, any Linux host with x86_64 should work fine.

1 Like

Yeah definitely have the latest version, Docker Desktop 4.1.1 (69879) is currently the newest version available. They do mention in the original post back in Feb that it’s a Apple Silicon preview and If you are trying to run Intel-based containers on an M1 machine, they can sometimes crash. We are using a piece of software called qemu to emulate Intel chips on M1 but it occasionally fails to run the container. Where possible we recommend sticking to arm64 containers on M1 machines; they will also be faster. Obviously an additional build step less optimal than just building it on your servers as part of deployment. Can I just double check that the documentation on Fly is up to date since the change to using esbuild? I did notice some references in the Dockerfile were giving me errors. Elixir and Phoenix do seem particularly tricky to deploy in general and was really hoping to finally something quick enough to use for prototyping through to small scale production. Thanks for the quick reply by the way - much appreciated.

I don’t think we’ve updated the docs for es build yet. @mark or @joshua would know for sure.

Did you give the remote build another shot after I deleted the builder earlier?

Still not happening for me, but I’m not going to give up - I’ll rewrite the Dockerfile and deploy from a remote x86_64 vm.

Hey! We don’t have official docs on deploying Phoenix apps yet, but we have a known, working example here.

The Dockerfile there should work for the latest esbuild setup.

Cheers Josh, I’ll try it now. Your main docs do actually use a phoenix app - Build, Deploy and Run an Elixir Application

Sorry, I miswrote. I meant that we don’t have docs for the new esbuild approach, linking to that repo. We’ll get that page updated soon!

1 Like

Still no dice, although it got much further. I tried first to copy the Dockerfile and redeploy my app. Then I cloned the example app tested it locally (without docker) then went through the deploy process and same error for both -

tcg@mbp hello-phoenix % fly deploy
Deploying wispy-cherry-1895
==> Validating app configuration
→ Validating app configuration done
Services
TCP 80/443 ⇢ 8080
==> Creating build context
→ Creating build context done
==> Building image with Docker
→ docker host: 20.10.8 linux aarch64
[+] Building 0.0s (0/1)
[+] Building 4.2s (16/29)
=> [internal] load remote build context 0.0s
=> copy /context / 0.1s
=> [internal] load metadata for docker.io/library/ubuntu:20.10 1.1s
=> [internal] load metadata for docker.io/hexpm/elixir:1.12.3-erlang-24.1.2-ubuntu-groovy-20210325 1.2s
=> [builder 1/17] FROM docker.io/hexpm/elixir:1.12.3-erlang-24.1.2-ubuntu-groovy-20210325@sha256:716 0.0s
=> [stage-1 1/7] FROM docker.io/library/ubuntu:20.10@sha256:a7b08558af07bcccca994b01e1c84f1d14a2156e0 0.0s
=> CACHED [stage-1 2/7] RUN apt-get update -y && apt-get install -y libstdc++6 openssl curl libncurse 0.0s
=> CACHED [stage-1 3/7] RUN sed -i ‘/en_US.UTF-8/s/^# //g’ /etc/locale.gen && locale-gen 0.0s
=> CACHED [stage-1 4/7] WORKDIR /app 0.0s
=> CACHED [stage-1 5/7] RUN chown nobody /app 0.0s
=> CACHED [builder 2/17] RUN apt-get update -y && apt-get install -y build-essential git python3 cur 0.0s
=> CACHED [builder 3/17] WORKDIR /app 0.0s
=> CACHED [builder 4/17] RUN mix local.hex --force && mix local.rebar --force 0.0s
=> CACHED copy /context / 0.0s
=> [builder 5/17] COPY mix.exs mix.lock ./ 0.0s
=> ERROR [builder 6/17] RUN mix deps.get --only prod 2.9s

[builder 6/17] RUN mix deps.get --only prod:
#16 2.837
#16 2.837 03:31:57.103 [info] Application hex exited: exited in: Hex.Application.start(:normal, [])
#16 2.837 ** (EXIT) an exception was raised:
#16 2.837 ** (MatchError) no match of right hand side value: :reduce_ok
#16 2.837 (elixir 1.12.3) src/elixir_def.erl:102: :elixir_def.store_definition/5
#16 2.837 (mix 1.12.3) lib/mix/scm.ex:130: Mix.SCM.available/0
#16 2.837 (hex 0.21.3) lib/hex/application.ex:9: Hex.Application.start/2
#16 2.837 (kernel 8.1.2) application_master.erl:293: :application_master.start_it_old/4
#16 2.841
#16 2.841 03:31:57.169 [info] Application inets exited: :stopped
#16 2.845
#16 2.845 03:31:57.174 [info] Application ssl exited: :stopped
#16 2.845
#16 2.845 03:31:57.174 [info] Application public_key exited: :stopped
#16 2.845
#16 2.845 03:31:57.174 [info] Application asn1 exited: :stopped
#16 2.845
#16 2.845 03:31:57.174 [info] Application crypto exited: :stopped
#16 2.850 Could not start Hex. Try fetching a new version with “mix local.hex” or uninstalling it with “mix archive.uninstall hex.ez”
#16 2.859 ** (MatchError) no match of right hand side value: {:error, {:hex, {:bad_return, {{Hex.Application, :start, [:normal, []]}, {:EXIT, {{:badmatch, :reduce_ok}, [{:elixir_def, :store_definition, 5, [file: ‘src/elixir_def.erl’, line: 102]}, {Mix.SCM, :available, 0, [file: ‘lib/mix/scm.ex’, line: 130]}, {Hex.Application, :start, 2, [file: ‘lib/hex/application.ex’, line: 9]}, {:application_master, :start_it_old, 4, [file: ‘application_master.erl’, line: 293]}]}}}}}}
#16 2.859 (hex 0.21.3) lib/hex.ex:5: Hex.start/0
#16 2.859 (mix 1.12.3) lib/mix/hex.ex:59: Mix.Hex.start/0
#16 2.859 (mix 1.12.3) lib/mix/dep/loader.ex:194: Mix.Dep.Loader.with_scm_and_app/4
#16 2.859 (mix 1.12.3) lib/mix/dep/loader.ex:147: Mix.Dep.Loader.to_dep/3
#16 2.859 (elixir 1.12.3) lib/enum.ex:1582: Enum."-map/2-lists^map/1-0-"/2
#16 2.859 (mix 1.12.3) lib/mix/dep/loader.ex:364: Mix.Dep.Loader.mix_children/2
#16 2.859 (mix 1.12.3) lib/mix/dep/loader.ex:18: Mix.Dep.Loader.children/0
#16 2.859 (mix 1.12.3) lib/mix/dep/converger.ex:57: Mix.Dep.Converger.all/4


Error error building: executor failed running [/bin/sh -c mix deps.get --only $MIX_ENV]: exit code: 1

Is it possible for someone to do a sanity test and confirm the example app deploys successfully from their machine?

==> Building image with Docker
→ docker host: 20.10.8 linux aarch64

This seems to be an issue so I used deploy --remote-only and it started building for x86_64 correctly and got much further before spitting out a different error -

Step 33/35 : COPY --from=builder --chown=nobody /app/_build/"${MIX_ENV}"/rel ./

Error error building: error rendering build status stream: unable to convert uid/gid chown string to host mapping: can’t find gid for group nobody: no such group: nobody

which seems related to the Dockerfile. Still would appreciate someone else confirming they got it to build and deploy :slight_smile:

Did you try cloning and deploying the example app? It works for me.

That’s exactly what I did. I’ll try again.

Are you using fly deploy --remote-only?

Yes, because I’m using an M1 Mac so if I don’t add the --remote-only flag it tries to build for aarch64.

Can you post your entire deployment output?

tcg@mbp hello-phoenix % fly launch

An existing fly.toml file was found for app hello-phoenix

? Would you like to copy its configuration to the new app? Yes

Creating app in /Users/tcg/Source/hello-phoenix

Scanning source code

Detected a Dockerfile app

? App Name (leave blank to use an auto-generated name): hello-firefly

Automatically selected personal organization: Tom Gosling

? Select region: lhr (London, United Kingdom)

Created app hello-firefly in organization personal

Wrote config file fly.toml

Your app is ready. Deploy with flyctl deploy

? Would you like to deploy now? No

tcg@mbp hello-phoenix % open .

tcg@mbp hello-phoenix % st .

tcg@mbp hello-phoenix % mix phx.gen.secret

05:29:29.467 [info] Compiling file system watcher for Mac…

05:29:30.222 [info] Done.

==> file_system

Compiling 7 files (.ex)

Generated file_system app

==> connection

Compiling 1 file (.ex)

Generated connection app

==> gettext

Compiling 1 file (.erl)

Compiling 20 files (.ex)

Generated gettext app

===> Analyzing applications…

===> Compiling ranch

===> Analyzing applications…

===> Compiling telemetry

==> telemetry_metrics

Compiling 7 files (.ex)

Generated telemetry_metrics app

===> Analyzing applications…

===> Compiling telemetry_poller

==> decimal

Compiling 4 files (.ex)

Generated decimal app

==> jason

Compiling 8 files (.ex)

Generated jason app

==> castore

Compiling 1 file (.ex)

Generated castore app

==> esbuild

Compiling 3 files (.ex)

Generated esbuild app

==> db_connection

Compiling 14 files (.ex)

Generated db_connection app

==> ecto

Compiling 56 files (.ex)

Generated ecto app

==> phoenix_pubsub

Compiling 11 files (.ex)

Generated phoenix_pubsub app

===> Analyzing applications…

===> Compiling cowlib

===> Analyzing applications…

===> Compiling cowboy

===> Analyzing applications…

===> Compiling cowboy_telemetry

==> mime

Compiling 1 file (.ex)

Generated mime app

==> postgrex

Compiling 62 files (.ex)

Generated postgrex app

==> ecto_sql

Compiling 26 files (.ex)

Generated ecto_sql app

==> plug_crypto

Compiling 5 files (.ex)

Generated plug_crypto app

==> plug

Compiling 1 file (.erl)

Compiling 41 files (.ex)

warning: System.stacktrace/0 is deprecated. Use STACKTRACE instead

lib/plug/conn/wrapper_error.ex:23: Plug.Conn.WrapperError.reraise/3

Generated plug app

==> phoenix_html

Compiling 8 files (.ex)

Generated phoenix_html app

==> phoenix_view

Compiling 5 files (.ex)

Generated phoenix_view app

==> plug_cowboy

Compiling 5 files (.ex)

Generated plug_cowboy app

==> phoenix

Compiling 68 files (.ex)

Generated phoenix app

==> phoenix_live_reload

Compiling 4 files (.ex)

Generated phoenix_live_reload app

==> phoenix_live_view

Compiling 28 files (.ex)

Generated phoenix_live_view app

==> phoenix_live_dashboard

Compiling 40 files (.ex)

Generated phoenix_live_dashboard app

==> swoosh

Compiling 36 files (.ex)

Generated swoosh app

==> phoenix_ecto

Compiling 7 files (.ex)

Generated phoenix_ecto app

==> hello_phoenix

bJIQpAIjwZjMqVglY2b8aR0hSqZNl+UwulNlvE8hJYMMqepLzteaiQssdFGe4KGs

tcg@mbp hello-phoenix % fly secrets set SECRET_KEY_BASE=bJIQpAIjwZjMqVglY2b8aR0hSqZNl+UwulNlvE8hJYMMqepLzteaiQssdFGe4KGs

Secrets are staged for the first deployment

tcg@mbp hello-phoenix % fly postgres create

? App Name: hello-firefly-db

Automatically selected personal organization: Tom Gosling

? Select region: lhr (London, United Kingdom)

? Select VM size: shared-cpu-1x - 256

? Volume size (GB): 10

Creating postgres cluster hello-firefly-db in organization personal

Postgres cluster hello-firefly-db created

Username: postgres

Password: 3420c6a2f99b8dc7c6f0bfc3f60042dbc6de29a8c0c53b9f

Hostname: hello-firefly-db.internal

Proxy Port: 5432

PG Port: 5433

Save your credentials in a secure place, you won’t be able to see them again!

Monitoring Deployment

2 desired, 2 placed, 2 healthy, 0 unhealthy [health checks: 6 total, 6 passing]

→ v0 deployed successfully

Connect to postgres

Any app within the personal organization can connect to postgres using the above credentials and the hostname “hello-firefly-db.internal.”

For example: postgres://postgres:3420c6a2f99b8dc7c6f0bfc3f60042dbc6de29a8c0c53b9f@hello-firefly-db.internal:5432

See the postgres docs for more information on next steps, managing postgres, connecting from outside fly: Postgres on Fly

tcg@mbp hello-phoenix % fly postgres attach --postgres-app hello-firefly-db

Postgres cluster hello-firefly-db is now attached to hello-firefly

The following secret was added to hello-firefly:

DATABASE_URL=postgres://hello_firefly_kzor184wdoxemvj5:ffbe30905abe7095b5c10d1499834f3a@hello-firefly-db.internal:5432/hello_firefly?sslmode=disable

tcg@mbp hello-phoenix % fly deploy --remote-only

Deploying hello-firefly

==> Validating app configuration

→ Validating app configuration done

Services

TCP 80/443 ⇢ 8080

Remote builder fly-builder-rough-sea-6798 ready

==> Creating build context

→ Creating build context done

==> Building image with Docker

→ docker host: 20.10.8 linux x86_64

Sending build context to Docker daemon 4.482MB

Step 1/35 : ARG BUILDER_IMAGE=“hexpm/elixir:1.12.3-erlang-24.1.2-ubuntu-groovy-20210325”

Step 2/35 : ARG RUNNER_IMAGE=“ubuntu:20.10”

Step 3/35 : ARG MIX_ENV=“prod”

Step 4/35 : FROM ${BUILDER_IMAGE} as builder

—> 2c890f6fd283

Step 5/35 : RUN apt-get update -y && apt-get install -y build-essential git python3 curl && apt-get clean && rm -f /var/lib/apt/lists/_

—> Using cache

—> 3442dfa0e6ad

Step 6/35 : WORKDIR /app

—> Using cache

—> 892bf729c8c3

Step 7/35 : RUN mix local.hex --force && mix local.rebar --force

—> Using cache

—> 16337a1600a6

Step 8/35 : ARG MIX_ENV

—> Using cache

—> 09485b7eb5da

Step 9/35 : ENV MIX_ENV="${MIX_ENV}"

—> Using cache

—> e123b4a0bc06

Step 10/35 : COPY mix.exs mix.lock ./

—> Using cache

—> f348bff2d66c

Step 11/35 : RUN mix deps.get --only $MIX_ENV

—> Using cache

—> 4cc6b88e0cef

Step 12/35 : RUN mkdir config

—> Using cache

—> 5924dea41dd7

Step 13/35 : COPY config/config.exs config/$MIX_ENV.exs config/

—> Using cache

—> f0add00dd678

Step 14/35 : RUN mix deps.compile

—> Using cache

—> a74179ef647d

Step 15/35 : COPY priv priv

—> 47e26ad3e033

Step 16/35 : COPY assets assets

—> 214969cdc5d9

Step 17/35 : RUN mix assets.deploy

—> Running in 4ca9bb908285

04:35:46.138 [debug] Downloading esbuild from https://registry.npmjs.org/esbuild-linux-64/-/esbuild-linux-64-0.12.18.tgz

…/priv/static/assets/app.js 79.1kb

…/priv/static/assets/app.css 10.8kb

:zap: Done in 27ms

Generated hello_phoenix app

Check your digested files at “priv/static”

—> 05d758c0f174

Step 18/35 : COPY lib lib

—> 76e7ecc75a60

Step 19/35 : RUN mix compile

—> Running in 89a98d850bd3

Compiling 15 files (.ex)

Generated hello_phoenix app

—> 8f20d2edcf58

Step 20/35 : COPY config/runtime.exs config/

—> dd2317fe09e8

Step 21/35 : COPY rel rel

—> 1451d881fb8a

Step 22/35 : RUN mix release

—> Running in adb42426a3d1

  • assembling hello_phoenix-0.1.0 on MIX_ENV=prod

  • using config/runtime.exs to configure the release at runtime

  • creating _build/prod/rel/hello_phoenix/releases/0.1.0/vm.args

  • creating _build/prod/rel/hello_phoenix/releases/0.1.0/remote.vm.args

  • creating _build/prod/rel/hello_phoenix/releases/0.1.0/env.sh

  • creating _build/prod/rel/hello_phoenix/releases/0.1.0/env.bat

  • skipping elixir.bat for windows (bin/elixir.bat not found in the Elixir installation)

  • skipping iex.bat for windows (bin/iex.bat not found in the Elixir installation)

Release created at _build/prod/rel/hello_phoenix!

To start your system

_build/prod/rel/hello_phoenix/bin/hello_phoenix start

Once the release is running:

To connect to it remotely

_build/prod/rel/hello_phoenix/bin/hello_phoenix remote

To stop it gracefully (you may also send SIGINT/SIGTERM)

_build/prod/rel/hello_phoenix/bin/hello_phoenix stop

To list all commands:

_build/prod/rel/hello_phoenix/bin/hello_phoenix

—> c861af3ef0f0

Step 23/35 : FROM ${RUNNER_IMAGE}

—> e508bd6d694e

Step 24/35 : ARG MIX_ENV

—> Using cache

—> 72397d377000

Step 25/35 : RUN apt-get update -y && apt-get install -y libstdc++6 openssl curl libncurses5 locales && apt-get clean && rm -f /var/lib/apt/lists/_

—> Using cache

—> 5520da1fbb3f

Step 26/35 : RUN sed -i ‘/en_US.UTF-8/s/^# //g’ /etc/locale.gen && locale-gen

—> Using cache

—> 5bfd6c84cab7

Step 27/35 : ENV LANG en_US.UTF-8

—> Using cache

—> bc95485a0a9d

Step 28/35 : ENV LANGUAGE en_US:en

—> Using cache

—> c97ce1fd319e

Step 29/35 : ENV LC_ALL en_US.UTF-8

—> Using cache

—> e7a73ca6210d

Step 30/35 : WORKDIR “/app”

—> Using cache

—> 7b2570c02518

Step 31/35 : RUN chown nobody /app

—> Using cache

—> fda2c889a59b

Step 32/35 : USER nobody

—> Using cache

—> afd1c2dc27ce

Step 33/35 : COPY --from=builder --chown=nobody /app/_build/"${MIX_ENV}"/rel ./

Error error building: error rendering build status stream: unable to convert uid/gid chown string to host mapping: can’t find gid for group nobody: no such group: nobody