ENOENT when unzipping temporary file

Questions / Help
1

I have deployed a Phoenix app with a PG database with the PostGIS extension enabled. I’m trying to run a one-time task on a running instance to seed the database with geometric data. I am downloading .zip files from an external source and saving them to the tmp/ directory, processing and writing the data to the database, and deleting the file. It works locally, but when I run it in production I get an ENOENT when trying to unzip the files. Any suggestions? Thanks in advance!

07:53:41.692 [error] Task #PID<9449.2121.0> started from #PID<9449.2111.0> terminating
** (stop) :enoent
    (elixir 1.13.4) lib/system.ex:1044: System.cmd("unzip", ["/tmp/tl_2021_us_cd116.zip", "-d", "/tmp/exshape_da72c5e8-4581-433b-835a-05f4b17e6f53"], [])
    lib/exshape.ex:40: Exshape.unzip/3
    lib/exshape.ex:100: Exshape.from_zip/2
    lib/pyrex/shapefile.ex:47: PYREx.Shapefile.from_zip/1
    lib/pyrex/shapefile.ex:31: PYREx.Shapefile.map_download/3
    lib/pyrex/loader.ex:17: anonymous fn/2 in PYREx.Loader.run/0
    (elixir 1.13.4) lib/task/supervised.ex:89: Task.Supervised.invoke_mfa/2
    (elixir 1.13.4) lib/task/supervised.ex:34: Task.Supervised.reply/4
Function: &:erlang.apply/2
    Args: [#Function<4.108565059/1 in Pyrex.Loader>, [#Function<11.108565059/0 in Pyrex.Loader>]]
2

You may need to create this directory before you can unzip to it.

3

@kurt Thanks for the response, that doesn’t seem to be it but the suggestion lead me to the right place. The library lets you pass a working_dir option:

tmp_dir = System.tmp_dir!()
working_dir = Path.join([tmp_dir, random_string()])
shapefile = Path.join([tmp_dir, filepath])
File.mkdir!(working_dir)
# download .zip file ...
[{_, _, shapes}] = Exshape.from_zip(shapefile, working_dir: working_dir)
# do stuff, then clean up
File.rm!(shapefile)
File.rm_rf!(working_dir)

I got another ENOENT error with my new custom dir:

** (ErlangError) Erlang error: :enoent
    (elixir 1.13.4) lib/system.ex:1044: System.cmd("unzip", ["/tmp/tl_2021_us_cd116.zip", "-d", "/tmp/GqYqPi2zmbPxYbr1w7LPe9R3CgMJpZKj"], [])

I made sure the directory exists:

iex(pyrex@c7eabe04)1> File.ls!("/tmp")
["GqYqPi2zmbPxYbr1w7LPe9R3CgMJpZKj", "un9e9lSo9ffMk6Wtk1CidCPsHeeM4Ni1",
 "tl_2021_us_cd116.zip"]

I tried maxing the permissions and executing the command manually:

File.chmod!("/tmp/GqYqPi2zmbPxYbr1w7LPe9R3CgMJpZKj", 0o777)
iex(pyrex@c7eabe04)6> System.cmd("unzip", ["/tmp/tl_2021_us_cd116.zip", "-d", "/tmp/GqYqPi2zmbPxYbr1w7LPe9R3CgMJpZKj"], [])
** (ErlangError) Erlang error: :enoent
    (elixir 1.13.4) lib/system.ex:1044: System.cmd("unzip", ["/tmp/tl_2021_us_cd116.zip", "-d", "/tmp/GqYqPi2zmbPxYbr1w7LPe9R3CgMJpZKj"], [])

I exited to the shell and tried it there:

$ unzip tmp/tl_2021_us_cd116.zip -d tmp/GqYqPi2zmbPxYbr1w7LPe9R3CgMJpZKj
/bin/sh: 11: unzip: not found

The unzip command is not available. The Erlang enoent error is very unhelpful, doesn’t seem to make much sense. I’m not sure how to make sure unzip is available in my image. I tried just apt install unzip in the shell but that didn’t work.

$ apt install unzip
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
E: Unable to locate package unzip

$ sudo apt install unzip
/bin/sh: 16: sudo: not found

I’m not very smart with Docker or Linux. Here’s the top of my Dockerfile.

ARG ELIXIR_VERSION=1.13.4
ARG OTP_VERSION=25.0.2
ARG DEBIAN_VERSION=bullseye-20210902-slim

ARG BUILDER_IMAGE="hexpm/elixir:${ELIXIR_VERSION}-erlang-${OTP_VERSION}-debian-${DEBIAN_VERSION}"
ARG RUNNER_IMAGE="debian:${DEBIAN_VERSION}"

FROM ${BUILDER_IMAGE} as builder

# install build dependencies
RUN apt-get update -y && apt-get install -y build-essential git \
  && apt-get clean && rm -f /var/lib/apt/lists/*_*
4

Nevermind, I just had to think for a second. Just adding unzip to the list of installed packages in the runner image did the trick:

FROM ${RUNNER_IMAGE}

RUN apt-get update -y && apt-get install -y unzip libstdc++6 openssl libncurses5 locales \
  && apt-get clean && rm -f /var/lib/apt/lists/*_*
2 Likes
5

Oh yes, good cache. ENOENT is super confusing when the executable is not found.

2 Likes