Dockerfile for elixir/phx umbrella app w/ tailscale, overmind, honeymarker

ryansch · June 24, 2022, 12:18pm

I’ve removed/changed some project specific names but this is what we’ve landed on with one of our production apps that needs to work with resources on AWS.

Dockerfile:

# Find eligible builder and runner images on Docker Hub. We use Ubuntu/Debian instead of
# Alpine to avoid DNS resolution issues in production.
#
# https://hub.docker.com/r/hexpm/elixir/tags?page=1&name=ubuntu
# https://hub.docker.com/_/ubuntu?tab=tags
#
#
# This file is based on these images:
#
#   - https://hub.docker.com/r/hexpm/elixir/tags - for the build image
#   - https://hub.docker.com/_/debian?tab=tags&page=1&name=bullseye-20210902-slim - for the release image
#   - https://pkgs.org/ - resource for finding needed packages
#
ARG BUILDER_IMAGE="hexpm/elixir:1.13.4-erlang-24.3.3-ubuntu-focal-20211006"
ARG RUNNER_IMAGE="ubuntu:focal-20211006"

FROM outstand/su-exec:latest as su-exec

FROM ${RUNNER_IMAGE} as build_honeymarker

ARG HONEYMARKER_VERSION=0.2.7
ENV HONEYMARKER_VERSION=${HONEYMARKER_VERSION}

SHELL ["/bin/bash", "-o", "pipefail", "-c"]
ENV DEBIAN_FRONTEND=noninteractive

RUN set -eux; \
      \
      mkdir -p /tmp/build; \
	    cd /tmp/build; \
      \
      apt-get update -y; \
      apt-get install -y \
        curl \
        ca-certificates \
      ; \
      \
      curl -fsSL https://github.com/honeycombio/honeymarker/releases/download/v${HONEYMARKER_VERSION}/honeymarker-linux-amd64 -o honeymarker; \
      chmod +x honeymarker; \
      mv honeymarker /usr/local/bin/honeymarker; \
      honeymarker --help; \
      \
      apt-get clean; \
      rm -f /var/lib/apt/lists/*_*/ \
      \
      cd; \
      rm -rf /tmp/build

FROM ${BUILDER_IMAGE} as builder

SHELL ["/bin/bash", "-o", "pipefail", "-c"]
ENV DEBIAN_FRONTEND=noninteractive

# install build dependencies
RUN set -eux; \
      \
      apt-get update -y; \
      apt-get install -y \
        curl \
        ca-certificates \
      ; \
      \
      curl -fsSL https://deb.nodesource.com/setup_14.x | bash -; \
      \
      apt-get update -y; \
      apt-get install -y \
        build-essential \
        git \
        nodejs \
        python3 \
      ; \
      \
      curl -sL https://dl.yarnpkg.com/debian/pubkey.gpg | gpg --dearmor | tee /usr/share/keyrings/yarnkey.gpg >/dev/null; \
      echo "deb [signed-by=/usr/share/keyrings/yarnkey.gpg] https://dl.yarnpkg.com/debian stable main" | tee /etc/apt/sources.list.d/yarn.list; \
      apt-get update -y; \
      apt-get install -y \
        yarn \
      ; \
      apt-get clean; \
      rm -f /var/lib/apt/lists/*_*

# prepare build dir
WORKDIR /app

# install hex + rebar
RUN mix local.hex --force && \
    mix local.rebar --force

# set build ENV
ENV MIX_ENV="prod"

# install mix dependencies
COPY mix.exs mix.lock ./
COPY apps/app_1/mix.exs apps/app_1/mix.exs
COPY apps/app_2/mix.exs apps/app_2/mix.exs
COPY apps/app_web/mix.exs apps/app_web/mix.exs
RUN mix deps.get --only $MIX_ENV
RUN mkdir config

# copy compile-time config files before we compile dependencies
# to ensure any relevant config change will trigger the dependencies
# to be re-compiled.
COPY config/config.exs config/${MIX_ENV}.exs config/
RUN mix deps.compile

COPY apps/app_web/priv apps/app_web/priv

COPY apps/app_1/lib apps/app_1/lib
COPY apps/app_2/lib apps/app_2/lib
COPY apps/app_web/lib apps/app_web/lib

COPY apps/app_web/assets apps/app_web/assets

# compile assets
RUN mix do \
  assets.install, \
  assets.deploy

# Compile the release
RUN mix compile

# Changes to config/runtime.exs don't require recompiling the code
COPY config/runtime.exs config/

COPY rel rel
RUN mix release

# start a new build stage so that the final image will only contain
# the compiled release and other runtime necessities
FROM ${RUNNER_IMAGE}

ARG TAILSCALE_VERSION=1.26.1
ARG OVERMIND_VERSION=2.2.2

COPY --from=su-exec /sbin/su-exec /sbin/su-exec
COPY --from=build_honeymarker /usr/local/bin/honeymarker /usr/local/bin/honeymarker

ENV TAILSCALE_VERSION=${TAILSCALE_VERSION}
RUN set -eux; \
      \
      apt-get update -y; \
      apt-get install -y \
        curl \
        ca-certificates \
      ; \
      curl -fsSL https://pkgs.tailscale.com/stable/debian/bullseye.noarmor.gpg | tee /usr/share/keyrings/tailscale-archive-keyring.gpg >/dev/null; \
      curl -fsSL https://pkgs.tailscale.com/stable/debian/bullseye.tailscale-keyring.list | tee /etc/apt/sources.list.d/tailscale.list; \
      \
      apt-get update -y; \
      apt-get install -y \
        libstdc++6 \
        openssl \
        libncurses5 \
        locales \
        nftables \
        tailscale=${TAILSCALE_VERSION} \
        tmux \
      ; \
      apt-get clean; \
      rm -f /var/lib/apt/lists/*_*

ENV OVERMIND_VERSION=${OVERMIND_VERSION}
RUN set -eux; \
      \
      mkdir -p /tmp/build; \
	    cd /tmp/build; \
      curl -fsSL https://github.com/DarthSim/overmind/releases/download/v${OVERMIND_VERSION}/overmind-v${OVERMIND_VERSION}-linux-amd64.gz | gunzip > overmind; \
      mv overmind /usr/bin/overmind; \
      chmod +x /usr/bin/overmind; \
      cd; \
      rm -rf /tmp/build

# Set the locale
RUN sed -i '/en_US.UTF-8/s/^# //g' /etc/locale.gen && locale-gen

ENV LANG en_US.UTF-8
ENV LANGUAGE en_US:en
ENV LC_ALL en_US.UTF-8

WORKDIR "/app"
COPY docker/Procfile.fly /app/Procfile
COPY docker/tailscale-up.sh docker/wait-for-tailscale.sh /app/docker/
RUN chown -R nobody /app

# Only copy the final release from the build stage
COPY --from=builder --chown=nobody:root /app/_build/prod/rel/umbrella ./

COPY docker/fly-entrypoint.sh /docker-entrypoint.sh

ENV OVERMIND_NO_PORT=1
ENV OVERMIND_CAN_DIE=tailscaleup
ENV OVERMIND_STOP_SIGNALS="umbrella=TERM"

ENTRYPOINT ["/docker-entrypoint.sh"]
CMD ["overmind", "start"]

# Appended by flyctl
ENV ECTO_IPV6 true
ENV ERL_AFLAGS "-proto_dist inet6_tcp"

ARG vcs_ref
LABEL org.label-schema.vcs-ref=$vcs_ref \j
  org.label-schema.vcs-url="https://github.com/outstand/umbrella-app" \
  SERVICE_TAGS=$vcs_ref
ENV VCS_REF ${vcs_ref}
ENV APP_REVISION ${vcs_ref}

fly-entrypoint.sh:

#!/bin/bash

set -euo pipefail

echo 'net.ipv4.ip_forward = 1' | tee -a /etc/sysctl.conf
echo 'net.ipv6.conf.all.forwarding = 1' | tee -a /etc/sysctl.conf
sysctl -p /etc/sysctl.conf

exec "$@"

Procfile.fly:

tailscaled: tailscaled --verbose=1 --port 41641
tailscaleup: /app/docker/tailscale-up.sh
umbrella: /app/docker/wait-for-tailscale.sh su-exec nobody /app/bin/server

tailscale-up.sh:

#!/bin/bash

set -euo pipefail

mkdir -p /var/lock/tailscale/

# From: https://gist.github.com/przemoc/571091
# SPDX-License-Identifier: MIT
 
## Copyright (C) 2009 Przemyslaw Pawelczyk <przemoc@gmail.com>
##
## This script is licensed under the terms of the MIT license.
## https://opensource.org/licenses/MIT
#
# Lockable script boilerplate
 
### HEADER ###
 
LOCKFILE="/var/lock/tailscale/up.lock"
LOCKFD=99
 
# PRIVATE
_lock()             { flock -$1 $LOCKFD; }
_no_more_locking()  { _lock u; _lock xn && rm -f $LOCKFILE; }
_prepare_locking()  { eval "exec $LOCKFD>\"$LOCKFILE\""; trap _no_more_locking EXIT; }
 
# ON START
_prepare_locking
 
# PUBLIC
exlock_now()        { _lock xn; }  # obtain an exclusive lock immediately or fail
exlock()            { _lock x; }   # obtain an exclusive lock
shlock()            { _lock s; }   # obtain a shared lock
unlock()            { _lock u; }   # drop a lock
 
### BEGIN OF SCRIPT ###
 
# Remember! Lock file is removed when one of the scripts exits and it is
#           the only script holding the lock or lock is not acquired at all.

exlock
rm -f /var/lock/tailscale/up.ran
unlock

echo "Waiting for /var/run/tailscale/tailscaled.sock..."
until [ -S /var/run/tailscale/tailscaled.sock ]; do
  sleep 0.1
done
echo "Done waiting!"

tailscale up \
  "--authkey=${TAILSCALE_AUTHKEY}" \
  "--hostname=CHANGEME-${FLY_REGION}" \
  --accept-routes=true

exlock
touch /var/lock/tailscale/up.ran
unlock

wait-for-tailscale.sh:

#!/bin/bash

set -euo pipefail

mkdir -p /var/lock/tailscale/

# From: https://gist.github.com/przemoc/571091
# SPDX-License-Identifier: MIT
 
## Copyright (C) 2009 Przemyslaw Pawelczyk <przemoc@gmail.com>
##
## This script is licensed under the terms of the MIT license.
## https://opensource.org/licenses/MIT
#
# Lockable script boilerplate
 
### HEADER ###
 
LOCKFILE="/var/lock/tailscale/up.lock"
LOCKFD=99
 
# PRIVATE
_lock()             { flock -$1 $LOCKFD; }
_no_more_locking()  { _lock u; _lock xn && rm -f $LOCKFILE; }
_prepare_locking()  { eval "exec $LOCKFD>\"$LOCKFILE\""; trap _no_more_locking EXIT; }
 
# ON START
_prepare_locking
 
# PUBLIC
exlock_now()        { _lock xn; }  # obtain an exclusive lock immediately or fail
exlock()            { _lock x; }   # obtain an exclusive lock
shlock()            { _lock s; }   # obtain a shared lock
unlock()            { _lock u; }   # drop a lock
 
### BEGIN OF SCRIPT ###
 
# Remember! Lock file is removed when one of the scripts exits and it is
#           the only script holding the lock or lock is not acquired at all.

echo "Waiting for tailscale up..."

exlock

until [ -f /var/lock/tailscale/up.ran ]; do
  unlock
  sleep 0.1
  exlock
done

unlock

echo "Done waiting!"

exec "$@"

megalithic · June 24, 2022, 8:41pm

!!

zachallaun · June 26, 2022, 9:04pm

Thank you for this, @ryansch!

In case it’s helpful for anyone else, I got this up and running in a minimal, new Phoenix project here: zachallaun/flytail

I removed a couple of things and made a few changes compared to OP’s version:

Use a standard, non-umbrella app (refer to OP’s version for extra build steps if using umbrella app)
Use latest Elixir/Erlang (1.13.4/25.0.2)
Remove honeymarker
Switch from su-exec to gosu (just to avoid relying on outstand/su-exec:latest)
Assets: only run mix assets.deploy, not assets.install (not there for brand new Phoenix app)
Enable Tailscale SSH

Quick “get started” guide to use:

Files you care about are Dockerfile and everything inside docker/
Make sure all .sh files inside docker/ are executable (chmod +x) to avoid somewhat opaque errors
Generate an ephemeral/reusable Tailscale auth token and run fly secrets set TAILSCALE_AUTHKEY=...
Update the REPOSITORY in Dockerfile and reference to flytail at bottom
Launch/deploy and debug the inevitable bits I missed. =)

Thanks again @ryansch for the template!

modellurgist · January 25, 2023, 12:14pm

trying this with Debian, instead of Ubuntu, and running into some issues, if you want to follow here: Deploy hanging at "Monitoring deployment" when using Overmind to provide DB access through Tailscale to CrunchyBridge

Topic		Replies	Views
Problems upgrading the Elixir version for my (phoenix) app Phoenix elixir	2	83	September 16, 2024
(mix phx.gen.release --docker) (RuntimeError) couldn't fetch https://hub.docker.com/v2/namespaces/hexpm/repositories/elixir/tags?name=1.15.7-erlang-26.0.2-debian-bullseye- Questions / Help elixir , phoenix-files	14	533	December 13, 2023
Possible bug report - default build dockerfile doesn't exist Phoenix elixir	5	500	September 19, 2022
Problems with the Elixir tutorial	7	777	September 27, 2022
Fly deploy failing after changing Dockerfile Questions / Help elixir	2	500	July 7, 2023

Dockerfile for elixir/phx umbrella app w/ tailscale, overmind, honeymarker

Related topics