SIGTERM cleanup function not running when VM is shut down

Questions / Help
1

I have a NodeJS app running in 1 VM. Whenever I deploy an update, Fly spins up a new VM and sends a SIGTERM signal to the original one. However, I can’t seem to get a cleanup function to run in my app upon receiving that signal.

I have a simple test function in my app like this:

process.on("SIGTERM", () => console.log("SIGTERM received"))

I don’t see that console log anywhere - I’ve checked my Fly CLI logs and the “Monitoring” section of the Fly UI.

I must be missing something. Any help would be appreciated. Thanks!

2

Hi @gloss

What does your fly.toml file look like?

3

Hi @charsleysa, here’s my fly.toml:

app = "gloss-api"
kill_signal = "SIGTERM"
kill_timeout = 5
processes = []

# Necessary for M1 Macs
[[build.args]]
  PLATFORM = "linux/amd64"

[env]
  CHROMIUM_EXECUTABLE_PATH = "/usr/bin/google-chrome"
  PORT = "8080"

[experimental]
  allowed_public_ports = []
  auto_rollback = true

[[services]]
  http_checks = []
  internal_port = 8080
  processes = ["app"]
  protocol = "tcp"
  script_checks = []
  [services.concurrency]
    hard_limit = 25
    soft_limit = 20
    type = "connections"

  [[services.ports]]
    force_https = true
    handlers = ["http"]
    port = 80

  [[services.ports]]
    handlers = ["tls", "http"]
    port = 443

  [[services.tcp_checks]]
    grace_period = "1s"
    interval = "15s"
    restart_limit = 0
    timeout = "2s"

And here’s my Dockerfile in case that helps:

FROM node:slim as base

ENV PUPPETEER_SKIP_CHROMIUM_DOWNLOAD true

RUN apt-get update \
    && apt-get install -y wget gnupg \
    && wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - \
    && sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list' \
    && apt-get update \
    && apt-get install -y google-chrome-stable fonts-ipafont-gothic fonts-wqy-zenhei fonts-thai-tlwg fonts-kacst fonts-freefont-ttf fonts-noto-color-emoji libxss1 \
      --no-install-recommends \
    && rm -rf /var/lib/apt/lists/*

FROM base as builder

RUN mkdir /app
WORKDIR /app

COPY package*.json ./
COPY tsconfig.json ./

RUN npm install

COPY src ./src

RUN npm run build

FROM base

RUN mkdir /app
WORKDIR /app

COPY package*.json ./

RUN npm install --omit=dev

COPY --from=builder /app/build ./build

CMD ["npm", "run", "start"]
4

Could be that the 5 second kill timeout is too low and the process is being force killed before it has a chance to output your log message. Could try a higher timeout of maybe 30 seconds.

5

Thanks @charsleysa – I tried increasing the kill timeout with no luck.

What did solve it, though was changing the last line of my Docker file to:

CMD ["node", "build/server.js"]

Reason behind this is documented really well in this post

1 Like