How to swap properly? "Out of memory: Killed process (node)" error on deploy

1

I’ve got two postgres apps, staging and prod. The prod one is pretty beefy, but the staging instance is the skimpiest one, because it really doesn’t get any traffic.

But it keeps running out of memory when I deploy, because I think a Prisma migration task. It’s run via the deploy command:

[deploy]
  release_command = "npx prisma migrate deploy"

In the docs it says:

This command runs in a temporary VM - using the successfully built release - before that release is deployed.

So, am I running out of memory on this temporary VM or in my main machine?

One solution

I used a start.sh script to set some swap and run the needed scripts:

fallocate -l 512M /swapfile
chmod 0600 /swapfile
mkswap /swapfile
echo 10 > /proc/sys/vm/swappiness
swapon /swapfile

npx prisma migrate deploy

# Turn off swap
swapoff /swapfile
rm /swapfile

npm run start

fly.toml:

[experimental]
  cmd = "start.sh"
  entrypoint = "sh"

Dockerfile:

COPY --from=build /myapp/start.sh /myapp/start.sh
COPY --from=build /myapp/prisma /myapp/prisma
ADD . .
ENTRYPOINT [ "./start.sh" ]

Is this the “correct” way to do it? I mean it works so I’m happy.

2

Hi!

Yep, that sounds like it’s the temporary VM failing.

That’s also the way we would setup swap as well - in the ENTRYPOINT script (err, well in your case, you set it as CMD, but if it works, that’s cool).

The key point is that you can’t setup swap in the Dockerfile when creating the image that gets deployed, but instead need to do that when the VM is booted, and hence the ENTRYPOINT or CMD is the place to do it :+1:

One note:

Your Dockerfile defines ENTRYPOINT as start.sh but then fly.toml is over-riding that by setting sh as ENTRYPOINT and start.sh as the CMD.

You could instead:

  1. Keep start.sh as entrypoint as defined in the Dockerfile, perhaps also set the full path (/myapp/start.sh) instead of relative path ./start.sh.
  2. Ensure start.sh is executable (chmod +x /myapp/start.sh)
  3. Ensure start.sh will use the correct shell by prepending #! /bin/sh to the top of the file
  4. Remove the [experimental] section from fly.toml so the settings in Dockerfile are used

That’s optional but is just a bit cleaner.

2 Likes
3

Thanks for the tips!

1 Like
4

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.