Litefs database migrations with multiple machines in primary - how to determine writeable primary machine?

With litefs, we have the below docker-entrypoint for a rails setup. It checks the primary region, and does a migration. The issue is, that we have multiple machines in the primary region, and the migrations can fail at times. Is there a way (with an ENV or otherwise) to modify this entrypoint to make sure we’re only running migrations on the “primary” machine in the primary region?

This is using the default fly-apps/dockerfile-rails setup, so I think others will run into this eventually as well.

#!/bin/bash -e

# mount litefs
sudo -E litefs mount &

# If running the rails server then create or migrate existing database
if [ "${1}" == "./bin/rails" ] && [ "${2}" == "server" ] && [ "$FLY_REGION" == "$PRIMARY_REGION" ]; then
  ./bin/rails db:prepare
fi

exec "${@}"

Added litefs

Added rails

If you run LiteFS as the supervisor for your Rails app (e.g. litefs mount starts and it calls rails server) then you can set it up in the litefs.yml to only run certain commands on the candidate nodes. You can set lease.promote to true to automatically promote nodes as they start up which will ensure that the command is run by the primary.

That’s exactly what I needed. Thank you for the help!

To save me the trouble of repeating your steps, any chance you can outline the changes? Or even better, author a pull request for the rails dockerfile generator? :slight_smile:

Was going to try a PR, but I think it might be quicker for you as you’re more familiar. The changes I made were to the last few lines of the Dockerfile.

# Deployment options
ENV DATABASE_URL="sqlite3:///litefs/production.sqlite3" \
    PORT="3001"

ENTRYPOINT litefs mount

# Start the server by default, this can be overwritten at runtime
EXPOSE 3000
VOLUME /data

No longer need the bin/docker-entrypoint with litefs.

Then add the following to the litefs.yml at the bottom:

exec:
  # Only run migrations on candidate nodes.
  - cmd: "./bin/rails db:prepare"
    if-candidate: true

  # Then run the application server on all nodes.
  - cmd: "./bin/rails server"

And in the lease section add promote: true

lease:
  promote: true

  # Required. Must be either "consul" or "static".
  type: "consul"   

I haven’t tested this extensively, but the server starts cleanly, so I presume it’s on the right path.

Please don’t let me talk you out of it! Unless you get stuck or lose interest, I would much rather have a contribution and another member of the team than do this myself.

I’ve opened an issue so that this won’t be forgotten:

1 Like

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.