How do I get a Node+Postgres app from Heroku running on Fly?

Questions / Help Build debugging
1

I’m looking to migrate all of my apps from Heroku, and I am having a lot of issues with Node+Postgres on Fly. The docs and support articles that exist don’t seem to cover any of the issues I’m having… and there’s a lot.

Let’s take a relatively simple app of mine that runs a NodeJS express server and connects to a Postgres database using TypeORM.

Run fly launch
Give my app a name
Choose a region
It generates a fly.toml with heroku/buildpacks:20 amongst other things
I choose not to deploy at this stage

I use fly secrets set to set up a few environment variables

I run fly deploy, and my app is built and then fails to connect to the database to run migrations (which are part of my build script). My build script runs a couple of package.json scripts with NPM, but during the migration step I get a connection refused error.

I try to fly ssh console to debug the script, but I cannot connect to my app because “Error host unavailable: host was not found in DNS”. My app isn’t running?

I remove my migration script from my build and fly deploy my app (which is now in a broken state).

I successfully fly ssh console and try to run my migrations manually with npm run migrate to see if I can connect to the database… “npm: not found”.

I try a which npm with no results.
I try a which node with no results.

At this point I have no idea how my app is even running or what the purpose of fly ssh console is if I can’t access the core programs that are required to run my app.

After extensive Googling I find out about the deploy release_command and so I put my migrations in here. I fly deploy again and everything succeeds and appears to run my migrations.

Now I refer back to an article about migrating from Heroku to transfer the content of my database: Migrate from Heroku · Fly Docs

I run the pg_dump command, and realize immediately that my console doesn’t have access to the database URLs that I’d just configured. I guess this is supposed to be run with fly?

I fly ssh console and run the command, which outputs a bunch of alter tables and such, followed by several ERROR: role "abc123etcetc" does not exist.

Within the fly console I try psql, but “error: connection to server on socket “/var/run/postgresql/.s.PGSQL.5432” failed: No such file or directory”.

I remember about reading another command to connect to Postgres so I run fly postgress connect -a postgres-app-name (correctly replacing the app name).

From here I run \td which results in “Did not find any relations.”. And a simple count(*) of my user table errors with “ERROR: relation “user” does not exist”.

I figure I’ve messed up at some point and so I decide to detach the database and try again from scratch.

Run fly postgres detach -a app-name postgres-app-name and get “Error error running user-delete: 500: ERROR: role “app_name” cannot be dropped because some objects depend on it (SQLSTATE 2BP01)”.

Now I have a completely broken app with a seemingly empty database, and no idea what to do next.

I delete my Postgres app from the Fly web UI, and attempt to detach it from the CLI again, and attach a new database, but everything I run just seems to result in errors because my app is no longer running, and any attempt to remove the database means the app fails to start, so it just rolls back to the previous broken variant.

  • What is the correct approach for running migrations during a deployment?
  • How do I manually run migrations (or other commands with access to Node/NPM)?
  • How do I actually transfer my database content from Heroku to Fly?
  • How do I run commands to setup my app before it is running?
  • How do I now get a database connected to my horribly broken app?
  • How to I reset the database if I need to try this all again?
2

Disclaimer: my focus is on rails, but I was involved in node.js a few year back.

First, you run fly pg help to see the postgres commands available.

This will sound odd, but go to Getting Started · Fly Docs, ignore the recommendation to set your memory to 512, add '-j esbuildto therails newcommand, and stop afterfly open. If you like, you can run fly ssh console`. Either don’t allocate a new postgres db and set DATABASE_URL to your previous database, or delete your old postgres db and start over.

At this point you will have a Dockerfile, .dockerignore. and ‘fly.toml’. Delete the rails parts of the dockerfile, and adjust the [deploy] section of the fly.toml to run your migrations, and you should have something that should work for your node application,

3

Update:

The issues importing the database and database being empty after attempting an import are both due to the pg_dump command mentioned in Migrate from Heroku · Fly Docs.

The correct way to move data from Heroku is (note: none of these command need to be run within fly via ssh or anything):

# Create a backup
heroku pg:backups:capture -a <app-name>
# Download the backup
heroku pg:backups:download -a <app-name>
# Add this backup to fly
pg_restore --verbose --clean --no-acl --no-owner -d <database-url> latest.dump

I am now likely to move my apps elsewhere, however, as since this post I was able to spin up all of my apps on another service before I got any answers to my questions here.

I will leave this post in the hope it helps others.