Transferring Postgres from Heroku

I’m attempting to migrate a Rails app from Heroku, and I’ve followed the instructions without any issues up to the section about database migration. I ran the command laid out from the guide:

$ pg_dump --no-owner -C -d $HEROKU_DATABASE_URL | psql -d $DATABASE_URL

However, the following error occurs:

pg_dump: option requires an argument -- d
Try "pg_dump --help" for more information.
psql: option requires an argument -- d
Try "psql --help" for more information.

I have tried replacing the two variables with the actual database URL’s, but the $DATABASE_URL is an internal address, which makes me think I’m missing something.

Am I supposed to run the command from within my VM? Hope someone can help :pray:

Update: I attempted to run the command from within the app container with fly ssh console but got the following error:

$ pg_dump --no-owner -C -d $HEROKU_DATABASE_URL | psql -d $DATABASE_URL
pg_dump: server version: 14.4 (Ubuntu 14.4-1.pgdg20.04+1); pg_dump version: 11.17 (Debian 11.17-0+deb10u1)
pg_dump: aborting because of server version mismatch
1 Like

I think that you have to ssh to your app before running these commands.

fly ssh console

Otherwise, you can also load a database dump from your machine by adjusting the following commands:

flyctl proxy 15432:5432 -a my-postgres-app
psql -U <your-user> -h localhost -p 15432 <your-db> < <your_dump_file.sql>
1 Like

This works for me from the command line when at my Rails project
The first may not be necessary depending on how you have Heroku set up

   heroku login -I
    heroku pg:backups:capture
    heroku pg:backups:download

This downloads the database to your project folder. Uploading to fly.io is another story and in fact that is what I’m trying to figure out.

I hadn’t read those instructions when I first posted, but it makes some sense to download and upload rather than trying to do in one step. When I deploy I’m getting errors about a database table not being seen, but I’ve got the rails app running in localhost also. It seems the fly deploy uploads the database too. I just started with this yesterday, so haven’t gotten very far and will separately post if I can’t figure it out.

It’s the fly proxy command with psql that I listed above.

flyctl proxy 15432:5432 -a my-postgres-app just hangs for me. No error.

That sets up a proxy from the app port 5432 to local port 15432. Then you can use psql against localhost with port 15432.

Understood. When I set up the proxy on the app port, it hangs. I can’t even get to step two.

It opens up a foreground process that should remain open. You’ll need to use psql in a second terminal.

$ fly proxy 15432:5432 -a my-app
Proxying local port 15432 to remote [my-app.internal]:5432

Ahhh, did not know that. Will try when I’m back.

I’m having the same issues as @wiznaibus here.
I have flyctl proxy 15432:5432 -a my-app in a terminal and it’s proxying :white_check_mark:

But when I run

psql -U hora -h localhost -p 15432 my-postgres-db < data.sql

I get

zsh: no such file or directory: data.sql

I have tried doing the same with latest.dump downloaded straight from heroku and that was saved at the root of my project:

psql -U hora -h localhost -p 15432 my-postgres-db < latest.dump

but this one just hangs :confused: Do you know how to get that .sql file to dump onto my db?

Trying to be clearer this time.

  1. Obtain a sql dump file from heroku and have it locally.

  2. Proxy the postgres from fly to localhost.

$ fly proxy 15432:5432 -a my-app
Proxying local port 15432 to remote [my-app.internal]:5432
  1. Use psql like you would normally, but on port 15432. For instance, you should be able to connect, etc.

$ psql -U your_user -h localhost -p 15432 
psql #

$ psql -U your_user -h localhost -p 15432 < your_dump_file
...this will load the database dump and give some output...

@dornby - see step 3. If updated the user/db info, make sure you can connect directly first before trying to load the dump file.

1 Like


@tj1 thank you for your approach
but i’m having trouble
hope you can assist
(this is google translate)

You’ve connected correctly, but you need to use the correct database name.

psql -U postgres -h localhost -p 15432 -d your_database < latest.dump

You can find out the name of your databases by connecting locally and listing them with \l

postgres=# \l
                                    List of databases
   Name    |   Owner    | Encoding |  Collate   |   Ctype    |     Access privileges   
  
-----------+------------+----------+------------+------------+-------------------------
--
 main      | flypgadmin | UTF8     | en_US.utf8 | en_US.utf8 | 
 postgres  | flypgadmin | UTF8     | en_US.utf8 | en_US.utf8 | 
 template0 | flypgadmin | UTF8     | en_US.utf8 | en_US.utf8 | =c/flypgadmin           
 +
           |            |          |            |            | flypgadmin=CTc/flypgadmi
n
 template1 | flypgadmin | UTF8     | en_US.utf8 | en_US.utf8 | =c/flypgadmin           
 +
           |            |          |            |            | flypgadmin=CTc/flypgadmi
n
(4 rows)


2 Likes

Wuhu :tada: it finally worked for me.

Thank you so much @tj1 for leading me on the way. Here is a precise breakdown of what I did for it to work, with a lot of repetition from @tj1 's advice:

  1. Get your latest dump by running
heroku login
heroku pg:backups:capture
heroku pg:backups:download
  1. In one terminal, launch:
fly proxy 15432:5432 -a my-db-app-name
# make sure it's the name of your db-app, not your main app
  1. In another terminal, list your db names to get the exact name of your DB with \l:
psql -U postgres -h localhost -p 15432
postgres=> \l

   Name    |   Owner    | Encoding |  Collate   |   Ctype    |     Access privileges
-----------+------------+----------+------------+------------+---------------------------
 foobar  | flypgadmin | UTF8     | en_US.utf8 | en_US.utf8 |
 postgres  | flypgadmin | UTF8     | en_US.utf8 | en_US.utf8 |
 template0 | flypgadmin | UTF8     | en_US.utf8 | en_US.utf8 | =c/flypgadmin            +
           |            |          |            |            | flypgadmin=CTc/flypgadmin
 template1 | flypgadmin | UTF8     | en_US.utf8 | en_US.utf8 | =c/flypgadmin            +
           |            |          |            |            | flypgadmin=CTc/flypgadmin
(4 rows)
  1. Once you have your name, here foobar, import your dump with pg_restore:
pg_restore -U postgres -h localhost -p 15432 -d foobar < latest.dump

And voilà, or at least it worked for me.

5 Likes

@tj1 @dornby
Thanks for the guidance of the two masters
I also succeeded :grinning:

But I have a new problem.
I use “pgadmin” or “tableplus” software to connect postgresql db.

An error occurs:
could not translate host name “dbname-db.internal” to address: nodename nor servname provided, or not known.

“host name” I have tried the following but failed:
top2.nearest.of.dbname-db.internal
dbname-db.internal

I have closed the fly proxy and connected again, but the same error occurs.
If I turn on fly proxy and use localhost:15432 to connect,
Can connect to db, but not production db.

I’m a beginner and would appreciate some guidance.

(this is google translate)

@jedi I too use TablePlus and all worked fine:

  1. Keep your fly proxy open
  2. Use the following in TablePlus:
  • host: localhost
  • user: postgres (well, your db user name)
  • password: your db password
  • port: 15432
  • database: the same name as we used in our last step in my previous message: foobar in the example
2 Likes

@dornby thanks for your reply

I have enabled “fly proxy”.
Follow your instructions to connect,
It can be successfully connected.

I tried to change the data in the db in TablePlus and save it,
Then “puts db data” on the Internet,
But it shows the old data (maintain the data of the original heroku db) :face_with_spiral_eyes:

Obviously the data I changed in TablePlus was not updated synchronously,
Or what action is required to upload the database?

please help thanks :joy:

(this is google translate)

@dornby

OMG :scream:
my DATABASE_URL=$(heroku config:get DATABASE_URL)

I’m such an idiot… :rofl:
It is normal to change DATABASE_URL to fly’s db…

Thank u :+1:

@jedi you’re not an idiot :slightly_smiling_face:
Well done on solving this one, I wouldn’t have managed to help :smile: