Postgres database service inside Docker

Questions / Help
,
1

Early disclaimer: I’m fairly new to these subjects, as I’m usually more into frontend.

I have this Fly app that’s running from a Docker container.
This Docker container is composed of 3 services: app (a Rails 7 app), cache (Redis), and db (Postgres database).

On my local machine — and I really thought that by using Docker I would stop saying that — I can run this app both on development and production with no issues.

My database.yml file connects to host: db (where my database is).
Now, on Fly.io I see the following error:

2023-04-07T14:19:00.624 app[92bb3694] mad [info] [4a840a1a-aa16-48f8-a6d3-264d4d42bbd0] ActiveRecord::ConnectionNotEstablished (could not translate host name "db" to address: Name or service not known

Looks like it isn’t being able to find the db service.
Am I wrong when assuming that one can run multiple services per Docker container on Fly.io?
Can one of those be a postgres database?

Or perhaps that should be running on a separate app?

This is also slightly tied to other questions I will need answers to in the future involving persistent data and backups, so some lights here will be incredibly helpful.
Thanks :pray:

2

Hi @pmpinto!

You’re on the right track here! While it’s possible to run multiple services per Docker container, it’s typical practice to run only one per container. On Fly.io, that usually translates to a single service per app.

Could you tell me a bit more about how are you running your app locally? Based on that, I might be able to give you some more specific/actionable advice. (From a few things you wrote, I’m wondering whether you’re using docker-compose?)

For data persistence, have a look here. Some specifics:

Fly has volumes that you can attach to your apps. We do daily backups of your volumes automatically.

For Postgres, you might be interested in using Fly Postgres. It’s not a fully managed database, but it will make it easier to get a Postgres app (backed by persistent volumes) running on Fly.

3

Hey Matthew!
Thanks a lot for the prompt reply, very much appreciated!

You’re right, I’m using Docker compose. I’m not sure what else you need but I’ll leave here the command I’m using to start local development, perhaps that helps?

docker compose -f docker-compose.development.yml up --build

and inside docker-compose.development.yml I have something like this:

version: '3'
services:
  app:
    build: .
    # ...
  cache:
    image: redis
    # ...
  db:
    image: postgres
    # ...
volumes:
  cache:
  db:

does this help?

4

Thanks, this does help!

Since you said you’re fairly new to this, I hope you won’t mind if I clarify a bit. Your Docker Compose file actually defines three separate Docker containers, one for each service. So in fact you’re running a single service per container already!

The Fly equivalent of a service in your Docker compose file would indeed be an individual app, so the usual way to run this on Fly would be as three apps (within a single organization, so that they are all connected to the same private network, much like Docker Compose networking). So far, you’ve deployed your Rails application, so you’ll want to create a Postgres app and a Redis app next, and configure your app to connect to them (probably with secrets).

As I mentioned, Fly Postgres is a possibility for the database. You can also run Upstash Redis on Fly. It’s also possible to run Postgres and Redis images manually on Fly if you really want to.

Finally, if you’re interested in using Fly Postgres and/or Upstash, you might consider deleting your app and running fly launch again. It should give you the option to set up both databases and will automatically configure DATABASE_URL and REDIS_URL secrets on your Rails app with the right values. Then you’ll just have to make sure that your app uses those environment variables to connect to each.

5

Again, Matthew, thanks a lot for all the info — and no, I don’t mind at all that you clarify my fuzzy mind! It’s very much appreciated!

I will indeed need to set up some secrets at some point. I’ll give this a try and report back if I need assistance :pray:

6

Quick follow-up question, if I may:
I have just now created a Fly Postgres app and it returned the connection details. Then I created some secrets on my Rails app with the details it will need.
But I’m confused as to which hostname I should use for my Rails app connection settings.

Let’s assume the name of my database is my-database:

  • In the returned data it mentions 'hostname: my-database.internal
  • In the ‘connection string’ it mentions my-database.flycast
  • In my dashboard it mentions my-database.fly.dev

The one I’m currently using is my-database.internal, but as soon as I attempted to reach to my database it failed with:

ActiveRecord::ConnectionNotEstablished (could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?

Is this the correct hostname and there’s something else I may be missing or am I using the wrong hostname?

7

That error looks like you don’t have the right secret set. Check with fly secrets list

8

Well, I did create all the secrets my Rails app is asking for. The only doubt is if the hostname I used is the correct one, is it?

9

Is DATABASE_URL set? Alternately you can set up the production database is your config/database.yml

10

That made me realise I was missing a fly postgres attach. Doing that defined a DATABASE_URL secret for me.

But why do I need that? Where is that secret used? Is there any other secret I may need for purposes I’m not aware of?

I mean, how does this translate to config/database.yml file in my Rails app?

11

Are you seeing other errors?

12

Thanks!

So I removed the DATABASE_URL secret in favor of keeping it in my config/database.yml as such:

default: &default
  adapter: postgresql
  pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
  timeout: 5000
  port: 5432

production:
  <<: *default
  host: <%= ENV['POSTGRES_HOST_PROD'] %>
  database: <%= ENV['POSTGRES_DB_PROD'] %>
  username: <%= ENV['POSTGRES_USER_PROD'] %>
  password: <%= ENV['POSTGRES_PASSWORD_PROD'] %>

Meanwhile, I also noticed that I was working in another Fly app due to the fact that I’m relying on GitHub - superfly/fly-pr-review-apps: Github Action for PR Review Apps on Fly.io to build preview URLs. And this app didn’t have any of the secrets set up — I’ll tackle that later.
So, I merged everything to the main branch so that I could use the app that does have the secrets already set up, as such:

  ﬌ fly secrets list
Update available 0.0.509 -> 0.0.513.
Run "fly version update" to upgrade.
NAME                  	DIGEST          	CREATED AT
POSTGRES_DB_PROD      	[REDACTED]      	2023-04-10T14:23:24Z
POSTGRES_HOST_PROD    	[REDACTED]      	2023-04-10T14:21:28Z
POSTGRES_PASSWORD_PROD	[REDACTED]      	2023-04-10T14:23:24Z
POSTGRES_USER_PROD    	[REDACTED]	        2023-04-10T14:23:24Z
RAILS_MASTER_KEY      	[REDACTED]      	2023-03-24T19:33:23Z

Even so, the deployment fails with this error:

ActiveRecord::NoDatabaseError: We could not find your database: my-database. Which can be found in the database configuration file located at config/database.yml.

So it sounds like the server is reachable now, but the database doesn’t exist. Is there any step left for me to do besides creating and attaching a postgres app to my other Fly app?

13

Just changed db:migrate to db:prepare in fly.toml and it seems to have solved it, but I honestly have no idea why, so some insights would be very much appreciated!

[deploy]
  release_command = "bundle exec rails db:prepare"
14

Rails, by default, tries to connect to the database that is defined in the DATABASE_URL environment variable. That is why Fly sets it automatically. Actually, it is better that you use that variable instead of setting other env variables manually. I recommend you to reset the config/database.yml file, you don’t really need to make any changes

15

Why is that @brunoprietog?
I honestly don’t see any strong preference for either option in Configuring Rails Applications — Ruby on Rails Guides

16

It’s always good to follow conventions, unless you have a very strong reason not to. I don’t think this is the case, even more so when Fly already provides the environment variable ready, because as I tell you, that’s the convention. Obviously, you can configure everything, but Rails tries to have a configuration with good defaults, and using the DATABASE_URL environment variable is one of them. And since all PaaS use it, I don’t see a strong reason not to, unless you have a configuration with several replicas or shards, which doesn’t seem to be your case.

17

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.