Can't deploy Phoenix application due to database not found (/app/bin/migrate)

I’m able to successfully deploy a newly generated hello_elixir app on fly.io just fine (following the 1.6.3 guide), so I know that some of my setup is OK.

However, when I try to deploying an existing app generated with Phoenix 1.6.0, and since updated to 1.6.9, the deployment repeatedly fails. This app runs fine locally and when I do manual deployments using mix assets.deploy on a Linux server.

With fly.io it fails at the /app/bin/migrate step:

	 Preparing to run: `/app/bin/migrate` as nobody
	 2022/06/09 22:49:03 listening on [fdaa:0:6842:a7b:28df:3125:261a:2]:22 (DNS: [fdaa::3]:53)
	 22:49:05.251 [error] GenServer #PID<0.135.0> terminating
	     (connection 1.1.0) lib/connection.ex:622: Connection.enter_connect/5
	     (stdlib 3.17.2) proc_lib.erl:226: :proc_lib.init_p_do_apply/3
	     (connection 1.1.0) lib/connection.ex:622: Connection.enter_connect/5
	     (stdlib 3.17.2) proc_lib.erl:226: :proc_lib.init_p_do_apply/3
	     (postgrex 0.16.3) lib/postgrex/protocol.ex:160: Postgrex.Protocol.connect_endpoints/6
	     (db_connection 2.4.2) lib/db_connection/connection.ex:82: DBConnection.Connection.connect/2
	 ** (RuntimeError) connect raised KeyError exception: key :database not found. The exception details are hidden, as they may contain sensitive data such as database credentials. You may set :show_sensitive_data_on_connection_error to true when starting your connection if you wish to see all of the details
	     (connection 1.1.0) lib/connection.ex:622: Connection.enter_connect/5
	     (stdlib 3.17.2) proc_lib.erl:226: :proc_lib.init_p_do_apply/3
	 Main child exited normally with code: 1
	 Starting clean up.
Error Release command failed, deployment aborted

There are no errors prior to this—the images are pushed/mounted etc.

I can see that my DATABASE_URL and SECRET_KEY_BASE are set using flyctl secrets list

I can see that the error message suggests setting show_sensitive_data_on_connection_error to true but it’s not clear where this needs to be done since the guide says that the Fly.io launcher takes care of the database configuration.

Is there perhaps an issue with an app that was first generated with 1.6.0 and subsequently upgrade to >1.6.3?

Are you using sqlite? If so, migrations don’t run in the same sense. This gist and this thread might help. If you’re not using sqlite… sorry - I’m not sure!

Edit: sorry - I see postgrex in the stacktrace now. Disregard!

** (RuntimeError) connect raised KeyError exception: key :database not found. The exception details are hidden, as they may contain sensitive data such as database credentials. You may set :show_sensitive_data_on_connection_error to true when starting your connection if you wish to see all of the details

I think this error is being raised by Postgrex, the Elixir library that is a postgres adapter for Ecto. It’s complaining about a KeyError that the :database key is not found, probably in a config/opts list that it depends on to connect to your database.

The error message comes from a lower level library called DBConnection, and :show_sensitive_data_on_connection_error is an option you can pass to that library. When you configure your Repo in your Phoenix config files, I think all options that are not used directly by Ecto.Repo are passed to Postgrex, which passes the rest to DBConnection, so I would try something like config :my_app, MyApp.Repo, show_sensitive_data_on_connection_error: true.

There might also be more info in your fly logs.

As for the KeyError itself that’s blowing up your deploy, to me it suggests that the database location is missing from your config. Are you configuring the DATABASE_URL in your config/runtime.exs file?

  database_url =
    System.get_env("DATABASE_URL") ||
      raise """
      environment variable DATABASE_URL is missing.
      For example: ecto://USER:PASS@HOST/DATABASE
      """

  maybe_ipv6 = if System.get_env("ECTO_IPV6"), do: [:inet6], else: []

  config :my_app, MyApp.Repo,
    # ssl: true,
    url: database_url,
    pool_size: String.to_integer(System.get_env("POOL_SIZE") || "20"),
    socket_options: maybe_ipv6

To answer my own question, I think the issue stems from differences in config files between apps generated with Phoenix 1.6.3 and an earlier version (1.6.0 in my case). Even though I had since upgraded to 1.6.10 the various config files created by earlier versions of Phoenix were still missing elements relied on by fly launch.

The particular issue seems to be that the following lines in runtime.exs are included in newly generated apps but AFAICT are missing from apps generated by Phoenix 1.6.0 (I’m pretty sure I didn’t alter this file):

if config_env() == :prod do
  database_url =
    System.get_env("DATABASE_URL") ||
      raise """
      environment variable DATABASE_URL is missing.
      For example: ecto://USER:PASS@HOST/DATABASE
      """

  maybe_ipv6 = if System.get_env("ECTO_IPV6"), do: [:inet6], else: []

  config :hello_elixir, HelloElixir.Repo,
    # ssl: true,
    url: database_url,
    pool_size: String.to_integer(System.get_env("POOL_SIZE") || "10"),
    socket_options: maybe_ipv6

You can see how the missing database_url code would cause things to fail completely.

This part of the deployment is now working :slight_smile:

1 Like

Thanks so much for your reply. As it happens I just worked this out myself by going through the config files for a newly generated hello world app and comparing with my existing one.

Thanks for the help!

1 Like

I’m glad you got it working!