Since Elixir 15, releases fail with error "Postgrex.Protocol failed to connect: ssl connect: Options can not be combined"

Hi everyone,

Since I updated to Elixir 1.15 and OTP 26, our staging releases fail to connect to our database with the error : Postgrex.Protocol (#PID<0.184.0>) failed to connect: ** (DBConnection.ConnectionError) ssl connect: Options (or their values) can not be combined: [{verify,verify_peer}, {cacerts,undefined}] - {:options, :incompatible, [verify: :verify_peer, cacerts: :undefined]}

I didn’t change anything in configs files, our staging website is still running well, our database is still running well (staging website can access it & I can access it in local), and I didn’t read any breaking change with Ecto & Elixir 15.

Is that Fly related ?

Thanks for any help.

Complete logs :

Running encheres-immo-staging release_command: /app/bin/encheres_immo eval EncheresImmo.Release.migrate
  Created release_command machine 9185ee9a4e4de8
  Waiting for 9185ee9a4e4de8 to have state: started
  Machine 9185ee9a4e4de8 has state: started
  Waiting for 9185ee9a4e4de8 to have state: destroyed
  Machine 9185ee9a4e4de8 has state: destroyed
  Waiting for 9185ee9a4e4de8 to get exit event
Error release_command failed running on machine 9185ee9a4e4de8 with exit code 1.
Check its logs: here's the last 100 lines below, or run 'fly logs -i 9185ee9a4e4de8':
  Pulling container image registry.fly.io/encheres-immo-staging:deployment-01H41S3P4NSVXDZGSTJDVQW75R
  Successfully prepared image registry.fly.io/encheres-immo-staging:deployment-01H41S3P4NSVXDZGSTJDVQW75R (7.337902863s)
  Configuring firecracker
   INFO Starting init (commit: db101a53)...
   INFO Preparing to run: `/app/bin/encheres_immo eval EncheresImmo.Release.migrate` as nobody
   INFO [fly api proxy] listening at /.fly/api
  2023/06/28 19:59:49 listening on [fdaa:0:33f5:a7b:21e0:bdf5:5ea9:2]:22 (DNS: [fdaa::3]:53)
   WARN Reaped child process with pid: 282 and signal: SIGUSR1, core dumped? false
Error:   19:59:51.697 [error] Postgrex.Protocol (#PID<0.164.0>) failed to connect: ** (Postgrex.Error) FATAL 53300 (too_many_connections) remaining connection slots are reserved for non-replication superuser connections
  The EventStore database is already migrated.
Error:   19:59:52.184 [error] Postgrex.Protocol (#PID<0.184.0>) failed to connect: ** (DBConnection.ConnectionError) ssl connect: Options (or their values) can not be combined: [{verify,verify_peer},
                                                  {cacerts,undefined}] - {:options, :incompatible, [verify: :verify_peer, cacerts: :undefined]}
Error:   19:59:52.185 [error] Postgrex.Protocol (#PID<0.185.0>) failed to connect: ** (DBConnection.ConnectionError) ssl connect: Options (or their values) can not be combined: [{verify,verify_peer},
                                                  {cacerts,undefined}] - {:options, :incompatible, [verify: :verify_peer, cacerts: :undefined]}
Error:   19:59:54.310 [error] Postgrex.Protocol (#PID<0.185.0>) failed to connect: ** (DBConnection.ConnectionError) ssl connect: Options (or their values) can not be combined: [{verify,verify_peer},
                                                  {cacerts,undefined}] - {:options, :incompatible, [verify: :verify_peer, cacerts: :undefined]}
Error:   19:59:54.810 [error] Postgrex.Protocol (#PID<0.184.0>) failed to connect: ** (DBConnection.ConnectionError) ssl connect: Options (or their values) can not be combined: [{verify,verify_peer},
                                                  {cacerts,undefined}] - {:options, :incompatible, [verify: :verify_peer, cacerts: :undefined]}
Error:   19:59:54.935 [error] Could not create schema migrations table. This error usually happens due to the following:
    * The database does not exist
    * The "schema_migrations" table, which Ecto uses for managing
      migrations, was defined by another library
    * There is a deadlock while migrating (such as using concurrent
      indexes with a migration_lock)
  To fix the first issue, run "mix ecto.create" for the desired MIX_ENV.
  To address the second, you can run "mix ecto.drop" followed by
  "mix ecto.create", both for the desired MIX_ENV. Alternatively you may
  configure Ecto to use another table and/or repository for managing
  migrations:
      config :encheres_immo, EncheresImmo.Repo,
        migration_source: "some_other_table_for_schema_migrations",
        migration_repo: AnotherRepoForSchemaMigrations
  The full error report is shown below.
  ** (DBConnection.ConnectionError) connection not available and request was dropped from queue after 2996ms. This means requests are coming in and your connection pool cannot serve them fast enough. You can address this by:
    1. Ensuring your database is available and that you can connect to it
    2. Tracking down slow queries and making sure they are running fast enough
    3. Increasing the pool_size (although this increases resource consumption)
    4. Allowing requests to wait longer by increasing :queue_target and :queue_interval
  See DBConnection.start_link/2 for more information
      (ecto_sql 3.10.1) lib/ecto/adapters/sql.ex:913: Ecto.Adapters.SQL.raise_sql_call_error/1
      (elixir 1.15.0) lib/enum.ex:1693: Enum."-map/2-lists^map/1-1-"/2
      (ecto_sql 3.10.1) lib/ecto/adapters/sql.ex:1005: Ecto.Adapters.SQL.execute_ddl/4
      (ecto_sql 3.10.1) lib/ecto/migrator.ex:738: Ecto.Migrator.verbose_schema_migration/3
      (ecto_sql 3.10.1) lib/ecto/migrator.ex:552: Ecto.Migrator.lock_for_migrations/4
      (ecto_sql 3.10.1) lib/ecto/migrator.ex:428: Ecto.Migrator.run/4
      (ecto_sql 3.10.1) lib/ecto/migrator.ex:170: Ecto.Migrator.with_repo/3
      nofile:1: (file)
   INFO Main child exited normally with code: 1
   WARN Reaped child process with pid: 284 and signal: SIGUSR1, core dumped? false
   INFO Starting clean up.
   WARN hallpass exited, pid: 232, status: signal: 15 (SIGTERM)
  2023/06/28 19:59:55 listening on [fdaa:0:33f5:a7b:21e0:bdf5:5ea9:2]:22 (DNS: [fdaa::3]:53)
  [    7.133215] reboot: Restarting system
  machine restart policy set to 'no', not restarting
Error: release command failed - aborting deployment. error release_command machine 9185ee9a4e4de8 exited with non-zero status of 1

Complete list of the updated dependencies:

List

appsignal 2.4.3 => 2.7.5
appsignal_phoenix 2.2.1 => 2.3.3
appsignal_plug 2.0.12 => 2.0.14
cachex 3.4.0 => 3.6.0
castore 0.1.20 => 0.1.22
commanded 1.4.1 => 1.4.2
cowboy 2.9.0 => 2.10.0
cowlib 2.11.0 => 2.12.1
credo 1.6.7 => 1.7.0
db_connection 2.4.3 => 2.5.0
decimal 2.0.0 => 2.1.1
earmark 1.4.34 => 1.4.38
earmark_parser 1.4.29 => 1.4.32
ecto 3.9.4 => 3.10.2
ecto_sql 3.9.2 => 3.10.1
esbuild 0.6.0 => 0.7.1 (minor)
eventstore 1.4.1 => 1.4.2
excoveralls 0.15.1 => 0.16.1 (minor)
finch 0.14.0 => 0.16.0 (minor)
floki 0.34.0 => 0.34.3
gen_stage 1.1.2 => 1.2.1
geo 3.4.3 => 3.5.1
geo_postgis 3.4.2 => 3.4.3
gettext 0.20.0 => 0.22.2 (minor)
goth 1.3.1 => 1.4.1
libcluster 3.3.1 => 3.3.3
mint 1.4.2 => 1.5.1
nimble_options 0.5.2 => 1.0.2 (major)
nimble_pool 0.2.6 => 1.0.0 (major)
oban 2.13.6 => 2.15.2
phoenix 1.6.15 => 1.6.16
phoenix_ecto 4.4.0 => 4.4.2
phoenix_html 3.2.0 => 3.3.1
phoenix_live_dashboard 0.7.2 => 0.8.0 (minor)
phoenix_live_view 0.18.3 => 0.19.3 (minor)
phoenix_pubsub 2.1.1 => 2.1.3
phoenix_template 1.0.0 => 1.0.1
plug 1.14.0 => 1.14.2
plug_cowboy 2.6.0 => 2.6.1
plug_crypto 1.2.3 => 1.2.5
postgrex 0.16.5 => 0.17.1 (minor)
pow 1.0.27 => 1.0.31
sourceror 0.11.2 => 0.12.3 (minor)
ssl_verify_fun 1.1.6 => 1.1.7
surface 0.9.1 => 0.11.0 (minor)
tailwind 0.1.9 => 0.1.10
telemetry 1.1.0 => 1.2.1
telemetry_registry 0.3.0 => 0.3.1
tesla 1.5.0 => 1.7.0

the default Fly Postgres server, last time I used it, doesn’t have SSL enabled for connections, so maybe a default has changed with hex package updates. Your error about SSL implies your trying to connect over SSL when the server isn’t configured for it.

In my config somewhere I would have something like this for a Fly DB:

    config :spandex_app, SpandexApp.Repo,
      socket_options: [:inet6],
      ssl: false
2 Likes

Hi, thanks for your answer ! Sadly, turning SSL to false changes nothing.

I wonder if the problem is not particularly related to :
Error: 19:59:51.697 [error] Postgrex.Protocol (#PID<0.164.0>) failed to connect: ** (Postgrex.Error) FATAL 53300 (too_many_connections) remaining connection slots are reserved for non-replication superuser connections

My pool_size is the default 10. But on the Google Cloud console, I see ~22 active connections, turning into ~44 since Fly asked me to clone the machine of my application. The GC limit is at 50 active connections.

Can that be related?

No okay, I’m stupid. Didn’t config SSL in the right place. It works now, thank you so much :slight_smile:

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.