I’m currently trying to deploy Clickhouse to an existing application that already has been provisioned with Postgres. The end state is to have the Phoenix, Postgres, and Clickhouse together.
The existing application is working fine and I have provisioned (successfully) Clickhouse, but I’m having trouble with the integration. I’m currently trying to deploy the Phoenix Application (which I have tested locally), but I am getting the following error during deployment.
Error: 4-12-07T22:30:39Z 22:30:39.211 [error] Ch.Connection (#PID<0.184.0>) failed to connect: ** (Mint.TransportError) non-existing domain
Error: 4-12-07T22:30:39Z 22:30:39.211 [error] Ch.Connection (#PID<0.183.0>) failed to connect: ** (Mint.TransportError) non-existing domain
Error: 4-12-07T22:30:40Z 22:30:40.374 [error] Ch.Connection (#PID<0.184.0>) failed to connect: ** (Mint.TransportError) non-existing domain
Error: 4-12-07T22:30:41Z 22:30:41.459 [error] Ch.Connection (#PID<0.183.0>) failed to connect: ** (Mint.TransportError) non-existing domain
Error: 4-12-07T22:30:42Z 22:30:42.034 [error] Could not create schema migrations table. This error usually happens due to the following:
2024-12-07T22:30:42Z * The database does not exist
2024-12-07T22:30:42Z * The "schema_migrations" table, which Ecto uses for managing
2024-12-07T22:30:42Z migrations, was defined by another library
2024-12-07T22:30:42Z * There is a deadlock while migrating (such as using concurrent
2024-12-07T22:30:42Z indexes with a migration_lock)
2024-12-07T22:30:42Z To fix the first issue, run "mix ecto.create" for the desired MIX_ENV.
2024-12-07T22:30:42Z To address the second, you can run "mix ecto.drop" followed by
2024-12-07T22:30:42Z "mix ecto.create", both for the desired MIX_ENV. Alternatively you may
2024-12-07T22:30:42Z configure Ecto to use another table and/or repository for managing
2024-12-07T22:30:42Z migrations:
2024-12-07T22:30:42Z config :app, App.IngestRepo,
2024-12-07T22:30:42Z migration_source: "some_other_table_for_schema_migrations",
2024-12-07T22:30:42Z migration_repo: AnotherRepoForSchemaMigrations
2024-12-07T22:30:42Z The full error report is shown below.
2024-12-07T22:30:42Z ** (DBConnection.ConnectionError) connection not available and request was dropped from queue after 2978ms. This means requests are coming in and your connection pool cannot serve them fast enough. You can address this by:
2024-12-07T22:30:42Z 1. Ensuring your database is available and that you can connect to it
2024-12-07T22:30:42Z 2. Tracking down slow queries and making sure they are running fast enough
2024-12-07T22:30:42Z 3. Increasing the pool_size (although this increases resource consumption)
2024-12-07T22:30:42Z 4. Allowing requests to wait longer by increasing :queue_target and :queue_interval
2024-12-07T22:30:42Z See DBConnection.start_link/2 for more information
2024-12-07T22:30:42Z (ecto_sql 3.12.1) lib/ecto/adapters/sql.ex:1096: Ecto.Adapters.SQL.raise_sql_call_error/1
2024-12-07T22:30:42Z (elixir 1.15.2) lib/enum.ex:1693: Enum."-map/2-lists^map/1-1-"/2
2024-12-07T22:30:42Z (ecto_sql 3.12.1) lib/ecto/adapters/sql.ex:1203: Ecto.Adapters.SQL.execute_ddl/4
2024-12-07T22:30:42Z (ecto_sql 3.12.1) lib/ecto/migrator.ex:779: Ecto.Migrator.verbose_schema_migration/3
2024-12-07T22:30:42Z (ecto_sql 3.12.1) lib/ecto/migrator.ex:563: Ecto.Migrator.lock_for_migrations/4
2024-12-07T22:30:42Z (ecto_sql 3.12.1) lib/ecto/migrator.ex:432: Ecto.Migrator.run/4
2024-12-07T22:30:42Z (ecto_sql 3.12.1) lib/ecto/migrator.ex:170: Ecto.Migrator.with_repo/3
2024-12-07T22:30:42Z nofile:1: (file)
It looks to me like a couple of things could be happening:
- The database does not exists.
- The service is not exposed correctly.
- The connection is not configured correclty.
The database does not exist
This makes the most sense to me, I’m unsure what fly does when it provisions an application for the first time but it must run ecto.create and then when subsequent deployments happen it runs ecto.migrate (we can see this here release_command = '/app/bin/migrate').
However its not totally clear to me with the documentation how I can run ecto.create when I have already created it.
To be absolutely clear running ecto.create locally provisions both postgres and clickhouse, so all I really need to be able to do is to run ecto.create on a system which is already running?
The service is not exposed correctly / The connection is not configured correclty.
The clickhouse service is configured (kinda) following this blog post Iago Cavalcante · Software Engineer
app = 'clickhouse'
primary_region = 'lhr'
kill_signal = 'SIGINT'
kill_timeout = '5s'
[experimental]
auto_rollback = true
[build]
[[mounts]]
source = 'clickhouse_data'
destination = '/var/lib/clickhouse'
[[services]]
internal_port = 8123
protocol = "tcp"
[[services.ports]]
port = 8123
[[services]]
internal_port = 9000
protocol = "tcp"
[[services.ports]]
port = 9000
[[vm]]
size = 'shared-cpu-2x'
My undersanding of the following is that I should be able to connect on
https://<name>.internal:8123/<db_name>, but maybe my understanding is wrong there?
Is there a command I can run to better debug this?