Deploying also fails intermittently
Command: /app/bin/indie_paper eval IndiePaper.Release.migrate
Starting instance
Configuring virtual machine
Pulling container image
Unpacking image
Preparing kernel init
Configuring firecracker
Starting virtual machine
Starting init (commit: 50ffe20)...
Preparing to run: `/app/bin/indie_paper eval IndiePaper.Release.migrate` as nobody
2021/10/05 04:43:38 listening on [fdaa:0:2d0a:a7b:14bf:1e2d:c443:2]:22 (DNS: [fdaa::3]:53)
Reaped child process with pid: 561 and signal: SIGUSR1, core dumped? false
04:43:42.070 [error] Could not create schema migrations table. This error usually happens due to the following:
* The database does not exist
* The "schema_migrations" table, which Ecto uses for managing
migrations, was defined by another library
* There is a deadlock while migrating (such as using concurrent
indexes with a migration_lock)
To fix the first issue, run "mix ecto.create".
To address the second, you can run "mix ecto.drop" followed by
"mix ecto.create". Alternatively you may configure Ecto to use
another table and/or repository for managing migrations:
config :indie_paper, IndiePaper.Repo,
migration_source: "some_other_table_for_schema_migrations",
migration_repo: AnotherRepoForSchemaMigrations
The full error report is shown below.
** (DBConnection.ConnectionError) connection not available and request was dropped from queue after 2976ms. This means requests are coming in and your connection pool cannot serve them fast enough. You can address this by:
1. Ensuring your database is available and that you can connect to it
2. Tracking down slow queries and making sure they are running fast enough
3. Increasing the pool_size (albeit it increases resource consumption)
4. Allowing requests to wait longer by increasing :queue_target and :queue_interval
See DBConnection.start_link/2 for more information
(ecto_sql 3.7.0) lib/ecto/adapters/sql.ex:756: Ecto.Adapters.SQL.raise_sql_call_error/1
(elixir 1.12.1) lib/enum.ex:1553: Enum."-map/2-lists^map/1-0-"/2
(ecto_sql 3.7.0) lib/ecto/adapters/sql.ex:844: Ecto.Adapters.SQL.execute_ddl/4
(ecto_sql 3.7.0) lib/ecto/migrator.ex:645: Ecto.Migrator.verbose_schema_migration/3
(ecto_sql 3.7.0) lib/ecto/migrator.ex:473: Ecto.Migrator.lock_for_migrations/4
(ecto_sql 3.7.0) lib/ecto/migrator.ex:388: Ecto.Migrator.run/4
(ecto_sql 3.7.0) lib/ecto/migrator.ex:146: Ecto.Migrator.with_repo/3
(indie_paper 0.1.0) lib/indie_paper/release.ex:12: anonymous fn/2 in IndiePaper.Release.migrate/0
Main child exited normally with code: 1
Reaped child process with pid: 563 and signal: SIGUSR1, core dumped? false
Starting clean up.
Here is my runtime.exs
import Config
# config/runtime.exs is executed for all environments, including
# during releases. It is executed after compilation and before the
# system starts, so it is typically used to load production configuration
# and secrets from environment variables or elsewhere. Do not define
# any compile-time configuration in here, as it won't be applied.
# The block below contains prod specific runtime configuration.
if config_env() == :prod do
database_url =
System.get_env("DATABASE_URL") ||
raise """
environment variable DATABASE_URL is missing.
For example: ecto://USER:PASS@HOST/DATABASE
"""
config :indie_paper, IndiePaper.Repo,
# ssl: true,
socket_options: [:inet6],
url: database_url,
pool_size: String.to_integer(System.get_env("POOL_SIZE") || "10")
secret_key_base =
System.get_env("SECRET_KEY_BASE") ||
raise """
environment variable SECRET_KEY_BASE is missing.
You can generate one by calling: mix phx.gen.secret
"""
app_name =
System.get_env("FLY_APP_NAME") ||
raise "FLY_APP_NAME not available"
indie_paper_deploy_domain =
System.get_env("INDIEPAPER_DEPLOY_DOMAIN") || raise "INDIEPAPER_DEPLOY_DOMAIN not available"
config :indie_paper, IndiePaperWeb.Endpoint,
server: true,
url: [host: indie_paper_deploy_domain, port: 443, scheme: "https"],
check_origin: [
"https://#{app_name}.fly.dev",
"https://*.#{indie_paper_deploy_domain}"
],
http: [
# Enable IPv6 and bind on all interfaces.
# Set it to {0, 0, 0, 0, 0, 0, 0, 1} for local network only access.
# See the documentation on https://hexdocs.pm/plug_cowboy/Plug.Cowboy.html
# for details about using IPv6 vs IPv4 and loopback vs public addresses.
ip: {0, 0, 0, 0, 0, 0, 0, 0},
port: String.to_integer(System.get_env("PORT") || "4000")
],
secret_key_base: secret_key_base,
cache_static_manifest: "priv/static/cache_manifest.json"
and fly.toml
# fly.toml file generated for indiepaper-dev on 2021-09-02T14:43:54+05:30
kill_signal = "SIGTERM"
kill_timeout = 5
[env]
[deploy]
release_command = "/app/bin/indie_paper eval IndiePaper.Release.migrate"
[[services]]
internal_port = 4000
protocol = "tcp"
[services.concurrency]
hard_limit = 25
soft_limit = 20
[[services.ports]]
handlers = ["http"]
port = 80
[[services.ports]]
handlers = ["tls", "http"]
port = 443
[[services.tcp_checks]]
grace_period = "30s" # allow some time for startup
interval = "15s"
restart_limit = 6
timeout = "2s"
Output of dig from indiepaper-dev
# dig aaaa indiepaper-dev-db
; <<>> DiG 9.16.20 <<>> aaaa indiepaper-dev-db
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 4895
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;indiepaper-dev-db. IN AAAA
;; AUTHORITY SECTION:
. 86397 IN SOA a.root-servers.net. nstld.verisign-grs.com. 2021100401 1800 900 604800 86400
;; Query time: 36 msec
;; SERVER: fdaa::3#53(fdaa::3)
;; WHEN: Tue Oct 05 04:50:06 UTC 2021
;; MSG SIZE rcvd: 121
.