Phoenix: "The app is not listening on the expected address" (8080) But under the processes list, the app seems to start each time under a different random port

[xji]

(I’ve seen this question asked before: WARNING: The app is listening on the incorrect address, but my case seems to be different.)

I’m trying to deploy an existing Phoenix app to fly.io. After a few tries it seems to work, but now I get the “The app is not listening on the expected address and will not be reachable by fly-proxy.” error.

The log then proceeds to print this:

Found these processes inside the machine with open listening sockets:
  PROCESS

                                                          | ADDRESSES
-----------------------------------------------------------------------------------------------------------------------------------------------------------------
-----------------------------------------------------------------------------------------------------------------------------------------------------------------
----------------------------------------------------------*---------------------------------------
  /app/erts-13.1.3/bin/beam.smp -- -root /app -bindir /app/erts-13.1.3/bin -progname erl -- -home /nonexistent -- -proto_dist inet6_tcp -- -noshell -s elixir sta
rt_cli -mode embedded -setcookie PWOVL6U2DQUEE7V7BNYSLJPK2XG7CYEZPH6MS4Y2QJQDIW6R7E4A==== -sname magpie -config /app/releases/2.3.4/sys -boot /app/releases/2.3.4
/start -boot_var RELEASE_LIB /app/lib -- -extra --no-halt | [::]:33127
  /.fly/hallpass

                                                          | [fdaa:2:6f01:a7b:141:65dd:fe42:2]:22
  /app/erts-13.1.3/bin/epmd -daemon

                                                          | 0.0.0.0:4369, [::]:4369

Curiously, the first item in the list seems to be my app? I have no idea why it’s listening on 33127 though.

I tried to run fly deploy again, and now it’s on 40627.

Is this the app process, or is this something irrelevant?

I checked my runtime.exs, prod.exs and config.exs against a vanilla new project, which I was able to deploy successfully, and I seem to find no difference in terms of the port configuration… (I still don’t know where the “8080” is supposed to be explicitly specified though.)

I wonder what the issue could be. Is 8080 some sort of default for Phoenix apps, which will be blocked if there was something at this address and the app automatically tried another port? Or has the app actually exited, and the first item under “processes” is actually not the app?

[xji]

More logs. Has the Elixir app actually crashed? Wasn’t sure from reading the logs.

2023-06-30T20:05:29Z runner[d891699f64e418] ams [info]Configuring firecracker
2023-06-30T20:05:30Z app[d891699f64e418] ams [info] INFO Starting init (commit: db101a53)...
2023-06-30T20:05:30Z app[d891699f64e418] ams [info] INFO Preparing to run: `/app/bin/server` as nobody
2023-06-30T20:05:30Z app[d891699f64e418] ams [info] INFO [fly api proxy] listening at /.fly/api
2023-06-30T20:05:30Z app[d891699f64e418] ams [info]2023/06/30 20:05:30 listening on [fdaa:2:6f01:a7b:141:75bf:8712:2]:22 (DNS: [fdaa::3]:53)
2023-06-30T20:05:33Z app[d891699f64e418] ams [info] WARN Reaped child process with pid: 289 and signal: SIGUSR1, core dumped? false
2023-06-30T20:05:36Z app[d891699f64e418] ams [info]20:05:36.043 [info] tzdata release in place is from a file last modified Tue, 22 Dec 2020 23:35:21 GMT. Releas
e file on server was last modified Tue, 28 Mar 2023 20:25:39 GMT.
2023-06-30T20:05:37Z app[d891699f64e418] ams [info]20:05:37.020 [info] Tzdata has updated the release from 2020e to 2023c
2023-06-30T20:05:43Z runner[2865603fee7798] ams [info]Pulling container image registry.fly.io/magpie-cogsciprag:deployment-01H46YHMFZQY1VYEMHA8NYBJYF
2023-06-30T20:05:45Z runner[2865603fee7798] ams [info]Successfully prepared image registry.fly.io/magpie-cogsciprag:deployment-01H46YHMFZQY1VYEMHA8NYBJYF (2.7323
74056s)
2023-06-30T20:05:47Z runner[2865603fee7798] ams [info]Configuring firecracker
2023-06-30T20:05:47Z app[2865603fee7798] ams [info][    0.030212] Spectre V2 : WARNING: Unprivileged eBPF is enabled with eIBRS on, data leaks possible via Spect
re v2 BHB attacks!
2023-06-30T20:05:48Z app[2865603fee7798] ams [info] INFO Starting init (commit: db101a53)...
2023-06-30T20:05:48Z app[2865603fee7798] ams [info] INFO Preparing to run: `/app/bin/server` as nobody
2023-06-30T20:05:48Z app[2865603fee7798] ams [info] INFO [fly api proxy] listening at /.fly/api
2023-06-30T20:05:48Z app[2865603fee7798] ams [info]2023/06/30 20:05:48 listening on [fdaa:2:6f01:a7b:10e:38e9:804a:2]:22 (DNS: [fdaa::3]:53)
2023-06-30T20:05:50Z app[2865603fee7798] ams [info] WARN Reaped child process with pid: 288 and signal: SIGUSR1, core dumped? false
2023-06-30T20:05:53Z app[2865603fee7798] ams [info]20:05:53.887 [info] tzdata release in place is from a file last modified Tue, 22 Dec 2020 23:35:21 GMT. Releas
e file on server was last modified Tue, 28 Mar 2023 20:25:39 GMT.
2023-06-30T20:05:54Z app[2865603fee7798] ams [info]20:05:54.934 [info] Tzdata has updated the release from 2020e to 2023c
error.message="instance refused connection. is your app listening on 0.0.0.0:8080? make sure it is not only listening on 127.0.0.1 (hint: look at your startup lo
gs, servers often print the address they are listening on)" 2023-06-30T20:06:34Z proxy[d891699f64e418] ams [error]request.method="GET" request.id="01H46YM7F794GE
PTNK35YGZE8Z-fra"

[xji]

fly.toml:

kill_signal = "SIGTERM"

[deploy]
  release_command = "/app/bin/migrate"

[env]
  PHX_HOST = {my-address}
  PORT = "8080"

[http_service]
  internal_port = 8080
  force_https = true
  auto_stop_machines = true
  auto_start_machines = true
  min_machines_running = 0
  processes = ["app"]
  [http_service.concurrency]
    type = "connections"
    hard_limit = 1000
    soft_limit = 1000

[xji]

OK I finally saw this line from the logs:

Configuration :server was not enabled for Magpie.Endpoint, http/https services won't start

I have no idea why this is the case, since my rel/overlays/bin/server has exactly the same content as the vanilla test app:

#!/bin/sh
cd -P -- "$(dirname -- "$0")"
PHX_SERVER=true exec ./magpie start

But anyways I removed the conditional part on

if System.get_env("PHX_SERVER") do
  config :magpie, MagpieWeb.Endpoint, server: true
end

in my runtime.exs and it finally worked.