Multiple ports on fly app not working using modified example toml

Questions / Help
1

Hello All, I am attempting to configure a fly.io application which has two services exposed over two different ports. The services run on ports 80 and 805. I am using a fly deploy command with the following fly.toml:


 app = "runner-06fed748-ae10-42ff-918c-356d5b285eca-web-api"
 kill_signal = "SIGINT"
 kill_timeout = "5s"
 processes = []
 
 [[vm]]
   size = "shared-cpu-4x"
   memory = "8GB"
 
 [env]
   <redacted>
 
 [experimental]
   allowed_public_ports = []
   auto_rollback = true
 
 [[services]]
   http_checks = []
   internal_port = 80
   processes = ["app"]
   protocol = "tcp"
   script_checks = []
 
   [services.concurrency]
     hard_limit = 25
     soft_limit = 20
     type = "connections"
 
   [[services.ports]]
     force_https = true
     handlers = ["http"]
     port = 80
 
   [[services.ports]]
     handlers = ["tls", "http"]
     port = 443
 
   [[services.ports]]
     handlers = ["tls", "http"]
     port = 80
 
   [[services.tcp_checks]]
     grace_period = "1s"
     interval = "15s"
     restart_limit = 0
     timeout = "2s"
 
 [[services]]
   http_checks = []
   internal_port = 805
   processes = ["app"]
   protocol = "tcp"
   script_checks = []
 
   [services.concurrency]
     hard_limit = 25
     soft_limit = 20
     type = "connections"
 
   [[services.ports]]
     force_https = true
     handlers = ["http"]
     port = 804
 
   [[services.ports]]
     handlers = ["tls", "http"]
     port = 805
 
   [[services.tcp_checks]]
     grace_period = "1s"
     interval = "15s"
     restart_limit = 0
     timeout = "2s"
 
 [build]
   dockerfile = "Dockerfile"
   build_args = { POETRY_VERSION = "1.4.2", PYTHON_VERSION = "3.9", POSTGRES_VERSION = "15", REDIS_VERSION = "7", CELERY_VERSION = "5.3.6" }

I based this file on this fly-apps example: GitHub - fly-apps/fly-app-with-multiple-internal-ports: Example of how to deploy an app that has multiple ports listened to. I have exposed both ports in the docker container, and both services start.

2025-03-20 14:13:18.126	
Health check on port 80 is now passing.
2025-03-20 14:13:08.448	
Health check on port 805 is now passing.
2025-03-20 14:13:06.574	
INFO:     Application startup complete.
2025-03-20 14:13:06.574	
INFO:     Waiting for application startup.
2025-03-20 14:13:06.574	
INFO:     Started server process [678]
2025-03-20 14:13:06.542	
DEBUG:asyncio:Using selector: EpollSelector
2025-03-20 14:13:05.349	
INFO:     Started reloader process [667] using StatReload
2025-03-20 14:13:05.349	
INFO:     Uvicorn running on http://0.0.0.0:805 (Press CTRL+C to quit)
2025-03-20 14:13:05.349	
INFO:     Will watch for changes in these directories: ['/hotswap-server']
2025-03-20 14:13:05.348	
INFO:__main__:Starting Hotswap Server on port 805...
2025-03-20 14:13:05.308	
INFO:     Application startup complete.
2025-03-20 14:13:05.308	
INFO:     Waiting for application startup.
2025-03-20 14:13:05.308	
INFO:     Started server process [676]
2025-03-20 14:13:03.724	
INFO:     Started reloader process [666] using StatReload
2025-03-20 14:13:03.724	
INFO:     Uvicorn running on http://0.0.0.0:80 (Press CTRL+C to quit)
2025-03-20 14:13:03.723	
INFO:     Will watch for changes in these directories: ['/app']
2025-03-20 14:13:03.017	
Creating virtualenv web-api-9TtSrW0h-py3.9 in /root/.cache/pypoetry/virtualenvs
2025-03-20 14:13:03.003	
Health check on port 80 has failed. Your app is not responding properly. Services exposed on ports [80, 80, 443] will have intermittent failures until the health check passes.
2025-03-20 14:13:02.828	
2025-03-20 21:13:02,827 INFO success: hotswap entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2025-03-20 14:13:02.828	
2025-03-20 21:13:02,819 INFO success: fastapi entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)

However, when attempting to connect to either service from the internet, I receive a “Site can’t be reached DNS_PROBE_FINISHED_NXDOMAIN.” Interestingly, when I omit the second service from the fly.toml, the config works for the first service.

Can anyone spot what I am doing wrong? Any help would be much appreciated.

3 
 
   [[services.ports]]
     force_https = true
     handlers = ["http"]
     port = 80
 
   [[services.ports]]
     handlers = ["tls", "http"]
     port = 443
 
   [[services.ports]]
     handlers = ["tls", "http"]
     port = 80

You have two services.ports for port 80, so they stomp on each other. Remove one. (I think you want to remove the second one that has tls and http handlers).

  • Daniel
1 Like
4

Thanks, Daniel. This was indeed an issue but a secondary one I introduced during troubleshooting, not the core issue.

Here is the updated fly.toml with the double port 80 removed that continues to have the issue. Also updated to use port 8080 for the secondary service:

app = "runner-81cb61d0-1a8d-49c5-8663-27040503a7c6-web-api"
kill_signal = "SIGINT"
kill_timeout = "5s"

[build]
dockerfile = "Dockerfile"

[deploy]
strategy = "immediate"

[env]
<redacted>

[experimental]
auto_rollback = true

  [experimental.attached]
  secrets = { }

[[services]]
internal_port = 80
processes = [ "app" ]
protocol = "tcp"

  [services.concurrency]
  hard_limit = 25
  soft_limit = 20
  type = "connections"

  [[services.ports]]
  force_https = true
  handlers = [ "http" ]
  port = 80

  [[services.ports]]
  handlers = [ "tls", "http" ]
  port = 443

  [[services.tcp_checks]]
  grace_period = "1s"
  interval = "15s"
  timeout = "2s"

[[services]]
internal_port = 8080
processes = [ "app" ]
protocol = "tcp"

  [services.concurrency]
  hard_limit = 25
  soft_limit = 20
  type = "connections"

  [[services.ports]]
  force_https = true
  handlers = [ "http" ]
  port = 8080

  [[services.ports]]
  handlers = [ "tls", "http" ]
  port = 8_081

  [[services.tcp_checks]]
  grace_period = "1s"
  interval = "15s"
  timeout = "2s"

[[vm]]
memory = "8GB"
size = "shared-cpu-4x"

If I delete this block, access to the service on port 80 / 443 works.

[[services]]
internal_port = 8080
processes = [ "app" ]
protocol = "tcp"

  [services.concurrency]
  hard_limit = 25
  soft_limit = 20
  type = "connections"

  [[services.ports]]
  force_https = true
  handlers = [ "http" ]
  port = 8080
5

What’s the error you get when you have the block for ports 8080 and 8081 enabled? (I bet it’s not the same error you posted initially).

  • Daniel
6

Unfortunately yields same error. This is on all four of 80/443 and 8080/8081. Just appears like the outside ports are not open.

This site can’t be reached
Check if there is a typo in runner-fd5e9810-06eb-4cda-b0a7-5d8fe3613892-web-api.fly.dev.
DNS_PROBE_FINISHED_NXDOMAIN

Interestingly it only occurs when both [[services]] blocks are present. When only the first services block is present, the first service is accessible on 80/443.

7

Logs (which you can check with fly logs) look normal to me. I can’t inspect your app further because you deleted it - a deleted app would get the NXDOMAIN error (the hostname is registered when the app is created and removed when the app is deleted) but I’m assuming you get that error while the app is running :slight_smile:

Is it possible to stand up one of these apps and leave it around for me to have a look? To set expectations, I’m not likely to be able to get to it today, but I’ll try to have a look tomorrow.

  • Daniel
8

Hi Daniel, appreciate your help. I spun up this one and will leave it running: runner-d144c86d-fde1-47ce-bd61-e6b734d2e52c-web-api

Yep, getting the error while the app is running and health checks look to be passing.

2025-03-20 19:16:13.600	
Health check on port 8080 is now passing.
2025-03-20 19:16:01.215	
Health check on port 80 is now passing.
2025-03-20 19:15:58.741	
Health check on port 8080 has failed. Your app is not responding properly. Services exposed on ports [8080, 8081] will have intermittent failures until the health check passes.
2025-03-20 19:15:58.144	
INFO:     Application startup complete.
2025-03-20 19:15:58.144	
INFO:     Waiting for application startup.
2025-03-20 19:15:58.144	
INFO:     Started server process [678]
2025-03-20 19:15:58.116	
DEBUG:asyncio:Using selector: EpollSelector
2025-03-20 19:15:57.175	
INFO:     Started reloader process [667] using StatReload
2025-03-20 19:15:57.175	
INFO:     Uvicorn running on http://0.0.0.0:8080 (Press CTRL+C to quit)
2025-03-20 19:15:57.175	
INFO:     Will watch for changes in these directories: ['/hotswap-server']
2025-03-20 19:15:57.174	
INFO:__main__:Starting Hotswap Server on port 8080...
2025-03-20 19:15:57.143	
INFO:     Application startup complete.
2025-03-20 19:15:57.143	
INFO:     Waiting for application startup.
2025-03-20 19:15:57.143	
INFO:     Started server process [676]
2025-03-20 19:15:55.668	
INFO:     Started reloader process [666] using StatReload
2025-03-20 19:15:55.668	
INFO:     Uvicorn running on http://0.0.0.0:80 (Press CTRL+C to quit)
2025-03-20 19:15:55.668	
2025-03-21 02:15:55,668 INFO success: hotswap entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2025-03-20 19:15:55.668	
2025-03-21 02:15:55,668 INFO success: fastapi entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2025-03-20 19:15:55.668	
INFO:     Will watch for changes in these directories: ['/app']
2025-03-20 19:15:55.113	
Creating virtualenv web-api-9TtSrW0h-py3.9 in /root/.cache/pypoetry/virtualenvs
2025-03-20 19:15:54.262	
2025-03-21 02:15:54,262 INFO spawned: 'hotswap' with pid 667
2025-03-20 19:15:54.259	
2025-03-21 02:15:54,259 INFO spawned: 'fastapi' with pid 666
2025-03-20 19:15:53.256	
2025-03-21 02:15:53,256 INFO supervisord started with pid 657
9

Hi, try assigning an IP to this app.

fly ips allocate-v4 --shared should do it.

The logs show the app started correctly and is listening on both ports:

2025-03-20 19:16:13.600	Health check on port 8080 is now passing.
2025-03-20 19:16:01.215	Health check on port 80 is now passing.

the proxy configuration also looks correct. I tested with your exact verbatim fly.toml and a small Dockerfile that just stands up simple http servers on those two ports, and it works well. But an external hostname won’t be assigned unless the app has an associated IP address (mine did, at this point it’s the only known difference between your app and mine).

Let me know if that works :slight_smile:

  • Daniel
1 Like
10

Daniel, you are amazing! Works like a charm. Appreciate the help! :slight_smile: