AMS issues

,
1

We are having issues with machines in AMS region. We downscaled AMS and upscaled LHR which resolved the issue. We have different machines running in LHR that didn’t have issues, only machines in AMS. But why would this happen? And why would simply changing to LHR resolve the issue? Is there something wrong on Fly’s side? There is nothing on status page.

ams [error] [PR03] could not find a good candidate within 2 attempts at load balancing. last error: [PC01] instance refused connection. is your app listening on 0.0.0.0:8080? make sure it is not only listening on 127.0.0.1 (hint: look at your startup logs, servers often print the address they are listening on)

ams [error] [PC01] instance refused connection. is your app listening on 0.0.0.0:8080? make sure it is not only listening on 127.0.0.1 (hint: look at your startup logs, servers often print the address they are listening on)

[PR04] could not find a good candidate within 20 attempts at load balancing

2

Exact same issue as here: [PR03] could not find a good candidate within 21 attempts at load balancing. last error: [PU03] unreachable worker host

As soon as I rely on AMS machine again, everything fails.

3

This error generally means that the process inside your machine didn’t start successfully, or otherwise refuses to listen on the specified port in your fly.toml, which is different from the other thread linked. If this only happens in ams, maybe this is due to some external dependency that, for some reason, is not accessible from our ams region? Do you see any other logs emitted from the ams machines?

4

event.provider = proxy

2025-06-17T21:30:33.093 proxy[—ID—] ams [error] [PC01] instance refused connection. is your app listening on 0.0.0.0:8080? make sure it is not only listening on 127.0.0.1 (hint: look at your startup logs, servers often print the address they are listening on)

2025-06-17T21:30:43.223 proxy[-- ID --] ams [error] [PR03] could not find a good candidate within 1 attempts at load balancing. last error: [PC01] instance refused connection. is your app listening on 0.0.0.0:8080? make sure it is not only listening on 127.0.0.1 (hint: look at your startup logs, servers often print the address they are listening on)

[PR04] could not find a good candidate within 20 attempts at load balancing

2025-06-17T21:30:56.392 proxy[-- ID --] ams [error] [PR03] could not find a good candidate within 1 attempts at load balancing. last error: [PC01] instance refused connection. is your app listening on 0.0.0.0:8080? make sure it is not only listening on 127.0.0.1 (hint: look at your startup logs, servers often print the address they are listening on)

[PR04] could not find a good candidate within 20 attempts at load balancing

2025-06-17T21:31:07.380 runner[-- ID --] fra [info] Pulling container image registry.fly.io/url

2025-06-17T21:31:12.494 runner[48e3371c135558] fra [info] Successfully prepared image registry.fly.io/url(5.114182633s)

And then LHR or FRA picks up the request with no issues, this machine has worked in the past!

5

Then neither LHR, FRA or AMS were able to handle the incoming requests.
[PR04] could not find a good candidate within 20 attempts at load balancing

Now after a few minutes, LHR can handle the requests.

7

Are you able to fly ssh console into one of the broken machines while this is happening and check whether your app process is running correctly in there and is listening on the right port(s)?

8

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.