IAD region connectivity issue

[dar]

is IAD region having connectivity issues ? Our apps can’t connect to Neon while apps in SJC work fine.

[Anarchologist]

Same Here IAD proxy connectivity error please fix

[vasanth]

Same. Works after moving to region ewr.

[vancesolar]

I’m having the same error. My app in IAD can’t connect to my db hosted on Neon. I’m able to connect to the db locally and from a Cloudflare worker, the issue seems to be isolated to Fly

Edit: moving machine to EWR solved the issue

[statim24]

Same problem.

iad can’t connect to neon (aws us-east1) since ~12 hours ago. I switched my fly app to sjc.

I just tried iad again to see if it had resolved, but it is still not working.

not sure how to alert fly to this problem

[Tv1]

I can confirm iad machines can’t reach AWS us-east-1. Traceroute gets lost at packet.net. From ord it works.

[Tv1]

Just started working for me!

[Direwolf]

Seeing the same issue here in IAD. Sometimes it works though, so I’ve just “fly deploy” multiple times until it goes through. Seems like luck of the draw when the connection works.

[Direwolf]

Still seeing the issue sporadically.

[Direwolf]

Anyone have any idea’s how to mitigate this? I can’t even spin up new deploys now, they all fail with a timeout:

 2025-06-23T15:23:51.861 app[e78493d5c04183] iad [info] 15:23:51.860 [error] Postgrex.Protocol (#PID<0.2664.0>) failed to connect: ** (DBConnection.ConnectionError) tcp connect (ep-old-scene-REDACTED.us-east-1.aws.neon.tech:5432): timeout 

[Tv1]

I’m seeing this again, from some machines, while others are happily talking to same AWS service, both good and bad machines are in iad.

I’m trying to use a neon.tech hosted database that’s in AWS us-east-1 from fly.io iad, traceroute gets lost inside packet.net.

[nlencina]

Seeing this too, deploys keep failing for IAD

[Direwolf]

I ended up just moving my compute into a region outside of IAD and that fixed the issue. I wish I could somehow submit a support ticket… maybe if someone here pays for the $29/m service they could send one in?

[halfer]

Here I link a conversation that may be instructive; it’s about how to deploy even though the primary/preferred region is failing. In that discussion the failure case is overcapacity, but I should think that the discussion applies to any failure type.

no capacity available in sjc

@lillian This topic comes up every week or so. The advice I have given a few times is that a capacity failure should be handled by the user, by falling back to a series of acceptable alternatives. I infer that from this text in the API docs: [Machine creation] can fail, and you’re responsible for handling that failure. If you ask for a large Machine, or a Machine in a region we happen to be at capacity for, you might need to retry the request, or to fall back to another region. If you’re worki…

[morse]

I started getting the same connectivity issues today, app can’t connect to the database, hosted by Supabase on AWS in the same region

[leslie]

Hi! Thanks all for flagging this and sharing workarounds. We’ve been investigating your reports in parallel, and narrowed this down to an IPv6 connectivity issue between AWS’ US East zones and a subset of our hosts in iad.

We’re working with our upstream to figure out what’s going on. Traceroutes from our servers to us-east-1 show traffic entering AWS’ network, but we lose visibility after that because of ICMP filtering.

While we continue looking into this, the most straightforward workaround is indeed changing regions. You can move your Machines to another region by:

1. Scaling up in the new region: fly scale count <number-of-machines> --region <new-region>
   If you have apps with volumes, you can pass --from-snapshot <snapshot-id> to create new, non-empty volumes from an existing volume snapshot
2. Scaling down in iad: fly scale count 0 --region iad
3. Optionally, if your primary_region in fly.toml is set to iad, update this to your new region and redeploy

Another option is connecting to your AWS-hosted databases over IPv4 instead of IPv6. Your provider may have an IPv4-only connection string you can swap to. Alternatively, you might be able to change your database connection settings to prefer resolving IPv4.

We’ll pop back in to update this thread with any news.

[lukas-w]

I can confirm that using IPv4 to connect to Neon works around the issue.

The following snipppet - added to an entrypoint script for example - should make most Postgres clients skip the DNS lookup so that they’ll connect via IPv4:

PGHOSTADDR=$(getent ahostsv4 "$PGHOST" | grep DGRAM | awk '{print $1}' | head -n 1)
export PGHOSTADDR

In our case using Doctrine, this environment variable works without any further config changes. One downside is that this will only use the first IP address returned, so there’ll likely be no fail-over to the next one.