I know, the status page lists everything OK today but we are experiencing strongly lagging communication between Germany (Berlin area) and Chicago (ORD region). The fly CLI times out on simple things as listing apps. Communication to our self-hosted PostgreSQL instances time out while after a few minutes, they succeed again. App logs look OK (when accessible).
Is there anything going on that can explain this?
Would you mind sharing your app name and can you run a mtr debug.fly.dev or curl -v -H "flyio-debug: doit" https://debug.fly.dev and share the results?
mtr:
catbus (2003:d3:c718:9300:5fb9:a059:578d:c808) -> debug.fly.dev (2a09:8280:1:763f:8bdd:34d1:c624:78cd) 2026-03-30T20:24:42+0200
Keys: Help Display mode Restart statistics Order of fields quit
Packets Pings
Host Loss% Snt Last Avg Best Wrst StDev
1. p200300d3c71893000000000000000001.dip0.t-ipconnect.de 0,0% 54 1,2 1,3 1,0 2,2 0,2
2. 2003:0:8001:d000::1 0,0% 54 2,5 4,2 2,2 29,5 4,0
3. 2003:0:f00::77a 24,1% 54 5,6 5,9 5,3 7,0 0,3
4. 2003:0:f00::77b 9,4% 54 16,3 16,3 15,4 18,4 0,4
5. 2001:668:0:2:ffff:0:8d88:6e29 0,0% 54 16,3 20,0 15,4 75,0 11,7
6. 2001:668:0:3:ffff:0:9a0e:4952 40,7% 54 16,2 28,1 15,7 233,6 40,1
7. debug.fly.dev 35,2% 54 15,8 15,8 15,4 16,8 0,4
curl:
20:25 $ curl -v -H "flyio-debug: doit" https://debug.fly.dev
* Host debug.fly.dev:443 was resolved.
* IPv6: 2a09:8280:1:763f:8bdd:34d1:c624:78cd, 2a09:8280:1::1:2
* IPv4: 37.16.16.7, 37.16.21.10, 77.83.140.164
* Trying [2a09:8280:1:763f:8bdd:34d1:c624:78cd]:443...
* ALPN: curl offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* SSL Trust Anchors:
* OpenSSL default paths (fallback)
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384 / x25519 / id-ecPublicKey
* ALPN: server accepted h2
* Server certificate:
* subject: CN=*.fly.dev
* start date: Feb 21 23:24:17 2026 GMT
* expire date: May 22 23:24:16 2026 GMT
* issuer: C=US; O=Let's Encrypt; CN=E8
* Certificate level 0: Public key type EC/prime256v1 (256/128 Bits/secBits), signed using ecdsa-with-SHA384
* Certificate level 1: Public key type EC/secp384r1 (384/192 Bits/secBits), signed using sha256WithRSAEncryption
* Certificate level 2: Public key type RSA (4096/152 Bits/secBits), signed using sha256WithRSAEncryption
* subjectAltName: "debug.fly.dev" matches cert's "*.fly.dev"
* SSL certificate verified via OpenSSL.
* Established connection to debug.fly.dev (2a09:8280:1:763f:8bdd:34d1:c624:78cd port 443) from 2003:d3:c718:9300:5fb9:a059:578d:c808 port 44726
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://debug.fly.dev/
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: debug.fly.dev]
* [HTTP/2] [1] [:path: /]
* [HTTP/2] [1] [user-agent: curl/8.18.0]
* [HTTP/2] [1] [accept: */*]
* [HTTP/2] [1] [flyio-debug: doit]
> GET / HTTP/2
> Host: debug.fly.dev
> User-Agent: curl/8.18.0
> Accept: */*
> flyio-debug: doit
>
* Request completely sent off
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
< HTTP/2 200
< fly-region: fra
< remote-addr: 172.16.0.2:49926
< date: Mon, 30 Mar 2026 18:25:42 GMT
< content-length: 534
< content-type: text/plain; charset=utf-8
< server: Fly/4087d4940 (2026-03-30)
< via: 2 fly.io
< fly-request-id: 01KMZZX61082TXHRHJ8M300NRD-fra
< flyio-debug: {"n":"edge-cf-fra2-ad2e","nr":"fra","ra":"2003:d3:c718:9300:5fb9:a059:578d:c808","rf":"Verbatim","sr":"fra","sdc":"fra1","sid":"217819721c9896","st":0,"nrtt":1,"bn":"worker-dp-fra1-816b","mhn":null,"mrtt":null}
<
=== Headers ===
Host: debug.fly.dev
X-Forwarded-Proto: https
X-Forwarded-Ssl: on
Fly-Forwarded-Port: 443
Via: 2 fly.io
X-Request-Start: t=1774895142944929
Fly-Forwarded-Proto: https
User-Agent: curl/8.18.0
Accept: */*
Fly-Client-Ip: 2003:d3:c718:9300:5fb9:a059:578d:c808
X-Forwarded-For: 2003:d3:c718:9300:5fb9:a059:578d:c808, 2a09:8280:1:763f:8bdd:34d1:c624:78cd
Fly-Forwarded-Ssl: on
X-Forwarded-Port: 443
Fly-Region: fra
Fly-Request-Id: 01KMZZX61082TXHRHJ8M300NRD-fra
2026-03-30 18:25:42.954108747 +0000 UTC m=+13561839.449129534
* Connection #0 to host debug.fly.dev:443 left intact
I can share the app names but not in public.
Feel free to email the app name to peter at fly dot io.
Do you mind trying over IPv4 too? The IPv6 path seems to have a ton of losses at the moment, I’ll look into that. I’m wondering whether IPv4 is affected though.
We just changed routing around a bit, are you still seeing the same slowness?
I think, it has improved. Thanks so far! I am still dealing with app issues, so cannot completely tell, yet. I will let you know here when problems persist.
2 posts were split to a new topic: Postgres ETIMEOUT error in ORD region