C:\sources>tracert debug.fly.dev
Tracing route to debug.fly.dev [213.188.203.128]
over a maximum of 30 hops:
1 1 ms 1 ms 1 ms XXX
2 19 ms 16 ms 17 ms cpe-72-229-160-1.nyc.res.rr.com [72.229.160.1]
3 13 ms 17 ms 14 ms lag-62.nyctny5002h.netops.charter.com [68.173.20
1.6]
4 18 ms 14 ms 27 ms lag-107.nyquny9101r.netops.charter.com [68.173.1
98.162]
5 17 ms 20 ms 18 ms lag-15.nycmny837aw-bcr00.netops.charter.com [66.
109.6.76]
6 20 ms 13 ms 19 ms lag-1.pr2.nyc20.netops.charter.com [66.109.9.5]
7 15 ms 17 ms 18 ms nyk-b1-link.ip.twelve99.net [62.115.156.214]
8 42 ms 15 ms 17 ms ae-17.a02.nycmny17.us.bb.gin.ntt.net [129.250.8.
229]
9 26 ms 23 ms 19 ms ae-15.r20.nwrknj03.us.bb.gin.ntt.net [129.250.3.
8]
10 24 ms 16 ms 19 ms ae-0.r21.nwrknj03.us.bb.gin.ntt.net [129.250.6.1
7]
11 31 ms 33 ms 36 ms ae-3.r22.chcgil09.us.bb.gin.ntt.net [129.250.2.1
66]
12 32 ms 37 ms 36 ms ae-1.r23.chcgil09.us.bb.gin.ntt.net [129.250.2.2
7]
13 88 ms 80 ms 83 ms ae-1.r24.snjsca04.us.bb.gin.ntt.net [129.250.5.1
7]
14 83 ms 81 ms 82 ms ae-0.r25.snjsca04.us.bb.gin.ntt.net [129.250.3.1
47]
15 192 ms 188 ms 190 ms ae-21.r30.tokyjp05.jp.bb.gin.ntt.net [129.250.5.
77]
16 171 ms 174 ms 184 ms ae-0.a00.tokyjp09.jp.bb.gin.ntt.net [129.250.7.1
4]
17 * * * Request timed out.
18 * * * Request timed out.
19 191 ms 189 ms 189 ms 213.188.203.128
Trace complete.
C:\sources>
=== Headers ===
Host: debug.fly.dev
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.0.0.0 Safari/537.36
Fly-Request-Id: 01H31E7S40AF8AM3SDHSYA2VNA-lhr
Cache-Control: max-age=0
X-Forwarded-Proto: https
Fly-Forwarded-Port: 443
X-Forwarded-Ssl: on
Sec-Fetch-Site: cross-site
Sec-Fetch-Mode: navigate
Accept-Language: en-US,en;q=0.9
X-Forwarded-For: 72.229.170.XXX, 213.188.204.128
X-Request-Start: t=1686896895104834
Fly-Tracestate:
Accept-Encoding: gzip, deflate, br
Fly-Client-Ip: 72.229.170.XXX
Fly-Forwarded-Proto: https
X-Forwarded-Port: 443
Fly-Region: lhr
Via: 2 fly.io
Fly-Traceparent: 00-83b5a86726b56e23e9d206d99fca159b-4069d65df45afa9c-00
Sec-Ch-Ua-Mobile: ?0
Sec-Ch-Ua-Platform: "Windows"
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
Sec-Fetch-User: ?1
Sec-Fetch-Dest: document
Referer: https://community.fly.io/
Fly-Forwarded-Ssl: on
Sec-Ch-Ua: "(Not(A:Brand";v="8", "Chromium";v="98"
2023-06-16 06:28:15.110094509 +0000 UTC m=+99474.153296469
I think debug.fly.dev is supposed to be anycast ed with instances in all regions. I am in NYC. I am reaching the London region, IDK why, and its going the LONG WAY around plant earth through Japan.
TMobile LTE NYC → LGA region. (ICMP blocked by TM)
M247 ASN VPN NYC → IAD region. Eghhhhh, traceroute included
~ $ traceroute debug.fly.dev
traceroute to debug.fly.dev (213.188.205.128), 30 hops max, 60 byte packets
1 10.128.0.1 (10.128.0.1) 40.458 ms 39.595 ms 43.641 ms
2 vlan102.as13.nyc1.us.m247.com (146.70.115.137) 44.209 ms 43.525 ms 42.841 ms
3 irb-0.agg3v.nyc1.us.m247.com (193.9.115.218) 61.462 ms 60.657 ms 59.761 ms
4 146.70.0.227 (146.70.0.227) 47.796 ms 47.083 ms 52.565 ms
5 146.70.0.231 (146.70.0.231) 51.158 ms 50.328 ms 49.414 ms
6 * * hu0-5-0-2.201.rcr21.ewr03.atlas.cogentco.com (38.88.8.21) 56.889 ms
7 be3020.ccr42.jfk02.atlas.cogentco.com (154.54.41.113) 56.325 ms 71.502 ms be2273.ccr41.jfk02.atlas.cogentco.com (154.54.83.205) 58.628 ms
8 be2806.ccr41.dca01.atlas.cogentco.com (154.54.40.106) 58.070 ms be2807.ccr42.dca01.atlas.cogentco.com (154.54.40.110) 57.540 ms 63.212 ms
9 be3084.ccr41.iad02.atlas.cogentco.com (154.54.30.66) 62.773 ms 62.313 ms 61.931 ms
10 * * *
11 * * *
=== Headers ===
Host: debug.fly.dev
Fly-Forwarded-Proto: http
X-Forwarded-Proto: http
Fly-Traceparent: 00-8d443c5eff516d8ab66ca6d8c9572957-760516fbfa0090cd-00
Fly-Tracestate:
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
X-Forwarded-For: 146.70.115.xxx, 213.188.205.128
X-Forwarded-Port: 80
Fly-Region: iad
Fly-Request-Id: 01H31F0XWA5RFDE8B9QBARXW03-iad
Via: 1.1 fly.io
Fly-Client-Ip: 146.70.115.xxx
X-Request-Start: t=1686897719178533
User-Agent: Mozilla/5.0 (Linux; Android 12; SM-A426U1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Mobile Safari/537.36
Accept-Encoding: gzip, deflate
Fly-Forwarded-Ssl: off
X-Forwarded-Ssl: off
Fly-Forwarded-Port: 80
Accept-Language: en-US,en;q=0.9
2023-06-16 06:41:59.183346137 +0000 UTC m=+52667.078770112
Plot thickens, its DNS load balancer, same Optimum Online cable modem, 2 traceroutes 5 minutes apart, 2 different IPs for debug.fly.dev, I see same bad IAD routing over Cogent on the 2nd IP.
C:\sources>tracert debug.fly.dev
Tracing route to debug.fly.dev [77.83.140.164]
over a maximum of 30 hops:
1 <1 ms <1 ms <1 ms 192.168.1.1
2 9 ms 8 ms 11 ms 10.240.186.97
3 10 ms 9 ms 9 ms ubr208-ge1-0-1.cmts.nyk4ny.cv.net [67.59.246.118
]
4 13 ms 16 ms 10 ms ool-4353e790.dyn.optonline.net [67.83.231.144]
5 11 ms 12 ms 11 ms 67.59.239.83
6 11 ms 15 ms 11 ms 64.15.3.230
7 * * * Request timed out.
8 12 ms 13 ms 11 ms be2073.ccr31.jfk05.atlas.cogentco.com [154.54.0.
229]
9 11 ms 10 ms 10 ms be2809.rcr21.b007023-2.jfk05.atlas.cogentco.com
[154.54.81.202]
10 * * * Request timed out.
11 * * * Request timed out.
12 10 ms 11 ms 12 ms 77.83.140.164
Trace complete.
C:\sources>tracert debug.fly.dev
Tracing route to debug.fly.dev [213.188.205.128]
over a maximum of 30 hops:
1 <1 ms <1 ms <1 ms 192.168.1.1
2 9 ms 9 ms 9 ms 10.240.186.97
3 10 ms 9 ms 9 ms 67.59.246.120
4 12 ms 13 ms 10 ms ool-4353e792.dyn.optonline.net [67.83.231.146]
5 11 ms 11 ms 11 ms 64.15.8.59
6 12 ms 11 ms 18 ms 65.19.121.220
7 * * * Request timed out.
8 14 ms 13 ms 10 ms be3363.ccr42.jfk02.atlas.cogentco.com [154.54.3.
125]
9 18 ms 18 ms 17 ms be2807.ccr42.dca01.atlas.cogentco.com [154.54.40
.110]
10 19 ms 19 ms 19 ms be3084.ccr41.iad02.atlas.cogentco.com [154.54.30
.66]
11 * * * Request timed out.
12 * * * Request timed out.
13 * * * Request timed out.
14 17 ms 16 ms 16 ms 213.188.205.128
Trace complete.
C:\sources>
But wait there is more, Chrome’s HTML page, or Chrome (don’t feel like checking if its ISP DNS or DOH), on Optimum Online NYC, is showing SEA region. Because Chrome internally thinks debug.fly.dev is “213.188.200.128”, not the other 2 domains. On same Optimum NYC machine, I did
C:\sources>nslookup debug.fly.dev
Server: one.one.one.one
Address: 1.1.1.1
Non-authoritative answer:
Name: debug.fly.dev
Addresses: 2a09:8280:68:b297:25bb:14b3:3817:5daf
2a09:8280:1:763f:8bdd:34d1:c624:78cd
213.188.202.2
213.188.206.128
213.188.201.128
77.83.140.164
213.188.203.128
213.188.200.128
213.188.205.128
213.188.204.128
C:\sources>
Ummmm, should 213.188.203.128 in London really be listed as an A record for a USA based UA? This DNS LB seems wonky.
C:\sources>tracert 213.188.200.128
Tracing route to 213.188.200.128 over a maximum of 30 hops
1 <1 ms <1 ms <1 ms 192.168.1.1
2 9 ms 9 ms 9 ms 10.240.186.97
3 10 ms 10 ms 9 ms 67.59.246.122
4 12 ms 10 ms 11 ms ool-4353e796.dyn.optonline.net [67.83.231.150]
5 12 ms 11 ms 11 ms 64.15.8.170
6 11 ms 11 ms 9 ms rtr8.mhe.whplny.cv.net [65.19.121.222]
7 * * * Request timed out.
8 12 ms 11 ms 11 ms be3363.ccr42.jfk02.atlas.cogentco.com [154.54.3.
125]
9 22 ms 23 ms 23 ms be2890.ccr22.cle04.atlas.cogentco.com [154.54.82
.245]
10 30 ms 30 ms 29 ms be2718.ccr42.ord01.atlas.cogentco.com [154.54.7.
129]
11 42 ms 41 ms 41 ms be2832.ccr22.mci01.atlas.cogentco.com [154.54.44
.169]
12 51 ms 52 ms 51 ms be3036.ccr22.den01.atlas.cogentco.com [154.54.31
.89]
13 65 ms 64 ms 64 ms be3038.ccr32.slc01.atlas.cogentco.com [154.54.42
.97]
14 83 ms 86 ms 83 ms be2029.ccr22.sea02.atlas.cogentco.com [154.54.86
.110]
15 * * * Request timed out.
16 * * * Request timed out.
17 * * * Request timed out.
18 83 ms 76 ms 75 ms 213.188.200.128
Trace complete.
C:\sources>