Fly Postgres Connections to Replicas Slow

Ouch… It looks like there still is at least an 800ms difference for European users…

$ fly console --region ams
# time curl -i 'https://splashcat.fly.dev/battles/7011/' | fgrep -i fly-
fly-region: ams
fly-request-id: 01HGR0J32E5Q53KW2YZS4SR22C-ams

real    0m1.292s

# time curl -i -H 'fly-prefer-region: iad' 'https://splashcat.fly.dev/battles/7013/' | fgrep -i fly-
fly-region: iad
fly-request-id: 01HGR0NQCT4H01PV8YZBBVDV59-ams

real    0m0.445s

Compared to…

$ fly console --region iad
# time curl -i 'https://splashcat.fly.dev/battles/7017/' | fgrep -i fly-
fly-region: iad
fly-request-id: 01HGR0VDY5AQE0P4G2YN3X129S-iad

real    0m0.301s

And (somewhat less reliably)…

$ fly curl 'https://splashcat.fly.dev/battles/7019/'
REGION  STATUS  DNS     CONNECT  TLS     TTFB            TOTAL    
ams     200     3.9ms    4.2ms   430.6ms 1812.2ms        1813.6ms
dfw     200     7.6ms    7.8ms   180.1ms  793.2ms         921.5ms 
ewr     200     4.3ms    4.6ms    92.9ms  411.5ms         431.2ms 
fra     200     5.4ms    5.5ms   405.2ms 1860.9ms        1884.6ms
iad     200     2.8ms    3  ms    63  ms  557.1ms         559.2ms 
lax     200     6.7ms    6.9ms   271.4ms  919.2ms        1159.7ms
lhr     200    80.3ms   80.8ms   546.9ms 1855  ms        1874.6ms
mia     200     2.4ms    2.5ms   197.5ms  891.6ms        1076.1ms
nrt     200     4.6ms    4.8ms   724.5ms 1075.6ms        1689.6ms
ord     200     5.3ms    5.4ms   222.9ms  773.5ms         869.1ms 
sjc     200     3.4ms    3.6ms   328.3ms  921.3ms        1162.9ms

If you haven’t tried it already, it might make sense to SSH in to the replica and see how a direct connection to its unix-domain socket might work out:

$ fly ssh console -a splashcat-db --region ams
# echo "$FLY_REGION"  # double-check
# su postgres
# time psql -p 5433 -c "select state, count(*) from pg_stat_activity group by state"

If even that takes hundreds of milliseconds, then it may be that your tweaks from July didn’t end up carrying over to the new node…

1 Like