Just double-checking my information here: if we want to use fly with anycast IPs and multiple regions, we need to make sure we don’t use any long-lived inbound connections, correct? Otherwise if the packets were to change route mid-session they could end up at a different instance/region than thinks it has the connection open?
Long lived inbound TCP connections work fine, as long as the protocol can handle dropped connections.
Route changes are very. Like weeks. When they happen, the effect is just a broken TCP connection. You won’t get TCP packets you don’t expect, you’ll just stop receiving them at all. The client should reconnect at this point.
Note that route changes are only one event that can break long lived connections. You’ll see similar behavior when we deploy proxy changes or move VMs.