Just a quick note on the state update lags Michael mentions in the “Verify WireGuard tunnel” section —
A few important bits of state for your organizations — most importantly, DNS entries and WireGuard peer information — are synchronized through HashiCorp Consul.
Consul is great, but we are pushing our deployment of it to some limits, and as a result there are some updates that should happen very fast that instead take dozens of seconds. The flyctl changes here patch around that lag (@rugwiro and @michael made my very bad error handling much more resilient).
But the lag is itself bad! We’re working on that too. DNS, in particular, is painful; your instance can be up and responsive to traffic and working for customers, but flyctl can’t talk to it directly until DNS propagates, which makes things feel sometimes like they aren’t working as well as they are. Not ok!
We’ll have updates there as well, a bunch of different things, so that state propagation will hopefully soon stop being a thing we have to think about much.
Thanks for bearing with us!