Hardware failure in LHR two of my DBs down for 24 days as a result

[wjordan]

@l_shepp

Unless I’m missing some big announcement

Email notifications for host issues was announced a couple weeks ago, so you should receive an email for any future issues affecting Machines.

Host Issue notifications

Hi Fly.io community A few weeks ago we experienced some novel hardware failures that caused volumes attached to the affected hosts to become unrecoverable. These hosts’ disks started failing during, or shortly after, a NATS cluster incident where a newly provisioned host reused an IP address from a decommissioned server. The IP reuse was not handled gracefully with the existing NATS configuration. We run a NATS super cluster. In this topology we run a NATS cluster in each region, which…