Can't restart or access machine

Questions / Help
,
1

My Postgres machine has been unreachable for ~2 hours and I cannot recover it through any CLI commands.

App: guido-postgres
Machine: d8939ddb600e48
Region: nrt

Symptoms:

  • host_status: "unreachable" in machines API

  • config: null returned for the machine

  • fly machine restart → 408 timeout

  • fly machine stop → 408 timeout

  • fly ssh console → flyctl nil pointer panic (because config is null)

  • Manual restart via dashboard → “Lease Failed”

  • Releases v14 and v15 stuck in running state since 12:40 UTC today

Last request ID: 01KRGNFGST33HYMYH3R3H97352-nrt

My backend app guido-backend is down because it can’t reach Postgres. Any help appreciated.

3

Hi! This host seems to be unresponsive, we’re looking into it. You should get a notification in your fly.io dashboard with more details if it takes us longer than a few minutes to bring the host back up.

This kind of thing is precisely why we recommend multiple-machine Postgres setups for critical data! If your Postgres server going down causes problems, it means you do care about uptime, and a multi-machine setup will be more resilient to host failures.

4

It works now, thanks for the help and recommendation. I will setup a multiple machines to host the postgres instance