Postgres downtime

I’m having a failure that seems to have randomly started in a postgres app that’s only a few days old.

Logs output:

2024-12-11T15:03:41.321 app[148e477c261d18] lhr [info] repmgrd | restarting in 5s [attempt 361]

2024-12-11T15:03:41.805 app[148e477c261d18] lhr [info] failed post-init: failed to establish connection to local node: failed to connect to `host=fdaa:b:c8d6:a7b:be65:4a23:6c42:2 user=postgres database=postgres`: dial error (dial tcp [fdaa:b:c8d6:a7b:be65:4a23:6c42:2]:5433: connect: connection refused). Retrying...

2024-12-11T15:03:42.805 app[148e477c261d18] lhr [info] failed post-init: failed to establish connection to local node: failed to connect to `host=fdaa:b:c8d6:a7b:be65:4a23:6c42:2 user=postgres database=postgres`: dial error (dial tcp [fdaa:b:c8d6:a7b:be65:4a23:6c42:2]:5433: connect: connection refused). Retrying...

2024-12-11T15:03:43.805 app[148e477c261d18] lhr [info] failed post-init: failed to establish connection to local node: failed to connect to `host=fdaa:b:c8d6:a7b:be65:4a23:6c42:2 user=postgres database=postgres`: dial error (dial tcp [fdaa:b:c8d6:a7b:be65:4a23:6c42:2]:5433: connect: connection refused). Retrying...

2024-12-11T15:03:44.031 app[148e477c261d18] lhr [info] monitor | failed to open local connection: failed to connect to `host=fdaa:b:c8d6:a7b:be65:4a23:6c42:2 user=flypgadmin database=postgres`: dial error (dial tcp [fdaa:b:c8d6:a7b:be65:4a23:6c42:2]:5433: connect: connection refused)

2024-12-11T15:03:44.805 app[148e477c261d18] lhr [info] failed post-init: failed to establish connection to local node: failed to connect to `host=fdaa:b:c8d6:a7b:be65:4a23:6c42:2 user=postgres database=postgres`: dial error (dial tcp [fdaa:b:c8d6:a7b:be65:4a23:6c42:2]:5433: connect: connection refused). Retrying...

Output of checks:

  NAME | STATUS   | MACHINE        | LAST UPDATED | OUTPUT
-------*----------*----------------*--------------*------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  pg   | critical | 148e477c261d18 | 1h12m ago    | connect: no route to host
-------*----------*----------------*--------------*------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  role | critical | 148e477c261d18 | 25m41s ago   | 500 Internal Server Error
       |          |                |              | failed to connect to local node: failed to connect to `host=fdaa:b:c8d6:a7b:be65:4a23:6c42:2 user=repmgr database=repmgr`: dial error (dial tcp [fdaa:b:c8d6:a7b:be65:4a23:6c42:2]:5433: connect: connection refused)
-------*----------*----------------*--------------*------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  vm   | passing  | 148e477c261d18 | 43m27s ago   | [✓] checkDisk: 824.05 MB (83.6%) free space on /data/ (34.11µs)
       |          |                |              | [✓] checkLoad: load averages: 0.00 0.00 0.00 (58.57µs)
       |          |                |              | [✓] memory: system spent 0s of the last 60s waiting on memory (24.59µs)
       |          |                |              | [✓] cpu: system spent 576ms of the last 60s waiting on cpu (19.87µs)
       |          |                |              | [✓] io: system spent 480ms of the last 60s waiting on io (30.83µs)
-------*----------*----------------*--------------*------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Anyone else having issues? I haven’t made any changes to the app, so wondering if this is a fly issue.

Small update - I can’t even detach this DB from my rails app, as the fly postgres detach command gives me the error:

Error: no active leader found

What a seamless experience this is.

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.