Postgres cannot be connect: "Error opening connection to database"

Questions / Help
1

Hi everyone,
The Postgres I deployed on Fly is consistently giving the following error today, and fly’s internal apps are not connecting properly:

2022-09-21T12:43:44.607 app[79135861] sjc [info] sentinel | 2022-09-21T12:43:44.607Z ERROR cmd/sentinel.go:1009 no eligible masters

2022-09-21T12:43:45.195 app[79135861] sjc [info] exporter | INFO[2365] Established new database connection to "fdaa:0:737f:a7b:ad0:1:f80d:2:5433". source="postgres_exporter.go:970"

2022-09-21T12:43:45.195 app[79135861] sjc [info] exporter | ERRO[2365] Error opening connection to database (postgresql://flypgadmin:PASSWORD_REMOVED@[fdaa:0:737f:a7b:ad0:1:f80d:2]:5433/postgres?sslmode=disable): dial tcp [fdaa:0:737f:a7b:ad0:1:f80d:2]:5433: connect: connection refused source="postgres_exporter.go:1658"

2022-09-21T12:43:46.195 app[79135861] sjc [info] exporter | ERRO[2366] Error opening connection to database (postgresql://flypgadmin:PASSWORD_REMOVED@[fdaa:0:737f:a7b:ad0:1:f80d:2]:5433/postgres?sslmode=disable): dial tcp [fdaa:0:737f:a7b:ad0:1:f80d:2]:5433: connect: connection refused source="postgres_exporter.go:1658"

2022-09-21T12:43:50.375 app[79135861] sjc [info] sentinel | 2022-09-21T12:43:50.374Z WARN cmd/sentinel.go:276 no keeper info available {"db": "2e3aca53", "keeper": "22951f80f2"}

2022-09-21T12:43:50.376 app[79135861] sjc [info] sentinel | 2022-09-21T12:43:50.376Z ERROR cmd/sentinel.go:1009 no eligible masters

I did the following but it did not resume running:

  • scale count 0 and scale count 1

  • Re-deploy Postgres project

  • Follow this thread for troubleshooting

  • Use fly proxy and try to connect locally, failed

=== Keepers ===

UID		HEALTHY	PG LISTENADDRESS			PG HEALTHY	PG WANTEDGENERATION	PG CURRENTGENERATION
22951f80f2	false	fdaa:0:737f:a7b:2295:1:f80f:2:5433	false		4			3
ad01f80d2	true	fdaa:0:737f:a7b:ad0:1:f80d:2:5433	false	1	0

The database is set to dedicated-cpu-1x 2gb. A total of 40GB of disk abd was used 2.5GB.

Now use fly status to view the status still visible service is in state running and has 1 instance ( running (failed to co)). Now what should I do to get Postgres back up and running?

UPDATE: Running again

I found that Fly Postgres does not work correctly at 1 node, I scaled to 0 and then to 2 and the database started running again.
The confusion is,

  • I didn’t change any configuration before the issue occurred.
  • I’ve had some other Postgres running on Fly for a while now as well, and they all seem to be 1 node and running well.

Please let me know if there is anything missing.