Unable to reach postgres instance

Anyone else having similar issues? I have made no changes ot my app but suddenly i cant connect to postgres from the app.

PG::ConnectionBad: connection to server at “fdaa:3:711f:0:1::8”, port 5432 failed: server closed the connection unexpectedly (PG::ConnectionBad)
This probably means the server terminated abnormally
before or while processing the request.

I’ve never seen this before.

I’ve tried: rebooting instances, destroying machines and re-creating them, deploying the app.

Nothing has changed.

I am seeing the same thing, both my apps are having issues connecting to the respective Postgres DBs i.e. I am seeing Postgrex.Protocol (#PID<0.2231.0>) failed to connect: ** (DBConnection.ConnectionError) tcp recv (idle): closed

Seems like one of them will connect off and on. Maybe a fly thing?

I’m not sure if i fixed it or if it fixed itself. I did the following:

  • updated my postgres to the latest version
  • destroyed all the existing machines (app - not postgres)
  • scaled the app back up

Then it came online. I did the destroy and and scale up before, but it didnt seem to work…not s sure why it did this time.

Thanks, I’ll give that a try, I’m just running some hobby apps at the moment, so at least not a panic haha.

Wish i could say the same…ugh nothing worse than customers emailing you about issues you didn’t pickup on.

2 Likes

Same issue here I think, my app suddenly cannot find my Postgres instance when it was working fine before. I tried destroying my app and scaling back up but it fails to connect everytime.

Added postgres

Can you reach the db through a wireshark tunnel? I could the entire time which is the only reason i knew the db was fine. I’d check that to rule out any other issues.

Hi, same here across 4 apps, they all started throwing the same error. It seems it’s only affecting apps in my personal organization, I have two apps in outside organizations and they’re fine.

Yep db looks fine internally, can reach it through a Fly Proxy and it responds to queries.

1 Like

same here!

Hm… Another thing that could be tried is switching to the .internal address instead of the .flycast one.

(In your web app’s DATABASE_URL—or similar.)

This will no longer auto-start the database or load balance, but it might at least help narrow down the source of the problem…

I was suffering from the same issue, the way i resolved it was i rollbacked my postgres instance to a snapshot before the latest snapshot. The new instance from the snapshot didn’t suffer from the same issue. I’m going to assume the daily snapshot was the issue ?

From my understanding a daily snapshot shouldn’t change anything on the live database.

But it is a probable cause.

Any additional insight here?

Waiting for a reply from the Fly.IO team here.

Should I wait for you guys to actually resolve the issue, or should I try @sonalranjit’s fix?

Seems the issue is on your end.

I can confirm this solution worked - I created a new secret “DATABASE_URL_ALT” which replaces “.flycast” with “.internal” and updated my app to use this new var

I am running pretty old versions of flyio/postgres-flex (0.0.39) and haven’t touched the databases forever. Fortunately one instance of my app remained able to connect to the .flycast url, but no other instances or new machines could

1 Like

Added proxy

Same for me, all my apps stopped working…

My apps conenct to database via ip address and I have similar issue

Yesterday my postgres instance went down the same way, I reprovisioned new with the latest image and did a failover, it seemed to fix the issue.

Now my app can connect to both Redis upstash instances … And as I’m writing this they just added https://status.flyio.net/incidents/ftd07gnytjl4 so it may be networking related :-/