Another scary finicky experience that started happening the last hour, where everything seems perfectly fine with our postgres app but none of our other apps will connect.
The only guess is credentials, as I had to hack the DATABASE_URL to include a ?schema=public since no one has helped with this issue: Expanding on ENV variables
I had a ton of issues attaching and detaching so i had to create a new database, which was already a bad experience to begin with, so then I just created a single DATABASE_URL from one app and just applied that same URL to all apps, so not sure what the attach magic does, but if it creates new pg auth username/passwords and “rotates” them, then maybe that is what happen during a deploy to an app server?
Either way, this whole experience with internal network connections and getting apps to be able to talk to each other has been very finicky.
The types of issues like this one could potentially take our entire system down, which can not happen as we are in the ecommerce space where our client’s client’s are trying to checkout 24/7 on our platform.
fly pg attach and
fly pg detach simply almost never work, I get “Error An unknown error occured” about 95% of the time I try to use it.
So now I am in the worst possible circumstances, every app (10-15 apps) are down because PG is not accessible to any of them, but there is no issues with the pg instance. We even restarted the instance, nothing. I also have no way of knowing what the issue is, nor can I fix the issue. Luckily we are just getting all the production stuff deployed this week so we don’t currently have any traffic on these servers, but if this happens come Monday, we will have some VERY BAD problems, what happens when this happens at 2am? That just simply can not happen.