DNS Resolver Issues last night (SEA)

1

Last night we experienced downtime on our apps due to some internal errors that seemed out of my control to be able to fix - and they ultimately did resolve on their own by the morning. Fortunately, we are still in beta so the downtime wasn’t a fatal issue, however this will no longer be the case in a few weeks.

I use Upstash for Redis and CrunchyBridge for Postgres. I could not make new connections to either due to an apparent DNS issue:

Error -3 connecting to http://XXX.upstash.io:6379. Temporary failure in name resolution.

OperationalError: could not translate host name "(http://XXX.db.postgresbridge.com/)" to address: Temporary failure in name resolution

I am curious if anyone else is seeing these issues, if there was a known issue, and what might possible mitigation measures be in the future? I am currently deployed in a single region (sea).

2

My first thought was that the DNS issues might have been at the Upstash end, but if this affected another provider, maybe not.

I agree that DNS is a service that needs to be reliable, but for the purposes of getting you out of a pickle in the future, I wonder if you could connect to a named endpoint, and fallback to a IPv4/6 endpoint in the case of DNS failure?

4

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.