Upstash redis times out after being idle

Questions / Help
1

I am running a go app on Fly. Whenever my machine tries to connect to upstash redis after being idle, the connection times out and throws an error. However, subsequent requests (refreshing the page) works. This is the error:

dial tcp: lookup x.upstash.io: i/o timeout

Here my connection code:

opt, err := redis.ParseURL("rediss://default:x@x.upstash.io:6379?dial_timeout=5&read_timeout=6s&max_retries=2")
tlsConfig := &tls.Config{
    InsecureSkipVerify: true,
}
opt.TLSConfig = tlsConfig

When I omit the InsecureSkipVerify option then the connection always fails.

tls: failed to verify certificate: x509: certificate signed by unknown authority

This is the line that fails:

r.SIsMember(context.TODO(), "x", y).Result()

Am I doing something wrong?

2

Hey there! You should be using just redis:// as Upstash doesn’t run with TLS inside Fly.io (all traffic is already encrypted). And you can skip the TLS config.

Were you seeing timeouts using redis:// instead of rediss://? Also, in which region are running your app and Redis?

3

It is a “regular” upstash redis account, not one that was generated via the fly upstash partnership.

The region is us-east-1 (I don’t seem to have an option to change this to a fly region.)

So you are saying that I need to use TLS when developing locally, but use non-tls when deploying to fly?

4

Sorry, I misunderstood you. You’ll have a much better experience using the Fly-based Upstash Redis. Are you able to switch to it?

If not, you’re doing the right thing. However, you are crossing network boundaries. If your app is hosted in iad, you should see decent performance. From elsewhere, there are fewer guarantees since there is physical distance between your app and Redis.

5

Got it - I did not know there was a difference between the two types of upstash! I’ll take a look.

6

The database you get on Upstash.com is hosted on AWS. fly redis create will get you a database securely hosted right next to your app.

I’m curious - where would have been a good place for you to discover this difference? The Upstash docs, our docs?

7

I also think it might be a go-redis issue, because I see the same problem when using non-upstash redis. I just can’t figure out the right golang config to automatically reconnect and retry when there is an idle timeout.

8

Good question. It is confusing. At some point I created a fly Upstash instance, months ago. However, when I go back to the Upstash website and log in, it doesn’t show me any of my old Fly DBs. I honestly have no idea how to access them. I’ve tried logging in via Google and GitHub but I still just get “regular” Upstash accounts.

So that told me that the best way to use Upstash is via a regular account rather than a fly integration. Because the fly integration was less an integration with an existing account, but actually a whole new thing. Like, imagine you have a PayPal account vs a “PayPal account that only works with Shopify”. Its confusing.

In terms of the ideal scenario:

  1. Regular Upstash should not break just because it is running on fly and you have different routing.

  2. There should be exactly 1 way to connect and exactly 1 connection string

  3. In Upstash, there is a button to click to choose where to deploy: AWS vs gcp. There should just be another option for Fly on that screen. Whatever connection string it spits out should work both on localhost and Fly.

9

Indeed it’s a bit confusing. Our Upstash integration works via flyctl. To access your Fly Upstash DB dashboard fly redis dashboard your-fly-org.

Can you let us know which Fly.io region you’re connecting to you Upstash database from? That would help here.

10

Got it - I did not know that’s how i needed to access! I’m running in ewr.

11

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.