Redis timeouts

1

Hi, I randomly have issues with connecting to redis on my rails app. Redis is used mostly for sidekiq jobs.

I don’t have any visibility in the redis app in fly.io dashboard (it doesn’t show up), so I am not sure what I need to do to investigate.

I migrated my 2+ year old app from Heroku a few weeks ago (where I almost never had redis issues, and even then it was typically my fault).

Any ideas on how to debug this? Is there a way I can see my redis memory usage?

2

Hi! In this post a user proposes using the internal direction instead of the public direction to avoid disconnections. Are you using the public host?

EDIT: I’ve deployed a standalone instance of this image GitHub - fly-apps/redis: Launch a Redis server on Fly and it works better than the default managed redis

3

I am using the one on *.upstash.io

flyctl redis status floral-flower-6924
Redis
  ID             = L5q...
  Name           = floral-...
  Plan           = Free
  Primary Region = sea
  Read Regions   =
  Private URL    = redis://default:password@fly-floral-....upstash.io

How do I find the internal direction?
If I use the “standalone instance”, I’d have to pay for that?

4

I’ve not been successful in finding the internal address of the provisioned free Redis, so I ended up deploying a standalone instance. I’m not paying for it since it’s using of the three free slots, but your mileage may vary

5

Upstash Redis is a beta product, and when I tried it a few weeks ago it was practically unusable. I would advise as well to create a regular Fly app if you want a stable Redis server.

6

Hi @kevin,

Managed Redis currently has a relatively short idle timeout that will result in periodic disconnects from applications that are mostly idle. You will need to configure your Redis client to automatically reconnect, for example using the redis-rb Ruby client, by setting reconnect_attempts > 0 and not using disable_reconnection. Our Rails guide has a patch for use with Action Cable.

As noted in the documentation, the managed Redis instance address is a private IPv6 address. There is no public address. It’s listed as Private URL in the flyctl redis status output.
The older thread you referenced was discussing a standalone app deployment, not the managed product released more recently.

The product was only just launched a few weeks ago, and we’ve been working hard on stability and bug-fixing improvements since the initial rollout.

7

Yes I’m aware. I will give it another shot in 6 months or so :slight_smile:

1 Like
8

I tried self-hosting redis of flyio, increasing reconnect_attempts to 5, and I still get 5-20 redis timeout emails per week. Specifically, in the last week, the connectivity is much worse.

I am going to try to create a fresh instance v2 instance and see if that will fix things.

v1 toml file

# fly.toml file generated for withheld on 2022-09-18T14:49:17-07:00

app = "withheld"

[metrics]
  port = 9091
  path = "/metrics"

# Use the Docker image instead of building from source

[build]
  image = "flyio/redis:6.2"

[[services]]
  internal_port = 6379
  protocol      = "tcp"