We just recently pulled the trigger on moving our full backend over to Fly, and we’re seeing some frustrating Upstash performance that we’re trying to pin down. Both when using Upstash through Fly (SJC) and connecting to an external (paid) Upstash account, I’m seeing brutal latency and timeouts. The timeouts are less frequent but the latency is absolutely killing our performance and end-user experience.
It is now much faster for us to skip Redis and make external calls to MongoDB Atlas (hosted at Mongo) than it is to use Redis, which was the reason we implemented redis caching in the first place. Our average Upstash latency seems to now be +500ms and in some cases much higher.
We’re using Node w/node-redis. Our connection params does include a pingInterval (4 * 60 * 1000) which we’ve needed in the past. Our services establish a long-running connection.
Anyone have any thoughts/insight here, this is definitely dampening the otherwise awesome full transition to fly for us.
The lack of any meaningful response or even follow-up from support is extremely disappointing. We reached out Tuesday night, received a confusing follow-up question from support Wednesday morning (which was great turnaround), we responded a few hours later, and have since heard nothing. We followed up with an email Thursday, just asking for any kind of progress update, and have still heard nothing back.
We’ve now had this issue in our production environment for nearly a week. Even a simple follow-up to confirm something is happening on Fly’s end, or if we need to do more investigation on our end as you guys don’t think its your issue - ANYTHING - would be a hell of a lot better than whatever this current experience you’re putting us through.
Hey there, sorry this slipped through the cracks. We had a lot of support tickets last week. I’ve followed up personally via support and will be following up this week as needed.
I’ve done a bit of testing in SJC where your Redis is hosted, and haven’t been able to reproduce those numbers, so will need to look deeper.
I myself am also trying to use redis but on a self hosted vm on fly io and i can’t manage to enqueue something through bull with it, my backend app seems to connect to my redis app through a private ip6 network but when actually trying to put something in redis, i get a max attemp failed 20 error. I’m currently getting helped by a kind individual on discord on this thread: Discord
