I’m running a Ruby on Rails app with Fly and having a major outage.
I’m having some wild redis problems.
Upstash changed their free plan so I no longer qualify. I hit my limit and my app went down.
Unfortunately the account was setup through Fly.io and I don’t seem to be able to upgrade from there. So I created a new account, made it paid, and then tried to update the REDIS_URL secret via the commandline.
Now my apps just partially broken. I can visit the homepage on my laptop but not my phone. When I try to reset my password I get a Redis connection issue.
This is what I ran to update the secret
fly secrets set REDIS_URL=redis://default:****@us1-diverse-banana-41351.upstash.io:41351
It’s strange that the app is available on your laptop, but not the phone. That kind of suggests some kind of cache somewhere along the line (the laptop perhaps having an older/newer versison?)
The command to set the secret looks correct to me. What I’d do to debug at least that part is SSH in to the app/VM to see the value that the app is seeing (to double-check it is indeed what you are setting). Fly stores secrets encrypted (as you’d hope) and they are only decrypted when injected into the vm when it boots. However you can access it since of course you have access to your own vms. Run fly ssh console from the folder your fly.toml is in. That should load you into the vm, and then run echo $REDIS_URL to see what it is. You should see the same value you set. Assuming you do, that’s Fly’s secrets working. So the issue can’t be that.
If it’s still showing the old value, the app may not have deployed (to trigger the new secret to be read-in). Setting a secret should trigger a new deploy so it shouldn’t be that.
I don’t have much experience with Ruby but I would assume it throws/shows some kind of error exception. Perhaps in fly logs? Like … if it says it can’t connect, or it timed out, or something to indicate what the issue is.
