Greetings! I have a Phoenix app I’ve had running on k8s without issue for a long time. I’m trying to move it to Fly, but am struggling to get a Fly.io Redis instance connected.
The app is running, and after enabling Telemetry for Redix, I get:
Connection :flyway_parser_aixm_redix failed to connect to Redis at fly-app-keydb.internal:6379: non-existing domain
I’ve added
[experimental]
private_network=true
to the app’s toml after finding some hints searching around, but this hasn’t helped.
I’ve set the REDIS_URL as a env var via the CLI, and I have the app printing it during starting the supervisor. The URL is correct. I can install redis-tools and connect to the redis instance from the command line in the ssh console using the same URI, ie.
> redis-cli -u $REDIS_URL
This works without issue, however, from within the running app, the host is not found. I also tried directly connecting to the IP in the url, and this times out. When trying the redis-cli connection to IP, it works, but it is slow.
I’ve been banging my head on this for a few days now. Any suggestions on where to look next?
… is no longer needed. That’s confirmed by Fly e.g
Hence that makes no difference. So yep, you can remove that.
Next, is the app in the same organization as the Redis app? They need to be in order to be able to resolve each other’s internal hostnames.
Next, if you are using Fly’s Redis, based on their guide, I recall that needs a password … Are you providing one from your app? Else that would explain why it can’t connect if not.
Next, it may be worth trying an alternative .internal hostname: Private Networking · Fly Docs Like … what if you swap it for top1.nearest.of.<appname>.internal? Does that work? No real reason why it should but doesn’t hurt to try.
If still no luck, make sure your Redis app binds to IPv6. Again, depends on how it’s set up, but Fly’s private network uses IPv6. So if the Redis app is only listening on IPv4, other apps won’t be able to connect to it using the private network. Look for any references to it binding to 0.0.0.0 or something IPv4-ish.
I tried using the exact docs for setting up Redis on their docs, and have set a password. I am able to connect to it via the ssh console using redis-cli so I’m sure that part is working correctly. Yes, they are in the same organization. I only have the one.
Would the binding affect DNS resolution within Elixir? I’ll give both (dns/binding) suggestions a shot!
Created a dummy app, dummy redis instance using the docs instructions. Enabled Redix (the Elixir library) telemetry and same result:
Connection :flyway_redix failed to connect to Redis at my-flyway-redis-test.internal:6379: non-existing domain
¯_(ツ)_/¯
Again, I can hit the same host via the ssh console in both telnet and redis-cli.
Within iex, I tried :inet_res.nslookup('my-flyway-redis-test.internal', :in, :txt) and received {:error, :nxdomain}… I feel like both my actual app and this dummy app aren’t respecting Fly’s private DNS or something. Tried also with “google.com” and worked fine.
