I have deployed a change that should tell you if you can’t deploy when using shared IPs.
This hopefully will prevent a bunch of confusion with shared IPs.
1 Like
The UI should now properly display shared IPv4s.
1 Like
I’ve been deploying an app successfully for some months that I suspect is broken by this change and I’m unsure why it’s broken and how to fix it.
kill_signal = "SIGINT"
kill_timeout = 5
[build]
image = "registry.fly.io/healthcheck-server:[[edited]]"
[experimental]
exec = [
"/server",
"--failure_rate=0.5",
"--changes_rate=15s",
"--endpoint=:50051",
"--services=,grpc.health.v1.Health"
]
private_network = true
[[services]]
internal_port = 50051
protocol = "tcp"
[services.concurrency]
hard_limit = 25
soft_limit = 20
[[services.ports]]
handlers = ["tls"]
tls_options = {"alpn" = ["h2"]}
port = "443"
#[metrics]
# port = 50051
# path = "/metrics"
On deployment (using docker.io/flyio/flyctl:v0.0.442):
==> Verifying app config
Configuration errors in /fly.toml:
✘ base: Services defined at indexes: 0 require a dedicated IP address. You currently have no dedicated IPs allocated. Please allocate at least one dedicated IP before deploying (`fly ips allocate-v4` and/or `fly ips allocate-v6`). Affected services:
[0] tcp/443 => 50051
Error App configuration is not valid
It meets one (!?) of the requirements:
- HTTP on port 80
- TLS + HTTP on port 443
If I understand correctly, I should allocate an IPv4:
podman run \
[[edited]] \
docker.io/flyio/flyctl:v0.0.442 \
ips allocate-v4 --shared --app=healthcheck-server
And this succeeds (and IPv4 is assigned) but the app remains “pending”.
Questions:
- Why isn’t my app config conformant?
- Should I
ips allocate-v4 --sharedbefore deploying?
Thanks!
Shared IPv4s don’t work with just TLS on port 443. It needs to be both TLS+HTTP.
This would work:
[[services]]
internal_port = 50051
protocol = "tcp"
[services.concurrency]
hard_limit = 25
soft_limit = 20
[[services.ports]]
handlers = ["tls", "http"]
tls_options = {"alpn" = ["h2"]}
port = "443"
It sounds like you need to do HTTP yourself? If that’s the case, you’ll need to have only dedicated IPs assigned.
Thanks @jerome
Yes, it’s a gRPC app and it doesn’t work with Fly’s http handler.
I think shared IPv4 did work with just TLS earlier. Is that possible? I assumed Fly was using SNI to identify the hostname so there was no technical reason for an HTTP handler. It would be really neat if it worked, because it would allow non-HTTP protocols (e.g. gRPC) to be used with shared IPv4s by wrapping them in TLS.
Thanks. Some feedback: The error message when running fly ips allocate-v4 --shared on an app with non-standard ports doesn’t make sense:
Services defined at indexes: 0 require a dedicated IP address. You currently have a shared IPv4 assigned to your app (
fly ips list). Release the shared IP (fly ips release <shared ip>) and/or allocate only dedicated IPs before deploying (fly ips allocate-v4and/orfly ips allocate-v6). Affected services:
[0] tcp/80,443,5000 => 8080
This message would make sense in the context of fly deploy, but in the context of ips allocate-v4 --shared it is plain wrong: the app doesn’t currently have a shared IPv4. I’m not sure what a better message would be. Maybe suggest to the user to allocate a dedicated IP by dropping the --shared option. It would also help if the error message explained how to remove the non-standard port (editing fly.toml wasn’t enough, I also had to run fly deploy; not sure if there is a simpler way).
1 Like
I allocated a dedicated ipv4 and then released it. I now have none. Is there a way go back to using a shared one?
1 Like
Yes… you probably missed this:
Yep, I just figured that out by guessing at the --shared flag and came back to share (no pun intended). Sorry for not reading more carefully—I’m having a very frustrating morning. And thank you for the response.
2 Likes
Have you already started charging dedicated IP? I didn’t know until I saw this post. Will I be charged?
1 Like
We haven’t started charging for them and we’ll give plenty of warning before this happens.
Maybe this is a dumb question, but how can I allocate a cert and a domain name to a shared ipv4 address? Is there going to be a CNAME setup or similar? I don’t need a dedicated one for the testing environment, but Stripe doesn’t seem to support ipv6 only domains yet
Everything works the same for domains / certificates w/ shared IPv4. Best to assign a CNAME if you’re using a subdomain, and a A record is you’re using an apex domain.
OK I added a CNAME to appNAME.fly.dev and that seems to work now. Perhaps the UI needs to be updated? By default on new apps it’ll only prompt you to create an AAAA record, regardless of whether it’s a apex domain or subdomain. Before this change, I used to create both a A and AAAA record and everything worked fine.
Just to be 100% certain…
A shared IPv4 will not change, right?
I can still point an A DNS record to it?
I’m migrating an app from v1 to v2 and wouldn’t want to mess that up.
Correct, the shred IPv4 for an app won’t change over time. In the unlikely event that we’d need to change it, we’d give a lot of warning and would be able to verify that the record has changed.
How are you migrating? If you’re creating a new app then it will get a different shared IPv4.
2 Likes
Yes, I’ve created a new app hence why I need to update the DNS records 
Edit:
BTW after adding the domain I had to manually create the cert.
I seem to remember this was automatic? 
It seems that you just now started charging for IPv4 adresses for old deployments, but forgot the part about notifying people ahead of time… A forum post, posted less than 90 days before the change, cannot serve this purpose!
1 Like
Can I confirm that the dedicated IPs can also be used for outbound allow listing? Or is it just for incoming traffic?
Unfortunately we don’t support static IPs for outbound traffic yet. It is something we’re working on though, we just can’t make any promises on when it’ll be ready. In the mean time you can use 3rd parties for this like Fixie