[rejected] Feature request: Bare TLS on shared IPv4 (for non-HTTP protocols)

1

(reposting from my comment here)

Please allow apps with a shared IPv4 address to have just a TLS handler on port 443, instead of requiring TLS+HTTP. This is useful because it allows these apps to support non-HTTP protocols such as gRPC by wrapping them in TLS.

(Currently the only way to create a non-HTTP service is to use a dedicated IPv4 address or IPv6; but dedicated IPv4 addresses will cost $2/month, and IPv6 is not supported everywhere.)

I don’t see any technical reason to require TLS+HTTP, because presumably Fly is using SNI to identify the hostname. In fact I think shared IPv4 did work with just TLS at first, until extra checks were added.

Steps to check if bare TLS is supported:

$ mkdir app && cd app
$ fly launch --image flyio/hellofly:latest # accept defaults
# change the handlers of external port 443 from TLS+HTTP to TLS
$ patch fly.toml <<"EOF"
--- fly.toml.orig
+++ fly.toml
@@ -32,3 +32,3 @@
   [[services.ports]]
-    handlers = ["tls", "http"]
+    handlers = ["tls"]
     port = 443
EOF
$ fly deploy
==> Verifying app config


Configuration errors in .../app/fly.toml:

    ✘ base: Services defined at indexes: 0 require a dedicated IP address. You currently have no dedicated IPs allocated. Please allocate at least one dedicated IP before deploying (`fly ips allocate-v4` and/or `fly ips allocate-v6`). Affected services:
  [0] tcp/80,443 => 8080

Error App configuration is not valid

Desired behaviour: Deploy should succeed, a shared IPv4 should be automatically allocated, and the app should be accessible on port 443. (This particular demo app happens to use HTTP, but any protocol should be supported as long as it is wrapped in TLS.)

(This post is distinct from Feature Request: SNI based TLS routing with shared IPv4, which is asking to provision own certificates with shared IPv4.)

Cc: @jerome

3 Likes
2

I’ve (hopefully, temporarily) stopped deploying a gRPC service to Fly for this reason.

Admittedly because the $2/month fee required me to rework my deployment but more because this feels like something of a grab; exclusively renting a static IP1 which I don’t need nor want.

I have a strongly positive opinion of Fly (technology and people) and I’d happily spend money on Fly services but this change (something that worked, now doesn’t and incurs $$$) soured me.

1 – I know IP(v4) addresses represent a significant capital investment.

3

Shared IPs will only ever support services with an HTTP handler. This is much simpler for us to support. We’re going to keep free services as simple as we can. And we’re going to keep paid services as cheap as possible.

I don’t think there’s another cloud you that has a managed TCP load balancer for the $2/mo it takes to add an IP.

2 Likes
4

Hi Kurt, thanks for the reply. Are you able to share any insight into why TLS-without-HTTP is less simple to support, especially given my (fuzzy) recollection that it worked out-of-the-box when shared IPv4s were first launched?

5

Not an eng at Fly, but I’d imagine, with SNI-based routing on TLS v1.2 things are simpler still, but ECH (encrypted client hello) that’s now part of the TLS v1.3 standard, makes this setup a bit more involved: (from what I know, to support ECH) Fly must also host the DNS nameserver for the host being identified by the now-encrypted SNI field: Terminate multiple TLS domains, with both Fly and custom certificates - #2 by ignoramous

Also, I am not quite sure if various ALPNs complicate things for SNI-driven routes.

QUIC support is another area of concern for reverse-proxies. So you may argue that things aren’t getting any simpler for Fly, anyways…

1 Like
6

Those sound like reasons why TLS is difficult on shared IPs in general, not reasons why TLS is more difficult than TLS+HTTP. (Or did I misunderstand your comment?)

My point is that SNI (and ECH) are required anyway in order to support TLS+HTTP on shared IPs, so it’s unclear to me why dropping the HTTP handler might be difficult. (ALPN is interesting, but I assume it can simply be disabled/ignored for apps that want bare TLS.)

A consequence of requiring an HTTP handler on shared IPs is that it won’t be possible to create non-HTTP apps that cost $0 when they are idle, because they’ll generally require a dedicated IPv4 which can’t be “scaled to zero”.

2 Likes
7

The answer is probably “it’s a feature that’d have to be added to fly-proxy, supported, and maintained”.