I’ve got an app which has two process groups:
web serves API requests.worker recieves background jobs from SQS and processes them.External requests all work just fine, requests are routed to a web process.
However, internal requests (e.g. http://top3.nearest.of.my-service.internal/ ) – seem to be routed to any process, causing requests to fail if they get sent to a non-web process.
How do I define that only web processes listen on a port and should serve requests?
This is not currently possible with internal DNS. There are two possible workarounds:
fly ips allocate-v6 --private
Okay, if I allocate a load balancer IP how do I pin it so it’s only the web processes which are routed?
From the Flycast Private Load Balancing documentation, it sounds like traffic should be automatically routed to the web processes, the same way it works for public traffic?
This traffic is routed through the Fly proxy - like public traffic - arriving in the geographically closest region with active VMs.
It isn’t clear to me from the documentation whether there are other differences between using a private Flycast IP instead of .internal addresses. We have a similar use case with web and worker processes for one app, and a second app that needs to connect via private networking.
Flycast will actually send your traffic through the proxy, where using .internal will using DNS to address your app instances directly. Flycast could work for your use case. This is actually how our managed Redis offering works.
It may not be clear from the docs, but Flycast will use the same [services] block from your fly.toml as the public proxy does. You can control which process groups respond by placing each one on a different internal port.
One caveat. If your Flycast target app has a public IPv4 address, any service you setup for use with Flycast will also be available to the public internet. We don’t have a workaround for this yet besides creating two separate apps.
@jsierles Thanks! It’s great to know that we’re looking in the right direction.
One problem we ran into when switching to the private IPv6 Flycast address is that we see an error in the proxy logs.
Error: no host specified in headers or uri
(The Flycast target app is using force_https.)
We resolved this by configuring a Host: <app-name>.fly.dev header in our HTTP client, but I’m wondering if there’s a more convenient way to configure this?
One caveat. If your Flycast target app has a public IPv4 address, any service you setup for use with Flycast will also be available to the public internet. We don’t have a workaround for this yet besides creating two separate apps.
Does this only apply to ports configured in [[services.ports]] blocks? Here’s a heavily edited snippet of our Flycast target app, for more context. If I’m understanding correctly, 80 will redirect, 443 will be passed through to port 8080 on the web process, and 8081 will not be public.
[processes]
web = "..."
worker = "..."
[[services]]
internal_port = 8080
processes = ["web"]
protocol = "tcp"
[[services.ports]]
force_https = true
handlers = ["http"]
port = 80
[[services.ports]]
handlers = ["tls", "http"]
port = 443
[[services.http_checks]]
(... snip ...)
[[services]]
internal_port = 8081
processes = ["worker"]
protocol = "tcp"
[[services.http_checks]]
(... snip ...)
Setting the Host header is probably the simplest approach for now - what else did you have in mind?
Flycast uses the exact same proxy rules as our public proxy. So your service on port 8081 will not be available over the Flycast IP address. You’d have to add a [[services.ports]] section for it. Right now I don’t think there’s a good workaround besides running a totally separate app to handle internal Flycast requests.