Public web service + private workers setup

Questions / Help
1

I’ve read about running multiple services inside a single Fly.io app but I’m still confused…

I want to achieve the following:

  • A public web service (server) that takes requests from the internet, performs some work itself, offloads some work to a private “worker” service and returns a response.
  • A private web service (worker) that takes requests from server.
  • Both services are simple Hono web servers using the same custom Dockerfile.
  • Services should be able to scale independently.
  • Both server and worker will perform heavy computing tasks, so I don’t think running both type of services in a single Machine would work well. So, one service per Machine.
  • All machines should be able to auto suspend/stop when no requests are coming in. Machines in server should be started automatically as soon as requests come in. Same with machines in worker when server makes a request to the Flycast endpoint.
  • Usually, one request to server will make 4 concurrent requests to worker and this should start 4 machines, each one taking only one requests at a time.

Can I make all this into a single fly.toml?
Or do I need 2 different directories for each service with its own fly.toml?
If so, is there a recommended monorepo setup for this to avoid duplicating files as their codebase is 90% shared.

2

You can read about it here: Multiple processes inside a Fly.io app · Fly Docs
A single fly.toml and Dockerfile but each process will be its own independent machine.

The orchestration of 4 workers doing 1 task at a time requires a little more work but should be pretty simple.

3

It looks like Process groups is what I’m after right? as this runs each thing in its own Machine: Run multiple process groups in an app · Fly Docs

4

This is what I have so far…

# fly.toml
app = "my-app"
primary_region = "iad"

[http_service]
auto_start_machines = true
auto_stop_machines = false
force_https = true
internal_port = 3_000
min_machines_running = 0
processes = [ "server" ]

  [http_service.concurrency]
  hard_limit = 1
  soft_limit = 1
  type = "requests"

[processes]
server = "bun run src/server.ts"
worker = "bun run src/worker.ts"

[[services]]
auto_start_machines = true
auto_stop_machines = false
internal_port = 3_001
min_machines_running = 0
processes = [ "worker" ]

  [[services.ports]]
  force_https = true
  handlers = [ "http" ]
  port = 80

[[vm]]
size = "shared-cpu-2x"

My Dockerfile ends with

EXPOSE 3000/tcp
EXPOSE 3001/tcp

Do I need both?

Now, how do I set/get the Flycast address for the worker service group so that main can make requests to it?

5

Yes that looks good. The flycast address is in this format: <app-name>.flycast:<port>
So you’d make a request to http://<app-name>.flycast:3001 from your web app.

Don’t forget to delete any public IPs for your worker machine and add a private ip6 address to it.

6

Ok, so are you saying that all services are public by default? I’m confused. What is this section doing?

  [[services.ports]]
  force_https = true
  handlers = [ "http" ]
  port = 80

Because right now, making a request to the app public domain name (https://app-name.fly.dev) is correctly being routed to a server/main process.

If my server/main process will be calling the workers via internal/private flycast over the 3001 port, I don’t need to expose the workers to port 80 or anything, right???

Regarding deleting public IPs from my worker machines and add private ipv6 to them… I thought the workers didn’t get a public IP address. Anway, is it not possible to define that in the fly.toml?
Everytime I create new worker machines, I’ll need to manually remove the public IP address and add a private ipv6 to them?

7

if you do fly ips list -a my-app it should list the ips. I usually remove all public ips just in case.
You’ll need to remove force_https = true for your worker since internal networking is done w/o TLS.

Yes you might be correct, try removing the EXPOSE for the worker port.

8

Uhm, that doesn’t seem to be working. I’m following this guide:

So I’ve just launched a new ubuntu shell inside my Fly.io network and I can ping my app’s flycast address and it resolves to the private ipv6 I’ve created for my app. However, trying to do a CURL request to http://my-app.flycast:3001 doesn’t work. Returns “Connection reset by peer” and I can see nothing’s happening in the logs. No worker machine is being started.

All the documentation I can find around Flycast and private apps assumes one has a single service per app. Are you sure I should just be able to reach my worker’s machines using the flycast address over port 3001???

9

Ok, I got it working!

  1. EXPOSE 3001/tcp in the Dockerfile
  2. Map another port (8080) to the internal process port
# Private Worker
[[services]]
  processes = ["worker"]
  internal_port = 3001
  auto_stop_machines = false
  auto_start_machines = true
  min_machines_running = 0

  [[services.ports]]
    handlers = ["http"]
    port = 8080
  1. Use http://{app-name}.flycast:8080 to reach workers from within your Fly.io network.

I believe this should all be better documented. I know there’s a lot of documentation, but I don’t know, I feel it’s all spread all over the place. We need more working examples. This public web server + private worker is a very common practice, should be a template easy to find.

10

Nice! Sorry I meant removing the EXPOSE 3001 in your dockerfile, not the ports in your fly.toml

I agree the documentation can be lacking in some places, but the silver lining is that it forces you to dig deep into how Fly works… a skill you’ll need to make use of all the neat features.

11

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.