Single VM vs. process groups vs. separate apps: current best practice for multiple processes?

Questions / Help
1

There are various threads discussing this, but I have no way of knowing how up-to-date the information is—the current mix of v1 and v2 information in the docs and community threads makes this difficult. So I’m asking again to see what the current answer is.

Suppose I have:

A) A caching reverse proxy
B) A web application server
C) A web service that handles long-running tasks

A proxies to B, and B makes calls to C. Only A should be publicly accessible.

As I understand it, I can:

  1. Run all three processes on a single VM.

  2. Create a “process group” that runs each process on a separate VM within the same app. (It is not clear to me how VMs within a process group communicate with one another. Using fly’s internal DNS?)

  3. Create three separate apps within a single organization, and use fly’s internal DNS to communicate.

As of September 2023, which of these three options is the recommended method?

I have experimented with option #3, and found that internal DNS is flaky when machines are suspended and restarted (often DNS lookups of .internal addresses fail).

I don’t understand how option #2 significantly differs from option #3.

Option #1 is least attractive, because I need to construct Frankenstein images with all the packages and libraries needed for all three processes; I can’t easily restart just the proxy, etc.

My main concern is choosing the option that fits with the Fly platform’s expected future evolution, so that I don’t have to revisit all this in 6 months.

2

I have been playing around with #2, and so far don’t have a working app. But may be unrelated.

A few things about option 2, it shares all ENV variables.

But, as far as I can see, there is no ENV to detect the process group documented. The meta endpoint seems to return the process group, but not sure how you access this too use it.

And you still need everything for every process in the same docker image. It just sets them up in different VMs, which allows you to scale them horizontally and vertically separate.

Option 2, and 3 require separate mounts if needed, because each machine can only have 1 volume, and each volume can only belong to one machine.

Option 1, has the larger image, like option 2, but can share volumes, in addition to the ENV variables.

In my case, I am trying to use it as a process group, so I can try and share a litefs instance between two apps, but only the backend process have write capabilities.

I am curious the preferred approach. But as I understand it, the first one has all three share and fight for resources, that may or may not be a benefit. The second still creates the cost of the 3rd, but adds complexity to the configuration, like the first one. The third allows you to divide up things easier, but now your managing multiple configs, and apps.

1 Like
3

And you still need everything for every process in the same docker image. It just sets them up in different VMs, which allows you to scale them horizontally and vertically separate.

OK, I hadn’t considered that. That’s not so appealing.

Option 2, and 3 require separate mounts if needed, because each machine can only have 1 volume, and each volume can only belong to one machine.

That’s fine in my case: only the web app server needs a mount.

Option 3 seems like the way to go, but I don’t have a good understanding of how private networking interacts with machines scaling to zero. In my case, C) needs to run on a machine with more memory and CPU so it can get its occasional task done as quickly as possible. But most of the time it should be suspended. It seems like it loses its .internal DNS address when it gets suspended, and I can’t wake it back up without restarting it…

4

Hi @ifiwasabird! I think that that design (option #3) is possible, but with some different tools. Sending traffic to a stopped machine via private networking does not automatically start it, and as a result internal DNS only includes addresses for running Machines. Instead, it’s our edge proxy that is capable of automatically starting and stopping Machines.

You can use the edge proxy internally with Flycast. You would create your app C, configuring a service with auto_start_machines = true in your fly.toml. You would add a private IPv6 address to expose the app on your private network (as opposed to the public Internet). Then, when app B calls app C (either through the IP address directly or the .flycast domain name associated with it), the request goes through the proxy, which will automatically start the app C Machine if it’s not already running.

The edge proxy can automatically shut down Machines when they have no active requests or connections (depending on the concurrency type), but in your case it sounds like tasks may still be running when that happens. Therefore, I’m guessing that you’ll want to set auto_stop_machines = false to disable this. Instead, you can design app C so that the process will automatically exit when it has no tasks running.

(One caveat is that if you have more than two or more Machines in app C, then the proxy may load-balance between them. When doing so it will not take into account the number running tasks on each; it only knows about the number of concurrent requests or connections to it. However, since you mention “occasional” tasks, it sounds like this won’t be an issue for you in practice!)

5

Sending traffic to a stopped machine via private networking does not automatically start it, and as a result internal DNS only includes addresses for running Machines. Instead, it’s our edge proxy that is capable of automatically starting and stopping Machines.

OK, this helps immensely. Thank you! I don’t think I had mentally categorized what I needed as “load balancing,” so I didn’t realize that Flycast was relevant to me.

The edge proxy can automatically shut down Machines when they have no active requests or connections (depending on the concurrency type), but in your case it sounds like tasks may still be running when that happens. Therefore, I’m guessing that you’ll want to set auto_stop_machines = false to disable this. Instead, you can design app C so that the process will automatically exit when it has no tasks running.

Yes, it already does shut itself down, so this approach should work nicely.

1 Like
6

Wow that is good to know about Flycast. Very cool that it can do that.

7

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.