Multi process machines

Machines now supports multiple processes within a VM. To use it, add processes to your machines config. processes has the structure:

"processes": [
  {
    "name": "web",
    "entrypoint": ["/bin/bash", "-c"],
    "cmd": ["some", "command"],
    "env": {
      "HELLO": "world"
    },
    "user": "flyio" // optional
  },
  {
    "name": "logger",
    "entrypoint": ["/bin/bash", "-c"],
    "cmd": ["some", "other", "command"],
    "env": {
      "WORLD": "hello"
    }
  },
  ...
]

An example config

{
  "config": {
    "image": "senyo/watagwan:latest",
    "processes": [
      {
        "name": "web",
        "entrypoint": ["/bin/bash"],
        "cmd": ["loop1.sh"],
        "env": {
          "PROC_ID": "1"
        }
      },
      {
        "name": "logger",
        "entrypoint": ["/bin/bash"],
        "cmd": ["loop2.sh"],
        "env": {
          "PROC_ID": "2"
        }
      }
    ],
    "env": {
      "VERSION": "1.2.3",
    },
    "services": [
      {
        "ports": [
          {
            "port": 443,
            "handlers": [
              "tls",
              "http"
            ]
          },
          {
            "port": 80,
            "handlers": [
              "http"
            ]
          }
        ],
        "protocol": "tcp",
        "internal_port": 8080
      }
    ]
  }
}

This is a first iteration to supporting the sidecar pattern where each process has its own image and can run in its own process namespace.

That’s all there is to it. Now, here’s what you need to know:

  1. As mentioned above, processes run within the same VM. This is different than the [processes] directive in an app config.
  2. When any of your processes exit/dies, the entire VM is shutdown.
  3. You can’t use a different image for each process (we’re working on this). Everything you need for each process to run must be contained within your docker image.
  4. Processes run in the same process namespace (we’re working on this too).
  5. Environment variables can be defined at a global scope and a process scope (see the env config option in the process definition).
    5.1. Each process will be spawned with the globally scoped environment variables and its process scoped environment variables. If the same environment variable is defined in at both scopes, the process scope wins.
    5.2. When you SSH into a running machine, only the globally scoped environment variables are in the environment.

Give it some mileage and let us know if you run into any issues or have questions.

9 Likes

This is great!

2 Questions:

  • I’m assuming I can limit the services section to a certain process similar to the fly.toml section
[[services]]
  processes = ["web"] # this service only applies to the web block
  • This also applies to the fly.toml processes section, but:

    Is there a way to start a process just using the default CMD defined in my Dockerfile? Quite often that is the “main” process and I might want to start auxiliary processes like background workers or logging processes with custom commands. The way it stands right now, I always have to maintain my CMD command in two places.

I tried omitting the cmd section in the JSON but I hit a 60 second timeout on the API with the following body:

{
	"error": "You hit a Fly API error with request ID: 01GGT5M0WGZVBBNYE424X8PBSS-fra"
}

service names

Previously (or…currently with “apps”, instead of “machines”), processes creates more than one VM.

In the setup with Machines, it’s one VM with multiple processes.

So the services section doesn’t make sense to be specific to a specific process as its essentially opening ports into one VM.

ENTRYPOINT + CMD

I’m not totally sure on that point yet - omitting the CMD is definitely what I would have tried also (or setting it to null).

I suspect the FLY API error was a coincidence but let me know if that seems to consistently timeout when you omit that.

Ah okay got it. Seems like I misunderstood the “apps” processes then. I’m currently using Overmind to manage multiple processes on one machine, but this seems like a nice replacement.

I think some kind of config for just running the container image “default” command as a named process would be really nice.

Gotcha! You may even want to keep using Overmind, depending on your needs. Multiple VM’s via processes (in apps) means more compute to pay for.

Machines lets you run multiple processes in one VM, which is why we’re calling it a “sidecar”

1 Like

Great question. cmd and entrypoint override the respective arguments in your Dockerfile. If you omit either of them, it will use the cmd and/or entrypoint set in the Dockerfile. In your case, you can omit both of them.

1 Like

I saw commits to flyctl for multi-proc the other week; and so, any examples for fly deploy / fly run / fly update? Also, does fly clone clone multi-procs just fine?

Btw, the forum could do with a new Machines tag / category?

Thanks.

The commits to flyctl for multi process machines was only to add configuration options. All flyctl commands regarding should work as per usual.

1 Like

I currently have frps running in an app (machines). I am think about running caddy in the same app, if possible? I understand that I currently have to modify the Dockerfile so that caddy is available.

@senyo wrote “add processes to your machines config”. Is this another config file? Or is it fly.toml? (but his example is a JSON). Excuse me for my confusion.

Also, a lot of things has happened around fly.io just the last months. Is this information still up-to-date? (this thread is linked in the docs)

Dockerfile

FROM alpine:latest
ENV VERSION 0.48.0
WORKDIR app/
RUN wget https://github.com/fatedier/frp/releases/download/v${VERSION}/frp_${VERSION}_linux_amd64.tar.gz && \
    tar -xf frp_${VERSION}_linux_amd64.tar.gz --strip-components 1 frp_${VERSION}_linux_amd64/frps && \
    rm frp_${VERSION}_linux_amd64.tar.gz
COPY frps.ini .
CMD ["/app/frps","-c","/app/frps.ini"]

And the fly.toml

app = REDACTED
kill_signal = "SIGTERM"
kill_timeout = 120
primary_region = "arn"

[[services]]
  protocol = "udp"
  internal_port = 8080

  [[services.ports]]
    port = 7000
  [services.concurrency]
    type = "connections"
    hard_limit = 5
    soft_limit = 5

[[services]]
  protocol = "tcp"
  internal_port = 8080

  [[services.ports]]
    port = 80
    handlers = ["http"]

  [[services.ports]]
    port = 443
    handlers = ["tls", "http"]
  [services.concurrency]
    type = "connections"
    hard_limit = 5
    soft_limit = 5

  [[services.tcp_checks]]
    interval = "1m0s"
    timeout = "2s"
    grace_period = "1s"
    restart_limit = 0

[[services]]
  protocol = "udp"
  internal_port = 7001

  [[services.ports]]
    port = 7001
  [services.concurrency]
    type = "connections"
    hard_limit = 5
    soft_limit = 5

[[services]]
  protocol = "tcp"
  internal_port = REDACTED

  [[services.ports]]
    port = REDACTED
  [services.concurrency]
    type = "connections"
    hard_limit = 5
    soft_limit = 5

Check docs, if you haven’t already. Might answer some of your questions.

From what I can tell, docs now exclusively focus on V2 (machines) over V1 (nomad).

I’m afraid there is a naming confusion around process groups (the docs @ignoramous linked too) and the multi-processes work that @senyo did.

  1. “Process groups” and “multi-processes” are very different things.
  2. “Process groups” can be defined in fly.toml with [processes] section
  3. “Multi processes” have no section or way to be configured using fly.toml
  4. A “Process group” is a collection of machines that all run the same single process on start
  5. “Multi processes” is a feature that allows running many processes within a single machine.
  6. In a “multi processes” machine, the “init” spawns and monitors multiple children directly when without it, it only spawn one child (the image entrypoint)

If the previous explanation is clear, then by now you know Senyo was not referring to fly.toml, but to the JSON payload accepted by machines API when creating a new machine.

1 Like

Probably doesn’t help that processes in the machine API is completely different to processes in the fly.toml. processes was never a good name for the configuration in the fly.toml

1 Like

Now I am thoroughly confused. Isn’t the fly.toml [processes] for AppsV2 the same as multi-process in a single VM (which is different to [processes] in AppsV1 which are spread across VMs)?

fly.toml’s [processes] has never been about multi-process in a single VM. Not for Nomad, not for Apps V2 (aka machines)

[processes] in fly.toml is how you create “process groups”. A process group is a group of VMs that all run the same command. You can have different groups, each group will run a different command when a machine for that group starts.

Above is very different to machine’s “multi process” (Senyo’s work). At the moment, there is no way to configure “multi process” in fly.toml. “multi process” is the basis for things like sidecars, so an idea is to add a “[[sidecars]]” section to fly.toml that flyctl converts into machine’s multi process on deploys.

And yes, naming hasn’t been the best, so the confusion is real.

The [[sidecars] thing is just an idea, willing to hear your input on ways to expose this feature through fly.toml.

2 Likes

I would avoid using sidecars as multi-process is not quite the same as a sidecar and will probably cause more confusion when things don’t work as people expect when they see sidecar.

IMO your best bet long term is to deprecate processes in fly.toml and replace it with something like service_groups or instance_groups (I personally prefer the former) which makes its much clearer that these are separate VMs and not multiple processes in the same VM. Ensure you go through all documentation and update as necessary.

Wait a couple of months for the change to propagate and have everyone switch over, making sure to have lots of comms about the change, then remove support for processes in fly.toml.

Wait another month or so to sort out any stragglers then reuse processes in fly.toml for multi-process. Update documentation and release a timestamped blog post so if anyone finds an old tutorial or other outdated resource and wonders why it doesn’t work, there is a clear history they can easily find to explain the changes.

2 Likes

Throwing my hat in the ring, serviceworkers or workers might work, too (no pun).

Interesting, even if (in my experience) it is never worth the trouble to break api guarantees.

May be introduce a compat_date entry in fly.toml (like Cloudflare does with wrangler.toml and use that as a marker to deprecate or reinterpret existing behaviour). But this complicates documentation and may induce some amount of frustration leading folks to never touch the compat_date field altogether.

1 Like