Machine Files

Fresh Produce
, ,
1

We’ve just rolled out some changes to the config definition used for creating Machines to make it easier to have files created and populated from two data sources in the config. All data must be provided to the system as a base64 encoded string.

Below are two examples of how you can take advantage of this new feature when working with the Machines API directly:

Raw Data

{
  ...
  "files": [
    {
      "guest_path": "/etc/foo/bar.json",
      "raw_value": "ewogICJjb25maWciOiB7CiAgICAic29tZXRoaW5nIjogImNvbmZpZ3VyYWJsZSIKICB9Cn0K"
    }
  ]
}

With the above example, a file will be created at /etc/foo/bar.json populated with the following content:

{
  "config": {
    "something": "configurable"
  }
}

Secret Data

{
  ...
  "files": [
    {
      "guest_path": "/etc/db/config.json",
      "secret_name": "DB_CONFIG"
    }
  ]
}

If you have added a secret DB_CONFIG to the app, with the above example, a file will be created at /etc/db/config.json populated the content from the secret. If you had previously been relying on the DB_CONFIG environment variable to access the secret, it would no longer be set in this scenario and the only way to access the secret is in the /etc/db/config.json file.

REMINDER: the value must be base64 encoded before creating it:

fly secrets set --app my-awesome-fly-app DB_CONFIG="<BASE_64_ENCODED>"

We’ve also made it possible to define files for the machine using CLI flags and infly.toml for use with fly deploy.

The following flags have been added to fly machine run, fly machine update and fly deploy:

--file-local=/path/inside/machine=<local/path>
--file-literal=/path/inside/machine=VALUE
--file-secret=/path/inside/machine=SECRET

And below is an example for defining files via fly.toml:

[[files]]
  guest_path = "/path/to/hello.txt"
  raw_value = "aGVsbG8gd29ybGQK"

[[files]]
  guest_path = "/path/to/secret.txt"
  secret_name = "SUPER_SECRET"

[[files]]
  guest_path = "/path/to/config.yaml"
  local_path = "/local/path/config.yaml"
  processes = ["web"]
9 Likes
2

I’ve been eagerly waiting on this because I feel like 90% (if not all) of the manual work I’ve done to make MongoDB work on Fly was generating files as you can see on the docker entrypoint so having this as a built-in feature for machines and fly.toml is amazing. I might revisit that project at some other point.

1 Like
3

Thanks :purple_heart:

I came across this feature a few days ago while checking fly deploy --help, and I must say, it’s a game changer! Now you don’t even need a custom Dockerfile to deploy most of the infra stuff.

Look at how easy it is to use Fly’s proxy (i.e. .flycast addresses) to run a formation of opentelemetry collectors in gateway mode:

fly launch --image="otel/opentelemetry-collector-contrib:0.81.0"
           --file-local="/etc/otelcol-contrib/config.yaml=infra/otel.yaml"
           --no-public-ips

Or to forward logs in a single command:

fly launch --image="timberio/vector:0.31.0-alpine"
           --file-local="/etc/vector/vector.toml=infra/vector.toml"
           --no-public-ips
7 Likes
4

Is there a way to set file permissions? Some apps require files (especially files with secrets) to have certain permissions, e.g. 0600

5

There’s not currently a way to set the permissions in the API right now. A possible option until then would be to set them as part of the entrypoint.