Format can be selected at fly launch time, or switched at any time.
Tests are in place to ensure all three formats “round trip” and produce identical results.
Order of fields is the same in all three formats.
TOML remains the default, at least for now. Format is determined by file extension.
The one intentional difference is that comments placed at the top of the file are omitted in the JSON serialization as JSON does not permit comments.
Usage:
Add --yaml or --json to fly launch to specify the configuration format you want.
Already got an application? No problem. Run fly config save and specify --yaml or --json. Just be sure to remove your fly.toml file when done as it takes precedence.
You can even run fly config show --local to preview different formats. This one command defaults to JSON, but you can now specify --toml or --yaml.
Questions:
Should we change the default based on language/framework? Perhaps Rails users would be more comfortable with YAML? Perhaps JavaScript users would prefer JSON?
What integration possibilities does this open up for you? Do you have tooling where it is easier to consume or produce YAML or JSON?
What functions does this unlock? YAML, for example, supports map merging. Perhaps we could use this to enable per environment configurations. A sketch of what this could look like:
Other overrides should be equally as straightforward: for example different vm sizes for staging and production.
While YAML specific, this would be easy for us to support - we would just need to agree on how the environment is specified and then only unmarshal just that “branch” of the input.
For me, JSON’s lack of comments makes it a non-starter for infrastructure config. Hashicorp tries to get around it by allowing a special "_comment" key, but that’s a hack, and you can only have one comment per object.
What integration possibilities does this open up for you?
JSON and/or YAML schema linting are extremely useful, both during editing and CI.
Perhaps we could use this to enable per environment configurations.
One file with multiple environments would be VERY nice. Today, keeping multiple environment files in sync is tricky and error-prone.
As a source file, absolutely, but as an intermediate format its (relative) no-surprises aspect makes it a welcome option.
For example, I’m planning on using .rktd (s-expressions) to define configuration. Those have their own comment syntax, own macro language, merging concepts, and so on.
I’d like to explore more, but feel the need to have my explorations anchored by real use cases. For example, it is clear that supporting multiple environments better would be useful, but I have no way of validating that YAML’s map merging facilities would be sufficient.
If there are developers out there capable of installing go and running make, these explorations could go quickly as I can create branches of flyctl that you can build and try out.
Plush fly toml templates - suppose we supported a real templating language which could consume both .env files and --env flags on the deploy command line. I’m hesitant to pull in a full templating engine into flyctl, but it may be worth exploring.
I would love to be able to use JSON instead of toml for sure. As someone else noted comments would.be nice. So maybe JSONC instead of plain JSON? There are eslint/prettier plugins for example to support them. Unsure on ide support but I like working with Json a lot more than toml (probably cause I’m used it in a TS monorepo)
There appears to be at least two different JSONC’s (or perhaps it is JSONC and JSON-C), as well as a JSON5.
But first, an interesting observation, one that I don’t think many are aware of. YAML intends to be a proper superset of JSON, so you can take your JSON with comments, and even with unquoted keys and other affordances, rename the file to fly.toml and use it as is.
Knowing this, I’d be inclined to continue to output strictly conforming JSON, but be more liberal in what we accept (a.k.a. Postel’s law). Since none of the various go libraries for JSONC/5 (with or without a dash) appear to be actively maintained, and given that yaml will happily consume JSON and we already include the yaml libraries in flyctl, doing something like the following would make sense:
If file extension is .json, first try to parse as JSON. If that fails, save the original error and retry parsing it as yaml. If that succeeds, go with that result, otherwise report the original error.
fly configuration files are small enough, and parsers (particularly ones written in go) are fast enough that the overhead of double parsing is unlikely to be noticeable.
Even this on its own would be great… I wish flyctl would actually go further and also complain ahead of time about settings combinations that other parts of the Fly infrastructure will consider illogical/invalid:
As in your existing patch, these would need to be warnings, to preserve backward compatibility. It would be nice to have a --strict flag—to opt into them being fatal errors, though.
(Maybe there could be “editions”, as well, to allow pickier standards to be introduced over time…)
A big +1 for shared configs between environments with yaml. I think that having just staging and production as keys is overly perscriptive. I would like to see those nested under an environments key so we can name our environments as we like.
At the risk of this being a bit OT: I got impatient and wrote a simple script that allows me to merge a common config with app-specific configs. It goes like this:
import fs from "fs";
import path from "path";
// @ts-ignore
import merge from "@alexlafroscia/yaml-merge";
// assumes running from package.json script
const scriptDir = path.join("./", "app", "scripts", "fly-config");
const main = async () => {
const baseConfigPath = path.join(scriptDir, "fly.base.yaml");
const envsDir = path.join(scriptDir, "envs");
const configPaths = (
await fs.promises.readdir(envsDir, {
recursive: false,
withFileTypes: false,
})
).filter((path) => {
// it's more robust to use stat isDirectory, don't care
return path.endsWith(".yaml");
});
for (const configPath of configPaths) {
const merged = merge(path.join(envsDir, configPath), baseConfigPath);
console.log(merged);
await fs.promises.writeFile(`./fly.${configPath}`, merged);
}
};
main();
It would be great if there was an environment variable FLY_CONFIG that functions similarly to FLY_APP, except it points to the config file and saves having to use -c flags all the time.