I’m using Fly.io to deploy a few apps in a single environment, and I’m trying to automate the process. I’ve been using Makefiles and Terraform, but I’m running into some challenges with that approach.
The Terraform approach doesn’t handle remote builds from the current working directory (at least in an easy way, not sure if using docker provider with perhaps a fly remote builder can work), which means I’m missing out on some cool features of flyctl imperative approach. I also need the ability to do cross-platform builds, which adds another layer of complexity. I’m using a Mac M1 to build and deploy to x86_64 machines, so I need an easy way to do this remotely.
Right now, I’m doing a two-phase deploy where I first build the apps remotely with fly deploy --build-only --push, then grab the image IDs and carry them over as Terraform variables before doing terraform apply. This isn’t the ideal solution, so I’m hoping the Fly.io community or team have better ideas
My ideal solution would be a declarative spec of an environment that includes multiple apps, which is something that Terraform partially handles. Ideally, I’d like to build and deploy from the current working directory using Fly remote builders, and use a config spec for everything else (env variables, secrets, memory settings, services config, etc.).
I’d love to hear your thoughts on this! Have any of you run into similar issues with deploying multiple apps in a single environment, especially with cross-platform builds? How did you solve them?
@lubien I’m aware of the machines API, that’s what we’re using and what Terraform handles (I think it only handles the machines api already).
But that actually doesn’t solve the challenges I’m mentioning in my post.
@amine I am looking for a declarative approach as well. Automation is the only way to ensure consistency. With transitions occurring and a significant amount of doc content labelled “legacy”, it’s increasingly difficult to parse information to determine a direction.
I would like to see something clearer on app-v2 and machines. If v2 apps are being prioritized over v1, I believe fly as an organization, ought to version the documentation to reduce the confusing mix of v1 apps, v2 apps and machines.
There are presently 4 approaches to interact with the infrastructure that have different capabilities. These are presently all in a different state, incomplete and/or inconsistent with each other. This means, you cannot perform all operations using any one approach. The terraform provider vs flyctl, as an example, inconsistently implements the same API with inputs and outputs or not at all.
flyctl
REST API
GraphQL API
Terraform provider
Typically, tooling implements the same APIs underneath. My expectation is that flyctl and terraform ought to implement the same APIs and the GraphQL and REST are only flavours of the same API, so that as a developer, we can just consume these things.
For automation, it’s critical to get outputs from the APIs for state, so if changes are made, tooling can discern the current state to transition to the desired state. Hopefully there will be improvements here that will make this possible.
@kurt what direction are you are taking for APIs so there is a much clearer and cleaner path to automation with consistent APIs?
@amine A declarative approach to the environment is what docker accomplishes together with the toml. I think the piece missing to effectively automate with fly is the lack of consistency in the tooling so that you are not needing a mix of the terraform provider, flyctl, REST and GraphQL APIs at different versions to accomplish something.
I do agree the transition from v1 to v2 was a bit bumpy. Terraform first supported this while flyctl was a bit behind iirc but recently it did eventually catch-up.
But actually I like fly.io so much that I’m willing to experiment with features when they’re in alpha/beta releases (as long as reliability of running apps is the top priority), that’s probably one of the reason I picked Fly.io as there is a lot of back and forth feedback between the community and the fly team <3.
The purpose of this post was more to get a general direction, perhaps the fly.io team is already working on a declarative multi-app solution (maybe that’s Terraform or another internal tool) or have ideas on how that should be handled by users or it is in the roadmap and just a matter of time to get picked. Using that feedback, I can get a sense whether I should invest in implementing an abstraction on top of the API myself to solve the challenges I mentioned or continue with quick hacks and wrappers while a more permanent feature is implemented.
@amine It might be helpful to understand what you mean by a multi-app declarative solution. Terraform is capable of automating anything like this provided the underlying APIs are available – even while they are not, if you consider cobbling scripts etc. I suspect you are looking for Docker compose to be implemented which is not supported by fly.
