Fly Terraform - Feedback & Wishlist

1

Fly Terraform is amazing - well done Fly team and @DAlperin

I was able to build my project, deploy to the Fly registry and deploy it via Fly Machines. Example code is here: GitHub - mikehostetler/fly-terraform-example: Spike of building Docker Images, pushing to Fly Registry and deploying as Fly Machines

I am starting this thread to collect my wishlist for the Terraform Provider and how I’d like to set up my future workflows.

Support for Fly Consul as a Terraform Backend
I’d love to easily manage Terraform state using the Fly Consul service (Sneak peak: global lock service). I’m tempted to set up S3 (or Minio), create another endpoint, or hook into another stateful backend - but given that Consul is maybe available in Fly already, I didn’t think it would hurt to add this to the wishlist.

Ability to push / manage images without needing a Fly App v1
To deploy a custom Docker image via Fly Terraform, I first need to create a V1 application - create and push the image - then reference it in my new Fly V2 Terraform application.

Example of managing Fly landscape in Terraform via Github Actions
There are plenty of Terraform + Github Actions examples - but smoothing out the flyctl tunnel plus state management would be cool.

If some of these things exist - or there are workarounds now - happy to experiment and put together examples for others to follow.

1 Like
2

Thanks for this. I believe I accidentally used your repo as a reference last night. It was super helpful.

You don’t have to create a v1 app to push images, but you do need ANY app to push images. This creates a chicken and egg problem in the API.

I prefer to have terraform deal with stable infrastructure and leave updating images to scripting. So what I did was have terraform create the app and machine with alpine as a placeholder, set lifecycle ignore on image, output the machine id, then run my deploy script to update the machine with latest build.

Alternatively, if you prefer to have terraform manage the deploys, you can create a null_resource with a local-exec provisioner that depends on the app, calls out to a script of yours to docker build and push to the registry (which will happen after the app is created). And have the fly_machine resource depend on that so the image is in place. You would have to play with the null_resource triggers / dependencies to get it to rebuild when you want it to…

My app is a nextjs app where most of my environment variables are needed at build time not runtime. This combined with the null_resource is awkward enough that I’m pretty happy about keeping deploys to other tooling. Keep terraform for laying the fabric, and other scripts/tools for operating on it. IMO.

An even bigger problem for me with the machines API that I can see anyway is the step backwards in release tooling. Right now this is a small app with one node and a volume, so I can pass around a machine id and do ok. I don’t know how I would manage this at scale, but there would probably be enough scripting to make having terraform in the mix just for show… I might just not know about the tooling, but I can’t tell from the docs anyway.

2 Likes
3

I agree with your statements, there’s a feature request to support private docker images in fly that I posted to. But, I don’t think that’ll really get attention for a loooong time.

Using gitlab ci to upload build/upload the docker image to fly registery first, then fly-terraform would push that image. Not as abc as I hoped, but gets the job done I suppose.

# .gitlab-ci.yml
# Note, gitlab ci has the FLY_API_TOKEN env value set already. 
# That's why fly auth works without requiring any details here. 
stages:
  - validate
  - test
  - build
  - docker
  - deploy
  - cleanup

sast:
  stage: test

include:
  - template: Security/SAST.gitlab-ci.yml
  - template: Security/Secret-Detection.gitlab-ci.yml
  - template: Terraform/Base.gitlab-ci.yml  # https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Terraform/Base.gitlab-ci.yml
  - template: Jobs/SAST-IaC.gitlab-ci.yml   # https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/SAST-IaC.gitlab-ci.yml

# region Terraform Stuff
fmt:
  extends: .terraform:fmt
  needs: []

validate:
  extends: .terraform:validate
  needs: []

build:
  extends: .terraform:build

deploy:
  extends: .terraform:deploy
  dependencies:
    - build
  environment:
    name: $TF_STATE_NAME
    action: start
  # Auto run deploy, no need for manual action to start.
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
      when: on_success

destroy:
  extends: .terraform:destroy
  environment:
    name: $TF_STATE_NAME
    action: stop
  when: manual
#endregion Terraform

#region Docker Build and Upload Image
variables:
  IMAGE_TAG: $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG
  FLY_IMAGE_TAG: registry.fly.io/$CI_PROJECT_NAME:$CI_COMMIT_REF_SLUG
  # Tell 'docker:dind' to enable TLS (recommended)
  # and generate certificates in the specified directory.
  DOCKER_TLS_CERTDIR: "/certs"

build-push-docker-image-job:
  stage: docker
  # Specify a Docker image to run the job in.
  image: docker:latest
  # Specify an additional image 'docker:dind' ("Docker-in-Docker") that
  # will start up the Docker daemon when it is brought up by a runner.
  # Add flyctl to the image
  before_script:
    - apk update && apk add gcompat && apk add curl
    - curl -L https://fly.io/install.sh | sh
    - export FLYCTL_INSTALL="/root/.fly"
    - export PATH="$FLYCTL_INSTALL/bin:$PATH"
  services:
    - docker:dind
  script:
    # Docker login and push image to gitlab
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
    - docker build -t $IMAGE_TAG -t $FLY_IMAGE_TAG .
    - docker push $IMAGE_TAG

    # Push image to fly
    - flyctl auth docker
    - docker push $FLY_IMAGE_TAG
#endregion Docker Build and Upload Image
1 Like