Pull Request Previews and Preview Environments

I’ve always been fascinated by Infrastructure as Code. I was wondering if this is something on fly’s radar? I know there is currently a fly.toml, but I think you can only define one app at a time in it. I think being able to define multiple apps, postgres, etc would be a cool abstraction that would then allow for duplicated the whole stack in a PR temporarily and then shutting down.

Example: Pull Request Previews | Render

Side note: Is it possible to just write JSON instead of TOML?

We’re using Render.com with their IaC for Preview environments and since we have two repos (backend, frontend) it causing us issues quite often.
I feel I would rather do it “manually” with the flyctl command called from GitHub actions than fighting the IaC limitations regularly. Just browse the https://feedback.render.com site. Lot of the feature requests / issues are IaC related.
I think it’s extremely difficult to find a balance between versatility, complexity and simplicity because the use cases can be so different and complex.
IMHO the time spent doing that would be better invested elsewhere. :slight_smile:
Just my two cents.

It seems to me that render didn’t implement it well. Heroku is used by many large companies and it seems to work very well.

IMO, I think most of what fly can do distributed wise can be also done by AWS/GCloud, so at the moment I feel like fly is targeted to companies that need scale but can’t afford to hire someone to manage the complex infrastructure. I feel like that’s a lot smaller pool than companies looking for a simple all in one solution (that happens to scale as they grow). ¯_(ツ)_/¯

Maybe I’m completely wrong, just wanted to put it out there

1 Like

I just spent some time setting up CI in GitHub to give us a simple preview environment with Phoenix and Postgres with the superfly/flyctl-actions GitHub Action – if there’s some interest I can redact some of our specific info and share?

2 Likes

That would be great!

I have 2 GH workflows. The first is from their docs, I call it main.yml. The second is called branch.yml.

The key for me is in addition to the fly.production.toml that I use for all my deploys (using the GH Actions formula from fly’s docs, with deploy -c fly.production.toml), I made a copy and named it a fly.development.toml with the following changes:

# fly.development.toml
app = "<REPLACED_IN_CI>-dev-<myapp>"
...
PHX_HOST = "<REPLACED_IN_CI>-dev-<myapp>.fly.dev"

The key part is changing these during the deployment pipeline. I just recreated the “manual” process using the GH Action from fly. The most difficult parts were figuring out the correct order and replacing the <REPLACED_IN_CI> portions with sed :joy:

# branch.yml
name: Dev Preview
on:
  pull_request:
    branches:
      - main
env:
  FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}
jobs:
  infrastructure:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout Github repo
        uses: actions/checkout@v2.4.0
      - name: Save short SHA to env
        uses: benjlevesque/short-sha@v1.2
        id: short-sha
      - name: Build DB
        uses: superfly/flyctl-actions@1.1
        with:
          args:
            "pg create --region atl --vm-size shared-cpu-1x --name ${{ env.SHA
            }}-dev-<myapp>-db --volume-size 1 --organization <myorg>
            --initial-cluster-size 1"
      - name: Create fly app
        uses: superfly/flyctl-actions@1.1
        with:
          args: "apps create ${{ env.SHA }}-dev-<myapp> --org <myorg>"
      - name: Connect app to DB
        uses: superfly/flyctl-actions@1.1
        with:
          args:
            "pg attach -a ${{ env.SHA }}-dev-<myapp> --postgres-app ${{ env.SHA
            }}-dev-<myapp>-db"
      - name: Set up Phoenix secret_key_base with fly secrets
        uses: superfly/flyctl-actions@1.1
        with:
          args:
            "secrets set
            SECRET_KEY_BASE=<set-a-secret-or-use-github-secrets>
            -a ${{ env.SHA }}-dev-<myapp>"
      - name: Modify fly.development.toml file
        run: |
          sed -i 's/<REPLACED_IN_CI>/${{ env.SHA }}/g' fly.development.toml
      - name: Deploy app
        uses: superfly/flyctl-actions@1.1
        with:
          args:
            "deploy --region atl --app ${{ env.SHA }}-dev-<myapp> -c
            fly.development.toml"
      - name: Post link to PR Preview
        uses: actions-ecosystem/action-create-comment@v1
        with:
          github_token: ${{ secrets.github_token }}
          body: |
            Your PR is deployed, see it [here](https://${{ env.SHA }}-dev-<myapp>.fly.dev)
            ...
            Delete the app [here](https://fly.io/apps/${{ env.SHA }}-dev-<myapp>/settings)
            Delete the db [here](https://fly.io/apps/${{ env.SHA }}-dev-<myapp>-db/settings)

This works pretty well if you only have phoenix + DB like me, just replace <myapp>, <myorg>, and the region with your info!

My strategy going forward for deleting them (currently I’m doing this manually) is to modify the above to add a pr number to the name, then create a third workflow to use fly list apps -<prnumber>-dev and then use fly destroy on those. I also haven’t needed to worry about multiple regions yet but I’ll cross that bridge when we come to it :slight_smile:

I would also love any and all feedback on this! I’m loving the experience so far.

4 Likes

This looks awesome! I think theres a way to run an action when a PR is merged, maybe you can use that to delete it? And yeah definitely think using the PR # is the “standard” practice so its much easier to find and delete

I feel like this creates a lot of spam in the UI (you might see tens of apps), and again could be improved if fly supports this with some sort of namespace or deploy environment. I think vercel is king when it comes to easy of use

Someone also built this: GitHub Action for deplying staging apps on fly.io · Actions · GitHub Marketplace · GitHub

3 Likes

Super appreciative of this thread, figured I would drop something here that seems to be working for my workflow (no pun intended) which is a Postgres DB + Nodejs & Prisma app (distroless container caused some quirks too so happy to expand on that for Prisma users).

Background:
I only really wanted one DB per PR (hence the if statement on this job), and to then deploy the app on synchronize (subsequent code changes).

So more simply put, I have taken from the approaches described above, pinned these to the PR number, and split db and app into separate jobs.

Happy to explain more, but just wanted to put this out here as I just got it work :tada:

# branch.yml
name: Deploy Preview
on:
  pull_request

env:
  FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}
  FLY_ORG: ${{ secrets.FLY_ORG }}
  FLY_PREVIEW_APP: ...

jobs:
  db:
    if: github.event.action == 'opened'
    runs-on: ubuntu-latest
    steps:
      - name: Checkout Github repo
        uses: actions/checkout@v3
      - name: Get PR number
        uses: jwalton/gh-find-current-pr@v1
        id: findPr
        with:
          # Can be "open", "closed", or "all".  Defaults to "open".
          state: open
      - name: Build DB
        uses: superfly/flyctl-actions@1.3
        with:
          args:
            'pg create --region ord --vm-size shared-cpu-1x --name pr-${{ steps.findPr.outputs.pr }}-dev-${{ env.FLY_PREVIEW_APP }}-db --volume-size 1 --organization ${{ env.FLY_ORG }}
            --initial-cluster-size 1 '
      - name: Create fly app
        uses: superfly/flyctl-actions@1.3
        with:
          args: 'apps create pr-${{ steps.findPr.outputs.pr }}-dev-${{ env.FLY_PREVIEW_APP }} --org ${{ env.FLY_ORG }}'
      - name: Connect app to DB
        uses: superfly/flyctl-actions@1.3
        env:
          FLY_APP: pr-${{ steps.findPr.outputs.pr }}-dev-${{ env.FLY_PREVIEW_APP }}
        with:
          args: 'pg attach --postgres-app pr-${{ steps.findPr.outputs.pr }}-dev-${{ env.FLY_PREVIEW_APP }}-db'
      - name: Set up secret_key_base with fly secrets
        uses: superfly/flyctl-actions@1.3
        env:
          FLY_APP: pr-${{ steps.findPr.outputs.pr }}-dev-${{ env.FLY_PREVIEW_APP }}
        with:
          args: 'secrets set ...'
      - name: Post link to PR Preview
        uses: actions-ecosystem/action-create-comment@v1
        with:
          github_token: ${{ secrets.github_token }}
          body: |
            Delete the db [here](https://fly.io/apps/pr-${{ steps.findPr.outputs.pr }}-dev-${{ env.FLY_PREVIEW_APP }}-db/settings)

  app:
    if: github.event.action == 'synchronize'
    runs-on: ubuntu-latest
    steps:
      - name: Checkout Github repo
        uses: actions/checkout@v3
      - name: Get PR number
        uses: jwalton/gh-find-current-pr@v1
        id: findPr
        with:
          # Can be "open", "closed", or "all".  Defaults to "open".
          state: open
      - name: Modify fly.preview.toml file
        run: |
          sed -i 's/<REPLACED_IN_CI>/pr-${{ steps.findPr.outputs.pr }}/g' fly.preview.toml
      - name: Deploy app
        uses: superfly/flyctl-actions@1.3
        with:
          args:
            'deploy --region ord --app pr-${{ steps.findPr.outputs.pr }}-dev-${{ env.FLY_PREVIEW_APP }} -c
            fly.preview.toml'
      - name: Post link to PR Preview
        uses: actions-ecosystem/action-create-comment@v1
        with:
          github_token: ${{ secrets.github_token }}
          body: |
            Your PR is deployed, see it [here](https://pr-${{ steps.findPr.outputs.pr }}-dev-${{ env.FLY_PREVIEW_APP }}.fly.dev)
            ...
            Delete the app [here](https://fly.io/apps/pr-${{ steps.findPr.outputs.pr }}-dev-${{ env.FLY_PREVIEW_APP }}/settings)

I’ve also set this up here: Demo: Heroku-style Review Apps