Simulating PR Review Apps via GitHub Actions

Hi, I am trying to auto deploy ephemeral apps via GitHub Actions upon Pull Requests.

The superfly/fly-pr-review-apps doesn’t seem maintained, I haven’t tested yet but having read the code it might not entirely suit.

I tried what I thought would be a simpler approach:

---
name: Fly Deploy (PR)

on:
  pull_request:

env:
  FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}

jobs:
  deploy:
      name: Deploy app
      runs-on: ubuntu-latest

      steps:
        - uses: actions/checkout@v3
        - uses: superfly/flyctl-actions/setup-flyctl@master
        - run: |
            flyctl apps destroy temp-app --yes
            flyctl apps create --name temp-app
            flyctl secrets set SOME_NECESSARY_ENV_VAR=1    --app temp-app
            flyctl postgres attach temp-app-db             --app temp-app
            flyctl deploy                                  --app temp-app
            flyctl scale memory 512                        --app temp-app
            flyctl ssh console -C "app/bin/rails db:seed"  --app temp-app

Yes, any push on any PR would destroy and rebuild the app. I’m fine with that behaviour for now, my problem is that this Action does not work.

First thing I noticed is that the fly command does not work but the flyctl does. I’m not sure why.
Most importantly, the run command breaks:

Run flyctl apps destroy temp-app --yes
  flyctl apps destroy temp-app --yes
  flyctl apps create --name temp-app
  flyctl secrets set SOME_NECESSARY_ENV_VAR=1    --app temp-app
  flyctl postgres attach temp-app-db             --app temp-app
  flyctl deploy                                  --app temp-app
  flyctl scale memory 512                        --app temp-app
  flyctl ssh console -C "app/bin/rails db:seed"  --app temp-app
  shell: /usr/bin/bash -e {0}
  env:
    FLY_API_TOKEN: ***
Destroyed app temp-app
automatically selected personal organization: Pilou
New app created: temp-app
Error prompt: non interactive


Secrets are staged for the first deployment
Error: Process completed with exit code 1.

I may be doing something totally wrong, I am this close of having a rather nice pipeline with Fly.
If anyone had successfully set up such a pipeline, I would love some help!

Not all flyctl commands are automate-able, ref:

  1. Some flyctl commands do not provide a flag to confirm prompts in advance · Issue #933 · superfly/flyctl · GitHub
  2. Need to automatically accept the warning when setting a secret in GitHub actions using the superfly/flyctl-actions@1.1 - #2 by ignoramous

You can set env vars with flyctl deploy command (either with the flyctl deploy -e K1=V1 -e K2=V2 ... switch or declaring an [env] section in your fly.toml) rather than execute a separate command. For secrets, only build-time secrets can be set with flyctl deploy. Runtime secrets will have to be set out-of-band, I am afraid.

If you’re up for it, you can consider sending a pull-request for non-interactive modes for the flyctl commands you want to use with gh-actions: GitHub - superfly/flyctl: Command line tools for fly.io services

is that repo being maintained? looks like some useful pull reqs are there unmerged and lots of people are forking and adding their own features. (which is what we’re doing)

Yeah, I guess. I see a stream of commits from Fly engs. As for unmerged PRs: I don’t see many open for more than 2 weeks, so likely that ones not merged yet may need more work? In any case, if you have an open PR, feel free to ping Fly engs in PR comments to get their attention. From what little I gather, flyctl seems very central to the way Fly thinks about dev-ex.

oops – my bad. i skimmed that gh link and thought you were referring to this one:

1 Like

Thanks for your reply. I tried to remove the secrets set line to push env var in the deploy line, but the process breaks at the create step.
I also tried the yes | workaround without any luck.

If you’re up for it, you can consider sending a pull-request for non-interactive modes

I wouldn’t even know where to start :hot_face:

I managed to reach the deploy stage by rewriting the Action this way:

name: Fly Deploy (PR)

on:
  pull_request:

env:
  FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}

jobs:
  deploy:
      name: Deploy app
      runs-on: ubuntu-latest

      steps:
        - uses: actions/checkout@v3
        - uses: superfly/flyctl-actions/setup-flyctl@master
        - run: flyctl postgres detach temp-app-db                                              --app temp-app-pr || true
        - run: flyctl apps destroy temp-app-pr --yes                                                             || true
        - run: flyctl apps create --name temp-app-pr                                                               
        - run: flyctl postgres attach temp-app-db --database-user=Z$(uuidgen | tr -d -) --force --app temp-app-pr
        - run: flyctl secrets set SOME_NECESSARY_ENV_VAR=1                                     --app temp-app-pr
        - run: flyctl deploy                                                                   --app temp-app-pr
        - run: flyctl scale memory 512                                                         --app temp-app-pr
        - run: flyctl ssh console -C "app/bin/rails db:seed"                                   --app temp-app-pr

Edit: Added the letter Z in front of the uuidgen command so that the database user is sure to start with a letter.

  1. Running line after line bypasses the interactive error

  2. I added the postgres detach step, just in case. I thought I could manage database users but I couldn’t and I got errors because the temp-app-pr generated was already in use. So I added this uuidgen to have a random user name. This is :poop:, but it works for now.
    Edit: I am pretty sure the detach command does not work from CI. That’s why my database user was not removed properly.

  3. Now I get an error I don’t understand upon deploy:

Step 19/38 : ARG DEV_PACKAGES="git build-essential libpq-dev wget vim curl gzip xz-utils libsqlite3-dev"
 ---> Running in 81b4909f7100
 ---> 5546dafa7214
Step 20/38 : ENV DEV_PACKAGES ${DEV_PACKAGES}
 ---> Running in 640fe52b9ddd
 ---> be1b57f5ea40
Step 21/38 : RUN --mount=type=cache,id=dev-apt-cache,sharing=locked,target=/var/cache/apt     --mount=type=cache,id=dev-apt-lib,sharing=locked,target=/var/lib/apt     apt-get update -qq &&     apt-get install --no-install-recommends -y ${DEV_PACKAGES}     && rm -rf /var/lib/apt/lists /var/cache/apt/archives
Error failed to fetch an image or build from source: error building: error rendering build status stream: the --mount option requires BuildKit. Refer to https://docs.docker.com/go/buildkit/ to learn how to build images with BuildKit enabled

I haven’t changed my Dockerfile a bit. The fly deploy command in my non-ephemeral app works fine, so I suppose this has to do with my dark magic above.

Edit: Running the same set of commands from my machine leads to a successful deploy.

1 Like

I added this in the GitHub Action:

env:
  FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}
  DOCKER_BUILDKIT: 1                          # <----- this line

But now I get this error:

image size: 313 MB
==> Creating release
Error Post "https://api.fly.io/graphql": http2: server sent GOAWAY and closed the connection; LastStreamID=2147483647, ErrCode=NO_ERROR, debug=""

So close…

Edit: looks like a different issue related to Fly: server sent GOAWAY and closed the connection - #9 by aaronmoodie, after several attempts, it worked :tada:

1 Like

I’m curious how flyctl without -c <config> or -a <app-name> or without setting FLY_APP_NAME worked?

You either should have scrolled horizontally, or I should have avoided tabs :upside_down_face:

1 Like

The solution I found was still hacky and for some reasons failed too much upon deployment.

I removed the DOCKER_BUILDKIT: 1 line because today I don’t get errors.
Deployments look much more stable now than during the weekend.
The postgres attach command can get i/o timeouts (happened only once).

Also, I removed the postgres detach command because it just didn’t work remotely (it’s interactive only). To make the Action still work on the same Postgres cluster, I need to “rotate” the database user, which I hack with the uuidgen tool and leads to stacked users in my cluster whom I cannot remove from a fly command.
I’m expecting the postgres detach to remove the related database user as well, but I may be wrong.

If anyone has any suggestion in improving this process I’m entirely open to test other approaches :+1:

1 Like

The final GitHub Action I’ll be using is below. I decided to create (and destroy) a dedicated PostgreSQL cluster for Review Apps instead of using the same cluster, to avoid messing with my database users. I’m not of fan of wasting that much resource and it’s probably slower, but for now it looks like it’s a much more stable approach.

It’s a fun ride, I’ll regularly check updates for the Rails stack :+1:

---
name: Deploy PR (Fly)

on:
  pull_request:

env:
  FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}

jobs:
  deploy:
    # Create and deploy Fly app upon any push on any Pull Request to https://temp-app.fly.dev/

      name: Deploy PR (Fly)
      runs-on: ubuntu-latest

      steps:
        - uses: actions/checkout@v3
        - uses: superfly/flyctl-actions/setup-flyctl@master

        # Destroy and rebuild app + PostgreSQL cluster
        - run: flyctl apps destroy temp-app --yes
        - run: flyctl apps destroy temp-app-db --yes
        - run: flyctl postgres create --name=temp-app-db --region=cdg --vm-size=shared-cpu-1x --volume-size=1 --initial-cluster-size=1
        - run: flyctl apps create --name temp-app
        - run: flyctl postgres attach temp-app-db --app temp-app

        # Scale app and set environment variables before deployment
        - run: flyctl scale memory 512 --app temp-app
        - run: > 
            flyctl secrets set --app temp-app
            RAILS_MASTER_KEY=${{ secrets.STAGING_RAILS_MASTER_KEY }}
            THIS_IS_STAGING=1

        # Deploy and seed data
        - run: flyctl deploy --remote-only --app temp-app
        - run: flyctl ssh console -C "app/bin/rails db:seed" --app temp-app
4 Likes