Fly "containers" as a CI runners

We use Gitlab runner to build our code (using docker in docker) – since it’s just a docker container, any reason we can’t setup a job as a runner for our project? I especially like the idea of bare metal for this. We’re happy to pay for the node but it’s more about whether it can be done. I have a hunch it can, but might wait for someone else to figure out the quirks :slight_smile:

You can! It’s relatively easy, we use it for Buildkite workers. The biggest problem right now is that there’s no way to autoscale them based on build activity. If you just want to run one it’ll work great, if you want to scale up and down you’ll have to write a little app to query the Gitlab API and scale your fly worker app.

We highly recommend using persistent volumes for this. They’re great for a build cache. And you can retain that cache when scale your app to 0, then back up.

This should be pretty easy to get going as a Fly app: Run GitLab Runner in a container | GitLab

Awesome, will send this to my devops guy :slight_smile:, Tailscale, all get me excited about elegant alternate solutions to the mainstream complexity of Kube, etc. (Hmm maybe I’m just a Wireguard fanboy).

Thank you and hope to use the product in coming weeks.

Have you played around with offering ramdisks as block devices for extra fast building? I’ve wanted to play around with this idea but I doubt IO is the bottleneck in our elixir builds, I think it’s actually CPU. But maybe for something like node builds it might make a difference.

We haven’t, so far the bottleneck in everything we’ve tried has been dependency/intermediate caching. This is true with Rust, in particular.

Our Elixir builds are also very slow because we don’t do a great job with buildpack caching. When we build from a Dockerfile, they’re pretty quick.

1 Like