fly.io instances as Jenkins agents?

1

I’m wondering if using fly.io as Jenkins Agents (workers) is a good idea or not.

First I would need to implement a plugin similar to these:

This plugin would automatically manage the lifecycle of an agent (e.g. spin up a new fly.io instance for every new build).

Right now I use the kubernetes-plugin, and I have a few different images for different jenkins jobs.

My “problem” is that I have to wait for k8s to schedule a pod for me, which takes ~30sec. On top, if the pod scheduling happens to propagate to a new VM request from AWS (because the k8s cluster is full), then this time increases to 2-3 minutes.

If I read correctly this blog post: Docker without Docker · Fly, then this process of spinning up a new Jenkins Agent, could potentially be significantly faster. Right?

Also, is there any similar ongoing effort that I’m not aware of? Maybe for some platform other than Jenkins?

Thanks in advance :slight_smile:

2

It’s a good idea! It’s just a little harder than it needs to be. We have an example Buildkite worker app that we use for our own builds.

The problem is that you have to manage scaling separately, we don’t have a good way to add/remove worker instances as needed. The simplest thing to do is run a process on a separate app, probably using shared-cpu-1x that uses our API to set the instance count you need on a builder app.

The other problem is that the built in disks for instances are only 5GB. You can add persistent volumes, which is helpful for caching between builds. But disks don’t autoscale, you have to keep disks around if you want to use them again. They’re relatively cheap, so it might be worth it.

3

Could I not add/remove worker instances from my Jenkins plugin, by using flyctl scale ?

Also, I can scale down to 0 with flyctl right?

4

Yes that will work great! I hadn’t thought about just scripting flyctl but that makes total sense. And scaling to 0 with flyctl works just fine.

5

Cool, thanks!
I will give this a go when I find some time :sweat_smile:

1 Like