A proposals for fly launchers

Declassified
1

We’ve been discussing ways to improve the experience of launching applications with flyctl. Here we gather all the ideas and propose a work path.

The overarching goals are to make it easy for:

  • Developers to configure and deploy existing applications with minimal effort
  • Developers to launch vetted services (Redis, imgproxy, etc) from existing images, Git repositories or both
  • Us to keep templates, docs and images up-to-date without modifying flyctl directly

My thought on commands:

fly launch: Detect apps in the current directory and try to generate a Fly config and a builder (Dockerfile or otherwise)
fly launch redis: Launch a vetted redis service with some prompts for secrets/vars, like our UI-based launchers have done
fly launch —from https://github.com/fly-apps/keydb: Launch a service from a repo which has a .fly directory and meets requirements for deployment

Configuring and deploying existing apps

fly launch simply scans the current directory for the existence of strings or files, and plants a fly.toml and other embedded files (like a Dockerfile). It also asks for required secrets, adds entries to statics, etc. This works, but it’s brittle and requires updating flyctl for every small change to a scanner.

To improve this, we’d want each detectable project type to have an associated git repo, under a separate Github org like fly-apps. We might even consider moving the detectors to a set of bash scripts that could be run by flyctl to encourage contributions.

These repos might contain:

  • A working example app for a particular runtime version, service, etc
  • Documentation about app configuration and concepts, in markdown, that can be displayed in flyctl, and also embedded in our docs pages
  • A .fly directory with templates we would inject into an app, like a Dockerfile

We’ve discussed that default-but-editable Dockerfiles are sometimes preferable to bulky, 3rd party build packs. Especially when we know the environment well and can design efficient default Dockerfiles (like for Phoenix).

That said, detecting accurately can be a tricky job, so we may want to restrict our work here only to newer, known good configurations of common frameworks. A fallback could be general buildpacks when we don’t know what else to do.

Launching vetted services

Devs want an easy way to launch things like Redis, KeyDB, imgproxy, etc. We want to build smart tooling around this like we have with Postgres, but that’s a pretty big effort that ties in with our plans to change the orchestration API, drop nomad, etc.

Today, we hand people a git repo and say ‘clone and follow docs’. These examples tend to get outdated. They may refer to a command or feature removed from Fly, or something else changing with the config or upstream package.

To improve on this, we could:

  • Run integration tests that deploy projects and test basic functionality, like test flight does today
  • Build docker images to use via fly launch redis or another, similar command, to avoid ‘docker drift’ people might run into with tool version updates
  • Allow pointing at any repo with fly launch —from https://github.com/fly-apps/keydb
  • Make it possible to ‘eject’ the current app config of a running app

Documentation

These commands could be too mysterious if we don’t provide docs both on our site and inline while running them.

For the CLI, adding some output about ‘what happened’ and ‘next steps’ can go a long way. This could live in the repo rather than in flyctl.

For the docs site, we could revive the ability to render repo README files in our docs site. For each vetted service or runtime, we create a section. This was already setup before, but was too disorganised and led a lot of people astray with outdated examples.

It would be useful to think about ways to template docs in such a way that repeated commands like fly volumes create are not written directly into the markdown files. This was a problem when we removed fly init and broke a bunch of examples. This would be easy to do on the docs site, but would need some work to fit into the git repo approach.

Maybe we could keep the common templates in another repo, and ensure the final README is built on push.

1 Like
2

Missed one thing here. @shaun’s work on the Postgres stuff is awesome DX and should be looked at while working on this.

In particular, we could adopt the concept of ‘image releases’ so people who just want ‘the latest good thing’ can get it, and we can reasonably support issues without having to go deep into support at the level of Postgres. Knowing which image people are using will allow us to reference release notes and debug more easily.

Then if we want services (like redis) to graduate to a full fly redis command, we have a path that has already been walked to some distance.

3

I really like everything here, except this one thing I want to bikeshed.

I hate that fly launch generates weird names for all apps. I’d rather make it easy for people to name their app. And I’d sort of expect fly launch <name> to create an app with a name I like.

For databases, I think we want to make them a first class UX when fly <db> connect becomes a thing.

Also! I’d love for Fly launch to offer people a postgres when they’re launching a phoenix/rails app. It could either connect them to an existing one or even create up a new one.

Now that I said all this, I realized we’re talking about two things really.

  1. fly launch for building and deploying an app with a known framework
  2. fly launch for deploying someone else’s project

These might be separate things?

1 Like
4

The latest version of flyctl will prompt you for a name, like fly init did. We discussed picking a nicer name based on the current directory name plus some junk.

We’ve discussing using the existing launcher json templates to get stuff like redis out the door quicker. Are you saying you think it’s worth doing fly redis from the start rather than a sort of ‘beta’ launcher that works with less formalized services?

Offering a db launch is also on the cards, for sure. I think that could come sooner or later depending on what the answer to the above question is.

And yes, we’re talking about those two things separately.

#1 is about improving the way we implement scanners and how we maintain template files/repos.

#2 is about how much infrastructure we put in place to support deploying a ‘service’ like redis in a less cumbersome way than today, if not going all the way towards a solution like Postgres.

1 Like
5

I didn’t realize fly launch prompted me for a name. That’s pretty nice!

I think having fly launch do all the Postgres setup for Phoenix apps is very valuable. I don’t think that about Redis, for what it’s worth.

The neat stuff @shaun did for Postgres would apply to anything, though. The fly image update for Redis would be quite nice. I am very curious how far we can get Redis withouth= introducing fly redis. I don’t think it’s worth doing fly redis from the start.

The work you do here will get into our UI too. fly.io/launch/livebook is a one off, but anyone who wants a Launch on Fly.io button will need that kind of page to land on.

Overall I think it’s more important for fly launch working amazingly for Phoenix apps than for the CLI to launch arbitrary services.

6

So for service launchers, do we feel good about sticking with the existing launcher JSON spec for now?