Assemble Composite Applications with fly launch --attach

Fresh Produce
,
1

Applications are rarely a single code base. Never were.

A typical application has one or more of a relational database, an object storage bucket, and/or an in memory database. Perhaps add to this application monitoring, mail handler, a security layer. And more: log shipper, cron job, AI machines. The list goes on and on.

For items listed early in the previous paragraph, integration is generally a matter of a single command, for example fly postgres create or fly storage create. For items listed later it is a matter of finding the right set of instructions, running a handful of commands, and editing a configuration file. Let’s be honest here: when things go right this can be a matter of minutes, but all too often it is a matter of a half a day, or more.

We can do better. Early results from a direction we are exploring:

fly launch --from https://github.com/fly-apps/node-dictaphone
fly launch --attach --from https://github.com/rubys/cog-whisper
fly deploy

This demo is based off of the MDN Web Dictaphone, and comes in two flavors. Feel free to substitute “rails” for “node” in the first line. Just be sure that you are running flyctl v0.2.100 or later.

The first command pulls a codebase from Github and launches it complete with a Tigris bucket (for clips), a PostgreSQL database (for tracking) and Upstash Redis (for realtime broadcasts). The Rails version will also install and configure Sidekiq and deploy a release machine to handle migrations.

The result of the first command alone is a complete and standalone application that you can try out.

The second command adds a second codebase, and sets a single secret connecting the two. This second codebase is a complete application, and can have it own set of volumes, databases, secrets. Everything is controlled by the fly.toml that exists in that repository. Highlights of what this definition requests:

  • A beefy NVIDIA l40s CGU, capable of quickly running AI models
  • auto_stop_machines set to stop so that the machine only runs when there are requests to be processed.
  • Port 5000 to match replicate/cog conventions
  • Health check endpoint used for monitoring the status of the machine
  • A volume for storing models
  • A secret to set on the parent application

In short, you get a completely preconfigured Cog Whisper API application that you can put to use immediately. All you need to do is focus on your application logic.

Both the node and rails demos extract clips from Tigris/S3, pass them through a whisper AI transcoding function, and store the results in PostgreSQL database. Sounds complicated? It really is not - feel free to browse the code: node, and rails.

Run fly deploy and this is what you will see:

fly-launch-attach

Congratulations: you’ve just composed a composite application. cog-whisper is truly a resuable part - you can attach it to other applications that have similar requirements

And now that you are up and running, you are free to tweak the fly.toml in both applications, deploy to more regions, pull new versions as they become available. And everything you are used to, from load balancing, to machines stopping when not in use, to being able to control everything via an API “just works”.

Behind the scenes

Now that you’ve seen it work, let’s reveal how it works:

  • When you run fly launch --attach inside the directory of a deployed codebase, flyctl will extract the organization of the current codebase and use it as the default for the new codebase.
  • git clone of the new codebase will pull the new code into a new directory. The name of this defaults to ./fly/apps/${BASENAME} where basename is taken from your URL. If you would prefer a different directory, this can be overridden with an --into flag.
  • Name of new application is prefixed by the name of the “parent” application by default.
  • The new application is launched and given a private IPv6 address only, so it is not available on the public internet, just to apps in your organization and to those that VPN in using wireguard.
  • The fly.toml for the new application is scanned for secrets to be set in the “parent” application. An example:
[experimental.attached.secrets.export]
WHISPER_URL = "http://${FLYCAST_URL}/predictions/test"

Note the substitutable variable here. At the present time, it is the only variable that you can use, and it must be typed exactly as above. Over time we expect to provide more variables that can be used. And everything from environment variables to .env files could be potential sources.

Futures

This is clearly just the beginning. A few things we are exploring:

  • A marketplace for applications that are ready to be attached. This could start out as a simple documentation page, and expand to be so much more.
  • Dashboard support. One should be able to visualize and interact with your application as a whole.
  • Replacing the need for attached applications to be git repositories. A simple OCI image should suffice. Configuration information can be extracted from LABELs.

These are merely aspirational at this point; feedback on what would be useful and what might be missing from that list is welcome.

Postscript: no lock-in here!

  • If you are the type of person who prefers to roll up your sleeves and enter a handful of commands and edit a configuration file or two, you are welcome to continue to do so.
  • Applications require no modifications to either provide or consume another attached appliance. To the extent that they already do, applications can be run on your own laptop or on other cloud providers.
  • If you would prefer to use an external provider like Replicate, you can do that too.
9 Likes