Multi tenant applications

I’m looking for a simple way to run multi tenant apps on fly. By multi tenancy I mean the following: I have the same docker container for each tenant, the only difference is configuration. So for each tenant I want to create an independent Application from the same docker image but with different env variables.

One stupid way I could imagine is: Create an app per tenant via the cli. Deploy it and configure regions for the tenant. This would mean I have to build the docker image multiple times etc…

Ideally, I’d like to just make an API call and clone an app with different settings but the same image and I’m done. No extra build steps etc. I had a look at your GraphQL API but a lot of the fields are not well documented. There are some Strings like e.g. “source” on some type without any explanation.

Would be nice if you can improve your API docs and add some example use cases like described above.

1 Like

It sounds like one app per customer is the way to go. You can deploy prebuilt images from flyctl directly with flyctl deploy -i image-name where image-name is either a publicly accessible docker image or an image tag on the local machine. You can then use either secrets or env variables in fly.toml files to configure each app.

An API to clone apps is interesting, we’ll think about that. In the meantime, we’ve made our GraphQL API available to folks but it’s still early and evolving. We’re hoping to have it documented with stability guarantees before long.

Let’s think this through.
A user signs up with my service and wants to get started.
For that to happen I need to deploy a container for them.
So they click a button and I need to deploy.
I don’t want to build a workaround to use your cli in that scenario.
I just want to make an API call to your service and, hopefully, after a few seconds, the pre-built image is deployed and I have my IP.
I guess your cli + the fly.toml will do nothing else but make API calls in the end so I assume it should be possible without the cli. I just don’t want to reverse engineer it.

That totally makes sense. Here’s roughly the queries and mutations you’ll need to get the job done along with a link to the flyctl source as a reference. This is certainly a lot of steps, but your use case is neat and we’d like to make it easier someday.

1. Get your organization ID (flyctl)

You’ll need this node id when creating the app

query {
  organization(slug: "your-org") {
    id
  }
}

2. Create App (flyctl)

This creates a pending app. Note the app id and name are the same right now, but it’s best to pass the value from the ID field when calling mutations.

Variables:

{
 "organizationId": "your-org-id", 
 "name": "app-name"
}

Mutation:

mutation($input: CreateAppInput!) {
  createApp(input: $input) {
    app {
      id
      name
    }
  }
}

3. Prepare Image (flyctl)

Before deploying a docker image you’ll need to prep it with the optimizeImage mutation. This generates a flattened rootfs file from the image in the background. Since it’s async you’ll need to poll until you get the READY status. These rootfs files are reused for identical images, so you typically only see a delay when the image changes.

Variables:


{
  "appId": "your-app",
  "image": "your/image:latest"
}

Mutation:

mutation($input: OptimizeImageInput!) {
  optimizeImage(input: $input) {
    status
  }
}

4. Deploy (flyctl)

All you need to deploy is the app id, image reference, and definition. The definition is the JSON representation of the fly.toml file.

This mutation returns immediately while the app deploys in the background.

Variables:

{
  "appId": "your-app",
  "image": "your/image:latest",
  "definition": {
    "env": {
      "LOG_LEVEL": "info",
      "DATABASE_URL": "postgres://...",
    },
    "services": [
      {
        "protocol": "tcp",
        "internal_port": 8080,
        "concurrency": {
          "soft_limit": 20,
          "hard_limit": 25
        },
        "ports": [
          {
            "port": "80",
            "handlers": ["http"]
          },
          {
            "port": "443",
            "handlers": ["http", "tls"]
          }
        ],
       "http_checks": [
          {
            "interval": 10000,
            "timeout": 2000,
            "method": "get",
            "protocol": "http",
            "path": "/status",
            "tls_skip_verify": false,
            "headers": {}
          }
        ],
      }
    ]
   }
}

Mutation

mutation($input: DeployImageInput!) {
  deployImage(input: $input) {
    release {
      id
    }
  }
}

5. Check Deployment Status (flyctl)

You’ll need to poll to monitor the deployment. You can use an empty string as the deployment id to query the most recent deployment. Look at inProgress and successful to determine if the deployment is complete. Note that sometimes it takes a few seconds for the deployment to be created after calling the deployImage mutation.

Variables:

{
  "appName": "your-app",
  "deploymentId": ""
}

Query:

query ($appName: String!, $deploymentId: ID!) {
   app(name: $appName) {
   	deploymentStatus(id: $deploymentId) {
   		id
   		inProgress
   		status
   		successful
   		description
   		version
   		desiredCount
   		placedCount
   		healthyCount
   		unhealthyCount
   	}
  }
}
2 Likes

Thanks for taking the time. This looks very helpful. Is it possible to use a private image and supply a pull secret somehow?

Not yet but it’s something on our radar. In the meantime, you can use the fly registry to store the source image in an app in the same org since those are protected by auth.

For example…

flyctl auth docker
docker tag source-image registry.fly.io/source-app-name:latest
docker push registry.fly.io/source-app-name:latest

Then use registry.fly.io/source-app-name:latest as the image when deploying to another app in the same org.

That would work, thanks.

1 Like

I understand that I need to optimize images before I can use them for apps. This is still the case, right?

What I’m not sure about is why do I need to pass an app ID to optimize an image?

Additional questions:
Is an image tied to an app ID?
Can I use an optimized image for multiple apps in the same org?
Can I use an optimized image for multiple apps across multiple orgs?

You do not need to optimize your images anymore. This changed a short while ago.

You can reuse an image within the same organization, I believe!

@Jens as Jerome said, we no longer need to optimize images before deploy. When VMs launch, image layers are cached and reused by any app from any org that references the same layer, just like docker’s cache.

Unfortunately not, but that restriction bit me twice this week… give me a few minutes and I’ll allow image refs to be shared across apps in the same org.

I’m still searching for the right way to implement multi tenancy using fly.

One org per tenant sounds good in terms of isolation. However, it’s currently not possible to setup billing for organisations programmatically.

Would it be possible to create multiple private networks without creating an organisation for each?

Otherwise, would it be possible to configure billing once for multiple orgs?

Any other ideas?

We do support multiple private networks, but haven’t exposed a way to create them yet. That’s probably the best way to do this, we’ll see if we can get network names exposed through the create app mutation.

It’s now later this week: when you create an app with GraphQL, you can now pass a network field. This will attach it to a network namespace for the org. You can use any string value that can be a subdomain here (ie: customer-network-1245).

It’s not available in flyctl yet, we have more work to do to make this a first class feature, but it will work for you what you need! The caveat is: Wireguard peering will only use the default network on your organization.

1 Like

Hey, Jens, can I bug you to say a bit more about what the architecture you’re thinking about is? What needs to be shared between the (isolated, per-your-customer) networks you’d be creating? Do we need to be able to peer between networks? Is that all-or-nothing or does it need to be configurable?

We’re noodling about this on Slack right now, is why I’m asking.

I want to create a custom network per tenant. For each tenant, I’d like to deploy multiple apps across the globe, all should see each other. No tenant should see internal services of other tenants. Some apps will be publicly exposed. Additionally, I’d like to allow tenants to join external services into their wireguard network. The idea is that tenants can have non-exposed services in their clouds which they can use from within the wireguard network.

A bit more context if that helps. I’d like to create an API mesh. Use any service you want to form a mesh of federated APIs over a secure network. Then have gateways on the edge to protect the APIs and cache responses.

As we’re already making it complicated. I was thinking about an easy way for tenants to add services to their mesh. The obvious choice would be to deploy to fly, too. In that case, would it be possible to allow apps from other orgs to join another internal network? I guess I wouldn’t be the only one, interested in this feature. If you find a generic enough solution to connect anything with anything plus a way to define access policies, that would enable a lot of use cases.

So there’s, in this design, no explicit need for tenant-A (on network-A) to ever talk to any Fly service on tenant-B’s network-B, or for that matter any other Fly network? There’s no bridging, at all, between networks? You just need it to be straightforward to deploy things into those separate networks, and to pass a network to flyctl wireguard (or its GQL mutations or whatever).

As I said above, it would be desirable to have a bridge between networks. I want to operate proprietary encapsulated software inside my own org. Each tenant should have their own network. However, if tenants want to run their own workloads on fly, they should be able to connect to “their” network if possible so that the edge gateways can securely talk to their apps while I can operate the gateway stack separately. I don’t want to wrap fly completely to allow them to schedule their own apps. My focus is on APIs alone. If that’s too complicated, it’s not a deal breaker. Connecting peers to custom networks is a good start.

Ok, let’s see how big a deal adding multi-network support to Fly WireGuard will be; I’m guessing it’s pretty easy. I’ll get back to you here.

1 Like

Hey @thomas, what’s the state of this?