[Feature Request] Better API docs or an official Fly SDK

1

Hello!

Flyctl is great, no doubt about it. It makes managing many things easy. For some (admittedly niche) use-cases, however, the Fly API allows us to interact with our Fly resources with more precission, even from within our own codebases.

The issue is that currently, the API documentation available publicly here lacks a lot of descriptions and the model naming convention is all over the place. I realise this is nitpicking and it shouldn’t be a priority, but it would be nice if that Swagger could be updated to provide some description to some less-than-obvious endpoints and if the naming was a little more standardized.

Or, another option would be to have access to an SDK from Fly, perhaps that used in Flyctl.

2 Likes
2

I started work on a TypeScript based Fly SDK I’m calling FlyKit. Very very early, but happy to collab and talk through your needs.

I’m building this for myself right now.

I have experience with AWS CDK, so my design so far is largely influenced by CDK’s approach of building Constructs & Stacks.

I opted to wrap flyctl rather then talk directly to the API. My experience has been that flyctl is the more reliable and cutting edge interface to work with.

Here’s the general idea based on my Minio template here: GitHub - mikehostetler/fly-minio: Template for running Minio on Fly Machines v2

export class MyMinioStack extends FlyKit.Stack {
  minio: FlyKitMinio | null;
  props: MyMinioStackProps;

  constructor(props: MyMinioStackProps) {
    super();

    // TODO - move this into FlyKitMinio Props properly
    const app_props: FlyKitAppProps = {
      name: props.name,
      org: props.org,
      primary_region: props.primary_region,
      path: props.path,
    };
    this.minio = new FlyKitMinio({
      app_props,
      opts: {
        minio_version: props.minio_version,
        volume_name: props.volume_name,
      },
    });
  }
}

Then you’d just run it with:

$ flykit init

$ flykit launch

$ flykit destroy

This exists too - I looked into it but needed a bit more:

2 Likes
3

@devpikachu I agree, we haven’t spent a lot of time on perfecting the API schema, and descriptions are something we’d like to add. Is there a particular field or model that’s confusing?

@mhostetler I’m happy to see people building SDKs for Fly! What functionality are you missing in our API that made you use flyctl instead?

2 Likes
4

@mhostetler Thanks so much for those resources! They can provide a lot of inspiration around how to do SDKs / clients for Fly.

Having said that, we’d like to have a similar solution for Golang, as our services are/will be made with it. Using the resources above, coupled with the published Swagger, it’ll accelerate development, I’m sure.

As soon as there’s an MVP ready, I’ll post it on the forums for anyone to use / contribute to.

@ben-io Currently I’ve noticed the following things:

  • The possible response list is not exhaustive (probably unlikely for it to ever be), but I’ve encountered, for example, a 422 Unprocessable Entity on the POST /apps endpoint. Having checked both the URL, headers and request body, it’s unclear why that request is failing. I would’ve expected a 400 Bad Request, but I’ll speculate that the 422 is returned by some form of proxy? Either way, I have sent a support email in regards to this for further investigation.

  • For the CreateAppRequest model, it is unclear what the "network": string field is or what format it expects. I’d speculate it’s about IPs, but I’m not sure.

  • It’s unclear what the /machines/{machine_id}/(un)cordon endpoints do, as there doesn’t seem to be such a command in the Flyctl.

Thank you both for your replies!

1 Like
5

@devpikachu CDK uses a tool called JSII (Introduction - The jsii reference) to let other languages interact with JavaScript. I don’t know anything about it other than it exists - but if you’d like to talk about collaborating with that I’d be happy to explore it.

@ben-io I haven’t hit any snags yet. I’ve been working with Fly a long time (2018) and felt it would be faster & more stable to wrap the CLI tool. There’s always been a little bit of magic in the CLI that would be helpful to capture.

From a developer experience perspective, I know the CLI tool really well and have several other Fly templates I’ve created - so writing Stacks would be a matter of just wrapping those in my SDK - easy and fast.

6

To reply to your listed questions:

422 status code is returned when, for instance, an app with the same name already exists. I agree 400 bad request makes more sense here; I believe 422 is generated somewhere in our proxy code (for this endpoint, Machines API proxies to GraphQL).

I can’t immediately find documentation about this (and I’ll ask the team about it on Monday), but: by default, each Fly.io organisation has a private internal network that can’t be accessed by any other organsation.
If, on top of this, you would like some apps to not be able to access your organisation’s default private network, you can specify a name for a different network to put the apps in, and all apps with the same network name will be able to talk to each other.
Of course this is scoped to your organisation; the same network name used in two different organisations will generate separate networks.

“Cordoning” a machine refers to disabling its services, so our proxy won’t route requests to it. In flyctl this is used by blue/green deployments; one set of machines is started up with services disabled, and when they are all healthy, the services are enabled on the new machines and disabled on the old ones.

1 Like
7

by the way: flyctl is open source (github).
I’m not sure how usable it is as a library for Machines API - github.com/superfly/flyctl/api package includes type definitions and GraphQL API functions, but Machines API functions are part of the main flyctl package (as /flaps).
If not directly usable as a library, it should hopefully still be useful as a reference for writing your own code, or to be able to see where we use a given endpoint.

1 Like
8

@mhostetler That’s an awesome find! I had no idea such a tool existed, but at first glance it does seem promising.

I’m more than happy to collaborate on a Fly CDK. Let me know what’s your preferred channel of communication and we can kick off the initial conversations!

@lillian Thank you for the detailed response! I do find it a bit odd that it returned 422 for a duplicate app name, as the org I was testing against had only 2 apps, none of them named test-app. Regardless, I’ll do some more testing today and report the findings, if any.

As for drawing inspiration from flyctl, I found the repo after initially posting this thread and whilst it does offer great insight into how one might go about building their own, it wasn’t quite made for external use so some of the interfaces are more verbose than necessary. Of course, this doesn’t mean it can’t be used, only that one needs to pay greater attention when doing so.

To summarize, I find that going a similar route as @mhostetler does in his CDK, by writing it in TypeScript and then cross-compiling to a bunch of other platforms using jsii would very quickly produce usable CDKs for a lot of different projects, which I believe would be highly beneficial to the community as a whole.

It would also be easier to maintain, as it’d be a singular code-base, as per Amazon’s claims:

jsii allows code in any language to naturally interact with JavaScript classes. It is the technology that enables the AWS Cloud Development Kit to deliver polyglot libraries from a single codebase!

A class library written in TypeScript can be used in projects authored in TypeScript or Javascript (as usual), but also in C# (and other languages from the .NET family), Go, Java, Python, … More languages will be added in the future!

The only, let’s call it concern, I have so far about Mike’s approach is that by wrapping flyctl, you either bundle it in the CDK itself somehow, or it has to be installed as a dependency in the individual Dockerfiles of the projects, thus making Paketo builds less trivial for the languages that support it.

Furthermore, using jsii also introduces a dependency on having a Node runtime installed.

L.E.: Looking through the AWS CDK in a bit more detail, as I am not familiar with it, it seems the CDK is more oriented towards an Infrastructure as Code approach, which is not my primary goal for the SDK. Whilst the SDK would be usable as an Infrastructure as Code engine, it wouldn’t be its primary purpose, but rather, to allow interacting with Fly in a programmatic manner.

Take the following example:

Suppose we have an application test-app with 3 machines all deployed in fra. These 3 machines work together as a cluster, communicating with eachother.

Let’s say we want this app to scale to zero when not in use, but because this is a cluster and the machines depend on eachother to achieve consistency, we can’t scale the machines independently but rather, all 3 must be turned off or on as a singular entity.

With the current fly.toml configuration, this is, as far as I’m aware, not possible. The machines would be turned off one after the other, and when the proxy comes to turn them on, it will only do so for 1 of the machines. Whilst the other 2 could be coerced into turning on using the proxy, it’s not a great experience and could lead to errors / data loss.

Having an SDK would allow for custom setups where when 1 machine is booted up by the proxy, trough configuration or similar it could detect that it needs to turn on 2 other machines for the cluster to function, doing so with less delay and in a more deterministic manner, perhaps even queueing connections until the cluster is up, similar to how the proxy does it when it’s waiting for a machine to come online.

Similarly, when scaling to zero, we could leverage the SDK to turn off the other 2 machines in the cluster so that they don’t error out trying to communicate with the now turned off 3rd machine.

I hope the example above made sense.

9

As promised, here’s the repo for the (very) early version of the Golang SDK:

https://github.com/omnilium/fly-sdk

Godoc:

https://pkg.go.dev/github.com/omnilium/fly-sdk#section-readme

Opinions & contributions welcome!

1 Like
10

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.