Hi,
I am wondering about best practices in limiting possible harm from developers using flyctl on their development machines.
So, I guess the first question would be, any tools or configurations that can help prevent long days?
And second, is it possible to allow deployments ONLY from github actions? That is my biggest concern at the moment…
You may find something here to limit fly operations: Access tokens · Fly Docs but it’s generally for CI/CD scoping.
One best practice is to create an API/script for thefly commands instead of calling it directly, eg a “deploy:dev” command that runs fly deploy -a app_dev -c ./path/to/app.dev.toml
I find that crazy.
Since any engineer can fly auth login, if they mistakingly run the wrong command on their laptop they can blow up the entire business. It’s as simple as choosing the wrong command in your bash history…
Without sane restrictions, I can’t see how an SME/enterprise with experienced infra engineers will be able to use fly.io. Amazon’s principle of “mechanisms over good intentions” makes perfect sense here.
No matter how many layers of safe guard of protection you have, a dev with highest permission can also make some silly mistake… See Amazon, cloud flare, and a bunch of other large corporations.
Humans make mistakes all the time.
Exactly! That is the point. You want to make it as error-proof as possible.
In big companies, the only way to trigger a deployment is from the CI/CD.
As an engineer, yes, you can commit a bug. yes, you can commit a catastrophic configuration change. But there is a way to make sure that every last bit is reviewed (not that it solves everything, but it helps).
Having a flyctl tool on your local system works around ALL of those measures.
Don’t you agree?
Sure if that works for you and your company, do whatever you can for protection.
But imo, there’s a balance. You can have 100% safety for fly, but if a senior engineer goes into the DB and accidentally deletes it … Oops (see Cloudflare and NetApp, eg).
I personally don’t worry about blowing up a machine on Fly. If I happen to destroy an app, it’s not the end of the world since relaunching it takes a few minutes at most.
Each person/company is entitled to do work their own way.
A balance is important. But good systems/software/interfaces/apis (the list goes on) are ones that is easy to get right and hard to misuse.
Engineers will make mistakes. You can only issue read-access to the DB, and I’m sure that some of these companies implemented those guardrails after the fact.
My entire goal for this discussion was to try and understand fly’s philosophy. I’ve got the answer I was looking for, even if I don’t like it 
What would a good solution look like for you here? (I haven’t used AWS in depth)
If an engineer can fly auth login but cannot deploy, what should they be able to do? What commands should they be allowed to run, that does not break production if they run the wrong command by accident?
(I don’t work on auth, just personal curiosity)
something that may be useful is creating a staging or development fly org, and having engineers be logged in to an account with only access to that
alternatively, yes, you can provision macaroons with specific permissions for local flyctl. that is more effort than having engineers fly auth login, but presumably AWS’s version of “you can’t deploy from CLI” is too
I haven’t given that enough though, but the first thing that comes to mind is to have a switch (in the UI?) that blocks all write operations from flyctl.
So developers can still see logs, check the status of deployments, configurations, etc. but they cannot deploy or scale, for example.
The challenge here would be to differentiate github actions using flyctl from a developer using it, but I’m sure this is a much smaller and more manageable problem.
IF a catastrophe happens, giving your devs their powers back is as easy as a single click in the UI.
I understand that the UI can be down, and you want developers to be able to keep working even if some components of the infra are down. A very specific fly command with a 2FA for “re-enabling” the write access from the flyctl tool for a configurable period of time is also reasonable.