Tokens for app "<my-app>":
ID NAME CREATED BY EXPIRES AT
<token-id> flyctl deploy token <my-email> <timestamp>
Then I attempt to revoke it:
fly tokens revoke -t <token-id>
It seems to complete successfully. However, when I list my tokens again:
fly tokens --app <my-app> list
Nothing changes.
Problem 1: I see the exact same output as before attempted revocation (token still listed).
Problem 2: I also went to the (new) Access Tokens dashboard page, and saw the token was still listed on this page. I manually revoke the token via the UI on this page. I refresh, and it seems to be gone from the list on this page. But when I list my tokens again with fly tokens list --app <my-app>, it’s still there!
I thought perhaps the fly tokens list command shows all tokens including revoked ones. But regardless, this seems problematic: listing via CLI has different behavior from listing via dashboard (given that the current state I observe is that CLI lists the token, but the dashboard listing is empty), and revoking via CLI has different behavior from revoking via dashboard (as revoking via dashboard actually removed it from the dashboard listing).
Also minor suggestion: The fly tokens list command should show additional columns such as scope (app or org), type (ssh or machine-exec), and readonly (boolean). Perhaps revocation status as well, if the list command shows revoked tokens (one theory as to the cause of one of my confusions above).
2. The org dropdown and token dashboard are confusing when displayed together
In the screenshot below, the selected org is my-org-1 (top-left), while the table displays all the tokens for my account, including for other orgs like my-org-2:
I think the right UX here is either the sidebar should be removed from the Access Tokens page, or the table should only show the tokens for the selected org.
Actually, the key vanishes from the dashboard when revoked but is still displayed in fly tokens list. That output should probably get a “REVOKED” column and the option to hide those.
That’s definitely not good! Are you able to confirm whether or not the token still works?
To add some detail here, we store tokens created via flyctl or the dashboard in our database (I think purely so we can list them). When you revoke a token, we should be marking it as revoked in this database.
But what actually stops the token from being usable is it being marked as revoked in another database we call “tkdb”, which is the service responsible for token creation+verification in our infrastructure. In other words, it could be showing up in the list despite having been revoked in practice.
If you’re able to confirm whether it works or not, that will potentially help us isolate the issue
The token revocation UX is seriously bad and I think the docs and CLI need changes. Insofar as bad UX and incorrect documentation can cause someone to think a token is revoked when it isn’t, this is a security issue:
The docs here are outdated, saying that the command is fly tokens revoke -t <token>, when in fact, the only command that works is fly tokens revoke <token>.
The non-working command above returns a status code of 0, misleading me into thinking that revocation was successful!
That said, fly tokens revoke <token> does actually revoke the token (tested with an ssh token).
However, fly tokens list --app and fly tokens list --org both continue showing me the tokens that I’ve revoked!
Additional UX feedback on CLI
The -s flag for fly tokens list seems useless. If used with -o, it can only be -s org. If used with -a, it doesn’t do anything.
fly tokens list (without flags) errors telling me to specify the -a flag. It doesn’t indicate that -o/--org is a valid alternative.
When creating tokens, the CLI does a bad job of distinguishing between token scope (app or org) and token kind (ssh or machine-exec). It lumps them all together! The command fly tokens create <x> takes org, ssh, machine-exec, and deploy. But ssh and machine-exec tokens are app-scoped, and imo they should instead be nested under a flag for fly tokens create app [--kind].
deploy tokens would more intuitively be called app tokens, given that they’re app-scoped tokens. And after all, org-scoped tokens are called org tokens.
Creating readonly org tokens is bad UX:
There’s no non-interactive way of creating them.
The command is fly tokens create readonly --name <name> (with interactive prompt), but would be way more intuitive as fly tokens create org -o <org> --readonly --name <name> given that it’s an org token.
FLY_API_TOKEN='invalid_token' fly ssh console -a my-app says “You must be authenticated to view this.” but it would be better to say “Invalid token”.
