How are you managing cert files with Fly?

I don’t mean certs for HTTPs but for third party services (eg: databases).

Typically I use env vars for passwords and DB URLs, but with cert files there’s always issues with line breaks and how to parse the strings.

Would be great if fly secrets set could use a .crt file (or any text file really) as an input for the value of the secret.

Of course, deploying the physical cert file from the local machine is always an option. Not sure how people with CI, Github, etc, manage this though.

2 Likes

base64 everything! :smiley:

That’s actually what I’ve had to do a couple of times, really. But yeah, not ideal, not easy to manage. But it does encode the line breaks correctly, so when it works it works.

I wonder if there’s a secrets service (like Hashicorp Vault) one could mount as a filesystem, that would be interesting for this kind of thing.

2 Likes

Google Cloud Secrets does this.

You can either mount a secret in the filesystem in a certain path (at least in Google Cloud Run), or inject it as an env var. The neat thing is you can use string or files are the source of the secret.

Oh, that makes sense. Google Cloud also do a lot of key based secrets as part of the offering itself (think most service & API keys are files), so it makes sense they’ve got first class support for this.

I’ll put this in as a feature request… how are you managing right now, though? Just setting secrets with escaped newlines / base encoding / copying them into the image?

1 Like

Awesome!

Honestly, I rarely need to use certs and get by with simple env vars.

Recently I started using CockroachDB which requires SSL. Locally, I either read the cert file from the disk, or put a string with double quotes and new line \n chars in an .env file. Not sure what magic the double quotes do, but it just works.

In GC I use the secrets manager, which injects the cert as an env var.

I now wanted to try to run that app on Fly and was wondering how to solve this.

BTW @sudhir.j how do you use fly secrets import? I’ve tried to paste the contents of an .env file and it does nothing. And then I don’t know how to exit :rofl:

Would be also great it there was a way to import .env file as secrets!

Sorry I don’t follow: Why would one need a vault when fly secrets itself can be that vault?

We have a similar use to include private cert to handle TLS termination inside our fly app, and are just now looking at ways to do so.

Of course, we would simply let fly do TLS termination, but we need the hostname from the uri passed on to our fly app. It isn’t clear from the docs if that’s the case (I doubt it is the case) for non-HTTP TLS workloads (that is, plain old TLS over TCP).

I’d assume piping them in would work like cat stuff.env | fly secrets import but let me double check and get back to you. That pretty much works as a way to import a .env file.

1 Like

The vault works fine, it’s more the mechanism of providing the secret to the running application.

With certain formats (especially certification files with lots of newlines) it’s more convenient to have it as files on the disk instead of environment variables — mostly because to set them you’ll need to carefully do some encoding to make sure newlines work.

Thanks that did it!

cat .env | fly secrets import

Would be great if you added this to your secrets docs :slight_smile:

2 Likes

Oh, good. Will update the docs.

1 Like

Every few months I need to search for this answer.

1 Like

Just found out there’s also

flyctl secrets set FROM_A_FILE=- < file.txt

which should actually work much better for certificates or newline-y files.

5 Likes

That’s the exact use case which led to that useful but hidden feature :smiley:

1 Like

Is this correct?

Is a hyphen needed after the =?

Thanks.

The b64 trick (edit: and from_a_file) for crts could be made explicit in the secrets docs? Working with Fly Applications and Secrets and Fly Apps

Re: TLS termination by Fly: Can we expect it to send over URI it terminated non-HTTP TLS for (useful in case of certs issued for wildcard domains)? If so, how would it to do so? I see some bits about support for HAProxy’s PROXY protocol in the docs but that isn’t helpful for our use-case, if it only contains ip/ports. Public Networking

The hyphen means “STDIN until EOF”, a pipe or redirection sets the source and the hyphen instructs where to place it.

	echo "long text..." | flyctl secrets set LONG_TEXT=-
	flyctl secrets set FROM_A_FILE=- < file.txt
4 Likes

Where can I report a bug for fly secrets import?

If it isn’t a security vulnerability, then I guess on GitHub? See: `flyctl secrets import` should be able to ignore comments · Issue #575 · superfly/flyctl · GitHub

2 Likes

Thanks @ignoramous !

Setting secrets, and general configs, via files makes it much easier to migrate applications from Kubernetes environments also (what I had asked about here: What's the recommended way to load config is via a volume?), where an application is receiving secrets and config injection as files rather than environment variables.

A few years ago I used custom built deployment system that was limited to base64 encoding secrets as well, and there was no end to the problems that arose, such as people forgetting which secrets were base64 encoded, accidentally double base64ing, forgetting to unwrap on the other side, or oddities happening just like the issue @pier opened in `fly secrets import`should take into account values between double quotes · Issue #589 · superfly/flyctl · GitHub with quotes being included or not.

First class support for being able to set secrets as files and those files be tmpfs/shm securely mounted would be awesome in avoiding these mistakes and problems :grinning_face_with_smiling_eyes:

2 Likes