release_command not setting ENV?

,
1

I have a rails app. I just ran a migration that copied data from one column over to another. I tested extensively in development.

But now that I’ve run this on production my data got corrupted. Notably, this column is using the ActiveRecord encryption which requires that the ENV be set in order for the proper secret key to be used.

I’m really stumped as to why the data seems to be encrypted with a DIFFERENT key than is in my ENV (and I’m trying to figure out what this key is so I can decrypt this data…).

Is it definitely the case that Fly secrets get set as ENV when the release_command runs? I’ve not figured out an easy way to verify this.

If anyone from Fly can peek at my logs for my app “the-bot”, the string to search for is “Configuring active record encryption from environment”. This is outputted in my rails initialization process based on the presence of certain ENV keys.

I’ve not yet been able to find how I search logs from a few hours ago. I’m still looking…

2

Try the following command:

fly console

And once you get to the rails console, type in:

ENV['RAILS_MASTER_KEY']

This should be the value that is used for the release command.

3

I did check that already and environment variables are present in console. So can I be certain that this will reflect what’s there for the release command?

4

Yes. Feel free to unset the secret: fly secrets unset · Fly Docs . I believe that will cause the release command to fail with a message that the secret was not set.

Instructions on how to set the secret again (for Linux/Mac and Windows/Powershell) can be found here:

5

What would help me do more forensics would be to review the logs at the time of that specific deploy/migrarion. It was earlier today and I read that the logs are stored for 48 hours, but how can I view them?

Online and with CLI I can see the last ~20 log lines but I can’t figure out how to look back a few hours.