RFC: have fly launch produce a lib/tasks/fly.rake

1

An all too common first time experience for Rails developers coming from other hosting providers is that they find out that their assets:precompile step needs to be run on the server instead of the build machine. They hop on community.fly.io and are told to modify both their Dockerfile and fly.toml files. Two files that they are not familiar with. This user experience is sub-optimal.

With this change, they can instead be told go to into their lib/tasks/fly.rake file and move => 'assets:precompile' from the :build step to the :server step.

Similarly, if they want to launch new services (sidekiq, anyone?), they can do so directly in the comfort of a language that they already are familiar with.

Here’s what the initial fly.rake would look like:

# commands used to deploy a Rails application
namespace :fly do
  # BUILD step:
  #  - changes to the fs make here DO get deployed
  #  - NO access to secrets, volumes, databases
  #  - Failures here prevent deployment
  task :build => 'assets:precompile'

  # RELEASE step:
  #  - changes to the fs make here are DISCARDED
  #  - access secrets, databases
  #  - Failures here prevent deployment
  task :release => 'db:migrate'

  # SERVER step:
  #  - changes to the fs make here are deployed
  #  - access secrets, databases
  #  - Failures here result in VM being stated, shutdown, and rollback
  #    to last successful VM.
  task :server do
    sh 'bin/rails server'
  end
end

Following are the changes to the Dockerfile and toml file that make this work:

Before, in Dockerfile:

RUN bundle exec rails assets:precompile

After, in Dockerfile:

RUN bin/rails fly:build

After, in lib/tasks/fly.rake:

task :build => 'assets:precompile'

Before, in fly.toml:

release_command = "bundle exec rails db:migrate"

After, in fly.toml:

release_command = "bin/rails fly:release"

After, in lib/tasks/fly.rake:

task :release => 'db:migrate'

Before, in fly.toml:

SERVER_COMMAND = "bundle exec puma"

After, in fly.toml:

SERVER_COMMAND = "bin/rails fly:server"

After, in lib/tasks/fly.rake:

task :server do
  sh 'bin/rails server'
end

Note: bin/rails server will run whatever web server you have in your Gemfile, and Rails generates a Gemfile including puma by default.

1 Like
2

I like that this simplifies the Rails ↔ Fly integration into just one file.

As it stands, we have to tell people in our docs stuff like this:

“To change how the server boots, change the SERVER_COMMAND in the [env] directive of the fly.toml file”

“To change the build command, look at this line in the Dockerfile”

In this new world, we’d tell people, “it’s all in the Rakefile!”. It’s self-evident and not surprising.

The use of a Rakefile isn’t too heavy either, so we don’t have to maintain a gem or worry about too much “magic” getting in between this interface and all the stuff Fly puts on a users system.

At first glance it did feel a little weird seeing the sever command in a Rakefile, but I quickly got over that when I ran mix phx.server to fire up ui-ex today.

Longer term it would be great to bake something like this into flyctl, but short term this feels like the best solution for smoothing things over for Rails devs. It’s very easy to throw this out when we come up with a better way.

3

I, like many others, have been looking to migrate my Heroku apps to Fly. Issues with ENV vars at build time have been by far the most painful. Luckily the community has had these issues before so I managed to figure out what needed to be done. But because of this change, all the docs and community answers are out of sync with instructions like “remove RUN bundle exec rails assets:precompile” row. :sweat_smile:

But the biggest issue is that most Rails apps require at least some secrets for asset precompilation. And you can’t have secrets without RAILS_MASTER_KEY. And as far as I can tell there’s no good way to expose it safely in the build step. So my current workaround is that I edited this rake file to

  task server: "assets:precompile" do
    sh "bin/rails server"
  end

Is there a better way? Should something like this be the default?

In a perfect world secrets would be accessible in the build step too or at least there would be a simple-ish way to do that.

4

For some values of most, perhaps. :slight_smile:

I’ll note that Rails itself is moving the other way, with things like import maps (now), and thing like GitHub - rails/propshaft: Deliver assets for Rails in the pipeline. But clearly we have to deal with the world as it is today.

As to your comments and questions:

  • Indeed, the docs are out of sync. Things are moving fast, and I hope to have most of the docs cleaned up over this holiday weekend in the US. The community answers all represent the point in time at which they are given, and hopefully you agree that moving this to the rakefile is a step forward.
  • Yes, the changes you made to the rake file are the recommended approach for applications that require access to secrets.
  • No, this is not the default. The downside to the change you made is that in the rare cases where your precompile fails, VMs will be brought up, down, and back up again.
  • As to what a perfect world would look like: that we are going to move very slowly on. Leaking secrets can lead to a lot of unhappy people. Fly provides a lot of flexibility as to where build steps can take place - including your own hardware! - and we need to work through the implications that balance flexibility without compromising safety.

So… you have got something that should work for you now (and, realistically, given how rarely assets:precompile fails, will not likely cause you problems). But please give us feedback on any problems you may encounter.

Welcome to fly! I hope to make things better for Rails developers like yourself.

1 Like
5

I actually take it all back, I’m very sorry! :sweat_smile: The defaults are great.

And just because someone might search this community I’ll add some more info, so they might be able to find this post.

The reason I had issues with asset precompilation is because of
has_one_attached :avatar, service: :cloudinary. The method calls validate_service_configuration on active_storage and since it can’t find it, it fallbacks to the default one, in my case being AWS. There’s no AWS config either so it raises Aws::Sigv4::Errors::MissingCredentialsError.

I got around it quite easily by adding

ENV AWS_ACCESS_KEY_ID=1
ENV AWS_SECRET_ACCESS_KEY=1

in Dockerfile right under

ENV SECRET_KEY_BASE 1

I don’t really need AWS for building assets so this now works perfectly.

Thanks again!

1 Like
6

I’ve adjusted the Dockerfile produced as follows:

# The following enable assets to precompile on the build server.  Adjust
# as necessary.  If no combination works for you, see:
# https://fly.io/docs/rails/getting-started/existing/#access-to-environment-variables-at-build-time
ENV SECRET_KEY_BASE 1
# ENV AWS_ACCESS_KEY_ID=1
# ENV AWS_SECRET_ACCESS_KEY=1

This will make a release of flyctl in the near future. Once it lands, I’ll update the documentation to suggest uncommenting out these lines if AWS is used by the config.

Thanks for following up!

1 Like
7

Awesome, thanks!

Not really related but another possible improvement to this auto generated files would be to detect package.json or yarn.lock or something and if none are detected, not include the node stuff in the Dockerfile and fly.toml. My rails 7 app, for example, works with importmaps so I don’t have any of this JS crap in the repo and no need for node :sweat_smile:

8

Already on my todo list! My book avoids the use of node, as do my personal apps - example.

1 Like