Rails Litefs Tutorial Incomplete?

So I’ve gone through this tutorial Litefs · Fly Docs and managed to have a Rails app with SQLite almost successfully running.

The app was deployed but couldn’t load any of the example routes. I just had a posts scaffold but looking at the logs:

2023-03-24T14:07:31Z app[eac7b8bc] jnb [info][6672a100-227c-4d0a-8c9d-ca56d77c153a] ActionView::Template::Error (SQLite3::SQLException: no such table: posts):
2023-03-24T14:07:31Z app[eac7b8bc] jnb [info][6672a100-227c-4d0a-8c9d-ca56d77c153a]     3: <h1>Posts</h1>
2023-03-24T14:07:31Z app[eac7b8bc] jnb [info][6672a100-227c-4d0a-8c9d-ca56d77c153a]     4:
2023-03-24T14:07:31Z app[eac7b8bc] jnb [info][6672a100-227c-4d0a-8c9d-ca56d77c153a]     5: <div id="posts">
2023-03-24T14:07:31Z app[eac7b8bc] jnb [info][6672a100-227c-4d0a-8c9d-ca56d77c153a]     6:   <% @posts.each do |post| %>
2023-03-24T14:07:31Z app[eac7b8bc] jnb [info][6672a100-227c-4d0a-8c9d-ca56d77c153a]     7:     <%= render post %>
2023-03-24T14:07:31Z app[eac7b8bc] jnb [info][6672a100-227c-4d0a-8c9d-ca56d77c153a]     8:     <p>
2023-03-24T14:07:31Z app[eac7b8bc] jnb [info][6672a100-227c-4d0a-8c9d-ca56d77c153a]     9:       <%= link_to "Show this post", post %>
2023-03-24T14:07:31Z app[eac7b8bc] jnb [info][6672a100-227c-4d0a-8c9d-ca56d77c153a]

I would have thought the pre-generated files would have included a step to setup and migrate the database on initial deploy? Or is the expectation that it would be added in manually?

Also the GitHub - rubys/fly.io-rails: Rails support for Fly-io gem doesn’t appear to include the DATABASE_URL step mentioned in the guide and no volumes appear to be created in Fly even though the guide states create two volumes with the same name and in two different regions.

Aside from that, the experience has been great so far (the gem is awesome for the value it provides). I would like to just resolve this last little snag.

Things have indeed changed since that guide was written. Sorry about that. I’ll add that to my list of things to revisit.

Adding the following to your fly.toml will address the creation/migration:

  release_command = "bin/rails db:prepare"

See Dockerfiles and fly.toml · Fly Docs for more details.

Volume setup is also done through fly.toml:

  DATABASE_URL = "sqlite3:///mnt/volume/production.sqlite3"

  source = "sqlite3_volume"
  destination = "/mnt/volume"

See: Dockerfiles and fly.toml · Fly Docs

1 Like

Thanks for the swift response. I added those suggestions and made some progress but it hangs on the final step:

--> This release will not be available until the release command succeeds.
	 Starting instance
	 Configuring virtual machine
	 Pulling container image
	 Unpacking image
	 Preparing kernel init
	 Configuring firecracker
	 Starting virtual machine
	 Starting init (commit: 8e03fa6)...
	 Preparing to run: `bin/rails db:prepare` as root
	 2023/03/26 15:32:50 listening on [fdaa:0:3a9a:a7b:c02c:a48f:b37d:2]:22 (DNS: [fdaa::3]:53)
	 Starting clean up.
	 Starting instance
	 Configuring virtual machine
	 Pulling container image
	 Unpacking image
	 Preparing kernel init
	 Configuring firecracker
	 Starting virtual machine
	 Starting init (commit: 8e03fa6)...
	 Preparing to run: `bin/rails db:prepare` as root
	 2023/03/26 15:32:50 listening on [fdaa:0:3a9a:a7b:c02c:a48f:b37d:2]:22 (DNS: [fdaa::3]:53)
	 W, [2023-03-26T15:32:52.171049 #522]  WARN -- : You are running SQLite in production, this is generally not recommended. You can disable this warning by setting "config.active_record.sqlite3_production_warning=false".
	 Starting clean up.
Running release task (running)... 🌏

@rubys So this hung for quite a long time until I had to manually terminate it. I noticed the tutorial mentioned something about VMs with Consul enabled never exit, making releases impossible is that what the issue is here? If so, can that tutorial be public? Isn’t it sort of blocked until there is a valid workaround?

Thanks again for writing it just trying to workout how feasible it is to get all of this up and running successfully at the moment.

Bit a long shot, but could it be that you run out of memory?
See Optimizing your deployment · Fly Docs to add swap to the instance.

I think the GitHub - rubys/fly.io-rails: Rails support for Fly-io gem already sets up the config to do that eg:

fly.rake file:

# commands used to deploy a Rails application
namespace :fly do
  # BUILD step:
  #  - changes to the filesystem made here DO get deployed
  #  - NO access to secrets, volumes, databases
  #  - Failures here prevent deployment
  task :build => 'assets:precompile'

  # SERVER step:
  #  - changes to the filesystem made here are deployed
  #  - full access to secrets, databases
  #  - failures here result in VM being stated, shutdown, and rolled back
  #    to last successful deploy (if any).
  task :server => :swapfile do
  rm_f 'tmp/pids/server.pid' if File.exist? 'tmp/pids/server.pid'
    sh "bin/rails server"

  # optional SWAPFILE task:
  #  - adjust fallocate size as needed
  #  - performance critical applications should scale memory to the
  #    point where swap is rarely used.  'fly scale help' for details.
  #  - disable by removing dependency on the :server task, thus:
  #        task :server do
  task :swapfile do
    sh 'fallocate -l 512M /swapfile'
    sh 'chmod 0600 /swapfile'
    sh 'mkswap /swapfile'
    sh 'echo 10 > /proc/sys/vm/swappiness'
    sh 'swapon /swapfile'

and the litefs.yml:

# The path to where the SQLite database will be accessed.
mount-dir: "/data"

# The path to where the underlying volume mount is.
data-dir: "/mnt/volume"

# Execute this subprocess once LiteFS connects to the cluster. 
exec: "bin/rails fly:server"

# These environment variables will be available in your Fly.io application.
# You must specify "experiement.enable_consul" for FLY_CONSUL_URL to be available.
  url: "${FLY_CONSUL_URL}"
  advertise-url: "http://${HOSTNAME}.vm.${FLY_APP_NAME}.internal:20202"
1 Like

@rubys Are you able to successfully deploy this tutorial with the same suggestions?

Not yet. I’ve asked the author of litefs how to proceed. I’ll post back here when I have results.

1 Like

@rubys Any update on this? I don’t mean to pester but I’ve got a screencast video which sort of hinges on the outcome of this :sweat_smile:. If there’s any other direction you can point me down to seeing this through I’d be grateful.

Some raw notes:

  • LiteFS 0.3.0 had some major changes
  • Default for Fly has changed from Nomad to Machines, and doesn’t yet have access to Consul
  • LiteFS 0.4.0 will have more major changes

When complete (assuming all goes to plan) the amount of configuration required will be dramatically reduced. That’s not yet the case now.

Not realizing this, I had scripted the configuration to make it easier. it probably doesn’t make sense to script the current state given that things are in flux. I’ll likely update that page to include the manual steps required.

@rubys Thanks for the response. Yup that would be great, I’m happy to run through the manual process if needed. Just want to get it up and running

@rubys Hey, any updates on the tutorial fixes? Thanks!

I’ve started working on it:

I’m going to see how close I can get to making the following “just work”:

fly launch
bin/rails generate dockerfile --litefs
fly deploy

OK, I’ve updated the page: Litefs · Fly Docs

This is so new that I don’t know what works and what doesn’t work, other than that exact scenario described on the page has worked at least one time for me.

If anybody wants to try it and has feedback, consider opening a new topic here.

1 Like

Works beautifully. One thing that the tutorial is missing is adding <%= ENV['DATABASE_URL'] %> to the database.yml file for the production group.

So should be:

  <<: *default
  database: <%= ENV['DATABASE_URL'] %>

Thank you for updating and sorting this out, really appreciate it.

That shouldn’t be necessary.