Volumes - Storing and Accessing Files

Questions / Help
, , ,
1

Hi there,

(not sure I posted in the right channel/category, let me know)

There is something I don’t understand.

With the help of Fly Volumes overview · Fly Docs, Add volume storage to a Fly Launch app · Fly Docs and this tuto https://www.youtube.com/watch?v=E6UCy20ZKtA, I tried to setup volumes to store files.

Here is my fly.toml extract

[mounts]  
  source= "lightpress_data"
  destination = "/storage"

And storage.yml

test:
  service: Disk
  root: <%= Rails.root.join("tmp/storage") %>

local:
  service: Disk
  root: <%= Rails.root.join("storage") %>

The issue: I upload a file, it is being saved. I can access it, everything is ok. After a few minutes - I guess when the machine restarts - it is no longer accessible, I have this kind of error in the logs:

Errno::ENOENT (No such file or directory @ rb_sysopen - /rails/storage/d1/iy/d1iypphokiffgs0y9o7htur3kvgv)

I reupload the file, wait for ~5 minutes, and get again the same kind of error

Errno::ENOENT (No such file or directory @ rb_sysopen - /rails/storage/8k/lo/8klo4d4o05w1bpj2n7ynkevtz79f):

I tried modifying both fly.toml and storage.yml files (thinking the volume had to be mounted on /data mandatory for instance, but nothing seems to work).

So the questions are:
• What am I doing wrong?
• Are the files stored just temporary or longer term?
• What’s the best ay to make sure to access the files?

Thanks, really love Fly by the way!

5

To answer your question, what you really want here is:

root: "/storage"

This needs to match destination in the [mounts] section of your fly.toml.

In general, a better option is Tigris S3. If you are interested, the process is basically fly storage create and what you would put in your config/storage.yml would look like the following:

tigris:
  service: S3
  access_key_id: <%= ENV["AWS_ACCESS_KEY_ID"] %>
  secret_access_key: <%= ENV["AWS_SECRET_ACCESS_KEY"] %>
  endpoint: <%= ENV["AWS_ENDPOINT_URL_S3"] %>
  bucket: <%= ENV["BUCKET_NAME"] %>
7

Thanks for your reply!

So, just to confirm, I tried

test:
  service: Disk
  # root: <%= Rails.root.join("tmp/storage") %>
  root: "/storage"

First, and still got the issue, so also updated

local:
  service: Disk
  # root: <%= Rails.root.join("storage") %>
  root: "/storage"

Machine restarted and I still have the issue.

So far I have (an

test:
  service: Disk
  # root: <%= Rails.root.join("tmp/storage") %>
  root: "/storage"

local:
  service: Disk
  # root: <%= Rails.root.join("storage") %>
  root: "/storage"

(and it of course broke everything on local).

Could I have something like this?

test:
  service: Disk
  root: <%= Rails.root.join("tmp/storage") %>

development:
  service: Disk
  root: <%= Rails.root.join("storage") %>

production:
  service: Disk
  root: "/storage"

Will try tigris if this does not work

8

Before you keep shooting from the hips, try to understand what’s happening w/ the mount path of the destination. That will explain why it’s not working locally.

9

Shooting from the hips because a bit lost :wink:

I understand both mount destination and storage.yml need to lead to the same folder. I assume test: and local: are for my local environment.

What I don’t understand or am not sure here, is what line Fly will take. What about production: ?

10

It is not clear to me what you mean by that.

In your config/environments directory you should have a few files. Look for the following lines:

development.rb:  config.active_storage.service = :local
production.rb:  config.active_storage.service = :local
test.rb:  config.active_storage.service = :test

It is quite likely that you want your development and production to be handled differently. Local with Rails.root.join("tmp/storage") is a good choice for development. Change production to be something different, that could be :production, :storage, :volume, or (my suggestion) :tigris.

Now back in config/storage.yml, define a new section with a matching name. If you want S3 Trigris, go with what I suggested previously. If you want to place the data on a volume, the outermost name must match what you specified in config/environments/production.rb, and the root much match what you specified in your fly.toml.

11

This makes waaaay more sense, thanks!

I didn’t know this part:

development.rb:  config.active_storage.service = :local
production.rb:  config.active_storage.service = :local
test.rb:  config.active_storage.service = :test

I still have the issue, but can’t reproduce systematically, will dig deeper :slight_smile:

Thanks!