Hello, for development on my local machine, I have a ‘volumes’ folder that contains some text files I am storing that my program runs. How can I exclude this folder from being deployed? When I put it in the docker ignore and upload, flyio forgets about it completely and my app can’t read the text file being stored in that volume. Thank you.
I’m not sure I follow the issue, you want the empty folder to be deployed but not its contents and currently your dockerignore exluces the folder too?
I’m sorry for the confusion. I will explain what is going on in my app. When called, it is an express API that will save the input parameter into a text file stored in a flyio volume called url_list. So the format of that directory hosted on flyio would be volumes/url_list/urls.txt. On my local machine, to test the app, I created the exact structure of the directory, so I have volumes/url_list/urls.txt. Whenever I deploy changes, the local folder ‘volumes’, along with everything else in it, gets uploaded and altogether removes the data on the app’s volume.
Please let me know if I need to explain any more.
I think Im understanding now!
Can you share your fly.toml and .dockerignore (feel free to hide confidential things like app name if you’d like).
What I’m looking for is trying to understand if your .dockerignore is not enough to prevent your Fly volume being wiped by our image.
fly.toml:
app = “*******”
primary_region = “iad”
[http_service]
internal_port = 3000
force_https = true
auto_stop_machines = false
auto_start_machines = true
min_machines_running = 0
processes = [“app”]
.dockerignore:
/.git
/node_modules
.dockerignore
.env
Dockerfile
fly.toml
Whenever I add the path to the dockerignore, it gets removed from the volume on flyio and makes the file not findable.
@blonddev it doesntseem you setup your mounts section, your volume might not be attached to your machine at all
Can you give it a try?
With my current understanding of the issue, here are some suggestions
Mount the volume by adding it to your fly.toml
You can pick any path such as /app/volumes or /volumes, as long as your app knows where to find it. If you want to keep consistent with your local dev setup you probably need it to be mounted on /app/volumes (assuming your Dockerfile put things on the /app folder)
When to add the urls.txt?
You mentioned this file is important and needs to be setup on your machine so your express API works. When the volume is started it will be empty. You will either need to fly ssh console and write it manually or make your app write it when it boots. Also worth mentioning you’d need to do this manually whenever you clone your machine to scale horizontally.
Consider using environment variables
Volumes cost money and as you can see you’d need some manual work or work to be done on your app to make things run. My suggestion for this: just create an environment variable to list all URLs and put either on your fly.toml on fly secrets. You’d need to make your app being able to use this variable but after that you’re set and won’t have to pay for volumes at all.
The only reason why I chose to use volumes was the fact that it persists across deploys and restarts. Do environment variables do that?
Also, now lets say that the mount is setup like this:
[mounts]
source=“urls”
destination=“/volumes/urls”
If I have the folder on my local machine ‘/volumes/urls/urls.txt’, on deployment, will it erase the data on the flyio volume?
You can find an examples where you upload a folder/directory but not the files contained in it in the Rails .dockerignore file:
Adjust the directory name as needed. That being said, you undoubtedly want to put this directory on a volume if you want the contents to stick around between deploys.
ENV and secrets are read-only within the VM so if your goal is to write after the app starts, volumes will help more but if you need to scale to multiple machines it will be tricky, a database could suit better your needs.
You should add it to your .dockerignore. I believe the current behavior is that when you first mount a volume it will be empty (probably will have a lost+found folder inside) and I don’t think your local urls.txt will be added.
I will give that a go now and let you know.
1 Like
Alright, I’ll try the dockerignore right now.
1 Like
@lubien @rubys I believe everything is working as intended now. I appreciate you taking the time to help me. Have an excellent rest of the day!
1 Like
Glad it’s working, have a nice day!
This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.