Is there a way to prepopulate a volume without launching the app? I’d like to copy up some images and their resized versions for an older cms-style site I’m trying to migrate.
Still haven’t found a good solution for this. It seems that scp is an “almost” working option as reported in a prior post. I was thinking I could deploy an app before transitioning to fly that was attached to a volume and include a custom script and use that script to pull the data from S3 but I wasn’t sure if you could access fly hosted environment variables in that case. Could you access fly secrets from a custom script via flyctl ssh console somehow? Maybe with -C? I’m trying to migrate from linode, the images and movies are about 9GB together.
The simplest way to do this is to run not-your-app, SSH to it, and run your custom script. The env vars should be available!
The ssh server now supports sftp. We haven’t taught our CLI how to send files over sftp yet, but it’s now possible.
1 Like
Thanks for getting back to me kurt. I was able to use flyctl ssh console and then execute rsync over ssh pulling from my existing Linode server (great service but I’m hoping to take a devops break). It was about 11GB and was pretty fast even though I didn’t run anything in parallel because I didn’t want to blow something up. I actually didn’t use env vars like I was planning previously because I just transferred via rsync+ssh instead of from S3. The env vars were available though as you said.
I had to install rsync into debian but that was expected from other posts I saw.
I was using a docker image based on buster but needed to run apt update before I could run apt install rsync.
I checked where my volume was mounted locally first. My volume was mounted as /app/data as my fly.toml file stated.
Then I ran the following command after setting src_port, src_user, src_host and src_path based on my preexisting server as well as dst_path based on where I had my volume mounted locally.
rsync -avz '-e ssh -psrc_port -lsrc_user' src_user@src_host:src_path dst_path
I used password authentication which is frowned upon so you might need to do something different to load up ssh keys to access the source server. For a one-off it worked for me.
I wanted to use rsync to speed things up for repeated runs for both staging and production until the final migration from my old server because it is still in use.
1 Like