I have an app that I originally set up in the atl datacenter, and a database running in the dfw datacenter. I realized these were not in the same place, so deleted the app and re-created it in dfw under the same name.
Now it appears that my app will spin up in either of the two data centers. My app has been crashing due to some bugs, and when it is spun up after a crash it appears to switch back to the original data center.
Is this because I used the same name for the app, and there’s a bug in flyctl which surfaces when the same app name is used with different data centers?
Basic repo case: (not specifically tested, but this is what I think is happening)
Create an app in atl data center
Deploy
Delete the app
Create an app in the dfw data center with the same name as in step 1
Deploy the app
The app is in the dfw data center, as expected
Let it crash for some reason (in my case it was memory exhaustion)
Sometimes the app appears back in the original data center from step #1
In the logs below, we see the app in atl, then a deploy, then the app in dfw. I don’t have logs showing it jumping from dfw to atl, but I can definitively say that I didn’t modify the app config between the first and second part of the log. (This log is contiguous output from fly logs, not a cut and paste.)
From what I’ve seen, Fly moves apps depending on demand and capacity.
I’ve read other people saying you can pin an app to a region by mounting a volume to it there. So you would create a volume in the region you want the app to stay in, e.g in dfw, mount that to the app, and then in theory the app should not be moved from there.
But maybe there is now a better/official way to enforce the region.
Thanks - I was mainly trying to keep the database and app server in the same data center because of 1) performance and 2) data costs between app server and database. Do you know if there’s any documentation about exactly what counts as data? Is all data free within in the fly.IO network regardless of data center?
I also noticed that the app builder was in a different region than my app, meaning the docker build process has to traverse data centers. To keep build times optimal (I deploy several times a day, and want that feedback loop to be tight), I was hoping to keep it all in the same data center.
Lastly, I’ll just point out that it’s a little confusing/misleading that the preferred data center entry when spinning up a new app is in fact a preference, or recommendation, or hint, as opposed to a hard and fast rule.
I still think it’s possible that there’s a bug here because I used the same app name in different data centers at different times. I hope someone from fly reads this and can comment.
When you create an app, we default to giving it backup regions. This was a feature when we first launched that makes less sense now.
If you run fly regions list you should see a list of primary and backup regions. Since you’re running a database in dfw, you’ll want to run fly regions backups dfw to set the backup region the same as the primary region. Does that make sense?
Is there a help doc on this? I thought I read things fairly carefully, but I missed these details.
Also, looks like the command is fly regions backup dfw (no “S” on backup):
It appears to have worked swimmingly. Thank you!
➜ fly regions list
Region Pool:
dfw
Backup Region:
atl
ord
➜ fly regions backup dfw
Region Pool:
dfw
Backup Region:
So it looks like it removed the backup regions, which I assume is what’s desired.
As a followup question, will this ensure that my runner for app deployment also runs in dfw? If not, is there anything I can do to keep the deployment runner in dfw so the docker layers don’t have to move between data centers?
There’s not a great doc on backup regions, no. It’s a bit of a misfeature we’re working really hard to get away from.
Are you asking about the release_command runner? Release commands will run in the same regions an app process is restricted to. So I think it will do what you want!
I was talking about an app I have in my fly.io dashboard (fly-builder-red-darkness-7135) which is “Dead”, I assumed this was the runner that gets spun up when I fly deploy. But if this is in fact something old that I can delete, and the deployment VM gets spun up in the target region and then destroyed after, then I’m all good.
Thanks for the thorough responses here, I really appreciate it! As a Heroku (and ruby/rails) refugee, I am finding fly.io to be a wonderful new home for my Elixir projects.