App with multiple processes that are stuck in one region?

Questions / Help
1

I added a cron process to my fly.toml. I deployed it and now my app process is stuck on dfw instead of being deployed in iad.

  1. Added processes to fly.toml
[processes]
  app = ""
  cron = "cron -f"
  1. Ran deploy
fly deploy --remote-only

  1. Ran fly status and saw app and cron transferred to dfw instead of remaining on iad liked before adding the processes block.

  2. Ran set

fly regions set iad --group cron

Now my cron process is on iad. Ran the same command but with fly regions set iad --group app and my app is still on dfw instead of iad.

Ran:

fly regions list

Output

Region Pool:
iad
Backup Region:

I tried to scale all the processes to see if the app process would change.

Currently, app is 2 and cron is 1.

Ran:

fly scale count app=3 cron=1

It scaled the app up, but it’s still in dfw.

I tried to remove processes from fly.toml and now I can’t deploy.

Description = Deployment is running pending automatic promotion.

Screen Shot 2022-09-20 at 2.36.33 PM
Screen Shot 2022-09-20 at 2.36.33 PM2684×304 112 KB

Removing the processes from fly.toml and now it still thinks that there are processes running.

Error Your app has multiple process groups: app, cron. Please specify a process group to set the region.
1 Like
3

Are you still experiencing this deployment issue?

I took a quick look at the nomad allocs, and didn’t see any with that deployment description.

4

I’m having a similar issue where the region is totally ignored, it’s frustrating as it’s causing major performance issues. My database is in iad and the web processes keep booting up in scl??

fly regions list
Region Pool:
iad
Backup Region:

fly status

Instances
ID      	PROCESS  	VERSION	REGION	DESIRED	STATUS 	HEALTH CHECKS     	RESTARTS	CREATED
e3e47aef	streaming	19     	gru   	run    	running	1 total, 1 passing	0       	50m31s ago
9a559470	worker   	19     	scl   	run    	running	                  	0       	1h5m ago
1f137197	web      	19     	ord   	run    	running	1 total, 1 passing	0       	1h5m ago
5

Hi @tomm,

It’s not clear what you’ve tried, but starting with the simplest possible scenario: the regions need to be set for each process individually, e.g. fly regions set iad --group worker

6

Yes, it makes no difference :frowning: For my main web instance I was able to fix it by creating a volume that i’m not using in the correct region but worker processes continue to be spawned around the world.

7

Hi @tomm,

OK. We definitely have a bug with volumes + indeterminate regions.

There’s also something screwy in process groups if you start in one way and then iteratively change your app config. It looks like this may be the issue here. Would it be OK if we remove your volume in order to fix up this job?

Otherwise, you should be able to pin all your processes by giving them all volumes, but that’s probably not what you want to hear.

8

Yea, volume subculture should be unused now.

9

Sorry for the trickle, @tomm: can you delete the volume entirely, please?

10

How I fixed this was to just recreate everything, but in production, I can’t do that.

I ended up feeling like the platform was a nice idea but very unstable for now. It took awhile to get a reply on this thread too.

I did end up moving my infrastructure to another provider.

11

:white_check_mark: Deleted

Sidenote: I’m surprised it allowed me to delete a volume that was mounted to an instance in use.

12

@tomm Did you get up and running?

13

Amazingly yes, all processes are starting in the same region now