Primary region is set to lax, but always placed on backup region (sjc)

I have a dedicated-cpu-2x app who’s primary region is lax, but for the last couple days it pops up on sjc most of the time. My database and other services are in lax, which is not super ideal. Could it be that lax is full?

Actually it looks like my app is not deploying at all anymore. :thinking: Might be related to this issue a few days ago: App deployment stuck in `pending` on lax - #6 by SebastianS

Looks like a bug where one of the vms thinks it’s running the newest version, but it’s definitely not:

Instances
ID      	PROCESS	VERSION	REGION	DESIRED	STATUS  	HEALTH CHECKS     	RESTARTS	CREATED
39c0aa36	app    	251 ⇡  	lax   	run    	running 	1 total, 1 passing	0       	3m36s ago
9fc4a89c	app    	251 ⇡  	sjc(B)	run    	running 	1 total, 1 passing	0       	34m18s ago
467a1bab	app    	248    	lax   	stop   	complete	1 total, 1 passing	0       	5m18s ago
68c122c7	app    	243    	lax   	stop   	complete	1 total, 1 passing	0       	40m31s ago
fb99e459	app    	242    	sjc(B)	stop   	complete	1 total, 1 passing	0       	2h29m ago
c9f0db6c	app    	241    	lax   	stop   	complete	1 total, 1 passing	0       	6h34m ago
982ef4d0	app    	240    	sjc(B)	stop   	complete	1 total, 1 passing	0       	10h43m ago
0cd9fd2d	app    	239    	lax   	stop   	complete	1 total, 1 passing	0       	10h47m ago
64173e7e	app    	238    	sjc(B)	stop   	complete	1 total, 1 passing	0       	12h35m ago

Looks like I solved it by scaling up and down vms until the stuck one in sjc disappeared. Not a fun experience on launch day…

Try running fly regions backup lax. If you look at fly regions list, you’ll likely see SJC listed as a backup region. We default to no backup regions these days, but a few months ago we’d add the next nearest regions when you created an ap.

I’ve just read up on the notes on backup regions in the docs. I think I had a bit of a misunderstanding what they are for. My understanding was that they would be only used in rare cases when somethings going on with lax to make sure my vms are not placed on the other side of the planet.

It seems like they kick in more frequent than expected. Disabled them for now :+1:t2: