We’ve been having trouble lately managing deployments for our app. Twice in the past couple of weeks, we’ve noticed machines in strange states:
-
The first time we discovered our DB was performing poorly. When we investigated, we found that we had 2 machines in San Jose and one in Atlanta where our database is. (At least I think that’s what we had… the dashboard and CLI didn’t agree.) The machines were not running the same image, which caused additional strange behavior. We had never done anything to scale into San Jose, so I called it a fluke, killed those, and scaled ATL back up to 2.
-
A week or so later, we found that the two machines in ATL were running different versions of our app again, which again leads to very strange user-facing behavior.
We’re deploying with flyctl deploy --remote-only -c ./fly.toml
(via Github Actions), and we haven’t changed our config file in forever. Can anybody help me figure out what we’re doing wrong so we can keep this kind of thing from happening in the future?