[yul region] app deploys are down again (504 timeouts)

[benmask]

It seems that the machines api is down and/or experiencing network issues in the yul region at the very least. This is blocking deploys. This is not reflected in the status page: https://status.flyio.net/

## Running fly deploy with rolling update
request returned non-2xx status: 504: upstream request timeout

## Running fly deploy with bluegreen update
Error: failed to launch VM: request returned non-2xx status: 504: upstream request timeout

[mayailurus]

Hi… Perhaps this is due to the following?

@grahamcox82

I received an email overnight about the YUL region being deprecated and that we need to move our apps to another region.

[benmask]

I saw that but don’t believe it’s related. I started pulling the image from the fly mirror and the timeout disappeared. It seems to be an issue with fly timing out when pulling large images from dockerhub

[halfer]

@benmask

This is blocking deploys.

This is an interesting problem. If you need to deploy in a multi-region app and you can’t deploy in one region, how does one deal with a deploy? My first instinct would be to stop the machine in the problematic region, then send the new app to all other regions, and then redeploy in the bad region once it becomes available again.