My “running” app is unresponsive and deploys are not working in multiple regions (ord, dfw). They are just sitting in “pending”
Now “pending” for 30+ min – this deployment is a simple NodeJS API – health checks should have either passed or failed in just a few minutes.
We have been working on resolving a hardware failure in ORD affecting deployments: Fly.io Status - Host Instability In Chicago
1 Like
Looks like the issue with ORD has still not been fully resolved. I’m able to access my app from Canada but not from within the US. I’ve moved the app to a different region for now and it’s working now.
The hardware issues in ORD should be fully resolved, we’re still actively monitoring ORD as a follow-up but nothing appears to be out of the ordinary at this point. If you’re still experiencing any remaining issues could you describe them in more detail?
Deployments and logs are working correctly. However many users in the US (specifically around the Texas area) reported they were not able to access the app at all, and saw the following message:
However, I was able to access the app without issues from Canada. Some users who were affected, tried accessing the app through a VPN to change their location and that worked for them.
This was just ~40ish minutes ago. I’ve since changed the region to DFW and we haven’t encountered any issues since.
Thanks for the extra info. I did some investigation and though there was nothing definitive, the edge-proxy in dfw is under pretty heavy load and may have been affected by some intermittent network contention (causing dropped TCP packets) that might explain the problems your users experienced. We are working on adding edge capacity, but in any case the issue was local to dfw edge server and unrelated to the earlier problem in ord.
Update – after installing docker desktop and building locally I had no hanging/unresponsive deployment.
(note: Until that experiment, deploys to sea, ord, and dfw were still unresponsive today.)