URGENT: Deployment in MAA/Sin

I am having issues deploying to MAA/SIN zone for few hours now. Any issue in those zones?

You can detach the terminal anytime without stopping the deployment

Monitoring Deployment

v69 is being deployed

79a995e4: sin pending

v69 failed - Failed due to unhealthy allocations - rolling back to job version 69

There’s was an issue earlier, but we caught it and fixed it.

Is this still a problem?

We’re seeing similar problems in Sydney and Hong Kong. Should all be cleared up in a few.

The problem continues at the same spot. Just FYI, its a socket io app. My other app is working fine, so I cannot take the risk of testing redeployment there.

It Deployed finally. But this is scary. What can we do if this reoccurs in future. It did not even rollback to the previous active version.

Tried to deploy my another app, and here’s what I get. Luckily it rolled back to previous version.

2021-09-11T01:05:27.232836873Z runner[25c36c47] maa [info] Pulling container image
2021-09-11T01:06:52.300030184Z runner[25c36c47] maa [info] Pull failed, retrying (attempt #0)
2021-09-11T01:08:01.959181998Z runner[25c36c47] maa [info] Pull failed, retrying (attempt #1)
2021-09-11T01:09:24.236114960Z runner[25c36c47] maa [info] Pull failed, retrying (attempt #2)
2021-09-11T01:09:24.236117780Z runner[25c36c47] maa [info] Pulling image failed

MAA is a new region and this appears like a network issue. If we can’t pull the image, then we definitely can’t deploy.

Apps with volumes don’t have the same deployment strategy as apps without volumes. It looks like it tried rolling back but couldn’t.

I agree that if you cant pull the image, you cannot deploy. But pushing the image to a repo is also part of deployment. We are not explicitly pushing the image. And before this, I had exactly same issue in SIN region.