Since I’m trying to deploy to a non-production server I changed it’s region to syd and tried again, that didn’t help. I tried deleting the builder again and that didn’t work either.
I did find this recent topic though talking about large images being a problem: Can't Push Images in the Morning
Mine that is failing is 312mb, and whenever I have this problem it’s always that one.