This is likely our slow service discovery (also related to 502s). When we get a connection to your app, it’ll try the VMs our edge node knows about. During a deploy, it “knows about” the soon-to-be-gone VMs for longer than it should, so I think what’s happening is that it tries the request, the request fails, and then it retries on the newly created VM.
@jerome is working on this, it should be better.