Unfortunately even at v0.1.52 it still happens about 25-50% of the time.
This is sort of possible to work around when doing manual testing (where you can just delete the app and launch it again from scratch), but for any kind of automation (like github actions) or my own “flycd” Project (a wannabe argocd imitation for fly.io) it kind of kills the functionality unfortunately.