No need for a prerelease anymore, latest release works. I should mention that, at least for the first deployment, you need to use --force-machines with fly deploy as well as fly launch, to establish the app as a v2 one.
I flipped my organization to default to Apps v2 because I kept forgetting that flag.
fly orgs apps-v2 default-on <org-slug>