Ah, the most likely problem here is just slow propagation of wireguard peers. CI builds create a new wireguard peer on each run and those can take 2-3 minutes to come up. They’re stored in .fly/config.yml, which doesn’t get reused between CI runs.
For CI, you should use the local Docker. Change your command to fly deploy --local-only to force that. This will save you a tremendous number of headaches.
It’s normal for a builder app to show as pending, you can ignore that.