Pretty sure Fly’s control plane is supposed to rid of VMs running older deploy versions. May be a bug on their end. What deployment strategy (ex: rolling, rolling_one, canary, bluegreen, immediate) are you using? Once upon a time, some folks had issues with bluegreen in particular; and others with using release_command + rolling deploys; but haven’t seen those reported lately.
I haven’t had to try this myself, but does fly vm stop <vm-id> -a <app-name> work (ref: other vm cmds)?