In your fly.toml, add force_https=true to the relevant service (docs, example) and check if that helps?
Verify with curl --head http://myappname.fly.dev
Alternatively, you could add a HSTS header to responses from the web server (MDN).
Or, add a 301 permanent redirect from http://... to Location: https://... like how we do it.