Yep, if you set that app.fly.dev as a CNAME record, they will pull from that using https.
My issue was getting random 525s (when behind Cloudflare, which I wanted to do to get its geo-ip header). But it did work 99% of the time.
Not sure from @Loonb if they are consistently getting a 525 or randomly, like me. I solved it by handing TLS in-app but given all the changes to the proxy/routing since then, that may not be needed as I’ve not tried reverting back.