It seems that it is likely an issue on my end, as I’m able to curl -6 fly.io in the terminal, but still does not load in my browser (FF). So I tried chrome, and things load as expected. This means that the issue is with FF with IPv6 (on my machine at least). So I pop open about:config, and disable IPv6 and can load things. This of course is not the optimal solution, as IPv6 is the future™, so I set ipv4 only for fly.io specific domains. This still has issues with other fly apps using custom domains, but is suitable enough for the short term.
I’m currently using a mac, and have disabled private IP relay (even though this only affects safari & mail).
I’ve tried changing DNS servers to see if that changes things, but alas it does not.
So, I’m just going to leave this as an unsolved mystery as I have a (somewhat janky) workaround.