That’s expected. Since a shared ipv4 is shared between multiple apps, fly-proxy needs a hostname or TLS SNI to route a request, since just the ipv4 address no longer uniquely maps to an app.
Does your network support ipv6? For example, can you reach ipv6.google.com?