I’m in a pickle trying to figure out my multiplayer game architecture on Fly. I currently have a setup of a game-gateway that acts as “load balancer” for game-server machines that are created on demand. I route requests, which are fully tcp right now, with this kind of setup:
app.get('*', (c) => {
const region = c.req.header('region') ?? c.req.query('region')
const instance = c.req.header('instance') ?? c.req.query('instance')
if (!region || !instance) {
throw new Error('Machine not found')
}
c.header('fly-replay', `region=${region};app=appname;instance=${instance};`)
return c.body('OK')
})
Now, running WebSockets I have not been able to get rid of network hiccups, and I’ve turned my head on the UDP solutions. I’ve built a POC solution using WebTransport locally, it works well when connecting using straight localhost addresses and no load balancing server in between, and with simple self/system assigned cert.
The problem I have is two sided; I do not understand the cert world and fly world enough to determine how to get this working on a production environment.
Firstly, I assume I cannot reuse the certs provided by Fly.io? I must create my own cert somehow, and that raises the question that I guess I must also roll out some specific cert renewal stuff as well.
Secondly, can I pipe the WebTransport traffic the same way than I do now with fly replay, or do I need some other handling there that is specific for WebTransport?
I don’t fully grok your question, but just as a (hopefully useful) foil for the conversation: could you expand on why you think you’d need to reuse Fly certs? Just let your external machine use TLS, and then communicate between your gateway and downstream game servers over the private network. The private network uses WireGuard, so is encrypted, but it is not TLS, so you don’t have to worry about certs or cert renewal.
And it requires a cert to be used and I’m not really that sure what kind of cert needs to be created and with what configuration. I am also getting more and more confused on how to do the communication between browser <=> gateway <=> actual game server instance via WebTransport.
I am not familiar with the tech, but I asked my AI provider this:
What is WebTransport, and does it specifically need certificates if my cloud host terminates TLS for me?
WebTransport is a new web API designed for low-latency, bidirectional communication between web applications and servers, built on HTTP/3 and the QUIC protocol. It can use certificates for secure connections, but if your cloud host terminates TLS, you may not need to provide your own certificates, as the host handles that aspect.
Take this with as much salt as you need, but for now, I personally would research on how to get that library configured in non-cert mode. Perhaps someone else will chime in, and say that the kind of cert required here, or the kind of transport, won’t work with the Fly proxy or the private network.
Can you find out if Fly-io’s proxy system can work with this?
Short answer: Yes—Fly Proxy terminates TLS by default and does not proxy QUIC/HTTP‑3/WebTransport for you, so to run WebTransport you must terminate QUIC/TLS in your app (or remove Fly’s TLS handler and expose UDP 443/TCP 443 yourself).
Key points from Fly docs and community:
Fly Proxy’s built‑in TLS handler terminates HTTPS and converts it to unencrypted TCP forwarded over their WireGuard mesh. That handler supports TLS 1.2/1.3 and lets Fly manage certs. (Docs: TLS termination by Fly Proxy / TLS Support.)
Fly Proxy is the routing layer and applies protocol handlers; if you keep the default TLS handler, QUIC/HTTP/3 (which requires UDP/QUIC + TLS) will be terminated at the edge and not forwarded as QUIC to your app, preventing end‑to‑end WebTransport.
Community posts and docs show a working approach for QUIC/HTTP/3 on Fly: omit Fly’s handlers (disable TLS termination), expose UDP 443 (and TCP 443) in fly.toml, and have your app terminate TLS/QUIC itself (install certs in the app). Users have run quic-go this way.
Fly also offers an option: remove the TLS handler so the platform forwards raw TCP/UDP to your app; then your app must listen on UDP 443 and handle QUIC/HTTP/3 and present the origin certificate. This requires privileged ports or configuration to bind low ports in the VM/container.
So my initial impulse may have been wrong; it looks like you should disable TLS at the edge, and then you’ll be free to do certs at your gateway yourself.
Yeah I’ve been trying to figure it out all day, still maybe waiting a Fly dev to comment on this if it can be done. I am really eager to know if I can utilize WebTransport like this on Fly. I must say I will be pretty dissapointed if it turns out to be that I cannot and have to look into alternatives (what they even could be). I just know that I need the UDP (with some sort of gateway server) on this project or else it will lag & stutter forever.
I think my penultimate post is actually the answer you need; WebTransport seems to be based on the protocol that that community member got working. Is it not worth a try?
I understand that the post was about a single WebTransport server and routing traffic directly to it, and it does seem that the user got it working… But I’m more worried about the routing via gateway similarly via fly-proxy setup, and how hard it would be to set up… I would like to know that before tinkering more into it, or even subscribing to Fly support
Hi there, cool that you have gotten it to work! Your site, however, gives me a cert error so I did not proceed further
Could you elaborate more on what kind of setup you have here on Fly? I am very curious about it, and if you have a gateway server before the “actual server”?
It’s a self-signed cert, but WT does work with Chrome and Safari. It could use ACME/LE to provision a cert with a little bit of effort.
The only proxy to my knowledge is Fly’s UDP networking layer. That this works at all is frankly amazing. Kudos to the team.
In theory Fly could support QUIC passthrough to applications that support it with its existing TLS. That would ease the deployment of native H3 or WebTransport applications on Fly.
I am most keen to know if you’ve looked into it. I’d start off with Traefik; it is a very good proxy system, and is available as a Docker image. I would not personally rely on the feature being integrated into the Fly proxy; I think we can take that as a “no” now, and at best “maybe” in the future. So, unless something on the platform prevents it, build it yourself. This is a Lego set so get building with it!
So do you think I would be able to use this to proxy things “manually” within the Fly private/internal network? Could this completely replace my current simple gateway app I would just need to tell the Traefik which machine instance to relay the tcp/udp message to?
So many questions, sorry… I am not that pro with actual networking configuring, would just like to get this solved so I can go back to game development stuff
Kind of stuck with this problem…
so get building with it!