Websocket connection issues

Questions / Help
1

I’m having trouble migrating an express-ws node.js app over from another hosting provider (Glitch.com)

The app acts as a server for a game which uses UE4’s WebSocket Module to open a websocket connection to the node.js server.

However at the moment, on attempting to connect the game client to the fly.io version of the server:

  1. The client’s OnConnected function fires
  2. The client then attempts to send a message to the server (a custom “authentication” payload)
  3. However it immediately executes OnClosed implying that a the other end asked to close the connection (sometimes with code “8943” sometimes with code “0”, but always with “Peer did not specify a reason for initiating the closing”.)
  4. The client retries to connect a few times, but always gets the same result until it times out.
  5. Watching the server logs, at no point do I seem to see any acknowledgement that a connection attempt was even made.

I’ve tried using this to test if I can connect to the server and it seems to actually work fine. When I “connect”, I see the relevant log output on the server indicating a new connection, and if I manually follow up with a well-formatted “authentication payload”, I see the relevant log output appear there as well as the correct response payload.

Is there a reason that fly.io might reject the attempted websocket connection from my UE4 client? On Glitch I remember having similar issues until I ensured that a “User-Agent” was added to the “UpgradeHeaders” when creating the WebSocket object before requesting a connection. Perhaps there is something similar that I’m missing in this instance?

2

There shouldn’t be any requirements in particular for websockets to work, as long as your instances accept them.

Now I’m wondering if there’s something going on with the UE4 websocket client. Do you know if it uses an outdated HTTP version for these requests? It’s possible our proxy is rejecting the payloads somehow (though we don’t parse them).

Does the client stay connected if you don’t send any payloads?

3

I’ve had a skim through the UE4 source & searched online for any clues as to what HTTP version UE4 uses but haven’t found any answers so far. I’m not particularly familiar with HTTP versions so there may be a way of checking this which I’m missing.

I’ve also tried not sending any payloads on connect, but the connection still closes immediately.

4

Is there a way for us to reproduce this? If you could provide a minimal failing example and some instructions, I’m sure we could figure it out!

5

That would be super helpful! I can try to put together stripped down versions of the relevant parts of the client & server & some instructions, but it may take a little while tbh!

6

I’m pretty sure this is going to be a client problem. I assume any simple UE4 websockets client would reproduce the issue. Maybe you can confirm that.

7

I looked into doing this myself but it’s quite a large endeavour!

If you can add headers to the initial websocket upgrade request, can you add flyio-debug: doit? We can probably find relevant logs internally if you do that.

8

Ok, I’ve put a minimal version of the server code on Glitch here as well as a minimal UE4 client here. If you are able to get that running (you’ll need git LFS, Visual Studio / XCode & UE4 version 4.25… though really don’t worry too much if getting the environment all set up takes too much effort, it can be a bit of a nightmare!)

In the mean time, I’ll try adding that header so see if you can see anything in the logs at your end as well.

At some point I’ll try to get the minimal server code deployed to Fly as well though I’m still getting up to speed a little with how all this stuff works!

9

Added the header (I think?!). Just tried “connecting” a couple of times after recompiling, so hopefully something will show in the logs at your end?

I saw this in the Fly server logs but unsure if it’s relevant? I don’t recognise the /fuN3 url, this message only showed up once & doesn’t seem to show up in relation to my connection attempts from UE4…

error.message="HTTP host header / pseudoheader is missing" 2022-06-27T18:27:11Z proxy fra [error]error.code=3003 request.method="GET" request.url="/fuN3" request.id="01G6K6NRXV2C3HD1SJZ23JC2GV-fra" response.status=400

Also came across this link which I’m guessing is what you were referring to wrt to UE4 potentially using an “outdated HTTP version”? It may be trying to use the HTTP/1.1 Upgrade Mechanism which is “explicitly disallowed” by HTTP/2?

10

I see logs!

HTTP/1.1 is supported for a websocket upgrade.

I see the connection is upgraded but it’s being closed by either end (not sure which).

Trying to investigate further right now.

11

Can you enable more logs from your app?

I see your app is returning a 101 response (which is good), then we upgrade the request and response and start copying back and forth. It just ends nearly instantaneously.

12

Have added DEBUG=express:* & looks like there’s some extra info coming through from that when I try to connect now!

13

Aha!

Tried comparing connecting via the “WebSocket Test Page Client” & connecting via the game & it looks like there’s an extra / being added by UE4!

This is the app’s log when connecting from the test page client:

express:router dispatching GET /.websocket
express:router query  : /.websocket
express:router expressInit  : /.websocket

Which DOES work, but when connecting from UE4:

express:router dispatching GET //.websocket
express:router query  : //.websocket
express:router expressInit  : //.websocket

It doesn’t!

14

(This may be irrelevant, but it’s the closest I’ve got to finding a meaningful difference between how the connection is being made! It also strikes me as the sort of grey area where “maybe // is ok? Do the specs explicitly allow or disallow an extra /? Some clients might be fine, others might balk. Maybe a regex somewhere thinks it’s a comment?” sort of thing)

15

Can you try with the WS test page, using 2 slashes (//) to see if it works (and shows 2 slashes in the logs)?

Is it possible express can’t match it?

16

Only just had a chance to look into this again today. Seems like only / was being explicitly handled and suspect that on Glitch the // may have been automatically collapsed to / hence why it appeared to “just work”, so adding an additional explicit route fixed the issue!

app.ws('/',  require('./src/ws-controllers/ws-index'));
app.ws('//', require('./src/ws-controllers/ws-index'));

Adding our solution here in case anyone else stumbles across the same issue in the future!

17

Oh good.

It sounds like you might just need to remove a slash from the URL your UE4 client connects to. If it adds one automatically.

We do not mess with URLs at all :slight_smile:

18

Yeah, I had a little hunt through the UE4 source code the other day trying (and my own!) to work out where the extra slash was coming from but couldn’t find the code in question. Also have been trying to avoid needing to recompile the UE4 source code on this project since it’s quite a beast.