How to set up session affinity with nginx for socket.io

[echoi]

May be a duplicate but none of the related posts seems to have an answer.

I’m having “Session Unknown” issue with socket.io, and looking into setting up a custom load balancer with session affinity to resolve this. However I’ve got no idea where to start because of my lack of knowledge on how Fly.io’s public load balancer works.

1. Can I customize the default fly.io load balancer to achieve this? adding a custom hashing logic (ip based) to the default load balancer perhaps?

2. If not, is it recommended that I setup a single dedicated nginx instance per region (which would likely get all the trafic from that region) as a secondary load balancer to my private machines?

3, Or should I be setting up nginx for every machine to proxy with each other? (between vms)

each of the strategy makes sense in my head but I need some starting guidance on how to achieve them in fly.io (ex. how do I dynamically proxy between machines when they are automatically scaled up and down)

Thanks in advance and much appreciate any help!

[rubys]

TL;DR: you might not even need nginx, fly-replay will likely do what you want.

This may turn out to be way simpler than you think. I have an application where requests have affinity - the details are quite different, but the result is the same. But first, let me back up.

Request come into the fly network and are handled by one of our edge servers. We have no control over where the come in. We route those requests to the nearest server that you have deployed your application. The result is both more random and more complicated than this, but this is the effective result.

Now to answer your first question, no you can’t currently customize the default fly.io load balancer to do what you describe. This is on the todo list, but I have no insight as to when that might be available and if the final result will meet your needs.

So now, a request comes in and it might be to the wrong server. All you need to do is to respond back with a fly-replay and our proxy will then forward it to the right place. This could be another application, another instance of the same application in a different region, or even a specific machine.

If this works for you… you are done! You haven’t mentioned what language and framework you are using but many frameworks allow you to define middleware that is run on every request. Have it check for a session/cookie/whatever and respond with a fly-replay
header if the request needs to be handled elsewhere. In my case, I’m already running nginx on every machine, so I handle this there.

The only caution is that this is limited to requests that have a payload of less than a megabyte. In your case, it looks like you are only looking to set up a socket so this is not likely a concern - the size of the data sent across the socket once established is not an issue, it is only the original HTTP request that matters. In my application, I successfully replay requests to set up websockets all the time.

My application does have a need to handle uploading of files that may be larger than one megabyte, and I handle that by doing a reverse proxy in nginx, and routing it using the .internal address. This also works.

[echoi]

You are a hero.

I’m guessing I need to send the machine id to the client on the initial handshake so i can target it with replay’s instance field?

I’m gonna try this out tomorrow.

Thanks a ton!

[rubys]

If your client is a browser, perhaps set it in a cookie. Your server can determine its own machine ID by looking at the FLY_MACHINE_ID environment variable.

If your client is an app, there is a fly prefer region header that might help, but its granularity is only at the region level.

[echoi]

Worked like a charm!

For anyone who stumbles upon similar problems,

I followed this example, with a minor tweak where I had to pass the FLY_ALLOC_ID in a cookie (as @rubys suggested), since my app is pure client side javascript.

[simos]

Hey, i’m having the same issue, i serve an html page and save the machine id in a cookie and then i include it when i call:

const socket = io({
query: { fly_instance_id: getCookie(“mid”) },
extraHeaders: { “fly-force-instance-id”: getCookie(“mid”) }
});

i followed the example but still no luck. When only one machine is running everything works as expected but with two i get a mismatch console log with the expected ids but i get a lot of connections on the other server (the behaviour varies) and in the end i still get connection failed. Any ideas? Did you save the id in a cookie in a similar way?

[echoi]

Hi,

It would help if you posted your server side code here as well.
I’m no expert but can try to help you.

It does look like you should be in server side rendering environment, so you should not follow anything I posted above, but the example link above precisely.