I’ve been reading: https://fly.io/blog/websockets-and-fly/
Unless I missed something, this seems like a pretty standard WS setup.
If an app opens a socket with a client does Fly know which packets should go to which region and which instance in that region? What if the app instance is shut down due to de-scaling?
@pier yeah it’s pretty standard. Connections between clients and VMs are maintained until one of them closes it. When a VM needs to be terminated we’ll send a SIGINT and give several seconds for you to gracefully close connections. Clients will be routed to the next available VM when they try to reconnect and the dance continues.
Thanks. It would be great if you added that on the linked article or at least somewhere in the docs.
@pier We do need to write about our HTTP routing. In general it’s designed to work with no surprises, so the answer to this could have been “probably how you expect it to work”. Most people who use websockets don’t even know enough to ask this question!
So… just to reiterate on this for my future self and other readers.
Fly’s network layer “knows” when there’s an open WS connection between a client and a VM and that connection counts towards the concurrent connection limit (typically 25 connections).
When the WS connection is closed for any reason, this decreases the number of concurrent connections which might turn off the VM or not. If the client reconnects, it might land on the same VM or in another one.
So basically this means that no state related to WS should be stored in-memory on the VM.
For example, my use case is using Fly to send GC PubSub updates to browsers. When the VM receives a message from PubSub it will check whether it has a WS subscriber that could be interested in that particular message. If yes, relay it, if not ignore it.
Is this correct?
Correct. It’s best not to attach state to WS clients in your app, especially if you’re using autoscaling. Even without autoscaling, your VM could move around and lose its state.
Only way to counteract this is by persisting state to disk using our persistent volumes.
Either way, a WS connection for the same client might/will go to different VMs. You’ll need coordination if you want to keep a consistent state. Perhaps using a centralized store.
@jerome hello
I wanna build a simple real-time app(real time text editor) for me and my friend to edit code in real-time. So I need storage just for 1 file, to store text, it could be up to 300kb(usually 200-300 lines of text), my question is what is the best and fastest way to store this data on fly.io. I had read about persisted volumes, fly redis by upstash and LiteFS but cannot figure out what will be best and fastest storage provider for my use case?
The complete Odyssey with like 12k lines is like 700kB.
You could definitely store 300 lines of text in a DB with versions etc.