TCP Window Size Update not received / TCP fly.io service

[golangcaf]

I am running a TCP service on fly.io written in Golang.

Localhost:

• service runs fine i can connect using nc -vz localhost 9898
• echo ‘hi’ | nc localhost 9898 returns the right data

Fly.io

• service runs fine i can connect using nc -vz localhost 9898
• echo ‘hi’ | nc localhost 9898 returns nothing

Upon inspecting the traffic with Wireshark i can see the difference between the two is the TCP Window Size update being received on localhost but not on fly.io.

Localhost:

• client to fly: SYN window size 65kb
• fly to client: SYN, ACK window size 65kb
• client to fly: ACK window size 380kb
• fly to client: ACK window size 380kb (TCP WINDOW SIZE UPDATE)
• client to fly: PSH ACK sends data same window size
• client to fly: FIN ACK ends data same window size
• fly to client: ACK same window size
• fly to client: PSH ACK sends data same window size
• fly to client: FIN ACK sends data same window size

on fly

• client to fly: SYN window size 65kb
• fly to client: SYN, ACK window size 65kb
• client to fly: ACK window size 130kb
• fly to client: ACK window size 130kb (THIS DOES NOT HAPPEN)
• client to fly: PSH ACK sends data 130kb window size
• client to fly: FIN ACK ends data 130kb window size
• fly to client: ACK 65kb old window size
• fly to client: RST ACK 65kb old window size

I can share more of my code but it’s just writing a fixed string 60 bytes to the tcp sockets and works locally

This does not happen in another VPS hosting provider where there are no firewalls or other unknown network filtering

[chonk]

Could be MTU? Running Fly.io Apps On UDP and TCP · Fly Docs

[golangcaf]

I read about MTU but MTU and TCP window size are different things. And I can’t really customise them in Golang. Also as you can see the tcp window size update is coming from the client, so not sure maybe it’s something on my fly.toml that is not correct