503 using fly-machine-id cookie

[joeygennari]

Following the session affinity doc found here, I’m setting a response cookie (fly-machine-id) to the value in the request header with an expiration of 6 days. However if I scale out to different regions within those 6 days, and those original Fly.io machine id’s go down, any requests to my app with the old fly-machine-id will return a 503 error.

My assumption would be if a machine ID is no longer associated with the domain/app, that the load balancer would strip the cookie and return a new machine id. The client would be completely oblivious to change in the machine IDs, so without it, the only way to reach the site is to wait for the cookie expiration, or clear the cookie.

Here’s the behavior:

[joeygennari]

Looks like @khuezy hit this too:

Support for Sticky Session

This works great but there might be an edge case that fly should consider addressing: if a machine that was set on the user’s cookie is destroyed, the request from that user will hang indefinitely. It looks like Fly replay isn’t making sure that the machine exists before replaying the request. After about a minute or so, the request errors w/ 503. I’m guessing the Fly edge should also also expire the cookie, but that cookie key is user defined, so it’s not as straightforward.