We have a new feature! Here’s the spoiler: You can now cache a fly-replay per cookie, or per header.
To explain why, we’ll talk a little about “sticky sessions”, or “session affinity” as it is sometimes known. If you aren’t familiar, it’s mostly in the name: You want to make sure that requests from user A go to one destination, and “stick” there (as in, future requests from user A continue to go to the same place). We have shared a few ways to do this in the past — we even have a blueprint just for sticky sessions! And that blueprint has just been updated with this new approach to the same idea.
There are two parts to a sticky session: Getting the session where it needs to go, and sticking it there.
For getting a session where it needs to go, a great tool is fly-replay. Your app receives a request, figures out what other app, machine, or region should handle that request, and informs the Fly Proxy what to do. It’s hard to beat your code looking at a request and telling us where it should go, so we tend to congregate around fly-replay as the best solution to this problem.
The sticky part is a little bit more involved. A heavy-handed option is just to replay every request, which does get the job done, but forces you to shape your design around that. (To really spell it out: your requests take longer, and use more of your resources).
One feature we released to address this is the fly-replay cache. Alongside a fly-replay response to our proxy, your app can instruct the proxy to cache that replay for a given path. To give an example: If you have a domain-per-customer, you can say that everything at https://customer-a.company.com/* should replay to the customer-a-app in your organization. This works well, as long as you have a URL structure that supports it.
For some apps, this doesn’t work. This wouldn’t work for us, for example, where everyone’s dashboard is at https://fly.io/dashboard. For these cases, it would be useful to cache on something else. Bonus: It would be really neat if you could cache on something that was already there, rather than adding some Fly.io-specific cookie.
That’s where this new feature slots in. If you’re replaying a customer’s request to a specific machine, there’s something about that request that your app uses to make that decision. In all likelihood, it’s going to be either a session cookie, or an authentication header, and each unique value is going to go to the same target every time. If you can teach our proxy that fact, we can continue to route future matching requests to the same target.
Here’s the config:
[http_service]
internal_port = 8080
# New:
[[http_service.http_options.replay_cache]]
path_prefix = "/"
ttl_seconds = 300
type = "cookie"
name = "session_id"
[[http_service.http_options.replay_cache]]
path_prefix = "/api"
ttl_seconds = 600
type = "header"
name = "Authorization"
For requests to this app that are replayed, if a session_id cookie is present, that replay will be cached for that session_id for five minutes by every proxy instance that processes it.
Longer matching paths win here, so if a request comes in to /api the above doesn’t apply. Instead, these replays will be cached for ten minutes based on their Authorization header.
Under the hood we hash the specified value, the replay cache settings, and the requested hostname together to form the “identity” to replay in the future. This means if a request comes through a different domain, or you change your cache settings, your users won’t get an errant replay. It also means we don’t have a big map of your API keys hanging around, which is nice for everyone’s peace of mind.
As always, the documentation has been updated to suit:
To deploy with this new configuration, ensure your flyctl is upgraded through v0.3.202, which includes support for the new fly.toml keys.