So the key is really knowing which requests are going to write to your database, and are therefore replayable. If, for example, you only write in mutations, the job is a lot simpler!
I see a few options with Apollo Server. I’ll need to dig in more to give you some code to work with. But here’s the general idea.
We’d want to be able to hook into the framework in two places: at the start of the request, and in cases where write exceptions from the database are raised.
For #1, we want to be able to stop request completely and return a HTTP 409 response with the
Fly-Replay header. Ideally this happens as far up the stack as possible. For example, you could simply check if the body of the request contains
mutation and replay it in your primary region. This might be good enough to start out!
One way to do this with Apollo Server would be to switch to using it as express middleware. Here we could pass in a middleware before Apollo Server that literally checks for
mutation in the body. We could start here as an experiment.
Another way might be to create an Apollo Server plugin and hook into its requestDidStart event, or another event deeper in the stack should you need more context.
One more thing to think about is whether your UI will be fetching data immediately after it writes. If you do this, there’s a risk that the query will return stale data from the nearby replica. There are a few ways to deal with this, such as setting an expiring cookie to temporarily send queries to the primary region.
I hope this all makes sense. I’ll look at setting up an example with the express middleware and apollo server.