Hi, I am evaluating using Tigris as a distributed storage layer for my application. I plan on storing data in multiple regions globally and will be frequently ‘updating’ objects (new put over already existing objects). I have a few questions about the specifics of Tigris’ multi-region consistency model.
What is the atomicity of a put operation to multiple regions? For example, I try to update an object in the iad and fra regions. For whatever reason, the put to fra fails. Could the iad put still be executed? Will the put request report a failure?
If the iad put still goes through, what happens if I then make a get request in the fra region with the x-tigris-cas: true header set? Would the request get routed to the iad region with the stored data that’s been more recently updated? Would Tigris eventually update the existing/outdated stored object in the fra region?
What is the default behavior for a get request with the x-tigris-cas: true header when there are multiple copies of data stored? Will it get routed to the closest region with the object stored?
What is the atomicity of a put operation to multiple regions? For example, I try to update an object in the iad and fra regions. For whatever reason, the put to fra fails. Could the iad put still be executed? Will the put request report a failure?
When you choose multiple regions for put operations, the operation is applied synchronously to the first region, while the remaining regions are processed asynchronously for performance reasons. For example, if you select iad and fra, the operation will be applied to iad immediately, and the write for fra will be queued. Any failure in the fra region will be automatically retried by our asynchronous framework to ensure it succeeds as well.
For your points (2) and (3), please also check our documentation on consistency. We have simplified this process by deprecating the CAS header(still works if you’re using it) and introducing the X-Tigris-Consistent header. When you use this header during a put operation and then in subsequent operations, it ensures consistent reads by routing those reads to the same leader that handled the write operation, thereby providing strong consistency.
If the iad put still goes through, what happens if I then make a get request in the fra region with the x-tigris-cas: true header set? Would the request get routed to the iad region with the stored data that’s been more recently updated? Would Tigris eventually update the existing/outdated stored object in the fra region?
FRA will have the updated data replicated eventually. If you choose X-Tigris-Consistent during write and then later during read then it will make sure to read from iad. Here we have documented it.
What is the default behavior for a get request with the x-tigris-cas: true header when there are multiple copies of data stored? Will it get routed to the closest region with the object stored?
The idea is that system needs to know which leader has handled the write to offer consistent read and therefore either you can just use X-Tigris-Consistent during write and then later during read operation and let Tigris choose a leader for you automatically or you can combine it with region header to enforce the region that can be leader. In your case, are you using multiple copies on a request level or setting it on bucket level? If you set it on bucket level and then if you just pass X-Tigris-Consistent during reads then it will route to first region of that list which would be the default leader of your bucket.