Tigris Regions and x-tigris-regions header

[Chaika]

Hey!

I have both a request and a question about Tigris regions.

The regions page for Tigris mentions Tigris is deployed in Singapore. I have a server in Singapore which debug.fly.dev shows hitting SIN
=== Headers ===
Host: debug.fly.dev
Fly-Request-Id: 01J16MRRX4PZYF27K629PY04A9-sin
…
Fly-Region: sin

However in the Grafana Metrics Panel for Tigris I don’t see any requests hitting the sin location. It would be really nice if there was more information exposed in Tigris’s response headers. Edge location hit, storage location retrieved from, would be really nice for debugging. Based off the latency ~250ms even after a few requests to a 10kb object I assume I’m going somewhere else. I do ~1-2ms when ICMP pinging the Tigris endpoint directly.

Secondly, I forced my object to be in all regions via the x-tigris-regions header, and I see some curious information being returned sometimes

< x-tigris-regions: fra,syd,sjc,hkg,jnb,lhr,mad,iad,ord,gru,nrt
Sometimes without ‘sin’?
< x-tigris-regions: fra,gru,jnb,nrt,mad,hkg,iad_mr,ord,lhr,syd,sjc_mr
Sometimes with _mr?
sometimes sin comes back?
< x-tigris-regions: fra,sjc_mr,sin,gru,syd,iad,ord,jnb,lhr,nrt,mad

What’s with that, and what does _mr mean?

Thanks for your help!

[ovaistariq]

Hey, @Chaika, thanks for reporting this. We will debug what’s going on with the requests to your bucket in the SIN region.

Regarding the *_mr value reported in the x-tigris-regions header, that is something internal and should not have been reported in the header value. We will fix that.

[Chaika]

Thanks for fixing the _mr header, those look gone now!
Any response on

@Chaika

more information exposed in Tigris’s response headers. Edge location hit, storage location retrieved from, would be really nice for debugging.

Even just the edge location hit would be nice, somewhere in the headers where it’d be parsable (part of the server or a separate custom header like x-tigris-region). It’d be helpful at just understanding a bit of what’s going on and if the latency is reasonable.