Cache Hits in Metrics

[Zane_Milakovic]

I noticed in the metrics sections of the app, there is an HTTP Cache Ratio. I was curious what cache this was referring to and how to use it correctly.

I have an in-memory cache and Redis cache. I was curious if Fly had a cache itself?

Is there a header that needs set for this to report correctly?

The current query is -

sum(rate(fly_app_http_responses_count{app="showdown-labs",cache=~"^.+$"}[15s])) by (cache) /
      ignoring (cache) group_left sum(rate(fly_app_http_responses_count{app="showdown-labs",cache=~"^.+$"}[15s]))

[FrequentFlyer]

Aloha,

It must be to do with the [[statics]].
https://community.fly.io/t/first-look-static-asset-caching/

[Zane_Milakovic]

It looks like the cache metrics existed before [[statics]]. They were added to support it also later in the thread. Good thinking though.

I am just wondering if I can return back an x-cache: hit, miss header or something to ensure requests trigger on the graph correctly.

[FrequentFlyer]

Yeah, it could just be that parts were built out one after another.
Meaning, first setup the metrics back-end part of it, then allow this new feature to plug into metrics, so this new info can also be pulled.

That thread does mention this:

There’s a fly-cache-status: HIT (or MISS) in the responses.

Is this not what you mean?

[Zane_Milakovic]

Yeah, but that is fly inserting that based on [[statics]]. I was curious if there is a userland header I can return in my app that allows it to be picked up in metrics. Normally the fly-* is for fly purposes only.

[jerome]

[[statics]] has nothing to do with this. Well… we do add add proper headers for cache hits.

If your server responds with the correct headers, we’ll log these metrics for you.

The header to add to your responses is fly-cache-status: HIT|MISS|<whatever>.

[Zane_Milakovic]

Ok, awesome. I was tempted to not add this because of the fly-* start. I didn’t want to cause issues downstream. But will add this and see how it works.

Thank you!

[Zane_Milakovic]

@jerome there seems to be an issue. When I apply the “HIT”, it gets appended to a “MISS” statement.

First request to my server -

Second request to my server -

Is this the expected behavior?

It is registering the “HIT”. Because I visited the wrong page first (without headers set), I don’t know if it’s recording both “HIT” and “MISS”. Will try again later and try and validate.

[kurt]

I’m 99% sure we don’t add to the fly-cache-status header. That graph doesn’t parse the status, it just shows exactly what we got. So HIT means our proxy saw just the header HIT, for example.

How are you setting that header?

[Zane_Milakovic]

It is just some assignment.

	headers := w.Header()
	headers["access-control-allow-origin"] = []string{"*"}
	headers["content-disposition"] = []string{"inline"}
	headers["cache-control"] = []string{"public, max-age=" + pageCache.GetStoreTTL() + ", stale-while-revalidate=" + pageCache.GetStaleTTL() + ", stale-if-error=" + pageCache.GetStaleTTL()}
	headers["fly-cache-status"] = []string{"MISS"}
	headers["vary"] = []string{"content-encoding"}
	fmt.Fprint(w, buf.String())

Nothing that would append , MISS to the value.

I have an alternate block of code when I find an entry in Redis that will return “HIT” or “STALE”, depending on if it plans to rebuild the file after writing the output of the cached version.

That being said, locally, I don’t show the extra , MISS when running. It only shows the HIT or MISS once.

[kurt]

We just dug through the code and we never set fly-cache-status on proxied requests. We read it on the way through and update metrics, but there’s no code to set it!

Are you proxying that request yourself somehow? That looks a lot like what would happen with a proxy, for sure. It’s very confusing.

[Zane_Milakovic]

So I only have two apps. One is a Go net/http server that is very basic. And then a KeyDB instance for a cache.

I don’t have anything sitting in front of fly.io.

When I run the Docker image, or the app by itself locally, I only see one verb in the fly-cache-status. But when running on fly.io I see two.

I don’t know if it impacts anything. I am seeing cache results that appear correct. So it is all a little confusing.