First look: static asset caching

The “statics” feature is not yet part of the metrics.

They should be though! Maybe there’s a quick fix, taking a look.

@OldhamMade cache metrics should now show up for statics.

1 Like

Wonderful! Thanks for the quick fix here.

I’m not seeing any HITs though, so I think maybe I’ve not configured the statics properly. Do files served via Statics have something like a fly-cache-statics: hit header so one can identify successes?

There’s a fly-cache-status: HIT (or MISS) in the responses.

1 Like

Confirmed, I’m seeing some HITs now. Thanks! :partying_face: :beers:

1 Like

This is fantastic! Nice side-effect is that it’s really cleaned up my fly logs so I only see document and data requests, and not requests for assets.

One thing I’d like is a way to control the cache headers. I’m thinking that I could provide some sort of glob/regex pattern in the config with the associated cache header for any matching resources. That would be really nice for my stuff. In my app, all files either rarely/never change or they have a hash in their filename. For the things that don’t, I could easily add a query string ?v=2 if I ever did need to change it. So for most of my stuff having an immutable setting for the browser cache would be great.

Also useful would be a 304 if the etag matches :slight_smile:

5 Likes

This is seriously cool. I understand the origin of this feature was mainly to handle the static assets for Rails/Django app. But having some means to purge the static asset by URL could also be cool. As this makes fly.io a CDN for both dynamic and static apps with programmatic control over cache purge of static assets.

What do you think?

Well thank you!

The way our static assets feature is designed, they don’t change over time unless there’s a deploy. When you deploy, it purges all previous static assets and puts in the new ones.

There’s no “caching” in the CDN sense. I don’t think a purge functionality would work in this case. We know beforehand, on deploy, all the files that we can serve and their paths. If you deploy a new version, we’ll swap everything.

2 Likes

@jerome how does the built-in static server work? Can I point it to a subdirectory of my app source code? Does it play nicely with this static asset serving?

Also, I made a really tiny HTTP server container: GitHub - ananthb/thttpd-container: Minimal HTTP container with a static thttpd binary.

It works like it’s described here: The statics section

It should be seamless. If you have any static files in your Docker image, you can use the [[statics]] section in your fly.toml to let our proxy serve these directly without hitting your VMs.

Imho, it’s a must have for customers of Fly to be able to set the cache-control headers for this, to control the caching behaviour of intermediate caches (e.g. ISP proxies) and clients (browsers).

This is another must have.
Browsers make conditional requests if they have an expired object in cache, and a 304 response by the server is much better than a 200 with the full payload.

@ kentcdodds’s ideas make sense: allow us to exercise granular control over the outgoing caching headers with URL path pattern matching.

I think [[statics]] can be quite useful for many use cases/app, but don’t agree with your statement here. Rate limiting, WAF, etc etc … CDNs like Cloudflare nowadays bring so much that is more than simple static caching and of real value for serious production (web) app. And I’m not even talking about CDN’s optimized H/2 and H/3 stacks or networks.

2 Likes

Yup, a Cache-Control override is super important here, that’s the only reason I wouldn’t use this at this point. Even in the Rails example the assets have hashed filenames and are immutable, so there’s no reason for them to be requested every time. Having the request just pass through to the app might actually be desirable if we can set the right cache headers - it’ll make the app much faster on subsequent requests than having the requests made again but hitting the Fly static server.

1 Like

Does it do subdirectories as well? Going from that example, it maps /app/public in the VM to the /public URL prefix. So if a file exists at /app/public/dir1/example.jpg, and I request https://app-name.fly.dev/public/dir1/example.jpg will it be served from the Fly proxy?

I ask because I have an app where files in the root of the mapping are served correctly but files from sub-directories hit my app instead.

I would also like to see a cache-control directive per static entry; it would make it much simpler to pass on things like immutable to the client.

Yes, it is recursive (I’m pretty sure, I’m not the one who worked on this and sometimes I forget).

@aaronpeters @jbergstroem @sudhir.j we do have a PR on our proxy for cache headers, but it’s gotten old and needs to be revisited. I might take a look and try to make it happen.

It’s recursive! The big caveat is that it won’t follow symlinks.

Any thoughts on doing an actual HTTP proxy? If requests hit via the HTTP router and have cache headers then cache them on the Fly frontend server itself? Think Google App Engine used to / still has this, it’s pretty cool.

More than thoughts; there’s a PR on our proxy that does this right now, managing a local cache on the edges populated with fetches from origin workers, with the http-cache-semantics crate to manage cache headers. It’s a (bad) running joke that I haven’t merged it yet. It’s a straightforward change but there’s a lot going on with our backend right now and I’m waiting for a quiet period to roll it out.

3 Likes

A few more questions about this, just to get the details right:

  • is there 304 / If-None-Match support? If a browser sends an etag that matches what’s in the cache is a 304 sent?
  • the outgoing bandwidth billed for static files is the same as the regular rate, right?
  • are the static files are only served from the regions where the app is enabled and currently running? Or the cache copied to all regions?
  • this works only for http handlers, right? I don’t see how it can possibly work if there’s only tls termination or tcp.

QUIC/http3 comes to mind