NGINX is a very fast proxy server that has a built-in caching system — useful if you want a caching CDN for your users, or a way to cache objects inside Fly for fast and repeated access from your applications.
The first and simplest configuration is to use a set of ephemeral instances that based on GitHub - fly-apps/nginx: A fly app nginx config — this will load up each instance with NGINX, and the nginx.conf has a line to set the origin URL:
set $origin_url https://example.org;
We can modify this to the URL we want to proxy, create the app with fly apps create, update the app name in the fly.toml and we’re off! The configuration sets up the following cache configuration:
proxy_cache_path /tmp/nginx-cache levels=1:2 keys_zone=static:64m max_size=512m use_temp_path=off;
We’ve given it a few options as well:
/tmp/nginx-cache, tells NGINX where to hold the cached data.levelstells it how many directories deep to organise the data;keys_zonetells it how much memory to use for key & metadata storage.max_sizetells it how big the cache can grow on the disk. Removingmax_sizeis also an option — the cache will simply grow to fill the disk and be managed by NGINX after that.use_temp_path=offtells NGINX to put its temporary files in the same folder as the cache, instead of using a separate folder.
The first problem with this setup is that /tmp/nginx-cache is on the instance’s ephemeral storage — Fly provides each instance with 5GB of storage on the root disk that is not persisted, not even over restarts. This is going to lead to a lot of repeated origin fetches, so let’s fix it by adding a volume:
fly volumes create nginx_data --region maa
We can then update our fly.toml to mount the volume on /data:
[mount]
source = "nginx_data"
destination = "/data"
And update the nginx.conf to use that as the cache directory:
proxy_cache_path /data/nginx-cache levels=1:2 keys_zone=static:128m max_size=10g
Now we can expect much better cache ratios thanks to our large disk. You can also override how long items stay in the cache using proxy_cache_valid while also telling NGINX to revalidate them with conditional methods (that don’t download the entire object, just the headers) using proxy_cache_revalidate.
This is going great so far, but what if we want to distribute to the cache a bit over multiple servers or use more caching space than we have in a single volume? To do that, we could maintain a list of all the servers we have on each server, and route all requests for any particular URL to a single server, distributing all URLs randomly over the list of servers we have.
We’d do this by changing the origin in our nginx.conf to
proxy_pass http://nginx-nodes/;
and then define nginx-nodes as an upstream:
upstream nginx-nodes {
hash "$scheme://$host$request_uri" consistent;
server [fdaa:0:33b5:a7b:14bf:0:641d:2]:8081 max_fails=5 fail_timeout=1s;
server [fdaa:0:33b5:a7b:14bf:0:641d:3]:8081 max_fails=5 fail_timeout=1s;
...
}
The hash "$scheme://$host$request_uri" consistent; directive tells NGINX that we want to route the request to one of the servers on the list based on the hash of the $scheme://$host$request_uri pattern, which is mostly just the URL and host. The consistent keyword tells NGINX to use consistent hashing, which reduces wasted work when the list of servers is changed.
We need to define a new server listening on port 8081 that fetches the actual data, and move our proxy_cache directive there instead of the main server:
server {
listen [::]:8081;
listen 8081;
...
location / {
proxy_pass https://example.com/;
proxy_cache static;
}
}
This set is what the GitHub - fly-apps/nginx-cluster: A horizontally scalable NGINX caching cluster repo does, along with scripts to automatically update the list of servers every few seconds by querying the Fly DNS reflection APIs.
A caveat is that NGINX will only cache requests that return a
Cache-Controlor aExpiresheader, so you need to make sure your origin is returning one or both of them.
S3 & NGINX Caching
Setting up an NGINX cache is a great way to add a layer in front of S3 that reduces costs and improves performance. A couple of extra measures need to be taken to make it work, though.
Dealing with no headers
First, assuming we’re using public download URLs on a bucket that has objects publicly accessible, S3 does not return the Cache-Control or Expires headers, which means NGINX will not cache requests. One way to get around this is to add another proxy server inside NGINX that does nothing but proxy S3 requests with the Cache-Control header added. We can do this by pointing the proxy_pass directive to another sever running on 8082, like this:
proxy_pass http://127.0.0.1:8082/;
with the server defined as
server {
listen [::]:8082;
listen 8082;
location / {
proxy_pass https://example.s3.amazonaws.com/;
add_header Cache-Control "public, max-age=315360000";
}
}
Dealing with signed URLs
If the URLs we’re trying to cache are signed, which will happen if the bucket and its objects are not publicly accessible, we’ll need to run another tool like the GitHub - awslabs/aws-sigv4-proxy: This project signs and proxies HTTP requests with Sigv4 alongside our NGINX cluster. Running the configure signing proxy will allow setting the proxy_pass to localhost:nnnn if we’re running it locally as another process, or http://top1.nearest.of.signing-proxy-app.internal to send the request via the nearest running app instance. The proxy_set_header Host $host; is going to come in handy in this case.