Fly volumes getting permission denied

I’ve set up some volumes in anticipation of anchor scaling, but am still doing a lot of app-level tweaking. This is an nginx cache service, and I’m starting to get permission denied errors when I scale and try to connect (or reconnect) to these volumes. I never explicitly set ownership, and I’m not sure if I should start or if that’s the problem.

2021-06-25T17:49:10.912502265Z app[9044a498] sea [info] 2021/06/25 17:49:10 [crit] 557#557: opendir() "/data/nginx-cache/3" failed (13: Permission denied)

We automatically chown the mount destination of your volume with your docker image’s user and group. Meaning if you’re dropping privileges in your entrypoint, your volume’s mount destination might be owned by root:root.

You’ll probably have to adjust the permissions / ownership of the mount destination yourself if you’re doing that.

If not, can you show us your Dockerfile and relevant files?

Hey @jerome
I tried a to set some ownership, but still end up with permission errors. Here’s the relevant files (before I started messing with users): Dockerfile · GitHub

This is basically modifying the 5 Hour CDN article

It should be noted it all worked fine initially. Things got wonky when I decided to swap OpentResty in place of Nginx. OpenResty doesn’t create an nginx user for one thing, and I’m not sure what user account is active or what owner is assigned. Moving between nginx & OpenResty and deploying at different times may have made a mess of volume ownership.

Thanks for any help you can provide!

You can flyctl ssh console into a running instance and look at what the permissions look like.

It’s possible initially it created the /data/nginx-cache/3 (and /data/nginx-cache) with permissions that are out of reach for the user you’re currently running as.

You might have to chown -R nginx:nginx /data. I’m not sure that’s the correct user:gid though. Once you flyctl ssh console in there, you can cat /etc/passwd to see what users have been created. There might be a clue there.

@jerome I did some more debugging, then stripped things back to where I’m not specifying any users/ownership. I blew away the volumes and redeployed, and things are working. Not calling it fixed, but maybe mitigated for the time being.

Thanks for the pointers about flyctl ssh console - that’s a useful tool.