Full disclosure: I am very much not an expert at Elasticsearch or Dockerfiles!
I’m using the ELK Dockerfile setup recommended in this Fly.io article. We’ve had our Elasticsearch app running fine for a while but want to add a persisted volume to it so we don’t need to reindex our data when it crashes or the machine restarts (which is rare).
Here is our actual Dockerfile. The other setup scripts and entrypoints are unmodified from the article. Here is our fly.toml and elasticsearch.yml (where the volume path is specified).
A minute after the app starts, it crashes with this Elasticsearch error:
failed to obtain node locks, tried [/usr/share/elasticsearch/data]; maybe these locations are not writable or multiple nodes were started on the same data path?
I’ve tried a bunch of changes to the Dockerfile, and any change I make results in the same error (or the app just failing to launch at all). I went back & forth with ChatGPT quite a bit to try other ideas, but nothing has worked.
- Use
VOLUME /usr/share/elasticsearch/data
aftermkdir
- End the script with
USER 1000:0
orUSER elasticearch
orUSER elasticsearch:elasticsearch
orUSER elasticsearch:root
- Move the
USER
before theENTRYPOINT
- Remove any
chown
that touches the data dir, or try owning withelasticsearch
I’ve tried several permutations of the above options too, nothing has worked. I can briefly fly ssh console
into the app and check the ls -lg
permissions of the data dir:
drwxr-xr-x 3 root 4096 Jul 18 17:24 data
Since Elasticsearch is running as elasticsearch
and this dir is accessible byroot
, I’m assuming that’s where the write error is happening?
The other error I will often get from some of those options is:
error: failed switching to “elasticsearch:root”: operation not permitted
Which I believe is from this line in setup.sh
:
gosu elasticsearch:root /usr/local/bin/docker-entrypoint.sh
I’ve spent a while on this and haven’t made any progress, so I’m writing to see if someone knows where I may be going wrong. I understand that the