Using sqlite from persistent volume for Django application

[acmshar]

I’d like to use sqlite as my database for a simple Django app without LiteFS (for now). I have created an app with a single machine, and created the volume with fly volumes create blog_data. I’ve also added to fly.toml:

[mounts]
  source="blog_data"
  destination="/data"

When I set my DATABASE to be

DATABASES = {
    "default": env.dj_db_url("DATABASE_URL", default="sqlite:////data/db.sqlite3"),
}

python manage.py migrate fails on startup with: django.db.utils.OperationalError: unable to open database file

I’m able to get this to succeed writing to other locations such as sqlite:///db.sqlite3 and sqlite:////mnt/db.sqlite3, but these are both ephemeral.

Is there some way to have this successfully write to the persistent storage that I’m missing?

[billy]

hi acmshar,
Is it possible you can’t created a file at /data/db.sql?. This is an issue I sometimes run into.

[acmshar]

Good question. I am able to create that file manually by sshing into the app machine. I thought maybe by creating the file first that it might solve the problem, but rerunning fly deploy with the database set to "sqlite:////data/db.sqlite3" still fails with the same error.

I wonder if it’s a timing issue and the volume isn’t attached when that command is initially run.

I’ve commented out the migration in the deploy step:

[deploy]
#  release_command = "python manage.py migrate"

That seems to fix the issue. I then run migrate manually after the deployment and it works. I can add a post to the database and when I deploy again, the post is still there.

I feel like this should be easier and documented somewhere. I’m going to write this up for my blog, and I’ll try to submit a new page for the docs.

[billy]

So I looked into it a bit more, and release_command’s don’t have access to volumes, since they’re ephermereal and can’t attach the same volume as the main machine. I recommend running your migration script either somewhere in your Dockerfile ENTRY, or running it before starting your main application.

[acmshar]

Thanks for confirming that. I was planning to look into where to run it so that I don’t have to do it manually. When you say ’ running it before starting your main application’, do you have any recommendations for where/how to do that? I’m wondering if there is a configuration in fly.toml that would run it after the machine has access to the volume, or if there is some other way you are thinking of.

[billy]

The way I would personally do it having a small shellscript as your Dockerfile entrypoint:

#!/bin/sh
python manage.py migrate
python manage.py runserver

[acmshar]

Instead of runserver, I am sticking with gunicorn which was already in my Dockerfile. So my migrate.sh looks like:

#!/bin/sh
python manage.py migrate
gunicorn --bind :8000 --workers 2 django_project.wsgi

And the end of my Dockerfile looks like:

RUN chmod +x migrate.sh
EXPOSE 8000
ENTRYPOINT ["./migrate.sh"]

I see the output from the migrate command running in my monitoring logs, so I’m pretty confident this is working. Thanks for all of your help!