Setting up an Django app to use a Fly volume for the SQLite database

, ,
1

I have a django that I’ve been running for years on a local Ubuntu server that I’m trying to move to fly.io without any success.

It isn’t clear what the relationship is between the name of the db.sqlite3 database and the values in the [mount] section of fly.toml.

If I try to create a volume with the name db.sqlite3, it fails because a period is not allowed in the name.

fly.toml has:
[mounts]
source=“myapp_data”
destination=“/data”

I ran:
E:\Django\mydb>fly volume create myapp_data -r iad -n 1
ID: vol_vde81kk67oyym064
Name: myapp_data
App: mydbdwa
Region: iad
Zone: 7579
Size GB: 1
Encrypted: true
Created at: 03 May 26 19:31 UTC
Snapshot retention: 5
Scheduled snapshots: true

When I run fly deploy I get:
Error: Process group ‘app’ needs volumes with name ‘myapp_data’ to fulfill mounts defined in fly.toml; Run fly volume create myapp_data -r REGION -n COUNT for the following regions and counts: lax=1

Also, when I use the following in setting.py

DATABASES = {
“default”: env.dj_db_url(“DATABASE_URL”, default=“sqlite:////data/db.sqlite3”),
}

it fails recognizing env.dj.db.url.

Is there some basic tutorial on all the steps to make this work. I’ve read through all the posts related to this subject and it hasn’t helped with basic understanding.

2

Volumes are folders, not files. So, the volume you probably want here, assuming that everything in /data/* can go in the volume and not on the machine, is /data. The SQLite file is db.sqlite3, and if you have that already, you’d copy that into the volume, once the volume is created.

3

I added a bit to the post that I should have put in initially.

Creating the volume seems to be successful but the deploy says it is missing. Is there a typo that I’m missing?

4

Note also that, assuming you want all machines in your app to share this database, you can only operate in one region (e.g. iad). This is because volumes only work within a single region, so if you scale your app to run in other regions, those volumes (and database files) will be an independent copy.

You could look into LiteFS if you want to operate multi-region, but otherwise, using several machines in one region may be the best trade-off, especially if you want to keep a lid on the technicalities.

5

This is your problem:

Your machine is in lax, but your volume is in iad. They need to be in the same region (move the machine or the volume depending on where you want them to be).

6

There are several parallel problems here, and in general it is easier to solve problems one at a time. Once the volume region is fixed, you can move onto this one. I assume you could just hardwire it:

DATABASES = {
"default": "sqlite:////data/db.sqlite3",
}

However, I am guessing at Python syntax here! (Make sure you use double quotes, and not oblique quotes).

Of course, this stops the env var from having an effect, and an env var may be useful if you want to run a local or test environment, since this value may be different in each env. Indeed, I recommend testing locally: if env.dj_db_url does not work remotely, you probably want to find that out before you deploy. Are you running your app locally in Docker?

7

Machines never share volumes, though, not even when they’re in the same region.

(This surprises a lot of people.)

SQLite has a reputation for simplicity in other contexts, but it’s really the opposite here on the Fly.io platform, in my opinion. Even experienced old hands forget the dangers, from time to time, :dragon:.

The best tip is to not use volumes on Fly.io, basically. Among other non-obvious problems, they risk permanent data loss, since each one is tied to an individual, physical host machine. When that hardware fails, which it will eventually, then the volume disappears.

They are ok-ish for people who have LiteStream and similar configured, although then the admins have occasional manual-intervention emergencies, where they need to nudge their site back into existence. (Also, the platform generally assumes that you have ≥2 Machines in each app, so it will take one offline for a while on the assumption that you have a spare.)

And they are fun for those like myself who want to experiment with distributed databases, etc., …

8

Oh right, my bad. Listen to Mayailurus, OP :squinting_face_with_tongue:

9

I actually moved away from SQLite on Fly. It’s not that I had some production problems; just that LiteFS felt cool and experimental all at the same time, and I prefer boring, generally speaking. So I moved to managed Postgres (not on Fly, but I assume theirs is worth a look).

11

Thank you for all your help.

When fly.io initially configured my app, it used the location ‘lax’. Creating the volume with this location worked. Can I set the location? When I changed fly.toml to use machine location ‘iad’, it didn’t seem to take. Works with ‘lax’.

Using the following values in settings.py seems to make everything work. The commented out values cause “This page isn’t working” in the web browser.

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': "/data/db.sqlite3",
    }
}
#DATABASES = {
#"default": "sqlite:////data/db.sqlite3",
#}

This is a one or two user app so the simplest database is fine.

I don’t use Docker. Is env.dj.db.url something specific to Docker or an additional Python include?

For now I will just comment / uncomment the DATABASES lines to switch between local testing and deployment. Or put in some conditional logic.

It works fine on my local server but we have power failures longer than my UPS duration and this usually happens when I am away so it is better to host on a public server.

Thank you again.

12

Super; if it works, it works. I assume the Django docs will confirm the syntax of this value for you, if you want to check.

It’s not a Docker thing; I assume it’s something in Python (language level) or Django (framework level).

You can run your app natively on your local computer, though Docker is good to learn. It runs very similarly to the virtual machine system that Fly uses, so your app inside Docker will be a closer match for your production app in Fly. But it depends on how critical the remote app is; if it is for a hobby project, or just a few people, then it is not important.