My production machine has reverted to the date I last deployed, and all changes I made through the panel and database content have also reverted to that date. My project is django+wagtail, and I’m currently using SQLite as my database. This could be the cause. (They say there’s an error in the status and they’re fixing it, but I don’t know if this is related to my problem.)
Hey there!
Usually you’d want to store SQLite in volumes so they dont get back to initial state when your app restarts (everything outside volumes is ephemeral). Can you confirm you app has volumes and your database is stored inside their mount path?
Another issue would be that your app has multiple machines with a volume on each of them and you’re not syncing the data anyhow.
Thank you. SQLite wasn’t in a separate unit, but my data was being written to this database. Also, the photos and documents are gone. Fly.io reverted everything to the version I deployed a month ago. How can I access the latest version? Did Fly.io keep a backup? For example, back to October 20th?
Unfortunately we do not store data from the ephemeral VMs by design.
We would have backups for volumes though: Create and manage volumes · Fly Docs
And also very very recently we release persistent rootfs but that is opt-in:
Since the last deploy date. My editors have been adding content via the wagtail admin panel for about a month now. Has all of this been deleted?
If there was any restart after the deployment, yes.
You can use fly ssh sftp get · Fly Docs to get the current state before its restarted.
1 Like
Quick disclaimer that persistent rootfs is not at all appropriate for this use-case! If you need to persist actual data, you should definitely use a volume or Tigris.
Persistent rootfs is intended for use-cases like keeping a disk cache warm or avoiding an expensive boot process. Ultimately, you should still be prepared for the data you store in the rootfs to disappear at any time, even with "persist_rootfs": "always".
3 Likes
I’d argue that “deleted” is the wrong verb, since it was not saved to start with. Containers are ephemeral in Docker, and VMs are ephemeral in Firecracker. In some ways this technology can be thought of like a RAM disk: if power goes, so does the data.
I sympathise with your predicament; it’s rotten luck to lose work.
You didn’t perchance have another machine live, and LiteFS running between them?
If you set this up again, I’d say that the prior advice is good, but also set up your own off-site backups. Make sure you include the file system, if you’re storing photos and documents outside of the database.
This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.