LiteFS: Two (primary) volumes in one region

[rsas]

In the LiteFS example documentation, the last paragraph says:

Make sure you have at least 2 volumes created in Denver so that the primary status can move between those two. Otherwise, you’ll see temporary write availability loss when you deploy your application.

I am struggling to understand how this can be configured for one app so that app updates do not result in write loss and temporary outage.

[tj1]

ok, I think I understand what is going on.

1. It is using the apps v1 platform.
2. litefs-example/litefs.yml at main · superfly/litefs-example · GitHub shows advertise url to be at the host level.
3. The secondary in denver should hopefully be up and replicating.
4. On failover, the secondary in Denver should be up to date and take over.

However, with nomad it is tough to guarantee the secondary is up.

This idea may work well with v2 app though with static leasing, but then again, maybe not.

[rsas]

Thank you @tj1 for the quick reply. How would that result in a single TOML file for an app?

I am looking forward to specifying one app config file with a DB path and then using the scale option to add more instances in the same or arbitrary other regions. My expectation is that LiteFS would transparently do the rest, figuring out the PRIMEs and handling the transient restarts.

So far, I have not seen an example with multiple instances within one region that make this work. The challenge is that there cannot be two instances in the same region mounting the same persistent disk, no?

[tj1]

So far, I have not seen an example with multiple instances within one region that make this work. The challenge is that there cannot be two instances in the same region mounting the same persistent disk, no?

No, I don’t believe so.

i’m trying to figure this out myself for apps v2. I believe a consul cluster is required.

Small consul cluster + litefs failover

Thanks for the additional information. So, in theory, what I’m looking for is a floating dns name that can be moved to the appropriate host on failover, however, this doesn’t fully solve the problem as litefs also has to decide that it is now the primary (there are a few old-school ways of doing this VeritasFS for Oracle comes to mind where it actually killed all the IO on the primary). We have two possibilities right now for advertise_url, but neither will work with a static lease. <alloc_i…

[benbjohnson]

@rsas Sorry for the confusion. Yes, the docs have not been updated for the apps v2 platform. I also have a bunch of docs changes for our v0.4.0 release that’s coming soon. I’ll get ensure those are working with the new apps platform as well.

LiteFS should handle the primaries being in the den region if you specify it in the fly.toml:

candidate: ${FLY_REGION == "den"}

[rsas]

Thank you Ben. I will eagerly wait for the updates and looking forward to how this project evolves further. Big fan of the ideas since the inception of Litestream.