Fly.io plans for Litestream / Sqlite

Questions / Help
1

Hi :slight_smile:

Super excited about litestream being part of fly.io

I am building a firebase alternative using sqlite (https://javascriptdb.com) and I want to use fly.io for the infrastructure.

  1. Are you guys planning to offer a sqlite service similar to your postgres service? Do you have a rough ETA?

  2. Is there a way to set the fly-replay header from the client? Because I control the client sdk and the server, I can tell if an HTTP call should go straight into the write instance.

I see two benefits in doing this:

First, you save wasted time doing any business logic and trying to write.
Second, this could be a work around stale data on read replicas after a write, here’s what I’m thinking:

  • Client sdk does a POST/PUT request (knows that is a write, adds ‘fly-replay’ header).
  • Client sdk then adds a timer to set the ‘fly-replay’ header to all requests for the next X milliseconds.
  • Within X milliseconds client sdk does a GET request to get the data that was just updated. Because the header is present, request goes to the write replica, data is consistent.
  • After X milliseconds client sdk does a GET request for the same data. Header is NOT present, request goes to the nearest read replica, data is consistent because replication already happened.

Forgive me if what I’m saying doesn’t make any sense.

1 Like
2

For litestream specifically, using fly-force-instance-id should direct the request to a VM instance, instance-id, iff fronted by Fly’s http/s proxy, ref: Is it possibly for a client to connect to a particular instance - #4 by jerome

Technically, you could also make use of the 6pn internal network to chain the request through to wherever; ie, to any fly app within that fly org (ref).

Forgive me if what I’m saying doesn’t make any sense.

Discounting the many failure modes (idempotent requests, replica unresponsive after write, client dead, network partitioned, timers running out of sync due to cpu starvation, writes interlaced between get requests, and so on…), it does make sense. But as you mention, I’d rather wait for Fly to integrate litestream themselves (and solve those distributed systems problems for me). In the meanwhile, as a placeholder, you could consider using another database like PlanetScale (read-replicas), Aurora Serverless v2, Xata, tiledb, and its ilk.

1 Like
3

Awesome! I think fly-force-instance-id will do it for me.

Thanks a lot @ignoramous

1 Like