Here is an early release of two libraries to help globally distributed Phoenix applications using Postgres DB with read-replicas.
- fly_rpc - Library for making RPC calls to nodes in other Fly.io regions.
- fly_postgres - Library for working with local read-replica postgres databases and performing writes through RPC calls to other nodes in the primary Fly.io region.
The short notes:
- Prereq: Your Phoenix app nodes are clustered together
- Prereq: You have primary/replica Postgres DBs setup
- Use
fly_postgresin your app (Follow the README installation steps) - DB read operations like
MyApp.Repo.all(User)are executed locally on the replica - DB write operations like
MyApp.insert(changeset)are proxied to one of your apps running in the primary region. The call to insert will block until it completes and the database replication that includes your change is received on the calling node.
This means you can write “normal” code that performs writes on the primary, then blocks and waits for the async replication to complete before continuing. Your app doesn’t have to be completely re-designed or reworked for dealing with primary and replica databases. 
More complex operations that do many inserts or updates will become slow while it pauses to wait for each insert or update to be replicated locally. For these operations, you can do a simple refactor of your code to make an explicit RPC call to the primary. It essentially says, “run this complex operation on the primary, close to the database”.
As an example, this version of a LiveView event is totally valid when you are close to the database. It iterates through a loop and performs many inserts.
def handle_event("do_hard_work", %{"list_of_stuff" => stuff_ids} = _params, socket) do
# create list of things from a set of IDs. May be a very large list and many inserts
Enum.each(stuff_ids, fn item_id ->
{:ok, _created} = ImportantStuff.create_stuff_from_item(item_id)
end)
{:noreply, assign(socket, :stuff_list, ImportantStuff.all()}
end
While it could be completely re-written to be more efficient for batch operations, etc. We can make a small change to let it work well in our system.
def handle_event("do_hard_work", %{"list_of_stuff" => stuff_ids} = _params, socket) do
Fly.Postgres.rpc_and_wait(__MODULE__, :create_list_of_stuff, [stuff_ids])
{:noreply, assign(socket, :stuff_list, ImportantStuff.all()}
end
def create_list_of_stuff(stuff_ids) do
# create list of things from a set of IDs. May be a very large list and many inserts
Enum.each(stuff_ids, fn item_id ->
{:ok, _created} = ImportantStuff.create_stuff_from_item(item_id)
end)
end
This refactors the work part out into a separate public function. We can then RPC to the primary and execute the function there! We still need to pass the arguments. The RPC waits for the execution to finish and the data replication to sync. Finally, we continue on as normal.
Using this approach, your Phoenix LiveView can be physically close to your users giving really snappy and responsive behavior. Then “writes” are a little slower as they are performed in the primary region. This works well for read-heavy applications, which most web applications tend to be.
This is still early but seems to be working well for me so far! I’d love to get your feedback.
Interested to hear how it interacts with the Ash framework too.