Configuring a Single FLAME Runner Across Multiple Deployment Instances

1

Hello all,

I’ve configured the following setup:

    {FLAME.Pool,
       name: Nexus.HTMLToPDFRunner,
       min: 1,
       max: 10,
       max_concurrency: 5,
       idle_shutdown_after: 10_000,
       log: :debug}

But, my application is deployed across multiple instances (fly scale count 3). As a result, I end up with 3 instances of the flame runner instead of the single one I intended to have.

Does anyone have insights on how to configure this so that I only get one flame runner, regardless of the number of deployed instances?

2

This is the way FLAME works by design – the parent node manages its own flame children to avoid any orchestration layer as well as guarantees that the child is spawned as close as possible to the parent node (same datacenter worst case, same physical server best case). If you want to to have a single global FLAME child instance for a pool, you’d need to add the orchestration layer yourself.

For example, you could hardcode one of your machines to be the “primary” and start the flame pool, then other nodes RPC the primary to perform the flame calls (see :erpc). Or you could come up with other strategies to elect the leader and start the pool on demand (all with their own tradeoffs). Also if you are going multi-region, then you’d pay the cost to send data back and forth to the FLAME instances and added coordination brings a bunch of failure mode considerations.

3

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.