We have a multi-machine Fly app (4 machines across two regions) using LiteFS with a Consul-based lease (`lease.type: consul` in litefs.yml, stock `flyio/litefs:0.5`). Since a `fly machine update --vm-size ` on one of our machines, LiteFS on that machine (and, we later found, on machines that get restarted for other reasons) can no longer register with Consul at all.
**Exact error from `litefs mount`:**
```
ERROR: cannot init consul: cannot connect to consul: register node “-/litefs”: Unexpected response code: 500 (No cluster leader)
```
And in the ongoing retry loop once past initial registration:
```
level=INFO msg=“cannot fetch cluster ID from \“consul\” lease, retrying: Unexpected response code: 500”
level=INFO msg=“: cannot acquire lease or find primary, retrying: fetch primary url: Unexpected response code: 500”
level=INFO msg=“: cannot find primary & ineligible to become primary, retrying: no primary”
```
**Timeline:**
1. Everything was healthy for months prior.
2. Ran `fly machine update <machine_id> --vm-size performance-2x -a ` on one machine (a spare, not the one serving traffic at the time).
3. On its next restart, that machine got stuck in the retry loop above indefinitely (tested restarting it cleanly several times over ~1 hour, no self-recovery).
4. Separately resized our then-primary machine the same way, but did not restart it afterward – it stayed healthy for the rest of that session, meaning the resize itself isn’t instantly fatal, only a subsequent restart is.
5. Eventually the primary also needed to restart (unrelated reason) and hit the identical error, despite never having been resized itself in that instance – so the problem is not confined to the one resized machine, it affects the whole app’s Consul attachment.
**What we tried (neither helped):**
- Changed the Consul KV key in litefs.yml (`lease.consul.key`) from one value to a different one, redeployed. Same error, unchanged.
- Ran `fly consul detach` then `fly consul attach` on the app, to force a fresh Consul attachment. Checked `FLY_CONSUL_URL` directly on the machines afterward – it came back byte-for-byte identical (same token, same host, same per-app KV path) to before the detach. So this appears to be idempotent / re-links to the same underlying assignment rather than provisioning a new one, and did not change the outcome.
**What we found by comparing against a different app in the same org:**
- The affected app’s `FLY_CONSUL_URL` points to `consul-fra-9.fly-shared.net`.
- A different app in the same org points to `consul-fra-7.fly-shared.net` and shows zero Consul-related errors.
This strongly suggests the issue is specific to the `consul-fra-9` host/shard having no Raft leader, and our app is simply the one (of several in this org) currently pinned to it – not something wrong in our app’s config, since a completely separate app with presumably the same litefs.yml shape works fine on a different host.
**Questions:**
1. Is `consul-fra-9.fly-shared.net` known to be degraded right now?
2. Is there any way to have an app reassigned to a different Consul host/shard, short of destroying and recreating the app? (We couldn’t find one in `flyctl` – `fly consul` only has `attach`/`detach`, no list/status/reassign.)
3. Is there a safer, supported way to change a running machine’s vm-size that avoids this class of issue with Consul-leased LiteFS apps?
Happy to provide more detail (machine IDs, timestamps, further log excerpts) via DM if useful for someone from Fly to look at the specific Consul cluster.