For over 6 hours now, deployments have been failing continuously and suddenly — without any clear reason.
I’ve already tried re-authenticating with fly auth logout and fly auth login, but it didn’t help.
Has anyone experienced a similar issue and found a solution?
Error: failed to update machine e2860152b71386: failed to update VM e2860152b71386: permission_denied: bolt token: failed to verify service token: no verified tokens; token f9837202-9e79-5b90-b65f-77de68eabc9f: verify: context deadline exceeded
request token: failed to verify service token: no verified tokens; token 78f09fe8-2915-5da1-9c19-1a97eedb5b7b: verify: context deadline exceeded (Request ID: 01K0BY6Z2MZ2AS4WPEKM9WA2A8-sjc) (Trace ID: e9e5409983cf3116cd95db1b8b5f360c)
The issue has now been resolved and deployment is working again.
Just documenting this here in case someone runs into the same problem later:
One option is to wait for fly.io to fix the issue on their end.
Another suggestion I found in the community was to manually stop/destroy every machine in the app to reset it to a clean state, then restart deployment from scratch.
It was… an interesting experience — but definitely not one I’d like to repeat. =)