I have an app restarting every 6 minutes. Like clockwork.
There are no workers on the application that could cause this.
Here is the log.
error.message
machine was recently stopped and is unavailable to service request
event.provider
proxy
fly.app.instance
d8d9e17bed0e98
fly.app.name
zaia-core-service-prod
fly.instance.id
d8d9e17bed0e98
fly.region
gru
host
5f0f
http.request.id
01HY967WV71PXSSGZBN3C59NQJ-gru
http.request.method
GET
http.request.version
1.1
log.level
error
message
[PM11] machine was recently stopped and is unavailable to service request
timestamp
2024-05-19T19:46:15.017308Z
http.response.status_code
0
Machine states are broadcast to the Fly proxies in an eventually consistent manner. So, edge proxies may not have an up-to-date picture about machine states. A machine that appears as to the edge proxy may actually have been recently stopped.
If the edge proxy forwards a request to a recently stopped machine, and there are other machines available to handle the request, the error will be returned to the edge proxy. The error informs the proxy about the state of the machine, and instructs the edge to forward the request to another machine.
If the edge proxy believes there’s only one machine that can service the request, this logic is bypassed. The request is forwarded to the machine even if it was stopped recently.
The current threshold for ‘recently stopped’ is 5 minutes.