Machine still attempting to start

[j-man]

I have an app running Forgjo (with a 1GB mounted volume) that frequently enters this state where it fails to start:

2025-08-15T16:38:07.362 proxy[e2860d74f462d8] fra [info] Starting machine 
2025-08-15T16:38:07.365 proxy[e2860d74f462d8] fra [error] [PM01] machines API returned an error: "machine still attempting to start"  
2025-08-15T16:38:09.679 proxy[e2860d74f462d8] fra [info] Starting machine  
2025-08-15T16:38:09.683 proxy[e2860d74f462d8] fra [error] [PM01] machines API returned an error: "machine still attempting to start" 

Does this mean that the machine fails to start because it’s already starting? Is it stuck in a “starting” phase somehow?

I can “temporarily fix” this by doing:

fly scale count 0

and then rescale back to one again:

fly scale count 1

But it seems to stop working again fairly regularly at a later point in time again (usually within 1 day, i’ll try to find some log output here).

I’ve seen a few others on fly have a similar problem & API response, but I found no good working solution to the problem (maybe it’s unique for everyone?).

Maybe it’s related to suspension? (I do have that currently, i reckon if i never suspend / stop my machine with the scale to zero feature, then I will probably not have any issues… but i’ll hold off on that solution for now.). Maybe i caused the machine to crash with my own config & software, but the fact that it starts up just fine if i scale it down and up again means that it should be able to start fine without manual intervention too right?

Sometimes the log output above is interspersed with some rate limiting printouts too:

2025-08-15T16:41:48.442 proxy[e2860d74f462d8] fra [error] [PM01] machines API returned an error: "rate limit exceeded"
2025-08-15T16:41:48.891 proxy[e2860d74f462d8] fra [info] Starting machine
2025-08-15T16:41:48.892 proxy[e2860d74f462d8] fra [error] [PM01] machines API returned an error: "rate limit exceeded" 

Maybe that narrows it down?

---

output of fly machine status:

Machine ID: e2860d74f462d8
Instance ID: 01K2F9SMKRABW65BBFKVCJYYDG
State: starting
HostStatus: ok

VM
  ID            = e2860d74f462d8
  Instance ID   = 01K2F9SMKRABW65BBFKVCJYYDG
  State         = starting
  Image         = ratatoskr:deployment-01K1DPPKRZ2RKF6S33MC50Y7ES
  Name          = young-sun-6971
  Private IP    = fdaa:3:67e1:a7b:51c:7ee5:2b12:2
  Region        = fra
  Process Group = app
  CPU Kind      = shared
  vCPUs         = 1
  Memory        = 512
  Created       = 2025-08-12T14:08:54Z
  Updated       = 2025-08-14T14:33:32Z
  Entrypoint    =
  Command       =
  Volume        = vol_re839n595goj8d5r

Event Logs
STATE           EVENT           SOURCE  TIMESTAMP                       INFO
starting        start           user    2025-08-14T16:33:32.689+02:00
suspended       suspension      flyd    2025-08-14T16:30:30.781+02:00
suspending      suspension      user    2025-08-14T16:30:29.015+02:00
started         start           flyd    2025-08-14T16:24:03.724+02:00
starting        start           user    2025-08-14T16:22:37.787+02:00

[j-man]

No other useful logs to be found … puzzling