No volumes involved here. Today it works again
Iâm seeing this in LHR:
Error: failed to update VM 5683dd6f13e2d8: aborted: could not reserve resource for machine: insufficient memory available to fulfill request
Sorry for the inconvenience. What is the machine ID?
The could not reserve resource for machine
error is about scheduling new machines, but existing machines should be able to respond.
The machine ID and error message are below.
The machine canât be restarted.
% flyctl apps restart *****
Restarting machine 3d8d9797b2d998
Error: could not stop machine 3d8d9797b2d998: failed to restart VM 3d8d9797b2d998: unknown: could not reserve resource for machine: insufficient CPUs available to fulfill request
Hi all, Iâm getting the same issue in the cdg region.
Updating existing machines in '****-staging' with rolling strategy
Error: failed to update VM 2874161c060558: aborted: could not reserve resource for machine: insufficient memory available to fulfill request
Edit:
I tried removing the free builder machine and restarting the deployment pipeline, but this made no difference. What worked for me was to just run fly deploy --remote-only -c deploy/fly/staging.toml
locally. I feel like Iâm missing something.
I doubled the CPU(from shared-1x-cpu@256MB to shared-cpu-2x@ 512MB), but the problem has not been resolved.
Please take action as soon as possible!!
% fly apps restart ******
Restarting machine 3d8d9797b2d998
Error: could not stop machine 3d8d9797b2d998: failed to restart VM 3d8d9797b2d998: unknown: could not reserve resource for machine: insufficient CPUs available to fulfill request
Having same issue in SG:
Iâm having problem deploying in SG region since yesterday:
Updating machine 5683dedf65318e
Error: could not update machine 5683dedf65318e: failed to update VM 5683dedf65318e: aborted: could not reserve resource for machine: insufficient memory available to fulfill request
Please help!
Having this issue in LHR, for a good hour. Any help would be very appreciated!
Error: failed to update VM 9080ee0b5464d8: aborted: could not reserve resource for machine: insufficient memory available to fulfill request
hey, if your machine has no volume attached, try to remove it with fly machines destroy MACHINEID
and redeploy.
We are looking at the capacity issues and scheduling at the moment.
Having the same issue at sin region.
Fly need to really scale up their machines!
I do not want to change region because of this. Hopefully this can be fixed asap! my machines are not starting up.
I am having the same problem in MAD⌠it is scary that the only solution is to migrate zone, but lots of zones are having the same issue!
Also, as many have noted, this should be on the status page!
I am almost ready to launch my (first) app
I had the same issue in cdg
and I was able to start a new machine by destroying the existing one, cloning it, and attaching the volume I had:
$ fly machine destroy 287xxx128
machine 287xxx128 was found and is currently in stopped state, attempting to destroy...
287xxx128 has been destroyed
$ flyctl machine clone 287xxx128 --attach-volume vol_nylzxxxmkp
Cloning machine 287xxx128 into region cdg
Attaching existing volume vol_nylzxxxmkp
Provisioning a new machine with image registry.fly.io/xxx:deployment-xyz@sha256:xyz
Machine 48exxx3e8 has been created...
Waiting for machine 48exxx3e8 to start...
Waiting for 48exxx3e8 to become healthy (started, 1/1)
Machine has been successfully cloned!
You can get your machines and volumes ID with:
$ fly machine list
$ fly volume list
EDIT: actually, itâs happening again
Itâs again in MAD region.
There is nothing on the status page about that.
The existing running app is affected and doesnât work after failed deployment.
The app is running with ~128mb with 256 available and according to others, scaling to 512 wonât help. Recreating the machine also doesnât help.
Can you provide more info and update the status page accordingly? The issue is just too critical and lasts too long already.
It is back for me in CDG. Any update from the fly team? I agree with @mif it would be nice to see such critical issues on the status page.
Also in WAW because of insufficient CPU.
Hello everyone. Very sorry youâre running into this.
Is this happening on normal rolling deploys, where there are no stopped Machines?
If you are cloning a machine with a volume, increasing the RAM or CPU specs on your Machines, or using canary or blue-green deployment, [I was wrong about this; canary and blue-green should actually have helped if you donât have volumes, because they replace Machines with new ones that would land on a host with capacity, rather than updating in place] you may run into this if you happen to be on a near-capacity host. This can also happen if your Machines have auto-stopped and the host doesnât have enough CPU or memory free to start them up again when you deploy.
In that case the best solution I know, unfortunately (for the moment) is to create a new Machine (and volume, if needed), which should land on a different host in the same region.
But if youâre deploying only to update the image, and your Machines are already running, you should not hit any resource errors. Please let us know if this is the case!
For me, it is exactly the latter â one normally running machine â fly deploy â insufficient resources error + âfailedâ machine.
Can you share the App name?
dessart
Ok, that app is configured for auto-stop (i.e. scale to zero) which is the failure condition described in Cannot deploy because of insufficient memory - #76 by catflydotio. The machine, 32871d4f1271d8
, is on a host that is low on capacity and will fail to start/update depending on the available resources. If you donât want this behavior, you can disable auto-stop by setting the following in your fly.toml
:
[http_service]
auto_start_machines = true
auto_stop_machines = false
force_https = true
internal_port = 3_000
min_machines_running = 0