Cannot deploy because of insufficient memory

Questions / Help Build debugging
,
1

Hello!
I am working on a side project so not really daily deployments and I am noticing today that I cannot deploy my app.
I am using the very basic unmodified Elixir template along with a Postgres DB. At the migration step, it fails with this logging:

Error: release command failed - aborting deployment. error running release_command machine: error creating a release_command machine: failed to launch VM: could not reserve resource for machine: insufficient memory available to fulfill request

I am not sure if itā€™s

  • a programmerā€™s mistake
  • a limit of the free allowances
  • an outage on fly.io side

I tried to deploy with the immediate strategy as well because I found topics mentionning that but it did not change anything. Anyone has any clue on this ?

2 Likes
2

Can you try a different region?

3

experiencing the same today, albeit with a Rails app.

could it be connected to the Apps v2 rollout? I noticed that my RAM scalings were not applied as well

4

Iā€™m running into the same trying to deploy an Elixir app to the Paris region.

Iā€™m trying to launch an open-source app called Accent, using fly launch

Provisioning ips for ww-accent-localization
  Dedicated ipv6: 2a09:8280:1::4e:2074
  Shared ipv4: 66.241.124.60
  Add a dedicated ipv4 with: fly ips allocate-v4
Running ww-accent-localization release_command: /app/bin/migrate
Error: release command failed - aborting deployment. error running release_command machine: error creating a release_command machine: failed to launch VM: could not reserve resource for machine: insufficient memory available to fulfill request

So I thought, maybe I can scale the memory:

āžœ  accent git:(5c4c5f4) āœ— fly scale memory 512
Error: No active machines in process group 'app', check `fly status` output

Nope. So, I subsequently tried to just use deploy command, but get the same error failed to launch VM: could not reserve resource for machine: insufficient memory available to fulfill request

1 Like
5

Update: It does seem that I can also not start machines via API in a dedicated machines app

6

Got the same with fly deploy with an Elixir app to Paris Region

Error: release command failed - aborting deployment. error running release_command machine: error creating a release_command machine: failed to launch VM: could not reserve resource for machine: insufficient memory available to fulfill request

7

Sorry for the trouble! Weā€™re seeing these errors in ord, fra and cdg, and are looking into the problem.

Weā€™ve opened an incident here: Fly.io Status - Deployment failures due to memory capacity issues

2 Likes
8

FYI: Iā€™m still getting the same error despite your status page indicating this should be resolved. Or am I misreading it?

Weā€™ve adjusted capacity in the affected regions and are monitoring to ensure machines and apps are being scheduled as expected.

9

Thanks for letting us know! We should have the capacity available in cdg shortly.

10

@rio517 We should be good now. Will you let me know if itā€™s still failing? What size machine are you creating? Weā€™re also adding even more capacity to those regions now so thereā€™s a lot of spare headroom.

1 Like
11

Itā€™s okay for me, thx!

1 Like
12

Also resolved for me.

1 Like
13

Iā€™m seeing this issue now trying to deploy an app to the LHR region.

14

I am too. My app failed at 13:57 and every attempt to deploy results in ā€œfailed to update VM 6e82920f710078: aborted: could not reserve resource for machine: insufficient memory available to fulfill requestā€

15

@gardener I cleared up some resources on the host your machine is on, you should be able to deploy now.

16

I still have the same error.

Logs say:
2023-06-05T12:28:37Z proxy[6e82920f710078] lhr [info]Starting machine
error.message=ā€œfailed to change machine state: machine getting replaced, refusing to startā€ 2023-06-05T12:28:37Z proxy[6e82920f710078] lhr [error]request.method=ā€œGETā€ request.id=ā€œ01H25RFM258WFJK0NF4CZB4MSD-fraā€

fly deploy says:
Updating existing machines in ā€˜smart-gardenā€™ with rolling strategy
[1/1] Updating 6e82920f710078 [app]
Error: failed to update VM 6e82920f710078: aborted: could not reserve resource for machine: insufficient memory available to fulfill request

or with strategy immediate:
Updating existing machines in ā€˜smart-gardenā€™ with immediate strategy
[1/1] Updating 6e82920f710078 [app]
Continuing after error: failed to update VM 6e82920f710078: aborted: could not reserve resource for machine: insufficient memory available to fulfill request

17

probably related:

Failed creating the Postgres cluster xxxxxx-db: failed to launch VM: aborted: could not reserve resource for machine: insufficient CPUs available to fulfill request
Error creating Postgresql database. Be warned that this may affect deploys

WAW region

18

Yeah, having issues waking up machines in WAW region.

19

Regarding WAW, there was a misconfiguration on our end. Weā€™ve fixed the issue now.

20

@kaz The issue still happening and machines randomly goes down from time to time. Any suggestions?