The long story short: GPU machines can request up to 256GB of RAM per attached GPU device.
Some of you have expressed the need for more RAM, not talking about GPU memory, but larger machine’s memory options to load those beefy models that requires intermediary memory to work.
The base machine size for all GPU models is performance-8x, it comes with 8 CPUs and 32GB while its memory limit is 64GB. It has been possible to raise memory to 128GB by using performance-16x as base size, but starting this week any GPU machine can scale up memory up to 256GB per card.
Below example launches a A10 GPU machine with 256GB on a performance-8x base size (the default):
fly m run -a dan-gpu -r ord --vm-size a10 --vm-memory 256gb --shell ubuntu:24.04
...
Success! A Machine has been successfully launched in app dan-gpu
Machine ID: 3d8dd947fe4989
...
root@3d8dd947fe4989:/# free -mh
total used free shared buff/cache available
Mem: 251Gi 2.0Gi 251Gi 153Mi 486Mi 249Gi
And this is how you scale up to 512GB and 2 GPU cards:
❯ fly m run -a dan-gpu -r ord --vm-size a10 --vm-memory 512gb --vm-gpus 2 --shell ubuntu:24.04
...
Success! A Machine has been successfully launched in app dan-gpu
Machine ID: 4d89019df63587
...
root@4d89019df63587:/# free -mh
total used free shared buff/cache available
Mem: 503Gi 3.5Gi 503Gi 153Mi 485Mi 500Gi
Swap: 0B 0B 0B
happy hacking!
