Unable to create a machine with a volume attached due to an "insufficient resources" error

[vishalrao]

Hello!

I created a volume with:

fly volume create ollama -s 300 -r ams

then ran:

fly machine create ollama/ollama -n ollama -r ams --restart no -v ollama:/root/.ollama --vm-gpu-kind a100-80gb --vm-gpus 4

which resulted in the “insufficient resources” error.

If I run:

fly machine create ollama/ollama -n ollama -r ams --restart no --vm-gpu-kind a100-80gb --vm-gpus 4

without the -v ollama:/root/.ollama volume option then the machine is created and runs fine.

The reason I’m trying to get 300 GB storage space is that I’m trying to run the llama 3.1 405b model with ollama.com and this workload needs about 230 GB of disk space and total GPU memory hence the machine specs.

Thanks for any advice!

[vishalrao]

This is the error:

Error: could not launch machine: failed to launch VM: insufficient resources to create new machine with existing volume ‘vol_r6gw1n2o6xl961dv’ (Request ID: 01J5GAY06JHH64R1YD9648KMMP-bom)

[ashitag]

Can you try fly scale count 0 command, redeploy and see if it works?

[vishalrao]

Thanks for your replies! I also tried a100-sxm4-80gb with the same results as in my initial post.

Since you mentioned looking at fly.toml I thought I should try that and the following actually worked!

app = 'ollama-xxxxxx'

primary_region = "ams"

[build]
  image = "ollama/ollama"

[mounts]
  source = "ollama"
  destination = "/root/.ollama"
  initial_size = 300

[[vm]]
  memory = "256GB"
  cpus = 16
  cpu_kind = "performance"
  gpus = 4
  gpu_kind = "a100-80gb"

[[restart]]
  policy = "never"

I was able to proceed with my quick ollama run.

I will leave this forum post unsolved - hopefully an employee of Fly.io might be able to look into why the CLI commands aren’t working for me.

Also, side note, a request to allow higher performance vCPU counts than just 16, would be nice to support, say, 64 vCPUs or more - I was able to get 256 GB memory just fine.

Thanks again!