Get help with Fly GPUs!

1

Fly GPUs are available for everyone to try. As you start using them, we want to make sure we’re here to answer questions and help you through deploying your first GPU-enabled Fly apps.

Here’s a few questions other Fly devs have had:

GPU access and org trust

Starting with GPU access and security! The most common error some of you may encounter is this message when you try to deploy:

Your organization does not have enough trust to create GPU machines. Please contact [billing@fly.io](mailto:billing@fly.io) to remove the restriction.

This means exactly what it says: our fraud algorithm is unsure about your account. We’re a little more picky about who has access to GPUs than some of our other stuff. If you’re an existing Fly user with a history, we’ll look at your account and remove it. Otherwise we might ask you to pre-auth your card.

Perfomance CPUs and Midjourney Bot

One of you reported an issue (thanks!!) with the (Not)Midjourney Bot blog post where a non-GPU-enabled instance was being created from the fly.toml in the sample.

This is because the original post specified a shared cpu in the [[vm]] section, but GPUs now require performance CPUs (they just work a lot better). We’ve since fixed the sample to use performance-8x CPUs, so if you hit this issue, go try again!

What else have you come across? We’re here to help.

4 Likes
3

The request, not specific to GPUs, but especially for running expensive machines like GPUs, is that it would be easier to estimate costs if you could configure the time interval between the fly proxy’s automatic shutdown process.

1 Like
4

To add, it doesn’t have to be immediately configurable, but it would be nice if the documentation was clear.

5

I got an error message looks little different.

failed to provision seed volumes: failed creating volume: failed to create volume: To create volumes on GPU hosts please contact billing@fly.io (Request ID: xxxxx)

I’ve sent emails to billing@fly.io but still not get any responses.

6

When you need control over Machine idle shutdown, we recommend using the fact that Machines stop when the main process exits.

Thanks for the suggestion; for now I’m adding the above link to the GPU quickstart doc.

7

Can I ask what size volume you’re trying to create? There are some org limits on this. Once we figure this out I’ll go back and make that error message clearer too.

8

I followed this Fly.io Blog article.
Scaling Large Language Models to zero with Ollama · The Fly Blog

Here’s my fly.toml

app = 'hellegpu'
primary_region = 'ord'

[http_service]
  internal_port = 8080
  force_https = true
  auto_stop_machines = true
  auto_start_machines = true
  min_machines_running = 0
  processes = ['app']

[[vm]]
  memory = '16gb'
  cpu_kind = 'performance'
  cpus = 8
  gpus = 1
  size = 'a100-40gb'

[build]
  image = "ollama/ollama"

[mounts]
  source = "models"
  destination = "/root/.ollama"
  initial_size = "100gb"
9

Oh I see the issue. Your org was flagged by our trust algorithm. I fixed that and made the error message better. Go ahead and try again.

1 Like
10

hey @nina , got the same error message and still haven’t gotten a response from billing@fly.io. I’ve been using fly.io for a while now. How can I get the restriction removed from my account?

11

You should be good now!

12

Thank Nina. Love to great customer support from Fly.io

13

It’s working now. Thanks!

14

I am also having the same issue trying to deploy an app with GPU. Can that restriction be removed? I emailed billing@fly.io. Thanks.

15

You should be all set.

1 Like
16

Awesome, thanks!

17

I am having the same issue and also sent an email to billing@fly.io. I’m relatively new to fly.io so I’m pretty sure that’s why my account got flagged. Could you kindly lift the restriction for me as well?

Thanks!

18

Same issue with me please can I have restriction removed from the GPU? Thanks!

19

Hello! I’m facing this issue at the moment. I was following this article when I encountered the same error message. Can you remove the restriction please? Thank you!

20

@jeryl888 You should be good to go!