Calculating GPU pricing for image generation

[wceolin]

I’m considering using FLAME to generate images with FLUX.1 [schnell] in a Phoenix app using Bumblebee.

I’m trying to calculate how much it would cost to generate an image on a L40s Fly GPU to see if that’s a viable option financially.

Assumptions

• Fly GPU costs $1.25/hour for the L40s machine.
• Fly GPU charges for the startup and teardown processes.
• Cold starts without volumes are around 30 seconds for a 7B model. Therefore, a cold start for a 12B model should be around 51.3 seconds without a volume or ~20 seconds with a cached model on a persistent volume.
• I’m guessing the teardown process should take ~3 seconds.
• Each image generation should take ~1 second.

Pricing estimation

• GPU usage time to generate an image (and destroy machine): ~24s
• Price per second: $0.00034722
• Price to generate an image: $0.0083

Are my assumptions and calculations correct, or am I missing something?

[mabis]

Have you had a chance to play with the official comprehensive calculator ?

Just use 0.007 in the hours field

[wceolin]

I hadn’t seen this tool before. But I think this calculator is more useful for a tradicional workflow - not the one I mentioned. For example, it doesn’t allow you to actually use 0.007 in the hours field. It rounds things down to zero.

Either way, I’m more interested to understand if my assumptions are correct here. But I’m thinking Fly GPUs + FLAME isn’t a good approach for me here. It’s probably cost saving only if you’d keep it running forever/have lots of users, which isn’t my case. I think a service that charges per request (I.e. image generated is better suited to my use case).

[khuezy]

Ya this is a little bit tricky. I think you might be better off using a 3rd party API for image generation until you hit some scale where your own machine would be cheaper. The 30s overhead of loading the model on top of the machine bootup time would eat into the savings until some threshold - whatever that is… you’ll have to experiment.