Large files seem to cause a large processing delay

Hi there, I run an file hosting service with @wowjesus. For a short description so you guys can get the gist of things, users can upload files on our service to be shared with a link.

We’ve been using Fly for a while now and it works fine with smaller files for the most part, but trying to upload a large file, like 1MB+, to our service seems to take an extremely long time, around 70 seconds, whereas uploading smaller files can take from 70ms-500ms which is expected. We’re coming from Google Cloud which handled these files fine, we moved to Fly as it was much cheaper.

What we’re wondering is if there is anything at all we’re doing wrong (such as VM sizes), that could be causing this huge processing delay for larger files? Or is it something to do with Fly?

Many thanks.

In this thread @wowjesus mentions that you’re writing directly over the private network instead of sending uploads requests to the primary region. Maybe this is related?

In that scenario, it’s possible that the upload is being transferred twice: once to the secondary region VM, then again across the private network to the primary.

Are uploads from the primary region also slow? Is there a way I could test your app?


Writing to the primary instance only happens with database calls, the uploaded files are handled on the current node.

  • Is there a way I could test your app?
    Yes of course, please visit our landing to register an account (

OK, thanks. What happens to the files once they are uploaded to the node?

A random string gets generated (name) and the response is sent back before parsing the file (or gets partially parsed if the user’s response parsing setting needs it), after that, the file gets uploaded to an AWS bucket and the write to the database happens.

I signed up and tried uploading, but got a 404 error. Tried in Chrome and Firefox.

Oh yes with our recently rewritten backend, we are facing a few issues, so uploading from the dashboard is no yet possible, here is a Postman collection to upload files:
Body and access_key query param needs to be inputted, you can create an API (access) key at: Fileglass — Dashboard

OK, thanks.

I was trying to setup an example with Nestjs, but I see that multipart form uploads are not supported with the Fastify adapter. How are you handling the uploads?

We use the fastify-multipart package

Will you add the fly-request-id header to your logs then trigger a slow upload for us? We can look at specific requests in logs. I see a fair number of errors caused by timeouts reading request bodies on your app, but I’m not sure these are exactly the same.

I just tried a 2.2MB file and it too 590ms:

I noticed this is proxying through Cloud Flare. It would be worth testing directly to without Cloud Flare in the mix. My upload was routed through Toronto (I’m in Chicago), so one possibility is that Cloud Flare is routing you through a region that’s not close.

After some discussion with the users experiencing this, turns out that they were being routed to Singapore (they are in Qatar), and Singapore is further away than European nodes (Frankfurt, Amsterdam), so that could be the issue, i’ll tweak the logs in a second

That’s bonkers! We have an edge in Dubai, so they should probably get sent there. But Cloud Flare does not tend to do the right thing.

What is the name of that edge? I don’t see it listed at Regions

We only have an edge there, so you can’t run workers. You’ll see a fly-region: dxb header on requests that come through that region.

I see, and are there any plans on making that available as a worker region? It would be OP since there aren’t any middle east regions.

Yep! We’d like to, possibly in the next 6 months.

Glad to hear!

1 Like

Did you all figure anything out with upload speeds?