I’m running a web scraper for my application, and I’m noticing that I have no issues with running the scraper on my M3 Pro laptop, but getting lots of HTTP request failures when running it on the Fly box.
With that said, it’s a basic, hobby level lowest spec fly instance - but I’m curious to know if there’s a way to determine if I’m being throttled by fly due to it’s run speed?
Not sure if that sounds silly or not to be honest…
You’ll have to be more specific on the errors. Your throttling might be due to your machine running low on memory.
I don’t believe it’s a memory issue as I’ve had those before, and Fly notifies me if it runs out of memory. They’re HTTP requests, server runs at about 70mb/s and the server has a total of 512mb.
Running low on memory isn’t necessarily the same as OOM crashes. Look for any spikes when your puppeteer/playwright/etc opens a new tab, it could eat up memory but not crash (OOM) which will cause your app to throttle significantly.
I assume there’s a memory graph I can watch when it runs? This is a Swift project.
Go to your app’s dashboard and click on the Metrics tab. You can launch the Grafana view from it.
Yea memory looks fine, you have to provide the error logs my guy.
This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.
