Block bots based on user agent

[deano-fury]

I have been getting scraped by a bot claiming to be the Amazon Bot for over a week, its stopping my machines sleeping. Is there a way of blocking certain user agents at the fly proxy level so they don’t keep waking my machine up? I have put rules into nginx to return a 403 instead so its now saving myself bandwidth, but the whole reason of moving the service to fly was to only pay for what my legitimate usage was.

Thanks

Dean

[khuezy]

“Fly isn’t in the web application business”.

Have you tried adding WAF?

[halfer]

Welcome to the internet

I don’t change any settings at the fly proxy level, so can’t advise on that. But you could put a small always-on proxy of your own in front of your sleeping machines; you’d be paying a low fixed cost for this, but it would stop your several, larger, machines from being incorrectly woken up.

[deano-fury]

@halfer

I don’t change any settings at the fly proxy level, so can’t advise on that. But you could put a small always-on proxy of your own in front of your sleeping machines; you’d be paying a low fixed cost for this, but it would stop your several, larger, machines from being incorrectly woken up.

Thanks Kheuzy, HaIfer, I might have to do that. Fly already does so much clever stuff at the proxy level, I would have thought that would have been a nice feature to have, could even be a chargeable feature.