Easy way to filter traffic from Cloud Services (AWS/Azure etc)

Avatar

Avatar
Boost 3
Level 1
dnw123
Level 1

Likes

4 likes

Total Posts

1 post

Correct reply

0 solutions
Top badges earned
Boost 3
Boost 1
View profile

Avatar
Boost 3
Level 1
dnw123
Level 1

Likes

4 likes

Total Posts

1 post

Correct reply

0 solutions
Top badges earned
Boost 3
Boost 1
View profile
dnw123
Level 1

14-09-2014

We have a significant volume of automated traffic hitting our site from AWS instances (likely crawlers). We've looked in the raw logs for something to identify this traffic, but there it comes from a wide range of IP addresses, sometimes showing the ISP domain as amazonaws.com, mostly from their Virginia data centre and doesn't give a helpful user agent.

 

To filter this out we'd need to create a complex IP filter rule and update it each time AWS acquire new IP addresses. Given that this is a common rule that many clients will require, and given the rapid growth of cloud compute services, it would make sense for Adobe to offer and maintain a bot rule to exclude AWS/Azure/rackspace etc.