Your achievements

Level 1

0% to

Level 2

Tip /
Sign in

Sign in to Community

to gain points, level up, and earn exciting badges like the new
Bedrock Mission!

Learn more

View all

Sign in to view all badges

Coffee Break: Join us on Wednesday, April 5th at 10am PT to chat with Ben Gaines, Director of Product Management for Adobe Analytics. Ben will be online to answer your Analytics and Adobe Summit questions.

Easy way to filter traffic from Cloud Services (AWS/Azure etc)

Avatar

Level 1

14-09-2014

We have a significant volume of automated traffic hitting our site from AWS instances (likely crawlers). We've looked in the raw logs for something to identify this traffic, but there it comes from a wide range of IP addresses, sometimes showing the ISP domain as amazonaws.com, mostly from their Virginia data centre and doesn't give a helpful user agent.

 

To filter this out we'd need to create a complex IP filter rule and update it each time AWS acquire new IP addresses. Given that this is a common rule that many clients will require, and given the rapid growth of cloud compute services, it would make sense for Adobe to offer and maintain a bot rule to exclude AWS/Azure/rackspace etc.