Bot Traffic - GoogleBot

[mubarakd5726232]

GoogleBot accounts for ~17% of all our server requests (over 12 months) which is a sizable cost. I appreciate Adobe still has to process bot activity requests and make this data available for reporting, but is it really needed for GoogleBot ie could Adobe not drop any google requests or allow the option to be able to do so via Processing Rules ?

[harish_kumar · Accepted solution]

If you have enabled IAB bot filtering on your reportsuite, Adobe filter bot which are identified by IAB as a Bot. You can setup custom Bot rules as well. To update or upload bot rules, navigate to Analytics > Admin > Report Suites. Select the correct Report Suite, and then go to Edit Settings > General > Bot Rules.

Refer the below document how Adobe bots rule work :

Bot Rules overview

[PratheepArunRaj]

Dear Hari,

Do we need to submit the request to IAB to get the international Spiders & Bots List?

Also, will it reduce the Server Calls Cost? Guess it will be still billed but will not be added to our traffic for reporting.

Thank You!

Arun

[harish_kumar]

PratheepArunRaj​ If can IAB list is required, you may need to submit request with IAB for complete list of IAB identified bots. Since hits are getting collected at Adobe Servers, it will be charged as per the cost set for each server call though hits are identified as bots. Bots report in Report & Analytics can help you get count of bots identifier in the report suite but those hits will not be considered for other reporting.

[PratheepArunRaj]

Dear Hari,

Thanks, the answer is comprehensive.

Dear Mubarak,

Hope this is fine with you.

Thank You!

Arun

[mubarakd5726232]

This does not address my issue.  The bot is being filtered, however at the expense of ~17% of our allocated server call usage, again I appreciate this is because the hit is still being processed.  What I'm querying/suggesting is an option to let the customer decide if they wish to have hits from their bots list not processed or in a way to not incur additional cost. 
Whilst to an extent customers could put blocks in place for certain bots to avoid incurring Adobe processing costs, this would not be something they should to for Googlebot to avoid any referral implications.  It's why we need a solution off Adobe to avoid having to incur additional costs (for no real benefit)

[PratheepArunRaj]

Dear Mubarak,

Understand your concern on Server Call, but the option is not available right now. Kindly check with your Account Manager to raise it to Adobe. Update us if you are able to get a solution!

Thank You!

Arun