There should be an option to filter out all spider and bot traffic from SiteCatalyst reports, very similar to what was available from the HBX User Agent Filtering option.
I think this should include a warning system that identifies suspicious IP Addresses (i.e. sening an enormous amount of traffic per day) and allow you to block that IP address from now on...
This is a great idea. We currently do this via a VISTA rule but this is at an additional cost, I think this should come as standard within Site Catalyst.
What, it's not already filtered out? Hmmm...
Yeah I was lead to believe spiders and bots are already automatically filtered out ... ?
We do see bots in our reporting that apparently execute the Java Script. Not many, but they are there. At this time, they do not seriously affect our reporting.
I have to agree that this is a big issue and an easy one to solve for Omniture. THey should simply maintain a list of known bot IPs and have a setting in the admin consule where we can click to have this IP filter applied to our data like a standard Vista Rule. And just like Vista rules; they would still get the server calls since it did hit their servers but we won't see the numbers in our data sets. That's a win / win for both parties. Clean data for us continued revenue for Omniture.
Bot/ spider filtering is critical for data accuracy, but we found out the hard way that this is not being done by Omniture. The cost to implement this via a Vista rule was also a surprise.
We requested that Omniture recreate the filter that was previously available through HBX, to no avail. In the end, we had to supply the bot/ spider list ourselves. They made a short list available, but it was incomplete and was not being updated.
Omniture should be using the IAB spider/ bot list (with regular updates) and implement this for all clients automatically.
Omniture can do this using a Vista rule, it is just a mnual task that needs to be done monthly. They can either use the IAB list of bots/spiders or in the UK ABCe publish a list of these. We are currently testing this facility which I believe should be offered to all customers as a matter of course - out of the box with no additional charge. Without credible data any insight is meaningless.