Your achievements

Level 1

0% to

Level 2

Tip /
Sign in

Sign in to Community

to gain points, level up, and earn exciting badges like the new
BedrockMission!

Learn more

View all

Sign in to view all badges

Filter out Spider and Bot traffic from SiteCatalyst

Avatar

Avatar
Boost 500
Level 10
vabeachkevin1
Level 10

Likes

556 likes

Total Posts

40 posts

Correct reply

0 solutions
Top badges earned
Boost 500
Boost 50
Boost 5
Boost 3
Boost 250
View profile

Avatar
Boost 500
Level 10
vabeachkevin1
Level 10

Likes

556 likes

Total Posts

40 posts

Correct reply

0 solutions
Top badges earned
Boost 500
Boost 50
Boost 5
Boost 3
Boost 250
View profile
vabeachkevin1
Level 10

11-02-2010

There should be an option to filter out all spider and bot traffic from SiteCatalyst reports, very similar to what was available from the HBX User Agent Filtering option. 

19 Comments

Avatar

Avatar
Employee
BretGundy
Employee

Likes

0 likes

Total Posts

99 posts

Correct reply

0 solutions
View profile

Avatar
Employee
BretGundy
Employee

Likes

0 likes

Total Posts

99 posts

Correct reply

0 solutions
View profile
BretGundy
Employee

08-04-2010

Excellent comments here. While bot traffic is currently a small percentage, we're looking into some ideas now, as more bots and spiders are starting to execute JavaScript and request images.

Avatar

Avatar
Boost 50
Employee
benjamingaines2
Employee

Likes

271 likes

Total Posts

874 posts

Correct reply

0 solutions
Top badges earned
Boost 50
Boost 5
Boost 3
Boost 250
Boost 25
View profile

Avatar
Boost 50
Employee
benjamingaines2
Employee

Likes

271 likes

Total Posts

874 posts

Correct reply

0 solutions
Top badges earned
Boost 50
Boost 5
Boost 3
Boost 250
Boost 25
View profile
benjamingaines2
Employee

22-11-2010

Marking this "In Progress," per Bret Gundersen.

Avatar

Avatar
Applaud 5
Level 1
miikka
Level 1

Likes

0 likes

Total Posts

2 posts

Correct reply

0 solutions
Top badges earned
Applaud 5
View profile

Avatar
Applaud 5
Level 1
miikka
Level 1

Likes

0 likes

Total Posts

2 posts

Correct reply

0 solutions
Top badges earned
Applaud 5
View profile
miikka
Level 1

08-12-2010

This is very good idea, not least regarding the Google Preview "spider" bloating traffic. I'd like to see this option also in Data Warehouse.

Avatar

Avatar
Boost 5
Level 2
jamest0
Level 2

Likes

19 likes

Total Posts

13 posts

Correct reply

0 solutions
Top badges earned
Boost 5
Boost 3
Boost 10
Boost 1
Applaud 5
View profile

Avatar
Boost 5
Level 2
jamest0
Level 2

Likes

19 likes

Total Posts

13 posts

Correct reply

0 solutions
Top badges earned
Boost 5
Boost 3
Boost 10
Boost 1
Applaud 5
View profile
jamest0
Level 2

18-07-2011

Any update on this growing issue?

Avatar

Avatar
Employee
BretGundy
Employee

Likes

0 likes

Total Posts

99 posts

Correct reply

0 solutions
View profile

Avatar
Employee
BretGundy
Employee

Likes

0 likes

Total Posts

99 posts

Correct reply

0 solutions
View profile
BretGundy
Employee

27-07-2011

We are making progress on bot filtering, but I can't provide a timeline at this point. 

Avatar

Avatar
Boost 50
Level 6
cisco_webteam
Level 6

Likes

145 likes

Total Posts

34 posts

Correct reply

0 solutions
Top badges earned
Boost 50
Boost 5
Boost 3
Boost 25
Boost 100
View profile

Avatar
Boost 50
Level 6
cisco_webteam
Level 6

Likes

145 likes

Total Posts

34 posts

Correct reply

0 solutions
Top badges earned
Boost 50
Boost 5
Boost 3
Boost 25
Boost 100
View profile
cisco_webteam
Level 6

10-10-2011

Yeah, i can't believe what's available in HBX is not available even in SC 15

Avatar

Avatar
Level 1
jdnyland
Level 1

Likes

0 likes

Total Posts

86 posts

Correct reply

0 solutions
View profile

Avatar
Level 1
jdnyland
Level 1

Likes

0 likes

Total Posts

86 posts

Correct reply

0 solutions
View profile
jdnyland
Level 1

03-11-2011

The bot handling reporting is looking really good.  Let me know if you would like to try it out in the beta environment.

Avatar

Avatar
Ignite 1
Level 4
jane_mason
Level 4

Likes

55 likes

Total Posts

13 posts

Correct reply

0 solutions
Top badges earned
Ignite 1
Boost 50
Boost 5
Boost 3
Boost 25
View profile

Avatar
Ignite 1
Level 4
jane_mason
Level 4

Likes

55 likes

Total Posts

13 posts

Correct reply

0 solutions
Top badges earned
Ignite 1
Boost 50
Boost 5
Boost 3
Boost 25
View profile
jane_mason
Level 4

16-12-2011

We have a very large amount of MSN bot traffic in our data - an MSN bot that executes java script. We had the same problem with Google preview bot last Nov.

 

Omniture needs to prevent bots that execute java script from getting in our data and more importantly we are PAYING for the server calls. How about refund???????? (lol)

Avatar

Avatar
Level 1
jdnyland
Level 1

Likes

0 likes

Total Posts

86 posts

Correct reply

0 solutions
View profile

Avatar
Level 1
jdnyland
Level 1

Likes

0 likes

Total Posts

86 posts

Correct reply

0 solutions
View profile
jdnyland
Level 1

26-04-2012

This one is now implemented.  You can turn it on in the admin console and enable the IAB bot list with a single click.  Custom bot rules can also be created for user agents or IP addresses.