A

Anonymous

Solved

Spam filters registering clicks?

Forum|Forum|11 years ago
May 20, 2014
15 replies
29209 views

Has there been any problems with spam filters scanning emails and registering clicks as they follow the links in the email? We are getting false positives on our email clicks.

Marketo

This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.

Best answer by

There are several posts here on Marketo about this issue, and my firm has been digging into it a lot over the last few days. The short answer is that yes, this does indeed happen - spam filters (like Barracuda) / bots / junk mail algorithms do indeed click on links in emails (see this interesting blog post from 2013 regarding the issue - Barracuda calls this "multilevel intent analysis"). The spam filter is looking for redirection or malware or something like that. There isn't a whole ton that we marketers can do about it, though. Here is what we've done and found:

First thing we did was download the entire Marketo activity log using the API, put it in a database, and started dissecting the "Click Email" event types. We also sat down with the system administrator here to review some of this data. In short: there is nothing in the User Agent, Platform, Device, etc. that will help spot these.
Then we started looking at the timing: what about people who click before they open? What about people who click really quickly after the "Send Email" activity is logged? Well...the "Send Email" event isn't indicative of when, exactly, the email leaves Marketo's servers, so that's not really accurate - you can't spot bots based on that.
The best way we've found right now is to include a one-pixel picture / link on the email - invisible to just about everyone (as suggested here). Anything that clicks on such a tiny little pixel you can consider a bot. True; someone might not load images and see a box, but most people won't see it at all.
Another possibility: see if you have a bunch of clicks that all happen at the same time (or people clicking every link in an email, every week - would a real person really need to read your Privacy Policy week-in and week-out?). Those are probably bots...but I personally would want to download the data into a real database before attempting this kind of query.
One more (really complex) possibility: when we went to our sysadmin (the guy who runs our own company's Barracuda machine) about a lot of these issues, he started to "ping" some of the IP addresses included in the suspicious "Click Link In Email" activities. One or more of them shot back a response indicating that it was a Barracuda box. If you are really, super-duper concerned with this problem, it should be possible to download all Marketo activities via the API and write some custom script / code to extract the IP addresses from the Marketo "Click Email" events and then to periodically ping all these servers to see if you can get them to self-identify as a spam filter (parse the text-strings of the responses for incriminating evidence).

We have not done this last thing, as our "one-pixel" solution has indicated (at least over the last two weeks) that it's likely not a major issue. Perhaps some day, when our organization has unlimited resources (heh), we will pursue this last option, but the reality is that we have a lot going on and better things to do to add more value to our marketing efforts.

I would also like the data to exist in a perfect world - one where our Users validate our TRON Data Discs and we can take down the evil Master Control Programs while we're on our light-cycles on the grid - but that gleaming world of perfect, neon data does not exist. For most of us, I would guess this statistical aberration will not significantly affect our analysis of content effectiveness.

Hope this helps.

A

Anonymous

Accepted solution

There are several posts here on Marketo about this issue, and my firm has been digging into it a lot over the last few days. The short answer is that yes, this does indeed happen - spam filters (like Barracuda) / bots / junk mail algorithms do indeed click on links in emails (see this interesting blog post from 2013 regarding the issue - Barracuda calls this "multilevel intent analysis"). The spam filter is looking for redirection or malware or something like that. There isn't a whole ton that we marketers can do about it, though. Here is what we've done and found:

First thing we did was download the entire Marketo activity log using the API, put it in a database, and started dissecting the "Click Email" event types. We also sat down with the system administrator here to review some of this data. In short: there is nothing in the User Agent, Platform, Device, etc. that will help spot these.
Then we started looking at the timing: what about people who click before they open? What about people who click really quickly after the "Send Email" activity is logged? Well...the "Send Email" event isn't indicative of when, exactly, the email leaves Marketo's servers, so that's not really accurate - you can't spot bots based on that.
The best way we've found right now is to include a one-pixel picture / link on the email - invisible to just about everyone (as suggested here). Anything that clicks on such a tiny little pixel you can consider a bot. True; someone might not load images and see a box, but most people won't see it at all.
Another possibility: see if you have a bunch of clicks that all happen at the same time (or people clicking every link in an email, every week - would a real person really need to read your Privacy Policy week-in and week-out?). Those are probably bots...but I personally would want to download the data into a real database before attempting this kind of query.
One more (really complex) possibility: when we went to our sysadmin (the guy who runs our own company's Barracuda machine) about a lot of these issues, he started to "ping" some of the IP addresses included in the suspicious "Click Link In Email" activities. One or more of them shot back a response indicating that it was a Barracuda box. If you are really, super-duper concerned with this problem, it should be possible to download all Marketo activities via the API and write some custom script / code to extract the IP addresses from the Marketo "Click Email" events and then to periodically ping all these servers to see if you can get them to self-identify as a spam filter (parse the text-strings of the responses for incriminating evidence).

We have not done this last thing, as our "one-pixel" solution has indicated (at least over the last two weeks) that it's likely not a major issue. Perhaps some day, when our organization has unlimited resources (heh), we will pursue this last option, but the reality is that we have a lot going on and better things to do to add more value to our marketing efforts.

I would also like the data to exist in a perfect world - one where our Users validate our TRON Data Discs and we can take down the evil Master Control Programs while we're on our light-cycles on the grid - but that gleaming world of perfect, neon data does not exist. For most of us, I would guess this statistical aberration will not significantly affect our analysis of content effectiveness.

Hope this helps.

Venus_Wills

Level 3

Thanks for the information, Matt. For us, it appears that the spam filter doesn't click on all the links but just one. The click is also registering in the activity log before the "delivered email" is registered. So the one-pixel image link won't work for my case. However, I am noticing that none of these links lead to a "visit page" activity (and it should). I will try to use that filter for lead scoring. Please keep us posted if you uncover additional information about this.

SanfordWhiteman

Level 10

I am noticing that none of these links lead to a "visit page" activity (and it should).

... as mentioned in the other thread, you shouldn't expect a VWP activity just because you saw the Clicked Email. The anti-spam technology in use isn't going to download all libraries and do JS logging (in fact it would be a pretty good DoS attack against such services if it did).

Chris_Saporito

Level 7

We just ran into this same issue today. After reading through Matt's response, not exactly sure what our next step should be. Lots of good info though!

Casey_Grimes2

Level 10

Matt gives a lot of great advice here, but I did want to just add as a footnote/call to action for anyone else annoyed by the current situation: I've been talking to a few different filtering companies about adding some unique, filter service-only string to their UA when checking links (they normally spoof specifically as IE/Win7) in order to correctly differentiate human vs. machine clicks. The problem affects more than marketing automation platforms; I find myself continually explaining this to transactional email provider users, for instance.

I'd highly encourage people to go bother Cloudmark/Symantec/Barracuda as well so I don't seem like a lone weird geek on this point.

SanfordWhiteman

Level 10

filter service-only string to their UA when checking links

Doesn't make sense though. That would make their service worthless because it's supposed to be prechecking for hostile sites. All a site would need to do is UA sniff and return a non-malicious payload. They need more randomness, not less (my experience is it isn't the same UA at all).

A

Anonymous

One of my big concerns is that we're passing this activity into SFDC, which our inside reps are using to follow up on what seemingly look like responses to email campaigns. Have others resulted to just disabling that type of activity from being passed to SFDC?

SanfordWhiteman

Level 10

One of my big concerns is that we're passing this activity into SFDC, which our inside reps are using to follow up on what seemingly look like responses to email campaigns. Have others resulted to just disabling that type of activity from being passed to SFDC?

If you mean replicating all activities (well, one per lead per activity type per day, technically), yes, you would have to turn that off if you want to perform any filtering. If you create a Smart Campaign that seems to work, like the one Conor proposes above, you can use that to create SFDC tasks and/or Interesting Moments that correspond to the filtered activities/sequences of activities.

A

Anonymous

Marketo support suggested we trigger off of Opens Email and Clicks Link in Email, but the issue we're seeing is that this is still generating false positives, because those if a company has the filter, they're still going to register a click. So if they even just open an email, it looks like they've satisfied both requirements. In many cases, the clicks are even logging far ahead of the time when someone opens.

Has anyone worked out a way to tell Marketo that the Email has to open BEFORE the click?

B

Beth_Rotach

Level 2

Just a quick note - we've been emailing pretty aggressively with Marketo support regarding this issue. We also found out that MANY other ESPs provide this "click filtering" as part of their service because it happens so often to folks. We recently spoke with about 8-10 other ESPs that automatically (and very easily) filter these clicks for their clients. Apparently the ESP can easily filter clicks by IP/known barracuda IPs and code. Marketo deliverability team assured that their are now working with the product side to try to implement this ASAP, especially because so many people are asking about it - KEEP ASKING!

SanfordWhiteman

Level 10

Apparently the ESP can easily filter clicks by IP/known barracuda IPs and code.

Nope, they cannot (and are not) doing it this way. It's a preposterous claim, and any ESP that claims to reliably filter automated clicks this way is lying.

Rather, they are using a mechanism that is closer to what I have described in this thread and elsewhere. It is possible for Marketo to attempt the same and achieve a high degree of coverage. But to the degree that it works, it is because of the defensive coding used by the mail scanner (to prevent an amplification attack against the scanner itself) and not because of any special brilliance or detective work by the ESP.

Robb_Barrett

Level 10

OK, here's a situation I was presented with this morning:

We use Marketo for our Contact Us page and we have workflows that fire off alerts. One of the alerts has two links: I've Handled This, or I Need to Re-Route This. There is a follow up workflow that is triggered on the I've Handled This click. One of my colleagues asked for help because a click is firing off the follow-up workflow 4 times.

One of the logs I looked at shows the initial alert delivered Sunday night. On Monday morning, the alert was clicked at 10:34am and there was a corresponding VWP. Then, also at 10:34 I see two more Clicks Link and only one VWP.

My first thought is that I have a double-clicker. I created a lead for myself following the process. I was very slow about waiting to click on the link the first time. I did, it registered one click, one VWP, then nothing more. I put in a filter for Not Clicks Link In Email in Past 1 minute to see if that would help. Then, about 5 minutes after my first click I double clicked. It registered two clicks and two VWPs.

A minute later, it registered 3 clicks and 1 VWP. These were not by me or anyone else.

Now, it's work noting that we have a URLDefensePoint system in place. All links in emails are re-coded by the server with DefensePoint to check. I'm thinking that it's testing the link for us to see what happens prior to allowing the browser to go to the link.

Thoughts?

@Sanford Whiteman

Robb Barrett

SanfordWhiteman

Level 10

Now, it's work noting that we have a URLDefensePoint system in place. All links in emails are re-coded by the server with DefensePoint to check. I'm thinking that it's testing the link for us to see what happens prior to allowing the browser to go to the link.

I think you're correct.

And this is a case where, unlike inbound scanners I know of, the outbound/opt-in service can afford to perform deep scanning because they only see a subset of links. That is, they are actually following the JS redirect, so they generate a Visit Web Page as well as a click. (Inbound scanners can't afford to do this because from a defensive programming standpoint they could tie up their own resources.)

Robb_Barrett

Level 10

This is the same thing I'm seeing on spam traps too. You get a flurry of clicks and few VWPs.

Robb Barrett

Carmi_Lopez-Jo1

Adobe Employee

Thanks to @Kiersti Esparza, Manager of Privacy/Deliverability at Marketo, who has just posted a community article on this topic. Understanding a Spike in Click Activity

Cheers!

Carmi