Skip to main content
July 10, 2015

Why Does My Smart List Count Not Match the Email Performance Report?

  • July 10, 2015
  • 7 replies
  • 6566 views

“Why does my smart list count not match the Email Performance Report?” If there was a ranking for questions we hear most frequently, this would surely be near the top. I imagine the support team must answer this one all of the time.

Let’s take a look at why these don’t match up.

Opens

First of all, hopefully you know by now that opens is an absolutely awful metric to rely on. They depend on someone loading a small pixel in the email. If they have images blocked or if they read the email in a preview pane, we won’t be able to track that as an open. That’s a problem universal to all email senders.

Now, imagine that this person who has images blocked clicked on the email. Logic would dictate to us that they must have opened the email also. Our Email Performance Report has this logic built in. The activity log does not. So there will be no Opened Email activity recorded in the person’s activity log, but the report will count them as having opened. Bam, instant mismatch in counts.

Delivered

But surely delivered is a reliable metric to look at in a smart list? Eh, not so much. Certain categories of soft and hard bounces only register after the delivered message. In this case, the activity log will show a Delivered activity followed by a Bounced activity. The report will register the person as delivered, but as soon as we get a bounced message, it will no longer count that email as having been delivered. This means your delivered numbers in your report could actually decline over time as we get more accurate information back from the email client.

The Basic Rules for Email Performance Report

  • Each email activity can be one and only one status: delivered, hard bounced, soft bounced, or pending.
  • If the email was opened, it counts as delivered.
  • If the email was clicked or unsubscribed, it counts as delivered and opened.
  • If the email was opened, we ignore bounce messages. If it was not opened, hard bounced wins out over soft bounced and delivered.

Deleted Leads

One final example. If you run a smart list for Opened Email, it would only show you a count for records that were in your database at that time. If you had previously deleted a record that had opened the email, the smart list would no longer count them. The email report, however, would still include that open in your count.

Moral of this story?

Your most accurate metrics are going to come from the email performance report.

This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.

7 replies

September 11, 2015

Hi,

I am not sure about your statement about permanent stats of deleted leads in the email report.

I am walking through the docs about reports and found an interesting answer from another member who says opposite thing:

Here's an update for you. Marketo support says: "Reports don't auto-refresh each time you open them the way Smart Lists do, so if you delete a bunch of leads, tomorrow you will still see the same data on the report--until you refresh it. Then the data in your report would change."

Re: When a lead is deleted, does it affect my email performance reports?

What is the right answer now? I am totally confused!

September 11, 2015

Good question. My information came from the product manager. I'll point out this conflict and see if I can get clarity on which is right.

September 11, 2015

I confirmed again that deleted leads are still shown in the email performance report. Sorry about that confusion.

September 14, 2015

Thanks for the information! This gives me peace.

If it was the other way the statistics does not make sense.

September 22, 2015

I did some tests on one of our emails and found out, that the numbers of sent emails are different depending on the report.

I compared the number of sent emails of the "Email Performance" report with the number of sent email in the camaign which sends out the tested email.

The test ran for one week, and I got different results for each day.

The email performance report always shows me less sent emails then the campaign results.

Why is that so? Is the performance report summarzing multiple emails from one lead?

This is the only way I can explain the difference.

September 22, 2015

What do you mean about it summarizing multiple emails from one lead? Are you looking at an email that's sent out via a trigger campaign? In a batch campaign, we will dedupe by email, so a single email address will only get sent one email, but in trigger campaigns, you get the email when you trigger it so if you have duplicates, the same email address could go through multiple times. It's true that the same person opening or clicking multiple times is aggregated in some of the reporting but I don't think you would see that behavior in sent emails if you have duplicates because of our automatic deduping on batch campaigns.

September 25, 2015

This confirms my assumption. Thanks