Level 10

Question

If duplicate leads are an issue, be aware of how Marketo determines which lead is active

Forum|Forum|10 years ago
April 4, 2016
8 replies
16569 views

Many of us deal with duplicate leads on a regular basis. Today, I just learned of something that's been causing issues for us, as we were always under the impression that Marketo considered the lead that was "updated last/most recent" as the active lead - and the lead that would be affected for things like list uploads, email sends, etc. That's not the case. This could have significant implications if you have a lot of duplicates in your instance. I just posted an idea that hopefully will get enough attention and change the way Marketo deals with duplicate leads:

NOTE: if you head over to the Idea page ( ), you'll note that a serious flaw has been identified - specifically around the way Marketo re-uses old LeadIDs (when the original lead is purged from Marketo) and allows new leads to take on these old IDs. This could have serious implications, for example, for those of us that leverage the API in our environments.

Marketo

This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.

Grégoire_Miche2

Level 10

Hi Dan,

Thx for posting this and the idea. Learnt something.

-Greg

Dan_Stevens_Author

Level 10

For your reference, here is a detailed breakdown of Marketo's duplicate rules processing:

Sending Batch Emails

If there are duplicates within your email list, only one email will go out. It will be the lowest Marketo ID (created first). This record is also used to populate tokens. Example:

Created	Updated	Marketo ID	First name	Email
3/13/2010	4/13/2010	2749	Elizabeth	beth@company.com
12/15/2012	3/15/2016	10936	Beth	beth@company.com

In this case, the email will be sent to the first record with Marketo ID 2749. It will say “Dear Elizabeth” if you use a token for first name. Also, the activity entries will be attached to that record (sent, delivered, opened, clicked). When the email click goes to a page with Munchkin code in it, any web visits will also be associated with this record.

[EDIT 1/30/2018]: apparently this is not the case. As you can see in the latest comments, here in the Idea related to this: , @Aaron Dear uncovered a serious issue/bug - especially if you have a lot duplicates in your instance (in some cases, these duplicates are intentional). Basically, there is no defined logic to determine which lead record is used when evaluating duplicates as part of a batch email campaign.

List Uploads

When a lead is included in a list import and the lead already exists in Marketo, it will affect the most recently updated record in Marketo (updated last) - in this case, the record with the first name "Beth". If you're intention was to ensure Beth's record had the most up-to-date information so that some of that info could be used in an email send, you're out of luck. Marketo will use "Elizabeth's" record for that send.

Sending Trigger Emails

While a batch email de-duplicates, trigger emails do not de-dupe because triggers “batch each record individually”. However, it is uncommon that trigger emails are used for batch emails, so this is not usually a problem. Also, it is rare that duplicate records will qualify for a trigger campaign at the same time.

Form Fill-outs

If someone fills out a form and their email address is in Marketo multiple times, the form submissions will be applied to the most recently updated record (by last modified date in the “updated” field). The data value changes in Marketo will only affect one record if there are duplicates; they will not affect the duplicate records. For example, if a record changes their title, the change will not occur for the duplicate records.

Assuming that the Marketo cookie information is included in the form (standard for Marketo landing pages or embedded Marketo forms), the web page visits will now be tracked on the most recently updated record. Luckily, the most recently updated record is often the record that is most up to date. (which is why the the way Marketo treats batch email sends is an issue).

Note: If there are duplicate records in Marketo and one of the records is cookied, the cookied record will be associated to the form fillout regardless if a duplicate record was updated last. As noted above, the form fill-out activity may not be associated to a record which was sent an email directing them to the form fill-out.

Web Activity

Web activity will only be associated with one record if there are duplicates. If a record has a duplicate record and he/she goes on the website, the munchkin code will only track activity for the record that most recently was cookied by submitting a form. If the visitor’s cookie is reset and visits the site again and fills out a form, the record may be associated to a different record in Marketo. Therefore, all web activity that is recorded may be spread across the duplicate records.

Merging Leads

When merging two leads, Marketo will always use the most recent one to define the "Created On" date. This is undesirable since most would prefer to keep the initial/first date of the older lead as the "Created On" date.

Thanks to Kristen Carmean for surfacing this comprehensive list with us!

Dan_Stevens_Author

Level 10

Here's a situation where you can mitigate the amount of risk (if duplicates are an issue). Let's say you have an event program with 25 registrants. 10 of those registrants exist as duplicates. But the most recent lead records are included in this program. When sending out a reminder email, for example, as long as you define your smart list properly (e.g., member of program; program status is registered; etc.), only those QUALIFIED leads will be sent the email. This is a perfect example where Marketo will, in fact, use the most recent lead record when sending out a batch email since that's the record that qualified in the smart list to begin with.

Hope this helps!

Jenn_DiMaria2

Level 10

Wow. Thanks for posting this. I always thought it defaulted to whatever was in SFDC (because that's what it always seemed like when I'd go to merge them). Learn something new every day

Dan_Stevens_Author

Level 10

Jenn - do you automatically sync all leads from Marketo to SFDC? If so, then your assumption may still be true. The issue arises when you have duplicate leads/contacts in SFDC (which, of course, also exist in Marketo).

Jenn_DiMaria2

Level 10

SFDC is our master and we don't sync all Marketo leads to it, but we still have lots of issues with duplicates. Namely because our main contact form on the website is an SFDC web-to-lead form and many of our customers think they can reach Support this way. Of course, that then creates duplicates, which makes my life super fun

In many cases with those duplicates, we then have a Contact on an Account and then a Lead. If they've merely submitted the W2L form, those are easy to merge, but I remember when we first migrated our database, there was some best practice from SFDC about holding a Lead separately from a Contact, despite being the same information. I wish I remembered the details, but it's such a data pain.

Dan_Stevens_Author

Level 10

As I posted within the idea I created ( ), one way to remedy this would be to offer an option in the Qualification rules of a batch smart campaign (this last option does not exist - this is just a mock-up):

Gerard_Donnell4

Level 9

Thanks for sharing this info! It definitely answers a few questions.

A

Anonymous

Thanks, everyone. Very helpful.

Dan_Stevens_Author

Level 10

@Niranjan Kumbi [PM], near the bottom of this thread ( ), you mentioned that you would investigate internally to determine if this is an issue. Do you have any updates for us?

@Sanford Whiteman

@Aaron Dear

Nicholas_Manojl

Level 8

The use case isn't really a one-size-fits-all approach though.

You have a very complete lead record that was updated 6 months ago.

You have a duplicate lead that was created 1 minute ago, but it contains just an email address.

In your scenario, you want to email the lead record that only has an email address.

I think the problem is much harder than this.. in reality we need a way to methodically judge the quality of a lead record. But how? It's a very difficult problem.

Dan_Stevens_Author

Level 10

Totally get that. The premise of the Idea was to have consistency across the entire environment - for batch/trigger email sends, list imports, form submits, etc. But that's not what I'm referring to here in my most recent post.

If you look at the Idea comments/replies - beginning with Aaron's post on January 29, 2018 - he uncovered a pretty serious flaw in the system where Marketo will reuse leadIDs (when leads are deleted/purged from the system) and allow new leads to assume those older IDs. This could have some significant implications for those of us that rely heavily on the API. As Sanford put it:

"This is far and away the most fatal and mind-boggling technical detail -- if indeed it's true -- I've heard about w/Marketo, ever."

Nicholas_Manojl

Level 8

well, I haven't really touched the API much in recent times. But my understanding was the leadID was on the way out and the GUID was on the way in. But I think broadly speaking you make some really excellent points, so I definitely support you on that.

Sign up

Login with SSO

Login to the community

Login with SSO

Scanning file for viruses.

This file cannot be downloaded