Level 10

Question

Webhook to create purposeful duplicates in Marketo is unreliable

Forum|Forum|8 years ago
June 14, 2017
3 replies
8868 views

As part of our purposeful duplicate process (when CRM contacts are created in MS Dynamics) a basic webhook is called to create a fresh new lead:

We're finding that this is working very inconsistently for us. Sometimes it works, sometimes it doesn't. What's odd is that we always receive a common error type "Duplicate Lead" Failed. Server Returned code 302" - even when the webhook runs successfully. When I asked Support about this, they came back with the following:

Engineering confirmed that the HTTP 302 message is not an error. Instead, it is the response code for a page redirect.
HTTP 302:
en.wikipedia.org/wiki/HTTP_302
With that being said; I've asked if Engineering can modify the wording being written in the activity log, to prevent any future confusion. I also pointed out that you are not aware of why the redirect is taking place in the first place, and asked if they could explain that as well.

But this doesn't explain why the webhook is working so inconsistently. I know @Sanford Whiteman commented last year that the loopback post is broken on some instances (The 3 Coolest Webhooks I've seen (and or) Implemented ). Could this be the issue? Any others experience this inconsistent behavior?

Marketo

This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.

SanfordWhiteman

Level 10

302 is the expected response from the legacy /save endpoint, though.

If it weren't, then old + noscript forms wouldn't work: the /save endpoint has to allow for browsers without JS to redirect to a follow-up page, and the only way to do that is via an HTTP redirect (should be HTTP 307 these days, but that's another matter). The new /save2 endpoint doesn't do this, because it requires JS support in the browser in the first place.

So I wouldn't be concerned about the 302. I'd be looking at whether the call is actually successful. If it's failing, are you sure you're not going over the limit of 1 form post every 2 seconds?

Dan_Stevens_Author

Level 10

We're still trying to find out what this issue is here. But when doing some further investigating, we came across another issue: when the webhook creates the duplicate lead, it will populate the email address with the email-address-temp (since you cannot create duplicate by populating the email address immediately for obvious reasons), but then the email address only exists when looking at the lead detail view. In the lead grid, the email address is blank. And the lead - which now resides in one of the country lead partitions - isn't even retrievable (e.g., via smart list) in that partition. Even when searching by last name. We're not sure of the extent of how large this issue is just yet.

As for ensuring we're not going over the limit of 1 form post every 2 seconds, how can we delay this? This would be most problemsome when a batch of contacts are added to CRM - which would trigger the webhook call. I don't think a wait step would affect this here.

SanfordWhiteman

Level 10

but then the email address only exists when looking at the lead detail view. In the lead grid, the email address is blank

Even after running it through the flow to copy Temp Email → Email? That doesn't make much sense, I mean, data sent to /save2 is literally the same path taken by a true client-side data.

However, there is something in your flow that I wouldn't do, and that is post the data without an Email. I'd send {{lead.id}}-{{system.datetime}} as the Email. You're going to change it anyway, and this creates a nice little audit trail while ensuring uniqueness.

As for ensuring we're not going over the limit of 1 form post every 2 seconds, how can we delay this?

You can't directly delay it -- as you point out, a Wait Step would not help, since the event of a flood of new contacts you'd just be delaying the whole flood by a certain amount of time.

If you can't batch up these 'hooks (which would require major rearchitecture) you can use the method I describe here: https://nation.marketo.com/message/154866-re-throttle-web-hook-requests#comment-154866

Jep_Castelein2

Level 10

Hi Dan,

I created the 'localhost' duplicate solution you are using in 2011 and it has always been an unsupported solution. However, it has worked successfully ever since. The '302' response is the expected response. It is because the Form is trying to redirect you to the thank you page. There may be a delay in the new record showing up in the search due to the indexing delay. It is fine to use this solution as long as it works, but indeed, if anything goes wrong, support won't be able to help with this.

It would be a better solution solution to use a Webhook to call a proxy service outside of Marketo, which then creates the new Lead record in Marketo using the REST API. Azure Functions would be an ideal environment to build such a service (Azure Functions—Serverless Architecture | Microsoft Azure ). Potentially even better is to add the Lead to a Static List and have the external service poll the List. This is more scalable and allows for better error handling and retries.

Nevertheless, based on the write-up above I am not sure that anything is going wrong in your setup. The 302 message is expected and the Lead not showing up in the search could be due to indexing delay, If there really is a Lead visibility issue, it should also occur with Leads created via normal Form submissions.

Hope this helps.

Jep

Dan_Stevens_Author

Level 10

Hi @Jep Castelein - appreciate your reply here. We learned a while back that the 302 message was not an actual error. But what we found was that there were certain leads that didn't get duplicated - even though they qualified for the trigger campaign that kicks off the process. Who knows - maybe it was one of the issues with trigger campaigns - and not the webhook - but nevertheless, the duplicate didn't get created. There was no pattern or timeframe - completely random/inconsistent.

To be on the safe side, it's probably best that we develop a more scalable, reliable and supported approach. We have some professional services hours that we need to use before the end of August. Do you have the time (and be willing) to help build this solution? I have a call with our internal team next week - to see if it's something they can support. But they may not be familiar with Marketo or working with the API. Send me an email if you want to discuss further (daniel.r.stevens@avanade.com).

Ishita_Chawra

Hi all,

Have the same problems stated here and we use MSD 365 online:

1. we create lead duplicates via webhook

2. are seeing the error 302

3. duplicate lead email address is missing in Marketo and CRM

Questions:

1. How to check if the webhook is working?

2. 302 error, not a problem? is this still true?

3. Any answers for the missing email address?

Would really appreciate if you could share any better solution you found in these years.

Thank you for sharing the details.

IC

@Dan Stevens.‌ @Jep Castelein @Trish Keenan

SanfordWhiteman

Level 10

302 is the expected HTTP response code from the forms /save endpoint. Not an error.

Sign up

Login with SSO

Login to the community

Login with SSO

Scanning file for viruses.

This file cannot be downloaded