Leads failing to qualify for triggers | Community
Skip to main content
L
Liliana_Cheng1
Level 4
March 7, 2017
Solved

Leads failing to qualify for triggers

  • March 7, 2017
  • 5 replies
  • 6493 views

Hey community, our instance has been experiencing major delays (two to four hours) in our triggers. Some leads who fill out a form do not qualify for the triggers they're supposed to, sometimes for hours, sometimes for over a day. This issue is not isolated to one particular form or trigger, it's across our instance and it's inconsistently happening because some of my test leads have qualified for the trigger without an issue.

I've spoken to two Marketo support reps who confirmed that our triggers are set up correctly to capture, and the affected leads SHOULD be qualifying for the triggers. They also confirmed that we do not have a heavy volume of leads in the trigger queue. I am still waiting for a manager to update me but this is troubling.

What confuses me is when leads just sit in Marketo for two+ hours without any further activity will suddenly qualify for their trigger. No further actions were taken from the lead and I didn't manually request the campaign but they qualified - why is this happening? So most of the leads who failed to trigger immediately yesterday are okay after four hours... Is anyone else experiencing this massive delay in triggered behaviors?

If it makes a difference, I noticed this yesterday 3/6/2017 around 1:50pm Eastern and leads kept failing to qualify until 3pm. Another round of trigger failures happened between 7pm and 9pm. Most of them were caught up this morning but the most recent round of failures started around 11:30am today.

Thanks.

    Best answer by Liliana_Cheng1

    5/1/17 UPDATE - this issue looks like it's been resolved (for now). We had a call with our account exec and a support escalation manager this past Friday, and they said the engineers planned to push a patch over the weekend. Since then, I've been monitoring and testing and our triggers seem to be working as they're built to be.

    They haven't given us an in-depth analysis of what happened, but they said it was related to "the high speed database used to capture activity and how it was tied to trigger processing". The patch that was pushed this weekend was to remove the dependency. I'm not sure what their plan is after this (whether or not they'll tie these two problematic things together again).

    If you are still seeing issues related to delayed triggers, you should contact support with some examples so they're aware this is still an issue for you.

    5 replies

    Dan_Stevens_
    Dan_Stevens_
    Level 10
    March 7, 2017

    We encountered something similar a few months ago (For those of you on Corona (part of Project Orion), are you experiencing issues explained here? ), and occasionally experience this behavior.  I wonder if this has anything to do with the Orion migration.

    L
    Liliana_Cheng1Author
    Level 4
    March 7, 2017

    Thanks for the link, @Dan Stevens​. I added it to the case that I opened and asked if it was related to the Orion migration. I see that Ajay Awatramani​ replied to your discussion saying this bug was patched in November, but you mentioned that you still experience this behavior occasionally. How does your team handle these occasional delays? Does Marketo know this is still happening to your instance even after their patch?

    Right now I'm just refreshing a smart list that catches leads who filled out any form, but do not have a SFDC created date, and then manually reconciling them using the "request campaign" action for the triggers they were supposed to qualify for. I'm also a little frustrated by the lack of communication from the support team so far - the ticket was pushed up to a manager over 24hrs ago and I haven't heard anything from them even though the reps I spoke to this morning said there are new internal notes on my case...

    Josh_Hill13
    Josh_Hill13
    Level 10
    March 7, 2017

    Pretty typical. Look in Campaign Inspector for a list of triggered flows and see if you can shut some down. The slowdowns occur on the Trigger Eval Queue and often the cause is Visits Web Page, Lead is Created, or DVC.

      SanfordWhiteman
      SanfordWhiteman
      Level 10
      March 8, 2017

      Liliana, in addition to the great tips from experience from Dan & Josh, bear in mind that trigger backlogs -- though always unwanted -- can theoretically happen regardless of the load on your instance or pod, by virtue of the very fact that there is a queue.

      That is, when a system has multiple services handing off data to each other -- as any scaleable system must -- instead of a direct throughline from the client to all the back-end servers, that means more moving parts. And it also means temporary failure of any of the parts that enqueue and dequeue data can result in a later "replay" of delayed actions. That replay may seem mysterious, even impossible, but actually is due to the (relative) resilience of the architecture. Of course we don't want any failures, but when they happen it's reassuring that the queue pipeline continues where it left off instead of losing data.

      L
      Liliana_Cheng1Author
      Level 4
      March 9, 2017

      Thanks for your insight, @Sanford Whiteman​. The issue isn't just the significant delay in triggers, but there are a few test leads who are still just sitting there without qualifying for the trigger that they should have. There might have been more leads that didn't qualify for their triggers, but I manually reconciled the legit leads so our sales team could work them.

      Molly_Cousins
      Molly_Cousins
      Level 2
      April 20, 2017

      I'm experiencing this issue as well - and on the Filled Out Form trigger, to boot! Nothing amiss in Campaign Queue for us...

      Has this issue resolved for you @Liliana Cheng

        L
        Liliana_Cheng1Author
        Level 4
        April 20, 2017

        @Molly Cousins​, Marketo acknowledged that this is "an issue with their infrastructure causing performance delays" and said it could take months to roll out upgrades to their infrastructure. Incredibly frustrating for us and we are considering other options...

        As of last week, the frequency and length of delays had slowed down for us - we noticed it happening once a week or so, and the leads would start moving through the triggers within two hours or so. For better or for worse, they had told us that these trigger delays seemed to be limited to just a handful of instances so it was not a widespread issue.

        But today is a totally different story. Today has been significantly worse - my test leads from 10am still have ZERO activity on them apart from the original filled out form activity and the list of leads failing to qualify has grown. The senior support rep on our case advised that there was a server outage today that is affecting performance across a lot more instances. They had a P1 alert banner in the support portal that talked about performance issues all morning and most of this afternoon, but it's gone now.

        I suggest that you open a ticket right away so they know your account is affected as well. They are planning to "replay the triggers" today but they have not confirmed what time this will happen.

        Edit: never mind, I still see the P1 alert banner in the support portal. And my list of leads who are failing to qualify for their triggers keeps growing. They haven't fixed anything yet.

        Molly_Cousins
        Molly_Cousins
        Level 2
        April 20, 2017

        @Liliana Cheng Wow. I opened a P2 ticket 3+ hrs ago and hadn't heard anything back so I guess that explains their slow response. Thanks for the info! This is nuts.

        Dan_Stevens_
        Dan_Stevens_
        Level 10
        April 27, 2017

        Here's an official statement by Marketo Support: Marketo Service Interruption Update

        Josh_Hill13
        Josh_Hill13
        Level 10
        April 27, 2017

        Support says it was fixed at 1028am. Sounds like there's still work to do though.

        Open a ticket if you aren't sure the leads were processed.

        Dan_Stevens_
        Dan_Stevens_
        Level 10
        April 27, 2017

        We never got any official communication around this. In fact, the link I shared here didn't even mention the specific pods that were affected (or if it was system-wide).

          L
          Liliana_Cheng1AuthorAccepted solution
          Level 4
          May 1, 2017

          5/1/17 UPDATE - this issue looks like it's been resolved (for now). We had a call with our account exec and a support escalation manager this past Friday, and they said the engineers planned to push a patch over the weekend. Since then, I've been monitoring and testing and our triggers seem to be working as they're built to be.

          They haven't given us an in-depth analysis of what happened, but they said it was related to "the high speed database used to capture activity and how it was tied to trigger processing". The patch that was pushed this weekend was to remove the dependency. I'm not sure what their plan is after this (whether or not they'll tie these two problematic things together again).

          If you are still seeing issues related to delayed triggers, you should contact support with some examples so they're aware this is still an issue for you.

          Terms & ConditionsAccessibility statement

          Loading footer...