Coming to Launch: Rule Component Sequencing

[#1]

This change was released on July 7th, 2020.

--

We've made a significant update to the Launch browser runtime to add a new capability. This article will clearly describe the legacy behavior, the new behavior, and some things you should consider as you make the move.  Let's dive in!

1) Legacy behavior

When a rule contains multiple actions, Launch will call those actions in the same order that you have them organized in the interface.  In this example, Launch will call Action A, then Action B, then Action C.

What Launch does not do is wait for Action A to finish before it calls Action B. If Action A is synchronous, then it doesn't matter much. But if Action A is asynchronous, then Action B probably starts to run while Action A is asynchronously carrying out its task. Over the long-term, we'd expect to see that sometimes Action A finishes before Action B, but sometimes it finishes afterward. The same is true for Action B and Action C.  It looks kinda like this:

As an illustration, our users frequently have trouble with this when using the Custom Code action from the Core extension.  When these actions are not on a Library Load or Page Bottom rule, they are stored as sub-resources of the main library (which means in separate files) and retrieved over the network when needed.  Retrieving a file over the network is an asynchronous operation.  This causes issues for users when the next action is dependent on whatever they were doing with their Custom Code.

This is only one common example, but there are many others.

This same principle holds for rule order as well.

Rule X and Rule Y both trigger off the Library Loaded event. When the library loads, Launch calls Rule X first because it has a lower rule order. But Launch does not wait for Rule X's actions to complete before it calls Rule Y's actions. We can get race conditions in this scenario as well. You now know that we should expect Rule X's actions to finish first sometimes, but not always.

There are many scenarios where this doesn't matter because the rule actions are not dependent on one another. But, there are other scenarios where it matters a lot. This is hard to solve without the new behavior that we've enabled.

2) New behavior

The new behavior is that actions will only run when the previous action finishes.

So Launch will call Action A and wait for it to complete, then call Action B and wait for it to complete, then call Action C.

Effectively, Launch is adding conditions and actions to a processing queue, then processing the queue on a FIFO basis. Conditions and actions for a rule enter the queue when the rule event occurs. Each condition or action executes when it reaches the top position in the queue.

For technical readers, this is accomplished by converting conditions and actions into promises and executing them in an asynchronous promise chain.

When multiple rules share a single event, Launch adds the conditions and actions from each rule to the queue based on the rule_order attribute on the rule event.

Each condition and action has a timeout. There is no rule-level timeout.

2.1) Property Settings

Each property has a new attribute. In the UI, you'll see a checkbox labeled Run rule components in sequence visible under the Advanced Settings section of the property edit screen. You can toggle this setting on and off.

If you are looking at the API, you'll see that the property now has an attribute called rule_component_sequencing. This attribute will have a value of true or false.

When sequencing is on, all rules and rule components within the property will adopt this new behavior. When the sequencing is off, then there are no changes to the existing runtime behavior.

In Phase 1 (more on this later) the default for new and existing properties is off. If you want to use this new behavior, you'll need to go to the property edit screen, enable the new setting, and publish a new library (it can be an empty library).  This setting is editable at any time.

When you turn sequencing on for an existing property, we recommend extensive testing for the same reason that we are unwilling to make this change across the board for everyone. The Launch team is unable to test whether this has no unexpected consequences across the world, so we rely on each user to test their setup when they enable the new behavior.

If you are creating a new property, we recommend turning it on because this is the future direction of Launch development and because there is no risk of breaking an existing implementation.

2.2) Condition Settings

As mentioned above, timeout is now an essential setting for conditions and actions so that a single one can't bring your processing queue to a stop. When you enable sequencing at the property level, then you'll see a new attribute on your conditions in the left-rail under Advanced Settings.

All existing conditions will adopt a timeout of 2000ms. Newly created conditions will also have a timeout of 2000ms unless you change it when creating or editing the condition.

Once an asynchronous process has begun, we cannot cancel it, so it will continue and may eventually finish in the future. But when a condition reaches its timeout, it will fail regardless of whether it ultimately finishes or not. When a condition fails, the rule's subsequent conditions and actions will be removed from the processing queue.

2.3) Action Settings

Actions have two additional settings that you'll find under the Advanced Settings:

• Checkbox for Wait to run next action
• Text field for Timeout in milliseconds

All existing and new actions will have Wait to run next action set to true. When true, this action will obey all the rules of the processing queue, as previously described. If you turn it off, at runtime, the processing queue moves on to the next action immediately.  If the completion of this action does not affect subsequent actions (from the current rule or later rules), you can turn this option off and explicitly tell the processing queue that it doesn't need to wait for this action to finish before moving on.

Timeout on actions has all the same defaults and behaviors as a timeout on conditions, with one notable callout. When an action fails to execute within its timeout, Launch will consider that action as failed and will remove all subsequent rule actions from the processing queue.

As we talked to customers, consultants, and partners who were trialing this behavior, we heard pretty unanimously, that this was the desired effect.  

Consider the following rule actions:

Our users told us that sending no data is better than sending partial/incorrect data.

2.4) Rule View

You can get a quick view of the Wait to run next action setting from the Rule view. If sequencing is on at the property level, then you'll see an additional "Wait" note after an action with the wait setting enabled.

2.5) Custom Code

Custom Code in Launch requires some further explanation. If you're using Custom Code for regular old JavaScript, then there is nothing special you need to do to take advantage of sequencing. It will be taken care of for you.

However, we can now support scenarios where you want your Custom Code to do something asynchronous, and you want subsequent actions to wait until your asynchronous process has finished.

JavaScript

If your Custom Code is JavaScript, then you can now return a Promise in conditions or actions. The next condition or action will wait for the promise to resolve before executing.  

If the promise does not resolve within the timeout period, then Launch will consider that component as failed. As described above, when a condition or an action fails, Launch will remove subsequent actions from that rule from the processing queue.

Promises in JavaScript custom code only have this behavior if you do not execute it globally. It's much harder to make it work when you execute globally, and nowadays, you shouldn't be doing that anyway.

HTML

If your Custom Code is HTML, then you can put a <script> tag in there and use JavaScript to do something asynchronous there. In this scenario, there are two new functions available to you.

• onCustomCodeSuccess() - call this function to indicate that your custom code has completed and Launch can move on to subsequent actions
• onCustomCodeFailure() - call this function to indicate that your custom code has failed in some way and that Launch should remove subsequent actions from the processing queue

If Launch does not find any call to onCustomCodeSuccess() or onCustomCodeFailure() within the custom code, it will assume the action is completed immediately and move on to the next item in the queue.

3) App Implications

This feature provides Launch users more power and flexibility when it comes to rule execution. We also expect this to solve many problems that our users have reported over the last few years of running Launch. But there are also a few things that you should watch out for.

This section will cover the implications for using Launch within the UI. The next section will cover the runtime implications as Launch executes in the browser.

3.1) Sequencing is not revisable

Sequencing behavior is stored at the property level so that it can be easily turned on or off without you having to edit dozens to thousands of individual rule components. 

However, properties are not revisable, and individual resources (conditions, actions, data elements, etc.) do not store that property attribute. When you look back at a build or an old library, you won't be able to tell whether rule component sequencing was on or off at the time.

You should treat this as a one-time change, enable it, test it, move forward, and don't go backward. But there are individual circumstances where you're going to want to roll it back, so we're not enforcing anything yet. Be aware of the implications and account for it in your process of moving over.

Property changes will still generate audit events so there is a historical record of the change, but you won't be adding this change to a library.  It will just be included automatically.

Recommendation: Make this a one-time change; do not turn it on and off.

3.2) Approved-but-not-really Libraries

Launch includes this setting in a build so that it knows how to behave at runtime. When you flip the switch, this setting will be in the next build.

But because properties are not revisable (see 3.1 above), this setting is not part of the regular dependency checking. Your next build will have different behavior than the last one.  You don't want this when your next build is for an Approved library to Publish to Production and Launch is not able to prevent you from doing that.

Recommendation: Do not turn this on when you have Submitted or Approved libraries. Clear your publishing queue before you enable this for an existing property.

3.3) Restoring Old Revisions

Because Launch doesn't store the sequencing attribute on individual resources, it is also not stored on historical revisions of those resources.

When you view old rule revisions (by selecting them in an old library or by comparing them to other revisions), the "Wait" notices on individual rule actions will reflect the current property settings rather than the property settings at the time Launch cut the revision.

Furthermore, if you restore that revision (copy its settings to latest), it will use the current property setting for sequencing, rather than whatever the property setting was when Launch cut the revision.

4) Runtime Implications

This section covers the runtime implications of this change as Launch executes within the browser. Of necessity, this section is much more technical than the previous one.

4.1) All Conditions and Actions are Asynchronous

To accomplish this strict sequencing, Launch is now running all conditions and actions asynchronously.

If you are loading Launch synchronously and particularly if you have rules with Library Loaded or Page Bottom events, then you'll want to pay very close attention to timing.

Let's take the following scenario where:

• I am loading Launch synchronously
• I have a rule with a Library Load event and a Custom Code action that creates a global variable
• I am loading Launch on my page with the following code (note: please don't do this, this is just simple example for illustration)

<script src="//assets.adobedtm.com/staging/launch-ENab18ee4295ad48988ddb241e2794e906-development.min.js"></script>
<script>
    console.log(window.globalVariableCreatedFromMyCustomCodeInLaunch.foo); 
</script>

With sequencing turned off, your Custom Code action will execute synchronously when the Library is loaded. The global variable you created will exist before the browser continues to call console.log().

With queuing turned on, your Custom Code action will execute asynchronously. It will still get done very soon -- with the next JavaScript event loop -- but there's a race condition, so there is no guarantee that it will exist before the browser calls console.log().

4.2) Conditions and Actions are Queued When Called

Conditions and actions are added to the processing queue when the rule is triggered.

Let's take the following scenario:

• Rule N and Rule O that both use the Library Loaded event
• Rule N has an order of 1, and Rule O has an order of 2
• Rule N has Action 2 that that invokes Direct Call Rule P

When the Library Loaded event occurs, the actual order of operations is:

1. Launch queues Rule N's conditions and actions
2. Launch queues Rule O's conditions and actions
3. Launch runs Rule N conditions and they pass
4. Launch runs Rule N - Action 1
5. Launch runs Rule N - Action 2
6. Launch queue's Direct Call Rule P's conditions and actions
7. Launch runs Rule O's conditions and actions - because they entered the queue before Direct Call Rule P's
8. Launch runs Direct Call Rule P's conditions and actions

If you have rule actions that trigger other rules, you may end up in scenarios where your queue order is different than expected.

4.3) External Actions Can Influence Outcomes

Launch runs in an environment where many other things are going on that are outside its control. With sequencing turned on, there is an increased possibility for external actors (other code on the page) to interact with the data layer between the time a rule is triggered and its conditions and actions execute.

As a result, if your conditions and actions reference data on the data layer, they may execute using newer/different data than what was on the data layer at the time the rule was triggered.

5) Rollout plans

This change represents the future direction for the Launch browser runtime, but we recognize change takes time.  We want to roll out this change in a responsible way so that our users have time to make any necessary adjustments.

5.1) Phase 1: Sequencing Optional

Phase 1 is currently scheduled to be released after most business hours are over on July 7th, 2020.

In this phase, the use of sequencing is optional. For newly created properties, the default will be off.

If you want to enable this behavior for your property (new or existing), you need to go to the property edit page to turn it on. You will then need to create a new build on your property to see the new behavior.

Once you enable sequencing at the property level, the Timeout and Wait to run next action settings will be visible on Conditions and Actions as described above. All conditions (existing and new) will get a default Timeout of 2000ms. All actions (existing and new) will get a default Timeout of 2000ms and will have Wait to run next action set to true. 

5.2) Phase 2: Sequencing Defaults to On for New Properties

After we've got more feedback from more customers who are using this setting, we plan to enable this by default for newly created properties. We anticipate that change will happen sometime in summer of 2020.

In this phase, you can still turn sequencing off at property creation or anytime after.

5.3) Phase 3: New Properties are Just On

The third phase would remove the on/off toggle for newly created properties. New properties will just use the new sequencing behavior always.

For existing properties, making this change would be a one-way street. Once you've enabled it, there would be no going back. To test the behavior (strongly recommended), you would want to make a copy of your property and test on the clone before you did it on the original.

We have no specific timeline for this phase.

5.4) Phase 4: There Is No More Off

At some far off future date, there will likely come a time where the sequencing behavior will be the only behavior. We do not have any specific plans for when this may happen, but if sequencing works out the way that we hope, we will not maintain the non-sequenced behavior perpetually.  So do not panic, but definitely start to make plans to make the move.

6) Note for Extension Developers

Before we wrap up, there is one final note for extension developers.  

If you do not have extension modules that need to do something asynchronous, then no action is required of you.

If you do have extension conditions or actions that need to do something asynchronous, you should ask yourself whether it makes sense to consider the action complete only after the asynchronous task is complete. If so, then your condition or action should return a Promise that resolves when the asynchronous task is complete.

All Adobe extensions are compatible with this new behavior.

7) Wrapping Up

If you've made it this far, congratulations!! This change fixes several issues that we know some of you are having. It also introduces Promises in Custom Code, which many of our more advanced users have been requesting. And finally, it is a pre-requisite for some other changes that we need to make to continue to evolve Launch to deal with the challenges of modern web development.

Ask your questions below or hit us up on our Slack Channels.

Happy tagging 

[#2]

@thebenrobb,

This is super exciting stuff!  Thanks for sharing!

[#3]

I foresee one feature request as a result of this change: Branching Actions.

This would solve the problem where you don’t want the Action sequence to terminate just because one Action screwed up.

E.g. there could be:

• one branch (B1) that sets Custom Code, then fires the AA beacon
• another branch (B2) that sets Custom Code, then branches out into:
  • a sub-branch (B2.1) that fires a Facebook beacon
  • another sub-branch (B2.2) that fires a GA beacon
• etc

[#4]

@analytics_union, this makes me smile because we knew that this would come up (meaning we were fairly confident).  We discussed it as part of the original scope, but it starts to get complicated.  As is our nature, we defined the scope to solve the problem we knew had and decided to wait on the one we thought that we might have.  This lets us collect the specific use cases to make sure we solve the real problem instead of guessing at how people want it to work.

For now (meaning later this month after we release), I think you could accomplish your scenario by creating two rules that use the same event.  The rule you setup to handle Branch B would need to have a higher rule order number on the event so it runs 2nd.  Then your Facebook beacon (B2.1) could uncheck the Wait to run next action box, so that the GA Beacon (B2.2) doesn't have to wait for Facebook to finish.

[#5]

@thebenrobb , I certainly wasn't expecting my idea to be worked on right now. But I think it will be something that is requested for down the road. Your suggestion for using different rules would work, of course, but it's not so neat.

[#6]

Hello @thebenrobb,

I am not sure to interpret correctly the part with custom code. 
Do you mean that we can return a promise from any of our custom code section (Codition, Event, Actions) so the code in an action could look like: 

var promiseObj = new Promise(function(resolve,reject){
myAPIresult = fetch('myAPIstuff'); //some async function
s.evarXX = myLocation;
if(myLocation != ""){
   resolve('Nice');
}else{
   reject('good try');
}
})
return promiseObj 

I am wondering if you are looking for a specific object to be returned in the resolve and reject. 
Following your explanation on how failure are handled within launch, I can imagine that we can use the reject to abort a rule execution if a dependency is not loading, that directly in the action (not in condition). 

[#7]

@analytics_union I believe you're right on both counts.  I do expect people will request it.  And the different rules should work, but is not so neat.

[#8]

@Julien_Piccini1, Promises specifically are supported in JavaScript conditions and actions when you're not running in the global scope.  We have an update to the Core extension docs with some code samples, but in the meantime, here's a copy/paste job.  I think you've got the general idea, to my eye, your example looks a lot like this action from the upcoming docs:

return new Promise(function(resolve, reject) {
  setTimeout(function() {
    if (new Date().getDay() === 5) {
      resolve();
    } else {
      reject();
    }
  }, 1000);
});

[#9]

@thebenrobb Is there any way to force the rule order to also have the "wait then" behaviour?

We often encounter race condition issues between rules that are hard to spot because they are intermittent (and are also hard to resolve without being forced to cram everything into a single rule!)

Any help would be appreciated!