A/B Testing



In A/B testing , if Delivery A has open rate 0.38 and open rate of delivery B is 0.37, the winning delivery is A, although the difference is just 0.01 .

Can we further set a threshold of the score difference before the final delivery is sent out in javascript? For example, like a difference of 0.1% at least (i.e. like open rate for delivery A is 0.5% and open rate of delivery B is 0.4%).





Hi Diksha,

For the A/B Testing, we use the below JS (Not Complete) :

// query the database to find the winner (best open rate)

   var winner = xtk.queryDef.create(

     <queryDef schema="nms:delivery" operation="get">


         <node expr="@id"/>

         <node expr="@label"/>

         <node expr="[@operation-id]"/>

         <node expr="[@workflow-id]"/>



         <condition expr={"@FCP=0 and [@workflow-id]= " + instance.id}/>



         <node expr="[indicators/@estimatedRecipientOpenRatio]" sortDesc="true"/>



So, to set the threshold you can change the WHERE condition in this QUERY.

Apart from the current where condition you can put this below condition as well :

<condition expr={"@FCP=0 and [indicators/@estimatedRecipientOpenRatio] > 0.1 and [@workflow-id]= " + instance.id}/>

I am not sure, but I think this should work.

Kapil Sharma