Highlighted

A/B Testing

dikshagoel

27-03-2018

In A/B testing , if Delivery A has open rate 0.38 and open rate of delivery B is 0.37, the winning delivery is A, although the difference is just 0.01 .

Can we further set a threshold of the score difference before the final delivery is sent out in javascript? For example, like a difference of 0.1% at least (i.e. like open rate for delivery A is 0.5% and open rate of delivery B is 0.4%).

Replies

Highlighted

kapilsharma0101

29-03-2018

Hi Diksha,

For the A/B Testing, we use the below JS (Not Complete) :

// query the database to find the winner (best open rate)

   var winner = xtk.queryDef.create(

     <queryDef schema="nms:delivery" operation="get">

       <select>

         <node expr="@id"/>

         <node expr="@label"/>

         <node expr="[@operation-id]"/>

         <node expr="[@workflow-id]"/>

       </select>

       <where>

         <condition expr={"@FCP=0 and [@workflow-id]= " + instance.id}/>

       </where>

       <orderBy>

         <node expr="[indicators/@estimatedRecipientOpenRatio]" sortDesc="true"/>

       </orderBy>

     </queryDef>).ExecuteQuery()

So, to set the threshold you can change the WHERE condition in this QUERY.

Apart from the current where condition you can put this below condition as well :

<condition expr={"@FCP=0 and [indicators/@estimatedRecipientOpenRatio] > 0.1 and [@workflow-id]= " + instance.id}/>

I am not sure, but I think this should work.

Regards,
Kapil Sharma