Expand my Community achievements bar.

SOLVED

AB test use case script does not assume the same percentage of cases.

Avatar

Level 2

Hello. I'm beginner.

The following AB test use case script does not assume the same percentage of cases.

 

https://experienceleague.adobe.com/docs/campaign-classic/using/sending-messages/a-b-testing/use-case...

 

Is there a way to control it in the the same percentage of cases?

For example, if the same percentage of cases, deliver A as the winner.

Thanks.

1 Accepted Solution

Avatar

Correct answer by
Community Advisor

Hello @tokuchan 

 

To add this scenario you can have multiple query def statements for the specific delivery.

 

For this you will have to add @id='YOUR_DELIVERY ID' in the where condition.

Once you have the queries for two deliveries you can compare the result if the result if same then you can select Delivery A else let the winner delivery be the final delivery

 

Here is an example for the first delivery.

var deliveryA = xtk.queryDef.create(
     <queryDef schema="nms:delivery" operation="get">
       <select>
         <node expr="@id"/>
         <node expr="@label"/>
         <node expr="[@operation-id]"/>
         <node expr="[@workflow-id]"/>
         <node expr="[indicators/@estimatedRecipientOpenRatio]" alias="openRatio/>
       </select>
       <where>
         <condition expr={"@FCP=0 and [@workflow-id]= " + instance.id +" and @id="+DELIVERY_PRIMARY_KEY_HERE}/>
       </where>
        
     </queryDef>).ExecuteQuery()
   

Similarly you can create the query for delivery B and then compare the openRatio to decide your final delivery.

 

Thanks,

Manoj


     Manoj
     Find me on LinkedIn

View solution in original post

11 Replies

Avatar

Correct answer by
Community Advisor

Hello @tokuchan 

 

To add this scenario you can have multiple query def statements for the specific delivery.

 

For this you will have to add @id='YOUR_DELIVERY ID' in the where condition.

Once you have the queries for two deliveries you can compare the result if the result if same then you can select Delivery A else let the winner delivery be the final delivery

 

Here is an example for the first delivery.

var deliveryA = xtk.queryDef.create(
     <queryDef schema="nms:delivery" operation="get">
       <select>
         <node expr="@id"/>
         <node expr="@label"/>
         <node expr="[@operation-id]"/>
         <node expr="[@workflow-id]"/>
         <node expr="[indicators/@estimatedRecipientOpenRatio]" alias="openRatio/>
       </select>
       <where>
         <condition expr={"@FCP=0 and [@workflow-id]= " + instance.id +" and @id="+DELIVERY_PRIMARY_KEY_HERE}/>
       </where>
        
     </queryDef>).ExecuteQuery()
   

Similarly you can create the query for delivery B and then compare the openRatio to decide your final delivery.

 

Thanks,

Manoj


     Manoj
     Find me on LinkedIn

Avatar

Level 2
Hello @_Manoj_Kumar_, Thank you for your reply. How can I find out "DELIVERY_PRIMARY_KEY"? Also, I use "Recurring-delivery" instead of "Email" to get the winner. Is there anything different? Thank you.

Avatar

Community Advisor
No it should be an issue. go to your delivery templates and there you will find the delivery id

     Manoj
     Find me on LinkedIn

Avatar

Level 2
Hello @_Manoj_Kumar_, Thank you for your reply. It compare "Distribution A. @ Open Rate" with "Distribution B. @ Open Rate"? I look for "Dlivery ID" and try compare. I need to delivery repeatedly, not once... Thank you.

Avatar

Level 2
Hello @_Manoj_Kumar_, I Could not find DELIVERY_PRIMARY_KEY in the delivery template. Does DELIVERY_PRIMARY_KEY mean deliveryId? I was able to get the deliveryId in JS after delivery. I was able to create another Workflow, get the open rates from the deliveryId and compare them. Is my perception correct? Thank you.

Avatar

Community Advisor
Yes, You are correct. DELIVERY_PRIMARY_KEY was a placeholder for your reference on what to put there.

     Manoj
     Find me on LinkedIn

Avatar

Level 2

Hello @_Manoj_Kumar,  Thank you for reply.

I tried the following script in another workflow.

 

 

 

 

 

 

//deliveryA
var deliveryA = xtk.queryDef.create (
	<queryDef schema="nms:delivery" operation="select">
		<select>
			<node expr="@id"/>
			<node expr="@label"/>
			<node expr="[@operation-id] "/>
			<node expr=" (@workflow-id]"/>
			<node expr="[indicators/@reactivity]" alias="openRatio"/>
		</select>
		<where>
			<condition expr={"@id=" + DELIVERY_ID }/>
		</where>
	</queryDef>)

var responceA = deliveryA.ExecuteQuery ();

for each (var variable in responceA) {
	vars.winidA = variable.@id
	vars.winlabelA = variable.@label
	vars.openRatioA = variable.openRatio

	logInfo ("vars.winidA = [" + vars.winidA + "]");
	logInfo ("vars.winlabelA = [" + vars.winlabelA + "1");
	logInfo ("vars.openRatioA = [" + vars.openRatioA + "]");
}

//deliveryB
var deliveryB = xtk. queryDef.create (
	<queryDef schema="nms:delivery" operation="select">
		<select>
			<node expr="@id"/>
			<node expr="@label"/>
			<node expr="[@operation-id]"/>
			<node expr=" [@workflow-id]"/>
			<node expr=" [indicators/@reactivity]" alias="openRatio"/>
		</select>
		<where>
			<condition expr={"@id=" + DELIVERY_ID }/>
		</where>
	</queryDef>)

var responceB = deliveryB.ExecuteQuery ();

for each (var variable in responceB) {
	vars.winidB = variable.@id
	vars.winlabelB = variable.@label
	vars.openRatioB = variable.openRatio
	
	logInfo ("vars.winidB = [" + vars.winidB + "]");
	logInfo ("vars.winlabelB = [" + vars.winlabelB + "]");
	logInfo ("vars.openRatioB = [" + vars.openRatioB + "]");
}

//Compare open rates
if (vars. openRatioA == vars.openRatioB) {
	var winner = deliveryA;
	vars.winner = "deliveryA";
} else if (vars.openRatioA > vars.openRatioB) {
	var winner = deliveryA;
	vars.winner = "deliveryA";
} else if (vars.openRatioA < vars.openRatioB) {
	var winner = deliveryB;
	vars.winner = "deliveryB";

//Confirm winner
logInfo ("var winner = [" + winner + "]");
logInfo ("vars.winner = (" + vars.winner + "]");

 

 

 

I was able to get the open rate and compare it.

The winner was 1 and the loser was 0.

 

But, when I incorporated it into the Mail delivery workflow and added logic to duplicate the winner, I got an "Invalid XML Name" error.

 

 

 

//Confirmation of acquisition of deliveryId
logInfo ("instance.vars.deliveryAid - [" + instance.vars.deliveryAid + "]");
logInfo ("instance.vars.deliveryBid = [" + instance.vars.deliveryBid + "]");

//deliveryA
var deliveryA = xtk.queryDef.create (
	<queryDef schema="nms:delivery" operation="select">
		<select>
			<node expr="@id"/>
			<node expr="@label"/>
			<node expr="[@operation-id]"/>
			<node expr="(@workflow-id] "/>
			<node expr="[indicators/@reactivity]" alias="openRatio"/>
		</select>
		<where>
			<condition expr={"@FCP=0 and [@workflow-id]=" + instance.id +"and @ID=" + instance.vars.deliveryAid }/>
		</where>
	</queryDef>)

var responceA = deliveryA.ExecuteQuery()

for each (var variable in responceA) {
	vars.winidA = variable.@id
	vars.winlabelA = variable.elabel
	vars.openRatioA = variable.openRatio

	logInfo ("vars.winidA = [" + vars.winidA + "]");
	logInfo ("vars.winlabelA = [" + vars.winlabelA + "]");
	logInfo ("vars.openRaticA = [" + vars.openRatioA + "]");
}


//deliveryB
var deliveryB xtk.queryDef.create (
	<queryDef schema="nms:delivery" operation="select">
		<select>
			<node expr="@id"/>
			<node expr="@label"/>
			<node expr="[@operation-id]"/>
			<node expr="[@workflow-id]"/>
			<node expr="[indicators/@reactivity]" alias="openRatio"/>
		</select>
	<where>
		<condition expr={"@FCF=0 and [@workflow-id]= " + instance.id +"and @ID= + instance.vars.deliveryBid }/>
	</where>
	</queryDef>)

var responceB = deliveryB.ExecuteQuery()

for each (var variable in responceB) {
	vars.winidB = variable.@id
	vars.winlabelB = variable.@label
	vars.openRatioB = variable.openRatio

	logInfo ("vars.winidB = [" + vars.winidB + "]");
	logInfo ("vars.winlabelB = [" + vars.winlabelB + "]");
	logInfo ("vars.openRatioB = [" + vars.openRatioB + "]");

//Compare open rates
if (vars.openRatioA = vars.openRatioB) {
	var winner = deliveryA;
	vars.winner = "deliveryA";
} else if (vars.openRatioA > vars.openRatioB) {
	var winner = deliveryA;
	vars.winner = "deliveryA";
} else if (vars.openRatioA < vars.openRatiOB) {
	var winner = deliveryB;
	vars.winner = "deliveryB";
}

//Confirm winner
logInfo ("var winner = [" + winner + "]");
logInfo ("vazs.winner = [" + vars.winner + "]");

//Below, the sample script
var delivery = nms.delivery.create ()
delivery.Duplicate ("nms delivery|" + winner.@id)

delivery.label = winner.@label + " final"

delivery.operation_id = winner.@["operation-id"]
delivery.wrorkflow_id = winner.@ ["workflow-id"]
delivery.scheduling.validationMode = "manual"
delivery.scheduling.delayed = 0

delivery.save ()

vars.deliveryId = delivery.id

//Output new delivery ID
logInfo ("vars.deliveryId = [" + vars.deliveryId + "]");

 

 

 

I'm not engineer...

I'm not familiar with javascript.

I don't know the cause.

Where is the mistake?

I'd really appreciate it if you could give me your advice.

Thank you.

Avatar

Level 2

Hello @_Manoj_Kumar_,  I made some corrections.
But, I get an "Invalid XML Name" error.

 

 

//Confirmation of acquisition of deliveryId
logInfo ("instance.vars.deliveryAid = [" + instance.vars.deliveryAid + "]");
logInfo ("instance.vars.deliveryBid = [" + instance.vars.deliveryBid + "]");
var deliveryAid = instance.vars.deliveryAid;
var deliveryBid = instance.vars.deliveryBid;


//deliveryA
var deliveryA = xtk. queryDef.create (
	<queryDef schema="nms :delivery" operation="get">
		<select>
			<node expr="@id"/>
			<node expr="@label"/>
			<node expr="[@operation-id] "/>
			<node expr="[@workflow-id]"/>
			<node expr="[indicators/@reactivity]" alias="openRatio"/>
		</select>
	<where>
		<condition expr-{"@id=" + deliveryAid }/>
	</where>
	</queryDef>).ExecuteQuery ()

for each (var variable in deliveryA) {
	var winidA = variable.@id
	var winlabelA = variable.@label
	var openRatioA = variable.openRatio
	logInfo ("var winidA = [" + winidA + "]");
	logInfo ("var winlabelA = [" + winlabelA + "]");
	logInfo ("var openRatioA = [" + openRatioA + "]");
}

//deliveryB
var deliveryB = xtk.queryDef.create (
	<queryDef schema="nms:delivery" operation="get">
		<select>
			<node expr="@id"/>
			<node expr="@label"/>
			<node expr="[@operation-id] "/>
			<node expr="[@workflow-id]"/>
			<node expr="[indicators/@reactivity]" alias="openRatio"/>
		</select>
	<where>
		<condition expr={"@id=" + deliveryBid }/>
	</where>
	</queryDef>).ExecuteQuery ()

for each (var variable in deliveryB) {
	var winidB = variable.@id
	var winlabelB = variable.@label
	var openRatioB variable.openRatio
	logInfo ("var winidB = [" + winidB + "]");
	logInfo ("var winlabelB = [" + winlabelB + "]");
	logInfo ("var openRatioB = [" + openRatioB + "]");
}

//Compare open rates
if (openRatioA == openRatioB) {
	var winner = "deliveryA";
} else if (openRatioA > openRatioB) {
	var winner = "deliveryA";
} else if (openRatioA < openRatioB) {
	var winner = "deliveryB";
}

//Confirm winner
logInfo ("var winner = [" + winner + "]");

//Below, the sample script
var delivery = nms.delivery.create ()
delivery.Duplicate ("nms:delivery " + winner.@id)

delivery.label = winner.@label + " final"

delivery.operation_id = winner.@["operation-id"]
delivery.wrorkflow_id = winner.@ ["workflow-id"]

delivery.scheduling.validationMode = "manual"
delivery.scheduling.delayed = 0

delivery.save ()

vars.deliveryId = delivery.id

//Output new delivery ID
logInfo ("vars.deliveryId = [" + vars.deliveryId + "]");

 


It seems that the delivery object cannot be duplicated.
cannot put @ FCP = 0, [@ workflow-id], [operation-id] in <where> </ where>.

I'd really appreciate it if you could give me your advice.
Thank you.

Avatar

Community Advisor
Your If else code should be if (openRatioA == openRatioB) { var winner = deliveryA; } else if (openRatioA > openRatioB) { var winner = deliveryA; } else if (openRatioA < openRatioB) { var winner = deliveryB; }

     Manoj
     Find me on LinkedIn

Avatar

Level 2
Hello @_Manoj_Kumar_, Thank you for your reply. I fixed ifelse, but I couldn't get the value. When I changed operation = select, I got the value, but I can't duplicate it. I get an error in the delivery activity. error : "The schema for recipient specified in the delivery transition (oma: RecipientContactHistory) is not compatible with the schema defined in the delivery template ('nms:recipient'). They should be identical." It seems that operation-id and workflow-id cannot be get. Also, with operation = select, an error will occur even if the where clause is as follows. <condition expr = {"@ FCP = 0 and [@ workflow-id] =" + instance.id + "and @ID =" + DELIVERY_ID} />. I'd really appreciate it if you could give me your advice. Thank you.

Avatar

Level 2
Hello @_Manoj_Kumar_, I solved it by getting the open rate of AB from the delivery ID with operation = select, then searching for the winner object by the winner's delivery ID and duplicating it. Thank you very much!