AB test use case script does not assume the same perc...

[tokuchan]

Hello. I'm beginner.

The following AB test use case script does not assume the same percentage of cases.

https://experienceleague.adobe.com/docs/campaign-classic/using/sending-messages/a-b-testing/use-case...

Is there a way to control it in the the same percentage of cases?

For example, if the same percentage of cases, deliver A as the winner.

Thanks.

[_Manoj_Kumar_ · Accepted solution]

Hello @tokuchan 

To add this scenario you can have multiple query def statements for the specific delivery.

For this you will have to add @id='YOUR_DELIVERY ID' in the where condition.

Once you have the queries for two deliveries you can compare the result if the result if same then you can select Delivery A else let the winner delivery be the final delivery

Here is an example for the first delivery.

var deliveryA = xtk.queryDef.create(
     <queryDef schema="nms:delivery" operation="get">
       <select>
         <node expr="@id"/>
         <node expr="@label"/>
         <node expr="[@operation-id]"/>
         <node expr="[@workflow-id]"/>
         <node expr="[indicators/@estimatedRecipientOpenRatio]" alias="openRatio/>
       </select>
       <where>
         <condition expr={"@FCP=0 and [@workflow-id]= " + instance.id +" and @id="+DELIVERY_PRIMARY_KEY_HERE}/>
       </where>
        
     </queryDef>).ExecuteQuery()
   

Similarly you can create the query for delivery B and then compare the openRatio to decide your final delivery.

Thanks,

Manoj

[tokuchan]

Hello @_Manoj_Kumar_, Thank you for your reply. How can I find out "DELIVERY_PRIMARY_KEY"? Also, I use "Recurring-delivery" instead of "Email" to get the winner. Is there anything different? Thank you.

[_Manoj_Kumar_]

No it should be an issue. go to your delivery templates and there you will find the delivery id

[tokuchan]

Hello @_Manoj_Kumar_, Thank you for your reply. It compare "Distribution A. @ Open Rate" with "Distribution B. @ Open Rate"? I look for "Dlivery ID" and try compare. I need to delivery repeatedly, not once... Thank you.

[tokuchan]

Hello @_Manoj_Kumar_, I Could not find DELIVERY_PRIMARY_KEY in the delivery template. Does DELIVERY_PRIMARY_KEY mean deliveryId? I was able to get the deliveryId in JS after delivery. I was able to create another Workflow, get the open rates from the deliveryId and compare them. Is my perception correct? Thank you.

[_Manoj_Kumar_]

Yes, You are correct. DELIVERY_PRIMARY_KEY was a placeholder for your reference on what to put there.

[tokuchan]

Hello @_Manoj_Kumar,  Thank you for reply.

I tried the following script in another workflow.

//deliveryA
var deliveryA = xtk.queryDef.create (
	<queryDef schema="nms:delivery" operation="select">
		<select>
			<node expr="@id"/>
			<node expr="@label"/>
			<node expr="[@operation-id] "/>
			<node expr=" (@workflow-id]"/>
			<node expr="[indicators/@reactivity]" alias="openRatio"/>
		</select>
		<where>
			<condition expr={"@id=" + DELIVERY_ID }/>
		</where>
	</queryDef>)

var responceA = deliveryA.ExecuteQuery ();

for each (var variable in responceA) {
	vars.winidA = variable.@id
	vars.winlabelA = variable.@label
	vars.openRatioA = variable.openRatio

	logInfo ("vars.winidA = [" + vars.winidA + "]");
	logInfo ("vars.winlabelA = [" + vars.winlabelA + "1");
	logInfo ("vars.openRatioA = [" + vars.openRatioA + "]");
}

//deliveryB
var deliveryB = xtk. queryDef.create (
	<queryDef schema="nms:delivery" operation="select">
		<select>
			<node expr="@id"/>
			<node expr="@label"/>
			<node expr="[@operation-id]"/>
			<node expr=" [@workflow-id]"/>
			<node expr=" [indicators/@reactivity]" alias="openRatio"/>
		</select>
		<where>
			<condition expr={"@id=" + DELIVERY_ID }/>
		</where>
	</queryDef>)

var responceB = deliveryB.ExecuteQuery ();

for each (var variable in responceB) {
	vars.winidB = variable.@id
	vars.winlabelB = variable.@label
	vars.openRatioB = variable.openRatio
	
	logInfo ("vars.winidB = [" + vars.winidB + "]");
	logInfo ("vars.winlabelB = [" + vars.winlabelB + "]");
	logInfo ("vars.openRatioB = [" + vars.openRatioB + "]");
}

//Compare open rates
if (vars. openRatioA == vars.openRatioB) {
	var winner = deliveryA;
	vars.winner = "deliveryA";
} else if (vars.openRatioA > vars.openRatioB) {
	var winner = deliveryA;
	vars.winner = "deliveryA";
} else if (vars.openRatioA < vars.openRatioB) {
	var winner = deliveryB;
	vars.winner = "deliveryB";

//Confirm winner
logInfo ("var winner = [" + winner + "]");
logInfo ("vars.winner = (" + vars.winner + "]");

I was able to get the open rate and compare it.

The winner was 1 and the loser was 0.

But, when I incorporated it into the Mail delivery workflow and added logic to duplicate the winner, I got an "Invalid XML Name" error.

//Confirmation of acquisition of deliveryId
logInfo ("instance.vars.deliveryAid - [" + instance.vars.deliveryAid + "]");
logInfo ("instance.vars.deliveryBid = [" + instance.vars.deliveryBid + "]");

//deliveryA
var deliveryA = xtk.queryDef.create (
	<queryDef schema="nms:delivery" operation="select">
		<select>
			<node expr="@id"/>
			<node expr="@label"/>
			<node expr="[@operation-id]"/>
			<node expr="(@workflow-id] "/>
			<node expr="[indicators/@reactivity]" alias="openRatio"/>
		</select>
		<where>
			<condition expr={"@FCP=0 and [@workflow-id]=" + instance.id +"and @ID=" + instance.vars.deliveryAid }/>
		</where>
	</queryDef>)

var responceA = deliveryA.ExecuteQuery()

for each (var variable in responceA) {
	vars.winidA = variable.@id
	vars.winlabelA = variable.elabel
	vars.openRatioA = variable.openRatio

	logInfo ("vars.winidA = [" + vars.winidA + "]");
	logInfo ("vars.winlabelA = [" + vars.winlabelA + "]");
	logInfo ("vars.openRaticA = [" + vars.openRatioA + "]");
}


//deliveryB
var deliveryB xtk.queryDef.create (
	<queryDef schema="nms:delivery" operation="select">
		<select>
			<node expr="@id"/>
			<node expr="@label"/>
			<node expr="[@operation-id]"/>
			<node expr="[@workflow-id]"/>
			<node expr="[indicators/@reactivity]" alias="openRatio"/>
		</select>
	<where>
		<condition expr={"@FCF=0 and [@workflow-id]= " + instance.id +"and @ID= + instance.vars.deliveryBid }/>
	</where>
	</queryDef>)

var responceB = deliveryB.ExecuteQuery()

for each (var variable in responceB) {
	vars.winidB = variable.@id
	vars.winlabelB = variable.@label
	vars.openRatioB = variable.openRatio

	logInfo ("vars.winidB = [" + vars.winidB + "]");
	logInfo ("vars.winlabelB = [" + vars.winlabelB + "]");
	logInfo ("vars.openRatioB = [" + vars.openRatioB + "]");

//Compare open rates
if (vars.openRatioA = vars.openRatioB) {
	var winner = deliveryA;
	vars.winner = "deliveryA";
} else if (vars.openRatioA > vars.openRatioB) {
	var winner = deliveryA;
	vars.winner = "deliveryA";
} else if (vars.openRatioA < vars.openRatiOB) {
	var winner = deliveryB;
	vars.winner = "deliveryB";
}

//Confirm winner
logInfo ("var winner = [" + winner + "]");
logInfo ("vazs.winner = [" + vars.winner + "]");

//Below, the sample script
var delivery = nms.delivery.create ()
delivery.Duplicate ("nms delivery|" + winner.@id)

delivery.label = winner.@label + " final"

delivery.operation_id = winner.@["operation-id"]
delivery.wrorkflow_id = winner.@ ["workflow-id"]
delivery.scheduling.validationMode = "manual"
delivery.scheduling.delayed = 0

delivery.save ()

vars.deliveryId = delivery.id

//Output new delivery ID
logInfo ("vars.deliveryId = [" + vars.deliveryId + "]");

I'm not engineer...

I'm not familiar with javascript.

I don't know the cause.

Where is the mistake?

I'd really appreciate it if you could give me your advice.

Thank you.

[tokuchan]

Hello @_Manoj_Kumar_,  I made some corrections.
But, I get an "Invalid XML Name" error.

//Confirmation of acquisition of deliveryId
logInfo ("instance.vars.deliveryAid = [" + instance.vars.deliveryAid + "]");
logInfo ("instance.vars.deliveryBid = [" + instance.vars.deliveryBid + "]");
var deliveryAid = instance.vars.deliveryAid;
var deliveryBid = instance.vars.deliveryBid;


//deliveryA
var deliveryA = xtk. queryDef.create (
	<queryDef schema="nms :delivery" operation="get">
		<select>
			<node expr="@id"/>
			<node expr="@label"/>
			<node expr="[@operation-id] "/>
			<node expr="[@workflow-id]"/>
			<node expr="[indicators/@reactivity]" alias="openRatio"/>
		</select>
	<where>
		<condition expr-{"@id=" + deliveryAid }/>
	</where>
	</queryDef>).ExecuteQuery ()

for each (var variable in deliveryA) {
	var winidA = variable.@id
	var winlabelA = variable.@label
	var openRatioA = variable.openRatio
	logInfo ("var winidA = [" + winidA + "]");
	logInfo ("var winlabelA = [" + winlabelA + "]");
	logInfo ("var openRatioA = [" + openRatioA + "]");
}

//deliveryB
var deliveryB = xtk.queryDef.create (
	<queryDef schema="nms:delivery" operation="get">
		<select>
			<node expr="@id"/>
			<node expr="@label"/>
			<node expr="[@operation-id] "/>
			<node expr="[@workflow-id]"/>
			<node expr="[indicators/@reactivity]" alias="openRatio"/>
		</select>
	<where>
		<condition expr={"@id=" + deliveryBid }/>
	</where>
	</queryDef>).ExecuteQuery ()

for each (var variable in deliveryB) {
	var winidB = variable.@id
	var winlabelB = variable.@label
	var openRatioB variable.openRatio
	logInfo ("var winidB = [" + winidB + "]");
	logInfo ("var winlabelB = [" + winlabelB + "]");
	logInfo ("var openRatioB = [" + openRatioB + "]");
}

//Compare open rates
if (openRatioA == openRatioB) {
	var winner = "deliveryA";
} else if (openRatioA > openRatioB) {
	var winner = "deliveryA";
} else if (openRatioA < openRatioB) {
	var winner = "deliveryB";
}

//Confirm winner
logInfo ("var winner = [" + winner + "]");

//Below, the sample script
var delivery = nms.delivery.create ()
delivery.Duplicate ("nms:delivery " + winner.@id)

delivery.label = winner.@label + " final"

delivery.operation_id = winner.@["operation-id"]
delivery.wrorkflow_id = winner.@ ["workflow-id"]

delivery.scheduling.validationMode = "manual"
delivery.scheduling.delayed = 0

delivery.save ()

vars.deliveryId = delivery.id

//Output new delivery ID
logInfo ("vars.deliveryId = [" + vars.deliveryId + "]");

It seems that the delivery object cannot be duplicated.
cannot put @ FCP = 0, [@ workflow-id], [operation-id] in <where> </ where>.

I'd really appreciate it if you could give me your advice.
Thank you.

[_Manoj_Kumar_]

Your If else code should be if (openRatioA == openRatioB) { var winner = deliveryA; } else if (openRatioA > openRatioB) { var winner = deliveryA; } else if (openRatioA < openRatioB) { var winner = deliveryB; }

[tokuchan]

Hello @_Manoj_Kumar_, Thank you for your reply. I fixed ifelse, but I couldn't get the value. When I changed operation = select, I got the value, but I can't duplicate it. I get an error in the delivery activity. error : "The schema for recipient specified in the delivery transition (oma: RecipientContactHistory) is not compatible with the schema defined in the delivery template ('nms:recipient'). They should be identical." It seems that operation-id and workflow-id cannot be get. Also, with operation = select, an error will occur even if the where clause is as follows. <condition expr = {"@ FCP = 0 and [@ workflow-id] =" + instance.id + "and @ID =" + DELIVERY_ID} />. I'd really appreciate it if you could give me your advice. Thank you.

[tokuchan]

Hello @_Manoj_Kumar_, I solved it by getting the open rate of AB from the delivery ID with operation = select, then searching for the winner object by the winner's delivery ID and duplicating it. Thank you very much!