Performance Issues in Recomputing logic

[rohanpat]

Hi,

For recomputing, we are using logic that works on entire nms:delivery schema. We came across following solution across the community but it doesn't seem to fix our problem either since the logic is similar and performing on entire nms:delivery schema,

<queryDef schema="nms:delivery" operation="select">

<select>

<node expr="@internalName"/>

<node expr="@id"/>

</select>

</queryDef> );

var res = query.ExecuteQuery();
var count = 0;
for each(var delivery in res.delivery){
var dId = delivery.@id;
//step 2: recompute statistics and tracking
var nmsDeliveryDirtyFlags_tracking = 1; // tracking logs
var nmsDeliveryDirtyFlags_messageCounters = 8; // broad logs
var flags = nmsDeliveryDirtyFlags_tracking | nmsDeliveryDirtyFlags_messageCounters;
NLWS.nmsDelivery.RecomputeStats(dId, flags);

This logic works, the only issue now is that the work-flow is pulling all deliveries from the Broadlog to check against and it’s causing performance issues given the amount of total deliveries. Can someone suggest what modifications could be done to this logic to only check for the delivery IDs that are imported and not all of them?

[rohanpat]

Anyone around?

[rohanpat]

Hi @rohanpat,

this query is realy "hard" and make recompute for each delivery ever sent. Not sure why you use this, as AC will do it  (not in realtime of course).
Tracking tables have link to the delivery table and this is a way to fetch impacted Delivery ID.
One way can be to use Incremental query to fetch all tracking logs since last execution of the workflow. Based on previously mentioned link, use distinct delivery IDs from result and pass it to your JS script (adapt script to read temp query table instead of nms:delivery)
Take care of temporary table behind Incremental query as can grow fast.

Regards,

Milan

[rohanpat]

Hi @Milan_Vucetic,

Actually we are not using Adobe MTAs for delivery. We have our custom MTAs, based on logs we filter out the delivery sends and with help of target flow, have the data inserted in resp tables. That's the reason we have use this route. Once entries are in place, we recompute the schema so that we have correct send count along with bounces.

So could you please help us with the recompute logic as to how we can simple play around only import delivery ids instead of entire stuff to help improve the performance...

[DavidKangni]

Hi Rohan,

This is what Milan's said by adapt your script to read temp query instead of nms delivery.

• If you have a initial query on deliveries schema that pulled the list of deliveries you need

<queryDef operation="select" schema={vars.targetSchema}>
<select>

<node expr="@label"/>
<node expr="@internalName"/>

<node expr="@id"/>

</select>
</queryDef>

• If you want to keep using nms:delivery add a where condition. below I'm selected deliveries created in August 2020

<queryDef schema="nms:delivery" operation="select">

<select>
<node expr="@label"/>
<node expr="@internalName"/>

<node expr="@id"/>

</select>

<where>

<condition boolOperator="AND" expr="@created &gt;= #2020-08-01 00:00:00.000Z#"/>

<condition boolOperator="AND" expr="@created &lt;= #2020-08-26 23:59:00.000Z#"/>

<condition boolOperator="AND" expr="@state = 95"/>

</where>

</queryDef> );

Thanks,

David

[rohanpat]

Thanks @Milan_Vucetic and @DavidKangni 

We are under the impression, we would need to do recompute on nms:delivery for everything to be in sync. Could you suggest what schema other than delivery could be a best practice? Like simply going with nms:broadlog or nms:broadlogmsg?

Also any way to update this loop : for each(var delivery in res.delivery) ? Instead of fetching all deliveries, could we just focus on ones imported from our files? We don't have any internal queries getting hit.

The filter date might work, but it seems a manual work, every-time the date needs to be changed. Any way to make it dynamic?

And what does this mean, like state = 95? : <condition boolOperator="AND" expr="@state = 95"/>

[DavidKangni]

Hi,

state = 95 is delivery status. for the loop query, search through the forum you should find some example. we cannot just build the query as is on the fly.

You need a process like mid sourcing and tracking process to update your logs.

Thanks,

David