Bringing Linked Data into a Delivery at scale

[davidh2892249]

Hi All,

I have a requirement where I need to bring in data linked to recipients at a large scale (e.g. sending to recipient volume > 2million, the linked table schema will have even more data than this).

I have created a custom schema and created a link between recipient and the custom schema.

In the delivery I have:

* Set the "edit the recipient profile loading query..." to retrieve the linked records by editing the expression using some simple filters (including a date expression. field <= GetDate(). Most the time we'll set the limit to 1 / fetch single record, but there may be the need to retrieve 2 or 3 records per recipient.

* Created a JS condition + loop that shows the data if the recipient has atleast one linked record, and iterates over it if multiple records.

<% if ( recipient.customSchemaLinkName.length > 0 ) { %>
<% for (var i=0; i < recipient.customSchemaLinkName.length; i++) { %>

Show linked record field 1 here: <%= recipient.customSchemaLinkName[i].field1%>

Show linked record field 2 here: <%= recipient.customSchemaLinkName[i].field2 %>
<% } %>
<% } %>

It's working perfectly when previewing, but when testing preparing a delivery with 1m+ recipients, the message preparation phase is taking an awful long time.

Any recommendations how to optimise?

Thanks in advance

David

[isahore · Accepted solution]

Hi @davidh2892249,

This script will indeed take a lot of time when you are running the delivery for a large number of recipients.

May be you could do the enrichment part (recipient profile loading) in a workflow activity, and then split the target population based on the number of linked records: 0, 1, 2, 3, etc., provided you have a known max value. For each of the split transitions you can have different deliveries with code like:

targetData.linkedData[0].value, targetData.linkedData[1].value, targetData.linkedData[2].value, etc.

Thanks,

Ishan

[Sukrity_Wadhwa]

Hi @davidh2892249,

Was the given solution helpful to resolve your query or do you still need more help here? Do let us know.

Thanks!