How to get the details of the profiles that are getting excluded from Destination Export

[SubeAEP]

Hi Team,

We have a destination export dataflow on a segment that runs everyday which sends the incremental data to a SFTP location. We are using first name, last name, member number and email id in the mapping. Now everyday there are some profiles (around 10%) that are getting excluded from the export. We want to know the details of the profiles that are getting excluded in everyday's run as well as the reason behind it.

Is there any way to get the excluded profiles? Your inputs will be very helpful in debugging this issue.

Thanks

[_Manoj_Kumar_]

Hello @SubeAEP 

You can click on the Dataflow Run Start link and it will show you the reason of the exclusion in the identities excluded section.

[SubeAEP]

Hi @_Manoj_Kumar_ , yes but it always shows a generalized error 'Records didn't match attribute filters.'. I went to the dataflow details through Flowservice API but didnt see any record details or detailed error of the exclusion. Actually, we want the excluded record details so that we can trace them and try to retarget them. 

[DavidRoss91]

Hi @SubeAEP 

Have you tried to open a support request, to find out if backend support can pull a more detailed error? I checked this on my side and also only seeing 'Records didn't match attribute filters'.

[SubeAEP · Accepted solution]

Well, upon further investigation into the issue, we discovered that the dataset profile_export_for_destination_merge_policy_**** stores the incremental records for each day's run. We have found that for some runs, there are few records without a member number, hence they are excluded. Moreover, there are duplicate records in the dataset and that is also one of the reason for exclusion.

[Rite18]

@SubeAEP  did you consider below 

-  new profiles might be not qualified for segment linked to the destination

-  new profiles might have an attribute that causes governance / label check blocking export to the destination

- destination limitation on the number of profiles to be exported

[SubeAEP]

Well, upon further investigation into the issue, we discovered that the dataset profile_export_for_destination_merge_policy_**** stores the incremental records for each day's run. We have found that for some runs, there are few records without a member number, hence they are excluded. Moreover, there are duplicate records in the dataset and that is also one of the reasons for exclusion.

[Rite18]

good to know @SubeAEP  you have got the root cause, yeah,  duplicate records are also a reason for the exclusion