Null values are greater in volume than the source

[ShivaniM]

Hello All,

While comparing the values for one attribute in source (snowflake) and in Adobe Experience Platform.

Count for all values are matching but null values are way higher in number in AEP to that of source.

Can anyone help me know, how AEP processes the null values coming from source.

Example field OrdersInLast12Months has multiple values like (0, 10, 24, 44, 56, null etc)

I'm checking with latest timestamp of thirds to match the two values, all are matching other than null.

Thank You,

Shivani

[brekrut · Accepted solution]

Hello @ShivaniM 

A null values in a database / data warehouse is really an indicator data does not exist in the column for the record. 

When data is ingested into the Adobe RTCDP platform you will need to consider how to evaluate the presence of null from an external source.  

When data does not exist in the source it may not make sense to bring in the non existence of data into Adobe RTCDP.  If the record is ingested with a NULL you may face data issue where the text 'NULL' may be present or an empty string ' '.  These named value pairs in the CDP will evaluate to a value. 

I would recommend not ingesting the column which is null in the database / data warehouse.  You can use the condition of exist or not exists in the audience selection logic to collect profiles who do not contain values for a named value pair.