Append values to array in profiles attributes using mapping functions

[edgar_herrera]

Hi. I am having a problem with arrays in profile attributes and need guidance.

I am performing a proof of concept of arrays on profile attributes applying mapping functions. I have an XDM Individual Profile schema that, among its fields, there is an array of strings. In the same way, I have a dataset based on that schema.

I am using the Local File Upload source to upload CSV files. I have the following CSV files:

test1.csv

ID_CLIENTE_CORE, ARRAY
T0000000001, Hello

test2.csv

ID_CLIENTE_CORE, ARRAY
T0000000001, Friend

When it comes to mapping, I create a calculated field as follows:

upsert_array_append(to_array(false, ARRAY))

 Let me explain why I use these mapping functions:

1. to_array(value): This function converts a string, for example 'Hello', into an array. This causes 'Hello' to become ['Hello']. [1]
2. upsert_array_append(value): This function, in theory and following the Data Prep documentation, adds all elements of the full input array to the end of the array in Profile. This generates, in theory, that if a profile has an array in the path _tenant.arrayTest.array as profile attribute, with value of ['Friend'], and value is equal to ['Hello'], subsequent to loading the CSV file, the value of the profile attribute is ['Friend','Hello',]. Please correct me if I am wrong. [1]

I have tried the following mappings:

Source data	Target fields
upsert_array_append(to_array(false, ARRAY))	_tenant.ArrayAppendTest.array
upsert_array_append(to_array(false, ARRAY))	_tenant.ArrayAppendTest.array[*]

However, every time I upload data using any of the above mappings, the to_array(ARRAY) field replaces the one that was previously in the profile.

It should be noted that I have the isUpsert flag enabled [2], so I think my problem is related to how I am doing the mapping.

Doc. reference.

1. https://experienceleague.adobe.com/en/docs/experience-platform/data-prep/functions#arrays
2. https://experienceleague.adobe.com/en/docs/experience-platform/catalog/datasets/enable-upsert#enable...

[Elizabethes1 · Accepted solution]

you're facing an issue with appending values to an array field in an XDM Individual Profile schema using Adobe Experience Platform. Based on your description, the problem likely lies in how the upsert_array_append function is applied during the data mapping process.

Here’s a breakdown of your situation:

1. Understanding to_array and upsert_array_append:

   • to_array(value): Converts a single string into an array. For instance, 'Hello' becomes ['Hello'].
   • upsert_array_append(value): Should append the new array elements to the existing array in the profile.

2. Issue Encountered:

   • You expect that after uploading test1.csv and test2.csv, the profile's array should contain ['Friend', 'Hello']. Instead, it seems to be replaced each time, resulting in only the last value being retained.

3. Possible Cause:

   • The problem might be related to how the array is being accessed and updated in the mapping process. Using [*] in the target field might be causing the array to be overwritten instead of appending new elements.

4. Potential Solution:

   • Ensure you are correctly mapping the fields and not inadvertently overwriting the array. Here's how you might approach it:

   Mapping Example:

   plaintext

   Source data Target fields upsert_array_append(to_array(false, ARRAY)) _tenant.ArrayAppendTest.array

   Avoid using [*] unless you're targeting specific elements within an array. If [*] is not used correctly, it could lead to the behavior where existing elements are replaced.

5. Enabling the isUpsert Flag:

   • You mentioned the isUpsert flag is enabled, which is essential to ensure that profiles are updated rather than replaced. Double-check that this flag is applied correctly in your dataset and schema setup.

Steps to Try:

• Remove the [*] in the target field mapping and test again.
• Verify the Schema: Ensure your schema supports the array field and is correctly set up for append operations.
• Test with a Smaller Dataset: Try uploading a smaller dataset to observe the behavior and confirm that the array is being appended as expected.

Additional Resource:

• You might want to revisit the documentation and ensure that all functions and mapping steps are correctly implemented:  Adobe Experience Platform - Array Functions. or check  innovative cbd for info about the cbd.

If the issue persists, it might be beneficial to reach out to Adobe Support for more detailed troubleshooting specific to your implementation.

[edgar_herrera]

I have tried with _tenant.ArrayAppendTest.array, without using [*], getting the same result. I also have the isUpsert flag enabled.

[arundeepjain]

Hi @edgar_herrera upsert_array_append works only when dataset is enabled for upsert. Check and enable it by going through following link. Also you will not be able to verify this in datat lake as it gets updated into profile only.

https://experienceleague.adobe.com/en/docs/platform-learn/tutorials/profiles/update-a-specific-attri...

Thanks,

@jainarundeep 

[edgar_herrera]

I have the isUpsert flag enabled. I have already mentioned it above