Data Ingestion for Object Array field using File Upload method?

[sandip_surse]

Hi team,
Please help me to ingest below data in data lake through file upload method

Sample file in below records:

PName	Birthdate
Jery|doll	2023-03-10T18:56:41+00:00|2023-08-10T18:56:41+00:00
Tom|Marry	2023-05-10T18:56:41+00:00|2023-08-10T18:56:41+00:00

on below schema for reference schema structure for reference:

Please provide the suggestion or guide me on the same.

We are expecting to getting similar records at dataset level:

PName	Birthdate
Jery|doll	2023-03-10T18:56:41+00:00|2023-08-10T18:56:41+00:00
Tom|Marry	2023-05-10T18:56:41+00:00|2023-08-10T18:56:41+00:00

Regards,
Sandip

[Verticurl]

Please confirm a proper file format (.Parquet or JSON) for manual file uploads, not csv.

https://experienceleague.adobe.com/docs/experience-platform/ingestion/tutorials/ingest-batch-data.ht...

[sandip_surse]

Hi Verticurl,

It's CSV file format data.
We are already across on this document also we are able to see data during the data preparation but still data is not yet ingested as we expected.

[Verticurl]

For source connector Local File Upload, make sure you enable Error diagnostics to generate detailed error messages, then check for errors on failed ingestion status.

Please review docs:
https://experienceleague.adobe.com/docs/experience-platform/sources/connectors/local-system/local-fi...
https://experienceleague.adobe.com/docs/experience-platform/dataflows/ui/monitor-sources.html?lang=e...

[arpan-garg · Accepted solution]

Hello, @sandip_surse 

This solution will suit your needs:

Transform the arrays using the following method:

arrays_to_objects("birthDate", explode(Birthdate, "\|"), "XYZName", explode(PName, "\|"))

I couldn't find the correct name for the 'Name' field in your schema, so I used 'XYZName' as a placeholder above.

Once you've completed this step, kindly map it directly to the object array. Make sure to remove [0] from the data mapping.
Thanks,

Arpan

[sandip_surse]

Thanks Arpan,
We are used similar way one more data prep function and now issue get fixed thanks...
to_array(true,split(attribute_Name,"|")