Multiple Files with Same File Format but Different File Names – SFTP Ingestion into AEP

Question

Hi everyone,I have a scenario where 10 files are available in an SFTP location, and I’d like to load all of them into Adobe Experience Platform (AEP) at once. However, these files don’t all have the same number of fields — some contain additional or fewer fields than others.My questions are:Can all 10 files be ingested into AEP in a single load?How does AEP handle files with different field counts or mismatched schemas?Is there a best practice or recommended approach for handling such variations during ingestion?  Thank You

Devyendar · Accepted Answer

@siddarthk there are 2 ways to resolve it, but first create a master Schema with all the fields included across the 10 files.

1) Single Dataflow:

Standardize all the files and add the columns/fields missing in other files and leave the values for those fields blank.
Create a standard data mapping with all the fields from source files to Schema fields.
Make sure you don't select the individual files in the Dataflow (for preview), select the folder (that has all the 10 files), you might not get a preview if you select folder but that is fine.

2) Multiple Dataflows:

Create 10 different Dataflow each selecting the individual file and mapping the fields available in the file.
In this approach, you don't have to standardize all the files and include the missing fields (fields from other files with blank values).

Sign up

Login with SSO

Login to the community

Login with SSO

Scanning file for viruses.

This file cannot be downloaded