Data Ingestion Job Schedule in AEP

[vikramrai]

Hi team, 

We have batch ingestion set up where in AEP picks up files from SFTP folders, as of now it is set for specific time. 

I was wondering is there a option to make it any time and files will be picked up as soon as it is available in SFTP folder, so that we can avoid specific time requirements ad dependeies 

Thank you 

[abhinavbalooni · Accepted solution]

Hey @vikramrai 

I am not sure if you can achieve this through UI scheduling of batch files. 

You can look at the batch ingestion API but that would need a few changes in the way the file might be consumed currently. 

It should work something like this. 

You place a file in a cloud storage (not sure about sftp), trigger a microservice which in turn calls the batch ingestion api and then created a new batch upload everytime.

You would need some sort of a microservice (cloud function) which does the API calling bit for you.

Here is the link to the API documentation: 

https://experienceleague.adobe.com/docs/experience-platform/ingestion/batch/overview.html?lang=en

[arpan-garg]

Hi @vikramrai  - As @abhinavbalooni  mentioned , AEP does not support this. If you want this then you have to implement something custom using a middleware or a cloud solution. This custom solution should be configured as an event based trigger. As soon as a file is added or the data is changed the service should pick up the data from the file and ingest that into AEP.

Thanks,

Arpan