Data Ingestion Job Schedule in AEP | Community
Skip to main content
V
vikramrai
December 1, 2023
Solved

Data Ingestion Job Schedule in AEP

  • December 1, 2023
  • 2 replies
  • 1199 views

Hi team, 

 

We have batch ingestion set up where in AEP picks up files from SFTP folders, as of now it is set for specific time. 

 

I was wondering is there a option to make it any time and files will be picked up as soon as it is available in SFTP folder, so that we can avoid specific time requirements ad dependeies 

 

Thank you 

This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.
Best answer by abhinavbalooni

Hey @vikramrai 

I am not sure if you can achieve this through UI scheduling of batch files. 

 

You can look at the batch ingestion API but that would need a few changes in the way the file might be consumed currently. 

 

It should work something like this. 

 

You place a file in a cloud storage (not sure about sftp), trigger a microservice which in turn calls the batch ingestion api and then created a new batch upload everytime.

 

You would need some sort of a microservice (cloud function) which does the API calling bit for you.

 

Here is the link to the API documentation: 

https://experienceleague.adobe.com/docs/experience-platform/ingestion/batch/overview.html?lang=en

 

 

2 replies

abhinavbalooni
Community Advisor
abhinavbalooniCommunity AdvisorAccepted solution
Community Advisor
December 1, 2023

Hey @vikramrai 

I am not sure if you can achieve this through UI scheduling of batch files. 

 

You can look at the batch ingestion API but that would need a few changes in the way the file might be consumed currently. 

 

It should work something like this. 

 

You place a file in a cloud storage (not sure about sftp), trigger a microservice which in turn calls the batch ingestion api and then created a new batch upload everytime.

 

You would need some sort of a microservice (cloud function) which does the API calling bit for you.

 

Here is the link to the API documentation: 

https://experienceleague.adobe.com/docs/experience-platform/ingestion/batch/overview.html?lang=en

 

 

    arpan-garg
    Community Advisor
    arpan-gargCommunity Advisor
    Community Advisor
    January 3, 2024

    Hi @vikramrai  - As @abhinavbalooni  mentioned , AEP does not support this. If you want this then you have to implement something custom using a middleware or a cloud solution. This custom solution should be configured as an event based trigger. As soon as a file is added or the data is changed the service should pick up the data from the file and ingest that into AEP.

     

    Thanks,

    Arpan

     

     

      Terms & ConditionsAccessibility statement

      Loading footer...