Data prep while ingesting data | Community
Skip to main content
arpan-garg
Community Advisor
arpan-gargCommunity Advisor
Community Advisor
May 11, 2023
Solved

Data prep while ingesting data

  • May 11, 2023
  • 2 replies
  • 1086 views

Hi ,

I have some data preparation that needs to be done while ingesting activity data from a source via streaming API's. Specifically, I am looking to set the eventType as "online.seller.event" if the person associated with the event is an online seller, and as "online.buyer.event" if the person associated with the event is an online buyer. However, information on whether the person is an online buyer or seller is only available in the profile dataset and is not present in the event table in the source system. I was wondering if there is a way to perform this calculation on-the-fly using the data-prep functions in AEP. Perhaps by querying the profile dataset associated with the ECID and setting the eventType to "online.buyer.event" if the person is an online buyer.

I would appreciate any guidance or suggestions you may have on this matter. Thank you for your time.

This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.
Best answer by arijitg

That's correct @arpan-garg it's recommended that we do all joins, lookup, aggregations etc to the source and send the finalized data to AEP.

2 replies

arijitg
Adobe Employee
arijitgAdobe Employee
Adobe Employee
May 11, 2023

Sorry @arpan-garg AEP has limited ETL capability and data-prep functions does not support such target table lookup feature that you're expecting.

arpan-garg
Community Advisor
arpan-gargCommunity AdvisorAuthor
Community Advisor
May 11, 2023

Thanks for the info @arijitg , whats the ideal way to achieve this ? Would you recommend doing all such calculations outside AEP and then ingest the data.

 

 

arijitg
Adobe Employee
arijitgAdobe EmployeeAccepted solution
Adobe Employee
May 15, 2023

That's correct @arpan-garg it's recommended that we do all joins, lookup, aggregations etc to the source and send the finalized data to AEP.

Tof_Jossic
Adobe Employee
Tof_JossicAdobe Employee
Adobe Employee
May 11, 2023

Not entirely sure if this helps but since ETL is mentioned here, I thought I'd point to the following document: 

Terms & ConditionsAccessibility statement

Loading footer...