Export aggregated data from AEP to S3 bucket | Community
Skip to main content
Y
yumera
Level 2
February 19, 2024
Solved

Export aggregated data from AEP to S3 bucket

  • February 19, 2024
  • 2 replies
  • 1739 views

Hello,

 

We are looking at an option to export data from AEP to S3 bucket but in a bit unusual way.

The high level process is:

1. Using a query service data from a large Event dataset has been aggregated into a small table.

2. Using again query service data from point 1 table has been extracted and send for detail reporting.

3. Due to the query service expiring credentials this cannot be automated. And we are looking at the option of optimizing this.

 

One of our options we look at is to create a lookup schema+dataset with a identity non customer profile. But lookup dataset is not allowed to be used inside AEP and destinations

We tried with creating a event schema with only 1 profile and add many events on a single profile but then destination do not allow to export the data as array and would like to export the data as single line since there is only 1 profile on the dataset.

We cannot use "Individual Profile" as we cannot create each profile per the aggregated data, since it can be 1000 line of aggregation. 

 

Has anyone got any idea how can we achieve that?

    This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.
    Best answer by yumera

    Thank's all for suggestions. Unfortunately there were not suitable for our our solution. 

    At the end we end up creating a query script which will populate a lookup dataset which could be exported on a daily based using dataset export activity.

    2 replies

    arpan-garg
    Community Advisor
    arpan-gargCommunity Advisor
    Community Advisor
    February 19, 2024

    Hi @yumera  - What do you mean by query service expiring credentials? Are you using a third party to connect to it?

      Y
      yumeraAuthor
      Level 2
      February 20, 2024

      Hi arpan-garg,

       

      By query service i mean following:

      psql "sslmode=require
  host=XXX
  port=80
  dbname=XXX
      ...

       The aggregated data is not PII if it's what your concern for third party connection.

       

      arpan-garg
      Community Advisor
      arpan-gargCommunity Advisor
      Community Advisor
      February 20, 2024

      Hi @yumera - You can use no-expiring credentials, in case you are not able to automate it because of expiring credentials. More information can be found here https://experienceleague.adobe.com/docs/experience-platform/query/ui/credentials.html?lang=en

       

       

        Y
        yumeraAuthorAccepted solution
        Level 2
        March 15, 2024

        Thank's all for suggestions. Unfortunately there were not suitable for our our solution. 

        At the end we end up creating a query script which will populate a lookup dataset which could be exported on a daily based using dataset export activity.

        Terms & ConditionsAccessibility statement

        Loading footer...