Exclude event columns in Adobe DataStream set up

Question

Hello everyone,

I understand that in DataStream set up you can apply some rules for filtering event data that comes from Web/Mobile SDK and then it send data to event dataset. But, Is there a way to send some columns to event dataset enabled for profile and the rest of them, send to other dataset that isn't enabled for profile? In other words, I want to exclude some columns for the first dataset (I don't want to send all of them) and send it to other dataset.

The purpose of this, is to reduce the usage of Total Storage License.

Any advice is grateful.

Thank you.

Harveer_SinghGi1 · Accepted Answer

Hi @harveer_singhgi1 ,

I have 2 questions:

1. When you mention Rule in DataPrep, do you mean Dynamic Setup or Edit Mapping?

2. With the second option, I must create two differents datastream for each dataset?

Thank you!

Hi @cristhian_alexisna ,

The two options work as below,

Drop the unwanted columns/fields through DataPrep (Edit Mapping) in DataStream: Steps to implement this solution are,
- Create two DataStream,
  - DataStream 1:
    - Configure it with dataset 1 that needs to be profile enabled
    - Add rule in data prep to drop columns that are not needed in profile
  - DataStream 2:
    - Configure it with dataset 2 that is not required for profile
    - Add rule in data prep to drop columns that are needed in profile
- In Adobe Launch use DataStream 1 as default DataStream for WebSDK configuration
- Send entire payload to AEP using sendEvent (This sends data to dataset 1 using DataStream 1).
- Send entire payload to AEP using sendEvent while using the datastream override option to select DataStream 2 and dataset 2 for this event
Drop the unwanted columns/fields through Adobe Launch: Steps to implement this solution are,
- Create one DataStream,
  - DataStream 1:
    - Configure primary event dataset as dataset 1 that needs to be profile enabled
    - Configure secondary dataset as dataset 2 that is not required for profile
- In Adobe Launch use DataStream 1 as default DataStream for WebSDK configuration
- Prepare the payload for dataset 1 (populate only needed columns/fields)
- Send dataset 1 specific payload to AEP using sendEvent. (This sends data to dataset 1 using DataStream 1)
- Prepare the payload for dataset 2 (populate only needed columns/fields)
- Send dataset 2 specific payload to AEP using sendEvent while using the datastream override option to select DataStream 1 and dataset 2 for this event

So, to answer your questions, DataPrep is "Edit Mapping" option only in DataStream. And in second option you only need to create one DataStream with both event datasets added to it and choose the datasets for respective sendEvents.

Cheers!

Harveer_SinghGi1 · Answer

Hi @cristhian_alexisna ,You cannot split a single event and send it to multiple datasets in AEP using datastreams data prep. However, you can track the same event twice using sendEvent and send it to different datasets with respective columns needed in that dataset.What you'll have to do is trigger the sendEvent twice and route data to different datasets which can be achieved through data collection following below steps,Create two datastream,DataStream 1:Configure it with dataset 1 that needs to be profile enabledAdd rule in data prep to drop columns that are not needed in profileDataStream 2:Configure it with dataset 2 that is not required for profileAdd rule in data prep to drop columns that are needed in profileIn Adobe Launch use datastream 1 as default datastream for WebSDK configurationPrepare the payload for dataset 1 (populate only needed columns/fields)Send it to AEP using sendEvent.Prepare the payload for dataset 2 (populate only needed columns/fields)Send it to AEP using sendEvent while using the datastream override option to select datastream 2 for this event.This approach however comes with few caveats as you might have noticed,It causes doubling the sendEvent calls, so it won't be an ideal solution if you need to split the data on every AEP tracking callSame AEP event will be split in two places so you won't be able to breakdown the attributes from one dataset with other as they will behave as two different eventsI'll suggest exploring other options within AEP where you track the events as it is from your property in a single event dataset and then using AEP capabilities like a CTAS or data distiller create another dataset with only the columns required for profile and have this dataset profile enabled.Cheers!

Sign up

Login with SSO

Login to the community

Login with SSO

Scanning file for viruses.

This file cannot be downloaded