filter the data ingestion from adobe analytics source connector

[Pradeep-Jaiswal]

There is a requirement to split the historical adobe analytics data into two dataset. dataset1 having key events and enabled for profile. dataset2 having rest of the events(low importance) and not enabled for profile. is there a way to split the data ingestion using dataPrep or other method ?

[supratim320]

Easiest way to do this would be using query service/ data distiller, as long as you know how to differentiate between these 2 datasets, it will be quick & easy:

INSERT INTO <target_dataset>

SELECT * FROM <source_dataset>

where <conditions>

[Harveer_SinghGi1]

Hi @Pradeep-Jaiswal ,

I think you can achieve this without splitting the events in two different datasets unless that is a hard requirement for you. When setting up Analytics Source Connector you have the ability of Filtering for Real-Time Customer Profile which allows you to selectively include or exclude data from ingestion to the RTCDP from the dataflow of Adobe Analytics. You can apply,

1. Row-level filtering: For filtering specific events which fulfill your criteria

2. Column-level filtering: For filtering out the entire column (AA variable/metric field) from ingestion to RTCDP

Cheers!

[Pradeep-Jaiswal]

filtering is reserved only for live data, not for backfill

[kautuk_sahni]

@Pradeep-Jaiswal Just checking in — were you able to resolve your issue?
We’d love to hear how things worked out. If the suggestions above helped, marking a response as correct can guide others with similar questions. And if you found another solution, feel free to share it — your insights could really benefit the community. Thanks again for being part of the conversation!