Updating "Attributes" in Local File upload , after data Ingestion

[navink386520]

Step 1: On day 1, I created Source data flow via Localfileupload

Step 2: Uploaded .csv with 10 attributes into Dataset : Dataset1

Step 3: On day 2, I wanted to remove 2 attributes from 10

Thus question, could i do it ? 

[Avinash_Gupta_ · Accepted solution]

@navink386520

Generally speaking, AEP is an additive platform which means there are limited options for deleting data once it has been ingested. From the UI, you can delete a batch of data from the underlying data lake, but that will not remove the Profile data. You can also remove the entire dataset using the UI, which will remove all the data from the data lake and related Profiles. However, that will remove all data associated with the dataset (think of it as dropping a table in SQL). You can delete the dataset and re ingest the data by removing the 2 attributes.

[Anil_Umachigi]

@navink386520 When you say remove it all depends on from where?  and also if the data set is a time series or Record (Attributes) data. 

Record data: If record data type, you can overwrite those attributes on day 2 if that work for you? else you can remove the entire profile using privacy API. Another option is to drop the entire dataset thus deleting all data. 

Time series data: This is time stamped data, no two rows will be same because of time stamp and hence there is not really any way to remove or update data in this case. Your only option is 

• Delete the batch which ingested the data ( will also delete other rows of data), if enabled for profile this method will not remove it from profile
• Delete dataset to remove all data & which will remove all of the profile related data as well

Hope that helps

Anil