Remove/Create dataset from schema

[jooyeon_han]

Hi everyone,

One of my peers has updated identify in production by mistake so we had to remove the existing dataset (customer data in daily batch load) and recreate.

After deleting the dataset, I created the dataset by selecting 'create dataset from schema', then selected the same schema that was used. After that, I enabled the profile.

there is daily batch load automated by our data team. This morning I received an automated email that the file has been ingested into AEP.

However, I see status 'no batch'. Does it take couple of days to process? Did I miss any steps to ingest batch data into the new dataset?

[Indra]

Hi @jooyeon_han 

Can you please share what did he update in Dataset?

how did he update using API ? if yes which API and pls share Request and Response

[jooyeon_han]

I wish it was done via API.. the person updated the identity by check off the mark in UI.

Since our customers shares email accounts.. it aggregated multiple individuals into a single profile. 

I've deleted the existing dataset, re-create using schema. 

Then I noticed the scheduled SFTP also need to be recreated as previous one is for specific dataset (old one).

Currently waiting for tomorrow's batch load of our customer but not sure if I am missing any steps.

[Indra]

so changes made at Schema level not at dataset level right? Unchecked the Identity checkbox ?

[Indra]

Is your issue fixed? Are you able to see the data in dataset ?

[jooyeon_han]

Yes it seems like I now see new data in the dataset after updating sftp to the new dataset.

I don't see this applied to the existing segments yet.. It may require to rebuild segments.. I am going to allow 2 days if there is any scale change.

[DavidRoss91]

You shouldn't have to rebuild the segments, if they are still using the same data points that are being re-ingested into the new dataset. Are you still seeing any issues?

[jooyeon_han]

yes that is correct. I was told the person checked identity box on email, then he unchecked after.

[brekrut · Accepted solution]

During data ingestion a dataflow is needed to ingest data into a dataset if you are bringing data from an SFTP location.  If the dataset was deleted and re-created you need to ensure the dataflow is updated to point to the correct dataSet.

[jooyeon_han]

Thank you ! yes you are right. 

I later noticed that the I will need to recreate the scheduled sftp source to direct to the right dataset (new one) I created. After several test, it works!