Expand my Community achievements bar.

SOLVED

How to verify that data ingested into a dataset processed to Profile successfully.

Avatar

Community Advisor

Hi All,

How can I validate/check if the data ingested into dataset processed to Profile successfully?

Thanks

1 Accepted Solution

Avatar

Correct answer by
Employee

HI @Luca_Lattarini 

 

1. Check for the status of the batch in the dataset activity tab.

2. You can use the Retrieve Batch Catalog API to receive the same metrics that you see in the dataset activity tab.

View solution in original post

8 Replies

Avatar

Level 10

one of the quick way is to grab few random IDs from the data set and try to search from the Profile.

Avatar

Community Advisor

Hi @Luca_Lattarini 

To get the stats on how many records were ingested, how many moved to profile, how many ids were generated... all this can be derived from the Monitoring dashboard. You can filter by days, check with specific dataflow as well.

 

For records that failed to move to the profile, I am not sure which API can be utilized to fetch that. But there should be a way to find that.

 

As @Lokesh_Shivalingaiah suggested you can randomly perform sanity check as well by searching. However for large data to find the profiles that couldn't make it to UPS - I will check if I can find more details.

 

Thanks,

Chetanya

Avatar

Correct answer by
Employee

HI @Luca_Lattarini 

 

1. Check for the status of the batch in the dataset activity tab.

2. You can use the Retrieve Batch Catalog API to receive the same metrics that you see in the dataset activity tab.

Avatar

Community Advisor

Hello @poojasinha @Lokesh_Shivalingaiah @ChetanyaJain ,

I did some test and I found that check Dataset> Activity and Identies > Browser are the two ways for my marketers. These two ways allow me to monitor from the ingestion part to the Profile side.Is it correct consider that Query Service and Monitoring Batch covers the same ingestion part of Dataset> Activity ?

Thanks

 

Avatar

Employee

Hi @Luca_Lattarini 

 

If marketers are comfortable using the UI they should rely on the Dataset>Activity and Monitoring Tabs to get the metrics/ information from ingestion to profile.  The identities tab will not give them this information.

 

Thanks,

Pooja

Avatar

Level 3

Hey,

I have data ingested successfully into dataset few days ago but it has not yet got added to profile. The schema is enabled for "Profile". I noticed that I had only one identifier which was also PK. Shall making an existing field as an additional Identifier, resolve the issue (data not yet added to profile)?

What about the existing data? Do I need to reimport the data?

Thanks.

Avatar

Employee

As long as their is one primary identity defined (not identity) on the dataset the data should get ingested in the profile store. Data enters profile store at the time of ingestion so you will have to reload the data to make sure it reaches profile store. Run the Catalog API to lookup the dataset and make sure it has the following two attributes set in the unifiedProfile tag: enabled=true and enabledAt=the timestamp of when the dataset was enabled for profile. If it is not set then disable/enable the profile toggle in the UI and check with the API again. Re-ingest the data again.